Shared-Memory Subsystem

Shared-memory segments, maps, queues and allocators used for inter-process communication in Psyclone. More...

Detailed Description

Shared-memory segments, maps, queues and allocators used for inter-process communication in Psyclone.

Why shared memory

Psyclone deliberately runs as multiple cooperating OS processes rather than one big multithreaded program: a crash in one component's process cannot take down the rest of the system (crash isolation), processes can be restarted and re-attach to the live state, and external tools can inspect a running node without stopping it. The price is that all shared state must live in named shared-memory segments instead of ordinary heap objects — and this group provides everything needed to make that practical: the MemoryManager singleton that creates/attaches the segments, typed name→id maps (MemoryMaps.h, DataMapsMemory.h), ring-buffer message queues (MemoryQueues.h, ProcessMemory.h), the temporal message store (TemporalMemory.h) and the page-based legacy allocator.

Why not plain OS APIs or Boost.Interprocess

The raw primitives differ per platform — CreateFileMapping/MapViewOfFile on Windows, shm_open/mmap on Linux/macOS, with different naming, permission and cleanup semantics — and neither the OS nor std provides message queues, typed maps or a temporal store on top of them. CMSDK wraps the primitives once (in Utility functions and objects, used by MemoryManager) and builds the higher-level containers in a way that integrates directly with Messaging — a DataMessage's flat layout means it can be memcpy'd into a queue slot with no serialisation, and named semaphores give cross-process blocking waits.

How to use it: producer/consumer via a shared queue

Unrelated processes rendezvous on a queue by global name; each message is copied into shared memory on add and copied back out on receive, so no pointers ever cross the process boundary.

// Process A (producer)
uint32 qid;
MessageQueue::CreateMessageQueue(qid, "vision.in");   // create/attach by name
DataMessage* msg = new DataMessage(myType, myComponentID);
msg->setString("camera", "front-left");
MessageQueue::AddMessageToQueue(qid, msg);            // copied into the segment
delete msg;                                           // producer keeps ownership
 
// Process B (consumer) — blocks up to 100 ms on a named semaphore
DataMessage* in = MessageQueue::WaitForMessageQueue(qid, 100);
if (in) {
    const char* cam = in->getString("camera");
    // ... process ...
    delete in;                                        // heap copy owned by caller
}

For keyed lookup instead of FIFO delivery, use the shared maps (GenericMemoryMap and the registries in DataMapsMemory.h), which map names/ids to entries inside a segment the same way.

Warning

Struct-layout stability is the cardinal rule of this group. Every struct stored in a segment is mapped simultaneously into several processes, so its size, field order and alignment form a silent ABI. After changing ANY such struct, rebuild everything (make clean && make); running old and new binaries against the same segment corrupts memory across processes with no compiler or runtime diagnostic.

Note

Segments are keyed by node port, so several Psyclone instances can coexist on one machine; the machine-global instance index (PsycloneIndexHeader) lets them discover each other.

See also

MemoryManager, MessageQueue, GenericMemoryMap, TemporalMemory.h, Messaging

Files
file	ComponentData.h
	Legacy per-component data record stored in the page-based allocator (predecessor of ComponentMemory).
file	ComponentMemory.h
	Shared-memory component registry: identity, location, parameters, private data and statistics for every component in the system.
file	DataMapsMemory.h
	Shared-memory name/id maps: message types, contexts, tags, cranks and cross-process requests.
file	DynamicMemory.h
	Empty placeholder header for the dynamic-memory subsystem.
file	MemoryCommander.h
	Stub for a shared-memory command dispatcher; currently a placeholder with no active role.
file	MemoryController.h
	Abstract interface that lets subsystem memories query/update their segment's id, size and resize serial.
file	MemoryMaps.h
	Legacy shared-memory maps for the page-based allocator: reserved page ids, map entry layouts and the static MemoryMaps facade.
file	MemoryMaps.tpl.h
	Template implementations of the GenericMemoryMap shared-map algorithms.
file	MemoryQueues.h
	Ring-buffer message queues stored in shared memory, plus a static request/reply facility.
file	MemoryRequestConnection.h
	Client-side endpoint of the named shared-memory request/reply service.
file	MemoryRequestQueues.h
	Static algorithms for request queues and the request-status map used by the shared-memory request/reply server.
file	MemoryRequestServer.h
	Server side of the named shared-memory request/reply service.
file	ProcessMemory.h
	Shared-memory process ("space") table plus per-process message queues.
file	TemporalMemory.h
	Slot/bin shared-memory store for time-limited DataMessages (the node's "dynamic" memory).
file	ComponentData.cpp
	Implementation of the legacy page-based ComponentData record accessor.
file	ComponentMemory.cpp
	Implementation of the shared component registry: record allocation/relocation, parameters, private data, stats and migration support.
file	DataMapsMemory.cpp
	Implementation of the shared name/id maps (types, contexts, tags, cranks) and the request map, including two-phase id allocation and cluster sync.
file	DynamicMemory.cpp
	Placeholder implementation file for the dynamic-memory subsystem (see TemporalMemory.cpp for the real store).
file	MemoryCommander.cpp
	Stub implementation of the MemoryCommander placeholder.
file	MemoryController.cpp
	Trivial constructor/destructor of the abstract MemoryController interface.
file	MemoryManager.cpp
	Implementation of MemoryManager, MasterMemory, PsycloneIndex and the legacy MemoryManagerX page allocator.
file	MemoryMaps.cpp
	Implementation of the legacy static MemoryMaps facade over the page-based shared maps.
file	MemoryQueues.cpp
	Implementation of the shared-memory MessageQueue ring buffers and the static request/reply helpers, plus their unit tests.
file	MemoryRequestConnection.cpp
	Implementation of the client endpoint of the named shared-memory request/reply service, including the out-of-order reply cache.
file	MemoryRequestQueues.cpp
	Implementation of the raw request-queue and request-status-map algorithms used by the request/reply service.
file	MemoryRequestServer.cpp
	Implementation of the request/reply server: segment creation, request dispatch and connection maintenance.
file	ProcessMemory.cpp
	Implementation of the shared process table and per-process command/message/signal/request queues with named-semaphore signalling.
file	TemporalMemory.cpp
	Implementation of the slot/bin temporal message store: bin selection, block allocation, growth/resize and EOL expiry sweeps.

Namespaces
namespace	cmlabs

Classes
class	cmlabs::ComponentData
	Legacy accessor wrapping one component record inside a shared page. More...
struct	cmlabs::ComponentStats
	Legacy per-component throughput counters (DataMessageHeader-based rings). More...
class	cmlabs::ComponentMemory
	Accessor for the shared component registry. More...
class	cmlabs::DataMapsMemory
	Accessor for the shared name/id maps and the request map. More...
class	cmlabs::GenericMemoryMap< T, ID >
	Static algorithms shared by all bitfield-indexed maps in the data-maps segment. More...
class	cmlabs::MemoryCommander
	Placeholder class for a future shared-memory command interface. More...
class	cmlabs::MemoryController
	Pure-virtual interface for shared-segment bookkeeping (implemented by MasterMemory). More...
class	cmlabs::PsycloneIndex
	Accessor for the machine-global instance index segment. More...
struct	cmlabs::PerfStats
	Raw per-component performance counters sampled inside shared memory. More...
struct	cmlabs::AveragePerfStats
	Sliding-window averages derived from successive PerfStats snapshots. More...
struct	cmlabs::MemoryMasterStruct
	Root header of the node's master shared-memory segment. More...
class	cmlabs::MasterMemory
	Handle to the node's master shared-memory segment (MemoryMasterStruct). More...
class	cmlabs::MemoryManager
	Top-level facade of the shared-memory subsystem for one process. More...
struct	cmlabs::CMRegister
	Legacy registry block listing the Psyclone system ids present on this machine. More...
struct	cmlabs::MemoryID
	Decomposed form of a 64-bit global memory-block id (legacy page allocator). More...
class	cmlabs::MemoryManagerX
	Legacy page-based shared-memory allocator (predecessor of MemoryManager). More...
class	cmlabs::MemoryMaps
	Legacy static facade over the page-based shared maps (types, contexts, topics, cranks). More...
class	cmlabs::MessageQueue
	Static facade over shared-memory message queues and a request/reply matcher. More...
class	cmlabs::MemoryRequestConnection
	Client endpoint connected to a named shared-memory request server. More...
class	cmlabs::MemoryRequestQueues
	Static helpers for request queues and the request-status map (raw shared memory). More...
class	cmlabs::MemoryRequestServer
	Owner of a named request/reply segment; serves incoming client requests. More...
struct	cmlabs::ProcessStats
	Per-process throughput counters and recent-message rings, kept in shared memory. More...
struct	cmlabs::ProcessInfoStruct
	One process (space) record in the shared process table. More...
class	cmlabs::ProcessMemory
	Accessor for the shared process table and per-process queues. More...
class	cmlabs::TemporalMemory
	Accessor for the temporal (dynamic) message store in shared memory. More...