Distributed Systems

A Psyclone system scales along two axes without changing a line of crank code: spaces (more processes on one machine) and nodes (more machines). In both cases the placement is a PsySpec attribute — the modules themselves are oblivious.

Spaces: process isolation on one node

Every OS process attached to a node is a space (cmlabs::PsySpace). All inter-process state — component registries, message stores, queues — lives in named shared-memory segments (cmlabs::MemoryManager), which is what buys crash isolation: a space can die without corrupting the rest of the system, and can re-attach on restart.

Any catalog, whiteboard or module can be placed in a process separate from the main node process:

<space name="MySpace" type="external" />
 
<module name="Test2" space="MySpace">
    <trigger name="Ball" type="ball.2" />
    <crank name="Test" />
    <post name="Ball" type="ball.3" />
</module>

If the space does not already exist it is started automatically as a separate system process, and it is destroyed when the node shuts down. A node can have any number of spaces; if a component in one space performs an illegal action and crashes it, only the components in that space are affected. Runnable demos: Examples/adhoc.xml (a module in an external space between two in-node modules) and Examples/externals.xml.

Because same-node message transport goes through shared memory — a cmlabs::DataMessage is one flat allocation copied verbatim — crossing a space boundary costs a memory copy, not serialisation.

The external-module program in Examples/src/ExternalModules.cpp shows the manual version of the same idea: any executable you write can create a PsySpace, cmlabs::PsySpace::connect() to a running node by port, obtain a cmlabs::PsyAPI with cmlabs::PsySpace::getCrankAPI() and behave exactly like an internal crank — including detecting disconnection and re-attaching (see Writing a Module (Crank)).

Nodes: multiple machines (or multiple runtimes)

A <node> entry declares another Psyclone runtime that belongs to this system, and node= attributes place components on it:

<node name="Node1" address="localhost" port="11000" />
 
<module name="Ping" node="Main"> ... </module>
<module name="Pong" node="Node1"> ... </module>
 
<whiteboard name="WB1" node="Node1" key="count" keytype="integer" />
<catalog name="MyFiles" node="Node1" type="FileCatalog" root="./" />

Examples/multinode.xml runs the full ping-pong / signals / catalogs test suite with the two players, the whiteboards and the catalogs deliberately split across two nodes; Examples/multinodesimple.xml is the reduced version. Everything from Core Concepts carries over unchanged:

• Triggers and posts work system-wide; the node routes copies across the TCP link between nodes. Posts can opt out of guaranteed delivery for speed with guaranteed="no" (unguaranteed/UDP-style transport) — see the Test.PingPongUDP phase of multinode.xml, which benchmarks both.
• Signals are broadcast to every space in the whole system, across nodes.
• Retrieves and queries are location-transparent: a module on Main retrieves from a whiteboard on Node1 with the same cmlabs::PsyAPI::retrieve() call; the QUERY_NOT_REACHABLE status exists precisely for the remote-down case.
• Contexts are system-wide states; a context post on one node gates modules on all nodes.

For quick experiments you don't even need a second machine or spec: ./Psyclone psytest=N spawns N nodes on the local computer and spreads the test modules across them automatically.

The networking underneath

Node-to-node and external-client communication is built on the CMSDK networking stack — the same stack available to your own programs:

• cmlabs::NetworkManager — connection/channel management, protocol auto-detection (binary messages, HTTP(S), WebSocket, telnet on the same port), and the built-in HTTP server that also serves PsyProbe.
• The request fabric — cmlabs::RequestClient / cmlabs::RequestGateway / cmlabs::RequestExecutor — for distributed request/response workloads with load balancing and failover (see Going Deeper).
• SSL support is available throughout when built with the SSL targets (see Compiling and linking with the CMSDK library).

Design guidance

• Start on one node, one space. Split out a space when a component is crashy or resource-hungry; split out a node when you need another machine's hardware (sensors, GPUs) or capacity.
• Keep high-rate message paths within one node where transport is shared-memory; cross nodes with lower-rate, coarser messages, or mark them guaranteed="no" when loss is acceptable.
• Let whiteboards/catalogs live near their heaviest writers; retrieves and queries are location-transparent for the readers either way.