Networking

Socket transports, wire protocols, and the NetworkManager connection/channel layer (including the built-in HTTP/WebSocket server). More...

Detailed Description

Socket transports, wire protocols, and the NetworkManager connection/channel layer (including the built-in HTTP/WebSocket server).

Why CMSDK ships its own networking layer

Writing directly against the operating system's socket API means maintaining two code paths (Winsock2 on Windows, BSD sockets on Linux/macOS), hand-rolling byte-order conversion, framing, buffering, timeouts, reconnection and threading — for every application. The CMSDK Networking subsystem packages all of that once, portably, and integrates it with the rest of the SDK:

• One API across platforms: all Winsock-vs-POSIX differences (WSAStartup, SOCKET vs int descriptors, error codes, non-blocking connect semantics) are hidden below NetworkConnection and TCPListener; user code is identical on Windows, Linux and macOS.
• Consistent framing and byte order: HTTP, WebSocket (RFC 6455) and Telnet framing are implemented in NetworkProtocols.h; multi-byte header fields go over the wire in network (big-endian) byte order via htons()/htonl()/ntoh64().
• Native DataMessage transport: cmlabs::DataMessage objects — the SDK's universal typed container — can be sent as-is over TCP or UDP (PROTOCOL_MESSAGE), which is what the higher-level request fabric (RequestGateway/RequestPort) is built on.
• Built-in threading: every listener and connection gets its own worker thread; parsed traffic is either pushed to a cmlabs::NetworkReceiver callback (async) or queued for blocking waitFor*() calls (sync). No select()/poll() loops to write.
• Extras for free: protocol auto-detection on a single port, automatic reconnection, throughput statistics, SSL/TLS, and a complete embedded HTTP/WebSocket server.

Architecture: three layers, two key classes

The subsystem is stacked in three headers:

• NetworkConnections.h — raw transports: TCPListener, cmlabs::NetworkConnection and its concrete subclasses TCPConnection, UDPConnection, SSLConnection. These own actual sockets and move bytes.
• NetworkProtocols.h — wire formats: HTTPRequest / HTTPReply, WebsocketData, TelnetLine, JSONM, plus the static protocol handlers (HTTPProtocol, MessageProtocol, TelnetProtocol) and per-protocol CheckBufferForCompatibility() sniffers used for auto-detection.
• NetworkManager.h — lifecycle and dispatch: cmlabs::NetworkManager and cmlabs::NetworkChannel.

The two classes you normally touch:

• cmlabs::NetworkManager is the single top-level owner in a process. Its create*() factories open listeners (createListener()) and outgoing TCP/UDP/WebSocket connections; it stores the SSL certificate for encrypted listeners (setSSLCertificate()), runs a supervision thread that completes delayed connects and performs auto-reconnection, offers a blocking HTTP(S) client (makeHTTPRequest()), and hosts the built-in HTTP/WebSocket server. It owns every channel and connection it creates.
• cmlabs::NetworkChannel groups related listeners/connections into one logical interface (e.g. "the web UI port" or "the gateway link") and is what the factories return. Each listener/connection under a channel runs on its own worker thread; the channel funnels parsed traffic to your NetworkReceiver (async mode) or into typed queues read with waitForMessage(), waitForHTTPRequest(), waitForWebsocketData(), etc. (sync mode). Sends are addressed by connection id (sendMessage(), sendHTTPReply(), sendWebsocketData(), ...). A single peer link is identified by its uint64 connection id; the packed uint64 "location" (IPv4 + port in one integer) identifies the endpoint.

Mode 1: binary DataMessage over TCP (the native mode)

The workhorse of CMSDK inter-process traffic: size-prefixed serialised DataMessage frames (PROTOCOL_MESSAGE). Synchronous server and client:

// Server: listen on a caller-chosen port, sync (queued) mode
NetworkManager net;
NetworkChannel* srv = net.createListener(9000, NOENC, PROTOCOL_MESSAGE,
                                         false, 3000, true, 0, NULL);
uint64 conid;
DataMessage* msg = srv->waitForMessage(conid, 5000);   // caller owns msg
if (msg) {
    DataMessage* reply = new DataMessage();
    reply->setString("Result", "ok");
    srv->sendMessage(reply, conid);    // send does NOT take ownership
    delete reply;
    delete msg;
}
 
// Client: connect and exchange
NetworkManager cli;
uint64 cid, loc;
NetworkChannel* ch = cli.createTCPConnection("127.0.0.1", 9000, NOENC,
        PROTOCOL_MESSAGE, false, true, 0, NULL, cid, loc); // isAsync=false, autoreconnect=true
DataMessage* out = new DataMessage();
out->setString("URI", "ping");
ch->sendMessage(out, cid);
delete out;
DataMessage* in = ch->waitForMessage(cid, 3000);       // caller owns in
delete in;

For push (async) delivery instead, pass isAsync=true and a NetworkReceiver whose receiveMessage() override handles each message on the connection's thread.

Mode 2: UDP datagrams

Connectionless DataMessage delivery — discovery, telemetry, best-effort notifications:

NetworkManager net;
uint64 conid;
net.createUDPConnection(9001, PROTOCOL_MESSAGE, false, true, 0, NULL, conid);
// ... elsewhere, fire a datagram at a packed uint64 endpoint:
DataMessage* m = new DataMessage();
m->setString("Event", "heartbeat");
net.sendUDPMessage(m, destination);   // delivery not guaranteed (UDP)
delete m;

Mode 3: raw byte streams

When no framing is wanted at all, use the transport layer directly: TCPConnection / TCPListener give you buffered, timeout-aware, thread-safe byte I/O without any protocol on top:

TCPConnection con;
uint64 location;
if (con.connect("192.168.0.10", 7000, location, 3000)) {
    con.send("HELLO\n", 6);
    char buf[256]; uint32 got = 0;
    if (con.receiveAvailable(buf, got, sizeof(buf), 2000))
        ; // got bytes in buf
    con.disconnect();
}

Pass a NetworkDataReceiver to connect() to have bytes pushed to you from a reader thread instead (push mode).

Mode 4: Telnet (line-based text)

Interactive consoles and diagnostics speak PROTOCOL_TELNET — plain text lines (TelnetLine) with CRLF/LF handling:

NetworkManager net;
NetworkChannel* tn = net.createListener(2323, NOENC, PROTOCOL_TELNET,
                                        false, 3000, true, 0, NULL);
uint64 conid;
TelnetLine* line = tn->waitForTelnetLine(conid, 10000);  // caller owns line
if (line) {
    TelnetLine* out = new TelnetLine(0);
    out->setLine("OK", true);
    tn->sendTelnetLine(out, conid);
    delete out;
    delete line;
}

Mode 5: the built-in HTTP server

Any port becomes a web server with PROTOCOL_HTTP_SERVER — this is exactly how the PsyProbe monitoring UI is served. Async style with a receiver:

class MyServer : public NetworkReceiver {
    bool receiveHTTPRequest(HTTPRequest* req, NetworkChannel* channel,
                            uint64 conid) {
        HTTPReply* reply = new HTTPReply();
        reply->createPage(HTTP_OK, GetTimeNow(), "MyServer", 0,
                          req->keepAlive, false, "text/html",
                          "<html><body>Hello</body></html>");
        channel->sendHTTPReply(reply, conid);
        delete reply;
        delete req;      // we took ownership by returning true
        return true;
    }
};
MyServer handler;
NetworkManager net;
net.createListener(8080, NOENC, PROTOCOL_HTTP_SERVER,
                   true, 3000, true, 0, &handler);  // isAsync=true -> push mode

Sync style: omit the receiver (isAsync=false) and loop on NetworkChannel::waitForHTTPRequest() + sendHTTPReply(). HTTPReply also serves files from disk (createFromFile(), with If-Modified-Since/304 support), canned error pages (createErrorPage()), Basic-auth challenges and CORS preflight responses.

Mode 6: HTTP client

One blocking call, URL in, reply out (https:// switches to SSL automatically; credentials may be embedded in the URL):

NetworkManager net;
HTTPReply* reply = net.makeHTTPRequest("http://example.com/index.html", 5000);
if (reply) {
    uint32 size;
    const char* body = reply->getContent(size);
    delete reply;    // caller owns the reply
}

Other overloads send pre-built HTTPRequest objects, custom headers, raw typed bodies, or multipart/form-data uploads (HTTPPostEntry).

Mode 7: SSL/TLS

Any TCP-based mode above can be encrypted by passing SSLENC instead of NOENC (build with _USE_SSL_ / OpenSSL). Servers need a certificate first; client-side peer verification policy is configurable per manager, per channel or process-wide:

NetworkManager net;
net.setSSLCertificate("server.pem", "server.key");     // before SSLENC listeners
net.createListener(8443, SSLENC, PROTOCOL_HTTP_SERVER,
                   true, 3000, true, 0, &handler);      // HTTPS server
 
net.setSSLAllowSelfSigned(true);                        // testing only!
HTTPReply* r = net.makeHTTPRequest("https://localhost:8443/", 5000);

Warning

Allowing self-signed certificates disables peer verification for those connections — use only for testing or closed networks. Custom CA files/paths can be set with setSSLCALocation().

Mode 8: WebSockets

The HTTP server upgrades to WebSocket (RFC 6455) on the same port: detect the handshake with HTTPRequest::isWebsocketUpgrade(), answer with HTTPReply::CreateWebsocketHTTPReply(), then exchange WebsocketData frames.

class WSServer : public NetworkReceiver {
    bool receiveHTTPRequest(HTTPRequest* req, NetworkChannel* channel,
                            uint64 conid) {
        if (req->isWebsocketUpgrade()) {
            HTTPReply* up = HTTPReply::CreateWebsocketHTTPReply(
                req->getHeaderEntry("Sec-WebSocket-Key"),
                req->getHeaderEntry("Sec-WebSocket-Version"));
            channel->sendHTTPReply(up, conid);
            delete up;
        }
        delete req;
        return true;
    }
    bool receiveWebsocketData(WebsocketData* wsData, NetworkChannel* channel,
                              uint64 conid) {
        uint64 size;
        const char* payload = wsData->getContent(size);   // echo it back:
        WebsocketData* out = new WebsocketData();
        // server-to-client frames are sent unmasked (maskData=false)
        out->setData(WebsocketData::TEXT, false, payload, size);
        channel->sendWebsocketData(out, conid);
        delete out;
        delete wsData;
        return true;
    }
};
WSServer ws;
NetworkManager net;
net.createListener(8080, NOENC, PROTOCOL_HTTP_SERVER, true, 3000, true, 0, &ws);
 
// Client side — one call does connect + upgrade handshake:
uint64 conid;
NetworkChannel* ch = net.createWebsocketConnection("ws://localhost:8080/live",
                                                   0, NULL, conid);
WebsocketData* frame = new WebsocketData();
frame->setData(WebsocketData::TEXT, true, "hi", 2); // client frames: masked
ch->sendWebsocketData(frame, conid);
delete frame;
WebsocketData* answer = ch->waitForWebsocketData(conid, 3000);  // caller owns
delete answer;

Multiple protocols on one port

PROTOCOL_* values are bit flags: OR them in createListener() and each incoming connection's first bytes are sniffed (NetworkChannel::autoDetectConnection() via the per-protocol CheckBufferForCompatibility() tests) to pick the right handler thread:

// One port serving web pages, binary messages AND a telnet console:
net.createListener(9000, NOENC,
                   PROTOCOL_HTTP_SERVER | PROTOCOL_MESSAGE | PROTOCOL_TELNET,
                   true, 3000, true, 0, &handler);

Threading, ownership and lifecycle rules

Note

Async (isAsync=true) callbacks arrive on each connection's own protocol thread — receivers must be thread-safe. Objects handed to receive*() callbacks become yours when you return true (delete them); objects returned by waitFor*() and makeHTTPRequest() are always owned by the caller. Send methods never take ownership of the object passed in. Connection lifecycle changes (connect/disconnect/reconnect, buffer pressure, protocol errors) are reported as NetworkEvent notifications (NETWORKEVENT_* types).

Ports are always caller-chosen; the SDK deliberately has no hard-coded default port. Connections created with autoreconnect=true are transparently re-established by the manager's supervision thread.

Warning

The binary DataMessage protocol sends the serialised message image without byte-order conversion; both peers must share the same endianness (fine for x86/ARM; see the NetworkProtocols.h byte-order notes). WebSocket and HTTP framing, by contrast, are fully byte-order safe on the wire.

Files
file	ConsoleOutput.h
	Levelled console logging: verbose, debug and error printf-style output with per-subject filtering.
file	HTML.h
	HTML/URL helper utilities: entity encoding/decoding, MIME type lookup and URL component parsing.
file	NetworkManager.h
	Connection/channel management layer: multi-protocol listeners, typed dispatch, HTTP client — and the CMSDK's built-in HTTP/WebSocket server.
file	NetworkProtocols.h
	Wire-protocol layer: HTTP request/reply, WebSocket frames, Telnet lines, binary DataMessages, and protocol auto-detection.
file	ConsoleOutput.cpp
	Implementation of the levelled console logging functions: per-subject verbose/debug/error filtering and printf-style output.
file	HTML.cpp
	Implementation of the HTML/URL helpers: entity encode/decode tables, extension-to-MIME-type mapping and URL component extraction.
file	NetworkConnections.cpp
	Implementation of the raw socket transports: TCPListener, TCPConnection, UDPConnection and SSLConnection (OpenSSL).
file	NetworkManager.cpp
	Implementation of NetworkManager, NetworkChannel and NetworkThread: channel/connection lifecycle, protocol worker threads (HTTP server/client, message, telnet, WebSocket), protocol auto-detection and the blocking makeHTTPRequest() client, plus networking unit tests.
file	NetworkProtocols.cpp
	Implementation of the wire-protocol layer: HTTPRequest/HTTPReply parsing and generation, WebSocket (RFC 6455) framing with masking and extended payload lengths in network byte order, Telnet lines, JSONM multipart containers and the protocol auto-detection checks.

Classes
class	cmlabs::NetworkConnectionReceiver
	Callback interface for asynchronous delivery of newly accepted connections. More...
class	cmlabs::NetworkDataReceiver
	Callback interface for asynchronous (push) delivery of received bytes. More...
class	cmlabs::TCPListener
	TCP server socket: binds a port and accepts inbound connections (plain or SSL). More...
class	cmlabs::NetworkConnection
	Abstract base class for all point-to-point network connections. More...
class	cmlabs::UDPConnection
	UDP datagram connection (bound port for input, or output-only sender). More...
class	cmlabs::TCPConnection
	Plain TCP stream connection (client-initiated or accepted from a listener). More...
class	cmlabs::SSLConnection
	SSL/TLS-encrypted TCP connection (OpenSSL) with configurable peer verification. More...
struct	cmlabs::NetworkEvent
	Notification of a connection lifecycle change (connect, disconnect, buffer state...). More...
class	cmlabs::NetworkReceiver
	Callback interface for asynchronous delivery of parsed network traffic. More...
class	cmlabs::HTTPTestServer
	Minimal HTTP server used by the unit tests: replies with a canned page. More...
class	cmlabs::WebsocketTestServer
	Minimal WebSocket echo server used by the unit tests (handles upgrade + echo). More...
class	cmlabs::NetworkManager
	Central owner of all channels, listeners and connections in a process. More...
class	cmlabs::NetworkThread
	Bookkeeping for one worker thread of a NetworkChannel (per listener or connection). More...
class	cmlabs::NetworkChannel
	One logical network interface: a group of listeners/connections with shared dispatch. More...
class	cmlabs::HTTPPostEntry
	One named entry of an HTTP POST body (form field or uploaded file part). More...
class	cmlabs::HTTPRequest
	A parsed or generated HTTP request (also used for WebSocket upgrade handshakes). More...
struct	cmlabs::JSONMEntry
	Metadata for one binary attachment chunk inside a JSONM container. More...
class	cmlabs::JSONM
	JSONM: a JSON document with attached binary chunks in one contiguous buffer. More...
class	cmlabs::WebsocketData
	One WebSocket frame/message (RFC 6455): parsing, generation and control frames. More...
class	cmlabs::TelnetLine
	One line of Telnet-style text traffic. More...
class	cmlabs::HTTPReply
	A parsed or generated HTTP response. More...
class	cmlabs::NetworkProtocol
	Base class for the protocol auto-detection framework. More...
class	cmlabs::HTTPProtocol
	Server-side HTTP/WebSocket protocol handler (static send/receive helpers). More...
class	cmlabs::HTTPClientProtocol
	Client-side HTTP protocol handler (mirror of HTTPProtocol for outgoing requests). More...
class	cmlabs::MessageProtocol
	Binary DataMessage protocol handler: size-prefixed serialised messages. More...
class	cmlabs::TelnetProtocol
	Telnet line protocol handler for interactive text connections. More...

Functions
THREAD_RET THREAD_FUNCTION_CALL	cmlabs::TCPListenerRun (THREAD_ARG arg)
	Thread entry point for a TCPListener's internal accept loop.
THREAD_RET THREAD_FUNCTION_CALL	cmlabs::TCPConnectionRun (THREAD_ARG arg)
	Thread entry point for a TCPConnection's push-mode reader loop.
THREAD_RET THREAD_FUNCTION_CALL	cmlabs::UDPConnectionRun (THREAD_ARG arg)
	Thread entry point for a UDPConnection's push-mode reader loop.
bool	cmlabs::NetworkTest_TCPServer (uint16 port, uint32 count=0)
	Loopback test server: listens on port and echoes test payloads.
bool	cmlabs::NetworkTest_TCPClient (const char *address, uint16 port, uint32 count=0)
	Loopback test client matching NetworkTest_TCPServer().
bool	cmlabs::NetworkTest_SendReceiveData (TCPConnection *con, char *data, uint32 dataLen, uint32 c, bool receiveFirst=false)
	Send-and-verify helper used by the TCP tests.

Function Documentation

NetworkTest_SendReceiveData()

bool cmlabs::NetworkTest_SendReceiveData	(	TCPConnection *	con,
		char *	data,
		uint32	dataLen,
		uint32	c,
		bool	receiveFirst = false )

Send-and-verify helper used by the TCP tests.

Parameters

con	Connection under test.
data	Payload buffer.
dataLen	Payload size.
c	Iteration index.
receiveFirst	Receive before sending when true.

Returns

true if the exchanged data matched.

Definition at line 2968 of file NetworkConnections.cpp.

References cmlabs::NetworkConnection::receive(), and cmlabs::TCPConnection::send().

NetworkTest_TCPClient()

bool cmlabs::NetworkTest_TCPClient	(	const char *	address,
		uint16	port,
		uint32	count = 0 )

Loopback test client matching NetworkTest_TCPServer().

Parameters

address	Server host.
port	Server port.
count	Exchanges (0 = default).

Returns

true when the test completes successfully.

Definition at line 2843 of file NetworkConnections.cpp.

References cmlabs::TCPConnection::connect(), GetTimeNow(), cmlabs::NetworkConnection::isConnected(), cmlabs::NetworkConnection::receive(), and cmlabs::TCPConnection::send().

NetworkTest_TCPServer()

bool cmlabs::NetworkTest_TCPServer	(	uint16	port,
		uint32	count = 0 )

Loopback test server: listens on port and echoes test payloads.

Parameters

port	TCP port to listen on.
count	Number of exchanges (0 = default).

Returns

true when the test completes successfully.

Definition at line 2766 of file NetworkConnections.cpp.

References cmlabs::TCPListener::acceptConnection(), cmlabs::TCPListener::init(), cmlabs::NetworkConnection::isConnected(), NOENC, cmlabs::NetworkConnection::receive(), and cmlabs::NetworkConnection::send().

TCPConnectionRun()

THREAD_RET THREAD_FUNCTION_CALL cmlabs::TCPConnectionRun

(

THREAD_ARG

arg

)

Thread entry point for a TCPConnection's push-mode reader loop.

Definition at line 2749 of file NetworkConnections.cpp.

TCPListenerRun()

THREAD_RET THREAD_FUNCTION_CALL cmlabs::TCPListenerRun

(

THREAD_ARG

arg

)

Thread entry point for a TCPListener's internal accept loop.

Definition at line 2744 of file NetworkConnections.cpp.

UDPConnectionRun()

THREAD_RET THREAD_FUNCTION_CALL cmlabs::UDPConnectionRun

(

THREAD_ARG

arg

)

Thread entry point for a UDPConnection's push-mode reader loop.

Definition at line 2754 of file NetworkConnections.cpp.