Named, node-owned connection to an LLM endpoint. More...

#include <LLMConnection.h>

Public Member Functions
	LLMConnection ()
	~LLMConnection ()
void	setOwner (void *owner, LLMOwnerEndCallback callback)
	Set, or clear with NULLs, the owner-end callback.
LLMResult	configure (const XMLNode &llmNode)
	Configure from a parsed `<llm>` element.
LLMResult	configureFromString (const char *xml)
	Convenience overload: parse the `<llm>` element from an XML string, then configure().
bool	isConfigured () const
	`true` once configure() has succeeded.
LLMResult	getLastError () const
	Most recent typed error recorded by this connection.
const std::string &	getName () const
	Connection name from `<llm name="...">`; code looks connections up by this name.
LLMVendorType	getVendorType () const
const std::string &	getVendorTypeName () const
const std::string &	getEndpoint () const
bool	isGlobal () const
	Value of the `global` attribute.
const std::string &	getSchemaVersion () const
const std::string &	getAPIKeyRef () const
	The credential reference text, e.g.
bool	hasResolvedSecret () const
	`true` if `apikeyref` resolved to a secret.
const std::string &	getRequestTemplate () const
const std::string &	getResponseTemplate () const
const std::vector< LLMHeader > &	getHeaders () const
const LLMConstraints &	getConstraints () const
std::vector< std::string >	listModels () const
	Model names from the parsed `<models>` list.
LLMResult	selectModel (const char *modelName)
	Select a model by name from the parsed list.
const std::string &	getSelectedModel () const
	Currently selected model (the `<models default="...">` value until selectModel() is called).
LLMResult	connect ()
	Validate configuration and resolved secret, and mark the connection live.
LLMResult	reset ()
	Drop the live connection.
LLMResult	interact (const char *input, std::string &reply)
	One blocking request/reply exchange (mode A).
LLMResult	interactStream (const char input, LLMStreamCallback callback, void userData, std::string *fullReply=NULL)
	Streaming request (mode B): delivers decoded tokens as they arrive.
bool	isConnected () const
	`true` between a successful connect() and reset().
LLMStreamTransport	getStreamTransport () const
	Streaming transport resolved at configure() time.
LLMStats	stats () const
	Snapshot of this connection's statistics.

Detailed Description

Named, node-owned connection to an LLM endpoint.

See also: LLMConnection.h for the XML shape, credential rules and call modes.

Definition at line 235 of file LLMConnection.h.

Constructor & Destructor Documentation

◆ LLMConnection()

cmlabs::LLMConnection::LLMConnection ( )

Definition at line 62 of file LLMConnection.cpp.

References cmlabs::LLM_OK, cmlabs::LLM_STREAM_NONE, and cmlabs::LLM_VENDOR_NONE.

◆ ~LLMConnection()

cmlabs::LLMConnection::~LLMConnection ( )

Definition at line 70 of file LLMConnection.cpp.

References cmlabs::GetTimeNow().

Member Function Documentation

◆ configure()

LLMResult cmlabs::LLMConnection::configure ( const XMLNode & llmNode )

Configure from a parsed <llm> element.

Resolves the credential reference (see LLMConnection.h) and the stream transport. On any error the connection stays or becomes unconfigured and a typed error is returned; bad configuration never crashes.

Parameters

llmNode The <llm> element, containing one <llmvendor> sub-block.

Returns: ::LLM_OK, or a typed configuration error such as ::LLM_ERR_CONFIG_NO_VENDOR or ::LLM_ERR_CONFIG_INLINE_SECRET.

Definition at line 197 of file LLMConnection.cpp.

References cmlabs::attrTrue(), cmlabs::XMLNode::getAttribute(), cmlabs::XMLNode::getChildNode(), cmlabs::XMLNode::getName(), cmlabs::XMLNode::isEmpty(), cmlabs::LLM_ERR_CONFIG_EMPTY, cmlabs::LLM_ERR_CONFIG_INLINE_SECRET, cmlabs::LLM_ERR_CONFIG_NO_NAME, cmlabs::LLM_ERR_CONFIG_NO_VENDOR, cmlabs::LLM_ERR_CONFIG_NOT_LLM, cmlabs::LLM_OK, cmlabs::XMLNode::nChildNode(), and stricmp.

Referenced by configureFromString().

◆ configureFromString()

LLMResult cmlabs::LLMConnection::configureFromString ( const char * xml )

Convenience overload: parse the <llm> element from an XML string, then configure().

Parameters

xml	XML text containing the `<llm>` element.

Returns: ::LLM_OK, or ::LLM_ERR_CONFIG_PARSE if the string does not parse, or any error from configure().

Definition at line 188 of file LLMConnection.cpp.

References configure(), cmlabs::XMLResults::error, cmlabs::eXMLErrorNone, cmlabs::XMLNode::isEmpty(), cmlabs::LLM_ERR_CONFIG_EMPTY, and cmlabs::LLM_ERR_CONFIG_PARSE.

◆ connect()

LLMResult cmlabs::LLMConnection::connect ( )

Validate configuration and resolved secret, and mark the connection live.

Starts a fresh stats record. The underlying HTTP client is connectionless/blocking, so there is nothing to dial eagerly: endpoint reachability is verified lazily by the first interact().

Returns: ::LLM_OK, ::LLM_ERR_NOT_CONFIGURED, or ::LLM_ERR_SECRET_UNRESOLVED if apikeyref is set but no secret could be resolved.

Definition at line 396 of file LLMConnection.cpp.

References cmlabs::GetTimeNow(), cmlabs::LLM_ERR_NOT_CONFIGURED, cmlabs::LLM_ERR_SECRET_UNRESOLVED, and cmlabs::LLM_OK.

◆ getAPIKeyRef()

const std::string & cmlabs::LLMConnection::getAPIKeyRef ( ) const

inline

The credential reference text, e.g.

env:MY_KEY.

Note: This is the reference, never the secret value.

Definition at line 285 of file LLMConnection.h.

◆ getConstraints()

const LLMConstraints & cmlabs::LLMConnection::getConstraints ( ) const

inline

Definition at line 293 of file LLMConnection.h.

◆ getEndpoint()

const std::string & cmlabs::LLMConnection::getEndpoint ( ) const

inline

Definition at line 277 of file LLMConnection.h.

◆ getHeaders()

const std::vector< LLMHeader > & cmlabs::LLMConnection::getHeaders ( ) const

inline

Definition at line 292 of file LLMConnection.h.

◆ getLastError()

LLMResult cmlabs::LLMConnection::getLastError ( ) const

inline

Most recent typed error recorded by this connection.

Definition at line 269 of file LLMConnection.h.

◆ getName()

const std::string & cmlabs::LLMConnection::getName ( ) const

inline

Connection name from <llm name="...">; code looks connections up by this name.

Definition at line 274 of file LLMConnection.h.

◆ getRequestTemplate()

const std::string & cmlabs::LLMConnection::getRequestTemplate ( ) const

inline

Definition at line 290 of file LLMConnection.h.

◆ getResponseTemplate()

const std::string & cmlabs::LLMConnection::getResponseTemplate ( ) const

inline

Definition at line 291 of file LLMConnection.h.

◆ getSchemaVersion()

const std::string & cmlabs::LLMConnection::getSchemaVersion ( ) const

inline

Definition at line 282 of file LLMConnection.h.

◆ getSelectedModel()

const std::string & cmlabs::LLMConnection::getSelectedModel ( ) const

inline

Currently selected model (the <models default="..."> value until selectModel() is called).

Definition at line 307 of file LLMConnection.h.

◆ getStreamTransport()

LLMStreamTransport cmlabs::LLMConnection::getStreamTransport ( ) const

inline

Streaming transport resolved at configure() time.

Returns: ::LLM_STREAM_NONE if unconfigured or the vendor does not stream.

Definition at line 357 of file LLMConnection.h.

◆ getVendorType()

LLMVendorType cmlabs::LLMConnection::getVendorType ( ) const

inline

Definition at line 275 of file LLMConnection.h.

◆ getVendorTypeName()

const std::string & cmlabs::LLMConnection::getVendorTypeName ( ) const

inline

Definition at line 276 of file LLMConnection.h.

◆ hasResolvedSecret()

bool cmlabs::LLMConnection::hasResolvedSecret ( ) const

inline

true if apikeyref resolved to a secret.

Warning: The secret value itself is never exposed through the public API.

Definition at line 289 of file LLMConnection.h.

◆ interact()

LLMResult cmlabs::LLMConnection::interact	(	const char *	input,
		std::string &	reply )

One blocking request/reply exchange (mode A).

Shapes the request from <requesttemplate> and the configured headers, sends it, and extracts the reply text via the <responsetemplate> JSON path.

Parameters

[in]	input	Prompt text; must not be empty.
[out]	reply	Extracted reply text on success.

Returns: ::LLM_OK, ::LLM_ERR_NOT_CONNECTED, ::LLM_ERR_NO_INPUT, ::LLM_ERR_HTTP_UNREACHABLE, ::LLM_ERR_HTTP or ::LLM_ERR_REPLY_PARSE.

Warning: Blocks the calling thread until the exchange completes.

Definition at line 696 of file LLMConnection.cpp.

References cmlabs::HTTPReply::getContent(), HTTP_MALFORMED_URL, HTTP_OK, HTTP_POST, HTTP_SERVER_UNAVAILABLE, cmlabs::LLM_ERR_HTTP, cmlabs::LLM_ERR_HTTP_UNREACHABLE, cmlabs::LLM_ERR_NO_INPUT, cmlabs::LLM_ERR_NOT_CONFIGURED, cmlabs::LLM_ERR_NOT_CONNECTED, cmlabs::LLM_ERR_REPLY_PARSE, cmlabs::LLM_OK, cmlabs::llmJSONUnescape(), and cmlabs::HTTPReply::type.

◆ interactStream()

LLMResult cmlabs::LLMConnection::interactStream	(	const char *	input,
		LLMStreamCallback	callback,
		void *	userData,
		std::string *	fullReply = NULL )

Streaming request (mode B): delivers decoded tokens as they arrive.

The wire decoder (SSE, AWS event-stream or raw chunked) is chosen from the transport resolved at configure() time, so callers never branch on vendor.

Parameters

[in]	input	Prompt text; must not be empty.
[in]	callback	Sink for incremental tokens; return `false` to cancel.
[in]	userData	Opaque pointer passed through to `callback`.
[out]	fullReply	If non-NULL, receives the assembled full text.

Returns: ::LLM_OK, ::LLM_ERR_STREAM_CANCELLED if the sink cancelled, or any error interact() can return.

Warning: Blocks until the stream completes, errors, or the callback cancels.

Definition at line 639 of file LLMConnection.cpp.

References cmlabs::SSLConnection::connect(), cmlabs::TCPConnection::connect(), cmlabs::HTTPRequest::createRequest(), cmlabs::NetworkConnection::disconnect(), cmlabs::html::GetHostFromURL(), cmlabs::html::GetPortFromURL(), cmlabs::html::GetProtocolFromURL(), cmlabs::html::GetURIFromURL(), HTTP_POST, cmlabs::LLM_ERR_CONFIG_BAD_TRANSPORT, cmlabs::LLM_ERR_CONFIG_INVALID, cmlabs::LLM_ERR_HTTP_UNREACHABLE, cmlabs::LLM_ERR_NO_INPUT, cmlabs::LLM_ERR_NOT_CONFIGURED, cmlabs::LLM_ERR_NOT_CONNECTED, cmlabs::LLM_OK, cmlabs::LLM_STREAM_NONE, cmlabs::HTTPProtocol::SendHTTPRequest(), and stricmp.

◆ isConfigured()

bool cmlabs::LLMConnection::isConfigured ( ) const

inline

true once configure() has succeeded.

Definition at line 267 of file LLMConnection.h.

◆ isConnected()

bool cmlabs::LLMConnection::isConnected ( ) const

inline

true between a successful connect() and reset().

Definition at line 353 of file LLMConnection.h.

◆ isGlobal()

bool cmlabs::LLMConnection::isGlobal ( ) const

inline

Value of the global attribute.

Warning: Parsed only: master-only proxying of a global connection is NOT implemented.

Definition at line 281 of file LLMConnection.h.

◆ listModels()

std::vector< std::string > cmlabs::LLMConnection::listModels ( ) const

Model names from the parsed <models> list.

Returns: Empty vector if the connection is unconfigured.

Definition at line 275 of file LLMConnection.cpp.

◆ reset()

LLMResult cmlabs::LLMConnection::reset ( )

Drop the live connection.

Configuration is kept and the stats record is retained with LLMStats::ended set. Fires the owner-end callback if one is registered, and aborts any live stream.

Returns: ::LLM_OK.

Definition at line 411 of file LLMConnection.cpp.

References cmlabs::GetTimeNow(), cmlabs::LLM_ERR_NOT_CONFIGURED, and cmlabs::LLM_OK.

◆ selectModel()

LLMResult cmlabs::LLMConnection::selectModel ( const char * modelName )

Select a model by name from the parsed list.

Parameters

modelName Must appear in listModels().

Returns: ::LLM_OK, or ::LLM_ERR_UNKNOWN_MODEL if the name is not in the list.

Definition at line 279 of file LLMConnection.cpp.

References cmlabs::LLM_ERR_NOT_CONFIGURED, cmlabs::LLM_ERR_UNKNOWN_MODEL, cmlabs::LLM_OK, and stricmp.

◆ setOwner()

void cmlabs::LLMConnection::setOwner	(	void *	owner,
		LLMOwnerEndCallback	callback )

Set, or clear with NULLs, the owner-end callback.

Parameters

owner	Opaque handle passed back to `callback`.
callback	Invoked at most once per connect()..end cycle, from reset() or the destructor.

Definition at line 79 of file LLMConnection.cpp.

◆ stats()

LLMStats cmlabs::LLMConnection::stats ( ) const

Snapshot of this connection's statistics.

Returns: Current record while live; the retained final record (with LLMStats::ended set) after reset().

Definition at line 751 of file LLMConnection.cpp.

References cmlabs::GetTimeNow(), and cmlabs::LLMStats::uptimeMS.

The documentation for this class was generated from the following files:

/home/ubuntu/c/partner/psyclone2/CMSDK/include/LLMConnection.h
/home/ubuntu/c/partner/psyclone2/CMSDK/src/LLMConnection.cpp

Public Member Functions

Detailed Description

Constructor & Destructor Documentation

◆ LLMConnection()

◆ ~LLMConnection()

Member Function Documentation

◆ configure()

◆ configureFromString()

◆ connect()

◆ getAPIKeyRef()

◆ getConstraints()

◆ getEndpoint()

◆ getHeaders()

◆ getLastError()

◆ getName()

◆ getRequestTemplate()

◆ getResponseTemplate()

◆ getSchemaVersion()

◆ getSelectedModel()

◆ getStreamTransport()

◆ getVendorType()

◆ getVendorTypeName()

◆ hasResolvedSecret()

◆ interact()

◆ interactStream()

◆ isConfigured()

◆ isConnected()

◆ isGlobal()

◆ listModels()

◆ reset()

◆ selectModel()

◆ setOwner()

◆ stats()