diff --git a/docs/CLIENT.md b/docs/CLIENT.md index 0f3f97528..6cd4f2321 100644 --- a/docs/CLIENT.md +++ b/docs/CLIENT.md @@ -2,11 +2,11 @@ SimpleX client libraries provide low-level protocol access to SimpleX routers. They implement the wire protocols ([SMP](../protocol/simplex-messaging.md), [XFTP](../protocol/xftp.md), [NTF](../protocol/push-notifications.md)) and handle connection lifecycle, but leave encryption, identity management, and connection orchestration to the application. -This is **Layer 2** of the [SimpleX Network architecture](../protocol/overview-tjr.md). Layer 1 is the routers themselves; Layer 3 is the [Agent](AGENT.md), which builds duplex encrypted connections on top of these libraries. +This is **Layer 2** of the [SimpleX Network architecture](../protocol/overview-tjr.md). Layer 1 is the routers themselves; Layer 3 is the [Agent](AGENT.md), which builds duplex encrypted connections on top of these libraries. For internal architecture diagrams (thread topology, command processing flows), see [`spec/clients.md`](../spec/clients.md). ## SMP Client -**Source**: [`Simplex.Messaging.Client`](../src/Simplex/Messaging/Client.hs) — **Module spec**: [`spec/modules/Simplex/Messaging/Client.md`](../spec/modules/Simplex/Messaging/Client.md) +**Source**: [`Simplex.Messaging.Client`](../src/Simplex/Messaging/Client.hs). For architecture and module specs, see [SMP Client](../spec/clients.md#smp-client-protocolclient). The SMP client connects to SMP routers and manages simplex messaging queues — the fundamental addressing primitive of the SimpleX Network. Each simplex queue is a unidirectional, ordered sequence of fixed-size packets (16,384 bytes) with separate cryptographic credentials for sending and receiving. The queue model and command set are defined in the [SMP protocol](../protocol/simplex-messaging.md). @@ -33,7 +33,7 @@ Routers are identified by the SHA-256 hash of their CA certificate fingerprint, ## XFTP Client -**Source**: [`Simplex.FileTransfer.Client`](../src/Simplex/FileTransfer/Client.hs) — **Module spec**: [`spec/modules/Simplex/FileTransfer/Client.md`](../spec/modules/Simplex/FileTransfer/Client.md) +**Source**: [`Simplex.FileTransfer.Client`](../src/Simplex/FileTransfer/Client.hs). For architecture and module specs, see [XFTP Client](../spec/clients.md#xftp-client). The XFTP client connects to XFTP routers and manages data packets — individually addressed blocks used for larger payload delivery. Data packets come in fixed sizes (64KB, 256KB, 1MB, 4MB), hiding the actual payload size. The XFTP protocol runs over HTTP/2, simplifying browser integration. The data packet lifecycle and command set are defined in the [XFTP protocol](../protocol/xftp.md). @@ -46,7 +46,7 @@ The XFTP client connects to XFTP routers and manages data packets — individual ## NTF Client -**Source**: [`Simplex.Messaging.Notifications.Client`](../src/Simplex/Messaging/Notifications/Client.hs) — **Module spec**: [`spec/modules/Simplex/Messaging/Notifications/Client.md`](../spec/modules/Simplex/Messaging/Notifications/Client.md) +**Source**: [`Simplex.Messaging.Notifications.Client`](../src/Simplex/Messaging/Notifications/Client.hs). For architecture and module specs, see [NTF Client](../spec/clients.md#ntf-client). The NTF client connects to NTF (notification) routers and manages push notification tokens and subscriptions. It implements the [Push Notifications protocol](../protocol/push-notifications.md). @@ -81,15 +81,3 @@ The following capabilities require the [Agent](AGENT.md) (Layer 3): - [SimpleX Messaging Protocol](../protocol/simplex-messaging.md) — SMP wire format, commands, and security properties - [XFTP Protocol](../protocol/xftp.md) — XFTP wire format, data packet lifecycle - [SimpleX Network overview](../protocol/overview-tjr.md) — architecture, trust model, and design rationale - -## Module specs - -- [SMP Client](../spec/modules/Simplex/Messaging/Client.md) — proxy forwarding, batching, connection lifecycle, keepalive -- [XFTP Client](../spec/modules/Simplex/FileTransfer/Client.md) — handshake, data packet operations, forward secrecy -- [NTF Client](../spec/modules/Simplex/Messaging/Notifications/Client.md) — token and subscription operations, batch commands -- [SMP Protocol types](../spec/modules/Simplex/Messaging/Protocol.md) — command types, queue addresses, message encoding -- [XFTP Protocol types](../spec/modules/Simplex/FileTransfer/Protocol.md) — data packet types, XFTP commands -- [NTF Protocol types](../spec/modules/Simplex/Messaging/Notifications/Protocol.md) — notification commands, token/subscription types -- [Transport](../spec/modules/Simplex/Messaging/Transport.md) — TLS transport, session handshake -- [HTTP/2 Client](../spec/modules/Simplex/Messaging/Transport/HTTP2/Client.md) — HTTP/2 transport layer -- [Crypto](../spec/modules/Simplex/Messaging/Crypto.md) — cryptographic primitives used by clients diff --git a/spec/clients.md b/spec/clients.md new file mode 100644 index 000000000..871d1b4fd --- /dev/null +++ b/spec/clients.md @@ -0,0 +1,165 @@ +# Client Architecture + +SimpleX clients are the Layer 2 libraries that connect to routers. This document shows their internal architecture: component topology and command processing flows. + +For deployment and usage, see [docs/CLIENT.md](../docs/CLIENT.md). For protocol specifications, see [SMP](../protocol/simplex-messaging.md), [XFTP](../protocol/xftp.md), [Push Notifications](../protocol/push-notifications.md). + +--- + +## SMP Client (ProtocolClient) + +**Module specs**: [Client](modules/Simplex/Messaging/Client.md) · [Protocol](modules/Simplex/Messaging/Protocol.md) · [Transport](modules/Simplex/Messaging/Transport.md) · [Crypto](modules/Simplex/Messaging/Crypto.md) + +Generic protocol client used for both SMP and NTF connections. Manages a single TLS connection with multiplexed command/response matching via correlation IDs. + +### Component topology + +![SMP Client — Component Topology](diagrams/smp-client.svg) + +### Command/response flow + +```mermaid +sequenceDiagram + participant C as Caller
(Agent / router) + + box ProtocolClient + participant SC as sentCommands
(TMap CorrId Request) + participant SQ as sndQ + participant S as send thread + participant R as receive thread + participant RQ as rcvQ + participant P as process thread + end + + participant Router as SMP Router + + C->>SC: mkTransmission (generate CorrId, create Request with empty responseVar) + C->>SQ: write (Request, encoded command) + S->>SQ: read + S-->>S: check pending flag (drop if timed out) + S->>Router: tPutLog (transmit bytes) + + Router->>R: tGetClient (receive batch) + R->>RQ: write transmissions + + P->>RQ: read + P->>SC: lookup CorrId + alt command response (CorrId matches, pending) + P->>SC: remove CorrId + fill responseVar (TMVar) + else expired response (CorrId matches, already timed out) + P->>C: write to msgQ (STResponse) + else server event (empty CorrId) + P->>C: write to msgQ (STEvent) + end + + Note over C: getResponse: takeTMVar with timeout +``` + +--- + +## SMPClientAgent + +**Module specs**: [Client Agent](modules/Simplex/Messaging/Client/Agent.md) + +Connection manager that multiplexes multiple ProtocolClient connections. Tracks subscriptions, handles reconnection with backoff, and forwards server messages and connection events upward. Used by SMP router (proxying) and NTF router (subscriptions). + +### Component topology + +![SMPClientAgent — Component Topology](diagrams/smp-client-agent.svg) + +### Connection lifecycle + +```mermaid +sequenceDiagram + participant C as Consumer
(router / app) + participant A as SMPClientAgent + participant PC as ProtocolClient + participant Router as SMP Router + + C->>A: getSMPServerClient'' (server) + alt client exists in smpClients + A->>C: return existing client + else no client + A->>PC: connectClient (create new ProtocolClient) + PC->>Router: TLS handshake + A->>A: register disconnect handler + A->>C: return new client + end + + C->>A: subscribeQueuesNtfs (queueIds) + A->>A: add to pendingQueueSubs + A->>PC: sendProtocolCommands (SUB batch) + PC->>Router: SUB commands + Router->>PC: OK responses + A->>A: move pending → activeQueueSubs + A->>C: CASubscribed (via agentQ) + + Note over Router: connection drops + + PC->>A: disconnect handler fires + A->>A: filter by SessionId (only remove subs matching disconnected session) + A->>A: move active → pending (queue subs + service subs) + A->>C: CAServiceDisconnected (via agentQ, if service sub existed) + A->>C: CADisconnected (via agentQ, if queue subs existed) + A->>A: spawn smpSubWorker (retry with backoff) + A->>PC: reconnect + resubscribe pending subs + A->>C: CAConnected + CASubscribed (via agentQ) +``` + +--- + +## XFTP Client + +**Module specs**: [Client](modules/Simplex/FileTransfer/Client.md) · [Protocol](modules/Simplex/FileTransfer/Protocol.md) · [HTTP/2 Client](modules/Simplex/Messaging/Transport/HTTP2/Client.md) + +Stateless wrapper around HTTP2Client. XFTPClient adds no threads of its own — each operation is a synchronous HTTP/2 request/response. Serialization and multiplexing happen inside HTTP2Client's internal request queue and process thread. + +### Component topology + +![XFTP Client — Component Topology](diagrams/xftp-client.svg) + +### Upload/download flow + +```mermaid +sequenceDiagram + participant C as Caller
(Agent / app) + participant X as XFTPClient + participant H as HTTP2Client + participant Router as XFTP Router + + C->>X: createXFTPChunk (FNEW) + X->>H: HTTP/2 POST (encoded command) + H->>Router: request + Router->>H: response (sender ID + recipient IDs) + H->>X: decode response + X->>C: return IDs + + C->>X: uploadXFTPChunk (FPUT + file data) + X->>H: HTTP/2 POST (streaming body) + H->>Router: request with file stream + Router->>H: OK + H->>X: OK + X->>C: return OK + + C->>X: downloadXFTPChunk (FGET + ephemeral DH key) + X->>H: HTTP/2 POST (command) + H->>Router: request + Router->>H: streaming response (server DH key + nonce + encrypted data) + H->>X: streaming body + X->>X: compute DH secret, decrypt + save to file + X->>C: return () +``` + +--- + +## NTF Client + +**Module specs**: [Client](modules/Simplex/Messaging/Notifications/Client.md) · [Protocol](modules/Simplex/Messaging/Notifications/Protocol.md) + +Type alias for ProtocolClient — same architecture as SMP Client: + +```haskell +type NtfClient = ProtocolClient NTFVersion ErrorType NtfResponse +``` + +Same threads (send, receive, process, monitor), same queues (sndQ, rcvQ, sentCommands, msgQ), same command/response flow. Different command types: TNEW, TVFY, TCHK, TRPL, TDEL, TCRN, SNEW, SCHK, SDEL, PING. diff --git a/spec/diagrams/smp-client-agent.svg b/spec/diagrams/smp-client-agent.svg new file mode 100644 index 000000000..4257726ec --- /dev/null +++ b/spec/diagrams/smp-client-agent.svg @@ -0,0 +1,146 @@ + + + + + + + + + + + + SMPClientAgent -- Component Topology + + + consumer + (NTF router / + SMP proxy / + application) + + + + msgQ + (TBQueue, server messages) + + + + + + + agentQ + (TBQueue SMPClientAgentEvent) + + + + + + CAConnected + CADisconnected + CASubscribed / CASubError + CAServiceDisconnected + CAServiceSubscribed / SubError + + + + SMPClientAgent (connection manager) + + + + smpClients + (TMap SMPServer SMPClientVar) + + + + activeQueueSubs / pendingQueueSubs + (TMap SMPServer (TMap QueueId ...)) + activeServiceSubs / pendingServiceSubs + (TMap SMPServer (TVar (Maybe ...))) + + + + + + + + + + smpSubWorkers + (one per server) + + + + reconnect + resubscribe + + + getSMPServerClient'': get or create client + connectClient: create ProtocolClient, register disconnect handler + on disconnect: filter by SessionId, move active → pending, notify agentQ, spawn worker + worker: retry connect with backoff, resubscribe pending subs + subscribeQueuesNtfs / subscribeServiceNtfs: subscribe + track state + + + + ProtocolClient connections (one per SMP Router) + + + + ProtocolClient + → SMP Router A + + + + ProtocolClient + → SMP Router B + + + + ProtocolClient + → SMP Router N + + + ... + + + + + + + + + + ProtocolClient + + + state / queue + + + background worker + + + Solid arrows: TBQueue flow. Dashed: reconnection / resubscription. + + + diff --git a/spec/diagrams/smp-client.svg b/spec/diagrams/smp-client.svg new file mode 100644 index 000000000..d537ec8d0 --- /dev/null +++ b/spec/diagrams/smp-client.svg @@ -0,0 +1,149 @@ + + + + + + + + + + + + SMP Client (ProtocolClient) -- Component Topology + + + + per connection (raceAny_ -- any thread exit tears down connection) + + + caller + (Agent/ + router) + + + + commands + + + + sndQ + (TBQueue, 64) + + + + + + + send + + + + + + + receive + + + + + + SMP + Router + (TLS) + + + + + + + rcvQ + (TBQueue, 64) + + + + + + + process + + + + sentCommands + (TMap CorrId Request) + + + + match + + + + responseVar + (TMVar) + + + + monitor + + + + PING + + + optional + + + + msgQ (optional) + (TBQueue, server events) + + + + events (empty CorrId) + + + + to Agent / SMPClientAgent + + + + + + thread + + + queue / state + + + optional + + + Solid arrows: TBQueue flow. Dashed: STM lookups / TMVar responses. + + + diff --git a/spec/diagrams/xftp-client.svg b/spec/diagrams/xftp-client.svg new file mode 100644 index 000000000..847922ed9 --- /dev/null +++ b/spec/diagrams/xftp-client.svg @@ -0,0 +1,83 @@ + + + + + + + + + + + + XFTP Client -- Component Topology + + + + per connection (XFTPClient adds no threads; serialization in HTTP2Client) + + + caller + (Agent/ + router) + + + + + + + XFTPClient + sendXFTPCommand + + + + + + + HTTP2Client + (TLS + HTTP/2 streams) + + + + XFTP + Router + (HTTP/2) + + + + thParams (negotiated) + + + uploads: streaming request body + downloads: ephemeral DH + streaming + response body (per-chunk forward secrecy) + + + + + + client wrapper + + + external connection + + + state + + + XFTPClient adds no threads. HTTP2Client has internal reqQ + process thread. + + +