diff --git a/docs/AGENT.md b/docs/AGENT.md index 83050013f..415ba08f5 100644 --- a/docs/AGENT.md +++ b/docs/AGENT.md @@ -2,7 +2,7 @@ The SimpleX Agent builds duplex encrypted connections on top of [SimpleX client libraries](CLIENT.md). It manages the full lifecycle of secure communication: connection establishment, end-to-end encryption, queue rotation, file transfer, and push notifications. -This is **Layer 3** of the [SimpleX Network architecture](../protocol/overview-tjr.md). Layer 1 is the routers; Layer 2 is the [client libraries](CLIENT.md) that speak the wire protocols. The Agent adds the connection semantics that applications need. +This is **Layer 3** of the [SimpleX Network architecture](../protocol/overview-tjr.md). Layer 1 is the routers; Layer 2 is the [client libraries](CLIENT.md) that speak the wire protocols. The Agent adds the connection semantics that applications need. For internal architecture diagrams (thread topology, message processing flows), see [`spec/agent.md`](../spec/agent.md). **Source**: [`Simplex.Messaging.Agent`](../src/Simplex/Messaging/Agent.hs). **Module spec**: [`spec/modules/Simplex/Messaging/Agent.md`](../spec/modules/Simplex/Messaging/Agent.md) diff --git a/spec/agent.md b/spec/agent.md index 250bf2253..f77ccd4a4 100644 --- a/spec/agent.md +++ b/spec/agent.md @@ -1,13 +1,133 @@ -# SMP Agent +# Agent Architecture -> SMP agent implementation: duplex connections, queue rotation, ratchet sync, and notification subscriptions. +The SimpleX Agent is the Layer 3 connection manager. It builds duplex encrypted connections on top of Layer 2 client libraries. This document shows its internal architecture: component topology and message processing flows. -## Duplex Connections +For usage and API overview, see [docs/AGENT.md](../docs/AGENT.md). For protocol specifications, see [Agent Protocol](../protocol/agent-protocol.md), [PQDR](../protocol/pqdr.md). -## Queue Rotation +--- -## Ratchet Sync +**Module specs**: [Agent](modules/Simplex/Messaging/Agent.md) · [Agent Client](modules/Simplex/Messaging/Agent/Client.md) · [Agent Protocol](modules/Simplex/Messaging/Agent/Protocol.md) · [Store Interface](modules/Simplex/Messaging/Agent/Store/Interface.md) · [NtfSubSupervisor](modules/Simplex/Messaging/Agent/NtfSubSupervisor.md) · [XFTP Agent](modules/Simplex/FileTransfer/Agent.md) · [Ratchet](modules/Simplex/Messaging/Crypto/Ratchet.md) -## Notification Subscriptions +### Component topology -## Functions +![Agent - Component Topology](diagrams/agent.svg) + +### Message receive flow + +```mermaid +sequenceDiagram + participant R as SMP Router + + box Agent + participant SC as smpClients
(ProtocolClient pool) + participant MQ as msgQ
(TBQueue) + participant S as subscriber + participant St as Store + participant SQ as subQ
(TBQueue) + end + + participant App as Application + + R->>SC: MSG (encrypted packet) + SC->>MQ: write batch + + S->>MQ: read batch + S->>S: withConnLock
(serialize per connection) + S->>St: load ratchet state
(lockConnForUpdate) + S->>S: agentRatchetDecrypt
(double ratchet) + S->>S: checkMsgIntegrity
(sequence + hash chain) + S->>St: store received message,
update ratchet + S->>SQ: write AEvt (MSG + metadata) + + App->>SQ: read event + + Note over App: application processes message + + App->>S: ackMessage (agentMsgId) + Note over S,R: ACK is async
(enqueued as internal command) + S->>SC: ACK + SC->>R: ACK +``` + +### Message send flow + +```mermaid +sequenceDiagram + participant App as Application + + box Agent + participant API as sendMessage + participant St as Store + participant DW as deliveryWorker
(per send queue) + participant SC as smpClients
(ProtocolClient pool) + end + + participant R as SMP Router + + App->>API: sendMessage(connId, body) + API->>St: agentRatchetEncryptHeader
(advance ratchet, store
encrypt key + pending message) + API->>DW: signal doWork (TMVar) + API->>App: return msgId + + DW->>St: getPendingQueueMsg + DW->>DW: rcEncryptMsg
(encrypt body with stored key) + DW->>DW: encode AgentMsgEnvelope + DW->>SC: sendAgentMessage
(per-queue encrypt + SEND) + SC->>R: SEND (encrypted packet) + R->>SC: OK + + DW->>St: delete pending message + DW->>App: SENT msgId (via subQ) +``` + +### Connection establishment flow + +```mermaid +sequenceDiagram + participant A as Alice (initiator) + + box Agent A + participant AA as Agent + end + + participant SMP as SMP Router + + box Agent B + participant AB as Agent + end + + participant B as Bob (joiner) + + A->>AA: createConnection + AA->>SMP: NEW (Alice's receive queue) + SMP->>AA: queue ID + keys + AA->>A: invitation URI
(queue address + DH keys) + + Note over A,B: invitation passed out-of-band
(QR code, link) + + B->>AB: joinConnection(invitation) + AB->>AB: initSndRatchet
(PQ X3DH key agreement) + AB->>SMP: NEW (Bob's receive queue) + SMP->>AB: queue ID + AB->>SMP: KEY (secure Alice's queue) + AB->>SMP: SEND confirmation to
Alice's queue (Bob's queue
address + ratchet keys) + + SMP->>AA: MSG (confirmation) + AA->>AA: initRcvRatchet
(PQ X3DH key agreement),
decrypt confirmation + AA->>A: CONF (request approval) + A->>AA: allowConnection(confId) + AA->>SMP: SKEY (secure Alice's rcv queue) + AA->>SMP: NEW (Alice's send queue) + AA->>SMP: SEND reply to Bob's queue
(Alice's connection info) + + SMP->>AB: MSG (reply) + AB->>SMP: SKEY (secure Bob's rcv queue) + AB->>SMP: SEND HELLO to Alice + + SMP->>AA: MSG (HELLO) + AA->>SMP: SEND HELLO to Bob + AA->>A: CON (connected) + + SMP->>AB: MSG (HELLO) + AB->>B: CON (connected) +``` diff --git a/spec/clients.md b/spec/clients.md index a99686ab0..10634d0de 100644 --- a/spec/clients.md +++ b/spec/clients.md @@ -122,7 +122,7 @@ Stateless wrapper around HTTP2Client. XFTPClient adds no threads of its own; eac ![XFTP Client - Component Topology](diagrams/xftp-client.svg) -### Upload/download flow +### Packet delivery flow ```mermaid sequenceDiagram diff --git a/spec/diagrams/agent.svg b/spec/diagrams/agent.svg new file mode 100644 index 000000000..4b7bf802c --- /dev/null +++ b/spec/diagrams/agent.svg @@ -0,0 +1,202 @@ + + + + + + + + + + + + + Application + + + + subQ + + + + main threads (raceAny_: any exit tears down all) + + + + subscriber + (reads msgQ) + + + ntfSubQ + + + + + ntfSupervisor + (reads ntfSubQ) + + + + cleanupManager + (periodic cleanup) + + + + logServersStats + (periodic stats) + + + + worker pools (on-demand, one per queue/connection/server) + + + + delivery + (per send queue) + + + asyncCmd + (per connection) + + + smpSub + (per session) + + + + xftpRcv + (per server) + + + xftpSnd + (per server) + + + xftpDel + (per server) + + + + ntfSMP + (per SMP server) + + + ntfWorkers + (per NTF server) + + + ntfTknDel + (per NTF server) + + + + ntf workers (dispatched by ntfSupervisor) + + + + + + + + store + + + + Store + (SQLite / Postgres) + + + currentSubs + (TSessionSubs) + + + Operation State + (5-op suspension cascade) + + + + protocol client pools (lazy singleton per router) + + + smpClients + (TMap SMPTransportSession) + + + xftpClients + (TMap XFTPTransportSession) + + + ntfClients + (TMap NtfTransportSession) + + + SMP Routers + + + XFTP Routers + + + NTF Routers + + + + + msgQ + + + + + + on-demand worker + + + singleton thread + + + storage / state + + + external connection + + + Solid arrows: TBQueue connections. Dashed: store access / dispatch. Workers connect to protocol clients in their column. + + +