From 3bde77da10a428f66264692f76a7b604aa23bfc7 Mon Sep 17 00:00:00 2001 From: "Evgeny @ SimpleX Chat" <259188159+evgeny-simplex@users.noreply.github.com> Date: Fri, 13 Mar 2026 17:17:31 +0000 Subject: [PATCH] update terms --- spec/TOPICS.md | 20 ++++++------- spec/modules/Simplex/FileTransfer/Agent.md | 18 +++++------ spec/modules/Simplex/FileTransfer/Client.md | 8 ++--- .../Simplex/FileTransfer/Client/Agent.md | 6 ++-- .../Simplex/FileTransfer/Client/Main.md | 18 +++++------ .../Simplex/FileTransfer/Description.md | 6 ++-- spec/modules/Simplex/FileTransfer/Protocol.md | 2 +- spec/modules/Simplex/FileTransfer/Server.md | 28 ++++++++--------- .../Simplex/FileTransfer/Server/Env.md | 6 ++-- .../Simplex/FileTransfer/Server/Main.md | 8 ++--- .../Simplex/FileTransfer/Server/Stats.md | 6 ++-- .../Simplex/FileTransfer/Server/Store.md | 2 +- .../Simplex/FileTransfer/Server/StoreLog.md | 6 ++-- spec/modules/Simplex/FileTransfer/Types.md | 2 +- spec/modules/Simplex/Messaging/Agent.md | 2 +- .../modules/Simplex/Messaging/Agent/Client.md | 30 +++++++++---------- .../Simplex/Messaging/Agent/Env/SQLite.md | 2 +- .../Messaging/Agent/NtfSubSupervisor.md | 10 +++---- .../Simplex/Messaging/Agent/Protocol.md | 4 +-- spec/modules/Simplex/Messaging/Agent/Stats.md | 2 +- .../Messaging/Notifications/Protocol.md | 6 ++-- .../Simplex/Messaging/Notifications/Server.md | 28 ++++++++--------- .../Messaging/Notifications/Server/Control.md | 2 +- .../Messaging/Notifications/Server/Env.md | 6 ++-- .../Messaging/Notifications/Server/Main.md | 2 +- .../Notifications/Server/Push/APNS.md | 2 +- .../Messaging/Notifications/Server/Stats.md | 14 ++++----- .../Messaging/Notifications/Server/Store.md | 4 +-- .../Notifications/Server/Store/Postgres.md | 8 ++--- .../Messaging/Notifications/Transport.md | 4 +-- .../Simplex/Messaging/Notifications/Types.md | 2 +- 31 files changed, 132 insertions(+), 132 deletions(-) diff --git a/spec/TOPICS.md b/spec/TOPICS.md index 0489d79df..a62e23c29 100644 --- a/spec/TOPICS.md +++ b/spec/TOPICS.md @@ -2,7 +2,7 @@ > Cross-cutting patterns noticed during module documentation. Each entry may become a topic doc in `spec/` after all module docs are complete. -- **Exception handling strategy**: `catchOwn`/`catchAll`/`tryAllErrors` pattern (defined in Util.hs) used across server, client, and agent modules. The three-category classification (synchronous, own-async, cancellation) and when to use which catch variant is not obvious from any single call site. +- **Exception handling strategy**: `catchOwn`/`catchAll`/`tryAllErrors` pattern (defined in Util.hs) used across router, client, and agent modules. The three-category classification (synchronous, own-async, cancellation) and when to use which catch variant is not obvious from any single call site. - **Padding schemes**: Three different padding formats across the codebase — Crypto.hs uses 2-byte Word16 length prefix (max ~65KB), Crypto/Lazy.hs uses 8-byte Int64 prefix (file-sized), and both use '#' fill character. Ratchet header padding uses fixed sizes (88 or 2310 bytes). All use `pad`/`unPad` but with incompatible formats. The relationship between padding, encryption, and message size limits spans Crypto, Lazy, Ratchet, and the protocol layer. @@ -16,17 +16,17 @@ - **Service certificate subscription model**: Service subscriptions (SUBS/NSUBS) and per-queue subscriptions (SUB/NSUB) coexist with complex state transitions. Client/Agent.hs manages dual active/pending subscription maps with session-aware cleanup. Protocol.hs defines useServiceAuth (only NEW/SUB/NSUB). Client.hs implements authTransmission with dual signing (entity key over cert hash + transmission, service key over transmission only). Transport.hs handles the service certificate handshake extension (v16+). The full subscription lifecycle — from DBService credentials through handshake to service subscription to disconnect/reconnect — spans all four modules. -- **Two agent layers**: Client/Agent.hs ("small agent") is used only in servers — SMP proxy and notification server — to manage client connections to other SMP servers. Agent.hs + Agent/Client.hs ("big agent") is used in client applications. Both manage SMP client connections with subscription tracking and reconnection, but the big agent adds the full messaging agent layer (connections, double ratchet, file transfer). When documenting Agent/Client.hs, Client/Agent.hs should be reviewed for shared patterns and differences. +- **Two agent layers**: Client/Agent.hs ("small agent") is used only in routers — SMP proxy and notification router — to manage client connections to other SMP routers. Agent.hs + Agent/Client.hs ("big agent") is used in client applications. Both manage SMP client connections with subscription tracking and reconnection, but the big agent adds the full messaging agent layer (connections, double ratchet, file transfer). When documenting Agent/Client.hs, Client/Agent.hs should be reviewed for shared patterns and differences. - **Handshake protocol family**: SMP (Transport.hs), NTF (Notifications/Transport.hs), and XFTP (FileTransfer/Transport.hs) all have handshake protocols with the same structure (version negotiation + session binding + key exchange) but different feature sets. NTF is a strict subset. XFTP doesn't use the TLS handshake at all (HTTP2 layer). The shared types (THandle, THandleParams, THandleAuth) mean changes to the handshake infrastructure affect all three protocols. -- **Server subscription architecture**: The SMP server's subscription model spans Server.hs (serverThread split-STM lifecycle, tryDeliverMessage sync/async, ProhibitSub/ServerSub state machine), Env/STM.hs (SubscribedClients TVar-of-Maybe continuity, Client three-queue architecture), and Client/Agent.hs (small agent dual subscription model). The interaction between service subscriptions, direct queue subscriptions, notification subscriptions, and the serverThread subQ processing is not visible from any single module. +- **Router subscription architecture**: The SMP router's subscription model spans Server.hs (serverThread split-STM lifecycle, tryDeliverMessage sync/async, ProhibitSub/ServerSub state machine), Env/STM.hs (SubscribedClients TVar-of-Maybe continuity, Client three-queue architecture), and Client/Agent.hs (small agent dual subscription model). The interaction between service subscriptions, direct queue subscriptions, notification subscriptions, and the serverThread subQ processing is not visible from any single module. - **Duplex connection handshake**: The SMP duplex connection procedure (standard 10-step and fast 7-step) spans Agent.hs (orchestration, state machine), Agent/Protocol.hs (message types: AgentConfirmation/AgentConnInfoReply/AgentInvitation/HELLO, queue status types), Client.hs (SMP command dispatch), Protocol.hs (SMP-level KEY/SKEY commands). The handshake involves two-layer encryption (per-queue E2E + double ratchet), version-dependent paths (v2+ duplex, v6+ sender auth key, v7+ ratchet on confirmation, v9+ fast handshake with SKEY), and the asymmetry between initiating and accepting parties (different message types, different confirmation processing). The protocol spec (`agent-protocol.md`) defines the procedure but the implementation details — error handling, state persistence across restarts, race conditions between confirmation and message delivery — are only visible by reading the code across these modules. - **Connection links**: Full connection links (URI format with `#/?` query parameters) and binary-encoded links (`Encoding` instances) serve different contexts — URIs for out-of-band sharing, binary for agent-to-agent messages. Each has independent version-conditional encoding with different backward-compat rules (URI parser adjusts agent version ranges for old contact links, binary parser patches `queueMode` for forward compat). The `VersionI`/`VersionRangeI` typeclasses convert between `SMPQueueInfo` (versioned, in confirmations) and `SMPQueueUri` (version-ranged, in links). Full picture requires Agent/Protocol.hs, Protocol.hs, and agent-protocol.md. -- **Short links**: Short links are a compact representation for sharing via URLs, not a replacement for full connection links — both are used. Short links store encrypted link data on the router and encode only a server hostname, link type character, and key hash in the URL. The link data lifecycle (creation, encryption with key derivation, owner chain-of-trust validation, mutable user data updates) spans Agent/Protocol.hs (types, serialization, owner validation, server shortening/restoration), Agent.hs (link creation and resolution API), and the router-side link storage. The `FixedLinkData`/`ConnLinkData` split (immutable vs mutable), `OwnerAuth` chain validation, and `PreparedLinkParams` pre-computation are not visible from any single module. +- **Short links**: Short links are a compact representation for sharing via URLs, not a replacement for full connection links — both are used. Short links store encrypted link data on the router and encode only a router hostname, link type character, and key hash in the URL. The link data lifecycle (creation, encryption with key derivation, owner chain-of-trust validation, mutable user data updates) spans Agent/Protocol.hs (types, serialization, owner validation, router shortening/restoration), Agent.hs (link creation and resolution API), and the router-side link storage. The `FixedLinkData`/`ConnLinkData` split (immutable vs mutable), `OwnerAuth` chain validation, and `PreparedLinkParams` pre-computation are not visible from any single module. - **Agent worker framework**: `getAgentWorker` (lifecycle, restart rate limiting, crash recovery) + `withWork`/`withWork_`/`withWorkItems` (task retrieval with doWork flag atomics) defined in Agent/Client.hs, consumed by Agent.hs (async commands, message delivery), NtfSubSupervisor.hs (notification workers), FileTransfer/Agent.hs (XFTP workers), and simplex-chat. The framework separates two concerns: worker lifecycle (create-or-reuse, fork async, rate-limit restarts, escalate to CRITICAL) and task pattern (get next task, do task, as separate parameters). The doWork TMVar flag choreography (clear before query to prevent race) and the work-item-error vs store-error distinction are not obvious from any single consumer. @@ -40,19 +40,19 @@ - **NTF push delivery pipeline**: Bounded TBQueue (`pushQ`) creates backpressure → `ntfPush` thread reads → `checkActiveTkn` gates PNMessage (but not PNVerification or PNCheckMessages) → APNS delivery with single retry on connection errors (new push client on retry) → PPTokenInvalid marks token NTInvalid. Spans [Server.hs](modules/Simplex/Messaging/Notifications/Server.md), [APNS.hs](modules/Simplex/Messaging/Notifications/Server/Push/APNS.md) (DER JWT signing, HTTP/2 serializing queue, fire-and-forget connection), [Env.hs](modules/Simplex/Messaging/Notifications/Server/Env.md) (push client caching with race tolerance). -- **NTF service subscription model**: Service-level subscriptions (SUBS/NSUBS on SMP) vs individual queue subscriptions, with fallback from service to individual when `CAServiceUnavailable`. Service credentials are lazily generated per SMP server with 25h backdating and ~2700yr validity. XOR hash triggers on PostgreSQL maintain subscription aggregate counts. Subscription status tracking uses `ntf_service_assoc` flag to distinguish service-associated from individually-subscribed queues. Spans [Server.hs](modules/Simplex/Messaging/Notifications/Server.md) (subscriber thread, service fallback), [Env.hs](modules/Simplex/Messaging/Notifications/Server/Env.md) (lazy credential generation, Weak ThreadId subscriber cleanup), [Store/Postgres.hs](modules/Simplex/Messaging/Notifications/Server/Store/Postgres.md) (XOR hash triggers, batch status updates, cursor-based pagination). +- **NTF service subscription model**: Service-level subscriptions (SUBS/NSUBS on SMP) vs individual queue subscriptions, with fallback from service to individual when `CAServiceUnavailable`. Service credentials are lazily generated per SMP router with 25h backdating and ~2700yr validity. XOR hash triggers on PostgreSQL maintain subscription aggregate counts. Subscription status tracking uses `ntf_service_assoc` flag to distinguish service-associated from individually-subscribed queues. Spans [Server.hs](modules/Simplex/Messaging/Notifications/Server.md) (subscriber thread, service fallback), [Env.hs](modules/Simplex/Messaging/Notifications/Server/Env.md) (lazy credential generation, Weak ThreadId subscriber cleanup), [Store/Postgres.hs](modules/Simplex/Messaging/Notifications/Server/Store/Postgres.md) (XOR hash triggers, batch status updates, cursor-based pagination). -- **NTF startup resubscription**: `resubscribe` runs as detached `forkIO` (not in `raceAny_` group), uses `mapConcurrently` across SMP servers, each with `subscribeLoop` using 100x database batch multiplier and cursor-based pagination. `ExitCode` exceptions from `exitFailure` on DB error propagate to main thread despite `forkIO`. `getServerNtfSubscriptions` claims subscriptions by batch-updating to `NSPending`. Spans [Server.hs](modules/Simplex/Messaging/Notifications/Server.md), [Store/Postgres.hs](modules/Simplex/Messaging/Notifications/Server/Store/Postgres.md). +- **NTF startup resubscription**: `resubscribe` runs as detached `forkIO` (not in `raceAny_` group), uses `mapConcurrently` across SMP routers, each with `subscribeLoop` using 100x database batch multiplier and cursor-based pagination. `ExitCode` exceptions from `exitFailure` on DB error propagate to main thread despite `forkIO`. `getServerNtfSubscriptions` claims subscriptions by batch-updating to `NSPending`. Spans [Server.hs](modules/Simplex/Messaging/Notifications/Server.md), [Store/Postgres.hs](modules/Simplex/Messaging/Notifications/Server/Store/Postgres.md). -- **XFTP file upload pipeline**: Agent-side encryption (streaming 64KB blocks, fixed-size padding) → chunk size selection (75% threshold algorithm) → per-server chunk creation with ID collision retry (3 attempts) → recipient registration (recursive batching up to `maxRecipients` per FADD) → per-server upload (command + file body in single HTTP/2 streaming request) → file description generation (cross-product: M chunks × R replicas × N recipients → N descriptions). Spans [Agent.hs](modules/Simplex/FileTransfer/Agent.md) (worker orchestration, description generation), [Client.hs](modules/Simplex/FileTransfer/Client.md) (upload protocol), [Server.hs](modules/Simplex/FileTransfer/Server.md) (quota reservation with rollback, skipCommitted idempotency), [Crypto.hs](modules/Simplex/FileTransfer/Crypto.md) (streaming encryption with embedded header), [Description.hs](modules/Simplex/FileTransfer/Description.md) (validation, first-replica-only digest optimization). +- **XFTP file upload pipeline**: Agent-side encryption (streaming 64KB blocks, fixed-size padding) → chunk size selection (75% threshold algorithm) → per-router chunk creation with ID collision retry (3 attempts) → recipient registration (recursive batching up to `maxRecipients` per FADD) → per-router upload (command + file body in single HTTP/2 streaming request) → file description generation (cross-product: M chunks × R replicas × N recipients → N descriptions). Spans [Agent.hs](modules/Simplex/FileTransfer/Agent.md) (worker orchestration, description generation), [Client.hs](modules/Simplex/FileTransfer/Client.md) (upload protocol), [Server.hs](modules/Simplex/FileTransfer/Server.md) (quota reservation with rollback, skipCommitted idempotency), [Crypto.hs](modules/Simplex/FileTransfer/Crypto.md) (streaming encryption with embedded header), [Description.hs](modules/Simplex/FileTransfer/Description.md) (validation, first-replica-only digest optimization). -- **XFTP file download pipeline**: Description parsing (ValidFileDescription validation, YAML or web URI) → per-server chunk download with ephemeral DH key pair per download (forward secrecy) → size and digest verification before decryption → streaming decryption with auth tag verification (output deleted on failure) → redirect resolution (depth-1 chain: decrypt redirect YAML, validate size/digest, download actual file). Spans [Agent.hs](modules/Simplex/FileTransfer/Agent.md) (worker orchestration, redirect handling), [Client.hs](modules/Simplex/FileTransfer/Client.md) (ephemeral DH, chunk-proportional timeout), [Client/Main.hs](modules/Simplex/FileTransfer/Client/Main.md) (web URI decoding, parallel download with server grouping), [Crypto.hs](modules/Simplex/FileTransfer/Crypto.md) (dual decrypt paths, auth tag deletion), [Description.hs](modules/Simplex/FileTransfer/Description.md) (redirect file descriptions). +- **XFTP file download pipeline**: Description parsing (ValidFileDescription validation, YAML or web URI) → per-router chunk download with ephemeral DH key pair per download (forward secrecy) → size and digest verification before decryption → streaming decryption with auth tag verification (output deleted on failure) → redirect resolution (depth-1 chain: decrypt redirect YAML, validate size/digest, download actual file). Spans [Agent.hs](modules/Simplex/FileTransfer/Agent.md) (worker orchestration, redirect handling), [Client.hs](modules/Simplex/FileTransfer/Client.md) (ephemeral DH, chunk-proportional timeout), [Client/Main.hs](modules/Simplex/FileTransfer/Client/Main.md) (web URI decoding, parallel download with router grouping), [Crypto.hs](modules/Simplex/FileTransfer/Crypto.md) (dual decrypt paths, auth tag deletion), [Description.hs](modules/Simplex/FileTransfer/Description.md) (redirect file descriptions). - **XFTP handshake state machine**: Three-state session-cached handshake (`No entry` → `HandshakeSent` → `HandshakeAccepted`) per HTTP/2 session. Web clients use `xftp-web-hello` header and challenge-response identity proof; native clients use standard ALPN. SNI presence gates CORS headers, web serving, and SESSION error for unrecognized connections. Key reuse on re-hello preserves existing DH keys. Spans [Server.hs](modules/Simplex/FileTransfer/Server.md) (handshake logic, CORS, web serving), [Client.hs](modules/Simplex/FileTransfer/Client.md) (ALPN selection, cert chain validation), [Transport.hs](modules/Simplex/FileTransfer/Transport.md) (block size, version). - **XFTP storage lifecycle**: Quota reservation via atomic `stateTVar` before upload → rollback on failure (subtract + delete partial file) → physical file deleted before store cleanup (crash risk: store references missing file) → `RoundedSystemTime 3600` for privacy-preserving expiration timestamps → expiration with configurable throttling (100ms between files) → startup storage reconciliation (override stats from live store). Spans [Server.hs](modules/Simplex/FileTransfer/Server.md), [Server/Store.hs](modules/Simplex/FileTransfer/Server/Store.md), [Server/Env.hs](modules/Simplex/FileTransfer/Server/Env.md), [Server/StoreLog.hs](modules/Simplex/FileTransfer/Server/StoreLog.md) (error-resilient replay, compaction). -- **XFTP worker architecture**: Five worker types in three categories: rcv (per-server download + local decryption), snd (local prepare/encrypt + per-server upload), del (per-server delete). TMVar-based connection sharing with async retry on temporary errors, permanent error cleanup (put Left + delete from TMap). `withRetryIntervalLimit` caps consecutive retries; exhausted temporary errors silently abandon work cycle (chunk stays pending). `assertAgentForeground` dual check (throw if inactive + wait if backgrounded) gates every chunk operation. Spans [Agent.hs](modules/Simplex/FileTransfer/Agent.md), [Client/Agent.hs](modules/Simplex/FileTransfer/Client/Agent.md). +- **XFTP worker architecture**: Five worker types in three categories: rcv (per-router download + local decryption), snd (local prepare/encrypt + per-router upload), del (per-router delete). TMVar-based connection sharing with async retry on temporary errors, permanent error cleanup (put Left + delete from TMap). `withRetryIntervalLimit` caps consecutive retries; exhausted temporary errors silently abandon work cycle (chunk stays pending). `assertAgentForeground` dual check (throw if inactive + wait if backgrounded) gates every chunk operation. Spans [Agent.hs](modules/Simplex/FileTransfer/Agent.md), [Client/Agent.hs](modules/Simplex/FileTransfer/Client/Agent.md). - **SessionVar protocol client lifecycle**: Protocol client connections (SMP, NTF, XFTP) use a lazy singleton pattern: `getSessVar` atomically checks TMap → `newProtocolClient` fills TMVar on success/failure → `waitForProtocolClient` reads with timeout. Error caching via `persistErrorInterval` prevents connection storms (failed connections cache the error with expiry; callers receive cached error without reconnecting). `removeSessVar` uses monotonic `sessionVarId` compare-and-swap to prevent stale disconnect callbacks from removing newer clients. SMP has additional complexity: `SMPConnectedClient` wraps client with per-connection proxied relay map, `updateClientService` synchronizes service credentials post-connect, disconnect callback moves subscriptions to pending with session-ID matching. XFTP always uses `NRMBackground` timing regardless of caller request. Spans [Session.md](modules/Simplex/Messaging/Session.md), [Agent/Client.md](modules/Simplex/Messaging/Agent/Client.md) (lifecycle, disconnect callbacks, reconnection workers), [Agent.md](modules/Simplex/Messaging/Agent.md) (subscriber loop consuming events). @@ -60,7 +60,7 @@ - **Deferred message encryption**: Message bodies are NOT encrypted at enqueue time. `enqueueMessageB` advances the ratchet header and validates padding, but stores only the body reference (`sndMsgBodyId`) and encryption key. Actual encryption (`rcEncryptMsg`) happens at delivery time in `runSmpQueueMsgDelivery`. This enables body deduplication via `VRValue`/`VRRef` — identical bodies (common for group messages) share one database row, but each connection's delivery encrypts independently with its own ratchet. Confirmation and ratchet key messages bypass deferred encryption (pre-encrypted at enqueue time). Spans [Agent.md](modules/Simplex/Messaging/Agent.md) (enqueue + delivery), [AgentStore.md](modules/Simplex/Messaging/Agent/Store/AgentStore.md) (`snd_message_bodies` storage). -- **NTF agent subscription lifecycle**: The agent-side notification subscription system uses a supervisor-worker architecture with three worker pools (NTF server, SMP server, token deletion). `NSCCreate` triggers a four-way partition (`partitionQueueSubActions`): new sub, reset sub (credential mismatch or null action), continue SMP work, continue NTF work. Workers coordinate with the supervisor via `updated_by_supervisor` flag — workers only update local fields when the flag is set, preventing overwrite of supervisor decisions. The null-action sentinel (`workerErrors` sets action to NULL on permanent failure) bridges worker failure recovery to supervisor-driven re-creation. `retrySubActions` uses a shrinking TVar — each iteration only retries subs with temporary errors, so batches get smaller over time. `rescheduleWork` handles time-scheduled health checks by forking a sleep thread that re-signals `doWork`. Spans [NtfSubSupervisor.md](modules/Simplex/Messaging/Agent/NtfSubSupervisor.md) (supervisor, worker pools), [AgentStore.md](modules/Simplex/Messaging/Agent/Store/AgentStore.md) (updated_by_supervisor, null-action sentinel), [Agent/Client.md](modules/Simplex/Messaging/Agent/Client.md) (worker framework). +- **NTF agent subscription lifecycle**: The agent-side notification subscription system uses a supervisor-worker architecture with three worker pools (NTF router, SMP router, token deletion). `NSCCreate` triggers a four-way partition (`partitionQueueSubActions`): new sub, reset sub (credential mismatch or null action), continue SMP work, continue NTF work. Workers coordinate with the supervisor via `updated_by_supervisor` flag — workers only update local fields when the flag is set, preventing overwrite of supervisor decisions. The null-action sentinel (`workerErrors` sets action to NULL on permanent failure) bridges worker failure recovery to supervisor-driven re-creation. `retrySubActions` uses a shrinking TVar — each iteration only retries subs with temporary errors, so batches get smaller over time. `rescheduleWork` handles time-scheduled health checks by forking a sleep thread that re-signals `doWork`. Spans [NtfSubSupervisor.md](modules/Simplex/Messaging/Agent/NtfSubSupervisor.md) (supervisor, worker pools), [AgentStore.md](modules/Simplex/Messaging/Agent/Store/AgentStore.md) (updated_by_supervisor, null-action sentinel), [Agent/Client.md](modules/Simplex/Messaging/Agent/Client.md) (worker framework). - **Session-aware SMP subscription management**: SMP queue subscriptions are tracked per transport session with session-ID validation at multiple points. `subscribeQueues` groups queues by transport session, subscribes concurrently, then validates `activeClientSession` post-RPC — if the client was replaced during the RPC, results are discarded and converted to temporary errors for retry. `removeClientAndSubs` (disconnect cleanup) only demotes subscriptions whose session ID matches the disconnecting client. Batch UP notifications are accumulated across transmissions and deduplicated against already-active subscriptions. When ALL results are temporary errors and no connections were already active, the SMP client is closed to force fresh connection. `maxPending` throttles concurrent pending subscriptions with STM retry backpressure. Spans [Agent/Client.md](modules/Simplex/Messaging/Agent/Client.md) (subscription state, session validation), [Agent.md](modules/Simplex/Messaging/Agent.md) (subscriber loop, processSMPTransmissions, UP accumulation). diff --git a/spec/modules/Simplex/FileTransfer/Agent.md b/spec/modules/Simplex/FileTransfer/Agent.md index fd2a361d0..e5f58e996 100644 --- a/spec/modules/Simplex/FileTransfer/Agent.md +++ b/spec/modules/Simplex/FileTransfer/Agent.md @@ -8,15 +8,15 @@ The XFTP agent uses five worker types organized in three categories: -| Worker | Key (server) | Purpose | +| Worker | Key (router) | Purpose | |--------|-------------|---------| -| `xftpRcvWorker` | `Just server` | Download chunks from a specific XFTP server | +| `xftpRcvWorker` | `Just server` | Download chunks from a specific XFTP router | | `xftpRcvLocalWorker` | `Nothing` | Decrypt completed downloads locally | -| `xftpSndPrepareWorker` | `Nothing` | Encrypt files and create chunks on servers | -| `xftpSndWorker` | `Just server` | Upload chunks to a specific XFTP server | -| `xftpDelWorker` | `Just server` | Delete chunks from a specific XFTP server | +| `xftpSndPrepareWorker` | `Nothing` | Encrypt files and create chunks on routers | +| `xftpSndWorker` | `Just server` | Upload chunks to a specific XFTP router | +| `xftpDelWorker` | `Just server` | Delete chunks from a specific XFTP router | -Workers are created on-demand via `getAgentWorker` and keyed by server address. The local workers (keyed by `Nothing`) handle CPU-bound operations that don't require network access. +Workers are created on-demand via `getAgentWorker` and keyed by router address. The local workers (keyed by `Nothing`) handle CPU-bound operations that don't require network access. ## Non-obvious behavior @@ -71,7 +71,7 @@ During upload, `addRecipients` recursively calls itself if a chunk needs more re ### 12. Delete workers skip files older than rcvFilesTTL -`runXFTPDelWorker` uses `rcvFilesTTL` (not a dedicated delete TTL) to filter pending deletions. Files older than this TTL would already be expired on the server, so attempting deletion is pointless. This reuses the receive TTL as a proxy for server-side expiration. +`runXFTPDelWorker` uses `rcvFilesTTL` (not a dedicated delete TTL) to filter pending deletions. Files older than this TTL would already be expired on the router, so attempting deletion is pointless. This reuses the receive TTL as a proxy for router-side expiration. ### 13. closeXFTPAgent atomically swaps worker maps @@ -81,6 +81,6 @@ During upload, `addRecipients` recursively calls itself if a chunk needs more re `assertAgentForeground` both throws if the agent is inactive (`throwWhenInactive`) and blocks until it's in the foreground (`waitUntilForeground`). This is called before every chunk operation to ensure the agent isn't suspended or backgrounded during file transfers. -### 15. Per-server stats tracking +### 15. Per-router stats tracking -Every chunk download, upload, and delete operation increments per-server statistics (`downloads`, `uploads`, `deletions`, `downloadAttempts`, `uploadAttempts`, `deleteAttempts`, and error variants). Size-based stats (`downloadsSize`, `uploadsSize`) track throughput in kilobytes. +Every chunk download, upload, and delete operation increments per-router statistics (`downloads`, `uploads`, `deletions`, `downloadAttempts`, `uploadAttempts`, `deleteAttempts`, and error variants). Size-based stats (`downloadsSize`, `uploadsSize`) track throughput in kilobytes. diff --git a/spec/modules/Simplex/FileTransfer/Client.md b/spec/modules/Simplex/FileTransfer/Client.md index 5cf87a594..27fb50bc3 100644 --- a/spec/modules/Simplex/FileTransfer/Client.md +++ b/spec/modules/Simplex/FileTransfer/Client.md @@ -12,13 +12,13 @@ - **`xftpALPNv1` or `httpALPN11`**: performs v1 handshake with key exchange (`httpALPN11` is used for web port connections) - **No ALPN or unrecognized**: uses legacy v1 transport parameters without handshake -### 2. Server certificate chain validation +### 2. Router certificate chain validation -`xftpClientHandshakeV1` validates the server's identity by checking that the CA fingerprint from the certificate chain matches the expected `keyHash` from the server address. The server signs an authentication public key (X25519) with its long-term key. The client verifies this signature against the certificate chain, then extracts the X25519 key for HMAC-based command authentication. This authentication key is distinct from the per-download ephemeral DH keys. +`xftpClientHandshakeV1` validates the router's identity by checking that the CA fingerprint from the certificate chain matches the expected `keyHash` from the router address. The router signs an authentication public key (X25519) with its long-term key. The client verifies this signature against the certificate chain, then extracts the X25519 key for HMAC-based command authentication. This authentication key is distinct from the per-download ephemeral DH keys. ### 3. Ephemeral DH key pair per download -`downloadXFTPChunk` generates a fresh X25519 key pair for each chunk download. The public key is sent with the FGET command; the server responds with its own ephemeral key. The derived shared secret encrypts the file data in transit. This provides forward secrecy — compromising a past DH key doesn't decrypt other downloads. +`downloadXFTPChunk` generates a fresh X25519 key pair for each chunk download. The public key is sent with the FGET command; the router responds with its own ephemeral key. The derived shared secret encrypts the file data in transit. This provides forward secrecy — compromising a past DH key doesn't decrypt other downloads. ### 4. Chunk-size-proportional download timeout @@ -30,7 +30,7 @@ ### 6. Upload sends file body after command response -`uploadXFTPChunk` sends the FPUT command and file body in the same streaming HTTP/2 request: the protocol command block is sent first, followed immediately by the raw file data via `hSendFile`. The server response (`FROk` or error) is received only after both the command and file body have been fully sent. This is a single HTTP/2 round trip, not a two-phase interaction. +`uploadXFTPChunk` sends the FPUT command and file body in the same streaming HTTP/2 request: the protocol command block is sent first, followed immediately by the raw file data via `hSendFile`. The router response (`FROk` or error) is received only after both the command and file body have been fully sent. This is a single HTTP/2 round trip, not a two-phase interaction. ### 7. Empty corrId as nonce diff --git a/spec/modules/Simplex/FileTransfer/Client/Agent.md b/spec/modules/Simplex/FileTransfer/Client/Agent.md index 6ff1eebb7..c03400d90 100644 --- a/spec/modules/Simplex/FileTransfer/Client/Agent.md +++ b/spec/modules/Simplex/FileTransfer/Client/Agent.md @@ -8,7 +8,7 @@ ### 1. TMVar-based connection sharing -`getXFTPServerClient` first checks the `TMap XFTPServer (TMVar (Either XFTPClientAgentError XFTPClient))`. If no entry exists, it atomically inserts an empty `TMVar` and initiates connection. Other threads requesting the same server block on `readTMVar` until the connection is established or fails. This prevents duplicate connections to the same server. +`getXFTPServerClient` first checks the `TMap XFTPServer (TMVar (Either XFTPClientAgentError XFTPClient))`. If no entry exists, it atomically inserts an empty `TMVar` and initiates connection. Other threads requesting the same router block on `readTMVar` until the connection is established or fails. This prevents duplicate connections to the same router. ### 2. Async retry on temporary errors @@ -20,8 +20,8 @@ On permanent error, `newXFTPClient` puts the `Left error` into the `TMVar` (unbl ### 4. Connection timeout -`waitForXFTPClient` wraps `readTMVar` in a timeout. If the connection establishment takes too long (e.g., server unreachable and retry loop is slow), the caller gets a timeout error rather than blocking indefinitely. The underlying connection attempt continues in the background. +`waitForXFTPClient` wraps `readTMVar` in a timeout. If the connection establishment takes too long (e.g., router unreachable and retry loop is slow), the caller gets a timeout error rather than blocking indefinitely. The underlying connection attempt continues in the background. ### 5. closeXFTPServerClient removes from TMap -Closing a server client deletes its entry from the TMap, so the next request will establish a fresh connection. This is called on connection errors during file operations to force reconnection. +Closing a router client deletes its entry from the TMap, so the next request will establish a fresh connection. This is called on connection errors during file operations to force reconnection. diff --git a/spec/modules/Simplex/FileTransfer/Client/Main.md b/spec/modules/Simplex/FileTransfer/Client/Main.md index 5f7b45af4..abb9eceb5 100644 --- a/spec/modules/Simplex/FileTransfer/Client/Main.md +++ b/spec/modules/Simplex/FileTransfer/Client/Main.md @@ -8,7 +8,7 @@ ### 1. Web URI encoding: base64url(deflate(YAML)) -`encodeWebURI` compresses the YAML-encoded file description with raw DEFLATE, then base64url-encodes the result. `decodeWebURI` reverses this. The compressed description goes in the URL fragment (after `#`), which is never sent to the server — the file description stays client-side. +`encodeWebURI` compresses the YAML-encoded file description with raw DEFLATE, then base64url-encodes the result. `decodeWebURI` reverses this. The compressed description goes in the URL fragment (after `#`), which is never sent to the router — the file description stays client-side. ### 2. CLI receive accepts both file paths and URLs @@ -18,17 +18,17 @@ `receive` tracks a `depth` parameter starting at 1. After following one redirect, `depth` becomes 0. A second redirect throws "Redirect chain too long". This prevents infinite redirect loops from malicious file descriptions. -### 4. Parallel chunk uploads with server grouping +### 4. Parallel chunk uploads with router grouping -`uploadFile` groups chunks by server via `groupAllOn`, then uses `pooledForConcurrentlyN 16` to process up to 16 server-groups concurrently. Within each group, chunks are uploaded sequentially (`mapM`). Errors from any chunk are collected and the first one is thrown. +`uploadFile` groups chunks by router via `groupAllOn`, then uses `pooledForConcurrentlyN 16` to process up to 16 router-groups concurrently. Within each group, chunks are uploaded sequentially (`mapM`). Errors from any chunk are collected and the first one is thrown. -### 5. Random server selection +### 5. Random router selection -`getXFTPServer` selects a random server from the provided list for each chunk. With a single server, it's deterministic. With multiple servers, it uses `StdGen` in a TVar for thread-safe random selection via `stateTVar`. +`getXFTPServer` selects a random router from the provided list for each chunk. With a single router, it's deterministic. With multiple routers, it uses `StdGen` in a TVar for thread-safe random selection via `stateTVar`. ### 6. withReconnect nests retry with reconnection -`withReconnect` wraps `withRetry` twice: the outer retry reconnects to the server, and the inner operation runs against the connection. On failure, the server connection is explicitly closed before retrying, forcing a fresh connection on the next attempt. +`withReconnect` wraps `withRetry` twice: the outer retry reconnects to the router, and the inner operation runs against the connection. On failure, the router connection is explicitly closed before retrying, forcing a fresh connection on the next attempt. ### 7. withRetry rejects zero retries @@ -36,8 +36,8 @@ ### 8. File description auto-deletion prompt -After successful receive or delete, `removeFD` either auto-deletes the file description (if `--yes` flag) or prompts the user. This prevents accidental reuse of one-time file descriptions — each receive consumes the description by ACKing chunks on the server. +After successful receive or delete, `removeFD` either auto-deletes the file description (if `--yes` flag) or prompts the user. This prevents accidental reuse of one-time file descriptions — each receive consumes the description by ACKing chunks on the router. -### 9. Sender description uses first replica's server +### 9. Sender description uses first replica's router -`createSndFileDescription` takes the server from the first replica of each chunk for the sender's `FileChunkReplica`. This reflects the current limitation that each chunk is uploaded to exactly one server — the sender description records that single server. +`createSndFileDescription` takes the router from the first replica of each chunk for the sender's `FileChunkReplica`. This reflects the current limitation that each chunk is uploaded to exactly one router — the sender description records that single router. diff --git a/spec/modules/Simplex/FileTransfer/Description.md b/spec/modules/Simplex/FileTransfer/Description.md index b4c7e2fe9..0edd0bee8 100644 --- a/spec/modules/Simplex/FileTransfer/Description.md +++ b/spec/modules/Simplex/FileTransfer/Description.md @@ -22,9 +22,9 @@ When encoding chunks to YAML via `unfoldChunksToReplicas`, the `digest` and non- The top-level `FileDescription` has a `chunkSize` field. Individual chunk replicas only serialize their `chunkSize` if it differs from this default. This saves space in the common case where most chunks are the same size (only the last chunk may be smaller). -### 4. YAML encoding groups replicas by server +### 4. YAML encoding groups replicas by router -`groupReplicasByServer` groups all chunk replicas by their server, producing `FileServerReplica` records. This is the serialization format — replicas are organized by server, not by chunk. The parser (`foldReplicasToChunks`) reverses this grouping back to per-chunk replica lists. +`groupReplicasByServer` groups all chunk replicas by their router, producing `FileServerReplica` records. This is the serialization format — replicas are organized by router, not by chunk. The parser (`foldReplicasToChunks`) reverses this grouping back to per-chunk replica lists. ### 5. FileDescriptionURI uses query-string encoding @@ -40,4 +40,4 @@ Two limits exist: `maxFileSize = 1GB` (soft limit, checked by CLI client) and `m ### 8. Redirect file descriptions -A `FileDescription` can contain a `redirect` field pointing to another file's metadata (`RedirectFileInfo` with size and digest). The outer description downloads an encrypted YAML file that, once decrypted, yields the actual `FileDescription` for the real file. This adds one level of indirection for privacy — the relay servers hosting the redirect don't know the actual file's servers. +A `FileDescription` can contain a `redirect` field pointing to another file's metadata (`RedirectFileInfo` with size and digest). The outer description downloads an encrypted YAML file that, once decrypted, yields the actual `FileDescription` for the real file. This adds one level of indirection for privacy — the relay routers hosting the redirect don't know the actual file's routers. diff --git a/spec/modules/Simplex/FileTransfer/Protocol.md b/spec/modules/Simplex/FileTransfer/Protocol.md index f31c90561..4bbcb8726 100644 --- a/spec/modules/Simplex/FileTransfer/Protocol.md +++ b/spec/modules/Simplex/FileTransfer/Protocol.md @@ -29,7 +29,7 @@ Even for single transmissions, `xftpEncodeBatch1` wraps the encoded transmission ### 5. FileParty GADT partitions command space -Commands are indexed by `FileParty` (`SFSender` / `SFRecipient`) at the type level via `FileCmd`. This ensures at compile time that sender commands (FNEW, FADD, FPUT, FDEL) and recipient commands (FGET, FACK, PING) cannot be confused. The server pattern-matches on `SFileParty` to determine which index (sender vs recipient) to look up in the file store. +Commands are indexed by `FileParty` (`SFSender` / `SFRecipient`) at the type level via `FileCmd`. This ensures at compile time that sender commands (FNEW, FADD, FPUT, FDEL) and recipient commands (FGET, FACK, PING) cannot be confused. The router pattern-matches on `SFileParty` to determine which index (sender vs recipient) to look up in the file store. ### 6. Empty corrId and implicit session ID diff --git a/spec/modules/Simplex/FileTransfer/Server.md b/spec/modules/Simplex/FileTransfer/Server.md index f3a01314d..99e17a427 100644 --- a/spec/modules/Simplex/FileTransfer/Server.md +++ b/spec/modules/Simplex/FileTransfer/Server.md @@ -1,16 +1,16 @@ # Simplex.FileTransfer.Server -> XFTP server: HTTP/2 request handling, handshake state machine, file operations, and statistics. +> XFTP router: HTTP/2 request handling, handshake state machine, file operations, and statistics. **Source**: [`FileTransfer/Server.hs`](../../../../src/Simplex/FileTransfer/Server.hs) ## Architecture -The XFTP server runs several concurrent threads via `raceAny_`: +The XFTP router runs several concurrent threads via `raceAny_`: | Thread | Purpose | |--------|---------| -| `runServer` | HTTP/2 server accepting file transfer requests | +| `runServer` | HTTP/2 router accepting file transfer requests | | `expireFiles` | Periodic file expiration with throttling | | `logServerStats` | Periodic stats flush to CSV | | `savePrometheusMetrics` | Periodic Prometheus metrics dump | @@ -20,24 +20,24 @@ The XFTP server runs several concurrent threads via `raceAny_`: ### 1. Three-state handshake with session caching -The server maintains a `TMap SessionId Handshake` with three states: -- **No entry**: first request — for non-SNI or `xftp-web-hello` requests, `processHello` generates DH key pair and sends server handshake; for SNI requests without `xftp-web-hello`, returns `SESSION` error -- **`HandshakeSent pk`**: server hello sent, waiting for client handshake with version negotiation +The router maintains a `TMap SessionId Handshake` with three states: +- **No entry**: first request — for non-SNI or `xftp-web-hello` requests, `processHello` generates DH key pair and sends router handshake; for SNI requests without `xftp-web-hello`, returns `SESSION` error +- **`HandshakeSent pk`**: router hello sent, waiting for client handshake with version negotiation - **`HandshakeAccepted thParams`**: handshake complete, subsequent requests use cached params -Web clients can re-send hello (`xftp-web-hello` header) even in `HandshakeSent` or `HandshakeAccepted` states — the server reuses the existing private key rather than generating a new one. +Web clients can re-send hello (`xftp-web-hello` header) even in `HandshakeSent` or `HandshakeAccepted` states — the router reuses the existing private key rather than generating a new one. ### 2. Web identity proof via challenge-response -When a web client sends a hello with a non-empty body, the server parses an `XFTPClientHello` containing a `webChallenge`. The server signs `challenge <> sessionId` with its long-term key and includes the signature in the handshake response. This proves server identity to web clients that cannot verify TLS certificates directly. +When a web client sends a hello with a non-empty body, the router parses an `XFTPClientHello` containing a `webChallenge`. The router signs `challenge <> sessionId` with its long-term key and includes the signature in the handshake response. This proves router identity to web clients that cannot verify TLS certificates directly. ### 3. skipCommitted drains request body on re-upload -If `receiveServerFile` detects the file is already uploaded (`filePath` TVar is `Just`), it cannot simply ignore the request body — the HTTP/2 client would block waiting for the server to consume it. Instead, `skipCommitted` reads and discards the entire body in `fileBlockSize` increments, returning `FROk` when complete. This makes FPUT idempotent from the client's perspective. +If `receiveServerFile` detects the file is already uploaded (`filePath` TVar is `Just`), it cannot simply ignore the request body — the HTTP/2 client would block waiting for the router to consume it. Instead, `skipCommitted` reads and discards the entire body in `fileBlockSize` increments, returning `FROk` when complete. This makes FPUT idempotent from the client's perspective. ### 4. Atomic quota reservation with rollback -`receiveServerFile` uses `stateTVar` to atomically check and reserve storage quota before receiving the file. If the upload fails (timeout, size mismatch, IO error), the reserved size is subtracted from `usedStorage` and the partial file is deleted. This prevents failed uploads from permanently consuming quota. +`receiveServerFile` uses `stateTVar` to atomically check and reserve storage quota before receiving the file. If the upload fails (timeout, size mismatch, IO error), the reserved size is subtracted from `usedStorage` and the partial file is deleted on the router. This prevents failed uploads from permanently consuming quota. ### 5. retryAdd generates new IDs on collision @@ -45,7 +45,7 @@ If `receiveServerFile` detects the file is already uploaded (`filePath` TVar is ### 6. Timing attack mitigation on entity lookup -`verifyXFTPTransmission` calls `dummyVerifyCmd` (imported from SMP server) when a file entity is not found. This equalizes response timing to prevent attackers from distinguishing "entity doesn't exist" from "signature invalid" based on latency. +`verifyXFTPTransmission` calls `dummyVerifyCmd` (imported from SMP router) when a file entity is not found. This equalizes response timing to prevent attackers from distinguishing "entity doesn't exist" from "signature invalid" based on latency. ### 7. BLOCKED vs EntityOff distinction @@ -62,11 +62,11 @@ Despite the name suggesting it only marks a file as blocked, `blockServerFile` a ### 9. Stats restore overrides counts from live store -`restoreServerStats` loads stats from the backup file but overrides `_filesCount` and `_filesSize` with values computed from the live file store (TMap size and `usedStorage` TVar). If the backup values differ, warnings are logged. This handles cases where files were expired or deleted while the server was down. +`restoreServerStats` loads stats from the backup file but overrides `_filesCount` and `_filesSize` with values computed from the live file store (TMap size and `usedStorage` TVar). If the backup values differ, warnings are logged. This handles cases where files were expired or deleted while the router was down. ### 10. File expiration with configurable throttling -`expireServerFiles` accepts an optional `itemDelay` (100ms when called from the periodic thread, `Nothing` at startup). Between each file check, `threadDelay itemDelay` prevents expiration from monopolizing IO. At startup, files are expired without delay to clean up quickly. +`expireServerFiles` accepts an optional `itemDelay` (100ms when called from the periodic thread, `Nothing` at router startup). Between each file check, `threadDelay itemDelay` prevents expiration from monopolizing IO. At startup, files are expired without delay to clean up quickly. ### 11. Stats log aligns to wall-clock midnight @@ -78,7 +78,7 @@ Despite the name suggesting it only marks a file as blocked, `blockServerFile` a ### 13. SNI-dependent CORS and web serving -CORS headers require both `sniUsed = True` and `addCORSHeaders = True` in the transport config. Static web page serving is enabled when `sniUsed = True`. Non-SNI connections (direct TLS without hostname) skip both CORS and web serving. This separates the web-facing and protocol-facing behaviors of the same port. +CORS headers require both `sniUsed = True` and `addCORSHeaders = True` in the transport config. Static web page serving is enabled when `sniUsed = True`. Non-SNI connections (direct TLS without hostname) skip both CORS and web serving. This separates the web-facing and protocol-facing behaviors of the same router port. ### 14. Control port file operations use recipient index diff --git a/spec/modules/Simplex/FileTransfer/Server/Env.md b/spec/modules/Simplex/FileTransfer/Server/Env.md index e9f509a1a..0b3bba3ff 100644 --- a/spec/modules/Simplex/FileTransfer/Server/Env.md +++ b/spec/modules/Simplex/FileTransfer/Server/Env.md @@ -1,6 +1,6 @@ # Simplex.FileTransfer.Server.Env -> XFTP server environment: configuration, storage quota tracking, and request routing. +> XFTP router environment: configuration, storage quota tracking, and request routing. **Source**: [`FileTransfer/Server/Env.hs`](../../../../../src/Simplex/FileTransfer/Server/Env.hs) @@ -8,7 +8,7 @@ ### 1. Startup storage accounting with quota warning -`newXFTPServerEnv` computes `usedStorage` by summing file sizes from the in-memory store at startup. If the computed usage exceeds the configured `fileSizeQuota`, a warning is logged but the server still starts. This allows the server to come up even if it's over quota (e.g., after a quota reduction), relying on expiration to reclaim space. +`newXFTPServerEnv` computes `usedStorage` by summing file sizes from the in-memory store at startup. If the computed usage exceeds the configured `fileSizeQuota`, a warning is logged but the router still starts. This allows the router to come up even if it's over quota (e.g., after a quota reduction), relying on expiration to reclaim space. ### 2. XFTPRequest ADT separates new files from commands @@ -21,4 +21,4 @@ This separation occurs after credential verification in `Server.hs`. `XFTPReqNew ### 3. fileTimeout for upload deadline -`fileTimeout` in `XFTPServerConfig` sets the maximum time allowed for a single file upload (FPUT). The server wraps the receive operation in `timeout fileTimeout`. Default is 5 minutes (for 4MB chunks). This prevents slow or stalled uploads from holding server resources indefinitely. +`fileTimeout` in `XFTPServerConfig` sets the maximum time allowed for a single file upload (FPUT). The router wraps the receive operation in `timeout fileTimeout`. Default is 5 minutes (for 4MB chunks). This prevents slow or stalled uploads from holding router resources indefinitely. diff --git a/spec/modules/Simplex/FileTransfer/Server/Main.md b/spec/modules/Simplex/FileTransfer/Server/Main.md index 54a45751f..c892e6bf5 100644 --- a/spec/modules/Simplex/FileTransfer/Server/Main.md +++ b/spec/modules/Simplex/FileTransfer/Server/Main.md @@ -1,12 +1,12 @@ # Simplex.FileTransfer.Server.Main -> XFTP server CLI: INI configuration parsing, TLS setup, and default constants. +> XFTP router CLI: INI configuration parsing, TLS setup, and default constants. **Source**: [`FileTransfer/Server/Main.hs`](../../../../../src/Simplex/FileTransfer/Server/Main.hs) ## Non-obvious behavior -### 1. Key server constants +### 1. Key router constants | Constant | Value | Purpose | |----------|-------|---------| @@ -17,7 +17,7 @@ ### 2. allowedChunkSizes defaults to all four sizes -If not configured, `allowedChunkSizes` defaults to `[kb 64, kb 256, mb 1, mb 4]`. The INI file can restrict this to a subset, controlling which chunk sizes the server accepts. +If not configured, `allowedChunkSizes` defaults to `[kb 64, kb 256, mb 1, mb 4]`. The INI file can restrict this to a subset, controlling which chunk sizes the router accepts. ### 3. Storage quota from INI with unit parsing @@ -25,4 +25,4 @@ If not configured, `allowedChunkSizes` defaults to `[kb 64, kb 256, mb 1, mb 4]` ### 4. Dual TLS credential support -The server supports both primary TLS credentials (`caCertificateFile`/`certificateFile`/`privateKeyFile`) and optional HTTP-specific credentials (`httpCaCertificateFile`/etc.). When HTTP credentials are present, the server uses `defaultSupportedParamsHTTPS` which enables broader TLS compatibility for web clients. +The router supports both primary TLS credentials (`caCertificateFile`/`certificateFile`/`privateKeyFile`) and optional HTTP-specific credentials (`httpCaCertificateFile`/etc.). When HTTP credentials are present, the router uses `defaultSupportedParamsHTTPS` which enables broader TLS compatibility for web clients. diff --git a/spec/modules/Simplex/FileTransfer/Server/Stats.md b/spec/modules/Simplex/FileTransfer/Server/Stats.md index 7e684c58a..7eb2ad47b 100644 --- a/spec/modules/Simplex/FileTransfer/Server/Stats.md +++ b/spec/modules/Simplex/FileTransfer/Server/Stats.md @@ -1,6 +1,6 @@ # Simplex.FileTransfer.Server.Stats -> XFTP server statistics: IORef-based counters with backward-compatible persistence. +> XFTP router statistics: IORef-based counters with backward-compatible persistence. **Source**: [`FileTransfer/Server/Stats.hs`](../../../../../src/Simplex/FileTransfer/Server/Stats.hs) @@ -8,11 +8,11 @@ ### 1. setFileServerStats is not thread safe -`setFileServerStats` directly writes to IORefs without synchronization. It is explicitly intended for server startup only (restoring from backup file), before any concurrent threads are running. +`setFileServerStats` directly writes to IORefs without synchronization. It is explicitly intended for router startup only (restoring from backup file), before any concurrent threads are running. ### 2. Backward-compatible parsing -The `strP` parser uses `opt` for newer fields, defaulting missing fields to 0. This allows reading stats files from older server versions that don't include fields like `filesBlocked` or `fileDownloadAcks`. +The `strP` parser uses `opt` for newer fields, defaulting missing fields to 0. This allows reading stats files from older router versions that don't include fields like `filesBlocked` or `fileDownloadAcks`. ### 3. PeriodStats for download tracking diff --git a/spec/modules/Simplex/FileTransfer/Server/Store.md b/spec/modules/Simplex/FileTransfer/Server/Store.md index 89b0c3b36..f2ded441e 100644 --- a/spec/modules/Simplex/FileTransfer/Server/Store.md +++ b/spec/modules/Simplex/FileTransfer/Server/Store.md @@ -36,4 +36,4 @@ File timestamps use `RoundedFileTime` which is `RoundedSystemTime 3600` — syst ### 8. blockFile conditional storage adjustment -`blockFile` takes a `deleted :: Bool` parameter. When `True` (file blocked with physical deletion), it subtracts the file size from `usedStorage`. When `False` (block without deletion), storage is unchanged. This allows blocking without physical deletion for audit purposes. Currently, both the server's `blockServerFile` and the store log replay path pass `True`. +`blockFile` takes a `deleted :: Bool` parameter. When `True` (file blocked with physical deletion), it subtracts the file size from `usedStorage`. When `False` (block without deletion), storage is unchanged. This allows blocking without physical deletion for audit purposes. Currently, both the router's `blockServerFile` and the store log replay path pass `True`. diff --git a/spec/modules/Simplex/FileTransfer/Server/StoreLog.md b/spec/modules/Simplex/FileTransfer/Server/StoreLog.md index 35a339515..6549c3666 100644 --- a/spec/modules/Simplex/FileTransfer/Server/StoreLog.md +++ b/spec/modules/Simplex/FileTransfer/Server/StoreLog.md @@ -1,6 +1,6 @@ # Simplex.FileTransfer.Server.StoreLog -> Append-only store log for XFTP file operations with error-resilient replay and compaction. +> Append-only store log for XFTP router file operations with error-resilient replay and compaction. **Source**: [`FileTransfer/Server/StoreLog.hs`](../../../../../src/Simplex/FileTransfer/Server/StoreLog.hs) @@ -8,7 +8,7 @@ ### 1. Error-resilient replay -`readFileStore` parses the store log line-by-line. Lines that fail to parse or fail to process (e.g., referencing a nonexistent sender ID) are logged as errors but do not halt replay. The store is reconstructed from whatever valid entries exist. This allows the server to recover from partial log corruption. +`readFileStore` parses the store log line-by-line. Lines that fail to parse or fail to process (e.g., referencing a nonexistent sender ID) are logged as errors but do not halt replay. The store is reconstructed from whatever valid entries exist. This allows the router to recover from partial log corruption. ### 2. Sender ID validation on recipient writes @@ -16,7 +16,7 @@ ### 3. Backward-compatible status parsing -`AddFile` log entries include an `EntityStatus` field. The parser uses `<|> pure EntityActive` as a fallback, defaulting to `EntityActive` when the status field is missing. This allows reading store logs from older server versions that didn't record entity status. +`AddFile` log entries include an `EntityStatus` field. The parser uses `<|> pure EntityActive` as a fallback, defaulting to `EntityActive` when the status field is missing. This allows reading store logs from older router versions that didn't record entity status. ### 4. Compaction on restart diff --git a/spec/modules/Simplex/FileTransfer/Types.md b/spec/modules/Simplex/FileTransfer/Types.md index 814e65195..14abc7b21 100644 --- a/spec/modules/Simplex/FileTransfer/Types.md +++ b/spec/modules/Simplex/FileTransfer/Types.md @@ -12,7 +12,7 @@ ### 2. Send file status state machine -`SndFileStatus` progresses: `SFSNew` → `SFSEncrypting` → `SFSEncrypted` → `SFSUploading` → `SFSComplete`, with `SFSError` as terminal. The prepare worker handles `SFSNew` → `SFSEncrypted` (including retry from `SFSEncrypting`), while per-server upload workers handle `SFSUploading` → `SFSComplete`. +`SndFileStatus` progresses: `SFSNew` → `SFSEncrypting` → `SFSEncrypted` → `SFSUploading` → `SFSComplete`, with `SFSError` as terminal. The prepare worker handles `SFSNew` → `SFSEncrypted` (including retry from `SFSEncrypting`), while per-router upload workers handle `SFSUploading` → `SFSComplete`. ### 3. Encrypted file path convention diff --git a/spec/modules/Simplex/Messaging/Agent.md b/spec/modules/Simplex/Messaging/Agent.md index a52be2156..e2cac0638 100644 --- a/spec/modules/Simplex/Messaging/Agent.md +++ b/spec/modules/Simplex/Messaging/Agent.md @@ -15,7 +15,7 @@ This module is the top-level SimpleX agent, consumed by simplex-chat and other c ### Agent startup — backgroundMode `getSMPAgentClient_` accepts a `backgroundMode` flag that fundamentally changes agent capabilities: -- **Normal mode** (`backgroundMode = False`): starts four threads raced via `raceAny_` — `subscriber` (main event loop), `runNtfSupervisor` (notification management), `cleanupManager` (garbage collection), `logServersStats` (statistics). Also restores persisted server statistics. If any thread crashes, all are cancelled; statistics are saved in a `finally` block. +- **Normal mode** (`backgroundMode = False`): starts four threads raced via `raceAny_` — `subscriber` (main event loop), `runNtfSupervisor` (notification management), `cleanupManager` (garbage collection), `logServersStats` (statistics). Also restores persisted router statistics. If any thread crashes, all are cancelled; statistics are saved in a `finally` block. - **Background mode** (`backgroundMode = True`): starts only the `subscriber` thread. No cleanup, no notifications, no stats persistence. Used when the agent needs minimal receive-only operation. Thread crashes are caught by the `run` wrapper: if the agent is still active (`acThread` is set), the exception is reported as `CRITICAL True` to `subQ`. If the agent is being disposed, crashes are silently ignored. diff --git a/spec/modules/Simplex/Messaging/Agent/Client.md b/spec/modules/Simplex/Messaging/Agent/Client.md index f1f4965b6..0177b4f70 100644 --- a/spec/modules/Simplex/Messaging/Agent/Client.md +++ b/spec/modules/Simplex/Messaging/Agent/Client.md @@ -29,7 +29,7 @@ The module is consumed by Agent.hs (which passes specific worker bodies, task qu - **Operation states**: `ntfNetworkOp`, `rcvNetworkOp`, `msgDeliveryOp`, `sndNetworkOp`, `databaseOp` - **Locking**: `connLocks`, `invLocks`, `deleteLock`, `getMsgLocks`, `clientNoticesLock` - **Service state**: `useClientServices` (per-user boolean controlling whether service certificates are used) -- **Proxy routing**: `smpProxiedRelays` (maps destination transport session → proxy server used) +- **Proxy routing**: `smpProxiedRelays` (maps destination transport session → proxy router used) - **Network state**: `userNetworkInfo`, `userNetworkUpdated`, `useNetworkConfig` (slow/fast pair) All TVars are initialized in `newAgentClient`. The `active` TVar is the global kill switch — `closeAgentClient` sets it to `False`, and all protocol client getters check it first. @@ -56,11 +56,11 @@ When `newProtocolClient` fails and `persistErrorInterval > 0`, the error is cach 1. **Session ID registration**: `SS.setSessionId` records the TLS session ID in `currentSubs`, linking the transport session to the actual TLS connection for later session validation. -2. **Service credential synchronization** (`updateClientService`): After connecting, compares client-side and server-side service state. Four cases: +2. **Service credential synchronization** (`updateClientService`): After connecting, compares client-side and router-side service state. Four cases: - Both have service and IDs match → update DB (no-op if same) - Both have service but IDs differ → update DB and remove old queue-service associations - - Client has service, server doesn't → delete client service (handles server version downgrade) - - Server has service, client doesn't → log error (should not happen in normal flow) + - Client has service, router doesn't → delete client service (handles router version downgrade) + - Router has service, client doesn't → log error (should not happen in normal flow) On connection failure, `smpConnectClient` triggers `resubscribeSMPSession` before re-throwing the error. This ensures pending subscriptions get retry logic even when the initial connection attempt fails. @@ -182,7 +182,7 @@ The `clientNoticesLock` TMVar serializes notice processing across concurrent sub ### processSubResults — partitioning Subscription results are partitioned into five categories: -1. **Failed with client notice** — error has an associated server-side notice (e.g., queue status change). Queue is treated as failed (removed from pending, added to `removedSubs`) AND the notice is recorded for processing. +1. **Failed with client notice** — error has an associated router-side notice (e.g., queue status change). Queue is treated as failed (removed from pending, added to `removedSubs`) AND the notice is recorded for processing. 2. **Failed permanently** — non-temporary error without notice, queue is removed from pending and added to `removedSubs` 3. **Failed temporarily** — error is transient, queue stays in pending unchanged for retry on reconnect 4. **Subscribed** — moved from pending to active. Further split into: queues whose service ID matches the session service (added as service-associated) and others. If the queue had a tracked `clientNoticeId`, it is cleared (notice resolved by successful subscription). @@ -205,18 +205,18 @@ Subscription results are partitioned into five categories: Implements SMP proxy/direct routing with fallback: -1. `shouldUseProxy` checks `smpProxyMode` (Always/Unknown/Unprotected/Never) and whether the destination server is "known" (in the user's server list) +1. `shouldUseProxy` checks `smpProxyMode` (Always/Unknown/Unprotected/Never) and whether the destination router is "known" (in the user's router list) 2. If proxying: `getSMPProxyClient` creates or reuses a proxy connection, then `connectSMPProxiedRelay` establishes the relay session. On `NO_SESSION` error, re-creates the relay session through the same proxy. 3. If proxying fails with a host error and `smpProxyFallback` allows it: falls back to direct connection 4. `deleteRelaySession` carefully validates that the current relay session matches the one that failed before removing it (prevents removing a concurrently-created replacement session) -**NO_SESSION retry limit**: On `NO_SESSION`, `sendViaProxy` is called recursively with `Just proxySrv` to reuse the same proxy server. If the recursive call also gets `NO_SESSION`, it throws `proxyError` instead of recursing again — `proxySrv_` is `Just`, so the `Nothing` branch (which recurses) is not taken. This limits retry to exactly one attempt. +**NO_SESSION retry limit**: On `NO_SESSION`, `sendViaProxy` is called recursively with `Just proxySrv` to reuse the same proxy router. If the recursive call also gets `NO_SESSION`, it throws `proxyError` instead of recursing again — `proxySrv_` is `Just`, so the `Nothing` branch (which recurses) is not taken. This limits retry to exactly one attempt. **Proxy selection caching** (`smpProxiedRelays`): When `getSMPProxyClient` selects a proxy for a destination, it atomically inserts the proxy→destination mapping into `smpProxiedRelays`. If a mapping already exists (another thread selected a proxy for the same destination), the existing mapping is used. On relay creation failure with non-host errors, both the relay session and proxy mapping are removed. On host errors, they are preserved to allow fallback logic. ## Service credentials lifecycle -`getServiceCredentials` manages per-user, per-server service certificate credentials: +`getServiceCredentials` manages per-user, per-router service certificate credentials: 1. Checks `useClientServices` — if the user has services disabled, returns `Nothing` 2. Looks up existing credentials in DB via `getClientServiceCredentials` @@ -235,15 +235,15 @@ The generated credentials are Ed25519 self-signed certificates with `simplex` or `withStoreBatch` / `withStoreBatch'` run multiple DB operations in a single transaction, catching exceptions per-operation to report individual failures. The entire batch is within one `agentOperationBracket`. -## Server selection — getNextServer / withNextSrv +## Router selection — getNextServer / withNextSrv -Server selection has two-level diversity: -1. **Operator diversity**: prefer servers from operators not already used (tracked by `usedOperators` set) -2. **Host diversity**: prefer servers with hosts not already used (tracked by `usedHosts` set) +Router selection has two-level diversity: +1. **Operator diversity**: prefer routers from operators not already used (tracked by `usedOperators` set) +2. **Host diversity**: prefer routers with hosts not already used (tracked by `usedHosts` set) -`filterOrAll` ensures that if all servers are "used," the full list is returned rather than an empty one. +`filterOrAll` ensures that if all routers are "used," the full list is returned rather than an empty one. -`withNextSrv` is designed for retry loops — it re-reads user servers on each call (allowing configuration changes during retries) and tracks `triedHosts` across attempts. When all hosts are tried, the tried set is reset (`S.empty`), creating a round-robin effect. +`withNextSrv` is designed for retry loops — it re-reads user routers on each call (allowing configuration changes during retries) and tracks `triedHosts` across attempts. When all hosts are tried, the tried set is reset (`S.empty`), creating a round-robin effect. ## Locking primitives @@ -295,6 +295,6 @@ Classifies errors as temporary (retryable) or permanent. Notable non-obvious cla - `CRITICAL True` is temporary — `True` means the error shows a restart button, implying the user should retry. `CRITICAL False` is permanent. - `INACTIVE` is temporary — the agent may be reactivated - `SMP.PROXY NO_SESSION` via proxy is temporary — session can be re-established -- `SMP.STORE _` is temporary — server-side store error, not a client issue +- `SMP.STORE _` is temporary — router-side store error, not a client issue `temporaryOrHostError` extends `temporaryAgentError` to also include host-related errors (`HOST`, `TRANSPORT TEVersion`). Used in subscription management where host errors should trigger resubscription rather than permanent failure. diff --git a/spec/modules/Simplex/Messaging/Agent/Env/SQLite.md b/spec/modules/Simplex/Messaging/Agent/Env/SQLite.md index 7bfb10bbc..ec7852acf 100644 --- a/spec/modules/Simplex/Messaging/Agent/Env/SQLite.md +++ b/spec/modules/Simplex/Messaging/Agent/Env/SQLite.md @@ -6,4 +6,4 @@ ## mkUserServers — silent fallback on all-disabled -See comment on `mkUserServers`. If filtering servers by `enabled && role` yields an empty list, `fromMaybe srvs` falls back to *all* servers regardless of enabled/role status. This prevents a configuration where all servers are disabled from leaving the user with no servers — but means disabled servers can still be used if every server in a role is disabled. +See comment on `mkUserServers`. If filtering routers by `enabled && role` yields an empty list, `fromMaybe srvs` falls back to *all* routers regardless of enabled/role status. This prevents a configuration where all routers are disabled from leaving the user with no routers — but means disabled routers can still be used if every router in a role is disabled. diff --git a/spec/modules/Simplex/Messaging/Agent/NtfSubSupervisor.md b/spec/modules/Simplex/Messaging/Agent/NtfSubSupervisor.md index d55cfd746..ac591c192 100644 --- a/spec/modules/Simplex/Messaging/Agent/NtfSubSupervisor.md +++ b/spec/modules/Simplex/Messaging/Agent/NtfSubSupervisor.md @@ -6,7 +6,7 @@ ## Architecture -The notification system uses a supervisor with **three worker pools**, each keyed by server address: +The notification system uses a supervisor with **three worker pools**, each keyed by router address: | Pool | Key | Purpose | |------|-----|---------| @@ -23,7 +23,7 @@ The supervisor (`runNtfSupervisor`) reads commands from `ntfSubQ` and dispatches `partitionQueueSubActions` classifies each (queue, subscription) pair into one of four buckets: - **New sub**: no existing subscription record — create from scratch -- **Reset sub**: credentials mismatch (SMP server changed, notifier ID changed, action was nulled by error, or action is a delete) — wipe and restart from SMP key exchange +- **Reset sub**: credentials mismatch (SMP router changed, notifier ID changed, action was nulled by error, or action is a delete) — wipe and restart from SMP key exchange - **Continue SMP work**: existing action is `NSASMP` and credentials are consistent — kick the SMP worker - **Continue NTF work**: existing action is `NSANtf` and credentials are consistent — kick the NTF worker @@ -54,11 +54,11 @@ Successful check responses with statuses not in `subscribeNtfStatuses` also trig Token deletion splits into two phases: 1. **Store phase**: Remove token from active store, persist `(server, privateKey, tokenId)` to a deletion queue via `addNtfTokenToDelete` -2. **Network phase**: `runNtfTknDelWorker` reads from the queue and performs the actual server-side deletion +2. **Network phase**: `runNtfTknDelWorker` reads from the queue and performs the actual router-side deletion On supervisor startup, `startTknDelete` scans for any pending deletion queue entries and launches workers. This ensures token cleanup survives agent restarts. -If the token has no server-side ID (`ntfTokenId = Nothing`), only the store phase runs — no worker is launched. +If the token has no router-side ID (`ntfTokenId = Nothing`), only the store phase runs — no worker is launched. ### 6. workerErrors nulls subscription action @@ -88,7 +88,7 @@ When token deletion gets a permanent (non-temporary, non-host) error, the deleti ### 12. getNtfServer — random selection from multiple -When multiple notification routers are configured, one is selected randomly using `randomR` with a session-stable `TVar` generator. Single-server configurations skip the randomness. +When multiple notification routers are configured, one is selected randomly using `randomR` with a session-stable `TVar` generator. Single-router configurations skip the randomness. ### 13. closeNtfSupervisor — atomic swap then cancel diff --git a/spec/modules/Simplex/Messaging/Agent/Protocol.md b/spec/modules/Simplex/Messaging/Agent/Protocol.md index ad95df809..c6e65fbdf 100644 --- a/spec/modules/Simplex/Messaging/Agent/Protocol.md +++ b/spec/modules/Simplex/Messaging/Agent/Protocol.md @@ -64,9 +64,9 @@ The semicolon separator for SMP queues in the URI query string is deliberate — Short links encode `ContactConnType` as a single lowercase letter in the URL path: `a` (contact), `c` (channel), `g` (group), `r` (relay). Invitation links use `i`. The parser uses `toUpper` before dispatching to `ctTypeP` (which expects uppercase), while the encoder uses `toLower` on `ctTypeChar` output. This case dance happens because the wire format wants lowercase URLs but the internal representation uses uppercase. -## Short link server shortening +## Short link router shortening -`shortenShortLink` strips port and key hash from preset servers, leaving only the hostname (`SMPServerOnlyHost` pattern). This makes short links shorter for well-known servers. `restoreShortLink` reverses this by looking up the full server definition from the preset list. Both functions match on primary hostname only (first in the `NonEmpty` list). +`shortenShortLink` strips port and key hash from preset routers, leaving only the hostname (`SMPServerOnlyHost` pattern). This makes short links shorter for well-known routers. `restoreShortLink` reverses this by looking up the full router definition from the preset list. Both functions match on primary hostname only (first in the `NonEmpty` list). `isPresetServer` has a non-obvious port matching rule: empty port in the preset matches `"443"` or `"5223"` in the link. This handles servers that use default ports without explicitly listing them. diff --git a/spec/modules/Simplex/Messaging/Agent/Stats.md b/spec/modules/Simplex/Messaging/Agent/Stats.md index d793564e7..d501c3f7e 100644 --- a/spec/modules/Simplex/Messaging/Agent/Stats.md +++ b/spec/modules/Simplex/Messaging/Agent/Stats.md @@ -1,6 +1,6 @@ # Simplex.Messaging.Agent.Stats -> Per-server statistics counters (SMP, XFTP, NTF) with TVar-based live state and serializable snapshots. +> Per-router statistics counters (SMP, XFTP, NTF) with TVar-based live state and serializable snapshots. **Source**: [`Agent/Stats.hs`](../../../../../src/Simplex/Messaging/Agent/Stats.hs) diff --git a/spec/modules/Simplex/Messaging/Notifications/Protocol.md b/spec/modules/Simplex/Messaging/Notifications/Protocol.md index 71daf771d..fb718fd80 100644 --- a/spec/modules/Simplex/Messaging/Notifications/Protocol.md +++ b/spec/modules/Simplex/Messaging/Notifications/Protocol.md @@ -28,7 +28,7 @@ When encoding `NRTkn` responses, the `NTInvalid` reason is only included if the ### 4. subscribeNtfStatuses migration invariant -The comment on `subscribeNtfStatuses` (`[NSNew, NSPending, NSActive, NSInactive]`) warns that changing these statuses requires a new database migration for queue ID hashes (see `m20250830_queue_ids_hash`). This is a cross-module invariant between protocol types and server storage. +The comment on `subscribeNtfStatuses` (`[NSNew, NSPending, NSActive, NSInactive]`) warns that changing these statuses requires a new database migration for queue ID hashes (see `m20250830_queue_ids_hash`). This is a cross-module invariant between protocol types and router storage. ### 5. allowNtfSubCommands permits NTInvalid and NTExpired @@ -36,7 +36,7 @@ Token status `NTInvalid` allows subscription commands (SNEW, SCHK, SDEL), which ### 6. PPApnsNull test provider -`PPApnsNull` is a push provider that never communicates with APNS. It's used for end-to-end testing of the notification server from clients without requiring actual push infrastructure. +`PPApnsNull` is a push provider that never communicates with APNS. It's used for end-to-end testing of the notification router from clients without requiring actual push infrastructure. ### 7. DeviceToken hex validation @@ -44,7 +44,7 @@ Token status `NTInvalid` allows subscription commands (SNEW, SCHK, SDEL), which ### 8. SMPQueueNtf parsing applies updateSMPServerHosts -Both `smpP` and `strP` for `SMPQueueNtf` apply `updateSMPServerHosts` to the parsed SMP server. This normalizes server host addresses on deserialization, ensuring consistent comparison even if the on-wire format uses different host representations. +Both `smpP` and `strP` for `SMPQueueNtf` apply `updateSMPServerHosts` to the parsed SMP server. This normalizes router host addresses on deserialization, ensuring consistent comparison even if the on-wire format uses different host representations. ### 9. NRTknId response tag comment diff --git a/spec/modules/Simplex/Messaging/Notifications/Server.md b/spec/modules/Simplex/Messaging/Notifications/Server.md index 5c74878d7..d77a30a00 100644 --- a/spec/modules/Simplex/Messaging/Notifications/Server.md +++ b/spec/modules/Simplex/Messaging/Notifications/Server.md @@ -1,12 +1,12 @@ # Simplex.Messaging.Notifications.Server -> NTF server: manages tokens, subscriptions, SMP subscriber connections, and push notification delivery. +> NTF router: manages tokens, subscriptions, SMP subscriber connections, and push notification delivery. **Source**: [`Notifications/Server.hs`](../../../../../src/Simplex/Messaging/Notifications/Server.hs) ## Architecture -The NTF server runs several concurrent threads via `raceAny_`: +The NTF router runs several concurrent threads via `raceAny_`: | Thread | Purpose | |--------|---------| @@ -26,7 +26,7 @@ When `verifyNtfTransmission` encounters an AUTH error (entity not found), it cal ### 2. TNEW idempotent re-registration -When TNEW is received for an already-registered token, the server: +When TNEW is received for an already-registered token, the router: 1. Looks up the existing token via `findNtfTokenRegistration` (matches on push provider, device token, AND verify key) 2. Verifies the DH secret matches (recomputed from the new `dhPubKey` and stored `tknDhPrivKey`) 3. If DH secrets differ → AUTH error (prevents token hijacking) @@ -36,7 +36,7 @@ If the verify key doesn't match in step 1, the lookup returns `Nothing` and a ne ### 3. SNEW idempotent subscription -When SNEW is received for an existing subscription (same token + SMP queue), the server returns the existing `ntfSubId` if the notifier key matches. If keys differ, AUTH error. New subscriptions are only created when no match exists in `findNtfSubscription`. +When SNEW is received for an existing subscription (same token + SMP queue), the router returns the existing `ntfSubId` if the notifier key matches. If keys differ, AUTH error. New subscriptions are only created when no match exists in `findNtfSubscription`. ### 4. PPApnsNull suppresses statistics @@ -44,7 +44,7 @@ When SNEW is received for an existing subscription (same token + SMP queue), the ### 5. END requires active session validation -SMP END messages are only processed when the originating session is the currently active session for that server (`activeClientSession'` check). This prevents stale END messages from previous (reconnected) sessions from incorrectly marking subscriptions as ended. +SMP END messages are only processed when the originating session is the currently active session for that router (`activeClientSession'` check). This prevents stale END messages from previous (reconnected) sessions from incorrectly marking subscriptions as ended. ### 6. waitForSMPSubscriber two-phase wait @@ -52,9 +52,9 @@ SMP END messages are only processed when the originating session is the currentl ### 7. CAServiceUnavailable triggers individual resubscription -When a service subscription becomes unavailable (SMP server rejects service credentials), the NTF server: +When a service subscription becomes unavailable (SMP router rejects service credentials), the NTF router: 1. Removes the service association from the database -2. Resubscribes all individual queues for that server via `subscribeSrvSubs` +2. Resubscribes all individual queues for that router via `subscribeSrvSubs` This is the fallback path from service-level to queue-level SMP subscriptions. @@ -70,9 +70,9 @@ On the second failure, the error is logged and returned. `PPTokenInvalid` marks Cron notification interval has a hard minimum of 20 minutes. `TCRN 0` disables cron notifications. `TCRN n` where `1 <= n < 20` returns `QUOTA` error. -### 10. Startup resubscription is concurrent per server +### 10. Startup resubscription is concurrent per router -`resubscribe` uses `mapConcurrently` to resubscribe to all known SMP servers in parallel. Within each server, subscriptions are paginated via `subscribeLoop` using cursor-based pagination (`afterSubId_`). +`resubscribe` uses `mapConcurrently` to resubscribe to all known SMP routers in parallel. Within each router, subscriptions are paginated via `subscribeLoop` using cursor-based pagination (`afterSubId_`). ### 11. receive separates error responses from commands @@ -80,7 +80,7 @@ The `receive` function processes incoming transmissions and partitions results: ### 12. Maintenance mode saves state then exits immediately -When `maintenance` is set in `startOptions`, the server restores stats, calls `stopServer` (closes DB, saves stats), and exits with `exitSuccess`. It never starts transport listeners, subscriber threads, or resubscription. This provides a way to run database migrations without the server serving traffic. +When `maintenance` is set in `startOptions`, the router restores stats, calls `stopServer` (closes DB, saves stats), and exits with `exitSuccess`. It never starts transport listeners, subscriber threads, or resubscription. This provides a way to run database migrations without the router serving traffic. ### 13. Resubscription runs as a detached fork @@ -88,7 +88,7 @@ When `maintenance` is set in `startOptions`, the server restores stats, calls `s ### 14. TNEW re-registration resets status for non-verifiable tokens -When a re-registration TNEW matches on DH secret but `allowTokenVerification tknStatus` is `False` (token is `NTNew`, `NTInvalid`, or `NTExpired`), the server resets status to `NTRegistered` before sending the verification push. This makes TNEW a "status repair" mechanism — clients with stuck tokens can restart the verification flow by re-registering with the same DH key. +When a re-registration TNEW matches on DH secret but `allowTokenVerification tknStatus` is `False` (token is `NTNew`, `NTInvalid`, or `NTExpired`), the router resets status to `NTRegistered` before sending the verification push. This makes TNEW a "status repair" mechanism — clients with stuck tokens can restart the verification flow by re-registering with the same DH key. ### 15. DELD unconditionally updates status (no session validation) @@ -96,7 +96,7 @@ Unlike `SMP.END` which checks `activeClientSession'` to prevent stale session me ### 16. TRPL generates new code but reuses the DH key -`TRPL` (token replace) creates a new registration code and resets status to `NTRegistered`, but does NOT generate a new server DH key pair. The existing `tknDhPrivKey` and `tknDhSecret` are preserved — only the push provider token and registration code change. The encrypted channel between client and NTF router persists across device token replacements. +`TRPL` (token replace) creates a new registration code and resets status to `NTRegistered`, but does NOT generate a new router DH key pair. The existing `tknDhPrivKey` and `tknDhSecret` are preserved — only the push provider token and registration code change. The encrypted channel between client and NTF router persists across device token replacements. ### 17. PNMessage delivery requires NTActive, verification and cron do not @@ -112,7 +112,7 @@ When a service subscription is confirmed, the NTF router compares expected and c ### 20. subscribeLoop calls exitFailure on database error -If `getServerNtfSubscriptions` returns `Left _` during startup resubscription, the server terminates via `exitFailure`. Since `resubscribe` runs in a forked thread (pattern 13), this `exitFailure` terminates the entire process — a transient database error during startup resubscription kills the server. +If `getServerNtfSubscriptions` returns `Left _` during startup resubscription, the router terminates via `exitFailure`. Since `resubscribe` runs in a forked thread (pattern 13), this `exitFailure` terminates the entire process — a transient database error during startup resubscription kills the router. ### 21. Stats log aligns to wall-clock time of day @@ -120,7 +120,7 @@ The stats logging thread calculates an `initialDelay` to synchronize the first f ### 22. NMSG AUTH errors silently counted, not logged -When `addTokenLastNtf` returns `Left AUTH` (notification for a queue whose subscription/token association is invalid), the server increments `ntfReceivedAuth` but takes no corrective action. Other error types are silently ignored. This is expected — subscriptions may be deleted while messages are in-flight. +When `addTokenLastNtf` returns `Left AUTH` (notification for a queue whose subscription/token association is invalid), the router increments `ntfReceivedAuth` but takes no corrective action. Other error types are silently ignored. This is expected — subscriptions may be deleted while messages are in-flight. ### 23. PNVerification delivery transitions token to NTConfirmed diff --git a/spec/modules/Simplex/Messaging/Notifications/Server/Control.md b/spec/modules/Simplex/Messaging/Notifications/Server/Control.md index 897f81c16..cbdb5b416 100644 --- a/spec/modules/Simplex/Messaging/Notifications/Server/Control.md +++ b/spec/modules/Simplex/Messaging/Notifications/Server/Control.md @@ -1,6 +1,6 @@ # Simplex.Messaging.Notifications.Server.Control -> Control port command protocol for NTF server administration. +> Control port command protocol for NTF router administration. **Source**: [`Notifications/Server/Control.hs`](../../../../../../src/Simplex/Messaging/Notifications/Server/Control.hs) diff --git a/spec/modules/Simplex/Messaging/Notifications/Server/Env.md b/spec/modules/Simplex/Messaging/Notifications/Server/Env.md index c266390d2..17ae63862 100644 --- a/spec/modules/Simplex/Messaging/Notifications/Server/Env.md +++ b/spec/modules/Simplex/Messaging/Notifications/Server/Env.md @@ -1,6 +1,6 @@ # Simplex.Messaging.Notifications.Server.Env -> NTF server environment: configuration, subscriber state, and push provider management. +> NTF router environment: configuration, subscriber state, and push provider management. **Source**: [`Notifications/Server/Env.hs`](../../../../../../src/Simplex/Messaging/Notifications/Server/Env.hs) @@ -8,7 +8,7 @@ ### 1. Service credentials are lazily generated -`mkDbService` in `newNtfServerEnv` generates service credentials on demand: when `getCredentials` is called for an SMP server, it checks the database. If the server is known and already has credentials, they are reused. If the server is known but has no credentials yet (first connection), new credentials are generated via `genCredentials`, stored in the database, and returned. If the server is not in the database at all, `PCEServiceUnavailable` is thrown (this case should not occur in practice, as clients only connect to servers already tracked in the database). +`mkDbService` in `newNtfServerEnv` generates service credentials on demand: when `getCredentials` is called for an SMP router, it checks the database. If the router is known and already has credentials, they are reused. If the router is known but has no credentials yet (first connection), new credentials are generated via `genCredentials`, stored in the database, and returned. If the router is not in the database at all, `PCEServiceUnavailable` is thrown (this case should not occur in practice, as clients only connect to routers already tracked in the database). Service credentials are only used when `useServiceCreds` is enabled in the config. @@ -18,7 +18,7 @@ Service credentials are only used when `useServiceCreds` is enabled in the confi ### 3. getPushClient lazy initialization -`getPushClient` looks up the push client by provider in `pushClients` TMap. If not found, it calls `newPushClient` to create and register one. Push provider connections are established on first use, not at server startup. +`getPushClient` looks up the push client by provider in `pushClients` TMap. If not found, it calls `newPushClient` to create and register one. Push provider connections are established on first use, not at router startup. ### 4. Service credential validity: 25h backdating, ~2700yr forward diff --git a/spec/modules/Simplex/Messaging/Notifications/Server/Main.md b/spec/modules/Simplex/Messaging/Notifications/Server/Main.md index 3719dcd97..54136f1c3 100644 --- a/spec/modules/Simplex/Messaging/Notifications/Server/Main.md +++ b/spec/modules/Simplex/Messaging/Notifications/Server/Main.md @@ -1,6 +1,6 @@ # Simplex.Messaging.Notifications.Server.Main -> CLI interface and INI configuration parsing for the NTF server. +> CLI interface and INI configuration parsing for the NTF router. **Source**: [`Notifications/Server/Main.hs`](../../../../../../src/Simplex/Messaging/Notifications/Server/Main.hs) diff --git a/spec/modules/Simplex/Messaging/Notifications/Server/Push/APNS.md b/spec/modules/Simplex/Messaging/Notifications/Server/Push/APNS.md index d2a49471d..3fd2bd880 100644 --- a/spec/modules/Simplex/Messaging/Notifications/Server/Push/APNS.md +++ b/spec/modules/Simplex/Messaging/Notifications/Server/Push/APNS.md @@ -72,4 +72,4 @@ The comment explicitly states `APNSErrorResponse` is `data` rather than `newtype ### 17. Connection initialization is fire-and-forget -`createAPNSPushClient` calls `connectHTTPS2` and discards the result with `void`. If the initial connection fails, the error is only logged — the client is still created. The first push delivery triggers `getApnsHTTP2Client` which reconnects. This means the server can start even if APNS is unreachable. +`createAPNSPushClient` calls `connectHTTPS2` and discards the result with `void`. If the initial connection fails, the error is only logged — the client is still created. The first push delivery triggers `getApnsHTTP2Client` which reconnects. This means the router can start even if APNS is unreachable. diff --git a/spec/modules/Simplex/Messaging/Notifications/Server/Stats.md b/spec/modules/Simplex/Messaging/Notifications/Server/Stats.md index 4a4439f54..d954f03d1 100644 --- a/spec/modules/Simplex/Messaging/Notifications/Server/Stats.md +++ b/spec/modules/Simplex/Messaging/Notifications/Server/Stats.md @@ -1,6 +1,6 @@ # Simplex.Messaging.Notifications.Server.Stats -> NTF server statistics collection with own-server breakdown and backward-compatible persistence. +> NTF router statistics collection with own-router breakdown and backward-compatible persistence. **Source**: [`Notifications/Server/Stats.hs`](../../../../../../src/Simplex/Messaging/Notifications/Server/Stats.hs) @@ -8,27 +8,27 @@ ### 1. incServerStat double lookup -`incServerStat` performs a non-STM IO lookup first. On cache hit, the STM transaction only touches the per-server `TVar Int` without reading the shared TMap, avoiding contention. On cache miss, the STM block re-checks the map to handle races (another thread may have inserted between the IO lookup and STM entry). +`incServerStat` performs a non-STM IO lookup first. On cache hit, the STM transaction only touches the per-router `TVar Int` without reading the shared TMap, avoiding contention. On cache miss, the STM block re-checks the map to handle races (another thread may have inserted between the IO lookup and STM entry). ### 2. setNtfServerStats is not thread safe -`setNtfServerStats` is explicitly documented as non-thread-safe and intended for server startup only (restoring from backup file). +`setNtfServerStats` is explicitly documented as non-thread-safe and intended for router startup only (restoring from backup file). ### 3. Backward-compatible parsing -The `strP` parser uses `opt` which defaults missing fields to 0. This allows reading stats files from older server versions that don't include newer fields (`ntfReceivedAuth`, `ntfFailed`, `ntfVrf*`, etc.). +The `strP` parser uses `opt` which defaults missing fields to 0. This allows reading stats files from older router versions that don't include newer fields (`ntfReceivedAuth`, `ntfFailed`, `ntfVrf*`, etc.). ### 4. getNtfServerStatsData is a non-atomic snapshot -`getNtfServerStatsData` reads each `IORef` and `TMap` field sequentially in plain `IO`, not inside a single STM transaction. The returned `NtfServerStatsData` is not a consistent point-in-time snapshot — invariants like "received >= delivered" may not hold. The same applies to `getStatsByServer`, which does one `readTVarIO` for the map root TVar, then a separate `readTVarIO` for each per-server TVar. This is acceptable for periodic reporting where approximate consistency suffices. +`getNtfServerStatsData` reads each `IORef` and `TMap` field sequentially in plain `IO`, not inside a single STM transaction. The returned `NtfServerStatsData` is not a consistent point-in-time snapshot — invariants like "received >= delivered" may not hold. The same applies to `getStatsByServer`, which does one `readTVarIO` for the map root TVar, then a separate `readTVarIO` for each per-router TVar. This is acceptable for periodic reporting where approximate consistency suffices. ### 5. Mixed IORef/TVar concurrency primitives -Aggregate counters (`ntfReceived`, `ntfDelivered`, etc.) use `IORef Int` incremented via `atomicModifyIORef'_`, while per-server breakdowns use `TMap Text (TVar Int)` incremented atomically via STM in `incServerStat`. Although both individual operations are atomic, the aggregate and per-server increments are separate operations, so their values can drift: a thread could increment the aggregate `IORef` before `incServerStat` runs, or vice versa. +Aggregate counters (`ntfReceived`, `ntfDelivered`, etc.) use `IORef Int` incremented via `atomicModifyIORef'_`, while per-router breakdowns use `TMap Text (TVar Int)` incremented atomically via STM in `incServerStat`. Although both individual operations are atomic, the aggregate and per-router increments are separate operations, so their values can drift: a thread could increment the aggregate `IORef` before `incServerStat` runs, or vice versa. ### 6. setStatsByServer replaces TMap atomically but orphans old TVars -`setStatsByServer` builds a fresh `Map Text (TVar Int)` in IO via `newTVarIO`, then atomically replaces the TMap's root TVar. Old per-server TVars are not reused — any other thread holding a reference from a prior `TM.lookupIO` would modify an orphaned counter. Safe only because it's called at startup (like `setNtfServerStats`), but lacks the explicit "not thread safe" comment. +`setStatsByServer` builds a fresh `Map Text (TVar Int)` in IO via `newTVarIO`, then atomically replaces the TMap's root TVar. Old per-router TVars are not reused — any other thread holding a reference from a prior `TM.lookupIO` would modify an orphaned counter. Safe only because it's called at startup (like `setNtfServerStats`), but lacks the explicit "not thread safe" comment. ### 7. Positional parser format despite key=value appearance diff --git a/spec/modules/Simplex/Messaging/Notifications/Server/Store.md b/spec/modules/Simplex/Messaging/Notifications/Server/Store.md index 05a7e70e2..d9deedbf4 100644 --- a/spec/modules/Simplex/Messaging/Notifications/Server/Store.md +++ b/spec/modules/Simplex/Messaging/Notifications/Server/Store.md @@ -36,7 +36,7 @@ When `stmDeleteNtfToken` removes a token, it deletes the entry from the inner `T ### 8. deleteTokenSubs returns SMP queues for upstream unsubscription -`deleteTokenSubs` atomically collects all `SMPQueueNtf` values from the deleted subscriptions and returns them. This is how the server layer knows which SMP notifier subscriptions to tear down. `stmRemoveInactiveTokenRegistrations` discards this list (`void $`), meaning rival-token cleanup does **not** trigger SMP unsubscription — only explicit token deletion does. +`deleteTokenSubs` atomically collects all `SMPQueueNtf` values from the deleted subscriptions and returns them. This is how the router layer knows which SMP notifier subscriptions to tear down. `stmRemoveInactiveTokenRegistrations` discards this list (`void $`), meaning rival-token cleanup does **not** trigger SMP unsubscription — only explicit token deletion does. ### 9. stmAddNtfSubscription always returns Just (vestigial Maybe) @@ -48,7 +48,7 @@ When `stmDeleteNtfSubscription` removes a subscription, it deletes the `subId` f ### 11. stmSetNtfService — asymmetric cleanup with Postgres store -`stmSetNtfService` uses `maybe TM.delete TM.insert` to either remove or set the service association for an SMP server. This is purely a key-value update with no cascading effects on subscriptions. The Postgres store's `removeServiceAndAssociations` handles subscription cleanup separately, meaning the STM and Postgres stores have **different cleanup semantics** for service removal. +`stmSetNtfService` uses `maybe TM.delete TM.insert` to either remove or set the service association for an SMP router. This is purely a key-value update with no cascading effects on subscriptions. The Postgres store's `removeServiceAndAssociations` handles subscription cleanup separately, meaning the STM and Postgres stores have **different cleanup semantics** for service removal. ### 12. Subscription index triple-write invariant diff --git a/spec/modules/Simplex/Messaging/Notifications/Server/Store/Postgres.md b/spec/modules/Simplex/Messaging/Notifications/Server/Store/Postgres.md index 440797539..bde863eb6 100644 --- a/spec/modules/Simplex/Messaging/Notifications/Server/Store/Postgres.md +++ b/spec/modules/Simplex/Messaging/Notifications/Server/Store/Postgres.md @@ -8,7 +8,7 @@ ### 1. deleteNtfToken exclusive row lock -`deleteNtfToken` acquires `FOR UPDATE` on the token row before cascading deletes. This prevents concurrent subscription inserts for this token during the deletion window. The subscriptions are aggregated by SMP server and returned for in-memory subscription cleanup. +`deleteNtfToken` acquires `FOR UPDATE` on the token row before cascading deletes. This prevents concurrent subscription inserts for this token during the deletion window. The subscriptions are aggregated by SMP router and returned for in-memory subscription cleanup. ### 2. addTokenLastNtf atomic CTE @@ -47,11 +47,11 @@ Only non-service-associated subscriptions (`NOT ntf_service_assoc`) are returned ### 9. Server upsert optimization -`addNtfSubscription` first tries a plain SELECT for the SMP server, then falls back to INSERT with ON CONFLICT only if the server doesn't exist. This avoids the upsert overhead in the common case where the server already exists. +`addNtfSubscription` first tries a plain SELECT for the SMP router, then falls back to INSERT with ON CONFLICT only if the router doesn't exist. This avoids the upsert overhead in the common case where the router already exists. ### 10. Service association tracking -`batchUpdateSrvSubStatus` atomically updates both subscription status and `ntf_service_assoc` flag. When notifications arrive via a service subscription (`newServiceId` is `Just`), all affected subscriptions are marked as service-associated. `removeServiceAndAssociations` resets all subscriptions for a server to `NSInactive` with `ntf_service_assoc = FALSE`. +`batchUpdateSrvSubStatus` atomically updates both subscription status and `ntf_service_assoc` flag. When notifications arrive via a service subscription (`newServiceId` is `Just`), all affected subscriptions are marked as service-associated. `removeServiceAndAssociations` resets all subscriptions for a router to `NSInactive` with `ntf_service_assoc = FALSE`. ### 11. uninterruptibleMask_ wraps most store operations @@ -63,7 +63,7 @@ Only non-service-associated subscriptions (`NOT ntf_service_assoc`) are returned ### 13. getUsedSMPServers uncorrelated EXISTS -The `EXISTS` subquery in `getUsedSMPServers` has no join condition to the outer `smp_servers` table — it returns ALL servers if ANY subscription anywhere has a subscribable status. This is intentional for server startup: the server needs all SMP server records (including `ServiceSub` data) to rebuild in-memory state, and the EXISTS clause is a cheap guard against an empty subscription table. +The `EXISTS` subquery in `getUsedSMPServers` has no join condition to the outer `smp_servers` table — it returns ALL servers if ANY subscription anywhere has a subscribable status. This is intentional for router startup: the router needs all SMP router records (including `ServiceSub` data) to rebuild in-memory state, and the EXISTS clause is a cheap guard against an empty subscription table. ### 14. Trigger-maintained XOR hash aggregates diff --git a/spec/modules/Simplex/Messaging/Notifications/Transport.md b/spec/modules/Simplex/Messaging/Notifications/Transport.md index df4021475..9b94d7e0d 100644 --- a/spec/modules/Simplex/Messaging/Notifications/Transport.md +++ b/spec/modules/Simplex/Messaging/Notifications/Transport.md @@ -33,6 +33,6 @@ NTF uses a 512-byte block size (`ntfBlockSize`), significantly smaller than SMP. `ntfTHandle` creates a THandle with `thVersion = VersionNTF 0` — a version that no real protocol supports. This is a placeholder value that gets overwritten during version negotiation. All feature gates check `v >= authBatchCmdsNTFVersion` (v2), so the v0 placeholder disables all optional features. -### 6. Server handshake always sends authPubKey +### 6. Router handshake always sends authPubKey -`ntfServerHandshake` always includes `authPubKey = Just sk` in the server handshake, regardless of the advertised version range. The encoding functions (`encodeAuthEncryptCmds`) then decide whether to actually serialize it based on the max version. This means the key is computed even when it won't be sent. +`ntfServerHandshake` always includes `authPubKey = Just sk` in the router handshake, regardless of the advertised version range. The encoding functions (`encodeAuthEncryptCmds`) then decide whether to actually serialize it based on the max version. This means the key is computed even when it won't be sent. diff --git a/spec/modules/Simplex/Messaging/Notifications/Types.md b/spec/modules/Simplex/Messaging/Notifications/Types.md index 97cc66913..576d9c088 100644 --- a/spec/modules/Simplex/Messaging/Notifications/Types.md +++ b/spec/modules/Simplex/Messaging/Notifications/Types.md @@ -16,4 +16,4 @@ ### 3. NSADelete and NSARotate are deprecated -These `NtfSubNTFAction` values are no longer generated by current code but are retained in the type for processing legacy database records. `NSARotate` is logically "delete + recreate" while `NSADelete` is "delete subscription on NTF server + delete notifier credentials on SMP server". +These `NtfSubNTFAction` values are no longer generated by current code but are retained in the type for processing legacy database records. `NSARotate` is logically "delete + recreate" while `NSADelete` is "delete subscription on NTF router + delete notifier credentials on SMP router".