From bde90500ea50ef180ba95abc4b7393f8c37119ef Mon Sep 17 00:00:00 2001 From: "Evgeny @ SimpleX Chat" <259188159+evgeny-simplex@users.noreply.github.com> Date: Fri, 13 Mar 2026 12:00:22 +0000 Subject: [PATCH] agent store and notifications specs --- .../Messaging/Agent/NtfSubSupervisor.md | 75 ++++++++++++++++++ spec/modules/Simplex/Messaging/Agent/Store.md | 44 +++++++++++ .../Messaging/Agent/Store/AgentStore.md | 76 ++++++++++++++++++ .../Simplex/Messaging/Agent/Store/Common.md | 7 ++ .../Simplex/Messaging/Agent/Store/DB.md | 7 ++ .../Simplex/Messaging/Agent/Store/Entity.md | 7 ++ .../Messaging/Agent/Store/Interface.md | 7 ++ .../Simplex/Messaging/Agent/Store/Postgres.md | 23 ++++++ .../Simplex/Messaging/Agent/Store/SQLite.md | 26 ++++++ .../Simplex/Messaging/Agent/Store/Shared.md | 7 ++ .../Simplex/Messaging/Notifications/Client.md | 15 ++++ .../Messaging/Notifications/Protocol.md | 43 ++++++++++ .../Simplex/Messaging/Notifications/Server.md | 79 +++++++++++++++++++ .../Messaging/Notifications/Server/Control.md | 7 ++ .../Messaging/Notifications/Server/Env.md | 21 +++++ .../Messaging/Notifications/Server/Main.md | 7 ++ .../Notifications/Server/Push/APNS.md | 35 ++++++++ .../Server/Push/APNS/Internal.md | 7 ++ .../Messaging/Notifications/Server/Stats.md | 19 +++++ .../Messaging/Notifications/Server/Store.md | 23 ++++++ .../Notifications/Server/Store/Postgres.md | 54 +++++++++++++ .../Notifications/Server/Store/Types.md | 7 ++ .../Messaging/Notifications/Transport.md | 36 ++++----- .../Simplex/Messaging/Notifications/Types.md | 19 +++++ 24 files changed, 630 insertions(+), 21 deletions(-) create mode 100644 spec/modules/Simplex/Messaging/Agent/NtfSubSupervisor.md create mode 100644 spec/modules/Simplex/Messaging/Agent/Store.md create mode 100644 spec/modules/Simplex/Messaging/Agent/Store/AgentStore.md create mode 100644 spec/modules/Simplex/Messaging/Agent/Store/Common.md create mode 100644 spec/modules/Simplex/Messaging/Agent/Store/DB.md create mode 100644 spec/modules/Simplex/Messaging/Agent/Store/Entity.md create mode 100644 spec/modules/Simplex/Messaging/Agent/Store/Interface.md create mode 100644 spec/modules/Simplex/Messaging/Agent/Store/Postgres.md create mode 100644 spec/modules/Simplex/Messaging/Agent/Store/SQLite.md create mode 100644 spec/modules/Simplex/Messaging/Agent/Store/Shared.md create mode 100644 spec/modules/Simplex/Messaging/Notifications/Client.md create mode 100644 spec/modules/Simplex/Messaging/Notifications/Protocol.md create mode 100644 spec/modules/Simplex/Messaging/Notifications/Server.md create mode 100644 spec/modules/Simplex/Messaging/Notifications/Server/Control.md create mode 100644 spec/modules/Simplex/Messaging/Notifications/Server/Env.md create mode 100644 spec/modules/Simplex/Messaging/Notifications/Server/Main.md create mode 100644 spec/modules/Simplex/Messaging/Notifications/Server/Push/APNS.md create mode 100644 spec/modules/Simplex/Messaging/Notifications/Server/Push/APNS/Internal.md create mode 100644 spec/modules/Simplex/Messaging/Notifications/Server/Stats.md create mode 100644 spec/modules/Simplex/Messaging/Notifications/Server/Store.md create mode 100644 spec/modules/Simplex/Messaging/Notifications/Server/Store/Postgres.md create mode 100644 spec/modules/Simplex/Messaging/Notifications/Server/Store/Types.md create mode 100644 spec/modules/Simplex/Messaging/Notifications/Types.md diff --git a/spec/modules/Simplex/Messaging/Agent/NtfSubSupervisor.md b/spec/modules/Simplex/Messaging/Agent/NtfSubSupervisor.md new file mode 100644 index 000000000..33cd3eacb --- /dev/null +++ b/spec/modules/Simplex/Messaging/Agent/NtfSubSupervisor.md @@ -0,0 +1,75 @@ +# Simplex.Messaging.Agent.NtfSubSupervisor + +> Supervisor-worker architecture for notification subscription lifecycle management. + +**Source**: [`Agent/NtfSubSupervisor.hs`](../../../../../src/Simplex/Messaging/Agent/NtfSubSupervisor.hs) + +## Architecture + +The notification system uses a supervisor with **three worker pools**, each keyed by server address: + +| Pool | Key | Purpose | +|------|-----|---------| +| `ntfWorkers` | NtfServer | Create/check/delete/rotate subscriptions on notification router | +| `ntfSMPWorkers` | SMPServer | Create/delete notifier credentials on messaging router | +| `ntfTknDelWorkers` | NtfServer | Delete tokens on notification router (background cleanup) | + +The supervisor (`runNtfSupervisor`) reads commands from `ntfSubQ` and dispatches work to the appropriate pools. Workers are created lazily via `getAgentWorker` and process batches from the database. + +## Non-obvious behavior + +### 1. NSCCreate four-way partition + +`partitionQueueSubActions` classifies each (queue, subscription) pair into one of four buckets: + +- **New sub**: no existing subscription record — create from scratch +- **Reset sub**: credentials mismatch (SMP server changed, notifier ID changed, action was nulled by error, or action is a delete) — wipe and restart from SMP key exchange +- **Continue SMP work**: existing action is `NSASMP` and credentials are consistent — kick the SMP worker +- **Continue NTF work**: existing action is `NSANtf` and credentials are consistent — kick the NTF worker + +The key decision point: when `subAction_` is `Nothing` (set by `workerErrors` after permanent failures), the subscription is treated as needing a full reset. This interacts with the null-action sentinel pattern from `AgentStore`. + +### 2. retrySubActions shrinking retry with TVar + +`retrySubActions` holds the list of subs-to-retry in a `TVar`. Each iteration, the action function returns only the subs that got temporary errors (via `splitResults`). The `TVar` is overwritten with this shrinking list. On success or permanent error, subs drop out. This means retry batches get smaller over time. + +`splitResults` implements a three-way partition: temporary errors → retry, permanent errors → null the action + notify, successes → continue pipeline. + +### 3. rescheduleWork deferred wake-up + +When the NTF worker finds that all pending `NSACheck` actions have future timestamps, it does not spin-wait. Instead it: +1. Takes itself out of the `doWork` TMVar (so the worker blocks on `waitForWork`) +2. Forks a thread that sleeps until the first action's timestamp +3. The forked thread re-signals `doWork` when the time arrives + +This is the mechanism for time-scheduled subscription health checks. + +### 4. checkSubs AUTH triggers full recreation + +When the notification router returns `AUTH` for a subscription check, the subscription is not simply marked as failed — it is fully recreated from scratch by resetting to `NSASMP NSASmpKey` state. This handles the case where the notification router has lost its subscription state (restart, data loss). The SMP worker is kicked to re-establish notifier credentials. + +Non-AUTH failure statuses that are not in `subscribeNtfStatuses` also trigger recreation. + +### 5. deleteToken two-phase with restart survival + +Token deletion splits into two phases: +1. **Store phase**: Remove token from active store, persist `(server, privateKey, tokenId)` to a deletion queue via `addNtfTokenToDelete` +2. **Network phase**: `runNtfTknDelWorker` reads from the queue and performs the actual server-side deletion + +On supervisor startup, `startTknDelete` scans for any pending deletion queue entries and launches workers. This ensures token cleanup survives agent restarts. + +If the token has no server-side ID (`ntfTokenId = Nothing`), only the store phase runs — no worker is launched. + +### 6. workerErrors nulls subscription action + +When permanent (non-temporary, non-host) errors occur in batch operations, `workerErrors` sets the subscription's action to `NULL` in the database and notifies the client. The next `NSCCreate` for that connection will see `subAction_ = Nothing` in `contOrReset` and trigger a full subscription reset. + +This null-action sentinel is the bridge between worker failure recovery and supervisor-driven re-creation. + +### 7. NSADelete and NSARotate are deprecated + +These NTF worker actions are no longer generated by current code but are kept for processing legacy database records. They are explicitly not batched (processed one at a time via `mapM`). `NSARotate` deletes the subscription then re-queues `NSCCreate` back to the supervisor. + +### 8. Stats counting groups by userId + +`incStatByUserId` groups batch subscriptions by `userId` before incrementing stats counters, ensuring per-user counts are accurate even when a single batch contains subscriptions from multiple users. diff --git a/spec/modules/Simplex/Messaging/Agent/Store.md b/spec/modules/Simplex/Messaging/Agent/Store.md new file mode 100644 index 000000000..0eecbf8d1 --- /dev/null +++ b/spec/modules/Simplex/Messaging/Agent/Store.md @@ -0,0 +1,44 @@ +# Simplex.Messaging.Agent.Store + +> Domain entity types for agent persistence — queues, connections, messages, commands, and store errors. + +**Source**: [`Agent/Store.hs`](../../../../../src/Simplex/Messaging/Agent/Store.hs) + +## Overview + +This module defines the data types that represent agent state. It contains no database operations — those are in [AgentStore.hs](./Store/AgentStore.md). The key abstractions are: + +- **Queue types** (`StoredRcvQueue`, `StoredSndQueue`) parameterized by `DBStored` phantom type for new vs persisted distinction +- **Connection GADT** (`Connection'`) encoding the connection state machine at the type level +- **Message containers** (`RcvMsgData`, `SndMsgData`, `PendingMsgData`) for the message lifecycle +- **Store errors** (`StoreError`) including two sentinel errors with special semantics + +## Connection' — type-level state machine + +The `Connection'` GADT encodes connection lifecycle as a type parameter: `CNew` → `CRcv`/`CSnd` → `CDuplex`, plus `CContact` for reusable contact connections. `SomeConn` wraps an existential to store connections of unknown type. + +`TestEquality SConnType` deliberately omits `SCNew` — `testEquality SCNew SCNew` returns `Nothing`. This is intentional: `NewConnection` has no queues and is not a valid target for type-level connection matching in store operations. + +## canAbortRcvSwitch — race condition boundary + +See comments on `canAbortRcvSwitch`. The `RSSendingQUSE` and `RSReceivedMessage` states cannot be aborted because the sender may have already deleted the original queue. Aborting (deleting the new queue) at that point would break the connection with no recovery path. + +## ratchetSyncAllowed / ratchetSyncSendProhibited — cross-repo contract + +See comments on `ratchetSyncAllowed`. Both functions carry the comment "this function should be mirrored in the clients" — simplex-chat must implement identical logic. The agent enforces these state checks, but the chat client also needs them for UI decisions (e.g., disabling send when `ratchetSyncSendProhibited`). + +## SEWorkItemError — worker suspension sentinel + +`SEWorkItemError` is a sentinel error that triggers worker suspension when encountered during work item retrieval. The `AnyStoreError` typeclass exposes `isWorkItemError` for the worker framework ([Agent/Client.hs](./Client.md)) to detect this case. The comment "do not use!" means it should not be thrown for normal error conditions — only when the work item itself is corrupt/unreadable and the worker should stop rather than retry. + +## SEAgentError — store-level error wrapping + +`SEAgentError` wraps `AgentErrorType` inside store operations. This allows store functions to return agent-level errors (e.g., connection state violations detected during a DB transaction) without breaking the `ExceptT StoreError` type. The "to avoid race conditions" rationale: checking a condition and acting on it must happen in the same DB transaction, so the agent error is returned through the store error channel. + +## InvShortLink — secure-on-read semantics + +See comment on `InvShortLink`. Stored separately from the connection because 1-time invitation short links have a "secure-on-read" property: accessing the link data on the router marks it as read, preventing undetected observation. The `sndPrivateKey` is persisted to allow retries of the link creation without generating new keys. + +## RcvQueueSub — subscription-optimized projection + +`RcvQueueSub` strips cryptographic fields from `RcvQueue`, keeping only what's needed for subscription tracking in [TSessionSubs](./TSessionSubs.md). This reduces memory pressure when tracking thousands of subscriptions in STM. diff --git a/spec/modules/Simplex/Messaging/Agent/Store/AgentStore.md b/spec/modules/Simplex/Messaging/Agent/Store/AgentStore.md new file mode 100644 index 000000000..9fbc2beb3 --- /dev/null +++ b/spec/modules/Simplex/Messaging/Agent/Store/AgentStore.md @@ -0,0 +1,76 @@ +# Simplex.Messaging.Agent.Store.AgentStore + +> Core CRUD operations for agent persistence — users, connections, queues, messages, ratchets, notifications, and file transfers. + +**Source**: [`Agent/Store/AgentStore.hs`](../../../../../../src/Simplex/Messaging/Agent/Store/AgentStore.hs) + +## Overview + +At ~3700 lines, this is the largest module in the codebase. It implements all database operations for the agent, compiled with CPP for both SQLite and PostgreSQL backends. Most functions are straightforward SQL CRUD, but several patterns are non-obvious. + +The module re-exports `withConnection`, `withTransaction`, `withTransactionPriority`, `firstRow`, `firstRow'`, `maybeFirstRow`, and `fromOnlyBI` from the backend-specific Common module. + +## Dual-backend compilation + +The module uses `#if defined(dbPostgres)` throughout. Key behavioral differences: +- **Row locking**: PostgreSQL uses `FOR UPDATE` on reads that precede writes (e.g., `getConnForUpdate`, `getRatchetForUpdate`, `retrieveLastIdsAndHashRcv_`). SQLite relies on its single-writer model instead. +- **Batch queries**: PostgreSQL uses `IN ?` with `In` wrapper for batch operations. SQLite falls back to per-row `forM` loops. +- **Constraint handling**: PostgreSQL uses `constraintViolation`, SQLite checks `SQL.ErrorConstraint`. + +## getWorkItem / getWorkItems — worker store pattern + +`getWorkItem` implements the store-side pattern for the [worker framework](../Client.md): `getId → getItem → markFailed`. If `getId` or `getItem` throws an IO exception, `handleWrkErr` wraps it as `SEWorkItemError` (via `mkWorkItemError`), which signals the worker to suspend rather than retry. This prevents crash loops on corrupt data. + +`getWorkItems` extends this to batch work items, where each item failure is independent. + +**Consumed by**: `getPendingQueueMsg`, `getPendingServerCommand`, `getNextNtfSubNTFActions`, `getNextNtfSubSMPActions`, `getNextDeletedSndChunkReplica`, `getNextNtfTokenToDelete`. + +## Notification subscription — supervisor/worker coordination + +`updateNtfSubscription`, `setNullNtfSubscriptionAction`, and `deleteNtfSubscription` all check `updated_by_supervisor` before writing. When `True`, the worker only updates local fields (ntf IDs, status) and skips action/server fields that the supervisor may have changed. This prevents the worker from overwriting supervisor decisions during concurrent execution. + +`markUpdatedByWorker` resets the flag to `False` before each work item is processed, so the worker "claims" the subscription for the duration of its operation. + +## createServer / getServerKeyHash_ — key hash migration + +`createServer` returns `Maybe KeyHash`: `Nothing` means the server was newly created with the passed hash; `Just kh` means the server already existed and the passed hash differs from the stored one. This `Just` value is stored as `server_key_hash` on queues to allow per-queue key hash overrides. + +The `COALESCE(q.server_key_hash, s.key_hash)` pattern appears throughout queries — queues can override the server-level hash, enabling gradual migration when a router's identity key changes. + +## updateRcvMsgHash / updateSndMsgHash — race condition guard + +Both functions include `AND last_internal_*_msg_id = ?` in their UPDATE WHERE clause. This prevents a race: if another message was processed between `updateIds` and `updateHash` (incrementing the last ID), the hash update is silently skipped rather than corrupting the chain. See comments on these functions. + +## deleteConn — conditional delivery wait + +Three deletion paths: +1. No timeout: immediate delete. +2. Timeout + no pending deliveries: immediate delete. +3. Timeout + pending deliveries + `deleted_at_wait_delivery` expired: delete. +4. Timeout + pending deliveries + not expired: return `Nothing` (skip). + +This allows graceful delivery completion before connection cleanup. + +## createSndConn — confirmed queue guard + +See comment on `createSndConn`. Checks `checkConfirmedSndQueueExists_` before creating, because `insertSndQueue_` uses `ON CONFLICT DO UPDATE` which would silently replace an existing confirmed send queue. The pre-check prevents this destructive upsert. + +## insertRcvQueue_ / insertSndQueue_ — queue ID preservation + +Both functions check if a queue already exists (by server + queue ID) and reuse the existing database `queue_id`. If not found, they generate the next sequential ID (`MAX + 1`). This preserves database IDs across retries of queue creation. + +## createClientService — service_id reset on upsert + +The `ON CONFLICT DO UPDATE` clause sets `service_id = NULL` when credentials are updated. This forces re-registration with the router after credential rotation — the old service ID is invalidated. + +## deleteSndMsgDelivery — conditional message retention + +After removing the delivery record, checks whether any pending deliveries remain for the message. If none remain and the receipt status is `MROk`, the entire message is deleted. Otherwise, if `keepForReceipt` is true, only the message body is cleared (for debugging receipt mismatches). Handles shared `snd_message_bodies` with `FOR UPDATE` locking on PostgreSQL to prevent concurrent deletion races. + +## createWithRandomId' — bounded retry + +Generates random 12-byte IDs (base64url encoded) and retries up to 3 times on constraint violations (unique ID collision). Returns `SEUniqueID` if all attempts fail. + +## setRcvQueuePrimary / setSndQueuePrimary — two-step primary swap + +First clears primary flag on all queues in the connection, then sets it on the target queue. Also clears `replace_*_queue_id` on the new primary — this completes the queue rotation by removing the "replacing" marker. diff --git a/spec/modules/Simplex/Messaging/Agent/Store/Common.md b/spec/modules/Simplex/Messaging/Agent/Store/Common.md new file mode 100644 index 000000000..45db84995 --- /dev/null +++ b/spec/modules/Simplex/Messaging/Agent/Store/Common.md @@ -0,0 +1,7 @@ +# Simplex.Messaging.Agent.Store.Common + +> CPP-conditional re-export of backend-specific common utilities (DBStore, withConnection, withTransaction). + +**Source**: [`Agent/Store/Common.hs`](../../../../../../src/Simplex/Messaging/Agent/Store/Common.hs) + +No non-obvious behavior. See source. One of three CPP re-export wrappers (Interface, Common, DB). diff --git a/spec/modules/Simplex/Messaging/Agent/Store/DB.md b/spec/modules/Simplex/Messaging/Agent/Store/DB.md new file mode 100644 index 000000000..70be997d6 --- /dev/null +++ b/spec/modules/Simplex/Messaging/Agent/Store/DB.md @@ -0,0 +1,7 @@ +# Simplex.Messaging.Agent.Store.DB + +> CPP-conditional re-export of backend-specific database primitives (Connection, FromField, ToField). + +**Source**: [`Agent/Store/DB.hs`](../../../../../../src/Simplex/Messaging/Agent/Store/DB.hs) + +No non-obvious behavior. See source. One of three CPP re-export wrappers (Interface, Common, DB). diff --git a/spec/modules/Simplex/Messaging/Agent/Store/Entity.md b/spec/modules/Simplex/Messaging/Agent/Store/Entity.md new file mode 100644 index 000000000..801398f1a --- /dev/null +++ b/spec/modules/Simplex/Messaging/Agent/Store/Entity.md @@ -0,0 +1,7 @@ +# Simplex.Messaging.Agent.Store.Entity + +> Phantom-typed database entity IDs distinguishing new (unsaved) from stored records. + +**Source**: [`Agent/Store/Entity.hs`](../../../../../../src/Simplex/Messaging/Agent/Store/Entity.hs) + +No non-obvious behavior. See source. diff --git a/spec/modules/Simplex/Messaging/Agent/Store/Interface.md b/spec/modules/Simplex/Messaging/Agent/Store/Interface.md new file mode 100644 index 000000000..923cbfca9 --- /dev/null +++ b/spec/modules/Simplex/Messaging/Agent/Store/Interface.md @@ -0,0 +1,7 @@ +# Simplex.Messaging.Agent.Store.Interface + +> CPP-conditional re-export of the active database backend (SQLite or PostgreSQL). + +**Source**: [`Agent/Store/Interface.hs`](../../../../../../src/Simplex/Messaging/Agent/Store/Interface.hs) + +No non-obvious behavior. See source. One of three CPP re-export wrappers (Interface, Common, DB) that select the active backend at compile time via `dbPostgres`. diff --git a/spec/modules/Simplex/Messaging/Agent/Store/Postgres.md b/spec/modules/Simplex/Messaging/Agent/Store/Postgres.md new file mode 100644 index 000000000..8cb29c1b0 --- /dev/null +++ b/spec/modules/Simplex/Messaging/Agent/Store/Postgres.md @@ -0,0 +1,23 @@ +# Simplex.Messaging.Agent.Store.Postgres + +> PostgreSQL backend — dual-pool connection management, schema lifecycle, and migration. + +**Source**: [`Agent/Store/Postgres.hs`](../../../../../../src/Simplex/Messaging/Agent/Store/Postgres.hs) + +## Dual pool architecture + +`connectPostgresStore` creates two connection pools (`dbPriorityPool` and `dbPool`), each with `poolSize` connections. Priority pool is used by `withTransactionPriority` for operations that shouldn't be blocked by regular queries. Both pools are TBQueue-based — connections are taken and returned after use. + +All connections are created eagerly at initialization, not lazily on demand. + +## uninterruptibleMask_ — pool atomicity invariant + +See comment on `connectStore`. `uninterruptibleMask_` prevents async exceptions from interrupting pool filling or draining. The invariant: when `dbClosed = True`, queues are empty; when `False`, queues are full (or connections are in-flight with threads that will return them). Interruption mid-fill would break this invariant. + +## Schema creation — fail-fast on missing + +If the PostgreSQL schema doesn't exist and `createSchema` is `False`, the process logs an error and calls `exitFailure`. This prevents silent operation against the wrong schema. + +## execSQL — not implemented + +`execSQL` throws "not implemented" — the PostgreSQL client doesn't support raw SQL execution via the agent API. The function exists only to satisfy the shared interface. diff --git a/spec/modules/Simplex/Messaging/Agent/Store/SQLite.md b/spec/modules/Simplex/Messaging/Agent/Store/SQLite.md new file mode 100644 index 000000000..2513882ff --- /dev/null +++ b/spec/modules/Simplex/Messaging/Agent/Store/SQLite.md @@ -0,0 +1,26 @@ +# Simplex.Messaging.Agent.Store.SQLite + +> SQLite backend — store creation, encrypted connection management, migration, and custom SQL functions. + +**Source**: [`Agent/Store/SQLite.hs`](../../../../../../src/Simplex/Messaging/Agent/Store/SQLite.hs) + +## Security-relevant PRAGMAs + +`connectDB` sets PRAGMAs at connection time: +- `secure_delete = ON`: data is overwritten (not just unlinked) on DELETE +- `auto_vacuum = FULL`: freed pages are reclaimed immediately +- `foreign_keys = ON`: referential integrity enforced + +These are set per-connection, not per-database — every new connection (including re-opens) gets them. + +## simplex_xor_md5_combine — custom SQLite function + +A C-exported SQLite function registered at connection time. Takes an existing `IdsHash` and a `RecipientId`, XORs the hash with the MD5 of the ID. This is the SQLite implementation of the accumulative IdsHash used by service subscriptions (see [TSessionSubs.md](../TSessionSubs.md#updateActiveService--accumulative-xor-merge)). PostgreSQL uses its native `md5()` and `decode()` functions instead. + +## openSQLiteStore_ — connection swap under MVar + +Uses `bracketOnError` with `takeMVar`/`tryPutMVar`: takes the connection MVar, creates a new connection, and puts the new one back. If connection fails, `tryPutMVar` restores the old connection. The `dbClosed` TVar is flipped atomically with the key update. + +## storeKey — conditional key retention + +`storeKey key keepKey` stores the encryption key in the `dbKey` TVar only if `keepKey` is true. This allows `reopenDBStore` to re-open without the caller re-supplying the key. If `keepKey` is false and the store is closed, `reopenDBStore` fails with "no key". diff --git a/spec/modules/Simplex/Messaging/Agent/Store/Shared.md b/spec/modules/Simplex/Messaging/Agent/Store/Shared.md new file mode 100644 index 000000000..bc60de14e --- /dev/null +++ b/spec/modules/Simplex/Messaging/Agent/Store/Shared.md @@ -0,0 +1,7 @@ +# Simplex.Messaging.Agent.Store.Shared + +> Migration types, error reporting, and confirmation modes shared across database backends. + +**Source**: [`Agent/Store/Shared.hs`](../../../../../../src/Simplex/Messaging/Agent/Store/Shared.hs) + +No non-obvious behavior. See source. diff --git a/spec/modules/Simplex/Messaging/Notifications/Client.md b/spec/modules/Simplex/Messaging/Notifications/Client.md new file mode 100644 index 000000000..d2c3eef0e --- /dev/null +++ b/spec/modules/Simplex/Messaging/Notifications/Client.md @@ -0,0 +1,15 @@ +# Simplex.Messaging.Notifications.Client + +> Typed wrappers around `ProtocolClient` for NTF protocol commands. + +**Source**: [`Notifications/Client.hs`](../../../../../src/Simplex/Messaging/Notifications/Client.hs) + +## Non-obvious behavior + +### 1. Subscription operations always use NRMBackground + +`ntfCreateSubscription`, `ntfCheckSubscription`, `ntfDeleteSubscription`, and their batch variants hardcode `NRMBackground` as the network request mode. Token operations (`ntfRegisterToken`, `ntfVerifyToken`, etc.) accept the mode as a parameter. This reflects that subscription management is a background activity driven by the supervisor, while token operations can be user-initiated. + +### 2. Batch operations return per-item errors + +`ntfCreateSubscriptions` and `ntfCheckSubscriptions` return `NonEmpty (Either NtfClientError result)` — individual items in a batch can fail independently. Callers must handle partial success (some created, some failed). The singular variants throw on any error. diff --git a/spec/modules/Simplex/Messaging/Notifications/Protocol.md b/spec/modules/Simplex/Messaging/Notifications/Protocol.md new file mode 100644 index 000000000..9354e2086 --- /dev/null +++ b/spec/modules/Simplex/Messaging/Notifications/Protocol.md @@ -0,0 +1,43 @@ +# Simplex.Messaging.Notifications.Protocol + +> NTF protocol entities, commands, responses, and wire encoding for the notification system. + +**Source**: [`Notifications/Protocol.hs`](../../../../../src/Simplex/Messaging/Notifications/Protocol.hs) + +## Non-obvious behavior + +### 1. Asymmetric credential validation + +`checkCredentials` enforces different rules per command category: + +| Category | Signature required | Entity ID | +|----------|-------------------|-----------| +| TNEW, SNEW | Yes | Must be empty (new entity) | +| PING | No | Must be empty | +| All others | Yes | Must be present | + +For responses, the rule inverts: `NRTknId`, `NRSubId`, and `NRPong` must NOT have entity IDs (they are returned before/without entity context), while `NRErr` optionally has one (errors can occur with or without entity context). + +### 2. PNMessageData semicolon separator + +`encodePNMessages` uses `;` as the separator between push notification message items instead of the standard `,` used by `NonEmpty` `strEncode`. This is because `SMPQueueNtf` contains an `SMPServer` whose host list encoding already uses commas, which would create ambiguous parsing. + +### 3. NTInvalid reason is version-gated + +When encoding `NRTkn` responses, the `NTInvalid` reason is only included if the negotiated protocol version is >= `invalidReasonNTFVersion` (v3). Older clients receive `NTInvalid Nothing`. This prevents parse failures on clients that don't understand the reason field. + +### 4. subscribeNtfStatuses migration invariant + +The comment on `subscribeNtfStatuses` (`[NSNew, NSPending, NSActive, NSInactive]`) warns that changing these statuses requires a new database migration for queue ID hashes (see `m20250830_queue_ids_hash`). This is a cross-module invariant between protocol types and server storage. + +### 5. allowNtfSubCommands permits NTInvalid and NTExpired + +Token status `NTInvalid` allows subscription commands (SNEW, SCHK, SDEL), which is counterintuitive. The rationale (noted in a TODO comment) is that invalidation can happen after verification, and existing subscriptions should remain manageable. `NTExpired` is also permitted for the same reason. + +### 6. PPApnsNull test provider + +`PPApnsNull` is a push provider that never communicates with APNS. It's used for end-to-end testing of the notification server from clients without requiring actual push infrastructure. + +### 7. DeviceToken hex validation + +`DeviceToken` string parsing has two paths: a hardcoded literal match for `"apns_null test_ntf_token"` (test tokens), and hex string validation for real tokens (must be even-length hex). The wire encoding (`smpP`) does not perform this validation — it accepts any `ByteString`. diff --git a/spec/modules/Simplex/Messaging/Notifications/Server.md b/spec/modules/Simplex/Messaging/Notifications/Server.md new file mode 100644 index 000000000..9c88cf7a0 --- /dev/null +++ b/spec/modules/Simplex/Messaging/Notifications/Server.md @@ -0,0 +1,79 @@ +# Simplex.Messaging.Notifications.Server + +> NTF server: manages tokens, subscriptions, SMP subscriber connections, and push notification delivery. + +**Source**: [`Notifications/Server.hs`](../../../../../src/Simplex/Messaging/Notifications/Server.hs) + +## Architecture + +The NTF server runs several concurrent threads via `raceAny_`: + +| Thread | Purpose | +|--------|---------| +| `ntfSubscriber` | Receives SMP messages (NMSG, END, DELD) and agent events (connect/disconnect/subscribe) | +| `ntfPush` | Reads push queue and delivers via APNS provider | +| `periodicNtfsThread` | Sends periodic "check messages" push notifications (cron) | +| `runServer` (per transport) | Accepts client connections and runs NTF protocol | +| Stats/Prometheus/Control | Optional monitoring and admin threads | + +Each client connection spawns `receive`, `send`, and `client` threads via `raceAny_`. + +## Non-obvious behavior + +### 1. Timing attack mitigation on entity lookup + +When `verifyNtfTransmission` encounters an AUTH error (entity not found), it calls `dummyVerifyCmd` to equalize response timing before returning the error. This prevents attackers from distinguishing "entity doesn't exist" from "signature invalid" based on response latency. + +### 2. TNEW idempotent re-registration + +When TNEW is received for an already-registered token, the server: +1. Looks up the existing token via `findNtfTokenRegistration` +2. Verifies the DH secret matches (recomputed from the new `dhPubKey` and stored `tknDhPrivKey`) +3. If DH secrets differ → AUTH error (prevents token hijacking) +4. If they match → re-sends verification push notification + +This makes TNEW safe for client retransmission after connection drops. + +### 3. SNEW idempotent subscription + +When SNEW is received for an existing subscription (same token + SMP queue), the server returns the existing `ntfSubId` if the notifier key matches. If keys differ, AUTH error. New subscriptions are only created when no match exists in `findNtfSubscription`. + +### 4. PPApnsNull suppresses statistics + +`incNtfStatT` skips all stat increments when the device token uses `PPApnsNull` provider. This prevents test tokens from polluting production metrics. + +### 5. END requires active session validation + +SMP END messages are only processed when the originating session is the currently active session for that server (`activeClientSession'` check). This prevents stale END messages from previous (reconnected) sessions from incorrectly marking subscriptions as ended. + +### 6. waitForSMPSubscriber two-phase wait + +`waitForSMPSubscriber` first tries a non-blocking `tryReadTMVar`. If the subscriber isn't ready yet, it falls back to a blocking `readTMVar` with a 10-second timeout. This avoids creating an extra timeout thread in the common case where the subscriber is already available. + +### 7. CAServiceUnavailable triggers individual resubscription + +When a service subscription becomes unavailable (SMP server rejects service credentials), the NTF server: +1. Removes the service association from the database +2. Resubscribes all individual queues for that server via `subscribeSrvSubs` + +This is the fallback path from service-level to queue-level SMP subscriptions. + +### 8. Push delivery single retry + +`deliverNotification` retries exactly once on connection errors (`PPConnection`) or `PPRetryLater`: +1. Creates a new push client (`newPushClient`) to get a fresh connection +2. Retries the delivery + +On the second failure, the error is logged and returned. `PPTokenInvalid` marks the token as `NTInvalid` on either the first or retry attempt. + +### 9. TCRN minimum interval enforcement + +Cron notification interval has a hard minimum of 20 minutes. `TCRN 0` disables cron notifications. `TCRN n` where `1 <= n < 20` returns `QUOTA` error. + +### 10. Startup resubscription is concurrent per server + +`resubscribe` uses `mapConcurrently` to resubscribe to all known SMP servers in parallel. Within each server, subscriptions are paginated via `subscribeLoop` using cursor-based pagination (`afterSubId_`). + +### 11. receive separates error responses from commands + +The `receive` function processes incoming transmissions and partitions results: malformed/unauthorized requests are written directly to `sndQ` as error responses, while valid commands go to `rcvQ` for processing. This ensures protocol errors get immediate responses without competing for the command processing queue. diff --git a/spec/modules/Simplex/Messaging/Notifications/Server/Control.md b/spec/modules/Simplex/Messaging/Notifications/Server/Control.md new file mode 100644 index 000000000..897f81c16 --- /dev/null +++ b/spec/modules/Simplex/Messaging/Notifications/Server/Control.md @@ -0,0 +1,7 @@ +# Simplex.Messaging.Notifications.Server.Control + +> Control port command protocol for NTF server administration. + +**Source**: [`Notifications/Server/Control.hs`](../../../../../../src/Simplex/Messaging/Notifications/Server/Control.hs) + +No non-obvious behavior. See source. diff --git a/spec/modules/Simplex/Messaging/Notifications/Server/Env.md b/spec/modules/Simplex/Messaging/Notifications/Server/Env.md new file mode 100644 index 000000000..96221a012 --- /dev/null +++ b/spec/modules/Simplex/Messaging/Notifications/Server/Env.md @@ -0,0 +1,21 @@ +# Simplex.Messaging.Notifications.Server.Env + +> NTF server environment: configuration, subscriber state, and push provider management. + +**Source**: [`Notifications/Server/Env.hs`](../../../../../../src/Simplex/Messaging/Notifications/Server/Env.hs) + +## Non-obvious behavior + +### 1. Service credentials are lazily generated + +`mkDbService` in `newNtfServerEnv` generates service credentials on demand: when `getCredentials` is called for an SMP server, it first checks the database. If credentials exist, they are used. If not (`Nothing`), new credentials are generated via `genCredentials`, stored in the database, and returned. This happens per SMP server on first connection. + +Service credentials are only used when `useServiceCreds` is enabled in the config. + +### 2. PPApnsNull creates a no-op push client + +`newPushClient` checks `apnsProviderHost` for the push provider. `PPApnsNull` returns `Nothing`, which creates a no-op client (`\_ _ -> pure ()`). Real providers create an actual APNS connection. This is the mechanism that allows `PPApnsNull` tokens to function without push infrastructure. + +### 3. getPushClient lazy initialization + +`getPushClient` looks up the push client by provider in `pushClients` TMap. If not found, it calls `newPushClient` to create and register one. Push provider connections are established on first use, not at server startup. diff --git a/spec/modules/Simplex/Messaging/Notifications/Server/Main.md b/spec/modules/Simplex/Messaging/Notifications/Server/Main.md new file mode 100644 index 000000000..3719dcd97 --- /dev/null +++ b/spec/modules/Simplex/Messaging/Notifications/Server/Main.md @@ -0,0 +1,7 @@ +# Simplex.Messaging.Notifications.Server.Main + +> CLI interface and INI configuration parsing for the NTF server. + +**Source**: [`Notifications/Server/Main.hs`](../../../../../../src/Simplex/Messaging/Notifications/Server/Main.hs) + +No non-obvious behavior. Standard CLI/config boilerplate. Notable defaults: `subsBatchSize = 900`, `periodicNtfsInterval = 5 minutes`, `pushQSize = 32768`, `persistErrorInterval = 0` (disables SMP client reconnection error persistence). diff --git a/spec/modules/Simplex/Messaging/Notifications/Server/Push/APNS.md b/spec/modules/Simplex/Messaging/Notifications/Server/Push/APNS.md new file mode 100644 index 000000000..2a6d8c0b1 --- /dev/null +++ b/spec/modules/Simplex/Messaging/Notifications/Server/Push/APNS.md @@ -0,0 +1,35 @@ +# Simplex.Messaging.Notifications.Server.Push.APNS + +> Apple Push Notification Service (APNS) client: JWT authentication, HTTP/2 delivery, and e2e encryption. + +**Source**: [`Notifications/Server/Push/APNS.hs`](../../../../../../../src/Simplex/Messaging/Notifications/Server/Push/APNS.hs) + +## Non-obvious behavior + +### 1. PNCheckMessages is not encrypted + +`PNVerification` and `PNMessage` notifications are encrypted with the shared DH secret (`C.cbEncrypt`) and padded to `paddedNtfLength` (3072 bytes) to prevent metadata leakage. `PNCheckMessages` is sent as a plain `{"checkMessages": true}` background notification — it carries no sensitive data and doesn't need e2e encryption. + +### 2. Fixed-length encryption padding + +All encrypted notifications are padded to `paddedNtfLength` (3072 bytes) regardless of actual content size. This prevents notification size from revealing whether it's a verification code (small) or a message batch (larger). + +### 3. JWT token caching with TTL refresh + +`getApnsJWTToken` caches the signed JWT and only regenerates it when the token age exceeds `tokenTTL` (30 minutes). No locking is used — if two threads race to refresh, last writer wins, which is acceptable since both produce valid tokens. + +### 4. HTTP/2 reconnect-on-use + +`createAPNSPushClient` registers a disconnect callback that sets `https2Client` to `Nothing`. `getApnsHTTP2Client` lazily reconnects on the next push delivery attempt. The connection is not proactively maintained. + +### 5. 503 triggers active disconnect before retry + +When APNS returns 503 (Service Unavailable), the client actively closes the HTTP/2 connection (`disconnectApnsHTTP2Client`) before throwing `PPRetryLater`. This ensures a fresh connection is established on retry rather than reusing a potentially degraded connection. + +### 6. ExpiredProviderToken is permanent + +403 errors for `ExpiredProviderToken` and `InvalidProviderToken` are classified as `PPPermanentError` rather than retryable. Since `getApnsJWTToken` just refreshed the JWT before the request, retrying with the same key would produce the same error. This indicates a configuration problem (wrong key/team ID). + +### 7. EC key type assumption + +`readECPrivateKey` uses a specific pattern match for EC keys (`PrivKeyEC_Named`). It will crash at runtime if the APNS key file contains a different key type. The comment acknowledges this limitation. diff --git a/spec/modules/Simplex/Messaging/Notifications/Server/Push/APNS/Internal.md b/spec/modules/Simplex/Messaging/Notifications/Server/Push/APNS/Internal.md new file mode 100644 index 000000000..b42753e98 --- /dev/null +++ b/spec/modules/Simplex/Messaging/Notifications/Server/Push/APNS/Internal.md @@ -0,0 +1,7 @@ +# Simplex.Messaging.Notifications.Server.Push.APNS.Internal + +> APNS HTTP header constants and JSON encoding options. + +**Source**: [`Notifications/Server/Push/APNS/Internal.hs`](../../../../../../../../src/Simplex/Messaging/Notifications/Server/Push/APNS/Internal.hs) + +No non-obvious behavior. See source. Defines APNS header names and JSON options (`UntaggedValue` sum encoding, `camelTo2 '-'` for hyphenated field names like `content-available`, `mutable-content`). diff --git a/spec/modules/Simplex/Messaging/Notifications/Server/Stats.md b/spec/modules/Simplex/Messaging/Notifications/Server/Stats.md new file mode 100644 index 000000000..971419abf --- /dev/null +++ b/spec/modules/Simplex/Messaging/Notifications/Server/Stats.md @@ -0,0 +1,19 @@ +# Simplex.Messaging.Notifications.Server.Stats + +> NTF server statistics collection with own-server breakdown and backward-compatible persistence. + +**Source**: [`Notifications/Server/Stats.hs`](../../../../../../src/Simplex/Messaging/Notifications/Server/Stats.hs) + +## Non-obvious behavior + +### 1. incServerStat double lookup + +`incServerStat` performs a non-STM IO lookup first, then only enters an STM transaction on cache miss. The STM block re-checks the map to handle races (another thread may have inserted between the IO lookup and STM entry). This avoids contention on the shared TMap in the common case where the server's counter TVar already exists. + +### 2. setNtfServerStats is not thread safe + +`setNtfServerStats` is explicitly documented as non-thread-safe and intended for server startup only (restoring from backup file). + +### 3. Backward-compatible parsing + +The `strP` parser uses `opt` which defaults missing fields to 0. This allows reading stats files from older server versions that don't include newer fields (`ntfReceivedAuth`, `ntfFailed`, `ntfVrf*`, etc.). diff --git a/spec/modules/Simplex/Messaging/Notifications/Server/Store.md b/spec/modules/Simplex/Messaging/Notifications/Server/Store.md new file mode 100644 index 000000000..33acdaad9 --- /dev/null +++ b/spec/modules/Simplex/Messaging/Notifications/Server/Store.md @@ -0,0 +1,23 @@ +# Simplex.Messaging.Notifications.Server.Store + +> STM-based in-memory store for notification tokens, subscriptions, and last-notification accumulation. + +**Source**: [`Notifications/Server/Store.hs`](../../../../../../src/Simplex/Messaging/Notifications/Server/Store.hs) + +## Non-obvious behavior + +### 1. Two-level token registration index + +`tokenRegistrations` uses a nested TMap: `DeviceToken -> TMap ByteString NtfTokenId`, where the inner key is the serialized verify key. This allows **multiple concurrent registrations** per device token (with different keys), protecting against malicious registration attempts if a token is compromised. The inner key is derived via `C.toPubKey C.pubKeyBytes`. + +### 2. stmRemoveInactiveTokenRegistrations cleans up rivals + +When a token is activated, `stmRemoveInactiveTokenRegistrations` removes ALL other registrations for the same device token, including their token records, last notifications, and all subscriptions. Only the activating token's registration survives. + +### 3. stmStoreTokenLastNtf guards against stale tokens + +`stmStoreTokenLastNtf` performs a non-STM IO lookup first, then enters STM. Within the STM block, it re-checks the map to handle the race where another thread modified the map between the IO lookup and STM entry. It only inserts for tokens that exist in the `tokens` map — stale token IDs are silently ignored. + +### 4. tokenLastNtfs accumulates via prepend + +New notifications are prepended to the `NonEmpty PNMessageData` list via `(<|)`. The list is unbounded in the STM store — bounding is handled at the push delivery layer (the Postgres store limits to 6). diff --git a/spec/modules/Simplex/Messaging/Notifications/Server/Store/Postgres.md b/spec/modules/Simplex/Messaging/Notifications/Server/Store/Postgres.md new file mode 100644 index 000000000..3cb5c9083 --- /dev/null +++ b/spec/modules/Simplex/Messaging/Notifications/Server/Store/Postgres.md @@ -0,0 +1,54 @@ +# Simplex.Messaging.Notifications.Server.Store.Postgres + +> PostgreSQL-backed persistent store for notification tokens, subscriptions, and last-notification delivery. + +**Source**: [`Notifications/Server/Store/Postgres.hs`](../../../../../../../src/Simplex/Messaging/Notifications/Server/Store/Postgres.hs) + +## Non-obvious behavior + +### 1. deleteNtfToken exclusive row lock + +`deleteNtfToken` acquires `FOR UPDATE` on the token row before cascading deletes. This prevents concurrent subscription inserts for this token during the deletion window. The subscriptions are aggregated by SMP server and returned for in-memory subscription cleanup. + +### 2. addTokenLastNtf atomic CTE + +`addTokenLastNtf` executes a single SQL statement with three CTEs that atomically: +1. **Upserts** the new notification into `last_notifications` (one row per token+subscription) +2. **Collects** the most recent notifications for the token (limited to `maxNtfs = 6`) +3. **Deletes** any older notifications beyond the limit + +This ensures the push notification always contains the most recent notifications across all of a token's subscriptions, with bounded storage. + +### 3. setTokenActive cleans duplicate registrations + +After activating a token, `setTokenActive` deletes all other tokens with the same `push_provider` + `push_provider_token` but different `token_id`. This cleans up incomplete or duplicate registration attempts. + +### 4. setTknStatusConfirmed conditional update + +Updates to `NTConfirmed` only if the current status is not already `NTConfirmed` or `NTActive`. This prevents downgrading an already-active token back to confirmed state when a delayed verification push arrives. + +### 5. Silent token date tracking + +`updateTokenDate` is called on every token read (`getNtfToken_`, `findNtfSubscription`, `getNtfSubscription`). It updates `updated_at` only when the current date differs from the stored date. This tracks token activity without explicit client action. + +### 6. getServerNtfSubscriptions marks as pending + +After reading subscriptions for resubscription, `getServerNtfSubscriptions` batch-updates their status to `NSPending`. This prevents the same subscriptions from being picked up by a concurrent resubscription pass — it acts as a "claim" mechanism. + +Only non-service-associated subscriptions (`NOT ntf_service_assoc`) are returned for individual resubscription. + +### 7. Approximate subscription count + +`getEntityCounts` uses `pg_class.reltuples` for the subscription count instead of `count(*)`. This returns an approximate value from PostgreSQL's statistics catalog, avoiding a full table scan on potentially large subscription tables. + +### 8. withFastDB vs withDB priority pools + +`withFastDB` uses `withTransactionPriority ... True` to run on the priority connection pool. Client-facing operations (token registration, subscription commands) use the priority pool, while background operations (batch status updates, resubscription) use the regular pool. + +### 9. Server upsert optimization + +`addNtfSubscription` first tries a plain SELECT for the SMP server, then falls back to INSERT with ON CONFLICT only if the server doesn't exist. This avoids the upsert overhead in the common case where the server already exists. + +### 10. Service association tracking + +`batchUpdateSrvSubStatus` atomically updates both subscription status and `ntf_service_assoc` flag. When notifications arrive via a service subscription (`newServiceId` is `Just`), all affected subscriptions are marked as service-associated. `removeServiceAndAssociations` resets all subscriptions for a server to `NSInactive` with `ntf_service_assoc = FALSE`. diff --git a/spec/modules/Simplex/Messaging/Notifications/Server/Store/Types.md b/spec/modules/Simplex/Messaging/Notifications/Server/Store/Types.md new file mode 100644 index 000000000..97f0fce46 --- /dev/null +++ b/spec/modules/Simplex/Messaging/Notifications/Server/Store/Types.md @@ -0,0 +1,7 @@ +# Simplex.Messaging.Notifications.Server.Store.Types + +> Pure record types and STM conversion for notification tokens and subscriptions. + +**Source**: [`Notifications/Server/Store/Types.hs`](../../../../../../../src/Simplex/Messaging/Notifications/Server/Store/Types.hs) + +No non-obvious behavior. `mkTknData`/`mkTknRec` convert between pure records and TVar-based STM data. `tknUpdatedAt` is parsed as optional for backward compatibility with store logs that predate it. diff --git a/spec/modules/Simplex/Messaging/Notifications/Transport.md b/spec/modules/Simplex/Messaging/Notifications/Transport.md index 7c7955154..263b2459a 100644 --- a/spec/modules/Simplex/Messaging/Notifications/Transport.md +++ b/spec/modules/Simplex/Messaging/Notifications/Transport.md @@ -1,36 +1,30 @@ # Simplex.Messaging.Notifications.Transport -> Notification Router Protocol transport: manages push notification subscriptions between client and NTF Router. +> NTF protocol version negotiation, TLS handshake, and transport handle setup. **Source**: [`Notifications/Transport.hs`](../../../../../src/Simplex/Messaging/Notifications/Transport.hs) -**Protocol spec**: [`protocol/push-notifications.md`](../../../../../protocol/push-notifications.md) — SimpleX Notification Router protocol. +## Non-obvious behavior -## Overview +### 1. ALPN-dependent version range -This module implements the transport layer for the **Notification Router Protocol**. Per the protocol spec: "To manage notification subscriptions to SMP routers, SimpleX Notification Router provides an RPC protocol with a similar design to SimpleX Messaging Protocol router." +`ntfServerHandshake` advertises `legacyServerNTFVRange` (v1 only) when ALPN is not available (`getSessionALPN` returns `Nothing`). When ALPN is present, it advertises the full `supportedServerNTFVRange`. This is the backward-compatibility mechanism for pre-ALPN clients that cannot negotiate newer protocol features. -The protocol spec diagram shows three separate protocols in the notification flow: -1. **Notification Router Protocol** (this module): client ↔ SimpleX Notification Router — subscription management -2. **SMP protocol**: SMP Router → SimpleX Notifications Subscriber — notification signals -3. **Push provider** (e.g., APN): SimpleX Push Router → device — per the spec: "the notifications are e2e encrypted between SimpleX Notification Router and the user's device" +### 2. Version-gated features -## Differences from SMP transport +Two feature gates exist in the NTF protocol: -The NTF protocol reuses SMP's transport infrastructure but with reduced parameters: +| Version | Feature | Effect | +|---------|---------|--------| +| v2 (`authBatchCmdsNTFVersion`) | Auth key exchange + batching | `authPubKey` sent in handshake, `implySessId` and `batch` enabled | +| v3 (`invalidReasonNTFVersion`) | Token invalid reasons | `NTInvalid` responses include the reason enum | -| Property | SMP | NTF | -|----------|-----|-----| -| Block size | 16384 | 512 | -| Block encryption | Yes (v11+) | No (`encryptBlock = Nothing`) | -| Service certificates | Yes (v16+) | No (`serviceAuth = False`) | -| Version range | 6–19 | 1–3 | -| Handshake messages | 2–3 | 2 | +Pre-v2 connections have no command encryption or batching — commands are sent in plaintext within TLS. -## Same ALPN/legacy fallback pattern as SMP +### 3. Unused Protocol typeclass parameters -`ntfServerHandshake` uses the same pattern as `smpServerHandshake`: if ALPN is not negotiated (`getSessionALPN` returns `Nothing`), the notification router offers only `legacyServerNTFVRange` (v1 only). +`ntfClientHandshake` accepts `_proxyServer` and `_serviceKeys` parameters that are ignored. These exist because the `Protocol` typeclass (shared with SMP) requires `protocolClientHandshake` to accept them. The NTF protocol does not support proxy routing or service authentication. -## NTF handshake uses SMP shared types +### 4. Block size -The handshake reuses SMP's `THandle`, `THandleParams`, `THandleAuth` types. The `encodeAuthEncryptCmds` and `authEncryptCmdsP` helper functions are defined locally in this module (with NTF-specific version thresholds). NTF never sets `sessSecret` / `sessSecret'`, `peerClientService`, or `clientService` — these are always `Nothing`. +NTF uses a 512-byte block size (`ntfBlockSize`), significantly smaller than SMP. Notification commands and responses are short — the main payload is the `PNMessageData` which contains encrypted message metadata. diff --git a/spec/modules/Simplex/Messaging/Notifications/Types.md b/spec/modules/Simplex/Messaging/Notifications/Types.md new file mode 100644 index 000000000..bb05ccefb --- /dev/null +++ b/spec/modules/Simplex/Messaging/Notifications/Types.md @@ -0,0 +1,19 @@ +# Simplex.Messaging.Notifications.Types + +> Agent-side notification token and subscription types with action state machines. + +**Source**: [`Notifications/Types.hs`](../../../../../src/Simplex/Messaging/Notifications/Types.hs) + +## Non-obvious behavior + +### 1. NASDeleted is a transient race condition artifact + +`NASDeleted` can only exist when the notification supervisor updates a subscription record while a worker is mid-operation on that same subscription. The worker's post-operation database update hits a record that was already modified by the supervisor, resulting in an update to `NASDeleted` status instead of a full deletion. This status should not persist — it is cleaned up on the next supervisor pass. + +### 2. Action space split across two worker types + +`NtfSubAction` is an `Either`-like sum of `NtfSubNTFAction` (handled by NTF router workers) and `NtfSubSMPAction` (handled by SMP router workers). The supervisor writes these to the database, and each worker pool only reads its own action type. `isDeleteNtfSubAction` classifies actions across both types for the supervisor's reset logic. + +### 3. NSADelete and NSARotate are deprecated + +These `NtfSubNTFAction` values are no longer generated by current code but are retained in the type for processing legacy database records. `NSARotate` is logically "delete + recreate" while `NSADelete` is "delete notifier on NTF router + delete credentials on SMP router".