diff --git a/spec/agent/infrastructure.md b/spec/agent/infrastructure.md index 5e3c828e6..784608b62 100644 --- a/spec/agent/infrastructure.md +++ b/spec/agent/infrastructure.md @@ -194,3 +194,18 @@ The agent supports SQLite and PostgreSQL via CPP compilation flags (`#if defined **Store access bracketing**: `withStore` wraps all database operations with `agentOperationBracket AODatabase`, connecting the store to the suspension cascade. `withStoreBatch` / `withStoreBatch'` run multiple operations in a single transaction with per-operation error catching. **Known bug**: `checkConfirmedSndQueueExists_` uses `#if defined(dpPostgres)` (typo - should be `dbPostgres`), so the `FOR UPDATE` clause is never included on either backend. + +### Migration framework + +**Source**: [Agent/Store/Migrations.hs](../../src/Simplex/Messaging/Agent/Store/Migrations.hs), [Agent/Store/Shared.hs](../../src/Simplex/Messaging/Agent/Store/Shared.hs) + +Migrations are Haskell modules under `Agent/Store/SQLite/Migrations/` and `Agent/Store/Postgres/Migrations/`. Each has `up` SQL and optional `down` SQL. + +**Key behaviors**: + +- `migrationsToRun` compares app migrations against the `migrations` table by name. Divergent histories (app has `[a,b]`, DB has `[a,c]`) produce `MTREDifferent` error - manual intervention required. +- Each migration runs in its own transaction with the `migrations` insert *before* the schema change - failure rolls back both. +- Downgrades require all intermediate migrations to have `down` SQL; missing any produces `MTRENoDown`. +- `MigrationConfirmation` controls whether upgrades/downgrades auto-apply, prompt, or error. + +**Special case**: `m20220811_onion_hosts` triggers `updateServers` to expand host entries with Tor addresses - this is data migration, not just schema. diff --git a/spec/topics/patterns.md b/spec/topics/patterns.md index a5d61500e..09bd9a7e1 100644 --- a/spec/topics/patterns.md +++ b/spec/topics/patterns.md @@ -10,6 +10,8 @@ For protocol-specific encoding details, see [transport.md](transport.md). For cr - [Compression](#compression) - [Concurrent data structures](#concurrent-data-structures) - [Batch processing](#batch-processing) +- [Time encoding](#time-encoding) +- [Utilities](#utilities) --- @@ -335,3 +337,21 @@ withStoreBatch' :: Traversable t ``` Use when operations cannot fail (or failures should become `INTERNAL` errors). + +--- + +## Time encoding + +**Source**: [SystemTime.hs](../../src/Simplex/Messaging/SystemTime.hs) + +`RoundedSystemTime t` uses a phantom type-level `Nat` for precision. `SystemDate` (precision 86400) provides k-anonymity for file creation times - all timestamps within a day collapse to the same value, preventing correlation attacks. + +--- + +## Utilities + +**Source**: [Util.hs](../../src/Simplex/Messaging/Util.hs) + +**Functor combinators**: `<$$>` (double fmap), `<$$` (double fmap const), and `<$?>` (fmap with `MonadFail` on `Left`) are used throughout for nested functor manipulation and fallible parsing chains. + +**`threadDelay'`**: Handles `Int64` delays that exceed `maxBound::Int` by looping with `maxBound`-sized chunks. diff --git a/spec/topics/transport.md b/spec/topics/transport.md index 2cce46f77..a9ce85f1b 100644 --- a/spec/topics/transport.md +++ b/spec/topics/transport.md @@ -10,6 +10,8 @@ For service certificate handshake extensions, see [client-services.md](client-se - [Transmission encoding and signing](#transmission-encoding-and-signing) - [Version negotiation](#version-negotiation) - [Connection management](#connection-management) +- [HTTP/2 sessions](#http2-sessions) +- [WebSocket adapter](#websocket-adapter) --- @@ -321,3 +323,27 @@ All four threads run inside `raceAny_` with `E.finally disconnected`. When any t 2. The agent callback demotes subscriptions, fires DOWN events, and initiates resubscription The `connected` TVar is set to `True` after the handshake succeeds and before the threads start. Note: in the protocol client, this TVar is not reset on disconnect - disconnect detection relies on thread cancellation via `raceAny_` and the `disconnected` callback, not STM re-evaluation. (The server-side `Client` type has a separate `connected` TVar that is reset in `clientDisconnected`.) + +--- + +## HTTP/2 sessions + +**Source**: [Transport/HTTP2/Client.hs](../../src/Simplex/Messaging/Transport/HTTP2/Client.hs), [Transport/HTTP2/Server.hs](../../src/Simplex/Messaging/Transport/HTTP2/Server.hs) + +HTTP/2 is used for XFTP file transfers and notifications to push providers (APNs). + +**Why the request queue**: `sendRequest` serializes requests through a `TBQueue` because the underlying http2 library is not thread-safe for concurrent stream creation. `sendRequestDirect` exists but is explicitly marked unsafe. + +**Inactivity expiration**: Server connections track `activeAt` and are closed by a background thread when idle beyond `checkInterval`. This is necessary because HTTP/2 has no application-level keepalive - abandoned connections would otherwise persist indefinitely. + +--- + +## WebSocket adapter + +**Source**: [Transport/WebSockets.hs](../../src/Simplex/Messaging/Transport/WebSockets.hs) + +WebSocket wraps TLS for browser clients, implementing the `Transport` typeclass. + +**Strict size matching**: Unlike raw TLS where `cGet` may accumulate multiple reads, WebSocket `cGet` expects a single `receiveData` to return exactly the requested size. Mismatch throws `TEBadBlock` immediately - WebSocket messages are atomic, so partial reads indicate a protocol error. + +**No compression**: `connectionCompressionOptions = NoCompression` because the payload is already encrypted. Compressing ciphertext wastes CPU and leaks information about plaintext structure.