From f56b5940368c5d8773a4f1534c5f5c87daaa95d4 Mon Sep 17 00:00:00 2001 From: "Evgeny @ SimpleX Chat" <259188159+evgeny-simplex@users.noreply.github.com> Date: Sun, 15 Mar 2026 11:03:30 +0000 Subject: [PATCH] remove old topic stubs --- spec/agent-protocol.md | 13 -- spec/compression.md | 84 --------- spec/crypto-ratchet.md | 13 -- spec/crypto-tls.md | 11 -- spec/crypto.md | 19 --- spec/encoding.md | 332 ------------------------------------ spec/ntf-protocol.md | 15 -- spec/ntf-server.md | 11 -- spec/remote-control.md | 11 -- spec/smp-client.md | 11 -- spec/smp-protocol.md | 13 -- spec/smp-server.md | 13 -- spec/storage-agent.md | 11 -- spec/storage-server.md | 9 - spec/transport-http2.md | 13 -- spec/transport-websocket.md | 7 - spec/transport.md | 11 -- spec/version.md | 177 ------------------- spec/xftp-client.md | 11 -- spec/xftp-protocol.md | 13 -- spec/xftp-server.md | 11 -- spec/xrcp-protocol.md | 13 -- 22 files changed, 822 deletions(-) delete mode 100644 spec/agent-protocol.md delete mode 100644 spec/compression.md delete mode 100644 spec/crypto-ratchet.md delete mode 100644 spec/crypto-tls.md delete mode 100644 spec/crypto.md delete mode 100644 spec/encoding.md delete mode 100644 spec/ntf-protocol.md delete mode 100644 spec/ntf-server.md delete mode 100644 spec/remote-control.md delete mode 100644 spec/smp-client.md delete mode 100644 spec/smp-protocol.md delete mode 100644 spec/smp-server.md delete mode 100644 spec/storage-agent.md delete mode 100644 spec/storage-server.md delete mode 100644 spec/transport-http2.md delete mode 100644 spec/transport-websocket.md delete mode 100644 spec/transport.md delete mode 100644 spec/version.md delete mode 100644 spec/xftp-client.md delete mode 100644 spec/xftp-protocol.md delete mode 100644 spec/xftp-server.md delete mode 100644 spec/xrcp-protocol.md diff --git a/spec/agent-protocol.md b/spec/agent-protocol.md deleted file mode 100644 index b84ffb9ce..000000000 --- a/spec/agent-protocol.md +++ /dev/null @@ -1,13 +0,0 @@ -# Agent Protocol Implementation - -> Implements agent connection procedures, queue rotation, and duplex messaging. - -**Protocol reference**: [`protocol/agent-protocol.md`](../protocol/agent-protocol.md) - -## Types - -## Connection Procedures - -## Queue Rotation - -## Functions diff --git a/spec/compression.md b/spec/compression.md deleted file mode 100644 index faa8c275f..000000000 --- a/spec/compression.md +++ /dev/null @@ -1,84 +0,0 @@ -# Compression - -> Zstd compression for SimpleX protocol messages. - -**Source file**: [`Compression.hs`](../src/Simplex/Messaging/Compression.hs) - -## Overview - -Optional Zstd compression for SMP message bodies. Short messages bypass compression entirely to avoid overhead. The `Compressed` type carries a tag byte indicating whether the payload is compressed or passthrough, making it self-describing on the wire. - -## Types - -### `Compressed` - -**Source**: `Compression.hs:17-22` - -```haskell -data Compressed - = Passthrough ByteString -- short messages, left intact - | Compressed Large -- Zstd-compressed, 2-byte length prefix -``` - -Wire encoding (`Compression.hs:30-38`): - -``` -Passthrough → '0' ++ smpEncode ByteString (1-byte tag + 1-byte length + data) -Compressed → '1' ++ smpEncode Large (1-byte tag + 2-byte length + data) -``` - -Tags are `'0'` (0x30) and `'1'` (0x31) — same ASCII convention as `Maybe` encoding. - -`Passthrough` uses standard `ByteString` encoding (max 255 bytes, 1-byte length prefix). `Compressed` uses `Large` encoding (max 65535 bytes, 2-byte Word16 length prefix), since compressed output can exceed 255 bytes for larger inputs. - -## Constants - -| Constant | Value | Purpose | Source | -|----------|-------|---------|--------| -| `maxLengthPassthrough` | 180 | Messages at or below this length are not compressed | `Compression.hs:24-25` | -| `compressionLevel` | 3 | Zstd compression level | `Compression.hs:27-28` | - -The 180-byte threshold was "sampled from real client data" — messages above this length show rapidly increasing compression ratio. Below 180 bytes, compression overhead (FFI call, dictionary-less Zstd startup) outweighs savings. - -## Functions - -### `compress1` - -**Source**: `Compression.hs:40-43` - -```haskell -compress1 :: ByteString -> Compressed -``` - -Compress a message body: -- If `B.length bs <= 180` → `Passthrough bs` -- Otherwise → `Compressed (Large (Z1.compress 3 bs))` - -No context or dictionary — each message is independently compressed ("1" in `compress1` refers to single-shot compression). - -### `decompress1` - -**Source**: `Compression.hs:45-53` - -```haskell -decompress1 :: Int -> Compressed -> Either String ByteString -``` - -Decompress with size limit: -- `Passthrough bs` → `Right bs` (no check needed — already bounded by encoding) -- `Compressed (Large bs)` → check `Z1.decompressedSize bs`: - - If size is known and within `limit` → decompress - - If size unknown or exceeds `limit` → `Left` error - -The size limit check happens **before** decompression, using Zstd's frame header (which includes the decompressed size when the compressor wrote it). This prevents decompression bombs — an attacker cannot cause unbounded memory allocation by sending a small compressed payload that expands to gigabytes. - -The `Z1.decompress` result is pattern-matched for three cases: -- `Z1.Error e` → `Left e` -- `Z1.Skip` → `Right mempty` (zero-length output) -- `Z1.Decompress bs'` → `Right bs'` - -## Security notes - -- **Decompression bomb protection**: `decompress1` requires an explicit size limit and checks `decompressedSize` before allocating. Callers must pass an appropriate limit (typically the SMP block size). -- **No dictionary/context**: Each message is independently compressed. No shared state between messages that could leak information across compression boundaries. -- **Passthrough for short messages**: Messages ≤ 180 bytes are never compressed, avoiding timing side channels from compression ratio differences on short, potentially-predictable messages. diff --git a/spec/crypto-ratchet.md b/spec/crypto-ratchet.md deleted file mode 100644 index de5af38a2..000000000 --- a/spec/crypto-ratchet.md +++ /dev/null @@ -1,13 +0,0 @@ -# Double Ratchet & PQDR - -> Implements the double ratchet algorithm with post-quantum extensions (PQDR). - -**Protocol reference**: [`protocol/pqdr.md`](../protocol/pqdr.md) - -## State - -## Transitions - -## Key Derivation - -## Functions diff --git a/spec/crypto-tls.md b/spec/crypto-tls.md deleted file mode 100644 index 9327ae69a..000000000 --- a/spec/crypto-tls.md +++ /dev/null @@ -1,11 +0,0 @@ -# TLS & Certificate Chains - -> TLS session setup, certificate chain construction, and server identity validation. - -## TLS Setup - -## Certificate Validation - -## Trust Anchoring - -## Functions diff --git a/spec/crypto.md b/spec/crypto.md deleted file mode 100644 index ec8fb0a49..000000000 --- a/spec/crypto.md +++ /dev/null @@ -1,19 +0,0 @@ -# Cryptographic Primitives - -> All cryptographic primitives used across SimpleX protocols. - -## Ed25519 - -## X25519 - -## NaCl - -## AES-GCM - -## SHA - -## HKDF - -## Key Generation - -## Functions diff --git a/spec/encoding.md b/spec/encoding.md deleted file mode 100644 index f5501cfab..000000000 --- a/spec/encoding.md +++ /dev/null @@ -1,332 +0,0 @@ -# Encoding - -> Binary and string encoding used across all SimpleX protocols. - -**Source files**: [`Encoding.hs`](../src/Simplex/Messaging/Encoding.hs), [`Encoding/String.hs`](../src/Simplex/Messaging/Encoding/String.hs), [`Parsers.hs`](../src/Simplex/Messaging/Parsers.hs) - -## Overview - -Two encoding layers serve different purposes: - -- **`Encoding`** — Binary wire format for SMP protocol transmissions. Compact, no delimiters between fields. Used in all on-the-wire protocol messages. -- **`StrEncoding`** — Human-readable string format for configuration, URIs, logs, and JSON serialization. Uses base64url for binary data, decimal for numbers, comma-separated lists, space-separated tuples. - -Both are typeclasses with `MINIMAL` pragmas requiring `encode` + (`decode` | `parser`), with the missing one derived from the other. - -## Binary Encoding (`Encoding` class) - -```haskell -class Encoding a where - smpEncode :: a -> ByteString - smpDecode :: ByteString -> Either String a -- default: parseAll smpP - smpP :: Parser a -- default: smpDecode <$?> smpP -``` - -### Length-prefix conventions - -| Type | Prefix | Max size | -|------|--------|----------| -| `ByteString` | 1-byte length (Word8 as Char) | 255 bytes | -| `Large` (newtype) | 2-byte length (Word16 big-endian) | 65535 bytes | -| `Tail` (newtype) | None — consumes rest of input | Unlimited | -| Lists (`smpEncodeList`) | 1-byte count prefix, then concatenated items | 255 items | -| `NonEmpty` | Same as list (fails on count=0) | 255 items | - -### Scalar types - -| Type | Encoding | Bytes | -|------|----------|-------| -| `Char` | Raw byte | 1 | -| `Bool` | `'T'` / `'F'` (0x54 / 0x46) | 1 | -| `Word16` | Big-endian | 2 | -| `Word32` | Big-endian | 4 | -| `Int64` | Two big-endian Word32s (high then low) | 8 | -| `SystemTime` | `systemSeconds` as Int64 (nanoseconds dropped) | 8 | -| `Text` | UTF-8 then ByteString encoding (1-byte length prefix) | 1 + len | -| `String` | `B.pack` then ByteString encoding | 1 + len | - -### `Maybe a` - -``` -Nothing → '0' (0x30) -Just x → '1' (0x31) ++ smpEncode x -``` - -Tags are ASCII characters `'0'`/`'1'`, not binary 0x00/0x01. - -### Tuples - -Tuples (2 through 8) encode as simple concatenation — no length prefix, no separator. Fields are parsed sequentially using each component's `smpP`. This works because each component's parser knows how many bytes to consume (via its own length prefix or fixed size). - -### Combinators - -| Function | Signature | Purpose | -|----------|-----------|---------| -| `_smpP` | `Parser a` | Space-prefixed parser (`A.space *> smpP`) | -| `smpEncodeList` | `[a] -> ByteString` | 1-byte count + concatenated items | -| `smpListP` | `Parser [a]` | Parse count then that many items | -| `lenEncode` | `Int -> Char` | Int to single-byte length char | - -## String Encoding (`StrEncoding` class) - -```haskell -class StrEncoding a where - strEncode :: a -> ByteString - strDecode :: ByteString -> Either String a -- default: parseAll strP - strP :: Parser a -- default: strDecode <$?> base64urlP -``` - -Key difference from `Encoding`: the default `strP` parses base64url input first, then applies `strDecode`. This means types that only implement `strDecode` will automatically accept base64url-encoded input. - -### Instance conventions - -| Type | Encoding | -|------|----------| -| `ByteString` | base64url (non-empty required) | -| `Word16`, `Word32` | Decimal string | -| `Int`, `Int64` | Signed decimal | -| `Char`, `Bool` | Delegates to `Encoding` (`smpEncode`/`smpP`) | -| `Maybe a` | Empty string = `Nothing`, otherwise `strEncode a` | -| `Text` | UTF-8 bytes, parsed until space/newline | -| `SystemTime` | `systemSeconds` as Int64 (decimal) | -| `UTCTime` | ISO 8601 string | -| `CertificateChain` | Comma-separated base64url blobs | -| `Fingerprint` | base64url of fingerprint bytes | - -### Collection encoding - -| Type | Separator | -|------|-----------| -| Lists (`strEncodeList`) | Comma `,` | -| `NonEmpty` | Comma (fails on empty) | -| `Set a` | Comma | -| `IntSet` | Comma | -| Tuples (2-6) | Space (` `) | - -### `Str` newtype - -Raw string (not base64url-encoded). Parses until space, consumes trailing space. Used for string-valued protocol fields that should not be base64-encoded. - -### `TextEncoding` class - -```haskell -class TextEncoding a where - textEncode :: a -> Text - textDecode :: Text -> Maybe a -``` - -Separate from `StrEncoding` — operates on `Text` rather than `ByteString`. Used for types that need Text representation (e.g., enum display names). - -### JSON bridge functions - -| Function | Purpose | -|----------|---------| -| `strToJSON` | `StrEncoding a => a -> J.Value` via `decodeLatin1 . strEncode` | -| `strToJEncoding` | Same, for Aeson encoding | -| `strParseJSON` | `StrEncoding a => String -> J.Value -> JT.Parser a` — parse JSON string via `strP` | -| `textToJSON` | `TextEncoding a => a -> J.Value` | -| `textToEncoding` | Same, for Aeson encoding | -| `textParseJSON` | `TextEncoding a => String -> J.Value -> JT.Parser a` | - -## Parsers - -**Source**: [`Parsers.hs`](../src/Simplex/Messaging/Parsers.hs) - -### Core parsing functions - -| Function | Signature | Purpose | -|----------|-----------|---------| -| `parseAll` | `Parser a -> ByteString -> Either String a` | Parse consuming all input (fails if bytes remain) | -| `parse` | `Parser a -> e -> ByteString -> Either e a` | `parseAll` with custom error type (discards error string) | -| `parseE` | `(String -> e) -> Parser a -> ByteString -> ExceptT e IO a` | `parseAll` lifted into `ExceptT` | -| `parseE'` | `(String -> e) -> Parser a -> ByteString -> ExceptT e IO a` | Like `parseE` but allows trailing input | -| `parseRead1` | `Read a => Parser a` | Parse a word then `readMaybe` it | -| `parseString` | `(ByteString -> Either String a) -> String -> a` | Parse from `String` (errors with `error`) | - -### `base64P` - -Standard base64 parser (not base64url — uses `+`/`/` alphabet). Takes alphanumeric + `+`/`/` characters, optional `=` padding, then decodes. Contrast with `base64urlP` in `Encoding/String.hs` which uses `-`/`_` alphabet. - -### JSON options helpers - -Platform-conditional JSON encoding for cross-platform compatibility (Haskell ↔ Swift). - -| Function | Purpose | -|----------|---------| -| `enumJSON` | All-nullary constructors as strings, with tag modifier | -| `sumTypeJSON` | Platform-conditional: `taggedObjectJSON` on non-Darwin, `singleFieldJSON` on Darwin | -| `taggedObjectJSON` | `{"type": "Tag", "data": {...}}` format | -| `singleFieldJSON` | `{"Tag": value}` format | -| `defaultJSON` | Default options with `omitNothingFields = True` | - -Pattern synonyms for JSON field names: -- `TaggedObjectJSONTag = "type"` -- `TaggedObjectJSONData = "data"` -- `SingleFieldJSONTag = "_owsf"` - -### String helpers - -| Function | Purpose | -|----------|---------| -| `fstToLower` | Lowercase first character | -| `dropPrefix` | Remove prefix string, lowercase remainder | -| `textP` | Parse rest of input as UTF-8 `String` | - -## Auxiliary Types and Utilities - -### TMap - -**Source**: [`TMap.hs`](../src/Simplex/Messaging/TMap.hs) - -```haskell -type TMap k a = TVar (Map k a) -``` - -STM-based concurrent map. Wraps `Data.Map.Strict` in a `TVar`. All mutations use `modifyTVar'` (strict) to prevent thunk accumulation. - -| Function | Notes | -|----------|-------| -| `emptyIO` | IO allocation (`newTVarIO`) | -| `singleton` | STM allocation | -| `clear` | Reset to empty | -| `lookup` / `lookupIO` | STM / non-transactional IO read | -| `member` / `memberIO` | STM / non-transactional IO membership | -| `insert` / `insertM` | Insert value / insert from STM action | -| `delete` | Remove key | -| `lookupInsert` | Atomic lookup-then-insert (returns old value) | -| `lookupDelete` | Atomic lookup-then-delete | -| `adjust` / `update` / `alter` / `alterF` | Standard Map operations lifted to STM | -| `union` | Merge `Map` into `TMap` | - -`lookupIO`/`memberIO` use `readTVarIO` — single-read outside STM transaction, useful when you need a snapshot without composing with other STM operations. - -### SessionVar - -**Source**: [`Session.hs`](../src/Simplex/Messaging/Session.hs) - -Race-safe session management using TMVar + monotonic ID. - -```haskell -data SessionVar a = SessionVar - { sessionVar :: TMVar a -- result slot - , sessionVarId :: Int -- monotonic ID from TVar counter - , sessionVarTs :: UTCTime -- creation timestamp - } -``` - -| Function | Purpose | -|----------|---------| -| `getSessVar` | Lookup or create session. Returns `Left new` or `Right existing` | -| `removeSessVar` | Delete session only if ID matches (prevents removing a replacement) | -| `tryReadSessVar` | Non-blocking read of session result | - -The ID-match check in `removeSessVar` prevents a race where: -1. Thread A creates session #5, starts work -2. Thread B creates session #6 (replacing #5 in TMap) -3. Thread A finishes, tries to remove — ID mismatch, removal blocked - -### ServiceScheme - -**Source**: [`ServiceScheme.hs`](../src/Simplex/Messaging/ServiceScheme.hs) - -```haskell -data ServiceScheme = SSSimplex | SSAppServer SrvLoc -data SrvLoc = SrvLoc HostName ServiceName -``` - -URI scheme for SimpleX service addresses. `SSSimplex` encodes as `"simplex:"`, `SSAppServer` as `"https://host:port"`. - -`simplexChat` is the constant `SSAppServer (SrvLoc "simplex.chat" "")`. - -### SystemTime - -**Source**: [`SystemTime.hs`](../src/Simplex/Messaging/SystemTime.hs) - -```haskell -newtype RoundedSystemTime (t :: Nat) = RoundedSystemTime { roundedSeconds :: Int64 } -type SystemDate = RoundedSystemTime 86400 -- day precision -type SystemSeconds = RoundedSystemTime 1 -- second precision -``` - -Phantom-typed time rounding. The `Nat` type parameter specifies rounding granularity in seconds. - -| Function | Purpose | -|----------|---------| -| `getRoundedSystemTime` | Get current time rounded to `t` seconds | -| `getSystemDate` | Alias for day-rounded time | -| `getSystemSeconds` | Second-precision (no rounding needed, just drops nanoseconds) | -| `roundedToUTCTime` | Convert back to `UTCTime` | - -`RoundedSystemTime` derives `FromField`/`ToField` for SQLite storage and `FromJSON`/`ToJSON` for API serialization. - -### Util - -**Source**: [`Util.hs`](../src/Simplex/Messaging/Util.hs) - -Selected utilities used across the codebase: - -**Monadic combinators**: - -| Function | Signature | Purpose | -|----------|-----------|---------| -| `<$?>` | `MonadFail m => (a -> Either String b) -> m a -> m b` | Lift fallible function into parser | -| `$>>=` | `(Monad m, Monad f, Traversable f) => m (f a) -> (a -> m (f b)) -> m (f b)` | Monadic bind through nested monad | -| `ifM` / `whenM` / `unlessM` | Monadic conditionals | | -| `anyM` | Short-circuit `any` for monadic predicates (strict) | | - -**Error handling**: - -| Function | Purpose | -|----------|---------| -| `tryAllErrors` | Catch all exceptions (including async) into `ExceptT` | -| `catchAllErrors` | Same with handler | -| `tryAllOwnErrors` | Catch only "own" exceptions (re-throws async cancellation) | -| `catchAllOwnErrors` | Same with handler | -| `isOwnException` | `StackOverflow`, `HeapOverflow`, `AllocationLimitExceeded` | -| `isAsyncCancellation` | Any `SomeAsyncException` except own exceptions | -| `catchThrow` | Catch exceptions, wrap in Left | -| `allFinally` | `tryAllErrors` + `final` + `except` (like `finally` for ExceptT) | - -The own-vs-async distinction is critical: `catchOwn`/`tryAllOwnErrors` never swallow async cancellation (`ThreadKilled`, `UserInterrupt`, etc.), only synchronous exceptions and resource exhaustion (`StackOverflow`, `HeapOverflow`, `AllocationLimitExceeded`). - -**STM**: - -| Function | Purpose | -|----------|---------| -| `tryWriteTBQueue` | Non-blocking bounded queue write, returns success | - -**Database result helpers**: - -| Function | Purpose | -|----------|---------| -| `firstRow` | Extract first row with transform, or Left error | -| `maybeFirstRow` | Extract first row as Maybe | -| `firstRow'` | Like `firstRow` but transform can also fail | - -**Collection utilities**: - -| Function | Purpose | -|----------|---------| -| `groupOn` | `groupBy` using equality on projected key | -| `groupAllOn` | `groupOn` after `sortOn` (groups non-adjacent elements) | -| `toChunks` | Split list into `NonEmpty` chunks of size n | -| `packZipWith` | Optimized ByteString zipWith (direct memory access) | - -**Miscellaneous**: - -| Function | Purpose | -|----------|---------| -| `safeDecodeUtf8` | Decode UTF-8 replacing errors with `'?'` | -| `bshow` / `tshow` | `show` to `ByteString` / `Text` | -| `threadDelay'` | `Int64` delay (handles overflow by looping) | -| `diffToMicroseconds` / `diffToMilliseconds` | `NominalDiffTime` conversion | -| `labelMyThread` | Label current thread for debugging | -| `encodeJSON` / `decodeJSON` | `ToJSON a => a -> Text` / `FromJSON a => Text -> Maybe a` | -| `traverseWithKey_` | `Map` traversal discarding results | - -## Security notes - -- **Length prefix overflow**: `ByteString` encoding uses 1-byte length — silently truncates strings > 255 bytes. Callers must ensure size bounds before encoding. `Large` extends to 65535 bytes via Word16 prefix. -- **`Tail` unbounded**: `Tail` consumes all remaining input with no size check. Only safe when total message size is already bounded (e.g., within a padded SMP block). -- **base64 vs base64url**: `Parsers.base64P` uses standard alphabet (`+`/`/`), while `String.base64urlP` uses URL-safe alphabet (`-`/`_`). Mixing them causes silent decode failures. -- **`safeDecodeUtf8`**: Replaces invalid UTF-8 with `'?'` rather than failing. Suitable for logging/display, not for security-critical string comparison. diff --git a/spec/ntf-protocol.md b/spec/ntf-protocol.md deleted file mode 100644 index c826e7e72..000000000 --- a/spec/ntf-protocol.md +++ /dev/null @@ -1,15 +0,0 @@ -# NTF Protocol Implementation - -> Implements NTF commands, token registration, and subscription lifecycle for push notifications. - -**Protocol reference**: [`protocol/push-notifications.md`](../protocol/push-notifications.md) - -## Types - -## Commands - -## Token Lifecycle - -## Subscription Lifecycle - -## Functions diff --git a/spec/ntf-server.md b/spec/ntf-server.md deleted file mode 100644 index 4a39957e3..000000000 --- a/spec/ntf-server.md +++ /dev/null @@ -1,11 +0,0 @@ -# Notification Server - -> Notification server implementation: token management, subscriptions, and APNS integration. - -## Token Management - -## Subscription Management - -## APNS Integration - -## Functions diff --git a/spec/remote-control.md b/spec/remote-control.md deleted file mode 100644 index 5a064437c..000000000 --- a/spec/remote-control.md +++ /dev/null @@ -1,11 +0,0 @@ -# Remote Control (XRCP) - -> XRCP implementation: discovery, invitation, and session management. - -## Discovery - -## Invitation - -## Session Management - -## Functions diff --git a/spec/smp-client.md b/spec/smp-client.md deleted file mode 100644 index 39ae87f9a..000000000 --- a/spec/smp-client.md +++ /dev/null @@ -1,11 +0,0 @@ -# SMP Client - -> SMP client implementation: protocol operations, proxy relay, and reconnection logic. - -## Protocol Operations - -## Proxy Relay - -## Reconnection - -## Functions diff --git a/spec/smp-protocol.md b/spec/smp-protocol.md deleted file mode 100644 index 0def97941..000000000 --- a/spec/smp-protocol.md +++ /dev/null @@ -1,13 +0,0 @@ -# SMP Protocol Implementation - -> Implements SMP commands, types, and binary encoding for the SimpleX Messaging Protocol. - -**Protocol reference**: [`protocol/simplex-messaging.md`](../protocol/simplex-messaging.md) - -## Types - -## Commands - -## Encoding - -## Functions diff --git a/spec/smp-server.md b/spec/smp-server.md deleted file mode 100644 index 696d19067..000000000 --- a/spec/smp-server.md +++ /dev/null @@ -1,13 +0,0 @@ -# SMP Server - -> SMP server implementation: connection handling, queue operations, proxying, and control port. - -## Connection Handling - -## Queue Operations - -## Proxying - -## Control - -## Functions diff --git a/spec/storage-agent.md b/spec/storage-agent.md deleted file mode 100644 index 4ba4c414c..000000000 --- a/spec/storage-agent.md +++ /dev/null @@ -1,11 +0,0 @@ -# Agent Storage - -> Agent storage backends: SQLite, Postgres, and migration framework. - -## SQLite Backend - -## Postgres Backend - -## Migration Framework - -## Functions diff --git a/spec/storage-server.md b/spec/storage-server.md deleted file mode 100644 index b2dec1842..000000000 --- a/spec/storage-server.md +++ /dev/null @@ -1,9 +0,0 @@ -# Server Storage - -> Server storage backends: STM queues and message stores (STM, Journal, Postgres). - -## STM Queues - -## Message Stores (STM, Journal, Postgres) - -## Functions diff --git a/spec/transport-http2.md b/spec/transport-http2.md deleted file mode 100644 index 2594b8431..000000000 --- a/spec/transport-http2.md +++ /dev/null @@ -1,13 +0,0 @@ -# HTTP/2 Transport - -> HTTP/2 framing, client and server sessions, and file streaming for XFTP. - -## Framing - -## Client Sessions - -## Server Sessions - -## File Streaming - -## Functions diff --git a/spec/transport-websocket.md b/spec/transport-websocket.md deleted file mode 100644 index 182a43c47..000000000 --- a/spec/transport-websocket.md +++ /dev/null @@ -1,7 +0,0 @@ -# WebSocket Transport - -> WebSocket adapter for browser-based SimpleX clients. - -## Adapter - -## Functions diff --git a/spec/transport.md b/spec/transport.md deleted file mode 100644 index 0e50a67d9..000000000 --- a/spec/transport.md +++ /dev/null @@ -1,11 +0,0 @@ -# Transport Layer - -> Transport abstraction, handshake protocol, and block padding for metadata privacy. - -## Abstraction - -## Handshake Protocol - -## Block Padding - -## Functions diff --git a/spec/version.md b/spec/version.md deleted file mode 100644 index 19ad786fe..000000000 --- a/spec/version.md +++ /dev/null @@ -1,177 +0,0 @@ -# Version Negotiation - -> Version ranges and compatibility checking for protocol evolution. - -**Source files**: [`Version.hs`](../src/Simplex/Messaging/Version.hs), [`Version/Internal.hs`](../src/Simplex/Messaging/Version/Internal.hs) - -## Overview - -All SimpleX protocols use version negotiation during handshake. Each party advertises a `VersionRange` (min..max supported), and negotiation produces a `Compatible` proof value if the ranges overlap — choosing the highest mutually-supported version. - -The `Compatible` newtype can only be constructed internally (constructor is not exported), so the type system enforces that compatibility was actually checked. - -## Types - -### `Version v` - -```haskell -newtype Version v = Version Word16 -``` - -Phantom-typed version number. The phantom `v` distinguishes version spaces (e.g., SMP versions vs Agent versions vs XFTP versions) at the type level, preventing accidental comparison across protocols. - -- `Encoding`: 2 bytes big-endian (via Word16 instance) -- `StrEncoding`: decimal string -- JSON: numeric value -- Derives: `Eq`, `Ord`, `Show` - -The constructor is exported from `Version.Internal` but not from `Version`, so application code cannot fabricate versions — they must come from protocol constants or parsing. - -### `VersionRange v` - -```haskell -data VersionRange v = VRange - { minVersion :: Version v - , maxVersion :: Version v - } -``` - -Invariant: `minVersion <= maxVersion` (enforced by smart constructors). - -The `VRange` constructor is not exported — only the pattern synonym `VersionRange` (read-only) is public. - -- `Encoding`: two Word16s concatenated (4 bytes total) -- `StrEncoding`: `"min-max"` or `"v"` if min == max -- JSON: `{"minVersion": n, "maxVersion": n}` - -### `VersionScope v` - -```haskell -class VersionScope v -``` - -Empty typeclass used as a constraint on version operations. Each protocol declares its version scope: - -```haskell -instance VersionScope SMP -instance VersionScope Agent -``` - -This prevents accidentally mixing version ranges from different protocols in negotiation functions. - -### `Compatible a` - -```haskell -newtype Compatible a = Compatible_ a - -pattern Compatible :: a -> Compatible a -pattern Compatible a <- Compatible_ a -``` - -Proof that compatibility was checked. The `Compatible_` constructor is not exported — `Compatible` is a read-only pattern synonym. The only way to obtain a `Compatible` value is through `compatibleVersion`, `compatibleVRange`, `proveCompatible`, or the internal `mkCompatibleIf`. - -### `VersionI` / `VersionRangeI` type classes - -Multi-param typeclasses with functional dependencies for generic version/range operations. Allow extension types that wrap `Version` or `VersionRange` to participate in negotiation: - -```haskell -class VersionScope v => VersionI v a | a -> v where - type VersionRangeT v a -- associated type: range form - version :: a -> Version v - toVersionRangeT :: a -> VersionRange v -> VersionRangeT v a - -class VersionScope v => VersionRangeI v a | a -> v where - type VersionT v a -- associated type: version form - versionRange :: a -> VersionRange v - toVersionRange :: a -> VersionRange v -> a - toVersionT :: a -> Version v -> VersionT v a -``` - -Identity instances exist for `Version v` and `VersionRange v` themselves. - -## Functions - -### Construction - -| Function | Signature | Purpose | -|----------|-----------|---------| -| `mkVersionRange` | `Version v -> Version v -> VersionRange v` | Construct range, `error` if min > max | -| `safeVersionRange` | `Version v -> Version v -> Maybe (VersionRange v)` | Safe construction, `Nothing` if invalid | -| `versionToRange` | `Version v -> VersionRange v` | Singleton range (min == max) | - -### Compatibility checking - -### isCompatible - -**Purpose**: Check if a single version falls within a range. - -```haskell -isCompatible :: VersionI v a => a -> VersionRange v -> Bool -``` - -### isCompatibleRange - -**Purpose**: Check if two version ranges overlap: `min1 <= max2 && min2 <= max1`. - -```haskell -isCompatibleRange :: VersionRangeI v a => a -> VersionRange v -> Bool -``` - -### proveCompatible - -**Purpose**: If version is compatible, wrap in `Compatible` proof. Returns `Nothing` if out of range. - -```haskell -proveCompatible :: VersionI v a => a -> VersionRange v -> Maybe (Compatible a) -``` - -### Negotiation - -### compatibleVersion - -**Purpose**: Negotiate a single version from two ranges. Returns `min(max1, max2)` — the highest mutually-supported version. Returns `Nothing` if ranges don't overlap. - -```haskell -compatibleVersion :: VersionRangeI v a => a -> VersionRange v -> Maybe (Compatible (VersionT v a)) -``` - -### compatibleVRange - -**Purpose**: Compute the intersection of two version ranges: `(max(min1,min2), min(max1,max2))`. Returns `Nothing` if the intersection is empty. - -```haskell -compatibleVRange :: VersionRangeI v a => a -> VersionRange v -> Maybe (Compatible a) -``` - -### compatibleVRange' - -**Purpose**: Cap a version range's maximum at a given version. Returns `Nothing` if the cap is below the range's minimum. - -```haskell -compatibleVRange' :: VersionRangeI v a => a -> Version v -> Maybe (Compatible a) -``` - -## Protocol version constants - -Version constants for each protocol are defined in their respective Transport modules. For SMP, key gates include: - -- `currentSMPAgentVersion`, `supportedSMPAgentVRange` — current negotiation range -- `serviceCertsSMPVersion = 16` — service certificate handshake -- `rcvServiceSMPVersion = 19` — service subscription commands - -See [`transport.md`](transport.md) and [`rcv-services.md`](rcv-services.md) for protocol-specific version constants. - -## Negotiation protocol - -During handshake: -1. Client sends its `VersionRange` to server -2. Server computes `compatibleVRange clientRange serverRange` -3. If `Nothing` → reject connection (incompatible) -4. If `Just (Compatible agreedRange)` → use `maxVersion agreedRange` as the effective protocol version - -The `Compatible` proof flows through the connection setup, ensuring all subsequent version-gated code paths have evidence that negotiation occurred. - -## Security notes - -- **No downgrade attack protection in negotiation itself** — an active MITM could modify the version range to force a lower version. Protection comes from the TLS layer (authentication prevents MITM) and from servers setting minimum version floors. -- **`mkVersionRange` uses `error`** — only safe for compile-time constants. Runtime construction must use `safeVersionRange`. diff --git a/spec/xftp-client.md b/spec/xftp-client.md deleted file mode 100644 index 99306bb73..000000000 --- a/spec/xftp-client.md +++ /dev/null @@ -1,11 +0,0 @@ -# XFTP Client - -> XFTP client implementation: file operations, CLI interface, and agent integration. - -## File Operations - -## CLI - -## Agent - -## Functions diff --git a/spec/xftp-protocol.md b/spec/xftp-protocol.md deleted file mode 100644 index 26eb950be..000000000 --- a/spec/xftp-protocol.md +++ /dev/null @@ -1,13 +0,0 @@ -# XFTP Protocol Implementation - -> Implements XFTP commands, types, and chunk operations for the SimpleX File Transfer Protocol. - -**Protocol reference**: [`protocol/xftp.md`](../protocol/xftp.md) - -## Types - -## Commands - -## Chunk Operations - -## Functions diff --git a/spec/xftp-server.md b/spec/xftp-server.md deleted file mode 100644 index bdcbbb9aa..000000000 --- a/spec/xftp-server.md +++ /dev/null @@ -1,11 +0,0 @@ -# XFTP Server - -> XFTP server implementation: chunk storage, recipient management, and control port. - -## Chunk Storage - -## Recipient Management - -## Control - -## Functions diff --git a/spec/xrcp-protocol.md b/spec/xrcp-protocol.md deleted file mode 100644 index 8f084f7ca..000000000 --- a/spec/xrcp-protocol.md +++ /dev/null @@ -1,13 +0,0 @@ -# XRCP Protocol Implementation - -> Implements XRCP session handshake and commands for remote control of SimpleX clients. - -**Protocol reference**: [`protocol/xrcp.md`](../protocol/xrcp.md) - -## Types - -## Session Handshake - -## Commands - -## Functions