simplexmq/CODE.md

simplexmq — LLM Navigation Guide

This file is the entry point for LLMs working on simplexmq. Read it before making any code changes.

Three-Layer Architecture

simplexmq maintains three documentation layers alongside source code:

Layer	Directory	Answers	Audience
Product	product/	What does this do? Who uses it? What must never break?	Anyone reasoning about behavior, privacy, security
Spec	spec/	How does the code work? What does each function do? What are the security invariants?	LLMs and developers modifying code
Protocol	protocol/	What is the wire protocol? What are the message formats and state machines?	Protocol implementors, formal verification

Additionally:

• rfcs/ — Protocol evolution: each RFC describes a delta to a protocol spec
• product/threat-model.md — Comprehensive threat model across all protocols
• spec/security-invariants.md — Every security invariant with enforcement and test coverage

Navigation Workflow

When modifying code, follow this sequence:

1. Identify scope — Find the relevant component in product/concepts.md
2. Load product context — Read the component file in product/components/ to understand what users depend on
3. Load spec context — Read the relevant spec/ file(s) for implementation details and call graphs
4. Check security — Read spec/security-invariants.md for any invariants enforced by the code you're changing
5. Load source — Read the actual source files referenced in spec/
6. Identify impact — Trace the call graph to understand what your change affects
7. Implement — Make the change
8. Update all layers — Update spec/, product/, and protocol/ (if wire protocol changed) to stay coherent

Protocol Specifications

Consolidated protocol specs live in protocol/. These describe the wire protocols as originally specified. Code has advanced beyond these versions — Phase 2 of this project will synchronize them.

File	Protocol	Spec version	Code version
simplex-messaging.md	SMP (simplex messaging)	v9	SMP relay v18, SMP client v4
agent-protocol.md	Agent (duplex connections)	v5	Agent v7
xftp.md	XFTP (file transfer)	v2	XFTP v3
xrcp.md	XRCP (remote control)	v1	RCP v1
push-notifications.md	Push notifications	v2	NTF v3
pqdr.md	PQDR (post-quantum double ratchet)	v1	E2E v3
overview-tjr.md	Cross-protocol overview	—	—

Note: SMP has multiple version axes — VersionSMP (relay/transport, currently 18), VersionSMPC (client protocol, currently 4), and VersionSMPA (agent, currently 7). These are negotiated independently.

Protocol specs are amended in place when implementation changes. RFCs in rfcs/ track the evolution history.

Source Structure

src/Simplex/
  Messaging/
    Protocol.hs, Protocol/Types.hs      — SMP wire protocol types + encoding
    Client.hs                            — SMP client (protocol operations, proxy relay)
    Client/Agent.hs                      — Low-level async SMP agent
    Server.hs                            — SMP server request handling
    Server/Env/STM.hs                    — Server environment + STM state
    Server/Main.hs, Server/Main/Init.hs  — Server CLI + initialization
    Server/QueueStore/                   — Queue storage (STM, Postgres)
    Server/MsgStore/                     — Message storage (STM, Journal, Postgres)
    Server/MsgStore/Journal.hs           — Journal message store (1000 lines)
    Server/StoreLog/                     — Store log (append-only write, read-compact-rewrite restore)
    Server/NtfStore.hs                   — Message notification store
    Server/Control.hs, Server/CLI.hs     — Control protocol + CLI utilities
    Server/Stats.hs, Server/Prometheus.hs — Metrics
    Server/Information.hs                — Server public information / metadata
    Agent.hs                             — SMP agent: duplex connections, queue rotation
    Agent/Client.hs                      — Agent's SMP/XFTP/NTF client management
    Agent/Protocol.hs                    — Agent wire protocol types + encoding (2200 lines)
    Agent/Store.hs                       — Agent storage types (queues, connections, messages)
    Agent/Store/AgentStore.hs              — Agent storage implementation (3500 lines)
    Agent/Store/                          — Agent storage backends (SQLite, Postgres)
    Agent/Env/SQLite.hs                  — Agent environment + configuration
    Agent/NtfSubSupervisor.hs            — Notification subscription management
    Agent/TSessionSubs.hs                — Transport session subscriptions
    Agent/Stats.hs                       — Agent statistics
    Agent/RetryInterval.hs               — Retry interval logic
    Agent/Lock.hs                        — Named locks
    Agent/QueryString.hs                 — Query string parsing
    Transport.hs                         — TLS transport abstraction + handshake
    Transport/Client.hs, Transport/Server.hs — TLS client + server
    Transport/HTTP2.hs                   — HTTP/2 transport setup
    Transport/HTTP2/Client.hs            — HTTP/2 client
    Transport/HTTP2/Server.hs            — HTTP/2 server
    Transport/HTTP2/File.hs              — HTTP/2 file streaming
    Transport/WebSockets.hs              — WebSocket adapter
    Transport/Buffer.hs                  — Transport buffering
    Transport/KeepAlive.hs               — TCP keepalive
    Transport/Shared.hs                  — Certificate chain validation
    Transport/Credentials.hs             — TLS credential generation
    Crypto.hs                            — All cryptographic primitives
    Crypto/File.hs                       — File encryption (NaCl secret box + lazy)
    Crypto/Lazy.hs                       — Lazy hashing + encryption
    Crypto/Ratchet.hs                    — Double ratchet + PQDR
    Crypto/ShortLink.hs                  — Short link key derivation
    Crypto/SNTRUP761.hs                  — Post-quantum KEM hybrid secret
    Crypto/SNTRUP761/Bindings.hs         — sntrup761 C FFI bindings
    Notifications/Protocol.hs            — NTF wire protocol types + encoding
    Notifications/Types.hs               — NTF agent types (tokens, subscriptions)
    Notifications/Transport.hs           — NTF transport handshake
    Notifications/Client.hs              — NTF client operations
    Notifications/Server.hs              — NTF server
    Notifications/Server/Env.hs          — NTF server environment + config
    Notifications/Server/Store.hs        — NTF server storage (STM)
    Notifications/Server/Store/Postgres.hs — NTF server storage (Postgres)
    Notifications/Server/Push/APNS.hs    — Apple push notification integration
    Notifications/Server/Push/APNS/Internal.hs — APNS HTTP/2 client
    Notifications/Server/Main.hs         — NTF server CLI
    Notifications/Server/Stats.hs        — NTF server metrics
    Notifications/Server/Prometheus.hs   — NTF Prometheus metrics
    Notifications/Server/Control.hs      — NTF server control
    Encoding.hs, Encoding/String.hs      — Binary + string encoding
    Version.hs, Version/Internal.hs      — Version ranges + negotiation
    Util.hs                              — Utilities (error handling, STM, grouping)
    Parsers.hs                           — Attoparsec parser combinators
    TMap.hs                              — Transactional map (STM)
    Compression.hs                       — Zstd compression
    ServiceScheme.hs                     — Service scheme + server location types
    Session.hs                           — Session variables (TVar-based)
    SystemTime.hs                        — Rounded system time types
  FileTransfer/
    Protocol.hs                          — XFTP wire protocol types + encoding
    Client.hs                            — XFTP client operations
    Client/Agent.hs                      — XFTP client agent (connection pooling)
    Client/Main.hs                       — XFTP CLI client implementation
    Client/Presets.hs                    — Default XFTP servers
    Server.hs                            — XFTP server request handling
    Server/Env.hs                        — XFTP server environment + config
    Server/Store.hs                      — XFTP server storage
    Server/StoreLog.hs                   — XFTP server store log
    Server/Main.hs                       — XFTP server CLI
    Server/Stats.hs                      — XFTP server metrics
    Server/Prometheus.hs                 — XFTP Prometheus metrics
    Server/Control.hs                    — XFTP server control
    Agent.hs                             — XFTP agent operations
    Description.hs                       — File description format
    Transport.hs                         — XFTP transport
    Crypto.hs                            — File encryption for transfer
    Types.hs                             — File transfer types
    Chunks.hs                            — Chunk sizing
  RemoteControl/
    Client.hs                            — XRCP client (ctrl device)
    Invitation.hs                        — XRCP invitation handling
    Discovery.hs                         — Local network discovery
    Discovery/Multicast.hsc              — Multicast discovery (C FFI)
    Types.hs                             — XRCP types + version

apps/
  smp-server/Main.hs                    — SMP server executable
  smp-server/web/Static.hs              — SMP server web static files
  xftp-server/Main.hs                   — XFTP server executable
  xftp/Main.hs                          — XFTP CLI executable
  ntf-server/Main.hs                    — Notification server executable
  smp-agent/Main.hs                     — SMP agent (experimental, not in cabal)

Linking Conventions

spec → src

Fully qualified exported function names inline in prose: Simplex.Messaging.Client.connectSMPProxiedRelay. Use Grep/Glob to locate in source. For app targets: xftp/Main.main.

src → spec

Comment above function:

-- spec/crypto-tls.md#certificate-chain-validation
-- Validates relay certificate chain to prevent proxy MITM (SI-XX)
connectSMPProxiedRelay :: ...

spec ↔ spec

Named markdown heading anchors: spec/crypto.md#ed25519-signing

spec ↔ product

Cross-references: product/rules.md#pr-05, spec/security-invariants.md#si-01

protocol/ references

protocol/simplex-messaging.md with section name

Build Flags

simplexmq builds with several flag combinations:

Flag	Effect
(none)	Default: SQLite storage, all executables
-fserver_postgres	Postgres backend for SMP server
-fclient_postgres	Postgres backend for agent storage
-fclient_library	Library-only build (no server executables)
-fswift	Swift JSON format for mobile bindings
-fuse_crypton	Use crypton in cryptostore

All flag combinations must compile with --enable-tests. Verify with:

cabal build all --ghc-options="-O0" [-flags] [--enable-tests]

Change Protocol

Every code change must maintain coherence across all three layers:

1. Code change — Implement in src/
2. Spec update — Update the relevant spec/ file(s): types, call graphs, security notes
3. Product update — If user-visible behavior changed, update product/ files
4. Protocol update — If wire protocol changed, amend protocol/ spec (requires user approval)
5. Security check — If the change touches a trust boundary, update spec/security-invariants.md

Protocol spec amendments require explicit user approval before committing.