mirror of
https://github.com/simplex-chat/simplexmq.git
synced 2026-10-23 21:43:44 +00:00
links
This commit is contained in:
Evgeny @ SimpleX Chat committed
1 parent
388b13d417
commit
13e8b7b411
4 files changed
+106
-59
No files matched your search
@@ -1,18 +1,16 @@
|
||||
# SimpleXMQ
|
||||
# SimpleX Network
|
||||
|
||||
[](https://github.com/simplex-chat/simplexmq/actions/workflows/build.yml)
|
||||
[](https://github.com/simplex-chat/simplexmq/releases)
|
||||
|
||||
## SimpleX Network software
|
||||
|
||||
SimpleXMQ provides the software for [SimpleX Network](./protocol/overview-tjr.md) — a general-purpose packet routing network where endpoints exchange data through independently operated routers using resource-based addressing. Unlike IP networks, SimpleX addresses identify resources on routers (queues, data packets), not endpoint devices. Participants do not need globally unique identifiers to communicate.
|
||||
The simplexmq package provides the software for [SimpleX Network](./protocol/overview-tjr.md) — a general-purpose packet routing network where endpoints exchange data through independently operated routers using resource-based addressing. Unlike IP networks, SimpleX addresses identify resources on routers (queues, data packets), not endpoint devices. Participants do not need globally unique identifiers to communicate.
|
||||
|
||||
The software is organized in three layers:
|
||||
|
||||
```
|
||||
Application (e.g. SimpleX Chat)
|
||||
+----------------------------------+
|
||||
| SimpleX Agent | Layer 3 — bidirectional connections, e2e encryption
|
||||
| SimpleX Agent | Layer 3 — duplex connections, e2e encryption
|
||||
+----------------------------------+
|
||||
| SimpleX Client Libraries | Layer 2 — protocol clients for SMP, XFTP
|
||||
+----------------------------------+
|
||||
@@ -22,9 +20,9 @@ The software is organized in three layers:
|
||||
|
||||
[SimpleX Chat](https://github.com/simplex-chat/simplex-chat) is one application built on Layer 3. IoT devices, AI services, monitoring systems, and automated services are other applications that can use Layers 2 or 3 directly.
|
||||
|
||||
SimpleXMQ is implemented in Haskell, benefiting from robust software transactional memory (STM) and concurrency primitives.
|
||||
The simplexmq package is implemented in Haskell, benefiting from robust software transactional memory (STM) and concurrency primitives.
|
||||
|
||||
See the [SimpleX Network overview](./protocol/overview-tjr.md) for the full protocol architecture, trust model, and security analysis.
|
||||
See the [SimpleX Network overview](./protocol/overview-tjr.md) for the full protocol architecture, trust model, and [security analysis](./protocol/security.md).
|
||||
|
||||
## Architecture
|
||||
|
||||
@@ -32,9 +30,9 @@ See the [SimpleX Network overview](./protocol/overview-tjr.md) for the full prot
|
||||
|
||||
Routers are the network infrastructure — they accept, buffer, and deliver packets. Three router types serve different purposes:
|
||||
|
||||
- **SMP routers** provide messaging queues — unidirectional, ordered sequences of fixed-size packets (16,384 bytes). Protocol: [SMP](./protocol/simplex-messaging.md).
|
||||
- **XFTP routers** provide data packet storage — individually addressed blocks in fixed sizes (64KB–4MB) for larger payloads. Protocol: [XFTP](./protocol/xftp.md).
|
||||
- **NTF routers** bridge to platform push services (APNS) for mobile notification delivery. Protocol: [Push Notifications](./protocol/push-notifications.md).
|
||||
- **SMP routers** provide messaging queues — unidirectional, ordered sequences of fixed-size packets (16,384 bytes). Protocol: [SMP](./protocol/simplex-messaging.md). Module spec: [`Simplex.Messaging.Server`](./spec/modules/Simplex/Messaging/Server.md).
|
||||
- **XFTP routers** accept and deliver data packets — individually addressed blocks in fixed sizes (64KB–4MB) for larger payloads. Protocol: [XFTP](./protocol/xftp.md). Module spec: [`Simplex.FileTransfer.Server`](./spec/modules/Simplex/FileTransfer/Server.md).
|
||||
- **NTF routers** bridge to platform push services (APNS) for mobile notification delivery. Protocol: [Push Notifications](./protocol/push-notifications.md). Module spec: [`Simplex.Messaging.Notifications.Server`](./spec/modules/Simplex/Messaging/Notifications/Server.md).
|
||||
|
||||
#### Running an SMP router
|
||||
|
||||
@@ -54,21 +52,21 @@ See [docs/ROUTERS.md](./docs/ROUTERS.md) for XFTP/NTF router setup, advanced con
|
||||
|
||||
[Client libraries](./docs/CLIENT.md) provide low-level protocol access to SimpleX routers. They implement the wire protocols (SMP, XFTP) and handle connection lifecycle, command authentication, and keep-alive.
|
||||
|
||||
The [SMP client](./src/Simplex/Messaging/Client.hs) offers a functional Haskell API with STM queues for asynchronous event delivery. The [XFTP client](./src/Simplex/FileTransfer/Client.hs) handles data packet upload/download with per-download forward secrecy.
|
||||
The [SMP client](./src/Simplex/Messaging/Client.hs) ([module spec](./spec/modules/Simplex/Messaging/Client.md)) offers a functional Haskell API with STM queues for asynchronous event delivery. The [XFTP client](./src/Simplex/FileTransfer/Client.hs) ([module spec](./spec/modules/Simplex/FileTransfer/Client.md)) sends and receives data packets with per-request forward secrecy.
|
||||
|
||||
Applications that manage their own encryption and connection logic — IoT devices, sensors, simple data pipelines — can use this layer directly. See [docs/CLIENT.md](./docs/CLIENT.md).
|
||||
|
||||
### SimpleX Agent
|
||||
|
||||
The [Agent](./docs/AGENT.md) builds bidirectional encrypted connections on top of the client libraries. It manages:
|
||||
The [Agent](./docs/AGENT.md) builds duplex encrypted connections on top of the client libraries. It manages:
|
||||
|
||||
- Duplex connections from unidirectional queue pairs
|
||||
- Duplex connections from simplex queue pairs
|
||||
- End-to-end encryption with double ratchet and post-quantum extensions
|
||||
- File transfer with chunking, encryption, and multi-router distribution
|
||||
- Queue rotation for metadata privacy
|
||||
- Push notification subscriptions
|
||||
|
||||
The [Agent library](./src/Simplex/Messaging/Agent.hs) communicates via STM queues using the [ACommand](./src/Simplex/Messaging/Agent/Protocol.hs) type — no serialization needed.
|
||||
The [Agent library](./src/Simplex/Messaging/Agent.hs) ([module spec](./spec/modules/Simplex/Messaging/Agent.md)) communicates via STM queues using the [ACommand](./src/Simplex/Messaging/Agent/Protocol.hs) type — no serialization needed. The Agent implements the [Agent protocol](./protocol/agent-protocol.md) for duplex connections and uses the [PQDR protocol](./protocol/pqdr.md) for end-to-end encryption. Cross-device remote control uses the [XRCP protocol](./protocol/xrcp.md).
|
||||
|
||||
See [docs/AGENT.md](./docs/AGENT.md).
|
||||
|
||||
|
||||
+39
-23
@@ -1,16 +1,16 @@
|
||||
# SimpleX Agent
|
||||
|
||||
The SimpleX Agent builds bidirectional encrypted connections on top of [SimpleX client libraries](CLIENT.md). It manages the full lifecycle of secure communication: connection establishment, end-to-end encryption, queue rotation, file transfer, and push notifications.
|
||||
The SimpleX Agent builds duplex encrypted connections on top of [SimpleX client libraries](CLIENT.md). It manages the full lifecycle of secure communication: connection establishment, end-to-end encryption, queue rotation, file transfer, and push notifications.
|
||||
|
||||
This is **Layer 3** of the [SimpleX Network architecture](../protocol/overview-tjr.md). Layer 1 is the routers; Layer 2 is the [client libraries](CLIENT.md) that speak the wire protocols. The Agent adds the connection semantics that applications need.
|
||||
|
||||
**Source**: [`Simplex.Messaging.Agent`](../src/Simplex/Messaging/Agent.hs)
|
||||
**Source**: [`Simplex.Messaging.Agent`](../src/Simplex/Messaging/Agent.hs) — **Module spec**: [`spec/modules/Simplex/Messaging/Agent.md`](../spec/modules/Simplex/Messaging/Agent.md)
|
||||
|
||||
## Connections
|
||||
|
||||
The Agent turns unidirectional SMP queues into bidirectional connections:
|
||||
The Agent turns simplex (unidirectional) SMP queues into duplex connections, implementing the [Agent protocol](../protocol/agent-protocol.md):
|
||||
|
||||
- **Duplex connections**: each connection uses a pair of SMP queues — one for each direction. The queues can be on different routers chosen independently by each party.
|
||||
- **Duplex connections**: each connection uses a pair of SMP queues — one for each direction. The queues can be on different routers chosen independently by each party. See the [duplex connection procedure](../protocol/agent-protocol.md) for the full handshake.
|
||||
- **Connection establishment**: one party creates a connection and generates an invitation (containing router address, queue ID, and public keys). The invitation is passed out-of-band (QR code, link, etc.). The other party joins by creating a reverse queue and completing the handshake.
|
||||
- **Connection links**: the Agent supports connection links (long and short) for sharing connection invitations via URLs. Short links use a separate SMP queue to store the full invitation, allowing compact QR codes.
|
||||
- **Queue rotation**: the Agent periodically rotates the underlying SMP queues, limiting the window for metadata correlation. Rotation is transparent to the application — the connection identity is stable while the underlying queues change.
|
||||
@@ -18,53 +18,53 @@ The Agent turns unidirectional SMP queues into bidirectional connections:
|
||||
|
||||
## Encryption
|
||||
|
||||
The Agent provides end-to-end encryption with forward secrecy and break-in recovery:
|
||||
The Agent provides end-to-end encryption with forward secrecy and break-in recovery, specified in the [Post-Quantum Double Ratchet protocol](../protocol/pqdr.md):
|
||||
|
||||
- **Double ratchet**: messages are encrypted using a double ratchet protocol derived from the Signal protocol. Each message uses a unique key; compromising one key does not reveal past or future messages.
|
||||
- **Post-quantum extensions**: the ratchet supports hybrid key exchange using SNTRUP761 (a lattice-based KEM) combined with X25519 DH. This provides protection against future quantum computers that could break classical DH.
|
||||
- **Double ratchet**: messages are encrypted using a double ratchet protocol derived from the Signal protocol. Each message uses a unique key; compromising one key does not reveal past or future messages. See the [PQDR specification](../protocol/pqdr.md) for the full ratchet state machine.
|
||||
- **Post-quantum extensions**: the ratchet supports hybrid key exchange using SNTRUP761 (a lattice-based KEM) combined with X25519 DH. This provides protection against future quantum computers that could break classical DH. See the [SNTRUP761 module spec](../spec/modules/Simplex/Messaging/Crypto/SNTRUP761.md) and [Ratchet module spec](../spec/modules/Simplex/Messaging/Crypto/Ratchet.md) for implementation details.
|
||||
- **Ratchet synchronization**: if the ratchet state becomes desynchronized (e.g., due to message loss or device restore), the Agent detects this and can negotiate resynchronization with the peer.
|
||||
- **Per-queue encryption**: in addition to end-to-end encryption, each queue has a separate encryption layer between sender and router, preventing traffic correlation even if TLS is compromised.
|
||||
- **Per-queue encryption**: in addition to end-to-end encryption, each queue has a separate encryption layer between sender and router, preventing traffic correlation even if TLS is compromised. See the [SMP protocol security model](../protocol/simplex-messaging.md).
|
||||
|
||||
## File Transfer
|
||||
|
||||
The Agent handles file transfer over [XFTP](../protocol/xftp.md) routers:
|
||||
The Agent handles file transfer over [XFTP](../protocol/xftp.md) routers. File transfer orchestration is implemented in the [XFTP Agent module](../spec/modules/Simplex/FileTransfer/Agent.md):
|
||||
|
||||
- **Chunking**: files are split into chunks, each stored as a data packet on an XFTP router. Chunk sizes are fixed powers of 2 (64KB to 4MB), hiding the actual file size.
|
||||
- **Client-side encryption**: files are encrypted and padded before upload. The recipient decrypts after downloading all chunks. The encryption key and file metadata are sent through the SMP connection, not through XFTP.
|
||||
- **Multi-router distribution**: chunks can be uploaded to different XFTP routers, and each chunk can have multiple replicas on different routers for redundancy.
|
||||
- **Redirect chains**: for metadata privacy, file descriptors can be stored as XFTP data packets themselves, creating an indirection layer between the SMP message and the actual file location.
|
||||
- **Chunking**: files are split into chunks, each sent as a data packet to an XFTP router. Chunk sizes are fixed powers of 2 (64KB to 4MB), hiding the actual file size. See the [file description module spec](../spec/modules/Simplex/FileTransfer/Description.md) for chunk size selection and file descriptor format.
|
||||
- **Client-side encryption**: files are encrypted and padded before being sent to XFTP routers. The recipient decrypts after receiving all chunks. The encryption key and file metadata are sent through the SMP connection, not through XFTP. See [file crypto module spec](../spec/modules/Simplex/FileTransfer/Crypto.md).
|
||||
- **Multi-router distribution**: chunks can be sent to different XFTP routers, and each chunk can have multiple replicas on different routers for redundancy.
|
||||
- **Redirect chains**: for metadata privacy, file descriptors can be sent as XFTP data packets themselves, creating an indirection layer between the SMP message and the actual file location.
|
||||
|
||||
## Notifications
|
||||
|
||||
The Agent manages push notification subscriptions for mobile devices:
|
||||
The Agent manages push notification subscriptions for mobile devices, using the [Push Notifications protocol](../protocol/push-notifications.md). Notification supervision is handled by the [NtfSubSupervisor](../spec/modules/Simplex/Messaging/Agent/NtfSubSupervisor.md):
|
||||
|
||||
- **Token registration**: registers device push tokens with NTF (notification) routers, which bridge to platform push services (APNS).
|
||||
- **Token registration**: registers device push tokens with NTF (notification) routers, which bridge to platform push services (APNS). See the [NTF client module spec](../spec/modules/Simplex/Messaging/Notifications/Client.md).
|
||||
- **Notification subscriptions**: creates NTF subscriptions for SMP queues so that incoming messages trigger push notifications without requiring persistent connections.
|
||||
- **Privacy preservation**: push notifications contain only a notification ID, not message content. The device wakes, connects to the SMP router, and retrieves the actual message.
|
||||
- **Privacy preservation**: push notifications contain only a notification ID, not message content. The device wakes, connects to the SMP router, and retrieves the actual message. See the [Push Notifications protocol](../protocol/push-notifications.md) for the full flow.
|
||||
|
||||
## Integration
|
||||
|
||||
The Agent is designed to be embedded as a Haskell library:
|
||||
|
||||
- **STM queues**: the application communicates with the Agent via STM queues. Commands go in (`ACommand`), events come out (`AEvent`). No serialization or parsing — direct Haskell values.
|
||||
- **Async operation**: all network operations are asynchronous. The Agent manages internal worker threads for each router connection, message processing, and background tasks (cleanup, statistics, notification supervision).
|
||||
- **STM queues**: the application communicates with the Agent via STM queues. Commands go in (`ACommand`), events come out (`AEvent`). No serialization or parsing — direct Haskell values. The command/event types are defined in the [Agent Protocol module](../spec/modules/Simplex/Messaging/Agent/Protocol.md).
|
||||
- **Async operation**: all network operations are asynchronous. The Agent manages internal worker threads for each router connection, message processing, and background tasks (cleanup, statistics, notification supervision). See the [Agent Client module spec](../spec/modules/Simplex/Messaging/Agent/Client.md) for worker architecture.
|
||||
- **Background mode**: on mobile platforms, the Agent can run in a reduced mode with only the message receiver active, minimizing resource usage when the app is backgrounded.
|
||||
- **Dual database backends**: the Agent supports both SQLite (for mobile/desktop) and PostgreSQL (for server deployments) as persistence backends, selected at compile time.
|
||||
- **Dual database backends**: the Agent supports both SQLite (for mobile/desktop) and PostgreSQL (for server deployments) as persistence backends, selected at compile time. See [Agent Store Interface](../spec/modules/Simplex/Messaging/Agent/Store/Interface.md) and [Agent Store Postgres](../spec/modules/Simplex/Messaging/Agent/Store/Postgres.md).
|
||||
|
||||
## Use cases
|
||||
|
||||
- **Chat applications**: [SimpleX Chat](https://github.com/simplex-chat/simplex-chat) is the reference application, using the full Agent API for messaging, file sharing, groups, and calls.
|
||||
- **Bots and automated services**: services that need bidirectional encrypted communication with SimpleX Chat users or other Agent-based applications.
|
||||
- **Any application needing secure bidirectional communication** over the SimpleX Network without implementing the connection management, encryption, and queue rotation logic directly.
|
||||
- **Bots and automated services**: services that need duplex encrypted communication with SimpleX Chat users or other Agent-based applications.
|
||||
- **Any application needing secure duplex communication** over the SimpleX Network without implementing the connection management, encryption, and queue rotation logic directly.
|
||||
|
||||
## What this layer adds over client libraries
|
||||
|
||||
| Capability | Client (Layer 2) | Agent (Layer 3) |
|
||||
|---|---|---|
|
||||
| Queue operations | Direct | Managed transparently |
|
||||
| Connection model | Unidirectional queues | Bidirectional connections |
|
||||
| Connection model | Simplex (unidirectional) queues | Duplex connections |
|
||||
| Encryption | Application's responsibility | Double ratchet with PQ extensions |
|
||||
| File transfer | Raw data packet upload/download | Chunking, encryption, reassembly |
|
||||
| File transfer | Raw data packet send/receive | Chunking, encryption, reassembly |
|
||||
| Identity | Per-queue keys | Per-connection, rotatable |
|
||||
| Notifications | Not available | NTF router integration |
|
||||
|
||||
@@ -73,3 +73,19 @@ The Agent is designed to be embedded as a Haskell library:
|
||||
- [Agent Protocol](../protocol/agent-protocol.md) — duplex connection procedure, message format
|
||||
- [SimpleX Network overview](../protocol/overview-tjr.md) — architecture, trust model
|
||||
- [PQDR](../protocol/pqdr.md) — post-quantum double ratchet specification
|
||||
- [SimpleX Messaging Protocol](../protocol/simplex-messaging.md) — SMP queue operations used by the Agent
|
||||
- [XFTP Protocol](../protocol/xftp.md) — data packet operations for file transfer
|
||||
- [Push Notifications Protocol](../protocol/push-notifications.md) — NTF token and subscription management
|
||||
- [XRCP Protocol](../protocol/xrcp.md) — remote control protocol for cross-device Agent access
|
||||
|
||||
## Module specs
|
||||
|
||||
- [Agent](../spec/modules/Simplex/Messaging/Agent.md) — main Agent module, connection lifecycle, message processing
|
||||
- [Agent Client](../spec/modules/Simplex/Messaging/Agent/Client.md) — worker threads, router connections, subscription management
|
||||
- [Agent Protocol](../spec/modules/Simplex/Messaging/Agent/Protocol.md) — ACommand/AEvent types, connection invitations
|
||||
- [Agent Store Interface](../spec/modules/Simplex/Messaging/Agent/Store/Interface.md) — database abstraction for SQLite/Postgres
|
||||
- [Agent Store (AgentStore)](../spec/modules/Simplex/Messaging/Agent/Store/AgentStore.md) — connection, queue, and message persistence
|
||||
- [NtfSubSupervisor](../spec/modules/Simplex/Messaging/Agent/NtfSubSupervisor.md) — notification subscription management
|
||||
- [XFTP Agent](../spec/modules/Simplex/FileTransfer/Agent.md) — file transfer orchestration
|
||||
- [Ratchet](../spec/modules/Simplex/Messaging/Crypto/Ratchet.md) — double ratchet implementation
|
||||
- [SNTRUP761](../spec/modules/Simplex/Messaging/Crypto/SNTRUP761.md) — post-quantum KEM
|
||||
+24
-18
@@ -2,21 +2,21 @@
|
||||
|
||||
SimpleX client libraries provide low-level protocol access to SimpleX routers. They implement the wire protocols ([SMP](../protocol/simplex-messaging.md), [XFTP](../protocol/xftp.md)) and handle connection lifecycle, but leave encryption, identity management, and connection orchestration to the application.
|
||||
|
||||
This is **Layer 2** of the [SimpleX Network architecture](../protocol/overview-tjr.md). Layer 1 is the routers themselves; Layer 3 is the [Agent](AGENT.md), which builds bidirectional encrypted connections on top of these libraries.
|
||||
This is **Layer 2** of the [SimpleX Network architecture](../protocol/overview-tjr.md). Layer 1 is the routers themselves; Layer 3 is the [Agent](AGENT.md), which builds duplex encrypted connections on top of these libraries.
|
||||
|
||||
## SMP Client
|
||||
|
||||
**Source**: [`Simplex.Messaging.Client`](../src/Simplex/Messaging/Client.hs)
|
||||
**Source**: [`Simplex.Messaging.Client`](../src/Simplex/Messaging/Client.hs) — **Module spec**: [`spec/modules/Simplex/Messaging/Client.md`](../spec/modules/Simplex/Messaging/Client.md)
|
||||
|
||||
The SMP client connects to SMP routers and manages messaging queues — the fundamental addressing primitive of the SimpleX Network. Each queue is a unidirectional, ordered sequence of fixed-size packets (16,384 bytes) with separate cryptographic credentials for sending and receiving.
|
||||
The SMP client connects to SMP routers and manages simplex messaging queues — the fundamental addressing primitive of the SimpleX Network. Each simplex queue is a unidirectional, ordered sequence of fixed-size packets (16,384 bytes) with separate cryptographic credentials for sending and receiving. The queue model and command set are defined in the [SMP protocol](../protocol/simplex-messaging.md).
|
||||
|
||||
### Capabilities
|
||||
|
||||
- **Queue management**: create, secure, subscribe to, and delete queues on any SMP router
|
||||
- **Queue management**: create, secure, subscribe to, and delete queues on any SMP router. Queue operations use the [SMP command set](../protocol/simplex-messaging.md) (NEW, KEY, SUB, DEL, etc.).
|
||||
- **Message sending and receiving**: send messages to a queue's sender address; receive messages from a queue's recipient address
|
||||
- **Command authentication**: each queue operation is authenticated with per-queue cryptographic keys (Ed25519, Ed448, or X25519)
|
||||
- **Command authentication**: each queue operation is authenticated with per-queue cryptographic keys (Ed25519, Ed448, or X25519). See the [SMP protocol security model](../protocol/simplex-messaging.md) for key roles.
|
||||
- **Keep-alive**: automatic ping loop detects and recovers from half-open connections
|
||||
- **Proxy forwarding**: send messages through a proxy router via 2-hop onion routing (PRXY/PFWD/RFWD commands), protecting the sender's IP address from the destination router
|
||||
- **Proxy forwarding**: send messages through a proxy router via 2-hop onion routing (PRXY/PFWD/RFWD commands), protecting the sender's IP address from the destination router. See [proxy forwarding details](../spec/modules/Simplex/Messaging/Client.md) in the module spec.
|
||||
- **Batched commands**: multiple commands can be sent in a single transmission for efficiency
|
||||
|
||||
### API model
|
||||
@@ -33,37 +33,33 @@ Routers are identified by the SHA-256 hash of their CA certificate fingerprint,
|
||||
|
||||
## XFTP Client
|
||||
|
||||
**Source**: [`Simplex.FileTransfer.Client`](../src/Simplex/FileTransfer/Client.hs)
|
||||
**Source**: [`Simplex.FileTransfer.Client`](../src/Simplex/FileTransfer/Client.hs) — **Module spec**: [`spec/modules/Simplex/FileTransfer/Client.md`](../spec/modules/Simplex/FileTransfer/Client.md)
|
||||
|
||||
The XFTP client connects to XFTP routers and manages data packets — individually addressed blocks used for larger payload delivery. Data packets come in fixed sizes (64KB, 256KB, 1MB, 4MB), hiding the actual payload size.
|
||||
The XFTP client connects to XFTP routers and manages data packets — individually addressed blocks used for larger payload delivery. Data packets come in fixed sizes (64KB, 256KB, 1MB, 4MB), hiding the actual payload size. The data packet lifecycle and command set are defined in the [XFTP protocol](../protocol/xftp.md).
|
||||
|
||||
### Capabilities
|
||||
|
||||
- **Data packet creation**: create data packets on routers with sender, recipient, and optional additional recipient credentials
|
||||
- **Upload**: send encrypted data in a single HTTP/2 streaming request (command + body)
|
||||
- **Download**: retrieve data packets with per-download ephemeral Diffie-Hellman key exchange, providing forward secrecy — compromising one download key does not reveal other downloads
|
||||
- **Data packet creation**: create data packets on routers with sender, recipient, and optional additional recipient credentials. See the [XFTP protocol](../protocol/xftp.md) for credential roles and packet lifecycle.
|
||||
- **Send** (FPUT): send encrypted data to the router in a single HTTP/2 streaming request (command + body)
|
||||
- **Receive** (FGET): receive data packets with per-request ephemeral Diffie-Hellman key exchange, providing forward secrecy — compromising one DH key does not reveal other received data packets
|
||||
- **Acknowledgment and deletion**: recipients acknowledge receipt; senders delete data packets after delivery
|
||||
|
||||
### Size selection
|
||||
|
||||
`prepareChunkSizes` selects data packet sizes using a threshold algorithm: if the remaining payload exceeds 75% of the next larger size, it uses the larger size. This balances storage efficiency against the number of round trips. Single-chunk payloads (e.g., redirect descriptors) can use `singleChunkSize` to verify they fit in one data packet.
|
||||
|
||||
## Use cases
|
||||
|
||||
These libraries are appropriate when the application manages its own encryption and connection logic:
|
||||
|
||||
- **IoT sensor data collection**: a sensor creates an SMP queue and sends readings; a collector subscribes and receives them. The queue address (router + queue ID + keys) is provisioned once, out-of-band.
|
||||
- **Device control**: a controller sends commands to an actuator's queue. Separate queues for commands and telemetry provide unidirectional isolation.
|
||||
- **Bulk data delivery**: an application encrypts and chunks a file, uploads data packets to XFTP routers, and shares the packet addresses with the recipient out-of-band.
|
||||
- **Bulk data delivery**: an application encrypts and chunks a file, sends data packets to XFTP routers, and shares the packet addresses with the recipient out-of-band.
|
||||
- **Custom protocols**: any application that needs unidirectional, router-mediated packet delivery without the overhead of the Agent's connection management.
|
||||
|
||||
## What this layer does NOT provide
|
||||
|
||||
The following capabilities require the [Agent](AGENT.md) (Layer 3):
|
||||
|
||||
- **Bidirectional connections** — the Agent pairs two unidirectional queues into a duplex connection
|
||||
- **Duplex connections** — the Agent pairs two simplex queues into a duplex connection
|
||||
- **End-to-end encryption** — the Agent manages double ratchet with post-quantum extensions
|
||||
- **File transfer** — the Agent handles chunking, encryption, padding, multi-router upload, and reassembly
|
||||
- **File transfer** — the Agent handles chunking, encryption, padding, multi-router distribution, and reassembly
|
||||
- **Queue rotation** — the Agent transparently rotates queues to limit metadata correlation
|
||||
- **Connection discovery** — connection links, short links, and contact addresses are Agent-level abstractions
|
||||
- **Push notifications** — notification token management and subscription is Agent-level
|
||||
@@ -73,3 +69,13 @@ The following capabilities require the [Agent](AGENT.md) (Layer 3):
|
||||
- [SimpleX Messaging Protocol](../protocol/simplex-messaging.md) — SMP wire format, commands, and security properties
|
||||
- [XFTP Protocol](../protocol/xftp.md) — XFTP wire format, data packet lifecycle
|
||||
- [SimpleX Network overview](../protocol/overview-tjr.md) — architecture, trust model, and design rationale
|
||||
|
||||
## Module specs
|
||||
|
||||
- [SMP Client](../spec/modules/Simplex/Messaging/Client.md) — proxy forwarding, batching, connection lifecycle, keepalive
|
||||
- [XFTP Client](../spec/modules/Simplex/FileTransfer/Client.md) — handshake, data packet operations, forward secrecy
|
||||
- [SMP Protocol types](../spec/modules/Simplex/Messaging/Protocol.md) — command types, queue addresses, message encoding
|
||||
- [XFTP Protocol types](../spec/modules/Simplex/FileTransfer/Protocol.md) — data packet types, XFTP commands
|
||||
- [Transport](../spec/modules/Simplex/Messaging/Transport.md) — TLS transport, session handshake
|
||||
- [HTTP/2 Client](../spec/modules/Simplex/Messaging/Transport/HTTP2/Client.md) — HTTP/2 transport layer
|
||||
- [Crypto](../spec/modules/Simplex/Messaging/Crypto.md) — cryptographic primitives used by clients
|
||||
+31
-4
@@ -6,7 +6,7 @@ This document covers deployment and advanced configuration. For an overview of t
|
||||
|
||||
## SMP Router
|
||||
|
||||
The SMP router provides messaging queues — unidirectional, ordered sequences of fixed-size packets (16,384 bytes each). It implements the [SimpleX Messaging Protocol](../protocol/simplex-messaging.md).
|
||||
The SMP router provides messaging queues — unidirectional, ordered sequences of fixed-size packets (16,384 bytes each). It implements the [SimpleX Messaging Protocol](../protocol/simplex-messaging.md). **Module spec**: [`spec/modules/Simplex/Messaging/Server.md`](../spec/modules/Simplex/Messaging/Server.md).
|
||||
|
||||
### Advanced configuration
|
||||
|
||||
@@ -35,13 +35,13 @@ echo 'PATH="/opt/homebrew/opt/openssl@3/bin:$PATH"' >> ~/.zprofile
|
||||
|
||||
## XFTP Router
|
||||
|
||||
The XFTP router provides data packet storage — individually addressed blocks in fixed sizes (64KB, 256KB, 1MB, 4MB). It implements the [XFTP protocol](../protocol/xftp.md). Data packets are used for larger payload delivery (files, media) where SMP queue packet sizes would be inefficient.
|
||||
The XFTP router accepts and delivers data packets — individually addressed blocks in fixed sizes (64KB, 256KB, 1MB, 4MB). It implements the [XFTP protocol](../protocol/xftp.md). Data packets are used for larger payload delivery (files, media) where SMP queue packet sizes would be inefficient. **Module spec**: [`spec/modules/Simplex/FileTransfer/Server.md`](../spec/modules/Simplex/FileTransfer/Server.md).
|
||||
|
||||
Initialize with `xftp-server init` and configure storage quota in `xftp-server.ini`.
|
||||
|
||||
## NTF Router
|
||||
|
||||
The NTF router bridges SimpleX Network to platform push notification services (APNS). It implements the [Push Notifications protocol](../protocol/push-notifications.md). Mobile clients register push tokens with the NTF router, which subscribes to their SMP queues and sends push notifications when messages arrive. The push notification contains only a notification ID, not message content.
|
||||
The NTF router bridges SimpleX Network to platform push notification services (APNS). It implements the [Push Notifications protocol](../protocol/push-notifications.md). Mobile clients register push tokens with the NTF router, which subscribes to their SMP queues and sends push notifications when messages arrive. The push notification contains only a notification ID, not message content. **Module spec**: [`spec/modules/Simplex/Messaging/Notifications/Server.md`](../spec/modules/Simplex/Messaging/Notifications/Server.md).
|
||||
|
||||
Initialize with `ntf-server init` and configure APNS credentials in `ntf-server.ini`.
|
||||
|
||||
@@ -184,7 +184,7 @@ smp-server init [-l] -n <fqdn>
|
||||
|
||||
## Monitoring
|
||||
|
||||
SMP and XFTP routers expose Prometheus metrics via a control port. The control port also supports commands for runtime inspection (queue counts, client counts, statistics). See module specs for details on available metrics and control commands.
|
||||
SMP and XFTP routers expose Prometheus metrics via a control port. The control port also supports commands for runtime inspection (queue counts, client counts, statistics). See [SMP Server Prometheus](../spec/modules/Simplex/Messaging/Server/Prometheus.md), [SMP Server Control](../spec/modules/Simplex/Messaging/Server/Control.md), and [NTF Server Control](../spec/modules/Simplex/Messaging/Notifications/Server/Control.md) module specs for available metrics and control commands.
|
||||
|
||||
## Protocol references
|
||||
|
||||
@@ -192,3 +192,30 @@ SMP and XFTP routers expose Prometheus metrics via a control port. The control p
|
||||
- [XFTP Protocol](../protocol/xftp.md) — data packet protocol
|
||||
- [Push Notifications Protocol](../protocol/push-notifications.md) — NTF protocol
|
||||
- [SimpleX Network overview](../protocol/overview-tjr.md) — architecture and trust model
|
||||
|
||||
## Module specs
|
||||
|
||||
### SMP Router
|
||||
- [Server](../spec/modules/Simplex/Messaging/Server.md) — main server module, client handling, message routing
|
||||
- [Server Main](../spec/modules/Simplex/Messaging/Server/Main.md) — server startup, initialization
|
||||
- [QueueStore](../spec/modules/Simplex/Messaging/Server/QueueStore.md) — queue persistence abstraction
|
||||
- [QueueStore Postgres](../spec/modules/Simplex/Messaging/Server/QueueStore/Postgres.md) — PostgreSQL queue store
|
||||
- [MsgStore](../spec/modules/Simplex/Messaging/Server/MsgStore.md) — message storage abstraction
|
||||
- [StoreLog](../spec/modules/Simplex/Messaging/Server/StoreLog.md) — append-only store log for queue persistence
|
||||
- [Server Control](../spec/modules/Simplex/Messaging/Server/Control.md) — control port commands
|
||||
- [Server Prometheus](../spec/modules/Simplex/Messaging/Server/Prometheus.md) — metrics export
|
||||
- [Server Stats](../spec/modules/Simplex/Messaging/Server/Stats.md) — statistics collection
|
||||
|
||||
### XFTP Router
|
||||
- [Server](../spec/modules/Simplex/FileTransfer/Server.md) — main server module, data packet handling
|
||||
- [Server Main](../spec/modules/Simplex/FileTransfer/Server/Main.md) — server startup
|
||||
- [Server Store](../spec/modules/Simplex/FileTransfer/Server/Store.md) — data packet storage
|
||||
- [Server StoreLog](../spec/modules/Simplex/FileTransfer/Server/StoreLog.md) — store log for packet persistence
|
||||
- [Server Stats](../spec/modules/Simplex/FileTransfer/Server/Stats.md) — statistics
|
||||
|
||||
### NTF Router
|
||||
- [Server](../spec/modules/Simplex/Messaging/Notifications/Server.md) — main server module
|
||||
- [Server Main](../spec/modules/Simplex/Messaging/Notifications/Server/Main.md) — server startup
|
||||
- [Server Store Postgres](../spec/modules/Simplex/Messaging/Notifications/Server/Store/Postgres.md) — PostgreSQL store for tokens and subscriptions
|
||||
- [APNS Push](../spec/modules/Simplex/Messaging/Notifications/Server/Push/APNS.md) — Apple push notification delivery
|
||||
- [Server Control](../spec/modules/Simplex/Messaging/Notifications/Server/Control.md) — control port commands
|
||||
Reference in new issue
Block a user