dandri/simplexmq
1
0
Fork 0
mirror of https://github.com/simplex-chat/simplexmq.git synced 2026-08-28 13:44:26 +00:00
Code Issues Packages Projects Releases Wiki Activity

update RFCs (#1730)

Browse Source 
* update RFCs

* update

* update overview

* update terminology

* original language in threat model

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
This commit is contained in:
Evgeny
2026-03-09 23:35:41 +00:00
committed by GitHub
co-authored by Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
parent 3c5752383d
commit 583f4e059d
64 changed files with 1476 additions and 1214 deletions
Download Patch File Download Diff File Expand all files Collapse all files
protocol/agent-protocol.md
+60 -62
View File
@@ -6,7 +6,7 @@ Version 7, 2025-01-24
- [Abstract](#abstract)
- [SMP agent](#smp-agent)
- [SMP servers management](#smp-servers-management)
- [SMP routers management](#smp-routers-management)
- [SMP agent protocol scope](#smp-agent-protocol-scope)
- [Duplex connection procedure](#duplex-connection-procedure)
- [Fast duplex connection procedure](#fast-duplex-connection-procedure)
@@ -35,13 +35,13 @@ Version 7, 2025-01-24
## Abstract
The purpose of SMP agent protocol is to define the syntax and the semantics of communications between the client and the agent that connects to [SMP](./simplex-messaging.md) servers.
The purpose of SMP agent protocol is to define the syntax and the semantics of communications between the client and the agent that connects to [SMP](./simplex-messaging.md) routers.
It provides:
- API to create and manage bi-directional (duplex) connections between the users of SMP agents consisting of two (or more) separate unidirectional (simplex) SMP queues, abstracting away multiple steps required to establish bi-directional connections and any information about the servers location from the users of the agent protocol.
- API to create and manage bi-directional (duplex) connections between the users of SMP agents consisting of two (or more) separate unidirectional (simplex) SMP queues, abstracting away multiple steps required to establish bi-directional connections and any information about the routers location from the users of the agent protocol.
- management of E2E encryption between SMP agents, generating ephemeral asymmetric keys for each connection.
- SMP command authentication on SMP servers, generating ephemeral keys for each SMP queue.
- TCP/TLS transport handshake with SMP servers.
- SMP command authentication on SMP routers, generating ephemeral keys for each SMP queue.
- TCP/TLS transport handshake with SMP routers.
- validation of message integrity.
SMP agent API provides no security between the agent and the client - it is assumed that the agent is executed in the trusted and secure environment, via the agent library, when the agent logic is included directly into the client application - [SimpleX Chat for terminal](https://github.com/simplex-chat/simplex-chat) uses this approach.
@@ -58,22 +58,22 @@ This document describes SMP agent protocol version 7. The version history:
## SMP agent
SMP agents communicate with each other via SMP servers using [simplex messaging protocol (SMP)](./simplex-messaging.md) according to the API calls used by the client applications. This protocol is a middle layer in SimpleX protocols (above SMP protocol but below any application level protocol) - it is intended to be used by client-side applications that need secure asynchronous bi-directional communication channels ("connections").
SMP agents communicate with each other via SMP routers using [simplex messaging protocol (SMP)](./simplex-messaging.md) according to the API calls used by the client applications. This protocol is a middle layer in SimpleX protocols (above SMP protocol but below any application level protocol) - it is intended to be used by client-side applications that need secure asynchronous bi-directional communication channels ("connections").
The agent must have a persistent storage to manage the states of known connections and of the client-side information of SMP queues that each connection consists of, and also the buffer of the most recent sent and received messages. The number of the messages that should be stored is implementation specific, depending on the error management approach that the agent implements; at the very least the agent must store the hashes and IDs of the last received and sent messages.
## SMP servers management
## SMP routers management
SMP agent API does not use the addresses of the SMP servers that the agent will use to create and use the connections (excluding the server address in queue URIs used in JOIN command). The list of the servers is a part of the agent configuration and can be dynamically changed by the agent implementation:
SMP agent API does not use the addresses of the SMP routers that the agent will use to create and use the connections (excluding the router address in queue URIs used in JOIN command). The list of the routers is a part of the agent configuration and can be dynamically changed by the agent implementation:
- by the client applications via any API that is outside of scope of this protocol.
- by the agents themselves based on availability and latency of the configured servers.
- by the agents themselves based on availability and latency of the configured routers.
## SMP agent protocol scope
SMP agent protocol has 2 main parts:
- the messages that SMP agents exchange with each other in order to:
- negotiate establishing unidirectional (simplex) encrypted queues on SMP servers.
- negotiate establishing unidirectional (simplex) encrypted queues on SMP routers.
- exchange client messages and delivery notifications, providing sequential message IDs and message integrity (by including the hash of the previous message).
- re-negotiate messaging queues to use and connection e2e encryption.
- the messages that the clients of SMP agents should send out-of-band (as pre-shared "invitation" including queue URIs) to protect [E2E encryption][1] from active attacks ([MITM attacks][2]).
@@ -86,19 +86,19 @@ SMP agent protocol has 2 main parts:
![Duplex connection procedure](./diagrams/duplex-messaging/duplex-creating.svg)
The procedure of establishing a duplex connection is explained on the example of Alice and Bob creating a bi-directional connection consisting of two unidirectional (simplex) queues, using SMP agents (A and B) to facilitate it, and two different SMP servers (which could be the same server). It is shown on the diagram above and has these steps:
The procedure of establishing a duplex connection is explained on the example of Alice and Bob creating a bi-directional connection consisting of two unidirectional (simplex) queues, using SMP agents (A and B) to facilitate it, and two different SMP routers (which could be the same router). It is shown on the diagram above and has these steps:
1. Alice requests the new connection from the SMP agent A using agent `createConnection` api function.
2. Agent A creates an SMP queue on the server (using [SMP protocol](./simplex-messaging.md) `NEW` command) and responds to Alice with the invitation that contains queue information and the encryption keys Bob's agent B should use. The invitation format is described in [Connection link](connection-link-1-time-invitation-and-contact-address).
2. Agent A creates an SMP queue on the router (using [SMP protocol](./simplex-messaging.md) `NEW` command) and responds to Alice with the invitation that contains queue information and the encryption keys Bob's agent B should use. The invitation format is described in [Connection link](connection-link-1-time-invitation-and-contact-address).
3. Alice sends the [connection link](#connection-link-1-time-invitation-and-contact-address) to Bob via any secure channel (out-of-band message) - as a link or as a QR code.
4. Bob uses agent `joinConnection` api function with the connection link as a parameter to agent B to accept the connection.
5. Agent B creates Bob's SMP reply queue with SMP server `NEW` command.
6. Agent B confirms the connection: sends an "SMP confirmation" with SMP server `SEND` command to the SMP queue specified in the connection link - SMP confirmation is an unauthenticated message with an ephemeral key that will be used to authenticate Bob's commands to the queue, as described in SMP protocol, and Bob's info (profile, public key for E2E encryption, and the connection link to this 2nd queue to Agent A - this connection link SHOULD use "simplex" URI scheme). This message is encrypted using key passed in the connection link (or with the derived shared secret, in which case public key for key derivation should be sent in clear text).
5. Agent B creates Bob's SMP reply queue with SMP router `NEW` command.
6. Agent B confirms the connection: sends an "SMP confirmation" with SMP router `SEND` command to the SMP queue specified in the connection link - SMP confirmation is an unauthenticated message with an ephemeral key that will be used to authenticate Bob's commands to the queue, as described in SMP protocol, and Bob's info (profile, public key for E2E encryption, and the connection link to this 2nd queue to Agent A - this connection link SHOULD use "simplex" URI scheme). This message is encrypted using key passed in the connection link (or with the derived shared secret, in which case public key for key derivation should be sent in clear text).
6. Alice confirms and continues the connection:
- Agent A receives the SMP confirmation containing Bob's key, reply queue and info as SMP server `MSG`.
- Agent A receives the SMP confirmation containing Bob's key, reply queue and info as SMP router `MSG`.
- Agent A notifies Alice sending `CONF` notification with Bob's info.
- Alice allows connection to continue with agent `allowConnection` api function.
- Agent A secures the queue with SMP server `KEY` command.
- Agent A secures the queue with SMP router `KEY` command.
- Agent A sends SMP confirmation with ephemeral sender key, ephemeral public encryption key and profile (but without reply queue).
7. Agent B confirms the connection:
- receives the confirmation.
@@ -114,7 +114,7 @@ The procedure of establishing a duplex connection is explained on the example of
At this point the duplex connection between Alice and Bob is established, they can use `SEND` command to send messages. The diagram also shows how the connection status changes for both parties, where the first part is the status of the SMP queue to receive messages, and the second part - the status of the queue to send messages.
The most communication happens between the agents and servers, from the point of view of Alice and Bob there are 4 steps (not including notifications):
The most communication happens between the agents and routers, from the point of view of Alice and Bob there are 4 steps (not including notifications):
1. Alice requests a new connection with `createConnection` agent API function and receives the connection link.
2. Alice passes connection link out-of-band to Bob.
@@ -137,14 +137,14 @@ Faster duplex connection process is possible with the `SKEY` command added in v9
![Fast duplex connection procedure](./diagrams/duplex-messaging/duplex-creating-fast.svg)
1. Alice requests the new connection from the SMP agent A using agent `createConnection` api function
2. Agent A creates an SMP queue on the server (using [SMP protocol](./simplex-messaging.md) `NEW` command with the flag allowing the sender to secure the queue) and responds to Alice with the invitation that contains queue information and the encryption keys Bob's agent B should use. The invitation format is described in [Connection link](connection-link-1-time-invitation-and-contact-address).
2. Agent A creates an SMP queue on the router (using [SMP protocol](./simplex-messaging.md) `NEW` command with the flag allowing the sender to secure the queue) and responds to Alice with the invitation that contains queue information and the encryption keys Bob's agent B should use. The invitation format is described in [Connection link](connection-link-1-time-invitation-and-contact-address).
3. Alice sends the [connection link](connection-link-1-time-invitation-and-contact-address) to Bob via any secure channel (out-of-band message) - as a link or as a QR code. This link contains the flag that the queue can be secured by the sender.
4. Bob uses agent `joinConnection` api function with the connection link as a parameter to agent B to accept the connection.
5. Agent B secures Alice's queue with SMP command `SKEY` - this command can be proxied.
6. Agent B creates Bob's SMP reply queue with SMP server `NEW` command (with the flag allowing the sender to secure the queue).
7. Agent B confirms the connection: sends an "SMP confirmation" with SMP server `SEND` command to the SMP queue specified in the connection link - SMP confirmation is an unauthenticated message with an ephemeral key that will be used to authenticate Bob's commands to the queue, as described in SMP protocol, and Bob's info (profile, public key for E2E encryption, and the connection link to this 2nd queue to Agent A - this connection link SHOULD use "simplex" URI scheme). This message is encrypted using key passed in the connection link (or with the derived shared secret, in which case public key for key derivation should be sent in clear text).
6. Agent B creates Bob's SMP reply queue with SMP router `NEW` command (with the flag allowing the sender to secure the queue).
7. Agent B confirms the connection: sends an "SMP confirmation" with SMP router `SEND` command to the SMP queue specified in the connection link - SMP confirmation is an unauthenticated message with an ephemeral key that will be used to authenticate Bob's commands to the queue, as described in SMP protocol, and Bob's info (profile, public key for E2E encryption, and the connection link to this 2nd queue to Agent A - this connection link SHOULD use "simplex" URI scheme). This message is encrypted using key passed in the connection link (or with the derived shared secret, in which case public key for key derivation should be sent in clear text).
8. Alice confirms the connection:
- Agent A receives the SMP confirmation containing Bob's key, reply queue and info as SMP server `MSG`.
- Agent A receives the SMP confirmation containing Bob's key, reply queue and info as SMP router `MSG`.
- Agent A notifies Alice sending `CONF` notification with Bob's info (that indicates that Agent B already secured the queue).
- Alice allows connection to continue with agent `allowConnection` api function.
- Agent A secures Bob's queue with SMP command `SKEY`.
@@ -159,11 +159,11 @@ Faster duplex connection process is possible with the `SKEY` command added in v9
SMP agents support creating a special type of connection - a contact address - that allows to connect to multiple network users who can send connection requests by sending 1-time connection links to the message queue.
This connection address uses a messaging queue on SMP server to receive invitations to connect - see `agentInvitation` message below. Once connection request is accepted, a new connection is created and the address itself is no longer used to send the messages - deleting this address does not disrupt the connections that were created via it.
This connection address uses a messaging queue on SMP router to receive invitations to connect - see `agentInvitation` message below. Once connection request is accepted, a new connection is created and the address itself is no longer used to send the messages - deleting this address does not disrupt the connections that were created via it.
## Communication between SMP agents
To establish duplex connections and to send messages on behalf of their clients, SMP agents communicate via SMP servers.
To establish duplex connections and to send messages on behalf of their clients, SMP agents communicate via SMP routers.
Agents use SMP message client body (the part of the SMP message after header - see [SMP protocol](./simplex-messaging.md)) to transmit agent client messages and exchange messages between each other.
@@ -217,7 +217,7 @@ Decrypted SMP message client body can be one of 4 types:
- to confirm that the new double ratchet encryption is agreed (`EREADY`).
- to notify another party that it can continue sending messages after queue capacity was exceeded (`A_QCONT`).
- to manage SMP queue rotation (`QADD`, `QKEY`, `QUSE`, `QTEST`).
- `msgPadding` - an optional message padding to make all SMP messages have constant size, to prevent servers from observing the actual message size. The only case the message padding can be absent is when the message has exactly the maximum size, in all other cases the message MUST be padded to a fixed size.
- `msgPadding` - an optional message padding to make all SMP messages have constant size, to prevent routers from observing the actual message size. The only case the message padding can be absent is when the message has exactly the maximum size, in all other cases the message MUST be padded to a fixed size.
### Messages between SMP agents
@@ -256,14 +256,14 @@ A_QCONT = %s"QC" sndQueueAddr
QADD = %s"QA" sndQueues
sndQueues = length 1*(newQueueUri replacedSndQueue)
newQueueUri = clientVRange smpServer senderId dhPublicKey [queueMode]
newQueueUri = clientVRange smpRouter senderId dhPublicKey [queueMode]
dhPublicKey = length x509encoded
queueMode = %s"M" / %s"C" ; M - messaging (sender can secure), C - contact
replacedSndQueue = "0" / "1" sndQueueAddr
QKEY = %s"QK" sndQueueKeys
sndQueueKeys = length 1*(newQueueInfo senderKey)
newQueueInfo = version smpServer senderId dhPublicKey [queueMode]
newQueueInfo = version smpRouter senderId dhPublicKey [queueMode]
senderKey = length x509encoded
QUSE = %s"QU" sndQueuesReady
@@ -273,8 +273,8 @@ primary = %s"T" / %s"F"
QTEST = %s"QT" sndQueueAddrs
sndQueueAddrs = length 1*sndQueueAddr
sndQueueAddr = smpServer senderId
smpServer = hosts port keyHash
sndQueueAddr = smpRouter senderId
smpRouter = hosts port keyHash
hosts = length 1*host
host = shortString
port = shortString
@@ -298,7 +298,7 @@ This message is not used with [fast duplex connection](#fast-duplex-connection-p
#### A_MSG message
This is the agent envelope used to send client messages once the connection is established. This is different from the MSG sent by SMP server to the agent and MSG event from SMP agent to the client that are sent in different contexts.
This is the agent envelope used to send client messages once the connection is established. This is different from the MSG sent by SMP router to the agent and MSG event from SMP agent to the client that are sent in different contexts.
#### A_RCVD message
@@ -314,7 +314,7 @@ This message is sent to notify the sender client that it can continue sending th
### Rotating messaging queue
SMP agents SHOULD support 4 messages to rotate message reception to another messaging server:
SMP agents SHOULD support 4 messages to rotate message reception to another messaging router:
`QADD`: add the new queue address(es) to the connection - sent by the client that initiates rotation.
`QKEY`: pass sender's key via existing connection (SMP confirmation message will not be used, to avoid the same "race" of the initial key exchange that would create the risk of intercepting the queue for the attacker) - sent by the client accepting the rotation
`QUSE`: instruct the sender to use the new queue with sender's queue ID as parameter. From this point some messages can be sent to both the new queue and the old queue.
@@ -384,7 +384,7 @@ connectionLink = connectionScheme "/" connLinkType "#/?v=" versionRange "&smp="
connLinkType = %s"invitation" / %s"contact"
connectionScheme = (%s"https://" clientAppServer) | %s"simplex:"
clientAppServer = hostname [ ":" port ]
; client app server, e.g. simplex.chat
; client app router, e.g. simplex.chat
versionRange = 1*DIGIT / 1*DIGIT "-" 1*DIGIT ; agent version range
e2eEncryption = <e2e encryption parameters for double ratchet>
smpQueues = smpQueue *(";" smpQueue) ; SMP queues for the connection (semicolon-separated)
@@ -392,17 +392,17 @@ smpQueue = <URL-encoded queueURI defined in SMP protocol>
clientData = <URL-encoded application-specific data>
```
All parameters are passed via URI hash to avoid sending them to the server (in case "https" scheme is used) - they can be used by the client-side code and processed by the client application. Parameters can be present in any order, any unknown additional parameters SHOULD be ignored.
All parameters are passed via URI hash to avoid sending them to the router (in case "https" scheme is used) - they can be used by the client-side code and processed by the client application. Parameters can be present in any order, any unknown additional parameters SHOULD be ignored.
`clientAppServer` is not an SMP server - it is a server that shows the instruction on how to download the client app that will connect using this connection link. This server can also host a mobile or desktop app manifest so that this link is opened directly in the app if it is installed on the device.
`clientAppServer` is not an SMP router - it is a server that shows the instruction on how to download the client app that will connect using this connection link. This server can also host a mobile or desktop app manifest so that this link is opened directly in the app if it is installed on the device.
"simplex" URI scheme in `connectionProtocol` can be used instead of client app server, to connect without creating any web traffic. Client apps MUST support this URI scheme.
"simplex" URI scheme in `connectionProtocol` can be used instead of client app router, to connect without creating any web traffic. Client apps MUST support this URI scheme.
See SMP protocol [out-of-band messages](./simplex-messaging.md#out-of-band-messages) for syntax of `queueURI`.
### Short connection link syntax
Short links provide a more compact representation by storing connection data on the server:
Short links provide a more compact representation by storing connection data on the router:
```
shortLink = shortLinkScheme "/" linkType "#" [linkId "/"] linkKey ["?" shortLinkParams]
@@ -415,7 +415,7 @@ shortLinkParams = hostParam ["&" portParam] ["&" keyHashParam]
hostParam = %s"h=" hostList
hostList = host *("," host)
portParam = %s"p=" port
keyHashParam = %s"c=" base64url ; server certificate fingerprint
keyHashParam = %s"c=" base64url ; router certificate fingerprint
```
Contact types:
@@ -424,11 +424,11 @@ Contact types:
- `g` (CCTGroup) - group connection
- `r` (CCTRelay) - relay connection
Short links can use either the `simplex:` scheme or `https://` with a server hostname. When using the simplex scheme, server information is included in query parameters.
Short links can use either the `simplex:` scheme or `https://` with a router hostname. When using the simplex scheme, router information is included in query parameters.
## Short links
Short links provide a compact representation of connection links by storing encrypted connection data on the SMP server. The link key in the URI fragment (after `#`) is never sent to the server, ensuring the server cannot decrypt the stored connection data.
Short links provide a compact representation of connection links by storing encrypted connection data on the SMP router. The link key in the URI fragment (after `#`) is never sent to the router, ensuring the router cannot decrypt the stored connection data.
### Link key derivation
@@ -441,7 +441,7 @@ linkKey = SHA3-256(fixedLinkData)
The fixed link data includes:
- Agent version range
- Root public key (Ed25519) for signing
- SMP queue connection request (server, queue IDs, encryption keys)
- SMP queue connection request (router, queue IDs, encryption keys)
- Optional link entity ID
For contact links, the link ID and encryption key are derived from the link key using HKDF:
@@ -459,7 +459,7 @@ encryptionKey = HKDF(info="SimpleXInvLink", key=linkKey, outputLen=32)
### Link data encryption
Link data stored on the server consists of two encrypted parts: fixed data and user data. Both are encrypted using NaCl secret_box (XSalsa20-Poly1305) with the derived encryption key:
Link data stored on the router consists of two encrypted parts: fixed data and user data. Both are encrypted using NaCl secret_box (XSalsa20-Poly1305) with the derived encryption key:
```abnf
queueLinkData = encFixedData encUserData
@@ -500,8 +500,8 @@ The fixed data is signed with the root key and its hash becomes the link key. Th
When a user receives a short link, the agent resolves it as follows:
1. Extract the link key from the URI fragment
2. Send `LGET` command to the SMP server with the link ID
3. Receive encrypted link data from the server
2. Send `LGET` command to the SMP router with the link ID
3. Receive encrypted link data from the router
4. Decrypt the link data using the link key
5. Extract the full connection information (SMP queue URI, encryption keys, profile)
6. Proceed with the standard connection procedure using `joinConnection`
@@ -513,7 +513,7 @@ For invitation links, the `LKEY` command is used to set the sender key when gett
The recipient who created the queue can manage the short link data:
- **LSET** - Set or update the link data associated with a queue. This is used when creating a short link or updating the user data (e.g., profile changes).
- **LDEL** - Delete the link data from the server. This effectively invalidates the short link.
- **LDEL** - Delete the link data from the router. This effectively invalidates the short link.
Short links support different connection modes:
- **invitation** - One-time invitation links that can only be used once
@@ -565,13 +565,13 @@ Client can `acceptContact` and `rejectContact`, with `OK` and `ERR` events in ca
#### Send message
`sendMessage` api is always asynchronous. The api call returns message ID, `SENT` event once the message is sent to the server, `MWARN` event in case of temporary delivery failure that can be resolved by the user (e.g., by connecting via Tor or by upgrading the client) and `MERR` in case of permanent delivery failure.
`sendMessage` api is always asynchronous. The api call returns message ID, `SENT` event once the message is sent to the router, `MWARN` event in case of temporary delivery failure that can be resolved by the user (e.g., by connecting via Tor or by upgrading the client) and `MERR` in case of permanent delivery failure.
#### Acknowledge received message
Messages are delivered to the client application via `MSG` event.
Client application must always `ackMessage` to receive the next one - failure to call it in reference implementation will prevent the delivery of subsequent messages until the client reconnects to the server.
Client application must always `ackMessage` to receive the next one - failure to call it in reference implementation will prevent the delivery of subsequent messages until the client reconnects to the router.
This api is also used to acknowledge message delivery to the sending party - that party client application will receive `RCVD` event.
@@ -589,11 +589,11 @@ This api is also used to acknowledge message delivery to the sending party - tha
#### Get short link data
`getConnectionLink` api (`LGET` command) is used to retrieve and decrypt the short link data from the server. Returns `LDATA` event with the decrypted link data.
`getConnectionLink` api (`LGET` command) is used to retrieve and decrypt the short link data from the router. Returns `LDATA` event with the decrypted link data.
#### Rotate message queue to another server
#### Rotate message queue to another router
`switchConnection` api is used to rotate connection queues to another messaging server.
`switchConnection` api is used to rotate connection queues to another messaging router.
#### Renegotiate e2e encryption
@@ -616,20 +616,20 @@ Agent API uses these events dispatch to notify client application about events r
- `INFO` - information from the party that initiated the connection with `createConnection` sent to the party accepting the connection with `joinConnection`.
- `CON` - notification that connection is established sent to both parties of the connection.
- `END` - notification that connection subscription is terminated when another client subscribed to the same messaging queue.
- `DOWN` - notification that connection server is temporarily unavailable.
- `UP` - notification that the subscriptions made in the current client session are resumed after the server became available.
- `DOWN` - notification that connection router is temporarily unavailable.
- `UP` - notification that the subscriptions made in the current client session are resumed after the router became available.
- `SWITCH` - notification about queue rotation process.
- `RSYNC` - notification about e2e encryption re-negotiation process.
- `SENT` - notification to confirm that the message was delivered to at least one of SMP servers. This notification contains the same message ID as returned to `sendMessage` api. `SENT` notification, depending on network availability, can be sent at any time later, potentially in the next client session.
- `SENT` - notification to confirm that the message was delivered to at least one of SMP routers. This notification contains the same message ID as returned to `sendMessage` api. `SENT` notification, depending on network availability, can be sent at any time later, potentially in the next client session.
- `MWARN` - temporary delivery failure that can be resolved by the user (e.g., by connecting via Tor or by upgrading the client).
- `MERR` - notification about permanent message delivery failure.
- `MERRS` - notification about permanent message delivery failure for multiple messages (e.g., when multiple messages expire).
- `MSG` - sent when agent receives the message from the SMP server.
- `MSG` - sent when agent receives the message from the SMP router.
- `MSGNTF` - sent after agent received and processed the message referenced in the push notification.
- `RCVD` - notification confirming message receipt by another party.
- `QCONT` - notification that the agent continued sending messages after queue capacity was exceeded and recipient received all messages.
- `LINK` - short link URI created or updated for a contact address.
- `LDATA` - decrypted short link data received from the server.
- `LDATA` - decrypted short link data received from the router.
- `DELD` - notification that the connection was deleted.
- `JOINED` - notification that a member joined via a contact address.
- `STAT` - connection statistics event.
@@ -642,9 +642,7 @@ This list of events is not exhaustive and provided for information only. Please
## Threat model
This threat model complements SimpleX Messaging Protocol [threat model](./overview-tjr.md#threat-model) with agent-level concerns: duplex connections, end-to-end encryption with [post-quantum double ratchet](./pqdr.md), message integrity, connection establishment and queue rotation. Only additional properties not covered in the SMP threat model are listed below.
This section uses network architecture terminology: SMP servers are referred to as routers, SMP queues as streams, and transport messages as packets — see [SimpleX Network architecture](../docs/network-architecture-v2.md).
This threat model complements SimpleX Messaging Protocol [threat model](./security.md#threat-model) with agent-level concerns: duplex connections, end-to-end encryption with [post-quantum double ratchet](./pqdr.md), message integrity, connection establishment and queue rotation. Only additional properties not covered in the SMP threat model are listed below.
#### Additional global assumptions
@@ -654,22 +652,22 @@ This section uses network architecture terminology: SMP servers are referred to
#### A passive adversary
*cannot:*
- learn the contents of packets, which are additionally encrypted with the double ratchet independently from per-stream encryption.
- learn the contents of packets, which are additionally encrypted with the double ratchet independently from per-queue encryption.
#### Destination router (chosen by the receiving client application)
*can:*
- correlate streams belonging to the same duplex connection when queue rotation creates a new stream on the same router.
- correlate queues belonging to the same duplex connection when queue rotation creates a new queue on the same router.
- when both peers of a connection chose the same router, correlate the two directions of the duplex connection.
*cannot:*
- compromise end-to-end encryption even with full access to the per-stream NaCl DH secret.
- correlate streams belonging to the same connection after queue rotation to a different router.
- compromise end-to-end encryption even with full access to the per-queue NaCl DH secret.
- correlate queues belonging to the same connection after queue rotation to a different router.
#### An attacker who obtained a client application's (decrypted) database
*can:*
- learn the full communication graph: all communication peers, associated router addresses, and stream identifiers.
- learn the full communication graph: all communication peers, associated router addresses, and queue identifiers.
*cannot:*
- decrypt future messages once the client application resumes communication and the double ratchet completes a new ratchet step, provided PQDR is active.
@@ -682,7 +680,7 @@ This section uses network architecture terminology: SMP servers are referred to
- prevent double ratchet advancement by not sending messages, delaying break-in recovery.
*cannot:*
- disrupt packet delivery in other streams.
- disrupt packet delivery in other queues.
#### An attacker who obtained a connection link
protocol/overview-tjr.md
+86 -227
View File
@@ -1,4 +1,4 @@
Revision 2, 2024-06-22
Revision 4, 2026-03-09
Evgeny Poberezkin
@@ -8,16 +8,17 @@ Evgeny Poberezkin
- [Introduction](#introduction)
- [What is SimpleX](#what-is-simplex)
- [Network model](#network-model)
- [Applications](#applications)
- [SimpleX objectives](#simplex-objectives)
- [In Comparison](#in-comparison)
- [Technical Details](#technical-details)
- [Trust in Servers](#trust-in-servers)
- [Client -> Server Communication](#client---server-communication)
- [Trust in Routers](#trust-in-routers)
- [Client -> Router Communication](#client---router-communication)
- [2-hop Onion Message Routing](#2-hop-onion-message-routing)
- [SimpleX Messaging Protocol](#simplex-messaging-protocol)
- [SimpleX Agents](#simplex-agents)
- [Encryption Primitives Used](#encryption-primitives-used)
- [Threat model](#threat-model)
- [Security](#security)
- [Acknowledgements](#acknowledgements)
@@ -27,27 +28,27 @@ Evgeny Poberezkin
SimpleX as a whole is a platform upon which applications can be built. [SimpleX Chat](https://github.com/simplex-chat/simplex-chat) is one such application that also serves as an example and reference application.
- [SimpleX Messaging Protocol](./simplex-messaging.md) (SMP) is a protocol to send messages in one direction to a recipient, relying on a server in-between. The messages are delivered via uni-directional queues created by recipients.
- SMP protocol allows to send message via a SMP server playing proxy role using 2-hop onion routing (referred to as "private routing" in messaging clients) to protect transport information of the sender (IP address and session) from the server chosen (and possibly controlled) by the recipient.
- [SimpleX Messaging Protocol](./simplex-messaging.md) (SMP) is a protocol to send messages in one direction to a recipient, relying on a router in-between. The messages are delivered via uni-directional queues created by recipients.
- SMP protocol allows to send message via a SMP router playing proxy role using 2-hop onion routing (referred to as "private routing" in messaging clients) to protect transport information of the sender (IP address and session) from the router chosen (and possibly controlled) by the recipient.
- SMP runs over a transport protocol (shown below as TLS) that provides integrity, server authentication, confidentiality, and transport channel binding.
- A SimpleX Server is one of those servers.
- A SimpleX router is one of those routers.
- The SimpleX Network is the term used for the collective of SimpleX Servers that facilitate SMP.
- The SimpleX Network is the term used for the collective of SimpleX routers that facilitate SMP.
- SimpleX Client libraries speak SMP to SimpleX Servers and provide a low-level API not generally intended to be used by applications.
- SimpleX Client libraries speak SMP to SimpleX routers and provide a low-level API not generally intended to be used by applications.
- SimpleX Agents interface with SimpleX Clients to provide a more high-level API intended to be used by applications. Typically they are embedded as libraries, but can also be abstracted into local services.
- SimpleX Agents communicate with other agents inside e2e encrypted envelopes provided by SMP protocol - the syntax and semantics of the messages exchanged by the agent are defined by [SMP agent protocol](./agent-protocol.md)
*Diagram showing the SimpleX Chat app, with logical layers of the chat application interfacing with a SimpleX Agent library, which in turn interfaces with a SimpleX Client library. The Client library in turn speaks the Messaging Protocol to a SimpleX Server.*
*Diagram showing the SimpleX Chat app, with logical layers of the chat application interfacing with a SimpleX Agent library, which in turn interfaces with a SimpleX Client library. The Client library in turn speaks the Messaging Protocol to a SimpleX router.*
```
User's Computer Internet Third-Party Server
User's Computer Internet Third-Party Router
------------------ | ---------------------- | -------------------------
| |
SimpleX Chat | |
@@ -57,11 +58,43 @@ SimpleX as a whole is a platform upon which applications can be built. [SimpleX
+----------------+ | |
| SimpleX Agent | | |
+----------------+ -------------- TLS ---------------- +----------------+
| SimpleX Client | ------ SimpleX Messaging Protocol ------> | SimpleX Server |
| SimpleX Client | ------ SimpleX Messaging Protocol ------> | SimpleX router |
+----------------+ ----------------------------------- +----------------+
| |
```
#### Network model
SimpleX is a general-purpose packet routing network built on top of the Internet. Network endpoints — end-user devices, automated services, AI-enabled applications, IoT devices — exchange data packets through SimpleX network nodes (SMP routers), which accept, buffer, and deliver packets. Each router operates independently and can be operated by any party on standard computing hardware.
SimpleX routers use resource-based addressing: each address identifies a resource on a router, similar to how the World Wide Web addresses resources via URLs. Internet routers, by comparison, use endpoint-based addressing, where IP addresses identify destination devices. Because of this design, SimpleX network participants do not need globally unique addresses to communicate.
SimpleX network has two resource-based addressing schemes:
- *Messaging queues* ([SMP](./simplex-messaging.md)). A queue is a unidirectional, ordered sequence of fixed-size data packets (16,384 bytes each). Each queue has a resource address on a specific router, gated by cryptographic credentials that separately authorize sending and receiving.
- *Data packets* ([XFTP](./xftp.md)). A data packet is an individually addressed block in one of the standard sizes. Each packet has a unique resource address on a specific router, gated by cryptographic credentials. Data packet addressing is more efficient for delivery of larger payloads than queues.
Packet delivery follows a two-router path. The sending endpoint submits a packet to a first router, which forwards it to a second router, where the receiving endpoint retrieves it. The sending endpoint's IP address is known only to the first router; the receiving endpoint's IP address is known only to the second router. See [2-hop Onion Message Routing](#2-hop-onion-message-routing) for details.
Routers buffer packets between submission and retrieval — from seconds to days, enabling asynchronous delivery when endpoints are online at different times. Packets are removed after delivery or after a configured expiration period.
#### Applications
Applications currently using SimpleX network:
- **SimpleX Chat** — a peer-to-peer messenger using SimpleX network as a transport layer, in the same way that communication applications use WebRTC, Tor, i2p, or Nym. All communication logic — contacts, conversations, groups, message formats, end-to-end encryption — runs on endpoint devices.
- **IoT devices** — using the SimpleX queue protocol directly for sensor data collection and device control.
- **AI-based services** — automated services built on the SimpleX Chat application core.
- **Secure monitoring and control systems** — applications for equipment monitoring and control, including robotics, using the network for command delivery and telemetry collection.
[SimpleGo](https://simplego.dev), developed by an independent organization, is a microcontroller-based device running a SimpleX Chat-compatible messenger directly on a microcontroller without a general-purpose operating system. Running over 20 days on a single battery charge, it demonstrates the energy efficiency of resource-based addressing: the device receives packets without continuous polling. A microcontroller-based router implementation that functions simultaneously as a WiFi router is also in development.
#### SimpleX objectives
1. Provide messaging infrastructure for distributed applications. This infrastructure needs to have the following qualities:
@@ -70,7 +103,7 @@ SimpleX as a whole is a platform upon which applications can be built. [SimpleX
- Privacy: protect against traffic correlation attacks to determine the contacts that the users communicate with.
- Reliability: the messages should be delivered even if some participating network servers or receiving clients fail, with “at least once” delivery guarantee.
- Reliability: the messages should be delivered even if some participating network routers or receiving clients fail, with "at least once" delivery guarantee.
- Integrity: the messages sent in one direction are ordered in a way that sender and recipient agree on; the recipient can detect when a message was removed or changed.
@@ -78,63 +111,63 @@ SimpleX as a whole is a platform upon which applications can be built. [SimpleX
- Low latency: the delay introduced by the network should not be higher than 100ms-1s in addition to the underlying TCP network latency.
2. Provide better communication security and privacy than the alternative instant messaging solutions. In particular SimpleX provides better privacy of metadata (who talks to whom and when) and better security against active network attackers and malicious servers.
2. Provide better communication security and privacy than the alternative instant messaging solutions. In particular SimpleX provides better privacy of metadata (who talks to whom and when) and better security against active network attackers and malicious routers.
3. Balance user experience with privacy requirements, prioritizing experience of mobile device users.
#### In Comparison
SimpleX network has a design similar to P2P networks, but unlike most P2P networks it consists of clients and servers without depending on any centralized component.
SimpleX network has a design similar to P2P networks, but unlike most P2P networks it consists of clients and routers without depending on any centralized component.
In comparison to more traditional messaging applications (e.g. WhatsApp, Signal, Telegram) the key differences of SimpleX network are:
- participants do not need to have globally unique addresses to communicate, instead they use redundant unidirectional (simplex) messaging queues, with a separate set of queues for each contact.
- connection requests are passed out-of-band, non-optionally protecting key exchange against man-in-the-middle attack.
- simple message queues provided by network servers are used by the clients to create more complex communication scenarios, such as duplex one-to-one communication, transmitting files, group communication without central servers, and content/communication channels.
- simple message queues provided by network routers are used by the clients to create more complex communication scenarios, such as duplex one-to-one communication, transmitting files, group communication without central routers, and content/communication channels.
- servers do not store any user information (no user profiles or contacts, or messages once they are delivered), and primarily use in-memory persistence.
- routers do not store any user information (no user profiles or contacts, or messages once they are delivered), and primarily use in-memory persistence.
- users can change servers with minimal disruption - even after an in-use server disappears, simply by changing the configuration on which servers the new queues are created.
- users can change routers with minimal disruption - even after an in-use router disappears, simply by changing the configuration on which routers the new queues are created.
## Technical Details
#### Trust in Servers
#### Trust in Routers
Clients communicate directly with servers (but not with other clients) using SimpleX Messaging Protocol (SMP) running over some transport protocol that provides integrity, server authentication, confidentiality, and transport channel binding. By default, we assume this transport protocol is TLS.
Clients communicate directly with routers (but not with other clients) using SimpleX Messaging Protocol (SMP) running over some transport protocol that provides integrity, server authentication, confidentiality, and transport channel binding. By default, we assume this transport protocol is TLS.
Users use multiple servers, and choose where to receive their messages. Accordingly, they send messages to their communication partners' chosen servers either directly, if this is a known/trusted server, or via another SMP server providing proxy functionality to protect IP address and session of the sender.
Users use multiple routers, and choose where to receive their messages. Accordingly, they send messages to their communication partners' chosen routers either directly, if this is a known/trusted router, or via another SMP router providing proxy functionality to protect IP address and session of the sender.
Although end-to-end encryption is always present, users place a degree of trust in servers they connect to. This trust decision is very similar to a user's choice of email provider; however the trust placed in a SimpleX server is significantly less. Notably, there is no re-used identifier or credential between queues on the same (or different) servers. While a user *may* re-use a transport connection to fetch messages from multiple queues, or connect to a server from the same IP address, both are choices a user may opt into to break the promise of un-correlatable queues.
Although end-to-end encryption is always present, users place a degree of trust in routers they connect to. This trust decision is very similar to a user's choice of email provider; however the trust placed in a SimpleX router is significantly less. Notably, there is no re-used identifier or credential between queues on the same (or different) routers. While a user *may* re-use a transport connection to fetch messages from multiple queues, or connect to a router from the same IP address, both are choices a user may opt into to break the promise of un-correlatable queues.
Users may trust a server because:
Users may trust a router because:
- They deploy and control the servers themselves from the available open-source code. This has the trade-offs of strong trust in the server but limited metadata obfuscation to a passive network observer. Techniques such as noise traffic, traffic mixing (incurring latency), and using an onion routing transport protocol can mitigate that.
- They deploy and control the routers themselves from the available open-source code. This has the trade-offs of strong trust in the router but limited metadata obfuscation to a passive network observer. Techniques such as noise traffic, traffic mixing (incurring latency), and using an onion routing transport protocol can mitigate that.
- They use servers from a trusted commercial provider. The more clients the provider has, the less metadata about the communication times is leaked to the network observers.
- They use routers from a trusted commercial provider. The more clients the provider has, the less metadata about the communication times is leaked to the network observers.
By default, servers do not retain access logs, and permanently delete messages and queues when requested. Messages persist only in memory until they cross a threshold of time, typically on the order of days.[0] There is still a risk that a server maliciously records all queues and messages (even though encrypted) sent via the same transport connection to gain a partial knowledge of the user’s communications graph and other meta-data.
By default, routers do not retain access logs, and permanently delete messages and queues when requested. Messages persist in memory or in a database until they cross a threshold of time, typically on the order of days.[0] There is still a risk that a router maliciously records all queues and messages (even though encrypted) sent via the same transport connection to gain a partial knowledge of the user's communications graph and other meta-data.
SimpleX supports measures (managed transparently to the user at the agent level) to mitigate the trust placed in servers. These include rotating the queues in use between users, noise traffic, supporting overlay networks such as Tor, and isolating traffic to different queues to different transport connections (and Tor circuits, if Tor is used).
SimpleX supports measures (managed transparently to the user at the agent level) to mitigate the trust placed in routers. These include rotating the queues in use between users, noise traffic, supporting overlay networks such as Tor, and isolating traffic to different queues to different transport connections (and Tor circuits, if Tor is used).
[0] While configurable by servers, a minimum value is enforced by the default software. SimpleX Agents can provide redundant routing over queues to mitigate against message loss.
[0] While configurable by routers, a minimum value is enforced by the default software. SimpleX Agents can provide redundant routing over queues to mitigate against message loss.
#### Client -> Server Communication
#### Client -> Router Communication
Utilizing TLS grants the SimpleX Messaging Protocol (SMP) server authentication and metadata protection to a passive network observer. But SMP does not rely on the transport protocol for message confidentiality or client authentication. The SMP protocol itself provides end-to-end confidentiality, authentication, and integrity of messages between communicating parties.
Servers have long-lived, self-signed, offline certificates whose hash is pre-shared with clients over secure channels - either provided with the client library or provided in the secure introduction between clients, as part of the server address. The offline certificate signs an online certificate used in the transport protocol handshake. [0]
Routers have long-lived, self-signed, offline certificates whose hash is pre-shared with clients over secure channels - either provided with the client library or provided in the secure introduction between clients, as part of the router address. The offline certificate signs an online certificate used in the transport protocol handshake. [0]
If the transport protocol's confidentiality is broken, incoming and outgoing messages to the server cannot be correlated by message contents. Additionally, because of encryption at the SMP layer, impersonating the server is not sufficient to pass (and therefore correlate) a message from a sender to recipient - the only attack possible is to drop the messages. Only by additionally *compromising* the server can one pass and correlate messages.
If the transport protocol's confidentiality is broken, incoming and outgoing messages to the router cannot be correlated by message contents. Additionally, because of encryption at the SMP layer, impersonating the router is not sufficient to pass (and therefore correlate) a message from a sender to recipient - the only attack possible is to drop the messages. Only by additionally *compromising* the router can one pass and correlate messages.
It's important to note that the SMP protocol does not do server authentication. Instead we rely upon the fact that an attacker who tricks the transport protocol into authenticating the server incorrectly cannot do anything with the SMP messages except drop them.
It's important to note that the SMP protocol does not do server authentication. Instead we rely upon the fact that an attacker who tricks the transport protocol into authenticating the router incorrectly cannot do anything with the SMP messages except drop them.
After the connection is established, the client sends blocks of a fixed size 16KB, and the server replies with the blocks of the same size to reduce metadata observable to a network adversary. The protocol has been designed to make traffic correlation attacks difficult, adapting ideas from Tor, remailers, and more general onion and mix networks. It does not try to replace Tor though - SimpleX servers can be deployed as onion services and SimpleX clients can communicate with servers over Tor to further improve participants privacy.
After the connection is established, the client sends blocks of a fixed size 16KB, and the router replies with the blocks of the same size to reduce metadata observable to a network adversary. The protocol has been designed to make traffic correlation attacks difficult, adapting ideas from Tor, remailers, and more general onion and mix networks. It does not try to replace Tor though - SimpleX routers can be deployed as onion services and SimpleX clients can communicate with routers over Tor to further improve participants privacy.
By using fixed-size blocks, oversized for the expected content, the vast majority of traffic is uniform in nature. When enough traffic is transiting a server simultaneously, the server acts as a low-latency mix node. We can't rely on this behavior to make a security claim, but we have engineered to take advantage of it when we can. As mentioned, this holds true even if the transport connection is compromised.
By using fixed-size blocks, oversized for the expected content, the vast majority of traffic is uniform in nature. When enough traffic is transiting a router simultaneously, the router acts as a low-latency mix node. We can't rely on this behavior to make a security claim, but we have engineered to take advantage of it when we can. As mentioned, this holds true even if the transport connection is compromised.
The protocol does not protect against attacks targeted at particular users with known identities - e.g., if the attacker wants to prove that two known users are communicating, they can achieve it by observing their local traffic. At the same time, it substantially complicates large-scale traffic correlation, making determining the real user identities much less effective.
@@ -143,39 +176,39 @@ The protocol does not protect against attacks targeted at particular users with
#### 2-hop Onion Message Routing
As SimpleX Messaging Protocol servers providing messaging queues are chosen by the recipients, in case senders connect to these servers directly the server owners (who potentially can be the recipients themselves) can learn senders' IP addresses (if Tor is not used) and which other queues on the same server are accessed by the user in the same transport connection (even if Tor is used).
As SimpleX Messaging Protocol routers providing messaging queues are chosen by the recipients, in case senders connect to these routers directly the router owners (who potentially can be the recipients themselves) can learn senders' IP addresses (if Tor is not used) and which other queues on the same router are accessed by the user in the same transport connection (even if Tor is used).
While the clients support isolating the messages sent to different queues into different transport connections (and Tor circuits), this is not practical, as it consumes additional traffic and system resources.
To mitigate this problem SimpleX Messaging Protocol servers support 2-hop onion message routing when the SMP server chosen by the sender forwards the messages to the servers chosen by the recipients, thus protecting both the senders IP addresses and sessions, even if connection isolation and Tor are not used.
To mitigate this problem SimpleX Messaging Protocol routers support 2-hop onion message routing when the SMP router chosen by the sender forwards the messages to the routers chosen by the recipients, thus protecting both the senders IP addresses and sessions, even if connection isolation and Tor are not used.
The design of 2-hop onion message routing prevents these potential attacks:
- MITM by proxy (SMP server that forwards the messages).
- MITM by proxy (SMP router that forwards the messages).
- Identification by the proxy which and how many queues the sender sends messages to (as messages are additionally e2e encrypted between the sender and the destination SMP server).
- Identification by the proxy which and how many queues the sender sends messages to (as messages are additionally e2e encrypted between the sender and the destination SMP router).
- Correlation of messages sent to different queues via the same user session (as random correlation IDs and keys are used for each message).
See more details about 2-hop onion message routing design in [SimpleX Messaging Protocol](./simplex-messaging.md#proxying-sender-commands)
Also see [Threat model](#threat-model)
Also see [Security](./security.md)
#### SimpleX Messaging Protocol
SMP is initialized with an in-person or out-of-band introduction message, where Alice provides Bob with details of a server (including IP address or host name, port, and hash of the long-lived offline certificate), a queue ID, and Alice's public keys to agree e2e encryption. These introductions are similar to the PANDA key-exchange, in that if observed, the adversary can race to establish the communication channel instead of the intended participant. [0]
SMP is initialized with an in-person or out-of-band introduction message, where Alice provides Bob with details of a router (including IP address or host name, port, and hash of the long-lived offline certificate), a queue ID, and Alice's public keys to agree e2e encryption. These introductions are similar to the PANDA key-exchange, in that if observed, the adversary can race to establish the communication channel instead of the intended participant. [0]
Because queues are uni-directional, Bob provides an identically-formatted introduction message to Alice over Alice's now-established receiving queue.
When setting up a queue, the server will create separate sender and recipient queue IDs (provided to Alice during set-up and Bob during initial connection). Additionally, during set-up Alice will perform a DH exchange with the server to agree upon a shared secret. This secret will be used to re-encrypt Bob's incoming message before Alice receives it, creating the anti-correlation property earlier-described should the transport encryption be compromised.
When setting up a queue, the router will create separate sender and recipient queue IDs (provided to Alice during set-up and Bob during initial connection). Additionally, during set-up Alice will perform a DH exchange with the router to agree upon a shared secret. This secret will be used to re-encrypt Bob's incoming message before Alice receives it, creating the anti-correlation property earlier-described should the transport encryption be compromised.
[0] Users can additionally create public 'contact queues' that are only used to receive connection requests.
[0] Users can additionally create public 'contact queues' that are only used to receive connection requests.
#### SimpleX Agents
SimpleX agents provide higher-level operations compared to SimpleX Clients, who are primarily concerned with creating queues and communicating with servers using SMP. Agent operations include:
SimpleX agents provide higher-level operations compared to SimpleX Clients, who are primarily concerned with creating queues and communicating with routers using SMP. Agent operations include:
- Managing sets of bi-directional, redundant queues for communication partners
@@ -186,195 +219,21 @@ SimpleX agents provide higher-level operations compared to SimpleX Clients, who
- Noise traffic
#### Encryption Primitives Used
## Security
- Ed25519 or Curve25519 to authorize/verify commands to SMP servers (authorization algorithm is set via client/server configuration).
- Curve25519 for DH exchange to agree:
- the shared secret between server and recipient (to encrypt message bodies - it avoids shared cipher-text in sender and recipient traffic)
- the shared secret between sender and recipient (to encrypt messages end-to-end in each queue - it avoids shared cipher-text in redundant queues).
- [NaCl crypto_box](https://nacl.cr.yp.to/box.html) encryption scheme (curve25519xsalsa20poly1305) for message body encryption between server and recipient and for E2E per-queue encryption.
- SHA256 to validate server offline certificates.
- [double ratchet](https://signal.org/docs/specifications/doubleratchet/) protocol for end-to-end message encryption between the agents:
- Curve448 keys to agree shared secrets required for double ratchet initialization (using [X3DH](https://signal.org/docs/specifications/x3dh/) key agreement with 2 ephemeral keys for each side),
- AES-GCM AEAD cipher,
- SHA512-based HKDF for key derivation.
For encryption primitives, threat model, and detailed security analysis, see [Security](./security.md).
SimpleX provides these security properties:
## Threat Model
- **End-to-end encryption** with forward secrecy via double ratchet protocol, with optional post-quantum protection.
#### Global Assumptions
- **No shared identifiers** across connections — contacts cannot prove they communicate with the same user.
- A user protects their local database and key material.
- The user's application is authentic, and no local malware is running.
- The cryptographic primitives in use are not broken.
- A user's choice of servers is not directly tied to their identity or otherwise represents distinguishing information about the user.
- The user's client uses 2-hop onion message routing.
- **Sender deniability** — neither routers nor recipients can cryptographically prove message origin.
#### A passive adversary able to monitor the traffic of one user
- **Transport metadata protection** — fixed-size blocks, 2-hop onion routing, and connection isolation frustrate traffic correlation.
*can:*
- identify that and when a user is using SimpleX.
- determine which servers the user receives the messages from.
- observe how much traffic is being sent, and make guesses as to its purpose.
*cannot:*
- see who sends messages to the user and who the user sends the messages to.
- determine the servers used by users' contacts.
#### A passive adversary able to monitor a set of senders and recipients
*can:*
- identify who and when is using SimpleX.
- learn which SimpleX Messaging Protocol servers are used as receive queues for which users.
- learn when messages are sent and received.
- perform traffic correlation attacks against senders and recipients and correlate senders and recipients within the monitored set, frustrated by the number of users on the servers.
- observe how much traffic is being sent, and make guesses as to its purpose
*cannot, even in case of a compromised transport protocol:*
- perform traffic correlation attacks with any increase in efficiency over a non-compromised transport protocol
#### SimpleX Messaging Protocol server
*can:*
- learn when a queue recipient is online
- know how many messages are sent via the queue (although some may be noise or not content messages).
- learn which messages would trigger notifications even if a user does not use [push notifications](./push-notifications.md).
- perform the correlation of the queue used to receive messages (matching multiple queues to a single user) via either a re-used transport connection, user's IP Address, or connection timing regularities.
- learn a recipient's IP address, track them through other IP addresses they use to access the same queue, and infer information (e.g. employer) based on the IP addresses, as long as Tor is not used.
- drop all future messages inserted into a queue, detectable only over other, redundant queues.
- lie about the state of a queue to the recipient and/or to the sender (e.g. suspended or deleted when it is not).
- spam a user with invalid messages.
*cannot:*
- undetectably add, duplicate, or corrupt individual messages.
- undetectably drop individual messages, so long as a subsequent message is delivered.
- learn the contents or type of messages.
- distinguish noise messages from regular messages except via timing regularities.
- compromise the users' end-to-end encryption with an active attack.
- learn a sender's IP address, track them through other IP addresses they use to access the same queue, and infer information (e.g. employer) based on the IP addresses, even if Tor is not used (provided messages are sent via proxy SMP server).
- perform senders' queue correlation (matching multiple queues to a single sender) via either a re-used transport connection, user's IP Address, or connection timing regularities, unless it has additional information from the proxy SMP server (provided messages are sent via proxy SMP server).
#### SimpleX Messaging Protocol server that proxies the messages to another SMP server
*can:*
- learn a sender's IP address, as long as Tor is not used.
- learn when a sender with a given IP address is online.
- know how many messages are sent from a given IP address and to a given destination SMP server.
- drop all messages from a given IP address or to a given destination server.
- unless destination SMP server detects repeated public DH keys of senders, replay messages to a destination server within a single session, causing either duplicate message delivery (which will be detected and ignored by the receiving clients), or, when receiving client is not connected to SMP server, exhausting capacity of destination queues used within the session.
*cannot:*
- perform queue correlation (matching multiple queues to a single user), unless it has additional information from the destination SMP server.
- undetectably add, duplicate, or corrupt individual messages.
- undetectably drop individual messages, so long as a subsequent message is delivered.
- learn the contents or type of messages.
- learn which messages would trigger notifications.
- learn the destination queues of messages.
- distinguish noise messages from regular messages except via timing regularities.
- compromise the user's end-to-end encryption with another user via an active attack.
- compromise the user's end-to-end encryption with the destination SMP servers via an active attack.
#### An attacker who obtained Alice's (decrypted) chat database
*can:*
- see the history of all messages exchanged by Alice with her communication partners.
- see shared profiles of contacts and groups.
- surreptitiously receive new messages sent to Alice via existing queues; until communication queues are rotated or the Double-Ratchet advances forward.
- prevent Alice from receiving all new messages sent to her - either surreptitiously by emptying the queues regularly or overtly by deleting them.
- send messages from the user to their contacts; recipients will detect it as soon as the user sends the next message, because the previous message hash won’t match (and potentially won’t be able to decrypt them in case they don’t keep the previous ratchet keys).
*cannot:*
- impersonate a sender and send messages to the user whose database was stolen. Doing so requires also compromising the server (to place the message in the queue, that is possible until the Double-Ratchet advances forward) or the user's device at a subsequent time (to place the message in the database).
- undetectably communicate at the same time as Alice with her contacts. Doing so would result in the contact getting different messages with repeated IDs.
- undetectably monitor message queues in realtime without alerting the user they are doing so, as a second subscription request unsubscribes the first and notifies the second.
#### A user’s contact
*can:*
- spam the user with messages.
- forever retain messages from the user.
*cannot:*
- cryptographically prove to a third-party that a message came from a user (assuming the user’s device is not seized).
- prove that two contacts they have is the same user.
- cannot collaborate with another of the user's contacts to confirm they are communicating with the same user.
#### An attacker who observes Alice showing an introduction message to Bob
*can:*
- Impersonate Bob to Alice.
*cannot:*
- Impersonate Alice to Bob.
#### An attacker with Internet access
*can:*
- Denial of Service SimpleX messaging servers.
- spam a user's public “contact queue” with connection requests.
*cannot:*
- send messages to a user who they are not connected with.
- enumerate queues on a SimpleX server.
- **Out-of-band key exchange** — connection requests passed outside the network protect against MITM attacks.
## Acknowledgements
protocol/push-notifications.md
+45 -45
View File
@@ -1,8 +1,8 @@
Version 3, 2025-01-24
# Overview of push notifications for SimpleX Messaging Servers
# Overview of push notifications for SimpleX Messaging Routers
This document describes Notification Server protocol version 3. Version history:
This document describes Notification Router protocol version 3. Version history:
- v1: initial version
- v2: authenticated commands, command batching
- v3: detailed invalid token reason
@@ -10,10 +10,10 @@ This document describes Notification Server protocol version 3. Version history:
## Table of contents
- [Introduction](#introduction)
- [Participating servers](#participating-servers)
- [Participating routers](#participating-routers)
- [Register device token to receive push notifications](#register-device-token-to-receive-push-notifications)
- [Subscribe to connection notifications](#subscribe-to-connection-notifications)
- [SimpleX Notification Server protocol](#simplex-notification-server-protocol)
- [SimpleX Notification Router protocol](#simplex-notification-router-protocol)
- [Register new notification token](#register-new-notification-token)
- [Verify notification token](#verify-notification-token)
- [Check notification token status](#check-notification-token-status)
@@ -28,35 +28,35 @@ This document describes Notification Server protocol version 3. Version history:
## Introduction
SimpleX Messaging servers already operate as push servers and deliver the messages to subscribed clients as soon as they are sent to the servers.
SimpleX Messaging routers already operate as push routers and deliver the messages to subscribed clients as soon as they are sent to the routers.
The reason for push notifications is to support instant message notifications on iOS that does not allow background services.
## Participating servers
## Participating routers
The diagram below shows which servers participate in message notification delivery.
The diagram below shows which routers participate in message notification delivery.
While push provider (e.g., APN) can learn how many notifications are delivered to the user, it cannot access message content, even encrypted, or any message metadata - the notifications are e2e encrypted between SimpleX Notification Server and the user's device.
While push provider (e.g., APN) can learn how many notifications are delivered to the user, it cannot access message content, even encrypted, or any message metadata - the notifications are e2e encrypted between SimpleX Notification Router and the user's device.
```
User's iOS device Internet Servers
User's iOS device Internet Routers
--------------------- . ------------------------ . -----------------------------
. .
. . can be self-hosted now
+--------------+ . . +----------------+
| SimpleX Chat | -------------- TLS --------------- | SimpleX |
| client |------> SimpleX Messaging Protocol (SMP) ------> | Messaging |
+--------------+ ---------------------------------- | Server |
+--------------+ ---------------------------------- | Router |
^ | . . +----------------+
| | . . . . . | . . .
| | . . | V |
| | . . |SMP| TLS
| | . . | | | SimpleX
| | . . . . . V . . . NTF Server
| | . . . . . V . . . NTF Router
| | . . +----------------------------------+
| | . . | +---------------+ |
| | -------------- TLS --------------- | | SimpleX | can be |
| |-----------> Notification Server Protocol -----> | | Notifications | self-hosted |
| |-----------> Notification Router Protocol -----> | | Notifications | self-hosted |
| ---------------------------------- | | Subscriber | in the future |
| . . | +---------------+ |
| . . | | |
@@ -64,7 +64,7 @@ While push provider (e.g., APN) can learn how many notifications are delivered t
| . . | +---------------+ |
| . . | | SimpleX | |
| . . | | Push | |
| . . | | Server | |
| . . | | Router | |
| . . | +---------------+ |
| . . +----------------------------------+
| . . . . . | . . .
@@ -90,27 +90,27 @@ This diagram shows the process of subscription to notifications, notification de
![Subscribe to notifications](./diagrams/notifications/subscription.svg)
## SimpleX Notification Server protocol
## SimpleX Notification Router protocol
To manage notification subscriptions to SMP servers, SimpleX Notification Server provides an RPC protocol with a similar design to SimpleX Messaging Protocol server.
To manage notification subscriptions to SMP routers, SimpleX Notification Router provides an RPC protocol with a similar design to SimpleX Messaging Protocol router.
This protocol sends requests and responses in a fixed size blocks of 512 bytes over TLS, uses the same [syntax of protocol transmissions](./simplex-messaging.md#smp-transmission-and-transport-block-structure) as SMP protocol, and has the same transport [handshake syntax](./simplex-messaging.md#transport-handshake) (except the server certificate is not included in the handshake).
This protocol sends requests and responses in a fixed size blocks of 512 bytes over TLS, uses the same [syntax of protocol transmissions](./simplex-messaging.md#smp-transmission-and-transport-block-structure) as SMP protocol, and has the same transport [handshake syntax](./simplex-messaging.md#transport-handshake) (except the router certificate is not included in the handshake).
The client and server use ALPN extension with `ntf/1` protocol name to agree handshake version.
The client and router use ALPN extension with `ntf/1` protocol name to agree handshake version.
Protocol commands have this syntax:
```
ntfServerTransmission =
ntfServerCmd = newTokenCmd / verifyTokenCmd / checkTokenCmd /
ntfRouterTransmission =
ntfRouterCmd = newTokenCmd / verifyTokenCmd / checkTokenCmd /
replaceTokenCmd / deleteTokenCmd / cronCmd /
newSubCmd / checkSubCmd / deleteSubCmd / pingCmd
```
### Register new notification token
This command should be used after the client app obtains a token from push notifications provider to register the token with the server.
This command should be used after the client app obtains a token from push notifications provider to register the token with the router.
Having received this command the server will deliver a test notification via the push provider to validate that the client has this token.
Having received this command the router will deliver a test notification via the push provider to validate that the client has this token.
The command syntax:
@@ -121,20 +121,20 @@ deviceToken = pushProvider tokenString
pushProvider = apnsDev / apnsProd / apnsNull
apnsDev = "AD" ; APNS token for development environment
apnsProd = "AP" ; APNS token for production environment
apnsNull = "AN" ; token that does not trigger any notification delivery - used for server testing
apnsNull = "AN" ; token that does not trigger any notification delivery - used for router testing
tokenString = shortString
authPubKey = length x509encoded ; Ed25519 key used to verify clients commands
clientDhPubKey = length x509encoded ; X25519 key to agree e2e encryption between the server and client
clientDhPubKey = length x509encoded ; X25519 key to agree e2e encryption between the router and client
shortString = length *OCTET
length = 1*1 OCTET
```
The server response syntax:
The router response syntax:
```abnf
tokenIdResp = %s"IDTKN" SP entityId serverDhPubKey
tokenIdResp = %s"IDTKN" SP entityId routerDhPubKey
entityId = shortString
serverDhPubKey = length x509encoded ; X25519 key to agree e2e encryption between the server and client
routerDhPubKey = length x509encoded ; X25519 key to agree e2e encryption between the router and client
```
### Verify notification token
@@ -209,8 +209,8 @@ After this command all message notification subscriptions will be removed and no
This command enables or disables periodic notifications sent to the client device irrespective of message notifications.
This is useful for two reasons:
- it provides better privacy from notification server, as while the server learns the device token, it doesn't learn anything else about user communications.
- it allows to receive messages when notifications were dropped by push provider, e.g. while the device was offline, or lost by notification server, e.g. while it was restarting.
- it provides better privacy from notification router, as while the router learns the device token, it doesn't learn anything else about user communications.
- it allows to receive messages when notifications were dropped by push provider, e.g. while the device was offline, or lost by notification router, e.g. while it was restarting.
The command syntax:
@@ -223,18 +223,18 @@ The interval for periodic notifications is set in minutes, with the minimum of 2
### Create SMP message notification subscription
This command makes notification server subscribe to message notifications from SMP server and to deliver them to push provider:
This command makes notification router subscribe to message notifications from SMP router and to deliver them to push provider:
```abnf
newSubCmd = %s"SNEW" newSub
newSub = %s "S" tokenId smpServer notifierId notifierKey
newSub = %s "S" tokenId smpRouter notifierId notifierKey
tokenId = shortString ; returned in response to `TNEW` command
smpServer = smpServer = hosts port fingerprint
smpRouter = smpRouter = hosts port fingerprint
hosts = length 1*host
host = shortString
port = shortString
fingerprint = shortString
notifierId = shortString ; returned by SMP server in response to `NKEY` SMP command
notifierId = shortString ; returned by SMP router in response to `NKEY` SMP command
notifierKey = length x509encoded ; private key used to authorize requests to subscribe to message notifications
```
@@ -256,8 +256,8 @@ The response:
```abnf
subStatusResp = %s"SUB" SP subStatus
subStatus = %s"NEW" / %s"PENDING" / ; e.g., after SMP server disconnect/timeout while ntf server is retrying to connect
%s"ACTIVE" / %s"INACTIVE" / %s"END" / ; if another server subscribed to notifications
subStatus = %s"NEW" / %s"PENDING" / ; e.g., after SMP router disconnect/timeout while ntf router is retrying to connect
%s"ACTIVE" / %s"INACTIVE" / %s"END" / ; if another router subscribed to notifications
%s"AUTH" / %s"DELETED" / %s"SERVICE" / subErrStatus
subErrStatus = %s"ERR" SP shortString
```
@@ -297,7 +297,7 @@ Where `errorType` has the same syntax as in [SimpleX Messaging Protocol](./simpl
## Threat Model
This threat model compliments SimpleX Messaging Protocol [threat model](./overview-tjr.md#threat-model)
This threat model compliments SimpleX Messaging Protocol [threat model](./security.md#threat-model)
#### A passive adversary able to monitor the traffic of one user
@@ -307,21 +307,21 @@ This threat model compliments SimpleX Messaging Protocol [threat model](./overvi
*cannot:*
- determine which servers a user subscribed to the notifications from.
- determine which routers a user subscribed to the notifications from.
#### A passive adversary able to monitor a set of senders and recipients
*can:*
- perform more efficient traffic correlation attacks against senders and recipients and correlate senders and recipients within the monitored set, frustrated by the number of users on the servers.
- perform more efficient traffic correlation attacks against senders and recipients and correlate senders and recipients within the monitored set, frustrated by the number of users on the routers.
#### SimpleX Messaging Protocol server
#### SimpleX Messaging Protocol router
*can:*
- learn which messages trigger push notifications.
- learn IP address of SimpleX notification servers used by the user.
- learn IP address of SimpleX notification routers used by the user.
- drop message notifications.
@@ -333,13 +333,13 @@ This threat model compliments SimpleX Messaging Protocol [threat model](./overvi
- learn which queues belong to the same users with any additional efficiency compared with not using push notifications.
#### SimpleX Notification Server subscribed to message notifications
#### SimpleX Notification Router subscribed to message notifications
*can:*
- learn a user device token.
- learn how many messaging queues and servers a user receives messages from.
- learn how many messaging queues and routers a user receives messages from.
- learn how many message notifications are delivered to the user from each queue.
@@ -359,7 +359,7 @@ This threat model compliments SimpleX Messaging Protocol [threat model](./overvi
- add, duplicate, or corrupt individual messages that will be shown to the user.
#### SimpleX Notification Server subscribed ONLY to periodic notifications
#### SimpleX Notification Router subscribed ONLY to periodic notifications
*can:*
@@ -371,7 +371,7 @@ This threat model compliments SimpleX Messaging Protocol [threat model](./overvi
*cannot:*
- learn how many messaging queues and servers a user receives messages from.
- learn how many messaging queues and routers a user receives messages from.
- learn how many message notifications are delivered to the user from each queue.
@@ -403,7 +403,7 @@ This threat model compliments SimpleX Messaging Protocol [threat model](./overvi
*cannot:*
- learn which SimpleX Messaging Protocol servers are used by a user (notifications are e2e encrypted).
- learn which SimpleX Messaging Protocol routers are used by a user (notifications are e2e encrypted).
- learn which or how many messaging queues a user receives notifications from.
@@ -415,4 +415,4 @@ This threat model compliments SimpleX Messaging Protocol [threat model](./overvi
- register notification token not present on attacker's device.
- enumerate tokens or subscriptions on a SimpleX Notification Server.
- enumerate tokens or subscriptions on a SimpleX Notification Router.
protocol/security.md
+215
View File
@@ -0,0 +1,215 @@
Revision 1, 2026-03-09
# SimpleX Network: Security
This document describes the cryptographic primitives and threat model for the SimpleX network. For a general introduction, see [SimpleX: messaging and application platform](./overview-tjr.md).
## Table of contents
- [Encryption primitives](#encryption-primitives)
- [Threat model](#threat-model)
- [Global Assumptions](#global-assumptions)
- [A passive adversary able to monitor the traffic of one user](#a-passive-adversary-able-to-monitor-the-traffic-of-one-user)
- [A passive adversary able to monitor a set of senders and recipients](#a-passive-adversary-able-to-monitor-a-set-of-senders-and-recipients)
- [SimpleX Messaging Protocol router](#simplex-messaging-protocol-router)
- [SimpleX Messaging Protocol router that proxies the messages to another SMP router](#simplex-messaging-protocol-router-that-proxies-the-messages-to-another-smp-router)
- [An attacker who obtained Alice's (decrypted) chat database](#an-attacker-who-obtained-alices-decrypted-chat-database)
- [A user's contact](#a-users-contact)
- [An attacker who observes Alice showing an introduction message to Bob](#an-attacker-who-observes-alice-showing-an-introduction-message-to-bob)
- [An attacker with Internet access](#an-attacker-with-internet-access)
## Encryption primitives
- **Router command authorization**: X25519 DH-based authenticated encryption (SMP v7+), providing sender deniability. Ed25519 signatures used for recipient commands and notifier commands.
- **Per-queue key agreement**: Curve25519 DH exchange to agree:
- the shared secret between router and recipient (to encrypt message bodies — avoids shared ciphertext in sender and recipient traffic),
- the shared secret between sender and recipient (to encrypt messages end-to-end in each queue — avoids shared ciphertext in redundant queues).
- **SMP-layer encryption**: [NaCl crypto_box](https://nacl.cr.yp.to/box.html) (curve25519xsalsa20poly1305) for message body encryption between router and recipient, and for e2e per-queue encryption.
- **Certificate validation**: SHA256 to validate router offline certificates.
- **End-to-end encryption**: [Double ratchet](https://signal.org/docs/specifications/doubleratchet/) protocol:
- Curve448 keys for shared secret agreement via [X3DH](https://signal.org/docs/specifications/x3dh/) with 2 ephemeral keys per side,
- optional [SNTRUP761](https://ntruprime.cr.yp.to/) post-quantum KEM running in parallel with the DH ratchet (see [PQDR](./pqdr.md)), providing post-quantum forward secrecy,
- AES-GCM AEAD cipher,
- SHA512-based HKDF for key derivation.
## Threat Model
### Global Assumptions
- A user protects their local database and key material.
- The user's application is authentic, and no local malware is running.
- The cryptographic primitives in use are not broken.
- A user's choice of routers is not directly tied to their identity or otherwise represents distinguishing information about the user.
- The user's client uses 2-hop onion message routing.
### A passive adversary able to monitor the traffic of one user
*can:*
- identify that and when a user is using SimpleX.
- determine which routers the user receives messages from.
- observe how much traffic is being sent, and make guesses as to its purpose.
*cannot:*
- see who sends messages to the user and who the user sends messages to.
- determine the routers used by users' contacts.
### A passive adversary able to monitor a set of senders and recipients
*can:*
- identify who and when is using SimpleX.
- learn which SimpleX Messaging Protocol routers are used as receive queues for which users.
- learn when messages are sent and received.
- perform traffic correlation attacks against senders and recipients and correlate senders and recipients within the monitored set, frustrated by the number of users on the routers.
- observe how much traffic is being sent, and make guesses as to its purpose.
*cannot, even in case of a compromised transport protocol:*
- perform traffic correlation attacks with any increase in efficiency over a non-compromised transport protocol.
### SimpleX Messaging Protocol router
*can:*
- learn when a queue recipient is online.
- know how many messages are sent via the queue (although some may be noise or not content messages).
- learn which messages would trigger notifications even if a user does not use [push notifications](./push-notifications.md).
- perform the correlation of the queue used to receive messages (matching multiple queues to a single user) via either a re-used transport connection, user's IP Address, or connection timing regularities.
- learn a recipient's IP address, track them through other IP addresses they use to access the same queue, and infer information (e.g. employer) based on the IP addresses, as long as Tor is not used.
- drop all future messages inserted into a queue, detectable only over other, redundant queues.
- lie about the state of a queue to the recipient and/or to the sender (e.g. suspended or deleted when it is not).
- spam a user with invalid messages.
*cannot:*
- undetectably add, duplicate, or corrupt individual messages.
- undetectably drop individual messages, so long as a subsequent message is delivered.
- learn the contents or type of messages.
- distinguish noise messages from regular messages except via timing regularities.
- compromise the users' end-to-end encryption with an active attack.
- learn a sender's IP address, track them through other IP addresses they use to access the same queue, and infer information (e.g. employer) based on the IP addresses, even if Tor is not used (provided messages are sent via proxy SMP router).
- perform senders' queue correlation (matching multiple queues to a single sender) via either a re-used transport connection, user's IP Address, or connection timing regularities, unless it has additional information from the proxy SMP router (provided messages are sent via proxy SMP router).
### SimpleX Messaging Protocol router that proxies the messages to another SMP router
*can:*
- learn a sender's IP address, as long as Tor is not used.
- learn when a sender with a given IP address is online.
- know how many messages are sent from a given IP address and to a given destination SMP router.
- drop all messages from a given IP address or to a given destination router.
- unless destination SMP router detects repeated public DH keys of senders, replay messages to a destination router within a single session, causing either duplicate message delivery (which will be detected and ignored by the receiving clients), or, when receiving client is not connected to SMP router, exhausting capacity of destination queues used within the session.
*cannot:*
- perform queue correlation (matching multiple queues to a single user), unless it has additional information from the destination SMP router.
- undetectably add, duplicate, or corrupt individual messages.
- undetectably drop individual messages, so long as a subsequent message is delivered.
- learn the contents or type of messages.
- learn which messages would trigger notifications.
- learn the destination queues of messages.
- distinguish noise messages from regular messages except via timing regularities.
- compromise the user's end-to-end encryption with another user via an active attack.
- compromise the user's end-to-end encryption with the destination SMP routers via an active attack.
### An attacker who obtained Alice's (decrypted) chat database
*can:*
- see the history of all messages exchanged by Alice with her communication partners.
- see shared profiles of contacts and groups.
- surreptitiously receive new messages sent to Alice via existing queues; until communication queues are rotated or the Double-Ratchet advances forward.
- prevent Alice from receiving all new messages sent to her - either surreptitiously by emptying the queues regularly or overtly by deleting them.
- send messages from the user to their contacts; recipients will detect it as soon as the user sends the next message, because the previous message hash won't match (and potentially won't be able to decrypt them in case they don't keep the previous ratchet keys).
*cannot:*
- impersonate a sender and send messages to the user whose database was stolen. Doing so requires also compromising the router (to place the message in the queue, that is possible until the Double-Ratchet advances forward) or the user's device at a subsequent time (to place the message in the database).
- undetectably communicate at the same time as Alice with her contacts. Doing so would result in the contact getting different messages with repeated IDs.
- undetectably monitor message queues in realtime without alerting the user they are doing so, as a second subscription request unsubscribes the first and notifies the second.
### A user's contact
*can:*
- spam the user with messages.
- forever retain messages from the user.
*cannot:*
- cryptographically prove to a third-party that a message came from a user (assuming the user's device is not seized).
- prove that two contacts they have is the same user.
- cannot collaborate with another of the user's contacts to confirm they are communicating with the same user.
### An attacker who observes Alice showing an introduction message to Bob
*can:*
- Impersonate Bob to Alice.
*cannot:*
- Impersonate Alice to Bob.
### An attacker with Internet access
*can:*
- Denial of Service SimpleX messaging routers.
- spam a user's public "contact queue" with connection requests.
*cannot:*
- send messages to a user who they are not connected with.
- enumerate queues on a SimpleX router.
protocol/simplex-messaging.md
+243 -243
View File
File diff suppressed because it is too large Load Diff
protocol/xftp.md
+158 -156
View File
@@ -11,12 +11,12 @@ Version 3, 2025-01-24
- [XFTP procedure](#xftp-procedure)
- [File description](#file-description)
- [URIs syntax](#uris-syntax)
- [XFTP server URI](#xftp-server-uri)
- [XFTP router URI](#xftp-router-uri)
- [File description URI](#file-description-URI)
- [XFTP qualities and features](#xftp-qualities-and-features)
- [Cryptographic algorithms](#cryptographic-algorithms)
- [File chunk IDs](#file-chunk-ids)
- [Server security requirements](#server-security-requirements)
- [Data packet IDs](#data-packet-ids)
- [Router security requirements](#router-security-requirements)
- [Transport protocol](#transport-protocol)
- [TLS ALPN](#tls-alpn)
- [Connection handshake](#connection-handshake)
@@ -26,13 +26,13 @@ Version 3, 2025-01-24
- [Command authentication](#command-authentication)
- [Keep-alive command](#keep-alive-command)
- [File sender commands](#file-sender-commands)
- [Register new file chunk](#register-new-file-chunk)
- [Add file chunk recipients](#add-file-chunk-recipients)
- [Upload file chunk](#upload-file-chunk)
- [Delete file chunk](#delete-file-chunk)
- [Register new data packet](#register-new-data-packet)
- [Add data packet recipients](#add-data-packet-recipients)
- [Upload data packet](#upload-data-packet)
- [Delete data packet](#delete-data-packet)
- [File recipient commands](#file-recipient-commands)
- [Download file chunk](#download-file-chunk)
- [Acknowledge file chunk download](#acknowledge-file-chunk-download)
- [Download data packet](#download-data-packet)
- [Acknowledge data packet download](#acknowledge-data-packet-download)
- [Error responses](#error-responses)
- [Threat model](#threat-model)
@@ -46,7 +46,7 @@ It is designed as a application level protocol to solve the problem of secure an
## Introduction
The objective of SimpleX File Transfer Protocol (XFTP) is to facilitate the secure and private unidirectional transfer of files from senders to recipients via persistent file chunks stored by the xftp server.
The objective of SimpleX File Transfer Protocol (XFTP) is to facilitate the secure and private unidirectional transfer of files from senders to recipients via persistent data packets stored by the xftp router.
XFTP is implemented as an application level protocol on top of HTTP2 and TLS.
@@ -56,19 +56,21 @@ This document describes XFTP protocol version 3. The version history:
- v2: authenticated commands - added basic auth support for commands
- v3: blocked files - added BLOCKED error type for policy violations
The protocol describes the set of commands that senders and recipients can send to XFTP servers to create, upload, download and delete file chunks of several pre-defined sizes. XFTP servers SHOULD support chunks of 4 sizes: 64KB, 256KB, 1MB and 4MB (1KB = 1024 bytes, 1MB = 1024KB).
The protocol describes the set of commands that senders and recipients can send to XFTP routers to create, upload, download and delete data packets of several pre-defined sizes. XFTP routers SHOULD support packets of 4 sizes: 64KB, 256KB, 1MB and 4MB (1KB = 1024 bytes, 1MB = 1024KB).
The protocol is designed with the focus on meta-data privacy and security. While using TLS, the protocol does not rely on TLS security by using additional encryption to achieve that there are no identifiers or ciphertext in common in received and sent server traffic, frustrating traffic correlation even if TLS is compromised.
The protocol is designed with the focus on meta-data privacy and security. While using TLS, the protocol does not rely on TLS security by using additional encryption to achieve that there are no identifiers or ciphertext in common in received and sent router traffic, frustrating traffic correlation even if TLS is compromised.
XFTP does not use any form of participants' identities. It relies on out-of-band passing of "file description" - a human-readable YAML document with the list of file chunk locations, hashes and necessary cryptographic keys.
XFTP does not use any form of participants' identities. It relies on out-of-band passing of "file description" - a human-readable YAML document with the list of data packet locations, hashes and necessary cryptographic keys.
> **Note:** While this protocol was originally designed for file transfer, it handles generic addressed data packets. File-specific semantics (splitting files into packets, assembly, naming) are application-level concerns defined in the [agent protocol](./agent-protocol.md).
## XFTP Model
The XFTP model has three communication participants: the recipient, the file server (XFTP server) that is chosen and, possibly, controlled by the sender, and the sender.
The XFTP model has three communication participants: the recipient, the XFTP router that is chosen and, possibly, controlled by the sender, and the sender.
XFTP server allows uploading fixed size file chunks, with or without basic authentication. The same party that can be the sender of one file chunk can be the recipient of another, without exposing it to the server.
XFTP router allows uploading fixed size data packets, with or without basic authentication. The same party that can be the sender of one data packet can be the recipient of another, without exposing it to the router.
Each file chunk allows multiple recipients, each recipient can download the same chunk multiple times. It allows depending on the threat model use the same recipient credentials for multiple parties, thus reducing server ability to understand the number of intended recipients (but server can still track IP addresses to determine it), or use one unique set of credentials for each recipient, frustrating traffic correlation on the assumption of compromised TLS. In the latter case, senders can create a larger number of recipient credentials to hide the actual number of intended recipients from the servers (which is what SimpleX clients do).
Each data packet allows multiple recipients, each recipient can download the same packet multiple times. It allows depending on the threat model use the same recipient credentials for multiple parties, thus reducing router ability to understand the number of intended recipients (but router can still track IP addresses to determine it), or use one unique set of credentials for each recipient, frustrating traffic correlation on the assumption of compromised TLS. In the latter case, senders can create a larger number of recipient credentials to hide the actual number of intended recipients from the routers (which is what SimpleX clients do).
```
Sender Internet XFTP relays Internet Recipient
@@ -76,7 +78,7 @@ Each file chunk allows multiple recipients, each recipient can download the same
| | | |
| | (can be self-hosted) | |
| | +---------+ | |
chunk 1 ----- HTTP2 over TLS ------ | XFTP | ---- HTTP2 / TLS ----- chunk 1
packet 1 ----- HTTP2 over TLS ------ | XFTP | ---- HTTP2 / TLS ----- packet 1
|---> SimpleX File Transfer Protocol (XFTP) --> | Relay | ---> XFTP ------------->|
| --------------------------- +---------+ ---------------------- |
| | | | | |
@@ -90,21 +92,21 @@ file ---> | XFTP | ------> XFTP ----> | Relay | --->
| | | +---------+ | | |
| ------- HTTP2 / TLS ------- | XFTP | ---- HTTP2 / TLS ---- |
|-------------> XFTP ----> | Relay | ---> XFTP ------------->|
chunk N --------------------------- +---------+ --------------------- chunk N
| | (store file chunks) | |
packet N --------------------------- +---------+ --------------------- packet N
| | (store data packets) | |
| | | |
| | | |
```
When sender client uploads a file chunk, it has to register it first with one sender ID and multiple recipient IDs, and one random unique key per ID to authenticate sender and recipients, and also provide its size and hash that will be validated when chunk is uploaded.
When sender client uploads a data packet, it has to register it first with one sender ID and multiple recipient IDs, and one random unique key per ID to authenticate sender and recipients, and also provide its size and hash that will be validated when packet is uploaded.
To send the actual file, the sender client MUST pad it and encrypt it with a random symmetric key and distribute chunks of fixed sized across multiple XFTP servers. Information about chunk locations, keys, hashes and required keys is passed to the recipients as "[file description](#file-description)" out-of-band.
To send the actual file, the sender client MUST pad it and encrypt it with a random symmetric key and distribute packets of fixed sized across multiple XFTP routers. Information about packet locations, keys, hashes and required keys is passed to the recipients as "[file description](#file-description)" out-of-band.
Creating, uploading, downloading and deleting file chunks requires sending commands to the XFTP server - they are described in detail in [XFTP commands](#xftp-commands) section.
Creating, uploading, downloading and deleting data packets requires sending commands to the XFTP router - they are described in detail in [XFTP commands](#xftp-commands) section.
## Persistence model
Server stores file chunk records in memory, with optional adding to append-only log, to allow restoring them on server restart. File chunk bodies can be stored as files or as objects in any object store (e.g. S3).
Router stores data packet records in memory, with optional adding to append-only log, to allow restoring them on router restart. Data packet bodies can be stored as files or as objects in any object store (e.g. S3).
## XFTP procedure
@@ -114,28 +116,28 @@ To send the file, the sender will:
1) Prepare file
- compute its SHA512 digest.
- prepend header with the name and pad the file to match the whole number of chunks in size. It is RECOMMENDED to use 2 of 4 allowed chunk sizes, to balance upload size and metadata privacy.
- prepend header with the name and pad the file to match the whole number of packets in size. It is RECOMMENDED to use 2 of 4 allowed packet sizes, to balance upload size and metadata privacy.
- encrypt it with a randomly chosen symmetric key and IV (e.g., using NaCL secret_box).
- split into allowed size chunks.
- split into allowed size packets.
- generate per-recipient keys. It is recommended that the sending client generates more per-recipient keys than the actual number of recipients, rounding up to a power of 2, to conceal the actual number of intended recipients.
2) Upload file chunks
- register each chunk record with randomly chosen one or more (for redundancy) XFTP server(s).
2) Upload data packets
- register each packet record with randomly chosen one or more (for redundancy) XFTP router(s).
- optionally request additional recipient IDs, if required number of recipient keys didn't fit into register request.
- upload each chunk to chosen server(s).
- upload each packet to chosen router(s).
3) Prepare file descriptions, one per recipient.
The sending client combines addresses of all chunks and other information into "file description", different for each file recipient, that will include:
The sending client combines addresses of all packets and other information into "file description", different for each file recipient, that will include:
- an encryption key used to encrypt/decrypt the full file (the same for all recipients).
- file SHA512 digest to validate download.
- list of chunk descriptions; information for each chunk:
- private Ed25519 key to sign commands for file transfer server.
- chunk address (server host and chunk ID).
- chunk sha512 digest.
- list of packet descriptions; information for each packet:
- private Ed25519 key to sign commands for file transfer router.
- packet address (router host and packet ID).
- packet sha512 digest.
To reduce the size of file description, chunks are grouped by the server host.
To reduce the size of file description, packets are grouped by the router host.
4) Send file description(s) to the recipient(s) out-of-band, via pre-existing secure and authenticated channel. E.g., SimpleX clients send it as messages via SMP protocol, but it can be done via any other channel.
@@ -145,16 +147,16 @@ To reduce the size of file description, chunks are grouped by the server host.
Having received the description, the recipient will:
1) Download all chunks.
1) Download all packets.
The receiving client can fall back to secondary servers, if necessary:
- if the server is not available.
- if the chunk is not present on the server (ERR AUTH response).
- if the hash of the downloaded file chunk does not match the description.
The receiving client can fall back to secondary routers, if necessary:
- if the router is not available.
- if the packet is not present on the router (ERR AUTH response).
- if the hash of the downloaded data packet does not match the description.
Optionally recipient can acknowledge file chunk reception to delete file ID from server for this recipient.
Optionally recipient can acknowledge data packet reception to delete file ID from router for this recipient.
2) Combine the chunks into a file.
2) Combine the packets into a file.
3) Decrypt the file using the key in file description.
@@ -170,35 +172,35 @@ Optionally recipient can acknowledge file chunk reception to delete file ID from
It includes these fields:
- `party` - "sender" or "recipient". Sender's file description is required to delete the file.
- `size` - padded file size equal to total size of all chunks, see `fileSize` syntax below.
- `size` - padded file size equal to total size of all packets, see `fileSize` syntax below.
- `digest` - SHA512 hash of encrypted file, base64url encoded string.
- `key` - symmetric encryption key to decrypt the file, base64url encoded string.
- `nonce` - nonce to decrypt the file, base64url encoded string.
- `chunkSize` - default chunk size, see `fileSize` syntax below.
- `replicas` - the array of file chunk replicas descriptions.
- `packetSize` - default packet size, see `fileSize` syntax below.
- `replicas` - the array of data packet replicas descriptions.
- `redirect` - optional property for redirect information indicating that the file is itself a description to another file, allowing to use file description as a short URI.
Each replica description is an object with 2 fields:
- `chunks` - and array of chunk replica descriptions stored on one server.
- `server` - [server address](#xftp-server-uri) where the chunks can be downloaded from.
- `packets` - and array of packet replica descriptions stored on one router.
- `router` - [router address](#xftp-router-uri) where the packets can be downloaded from.
Each server replica description is a string with this syntax:
Each router replica description is a string with this syntax:
```abnf
chunkReplica = chunkNo ":" replicaId ":" replicaKey [":" chunkDigest [":" chunkSize]]
chunkNo = 1*DIGIT
; a sequential 1-based chunk number in the original file.
packetReplica = packetNo ":" replicaId ":" replicaKey [":" packetDigest [":" packetSize]]
packetNo = 1*DIGIT
; a sequential 1-based packet number in the original file.
replicaId = base64url
; server-assigned random chunk replica ID.
; router-assigned random packet replica ID.
replicaKey = base64url
; sender-generated random key to receive (or to delete, in case of sender's file description) the chunk replica.
chunkDigest = base64url
; chunk digest that MUST be specified for the first replica of each chunk,
; sender-generated random key to receive (or to delete, in case of sender's file description) the packet replica.
packetDigest = base64url
; packet digest that MUST be specified for the first replica of each packet,
; and SHOULD be omitted (or be the same) on the subsequent replicas
chunkSize = fileSize
packetSize = fileSize
fileSize = sizeInBytes / sizeInUnits
; chunk size SHOULD only be specified on the first replica and only if it is different from default chunk size
; packet size SHOULD only be specified on the first replica and only if it is different from default packet size
sizeInBytes = 1*DIGIT
sizeInUnits = 1*DIGIT sizeUnit
sizeUnit = %s"kb" / %s"mb" / %s"gb"
@@ -211,22 +213,22 @@ Optional redirect information has two fields:
## URIs syntax
### XFTP server URI
### XFTP router URI
The XFTP server address is a URI with the following syntax:
The XFTP router address is a URI with the following syntax:
```abnf
xftpServerURI = %s"xftp://" xftpServer
xftpServer = serverIdentity [":" basicAuth] "@" srvHost [":" port]
xftpRouterURI = %s"xftp://" xftpRouter
xftpRouter = routerIdentity [":" basicAuth] "@" srvHost [":" port]
srvHost = <hostname> ; RFC1123, RFC5891
port = 1*DIGIT
serverIdentity = base64url
routerIdentity = base64url
basicAuth = base64url
```
### File description URI
This file description URI can be generated by the client application to share a small file description as a QR code or as a link. Practically, to be able to scan a QR code it should be under 1000 characters, so only file descriptions with 1-2 chunks can be used in this case. This is supported with `redirect` property when file description leads to a file which in itself is a larger file description to another file - akin to URL shortener.
This file description URI can be generated by the client application to share a small file description as a QR code or as a link. Practically, to be able to scan a QR code it should be under 1000 characters, so only file descriptions with 1-2 packets can be used in this case. This is supported with `redirect` property when file description leads to a file which in itself is a larger file description to another file - akin to URL shortener.
File description URI syntax:
@@ -247,47 +249,47 @@ clientAppServer is not a server the client connects to - it is a server that sho
XFTP stands for SimpleX File Transfer Protocol. Its design is based on the same ideas and has some of the qualities of SimpleX Messaging Protocol:
- recipient cannot see sender's IP address, as the file fragments (chunks) are temporarily stored on multiple XFTP relays.
- recipient cannot see sender's IP address, as the file fragments (packets) are temporarily stored on multiple XFTP relays.
- file can be sent asynchronously, without requiring the sender to be online for file to be received.
- there is no network of peers that can observe this transfer - sender chooses which XFTP relays to use, and can self-host their own.
- XFTP relays do not have any file metadata - they only see individual chunks, with access to each chunk authorized with anonymous credentials (using Edwards curve cryptographic signature) that are random per chunk.
- chunks have one of the sizes allowed by the servers - 64KB, 256KB, 1MB and 4MB chunks, so sending a large file looks indistinguishable from sending many small files to XFTP server. If the same transport connection is reused, server would only know that chunks are sent by the same user.
- each chunk can be downloaded by multiple recipients, but each recipient uses their own key and chunk ID to authorize access, and the chunk is encrypted by a different key agreed via ephemeral DH keys (NaCl crypto_box (SalsaX20Poly1305 authenticated encryption scheme ) with shared secret derived from Curve25519 key exchange) on the way from the server to each recipient. XFTP protocol as a result has the same quality as SMP protocol - there are no identifiers and ciphertext in common between sent and received traffic inside TLS connection, so even if TLS is compromised, it complicates traffic correlation attacks.
- XFTP protocol supports redundancy - each file chunk can be sent via multiple relays, and the recipient can choose the one that is available. Current implementation of XFTP protocol in SimpleX Chat does not support redundancy though.
- XFTP relays do not have any file metadata - they only see individual packets, with access to each packet authorized with anonymous credentials (using Edwards curve cryptographic signature) that are random per packet.
- packets have one of the sizes allowed by the routers - 64KB, 256KB, 1MB and 4MB packets, so sending a large file looks indistinguishable from sending many small files to XFTP router. If the same transport connection is reused, router would only know that packets are sent by the same user.
- each packet can be downloaded by multiple recipients, but each recipient uses their own key and packet ID to authorize access, and the packet is encrypted by a different key agreed via ephemeral DH keys (NaCl crypto_box (SalsaX20Poly1305 authenticated encryption scheme ) with shared secret derived from Curve25519 key exchange) on the way from the router to each recipient. XFTP protocol as a result has the same quality as SMP protocol - there are no identifiers and ciphertext in common between sent and received traffic inside TLS connection, so even if TLS is compromised, it complicates traffic correlation attacks.
- XFTP protocol supports redundancy - each data packet can be sent via multiple relays, and the recipient can choose the one that is available. Current implementation of XFTP protocol in SimpleX Chat does not support redundancy though.
- the file as a whole is encrypted with a random symmetric key using NaCl secret_box.
## Cryptographic algorithms
Clients must cryptographically authorize XFTP commands, see [Command authentication](#command-authentication).
To authorize/verify transmissions clients and servers MUST use either signature algorithm Ed25519 algorithm defined in RFC8709 or using deniable authentication scheme based on NaCL crypto_box (see Simplex Messaging Protocol).
To authorize/verify transmissions clients and routers MUST use either signature algorithm Ed25519 algorithm defined in RFC8709 or using deniable authentication scheme based on NaCL crypto_box (see Simplex Messaging Protocol).
To encrypt/decrypt file chunk bodies delivered to the recipients, servers/clients MUST use NaCL crypto_box.
To encrypt/decrypt data packet bodies delivered to the recipients, routers/clients MUST use NaCL crypto_box.
Clients MUST encrypt file chunk bodies sent via XFTP servers using use NaCL crypto_box.
Clients MUST encrypt data packet bodies sent via XFTP routers using use NaCL crypto_box.
## File chunk IDs
## Data packet IDs
XFTP servers MUST generate a separate new set of IDs for each new chunk - for the sender (that uploads the chunk) and for each intended recipient. It is REQUIRED that:
XFTP routers MUST generate a separate new set of IDs for each new packet - for the sender (that uploads the packet) and for each intended recipient. It is REQUIRED that:
- These IDs are different and unique within the server.
- These IDs are different and unique within the router.
- Based on random bytes generated with cryptographically strong pseudo-random number generator.
## Server security requirements
## Router security requirements
XFTP server implementations MUST NOT create, store or send to any other servers:
XFTP router implementations MUST NOT create, store or send to any other routers:
- Logs of the client commands and transport connections in the production environment.
- History of retrieved files.
- Snapshots of the database they use to store file chunks (instead clients can manage redundancy by creating chunk replicas using more than one XFTP server). In-memory persistence is recommended for file chunks records.
- Snapshots of the database they use to store data packets (instead clients can manage redundancy by creating packet replicas using more than one XFTP router). In-memory persistence is recommended for data packets records.
- Any other information that may compromise privacy or [forward secrecy][4] of communication between clients using XFTP servers.
- Any other information that may compromise privacy or [forward secrecy][4] of communication between clients using XFTP routers.
## Transport protocol
- binary-encoded commands sent as fixed-size padded block in the body of HTTP2 POST request, similar to SMP and notifications server protocol transmission encodings.
- binary-encoded commands sent as fixed-size padded block in the body of HTTP2 POST request, similar to SMP and notifications router protocol transmission encodings.
- HTTP2 POST with a fixed size padded block body for file upload and download.
Block size - 16384 bytes (it would fit ~350 Ed25519 recipient keys).
@@ -306,41 +308,41 @@ The reason not to use URI segments / HTTP verbs / REST semantics is to have cons
### ALPN to agree handshake version
Client and server use [ALPN extension][18] of TLS to agree handshake version.
Client and router use [ALPN extension][18] of TLS to agree handshake version.
Server SHOULD send `xftp/1` protocol name and the client should confirm this name in order to use the current protocol version. This is added to allow support of older clients without breaking backward compatibility and to extend or modify handshake syntax.
Router SHOULD send `xftp/1` protocol name and the client should confirm this name in order to use the current protocol version. This is added to allow support of older clients without breaking backward compatibility and to extend or modify handshake syntax.
If the client does not confirm this protocol name, the server would fall back to v1 of XFTP protocol.
If the client does not confirm this protocol name, the router would fall back to v1 of XFTP protocol.
### Transport handshake
When a client and a server agree on handshake version using ALPN extension, they should proceed with XFTP handshake.
When a client and a router agree on handshake version using ALPN extension, they should proceed with XFTP handshake.
As with SMP, a client doesn't reveal its version range to avoid version fingerprinting. Unlike SMP, XFTP runs a HTTP2 protocol over TLS and the server can't just send its handshake right away. So a session handshake is driven by client-sent requests:
As with SMP, a client doesn't reveal its version range to avoid version fingerprinting. Unlike SMP, XFTP runs a HTTP2 protocol over TLS and the router can't just send its handshake right away. So a session handshake is driven by client-sent requests:
1. To pass initiative to the server, the client sends a request with empty body.
2. Server responds with its `paddedServerHello` block.
1. To pass initiative to the router, the client sends a request with empty body.
2. Router responds with its `paddedRouterHello` block.
3. Clients sends a request containing `paddedClientHello` block,
4. Server sends an empty response, finalizing the handshake.
4. Router sends an empty response, finalizing the handshake.
Once TLS handshake is complete, client and server will exchange blocks of fixed size (16384 bytes).
Once TLS handshake is complete, client and router will exchange blocks of fixed size (16384 bytes).
```abnf
paddedServerHello = <padded(serverHello, 16384)>
serverHello = xftpVersionRange sessionIdentifier serverCerts signedServerKey ignoredPart
paddedRouterHello = <padded(routerHello, 16384)>
routerHello = xftpVersionRange sessionIdentifier routerCerts signedRouterKey ignoredPart
xftpVersionRange = minXftpVersion maxXftpVersion
minXftpVersion = xftpVersion
maxXftpVersion = xftpVersion
sessionIdentifier = shortString
; unique session identifier derived from transport connection handshake
serverCerts = length 1*serverCert ; NonEmpty list of certificates in chain
serverCert = originalLength <x509encoded>
signedServerKey = originalLength <x509encoded> ; signed by server certificate
routerCerts = length 1*routerCert ; NonEmpty list of certificates in chain
routerCert = originalLength <x509encoded>
signedRouterKey = originalLength <x509encoded> ; signed by router certificate
paddedClientHello = <padded(clientHello, 16384)>
clientHello = xftpVersion keyHash ignoredPart
; chosen XFTP protocol version - must be the maximum supported version
; within the range offered by the server
; within the range offered by the router
xftpVersion = 2*2OCTET ; Word16 version number
keyHash = shortString
@@ -350,47 +352,47 @@ originalLength = 2*2OCTET
ignoredPart = *OCTET
```
In XFTP v2 the handshake is only used for version negotiation, but `serverCert` and `signedServerKey` must be validated by the client.
In XFTP v2 the handshake is only used for version negotiation, but `routerCert` and `signedRouterKey` must be validated by the client.
`keyHash` is the CA fingerprint used by client to validate TLS certificate chain and is checked by a server against its own key.
`keyHash` is the CA fingerprint used by client to validate TLS certificate chain and is checked by a router against its own key.
`ignoredPart` in handshake allows to add additional parameters in handshake without changing protocol version - the client and servers must ignore any extra bytes within the original block length.
`ignoredPart` in handshake allows to add additional parameters in handshake without changing protocol version - the client and routers must ignore any extra bytes within the original block length.
For TLS transport client should assert that `sessionIdentifier` is equal to `tls-unique` channel binding defined in [RFC 5929][14] (TLS Finished message struct); we pass it in `serverHello` block to allow communication over some other transport protocol (possibly, with another channel binding).
For TLS transport client should assert that `sessionIdentifier` is equal to `tls-unique` channel binding defined in [RFC 5929][14] (TLS Finished message struct); we pass it in `routerHello` block to allow communication over some other transport protocol (possibly, with another channel binding).
### Requests and responses
- File sender:
- create file chunk record.
- create data packet record.
- Parameters:
- Ed25519 key for subsequent sender commands and Ed25519 keys for commands of each recipient.
- chunk size.
- packet size.
- Response:
- chunk ID for the sender and different IDs for all recipients.
- add recipients to file chunk
- packet ID for the sender and different IDs for all recipients.
- add recipients to data packet
- Parameters:
- sender's chunk ID
- sender's packet ID
- Ed25519 keys for commands of each recipient.
- Response:
- chunk IDs for new recipients.
- upload file chunk.
- delete file chunk (invalidates all recipient IDs).
- packet IDs for new recipients.
- upload data packet.
- delete data packet (invalidates all recipient IDs).
- File recipient:
- download file chunk:
- chunk ID
- DH key for additional encryption of the chunk.
- command should be signed with the key passed by the sender when creating chunk record.
- delete file chunk ID (only for one recipient): signed with the same key.
- download data packet:
- packet ID
- DH key for additional encryption of the packet.
- command should be signed with the key passed by the sender when creating packet record.
- delete data packet ID (only for one recipient): signed with the same key.
## XFTP commands
Commands syntax below is provided using ABNF with case-sensitive strings extension.
```abnf
xftpCommand = ping / senderCommand / recipientCmd / serverMsg
xftpCommand = ping / senderCommand / recipientCmd / routerMsg
senderCommand = register / add / put / delete
recipientCmd = get / ack
serverMsg = pong / sndIds / rcvIds / ok / file / error
routerMsg = pong / sndIds / rcvIds / ok / file / error
```
The syntax of specific commands and responses is defined below.
@@ -401,11 +403,11 @@ Commands are made via HTTP2 requests, responses to commands are correlated as HT
### Command authentication
XFTP servers must authenticate all transmissions (excluding `ping`) by verifying the client signatures. Command signature should be generated by applying the algorithm specified for the file to the `signed` block of the transmission, using the key associated with the file chunk ID (recipient's or sender's depending on which file chunk ID is used).
XFTP routers must authenticate all transmissions (excluding `ping`) by verifying the client signatures. Command signature should be generated by applying the algorithm specified for the file to the `signed` block of the transmission, using the key associated with the data packet ID (recipient's or sender's depending on which data packet ID is used).
### Keep-alive command
To keep the transport connection alive and to generate noise traffic the clients should use `ping` command to which the server responds with `pong` response. This command should be sent unsigned and without file chunk ID.
To keep the transport connection alive and to generate noise traffic the clients should use `ping` command to which the router responds with `pong` response. This command should be sent unsigned and without data packet ID.
```abnf
ping = %s"PING"
@@ -421,13 +423,13 @@ pong = %s"PONG"
### File sender commands
Sending any of the commands in this section (other than `register`, that is sent without file chunk ID) is only allowed with sender's ID.
Sending any of the commands in this section (other than `register`, that is sent without data packet ID) is only allowed with sender's ID.
#### Register new file chunk
#### Register new data packet
This command is sent by the sender to the XFTP server to register a new file chunk.
This command is sent by the sender to the XFTP router to register a new data packet.
Servers SHOULD support basic auth with this command, to allow only server owners and trusted users to create file chunks on the servers.
Routers SHOULD support basic auth with this command, to allow only router owners and trusted users to create data packets on the routers.
The syntax is:
@@ -446,7 +448,7 @@ x509encoded = <binary X509 key encoding>
length = 1*1 OCTET
```
If the file chunk is registered successfully, the server must send `sndIds` response with the sender's and recipients' file chunk IDs:
If the data packet is registered successfully, the router must send `sndIds` response with the sender's and recipients' data packet IDs:
```abnf
sndIds = %s"SIDS " senderId recipientIds
@@ -455,9 +457,9 @@ recipientIds = length 1*recipientId
recipientId = length *OCTET
```
#### Add file chunk recipients
#### Add data packet recipients
This command is sent by the sender to the XFTP server to add additional recipient keys to the file chunk record, in case number of keys requested by client didn't fit into `register` command. The syntax is:
This command is sent by the sender to the XFTP router to add additional recipient keys to the data packet record, in case number of keys requested by client didn't fit into `register` command. The syntax is:
```abnf
add = %s"FADD " rcvPublicAuthKeys
@@ -465,7 +467,7 @@ rcvPublicAuthKeys = length 1*rcvPublicAuthKey
rcvPublicAuthKey = length x509encoded
```
If additional keys were added successfully, the server must send `rcvIds` response with the added recipients' file chunk IDs:
If additional keys were added successfully, the router must send `rcvIds` response with the added recipients' data packet IDs:
```abnf
rcvIds = %s"RIDS " recipientIds
@@ -473,46 +475,46 @@ recipientIds = length 1*recipientId
recipientId = length *OCTET
```
#### Upload file chunk
#### Upload data packet
This command is sent by the sender to the XFTP server to upload file chunk body to server. The syntax is:
This command is sent by the sender to the XFTP router to upload data packet body to router. The syntax is:
```abnf
put = %s"FPUT"
```
Chunk body is streamed via HTTP2 request.
Packet body is streamed via HTTP2 request.
If file chunk body was successfully received, the server must send `ok` response.
If data packet body was successfully received, the router must send `ok` response.
```abnf
ok = %s"OK"
```
#### Delete file chunk
#### Delete data packet
This command is sent by the sender to the XFTP server to delete file chunk from the server. The syntax is:
This command is sent by the sender to the XFTP router to delete data packet from the router. The syntax is:
```abnf
delete = %s"FDEL"
```
Server should delete file chunk record, invalidating all recipient IDs, and delete file body from file storage. If file chunk was successfully deleted, the server must send `ok` response.
Router should delete data packet record, invalidating all recipient IDs, and delete file body from file storage. If data packet was successfully deleted, the router must send `ok` response.
### File recipient commands
Sending any of the commands in this section is only allowed with recipient's ID.
#### Download file chunk
#### Download data packet
This command is sent by the recipient to the XFTP server to download file chunk body from the server. The syntax is:
This command is sent by the recipient to the XFTP router to download data packet body from the router. The syntax is:
```abnf
get = %s"FGET " rDhKey
rDhKey = length x509encoded
```
If requested file is successfully located, the server must send `file` response. File chunk body is sent as HTTP2 response body.
If requested file is successfully located, the router must send `file` response. Data packet body is sent as HTTP2 response body.
```abnf
file = %s"FILE " sDhKey cbNonce
@@ -520,23 +522,23 @@ sDhKey = length x509encoded
cbNonce = 24*24 OCTET ; NaCl crypto_box nonce
```
Chunk is additionally encrypted on the way from the server to the recipient using a key agreed via ephemeral DH keys `rDhKey` and `sDhKey`, so there is no ciphertext in common between sent and received traffic inside TLS connection, in order to complicate traffic correlation attacks, if TLS is compromised.
Packet is additionally encrypted on the way from the router to the recipient using a key agreed via ephemeral DH keys `rDhKey` and `sDhKey`, so there is no ciphertext in common between sent and received traffic inside TLS connection, in order to complicate traffic correlation attacks, if TLS is compromised.
#### Acknowledge file chunk download
#### Acknowledge data packet download
This command is sent by the recipient to the XFTP server to acknowledge file reception, deleting file ID from server for this recipient. The syntax is:
This command is sent by the recipient to the XFTP router to acknowledge file reception, deleting file ID from router for this recipient. The syntax is:
```abnf
ack = %s"FACK"
```
If file recipient ID is successfully deleted, the server must send `ok` response.
If file recipient ID is successfully deleted, the router must send `ok` response.
In current implementation of XFTP protocol in SimpleX Chat clients don't use FACK command. Files are automatically expired on servers after configured time interval.
In current implementation of XFTP protocol in SimpleX Chat clients don't use FACK command. Files are automatically expired on routers after configured time interval.
### Error responses
The server responds with `ERR` followed by the error type:
The router responds with `ERR` followed by the error type:
```abnf
error = %s"ERR " errorType
@@ -556,17 +558,17 @@ Error types:
- `SESSION` - incorrect session ID (TLS Finished message / tls-unique binding).
- `HANDSHAKE` - incorrect handshake command.
- `CMD` - command syntax errors (UNKNOWN, SYNTAX, PROHIBITED, NO_AUTH, HAS_AUTH, NO_ENTITY).
- `AUTH` - command authorization error - bad signature or non-existing file chunk.
- `BLOCKED` - file chunk was blocked due to policy violation (added in v3). Contains blocking reason and optional notice.
- `AUTH` - command authorization error - bad signature or non-existing data packet.
- `BLOCKED` - data packet was blocked due to policy violation (added in v3). Contains blocking reason and optional notice.
- `SIZE` - incorrect file size.
- `QUOTA` - storage quota exceeded.
- `DIGEST` - incorrect file digest.
- `CRYPTO` - file encryption/decryption failed.
- `NO_FILE` - no expected file body in request/response or no file on the server.
- `NO_FILE` - no expected file body in request/response or no file on the router.
- `HAS_FILE` - unexpected file body.
- `FILE_IO` - file IO error.
- `TIMEOUT` - file sending or receiving timeout.
- `INTERNAL` - internal server error.
- `INTERNAL` - internal router error.
## Threat model
@@ -575,7 +577,7 @@ Error types:
- A user protects their local database and key material.
- The user's application is authentic, and no local malware is running.
- The cryptographic primitives in use are not broken.
- A user's choice of servers is not directly tied to their identity or otherwise represents distinguishing information about the user.
- A user's choice of routers is not directly tied to their identity or otherwise represents distinguishing information about the user.
#### A passive adversary able to monitor the traffic of one user
@@ -583,7 +585,7 @@ Error types:
- identify that and when a user is sending files over XFTP protocol.
- determine which servers the user sends/receives files to/from.
- determine which routers the user sends/receives files to/from.
- observe how much traffic is being sent, and make guesses as to its purpose.
@@ -595,11 +597,11 @@ Error types:
*can:*
- learn which XFTP servers are used to send and receive files for which users.
- learn which XFTP routers are used to send and receive files for which users.
- learn when files are sent and received.
- perform traffic correlation attacks against senders and recipients and correlate senders and recipients within the monitored set, frustrated by the number of users on the servers.
- perform traffic correlation attacks against senders and recipients and correlate senders and recipients within the monitored set, frustrated by the number of users on the routers.
- observe how much traffic is being sent, and make guesses as to its purpose.
@@ -609,31 +611,31 @@ Error types:
- perform traffic correlation attacks.
#### XFTP server
#### XFTP router
*can:*
- learn when file senders and recipients are online.
- know how many file chunks and chunk sizes are sent via the server.
- know how many data packets and packet sizes are sent via the router.
- perform the correlation of the file chunks as belonging to one file via either a re-used transport connection, user's IP address, or connection timing regularities.
- perform the correlation of the data packets as belonging to one file via either a re-used transport connection, user's IP address, or connection timing regularities.
- learn file senders' and recipients' IP addresses, and infer information (e.g. employer) based on the IP addresses, as long as Tor is not used.
- delete file chunks, preventing file delivery, as long as redundant delivery is not used.
- delete data packets, preventing file delivery, as long as redundant delivery is not used.
- lie about the state of a file chunk to the recipient and/or to the sender (e.g. deleted when it is not).
- lie about the state of a data packet to the recipient and/or to the sender (e.g. deleted when it is not).
- refuse deleting the file when instructed by the sender.
*cannot:*
- undetectably corrupt file chunks.
- undetectably corrupt data packets.
- learn the contents, name or the exact size of sent files.
- learn approximate size of sent files, as long as more than one server is used to send file chunks.
- learn approximate size of sent files, as long as more than one router is used to send data packets.
- compromise the users' end-to-end encryption of files with an active attack.
@@ -645,7 +647,7 @@ Error types:
- receive all files sent and received by Alice that did not expire yet, as long as information about these files was not removed from the database.
- prevent Alice's contacts from receiving the files she sent by deleting all or some of the file chunks from XFTP servers.
- prevent Alice's contacts from receiving the files she sent by deleting all or some of the data packets from XFTP routers.
#### A user's contact
@@ -667,10 +669,10 @@ Error types:
*can:*
- Denial of Service XFTP servers.
- Denial of Service XFTP routers.
*cannot:*
- send files to a user who they are not connected with.
- enumerate file chunks on an XFTP server.
- enumerate data packets on an XFTP router.