From 01e8c232f03d262c511c98ec1fda8cda53b15db6 Mon Sep 17 00:00:00 2001 From: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com> Date: Mon, 22 Nov 2021 19:08:19 +0000 Subject: [PATCH] change syntax for queue URI and connection request (#205) * change syntax for queue URI and connection request * separate queue URI into separate section * correction --- protocol/agent-protocol.md | 53 ++++++++++++++++++++--------------- protocol/simplex-messaging.md | 52 +++++++++++++++++----------------- protocol/smp-uri-request.txt | 16 +++++++++++ 3 files changed, 72 insertions(+), 49 deletions(-) create mode 100644 protocol/smp-uri-request.txt diff --git a/protocol/agent-protocol.md b/protocol/agent-protocol.md index 9d80beef0..417a75853 100644 --- a/protocol/agent-protocol.md +++ b/protocol/agent-protocol.md @@ -24,7 +24,7 @@ - [END notification](#end-notification) - [OFF command](#off-command) - [DEL command](#del-command) -- [Connection invitation](#connection-invitation) +- [Connection request](#connection-request) ## Abstract @@ -32,28 +32,27 @@ The purpose of SMP agent protocol is to define the syntax and the semantics of c It provides: - convenient protocol to create and manage bi-directional (duplex) connections between the users of SMP agents consisting of two (or more) separate unidirectional (simplex) SMP queues, abstracting away multiple steps required to establish bi-directional connections and any information about the servers location from the users of the protocol. -- management of E2E encryption between SMP agents, generating ephemeral RSA keys for each connection. -- SMP command authentication on SMP servers, generating ephemeral RSA keys for each SMP queue. -- TCP transport handshake and encryption with SMP servers. +- management of E2E encryption between SMP agents, generating ephemeral asymmetric keys for each connection. +- SMP command authentication on SMP servers, generating ephemeral keys for each SMP queue. +- TCP/TLS transport handshake with SMP servers. - validation of message integrity. -SMP agent protocols provides no encryption or any security on the client side - it is assumed that the agent is executed in the trusted and secure environment. +SMP agent protocol provides no encryption or security on the client side - it is assumed that the agent is executed in the trusted and secure environment, in one of three ways: +- via TCP network using secure connection. +- via local port (when the agent runs on the same device as a separate process). +- via agent library, when the agent logic is included directly into the client application. -The future versions of this protocol could provide: -- managing redundant SMP queues with more than 1 queue in each direction. -- managing simple symmetric groups as a foundation for chat groups and device synchronization. -- agent cluster - synchronizing states of multiple agents. -- secure "synchronous" streams with symmetric message encryption and connection-level authentication (requires extending [SMP protocol](./simplex-messaging.md)) - it can be used, e.g., for file transfers. +The last option is the most secure, as it reduces the number of attack vectors in comparison with other options. [SimpleX Chat for terminal](https://github.com/simplex-chat/simplex-chat) uses this approach. ## SMP agent -SMP agent is a client-side process or library that communicates via SMP servers using [simplex messaging protocol (SMP)](./simplex-messaging.md) with other SMP agents according to the commands received from its users. This protocol is a middle layer in SMP protocols stack (above SMP protocol but below any application level protocol) - it is intended to be used by client-side applications that need secure asynchronous bi-directional communication channels ("connections"). +SMP agent communicates via SMP servers using [simplex messaging protocol (SMP)](./simplex-messaging.md) with other SMP agents according to the commands received from its users. This protocol is a middle layer in SimpleX protocols (above SMP protocol but below any application level protocol) - it is intended to be used by client-side applications that need secure asynchronous bi-directional communication channels ("connections"). -The agent must have a persistent storage to manage the states of known connections and of the client-side information of two SMP queues that each connection consists of, and also the buffer of the most recent messages. The number of the messages that should be stored is implementation specific, depending on the error management approach that the agent implements; at the very least the agent must store the hash and id of the last received message. +The agent must have a persistent storage to manage the states of known connections and of the client-side information of SMP queues that each connection consists of, and also the buffer of the most recent sent and received messages. The number of the messages that should be stored is implementation specific, depending on the error management approach that the agent implements; at the very least the agent must store the hash and id of the last received and sent message. ## SMP servers management -SMP agent protocol commands do not contain SMP servers that the agent will use to establish the connections between their users. The servers are part of the agent configuration and can be dynamically added and removed by the agent implementation: +SMP agent protocol commands do not contain SMP servers that the agent will use to establish the connections. The servers are part of the agent configuration and can be dynamically added and removed by the agent implementation: - by the client applications via any API that is outside of scope of this protocol. - by the agents themselves based on servers availability and latency. @@ -64,7 +63,7 @@ SMP agent protocol has 3 main parts: - the syntax and semantics of messages that SMP agents exchange between each other in order to: - negotiate establishing unidirectional (simplex) encrypted queues on SMP server(s) - exchange client messages and delivery notifications, providing sequential message IDs and message integrity (by including the hash of the previous message). -- the syntax and semantics of the commands (a higher level interface than SMP protocol) that are sent over TCP or other sequential protocol by agent clients to the agents. This protocol allows to create and manage multiple connections, each consisting of two simplex SMP queues. +- the syntax and semantics of the commands that are sent over TCP or other sequential protocol by agent clients to the agents. This protocol allows to create and manage multiple connections, each consisting of two or more SMP queues. - the syntax and semantics of the message that the clients of SMP agents should send out-of-band (as pre-shared "invitation" including SMP server, queue ID and encryption key) to ensure [E2E encryption][1] the integrity of SMP queues and protection against active attacks ([MITM attacks][2]). ## Duplex connection procedure @@ -74,7 +73,7 @@ SMP agent protocol has 3 main parts: The procedure of establishing a duplex connection is explained on the example of Alice and Bob creating a bi-directional connection comprised of two unidirectional (simplex) queues, using SMP agents (A and B) to facilitate it, and two different SMP servers (which could be the same server). It is shown on the diagram above and has these steps: 1. Alice requests the new connection from the SMP agent A using `NEW` command. -2. Agent A creates an SMP connection on the server (using [SMP protocol](./simplex-messaging.md)) and responds to Alice with the invitation that contains queue information and the encryption key Bob's agent B should use. The invitation format is described in [Connection invitation](#connection-invitation). +2. Agent A creates an SMP connection on the server (using [SMP protocol](./simplex-messaging.md)) and responds to Alice with the invitation that contains queue information and the encryption key Bob's agent B should use. The invitation format is described in [Connection request](#connection-request). 3. Alice sends the invitation to Bob via any secure channel they have (out-of-band message). 4. Bob sends `JOIN` command with the invitation as a parameter to agent B to accept the connection. 5. Establishing Alice's SMP connection (with SMP protocol commands): @@ -138,7 +137,7 @@ helloMsg = %s"HELLO" SP signatureVerificationKey [SP %s"NO_ACK"] ; NO_ACK means that acknowledgements to client messages will NOT be sent in this connection by the agent that sent `HELLO` message. signatureVerificationKey = encoded -replyQueueMsg = %s"REPLY" SP ; `queueInfo` is the same as in out-of-band message, see SMP protocol +replyQueueMsg = %s"REPLY" SP ; `connectionRequest` is defined below ; this message can only be sent by the second connection party deleteQueueMsg = %s"DEL" ; notification that recipient queue will be deleted @@ -204,7 +203,7 @@ agentMsg = invitation / connRequest / connInfo / connected / unsubscribed / conn newCmd = %s"NEW" [SP %s"NO_ACK"] ; response is `invitation` or `error` ; NO_ACK parameter currently not supported -invitation = %s"INV" SP ; `queueInfo` is the same as in out-of-band message, see SMP protocol +invitation = %s"INV" SP ; `connectionRequest` is defined below connRequest = %s"REQ" SP confirmationId SP msgBody ; msgBody here is any binary information identifying connection request @@ -231,8 +230,8 @@ connDown = %s"DOWN" connUp = %s"UP" ; restored connection -joinCmd = %s"JOIN" SP [SP %s"NO_REPLY"] [SP %s"NO_ACK"] -; `queueInfo` is the same as in out-of-band message, see SMP protocol +joinCmd = %s"JOIN" SP [SP %s"NO_REPLY"] [SP %s"NO_ACK"] +; `connectionRequest` is defined below ; response is `connected` or `error` suspendCmd = %s"OFF" ; can be sent by either party, response `ok` or `error` @@ -348,11 +347,21 @@ It is used to suspend the receiving SMP queue - sender will no longer be able to It is used to delete the connection and all messages in it, as well as the receiving SMP queue and all messages in it that were remaining on the server. Agent response to this command can be `OK` or `ERR`. This command is irreversible. -## Connection invitation +## Connection request -Connection invitation `queueInfo` is generated by SMP agent in response to `newCmd` command (`"NEW"`), used by another party user with `joinCmd` command (`"JOIN"`), and then another invitation is sent by the agent in `replyQueueMsg` and used by the first party agent to connect to the reply queue (the second part of the process is invisible to the users). +Connection request `connectionRequest` is generated by SMP agent in response to `newCmd` command (`"NEW"`), used by another party user with `joinCmd` command (`"JOIN"`), and then another invitation is sent by the agent in `replyQueueMsg` and used by the first party agent to connect to the reply queue (the second part of the process is invisible to the users). -See SMP protocol [out-of-band messages](./simplex-messaging.md#out-of-band-messages) for connection invitation syntax. +Connection request syntax: + +``` +connectionRequest = queueURI "#/connect/" encryptionScheme ":" publicKey +encryptionScheme = %s"rsa" ; end-to-end encryption and key exchange protocols, + ; the current hybrid encryption scheme (RSA-OAEP/AES-256-GCM-SHA256) + ; will be replaced with double ratchet protocol and DH key exchange. +publicKey = +``` + +See SMP protocol [out-of-band messages](./simplex-messaging.md#out-of-band-messages) for syntax for queueURI. [1]: https://en.wikipedia.org/wiki/End-to-end_encryption [2]: https://en.wikipedia.org/wiki/Man-in-the-middle_attack diff --git a/protocol/simplex-messaging.md b/protocol/simplex-messaging.md index a1b4ff5e9..ad08ccb89 100644 --- a/protocol/simplex-messaging.md +++ b/protocol/simplex-messaging.md @@ -7,6 +7,7 @@ - [SMP Model](#smp-model) - [Out-of-band messages](#out-of-band-messages) - [Simplex queue](#simplex-queue) +- [SMP queue URI](#smp-queue-uri) - [SMP procedure](#smp-procedure) - [SMP qualities and features](#smp-qualities-and-features) - [Cryptographic algorithms](#cryptographic-algorithms) @@ -78,34 +79,9 @@ Creating and using the queue requires sending commands to the SMP server from th ## Out-of-band messages -The out-of-band message with the queue information is sent via some trusted alternative channel from the recipient to the sender. This message is used to share the encryption (a.k.a. "public") key that the sender will use to encrypt the messages (to be decrypted by the recipient), sender queue ID, server hostname and any other information necessary to establish secure encrypted connection with SMP server (see [Appendix A](#appendix-a) for SMP transport protocol). +The out-of-band message with the queue information is sent via some trusted alternative channel from the recipient to the sender. This message is used to share one or several [queue URIs](#smp-queue-uri) that parties can use to establish the initial connection, the encryption scheme and, it can include the public key(s) for end-to-end encryption. -The [ABNF][8] syntax of the message is: - -```abnf -queueInfo = %s"smp::" smpServer "::" queueId "::" encryptionKey -smpServer = srvHost [":" port] "#" serverIdentity -srvHost = ; RFC1123, RFC5891 -port = 1*DIGIT -serverIdentity = encoded -queueId = encoded -encryptionKey = encryptionScheme ":" x509encoded ; the recipient's RSA public key for sender to encrypt messages -encryptionScheme = %s"rsa" ; end-to-end encryption and key exchange protocols, - ; the current hybrid encryption scheme (RSA-OAEP/AES-256-GCM-SHA256) - ; will be replaced with double ratchet protocol and DH key exchange. -x509encoded = -encoded = -``` - -`hostname` can be IP address or domain name, as defined in RFC 1123, section 2.1. - -`port` is optional, the default TCP port for SMP protocol is 5223. - -`serverIdentity` is a required hash of the server certificate SPKI block (without line breaks, header and footer) used by the client to validate server certificate during transport handshake (see [Appendix A](#appendix-a)). - -Encryption keys are encoded using [X509][11] specification. - -Defining the approach to out-of-band message passing is out of scope of this protocol. +The approach to out-of-band message passing and their syntax should be defined in application-level protocols. ## Simplex queue @@ -133,6 +109,28 @@ Queue is defined by recipient ID `RID` and sender ID `SID`, unique for the serve The protocol uses different IDs for sender and recipient in order to provide an additional privacy by preventing the correlation of senders and recipients commands sent over the network - in case the encrypted transport is compromised, it would still be difficult to correlate senders and recipients without access to the queue records on the server. +## SMP queue URI + +The SMP queue URI should include queue hostname, an optional port, sender queue ID and server identity to establish secure connection with SMP server (see [Appendix A](#appendix-a) for SMP transport protocol). + +The [ABNF][8] syntax of the queue URI is: + +```abnf +queueURI = %s"smp://" smpServer "/" queueId +smpServer = serverIdentity "@" srvHost [":" port] +srvHost = ; RFC1123, RFC5891 +port = 1*DIGIT +serverIdentity = base64url +queueId = base64url +base64url = ; RFC4648, section 5 +``` + +`hostname` can be IP address or domain name, as defined in RFC 1123, section 2.1. + +`port` is optional, the default TCP port for SMP protocol is 5223. + +`serverIdentity` is a required hash of the server certificate SPKI block (without line breaks, header and footer) used by the client to validate server certificate during transport handshake (see [Appendix A](#appendix-a)) + ## SMP procedure The SMP procedure of creating a simplex queue on SMP server is explained using participants Alice (the recipient) who wants to receive messages from Bob (the sender). diff --git a/protocol/smp-uri-request.txt b/protocol/smp-uri-request.txt new file mode 100644 index 000000000..ffa3fd263 --- /dev/null +++ b/protocol/smp-uri-request.txt @@ -0,0 +1,16 @@ +Scheme name: smp + +Status: Provisional + +Applications/protocols that use this scheme name: +This scheme is used for URIs of message queues in SimpleX Messaging Protocol, +a client-server protocol for asynchronous distributed unidirectional +message transmission via persistent message queues. + +Contact: Evgeny Poberezkin + +Change controller: Evgeny Poberezkin + +References: +The syntax for message queue URIs in the latest version of SimpleX Messaging Protocol: +https://github.com/simplex-chat/simplexmq/blob/v5/protocol/simplex-messaging.md#smp-queue-uri