dandri/simplexmq
1
0
Fork 0
mirror of https://github.com/simplex-chat/simplexmq.git synced 2026-03-30 20:45:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

SMP agent protocol - duplex messaging (#39)

Browse Source 
* duplex messaging commands syntax

* update duplex messaging commands

* update duplex commands/responses

* SMP messages between agents

* error for multiple skipped messages

* more syntax

* more syntax

* add diagram: creating duplex connection

* fix diagram link

* update diagram

* update duplex diagram

* add queue statuses to the diagram

* add "try sending" periods to duplex diagram

* diagram: queue status (receive/send)

* update queue status

* simplify duplex connection to only have two queues

* remove error notification sent to another agent, only notify user

* remove unused commands, add "unsubscribed" notification

* simplified commands and added connection invitation syntax

* update SMP agent protocol

* duplex protocol correction

* corrections (#40)

* SMP agent protocol

* rename duplex-messaging to agent-protocol

* minor fixes

* SMP agent protocol corrections

Co-authored-by: Efim Poberezkin <8711996+efim-poberezkin@users.noreply.github.com>
This commit is contained in:
Evgeny Poberezkin
2021-04-16 19:56:53 +01:00
committed by GitHub
parent 28a3db5edc
commit 36dad0ba86
4 changed files with 307 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

228
agent-protocol.md Normal file
View File

@@ -0,0 +1,228 @@
# SMP agent protocol - duplex communication over SMP protocol
## Table of contents
- [Abstract](#abstract)
- [SMP agent](#smp-agent)
- [SMP agent protocol components](#smp-agent-protocol-components)
- [Duplex connection](#duplex-connection)
- [Communication between SMP agents](#communication-between-smp-agents)
- [Message syntax](#messages-between-smp-agents)
- [SMP agent commands](#smp-agent-commands)
- [Connection invitation](#connection-invitation)
## Abstract
The purpose of SMP agent protocol is to define the syntax and the semantics of communications between the client and the agent that connects to [SMP](./simplex-messaging.md) servers.
It provides:
- convenient protocol to create and manage a bi-directional (duplex) connection to the users of SMP agents consisting of two separate unidirectional (simplex) SMP queues, abstracting away multiple steps required to establish bi-directional connections.
- management of E2E encryption between SMP agents, generating ephemeral RSA keys for each connection.
- SMP command authentication on SMP servers, generating ephemeral RSA keys for each SMP queue.
- TCP transport handshake and encryption with SMP servers.
- validation of message integrity.
SMP agent protocols provides no encryption or any security on the client side - it is assumed that the agent is executed in the trusted and secure environment.
The future versions of this protocol could provide:
- managing redundant SMP queues with more than 1 queue in each direction.
- managing simple symmetric groups as a foundation for chat groups and device synchronization.
- agent cluster - synchronizing states of multiple agents.
- secure "synchronous" streams with symmetric message encryption and connection-level authentication (requires extending SMP protocol) - it can be used, e.g., for file transfers.
## SMP agent
SMP agent is a client-side process or library that communicates via SMP servers using [simplex messaging protocol (SMP)](./simplex-messaging.md) with other SMP agents according to the commands received from its users. This protocol is a middle layer in SMP protocols stack (above SMP protocol but below any application level protocol) - it is intended to be used by client-side applications that need secure asynchronous bi-directional communication channels ("connections").
The agent must have a persistent storage to manage the states of known connections and of the client-side information of two SMP queues that each connection consists of, and also the buffer of the most recent messages. The number of the messages that should be stored is implementation specific, depending on the error management approach that the agent implements; at the very least the agent must store the hash and id of the last received message.
## SMP agent protocol components
SMP agent protocol has 3 main parts:
- the syntax and semantics of messages that SMP agents exchange between each other in order to:
- negotiate establishing unidirectional (simplex) encrypted queues on SMP server(s)
- exchange client messages and delivery notifications, providing sequential message IDs and message integrity (by including the hash of the previous message).
- the syntax and semantics of the commands (a higher level interface than SMP protocol) that are sent over TCP or other sequential protocol by agent clients to the agents. This protocol allows to create and manage multiple connections, each consisting of two simplex SMP queues.
- the syntax and semantics of the message that the clients of SMP agents should send out-of-band (as pre-shared "invitation" including SMP server, queue ID and encryption key) to ensure [E2E encryption][1] the integrity of SMP queues and protection against active attacks ([MITM attacks][2]).
## Duplex connection
**Creating duplex connection between Alice and Bob:**
![Creating duplex connection](/diagrams/duplex-messaging/duplex-creating.svg)
## Communication between SMP agents
SMP agents communicate via SMP servers managing creation, deletion and operations of SMP queues.
Agents can use SMP message client body (the part of the SMP message after header - see SMP protocol) to transmit agent client messages and exchange messages between each other.
Each SMP message client body, once decrypted, contains 3 parts (one of them may include binary message body), as defined by `decryptedSmpMessageBody` syntax:
- `agentMsgHeader` - agent message header that contains sequential agent message ID for a particular SMP queue, agent timestamp (ISO8601) and the hash of the previous message.
- `agentMessage` - a command/message to the other SMP agent:
- to establish the connection with two SMP queues (`helloMsg`, `replyQueueMsg`)
- to send and to acknowledge user messages (`clientMsg`, `acknowledgeMsg`)
- to notify another agent about queue deletion (`deleteQueueMsg`)
- `msgPadding` - an optional message padding to make all SMP messages have consistent size as an additional privacy protection measure.
### Messages between SMP agents
Message syntax below uses [ABNF][1].
```abnf
decryptedSmpMessageBody = agentMsgHeader CRLF agentMessage CRLF msgPadding
agentMsgHeader = agentMsgId SP agentTimestamp SP previousMsgHash
agentMsgId = 1*DIGIT ; sequential agent message ID set by the sending agent
agentMessage = helloMsg / replyQueueMsg / deleteQueueMsg
/ clientMsg / acknowledgeMsg
msgPadding = *OCTET ; optional random bytes to get messages to the same size (as defined in SMP message size)
helloMsg = %s"HELLO" SP signatureVerificationKey [SP %s"NO_ACK"]
; NO_ACK means that acknowledgements to client messages will NOT be sent in this connection by the agent that sent `HELLO` message.
signatureVerificationKey = encoded ; base64 encoded
replyQueueMsg = %s"REPLY" SP qInfo ; `qInfo` is the same as in out-of-band message
; this message can only be sent by the second connection party
deleteQueueMsg = %s"DEL" ; notification that recipient queue will be deleted
; no need to notify the other party about suspending queue separately, as suspended and deleted queues are the same to the sender
clientMsg = %s"MSG" SP size CRLF clientMsgBody CRLF ; CRLF is in addition to CRLF in decryptedSmpMessageBody
size = 1*DIGIT
clientMsgBody = *OCTET
acknowledgeMsg = %s"ACK" SP agentMsgId SP ackStatus
ackStatus = %s"OK" / ackError
ackError = %s"ERR" SP ackErrorType
ackErrorType = ackUnknownMsg / ackProhibitedMsg / ackSyntaxErr
ackUnknownMsg = %s"UNKNOWN"
ackProhibitedMsg = %"PROHIBITED" ; e.g. "HELLO" or "REPLY"
ackSyntaxErr = %"SYNTAX" SP syntaxErrCode
syntaxErrCode = 1*DIGIT ; TODO
```
## SMP agent commands
This part describes the transmissions between users and client-side SMP agents: commands that the users send to create and operate duplex connections and SMP agent responses and messages they deliver.
Commands syntax below is provided using [ABNF][1].
Each transmission between the user and SMP agent must have this format/syntax:
```abnf
agentTransmission = [corrId] CRLF [cAlias] CRLF agentCommand
corrId = 1*(%x21-7F) ; any characters other than control/whitespace
cAlias = cId / cName
cId = encoded
cName = 1*(ALPHA / DIGIT / "_" / "-")
agentCommand = (userCmd / agentMsg) CRLF
userCmd = newCmd / joinCmd
/ acceptCmd / subscribeCmd
/ sendCmd / acknowledgeCmd
/ suspendCmd / deleteCmd
agentMsg = invitation / confirmation
/ connected / unsubscribed
/ message / sent / received
/ ok / error
newCmd = %s"NEW" SP smpServer [SP %s"NO_ACK"]
; response is `invitation` or `error`
smpServer = srvHost [":" port] ["#" keyFingerprint]
srvHost = hostname ; RFC1123, RFC5891
port = 1*DIGIT
keyFingerprint = encoded
invitation = %s"INV" SP qInfo
connected = %s"CON"
subscribeCmd = %s"SUB"
unsubscribed = %s"END"
; when another agent (or another client of the same agent)
; subscribes to the same SMP queue on the server
joinCmd = %s"JOIN" SP qInfo
[SP (smpServer / %s"NO_REPLY")] ; reply queue SMP server
; server from qInfo is used by default
[SP %s"NO_ACK"]
; response is `connection` or `error`
confirmation = %s"CONF" SP partyId SP partyInfo
; currently not implemented
acceptCmd = %s"LET" SP partyId ; response is `ok` or `error`
; currently not implemented
suspendCmd = %s"OFF" ; can be sent by either party, response `ok` or `error`
deleteCmd = %s"DEL" ; can be sent by either party, response `ok` or `error`
sendCmd = %s"SEND" SP msgBody
; send syntax is similar to that of SMP protocol, but it is wrapped in SMP message
msgBody = stringMsg | binaryMsg
stringMsg = ":" string ; until CRLF in the transmission
string = *(%x01-09 / %x0B-0C / %x0E-FF %) ; any characters other than NUL, CR and LF
binaryMsg = size CRLF msgBody CRLF ; the last CRLF is in addition to CRLF in the transmission
size = 1*DIGIT ; size in bytes
msgBody = *OCTET ; any content of specified size - safe for binary
sent = %s"SENT" SP agentMsgId
message = %s"MSG" SP msgStatus
SP %s"R=" agentMsgId "," agentTimestamp ; receiving agent
SP %s"B=" brokerMsgId "," srvTimestamp ; broker (server)
SP %s"S=" agentMsgId "," agentTimestamp ; sending agent
SP binaryMsg
agentMsgId = 1*DIGIT
srvTimestamp = date-time ; RFC3339
agentTimestamp = date-time
msgStatus = ok / messageError
messageError = %s"ERR" SP messageErrorType
messageErrorType = skippedMsgErr / badMsgIdErr / badHashErr
skippedMsgErr = %"IDS" SP missingFromMsgId SP missingToMsgId
badMsgIdErr = %"ID" SP previousMsgId ; ID is lower than the previous
badHashErr = %"HASH"
acknowledge = %s"ACK" SP agentMsgId ; ID assigned by receiving agent (in MSG "R")
received = %s"RCVD" SP agentMsgId ; ID assigned by sending agent (in SENT response)
ok = %s"OK"
error = %s"ERR" SP errorType
encoded = base64
```
## Connection invitation
Connection invitation `qInfo` is generated by SMP agent in response to `newCmd` command (`"NEW"`), used by another party user with `joinCmd` command (`"JOIN"`), and then another invitation is sent by the agent in `replyQueueMsg` and used by the first party agent to connect to the reply queue (the second part of the process is invisible to the users).
Connection invitation is a text with the following syntax:
```
qInfo = %s"smp::" smpServer "::" queueId "::" ephemeralPublicKey
queueId = encoded
ephemeralPublicKey = encoded ; RSA key for sender to encrypt messages X509 base64 encoded
```
[1]: https://en.wikipedia.org/wiki/End-to-end_encryption
[2]: https://en.wikipedia.org/wiki/Man-in-the-middle_attack
[3]: https://tools.ietf.org/html/rfc5234

72
diagrams/duplex-messaging/duplex-creating.mmd Normal file
View File

@@ -0,0 +1,72 @@
sequenceDiagram
participant A as Alice
participant AA as Alice's<br>agent
participant AS as Alice's<br>server
participant BS as Bob's<br>server
participant BA as Bob's<br>agent
participant B as Bob
note over AA, BA: status (receive/send): NONE/NONE
note over A, AA: 1. request connection from agent
A ->> AA: NEW: create<br>duplex connection
note over AA, AS: 2. create Alice's SMP queue
AA ->> AS: NEW: create SMP queue
AS ->> AA: IDS: SMP queue IDs
note over AA: status: NEW/NONE
A ->> AA: INVITE: invite<br>another party
AA ->> A: JOIN: invitation<br>to connect
note over AA: status: PENDING/NONE
note over A, B: 3. out-of-band invitation
A ->> B: OOB: invitation to connect
B ->> BA: CONNECT:<br>via invitation info
note over BA: status: NONE/NEW
note over BA, AA: 4. establish Alice's SMP queue
BA ->> AS: SEND: KEY: sender's key and Bob's profile
note over BA: status: NONE/CONFIRMED
activate BA
AS ->> AA: MSG: KEY: sender's key<br>and Bob's profile
note over AA: status: CONFIRMED/NONE
AA ->> A: Bob's profile<br>and party ID
A ->> AA: ACCEPT:<br>accept party ID
AA ->> AS: KEY: secure queue
note over AA: status: SECURED/NONE
BA ->> AS: SEND: HI: try sending until successful
deactivate BA
note over BA: status: NONE/ACTIVE
AS ->> AA: MSG: HI: Alice's agent<br>knows Bob can send
note over AA: status: ACTIVE/NONE
note over BA, BS: 5. create Bob's SMP queue
BA ->> BS: NEW: create SMP queue
BS ->> BA: IDS: SMP queue IDs
note over BA: status: NEW/ACTIVE
note over AA, BA: 6. establish Bob's SMP queue
BA ->> AS: SEND: REPLY: invitation to the connect
note over BA: status: PENDING/ACTIVE
AS ->> AA: MSG: REPLY: invitation<br>to connect
note over AA: status: ACTIVE/NEW
AA ->> BS: SEND: KEY: sender's key and Alice's profile
note over AA: status: ACTIVE/CONFIRMED
activate AA
BS ->> BA: MSG: KEY: sender's key<br>and Alice's profile
note over BA: status: CONFIRMED/ACTIVE
BA ->> BS: KEY: secure queue
note over BA: status: SECURED/ACTIVE
AA ->> BS: SEND: HI: try sending until successful
deactivate AA
note over AA: status: ACTIVE/ACTIVE
BS ->> BA: MSG: HI: Bob's agent<br>knows Alice can send
note over BA: status: ACTIVE/ACTIVE
note over A, B: 7. notify users about connection success
AA ->> A: CONNECTED
BA ->> B: CONNECTED

1
diagrams/duplex-messaging/duplex-creating.svg Normal file
View File

File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 31 KiB

11
simplex-messaging.md
View File

@@ -451,10 +451,11 @@ Each transmission between the client and the server must have this format/syntax
```abnf
transmission = [signature] CRLF signed CRLF
signed = [queueId] CRLF msg
signed = [corrId] CRLF [queueId] CRLF msg
msg = recipientCmd / send / serverMsg
recipientCmd = create / subscribe / secure / acknowledge / suspend / delete
serverMsg = queueIds / message / unsubscribed / ok / error
corrId = 1*(%x21-7F) ; any characters other than control/whitespace
queueId = encoded ; empty queue ID is used with "create" command
signature = encoded ; empty signature can be used with "create" and "send" commands
encoded = base64
@@ -644,7 +645,7 @@ The server must respond with `"ERR AUTH"` response in the following cases:
- queue is NOT secured but the transmission has a signature.
Until the queue is secured, the server should accept any number of unsigned
messages - it both enables the legimate sender to resend the confirmation in
messages - it both enables the legitimate sender to resend the confirmation in
case of failure and also allows the simplex messaging client to ignore any
confirmation messages that may be sent by the attackers (assuming they could
have intercepted the queue ID in the server response, but do not have a correct
@@ -654,14 +655,14 @@ The body should be encrypted with the recipient's "public" key (`EK`); once
decrypted it must have this format:
```abnf
decryptedBody = reserved LF clientBody LF
reserved = senderKeyMsg / *VCHAR
decryptedBody = clientHeader CRLF clientBody CRLF
clientHeader = senderKeyMsg / *VCHAR
senderKeyMsg = %s"KEY" SP senderKey
senderKey = encoded
clientBody = *OCTET
```
`reserved` in the initial unsigned message is used to transmit sender's server
`clientHeader` in the initial unsigned message is used to transmit sender's server
key and can be used in the future revisions of SMP protocol for other purposes.
### Server messages