package and module docs, remove Simplex.Markdown (moved to simplex-chat), rename Agent.Transmission to Agent.Protocol (#133)

* package and module docs, remove Simplex.Markdown (moved to simplex-chat), rename Agent.Transmission to Agent.Protocol

* move errors.md to haddock comments, Transport docs

* add CHANGELOG.md, add missing package versions

* changelog, copyright

* docs for Simplex.Messaging.Crypto

* consistent punctuation

* use absolute URLs in readme

* correction
This commit is contained in:
Evgeny Poberezkin
2021-05-09 09:36:08 +01:00
committed by GitHub
parent ad87442811
commit 5d59e4b2bd
22 changed files with 722 additions and 518 deletions

26
CHANGELOG.md Normal file
View File

@@ -0,0 +1,26 @@
# 0.3.1
- Released to hackage.org
- SMP agent protocol changes:
- move SMP server from agent commands NEW/JOIN to agent config
- send CON to user only when the 1st party responds HELLO
- Fix REPLY vulnerability
- Fix transaction busy error
# 0.3.0
- SMP encrypted transport over TCP
- Standard X509/PKCS8 encoding for RSA keys
- Sign and verify agent messages
- Verify message integrity based on previous message hash and ID
- Prevent timing attack allowing to determine if queue exists
- Only allow correct RSA keys and signature sizes
# 0.2.0
- SMP client library
- SMP agent with E2E encryption
# 0.1.0
- SMP protocol server implementation without encryption

View File

@@ -5,7 +5,7 @@
## Message broker for unidirectional (simplex) queues
SimpleXMQ is a message broker for managing message queues and sending messages over public network. It consists of SMP server, SMP client library and SMP agent that implement [SMP protocol](./protocol/simplex-messaging.md) for client-server communication and [SMP agent protocol](./protocol/agent-protocol.md) to manage duplex connections via simplex queues on multiple SMP servers.
SimpleXMQ is a message broker for managing message queues and sending messages over public network. It consists of SMP server, SMP client library and SMP agent that implement [SMP protocol](https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md) for client-server communication and [SMP agent protocol](https://github.com/simplex-chat/simplexmq/blob/master/protocol/agent-protocol.md) to manage duplex connections via simplex queues on multiple SMP servers.
SMP protocol is inspired by [Redis serialization protocol](https://redis.io/topics/protocol), but it is much simpler - it currently has only 8 client commands and 6 server responses.
@@ -13,8 +13,8 @@ SimpleXMQ is implemented in Haskell - it benefits from robust software transacti
## SimpleXMQ roadmap
- Streams - high performance message queues. See [Streams RFC](./rfcs/2021-02-28-streams.md) for details.
- "Small" connection groups, when each message will be sent by the SMP agent to multiple connections with a single client command. See [Groups RFC](./rfcs/2021-03-18-groups.md) for details.
- Streams - high performance message queues. See [Streams RFC](https://github.com/simplex-chat/simplexmq/blob/master/rfcs/2021-02-28-streams.md) for details.
- "Small" connection groups, when each message will be sent by the SMP agent to multiple connections with a single client command. See [Groups RFC](https://github.com/simplex-chat/simplexmq/blob/master/rfcs/2021-03-18-groups.md) for details.
- SMP agents cluster to share connections and message management by multiple agents (for example, it would enable multi-device use for [simplex-chat](https://github.com/simplex-chat/simplex-chat)).
- SMP queue redundancy and rotation in SMP agent duplex connections.
- "Large" groups design and implementation.
@@ -23,30 +23,30 @@ SimpleXMQ is implemented in Haskell - it benefits from robust software transacti
### SMP server
[SMP server](./apps/smp-server/Main.hs) can be run on any Linux distribution without any dependencies. It uses in-memory persistence with an optional append-only log of created queues that allows to re-start the server without losing the connections. This log is compacted on every server restart, permanently removing suspended and removed queues.
[SMP server](https://github.com/simplex-chat/simplexmq/blob/master/apps/smp-server/Main.hs) can be run on any Linux distribution without any dependencies. It uses in-memory persistence with an optional append-only log of created queues that allows to re-start the server without losing the connections. This log is compacted on every server restart, permanently removing suspended and removed queues.
To enable the queue logging, uncomment `enable: on` option in `smp-server.ini` configuration file that is created the first time the server is started.
On the first start the server generates an RSA key pair for encrypted transport handshake and outputs hash of the public key every time it runs - this hash should be used as part of the server address: `<hostname>:5223#<key hash>`.
SMP server implements [SMP protocol](./protocol/simplex-messaging.md).
SMP server implements [SMP protocol](https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md).
### SMP client library
[SMP client](./src/Simplex/Messaging/Client.hs) is a Haskell library to connect to SMP servers that allows to:
[SMP client](https://github.com/simplex-chat/simplexmq/blob/master/src/Simplex/Messaging/Client.hs) is a Haskell library to connect to SMP servers that allows to:
- execute commands with a functional API.
- receive messages and other notifications via STM queue.
- automatically send keep-alive commands.
### SMP agent
[SMP agent library](./src/Simplex/Messaging/Agent.hs) can be used to run SMP agent as part of another application and to communicate with the agent via STM queues, without serializing and parsing commands and responses.
[SMP agent library](https://github.com/simplex-chat/simplexmq/blob/master/src/Simplex/Messaging/Agent.hs) can be used to run SMP agent as part of another application and to communicate with the agent via STM queues, without serializing and parsing commands and responses.
Haskell type [ACommand](./src/Simplex/Messaging/Agent/Transmission.hs) represents SMP agent protocol to communicate via STM queues.
Haskell type [ACommand](https://github.com/simplex-chat/simplexmq/blob/master/src/Simplex/Messaging/Agent/Protocol.hs) represents SMP agent protocol to communicate via STM queues.
See [simplex-chat](https://github.com/simplex-chat/simplex-chat) terminal UI for the example of integrating SMP agent into another application.
[SMP agent executable](./apps/smp-agent/Main.hs) can be used to run a standalone SMP agent process that implements plaintext [SMP agent protocol](./protocol/agent-protocol.md) via TCP port 5224, so it can be used via telnet. It can be deployed in private networks to share access to the connections between multiple applications and services.
[SMP agent executable](https://github.com/simplex-chat/simplexmq/blob/master/apps/smp-agent/Main.hs) can be used to run a standalone SMP agent process that implements plaintext [SMP agent protocol](https://github.com/simplex-chat/simplexmq/blob/master/protocol/agent-protocol.md) via TCP port 5224, so it can be used via telnet. It can be deployed in private networks to share access to the connections between multiple applications and services.
## Using SMP server and SMP agent
@@ -58,12 +58,12 @@ It's the easiest to try SMP agent via a prototype [simplex-chat](https://github.
## SMP server design
![SMP server design](./design/server.svg)
![SMP server design](https://raw.githubusercontent.com/simplex-chat/simplexmq/master/design/server.svg)
## SMP agent design
![SMP agent design](./design/agent2.svg)
![SMP agent design](https://raw.githubusercontent.com/simplex-chat/simplexmq/master/design/agent2.svg)
## License
[AGPL v3](./LICENSE)
[AGPL v3](https://github.com/simplex-chat/simplexmq/blob/master/LICENSE)

View File

@@ -1,15 +1,25 @@
name: simplexmq
version: 0.3.0
#synopsis:
#description:
synopsis: SimpleXMQ message broker
description: |
This package includes <./docs/Simplex-Messaging-Server.html server>,
<./docs/Simplex-Messaging-Client.html client> and
<./docs/Simplex-Messaging-Agent.html agent> for SMP protocols:
.
* <https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md SMP protocol>
* <https://github.com/simplex-chat/simplexmq/blob/master/protocol/agent-protocol.md SMP agent protocol>
.
See <https://github.com/simplex-chat/simplex-chat terminal chat prototype> built with SimpleXMQ broker.
homepage: https://github.com/simplex-chat/simplexmq#readme
license: AGPL-3
author: Evgeny Poberezkin
maintainer: evgeny@poberezkin.com
copyright: 2020 Evgeny Poberezkin
author: simplex.chat
maintainer: chat@simplex.chat
copyright: 2020 simplex.chat
category: Web, System, Services, Cryptography
extra-source-files:
- README.md
- CHANGELOG.md
dependencies:
- ansi-terminal == 0.10.*
@@ -20,21 +30,21 @@ dependencies:
- base >= 4.7 && < 5
- base64-bytestring >= 1.0 && < 1.3
- bytestring == 0.10.*
- containers
- containers == 0.6.*
- cryptonite == 0.26.*
- directory == 1.3.*
- filepath == 1.4.*
- generic-random == 1.3.*
- iso8601-time == 0.1.*
- memory == 0.15.*
- mtl
- mtl == 2.2.*
- network == 3.1.*
- network-transport == 0.5.*
- QuickCheck == 2.13.*
- random == 1.1.*
- simple-logger == 0.1.*
- sqlite-simple == 0.4.*
- stm
- stm == 2.5.*
- template-haskell == 2.15.*
- text == 1.2.*
- time == 1.9.*

View File

@@ -1,138 +0,0 @@
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE OverloadedStrings #-}
module Simplex.Markdown where
import Control.Applicative ((<|>))
import Data.Attoparsec.Text (Parser)
import qualified Data.Attoparsec.Text as A
import Data.Either (fromRight)
import Data.Functor (($>))
import Data.Map.Strict (Map)
import qualified Data.Map.Strict as M
import Data.String
import Data.Text (Text)
import qualified Data.Text as T
import System.Console.ANSI.Types
data Markdown = Markdown Format Text | Markdown :|: Markdown
deriving (Eq, Show)
data Format
= Bold
| Italic
| Underline
| StrikeThrough
| Snippet
| Secret
| Colored Color
| NoFormat
deriving (Eq, Show)
instance Semigroup Markdown where (<>) = (:|:)
instance Monoid Markdown where mempty = unmarked ""
instance IsString Markdown where fromString = unmarked . T.pack
unmarked :: Text -> Markdown
unmarked = Markdown NoFormat
colorMD :: Char
colorMD = '!'
secretMD :: Char
secretMD = '#'
formats :: Map Char Format
formats =
M.fromList
[ ('*', Bold),
('_', Italic),
('+', Underline),
('~', StrikeThrough),
('`', Snippet),
(secretMD, Secret),
(colorMD, Colored White)
]
colors :: Map Text Color
colors =
M.fromList
[ ("red", Red),
("green", Green),
("blue", Blue),
("yellow", Yellow),
("cyan", Cyan),
("magenta", Magenta),
("r", Red),
("g", Green),
("b", Blue),
("y", Yellow),
("c", Cyan),
("m", Magenta),
("1", Red),
("2", Green),
("3", Blue),
("4", Yellow),
("5", Cyan),
("6", Magenta)
]
parseMarkdown :: Text -> Markdown
parseMarkdown s = fromRight (unmarked s) $ A.parseOnly (markdownP <* A.endOfInput) s
markdownP :: Parser Markdown
markdownP = merge <$> A.many' fragmentP
where
merge :: [Markdown] -> Markdown
merge [] = ""
merge fs = foldr1 (:|:) fs
fragmentP :: Parser Markdown
fragmentP =
A.anyChar >>= \case
' ' -> unmarked . T.cons ' ' <$> A.takeWhile (== ' ')
c -> case M.lookup c formats of
Just Secret -> secretP
Just (Colored White) -> coloredP
Just f -> formattedP c "" f
Nothing -> unformattedP c
formattedP :: Char -> Text -> Format -> Parser Markdown
formattedP c p f = do
s <- A.takeTill (== c)
(A.char c $> markdown c p f s) <|> noFormat (c `T.cons` p <> s)
markdown :: Char -> Text -> Format -> Text -> Markdown
markdown c p f s
| T.null s || T.head s == ' ' || T.last s == ' ' =
unmarked $ c `T.cons` p <> s `T.snoc` c
| otherwise = Markdown f s
secretP :: Parser Markdown
secretP = secret <$> A.takeWhile (== secretMD) <*> A.takeTill (== secretMD) <*> A.takeWhile (== secretMD)
secret :: Text -> Text -> Text -> Markdown
secret b s a
| T.null a || T.null s || T.head s == ' ' || T.last s == ' ' =
unmarked $ secretMD `T.cons` ss
| otherwise = Markdown Secret $ T.init ss
where
ss = b <> s <> a
coloredP :: Parser Markdown
coloredP = do
color <- A.takeWhile (\c -> c /= ' ' && c /= colorMD)
case M.lookup color colors of
Just c ->
let f = Colored c
in (A.char ' ' *> formattedP colorMD (color `T.snoc` ' ') f)
<|> noFormat (colorMD `T.cons` color)
_ -> noFormat (colorMD `T.cons` color)
unformattedP :: Char -> Parser Markdown
unformattedP c = unmarked . T.cons c <$> wordsP
wordsP :: Parser Text
wordsP = do
s <- (<>) <$> A.takeTill (== ' ') <*> A.takeWhile (== ' ')
A.peekChar >>= \case
Nothing -> pure s
Just c -> case M.lookup c formats of
Just _ -> pure s
Nothing -> (s <>) <$> wordsP
noFormat :: Text -> Parser Markdown
noFormat = pure . unmarked

View File

@@ -8,6 +8,18 @@
{-# LANGUAGE RankNTypes #-}
{-# LANGUAGE ScopedTypeVariables #-}
-- |
-- Module : Simplex.Messaging.Agent
-- Copyright : (c) simplex.chat
-- License : AGPL-3
--
-- Maintainer : chat@simplex.chat
-- Stability : experimental
-- Portability : non-portable
--
-- This module defines SMP protocol agent with SQLite persistence.
--
-- See https://github.com/simplex-chat/simplexmq/blob/master/protocol/agent-protocol.md
module Simplex.Messaging.Agent
( runSMPAgent,
runSMPAgentBlocking,
@@ -32,9 +44,9 @@ import Data.Time.Clock
import Database.SQLite.Simple (SQLError)
import Simplex.Messaging.Agent.Client
import Simplex.Messaging.Agent.Env.SQLite
import Simplex.Messaging.Agent.Protocol
import Simplex.Messaging.Agent.Store
import Simplex.Messaging.Agent.Store.SQLite (SQLiteStore, connectSQLiteStore)
import Simplex.Messaging.Agent.Transmission
import Simplex.Messaging.Client (SMPServerTransmission)
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Protocol (CorrId (..), MsgBody, SenderPublicKey)
@@ -47,9 +59,16 @@ import UnliftIO.Async (race_)
import qualified UnliftIO.Exception as E
import UnliftIO.STM
-- | Runs an SMP agent as a TCP service using passed configuration.
--
-- See a full agent executable here: https://github.com/simplex-chat/simplexmq/blob/master/apps/smp-agent/Main.hs
runSMPAgent :: (MonadRandom m, MonadUnliftIO m) => AgentConfig -> m ()
runSMPAgent cfg = newEmptyTMVarIO >>= (`runSMPAgentBlocking` cfg)
-- | Runs an SMP agent as a TCP service using passed configuration with signalling.
--
-- This function uses passed TMVar to signal when the server is ready to accept TCP requests (True)
-- and when it is disconnected from the TCP socket once the server thread is killed (False).
runSMPAgentBlocking :: (MonadRandom m, MonadUnliftIO m) => TMVar Bool -> AgentConfig -> m ()
runSMPAgentBlocking started cfg@AgentConfig {tcpPort} = runReaderT smpAgent =<< newSMPAgentEnv cfg
where
@@ -61,6 +80,7 @@ runSMPAgentBlocking started cfg@AgentConfig {tcpPort} = runReaderT smpAgent =<<
race_ (connectClient h c) (runSMPAgentClient c)
`E.finally` (closeSMPServerClients c >> logConnection c False)
-- | Creates an SMP agent instance that receives commands and sends responses via 'TBQueue's.
getSMPAgentClient :: (MonadUnliftIO m, MonadReader Env m) => m AgentClient
getSMPAgentClient = do
n <- asks clientCounter
@@ -75,6 +95,7 @@ logConnection c connected =
let event = if connected then "connected to" else "disconnected from"
in logInfo $ T.unwords ["client", showText (clientId c), event, "Agent"]
-- | Runs an SMP agent instance that receives commands and sends responses via 'TBQueue's.
runSMPAgentClient :: (MonadUnliftIO m, MonadReader Env m) => AgentClient -> m ()
runSMPAgentClient c = do
db <- asks $ dbFile . config

View File

@@ -45,8 +45,8 @@ import qualified Data.Set as S
import Data.Text.Encoding
import Data.Time.Clock
import Simplex.Messaging.Agent.Env.SQLite
import Simplex.Messaging.Agent.Protocol
import Simplex.Messaging.Agent.Store
import Simplex.Messaging.Agent.Transmission
import Simplex.Messaging.Client
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Protocol (ErrorType (AUTH), MsgBody, QueueId, SenderPublicKey)

View File

@@ -10,8 +10,8 @@ import Crypto.Random
import Data.List.NonEmpty (NonEmpty)
import Network.Socket
import Numeric.Natural
import Simplex.Messaging.Agent.Protocol (SMPServer)
import Simplex.Messaging.Agent.Store.SQLite
import Simplex.Messaging.Agent.Transmission (SMPServer)
import Simplex.Messaging.Client
import System.Random (StdGen, newStdGen)
import UnliftIO.STM

View File

@@ -11,7 +11,68 @@
{-# LANGUAGE TypeFamilies #-}
{-# OPTIONS_GHC -fno-warn-unticked-promoted-constructors #-}
module Simplex.Messaging.Agent.Transmission where
-- |
-- Module : Simplex.Messaging.Agent.Protocol
-- Copyright : (c) simplex.chat
-- License : AGPL-3
--
-- Maintainer : chat@simplex.chat
-- Stability : experimental
-- Portability : non-portable
--
-- Types, parsers, serializers and functions to send and receive SMP agent protocol commands and responses.
--
-- See https://github.com/simplex-chat/simplexmq/blob/master/protocol/agent-protocol.md
module Simplex.Messaging.Agent.Protocol
( -- * SMP agent protocol types
ACommand (..),
AParty (..),
SAParty (..),
SMPMessage (..),
AMessage (..),
SMPServer (..),
SMPQueueInfo (..),
AgentErrorType (..),
CommandErrorType (..),
ConnectionErrorType (..),
BrokerErrorType (..),
SMPAgentError (..),
ATransmission,
ATransmissionOrError,
ARawTransmission,
ConnAlias,
ReplyMode (..),
AckMode (..),
OnOff (..),
MsgIntegrity (..),
MsgErrorType (..),
QueueStatus (..),
SignatureKey,
VerificationKey,
EncryptionKey,
DecryptionKey,
-- * Parse and serialize
serializeCommand,
serializeSMPMessage,
serializeMsgIntegrity,
serializeServer,
serializeSmpQueueInfo,
serializeAgentError,
commandP,
parseSMPMessage,
smpServerP,
smpQueueInfoP,
msgIntegrityP,
agentErrorTypeP,
-- * TCP transport functions
tPut,
tGet,
tPutRaw,
tGetRaw,
)
where
import Control.Applicative (optional, (<|>))
import Control.Monad.IO.Class
@@ -35,29 +96,33 @@ import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Parsers
import Simplex.Messaging.Protocol
( CorrId (..),
Encoded,
ErrorType,
MsgBody,
MsgId,
SenderPublicKey,
)
import qualified Simplex.Messaging.Protocol as SMP
import Simplex.Messaging.Transport
import Simplex.Messaging.Transport (TransportError, getLn, putLn, serializeTransportError, transportErrorP)
import Simplex.Messaging.Util
import System.IO
import Test.QuickCheck (Arbitrary (..))
import Text.Read
import UnliftIO.Exception
-- | Raw (unparsed) SMP agent protocol transmission.
type ARawTransmission = (ByteString, ByteString, ByteString)
-- | Parsed SMP agent protocol transmission.
type ATransmission p = (CorrId, ConnAlias, ACommand p)
-- | SMP agent protocol transmission or transmission error.
type ATransmissionOrError p = (CorrId, ConnAlias, Either AgentErrorType (ACommand p))
-- | SMP agent protocol participants.
data AParty = Agent | Client
deriving (Eq, Show)
-- | Singleton types for SMP agent protocol participants.
data SAParty :: AParty -> Type where
SAgent :: SAParty Agent
SClient :: SAParty Client
@@ -75,6 +140,7 @@ data ACmd = forall p. ACmd (SAParty p) (ACommand p)
deriving instance Show ACmd
-- | Parameterized type for SMP agent protocol commands and responses from all participants.
data ACommand (p :: AParty) where
NEW :: ACommand Client -- response INV
INV :: SMPQueueInfo -> ACommand Agent
@@ -109,24 +175,38 @@ deriving instance Eq (ACommand p)
deriving instance Show (ACommand p)
type Message = ByteString
-- | SMP message formats.
data SMPMessage
= SMPConfirmation SenderPublicKey
| SMPMessage
{ senderMsgId :: AgentMsgId,
= -- | SMP confirmation
-- (see <https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md#send-message SMP protocol>)
SMPConfirmation SenderPublicKey
| -- | Agent message header and envelope for client messages
-- (see <https://github.com/simplex-chat/simplexmq/blob/master/protocol/agent-protocol.md#messages-between-smp-agents SMP agent protocol>)
SMPMessage
{ -- | sequential ID assigned by the sending agent
senderMsgId :: AgentMsgId,
-- | timestamp from the sending agent
senderTimestamp :: SenderTimestamp,
-- | digest of the previous message
previousMsgHash :: ByteString,
-- | messages sent between agents once queue is secured
agentMessage :: AMessage
}
deriving (Show)
-- | Messages sent between SMP agents once SMP queue is secured.
--
-- https://github.com/simplex-chat/simplexmq/blob/master/protocol/agent-protocol.md#messages-between-smp-agents
data AMessage where
-- | the first message in the queue to validate it is secured
HELLO :: VerificationKey -> AckMode -> AMessage
-- | reply queue information
REPLY :: SMPQueueInfo -> AMessage
-- | agent envelope for the client message
A_MSG :: MsgBody -> AMessage
deriving (Show)
-- | Parse SMP message.
parseSMPMessage :: ByteString -> Either AgentErrorType SMPMessage
parseSMPMessage = parse (smpMessageP <* A.endOfLine) $ AGENT A_MESSAGE
where
@@ -148,6 +228,7 @@ parseSMPMessage = parse (smpMessageP <* A.endOfLine) $ AGENT A_MESSAGE
<*> (base64P <|> pure "") <* A.endOfLine
<*> agentMessageP
-- | Serialize SMP message.
serializeSMPMessage :: SMPMessage -> ByteString
serializeSMPMessage = \case
SMPConfirmation sKey -> smpMessage ("KEY " <> C.serializePubKey sKey) "" ""
@@ -173,10 +254,12 @@ agentMessageP =
A_MSG <$> A.take size <* A.endOfLine
ackMode = AckMode <$> (" NO_ACK" $> Off <|> pure On)
-- | SMP queue information parser.
smpQueueInfoP :: Parser SMPQueueInfo
smpQueueInfoP =
"smp::" *> (SMPQueueInfo <$> smpServerP <* "::" <*> base64P <* "::" <*> C.pubKeyP)
-- | SMP server location parser.
smpServerP :: Parser SMPServer
smpServerP = SMPServer <$> server <*> optional port <*> optional kHash
where
@@ -184,23 +267,23 @@ smpServerP = SMPServer <$> server <*> optional port <*> optional kHash
port = A.char ':' *> (B.unpack <$> A.takeWhile1 A.isDigit)
kHash = C.KeyHash <$> (A.char '#' *> base64P)
parseAgentMessage :: ByteString -> Either AgentErrorType AMessage
parseAgentMessage = parse agentMessageP $ AGENT A_MESSAGE
serializeAgentMessage :: AMessage -> ByteString
serializeAgentMessage = \case
HELLO verifyKey ackMode -> "HELLO " <> C.serializePubKey verifyKey <> if ackMode == AckMode Off then " NO_ACK" else ""
REPLY qInfo -> "REPLY " <> serializeSmpQueueInfo qInfo
A_MSG body -> "MSG " <> serializeMsg body <> "\n"
-- | Serialize SMP queue information that is sent out-of-band.
serializeSmpQueueInfo :: SMPQueueInfo -> ByteString
serializeSmpQueueInfo (SMPQueueInfo srv qId ek) =
B.intercalate "::" ["smp", serializeServer srv, encode qId, C.serializePubKey ek]
-- | Serialize SMP server location.
serializeServer :: SMPServer -> ByteString
serializeServer SMPServer {host, port, keyHash} =
B.pack $ host <> maybe "" (':' :) port <> maybe "" (('#' :) . B.unpack . encode . C.unKeyHash) keyHash
-- | SMP server location and transport key digest (hash).
data SMPServer = SMPServer
{ host :: HostName,
port :: Maybe ServiceName,
@@ -211,79 +294,128 @@ data SMPServer = SMPServer
instance IsString SMPServer where
fromString = parseString . parseAll $ smpServerP
-- | SMP agent connection alias.
type ConnAlias = ByteString
type OtherPartyId = Encoded
-- | Connection modes.
data OnOff = On | Off deriving (Eq, Show, Read)
-- | Message acknowledgement mode of the connection.
newtype AckMode = AckMode OnOff deriving (Eq, Show)
-- | SMP queue information sent out-of-band.
--
-- https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md#out-of-band-messages
data SMPQueueInfo = SMPQueueInfo SMPServer SMP.SenderId EncryptionKey
deriving (Eq, Show)
-- | Connection reply mode (used in JOIN command).
newtype ReplyMode = ReplyMode OnOff deriving (Eq, Show)
-- | Public key used to E2E encrypt SMP messages.
type EncryptionKey = C.PublicKey
-- | Private key used to E2E decrypt SMP messages.
type DecryptionKey = C.SafePrivateKey
-- | Private key used to sign SMP commands
type SignatureKey = C.SafePrivateKey
-- | Public key used by SMP server to authorize (verify) SMP commands.
type VerificationKey = C.PublicKey
data QueueDirection = SND | RCV deriving (Show)
data QueueStatus = New | Confirmed | Secured | Active | Disabled
-- | SMP queue status.
data QueueStatus
= -- | queue is created
New
| -- | queue is confirmed by the sender
Confirmed
| -- | queue is secured with sender key (only used by the queue recipient)
Secured
| -- | queue is active
Active
| -- | queue is disabled (only used by the queue recipient)
Disabled
deriving (Eq, Show, Read)
type AgentMsgId = Int64
type SenderTimestamp = UTCTime
-- | Result of received message integrity validation.
data MsgIntegrity = MsgOk | MsgError MsgErrorType
deriving (Eq, Show)
-- | Error of message integrity validation.
data MsgErrorType = MsgSkipped AgentMsgId AgentMsgId | MsgBadId AgentMsgId | MsgBadHash | MsgDuplicate
deriving (Eq, Show)
-- | error type used in errors sent to agent clients
-- | Error type used in errors sent to agent clients.
data AgentErrorType
= CMD CommandErrorType -- command errors
| CONN ConnectionErrorType -- connection state errors
| SMP ErrorType -- SMP protocol errors forwarded to agent clients
| BROKER BrokerErrorType -- SMP server errors
| AGENT SMPAgentError -- errors of other agents
| INTERNAL String -- agent implementation errors
= -- | command or response error
CMD CommandErrorType
| -- | connection errors
CONN ConnectionErrorType
| -- | SMP protocol errors forwarded to agent clients
SMP ErrorType
| -- | SMP server errors
BROKER BrokerErrorType
| -- | errors of other agents
AGENT SMPAgentError
| -- | agent implementation or dependency errors
INTERNAL String
deriving (Eq, Generic, Read, Show, Exception)
-- | SMP agent protocol command or response error.
data CommandErrorType
= PROHIBITED -- command is prohibited
| SYNTAX -- command syntax is invalid
| NO_CONN -- connection alias is required with this command
| SIZE -- message size is not correct (no terminating space)
| LARGE -- message does not fit SMP block
= -- | command is prohibited
PROHIBITED
| -- | command syntax is invalid
SYNTAX
| -- | connection alias is required with this command
NO_CONN
| -- | message size is not correct (no terminating space)
SIZE
| -- | message does not fit in SMP block
LARGE
deriving (Eq, Generic, Read, Show, Exception)
-- | Connection error.
data ConnectionErrorType
= UNKNOWN -- connection alias not in database
| DUPLICATE -- connection alias already exists
| SIMPLEX -- connection is simplex, but operation requires another queue
= -- | connection alias is not in the database
UNKNOWN
| -- | connection alias already exists
DUPLICATE
| -- | connection is simplex, but operation requires another queue
SIMPLEX
deriving (Eq, Generic, Read, Show, Exception)
-- | SMP server errors.
data BrokerErrorType
= RESPONSE ErrorType -- invalid server response (failed to parse)
| UNEXPECTED -- unexpected response
| NETWORK -- network error
| TRANSPORT TransportError -- handshake or other transport error
| TIMEOUT -- command response timeout
= -- | invalid server response (failed to parse)
RESPONSE ErrorType
| -- | unexpected response
UNEXPECTED
| -- | network error
NETWORK
| -- | handshake or other transport error
TRANSPORT TransportError
| -- | command response timeout
TIMEOUT
deriving (Eq, Generic, Read, Show, Exception)
-- | Errors of another SMP agent.
data SMPAgentError
= A_MESSAGE -- possibly should include bytestring that failed to parse
| A_PROHIBITED -- possibly should include the prohibited SMP/agent message
| A_ENCRYPTION -- cannot RSA/AES-decrypt or parse decrypted header
| A_SIGNATURE -- invalid RSA signature
= -- | possibly should include bytestring that failed to parse
A_MESSAGE
| -- | possibly should include the prohibited SMP/agent message
A_PROHIBITED
| -- | cannot RSA/AES-decrypt or parse decrypted header
A_ENCRYPTION
| -- | invalid RSA signature
A_SIGNATURE
deriving (Eq, Generic, Read, Show, Exception)
instance Arbitrary AgentErrorType where arbitrary = genericArbitraryU
@@ -296,6 +428,7 @@ instance Arbitrary BrokerErrorType where arbitrary = genericArbitraryU
instance Arbitrary SMPAgentError where arbitrary = genericArbitraryU
-- | AMP agent command and response parser
commandP :: Parser ACmd
commandP =
"NEW" $> ACmd SClient NEW
@@ -328,6 +461,7 @@ commandP =
partyMeta idParser = (,) <$> idParser <* "," <*> tsISO8601P <* A.space
agentError = ACmd SAgent . ERR <$> agentErrorTypeP
-- | Message integrity validation result parser.
msgIntegrityP :: Parser MsgIntegrity
msgIntegrityP = "OK" $> MsgOk <|> "ERR " *> (MsgError <$> msgErrorType)
where
@@ -340,6 +474,7 @@ msgIntegrityP = "OK" $> MsgOk <|> "ERR " *> (MsgError <$> msgErrorType)
parseCommand :: ByteString -> Either AgentErrorType ACmd
parseCommand = parse commandP $ CMD SYNTAX
-- | Serialize SMP agent command.
serializeCommand :: ACommand p -> ByteString
serializeCommand = \case
NEW -> "NEW"
@@ -372,6 +507,7 @@ serializeCommand = \case
showTs :: UTCTime -> ByteString
showTs = B.pack . formatISO8601Millis
-- | Serialize message integrity validation result.
serializeMsgIntegrity :: MsgIntegrity -> ByteString
serializeMsgIntegrity = \case
MsgOk -> "OK"
@@ -383,6 +519,7 @@ serializeMsgIntegrity = \case
MsgBadHash -> "HASH"
MsgDuplicate -> "DUPLICATE"
-- | SMP agent protocol error parser.
agentErrorTypeP :: Parser AgentErrorType
agentErrorTypeP =
"SMP " *> (SMP <$> SMP.errorTypeP)
@@ -391,6 +528,7 @@ agentErrorTypeP =
<|> "INTERNAL " *> (INTERNAL <$> parseRead A.takeByteString)
<|> parseRead2
-- | Serialize SMP agent protocol error.
serializeAgentError :: AgentErrorType -> ByteString
serializeAgentError = \case
SMP e -> "SMP " <> SMP.serializeErrorType e
@@ -401,20 +539,23 @@ serializeAgentError = \case
serializeMsg :: ByteString -> ByteString
serializeMsg body = bshow (B.length body) <> "\n" <> body
-- | Send raw (unparsed) SMP agent protocol transmission to TCP connection.
tPutRaw :: Handle -> ARawTransmission -> IO ()
tPutRaw h (corrId, connAlias, command) = do
putLn h corrId
putLn h connAlias
putLn h command
-- | Receive raw (unparsed) SMP agent protocol transmission from TCP connection.
tGetRaw :: Handle -> IO ARawTransmission
tGetRaw h = (,,) <$> getLn h <*> getLn h <*> getLn h
-- | Send SMP agent protocol command (or response) to TCP connection.
tPut :: MonadIO m => Handle -> ATransmission p -> m ()
tPut h (CorrId corrId, connAlias, command) =
liftIO $ tPutRaw h (corrId, connAlias, serializeCommand command)
-- | get client and agent transmissions
-- | Receive client and agent transmissions from TCP connection.
tGet :: forall m p. MonadIO m => SAParty p -> Handle -> m (ATransmissionOrError p)
tGet party h = liftIO (tGetRaw h) >>= tParseLoadBody
where

View File

@@ -15,7 +15,7 @@ import Data.Int (Int64)
import Data.Kind (Type)
import Data.Time (UTCTime)
import Data.Type.Equality
import Simplex.Messaging.Agent.Transmission
import Simplex.Messaging.Agent.Protocol
import Simplex.Messaging.Protocol
( MsgBody,
MsgId,
@@ -272,11 +272,21 @@ type InternalTs = UTCTime
-- * Store errors
-- | Agent store error.
data StoreError
= SEInternal ByteString
| SEConnNotFound
| SEConnDuplicate
| SEBadConnType ConnType
| SEBadQueueStatus -- not used, planned to check strictly
| SENotImplemented -- TODO remove
= -- | IO exceptions in store actions.
SEInternal ByteString
| -- | Connection alias not found (or both queues absent).
SEConnNotFound
| -- | Connection alias already used.
SEConnDuplicate
| -- | Wrong connection type, e.g. "send" connection when "receive" or "duplex" is expected, or vice versa.
-- 'upgradeRcvConnToDuplex' and 'upgradeSndConnToDuplex' do not allow duplex connections - they would also return this error.
SEBadConnType ConnType
| -- | Currently not used. The intention was to pass current expected queue status in methods,
-- as we always know what it should be at any stage of the protocol,
-- and in case it does not match use this error.
SEBadQueueStatus
| -- | Used in `getMsg` that is not implemented/used. TODO remove.
SENotImplemented
deriving (Eq, Show, Exception)

View File

@@ -38,9 +38,9 @@ import Database.SQLite.Simple.Ok (Ok (Ok))
import Database.SQLite.Simple.QQ (sql)
import Database.SQLite.Simple.ToField (ToField (..))
import Network.Socket (ServiceName)
import Simplex.Messaging.Agent.Protocol
import Simplex.Messaging.Agent.Store
import Simplex.Messaging.Agent.Store.SQLite.Schema (createSchema)
import Simplex.Messaging.Agent.Transmission
import Simplex.Messaging.Parsers (blobFieldParser)
import qualified Simplex.Messaging.Protocol as SMP
import Simplex.Messaging.Util (bshow, liftIOEither)

View File

@@ -8,18 +8,35 @@
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE ScopedTypeVariables #-}
-- |
-- Module : Simplex.Messaging.Client
-- Copyright : (c) simplex.chat
-- License : AGPL-3
--
-- Maintainer : chat@simplex.chat
-- Stability : experimental
-- Portability : non-portable
--
-- This module provides a functional client API for SMP protocol.
--
-- See https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md
module Simplex.Messaging.Client
( SMPClient (blockSize),
( -- * Connect (disconnect) client to (from) SMP server
SMPClient (blockSize),
getSMPClient,
closeSMPClient,
-- * SMP protocol command functions
createSMPQueue,
subscribeSMPQueue,
secureSMPQueue,
sendSMPMessage,
ackSMPMessage,
sendSMPCommand,
suspendSMPQueue,
deleteSMPQueue,
sendSMPCommand,
-- * Supporting types and client configuration
SMPClientError (..),
SMPClientConfig (..),
smpDefaultConfig,
@@ -40,14 +57,20 @@ import qualified Data.Map.Strict as M
import Data.Maybe
import Network.Socket (ServiceName)
import Numeric.Natural
import Simplex.Messaging.Agent.Transmission (SMPServer (..))
import Simplex.Messaging.Agent.Protocol (SMPServer (..))
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Protocol
import Simplex.Messaging.Transport
import Simplex.Messaging.Transport (THandle (..), TransportError, clientHandshake, runTCPClient)
import Simplex.Messaging.Util (bshow, liftError, raceAny_)
import System.IO
import System.Timeout
-- | 'SMPClient' is a handle used to send commands to a specific SMP server.
--
-- The only exported selector is blockSize that is negotiated
-- with the server during the TCP transport handshake.
--
-- Use 'getSMPClient' to connect to an SMP server and create a client handle.
data SMPClient = SMPClient
{ action :: Async (),
connected :: TVar Bool,
@@ -61,16 +84,25 @@ data SMPClient = SMPClient
blockSize :: Int
}
-- | Type synonym for transmission from some SPM server queue.
type SMPServerTransmission = (SMPServer, RecipientId, Command 'Broker)
-- | SMP client configuration.
data SMPClientConfig = SMPClientConfig
{ qSize :: Natural,
{ -- | size of TBQueue to use for server commands and responses
qSize :: Natural,
-- | default SMP server port if port is not specified in SMPServer
defaultPort :: ServiceName,
-- | timeout of TCP commands (microseconds)
tcpTimeout :: Int,
-- | period for SMP ping commands (microseconds)
smpPing :: Int,
-- | estimated maximum size of SMP command excluding message body,
-- determines the maximum allowed message size
smpCommandSize :: Int
}
-- | Default SMP client configuration.
smpDefaultConfig :: SMPClientConfig
smpDefaultConfig =
SMPClientConfig
@@ -88,6 +120,11 @@ data Request = Request
type Response = Either SMPClientError Cmd
-- | Connects to 'SMPServer' using passed client configuration
-- and queue for messages and notifications.
--
-- A single queue can be used for multiple 'SMPClient' instances,
-- as 'SMPServerTransmission' includes server information.
getSMPClient :: SMPServer -> SMPClientConfig -> TBQueue SMPServerTransmission -> IO () -> IO (Either SMPClientError SMPClient)
getSMPClient
smpServer@SMPServer {host, port, keyHash}
@@ -172,19 +209,39 @@ getSMPClient
Right r -> Right r
else Left SMPUnexpectedResponse
-- | Disconnects SMP client from the server and terminates client threads.
closeSMPClient :: SMPClient -> IO ()
closeSMPClient = uninterruptibleCancel . action
-- | SMP client error type.
data SMPClientError
= SMPServerError ErrorType
| SMPResponseError ErrorType
| SMPUnexpectedResponse
| SMPResponseTimeout
| SMPNetworkError
| SMPTransportError TransportError
| SMPSignatureError C.CryptoError
= -- | Correctly parsed SMP server ERR response.
-- This error is forwarded to the agent client as `ERR SMP err`.
SMPServerError ErrorType
| -- | Invalid server response that failed to parse.
-- Forwarded to the agent client as `ERR BROKER RESPONSE`.
SMPResponseError ErrorType
| -- | Different response from what is expected to a certain SMP command,
-- e.g. server should respond `IDS` or `ERR` to `NEW` command,
-- other responses would result in this error.
-- Forwarded to the agent client as `ERR BROKER UNEXPECTED`.
SMPUnexpectedResponse
| -- | Used for TCP connection and command response timeouts.
-- Forwarded to the agent client as `ERR BROKER TIMEOUT`.
SMPResponseTimeout
| -- | Failure to establish TCP connection.
-- Forwarded to the agent client as `ERR BROKER NETWORK`.
SMPNetworkError
| -- | TCP transport handshake or some other transport error.
-- Forwarded to the agent client as `ERR BROKER TRANSPORT e`.
SMPTransportError TransportError
| -- | Error when cryptographically "signing" the command.
SMPSignatureError C.CryptoError
deriving (Eq, Show, Exception)
-- | Create a new SMP queue.
--
-- https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md#create-queue-command
createSMPQueue ::
SMPClient ->
RecipientPrivateKey ->
@@ -196,6 +253,9 @@ createSMPQueue c rpKey rKey =
Cmd _ (IDS rId sId) -> return (rId, sId)
_ -> throwE SMPUnexpectedResponse
-- | Subscribe to the SMP queue.
--
-- https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md#subscribe-to-queue
subscribeSMPQueue :: SMPClient -> RecipientPrivateKey -> RecipientId -> ExceptT SMPClientError IO ()
subscribeSMPQueue c@SMPClient {smpServer, msgQ} rpKey rId =
sendSMPCommand c (Just rpKey) rId (Cmd SRecipient SUB) >>= \case
@@ -204,15 +264,24 @@ subscribeSMPQueue c@SMPClient {smpServer, msgQ} rpKey rId =
lift . atomically $ writeTBQueue msgQ (smpServer, rId, cmd)
_ -> throwE SMPUnexpectedResponse
-- | Secure the SMP queue by adding a sender public key.
--
-- https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md#secure-queue-command
secureSMPQueue :: SMPClient -> RecipientPrivateKey -> RecipientId -> SenderPublicKey -> ExceptT SMPClientError IO ()
secureSMPQueue c rpKey rId senderKey = okSMPCommand (Cmd SRecipient $ KEY senderKey) c rpKey rId
-- | Send SMP message.
--
-- https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md#send-message
sendSMPMessage :: SMPClient -> Maybe SenderPrivateKey -> SenderId -> MsgBody -> ExceptT SMPClientError IO ()
sendSMPMessage c spKey sId msg =
sendSMPCommand c spKey sId (Cmd SSender $ SEND msg) >>= \case
Cmd _ OK -> return ()
_ -> throwE SMPUnexpectedResponse
-- | Acknowledge message delivery (server deletes the message).
--
-- https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md#acknowledge-message-delivery
ackSMPMessage :: SMPClient -> RecipientPrivateKey -> QueueId -> ExceptT SMPClientError IO ()
ackSMPMessage c@SMPClient {smpServer, msgQ} rpKey rId =
sendSMPCommand c (Just rpKey) rId (Cmd SRecipient ACK) >>= \case
@@ -221,9 +290,16 @@ ackSMPMessage c@SMPClient {smpServer, msgQ} rpKey rId =
lift . atomically $ writeTBQueue msgQ (smpServer, rId, cmd)
_ -> throwE SMPUnexpectedResponse
-- | Irreversibly suspend SMP queue.
-- The existing messages from the queue will still be delivered.
--
-- https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md#suspend-queue
suspendSMPQueue :: SMPClient -> RecipientPrivateKey -> QueueId -> ExceptT SMPClientError IO ()
suspendSMPQueue = okSMPCommand $ Cmd SRecipient OFF
-- | Irreversibly delete SMP queue and all messages in it.
--
-- https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md#delete-queue
deleteSMPQueue :: SMPClient -> RecipientPrivateKey -> QueueId -> ExceptT SMPClientError IO ()
deleteSMPQueue = okSMPCommand $ Cmd SRecipient DEL
@@ -233,6 +309,7 @@ okSMPCommand cmd c pKey qId =
Cmd _ OK -> return ()
_ -> throwE SMPUnexpectedResponse
-- | Send any SMP command ('Cmd' type).
sendSMPCommand :: SMPClient -> Maybe C.SafePrivateKey -> QueueId -> Cmd -> ExceptT SMPClientError IO Cmd
sendSMPCommand SMPClient {sndQ, sentCommands, clientCorrId, tcpTimeout} pKey qId cmd = do
corrId <- lift_ getNextCorrId

View File

@@ -7,39 +7,50 @@
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE TypeApplications #-}
-- |
-- Module : Simplex.Messaging.Crypto
-- Copyright : (c) simplex.chat
-- License : AGPL-3
--
-- Maintainer : chat@simplex.chat
-- Stability : experimental
-- Portability : non-portable
--
-- This module provides cryptography implementation for SMP protocols based on
-- <https://hackage.haskell.org/package/cryptonite cryptonite package>.
module Simplex.Messaging.Crypto
( PrivateKey (rsaPrivateKey),
( -- * RSA keys
PrivateKey (rsaPrivateKey),
SafePrivateKey, -- constructor is not exported
FullPrivateKey (..),
PublicKey (..),
Signature (..),
CryptoError (..),
SafeKeyPair,
FullKeyPair,
Key (..),
IV (..),
KeyHash (..),
generateKeyPair,
publicKey,
publicKeySize,
validKeySize,
safePrivateKey,
sign,
verify,
-- * E2E hybrid encryption scheme
encrypt,
decrypt,
-- * RSA OAEP encryption
encryptOAEP,
decryptOAEP,
-- * RSA PSS signing
Signature (..),
sign,
verify,
-- * AES256 AEAD-GCM scheme
Key (..),
IV (..),
encryptAES,
decryptAES,
serializePrivKey,
serializePubKey,
encodePubKey,
publicKeyHash,
sha256Hash,
privKeyP,
pubKeyP,
binaryPubKeyP,
authTagSize,
authTagToBS,
bsToAuthTag,
@@ -47,6 +58,21 @@ module Simplex.Messaging.Crypto
randomIV,
aesKeyP,
ivP,
-- * Encoding of RSA keys
serializePrivKey,
serializePubKey,
encodePubKey,
publicKeyHash,
privKeyP,
pubKeyP,
binaryPubKeyP,
-- * SHA256 hash
sha256Hash,
-- * Cryptography error type
CryptoError (..),
)
where
@@ -83,15 +109,27 @@ import Network.Transport.Internal (decodeWord32, encodeWord32)
import Simplex.Messaging.Parsers (base64P, blobFieldParser, parseAll, parseString)
import Simplex.Messaging.Util (liftEitherError, (<$?>))
-- | A newtype of 'Crypto.PubKey.RSA.PublicKey'.
newtype PublicKey = PublicKey {rsaPublicKey :: R.PublicKey} deriving (Eq, Show)
-- | A newtype of 'Crypto.PubKey.RSA.PrivateKey', with PublicKey removed.
--
-- It is not possible to recover PublicKey from SafePrivateKey.
-- The constructor of this type is not exported.
newtype SafePrivateKey = SafePrivateKey {unPrivateKey :: R.PrivateKey} deriving (Eq, Show)
-- | A newtype of 'Crypto.PubKey.RSA.PrivateKey' (with PublicKey inside).
newtype FullPrivateKey = FullPrivateKey {unPrivateKey :: R.PrivateKey} deriving (Eq, Show)
-- | Type-class used for both private key types: SafePrivateKey and FullPrivateKey.
class PrivateKey k where
-- unwraps 'Crypto.PubKey.RSA.PrivateKey'
rsaPrivateKey :: k -> R.PrivateKey
-- equivalent to data type constructor, not exported
_privateKey :: R.PrivateKey -> k
-- smart constructor removing public key from SafePrivateKey but keeping it in FullPrivateKey
mkPrivateKey :: R.PrivateKey -> k
instance PrivateKey SafePrivateKey where
@@ -119,28 +157,39 @@ instance FromField SafePrivateKey where fromField = blobFieldParser binaryPrivKe
instance FromField PublicKey where fromField = blobFieldParser binaryPubKeyP
-- | Tuple of RSA 'PublicKey' and 'PrivateKey'.
type KeyPair k = (PublicKey, k)
-- | Tuple of RSA 'PublicKey' and 'SafePrivateKey'.
type SafeKeyPair = (PublicKey, SafePrivateKey)
-- | Tuple of RSA 'PublicKey' and 'FullPrivateKey'.
type FullKeyPair = (PublicKey, FullPrivateKey)
-- | RSA signature newtype.
newtype Signature = Signature {unSignature :: ByteString} deriving (Eq, Show)
instance IsString Signature where
fromString = Signature . fromString
newtype Verified = Verified ByteString deriving (Show)
-- | Various cryptographic or related errors.
data CryptoError
= RSAEncryptError R.Error
| RSADecryptError R.Error
| RSASignError R.Error
| AESCipherError CE.CryptoError
| CryptoIVError
| AESDecryptError
| CryptoLargeMsgError
| CryptoHeaderError String
= -- | RSA OAEP encryption error
RSAEncryptError R.Error
| -- | RSA OAEP decryption error
RSADecryptError R.Error
| -- | RSA PSS signature error
RSASignError R.Error
| -- | AES initialization error
AESCipherError CE.CryptoError
| -- | IV generation error
CryptoIVError
| -- | AES decryption error
AESDecryptError
| -- | message does not fit in SMP block
CryptoLargeMsgError
| -- | failure parsing RSA-encrypted message header
CryptoHeaderError String
deriving (Eq, Show, Exception)
pubExpRange :: Integer
@@ -152,6 +201,7 @@ aesKeySize = 256 `div` 8
authTagSize :: Int
authTagSize = 128 `div` 8
-- | Generate RSA key pair with either SafePrivateKey or FullPrivateKey.
generateKeyPair :: PrivateKey k => Int -> IO (KeyPair k)
generateKeyPair size = loop
where
@@ -187,10 +237,13 @@ data Header = Header
msgSize :: Int
}
-- | AES key newtype.
newtype Key = Key {unKey :: ByteString}
-- | IV bytes newtype.
newtype IV = IV {unIV :: ByteString}
-- | Key hash newtype.
newtype KeyHash = KeyHash {unKeyHash :: ByteString} deriving (Eq, Ord, Show)
instance IsString KeyHash where
@@ -200,9 +253,11 @@ instance ToField KeyHash where toField = toField . encode . unKeyHash
instance FromField KeyHash where fromField = blobFieldParser $ KeyHash <$> base64P
-- | Digest (hash) of binary X509 encoding of RSA public key.
publicKeyHash :: PublicKey -> KeyHash
publicKeyHash = KeyHash . sha256Hash . encodePubKey
-- | SHA256 digest.
sha256Hash :: ByteString -> ByteString
sha256Hash = BA.convert . (hash :: ByteString -> Digest SHA256)
@@ -218,15 +273,22 @@ headerP = do
msgSize <- fromIntegral . decodeWord32 <$> A.take 4
return Header {aesKey, ivBytes, authTag, msgSize}
-- | AES256 key parser.
aesKeyP :: Parser Key
aesKeyP = Key <$> A.take aesKeySize
-- | IV bytes parser.
ivP :: Parser IV
ivP = IV <$> A.take (ivSize @AES256)
parseHeader :: ByteString -> Either CryptoError Header
parseHeader = first CryptoHeaderError . parseAll headerP
-- * E2E hybrid encryption scheme
-- | E2E encrypt SMP agent messages.
--
-- https://github.com/simplex-chat/simplexmq/blob/master/rfcs/2021-01-26-crypto.md#e2e-encryption
encrypt :: PublicKey -> Int -> ByteString -> ExceptT CryptoError IO ByteString
encrypt k paddedSize msg = do
aesKey <- liftIO randomAesKey
@@ -236,6 +298,9 @@ encrypt k paddedSize msg = do
encHeader <- encryptOAEP k $ serializeHeader header
return $ encHeader <> msg'
-- | E2E decrypt SMP agent messages.
--
-- https://github.com/simplex-chat/simplexmq/blob/master/rfcs/2021-01-26-crypto.md#e2e-encryption
decrypt :: PrivateKey k => k -> ByteString -> ExceptT CryptoError IO ByteString
decrypt pk msg'' = do
let (encHeader, msg') = B.splitAt (privateKeySize pk) msg''
@@ -244,6 +309,9 @@ decrypt pk msg'' = do
msg <- decryptAES aesKey ivBytes msg' authTag
return $ B.take msgSize msg
-- | AEAD-GCM encryption.
--
-- Used as part of hybrid E2E encryption scheme and for SMP transport blocks encryption.
encryptAES :: Key -> IV -> Int -> ByteString -> ExceptT CryptoError IO (AES.AuthTag, ByteString)
encryptAES aesKey ivBytes paddedSize msg = do
aead <- initAEAD @AES256 aesKey ivBytes
@@ -255,6 +323,9 @@ encryptAES aesKey ivBytes paddedSize msg = do
| len >= paddedSize = throwE CryptoLargeMsgError
| otherwise = return (msg <> B.replicate (paddedSize - len) '#')
-- | AEAD-GCM decryption.
--
-- Used as part of hybrid E2E encryption scheme and for SMP transport blocks decryption.
decryptAES :: Key -> IV -> ByteString -> AES.AuthTag -> ExceptT CryptoError IO ByteString
decryptAES aesKey ivBytes msg authTag = do
aead <- initAEAD @AES256 aesKey ivBytes
@@ -267,9 +338,11 @@ initAEAD (Key aesKey) (IV ivBytes) = do
cipher <- AES.cipherInit aesKey
AES.aeadInit AES.AEAD_GCM cipher iv
-- | Random AES256 key.
randomAesKey :: IO Key
randomAesKey = Key <$> getRandomBytes aesKeySize
-- | Random IV bytes for AES256 encryption.
randomIV :: IO IV
randomIV = IV <$> getRandomBytes (ivSize @AES256)
@@ -282,9 +355,11 @@ makeIV bs = maybeError CryptoIVError $ AES.makeIV bs
maybeError :: CryptoError -> Maybe a -> ExceptT CryptoError IO a
maybeError e = maybe (throwE e) return
-- | Convert AEAD 'AuthTag' to ByteString.
authTagToBS :: AES.AuthTag -> ByteString
authTagToBS = B.pack . map w2c . BA.unpack . AES.unAuthTag
-- | Convert ByteString to AEAD 'AuthTag'.
bsToAuthTag :: ByteString -> AES.AuthTag
bsToAuthTag = AES.AuthTag . BA.pack . map c2w . B.unpack
@@ -294,11 +369,17 @@ cryptoFailable = liftEither . first AESCipherError . CE.eitherCryptoError
oaepParams :: OAEP.OAEPParams SHA256 ByteString ByteString
oaepParams = OAEP.defaultOAEPParams SHA256
-- | RSA OAEP encryption.
--
-- Used as part of hybrid E2E encryption scheme and for SMP transport handshake.
encryptOAEP :: PublicKey -> ByteString -> ExceptT CryptoError IO ByteString
encryptOAEP (PublicKey k) aesKey =
liftEitherError RSAEncryptError $
OAEP.encrypt oaepParams k aesKey
-- | RSA OAEP decryption.
--
-- Used as part of hybrid E2E encryption scheme and for SMP transport handshake.
decryptOAEP :: PrivateKey k => k -> ByteString -> ExceptT CryptoError IO ByteString
decryptOAEP pk encKey =
liftEitherError RSADecryptError $
@@ -307,30 +388,47 @@ decryptOAEP pk encKey =
pssParams :: PSS.PSSParams SHA256 ByteString ByteString
pssParams = PSS.defaultPSSParams SHA256
-- | RSA PSS message signing.
--
-- Used by SMP clients to sign SMP commands and by SMP agents to sign messages.
sign :: PrivateKey k => k -> ByteString -> ExceptT CryptoError IO Signature
sign pk msg = ExceptT $ bimap RSASignError Signature <$> PSS.signSafer pssParams (rsaPrivateKey pk) msg
-- | RSA PSS signature verification.
--
-- Used by SMP servers to authorize SMP commands and by SMP agents to verify messages.
verify :: PublicKey -> Signature -> ByteString -> Bool
verify (PublicKey k) (Signature sig) msg = PSS.verify pssParams k msg sig
-- | Base-64 X509 encoding of RSA public key.
--
-- Used as part of SMP queue information (out-of-band message).
serializePubKey :: PublicKey -> ByteString
serializePubKey = ("rsa:" <>) . encode . encodePubKey
-- | Base-64 PKCS8 encoding of PSA private key.
--
-- Not used as part of SMP protocols.
serializePrivKey :: PrivateKey k => k -> ByteString
serializePrivKey = ("rsa:" <>) . encode . encodePrivKey
-- Base-64 X509 RSA public key parser.
pubKeyP :: Parser PublicKey
pubKeyP = decodePubKey <$?> ("rsa:" *> base64P)
-- Binary X509 RSA public key parser.
binaryPubKeyP :: Parser PublicKey
binaryPubKeyP = decodePubKey <$?> A.takeByteString
-- Base-64 PKCS8 RSA private key parser.
privKeyP :: PrivateKey k => Parser k
privKeyP = decodePrivKey <$?> ("rsa:" *> base64P)
-- Binary PKCS8 RSA private key parser.
binaryPrivKeyP :: PrivateKey k => Parser k
binaryPrivKeyP = decodePrivKey <$?> A.takeByteString
-- | Construct 'SafePrivateKey' from three numbers - used internally and in the tests.
safePrivateKey :: (Int, Integer, Integer) -> SafePrivateKey
safePrivateKey = SafePrivateKey . safeRsaPrivateKey
@@ -351,21 +449,25 @@ safeRsaPrivateKey (size, n, d) =
private_qinv = 0
}
-- Binary X509 encoding of 'PublicKey'.
encodePubKey :: PublicKey -> ByteString
encodePubKey = encodeKey . PubKeyRSA . rsaPublicKey
-- Binary PKCS8 encoding of 'PrivateKey'.
encodePrivKey :: PrivateKey k => k -> ByteString
encodePrivKey = encodeKey . PrivKeyRSA . rsaPrivateKey
encodeKey :: ASN1Object a => a -> ByteString
encodeKey k = toStrict . encodeASN1 DER $ toASN1 k []
-- Decoding of binary X509 'PublicKey'.
decodePubKey :: ByteString -> Either String PublicKey
decodePubKey =
decodeKey >=> \case
(PubKeyRSA k, []) -> Right $ PublicKey k
r -> keyError r
-- Decoding of binary PKCS8 'PrivateKey'.
decodePrivKey :: PrivateKey k => ByteString -> Either String k
decodePrivKey =
decodeKey >=> \case

View File

@@ -10,7 +10,58 @@
{-# LANGUAGE UndecidableInstances #-}
{-# OPTIONS_GHC -fno-warn-unticked-promoted-constructors #-}
module Simplex.Messaging.Protocol where
-- |
-- Module : Simplex.Messaging.Protocol
-- Copyright : (c) simplex.chat
-- License : AGPL-3
--
-- Maintainer : chat@simplex.chat
-- Stability : experimental
-- Portability : non-portable
--
-- Types, parsers, serializers and functions to send and receive SMP protocol commands and responses.
--
-- See https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md
module Simplex.Messaging.Protocol
( -- * SMP protocol types
Command (..),
Party (..),
Cmd (..),
SParty (..),
ErrorType (..),
CommandError (..),
Transmission,
SignedTransmission,
SignedTransmissionOrError,
RawTransmission,
SignedRawTransmission,
CorrId (..),
QueueId,
RecipientId,
SenderId,
RecipientPrivateKey,
RecipientPublicKey,
SenderPrivateKey,
SenderPublicKey,
Encoded,
MsgId,
MsgBody,
-- * Parse and serialize
serializeTransmission,
serializeCommand,
serializeErrorType,
transmissionP,
commandP,
errorTypeP,
-- * TCP transport functions
tPut,
tGet,
fromClient,
fromServer,
)
where
import Control.Applicative ((<|>))
import Control.Monad
@@ -29,13 +80,15 @@ import GHC.Generics (Generic)
import Generic.Random (genericArbitraryU)
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Parsers
import Simplex.Messaging.Transport
import Simplex.Messaging.Transport (THandle, TransportError (..), tGetEncrypted, tPutEncrypted)
import Simplex.Messaging.Util
import Test.QuickCheck (Arbitrary (..))
-- | SMP protocol participants.
data Party = Broker | Recipient | Sender
deriving (Show)
-- | Singleton types for SMP protocol participants.
data SParty :: Party -> Type where
SBroker :: SParty Broker
SRecipient :: SParty Recipient
@@ -43,28 +96,38 @@ data SParty :: Party -> Type where
deriving instance Show (SParty a)
-- | Type for command or response of any participant.
data Cmd = forall a. Cmd (SParty a) (Command a)
deriving instance Show Cmd
-- | SMP transmission without signature.
type Transmission = (CorrId, QueueId, Cmd)
-- | SMP transmission with signature.
type SignedTransmission = (C.Signature, Transmission)
type TransmissionOrError = (CorrId, QueueId, Either ErrorType Cmd)
-- | signed parsed transmission, with parsing error.
type SignedTransmissionOrError = (C.Signature, TransmissionOrError)
-- | unparsed SMP transmission with signature.
type RawTransmission = (ByteString, ByteString, ByteString, ByteString)
-- | unparsed SMP transmission with signature.
type SignedRawTransmission = (C.Signature, ByteString)
-- | SMP queue ID for the recipient.
type RecipientId = QueueId
-- | SMP queue ID for the sender.
type SenderId = QueueId
-- | SMP queue ID on the server.
type QueueId = Encoded
-- | Parameterized type for SMP protocol commands from all participants.
data Command (a :: Party) where
-- SMP recipient commands
NEW :: RecipientPublicKey -> Command Recipient
@@ -88,50 +151,76 @@ deriving instance Show (Command a)
deriving instance Eq (Command a)
-- | Base-64 encoded string.
type Encoded = ByteString
-- newtype to avoid accidentally changing order of transmission parts
-- | Transmission correlation ID.
--
-- A newtype to avoid accidentally changing order of transmission parts.
newtype CorrId = CorrId {bs :: ByteString} deriving (Eq, Ord, Show)
instance IsString CorrId where
fromString = CorrId . fromString
-- only used by Agent, kept here so its definition is close to respective public key
-- | Recipient's private key used by the recipient to authorize (sign) SMP commands.
--
-- Only used by SMP agent, kept here so its definition is close to respective public key.
type RecipientPrivateKey = C.SafePrivateKey
-- | Recipient's public key used by SMP server to verify authorization of SMP commands.
type RecipientPublicKey = C.PublicKey
-- only used by Agent, kept here so its definition is close to respective public key
-- | Sender's private key used by the recipient to authorize (sign) SMP commands.
--
-- Only used by SMP agent, kept here so its definition is close to respective public key.
type SenderPrivateKey = C.SafePrivateKey
-- | Sender's public key used by SMP server to verify authorization of SMP commands.
type SenderPublicKey = C.PublicKey
-- | SMP message server ID.
type MsgId = Encoded
-- | SMP message body.
type MsgBody = ByteString
-- | Type for protocol errors.
data ErrorType
= BLOCK
| CMD CommandError
| AUTH
| NO_MSG
| INTERNAL
| DUPLICATE_ -- TODO remove, not part of SMP protocol
= -- | incorrect block format, encoding or signature size
BLOCK
| -- | SMP command is unknown or has invalid syntax
CMD CommandError
| -- | command authorization error - bad signature or non-existing SMP queue
AUTH
| -- | ACK command is sent without message to be acknowledged
NO_MSG
| -- | internal server error
INTERNAL
| -- | used internally, never returned by the server (to be removed)
DUPLICATE_ -- TODO remove, not part of SMP protocol
deriving (Eq, Generic, Read, Show)
-- | SMP command error type.
data CommandError
= PROHIBITED
| KEY_SIZE
| SYNTAX
| NO_AUTH
| HAS_AUTH
| NO_QUEUE
= -- | server response sent from client or vice versa
PROHIBITED
| -- | bad RSA key size in NEW or KEY commands (only 1024, 2048 and 4096 bits keys are allowed)
KEY_SIZE
| -- | error parsing command
SYNTAX
| -- | transmission has no required credentials (signature or queue ID)
NO_AUTH
| -- | transmission has credentials that are not allowed for this command
HAS_AUTH
| -- | transmission has no required queue ID
NO_QUEUE
deriving (Eq, Generic, Read, Show)
instance Arbitrary ErrorType where arbitrary = genericArbitraryU
instance Arbitrary CommandError where arbitrary = genericArbitraryU
-- | SMP transmission parser.
transmissionP :: Parser RawTransmission
transmissionP = do
signature <- segment
@@ -142,6 +231,7 @@ transmissionP = do
where
segment = A.takeTill (== ' ') <* " "
-- | SMP command parser.
commandP :: Parser Cmd
commandP =
"NEW " *> newCmd
@@ -173,9 +263,12 @@ commandP =
serverError = Cmd SBroker . ERR <$> errorTypeP
-- TODO ignore the end of block, no need to parse it
-- | Parse SMP command.
parseCommand :: ByteString -> Either ErrorType Cmd
parseCommand = parse (commandP <* " " <* A.takeByteString) $ CMD SYNTAX
-- | Serialize SMP command.
serializeCommand :: Cmd -> ByteString
serializeCommand = \case
Cmd SRecipient (NEW rKey) -> "NEW " <> C.serializePubKey rKey
@@ -191,35 +284,44 @@ serializeCommand = \case
where
serializeMsg msgBody = bshow (B.length msgBody) <> " " <> msgBody <> " "
-- | SMP error parser.
errorTypeP :: Parser ErrorType
errorTypeP = "CMD " *> (CMD <$> parseRead1) <|> parseRead1
-- | Serialize SMP error.
serializeErrorType :: ErrorType -> ByteString
serializeErrorType = bshow
-- | Send signed SMP transmission to TCP transport.
tPut :: THandle -> SignedRawTransmission -> IO (Either TransportError ())
tPut th (C.Signature sig, t) =
tPutEncrypted th $ encode sig <> " " <> t <> " "
-- | Serialize SMP transmission.
serializeTransmission :: Transmission -> ByteString
serializeTransmission (CorrId corrId, queueId, command) =
B.intercalate " " [corrId, encode queueId, serializeCommand command]
-- | Validate that it is an SMP client command, used with 'tGet' by 'Simplex.Messaging.Server'.
fromClient :: Cmd -> Either ErrorType Cmd
fromClient = \case
Cmd SBroker _ -> Left $ CMD PROHIBITED
cmd -> Right cmd
-- | Validate that it is an SMP server command, used with 'tGet' by 'Simplex.Messaging.Client'.
fromServer :: Cmd -> Either ErrorType Cmd
fromServer = \case
cmd@(Cmd SBroker _) -> Right cmd
_ -> Left $ CMD PROHIBITED
-- | Receive and parse transmission from the TCP transport.
tGetParse :: THandle -> IO (Either TransportError RawTransmission)
tGetParse th = (>>= parse transmissionP TEBadBlock) <$> tGetEncrypted th
-- | get client and server transmissions
-- `fromParty` is used to limit allowed senders - `fromClient` or `fromServer` should be used
-- | Receive client and server transmissions.
--
-- The first argument is used to limit allowed senders.
-- 'fromClient' or 'fromServer' should be used here.
tGet :: forall m. MonadIO m => (Cmd -> Either ErrorType Cmd) -> THandle -> m SignedTransmissionOrError
tGet fromParty th = liftIO (tGetParse th) >>= decodeParseValidate
where

View File

@@ -10,8 +10,20 @@
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE TupleSections #-}
-- TODO move randomBytes to another module
module Simplex.Messaging.Server (runSMPServer, runSMPServerBlocking, randomBytes) where
-- |
-- Module : Simplex.Messaging.Server
-- Copyright : (c) simplex.chat
-- License : AGPL-3
--
-- Maintainer : chat@simplex.chat
-- Stability : experimental
-- Portability : non-portable
--
-- This module defines SMP protocol server with in-memory persistence
-- and optional append only log of SMP queue records.
--
-- See https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md
module Simplex.Messaging.Server (runSMPServer, runSMPServerBlocking) where
import Control.Concurrent.STM (stateTVar)
import Control.Monad
@@ -40,9 +52,16 @@ import UnliftIO.Exception
import UnliftIO.IO
import UnliftIO.STM
-- | Runs an SMP server using passed configuration.
--
-- See a full server here: https://github.com/simplex-chat/simplexmq/blob/master/apps/smp-server/Main.hs
runSMPServer :: (MonadRandom m, MonadUnliftIO m) => ServerConfig -> m ()
runSMPServer cfg = newEmptyTMVarIO >>= (`runSMPServerBlocking` cfg)
-- | Runs an SMP server using passed configuration with signalling.
--
-- This function uses passed TMVar to signal when the server is ready to accept TCP requests (True)
-- and when it is disconnected from the TCP socket once the server thread is killed (False).
runSMPServerBlocking :: (MonadRandom m, MonadUnliftIO m) => TMVar Bool -> ServerConfig -> m ()
runSMPServerBlocking started cfg@ServerConfig {tcpPort} = do
env <- newEnv cfg

View File

@@ -10,7 +10,37 @@
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE ScopedTypeVariables #-}
module Simplex.Messaging.Transport where
-- |
-- Module : Simplex.Messaging.Transport
-- Copyright : (c) simplex.chat
-- License : AGPL-3
--
-- Maintainer : chat@simplex.chat
-- Stability : experimental
-- Portability : non-portable
--
-- This module defines basic TCP server and client and SMP protocol encrypted transport over TCP.
--
-- See https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md#appendix-a
module Simplex.Messaging.Transport
( -- * TCP transport
runTCPServer,
runTCPClient,
putLn,
getLn,
trimCR,
-- * SMP encrypted transport
THandle (..),
TransportError (..),
serverHandshake,
clientHandshake,
tPutEncrypted,
tGetEncrypted,
serializeTransportError,
transportErrorP,
)
where
import Control.Applicative ((<|>))
import Control.Monad.Except
@@ -47,6 +77,9 @@ import UnliftIO.STM
-- * TCP transport
-- | Run TCP server on passed port and signal when server started and stopped via passed TMVar.
--
-- All accepted TCP connection handles are passed to the passed function.
runTCPServer :: MonadUnliftIO m => TMVar Bool -> ServiceName -> (Handle -> m ()) -> m ()
runTCPServer started port server = do
clients <- newTVarIO S.empty
@@ -79,6 +112,7 @@ startTCPServer started port = withSocketsDo $ resolve >>= open >>= setStarted
acceptTCPConn :: Socket -> IO Handle
acceptTCPConn sock = accept sock >>= getSocketHandle . fst
-- | Connect to passed TCP host:port and pass handle to the client.
runTCPClient :: MonadUnliftIO m => HostName -> ServiceName -> (Handle -> m a) -> m a
runTCPClient host port client = do
h <- liftIO $ startTCPClient host port
@@ -114,17 +148,20 @@ getSocketHandle conn = do
hSetBuffering h LineBuffering
return h
-- | Send ByteString to TCP connection handle terminating it with CRLF.
putLn :: Handle -> ByteString -> IO ()
putLn h = B.hPut h . (<> "\r\n")
-- | Receive ByteString from TCP connection handle, allowing LF or CRLF termination.
getLn :: Handle -> IO ByteString
getLn h = trimCR <$> B.hGetLine h
-- | Trim trailing CR from ByteString.
trimCR :: ByteString -> ByteString
trimCR "" = ""
trimCR s = if B.last s == '\r' then B.init s else s
-- * Encrypted transport
-- * SMP encrypted transport
data SMPVersion = SMPVersion Int Int Int Int
deriving (Eq, Ord)
@@ -143,6 +180,7 @@ smpVersionP =
let ver = A.decimal <* A.char '.'
in SMPVersion <$> ver <*> ver <*> ver <*> A.decimal
-- | The handle for SMP encrypted transport connection over TCP.
data THandle = THandle
{ handle :: Handle,
sndKey :: SessionKey,
@@ -162,29 +200,45 @@ data ClientHandshake = ClientHandshake
rcvKey :: SessionKey
}
-- | Error of SMP encrypted transport over TCP.
data TransportError
= TEBadBlock
| TEEncrypt
| TEDecrypt
| TEHandshake HandshakeError
= -- | error parsing transport block
TEBadBlock
| -- | block encryption error
TEEncrypt
| -- | block decryption error
TEDecrypt
| -- | transport handshake error
TEHandshake HandshakeError
deriving (Eq, Generic, Read, Show, Exception)
-- | Transport handshake error.
data HandshakeError
= ENCRYPT
| DECRYPT
| VERSION
| RSA_KEY
| HEADER
| AES_KEYS
| BAD_HASH
| MAJOR_VERSION
| TERMINATED
= -- | encryption error
ENCRYPT
| -- | decryption error
DECRYPT
| -- | error parsing protocol version
VERSION
| -- | error parsing RSA key
RSA_KEY
| -- | error parsing server transport header or invalid block size
HEADER
| -- | error parsing AES keys
AES_KEYS
| -- | not matching RSA key hash
BAD_HASH
| -- | lower major agent version than protocol version
MAJOR_VERSION
| -- | TCP transport terminated
TERMINATED
deriving (Eq, Generic, Read, Show, Exception)
instance Arbitrary TransportError where arbitrary = genericArbitraryU
instance Arbitrary HandshakeError where arbitrary = genericArbitraryU
-- | SMP encrypted transport error parser.
transportErrorP :: Parser TransportError
transportErrorP =
"BLOCK" $> TEBadBlock
@@ -192,6 +246,7 @@ transportErrorP =
<|> "AES_DECRYPT" $> TEDecrypt
<|> TEHandshake <$> parseRead1
-- | Serialize SMP encrypted transport error.
serializeTransportError :: TransportError -> ByteString
serializeTransportError = \case
TEEncrypt -> "AES_ENCRYPT"
@@ -199,12 +254,14 @@ serializeTransportError = \case
TEBadBlock -> "BLOCK"
TEHandshake e -> bshow e
-- | Encrypt and send block to SMP encrypted transport.
tPutEncrypted :: THandle -> ByteString -> IO (Either TransportError ())
tPutEncrypted THandle {handle = h, sndKey, blockSize} block =
encryptBlock sndKey (blockSize - C.authTagSize) block >>= \case
Left _ -> pure $ Left TEEncrypt
Right (authTag, msg) -> Right <$> B.hPut h (C.authTagToBS authTag <> msg)
-- | Receive and decrypt block from SMP encrypted transport.
tGetEncrypted :: THandle -> IO (Either TransportError ByteString)
tGetEncrypted THandle {handle = h, rcvKey, blockSize} =
B.hGet h blockSize >>= decryptBlock rcvKey >>= \case
@@ -232,8 +289,11 @@ makeNextIV SessionKey {baseIV, counter} = atomically $ do
(start, rest) = B.splitAt 4 $ C.unIV baseIV
iv c = C.IV $ (start `xor` encodeWord32 c) <> rest
-- | implements server transport handshake as per /rfcs/2021-01-26-crypto.md#transport-encryption
-- The numbers in function names refer to the steps in the document
-- | Server SMP encrypted transport handshake.
--
-- See https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md#appendix-a
--
-- The numbers in function names refer to the steps in the document.
serverHandshake :: Handle -> C.FullKeyPair -> ExceptT TransportError IO THandle
serverHandshake h (k, pk) = do
liftIO sendHeaderAndPublicKey_1
@@ -262,8 +322,11 @@ serverHandshake h (k, pk) = do
sendWelcome_6 :: THandle -> ExceptT TransportError IO ()
sendWelcome_6 th = ExceptT . tPutEncrypted th $ serializeSMPVersion currentSMPVersion <> " "
-- | implements client transport handshake as per /rfcs/2021-01-26-crypto.md#transport-encryption
-- The numbers in function names refer to the steps in the document
-- | Client SMP encrypted transport handshake.
--
-- See https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md#appendix-a
--
-- The numbers in function names refer to the steps in the document.
clientHandshake :: Handle -> Maybe C.KeyHash -> ExceptT TransportError IO THandle
clientHandshake h keyHash = do
(k, blkSize) <- getHeaderAndPublicKey_1_2

View File

@@ -1,99 +0,0 @@
# Errors
## Problems
- using numbers and strings to indicate errors (in protocol and in code) - ErrorType, AgentErrorType, TransportError
- re-using the same type in multiple contexts (with some constructors not applicable to all contexts) - ErrorType
## Error types
### ErrorType (Protocol.hs)
- BLOCK - incorrect block format, encoding or signature size
- CMD error - command is unknown or has invalid syntax, where `error` can be:
- PROHIBITED - server response sent from client or vice versa
- KEY_SIZE - bad RSA key size in NEW or KEY commands (only 1024, 2048 and 4096 bits keys are allowed)
- SYNTAX - error parsing command
- NO_AUTH - transmission has no required credentials (signature or queue ID)
- HAS_AUTH - transmission has not allowed credentials
- NO_QUEUE - transmission has not queue ID
- AUTH - command is not authorised (queue does not exist or signature verification failed).
- NO_MSG - acknowledging (ACK) the message without message
- INTERNAL - internal server error.
- DUPLICATE_ - it is used internally to signal that the queue ID is already used. This is NOT used in the protocol, instead INTERNAL is sent to the client. It has to be removed.
### AgentErrorType (Agent/Transmission.hs)
Some of these errors are not correctly serialized/parsed - see line 322 in Agent/Transmission.hs
- CMD e - command or response error
- PROHIBITED - server response sent as client command (and vice versa)
- SYNTAX - command is unknown or has invalid syntax.
- NO_CONN - connection is required in the command (and absent)
- SIZE - incorrect message size of messages (when parsing SEND and MSG)
- LARGE -- message does not fit SMP block
- CONN e - connection errors
- UNKNOWN - connection alias not in database
- DUPLICATE - connection alias already exists
- SIMPLEX - connection is simplex, but operation requires another queue
- SMP ErrorType - forwarding SMP errors (SMPServerError) to the agent client
- BROKER e - SMP server errors
- RESPONSE ErrorType - invalid SMP server response
- UNEXPECTED - unexpected response
- NETWORK - network TCP connection error
- TRANSPORT TransportError -- handshake or other transport error
- TIMEOUT - command response timeout
- AGENT e - errors of other agents
- A_MESSAGE - SMP message failed to parse
- A_PROHIBITED - SMP message is prohibited with the current queue status
- A_ENCRYPTION - cannot RSA/AES-decrypt or parse decrypted header
- A_SIGNATURE - invalid RSA signature
- INTERNAL ByteString - agent implementation or dependency error
### SMPClientError (Client.hs)
- SMPServerError ErrorType - this is correctly parsed server ERR response. This error is forwarded to the agent client as `ERR SMP err`
- SMPResponseError ErrorType - this is invalid server response that failed to parse - forwarded to the client as `ERR BROKER RESPONSE`.
- SMPUnexpectedResponse - different response from what is expected to a given command, e.g. server should respond `IDS` or `ERR` to `NEW` command, other responses would result in this error - forwarded to the client as `ERR BROKER UNEXPECTED`.
- SMPResponseTimeout - used for TCP connection and command response timeouts -> `ERR BROKER TIMEOUT`.
- SMPNetworkError - fails to establish TCP connection -> `ERR BROKER NETWORK`
- SMPTransportError e - fails connection handshake or some other transport error -> `ERR BROKER TRANSPORT e`
- SMPSignatureError C.CryptoError - error when cryptographically "signing" the command.
### StoreError (Agent/Store.hs)
- SEInternal ByteString - signals exceptions in store actions.
- SEConnNotFound - connection alias not found (or both queues absent).
- SEConnDuplicate - connection alias already used.
- SEBadConnType ConnType - wrong connection type, e.g. "send" connection when "receive" or "duplex" is expected, or vice versa. `updateRcvConnWithSndQueue` and `updateSndConnWithRcvQueue` do not allow duplex connections - they would also return this error.
- SEBadQueueStatus - the intention was to pass current expected queue status in methods, as we always know what it should be at any stage of the protocol, and in case it does not match use this error. **Currently not used**.
- SENotImplemented - used in `getMsg` that is not implemented/used.
### CryptoError (Crypto.hs)
- RSAEncryptError R.Error - RSA encryption error
- RSADecryptError R.Error - RSA decryption error
- RSASignError R.Error - RSA signature error
- AESCipherError CE.CryptoError - AES initialization error
- CryptoIVError - IV generation error
- AESDecryptError - AES decryption error
- CryptoLargeMsgError - message does not fit in SMP block
- CryptoHeaderError String - failure parsing RSA-encrypted message header
### TransportError (Transport.hs)
- TEBadBlock - error parsing block
- TEEncrypt - block encryption error
- TEDecrypt - block decryption error
- TEHandshake HandshakeError
### HandshakeError (Transport.hs)
- ENCRYPT - encryption error
- DECRYPT - decryption error
- VERSION - error parsing protocol version
- RSA_KEY - error parsing RSA key
- AES_KEYS - error parsing AES keys
- BAD_HASH - not matching RSA key hash
- MAJOR_VERSION - lower agent version than protocol version
- TERMINATED - transport terminated

View File

@@ -13,7 +13,7 @@ import Control.Concurrent
import Data.ByteString.Char8 (ByteString)
import qualified Data.ByteString.Char8 as B
import SMPAgentClient
import Simplex.Messaging.Agent.Transmission
import Simplex.Messaging.Agent.Protocol
import Simplex.Messaging.Protocol (ErrorType (..), MsgBody)
import System.IO (Handle)
import System.Timeout

View File

@@ -19,9 +19,9 @@ import Data.Word (Word32)
import qualified Database.SQLite.Simple as DB
import Database.SQLite.Simple.QQ (sql)
import SMPClient (testKeyHash)
import Simplex.Messaging.Agent.Protocol
import Simplex.Messaging.Agent.Store
import Simplex.Messaging.Agent.Store.SQLite
import Simplex.Messaging.Agent.Transmission
import qualified Simplex.Messaging.Crypto as C
import System.Random (Random (randomIO))
import Test.Hspec

View File

@@ -1,128 +0,0 @@
{-# LANGUAGE BlockArguments #-}
{-# LANGUAGE OverloadedStrings #-}
module MarkdownTests where
import Data.Text (Text)
import Simplex.Markdown
import System.Console.ANSI.Types
import Test.Hspec
markdownTests :: Spec
markdownTests = do
textFormat
secretText
textColor
textFormat :: Spec
textFormat = describe "text format (bold)" do
it "correct markdown" do
parseMarkdown "this is *bold formatted* text"
`shouldBe` "this is " <> Markdown Bold "bold formatted" <> " " <> "text"
parseMarkdown "*bold formatted* text"
`shouldBe` Markdown Bold "bold formatted" <> " " <> "text"
parseMarkdown "this is *bold*"
`shouldBe` "this is " <> Markdown Bold "bold"
parseMarkdown " *bold* text"
`shouldBe` " " <> Markdown Bold "bold" <> " " <> "text"
parseMarkdown " *bold* text"
`shouldBe` " " <> Markdown Bold "bold" <> " " <> "text"
parseMarkdown "this is *bold* "
`shouldBe` "this is " <> Markdown Bold "bold" <> " "
parseMarkdown "this is *bold* "
`shouldBe` "this is " <> Markdown Bold "bold" <> " "
it "ignored as markdown" do
parseMarkdown "this is * unformatted * text"
`shouldBe` "this is " <> "* unformatted *" <> " " <> "text"
parseMarkdown "this is *unformatted * text"
`shouldBe` "this is " <> "*unformatted *" <> " " <> "text"
parseMarkdown "this is * unformatted* text"
`shouldBe` "this is " <> "* unformatted*" <> " " <> "text"
parseMarkdown "this is **unformatted** text"
`shouldBe` "this is " <> "**" <> "unformatted** text"
parseMarkdown "this is*unformatted* text"
`shouldBe` "this is*unformatted* text"
parseMarkdown "this is *unformatted text"
`shouldBe` "this is " <> "*unformatted text"
it "ignored internal markdown" do
parseMarkdown "this is *long _bold_ (not italic)* text"
`shouldBe` "this is " <> Markdown Bold "long _bold_ (not italic)" <> " " <> "text"
parseMarkdown "snippet: `this is *bold text*`"
`shouldBe` "snippet: " <> Markdown Snippet "this is *bold text*"
secretText :: Spec
secretText = describe "secret text" do
it "correct markdown" do
parseMarkdown "this is #black_secret# text"
`shouldBe` "this is " <> Markdown Secret "black_secret" <> " " <> "text"
parseMarkdown "##black_secret### text"
`shouldBe` Markdown Secret "#black_secret##" <> " " <> "text"
parseMarkdown "this is #black secret# text"
`shouldBe` "this is " <> Markdown Secret "black secret" <> " " <> "text"
parseMarkdown "##black secret### text"
`shouldBe` Markdown Secret "#black secret##" <> " " <> "text"
parseMarkdown "this is #secret#"
`shouldBe` "this is " <> Markdown Secret "secret"
parseMarkdown " #secret# text"
`shouldBe` " " <> Markdown Secret "secret" <> " " <> "text"
parseMarkdown " #secret# text"
`shouldBe` " " <> Markdown Secret "secret" <> " " <> "text"
parseMarkdown "this is #secret# "
`shouldBe` "this is " <> Markdown Secret "secret" <> " "
parseMarkdown "this is #secret# "
`shouldBe` "this is " <> Markdown Secret "secret" <> " "
it "ignored as markdown" do
parseMarkdown "this is # unformatted # text"
`shouldBe` "this is " <> "# unformatted #" <> " " <> "text"
parseMarkdown "this is #unformatted # text"
`shouldBe` "this is " <> "#unformatted #" <> " " <> "text"
parseMarkdown "this is # unformatted# text"
`shouldBe` "this is " <> "# unformatted#" <> " " <> "text"
parseMarkdown "this is ## unformatted ## text"
`shouldBe` "this is " <> "## unformatted ##" <> " " <> "text"
parseMarkdown "this is#unformatted# text"
`shouldBe` "this is#unformatted# text"
parseMarkdown "this is #unformatted text"
`shouldBe` "this is " <> "#unformatted text"
it "ignored internal markdown" do
parseMarkdown "snippet: `this is #secret_text#`"
`shouldBe` "snippet: " <> Markdown Snippet "this is #secret_text#"
red :: Text -> Markdown
red = Markdown (Colored Red)
textColor :: Spec
textColor = describe "text color (red)" do
it "correct markdown" do
parseMarkdown "this is !1 red color! text"
`shouldBe` "this is " <> red "red color" <> " " <> "text"
parseMarkdown "!1 red! text"
`shouldBe` red "red" <> " " <> "text"
parseMarkdown "this is !1 red!"
`shouldBe` "this is " <> red "red"
parseMarkdown " !1 red! text"
`shouldBe` " " <> red "red" <> " " <> "text"
parseMarkdown " !1 red! text"
`shouldBe` " " <> red "red" <> " " <> "text"
parseMarkdown "this is !1 red! "
`shouldBe` "this is " <> red "red" <> " "
parseMarkdown "this is !1 red! "
`shouldBe` "this is " <> red "red" <> " "
it "ignored as markdown" do
parseMarkdown "this is !1 unformatted ! text"
`shouldBe` "this is " <> "!1 unformatted !" <> " " <> "text"
parseMarkdown "this is !1 unformatted ! text"
`shouldBe` "this is " <> "!1 unformatted !" <> " " <> "text"
parseMarkdown "this is !1 unformatted! text"
`shouldBe` "this is " <> "!1 unformatted!" <> " " <> "text"
-- parseMarkdown "this is !!1 unformatted!! text"
-- `shouldBe` "this is " <> "!!1" <> "unformatted!! text"
parseMarkdown "this is!1 unformatted! text"
`shouldBe` "this is!1 unformatted! text"
parseMarkdown "this is !1 unformatted text"
`shouldBe` "this is " <> "!1 unformatted text"
it "ignored internal markdown" do
parseMarkdown "this is !1 long *red* (not bold)! text"
`shouldBe` "this is " <> red "long *red* (not bold)" <> " " <> "text"
parseMarkdown "snippet: `this is !1 red text!`"
`shouldBe` "snippet: " <> Markdown Snippet "this is !1 red text!"

View File

@@ -1,6 +1,6 @@
module ProtocolErrorTests where
import Simplex.Messaging.Agent.Transmission (AgentErrorType, agentErrorTypeP, serializeAgentError)
import Simplex.Messaging.Agent.Protocol (AgentErrorType, agentErrorTypeP, serializeAgentError)
import Simplex.Messaging.Parsers (parseAll)
import Simplex.Messaging.Protocol (ErrorType, errorTypeP, serializeErrorType)
import Test.Hspec

View File

@@ -21,7 +21,7 @@ import SMPClient
)
import Simplex.Messaging.Agent (runSMPAgentBlocking)
import Simplex.Messaging.Agent.Env.SQLite
import Simplex.Messaging.Agent.Transmission
import Simplex.Messaging.Agent.Protocol
import Simplex.Messaging.Client (SMPClientConfig (..), smpDefaultConfig)
import Simplex.Messaging.Transport
import Test.Hspec

View File

@@ -1,5 +1,4 @@
import AgentTests
import MarkdownTests
import ProtocolErrorTests
import ServerTests
import System.Directory (createDirectoryIfMissing, removeDirectoryRecursive)
@@ -9,7 +8,6 @@ main :: IO ()
main = do
createDirectoryIfMissing False "tests/tmp"
hspec $ do
describe "SimpleX markdown" markdownTests
describe "Protocol errors" protocolErrorTests
describe "SMP server" serverTests
describe "SMP client agent" agentTests