simplexmq/spec/topics/patterns.md

Code Patterns

Cross-cutting patterns used throughout the codebase: exception handling, encoding utilities, compression, concurrent data structures, and batch processing. These patterns provide consistency, type safety, and correctness guarantees across all modules.

For protocol-specific encoding details, see transport.md. For cryptographic operations, see the inline documentation in Crypto.hs.

• Exception handling
• Binary encoding
• String encoding
• Compression
• Concurrent data structures
• Batch processing
• Time encoding
• Utilities

---

Exception handling

Source: Agent/Protocol.hs, Agent/Client.hs, Agent/Store/SQLite.hs

Error type hierarchy

The codebase uses a hierarchical error type structure:

AgentErrorType - top-level error type for agent client responses:

• CMD - command/response errors with context string
• CONN - connection errors with context (NOT_FOUND, DUPLICATE, SIMPLEX)
• SMP/NTF/XFTP - protocol-specific errors with server address
• BROKER - transport-level broker errors
• AGENT - internal agent errors (A_DUPLICATE, A_PROHIBITED)
• INTERNAL - implementation bugs (should never occur in production)
• CRITICAL - critical errors with optional restart offer

StoreError - database/storage layer errors:

• SEInternal - IO exceptions during database operations
• SEDatabaseBusy - database locked/busy (triggers CRITICAL with restart)
• SEConnNotFound/SEUserNotFound - entity lookup failures
• SEBadConnType - wrong connection type for operation

AgentCryptoError - cryptographic failures:

• DECRYPT_AES/DECRYPT_CB - decryption failures
• RATCHET_HEADER/RATCHET_EARLIER Word32/RATCHET_SKIPPED Word32/RATCHET_SYNC - double ratchet state issues

Monad stack

AM a  = ExceptT AgentErrorType (ReaderT Env IO) a  -- full error handling
AM' a = ReaderT Env IO a                            -- no error handling (for batch ops)

Store access patterns

Basic operations lift IO actions into the AM monad:

withStore :: AgentClient -> (DB.Connection -> IO (Either StoreError a)) -> AM a
withStore' :: AgentClient -> (DB.Connection -> IO a) -> AM a  -- wraps result in Right

Both wrap the action in a database transaction and convert StoreError to AgentErrorType via storeError.

Error mapping (key cases from storeError):

• SEConnNotFound/SERatchetNotFound -> CONN NOT_FOUND
• SEConnDuplicate -> CONN DUPLICATE
• SEBadConnType -> CONN SIMPLEX with context
• SEUserNotFound -> NO_USER
• SEAgentError e -> e (propagates wrapped error)
• SEDatabaseBusy -> CRITICAL True (offers restart)
• Other errors -> INTERNAL with error message

Error recovery patterns

tryError - attempt an operation, handle failure without throwing:

tryError (deleteQueue c NRMBackground rq') >>= \case
  Left e -> logError e >> continue
  Right () -> success

catchAllErrors - catch errors and run cleanup:

getQueueMessage c rq `catchAllErrors` \e ->
  atomically (releaseGetLock c rq) >> throwError e

catchAll_ - catch all exceptions, return default on failure:

notices <- liftIO $ withTransaction store (`getClientNotices` servers) `catchAll_` pure []

---

Binary encoding

Source: Encoding.hs

Encoding typeclass

class Encoding a where
  smpEncode :: a -> ByteString      -- encode to binary
  smpP :: Parser a                   -- attoparsec parser
  smpDecode :: ByteString -> Either String a  -- default via parseAll smpP

Primitive encoding

Type	Wire format
Char	Single byte
Bool	'T' or 'F'
Word16	2-byte big-endian
Word32	4-byte big-endian
Int64	Two Word32s combined
ByteString	1-byte length prefix + data (max 255 bytes)
Maybe a	'0' (Nothing) or '1' + encoded value
(a, b)	Concatenated encodings (no separator)

Special wrappers

Tail - takes remaining bytes without length prefix:

newtype Tail = Tail {unTail :: ByteString}
-- smpEncode = unTail (no prefix)
-- smpP = takeByteString

Large - for ByteStrings > 255 bytes:

newtype Large = Large {unLarge :: ByteString}
-- smpEncode = Word16 length prefix + data
-- smpP = read Word16, take that many bytes

List encoding

smpEncodeList :: Encoding a => [a] -> ByteString
-- 1-byte count prefix + concatenated encoded items (max 255 items)

instance Encoding (NonEmpty a)
-- Same format, fails on empty input during parsing

---

String encoding

Source: Encoding/String.hs

StrEncoding typeclass

class StrEncoding a where
  strEncode :: a -> ByteString      -- human-readable encoding
  strP :: Parser a                   -- parser (defaults to base64url)
  strDecode :: ByteString -> Either String a

Used for addresses, keys, and values displayed to users or in URIs.

Base64 URL encoding

ByteString instances use base64url encoding (RFC 4648):

• Alphabet: A-Z, a-z, 0-9, -, _
• No padding by default in output
• Accepts optional = padding on input

Tuple and list encoding

Tuples use space separation (via B.unwords):

strEncode (a, b) = B.unwords [strEncode a, strEncode b]

Lists use comma separation:

strEncodeList :: StrEncoding a => [a] -> ByteString
strEncodeList = B.intercalate "," . map strEncode

Numeric types

Int, Word16, Word32, Int64 encode as decimal strings (not binary).

JSON conversion utilities

strToJSON :: StrEncoding a => a -> J.Value
strParseJSON :: StrEncoding a => String -> J.Value -> JT.Parser a

Convert between StrEncoding and JSON string values for API serialization.

---

Compression

Source: Compression.hs

Algorithm and thresholds

Uses Zstandard (zstd) compression at level 3 (moderate compression/speed tradeoff).

maxLengthPassthrough :: Int
maxLengthPassthrough = 180  -- messages <= 180 bytes are not compressed

Wire format

data Compressed
  = Passthrough ByteString  -- tag '0' + 1-byte length + data
  | Compressed Large        -- tag '1' + 2-byte length + zstd data

Decompression bomb protection

decompress1 requires the compressed data to declare its decompressed size upfront:

decompress1 :: Int -> Compressed -> Either String ByteString

The function checks Z1.decompressedSize before decompressing. If the declared size exceeds the limit parameter (or is not specified), decompression is rejected. This prevents zip-bomb attacks where a small compressed payload would expand to exhaust memory.

Zstd's decompress can return Error, Skip (empty result), or Decompress bs' - all cases are handled explicitly.

---

Concurrent data structures

Source: TMap.hs

TMap

A TVar-wrapped immutable Data.Map, providing atomic read-modify-write operations via STM:

type TMap k a = TVar (Map k a)

STM operations (atomic)

Operation	Description
lookup k m	Read value for key
member k m	Check key existence
insert k v m	Insert/update value
delete k m	Remove key
lookupInsert k v m	Atomic lookup-then-insert, returns old value
lookupDelete k m	Atomic lookup-then-delete, returns deleted value

IO operations (non-transactional)

lookupIO :: Ord k => k -> TMap k a -> IO (Maybe a)
memberIO :: Ord k => k -> TMap k a -> IO Bool

These bypass STM for read-only access when atomicity with other operations is not needed.

Usage pattern

-- Within STM transaction (atomic with other STM ops)
atomically $ do
  existing <- TM.lookup key map
  case existing of
    Nothing -> TM.insert key newValue map
    Just _ -> pure ()

-- Outside transaction (simple read)
value <- TM.lookupIO key map

---

Batch processing

Source: Agent/Client.hs

withStoreBatch

Executes multiple database operations in a single transaction:

withStoreBatch :: Traversable t
  => AgentClient
  -> (DB.Connection -> t (IO (Either AgentErrorType a)))
  -> AM' (t (Either AgentErrorType a))

All operations run within one database transaction, ensuring:

• Atomicity: All operations succeed or all fail together
• Isolation: No partial updates visible to other readers
• Efficiency: Single transaction overhead instead of per-operation

Result semantics

Each batched operation produces an individual Either AgentErrorType a:

• Partial success is possible (some Right, some Left)
• If the transaction itself fails, all results become errors
• Fine-grained error handling per operation

Common patterns

Store multiple items:

void $ withStoreBatch' c $ \db ->
  map (storeDelivery db) deliveries

Fetch multiple items:

results <- withStoreBatch c $ \db ->
  map (getConnection db) connIds

Update multiple items:

void $ withStoreBatch' c $ \db ->
  map (\connId -> setConnPQSupport db connId PQSupportOn) connIds

withStoreBatch'

Convenience variant that wraps results in Right:

withStoreBatch' :: Traversable t
  => AgentClient
  -> (DB.Connection -> t (IO a))
  -> AM' (t (Either AgentErrorType a))

Use when operations cannot fail (or failures should become INTERNAL errors).

---

Time encoding

Source: SystemTime.hs

RoundedSystemTime t uses a phantom type-level Nat for precision. SystemDate (precision 86400) provides k-anonymity for file creation times - all timestamps within a day collapse to the same value, preventing correlation attacks.

---

Utilities

Source: Util.hs

Functor combinators: <$$> (double fmap), <$$ (double fmap const), and <$?> (fmap with MonadFail on Left) are used throughout for nested functor manipulation and fallible parsing chains.

threadDelay': Handles Int64 delays that exceed maxBound::Int by looping with maxBound-sized chunks.