smp web: client with tests (#1782)

* smp web: client with tests

* fixes

* support batching and SMP proxy

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
This commit is contained in:
Evgeny
2026-05-22 09:59:31 +01:00
committed by GitHub
parent 34e3d30c78
commit e56c12ab3c
8 changed files with 2052 additions and 57 deletions
@@ -0,0 +1,355 @@
# SMP Client for Browser
**Parent**: [SMP Agent Web Spike](./2026-03-20-smp-agent-web-spike.md)
**Depends on**: Spike 1 (merged) — transport, ratchet, encoding, per-queue E2E
## Context
The encoding spike proved all four encryption layers work cross-language. The next implementable and testable piece is the SMP client — the layer that sends commands, correlates responses by CorrId, authenticates with entity keys, and exposes typed async functions.
Faithful transpilation of `Simplex.Messaging.Client` (Client.hs). Transport is WebSocket (already working), protocol logic is identical to Haskell.
## Encoding path (per command)
Traced from `sendSMPMessage` through every function call:
```
1. encodeTransmission_(v, (corrId, entityId, command))
→ smpEncode(corrId, entityId) <> encodeProtocol(v, cmd)
Already have as encodeTransmission() in protocol.ts — update in place
2. encodeTransmissionForAuth(thParams, transmission)
→ tForAuth = smpEncode(sessionId) <> encodeTransmission_(...)
→ tToSend = encodeTransmission_(...) [when implySessId=true, which is always true for v>=7]
Note: implySessId means tToSend omits sessionId, but tForAuth includes it (for signing)
3. authTransmission(thAuth, serviceAuth=false, maybePrivKey, nonce, tForAuth)
→ thAuth contains serverPubKey (X25519) from handshake
→ maybePrivKey is Nothing for unauthenticated commands (LGET, SEND without key)
→ Nothing privKey: no auth, encode empty ByteString
→ Just X25519 privKey: TAAuthenticator(cbAuthenticate(serverPubKey, privKey, nonce, tForAuth))
→ Just Ed25519 privKey: TASignature(sign(privKey, tForAuth))
Note: nonce IS the CorrId (same 24 bytes used for both)
Note: serviceAuth is always false for browser client (no service certificates)
4. tEncodeAuth(serviceAuth=false, maybeAuth)
→ Nothing: smpEncode("") [1-byte 0x00]
→ Just (TAAuthenticator s, _): smpEncode(s) [1-byte len + 80 bytes]
→ Just (TASignature sig, _): smpEncode(signatureBytes sig) [1-byte len + 64 bytes]
Note: TAuthorizations = (TransmissionAuth, Maybe serviceSig) — serviceSig always Nothing for us
5. tEncode(serviceAuth, (auth, tToSend))
→ tEncodeAuth(auth) <> tToSend
6. tEncodeBatch1(serviceAuth, sentRawTransmission)
→ lenEncode(1) + smpEncode(Large(tEncode(...)))
Single-command batch. Always used when batch=true (v7+).
7. batchTransmissions_(blockSize, transmissions)
→ Pack multiple Large-wrapped transmissions into ≤blockSize blocks
→ Count byte prefix, up to 255 per block
→ blockSize' = blockSize - 19 (2 pad + 1 count + 16 auth tag)
```
## Parsing path (per received block)
```
1. tParse(thParams, blockBytes)
→ batch=true: parse count byte, then N Large-wrapped transmissions
→ Each: transmissionP(thParams) parses:
- authenticator (ByteString, 1-byte len + data) — ignored by client
- rest = authorized bytes
- re-parse authorized: corrId (ByteString) + entityId (ByteString) + command (rest)
- if implySessId=true: sessionId not in wire format, prepended from thParams for verification
→ Returns RawTransmission{authenticator, corrId, entityId, command}
2. tDecodeClient(thParams, rawTransmission)
→ Verify sessId matches (skipped when implySessId=true)
→ parseProtocol(v, command) → Either ErrorType BrokerMsg
→ Return (corrId, entityId, Right msg | Left err)
3. clientResp classification (Client.hs:708-712):
→ Left err (parse error) → PCEResponseError
→ Right msg, protocolError msg = Just err → PCEProtocolError (ERR response)
→ Right msg, protocolError msg = Nothing → Right msg (success)
4. Process: lookup corrId in pendingCommands
→ Found: resolve Promise with clientResp
→ Not found (empty corrId = server push): deliver to event callback
```
## Functions to implement
### Crypto (`src/crypto.ts` — extend)
| Function | Haskell | Implementation |
|---|---|---|
| `sha512Hash(msg)` | `Crypto.hs:1016` | `sha512(msg)` from `@noble/hashes/sha512` |
| `cbAuthenticator(serverPubKey, entityPrivKey, nonce, msg)` | `Crypto.hs:1367` | `cryptoBox(dh(serverPubKey, privKey), nonce, sha512Hash(msg))` → 80 bytes (16 tag + 64 hash) |
`cryptoBox` and `dh` already available from xftp-web. `sha512` from `@noble/hashes`.
Not needed in spike: `cbDecryptNoPad` (only used by `cbVerify` and proxy commands).
Ed25519 signing: `crypto_sign_detached` from libsodium (already loaded and initialized via xftp-web for secretbox — no second implementation needed).
Not needed: `cbVerify` (server-side only).
### Transport update (`src/transport/websockets.ts` — update)
**Gap: `connectSMP` must return `serverPubKey`** (raw X25519 public key bytes from the handshake). Currently it computes the DH secret and derives block keys, but discards the server's raw public key. The client needs it for `cbAuthenticate` on every command.
Update `SMPConnection` to include:
```typescript
interface SMPConnection {
ws: WebSocket
sessionId: Uint8Array
smpVersion: number
sndKey: Uint8Array | null
rcvKey: Uint8Array | null
serverPubKey: Uint8Array | null // raw X25519 public key — needed for command auth
}
```
### Protocol encoding (`src/protocol.ts` — update existing)
Update `encodeTransmission`, `encodeBatch`, `decodeTransmission` in place — these were spike throwaway. Replace with auth-aware versions and update existing tests accordingly.
| Function | Haskell ref | Notes |
|---|---|---|
| `encodeTransmission_(v, corrId, entityId, command)` | `Protocol.hs:2194` | Update existing `encodeTransmission`. Also fix `encodeNEW`: QueueReqData should be `Just (QRMessaging Nothing)` not `Nothing`, and rename `sndAuthKey` param to `basicAuth` (it's server auth, not a crypto key) |
| `encodeTransmissionForAuth(sessionId, corrId, entityId, command)` | `Protocol.hs:2186` | Returns `{tForAuth, tToSend}`. `implySessId` always true for v>=7 |
| `authTransmission(serverPubKey, maybePrivKey, nonce, tForAuth)` | `Client.hs:1372` | `maybePrivKey` is `{type: "x25519"|"ed25519", key} | null`. Null for unauthenticated commands. X25519 → cbAuthenticator. Ed25519 → sign. |
| `tEncodeAuth(auth)` | `Protocol.hs:507` | Handles null, authenticator (80 bytes), signature (64 bytes) |
| `tEncode(auth, tToSend)` | `Protocol.hs:2171` | `tEncodeAuth(auth) + tToSend` |
| `tEncodeBatch1(auth, tToSend)` | `Protocol.hs:2179` | `[count=1] + Large(tEncode(...))` |
| `tEncodeForBatch(auth, tToSend)` | `Protocol.hs:2175` | `Large(tEncode(...))` |
| `batchTransmissions(blockSize, transmissions)` | `Protocol.hs:2151` | Pack into ≤(blockSize-19)-byte blocks, count prefix |
| `transmissionP(sessionId, block)` | `Protocol.hs:1629` | Skip auth bytes (1-byte len + data), parse corrId + entityId + command from rest. `implySessId`=true (sessionId not in wire, no need to verify on client side), `serviceAuth`=false (no serviceSig to skip) |
| `tParse(sessionId, block)` | `Protocol.hs:2211` | Parse count, N×Large, each through `transmissionP` |
| `tDecodeClient(sessionId, version, rawTransmission)` | `Protocol.hs:2256` | Parse command bytes → typed BrokerMsg |
| `encodePING()` | | PING command for keepalive |
Update `decodeResponse`:
- Add `SOK` (subscribe response with optional serviceId, returned by SUB in v19)
- Add `INFO` (queue info response, for `getSMPQueueInfo`)
- Improve `ERR` parsing: currently reads just the tag string. Need to parse structured `ErrorType` (at minimum AUTH, QUOTA, NO_MSG, INTERNAL) for proper error handling in the client
### Client (`src/client.ts` — new)
```typescript
interface SMPClient {
sessionId: Uint8Array
smpVersion: number
serverPubKey: Uint8Array // for cbAuthenticate
// Core: send pre-encoded command, correlate response
// Lower-level than Haskell's sendProtocolCommand — takes pre-encoded command bytes
// privKey: {type: "x25519", key} | {type: "ed25519", key} | null
// Rejects with PCEProtocolError (ERR response), PCEResponseError (parse fail), PCEResponseTimeout
sendCommand(privKey: AuthKey | null, entityId: Uint8Array, command: Uint8Array): Promise<BrokerMsg>
// High-level commands (keys are DER-encoded unless noted)
// authKeyPair: {publicKey, privateKey, type: "x25519"} — public goes in NEW encoding, private for auth
createQueue(authKeyPair, dhKey, subMode): Promise<QueueIdsKeys>
subscribeQueue(privKey, rcvId): Promise<void> // SUB can return MSG (queued message) → pushed to onMessage
sendMessage(privKey, sndId, flags, msg): Promise<void> // privKey can be null (before queue secured)
ackMessage(privKey, rcvId, msgId): Promise<void> // ACK can return MSG → pushed to onMessage
secureQueue(privKey, rcvId, senderKey): Promise<void>
secureSndQueue(privKey, sndId): Promise<void>
getQueueLink(linkId): Promise<{senderId, linkData}>
getQueueInfo(privKey, queueId): Promise<QueueInfo>
deleteQueue(privKey, rcvId): Promise<void>
suspendQueue(privKey, rcvId): Promise<void>
close(): void
}
function createSMPClient(
url: string,
keyHash: Uint8Array,
onMessage: (entityId: Uint8Array, msg: BrokerMsg) => void,
onDisconnected: () => void,
wsOptions?: object,
): Promise<SMPClient>
```
Internally:
- `connectSMP` for WebSocket + handshake (existing, updated to return serverPubKey)
- `Map<string, {resolve, reject}>` for hex(corrId) → Promise correlation
- WebSocket `onmessage`: `receiveEncryptedBlock``tParse` → for each transmission: `tDecodeClient` → classify via `protocolError` → correlate by corrId or push to `onMessage`
- `sendCommand`: generate random 24-byte corrId/nonce → `encodeTransmissionForAuth``authTransmission``tEncodeBatch1``sendEncryptedBlock` → return Promise resolved by correlator
- `setInterval` ping: send PING, count timeouts, close after N consecutive
- Timeout per command: `setTimeout` on pending Promise, reject with PCEResponseTimeout
**Message delivery model:**
All MSGs reach `onMessage` regardless of how they arrive. Three sources:
1. **Server push** (empty corrId): receive handler calls `onMessage` directly
2. **SUB response**: `sendCommand` resolves with MSG → `subscribeQueue` pushes to `onMessage`, returns success to caller
3. **ACK response**: same — `ackMessage` pushes to `onMessage`, returns success
High-level functions never expose MSG to their callers. This mirrors Haskell's `processSUBResponse_` (Client.hs:858-862) and `ackSMPMessage` (Client.hs:1042-1044) which both call `writeSMPMessage` to forward MSGs to msgQ and return OK-equivalent.
### Client REPL (`smp-web/tests/client-repl.ts` — new, separate from ratchet-repl.ts)
Separate REPL process holding a WebSocket connection + SMP client state. Same stdin/stdout line protocol approach, different state and commands.
**Message queue:** The REPL maintains an internal `Message[]` queue. The SMPClient's `onMessage` callback pushes to this queue. MSGs arrive here from three sources: server pushes (no corrId), SUB responses, and ACK responses — all handled identically by the client internals. The `RECV` command dequeues from this queue (or waits with timeout).
**Concurrency:** Unlike the ratchet REPL (pure, no network), the client REPL receives messages concurrently with stdin. This works because Node's event loop handles WebSocket `onmessage` events between readline callbacks — no explicit threading needed.
```
CONNECT <url> <keyHashHex> [wsOptions]
→ Creates SMPClient, returns "ok"
NEW <rcvAuthKeyHex> <rcvDhKeyHex> [subMode]
→ createQueue (defaults: no basic auth, SMSubscribe, QRMessaging, no ntf creds)
→ returns "ok: <rcvIdHex> <sndIdHex> <srvDhKeyHex>"
SUB <rcvIdHex> <rcvPrivKeyHex>
→ subscribeQueue, returns "ok"
SEND <sndIdHex> <sndPrivKeyHex> <bodyHex>
→ sendMessage, returns "ok"
ACK <rcvIdHex> <rcvPrivKeyHex> <msgIdHex>
→ ackMessage, returns "ok"
KEY <rcvIdHex> <rcvPrivKeyHex> <senderKeyHex>
→ secureQueue, returns "ok"
SKEY <sndIdHex> <sndPrivKeyHex>
→ secureSndQueue, returns "ok"
LGET <linkIdHex>
→ getQueueLink, returns "ok: <senderIdHex> <linkDataHex>"
RECV [timeoutMs]
→ Dequeue next server-pushed MSG, returns "ok: <entityIdHex> <msgIdHex> <bodyHex>"
→ Times out with "error: timeout" if no message arrives
```
### Polymorphic testing
Same pattern as ratchet tests: `TestPeer` sum type with `TestPeerHS` / `TestPeerJS` dispatch. For SMP client tests:
```haskell
data TestSMPClient
= TestClientHS SMPClient
| TestClientJS Handle Handle ProcessHandle -- stdin, stdout, process
-- Dispatch functions
tcCreateQueue :: TestSMPClient -> ... -> IO QueueIdsKeys
tcSubscribe :: TestSMPClient -> ... -> IO ()
tcSendMessage :: TestSMPClient -> ... -> IO ()
tcReceiveMessage :: TestSMPClient -> IO (EntityId, MsgId, ByteString)
tcSecureQueue :: TestSMPClient -> ... -> IO ()
tcAckMessage :: TestSMPClient -> ... -> IO ()
```
Then the same test function runs against HS↔HS, HS↔JS, JS↔HS, JS↔JS peer combinations. The test creates two clients (one receiver, one sender) on the same SMP server, creates a queue, exchanges keys, sends messages — proving protocol compatibility.
## Tests
### Unit tests (callNode, no server)
1. `sha512Hash` — same input → same output as Haskell
2. `cbAuthenticator` — same serverPubKey + entityPrivKey + nonce + message → same 80 bytes as Haskell
3. `encodeTransmissionForAuth` — same sessionId + corrId + entityId + command (encoded at v19) → same `{tForAuth, tToSend}` as Haskell
4. `authTransmission` with X25519 key — same keys + nonce + tForAuth → same authenticated bytes as Haskell
5. `authTransmission` with Ed25519 key — same key + tForAuth → same signature bytes as Haskell
6. `authTransmission` with no key (Nothing) — produces empty auth, matches Haskell
7. `tEncodeBatch1` — same auth + transmission → same block bytes as Haskell
8. `tParse` + `tDecodeClient` — TS parses Haskell-encoded response block, extracts corrId + entityId + typed response
9. `batchTransmissions` — given N transmissions, produces same batch boundaries and block bytes as Haskell
### Integration tests (with SMP server, using REPL)
10. JS client connects, sends PING, receives PONG
11. JS client creates queue (NEW → IDS)
12. JS receiver creates queue + subscribes, JS sender sends message, receiver gets MSG
13. Full handshake: create queue → secure (KEY) → subscribe → send → receive MSG → ack
### Polymorphic integration tests
14. Same test function, peer combinations:
- HS sender, JS receiver
- JS sender, HS receiver
- JS sender, JS receiver
## Implementation order
1. Transport update — `connectSMP` returns `serverPubKey`
2. Crypto additions — `sha512Hash`, `cbAuthenticator`, Ed25519 `sign`
3. Protocol encoding updates — `encodeTransmissionForAuth`, `authTransmission`, `tEncode`, `tEncodeBatch1`, `batchTransmissions`, `encodePING`
4. Protocol parsing updates — `transmissionP`, `tParse`, `tDecodeClient`, update `decodeResponse` (add `SOK`, `INFO`, structured `ERR`)
5. Unit tests for steps 1-4
6. Client core — `createSMPClient`, `sendCommand`, corrId correlation, receive dispatch, ping
7. High-level command functions
8. Client REPL
9. Integration tests with server
10. Polymorphic test wiring
## Files
| File | Action |
|---|---|
| `smp-web/src/transport/websockets.ts` | Update `connectSMP` to return `serverPubKey` |
| `smp-web/src/crypto.ts` | Add `sha512Hash`, `cbAuthenticator`, Ed25519 `sign` |
| `smp-web/src/protocol.ts` | Update transmission encoding/parsing, add auth, batching |
| `smp-web/src/client.ts` | New — SMP client |
| `smp-web/tests/client-repl.ts` | New — SMP client REPL for integration tests |
| `tests/SMPWebTests.hs` | Unit + integration tests |
## Scope
### Client spike (this plan)
Core client: connect, auth, send/receive, correlate, ping. High-level commands: NEW, SUB, KEY, SKEY, SEND, ACK, OFF, DEL, LGET, GET, QUE (getSMPQueueInfo). Single-command path. Tests against real server.
### Client MVP (next, after spike)
- Proxy commands (PRXY, PFWD, PRES) — essential for privacy, users must not connect directly to untrusted servers
- Batch subscribe (subscribeSMPQueues) — needed for groups
- `reverseNonce` — needed for proxy
- Batch delete (deleteSMPQueues)
### Post-MVP
| What | Why |
|---|---|
| Notification commands (NKEY, NDEL, NSUB) | Value only with webpush support |
| Service certificates (serviceAuth, serviceSig) | Browser doesn't use |
| Stream commands (streamSubscribeSMPQueues) | Not used in Haskell client either |
| NetworkConfig, SOCKS, host mode, transport selection | Browser connects via WebSocket directly |
| Queue link management (LSET, LDEL, LKEY) | Only needed to create links, not join them |
| `cbVerify` | Server-side only |
## Haskell references
- `Client.hs:179-200` — ProtocolClient, PClient types
- `Client.hs:248``type SMPClient = ProtocolClient SMPVersion ErrorType BrokerMsg`
- `Client.hs:506-512` — Request type
- `Client.hs:628-642` — client connection, handshake, raceAny_ [send, process, receive, monitor]
- `Client.hs:644-658` — send loop, receive loop
- `Client.hs:660-678` — monitor/ping loop
- `Client.hs:680-719` — process loop, processMsg (corrId correlation, clientResp classification)
- `Client.hs:810-828` — createSMPQueue
- `Client.hs:833-836` — subscribeSMPQueue
- `Client.hs:938-939` — secureSMPQueue
- `Client.hs:1027-1031` — sendSMPMessage
- `Client.hs:1040-1045` — ackSMPMessage (note: ACK can return MSG)
- `Client.hs:1239-1243` — okSMPCommand pattern
- `Client.hs:1300-1326` — sendProtocolCommand_, sendRecv, size check, tEncodeBatch1
- `Client.hs:1333-1344` — getResponse, timeout handling
- `Client.hs:1349-1370` — mkTransmission_, CorrId=nonce, encodeTransmissionForAuth, authTransmission
- `Client.hs:1372-1391` — authTransmission, authenticate (X25519 vs Ed25519), service sig
- `Protocol.hs:488-525` — RawTransmission, TransmissionAuth, TAuthorizations, tEncodeAuth
- `Protocol.hs:1629-1643` — transmissionP
- `Protocol.hs:2129-2198` — batching, tEncode, tEncodeBatch1, batchTransmissions_
- `Protocol.hs:2207-2267` — tGetClient, tParse, tDecodeClient
- `Crypto.hs:1016` — sha512Hash
- `Crypto.hs:1296-1298` — cbEncryptNoPad (= cryptoBox without padding)
- `Crypto.hs:1330-1331` — cbDecryptNoPad
- `Crypto.hs:1366-1371` — cbAuthenticate, cbVerify
@@ -0,0 +1,238 @@
# SMP Client MVP: Proxy + Batching — Transpilation Plan
**Parent**: [SMP Client Spike](./2026-05-17-smp-client.md)
## Rule
Every TypeScript function is a faithful transpilation of a specific Haskell function at specific lines. Same name, same steps, same call chain. No inferences, no approximations. Each entry below gives the exact source to transpile from.
## Crypto functions
### `reverseNonce` → transpile `Crypto.hs:1409-1410`
```haskell
reverseNonce (CryptoBoxNonce s) = CryptoBoxNonce (B.reverse s)
```
TS: `function reverseNonce(nonce: Uint8Array): Uint8Array` — reverse the 24 bytes.
### `cbDecryptNoPad` → transpile `Crypto.hs:1330-1331`
```haskell
cbDecryptNoPad (DhSecretX25519 secret) = sbDecryptNoPad_ secret
```
Which is `sbDecryptNoPad_` from secretbox. xftp-web's `cbDecrypt` does decrypt+unpad. Need decrypt without unpad — extract tag(16) + cipher, decrypt, verify tag, return raw (no unpad). Use xftp-web's `sbInit`/`sbDecryptChunk`/`sbAuth` directly.
## Protocol encoding functions
### `encodeProtocolServer` → transpile `Protocol.hs:1264-1266`
```haskell
smpEncode ProtocolServer {host, port, keyHash} = smpEncode (host, port, keyHash)
```
Where:
- `host :: NonEmpty TransportHost``smpEncodeList` (1-byte count + items)
- Each `TransportHost``smpEncode (strEncode host)``encodeBytes(ascii(hostname))` (`Transport/Client.hs:77-78`)
- `port :: ServiceName` = ByteString → `encodeBytes(port)`
- `keyHash :: KeyHash` = ByteString → `encodeBytes(keyHash)`
File: `src/protocol.ts`
### `encodePRXY` → transpile `Protocol.hs:1710`
```haskell
PRXY host auth_ -> e (PRXY_, ' ', host, auth_)
```
= `"PRXY " + smpEncode(server) + smpEncode(Maybe BasicAuth)`
Where `Maybe BasicAuth` = `encodeMaybe(encodeBytes, auth)`.
### `encodePFWD` → transpile `Protocol.hs:1711`
```haskell
PFWD fwdV pubKey (EncTransmission s) -> e (PFWD_, ' ', fwdV, pubKey, Tail s)
```
= `"PFWD " + encodeWord16(version) + encodeBytes(pubKeyDer) + encTransmission` (Tail = no length prefix)
### `decodePKEY` → transpile `Protocol.hs:1894`
```haskell
PKEY_ -> PKEY <$> _smpP <*> smpP <*> smpP
```
= space + `decodeBytes(d)` (sessionId) + `decodeVersionRange(d)` + `decodeCertChainPubKey(d)`
`VersionRange` encoding (`Version.hs`): `smpEncode (minVersion, maxVersion)` = two Word16.
`CertChainPubKey` encoding (`Transport.hs:663-667`): `smpEncode (encodeCertChain chain, SignedObject signedPubKey)``encodeCertChain` is `Large`-encoded DER bytes, `SignedObject` is `Large`-encoded DER bytes.
### `decodePRES` → transpile `Protocol.hs:1896`
```haskell
PRES_ -> PRES <$> (EncResponse . unTail <$> _smpP)
```
= space + rest of bytes (Tail) → `EncResponse`
### Add to `decodeResponse`: `PKEY` and `PRES` cases.
## Client functions
### `sendProtocolCommands` → transpile `Client.hs:1262-1278`
Call chain:
1. `mapM (mkTransmission c) cs` — for each command: generate corrId, encode, auth, register pending request
2. `batchTransmissions' thParams` — pack into blocks
3. `mapM (sendBatch c nm) bs` — send each block, collect responses
4. `validate` — verify response count matches command count
In TS: `mkTransmission` = the existing `sendCommand` logic (corrId generation, `encodeTransmissionForAuth`, `authTransmission`) but separated into encode+register vs send+await. Need to refactor `sendCommand` to split these.
### `batchTransmissions'` → transpile `Protocol.hs:2135-2148`
Already have `batchTransmissions` in protocol.ts that does `batchTransmissions_`. Need `batchTransmissions'` which wraps with `tEncodeForBatch` before batching. Currently the TS `batchTransmissions` takes pre-encoded Large-wrapped bytes. Need to match the Haskell call chain exactly:
```haskell
batchTransmissions' params ts
| batch = batchTransmissions_ bSize $ L.map (first $ fmap $ tEncodeForBatch serviceAuth) ts
```
### `sendBatch` → transpile `Client.hs:1285-1298`
Three cases:
- `TBError`: return error response
- `TBTransmissions s n rs`: send block `s`, await all `n` responses concurrently
- `TBTransmission s r`: send block `s`, await one response
In browser: "concurrently" = all promises pending simultaneously, resolved by `onBlock` handler as responses arrive.
### `subscribeSMPQueues` → transpile `Client.hs:840-845`
```haskell
subscribeSMPQueues c qs = do
liftIO $ enablePings c
sendProtocolCommands c NRMBackground cs >>= mapM (processSUBResponse c)
where
cs = L.map (\(rId, rpKey) -> (rId, Just rpKey, Cmd SRecipient SUB)) qs
```
### `processSUBResponse` → transpile `Client.hs:854-862`
```haskell
processSUBResponse c (Response rId r) = pure r $>>= processSUBResponse_ c rId
processSUBResponse_ c rId = \case
OK -> pure $ Right Nothing
SOK serviceId_ -> pure $ Right serviceId_
cmd@MSG {} -> writeSMPMessage c rId cmd $> Right Nothing
r' -> pure . Left $ unexpectedResponse r'
```
MSG → push to `onMessage`, return success. Same pattern as single subscribe.
### `deleteSMPQueues` → transpile `Client.hs:1062-1065`
```haskell
deleteSMPQueues = okSMPCommands DEL
```
Uses `okSMPCommands` (`Client.hs:1245-1253`) which calls `sendProtocolCommands` and checks each response is OK.
### `connectSMPProxiedRelay` → transpile `Client.hs:1069-1093`
Call chain:
1. Send `PRXY relayServ proxyAuth` to proxy (via `sendProtocolCommand_`, entityId = NoEntity)
2. Receive `PKEY sessionId versionRange certChainPubKey`
3. Check version compatibility
4. `validateRelay chain key` — validate cert chain against relay's keyHash, extract X25519 key
5. Return `ProxiedRelay {sessionId, version, auth, relayKey}`
`validateRelay` (`Client.hs:1085-1093`):
1. `chainIdCaCerts chain` → extract leaf, id, ca certs
2. Check `Fingerprint kh == getFingerprint idCert SHA256`
3. `x509validate caCert (hostName, port) chain`
4. Extract server key from leaf cert
5. Verify signed key against server key
In browser: we already have `verifyIdentityProof` and `extractSignedKey` from xftp-web. Need to adapt for relay validation where we receive the cert chain in the PKEY response (DER-encoded, not from TLS handshake).
### `proxySMPCommand` → transpile `Client.hs:1157-1206`
Call chain:
1. Construct `serverThParams` = `smpTHParamsSetVersion v proxyThParams {sessionId, thAuth = serverThAuth}`
- `serverThAuth = thAuth proxyThParams with peerServerPubKey = relayKey`
2. Generate ephemeral X25519 keypair: `(cmdPubKey, cmdPrivKey)`
3. `cmdSecret = dh(relayKey, cmdPrivKey)`
4. Generate random nonce (also used as corrId)
5. `encodeTransmissionForAuth serverThParams (CorrId corrId, sId, Cmd sParty command)` — encode as if sending to relay
6. `authTransmission serverThAuth False spKey nonce tForAuth` — authenticate with entity key against relay
7. `batchTransmissions serverThParams [Right (auth, tToSend)]` — batch into single block
8. `cbEncrypt cmdSecret nonce batchBlock paddedProxiedTLength``EncTransmission`
9. Send `PFWD version cmdPubKey encTransmission` to proxy (entityId = sessionId)
10. Receive `PRES (EncResponse encResponse)`
11. `cbDecrypt cmdSecret (reverseNonce nonce) encResponse` — decrypt relay's response
12. `tParse serverThParams decrypted` — parse as relay's response
13. `tDecodeClient serverThParams parsed` — decode command
14. Classify: `Right (ERR e)` → throw PCEProtocolError, `Right r` → return Right r, `Left e` → throw PCEResponseError
Error wrapping (`Client.hs:1200-1206`): proxy-level errors (from PFWD response itself) → `ProxyClientError` returned as `Left`. Relay-level errors (inside PRES) → `PCEProtocolError` thrown.
### `paddedProxiedTLength` → `Protocol.hs:306-307` = 16226
## Constants
```
paddedProxiedTLength = 16226 -- Protocol.hs:306
serviceCertsSMPVersion = 16 -- Transport.hs:213
```
## Testing
### Unit tests (callNode, no server)
Each encoding function tested byte-for-byte against Haskell:
1. `reverseNonce` — reverse known bytes, compare
2. `encodeProtocolServer` — encode known server, compare with `smpEncode @SMPServer`
3. `encodePRXY` — encode PRXY command, compare with `encodeProtocol v (Cmd SProxiedClient (PRXY srv auth))`
4. `encodePFWD` — encode PFWD command, compare with `encodeProtocol v (Cmd SProxiedClient (PFWD v pk et))`
5. `batchTransmissions` with multiple commands — same batch boundaries as `batchTransmissions_` in Haskell
### Integration tests (with two SMP servers, from SMPProxyTests.hs pattern)
6. `connectProxiedRelay` — JS connects to proxy, sends PRXY for relay, gets PKEY, validates cert, extracts key
7. `proxySMPMessage` — JS sends SEND via proxy to relay, HS receiver gets MSG
8. Full proxy roundtrip — JS creates queue on relay via proxy, sends message via proxy, HS receives
### Batch tests
9. `subscribeSMPQueues` — JS batch-subscribes to N queues, verifies all subscribed
10. `deleteSMPQueues` — JS batch-deletes N queues
## Implementation order
1. `reverseNonce`, `cbDecryptNoPad`
2. `encodeProtocolServer`, `encodePRXY`, `encodePFWD`
3. `decodePKEY`, `decodePRES`, update `decodeResponse`
4. Unit tests for steps 1-3
5. Refactor `sendCommand` → split into `mkTransmission` (encode+register) and send
6. `sendProtocolCommands`, `sendBatch`
7. `subscribeSMPQueues`, `deleteSMPQueues`
8. Batch integration tests
9. `connectSMPProxiedRelay` (cert validation, PRXY/PKEY)
10. `proxySMPCommand`, `proxySMPMessage`
11. Proxy integration tests
## Files
| File | Action |
|---|---|
| `smp-web/src/crypto.ts` | `reverseNonce`, `cbDecryptNoPad` |
| `smp-web/src/protocol.ts` | `encodeProtocolServer`, `encodePRXY`, `encodePFWD`, `decodePKEY`, `decodePRES` |
| `smp-web/src/client.ts` | `sendProtocolCommands`, `sendBatch`, `subscribeSMPQueues`, `deleteSMPQueues`, `connectSMPProxiedRelay`, `proxySMPCommand` |
| `smp-web/tests/client-repl.ts` | Proxy + batch REPL commands |
| `tests/SMPWebTests.hs` | Tests |
## Haskell source — exact lines to transpile
| TS function | Haskell function | File:lines |
|---|---|---|
| `reverseNonce` | `reverseNonce` | `Crypto.hs:1409-1410` |
| `cbDecryptNoPad` | `cbDecryptNoPad` / `sbDecryptNoPad_` | `Crypto.hs:1330-1331` |
| `encodeProtocolServer` | `instance Encoding (ProtocolServer p)` | `Protocol.hs:1264-1266` |
| `encodePRXY` | `encodeProtocol v (PRXY ...)` | `Protocol.hs:1710` |
| `encodePFWD` | `encodeProtocol v (PFWD ...)` | `Protocol.hs:1711` |
| `decodePKEY` | `protocolP v PKEY_` | `Protocol.hs:1894` |
| `decodePRES` | `protocolP v PRES_` | `Protocol.hs:1896` |
| `sendProtocolCommands` | `sendProtocolCommands` | `Client.hs:1262-1278` |
| `sendBatch` | `sendBatch` | `Client.hs:1285-1298` |
| `subscribeSMPQueues` | `subscribeSMPQueues` | `Client.hs:840-845` |
| `processSUBResponse` | `processSUBResponse` + `processSUBResponse_` | `Client.hs:854-862` |
| `deleteSMPQueues` | `deleteSMPQueues` via `okSMPCommands` | `Client.hs:1062-1065, 1245-1253` |
| `connectSMPProxiedRelay` | `connectSMPProxiedRelay` | `Client.hs:1069-1093` |
| `validateRelay` | `validateRelay` (inside `connectSMPProxiedRelay`) | `Client.hs:1085-1093` |
| `proxySMPCommand` | `proxySMPCommand` | `Client.hs:1157-1206` |
| `proxyOKSMPCommand` | `proxyOKSMPCommand` | `Client.hs:1150-1155` |
| `smpTHParamsSetVersion` | `smpTHParamsSetVersion` | `Transport.hs:921-926` |
| `batchTransmissions'` | `batchTransmissions'` | `Protocol.hs:2135-2148` |
| `batchTransmissions_` | `batchTransmissions_` | `Protocol.hs:2150-2169` |
+490
View File
@@ -0,0 +1,490 @@
// SMP client: command/response correlation, authentication, typed async API.
// Mirrors: Simplex.Messaging.Client
import {
encodeTransmission, encodeTransmissionForAuth, authTransmission,
tEncodeBatch1, tEncodeForBatch, batchTransmissions, tEncode,
tParse, tDecodeClient, protocolError, encodePING,
decodeResponse, paddedProxiedTLength, encodePRXY, encodePFWD,
type AuthKey, type SMPResponse, type RawTransmission,
encodeNEW, encodeKEY, encodeSKEY, encodeSUB, encodeACK,
encodeSEND, encodeOFF, encodeDEL, encodeGET, encodeQUE, encodeLGET,
type IDSResponse, type MSGResponse,
} from "./protocol.js"
import {
connectSMP,
type SMPConnection,
} from "./transport/websockets.js"
import {SMP_BLOCK_SIZE} from "./transport.js"
import {sbEncryptBlock, sbDecryptBlock, cbAuthenticator, reverseNonce, cbDecryptNoPad} from "./crypto.js"
import {blockPad, blockUnpad} from "@simplex-chat/xftp-web/dist/protocol/transmission.js"
import {Decoder} from "@simplex-chat/xftp-web/dist/protocol/encoding.js"
import {generateX25519KeyPair, x25519KeyPairFromPrivate, dh, encodePubKeyX25519} from "@simplex-chat/xftp-web/dist/crypto/keys.js"
import {cbEncrypt, cbDecrypt} from "@simplex-chat/xftp-web/dist/crypto/secretbox.js"
import {extractSignedKey} from "@simplex-chat/xftp-web/dist/protocol/handshake.js"
// -- Error types (Client.hs:741-770)
// ProxiedRelay (Client.hs:1095-1100)
export interface ProxiedRelay {
sessionId: Uint8Array
version: number // negotiated version with relay
basicAuth: Uint8Array | null
relayKey: Uint8Array // relay's X25519 public key (raw 32 bytes)
}
// ProxyClientError (Client.hs:1102-1109)
export type ProxyClientError =
| {type: "ProxyProtocolError", error: string}
| {type: "ProxyUnexpectedResponse", response: string}
| {type: "ProxyResponseError", error: string}
export type SMPClientError =
| {type: "PROTOCOL", error: string} // ERR response from server
| {type: "RESPONSE", error: string} // failed to parse response
| {type: "UNEXPECTED", raw: string} // wrong response type for command
| {type: "TIMEOUT"} // response timeout
| {type: "NETWORK", error: string} // connection failure
| {type: "TRANSPORT", error: string} // handshake/transport error
// -- SMPClient
export interface SMPClient {
readonly sessionId: Uint8Array
readonly smpVersion: number
readonly serverPubKey: Uint8Array
// Core: send pre-encoded command, await correlated response
sendCommand(privKey: AuthKey | null, entityId: Uint8Array, command: Uint8Array): Promise<SMPResponse>
// High-level commands
createQueue(authKeyPair: {publicKey: Uint8Array, privateKey: Uint8Array}, dhKey: Uint8Array, subscribe: boolean): Promise<IDSResponse>
subscribeQueue(privKey: AuthKey, rcvId: Uint8Array): Promise<void>
getMessage(privKey: AuthKey, rcvId: Uint8Array): Promise<MSGResponse | null>
sendMessage(privKey: AuthKey | null, sndId: Uint8Array, notification: boolean, msg: Uint8Array): Promise<void>
ackMessage(privKey: AuthKey, rcvId: Uint8Array, msgId: Uint8Array): Promise<void>
secureQueue(privKey: AuthKey, rcvId: Uint8Array, senderKey: Uint8Array): Promise<void>
secureSndQueue(privKey: AuthKey, sndId: Uint8Array): Promise<void>
getQueueLink(linkId: Uint8Array): Promise<SMPResponse>
deleteQueue(privKey: AuthKey, rcvId: Uint8Array): Promise<void>
suspendQueue(privKey: AuthKey, rcvId: Uint8Array): Promise<void>
// Batch commands (Client.hs:840-845, 1062-1065)
subscribeQueues(queues: Array<{rcvId: Uint8Array, privKey: AuthKey}>): Promise<void[]>
deleteQueues(queues: Array<{rcvId: Uint8Array, privKey: AuthKey}>): Promise<void[]>
// Proxy commands (Client.hs:1069-1206)
connectProxiedRelay(relayHosts: string[], relayPort: string, relayKeyHash: Uint8Array, basicAuth: Uint8Array | null): Promise<ProxiedRelay>
proxySMPCommand(relay: ProxiedRelay, privKey: AuthKey | null, entityId: Uint8Array, command: Uint8Array): Promise<SMPResponse>
proxySendMessage(relay: ProxiedRelay, privKey: AuthKey | null, sndId: Uint8Array, notification: boolean, msg: Uint8Array): Promise<void>
close(): void
}
interface PendingRequest {
resolve: (resp: SMPResponse) => void
reject: (err: SMPClientError) => void
timer: ReturnType<typeof setTimeout>
}
function toHex(bytes: Uint8Array): string {
return Array.from(bytes, b => b.toString(16).padStart(2, "0")).join("")
}
export async function createSMPClient(
url: string,
keyHash: Uint8Array,
onMessage: (entityId: Uint8Array, msg: SMPResponse) => void,
onDisconnected: () => void,
config?: {timeout?: number, pingInterval?: number, pingMaxCount?: number, wsOptions?: object},
): Promise<SMPClient> {
const timeout_ = config?.timeout ?? 10_000
const pingInterval = config?.pingInterval ?? 600_000
const pingMaxCount = config?.pingMaxCount ?? 3
const conn = await connectSMP(url, keyHash, config?.wsOptions)
if (!conn.serverPubKey) throw new Error("createSMPClient: server has no auth key")
const serverPubKey = conn.serverPubKey
const pending = new Map<string, PendingRequest>()
let closed = false
let pingTimer: ReturnType<typeof setInterval> | null = null
let timeoutCount = 0
// -- Receive loop
function onBlock(data: ArrayBuffer | Buffer) {
if (closed) return
timeoutCount = 0
try {
const raw = data instanceof ArrayBuffer ? data : data.buffer.slice(data.byteOffset, data.byteOffset + data.byteLength)
const block = new Uint8Array(raw)
// Decrypt block
const decrypted = decryptBlock(block)
// Parse batch
const transmissions = tParse(decrypted)
for (const raw of transmissions) {
dispatch(raw)
}
} catch (e: any) {
// Parse error — log to stderr
process.stderr.write("SMP client receive error: " + e.message + "\n")
}
}
function decryptBlock(block: Uint8Array): Uint8Array {
if (conn.rcvKey) {
const {decrypted, nextChainKey} = sbDecryptBlock(conn.rcvKey, block)
conn.rcvKey = nextChainKey
return decrypted
}
// No block encryption — strip padding
return blockUnpad(block)
}
function dispatch(raw: RawTransmission) {
// Parse response
let response: SMPResponse
try {
response = decodeResponse(new Decoder(raw.command))
} catch (e: any) {
process.stderr.write("dispatch parse error: " + e.message + " command=" + toHex(raw.command) + "\n")
// If we can correlate, reject the pending request
const key = toHex(raw.corrId)
const req = pending.get(key)
if (req) {
pending.delete(key)
clearTimeout(req.timer)
req.reject({type: "RESPONSE", error: e.message})
}
return
}
// Classify: ERR → PCEProtocolError
const err = protocolError(response)
// Correlate by corrId
const corrIdBytes = raw.corrId
if (corrIdBytes.length === 0) {
// Server push (no corrId) — deliver to event callback
onMessage(raw.entityId, response)
return
}
const key = toHex(corrIdBytes)
const req = pending.get(key)
if (req) {
pending.delete(key)
clearTimeout(req.timer)
if (err) {
req.reject({type: "PROTOCOL", error: err})
} else {
req.resolve(response)
}
} else {
// No pending request — might be a late response or server push with corrId
// Deliver as event
if (!err) onMessage(raw.entityId, response)
}
}
// Wire up WebSocket receive
conn.ws.onmessage = (event) => onBlock(event.data as ArrayBuffer)
conn.ws.onclose = () => {
if (!closed) {
closed = true
cleanup()
onDisconnected()
}
}
conn.ws.onerror = () => {}
// -- Ping
function startPing() {
if (pingInterval <= 0) return
pingTimer = setInterval(async () => {
try {
await client.sendCommand(null, new Uint8Array(0), encodePING())
} catch {
timeoutCount++
if (pingMaxCount > 0 && timeoutCount >= pingMaxCount) {
client.close()
}
}
}, pingInterval)
}
function cleanup() {
if (pingTimer) {
clearInterval(pingTimer)
pingTimer = null
}
// Reject all pending requests
for (const [, req] of pending) {
clearTimeout(req.timer)
req.reject({type: "NETWORK", error: "disconnected"})
}
pending.clear()
}
// -- Send
// mkTransmission (Client.hs:1349-1370)
// Encode, authenticate, register pending request. Returns encoded transmission + promise.
// nonce_ parameter: if provided, used as corrId (for proxy commands where nonce = corrId)
function mkTransmission(privKey: AuthKey | null, entityId: Uint8Array, command: Uint8Array, nonce_?: Uint8Array): {auth: Uint8Array | null, tToSend: Uint8Array, promise: Promise<SMPResponse>} {
const nonce = nonce_ ?? crypto.getRandomValues(new Uint8Array(24))
const {tForAuth, tToSend} = encodeTransmissionForAuth(conn.sessionId, nonce, entityId, command)
const auth = authTransmission(serverPubKey, privKey, nonce, tForAuth)
const promise = new Promise<SMPResponse>((resolve, reject) => {
const key = toHex(nonce)
const timer = setTimeout(() => {
pending.delete(key)
timeoutCount++
reject({type: "TIMEOUT"} as SMPClientError)
}, timeout_)
pending.set(key, {resolve, reject, timer})
})
return {auth, tToSend, promise}
}
// Send a pre-encoded block (encrypt + write to WebSocket)
function sendBlock(block: Uint8Array): void {
if (conn.sndKey) {
const {encrypted, nextChainKey} = sbEncryptBlock(conn.sndKey, block, SMP_BLOCK_SIZE - 16)
conn.sndKey = nextChainKey
conn.ws.send(encrypted)
} else {
conn.ws.send(blockPad(block, SMP_BLOCK_SIZE))
}
}
// sendProtocolCommand (Client.hs:1300-1326) — single command
// nonce_: if provided, used as corrId (for proxy where nonce = corrId)
function sendCommand(privKey: AuthKey | null, entityId: Uint8Array, command: Uint8Array, nonce_?: Uint8Array): Promise<SMPResponse> {
if (closed) return Promise.reject({type: "NETWORK", error: "closed"} as SMPClientError)
const {auth, tToSend, promise} = mkTransmission(privKey, entityId, command, nonce_)
sendBlock(tEncodeBatch1(auth, tToSend))
return promise
}
// sendProtocolCommands (Client.hs:1262-1298) — batch multiple commands
function sendCommands(commands: Array<{privKey: AuthKey | null, entityId: Uint8Array, command: Uint8Array}>): Promise<SMPResponse>[] {
if (closed) return commands.map(() => Promise.reject({type: "NETWORK", error: "closed"} as SMPClientError))
// mkTransmission for each
const transmissions = commands.map(c => mkTransmission(c.privKey, c.entityId, c.command))
// Encode for batching: tEncodeForBatch each
const encoded = transmissions.map(t => tEncodeForBatch(t.auth, t.tToSend))
// Pack into blocks
const blocks = batchTransmissions(SMP_BLOCK_SIZE, encoded)
// Send each block
for (const block of blocks) sendBlock(block)
// Return all promises
return transmissions.map(t => t.promise)
}
// -- High-level commands
// okSMPCommand (Client.hs:1239-1243) — only accepts OK, not SOK
async function okCommand(privKey: AuthKey | null, entityId: Uint8Array, command: Uint8Array): Promise<void> {
const resp = await sendCommand(privKey, entityId, command)
if (resp.type !== "OK") {
throw {type: "UNEXPECTED", raw: resp.type} as SMPClientError
}
}
const client: SMPClient = {
sessionId: conn.sessionId,
smpVersion: conn.smpVersion,
serverPubKey,
sendCommand,
// createQueue (Client.hs:813-827)
async createQueue(authKeyPair, dhKey, subscribe) {
const command = encodeNEW(authKeyPair.publicKey, dhKey, null, subscribe)
// Auth with the X25519 private key from the keypair
const privKey: AuthKey = {type: "x25519", key: authKeyPair.privateKey}
const resp = await sendCommand(privKey, new Uint8Array(0), command)
if (resp.type !== "IDS") throw {type: "UNEXPECTED", raw: resp.type} as SMPClientError
return resp.response
},
// subscribeSMPQueue (Client.hs:833-836)
async subscribeQueue(privKey, rcvId) {
const resp = await sendCommand(privKey, rcvId, encodeSUB())
// SUB can return MSG (queued message) — push to onMessage
if (resp.type === "MSG") {
onMessage(rcvId, resp)
return
}
if (resp.type !== "OK" && resp.type !== "SOK") {
throw {type: "UNEXPECTED", raw: resp.type} as SMPClientError
}
},
// getSMPMessage (Client.hs:875-880)
async getMessage(privKey, rcvId) {
const resp = await sendCommand(privKey, rcvId, encodeGET())
if (resp.type === "OK") return null
if (resp.type === "MSG") {
onMessage(rcvId, resp)
return resp.response
}
throw {type: "UNEXPECTED", raw: resp.type} as SMPClientError
},
// sendSMPMessage (Client.hs:1027-1031)
async sendMessage(privKey, sndId, notification, msg) {
await okCommand(privKey, sndId, encodeSEND(notification, msg))
},
// ackSMPMessage (Client.hs:1040-1045)
async ackMessage(privKey, rcvId, msgId) {
const resp = await sendCommand(privKey, rcvId, encodeACK(msgId))
// ACK can return MSG — push to onMessage
if (resp.type === "MSG") {
onMessage(rcvId, resp)
return
}
if (resp.type !== "OK") {
throw {type: "UNEXPECTED", raw: resp.type} as SMPClientError
}
},
// secureSMPQueue (Client.hs:938-939)
async secureQueue(privKey, rcvId, senderKey) {
await okCommand(privKey, rcvId, encodeKEY(senderKey))
},
// secureSndSMPQueue (Client.hs:943-944)
// SKEY sends the public key derived from the private key
async secureSndQueue(privKey, sndId) {
// x25519KeyPairFromPrivate derives public from private
const pubKey = x25519KeyPairFromPrivate(privKey.key).publicKey
await okCommand(privKey, sndId, encodeSKEY(encodePubKeyX25519(pubKey)))
},
// getSMPQueueLink (Client.hs:976-980)
async getQueueLink(linkId) {
return sendCommand(null, linkId, encodeLGET())
},
// deleteSMPQueue (Client.hs:1058-1059)
async deleteQueue(privKey, rcvId) {
await okCommand(privKey, rcvId, encodeDEL())
},
// suspendSMPQueue (Client.hs:1051-1052)
async suspendQueue(privKey, rcvId) {
await okCommand(privKey, rcvId, encodeOFF())
},
// subscribeSMPQueues (Client.hs:840-845)
async subscribeQueues(queues) {
const commands = queues.map(q => ({privKey: q.privKey, entityId: q.rcvId, command: encodeSUB()}))
const promises = sendCommands(commands)
return Promise.all(promises.map(async (p, i) => {
const resp = await p
// processSUBResponse_ (Client.hs:857-862)
if (resp.type === "MSG") {
onMessage(queues[i].rcvId, resp)
return
}
if (resp.type !== "OK" && resp.type !== "SOK") {
throw {type: "UNEXPECTED", raw: resp.type} as SMPClientError
}
}))
},
// deleteSMPQueues (Client.hs:1062-1065) via okSMPCommands (Client.hs:1245-1253)
async deleteQueues(queues) {
const commands = queues.map(q => ({privKey: q.privKey, entityId: q.rcvId, command: encodeDEL()}))
const promises = sendCommands(commands)
return Promise.all(promises.map(async (p) => {
const resp = await p
if (resp.type !== "OK") {
throw {type: "UNEXPECTED", raw: resp.type} as SMPClientError
}
}))
},
// connectSMPProxiedRelay (Client.hs:1069-1093)
async connectProxiedRelay(relayHosts, relayPort, relayKeyHash, basicAuth) {
// Send PRXY to proxy server
const command = encodePRXY(relayHosts, relayPort, relayKeyHash, basicAuth)
const resp = await sendCommand(null, new Uint8Array(0), command)
if (resp.type !== "PKEY") throw {type: "UNEXPECTED", raw: resp.type} as SMPClientError
const {sessionId: relaySessId, versionRange, signedKeyDer} = resp.response
// Check version compatibility
const version = Math.min(versionRange.max, conn.smpVersion)
if (version < versionRange.min) throw {type: "TRANSPORT", error: "incompatible relay version"} as SMPClientError
// Extract relay's X25519 DH key from signed key (same as connectSMP handshake)
const relayKey = extractSignedKey(signedKeyDer).dhKey
// TODO: full certificate chain validation against relayKeyHash
// For now we trust the proxy's PKEY response (proxy already validated the relay)
return {sessionId: relaySessId, version, basicAuth, relayKey}
},
// proxySMPCommand (Client.hs:1157-1206)
async proxySMPCommand(relay, privKey, entityId, command) {
// Prepare relay params — encode as if sending directly to relay
const relaySessionId = relay.sessionId
// Generate ephemeral X25519 keypair for this command
const cmdKp = generateX25519KeyPair()
const cmdSecret = dh(relay.relayKey, cmdKp.privateKey)
const nonce = crypto.getRandomValues(new Uint8Array(24))
// Encode transmission for relay (using relay's sessionId)
const {tForAuth, tToSend} = encodeTransmissionForAuth(relaySessionId, nonce, entityId, command)
// Authenticate against relay's key
const auth = privKey
? cbAuthenticator(relay.relayKey, privKey.key, nonce, tForAuth)
: null
// Batch into single block (for relay)
const batchBlock = tEncodeBatch1(auth, tToSend)
// Encrypt for relay: cbEncrypt(cmdSecret, nonce, batchBlock, paddedProxiedTLength)
const encTransmission = cbEncrypt(cmdSecret, nonce, batchBlock, paddedProxiedTLength)
// Send PFWD to proxy (entityId = relay sessionId from PKEY)
// IMPORTANT: nonce is also used as corrId for PFWD (Client.hs:1175,1188)
// The relay extracts it from FwdTransmission.fwdCorrId to decrypt
const cmdPubKeyDer = encodePubKeyX25519(cmdKp.publicKey)
const pfwdCommand = encodePFWD(relay.version, cmdPubKeyDer, encTransmission)
const pfwdResp = await sendCommand(null, relay.sessionId, pfwdCommand, nonce)
// Handle response
if (pfwdResp.type === "PRES") {
// Decrypt relay's response: cbDecrypt(cmdSecret, reverseNonce(nonce), encResponse)
const decrypted = cbDecrypt(cmdSecret, reverseNonce(nonce), pfwdResp.encResponse)
// Parse as relay's response
const transmissions = tParse(decrypted)
if (transmissions.length !== 1) throw {type: "TRANSPORT", error: "bad proxy response block"} as SMPClientError
const decoded = tDecodeClient(transmissions[0])
const relayResp = decoded.response
const err = protocolError(relayResp)
if (err) throw {type: "PROTOCOL", error: err} as SMPClientError
return relayResp
}
if (pfwdResp.type === "ERR") {
throw {type: "PROTOCOL", error: pfwdResp.error} as SMPClientError
}
throw {type: "UNEXPECTED", raw: pfwdResp.type} as SMPClientError
},
// proxySMPMessage — convenience for SEND via proxy
async proxySendMessage(relay, privKey, sndId, notification, msg) {
const command = encodeSEND(notification, msg)
const resp = await client.proxySMPCommand(relay, privKey, sndId, command)
if (resp.type !== "OK") throw {type: "UNEXPECTED", raw: resp.type} as SMPClientError
},
close() {
if (closed) return
closed = true
cleanup()
conn.ws.close()
},
}
startPing()
return client
}
+42 -1
View File
@@ -4,7 +4,8 @@
import {hkdf as nobleHkdf} from "@noble/hashes/hkdf"
import {sha512} from "@noble/hashes/sha512"
import {gcm} from "@noble/ciphers/aes.js"
import {cbEncrypt, cbDecrypt} from "@simplex-chat/xftp-web/dist/crypto/secretbox.js"
import {cbEncrypt, cbDecrypt, cryptoBox, sbInit, sbDecryptChunk, sbAuth} from "@simplex-chat/xftp-web/dist/crypto/secretbox.js"
import {dh} from "@simplex-chat/xftp-web/dist/crypto/keys.js"
import {concatBytes} from "@simplex-chat/xftp-web/dist/protocol/encoding.js"
import {pad, unPad} from "@simplex-chat/xftp-web/dist/crypto/padding.js"
@@ -87,3 +88,43 @@ export function decryptAEAD(
const padded = cipher.decrypt(encrypted)
return unPad(padded)
}
// -- SHA-512 hash (Crypto.hs:1016)
export function sha512Hash(msg: Uint8Array): Uint8Array {
return sha512(msg)
}
// -- Command authentication (Crypto.hs:1366-1367)
// cbAuthenticate (Crypto.hs:1367)
// cryptoBox(dh(serverPubKey, entityPrivKey), nonce, sha512Hash(msg)) → 80 bytes (16 tag + 64 hash)
export function cbAuthenticator(serverPubKey: Uint8Array, entityPrivKey: Uint8Array, nonce: Uint8Array, msg: Uint8Array): Uint8Array {
const dhSecret = dh(serverPubKey, entityPrivKey)
return cryptoBox(dhSecret, nonce, sha512Hash(msg))
}
// -- reverseNonce (Crypto.hs:1409-1410)
export function reverseNonce(nonce: Uint8Array): Uint8Array {
const reversed = new Uint8Array(nonce.length)
for (let i = 0; i < nonce.length; i++) reversed[i] = nonce[nonce.length - 1 - i]
return reversed
}
// -- cbDecryptNoPad (Crypto.hs:1330-1331)
// Decrypt without unpadding. Used for proxy responses.
// Same as cbDecrypt but returns raw decrypted bytes without unPad.
export function cbDecryptNoPad(dhSecret: Uint8Array, nonce: Uint8Array, packet: Uint8Array): Uint8Array {
const tag = packet.subarray(0, 16)
const cipher = packet.subarray(16)
const state = sbInit(dhSecret, nonce)
const plaintext = sbDecryptChunk(state, cipher)
const computedTag = sbAuth(state)
// constant-time compare
let diff = 0
for (let i = 0; i < 16; i++) diff |= tag[i] ^ computedTag[i]
if (diff !== 0) throw new Error("cbDecryptNoPad: authentication failed")
return plaintext
}
+248 -31
View File
@@ -1,5 +1,5 @@
// SMP protocol commands and transmission format.
// Mirrors: Simplex.Messaging.Protocol
// Mirrors: Simplex.Messaging.Protocol + Simplex.Messaging.Client (auth)
import {
Decoder, concatBytes,
@@ -10,30 +10,113 @@ import {
encodeMaybe, decodeMaybe,
} from "@simplex-chat/xftp-web/dist/protocol/encoding.js"
import {cbEncrypt, cbDecrypt} from "@simplex-chat/xftp-web/dist/crypto/secretbox.js"
import {sign} from "@simplex-chat/xftp-web/dist/crypto/keys.js"
import {readTag, readSpace} from "@simplex-chat/xftp-web/dist/protocol/commands.js"
import {cbAuthenticator} from "./crypto.js"
// -- Transmission encoding (Protocol.hs:2201-2203)
// encodeTransmission_ v (CorrId corrId, queueId, command) =
// smpEncode (corrId, queueId) <> encodeProtocol v command
// -- Auth key type for command authentication (Client.hs:1372-1391)
export type AuthKey =
| {type: "x25519", key: Uint8Array} // raw 32-byte private key → cbAuthenticator
| {type: "ed25519", key: Uint8Array} // raw 64-byte private key → sign
// -- Transmission encoding (Protocol.hs:2186-2198)
// encodeTransmission_ (Protocol.hs:2194-2198)
// smpEncode (corrId, entityId) <> encodeProtocol v command
// (command is pre-encoded bytes)
export function encodeTransmission(corrId: Uint8Array, entityId: Uint8Array, command: Uint8Array): Uint8Array {
return concatBytes(
encodeBytes(new Uint8Array(0)), // empty auth
encodeBytes(corrId),
encodeBytes(entityId),
command
)
return concatBytes(encodeBytes(corrId), encodeBytes(entityId), command)
}
// Batch encoding (Protocol.hs:2175-2180)
// Each transmission is Large-wrapped, then prefixed with 1-byte count.
export function encodeBatch(...transmissions: Uint8Array[]): Uint8Array {
if (transmissions.length === 0 || transmissions.length > 255) throw new Error("encodeBatch: invalid count")
return concatBytes(new Uint8Array([transmissions.length]), ...transmissions.map(t => encodeLarge(t)))
// encodeTransmissionForAuth (Protocol.hs:2186-2192)
// implySessId = true for v>=7 (always true for web client v19)
// tForAuth = sessionId <> encodeTransmission_(...)
// tToSend = encodeTransmission_(...)
export function encodeTransmissionForAuth(
sessionId: Uint8Array, corrId: Uint8Array, entityId: Uint8Array, command: Uint8Array,
): {tForAuth: Uint8Array, tToSend: Uint8Array} {
const tToSend = encodeTransmission(corrId, entityId, command)
const tForAuth = concatBytes(encodeBytes(sessionId), tToSend)
return {tForAuth, tToSend}
}
// -- Transmission parsing (Protocol.hs:1629-1642)
// For implySessId = True (v7+): no sessId on wire
// -- Command authentication (Client.hs:1372-1391)
// authTransmission: produce auth bytes for a transmission
// Returns null for unauthenticated commands, Uint8Array of auth bytes otherwise
export function authTransmission(
serverPubKey: Uint8Array, // server's X25519 public key from handshake
privKey: AuthKey | null, // null for unauthenticated commands (LGET, SEND without key)
nonce: Uint8Array, // 24-byte CorrId/nonce (same bytes)
tForAuth: Uint8Array, // transmission bytes to authenticate
): Uint8Array | null {
if (privKey === null) return null
switch (privKey.type) {
case "x25519":
// TAAuthenticator: cbAuthenticate(serverPubKey, entityPrivKey, nonce, tForAuth)
return cbAuthenticator(serverPubKey, privKey.key, nonce, tForAuth)
case "ed25519":
// TASignature: sign(entityPrivKey, tForAuth)
return sign(privKey.key, tForAuth)
}
}
// tEncodeAuth (Protocol.hs:507-516)
// For v16+ (serviceAuth=true): when auth is present, encode serviceSig as Nothing (0x30) after auth.
// When auth is absent: just empty ByteString.
export function tEncodeAuth(auth: Uint8Array | null): Uint8Array {
if (auth === null) return encodeBytes(new Uint8Array(0)) // empty ByteString: [0x00]
// serviceAuth=true for v16+: smpEncode (authBytes, serviceSig) where serviceSig = Nothing
return concatBytes(encodeBytes(auth), new Uint8Array([0x30])) // auth + Nothing
}
// tEncode (Protocol.hs:2171-2172)
export function tEncode(auth: Uint8Array | null, tToSend: Uint8Array): Uint8Array {
return concatBytes(tEncodeAuth(auth), tToSend)
}
// tEncodeBatch1 (Protocol.hs:2179-2180)
// Single-command batch: count=1 + Large(tEncode(...))
export function tEncodeBatch1(auth: Uint8Array | null, tToSend: Uint8Array): Uint8Array {
return concatBytes(new Uint8Array([1]), encodeLarge(tEncode(auth, tToSend)))
}
// tEncodeForBatch (Protocol.hs:2175-2176)
// Large(tEncode(...)) — for multi-command batches
export function tEncodeForBatch(auth: Uint8Array | null, tToSend: Uint8Array): Uint8Array {
return encodeLarge(tEncode(auth, tToSend))
}
// batchTransmissions (Protocol.hs:2151-2168)
// Pack multiple encoded transmissions into ≤blockSize blocks.
// Each input is an already-encoded Large-wrapped transmission.
// Returns array of blocks, each prefixed with count byte.
export function batchTransmissions(blockSize: number, transmissions: Uint8Array[]): Uint8Array[] {
const maxPayload = blockSize - 19 // 2 pad + 1 count + 16 auth tag
const blocks: Uint8Array[] = []
let currentParts: Uint8Array[] = []
let currentLen = 0
let count = 0
for (const t of transmissions) {
const tLen = t.length
if (tLen > maxPayload) throw new Error("batchTransmissions: transmission too large")
if (currentLen + tLen > maxPayload || count >= 255) {
if (count > 0) blocks.push(concatBytes(new Uint8Array([count]), ...currentParts))
currentParts = [t]
currentLen = tLen
count = 1
} else {
currentParts.push(t)
currentLen += tLen
count++
}
}
if (count > 0) blocks.push(concatBytes(new Uint8Array([count]), ...currentParts))
return blocks
}
// -- Transmission parsing (Protocol.hs:1629-1643, 2211-2267)
export interface RawTransmission {
corrId: Uint8Array
@@ -41,14 +124,47 @@ export interface RawTransmission {
command: Uint8Array
}
export function decodeTransmission(d: Decoder): RawTransmission {
const _auth = decodeBytes(d) // authenticator (empty for unsigned)
const corrId = decodeBytes(d)
const entityId = decodeBytes(d)
const command = d.takeAll()
// transmissionP (Protocol.hs:1629-1642)
// Parse a single transmission from block bytes.
// implySessId=true, serviceAuth=false for web client.
export function transmissionP(data: Uint8Array): RawTransmission {
const d = new Decoder(data)
const auth = decodeBytes(d) // authenticator
// serviceAuth=true for v16+: if auth is non-empty, skip serviceSig (Maybe Signature)
if (auth.length > 0) {
decodeMaybe(decodeBytes, d) // skip serviceSig
}
const rest = d.takeAll() // authorized bytes
// re-parse authorized: corrId + entityId + command
const d2 = new Decoder(rest)
// implySessId=true: no sessionId in wire format
const corrId = decodeBytes(d2)
const entityId = decodeBytes(d2)
const command = d2.takeAll()
return {corrId, entityId, command}
}
// tParse (Protocol.hs:2211-2217)
// Parse a received block into individual transmissions.
// batch=true: count byte + N Large-wrapped transmissions
export function tParse(block: Uint8Array): RawTransmission[] {
const d = new Decoder(block)
const count = d.anyByte()
const transmissions: RawTransmission[] = []
for (let i = 0; i < count; i++) {
const data = decodeLarge(d)
transmissions.push(transmissionP(data))
}
return transmissions
}
// tDecodeClient (Protocol.hs:2256-2266)
// Parse command bytes into typed response.
export function tDecodeClient(raw: RawTransmission): {corrId: Uint8Array, entityId: Uint8Array, response: SMPResponse} {
const response = decodeResponse(new Decoder(raw.command))
return {corrId: raw.corrId, entityId: raw.entityId, response}
}
// -- SMP command tags
const SPACE = 0x20
@@ -85,15 +201,31 @@ export function decodeLNK(d: Decoder): LNKResponse {
// -- Response dispatch (same pattern as xftp-web decodeResponse)
export interface PKEYResponse {
sessionId: Uint8Array
versionRange: {min: number, max: number}
certChainDer: Uint8Array // Large-encoded DER certificate chain
signedKeyDer: Uint8Array // Large-encoded DER signed public key
}
export type SMPResponse =
| {type: "LNK", response: LNKResponse}
| {type: "IDS", response: IDSResponse}
| {type: "MSG", response: MSGResponse}
| {type: "OK"}
| {type: "SOK", serviceId: Uint8Array | null}
| {type: "PKEY", response: PKEYResponse}
| {type: "PRES", encResponse: Uint8Array}
| {type: "PONG"}
| {type: "END"}
| {type: "DELD"}
| {type: "ERR", message: string}
| {type: "ERR", error: string}
// protocolError check (Client.hs:710-712)
// Returns the error string if this is an ERR response, null otherwise
export function protocolError(resp: SMPResponse): string | null {
return resp.type === "ERR" ? resp.error : null
}
export function decodeResponse(d: Decoder): SMPResponse {
const tag = readTag(d)
@@ -111,12 +243,45 @@ export function decodeResponse(d: Decoder): SMPResponse {
return {type: "MSG", response: decodeMSG(d)}
}
case "OK": return {type: "OK"}
case "SOK": {
// SOK serviceId_ → e(SOK_, ' ', serviceId_)
readSpace(d)
const serviceId = d.remaining() > 0 ? decodeMaybe(decodeBytes, d) : null
return {type: "SOK", serviceId}
}
case "PKEY": {
// PKEY sessionId versionRange certChainPubKey (Protocol.hs:1894)
// PKEY_ -> PKEY <$> _smpP <*> smpP <*> smpP
// sessionId: ByteString, versionRange: (Word16, Word16)
// certChainPubKey: (NonEmpty Large, SignedObject) (Transport.hs:663-664)
// certChain = NonEmpty Large = 1-byte count + N × Large(2-byte len + DER)
// signedKey = Large(2-byte len + DER)
readSpace(d)
const sessionId = decodeBytes(d)
const min = decodeWord16(d)
const max = decodeWord16(d)
// certChain: NonEmpty Large (1-byte count + N × Large-encoded DER certs)
const certCount = d.anyByte()
const certChainDers: Uint8Array[] = []
for (let i = 0; i < certCount; i++) certChainDers.push(decodeLarge(d))
// signedKey: Large-encoded DER
const signedKeyDer = decodeLarge(d)
return {type: "PKEY", response: {sessionId, versionRange: {min, max}, certChainDer: certChainDers[0] ?? new Uint8Array(0), signedKeyDer}}
}
case "PRES": {
// PRES (EncResponse encBlock) (Protocol.hs:1896)
// PRES_ -> PRES <$> (EncResponse . unTail <$> _smpP)
readSpace(d)
return {type: "PRES", encResponse: d.takeAll()}
}
case "PONG": return {type: "PONG"}
case "END": return {type: "END"}
case "DELD": return {type: "DELD"}
case "ERR": {
readSpace(d)
return {type: "ERR", message: readTag(d)}
// Read the full error string (may be multi-word like "AUTH" or "QUOTA")
const errBytes = d.takeAll()
return {type: "ERR", error: new TextDecoder().decode(errBytes)}
}
default: throw new Error("unknown SMP response: " + tag)
}
@@ -138,23 +303,23 @@ export function encodeSubMode(subscribe: boolean): Uint8Array {
// NEW (Protocol.hs:1682-1689)
// For v19: e(NEW_, ' ', rKey, dhKey) <> e(auth_, subMode, queueReqData, ntfCreds)
// auth_ = Maybe SndPublicAuthKey (DER-encoded)
// queueReqData = Maybe QueueReqData
// ntfCreds = Maybe NewNtfCreds (not needed for widget)
// QueueReqData: QRMessaging Nothing = 'M' + Nothing(0x30)
export function encodeNEW(
rcvAuthKey: Uint8Array, // DER-encoded Ed25519 or X25519 public key
rcvDhKey: Uint8Array, // DER-encoded X25519 public key
sndAuthKey: Uint8Array | null, // DER-encoded, for TOFU sender auth
basicAuth: Uint8Array | null, // Maybe BasicAuth (server auth, not a crypto key)
subscribe: boolean,
): Uint8Array {
// QRMessaging Nothing: Just('M', Nothing) = 0x31 0x4D 0x30
const queueReqData = new Uint8Array([0x31, 0x4D, 0x30])
return concatBytes(
ascii("NEW "),
encodeBytes(rcvAuthKey),
encodeBytes(rcvDhKey),
encodeMaybe(encodeBytes, sndAuthKey),
encodeMaybe(encodeBytes, basicAuth),
encodeSubMode(subscribe),
encodeMaybe(() => new Uint8Array(0), null), // queueReqData = Nothing (widget doesn't create links)
encodeMaybe(() => new Uint8Array(0), null), // ntfCreds = Nothing
queueReqData,
new Uint8Array([0x30]), // ntfCreds = Nothing
)
}
@@ -202,6 +367,58 @@ export function encodeDEL(): Uint8Array {
return ascii("DEL")
}
// GET (Protocol.hs:1698)
export function encodeGET(): Uint8Array {
return ascii("GET")
}
// QUE (Protocol.hs:1702)
export function encodeQUE(): Uint8Array {
return ascii("QUE")
}
// PING (Protocol.hs:1705)
export function encodePING(): Uint8Array {
return ascii("PING")
}
// -- Proxy commands (Protocol.hs:1710-1711)
// encodeProtocolServer (Protocol.hs:1264-1266)
// smpEncode ProtocolServer {host, port, keyHash} = smpEncode (host, port, keyHash)
// host :: NonEmpty TransportHost → smpEncodeList (1-byte count + encodeBytes(strEncode(host)) for each)
// port :: ServiceName = ByteString → encodeBytes
// keyHash :: KeyHash = ByteString → encodeBytes
export function encodeProtocolServer(hosts: string[], port: string, keyHash: Uint8Array): Uint8Array {
const encodedHosts = hosts.map(h => encodeBytes(ascii(h)))
const hostList = concatBytes(new Uint8Array([hosts.length]), ...encodedHosts)
return concatBytes(hostList, encodeBytes(ascii(port)), encodeBytes(keyHash))
}
// PRXY (Protocol.hs:1710)
// PRXY host auth_ -> e(PRXY_, ' ', host, auth_)
export function encodePRXY(hosts: string[], port: string, keyHash: Uint8Array, basicAuth: Uint8Array | null): Uint8Array {
return concatBytes(
ascii("PRXY "),
encodeProtocolServer(hosts, port, keyHash),
encodeMaybe(encodeBytes, basicAuth),
)
}
// PFWD (Protocol.hs:1711)
// PFWD fwdV pubKey (EncTransmission s) -> e(PFWD_, ' ', fwdV, pubKey, Tail s)
export function encodePFWD(version: number, pubKeyDer: Uint8Array, encTransmission: Uint8Array): Uint8Array {
return concatBytes(
ascii("PFWD "),
encodeWord16(version),
encodeBytes(pubKeyDer),
encTransmission, // Tail — no length prefix
)
}
// paddedProxiedTLength (Protocol.hs:306-307)
export const paddedProxiedTLength = 16226
// -- SMP response decoders
// IDS (Protocol.hs:1914-1921)
+6 -3
View File
@@ -19,6 +19,8 @@ export interface SMPConnection {
// Block encryption state (null if no auth)
sndKey: Uint8Array | null
rcvKey: Uint8Array | null
// Server's raw X25519 public key — needed for command auth (cbAuthenticate)
serverPubKey: Uint8Array | null
}
export async function connectSMP(url: string, keyHash: Uint8Array, wsOptions?: object): Promise<SMPConnection> {
@@ -46,6 +48,7 @@ export async function connectSMP(url: string, keyHash: Uint8Array, wsOptions?: o
let sndKey: Uint8Array | null = null
let rcvKey: Uint8Array | null = null
let clientAuthPubKey: Uint8Array | null = null
let serverPubKey: Uint8Array | null = null
if (serverHs.authPubKey) {
// Verify server identity if server supports web challenge (v19+)
@@ -62,10 +65,10 @@ export async function connectSMP(url: string, keyHash: Uint8Array, wsOptions?: o
}
// DH key exchange for block encryption (v11+)
const serverDhKey = extractSignedKey(serverHs.authPubKey.signedKeyDer).dhKey
serverPubKey = extractSignedKey(serverHs.authPubKey.signedKeyDer).dhKey
const clientKp = generateX25519KeyPair()
clientAuthPubKey = encodePubKeyX25519(clientKp.publicKey)
const dhSecret = dh(serverDhKey, clientKp.privateKey)
const dhSecret = dh(serverPubKey, clientKp.privateKey)
// Client swaps snd/rcv vs server (Transport.hs:880)
const keys = sbcInit(serverHs.sessionId, dhSecret)
sndKey = keys.rcvKey
@@ -82,7 +85,7 @@ export async function connectSMP(url: string, keyHash: Uint8Array, wsOptions?: o
})
sendBlock(ws, blockPad(clientHs, SMP_BLOCK_SIZE))
return {ws, sessionId: serverHs.sessionId, smpVersion: version, sndKey, rcvKey}
return {ws, sessionId: serverHs.sessionId, smpVersion: version, sndKey, rcvKey, serverPubKey}
}
export function receiveBlock(ws: WebSocket): Promise<Uint8Array> {
+257
View File
@@ -0,0 +1,257 @@
// SMP client REPL for cross-language testing.
// Holds one SMPClient, reads commands from stdin, writes results to stdout.
//
// Commands:
// CONNECT <url> <keyHashHex> [wsOptionsJson]
// NEW <rcvAuthKeyHex> <rcvDhKeyHex> <rcvPrivKeyHex>
// SUB <rcvIdHex> <rcvPrivKeyHex>
// SEND <sndIdHex> <sndPrivKeyHex|none> <notification 0|1> <bodyHex>
// ACK <rcvIdHex> <rcvPrivKeyHex> <msgIdHex>
// KEY <rcvIdHex> <rcvPrivKeyHex> <senderKeyHex>
// SKEY <sndIdHex> <sndPrivKeyHex>
// DEL <rcvIdHex> <rcvPrivKeyHex>
// OFF <rcvIdHex> <rcvPrivKeyHex>
// PING
// RECV [timeoutMs]
// CLOSE
import {createInterface} from "readline"
import {createSMPClient, type SMPClient} from "../dist/client.js"
import type {SMPResponse, AuthKey} from "../dist/protocol.js"
import {generateX25519KeyPair, dh, encodePubKeyX25519, decodePubKeyX25519} from "@simplex-chat/xftp-web/dist/crypto/keys.js"
import {cbDecrypt} from "@simplex-chat/xftp-web/dist/crypto/secretbox.js"
import {Decoder, decodeBytes, decodeBool} from "@simplex-chat/xftp-web/dist/protocol/encoding.js"
import type {ProxiedRelay} from "../dist/client.js"
// -- State
let client: SMPClient | null = null
let proxiedRelay: ProxiedRelay | null = null
// Per-queue DH shared secrets for decrypting received messages (keyed by rcvId hex)
const queueSecrets = new Map<string, Uint8Array>()
const messageQueue: Array<{entityId: Uint8Array, msg: SMPResponse}> = []
let messageWaiter: {resolve: (m: {entityId: Uint8Array, msg: SMPResponse}) => void, timer: ReturnType<typeof setTimeout>} | null = null
// -- Hex helpers
function toHex(bytes: Uint8Array): string {
return Array.from(bytes, b => b.toString(16).padStart(2, "0")).join("")
}
function fromHex(hex: string): Uint8Array {
const bytes = new Uint8Array(hex.length / 2)
for (let i = 0; i < hex.length; i += 2)
bytes[i / 2] = parseInt(hex.substring(i, i + 2), 16)
return bytes
}
// -- Message delivery
function onMessage(entityId: Uint8Array, msg: SMPResponse): void {
if (messageWaiter) {
const w = messageWaiter
messageWaiter = null
clearTimeout(w.timer)
w.resolve({entityId, msg})
} else {
messageQueue.push({entityId, msg})
}
}
function waitForMessage(timeoutMs: number): Promise<{entityId: Uint8Array, msg: SMPResponse}> {
if (messageQueue.length > 0) {
return Promise.resolve(messageQueue.shift()!)
}
return new Promise((resolve, reject) => {
const timer = setTimeout(() => {
messageWaiter = null
reject(new Error("timeout"))
}, timeoutMs)
messageWaiter = {resolve, timer}
})
}
function makeAuthKey(hexKey: string): AuthKey {
return {type: "x25519", key: fromHex(hexKey)}
}
// -- Command parser
async function parseLine(line: string): Promise<string> {
const parts = line.split(" ")
const cmd = parts[0]
try {
switch (cmd) {
case "CONNECT": {
const url = parts[1]
const keyHash = fromHex(parts[2])
const wsOptions = parts[3] ? JSON.parse(parts[3]) : undefined
client = await createSMPClient(url, keyHash, onMessage, () => {
process.stderr.write("disconnected\n")
}, {wsOptions, timeout: 15000})
return "ok"
}
case "NEW": {
if (!client) return "error: not connected"
const rcvAuthKey = fromHex(parts[1])
const rcvPrivKey = fromHex(parts[2])
// Generate DH keypair for per-queue E2E
const dhKp = generateX25519KeyPair()
const dhPubDer = encodePubKeyX25519(dhKp.publicKey)
const resp = await client.createQueue(
{publicKey: rcvAuthKey, privateKey: rcvPrivKey},
dhPubDer,
true,
)
// Compute and store DH shared secret for decrypting received messages
const srvDhRaw = decodePubKeyX25519(resp.srvDhKey)
const dhShared = dh(srvDhRaw, dhKp.privateKey)
queueSecrets.set(toHex(resp.rcvId), dhShared)
return "ok: " + toHex(resp.rcvId) + " " + toHex(resp.sndId) + " " + toHex(resp.srvDhKey)
}
case "SUB": {
if (!client) return "error: not connected"
await client.subscribeQueue(makeAuthKey(parts[2]), fromHex(parts[1]))
return "ok"
}
case "SEND": {
if (!client) return "error: not connected"
const sndId = fromHex(parts[1])
const privKey: AuthKey | null = parts[2] === "none" ? null : makeAuthKey(parts[2])
const notification = parts[3] === "1"
const body = fromHex(parts[4])
await client.sendMessage(privKey, sndId, notification, body)
return "ok"
}
case "ACK": {
if (!client) return "error: not connected"
await client.ackMessage(makeAuthKey(parts[2]), fromHex(parts[1]), fromHex(parts[3]))
return "ok"
}
case "KEY": {
if (!client) return "error: not connected"
await client.secureQueue(makeAuthKey(parts[2]), fromHex(parts[1]), fromHex(parts[3]))
return "ok"
}
case "SKEY": {
if (!client) return "error: not connected"
await client.secureSndQueue(makeAuthKey(parts[2]), fromHex(parts[1]))
return "ok"
}
case "DEL": {
if (!client) return "error: not connected"
await client.deleteQueue(makeAuthKey(parts[2]), fromHex(parts[1]))
return "ok"
}
case "OFF": {
if (!client) return "error: not connected"
await client.suspendQueue(makeAuthKey(parts[2]), fromHex(parts[1]))
return "ok"
}
case "PING": {
if (!client) return "error: not connected"
// Optional auth key as second arg: PING <privKeyHex>
const pingKey: AuthKey | null = parts[1] ? makeAuthKey(parts[1]) : null
const resp = await client.sendCommand(pingKey, new Uint8Array(0), new TextEncoder().encode("PING"))
return resp.type === "PONG" ? "ok" : "error: unexpected " + resp.type
}
case "RECV": {
if (!client) return "error: not connected"
const timeoutMs = parts[1] ? parseInt(parts[1]) : 5000
const m = await waitForMessage(timeoutMs)
if (m.msg.type === "MSG") {
const {msgId, msgBody} = m.msg.response
// Decrypt per-queue E2E: cbDecrypt(dhShared, cbNonce(msgId), body)
const dhShared = queueSecrets.get(toHex(m.entityId))
if (dhShared) {
// decryptMsgV3: cbDecrypt then parse ClientRcvMsgBody (msgTs + msgFlags + space + Tail msgBody)
const decrypted = cbDecrypt(dhShared, msgId, msgBody)
const dd = new Decoder(decrypted)
dd.take(8) // skip msgTs (SystemTime = Int64 = 8 bytes)
dd.take(1) // skip msgFlags (Bool = 1 byte)
dd.take(1) // skip space (0x20)
const body = dd.takeAll()
return "ok: " + toHex(m.entityId) + " " + toHex(msgId) + " " + toHex(body)
}
// No DH secret (sender queue) — return raw
return "ok: " + toHex(m.entityId) + " " + toHex(msgId) + " " + toHex(msgBody)
}
return "ok: " + toHex(m.entityId) + " " + m.msg.type
}
// BSUB <rcvId1Hex>:<privKey1Hex> <rcvId2Hex>:<privKey2Hex> ...
case "BSUB": {
if (!client) return "error: not connected"
const queues = parts.slice(1).map(p => {
const [rcvIdHex, privKeyHex] = p.split(":")
return {rcvId: fromHex(rcvIdHex), privKey: makeAuthKey(privKeyHex)}
})
await client.subscribeQueues(queues)
return "ok"
}
// PRXY <host1,host2,...> <port> <keyHashHex> [basicAuthHex]
case "PRXY": {
if (!client) return "error: not connected"
const hosts = parts[1].split(",")
const port = parts[2]
const keyHash = fromHex(parts[3])
const auth = parts[4] ? fromHex(parts[4]) : null
proxiedRelay = await client.connectProxiedRelay(hosts, port, keyHash, auth)
return "ok: " + toHex(proxiedRelay.sessionId) + " " + proxiedRelay.version
}
// PSEND <sndIdHex> <sndPrivKeyHex|none> <notification 0|1> <bodyHex>
case "PSEND": {
if (!client || !proxiedRelay) return "error: not connected or no proxy session"
const sndId = fromHex(parts[1])
const privKey: AuthKey | null = parts[2] === "none" ? null : makeAuthKey(parts[2])
const notification = parts[3] === "1"
const body = fromHex(parts[4])
await client.proxySendMessage(proxiedRelay, privKey, sndId, notification, body)
return "ok"
}
case "CLOSE": {
if (client) client.close()
client = null
return "ok"
}
default:
return "error: unknown command: " + cmd
}
} catch (e: any) {
if (e.type) return "error: " + e.type + (e.error ? " " + e.error : "")
return "error: " + (e.message || String(e))
}
}
// -- Main
async function main() {
const rl = createInterface({input: process.stdin, terminal: false})
for await (const line of rl) {
const trimmed = line.trim()
if (!trimmed) continue
const response = await parseLine(trimmed)
process.stdout.write(response + "\n")
}
}
main().catch(e => {
process.stderr.write("FATAL: " + e.message + "\n")
process.exit(1)
})
+416 -22
View File
@@ -14,7 +14,7 @@
module SMPWebTests (smpWebTests) where
import Control.Concurrent.STM
import Control.Monad (when)
import Control.Monad (forM, forM_, when)
import Data.Bifunctor (first)
import Control.Exception (bracket)
import Control.Monad.Except (ExceptT, liftEither, runExceptT, throwError, withExceptT)
@@ -24,14 +24,14 @@ import System.IO (Handle, hFlush, hGetLine, hPutStr, hSetBuffering, BufferMode (
import System.Process (CreateProcess (..), StdStream (..), ProcessHandle, createProcess, proc, terminateProcess)
import qualified Data.ByteString as B
import qualified Data.ByteString.Char8 as BC
import Data.List (isInfixOf)
import Data.List (isInfixOf, isPrefixOf)
import Data.List.NonEmpty (NonEmpty (..))
import System.Directory (doesDirectoryExist)
import Data.Word (Word16)
import qualified Simplex.Messaging.Agent as A
import qualified Simplex.Messaging.Agent.Protocol as AP
import Simplex.Messaging.Agent.Protocol (CreatedConnLink (..), UserLinkData (..), UserContactData (..), UserConnLinkData (..))
import Simplex.Messaging.Client (pattern NRMInteractive)
import Simplex.Messaging.Client (pattern NRMInteractive, authTransmission, getProtocolClient, defaultSMPClientConfig, ProtocolClientConfig (..), connectSMPProxiedRelay, proxySMPMessage, closeProtocolClient, ProxyClientError (..))
import Simplex.Messaging.Version (mkVersionRange)
import Simplex.Messaging.Version.Internal (Version (..))
import qualified Simplex.Messaging.Crypto as C
@@ -44,14 +44,19 @@ import qualified Data.ByteArray as BA
import Simplex.Messaging.Crypto.ShortLink (contactShortLinkKdf, invShortLinkKdf)
import Simplex.Messaging.Encoding
import Simplex.Messaging.Encoding.String (Str (..), strEncode)
import Simplex.Messaging.Protocol (EntityId (..), SMPServer, SubscriptionMode (..), MsgFlags (..), pattern SMPServer, encodeProtocol, Command (..), NewQueueReq (..), BrokerMsg (..), RcvMessage (..), EncRcvMsgBody (..), QueueIdsKeys (..), PubHeader (..), PrivHeader (..), ClientMessage (..), ClientMsgEnvelope (..), pattern VersionSMPC)
import Simplex.Messaging.Server.Env.STM (AStoreType (..))
import Simplex.Messaging.Protocol (EntityId (..), SMPServer, SubscriptionMode (..), MsgFlags (..), noMsgFlags, pattern SMPServer, pattern NoEntity, encodeProtocol, Cmd (..), SParty (..), Command (..), NewQueueReq (..), QueueReqData (..), BrokerMsg (..), RcvMessage (..), EncRcvMsgBody (..), QueueIdsKeys (..), PubHeader (..), PrivHeader (..), ClientMessage (..), ClientMsgEnvelope (..), pattern VersionSMPC)
import Simplex.Messaging.Server.Env.STM (AStoreType (..), ServerConfig (..))
import Simplex.Messaging.Server.MsgStore.Types (SMSType (..), SQSType (..))
import Simplex.Messaging.Server.Web (attachStaticAndWS)
import Simplex.Messaging.Transport (TLS, smpBlockSize, currentServerSMPRelayVersion)
import Data.Time.Clock (getCurrentTime)
import Simplex.Messaging.Transport (TLS, transport, smpBlockSize, currentServerSMPRelayVersion, currentClientSMPRelayVersion, minServerSMPRelayVersion, supportedClientSMPRelayVRange, alpnSupportedSMPHandshakes)
import Simplex.Messaging.Version (mkVersionRange)
import Simplex.Messaging.Transport.Server (ServerCredentials (..), mkTransportServerConfig)
import Simplex.Messaging.Transport.HTTP2 (httpALPN)
import Simplex.Messaging.Transport.Client (TransportHost (..))
import SMPAgentClient (agentCfg, initAgentServers, testDB)
import SMPClient (cfgWebOn, testKeyHash, testPort, withSmpServerConfig)
import SMPClient (cfgWebOn, cfgMS, proxyCfgMS, updateCfg, testKeyHash, testPort, testPort2, testSMPClient, testSMPClient_, testHost2, testStoreLogFile2, testStoreMsgsDir2, journalCfg, withSmpServerConfig, withSmpServerConfigOn)
import ServerTests (sendRecv, signSendRecv, tGet1, decryptMsgV3, _SEND, pattern Resp, pattern Ids, pattern Msg, pattern New)
import AgentTests.DoubleRatchetTests (testEncryptDecrypt, testSkippedMessages, testManyMessages, testSkippedAfterRatchetAdvance)
import AgentTests.FunctionalAPITests (withAgent)
import Test.Hspec hiding (it)
@@ -68,7 +73,7 @@ impEnc :: String
impEnc = "import { Decoder, decodeBytes, decodeLarge, encodeBytes, encodeWord16 } from '@simplex-chat/xftp-web/dist/protocol/encoding.js';"
impProto_ :: String
impProto_ = "import { encodeTransmission, encodeBatch, decodeTransmission, encodeLGET, decodeLNK, decodeResponse, encodeNEW, encodeKEY, encodeSKEY, encodeSUB, encodeACK, encodeSEND, encodeOFF, encodeDEL } from './dist/protocol.js';"
impProto_ = "import { encodeTransmission, encodeTransmissionForAuth, authTransmission, tEncodeAuth, tEncode, tEncodeBatch1, tEncodeForBatch, batchTransmissions, transmissionP, tParse, tDecodeClient, protocolError, encodeLGET, decodeLNK, decodeResponse, encodeNEW, encodeKEY, encodeSKEY, encodeSUB, encodeACK, encodeSEND, encodeOFF, encodeDEL, encodeGET, encodeQUE, encodePING, encodeProtocolServer, encodePRXY, encodePFWD, paddedProxiedTLength } from './dist/protocol.js';"
impProto :: String
impProto = impEnc <> impProto_
@@ -222,6 +227,14 @@ spawnJsRatchet = do
hSetBuffering hOut LineBuffering
pure (hIn, hOut, ph)
spawnJsClient :: IO (Handle, Handle, ProcessHandle)
spawnJsClient = do
let cp = (proc "node" ["dist-test/client-repl.js"]) {cwd = Just "smp-web", std_in = CreatePipe, std_out = CreatePipe, std_err = Inherit}
(Just hIn, Just hOut, _, ph) <- createProcess cp
hSetBuffering hIn LineBuffering
hSetBuffering hOut LineBuffering
pure (hIn, hOut, ph)
destroyJsRatchet :: TestPeer -> IO ()
destroyJsRatchet (TestPeerJS _ _ ph) = terminateProcess ph
destroyJsRatchet _ = pure ()
@@ -353,18 +366,19 @@ smpWebTests_ = do
<> jsUint8 entityId <> ","
<> "new Uint8Array([0x4C,0x47,0x45,0x54])"
<> ")")
tsEncoded `shouldBe` (B.singleton 0 <> hsEncoded)
tsEncoded `shouldBe` hsEncoded
it "decodeTransmission parses Haskell-encoded" $ do
it "transmissionP parses Haskell-encoded" $ do
let corrId = "abc"
entityId = B.pack [10..33]
command = "TEST"
-- Wire format: auth(ByteString) + corrId(ByteString) + entityId(ByteString) + command(rest)
encoded = smpEncode (B.empty :: B.ByteString)
<> smpEncode corrId
<> smpEncode entityId
<> command
tsResult <- callNode $ impProto
<> "const t = decodeTransmission(new Decoder(" <> jsUint8 encoded <> "));"
<> "const t = transmissionP(" <> jsUint8 encoded <> ");"
<> jsOut ("new Uint8Array([...t.corrId, ...t.entityId, ...t.command])")
tsResult `shouldBe` (corrId <> entityId <> command)
@@ -405,6 +419,18 @@ smpWebTests_ = do
describe "commands" $ do
let v = currentServerSMPRelayVersion
it "encodeNEW matches Haskell" $ do
g <- C.newRandom
(rcvAuthPub, _) <- atomically $ C.generateAuthKeyPair C.SX25519 g
(rcvDhPub, _) <- atomically $ C.generateKeyPair @'C.X25519 g
let rcvAuthPubDer = C.encodePubKey rcvAuthPub
rcvDhPubDer = C.encodePubKey rcvDhPub
cmd = NEW $ NewQueueReq rcvAuthPub rcvDhPub Nothing SMSubscribe (Just $ QRMessaging Nothing) Nothing
hsEncoded = encodeProtocol v cmd
tsEncoded <- callNode $ impProto
<> jsOut ("encodeNEW(" <> jsUint8 rcvAuthPubDer <> "," <> jsUint8 rcvDhPubDer <> ", null, true)")
tsEncoded `shouldBe` hsEncoded
it "encodeSUB matches Haskell" $ do
let hsEncoded = encodeProtocol v SUB
tsEncoded <- callNode $ impProto <> jsOut "encodeSUB()"
@@ -1084,13 +1110,11 @@ smpWebTests_ = do
<> "try {"
<> "const conn = await connectSMP('wss://localhost:" <> testPort <> "', " <> jsUint8 kh <> ", {rejectUnauthorized: false, ALPNProtocols: ['http/1.1']});"
<> "if (!conn.sndKey || !conn.rcvKey) throw new Error('no block encryption keys');"
<> "const ping = encodeBatch(encodeTransmission(new Uint8Array([0x31]), new Uint8Array(0), new Uint8Array([0x50,0x49,0x4E,0x47])));"
<> "const ping = tEncodeBatch1(null, encodeTransmission(new Uint8Array([0x31]), new Uint8Array(0), encodePING()));"
<> "sendEncryptedBlock(conn, ping);"
<> "const resp = await receiveEncryptedBlock(conn);"
<> "const d = new Decoder(resp);"
<> "d.anyByte();" -- batch count
<> "const inner = decodeLarge(d);"
<> "const t = decodeTransmission(new Decoder(inner));"
<> "const ts = tParse(resp);"
<> "const t = ts[0];"
<> jsOut ("t.command")
<> "conn.ws.close(); setTimeout(() => process.exit(0), 100);"
<> "} catch(e) { process.stderr.write('ERROR: ' + e.message + '\\n'); process.exit(1); }"
@@ -1119,15 +1143,12 @@ smpWebTests_ = do
-- 3. Connect via WSS (with block encryption)
<> "const conn = await connectSMP('wss://localhost:" <> testPort <> "', " <> jsUint8 (C.unKeyHash testKeyHash) <> ", {rejectUnauthorized: false, ALPNProtocols: ['http/1.1']});"
-- 4. Send LGET (encrypted)
<> "const lget = encodeBatch(encodeTransmission(new Uint8Array([0x31]), linkId, encodeLGET()));"
<> "const lget = tEncodeBatch1(null, encodeTransmission(new Uint8Array([0x31]), linkId, encodeLGET()));"
<> "sendEncryptedBlock(conn, lget);"
-- 5. Receive LNK response (encrypted)
<> "const resp = await receiveEncryptedBlock(conn);"
<> "const rd = new Decoder(resp);"
<> "rd.anyByte();" -- batch count
<> "const inner = decodeLarge(rd);"
<> "const t = decodeTransmission(new Decoder(inner));"
<> "const r = decodeResponse(new Decoder(t.command));"
<> "const ts = tParse(resp);"
<> "const r = decodeResponse(new Decoder(ts[0].command));"
<> "if (r.type !== 'LNK') throw new Error('expected LNK, got ' + r.type);"
-- 6. Decrypt link data
<> "const dec = decryptLinkData(sbKey, r.response.encFixedData, r.response.encUserData);"
@@ -1308,6 +1329,109 @@ smpWebTests_ = do
_ -> Left "unexpected PrivHeader"
decoded `shouldBe` Right "hello from typescript e2e"
describe "protocol/transmission" $ do
describe "sha512Hash" $ do
it "matches Haskell" $ do
let msg = "test message for hashing"
hsHash = C.sha512Hash msg
tsHash <- callNode $ impEnc
<> "import { sha512Hash } from './dist/crypto.js';"
<> jsOut ("sha512Hash(new TextEncoder().encode('test message for hashing'))")
tsHash `shouldBe` hsHash
describe "cbAuthenticator" $ do
it "matches Haskell" $ do
g <- C.newRandom
(serverPub, _) <- atomically $ C.generateKeyPair @'C.X25519 g
(_, entityPriv) <- atomically $ C.generateKeyPair @'C.X25519 g
let nonce = C.cbNonce $ B.pack [1..24]
msg = "transmission bytes to authenticate"
C.CbAuthenticator hsAuth = C.cbAuthenticate serverPub entityPriv nonce msg
C.PrivateKeyX25519 sk = entityPriv
entityPrivBytes = BA.convert sk :: B.ByteString
tsAuth <- callNode $ impSodium <> impEnc
<> "import { cbAuthenticator } from './dist/crypto.js';"
<> jsOut ("cbAuthenticator("
<> jsUint8 (C.pubKeyBytes serverPub) <> ","
<> jsUint8 entityPrivBytes <> ","
<> jsUint8 (B.pack [1..24]) <> ","
<> "new TextEncoder().encode('transmission bytes to authenticate'))")
tsAuth `shouldBe` hsAuth
describe "encodeTransmissionForAuth" $ do
it "matches Haskell" $ do
let sessId = B.pack [1..32]
corrId = B.pack [10..33]
entityId = B.pack [40..63]
command = "PING"
-- Haskell: tForAuth = sessionId <> encodeTransmission_(corrId, entityId, command)
-- tToSend = encodeTransmission_(corrId, entityId, command) [implySessId=true]
tToSend = smpEncode (corrId :: B.ByteString, entityId :: B.ByteString) <> command
tForAuth = smpEncode sessId <> tToSend
tsResult <- callNode $ impProto
<> "const r = encodeTransmissionForAuth("
<> jsUint8 sessId <> ","
<> jsUint8 corrId <> ","
<> jsUint8 entityId <> ","
<> "new Uint8Array([0x50,0x49,0x4E,0x47]));"
<> jsOut ("new Uint8Array([...r.tForAuth, 0xFF, ...r.tToSend])")
let (tsTForAuth, rest) = B.break (== 0xFF) tsResult
tsTToSend = B.drop 1 rest
tsTForAuth `shouldBe` tForAuth
tsTToSend `shouldBe` tToSend
describe "tEncodeBatch1" $ do
it "matches Haskell tEncodeBatch1" $ do
-- Haskell: tEncodeBatch1 serviceAuth=false (auth, tToSend) = lenEncode 1 `cons` Large(tEncodeAuth auth <> tToSend)
let tToSend = "corrId-entity-command-bytes"
-- No auth: tEncodeAuth Nothing = smpEncode "" = [0x00]
encoded = B.singleton 1 <> smpEncode (Large (smpEncode (B.empty :: B.ByteString) <> tToSend))
tsEncoded <- callNode $ impProto
<> jsOut ("tEncodeBatch1(null, new TextEncoder().encode('corrId-entity-command-bytes'))")
tsEncoded `shouldBe` encoded
describe "tParse" $ do
it "parses Haskell-encoded batch response" $ do
-- Build a batch with one transmission: count=1 + Large(auth + corrId + entityId + command)
let corrId = B.pack [1..24]
entityId = B.pack [30..53]
command = "PONG"
auth = B.empty -- empty auth
inner = smpEncode auth <> smpEncode corrId <> smpEncode entityId <> command
block = B.singleton 1 <> smpEncode (Large inner)
tsResult <- callNode $ impProto
<> "const ts = tParse(" <> jsUint8 block <> ");"
<> "if (ts.length !== 1) throw new Error('expected 1 transmission');"
<> jsOut ("new Uint8Array([...ts[0].corrId, ...ts[0].entityId, ...ts[0].command])")
tsResult `shouldBe` (corrId <> entityId <> command)
describe "reverseNonce" $ do
it "matches Haskell" $ do
let nonce = B.pack [1..24]
hsReversed = B.reverse nonce
tsReversed <- callNode $ impProto
<> "import { reverseNonce } from './dist/crypto.js';"
<> jsOut ("reverseNonce(" <> jsUint8 nonce <> ")")
tsReversed `shouldBe` hsReversed
describe "encodeProtocolServer" $ do
it "matches Haskell" $ do
let srv = SMPServer ("smp1.example.com" :| ["smp2.example.com"]) "5223" (C.KeyHash $ B.pack [1..32])
hsEncoded = smpEncode srv
tsEncoded <- callNode $ impProto
<> jsOut ("encodeProtocolServer(['smp1.example.com','smp2.example.com'], '5223', " <> jsUint8 (B.pack [1..32]) <> ")")
tsEncoded `shouldBe` hsEncoded
describe "encodePRXY" $ do
it "matches Haskell" $ do
let srv = SMPServer ("relay.example.com" :| []) "" (C.KeyHash $ B.pack [1..32])
cmd = Cmd SProxiedClient $ PRXY srv Nothing
v = currentServerSMPRelayVersion
hsEncoded = encodeProtocol v cmd
tsEncoded <- callNode $ impProto
<> jsOut ("encodePRXY(['relay.example.com'], '', " <> jsUint8 (B.pack [1..32]) <> ", null)")
tsEncoded `shouldBe` hsEncoded
describe "full-stack" $ do
it "Haskell encodes all layers, TypeScript decodes" $ do
g <- C.newRandom
@@ -1441,3 +1565,273 @@ smpWebTests_ = do
Right (AP.AgentMessage _ (AP.A_MSG body)) <- pure $ smpDecode agentMsgBytes
body `shouldBe` "hello from ts full stack"
describe "client" $ do
it "JS client REPL: PING/PONG via SMP server" $ do
let msType = ASType SQSMemory SMSJournal
attachStaticAndWS "tests/fixtures" $ \attachHTTP ->
withSmpServerConfig (cfgWebOn msType testPort) (Just attachHTTP) $ \_ -> do
(hIn, hOut, ph) <- spawnJsClient
let C.KeyHash kh = testKeyHash
resp <- jsCmd hIn hOut $ "CONNECT wss://localhost:" <> testPort <> " " <> bsToHex kh <> " " <> "{\"rejectUnauthorized\":false,\"ALPNProtocols\":[\"http/1.1\"]}"
resp `shouldBe` "ok"
pingResp <- jsCmd hIn hOut "PING"
pingResp `shouldBe` "ok"
_ <- jsCmd hIn hOut "CLOSE"
terminateProcess ph
it "authTransmission full batch block matches Haskell" $ do
g <- C.newRandom
-- Known inputs
let sessId = B.pack [1..48] -- 48-byte sessionId like real server
corrId = B.pack [50..73] -- 24-byte corrId/nonce
(serverPub, _serverPriv) <- atomically $ C.generateKeyPair @'C.X25519 g
(_, entityPriv) <- atomically $ C.generateKeyPair @'C.X25519 g
let C.PrivateKeyX25519 sk = entityPriv
entityPrivBytes = BA.convert sk :: B.ByteString
nonce = C.cbNonce corrId
-- Haskell: encodeTransmissionForAuth
tToSend = smpEncode (corrId :: B.ByteString, B.empty :: B.ByteString) <> "PING"
tForAuth = smpEncode sessId <> tToSend
-- Haskell: cbAuthenticate
hsAuth = C.cbAuthenticate serverPub entityPriv nonce tForAuth
C.CbAuthenticator hsAuthBytes = hsAuth
-- Haskell: tEncodeBatch1 (with serviceAuth=true: auth + Nothing serviceSig)
hsBlock = B.singleton 1 <> smpEncode (Large (smpEncode hsAuthBytes <> smpEncode (Nothing :: Maybe B.ByteString) <> tToSend))
-- TypeScript: same computation
tsBlock <- callNode $ impSodium <> impProto
<> "const sessId = " <> jsUint8 sessId <> ";"
<> "const corrId = " <> jsUint8 corrId <> ";"
<> "const serverPub = " <> jsUint8 (C.pubKeyBytes serverPub) <> ";"
<> "const entityPriv = " <> jsUint8 entityPrivBytes <> ";"
<> "const {tForAuth, tToSend} = encodeTransmissionForAuth(sessId, corrId, new Uint8Array(0), encodePING());"
<> "const auth = authTransmission(serverPub, {type:'x25519', key:entityPriv}, corrId, tForAuth);"
<> jsOut ("tEncodeBatch1(auth, tToSend)")
tsBlock `shouldBe` hsBlock
it "JS client REPL: create queue and send/receive message" $ do
let msType = ASType SQSMemory SMSJournal
attachStaticAndWS "tests/fixtures" $ \attachHTTP ->
withSmpServerConfig (cfgWebOn msType testPort) (Just attachHTTP) $ \_ -> do
-- Generate auth keys (X25519 DH auth for v7+)
g <- C.newRandom
(rcvAuthPub, rcvAuthPriv) <- atomically $ C.generateKeyPair @'C.X25519 g
let rcvAuthPubDer = C.encodePubKey rcvAuthPub
C.PrivateKeyX25519 sk = rcvAuthPriv
rcvAuthPrivBytes = BA.convert sk :: B.ByteString
(sndAuthPub, sndAuthPriv) <- atomically $ C.generateKeyPair @'C.X25519 g
let sndAuthPubDer = C.encodePubKey sndAuthPub
C.PrivateKeyX25519 sndSk = sndAuthPriv
sndAuthPrivBytes = BA.convert sndSk :: B.ByteString
-- Spawn receiver and sender clients
(rcvIn, rcvOut, rcvPh) <- spawnJsClient
(sndIn, sndOut, sndPh) <- spawnJsClient
let connectCmd = "CONNECT wss://localhost:" <> testPort <> " " <> bsToHex (C.unKeyHash testKeyHash) <> " {\"rejectUnauthorized\":false,\"ALPNProtocols\":[\"http/1.1\"]}"
"ok" <- jsCmd rcvIn rcvOut connectCmd
"ok" <- jsCmd sndIn sndOut connectCmd
-- Receiver: create queue (REPL generates DH keypair internally)
newResp <- jsCmd rcvIn rcvOut $ "NEW " <> bsToHex rcvAuthPubDer <> " " <> bsToHex rcvAuthPrivBytes
let newParts = words newResp
when (head newParts /= "ok:") $ expectationFailure $ "NEW failed: " <> newResp
head newParts `shouldBe` "ok:"
let rcvIdHex = newParts !! 1
sndIdHex = newParts !! 2
-- Receiver: secure queue with sender's public key
keyResp <- jsCmd rcvIn rcvOut $ "KEY " <> rcvIdHex <> " " <> bsToHex rcvAuthPrivBytes <> " " <> bsToHex sndAuthPubDer
when (keyResp /= "ok") $ expectationFailure $ "KEY failed: " <> keyResp
-- Receiver: subscribe
subResp <- jsCmd rcvIn rcvOut $ "SUB " <> rcvIdHex <> " " <> bsToHex rcvAuthPrivBytes
when (subResp /= "ok") $ expectationFailure $ "SUB failed: " <> subResp
-- Sender: send message (with auth)
let testMsg = "hello from sender"
"ok" <- jsCmd sndIn sndOut $ "SEND " <> sndIdHex <> " " <> bsToHex sndAuthPrivBytes <> " 1 " <> bsToHex testMsg
-- Receiver: receive and decrypt message
recvResp <- jsCmd rcvIn rcvOut "RECV 5000"
let recvParts = words recvResp
head recvParts `shouldBe` "ok:"
length recvParts `shouldSatisfy` (>= 4)
let bodyHex = recvParts !! 3
hexToBS bodyHex `shouldBe` testMsg
-- Cleanup
_ <- jsCmd rcvIn rcvOut "CLOSE"
_ <- jsCmd sndIn sndOut "CLOSE"
terminateProcess rcvPh
terminateProcess sndPh
it "cross-language: HS sender, JS receiver" $ do
let msType = ASType SQSMemory SMSJournal
attachStaticAndWS "tests/fixtures" $ \attachHTTP ->
withSmpServerConfig (cfgWebOn msType testPort) (Just attachHTTP) $ \_ -> do
g <- C.newRandom
-- JS receiver: connect, create queue
(rcvIn, rcvOut, rcvPh) <- spawnJsClient
"ok" <- jsCmd rcvIn rcvOut $ "CONNECT wss://localhost:" <> testPort <> " " <> bsToHex (C.unKeyHash testKeyHash) <> " {\"rejectUnauthorized\":false,\"ALPNProtocols\":[\"http/1.1\"]}"
(rcvAuthPub, rcvAuthPriv) <- atomically $ C.generateKeyPair @'C.X25519 g
let C.PrivateKeyX25519 sk = rcvAuthPriv
rcvAuthPrivBytes = BA.convert sk :: B.ByteString
newResp <- jsCmd rcvIn rcvOut $ "NEW " <> bsToHex (C.encodePubKey rcvAuthPub) <> " " <> bsToHex rcvAuthPrivBytes
let newParts = words newResp
when (head newParts /= "ok:") $ expectationFailure $ "NEW failed: " <> newResp
let sndIdHex = newParts !! 2
rcvIdHex = newParts !! 1
sndId = hexToBS sndIdHex
-- JS receiver: subscribe (no KEY — sender sends unsigned)
subResp <- jsCmd rcvIn rcvOut $ "SUB " <> rcvIdHex <> " " <> bsToHex rcvAuthPrivBytes
when (subResp /= "ok") $ expectationFailure $ "SUB failed: " <> subResp
-- HS sender: connect via TLS, send message
testSMPClient @TLS $ \sh -> do
let testMsg = "hello from haskell"
Resp _ _ ok <- sendRecv sh (Nothing, "1234", EntityId sndId, _SEND testMsg)
ok `shouldBe` OK
-- JS receiver: receive and decrypt
recvResp <- jsCmd rcvIn rcvOut "RECV 5000"
let recvParts = words recvResp
head recvParts `shouldBe` "ok:"
let bodyHex = recvParts !! 3
hexToBS bodyHex `shouldBe` testMsg
_ <- jsCmd rcvIn rcvOut "CLOSE"
terminateProcess rcvPh
it "cross-language: JS sender, HS receiver" $ do
let msType = ASType SQSMemory SMSJournal
attachStaticAndWS "tests/fixtures" $ \attachHTTP ->
withSmpServerConfig (cfgWebOn msType testPort) (Just attachHTTP) $ \_ -> do
g <- C.newRandom
-- HS receiver: connect via TLS, create queue
testSMPClient @TLS $ \rh -> do
(rPub, rKey) <- atomically $ C.generateAuthKeyPair C.SEd448 g
(dhPub, dhPriv :: C.PrivateKeyX25519) <- atomically $ C.generateKeyPair g
Resp "abcd" _ (Ids _rId sId srvDh) <- signSendRecv rh rKey ("abcd", NoEntity, New rPub dhPub)
let dec = decryptMsgV3 $ C.dh' srvDh dhPriv
-- JS sender: connect via WebSocket, send message
(sndIn, sndOut, sndPh) <- spawnJsClient
"ok" <- jsCmd sndIn sndOut $ "CONNECT wss://localhost:" <> testPort <> " " <> bsToHex (C.unKeyHash testKeyHash) <> " {\"rejectUnauthorized\":false,\"ALPNProtocols\":[\"http/1.1\"]}"
let testMsg = "hello from typescript"
"ok" <- jsCmd sndIn sndOut $ "SEND " <> bsToHex (unEntityId sId) <> " none 1 " <> bsToHex testMsg
-- HS receiver: receive and decrypt
Resp "" _ (Msg mId1 msg1) <- tGet1 rh
dec mId1 msg1 `shouldBe` Right testMsg
Resp "bcda" _ OK <- signSendRecv rh rKey ("bcda", _rId, ACK mId1)
_ <- jsCmd sndIn sndOut "CLOSE"
terminateProcess sndPh
it "cross-language: JS sends via proxy to HS receiver" $ do
let msType = ASType SQSMemory SMSJournal
-- Proxy server with WebSocket: enable proxy on the web-enabled config
proxyCfgWeb = updateCfg (cfgWebOn msType testPort) $ \cfg' ->
cfg' {allowSMPProxy = True}
-- Relay server on testPort2: standard config
relayCfg = journalCfg (cfgMS msType) testStoreLogFile2 testStoreMsgsDir2
attachStaticAndWS "tests/fixtures" $ \attachHTTP ->
withSmpServerConfig proxyCfgWeb (Just attachHTTP) $ \_ ->
withSmpServerConfigOn (transport @TLS) relayCfg testPort2 $ \_ -> do
g <- C.newRandom
-- HS receiver: create queue on RELAY server via TLS
let (h :| _) = testHost2
testSMPClient_ @TLS h testPort2 supportedClientSMPRelayVRange $ \rh -> do
(rPub, rKey) <- atomically $ C.generateAuthKeyPair C.SEd448 g
(dhPub, dhPriv :: C.PrivateKeyX25519) <- atomically $ C.generateKeyPair g
Resp "abcd" _ (Ids _rId sId srvDh) <- signSendRecv rh rKey ("abcd", NoEntity, New rPub dhPub)
let dec = decryptMsgV3 $ C.dh' srvDh dhPriv
-- JS sender: connect to PROXY via WebSocket, send PRXY for relay
(sndIn, sndOut, sndPh) <- spawnJsClient
"ok" <- jsCmd sndIn sndOut $ "CONNECT wss://localhost:" <> testPort <> " " <> bsToHex (C.unKeyHash testKeyHash) <> " {\"rejectUnauthorized\":false,\"ALPNProtocols\":[\"http/1.1\"]}"
-- Connect to relay via proxy
prxyResp <- jsCmd sndIn sndOut $ "PRXY localhost " <> testPort2 <> " " <> bsToHex (C.unKeyHash testKeyHash)
when (not $ "ok:" `isPrefixOf` prxyResp) $ expectationFailure $ "PRXY failed: " <> prxyResp
-- Send message via proxy
let testMsg = "hello via proxy"
sendResp <- jsCmd sndIn sndOut $ "PSEND " <> bsToHex (unEntityId sId) <> " none 1 " <> bsToHex testMsg
when (sendResp /= "ok") $ expectationFailure $ "PSEND failed: " <> sendResp
-- HS receiver: receive and decrypt
Resp "" _ (Msg mId1 msg1) <- tGet1 rh
dec mId1 msg1 `shouldBe` Right testMsg
_ <- jsCmd sndIn sndOut "CLOSE"
terminateProcess sndPh
it "JS batch subscribe: create 3 queues, batch subscribe, receive messages" $ do
let msType = ASType SQSMemory SMSJournal
attachStaticAndWS "tests/fixtures" $ \attachHTTP ->
withSmpServerConfig (cfgWebOn msType testPort) (Just attachHTTP) $ \_ -> do
g <- C.newRandom
-- JS receiver
(rcvIn, rcvOut, rcvPh) <- spawnJsClient
"ok" <- jsCmd rcvIn rcvOut $ "CONNECT wss://localhost:" <> testPort <> " " <> bsToHex (C.unKeyHash testKeyHash) <> " {\"rejectUnauthorized\":false,\"ALPNProtocols\":[\"http/1.1\"]}"
-- Create 3 queues
sndIds <- forM [1..3 :: Int] $ \_ -> do
(authPub, authPriv) <- atomically $ C.generateKeyPair @'C.X25519 g
let C.PrivateKeyX25519 sk = authPriv
privBytes = BA.convert sk :: B.ByteString
newResp <- jsCmd rcvIn rcvOut $ "NEW " <> bsToHex (C.encodePubKey authPub) <> " " <> bsToHex privBytes
let newParts = words newResp
when (head newParts /= "ok:") $ expectationFailure $ "NEW failed: " <> newResp
pure (newParts !! 1, newParts !! 2, bsToHex privBytes) -- rcvIdHex, sndIdHex, privKeyHex
-- Batch subscribe
let bsubArg = unwords [rcvId <> ":" <> pk | (rcvId, _, pk) <- sndIds]
bsubResp <- jsCmd rcvIn rcvOut $ "BSUB " <> bsubArg
when (bsubResp /= "ok") $ expectationFailure $ "BSUB failed: " <> bsubResp
-- HS sender: send a message to each queue
testSMPClient @TLS $ \sh -> do
forM_ (zip [1..] sndIds) $ \(i :: Int, (_, sndIdHex, _)) -> do
let sndId = hexToBS sndIdHex
msg = "batch msg " <> BC.pack (show i)
Resp _ _ OK <- sendRecv sh (Nothing, BC.pack (show i), EntityId sndId, _SEND msg)
pure ()
-- JS receiver: receive 3 messages
forM_ [1..3 :: Int] $ \i -> do
recvResp <- jsCmd rcvIn rcvOut "RECV 5000"
let recvParts = words recvResp
head recvParts `shouldBe` "ok:"
let bodyHex = recvParts !! 3
hexToBS bodyHex `shouldBe` ("batch msg " <> BC.pack (show i))
_ <- jsCmd rcvIn rcvOut "CLOSE"
terminateProcess rcvPh
it "cross-language: HS sends via proxy to JS receiver (one server)" $ do
let msType = ASType SQSMemory SMSJournal
-- One server: WebSocket + proxy enabled (like oneServer in SMPProxyTests)
proxyCfgWeb = updateCfg (cfgWebOn msType testPort) $ \cfg' ->
cfg' {allowSMPProxy = True}
attachStaticAndWS "tests/fixtures" $ \attachHTTP ->
withSmpServerConfig proxyCfgWeb (Just attachHTTP) $ \_ -> do
g <- C.newRandom
-- JS receiver: create queue on server via WebSocket
(rcvIn, rcvOut, rcvPh) <- spawnJsClient
"ok" <- jsCmd rcvIn rcvOut $ "CONNECT wss://localhost:" <> testPort <> " " <> bsToHex (C.unKeyHash testKeyHash) <> " {\"rejectUnauthorized\":false,\"ALPNProtocols\":[\"http/1.1\"]}"
(rcvAuthPub, rcvAuthPriv) <- atomically $ C.generateKeyPair @'C.X25519 g
let C.PrivateKeyX25519 sk = rcvAuthPriv
rcvAuthPrivBytes = BA.convert sk :: B.ByteString
newResp <- jsCmd rcvIn rcvOut $ "NEW " <> bsToHex (C.encodePubKey rcvAuthPub) <> " " <> bsToHex rcvAuthPrivBytes
let newParts = words newResp
when (head newParts /= "ok:") $ expectationFailure $ "NEW failed: " <> newResp
let sndIdHex = newParts !! 2
rcvIdHex = newParts !! 1
sndId = hexToBS sndIdHex
-- JS receiver: subscribe
subResp <- jsCmd rcvIn rcvOut $ "SUB " <> rcvIdHex <> " " <> bsToHex rcvAuthPrivBytes
when (subResp /= "ok") $ expectationFailure $ "SUB failed: " <> subResp
-- HS sender: connect via TLS, use proxy to send to SAME server
let srv = SMPServer ("localhost" :| []) testPort testKeyHash
ts <- getCurrentTime
Right pc <- getProtocolClient g NRMInteractive (1, srv, Nothing)
defaultSMPClientConfig {serverVRange = mkVersionRange minServerSMPRelayVersion currentClientSMPRelayVersion}
[] Nothing ts (\_ -> pure ())
-- Connect proxy session to same server
sess <- runRight $ connectSMPProxiedRelay pc NRMInteractive srv Nothing
-- Send via proxy
let testMsg = "hello from haskell via proxy"
Right (Right ()) <- runExceptT $ proxySMPMessage pc NRMInteractive sess Nothing (EntityId sndId) noMsgFlags testMsg
-- JS receiver: receive and decrypt
recvResp <- jsCmd rcvIn rcvOut "RECV 5000"
let recvParts = words recvResp
head recvParts `shouldBe` "ok:"
let bodyHex = recvParts !! 3
hexToBS bodyHex `shouldBe` testMsg
closeProtocolClient pc
_ <- jsCmd rcvIn rcvOut "CLOSE"
terminateProcess rcvPh