mirror of
https://github.com/simplex-chat/simplexmq.git
synced 2026-07-28 18:40:28 +00:00
smp web: agent store
This commit is contained in:
Evgeny @ SimpleX Chat
@@ -0,0 +1,195 @@
|
||||
# Agent for Browser: Transpilation Breakdown
|
||||
|
||||
**Parent**: [SMP Client MVP](./2026-05-20-client-mvp.md)
|
||||
**Depends on**: SMP Client (complete, 96 tests), encoding/encryption spike (complete)
|
||||
|
||||
## Rule
|
||||
|
||||
Every TypeScript function is a faithful transpilation of a specific Haskell function. Same name, same steps, same call chain. No inferences.
|
||||
|
||||
## Scope
|
||||
|
||||
The web widget JOINS connections (never creates addresses). It sends and receives messages. It handles the connection handshake. It does NOT create invitations, manage notifications, transfer files, or do remote control.
|
||||
|
||||
## Architecture difference from Haskell
|
||||
|
||||
Haskell agent uses SQLite + multiple background threads (subscriber, delivery workers, cleanup manager, NTF supervisor). Browser agent uses IndexedDB + event-driven architecture (WebSocket onmessage, Promises, no threads).
|
||||
|
||||
The protocol logic is identical. The concurrency model differs. The store interface differs. The transpilation focuses on the protocol logic.
|
||||
|
||||
## Breakdown into testable pieces
|
||||
|
||||
### Piece 1: Agent protocol types (Agent/Protocol.hs)
|
||||
|
||||
Already partially done (AgentMsgEnvelope, AgentMessage, APrivHeader, AMessage). What's missing for the handshake:
|
||||
|
||||
| Type | Haskell location | What's needed |
|
||||
|------|-----------------|---------------|
|
||||
| `SMPQueueInfo` | `Agent/Protocol.hs:1310-1327` | Binary encode/decode — version-dependent, complex |
|
||||
| `SMPQueueUri` | `Agent/Protocol.hs:1344-1431` | Binary encode/decode + string encode/decode |
|
||||
| `SMPQueueAddress` | `Agent/Protocol.hs:1350-1356` | `{smpServer, senderId, dhPublicKey, queueMode}` |
|
||||
| `ConnectionRequestUri` | `Agent/Protocol.hs:1436-1441` | Binary encode/decode: `CRInvitationUri` + `CRContactUri` |
|
||||
| `ConnReqUriData` | `Agent/Protocol.hs:1728-1734` | Binary encode/decode: `{crAgentVRange, crSmpQueues, crClientData}` |
|
||||
| `SMPConfirmation` | `Agent/Protocol.hs:798-810` | Not wire-encoded — internal data structure for confirmation handling |
|
||||
| `E2ERatchetParams` | `Crypto/Ratchet.hs:223-241` | Already have encode/decode in ratchet.ts — need to verify completeness |
|
||||
|
||||
**Test**: each encode/decode function tested byte-for-byte against Haskell via callNode.
|
||||
|
||||
### Piece 2: Connection handshake — joinConnection (Agent.hs)
|
||||
|
||||
The join flow, transpiled step by step:
|
||||
|
||||
```
|
||||
joinConnection (Agent.hs:~1200-1300)
|
||||
1. Parse ConnectionRequestUri (already have URI parsing)
|
||||
2. Create RcvQueue on a selected server (newRcvQueue — Agent/Client.hs:1373)
|
||||
- Generate X25519 DH keypair for queue
|
||||
- Generate X25519/Ed25519 auth keypair
|
||||
- Call createSMPQueue on SMP client
|
||||
- Get back rcvId, sndId, srvDhKey
|
||||
3. Store connection + queue in database
|
||||
4. Generate X448 E2E ratchet params (generateRcvE2EParams — already have)
|
||||
5. Build ConnInfo (profile data)
|
||||
6. Encrypt ConnInfo with ratchet → encConnInfo
|
||||
7. Build AgentConfirmation envelope
|
||||
8. Wrap in ClientMessage + per-queue E2E encrypt → ClientMsgEnvelope
|
||||
9. Send via SMP SEND to the contact address queue
|
||||
10. Subscribe to own receive queue (SUB)
|
||||
11. Return connection ID
|
||||
```
|
||||
|
||||
Each step is independently testable. The full flow is an integration test.
|
||||
|
||||
**Key Haskell functions to transpile:**
|
||||
|
||||
| Function | File:lines | What it does |
|
||||
|----------|-----------|--------------|
|
||||
| `joinConnection` | `Agent.hs:~1200` | Top-level join |
|
||||
| `joinConn` | `Agent.hs:~1230` | Internal join logic |
|
||||
| `newRcvQueue` | `Agent/Client.hs:1373-1420` | Create queue on server |
|
||||
| `sendConfirmation` | `Agent/Client.hs:1788-1794` | Encrypt+send confirmation |
|
||||
| `sendInvitation` | `Agent/Client.hs:1796-1806` | Encrypt+send invitation |
|
||||
| `mkAgentConfirmation` | `Agent.hs:~3700` | Build confirmation envelope |
|
||||
| `agentCbEncrypt` | `Agent/Client.hs:2074-2082` | Per-queue E2E encrypt (already have) |
|
||||
|
||||
### Piece 3: Message processing — subscriber (Agent.hs)
|
||||
|
||||
Incoming message handling:
|
||||
|
||||
```
|
||||
subscriber (Agent.hs:2912-2919)
|
||||
→ reads from msgQ (populated by SMP client's onMessage callback)
|
||||
→ processSMPTransmissions (Agent.hs:2997-3297)
|
||||
→ for each transmission:
|
||||
→ STEvent (server push MSG):
|
||||
→ decryptClientMessage (per-queue E2E decrypt)
|
||||
→ parse AgentMsgEnvelope
|
||||
→ for AgentMsgEnvelope 'M':
|
||||
→ agentRatchetDecrypt (double ratchet decrypt)
|
||||
→ parse AgentMessage
|
||||
→ dispatch on AMessage type:
|
||||
→ HELLO: complete handshake
|
||||
→ A_MSG body: deliver to user
|
||||
→ A_RCVD: delivery receipt
|
||||
→ QADD/QKEY/QUSE/QTEST: queue switching (skip for MVP)
|
||||
→ EREADY: ratchet sync (skip for MVP)
|
||||
```
|
||||
|
||||
**Key Haskell functions to transpile:**
|
||||
|
||||
| Function | File:lines | What it does |
|
||||
|----------|-----------|--------------|
|
||||
| `processSMPTransmissions` | `Agent.hs:2997-3297` | Top-level message dispatcher |
|
||||
| `decryptClientMessage` | `Agent.hs:3282-3293` | Per-queue E2E decrypt + parse envelope |
|
||||
| `agentRatchetDecrypt` | `Agent.hs:3757-3767` | Ratchet decrypt + store update |
|
||||
| `helloMsg` | `Agent.hs:~3400` | Process HELLO (complete handshake) |
|
||||
| `smpConfirmation` | `Agent.hs:~3500` | Process received confirmation |
|
||||
| `smpInvitation` | `Agent.hs:~3600` | Process received invitation |
|
||||
|
||||
### Piece 4: Message sending — sendMessage (Agent.hs)
|
||||
|
||||
```
|
||||
sendMessage (Agent.hs:~1500)
|
||||
→ enqueueMessageB
|
||||
→ agentRatchetEncryptHeader (get ratchet encrypt key)
|
||||
→ rcEncryptMsg (encrypt message body)
|
||||
→ store encrypted message in DB
|
||||
→ createSndMsgDelivery (link to queue)
|
||||
→ delivery worker sends via SMP SEND
|
||||
```
|
||||
|
||||
**Key Haskell functions to transpile:**
|
||||
|
||||
| Function | File:lines | What it does |
|
||||
|----------|-----------|--------------|
|
||||
| `sendMessage` | `Agent.hs:~1500` | Top-level send |
|
||||
| `enqueueMessageB` | `Agent.hs:~2020-2060` | Encode + encrypt + store |
|
||||
| `agentRatchetEncrypt` | `Agent.hs:3742-3746` | Ratchet encrypt |
|
||||
| `agentRatchetEncryptHeader` | `Agent.hs:3748-3754` | Ratchet encrypt header |
|
||||
| `encodeAgentMsgStr` | `Agent.hs:2050-2054` | Encode AgentMessage to bytes |
|
||||
| `runSmpQueueMsgDelivery` | `Agent.hs:2092-2220` | Delivery worker — read from DB, send via SMP |
|
||||
|
||||
### Piece 5: Message acknowledgment (Agent.hs)
|
||||
|
||||
```
|
||||
ackMessage (Agent.hs:~1550)
|
||||
→ withStore: mark message as acknowledged
|
||||
→ send ACK to SMP server
|
||||
→ optionally send delivery receipt (A_RCVD)
|
||||
```
|
||||
|
||||
### Piece 6: Store interface (IndexedDB)
|
||||
|
||||
The agent uses ~50 distinct store operations. For the web widget MVP, we need:
|
||||
|
||||
| Store operation | What it does | Used by |
|
||||
|----------------|--------------|---------|
|
||||
| `createConnection` | Create connection record | joinConnection |
|
||||
| `getConn` | Get connection by ID | all operations |
|
||||
| `updateRcvIds` | Increment receive IDs | message processing |
|
||||
| `createRcvMsg` | Store received message | message processing |
|
||||
| `getRatchetForUpdate` | Get ratchet state for modify | encrypt/decrypt |
|
||||
| `updateRatchet` | Store updated ratchet | encrypt/decrypt |
|
||||
| `getSkippedMsgKeys` | Get skipped message keys | ratchet decrypt |
|
||||
| `createSndMsg` | Store sent message | sendMessage |
|
||||
| `createSndMsgDelivery` | Link message to queue | sendMessage |
|
||||
| `getPendingQueueMsg` | Get next message to send | delivery worker |
|
||||
| `updateSndMsgStatus` | Update delivery status | delivery worker |
|
||||
| `deleteMsg` | Delete after ACK | ackMessage |
|
||||
|
||||
## Implementation order
|
||||
|
||||
1. **Piece 1**: Agent protocol types — `SMPQueueInfo`, `SMPQueueUri`, `ConnectionRequestUri`, `ConnReqUriData` encode/decode. Test each against Haskell.
|
||||
2. **Piece 6**: Store interface — define TypeScript interface matching the needed operations. Implement with in-memory Map first (for testing), IndexedDB later.
|
||||
3. **Piece 4**: Message sending — `sendMessage` + `agentRatchetEncrypt` + delivery. Test: encrypt in TS, decrypt in HS.
|
||||
4. **Piece 3**: Message processing — `processSMPTransmissions` + `agentRatchetDecrypt`. Test: encrypt in HS, decrypt in TS.
|
||||
5. **Piece 2**: Connection handshake — `joinConnection`. Test: TS joins, HS accepts, messages flow.
|
||||
6. **Piece 5**: Message acknowledgment — `ackMessage`. Test: full roundtrip with ack.
|
||||
|
||||
Each piece is independently testable. Piece 1 uses callNode (no server). Pieces 3-5 use the SMP client REPL + real server. Piece 2 is the integration test that ties everything together.
|
||||
|
||||
## What to skip
|
||||
|
||||
| Feature | Why skip | Haskell functions |
|
||||
|---------|----------|-------------------|
|
||||
| Queue switching | Additive, not needed for MVP | `switchConnection`, QADD/QKEY/QUSE/QTEST handling |
|
||||
| Ratchet sync | MVP shows error, suggests reconnecting | `synchronizeRatchet`, EREADY handling |
|
||||
| Notifications | No webpush yet | All NTF functions |
|
||||
| File transfer | Separate protocol | All XFTP functions |
|
||||
| Remote control | Not in scope | All RC functions |
|
||||
| Client notices | Server admin feature | `processClientNotices` |
|
||||
| Cleanup manager | Can do manual cleanup | `cleanupManager` |
|
||||
| Server management | Configured at init | `setProtocolServers`, `testProtocolServer` |
|
||||
| Connection creation | Widget only joins | `createConnection`, short links |
|
||||
| Delivery receipts | Can add later | A_RCVD handling, receipt sending |
|
||||
| Multiple receive queues | Single queue per connection for MVP | Queue replacement logic |
|
||||
|
||||
## Files
|
||||
|
||||
| File | Action |
|
||||
|------|--------|
|
||||
| `smp-web/src/agent/protocol.ts` | Extend with SMPQueueInfo, ConnectionRequestUri, etc. |
|
||||
| `smp-web/src/agent/store.ts` | New — store interface + in-memory implementation |
|
||||
| `smp-web/src/agent/agent.ts` | New — agent logic: join, send, receive, ack |
|
||||
| `smp-web/tests/agent-repl.ts` | New — agent REPL for testing (or extend client-repl) |
|
||||
| `tests/SMPWebTests.hs` | Agent tests |
|
||||
@@ -0,0 +1,260 @@
|
||||
# Agent API Inventory for Web Widget
|
||||
|
||||
Every exported function from `Agent.hs`, classified as MVP / post-MVP / skip, with reasoning.
|
||||
|
||||
## Context
|
||||
|
||||
The web widget:
|
||||
- Joins existing connections via addresses hardcoded in the widget or sent as simplex links
|
||||
- Sends and receives messages
|
||||
- Does NOT create addresses, invitation links, or short links
|
||||
- Does NOT transfer files (post-MVP)
|
||||
- Does NOT manage notifications (post-MVP)
|
||||
- Does NOT rotate queues
|
||||
- Must accept ratchet re-sync initiated by the other side (but does not initiate)
|
||||
|
||||
## Lifecycle
|
||||
|
||||
| Function | MVP? | Reason |
|
||||
|----------|------|--------|
|
||||
| `getSMPAgentClient` | YES | Initialize agent — required |
|
||||
| `getSMPAgentClient_` | NO | Variant with extra params, not needed |
|
||||
| `disconnectAgentClient` | YES | Clean shutdown — required |
|
||||
| `disposeAgentClient` | NO | Hard shutdown — disconnectAgentClient is sufficient |
|
||||
| `resumeAgentClient` | NO | Widget doesn't suspend/resume — runs while page is open |
|
||||
| `foregroundAgent` | NO | Mobile-only concept |
|
||||
| `suspendAgent` | NO | Mobile-only concept |
|
||||
|
||||
## User management
|
||||
|
||||
| Function | MVP? | Reason |
|
||||
|----------|------|--------|
|
||||
| `createUser` | YES | Widget needs at least one user to own connections |
|
||||
| `deleteUser` | NO | Widget doesn't delete users — page reload is cleanup |
|
||||
|
||||
## Connection creation (widget never creates)
|
||||
|
||||
| Function | MVP? | Reason |
|
||||
|----------|------|--------|
|
||||
| `createConnection` | NO | Widget joins, never creates |
|
||||
| `createConnectionAsync` | NO | Same |
|
||||
| `prepareConnectionLink` | NO | For creating links |
|
||||
| `createConnectionForLink` | NO | For creating links |
|
||||
| `setConnShortLink` | NO | For creating short links |
|
||||
| `setConnShortLinkAsync` | NO | Same |
|
||||
| `deleteConnShortLink` | NO | For managing short links |
|
||||
| `getConnShortLink` | NO | For reading short links — widget uses hardcoded address |
|
||||
| `getConnShortLinkAsync` | NO | Same |
|
||||
| `getConnLinkPrivKey` | NO | For link management |
|
||||
| `deleteLocalInvShortLink` | NO | For link cleanup |
|
||||
| `changeConnectionUser` | NO | Widget has one user |
|
||||
|
||||
## Joining connections
|
||||
|
||||
| Function | MVP? | Reason |
|
||||
|----------|------|--------|
|
||||
| `prepareConnectionToJoin` | YES | Create connection record before join — prevents race with incoming confirmation |
|
||||
| `joinConnection` | YES | Core function — join via address URI |
|
||||
| `joinConnectionAsync` | NO | Async variant — widget can use sync joinConnection with await |
|
||||
| `connRequestPQSupport` | YES | Determine PQ support from connection request — needed for join |
|
||||
|
||||
## Handshake (accepting incoming)
|
||||
|
||||
| Function | MVP? | Reason |
|
||||
|----------|------|--------|
|
||||
| `allowConnection` | YES | Allow connection after receiving CONF — part of handshake |
|
||||
| `allowConnectionAsync` | NO | Async variant |
|
||||
| `acceptContact` | YES | Accept contact request (for group join flow) |
|
||||
| `acceptContactAsync` | NO | Async variant |
|
||||
| `prepareConnectionToAccept` | YES | Prepare for accept — same race prevention as prepareConnectionToJoin |
|
||||
| `rejectContact` | NO | Widget doesn't reject — it always accepts |
|
||||
|
||||
## Subscription
|
||||
|
||||
| Function | MVP? | Reason |
|
||||
|----------|------|--------|
|
||||
| `subscribeConnection` | YES | Subscribe to receive messages on one connection |
|
||||
| `subscribeConnections` | YES | Batch subscribe — needed after page reload |
|
||||
| `subscribeAllConnections` | YES | Subscribe everything for a user — simplest for widget |
|
||||
| `resubscribeConnection` | YES | Resubscribe after network recovery |
|
||||
| `resubscribeConnections` | YES | Batch resubscribe |
|
||||
| `getConnectionMessages` | NO | Fetch stored messages — widget processes messages as they arrive |
|
||||
| `getNotificationConns` | NO | Push notification management |
|
||||
| `subscribeClientService` | NO | Service certificate management |
|
||||
|
||||
## Messaging
|
||||
|
||||
| Function | MVP? | Reason |
|
||||
|----------|------|--------|
|
||||
| `sendMessage` | YES | Send a message — core function |
|
||||
| `sendMessages` | NO | Batch send — sendMessage is sufficient for MVP |
|
||||
| `sendMessagesB` | NO | Batch send with error handling — optimization |
|
||||
| `ackMessage` | YES | Acknowledge received message — required for protocol correctness |
|
||||
| `ackMessageAsync` | NO | Async variant |
|
||||
|
||||
## Queue management
|
||||
|
||||
| Function | MVP? | Reason |
|
||||
|----------|------|--------|
|
||||
| `switchConnection` | NO | Queue rotation — not needed |
|
||||
| `switchConnectionAsync` | NO | Same |
|
||||
| `abortConnectionSwitch` | NO | Cancel rotation |
|
||||
| `getConnectionQueueInfo` | NO | Debug info |
|
||||
| `suspendConnection` | NO | Widget deletes or ignores, doesn't suspend |
|
||||
|
||||
## Ratchet
|
||||
|
||||
| Function | MVP? | Reason |
|
||||
|----------|------|--------|
|
||||
| `synchronizeRatchet` | NO | Widget doesn't initiate ratchet sync. But MUST handle incoming EREADY — that's in processSMPTransmissions, not a separate API call. |
|
||||
| `getConnectionRatchetAdHash` | NO | Verification UI not in widget |
|
||||
|
||||
## Connection cleanup
|
||||
|
||||
| Function | MVP? | Reason |
|
||||
|----------|------|--------|
|
||||
| `deleteConnection` | YES | User may want to delete a conversation |
|
||||
| `deleteConnectionAsync` | NO | Async variant |
|
||||
| `deleteConnections` | NO | Batch delete — single delete sufficient |
|
||||
| `deleteConnectionsAsync` | NO | Same |
|
||||
| `getConnectionServers` | NO | Info only |
|
||||
| `compareConnections` | NO | Database sync tool |
|
||||
| `syncConnections` | NO | Database sync tool |
|
||||
|
||||
## Server configuration
|
||||
|
||||
| Function | MVP? | Reason |
|
||||
|----------|------|--------|
|
||||
| `setProtocolServers` | NO | Widget initialized with servers, doesn't change them |
|
||||
| `checkUserServers` | NO | Admin function |
|
||||
| `testProtocolServer` | NO | Admin function |
|
||||
| `setNtfServers` | NO | No notifications in MVP |
|
||||
| `setNetworkConfig` | NO | Widget uses default network config |
|
||||
| `setUserNetworkInfo` | NO | Widget doesn't track network state changes |
|
||||
| `reconnectAllServers` | NO | Widget handles reconnection via SMP client |
|
||||
| `reconnectSMPServer` | NO | Same |
|
||||
|
||||
## Notifications (all post-MVP)
|
||||
|
||||
| Function | MVP? | Reason |
|
||||
|----------|------|--------|
|
||||
| `registerNtfToken` | NO | No webpush yet |
|
||||
| `verifyNtfToken` | NO | Same |
|
||||
| `checkNtfToken` | NO | Same |
|
||||
| `deleteNtfToken` | NO | Same |
|
||||
| `getNtfToken` | NO | Same |
|
||||
| `getNtfTokenData` | NO | Same |
|
||||
| `toggleConnectionNtfs` | NO | Same |
|
||||
|
||||
## File transfer (all post-MVP)
|
||||
|
||||
| Function | MVP? | Reason |
|
||||
|----------|------|--------|
|
||||
| `xftpStartWorkers` | NO | Post-MVP |
|
||||
| `xftpStartSndWorkers` | NO | Same |
|
||||
| `xftpReceiveFile` | NO | Same |
|
||||
| `xftpDeleteRcvFile` | NO | Same |
|
||||
| `xftpDeleteRcvFiles` | NO | Same |
|
||||
| `xftpSendFile` | NO | Same |
|
||||
| `xftpSendDescription` | NO | Same |
|
||||
| `xftpDeleteSndFileInternal` | NO | Same |
|
||||
| `xftpDeleteSndFilesInternal` | NO | Same |
|
||||
| `xftpDeleteSndFileRemote` | NO | Same |
|
||||
| `xftpDeleteSndFilesRemote` | NO | Same |
|
||||
|
||||
## Remote control (all skip)
|
||||
|
||||
| Function | MVP? | Reason |
|
||||
|----------|------|--------|
|
||||
| `rcNewHostPairing` | NO | Not in scope |
|
||||
| `rcConnectHost` | NO | Same |
|
||||
| `rcConnectCtrl` | NO | Same |
|
||||
| `rcDiscoverCtrl` | NO | Same |
|
||||
|
||||
## Debug/stats (all skip)
|
||||
|
||||
| Function | MVP? | Reason |
|
||||
|----------|------|--------|
|
||||
| `getAgentSubsTotal` | NO | Debug |
|
||||
| `getAgentServersSummary` | NO | Debug |
|
||||
| `resetAgentServersStats` | NO | Debug |
|
||||
| `execAgentStoreSQL` | NO | Debug |
|
||||
| `getAgentMigrations` | NO | Debug |
|
||||
| `debugAgentLocks` | NO | Debug |
|
||||
| `getAgentSubscriptions` | NO | Debug |
|
||||
| `logConnection` | NO | Debug |
|
||||
| `withAgentEnv` | NO | Test utility |
|
||||
|
||||
## Summary
|
||||
|
||||
### Corrections after reviewing simplex-chat-2 Subscriber.hs + Commands.hs
|
||||
|
||||
The widget handles business chats (groups). Group flows trigger agent calls the widget doesn't initiate but must support:
|
||||
|
||||
- Member introductions create connections asynchronously → `createConnectionAsync`
|
||||
- Members join via introductions → `joinConnectionAsync`, `prepareConnectionToJoin`
|
||||
- Members leave/deleted → `deleteConnectionAsync`, `deleteConnectionsAsync`
|
||||
- Group messages go to all members → `sendMessages`
|
||||
- All message acks use async variant → `ackMessageAsync`
|
||||
- Accepting group invitations → `allowConnectionAsync`
|
||||
- `acceptContact` — used in contact request acceptance flow (APIAcceptContactRequest in Commands.hs)
|
||||
- `toggleConnectionNtfs` — used when CON received for group member (Subscriber.hs:850)
|
||||
|
||||
### MVP functions (27 of 90):
|
||||
|
||||
**Lifecycle:** `getSMPAgentClient`, `disconnectAgentClient`
|
||||
|
||||
**User:** `createUser`
|
||||
|
||||
**Join:** `prepareConnectionToJoin`, `joinConnection`, `joinConnectionAsync`, `connRequestPQSupport`
|
||||
|
||||
**Handshake:** `allowConnection`, `allowConnectionAsync`, `acceptContact`, `acceptContactAsync`, `prepareConnectionToAccept`
|
||||
|
||||
**Connection creation (for group member flows):** `createConnectionAsync`, `deleteConnectionAsync`, `deleteConnectionsAsync`
|
||||
|
||||
**Subscribe:** `subscribeConnection`, `subscribeConnections`, `subscribeAllConnections`, `resubscribeConnection`, `resubscribeConnections`
|
||||
|
||||
**Message:** `sendMessage`, `sendMessages`, `ackMessage`, `ackMessageAsync`
|
||||
|
||||
**Cleanup:** `deleteConnection`
|
||||
|
||||
**Notification toggle:** `toggleConnectionNtfs`
|
||||
|
||||
**Internal (not exported but required):** `subscriber`/`processSMPTransmissions` (message processing), `runSmpQueueMsgDelivery` (delivery worker), all encryption/decryption functions, store operations for the above.
|
||||
|
||||
### Message types to handle in processSMPTransmissions:
|
||||
|
||||
| AMessage | Handle? | Reason |
|
||||
|----------|---------|--------|
|
||||
| `HELLO` | YES | Complete handshake |
|
||||
| `A_MSG body` | YES | Deliver message to user |
|
||||
| `A_RCVD receipts` | YES | Process delivery receipts (show checkmarks) |
|
||||
| `A_QCONT addr` | NO | Queue continuation after quota — skip |
|
||||
| `QADD qs` | NO | Queue rotation — skip |
|
||||
| `QKEY qs` | NO | Queue rotation — skip |
|
||||
| `QUSE qs` | NO | Queue rotation — skip |
|
||||
| `QTEST qs` | NO | Queue rotation — skip |
|
||||
| `EREADY msgId` | ACCEPT | Must handle incoming (don't initiate) — reset ratchet sync state |
|
||||
|
||||
### AgentMsgEnvelope types to handle:
|
||||
|
||||
| Variant | Handle? | Reason |
|
||||
|---------|---------|--------|
|
||||
| `AgentConfirmation` | YES | Handshake — received when peer confirms |
|
||||
| `AgentMsgEnvelope` | YES | Normal encrypted messages |
|
||||
| `AgentInvitation` | YES | Received when joining contact address |
|
||||
| `AgentRatchetKey` | ACCEPT | Must handle incoming ratchet key — don't initiate |
|
||||
|
||||
### Store operations needed:
|
||||
|
||||
Based on the 18 MVP functions, the store needs (rough count):
|
||||
|
||||
- Connection CRUD: ~8 operations
|
||||
- Queue CRUD: ~6 operations
|
||||
- Ratchet state: ~4 operations (get, update, skipped keys)
|
||||
- Message storage: ~6 operations (create rcv/snd msg, update status, delete)
|
||||
- Message delivery: ~4 operations (create delivery, get pending, update status)
|
||||
- User: ~2 operations (create, get)
|
||||
- Confirmation: ~3 operations (create, get, delete)
|
||||
|
||||
**Estimated: ~33 store operations.** This is what determines whether SQLite WASM or IndexedDB direct is more practical.
|
||||
@@ -0,0 +1,121 @@
|
||||
# Agent Store: IndexedDB Design
|
||||
|
||||
**Parent**: [Agent Plan](./2026-05-22-agent.md)
|
||||
|
||||
## Schema
|
||||
|
||||
IndexedDB object stores, mapped from SQLite tables. Each store mirrors the Haskell schema from `agent_schema.sql`.
|
||||
|
||||
### Object stores needed (16)
|
||||
|
||||
```
|
||||
users
|
||||
key: userId (autoincrement)
|
||||
fields: deleted
|
||||
|
||||
connections
|
||||
key: connId (Uint8Array)
|
||||
fields: connMode, lastInternalMsgId, lastInternalRcvMsgId, lastInternalSndMsgId,
|
||||
lastExternalSndMsgId, lastRcvMsgHash, lastSndMsgHash, smpAgentVersion,
|
||||
duplexHandshake, enableNtfs, deleted, userId, ratchetSyncState, pqSupport
|
||||
|
||||
rcv_queues
|
||||
key: [host, port, rcvId] (compound)
|
||||
index: [connId], [host, port, sndId]
|
||||
fields: connId, rcvPrivateKey, rcvDhSecret, e2ePrivKey, e2eDhSecret, sndId, sndKey,
|
||||
status, smpClientVersion, rcvQueueId, rcvPrimary, replaceRcvQueueId, queueMode,
|
||||
serverKeyHash, lastBrokerTs
|
||||
|
||||
snd_queues
|
||||
key: [host, port, sndId] (compound)
|
||||
index: [connId]
|
||||
fields: connId, sndPrivateKey, e2eDhSecret, status, smpClientVersion,
|
||||
sndPublicKey, e2ePubKey, sndQueueId, sndPrimary, queueMode, serverKeyHash
|
||||
|
||||
messages
|
||||
key: [connId, internalId] (compound)
|
||||
index: [connId]
|
||||
fields: internalTs, internalRcvId, internalSndId, msgType, msgBody, msgFlags, pqEncryption
|
||||
|
||||
rcv_messages
|
||||
key: [connId, internalRcvId] (compound)
|
||||
index: [connId, internalId]
|
||||
fields: internalId, externalSndId, brokerId, brokerTs, internalHash,
|
||||
externalPrevSndHash, integrity, userAck, rcvQueueId, receiveAttempts
|
||||
|
||||
snd_messages
|
||||
key: [connId, internalSndId] (compound)
|
||||
index: [connId, internalId]
|
||||
fields: internalId, internalHash, previousMsgHash, retryIntSlow, retryIntFast,
|
||||
rcptInternalId, rcptStatus, msgEncryptKey, paddedMsgLen, sndMessageBodyId
|
||||
|
||||
snd_message_deliveries
|
||||
key: sndMessageDeliveryId (autoincrement)
|
||||
index: [connId, sndQueueId]
|
||||
fields: connId, sndQueueId, internalId, failed
|
||||
|
||||
snd_message_bodies
|
||||
key: sndMessageBodyId (autoincrement)
|
||||
fields: agentMsg
|
||||
|
||||
conn_confirmations
|
||||
key: confirmationId (Uint8Array)
|
||||
index: [connId]
|
||||
fields: connId, e2eSndPubKey, senderKey, ratchetState, senderConnInfo,
|
||||
accepted, ownConnInfo, smpReplyQueues, smpClientVersion
|
||||
|
||||
conn_invitations
|
||||
key: invitationId (Uint8Array)
|
||||
index: [contactConnId]
|
||||
fields: contactConnId, crInvitation, recipientConnInfo, accepted, ownConnInfo
|
||||
|
||||
ratchets
|
||||
key: connId (Uint8Array)
|
||||
fields: x3dhPrivKey1, x3dhPrivKey2, ratchetState, e2eVersion,
|
||||
x3dhPubKey1, x3dhPubKey2, pqPrivKem, pqPubKem
|
||||
|
||||
skipped_messages
|
||||
key: skippedMessageId (autoincrement)
|
||||
index: [connId]
|
||||
fields: connId, headerKey, msgN, msgKey
|
||||
|
||||
servers
|
||||
key: [host, port] (compound)
|
||||
fields: keyHash
|
||||
|
||||
commands
|
||||
key: commandId (autoincrement)
|
||||
index: [connId], [host, port]
|
||||
fields: connId, host, port, corrId, commandTag, command, agentVersion, serverKeyHash, failed
|
||||
|
||||
encrypted_rcv_message_hashes
|
||||
key: id (autoincrement)
|
||||
index: [connId, hash]
|
||||
fields: connId, hash, createdAt
|
||||
```
|
||||
|
||||
## Interface
|
||||
|
||||
TypeScript interface matching the ~60 store operations. Each method maps to a specific Haskell function in `AgentStore.hs`.
|
||||
|
||||
The interface will be defined in `src/agent/store.ts`. Implementation in `src/agent/store-idb.ts` (IndexedDB).
|
||||
|
||||
## Implementation approach
|
||||
|
||||
1. Define the TypeScript interface first — every method name matches the Haskell function name
|
||||
2. Implement with IndexedDB transactions
|
||||
3. Test each operation in isolation before wiring to agent
|
||||
|
||||
IndexedDB transactions map to SQLite transactions — both are ACID within a single store/table. Cross-store atomicity in IndexedDB requires putting multiple stores in one transaction, which is supported.
|
||||
|
||||
## Key differences from SQLite
|
||||
|
||||
1. **No SQL joins** — denormalize where needed, or do application-level joins
|
||||
2. **No AUTO INCREMENT guaranteed ordering** — use explicit counters
|
||||
3. **Blob keys** — IndexedDB supports ArrayBuffer keys natively
|
||||
4. **Compound keys** — IndexedDB supports array keys: `[host, port, rcvId]`
|
||||
5. **Indexes** — must be declared upfront in `onupgradeneeded`
|
||||
|
||||
## Testing
|
||||
|
||||
Each store operation tested by: write data, read it back, verify it matches. No server needed — pure store tests using `fake-indexeddb` in Node.js.
|
||||
@@ -29,6 +29,7 @@
|
||||
"devDependencies": {
|
||||
"@types/node": "^25.5.0",
|
||||
"@types/ws": "^8.18.1",
|
||||
"fake-indexeddb": "^6.2.5",
|
||||
"typescript": "^5.4.0",
|
||||
"ws": "^8.0.0"
|
||||
}
|
||||
|
||||
@@ -3,8 +3,15 @@
|
||||
|
||||
import {base64urlDecode} from "@simplex-chat/xftp-web/dist/protocol/description.js"
|
||||
import {
|
||||
Decoder, decodeBytes, decodeLarge, decodeWord16, decodeBool,
|
||||
Decoder, concatBytes,
|
||||
encodeBytes, decodeBytes,
|
||||
encodeLarge, decodeLarge,
|
||||
encodeWord16, decodeWord16,
|
||||
encodeBool, decodeBool,
|
||||
encodeMaybe, decodeMaybe,
|
||||
encodeNonEmpty, decodeNonEmpty,
|
||||
} from "@simplex-chat/xftp-web/dist/protocol/encoding.js"
|
||||
import {encodeProtocolServer} from "../protocol.js"
|
||||
|
||||
// -- Short link types (Agent/Protocol.hs:1462-1470)
|
||||
|
||||
@@ -164,6 +171,196 @@ export function decodeFixedLinkData(d: Decoder): FixedLinkData {
|
||||
return {agentVRange: {min, max}, rootKey, rest}
|
||||
}
|
||||
|
||||
// -- SMPQueueAddress (Agent/Protocol.hs:1350-1356)
|
||||
|
||||
export interface SMPQueueAddress {
|
||||
smpServer: {hosts: string[], port: string, keyHash: Uint8Array} // ProtocolServer
|
||||
senderId: Uint8Array // EntityId (ByteString)
|
||||
dhPublicKey: Uint8Array // PublicKeyX25519 (DER-encoded ByteString)
|
||||
queueMode: string | null // Maybe QueueMode: 'M' = Messaging, 'C' = Contact
|
||||
}
|
||||
|
||||
// -- SMPQueueInfo (Agent/Protocol.hs:1310-1327)
|
||||
// Version-dependent encoding
|
||||
|
||||
// SMP client version constants (Protocol.hs:281-294)
|
||||
const initialSMPClientVersion = 1
|
||||
const sndAuthKeySMPClientVersion = 3
|
||||
const shortLinksSMPClientVersion = 4
|
||||
|
||||
export interface SMPQueueInfo {
|
||||
clientVersion: number // VersionSMPC (Word16)
|
||||
queueAddress: SMPQueueAddress
|
||||
}
|
||||
|
||||
// smpEncode (Agent/Protocol.hs:1313-1321)
|
||||
export function encodeSMPQueueInfo(q: SMPQueueInfo): Uint8Array {
|
||||
const {clientVersion, queueAddress: {smpServer, senderId, dhPublicKey, queueMode}} = q
|
||||
const addrEnc = concatBytes(
|
||||
encodeWord16(clientVersion),
|
||||
encodeProtocolServer(smpServer.hosts, smpServer.port, smpServer.keyHash),
|
||||
encodeBytes(senderId),
|
||||
encodeBytes(dhPublicKey),
|
||||
)
|
||||
if (clientVersion >= shortLinksSMPClientVersion) {
|
||||
// encode queueMode directly (Maybe QueueMode as char or empty)
|
||||
const qmBytes = queueMode ? new Uint8Array([queueMode.charCodeAt(0)]) : new Uint8Array(0)
|
||||
return concatBytes(addrEnc, qmBytes)
|
||||
}
|
||||
if (clientVersion >= sndAuthKeySMPClientVersion && senderCanSecure(queueMode)) {
|
||||
return concatBytes(addrEnc, encodeBool(true))
|
||||
}
|
||||
if (clientVersion > initialSMPClientVersion) {
|
||||
return addrEnc
|
||||
}
|
||||
// v1 legacy — not supported by web widget
|
||||
throw new Error("encodeSMPQueueInfo: legacy v1 not supported")
|
||||
}
|
||||
|
||||
// smpP (Agent/Protocol.hs:1322-1327)
|
||||
export function decodeSMPQueueInfo(d: Decoder): SMPQueueInfo {
|
||||
const clientVersion = decodeWord16(d)
|
||||
// v1 legacy server encoding not supported
|
||||
if (clientVersion <= initialSMPClientVersion) throw new Error("decodeSMPQueueInfo: legacy v1 not supported")
|
||||
const smpServer = decodeProtocolServerTyped(d)
|
||||
const senderId = decodeBytes(d)
|
||||
const dhPublicKey = decodeBytes(d)
|
||||
const queueMode = decodeQueueMode(d)
|
||||
return {clientVersion, queueAddress: {smpServer, senderId, dhPublicKey, queueMode}}
|
||||
}
|
||||
|
||||
// -- SMPQueueUri (Agent/Protocol.hs:1347-1431)
|
||||
|
||||
export interface SMPQueueUri {
|
||||
clientVRange: {min: number, max: number} // VersionRangeSMPC
|
||||
queueAddress: SMPQueueAddress
|
||||
}
|
||||
|
||||
// smpEncode (Agent/Protocol.hs:1417-1427)
|
||||
export function encodeSMPQueueUri(q: SMPQueueUri): Uint8Array {
|
||||
const {clientVRange: {min: minV, max: maxV}, queueAddress: {smpServer, senderId, dhPublicKey, queueMode}} = q
|
||||
const addrEnc = concatBytes(
|
||||
encodeWord16(minV), encodeWord16(maxV),
|
||||
encodeProtocolServer(smpServer.hosts, smpServer.port, smpServer.keyHash),
|
||||
encodeBytes(senderId),
|
||||
encodeBytes(dhPublicKey),
|
||||
)
|
||||
if (minV >= shortLinksSMPClientVersion) {
|
||||
const qmBytes = queueMode ? new Uint8Array([queueMode.charCodeAt(0)]) : new Uint8Array(0)
|
||||
return concatBytes(addrEnc, qmBytes)
|
||||
}
|
||||
if (minV >= sndAuthKeySMPClientVersion || (maxV >= sndAuthKeySMPClientVersion && senderCanSecure(queueMode))) {
|
||||
return concatBytes(addrEnc, encodeBool(senderCanSecure(queueMode)))
|
||||
}
|
||||
return addrEnc
|
||||
}
|
||||
|
||||
// smpP (Agent/Protocol.hs:1428-1431)
|
||||
export function decodeSMPQueueUri(d: Decoder): SMPQueueUri {
|
||||
const min = decodeWord16(d)
|
||||
const max = decodeWord16(d)
|
||||
const smpServer = decodeProtocolServerTyped(d)
|
||||
const senderId = decodeBytes(d)
|
||||
const dhPublicKey = decodeBytes(d)
|
||||
const queueMode = decodeQueueMode(d)
|
||||
return {clientVRange: {min, max}, queueAddress: {smpServer, senderId, dhPublicKey, queueMode}}
|
||||
}
|
||||
|
||||
// -- ConnReqUriData (Agent/Protocol.hs:1728-1734, 1145-1158)
|
||||
|
||||
export interface ConnReqUriData {
|
||||
crAgentVRange: {min: number, max: number} // VersionRangeSMPA
|
||||
crSmpQueues: SMPQueueUri[] // NonEmpty SMPQueueUri
|
||||
crClientData: string | null // Maybe CRClientData (Text)
|
||||
}
|
||||
|
||||
// smpEncode (Agent/Protocol.hs:1145-1147)
|
||||
export function encodeConnReqUriData(d: ConnReqUriData): Uint8Array {
|
||||
const vr = concatBytes(encodeWord16(d.crAgentVRange.min), encodeWord16(d.crAgentVRange.max))
|
||||
const queues = encodeNonEmpty(encodeSMPQueueUri, d.crSmpQueues)
|
||||
const clientData = d.crClientData !== null
|
||||
? concatBytes(new Uint8Array([0x31]), encodeLarge(new TextEncoder().encode(d.crClientData)))
|
||||
: new Uint8Array([0x30])
|
||||
return concatBytes(vr, queues, clientData)
|
||||
}
|
||||
|
||||
// smpP (Agent/Protocol.hs:1148-1158)
|
||||
export function decodeConnReqUriData(d: Decoder): ConnReqUriData {
|
||||
const min = decodeWord16(d)
|
||||
const max = decodeWord16(d)
|
||||
const crSmpQueues = decodeNonEmpty(decodeSMPQueueUri, d)
|
||||
// Patch queueMode: if Nothing, set to QMContact (Agent/Protocol.hs:1156-1158)
|
||||
for (const q of crSmpQueues) {
|
||||
if (q.queueAddress.queueMode === null) q.queueAddress.queueMode = "C"
|
||||
}
|
||||
const clientData = decodeMaybe((dd) => {
|
||||
const large = decodeLarge(dd)
|
||||
return new TextDecoder().decode(large)
|
||||
}, d)
|
||||
return {crAgentVRange: {min, max}, crSmpQueues, crClientData: clientData}
|
||||
}
|
||||
|
||||
// -- ConnectionRequestUri (Agent/Protocol.hs:1130-1143, 1436-1441)
|
||||
|
||||
export type ConnectionRequestUri =
|
||||
| {mode: "invitation", crData: ConnReqUriData, e2eParams: Uint8Array} // raw smpEncoded E2ERatchetParams
|
||||
| {mode: "contact", crData: ConnReqUriData}
|
||||
|
||||
// smpEncode (Agent/Protocol.hs:1130-1133)
|
||||
export function encodeConnectionRequestUri(cr: ConnectionRequestUri): Uint8Array {
|
||||
switch (cr.mode) {
|
||||
case "invitation":
|
||||
return concatBytes(new Uint8Array([0x49]), encodeConnReqUriData(cr.crData), cr.e2eParams) // 'I' + crData + e2eParams
|
||||
case "contact":
|
||||
return concatBytes(new Uint8Array([0x43]), encodeConnReqUriData(cr.crData)) // 'C' + crData
|
||||
}
|
||||
}
|
||||
|
||||
// smpP (Agent/Protocol.hs:1140-1143)
|
||||
export function decodeConnectionRequestUri(d: Decoder): ConnectionRequestUri {
|
||||
const mode = d.anyByte()
|
||||
if (mode === 0x49) { // 'I' Invitation
|
||||
const crData = decodeConnReqUriData(d)
|
||||
const e2eParams = d.takeAll() // E2ERatchetParams consumes rest
|
||||
return {mode: "invitation", crData, e2eParams}
|
||||
}
|
||||
if (mode === 0x43) { // 'C' Contact
|
||||
const crData = decodeConnReqUriData(d)
|
||||
return {mode: "contact", crData}
|
||||
}
|
||||
throw new Error("decodeConnectionRequestUri: unknown mode 0x" + mode.toString(16))
|
||||
}
|
||||
|
||||
// -- Helpers
|
||||
|
||||
function senderCanSecure(queueMode: string | null): boolean {
|
||||
return queueMode === "M"
|
||||
}
|
||||
|
||||
// queueModeP (Agent/Protocol.hs:1433-1434)
|
||||
// Just <$> smpP <|> optional ((\case True -> QMMessaging; _ -> QMContact) <$> smpP)
|
||||
function decodeQueueMode(d: Decoder): string | null {
|
||||
if (d.remaining() === 0) return null
|
||||
const b = d.anyByte()
|
||||
if (b === 0x4D) return "M" // QMMessaging
|
||||
if (b === 0x43) return "C" // QMContact
|
||||
// Could be a Bool (sndSecure) for older versions — True='T'(0x54) → QMMessaging, False='F'(0x46) → QMContact
|
||||
if (b === 0x54) return "M" // True → QMMessaging
|
||||
if (b === 0x46) return null // False → no queueMode (not secured)
|
||||
return null
|
||||
}
|
||||
|
||||
// Decode ProtocolServer into typed format with string hosts
|
||||
function decodeProtocolServerTyped(d: Decoder): {hosts: string[], port: string, keyHash: Uint8Array} {
|
||||
const hostCount = d.anyByte()
|
||||
if (hostCount === 0) throw new Error("empty server host list")
|
||||
const hosts: string[] = []
|
||||
for (let i = 0; i < hostCount; i++) hosts.push(new TextDecoder().decode(decodeBytes(d)))
|
||||
const port = new TextDecoder().decode(decodeBytes(d))
|
||||
const keyHash = decodeBytes(d)
|
||||
return {hosts, port, keyHash}
|
||||
}
|
||||
|
||||
// -- Profile extraction
|
||||
|
||||
export function parseProfile(userData: Uint8Array): unknown {
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,279 @@
|
||||
// Agent store interface for IndexedDB.
|
||||
// Each method mirrors a Haskell function in AgentStore.hs.
|
||||
// Implementation in store-idb.ts.
|
||||
|
||||
// -- Types matching Haskell store types
|
||||
|
||||
export type ConnId = Uint8Array
|
||||
export type UserId = number
|
||||
export type EntityId = Uint8Array
|
||||
export type InternalId = number
|
||||
export type InternalRcvId = number
|
||||
export type InternalSndId = number
|
||||
|
||||
export type QueueStatus = "new" | "confirmed" | "secured" | "active" | "disabled" | "deleted"
|
||||
export type ConnectionMode = "INV" | "CON" // SCMInvitation | SCMContact
|
||||
export type RatchetSyncState = "ok" | "allowed" | "required" | "started" | "agreed"
|
||||
|
||||
export interface ConnData {
|
||||
connId: ConnId
|
||||
connMode: ConnectionMode
|
||||
userId: UserId
|
||||
smpAgentVersion: number
|
||||
enableNtfs: boolean
|
||||
duplexHandshake: boolean
|
||||
deleted: boolean
|
||||
ratchetSyncState: RatchetSyncState
|
||||
pqSupport: boolean
|
||||
// Message ID counters
|
||||
lastInternalMsgId: number
|
||||
lastInternalRcvMsgId: number
|
||||
lastInternalSndMsgId: number
|
||||
lastExternalSndMsgId: number
|
||||
lastRcvMsgHash: Uint8Array
|
||||
lastSndMsgHash: Uint8Array
|
||||
}
|
||||
|
||||
export interface RcvQueue {
|
||||
host: string
|
||||
port: string
|
||||
rcvId: Uint8Array
|
||||
connId: ConnId
|
||||
rcvPrivateKey: Uint8Array
|
||||
rcvDhSecret: Uint8Array
|
||||
e2ePrivKey: Uint8Array
|
||||
e2eDhSecret: Uint8Array | null
|
||||
sndId: Uint8Array
|
||||
sndKey: Uint8Array | null
|
||||
status: QueueStatus
|
||||
smpClientVersion: number | null
|
||||
dbQueueId: number
|
||||
primary: boolean
|
||||
replaceRcvQueueId: number | null
|
||||
queueMode: string | null
|
||||
serverKeyHash: Uint8Array | null
|
||||
lastBrokerTs: string | null
|
||||
}
|
||||
|
||||
export interface SndQueue {
|
||||
host: string
|
||||
port: string
|
||||
sndId: Uint8Array
|
||||
connId: ConnId
|
||||
sndPrivateKey: Uint8Array
|
||||
e2eDhSecret: Uint8Array
|
||||
status: QueueStatus
|
||||
smpClientVersion: number
|
||||
sndPublicKey: Uint8Array | null
|
||||
e2ePubKey: Uint8Array | null
|
||||
dbQueueId: number
|
||||
primary: boolean
|
||||
queueMode: string | null
|
||||
serverKeyHash: Uint8Array | null
|
||||
}
|
||||
|
||||
export interface MsgMeta {
|
||||
integrity: string // "OK" or error
|
||||
recipient: [number, string] // (internalId, internalTs)
|
||||
broker: [Uint8Array, string] // (brokerId/msgId, brokerTs)
|
||||
sndMsgId: number
|
||||
pqEncryption: boolean
|
||||
}
|
||||
|
||||
export interface RcvMsgData {
|
||||
msgMeta: MsgMeta
|
||||
msgType: string
|
||||
msgFlags: number
|
||||
msgBody: Uint8Array
|
||||
internalRcvId: number
|
||||
internalHash: Uint8Array
|
||||
externalPrevSndHash: Uint8Array
|
||||
encryptedMsgHash: Uint8Array
|
||||
}
|
||||
|
||||
export interface SndMsgData {
|
||||
internalId: number
|
||||
internalSndId: number
|
||||
internalTs: string
|
||||
msgType: string
|
||||
msgFlags: number
|
||||
msgBody: Uint8Array
|
||||
pqEncryption: boolean
|
||||
internalHash: Uint8Array
|
||||
prevMsgHash: Uint8Array
|
||||
msgEncryptKey: Uint8Array | null
|
||||
paddedMsgLen: number | null
|
||||
sndMessageBodyId: number | null
|
||||
}
|
||||
|
||||
export interface Confirmation {
|
||||
confirmationId: Uint8Array
|
||||
connId: ConnId
|
||||
e2eSndPubKey: Uint8Array
|
||||
senderKey: Uint8Array | null
|
||||
ratchetState: Uint8Array
|
||||
senderConnInfo: Uint8Array
|
||||
accepted: boolean
|
||||
ownConnInfo: Uint8Array | null
|
||||
smpReplyQueues: Uint8Array | null // serialized
|
||||
smpClientVersion: number | null
|
||||
}
|
||||
|
||||
export interface Invitation {
|
||||
invitationId: Uint8Array
|
||||
contactConnId: ConnId | null
|
||||
crInvitation: Uint8Array
|
||||
recipientConnInfo: Uint8Array
|
||||
accepted: boolean
|
||||
ownConnInfo: Uint8Array | null
|
||||
}
|
||||
|
||||
export interface RcvMsg {
|
||||
internalId: number
|
||||
msgMeta: MsgMeta
|
||||
msgType: string
|
||||
msgBody: Uint8Array
|
||||
userAck: boolean
|
||||
msgReceipt: Uint8Array | null
|
||||
}
|
||||
|
||||
export interface PendingQueueMsg {
|
||||
connId: ConnId
|
||||
sndQueueId: number
|
||||
internalId: number
|
||||
msgType: string
|
||||
msgFlags: number
|
||||
msgBody: Uint8Array
|
||||
internalHash: Uint8Array
|
||||
prevMsgHash: Uint8Array
|
||||
pqEncryption: boolean
|
||||
msgEncryptKey: Uint8Array | null
|
||||
paddedMsgLen: number | null
|
||||
sndMessageBodyId: number | null
|
||||
}
|
||||
|
||||
export interface AsyncCommand {
|
||||
commandId: number
|
||||
connId: ConnId
|
||||
host: string | null
|
||||
port: string | null
|
||||
corrId: Uint8Array
|
||||
commandTag: string
|
||||
command: Uint8Array
|
||||
agentVersion: number
|
||||
serverKeyHash: Uint8Array | null
|
||||
failed: boolean
|
||||
}
|
||||
|
||||
// -- Store interface
|
||||
// Each method name matches the Haskell function in AgentStore.hs.
|
||||
|
||||
export interface AgentStore {
|
||||
// -- Users (AgentStore.hs:201-230)
|
||||
createUserRecord(): Promise<UserId>
|
||||
getUserIds(): Promise<UserId[]>
|
||||
deleteUserRecord(userId: UserId): Promise<void>
|
||||
setUserDeleted(userId: UserId): Promise<void>
|
||||
|
||||
// -- Servers (AgentStore.hs:233-240)
|
||||
createServer(host: string, port: string, keyHash: Uint8Array): Promise<void>
|
||||
|
||||
// -- Connections (AgentStore.hs:242-500)
|
||||
createNewConn(connData: ConnData, connMode: ConnectionMode): Promise<ConnId>
|
||||
getConn(connId: ConnId): Promise<{connData: ConnData, rcvQueues: RcvQueue[], sndQueues: SndQueue[]} | null>
|
||||
getRcvConn(host: string, port: string, rcvId: Uint8Array): Promise<{connData: ConnData, rcvQueue: RcvQueue} | null>
|
||||
getConnSubs(connIds: ConnId[]): Promise<Map<string, ConnData>>
|
||||
getConnsData(connIds: ConnId[]): Promise<Map<string, ConnData>>
|
||||
lockConnForUpdate(connId: ConnId): Promise<void> // no-op in IndexedDB (single-threaded)
|
||||
setConnDeleted(connId: ConnId, waitDelivery: boolean): Promise<void>
|
||||
setConnUserId(oldUserId: UserId, connId: ConnId, newUserId: UserId): Promise<void>
|
||||
setConnAgentVersion(connId: ConnId, version: number): Promise<void>
|
||||
setConnPQSupport(connId: ConnId, pqSupport: boolean): Promise<void>
|
||||
setConnRatchetSync(connId: ConnId, state: RatchetSyncState): Promise<void>
|
||||
updateNewConnJoin(connId: ConnId, agentVersion: number, pqSupport: boolean, enableNtfs: boolean): Promise<void>
|
||||
updateNewConnRcv(connId: ConnId, rcvQueue: RcvQueue, subMode: string): Promise<RcvQueue>
|
||||
getDeletedConnIds(): Promise<ConnId[]>
|
||||
getDeletedWaitingDeliveryConnIds(): Promise<ConnId[]>
|
||||
getConnIds(): Promise<ConnId[]>
|
||||
|
||||
// -- Queues (AgentStore.hs:500-700)
|
||||
addConnRcvQueue(connId: ConnId, rcvQueue: RcvQueue, subMode: string): Promise<RcvQueue>
|
||||
addConnSndQueue(connId: ConnId, sndQueue: SndQueue): Promise<void>
|
||||
setRcvQueueStatus(rcvQueue: RcvQueue, status: QueueStatus): Promise<void>
|
||||
setSndQueueStatus(sndQueue: SndQueue, status: QueueStatus): Promise<void>
|
||||
setRcvQueueConfirmedE2E(rcvQueue: RcvQueue, dhSecret: Uint8Array, smpClientVersion: number): Promise<void>
|
||||
setRcvQueuePrimary(connId: ConnId, rcvQueue: RcvQueue): Promise<void>
|
||||
deleteConnRcvQueue(rcvQueue: RcvQueue): Promise<void>
|
||||
deleteConnRecord(connId: ConnId): Promise<void>
|
||||
upgradeRcvConnToDuplex(connId: ConnId, sndQueue: SndQueue): Promise<void>
|
||||
upgradeSndConnToDuplex(connId: ConnId, rcvQueue: RcvQueue, subMode: string): Promise<RcvQueue>
|
||||
getPrimaryRcvQueue(connId: ConnId): Promise<RcvQueue | null>
|
||||
getRcvQueue(connId: ConnId, host: string, port: string, rcvId: Uint8Array): Promise<RcvQueue | null>
|
||||
getDeletedRcvQueue(connId: ConnId, host: string, port: string, rcvId: Uint8Array): Promise<RcvQueue | null>
|
||||
setConnectionNtfs(connId: ConnId, enable: boolean): Promise<void>
|
||||
|
||||
// -- Subscriptions (AgentStore.hs:700-800)
|
||||
getSubscriptionServers(onlyNeeded: boolean): Promise<Array<{userId: UserId, host: string, port: string}>>
|
||||
getUserServerRcvQueueSubs(userId: UserId, host: string, port: string, onlyNeeded: boolean, batchSize: number, cursor: number | null): Promise<{queues: RcvQueue[], nextCursor: number | null}>
|
||||
unsetQueuesToSubscribe(): Promise<void>
|
||||
getConnectionsForDelivery(): Promise<ConnId[]>
|
||||
getAllSndQueuesForDelivery(): Promise<SndQueue[]>
|
||||
|
||||
// -- Confirmations (AgentStore.hs:800-870)
|
||||
createConfirmation(confirmation: Confirmation): Promise<Uint8Array>
|
||||
acceptConfirmation(confirmationId: Uint8Array, ownConnInfo: Uint8Array): Promise<Confirmation>
|
||||
getAcceptedConfirmation(connId: ConnId): Promise<Confirmation | null>
|
||||
removeConfirmations(connId: ConnId): Promise<void>
|
||||
|
||||
// -- Invitations (AgentStore.hs:870-920)
|
||||
createInvitation(invitation: Invitation): Promise<Uint8Array>
|
||||
getInvitation(invitationId: Uint8Array): Promise<Invitation | null>
|
||||
acceptInvitation(invitationId: Uint8Array, ownConnInfo: Uint8Array): Promise<void>
|
||||
unacceptInvitation(invitationId: Uint8Array): Promise<void>
|
||||
deleteInvitation(invitationId: Uint8Array): Promise<void>
|
||||
|
||||
// -- Messages (AgentStore.hs:873-1050)
|
||||
updateRcvIds(connId: ConnId): Promise<{internalId: number, internalRcvId: number, prevExternalSndId: number, prevRcvMsgHash: Uint8Array}>
|
||||
createRcvMsg(connId: ConnId, rcvQueue: RcvQueue, rcvMsgData: RcvMsgData): Promise<void>
|
||||
setLastBrokerTs(connId: ConnId, dbQueueId: number, brokerTs: string): Promise<void>
|
||||
updateRcvMsgHash(connId: ConnId, sndMsgId: number, internalRcvId: number, hash: Uint8Array): Promise<void>
|
||||
createSndMsgBody(agentMsg: Uint8Array): Promise<number>
|
||||
updateSndIds(connId: ConnId): Promise<{internalId: number, internalSndId: number, prevSndMsgHash: Uint8Array}>
|
||||
createSndMsg(connId: ConnId, sndMsgData: SndMsgData): Promise<void>
|
||||
updateSndMsgHash(connId: ConnId, internalSndId: number, hash: Uint8Array): Promise<void>
|
||||
createSndMsgDelivery(connId: ConnId, sndQueue: SndQueue, internalId: number): Promise<void>
|
||||
getPendingQueueMsg(connId: ConnId, sndQueue: SndQueue): Promise<PendingQueueMsg | null>
|
||||
updatePendingMsgRIState(connId: ConnId, msgId: number, retryIntSlow: number | null, retryIntFast: number | null): Promise<void>
|
||||
setMsgUserAck(connId: ConnId, internalId: number): Promise<{rcvQueue: RcvQueue, brokerId: Uint8Array}>
|
||||
getRcvMsg(connId: ConnId, internalId: number): Promise<RcvMsg | null>
|
||||
getLastMsg(connId: ConnId, brokerId: Uint8Array): Promise<RcvMsg | null>
|
||||
incMsgRcvAttempts(connId: ConnId, internalId: number): Promise<number>
|
||||
checkRcvMsgHashExists(connId: ConnId, hash: Uint8Array): Promise<boolean>
|
||||
getRcvMsgBrokerTs(connId: ConnId, brokerId: Uint8Array): Promise<string | null>
|
||||
deleteMsg(connId: ConnId, internalId: number): Promise<void>
|
||||
deleteDeliveredSndMsg(connId: ConnId, internalSndId: number): Promise<void>
|
||||
deleteSndMsgDelivery(connId: ConnId, sndQueue: SndQueue, msgId: number, keepForReceipt: boolean): Promise<void>
|
||||
getSndMsgViaRcpt(connId: ConnId, sndMsgId: number): Promise<{internalId: number, msgType: string, internalHash: Uint8Array, msgReceipt: Uint8Array | null} | null>
|
||||
updateSndMsgRcpt(connId: ConnId, sndMsgId: number, receipt: Uint8Array): Promise<void>
|
||||
|
||||
// -- Ratchet (AgentStore.hs:1300-1400)
|
||||
createRatchetX3dhKeys(connId: ConnId, privKey1: Uint8Array, privKey2: Uint8Array, pqKem: Uint8Array | null): Promise<void>
|
||||
getRatchetX3dhKeys(connId: ConnId): Promise<{privKey1: Uint8Array, privKey2: Uint8Array, pqKem: Uint8Array | null} | null>
|
||||
createRatchet(connId: ConnId, ratchetState: Uint8Array): Promise<void>
|
||||
getRatchet(connId: ConnId): Promise<Uint8Array | null>
|
||||
getRatchetForUpdate(connId: ConnId): Promise<Uint8Array | null> // same as getRatchet in IndexedDB (single-threaded)
|
||||
getSkippedMsgKeys(connId: ConnId): Promise<Map<string, Map<number, {mk: Uint8Array, iv: Uint8Array}>>>
|
||||
updateRatchet(connId: ConnId, ratchetState: Uint8Array, skippedMsgDiff: unknown): Promise<void>
|
||||
|
||||
// -- Commands (AgentStore.hs:1400-1480)
|
||||
createCommand(corrId: Uint8Array, connId: ConnId, host: string | null, port: string | null, command: AsyncCommand): Promise<number>
|
||||
getPendingCommandServers(connIds: ConnId[]): Promise<Array<{connId: ConnId, host: string, port: string}>>
|
||||
getAllPendingCommandConns(): Promise<Array<{connId: ConnId, host: string, port: string}>>
|
||||
getPendingServerCommand(host: string, port: string): Promise<AsyncCommand | null>
|
||||
updateCommandServer(commandId: number, host: string, port: string): Promise<void>
|
||||
deleteCommand(commandId: number): Promise<void>
|
||||
|
||||
// -- Encrypted message hash dedup (AgentStore.hs:1200-1220)
|
||||
checkRcvMsgHashExists_encrypted(connId: ConnId, hash: Uint8Array): Promise<boolean>
|
||||
addEncryptedRcvMsgHash(connId: ConnId, hash: Uint8Array): Promise<void>
|
||||
}
|
||||
Vendored
+15
@@ -0,0 +1,15 @@
|
||||
// Override IDBValidKey to accept Uint8Array (supported in modern browsers)
|
||||
interface IDBObjectStore {
|
||||
get(query: any): IDBRequest<any>
|
||||
getAll(query?: any, count?: number): IDBRequest<any[]>
|
||||
getAllKeys(query?: any, count?: number): IDBRequest<any[]>
|
||||
add(value: any, key?: any): IDBRequest<any>
|
||||
put(value: any, key?: any): IDBRequest<any>
|
||||
delete(query: any): IDBRequest<undefined>
|
||||
}
|
||||
|
||||
interface IDBIndex {
|
||||
get(query: any): IDBRequest<any>
|
||||
getAll(query?: any, count?: number): IDBRequest<any[]>
|
||||
getAllKeys(query?: any, count?: number): IDBRequest<any[]>
|
||||
}
|
||||
@@ -0,0 +1,818 @@
|
||||
// Agent store scenario tests using fake-indexeddb.
|
||||
// Each scenario exercises a full lifecycle: create → update → read → delete → verify gone.
|
||||
// Every store method is called from at least one test.
|
||||
|
||||
import "fake-indexeddb/auto"
|
||||
import {openAgentStore} from "../dist/agent/store-idb.js"
|
||||
import type {AgentStore} from "../dist/agent/store.js"
|
||||
|
||||
// The store returns raw IndexedDB rows with snake_case field names.
|
||||
// The TypeScript interface types use camelCase but the underlying data is snake_case.
|
||||
// We use `any` casts in assertions to access the actual field names.
|
||||
type Row = any
|
||||
|
||||
// -- Helpers
|
||||
|
||||
function bytes(hex: string): Uint8Array {
|
||||
const b = new Uint8Array(hex.length / 2)
|
||||
for (let i = 0; i < hex.length; i += 2) b[i / 2] = parseInt(hex.slice(i, i + 2), 16)
|
||||
return b
|
||||
}
|
||||
|
||||
function hex(b: Uint8Array): string {
|
||||
return Array.from(b, x => x.toString(16).padStart(2, "0")).join("")
|
||||
}
|
||||
|
||||
function randomBytes(n: number): Uint8Array {
|
||||
const b = new Uint8Array(n)
|
||||
for (let i = 0; i < n; i++) b[i] = Math.floor(Math.random() * 256)
|
||||
return b
|
||||
}
|
||||
|
||||
let passed = 0
|
||||
let failed = 0
|
||||
|
||||
function assert(cond: boolean, msg: string) {
|
||||
if (!cond) {
|
||||
console.error("FAIL:", msg)
|
||||
failed++
|
||||
} else {
|
||||
passed++
|
||||
}
|
||||
}
|
||||
|
||||
function assertEq(a: any, b: any, msg: string) {
|
||||
const av = a instanceof Uint8Array ? hex(a) : JSON.stringify(a)
|
||||
const bv = b instanceof Uint8Array ? hex(b) : JSON.stringify(b)
|
||||
assert(av === bv, `${msg}: expected ${bv}, got ${av}`)
|
||||
}
|
||||
|
||||
async function assertThrows(fn: () => Promise<any>, msg: string) {
|
||||
try {
|
||||
await fn()
|
||||
assert(false, `${msg}: expected throw`)
|
||||
} catch {
|
||||
passed++
|
||||
}
|
||||
}
|
||||
|
||||
// -- Scenarios
|
||||
|
||||
async function testUsers(store: AgentStore) {
|
||||
console.log(" users...")
|
||||
// createUserRecord, getUserIds
|
||||
const uid1 = await store.createUserRecord()
|
||||
const uid2 = await store.createUserRecord()
|
||||
let ids = await store.getUserIds()
|
||||
assert(ids.includes(uid1) && ids.includes(uid2), "getUserIds returns both users")
|
||||
|
||||
// setUserDeleted — marks user as deleted, getUserIds should exclude it
|
||||
await store.setUserDeleted(uid1)
|
||||
ids = await store.getUserIds()
|
||||
assert(!ids.includes(uid1), "deleted user not in getUserIds")
|
||||
assert(ids.includes(uid2), "non-deleted user still in getUserIds")
|
||||
|
||||
// deleteUserRecord — hard delete
|
||||
await store.deleteUserRecord(uid2)
|
||||
ids = await store.getUserIds()
|
||||
assert(!ids.includes(uid2), "hard-deleted user gone")
|
||||
}
|
||||
|
||||
async function testServers(store: AgentStore) {
|
||||
console.log(" servers...")
|
||||
// createServer — insert or ignore
|
||||
const kh = randomBytes(32)
|
||||
await store.createServer("smp1.example.com", "5223", kh)
|
||||
// Duplicate should not throw
|
||||
await store.createServer("smp1.example.com", "5223", kh)
|
||||
}
|
||||
|
||||
async function testConnectionsAndQueues(store: AgentStore) {
|
||||
console.log(" connections and queues...")
|
||||
const userId = await store.createUserRecord()
|
||||
const connId = randomBytes(24)
|
||||
const connId2 = randomBytes(24)
|
||||
|
||||
// createNewConn
|
||||
const created = await store.createNewConn({
|
||||
connId, connMode: "INV", userId, smpAgentVersion: 7,
|
||||
enableNtfs: true, duplexHandshake: true, deleted: false,
|
||||
ratchetSyncState: "ok", pqSupport: true,
|
||||
lastInternalMsgId: 0, lastInternalRcvMsgId: 0, lastInternalSndMsgId: 0,
|
||||
lastExternalSndMsgId: 0, lastRcvMsgHash: new Uint8Array(0), lastSndMsgHash: new Uint8Array(0),
|
||||
}, "INV")
|
||||
assertEq(created, connId, "createNewConn returns connId")
|
||||
|
||||
// getConn — should find it
|
||||
const got = await store.getConn(connId)
|
||||
assert(got !== null, "getConn finds connection")
|
||||
assertEq((got!.connData as Row).conn_id, connId, "getConn connId matches")
|
||||
|
||||
// getConnIds
|
||||
const allIds = await store.getConnIds()
|
||||
assert(allIds.some(id => hex(id) === hex(connId)), "getConnIds includes new conn")
|
||||
|
||||
// setConnAgentVersion
|
||||
await store.setConnAgentVersion(connId, 8)
|
||||
const got2 = await store.getConn(connId)
|
||||
assertEq((got2!.connData as Row).smp_agent_version, 8, "setConnAgentVersion updated")
|
||||
|
||||
// setConnPQSupport
|
||||
await store.setConnPQSupport(connId, false)
|
||||
const got3 = await store.getConn(connId)
|
||||
assertEq((got3!.connData as Row).pq_support, 0, "setConnPQSupport updated")
|
||||
|
||||
// setConnRatchetSync
|
||||
await store.setConnRatchetSync(connId, "required")
|
||||
const got4 = await store.getConn(connId)
|
||||
assertEq((got4!.connData as Row).ratchet_sync_state, "required", "setConnRatchetSync updated")
|
||||
|
||||
// setConnectionNtfs
|
||||
await store.setConnectionNtfs(connId, false)
|
||||
const got5 = await store.getConn(connId)
|
||||
assertEq((got5!.connData as Row).enable_ntfs, 0, "setConnectionNtfs updated")
|
||||
|
||||
// lockConnForUpdate — no-op, should not throw
|
||||
await store.lockConnForUpdate(connId)
|
||||
|
||||
// addConnRcvQueue
|
||||
const rcvQ = await store.addConnRcvQueue(connId, {
|
||||
host: "smp1.example.com", port: "5223", rcvId: randomBytes(24),
|
||||
connId, rcvPrivateKey: randomBytes(32), rcvDhSecret: randomBytes(32),
|
||||
e2ePrivKey: randomBytes(32), e2eDhSecret: null,
|
||||
sndId: randomBytes(24), sndKey: null, status: "new" as const,
|
||||
smpClientVersion: 7, dbQueueId: 0, primary: true,
|
||||
replaceRcvQueueId: null, queueMode: null,
|
||||
serverKeyHash: randomBytes(32), lastBrokerTs: null,
|
||||
}, "SMSubscribe")
|
||||
assert(rcvQ.dbQueueId >= 1, "addConnRcvQueue assigns dbQueueId")
|
||||
|
||||
// getPrimaryRcvQueue
|
||||
const primary = await store.getPrimaryRcvQueue(connId)
|
||||
assert(primary !== null, "getPrimaryRcvQueue finds queue")
|
||||
assertEq((primary as Row).rcv_primary, 1, "primary queue is marked primary")
|
||||
|
||||
// getRcvConn
|
||||
const rcvConn = await store.getRcvConn(rcvQ.host, rcvQ.port, rcvQ.rcvId)
|
||||
assert(rcvConn !== null, "getRcvConn finds by host/port/rcvId")
|
||||
|
||||
// getRcvQueue
|
||||
const rq = await store.getRcvQueue(connId, rcvQ.host, rcvQ.port, rcvQ.rcvId)
|
||||
assert(rq !== null, "getRcvQueue finds queue")
|
||||
|
||||
// setRcvQueueStatus
|
||||
await store.setRcvQueueStatus(rcvQ, "confirmed")
|
||||
const rq2 = await store.getRcvQueue(connId, rcvQ.host, rcvQ.port, rcvQ.rcvId)
|
||||
assertEq(rq2!.status, "confirmed", "setRcvQueueStatus updated")
|
||||
|
||||
// setRcvQueueConfirmedE2E
|
||||
const dhSecret = randomBytes(32)
|
||||
await store.setRcvQueueConfirmedE2E(rcvQ, dhSecret, 7)
|
||||
const rq3 = await store.getRcvQueue(connId, rcvQ.host, rcvQ.port, rcvQ.rcvId)
|
||||
assertEq((rq3 as Row).e2e_dh_secret, dhSecret, "setRcvQueueConfirmedE2E updated dh secret")
|
||||
assertEq((rq3 as Row).status, "confirmed", "setRcvQueueConfirmedE2E sets confirmed")
|
||||
|
||||
// addConnSndQueue + upgradeRcvConnToDuplex (same operation)
|
||||
const sndQ = {
|
||||
host: "smp2.example.com", port: "5223", sndId: randomBytes(24),
|
||||
connId, sndPrivateKey: randomBytes(32), e2eDhSecret: randomBytes(32),
|
||||
status: "confirmed" as const, smpClientVersion: 7,
|
||||
sndPublicKey: randomBytes(32), e2ePubKey: randomBytes(32),
|
||||
dbQueueId: 0, primary: true, queueMode: null, serverKeyHash: randomBytes(32),
|
||||
}
|
||||
await store.upgradeRcvConnToDuplex(connId, sndQ)
|
||||
const gotDuplex = await store.getConn(connId)
|
||||
assert(gotDuplex!.sndQueues.length >= 1, "upgradeRcvConnToDuplex added snd queue")
|
||||
|
||||
// setSndQueueStatus
|
||||
await store.setSndQueueStatus(sndQ, "active")
|
||||
|
||||
// getConnSubs, getConnsData
|
||||
const subs = await store.getConnSubs([connId])
|
||||
assert(subs.size === 1, "getConnSubs returns 1 entry")
|
||||
const connsData = await store.getConnsData([connId])
|
||||
assert(connsData.size === 1, "getConnsData returns 1 entry")
|
||||
|
||||
// Create second connection for setConnUserId
|
||||
await store.createNewConn({
|
||||
connId: connId2, connMode: "CON", userId, smpAgentVersion: 7,
|
||||
enableNtfs: true, duplexHandshake: true, deleted: false,
|
||||
ratchetSyncState: "ok", pqSupport: false,
|
||||
lastInternalMsgId: 0, lastInternalRcvMsgId: 0, lastInternalSndMsgId: 0,
|
||||
lastExternalSndMsgId: 0, lastRcvMsgHash: new Uint8Array(0), lastSndMsgHash: new Uint8Array(0),
|
||||
}, "CON")
|
||||
const userId2 = await store.createUserRecord()
|
||||
await store.setConnUserId(userId, connId2, userId2)
|
||||
const got6 = await store.getConn(connId2)
|
||||
assertEq((got6!.connData as Row).user_id, userId2, "setConnUserId updated")
|
||||
|
||||
// updateNewConnJoin
|
||||
await store.updateNewConnJoin(connId, 9, true, false)
|
||||
const got7 = await store.getConn(connId)
|
||||
assertEq((got7!.connData as Row).smp_agent_version, 9, "updateNewConnJoin updated version")
|
||||
|
||||
// updateNewConnRcv — adds a rcv queue (same as addConnRcvQueue)
|
||||
const rcvQ2 = await store.updateNewConnRcv(connId2, {
|
||||
host: "smp3.example.com", port: "5223", rcvId: randomBytes(24),
|
||||
connId: connId2, rcvPrivateKey: randomBytes(32), rcvDhSecret: randomBytes(32),
|
||||
e2ePrivKey: randomBytes(32), e2eDhSecret: null,
|
||||
sndId: randomBytes(24), sndKey: null, status: "new" as const,
|
||||
smpClientVersion: 7, dbQueueId: 0, primary: true,
|
||||
replaceRcvQueueId: null, queueMode: null,
|
||||
serverKeyHash: randomBytes(32), lastBrokerTs: null,
|
||||
}, "SMSubscribe")
|
||||
assert(rcvQ2.dbQueueId >= 1, "updateNewConnRcv assigns dbQueueId")
|
||||
|
||||
// upgradeSndConnToDuplex — add rcv queue to a snd-only connection
|
||||
const connId3 = randomBytes(24)
|
||||
await store.createNewConn({
|
||||
connId: connId3, connMode: "INV", userId, smpAgentVersion: 7,
|
||||
enableNtfs: true, duplexHandshake: true, deleted: false,
|
||||
ratchetSyncState: "ok", pqSupport: false,
|
||||
lastInternalMsgId: 0, lastInternalRcvMsgId: 0, lastInternalSndMsgId: 0,
|
||||
lastExternalSndMsgId: 0, lastRcvMsgHash: new Uint8Array(0), lastSndMsgHash: new Uint8Array(0),
|
||||
}, "INV")
|
||||
await store.addConnSndQueue(connId3, {
|
||||
host: "smp4.example.com", port: "5223", sndId: randomBytes(24),
|
||||
connId: connId3, sndPrivateKey: randomBytes(32), e2eDhSecret: randomBytes(32),
|
||||
status: "confirmed" as const, smpClientVersion: 7,
|
||||
sndPublicKey: null, e2ePubKey: null, dbQueueId: 0, primary: true,
|
||||
queueMode: null, serverKeyHash: randomBytes(32),
|
||||
})
|
||||
const rcvQ3 = await store.upgradeSndConnToDuplex(connId3, {
|
||||
host: "smp4.example.com", port: "5223", rcvId: randomBytes(24),
|
||||
connId: connId3, rcvPrivateKey: randomBytes(32), rcvDhSecret: randomBytes(32),
|
||||
e2ePrivKey: randomBytes(32), e2eDhSecret: null,
|
||||
sndId: randomBytes(24), sndKey: null, status: "new" as const,
|
||||
smpClientVersion: 7, dbQueueId: 0, primary: true,
|
||||
replaceRcvQueueId: null, queueMode: null,
|
||||
serverKeyHash: randomBytes(32), lastBrokerTs: null,
|
||||
}, "SMSubscribe")
|
||||
assert(rcvQ3.dbQueueId >= 1, "upgradeSndConnToDuplex added rcv queue")
|
||||
|
||||
// setRcvQueuePrimary — add second rcv queue, make it primary
|
||||
const rcvQ4 = await store.addConnRcvQueue(connId, {
|
||||
host: "smp5.example.com", port: "5223", rcvId: randomBytes(24),
|
||||
connId, rcvPrivateKey: randomBytes(32), rcvDhSecret: randomBytes(32),
|
||||
e2ePrivKey: randomBytes(32), e2eDhSecret: null,
|
||||
sndId: randomBytes(24), sndKey: null, status: "new" as const,
|
||||
smpClientVersion: 7, dbQueueId: 0, primary: false,
|
||||
replaceRcvQueueId: null, queueMode: null,
|
||||
serverKeyHash: randomBytes(32), lastBrokerTs: null,
|
||||
}, "SMSubscribe")
|
||||
await store.setRcvQueuePrimary(connId, rcvQ4)
|
||||
const newPrimary = await store.getPrimaryRcvQueue(connId)
|
||||
assertEq((newPrimary as Row).rcv_queue_id, rcvQ4.dbQueueId, "setRcvQueuePrimary changed primary")
|
||||
|
||||
// getDeletedRcvQueue — first delete a queue, then find it
|
||||
// deleteConnRcvQueue physically deletes, so getDeletedRcvQueue won't find it
|
||||
// We need to test the soft-delete path — but deleteConnRcvQueue does hard delete
|
||||
// Just verify it returns null for non-deleted
|
||||
const drq = await store.getDeletedRcvQueue(connId, rcvQ.host, rcvQ.port, rcvQ.rcvId)
|
||||
assert(drq === null, "getDeletedRcvQueue returns null for non-deleted queue")
|
||||
|
||||
// deleteConnRcvQueue
|
||||
await store.deleteConnRcvQueue(rcvQ4)
|
||||
const afterDel = await store.getRcvQueue(connId, rcvQ4.host, rcvQ4.port, rcvQ4.rcvId)
|
||||
assert(afterDel === null, "deleteConnRcvQueue removes queue")
|
||||
|
||||
// setConnDeleted (waitDelivery=true)
|
||||
await store.setConnDeleted(connId2, true)
|
||||
const waitDel = await store.getDeletedWaitingDeliveryConnIds()
|
||||
assert(waitDel.some(id => hex(id) === hex(connId2)), "setConnDeleted waitDelivery appears in getDeletedWaitingDeliveryConnIds")
|
||||
|
||||
// setConnDeleted (waitDelivery=false)
|
||||
await store.setConnDeleted(connId3, false)
|
||||
const delIds = await store.getDeletedConnIds()
|
||||
assert(delIds.some(id => hex(id) === hex(connId3)), "setConnDeleted appears in getDeletedConnIds")
|
||||
|
||||
// deleteConnRecord
|
||||
await store.deleteConnRecord(connId3)
|
||||
const gone = await store.getConn(connId3)
|
||||
assert(gone === null, "deleteConnRecord removes connection")
|
||||
}
|
||||
|
||||
async function testSubscriptions(store: AgentStore) {
|
||||
console.log(" subscriptions...")
|
||||
const userId = await store.createUserRecord()
|
||||
const connId = randomBytes(24)
|
||||
await store.createNewConn({
|
||||
connId, connMode: "INV", userId, smpAgentVersion: 7,
|
||||
enableNtfs: true, duplexHandshake: true, deleted: false,
|
||||
ratchetSyncState: "ok", pqSupport: false,
|
||||
lastInternalMsgId: 0, lastInternalRcvMsgId: 0, lastInternalSndMsgId: 0,
|
||||
lastExternalSndMsgId: 0, lastRcvMsgHash: new Uint8Array(0), lastSndMsgHash: new Uint8Array(0),
|
||||
}, "INV")
|
||||
await store.createServer("sub.example.com", "5223", randomBytes(32))
|
||||
const rcvQ = await store.addConnRcvQueue(connId, {
|
||||
host: "sub.example.com", port: "5223", rcvId: randomBytes(24),
|
||||
connId, rcvPrivateKey: randomBytes(32), rcvDhSecret: randomBytes(32),
|
||||
e2ePrivKey: randomBytes(32), e2eDhSecret: null,
|
||||
sndId: randomBytes(24), sndKey: null, status: "new" as const,
|
||||
smpClientVersion: 7, dbQueueId: 0, primary: true,
|
||||
replaceRcvQueueId: null, queueMode: null,
|
||||
serverKeyHash: randomBytes(32), lastBrokerTs: null,
|
||||
}, "SMOnlyCreate")
|
||||
|
||||
// getSubscriptionServers — onlyNeeded=true should find our to_subscribe=1 queue
|
||||
const srvs = await store.getSubscriptionServers(true)
|
||||
assert(srvs.some(s => s.host === "sub.example.com"), "getSubscriptionServers finds queue with to_subscribe")
|
||||
|
||||
// getSubscriptionServers — onlyNeeded=false
|
||||
const allSrvs = await store.getSubscriptionServers(false)
|
||||
assert(allSrvs.some(s => s.host === "sub.example.com"), "getSubscriptionServers(false) finds all")
|
||||
|
||||
// getUserServerRcvQueueSubs
|
||||
const {queues} = await store.getUserServerRcvQueueSubs(userId, "sub.example.com", "5223", true, 10, null)
|
||||
assert(queues.length >= 1, "getUserServerRcvQueueSubs finds queues")
|
||||
|
||||
// unsetQueuesToSubscribe
|
||||
await store.unsetQueuesToSubscribe()
|
||||
const srvsAfter = await store.getSubscriptionServers(true)
|
||||
assert(!srvsAfter.some(s => s.host === "sub.example.com"), "unsetQueuesToSubscribe clears to_subscribe")
|
||||
|
||||
// getConnectionsForDelivery, getAllSndQueuesForDelivery — need snd deliveries
|
||||
// Add snd queue and delivery
|
||||
const sndQ = {
|
||||
host: "sub.example.com", port: "5223", sndId: randomBytes(24),
|
||||
connId, sndPrivateKey: randomBytes(32), e2eDhSecret: randomBytes(32),
|
||||
status: "active" as const, smpClientVersion: 7,
|
||||
sndPublicKey: null, e2ePubKey: null, dbQueueId: 0, primary: true,
|
||||
queueMode: null, serverKeyHash: randomBytes(32),
|
||||
}
|
||||
await store.addConnSndQueue(connId, sndQ)
|
||||
// Need to create a message first
|
||||
const {internalId, internalSndId, prevSndMsgHash} = await store.updateSndIds(connId)
|
||||
await store.createSndMsg(connId, {
|
||||
internalId, internalSndId, internalTs: new Date().toISOString(),
|
||||
msgType: "HELLO", msgFlags: 0, msgBody: randomBytes(10),
|
||||
pqEncryption: false, internalHash: randomBytes(32),
|
||||
prevMsgHash: prevSndMsgHash, msgEncryptKey: null, paddedMsgLen: null, sndMessageBodyId: null,
|
||||
})
|
||||
// Get the snd queue with its actual dbQueueId
|
||||
const gotConn = await store.getConn(connId)
|
||||
const actualSndQ = gotConn!.sndQueues[0] as Row
|
||||
// Raw IDB row has snd_queue_id, interface expects dbQueueId
|
||||
await store.createSndMsgDelivery(connId, {dbQueueId: actualSndQ.snd_queue_id} as any, internalId)
|
||||
|
||||
const deliveryConns = await store.getConnectionsForDelivery()
|
||||
assert(deliveryConns.some(id => hex(id) === hex(connId)), "getConnectionsForDelivery finds conn with delivery")
|
||||
|
||||
const deliverySndQs = await store.getAllSndQueuesForDelivery()
|
||||
assert(deliverySndQs.length >= 1, "getAllSndQueuesForDelivery finds queues")
|
||||
}
|
||||
|
||||
async function testConfirmations(store: AgentStore) {
|
||||
console.log(" confirmations...")
|
||||
const userId = await store.createUserRecord()
|
||||
const connId = randomBytes(24)
|
||||
await store.createNewConn({
|
||||
connId, connMode: "INV", userId, smpAgentVersion: 7,
|
||||
enableNtfs: true, duplexHandshake: true, deleted: false,
|
||||
ratchetSyncState: "ok", pqSupport: false,
|
||||
lastInternalMsgId: 0, lastInternalRcvMsgId: 0, lastInternalSndMsgId: 0,
|
||||
lastExternalSndMsgId: 0, lastRcvMsgHash: new Uint8Array(0), lastSndMsgHash: new Uint8Array(0),
|
||||
}, "INV")
|
||||
|
||||
const confId = randomBytes(16)
|
||||
// createConfirmation
|
||||
const returned = await store.createConfirmation({
|
||||
confirmationId: confId, connId,
|
||||
e2eSndPubKey: randomBytes(32), senderKey: randomBytes(32),
|
||||
ratchetState: randomBytes(64), senderConnInfo: randomBytes(50),
|
||||
accepted: false, ownConnInfo: null,
|
||||
smpReplyQueues: randomBytes(100), smpClientVersion: 7,
|
||||
})
|
||||
assertEq(returned, confId, "createConfirmation returns confirmationId")
|
||||
|
||||
// getAcceptedConfirmation — not accepted yet
|
||||
const notAccepted = await store.getAcceptedConfirmation(connId)
|
||||
assert(notAccepted === null, "getAcceptedConfirmation returns null before acceptance")
|
||||
|
||||
// acceptConfirmation
|
||||
const ownInfo = randomBytes(40)
|
||||
const accepted = await store.acceptConfirmation(confId, ownInfo)
|
||||
assert(accepted !== null, "acceptConfirmation returns confirmation")
|
||||
assertEq(accepted.accepted, 1, "acceptConfirmation sets accepted=1")
|
||||
|
||||
// getAcceptedConfirmation — now accepted
|
||||
const gotAccepted = await store.getAcceptedConfirmation(connId)
|
||||
assert(gotAccepted !== null, "getAcceptedConfirmation finds accepted confirmation")
|
||||
assertEq((gotAccepted as Row).own_conn_info, ownInfo, "accepted confirmation has ownConnInfo")
|
||||
|
||||
// removeConfirmations
|
||||
await store.removeConfirmations(connId)
|
||||
const afterRemove = await store.getAcceptedConfirmation(connId)
|
||||
assert(afterRemove === null, "removeConfirmations deletes all confirmations for conn")
|
||||
}
|
||||
|
||||
async function testInvitations(store: AgentStore) {
|
||||
console.log(" invitations...")
|
||||
const userId = await store.createUserRecord()
|
||||
const connId = randomBytes(24)
|
||||
await store.createNewConn({
|
||||
connId, connMode: "CON", userId, smpAgentVersion: 7,
|
||||
enableNtfs: true, duplexHandshake: true, deleted: false,
|
||||
ratchetSyncState: "ok", pqSupport: false,
|
||||
lastInternalMsgId: 0, lastInternalRcvMsgId: 0, lastInternalSndMsgId: 0,
|
||||
lastExternalSndMsgId: 0, lastRcvMsgHash: new Uint8Array(0), lastSndMsgHash: new Uint8Array(0),
|
||||
}, "CON")
|
||||
|
||||
const invId = randomBytes(16)
|
||||
// createInvitation
|
||||
const returned = await store.createInvitation({
|
||||
invitationId: invId, contactConnId: connId,
|
||||
crInvitation: randomBytes(200), recipientConnInfo: randomBytes(50),
|
||||
accepted: false, ownConnInfo: null,
|
||||
})
|
||||
assertEq(returned, invId, "createInvitation returns invitationId")
|
||||
|
||||
// getInvitation — not accepted, should find
|
||||
const got = await store.getInvitation(invId)
|
||||
assert(got !== null, "getInvitation finds unaccepted invitation")
|
||||
|
||||
// acceptInvitation
|
||||
const ownInfo = randomBytes(40)
|
||||
await store.acceptInvitation(invId, ownInfo)
|
||||
// getInvitation — accepted, should NOT find (WHERE accepted = 0)
|
||||
const gotAfterAccept = await store.getInvitation(invId)
|
||||
assert(gotAfterAccept === null, "getInvitation returns null for accepted invitation")
|
||||
|
||||
// unacceptInvitation
|
||||
await store.unacceptInvitation(invId)
|
||||
const gotAfterUnaccept = await store.getInvitation(invId)
|
||||
assert(gotAfterUnaccept !== null, "unacceptInvitation resets accepted to 0")
|
||||
assert((gotAfterUnaccept as Row).own_conn_info === null, "unacceptInvitation clears ownConnInfo")
|
||||
|
||||
// deleteInvitation
|
||||
await store.deleteInvitation(invId)
|
||||
const gotAfterDelete = await store.getInvitation(invId)
|
||||
assert(gotAfterDelete === null, "deleteInvitation removes invitation")
|
||||
}
|
||||
|
||||
async function testReceiveMessages(store: AgentStore) {
|
||||
console.log(" receive messages...")
|
||||
const userId = await store.createUserRecord()
|
||||
const connId = randomBytes(24)
|
||||
await store.createNewConn({
|
||||
connId, connMode: "INV", userId, smpAgentVersion: 7,
|
||||
enableNtfs: true, duplexHandshake: true, deleted: false,
|
||||
ratchetSyncState: "ok", pqSupport: false,
|
||||
lastInternalMsgId: 0, lastInternalRcvMsgId: 0, lastInternalSndMsgId: 0,
|
||||
lastExternalSndMsgId: 0, lastRcvMsgHash: new Uint8Array(0), lastSndMsgHash: new Uint8Array(0),
|
||||
}, "INV")
|
||||
await store.createServer("rcv.example.com", "5223", randomBytes(32))
|
||||
const rcvQ = await store.addConnRcvQueue(connId, {
|
||||
host: "rcv.example.com", port: "5223", rcvId: randomBytes(24),
|
||||
connId, rcvPrivateKey: randomBytes(32), rcvDhSecret: randomBytes(32),
|
||||
e2ePrivKey: randomBytes(32), e2eDhSecret: null,
|
||||
sndId: randomBytes(24), sndKey: null, status: "active" as const,
|
||||
smpClientVersion: 7, dbQueueId: 0, primary: true,
|
||||
replaceRcvQueueId: null, queueMode: null,
|
||||
serverKeyHash: randomBytes(32), lastBrokerTs: null,
|
||||
}, "SMSubscribe")
|
||||
|
||||
// updateRcvIds
|
||||
const {internalId, internalRcvId, prevExternalSndId, prevRcvMsgHash} = await store.updateRcvIds(connId)
|
||||
assertEq(internalId, 1, "updateRcvIds first internalId=1")
|
||||
assertEq(internalRcvId, 1, "updateRcvIds first internalRcvId=1")
|
||||
assertEq(prevExternalSndId, 0, "updateRcvIds prevExternalSndId=0")
|
||||
|
||||
const brokerId = randomBytes(24)
|
||||
const brokerTs = new Date().toISOString()
|
||||
const internalHash = randomBytes(32)
|
||||
const encryptedMsgHash = randomBytes(32)
|
||||
const msgBody = randomBytes(100)
|
||||
|
||||
// createRcvMsg — exercises insertRcvMsgBase_, insertRcvMsgDetails_, updateRcvMsgHash, setLastBrokerTs
|
||||
await store.createRcvMsg(connId, rcvQ, {
|
||||
msgMeta: {
|
||||
integrity: "OK",
|
||||
recipient: [internalId, new Date().toISOString()],
|
||||
broker: [brokerId, brokerTs],
|
||||
sndMsgId: 1,
|
||||
pqEncryption: false,
|
||||
},
|
||||
msgType: "MSG",
|
||||
msgFlags: 0,
|
||||
msgBody,
|
||||
internalRcvId,
|
||||
internalHash,
|
||||
externalPrevSndHash: randomBytes(32),
|
||||
encryptedMsgHash,
|
||||
})
|
||||
|
||||
// getRcvMsg
|
||||
const rcvMsg = await store.getRcvMsg(connId, internalId)
|
||||
assert(rcvMsg !== null, "getRcvMsg finds message")
|
||||
assertEq(rcvMsg!.msgType, "MSG", "getRcvMsg msgType matches")
|
||||
|
||||
// getLastMsg — verify the msg is "last" (conn.last_internal_msg_id matches)
|
||||
const lastMsg = await store.getLastMsg(connId, brokerId)
|
||||
assert(lastMsg !== null, "getLastMsg finds message by brokerId")
|
||||
|
||||
// getRcvMsgBrokerTs
|
||||
const ts = await store.getRcvMsgBrokerTs(connId, brokerId)
|
||||
assert(ts !== null, "getRcvMsgBrokerTs finds broker ts")
|
||||
|
||||
// checkRcvMsgHashExists — encrypted hash was inserted by createRcvMsg
|
||||
const hashExists = await store.checkRcvMsgHashExists(connId, encryptedMsgHash)
|
||||
assert(hashExists, "checkRcvMsgHashExists finds hash inserted by createRcvMsg")
|
||||
|
||||
// incMsgRcvAttempts
|
||||
const attempts = await store.incMsgRcvAttempts(connId, internalId)
|
||||
assertEq(attempts, 1, "incMsgRcvAttempts returns 1 after first increment")
|
||||
const attempts2 = await store.incMsgRcvAttempts(connId, internalId)
|
||||
assertEq(attempts2, 2, "incMsgRcvAttempts returns 2 after second increment")
|
||||
|
||||
// setMsgUserAck
|
||||
const {rcvQueue: ackQ, brokerId: ackBrokerId} = await store.setMsgUserAck(connId, internalId)
|
||||
assert(ackQ !== null, "setMsgUserAck returns rcvQueue")
|
||||
assertEq(ackBrokerId, brokerId, "setMsgUserAck returns correct brokerId")
|
||||
|
||||
// Verify user_ack was set
|
||||
const rcvMsgAfterAck = await store.getRcvMsg(connId, internalId)
|
||||
assert(rcvMsgAfterAck!.userAck, "setMsgUserAck sets userAck=true")
|
||||
|
||||
// setLastBrokerTs — standalone call
|
||||
const newTs = new Date().toISOString()
|
||||
await store.setLastBrokerTs(connId, rcvQ.dbQueueId, newTs)
|
||||
|
||||
// updateRcvMsgHash — standalone call
|
||||
await store.updateRcvMsgHash(connId, 2, internalRcvId, randomBytes(32))
|
||||
|
||||
// deleteMsg
|
||||
await store.deleteMsg(connId, internalId)
|
||||
const deletedMsg = await store.getRcvMsg(connId, internalId)
|
||||
assert(deletedMsg === null, "deleteMsg removes message")
|
||||
}
|
||||
|
||||
async function testSendMessages(store: AgentStore) {
|
||||
console.log(" send messages...")
|
||||
const userId = await store.createUserRecord()
|
||||
const connId = randomBytes(24)
|
||||
await store.createNewConn({
|
||||
connId, connMode: "INV", userId, smpAgentVersion: 7,
|
||||
enableNtfs: true, duplexHandshake: true, deleted: false,
|
||||
ratchetSyncState: "ok", pqSupport: false,
|
||||
lastInternalMsgId: 0, lastInternalRcvMsgId: 0, lastInternalSndMsgId: 0,
|
||||
lastExternalSndMsgId: 0, lastRcvMsgHash: new Uint8Array(0), lastSndMsgHash: new Uint8Array(0),
|
||||
}, "INV")
|
||||
await store.createServer("snd.example.com", "5223", randomBytes(32))
|
||||
await store.addConnSndQueue(connId, {
|
||||
host: "snd.example.com", port: "5223", sndId: randomBytes(24),
|
||||
connId, sndPrivateKey: randomBytes(32), e2eDhSecret: randomBytes(32),
|
||||
status: "active" as const, smpClientVersion: 7,
|
||||
sndPublicKey: null, e2ePubKey: null, dbQueueId: 0, primary: true,
|
||||
queueMode: null, serverKeyHash: randomBytes(32),
|
||||
})
|
||||
const gotConn = await store.getConn(connId)
|
||||
const sndQueue = gotConn!.sndQueues[0]
|
||||
|
||||
// createSndMsgBody
|
||||
const agentMsg = randomBytes(200)
|
||||
const bodyId = await store.createSndMsgBody(agentMsg)
|
||||
assert(bodyId >= 1, "createSndMsgBody returns positive id")
|
||||
|
||||
// updateSndIds
|
||||
const {internalId, internalSndId, prevSndMsgHash} = await store.updateSndIds(connId)
|
||||
assertEq(internalId, 1, "updateSndIds first internalId=1")
|
||||
assertEq(internalSndId, 1, "updateSndIds first internalSndId=1")
|
||||
|
||||
const internalHash = randomBytes(32)
|
||||
|
||||
// createSndMsg
|
||||
await store.createSndMsg(connId, {
|
||||
internalId, internalSndId, internalTs: new Date().toISOString(),
|
||||
msgType: "SEND", msgFlags: 0, msgBody: randomBytes(100),
|
||||
pqEncryption: false, internalHash,
|
||||
prevMsgHash: prevSndMsgHash, msgEncryptKey: randomBytes(32),
|
||||
paddedMsgLen: 16384, sndMessageBodyId: bodyId,
|
||||
})
|
||||
|
||||
// updateSndMsgHash — standalone
|
||||
await store.updateSndMsgHash(connId, internalSndId, internalHash)
|
||||
|
||||
// createSndMsgDelivery
|
||||
await store.createSndMsgDelivery(connId, sndQueue, internalId)
|
||||
|
||||
// getPendingQueueMsg
|
||||
const pending = await store.getPendingQueueMsg(connId, sndQueue)
|
||||
assert(pending !== null, "getPendingQueueMsg finds pending message")
|
||||
assertEq(pending!.msgType, "SEND", "getPendingQueueMsg msgType matches")
|
||||
assertEq(pending!.internalId, internalId, "getPendingQueueMsg internalId matches")
|
||||
|
||||
// updatePendingMsgRIState
|
||||
await store.updatePendingMsgRIState(connId, internalId, 30, 5)
|
||||
|
||||
// getSndMsgViaRcpt
|
||||
const sndMsg = await store.getSndMsgViaRcpt(connId, internalSndId)
|
||||
assert(sndMsg !== null, "getSndMsgViaRcpt finds message")
|
||||
assertEq(sndMsg!.internalId, internalId, "getSndMsgViaRcpt internalId matches")
|
||||
assertEq(sndMsg!.internalHash, internalHash, "getSndMsgViaRcpt hash matches")
|
||||
|
||||
// updateSndMsgRcpt
|
||||
const receipt = randomBytes(8)
|
||||
await store.updateSndMsgRcpt(connId, internalSndId, receipt)
|
||||
|
||||
// deleteSndMsgDelivery — deletes delivery, then msg if no more deliveries
|
||||
await store.deleteSndMsgDelivery(connId, sndQueue, internalId, false)
|
||||
|
||||
// Create a second message for deleteDeliveredSndMsg
|
||||
const ids2 = await store.updateSndIds(connId)
|
||||
await store.createSndMsg(connId, {
|
||||
internalId: ids2.internalId, internalSndId: ids2.internalSndId,
|
||||
internalTs: new Date().toISOString(),
|
||||
msgType: "SEND", msgFlags: 0, msgBody: randomBytes(50),
|
||||
pqEncryption: false, internalHash: randomBytes(32),
|
||||
prevMsgHash: ids2.prevSndMsgHash, msgEncryptKey: null, paddedMsgLen: null, sndMessageBodyId: null,
|
||||
})
|
||||
|
||||
// deleteDeliveredSndMsg — no deliveries exist, so should delete msg
|
||||
await store.deleteDeliveredSndMsg(connId, ids2.internalId)
|
||||
}
|
||||
|
||||
async function testRatchet(store: AgentStore) {
|
||||
console.log(" ratchet...")
|
||||
const connId = randomBytes(24)
|
||||
const userId = await store.createUserRecord()
|
||||
await store.createNewConn({
|
||||
connId, connMode: "INV", userId, smpAgentVersion: 7,
|
||||
enableNtfs: true, duplexHandshake: true, deleted: false,
|
||||
ratchetSyncState: "ok", pqSupport: true,
|
||||
lastInternalMsgId: 0, lastInternalRcvMsgId: 0, lastInternalSndMsgId: 0,
|
||||
lastExternalSndMsgId: 0, lastRcvMsgHash: new Uint8Array(0), lastSndMsgHash: new Uint8Array(0),
|
||||
}, "INV")
|
||||
|
||||
const privKey1 = randomBytes(56)
|
||||
const privKey2 = randomBytes(56)
|
||||
const pqKem = randomBytes(100)
|
||||
|
||||
// createRatchetX3dhKeys
|
||||
await store.createRatchetX3dhKeys(connId, privKey1, privKey2, pqKem)
|
||||
|
||||
// getRatchetX3dhKeys
|
||||
const keys = await store.getRatchetX3dhKeys(connId)
|
||||
assert(keys !== null, "getRatchetX3dhKeys finds keys")
|
||||
assertEq(keys!.privKey1, privKey1, "x3dh privKey1 matches")
|
||||
assertEq(keys!.privKey2, privKey2, "x3dh privKey2 matches")
|
||||
assertEq(keys!.pqKem, pqKem, "x3dh pqKem matches")
|
||||
|
||||
// getRatchet — no ratchet state yet
|
||||
const noRatchet = await store.getRatchet(connId)
|
||||
assert(noRatchet === null, "getRatchet returns null before createRatchet")
|
||||
|
||||
// createRatchet — upserts, clearing x3dh keys
|
||||
const ratchetState = randomBytes(200)
|
||||
await store.createRatchet(connId, ratchetState)
|
||||
|
||||
// getRatchet
|
||||
const gotRatchet = await store.getRatchet(connId)
|
||||
assertEq(gotRatchet, ratchetState, "getRatchet returns stored state")
|
||||
|
||||
// getRatchetForUpdate — same as getRatchet in IndexedDB
|
||||
const gotForUpdate = await store.getRatchetForUpdate(connId)
|
||||
assertEq(gotForUpdate, ratchetState, "getRatchetForUpdate returns stored state")
|
||||
|
||||
// x3dh keys should be cleared after createRatchet
|
||||
const keysAfter = await store.getRatchetX3dhKeys(connId)
|
||||
assert(keysAfter === null, "createRatchet clears x3dh keys")
|
||||
|
||||
// getSkippedMsgKeys — empty initially
|
||||
const noSkipped = await store.getSkippedMsgKeys(connId)
|
||||
assertEq(noSkipped.size, 0, "getSkippedMsgKeys empty initially")
|
||||
|
||||
// updateRatchet with SMDAdd
|
||||
const headerKey = randomBytes(32)
|
||||
const msgKey = randomBytes(32)
|
||||
const newRatchetState = randomBytes(200)
|
||||
const addKeys = new Map<Uint8Array, Map<number, Uint8Array>>()
|
||||
const inner = new Map<number, Uint8Array>()
|
||||
inner.set(0, msgKey)
|
||||
inner.set(1, randomBytes(32))
|
||||
addKeys.set(headerKey, inner)
|
||||
await store.updateRatchet(connId, newRatchetState, {type: "add", keys: addKeys})
|
||||
|
||||
// Verify ratchet state updated
|
||||
const updatedRatchet = await store.getRatchet(connId)
|
||||
assertEq(updatedRatchet, newRatchetState, "updateRatchet updates state")
|
||||
|
||||
// Verify skipped keys added
|
||||
const skipped = await store.getSkippedMsgKeys(connId)
|
||||
assert(skipped.size >= 1, "updateRatchet SMDAdd adds skipped keys")
|
||||
|
||||
// updateRatchet with SMDRemove
|
||||
const hkHex = Array.from(headerKey, x => x.toString(16).padStart(2, "0")).join("")
|
||||
await store.updateRatchet(connId, randomBytes(200), {type: "remove", headerKey, msgN: 0})
|
||||
const afterRemove = await store.getSkippedMsgKeys(connId)
|
||||
// Should have 1 key remaining (msgN=1) instead of 2
|
||||
let totalKeys = 0
|
||||
for (const [, m] of afterRemove) totalKeys += m.size
|
||||
assertEq(totalKeys, 1, "updateRatchet SMDRemove removes specific key")
|
||||
|
||||
// updateRatchet with noChange
|
||||
const stateBeforeNoChange = randomBytes(200)
|
||||
await store.updateRatchet(connId, stateBeforeNoChange, {type: "noChange"})
|
||||
const afterNoChange = await store.getRatchet(connId)
|
||||
assertEq(afterNoChange, stateBeforeNoChange, "updateRatchet SMDNoChange only updates state")
|
||||
}
|
||||
|
||||
async function testCommands(store: AgentStore) {
|
||||
console.log(" commands...")
|
||||
const userId = await store.createUserRecord()
|
||||
const connId = randomBytes(24)
|
||||
await store.createNewConn({
|
||||
connId, connMode: "INV", userId, smpAgentVersion: 7,
|
||||
enableNtfs: true, duplexHandshake: true, deleted: false,
|
||||
ratchetSyncState: "ok", pqSupport: false,
|
||||
lastInternalMsgId: 0, lastInternalRcvMsgId: 0, lastInternalSndMsgId: 0,
|
||||
lastExternalSndMsgId: 0, lastRcvMsgHash: new Uint8Array(0), lastSndMsgHash: new Uint8Array(0),
|
||||
}, "INV")
|
||||
|
||||
const corrId = randomBytes(24)
|
||||
// createCommand
|
||||
const cmdId = await store.createCommand(corrId, connId, "cmd.example.com", "5223", {
|
||||
commandId: 0, connId, host: "cmd.example.com", port: "5223",
|
||||
corrId, commandTag: "NEW", command: randomBytes(50),
|
||||
agentVersion: 7, serverKeyHash: randomBytes(32), failed: false,
|
||||
})
|
||||
assert(cmdId >= 1, "createCommand returns positive id")
|
||||
|
||||
// getPendingCommandServers
|
||||
const servers = await store.getPendingCommandServers([connId])
|
||||
assert(servers.some(s => s.host === "cmd.example.com"), "getPendingCommandServers finds command server")
|
||||
|
||||
// getAllPendingCommandConns
|
||||
const allConns = await store.getAllPendingCommandConns()
|
||||
assert(allConns.some(c => hex(c.connId) === hex(connId)), "getAllPendingCommandConns finds conn")
|
||||
|
||||
// getPendingServerCommand
|
||||
const pendingCmd = await store.getPendingServerCommand("cmd.example.com", "5223")
|
||||
assert(pendingCmd !== null, "getPendingServerCommand finds command")
|
||||
|
||||
// updateCommandServer
|
||||
await store.updateCommandServer(cmdId, "cmd2.example.com", "5224")
|
||||
const updated = await store.getPendingServerCommand("cmd2.example.com", "5224")
|
||||
assert(updated !== null, "updateCommandServer changes server")
|
||||
const oldHost = await store.getPendingServerCommand("cmd.example.com", "5223")
|
||||
assert(oldHost === null, "updateCommandServer — old host returns nothing")
|
||||
|
||||
// deleteCommand
|
||||
await store.deleteCommand(cmdId)
|
||||
const deleted = await store.getPendingServerCommand("cmd2.example.com", "5224")
|
||||
assert(deleted === null, "deleteCommand removes command")
|
||||
}
|
||||
|
||||
async function testHashDedup(store: AgentStore) {
|
||||
console.log(" hash dedup...")
|
||||
const connId = randomBytes(24)
|
||||
const userId = await store.createUserRecord()
|
||||
await store.createNewConn({
|
||||
connId, connMode: "INV", userId, smpAgentVersion: 7,
|
||||
enableNtfs: true, duplexHandshake: true, deleted: false,
|
||||
ratchetSyncState: "ok", pqSupport: false,
|
||||
lastInternalMsgId: 0, lastInternalRcvMsgId: 0, lastInternalSndMsgId: 0,
|
||||
lastExternalSndMsgId: 0, lastRcvMsgHash: new Uint8Array(0), lastSndMsgHash: new Uint8Array(0),
|
||||
}, "INV")
|
||||
|
||||
const hash = randomBytes(32)
|
||||
|
||||
// checkRcvMsgHashExists_encrypted — not yet
|
||||
const before = await store.checkRcvMsgHashExists_encrypted(connId, hash)
|
||||
assert(!before, "checkRcvMsgHashExists_encrypted returns false before add")
|
||||
|
||||
// addEncryptedRcvMsgHash
|
||||
await store.addEncryptedRcvMsgHash(connId, hash)
|
||||
|
||||
// checkRcvMsgHashExists_encrypted — now exists
|
||||
const after = await store.checkRcvMsgHashExists_encrypted(connId, hash)
|
||||
assert(after, "checkRcvMsgHashExists_encrypted returns true after add")
|
||||
|
||||
// Different hash should not exist
|
||||
const other = await store.checkRcvMsgHashExists_encrypted(connId, randomBytes(32))
|
||||
assert(!other, "checkRcvMsgHashExists_encrypted returns false for different hash")
|
||||
}
|
||||
|
||||
// -- Run all
|
||||
|
||||
async function main() {
|
||||
console.log("Agent store tests")
|
||||
const store = await openAgentStore()
|
||||
|
||||
await testUsers(store)
|
||||
await testServers(store)
|
||||
await testConnectionsAndQueues(store)
|
||||
await testSubscriptions(store)
|
||||
await testConfirmations(store)
|
||||
await testInvitations(store)
|
||||
await testReceiveMessages(store)
|
||||
await testSendMessages(store)
|
||||
await testRatchet(store)
|
||||
await testCommands(store)
|
||||
await testHashDedup(store)
|
||||
|
||||
console.log(`\n${passed} passed, ${failed} failed`)
|
||||
if (failed > 0) process.exit(1)
|
||||
}
|
||||
|
||||
main().catch(e => { console.error("FATAL:", e?.message || e, e?.stack); process.exit(1) })
|
||||
@@ -3,7 +3,7 @@
|
||||
"target": "ES2022",
|
||||
"module": "ES2022",
|
||||
"moduleResolution": "node",
|
||||
"lib": ["ES2022"],
|
||||
"lib": ["ES2022", "DOM"],
|
||||
"outDir": "dist",
|
||||
"rootDir": "src",
|
||||
"declaration": true,
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
"target": "ES2022",
|
||||
"module": "ES2022",
|
||||
"moduleResolution": "node",
|
||||
"lib": ["ES2022"],
|
||||
"lib": ["ES2022", "DOM"],
|
||||
"outDir": "dist-test",
|
||||
"rootDir": "tests",
|
||||
"strict": true,
|
||||
|
||||
@@ -46,6 +46,8 @@ import Simplex.Messaging.Encoding
|
||||
import Simplex.Messaging.Encoding.String (Str (..), strEncode)
|
||||
import Simplex.Messaging.Protocol (EntityId (..), SMPServer, SubscriptionMode (..), MsgFlags (..), noMsgFlags, pattern SMPServer, pattern NoEntity, encodeProtocol, Cmd (..), SParty (..), Command (..), NewQueueReq (..), QueueReqData (..), BrokerMsg (..), RcvMessage (..), EncRcvMsgBody (..), QueueIdsKeys (..), PubHeader (..), PrivHeader (..), ClientMessage (..), ClientMsgEnvelope (..), pattern VersionSMPC)
|
||||
import Simplex.Messaging.Server.Env.STM (AStoreType (..), ServerConfig (..))
|
||||
import Simplex.Messaging.Server.QueueStore.QueueInfo (QueueMode (..))
|
||||
import Simplex.Messaging.ServiceScheme (ServiceScheme (..))
|
||||
import Simplex.Messaging.Server.MsgStore.Types (SMSType (..), SQSType (..))
|
||||
import Simplex.Messaging.Server.Web (attachStaticAndWS)
|
||||
import Data.Time.Clock (getCurrentTime)
|
||||
@@ -1236,6 +1238,52 @@ smpWebTests_ = do
|
||||
<> jsOut ("e.encAgentMessage")
|
||||
tsResult `shouldBe` "decrypt me"
|
||||
|
||||
describe "agent/queueInfo" $ do
|
||||
let impAgentProtoEnc = "import { encodeSMPQueueInfo, decodeSMPQueueInfo, encodeSMPQueueUri, decodeSMPQueueUri, encodeConnReqUriData, decodeConnReqUriData, encodeConnectionRequestUri, decodeConnectionRequestUri } from './dist/agent/protocol.js';"
|
||||
|
||||
describe "SMPQueueInfo" $ do
|
||||
it "encoding matches Haskell" $ do
|
||||
g <- C.newRandom
|
||||
(dhPub, _) <- atomically $ C.generateKeyPair @'C.X25519 g
|
||||
let srv = SMPServer ("smp.example.com" :| []) "5223" (C.KeyHash $ B.pack [1..32])
|
||||
senderId = EntityId $ B.pack [10..33]
|
||||
qAddr = AP.SMPQueueAddress {AP.smpServer = srv, AP.senderId = senderId, AP.dhPublicKey = dhPub, AP.queueMode = Just QMMessaging}
|
||||
qi = AP.SMPQueueInfo (VersionSMPC 4) qAddr
|
||||
hsBytes = smpEncode qi
|
||||
tsBytes <- callNode $ impEnc <> impAgentProtoEnc
|
||||
<> "const qi = {clientVersion: 4, queueAddress: {smpServer: {hosts: ['smp.example.com'], port: '5223', keyHash: " <> jsUint8 (B.pack [1..32]) <> "}, senderId: " <> jsUint8 (B.pack [10..33]) <> ", dhPublicKey: " <> jsUint8 (C.encodePubKey dhPub) <> ", queueMode: 'M'}};"
|
||||
<> jsOut ("encodeSMPQueueInfo(qi)")
|
||||
tsBytes `shouldBe` hsBytes
|
||||
|
||||
it "TypeScript decodes Haskell-encoded" $ do
|
||||
g <- C.newRandom
|
||||
(dhPub, _) <- atomically $ C.generateKeyPair @'C.X25519 g
|
||||
let srv = SMPServer ("relay.test.com" :| ["relay2.test.com"]) "" (C.KeyHash $ B.pack [50..81])
|
||||
senderId = EntityId $ B.pack [1..24]
|
||||
qAddr = AP.SMPQueueAddress {AP.smpServer = srv, AP.senderId = senderId, AP.dhPublicKey = dhPub, AP.queueMode = Just QMContact}
|
||||
qi = AP.SMPQueueInfo (VersionSMPC 4) qAddr
|
||||
hsBytes = smpEncode qi
|
||||
tsResult <- callNode $ impEnc <> impAgentProtoEnc
|
||||
<> "const qi = decodeSMPQueueInfo(new Decoder(" <> jsUint8 hsBytes <> "));"
|
||||
<> jsOut ("new Uint8Array([qi.clientVersion >> 8, qi.clientVersion & 0xff, qi.queueAddress.smpServer.hosts.length, qi.queueAddress.queueMode ? qi.queueAddress.queueMode.charCodeAt(0) : 0])")
|
||||
tsResult `shouldBe` B.pack [0, 4, 2, 0x43] -- version=4, 2 hosts, queueMode='C'
|
||||
|
||||
describe "ConnectionRequestUri" $ do
|
||||
it "contact encoding matches Haskell" $ do
|
||||
g <- C.newRandom
|
||||
(dhPub, _) <- atomically $ C.generateKeyPair @'C.X25519 g
|
||||
let srv = SMPServer ("smp1.example.com" :| []) "" (C.KeyHash $ B.pack [1..32])
|
||||
senderId = EntityId $ B.pack [1..24]
|
||||
qAddr = AP.SMPQueueAddress {AP.smpServer = srv, AP.senderId = senderId, AP.dhPublicKey = dhPub, AP.queueMode = Just QMContact}
|
||||
qUri = AP.SMPQueueUri (mkVersionRange (VersionSMPC 4) (VersionSMPC 4)) qAddr
|
||||
crData = AP.ConnReqUriData {AP.crScheme = SSSimplex, AP.crAgentVRange = mkVersionRange (AP.VersionSMPA 2) (AP.VersionSMPA 7), AP.crSmpQueues = qUri :| [], AP.crClientData = Nothing}
|
||||
cr = AP.CRContactUri crData :: AP.ConnectionRequestUri 'AP.CMContact
|
||||
hsBytes = smpEncode cr
|
||||
tsBytes <- callNode $ impEnc <> impAgentProtoEnc
|
||||
<> "const cr = {mode: 'contact', crData: {crAgentVRange: {min: 2, max: 7}, crSmpQueues: [{clientVRange: {min: 4, max: 4}, queueAddress: {smpServer: {hosts: ['smp1.example.com'], port: '', keyHash: " <> jsUint8 (B.pack [1..32]) <> "}, senderId: " <> jsUint8 (B.pack [1..24]) <> ", dhPublicKey: " <> jsUint8 (C.encodePubKey dhPub) <> ", queueMode: 'C'}}], crClientData: null}};"
|
||||
<> jsOut ("encodeConnectionRequestUri(cr)")
|
||||
tsBytes `shouldBe` hsBytes
|
||||
|
||||
describe "protocol/e2e" $ do
|
||||
describe "PubHeader" $ do
|
||||
it "encoding without key matches Haskell" $ do
|
||||
|
||||
Reference in New Issue
Block a user