Compare commits

...
240 Commits
Author SHA1 Message Date
shum 32124da512 further analysis 2026-03-26 13:55:16 +00:00
shum d8c8d74d9c add new analysis 2026-03-26 10:58:24 +00:00
shum 958331837d smp-server: analyze slow queries 2026-03-25 14:14:29 +00:00
Evgeny @ SimpleX Chat 2012236f65 agent: correct log message 2026-03-23 14:02:16 +00:00
Evgeny @ SimpleX Chat a54518afe1 test: rcv service re-association on restart (#1746) 2026-03-23 13:12:16 +00:00
sh e762e84f46 prometheus: fix metrics names (#1747) 2026-03-23 13:11:29 +00:00
Evgeny Poberezkin 5f08457b7e Merge branch 'master' into rcv-services 2026-03-20 15:43:17 +00:00
sh efcef2d1fd xftp-web: add postgres schema cleanup for integration tests (#1741)
Stale postgres schema leaked pending XFTP operations between
cross-language tests, causing N-1 of N tests to fail.
2026-03-20 09:58:56 +00:00
Evgeny Poberezkin 1a255f2e2f Merge branch 'master' into rcv-services 2026-03-20 09:00:01 +00:00
Evgeny 963d7b2f75 fix small bugs (#1740)
* fix small bugs

* toChunks
2026-03-20 08:59:38 +00:00
EvgenyandEvgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com> 8f4274763b smp: service fixes (#1737)
* smp: deliver service subscription to correct client

* tests: more resilient to concurrency

* optimize PostgreSQL query

* fix service re-association after server "downgrade"

* correctly handle service removed from server (and ID changed)

* remove unused

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-03-20 07:54:26 +00:00
Evgeny Poberezkin 4e4e0a4f42 6.5.0.11 2026-03-17 09:26:02 +00:00
sh 082a6c6f22 web: serve on-the-fly compressed gzip static files (#1735)
* web: serve pre-compressed gzip static files

* web: compress static files on the fly instead of pre-compressed
2026-03-16 09:08:43 +00:00
shandEvgeny dc2921e4ce xftp-server: embed file download widget in XFTP server web page (#1733)
* xftp-server: embed file download widget in XFTP server web page

When a URL has a hash fragment (>50 chars), the server page shows the
file download UI instead of the server info page. Embeds xftp-web
assets (JS, CSS, crypto worker) and protocol overlay with matching
website content. Overlay renders below the server navbar.

* xftp-server: fix overlay scroll lock, remove extra margin, fix dark SVG

* xftp-server: move file transfer widget to standalone /file page

* web: collapse all repeated Nothing sections in render

section_ only collapsed the first occurrence of a section when content
was Nothing, leaving subsequent sections with the same label intact.
This caused SMP server pages to show raw <x-xftpConfig> tags.

* xftp-server: update bundled css/js

* xftp-server: move file.html to xftp-server, rename xftp bundle dir

* web: remove unused server-info wrapper div

* refactor

* fix

---------

Co-authored-by: Evgeny <evgeny@poberezkin.com>
2026-03-13 16:00:02 +00:00
sh 5a32e729e0 xftp-web: add "Upload your file" link after download completes (#1736)
* xftp-web: add "Upload your file" link after download completes

* xftp-web: call initUpload directly instead of hashchange dispatch
2026-03-13 15:01:15 +00:00
Evgeny Poberezkin 48eba591bc Merge branch 'master' into rcv-services 2026-03-12 17:05:51 +00:00
shandEvgeny Poberezkin 782cacfb3c fix: using simplexmq as dependency (move embedFile to executables) (#1734)
* web: parameterize generateSite, remove Embedded from library

Move embedFile/embedDir out of the library so it works when
simplexmq is consumed as a dependency. generateSite now accepts
mediaContent, wellKnown, and linkHtml as parameters.

* smp-server, xftp-server: embed static files in executables

Add shared apps/common/Embedded.hs with TH splices, update SMPWeb
and XFTPWeb to pass embedded content to generateSite, move
file-embed dependency from library to executables and test suite.

* refactor

* add export, move common files to Web subfolder

* fix .cabal

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2026-03-12 17:05:00 +00:00
sh 328d3b941a xftp-web: use XFTP server domain in share link, verify on download (#1732)
* xftp-web: use XFTP server domain in share link, verify on download

Use a random XFTP server host from the file description as the link
origin instead of the current page domain. On download, verify
that the hosting domain matches one of the servers in the description.

* xftp-web: use first description server as link origin

* xftp-web: show wrong server error in download UI instead of blank page
2026-03-11 15:59:26 +00:00
sh b6b87a6323 xftp-web: disable minification in vite build (#1731) 2026-03-11 09:08:23 +00:00
Evgeny @ SimpleX Chat b81670c414 docs: move implemented rfcs 2026-03-10 19:18:34 +00:00
Evgeny @ SimpleX Chat 98351cf097 docs: add missing encodings and other protocol corrections 2026-03-10 16:36:24 +00:00
Evgeny @ SimpleX Chat 01785d5de8 docs: add e2e encrypted message wire encoding to PQDR spec 2026-03-10 09:04:21 +00:00
Evgeny @ SimpleX Chat f745ce5ab6 docs: fix minor issues in protocols 2026-03-10 08:15:53 +00:00
EvgenyandEvgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com> 583f4e059d update RFCs (#1730)
* update RFCs

* update

* update overview

* update terminology

* original language in threat model

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-03-09 23:35:41 +00:00
Evgeny @ SimpleX Chat 3c5752383d update protocol docs 2026-03-09 16:29:05 +00:00
sh 33454444a4 xftp-web: new version (#1728)
* xftp-web: remove flux from preset

* xftp-web: new version
2026-03-09 14:26:40 +00:00
Evgeny Poberezkin 3a25561c78 Merge branch 'master' into rcv-services 2026-03-09 12:25:52 +00:00
Evgeny Poberezkin 1449e6418c 6.5.0.10 2026-03-09 12:25:08 +00:00
sh 3a627a9599 tests: use correct web CA certificate in CLI static files test (#1727) 2026-03-09 12:23:03 +00:00
EvgenyandEvgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com> d2d834ad16 agent: validate destination relay certificate, allow 3-4 certificate chains (#1717)
Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-03-09 12:22:42 +00:00
Evgeny @ SimpleX Chat c624a10e70 Merge branch 'master' into rcv-services 2026-03-09 10:23:01 +00:00
shandEvgeny Poberezkin 437cdde4a5 xftp: add web page for server information (#1724)
* xftp: add web page for server information

* web: rename XFTP.Web to XFTPWeb, remove XFTP subdirectory

* refactor(xftp): remove storage quota from web page

* refactor

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2026-03-09 08:44:28 +00:00
shandEvgeny Poberezkin eed1bf14c6 web: extract shared web module from smp-server (#1723)
* web: extract shared web module from smp-server

Move web serving infrastructure (warp, static files, HTML templating)
from apps/smp-server/web/Static.hs into library modules:
- Simplex.Messaging.Server.Web (generic web infra + templating)
- Simplex.Messaging.Server.Web.Embedded (TH-embedded assets)

Move static assets from apps/smp-server/static/ to
src/Simplex/Messaging/Server/Web/.

Move EmbeddedWebParams/WebHttpsParams from Server.Main to Server.Web.

Keep SMP-specific rendering (serverInformation) in apps/smp-server/SMP/Web.hs.

generateSite is now generic: takes pre-rendered HTML + link page paths,
enabling reuse by XFTP and NTF servers.

* web: add tests for templating engine

Tests for render, section_, item_, and timedTTLText functions
in Simplex.Messaging.Server.Web module.

* web: add serverInfoSubsts, serveStaticPageH2, safe port parsing

* web: rename SMP.Web to SMPWeb, remove SMP subdirectory

* fix(web): section_ collapsing sections with Just "" content

Commit e48bedea ("servers: fix server pages when source code is not
specified") changed section_ to treat Just "" the same as Nothing -
collapsing the section. The intent was to handle the sourceCode case
(empty string when not specified), but the guard
`not (B.null content)` also broke operator, admin, complaints, and
hosting - all of which legitimately use Just "" as a
section-present marker.

Before (correct):
  Nothing -> before <> next
  Just content -> before <> item_ label content inside <> ...

After (broken):
  Just content | not (B.null content) -> ...
  _ -> before <> next

Restore the original behavior: only Nothing collapses a section.

* refactor

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2026-03-09 08:42:38 +00:00
Evgeny @ SimpleX Chat 313e96513c docs: correction to governance process (#1725) 2026-03-08 17:27:56 +00:00
Evgeny @ SimpleX Chat 8518f6087b docs: agent threat model 2026-03-07 15:47:46 +00:00
Evgeny Poberezkin 1d3057985c Merge branch 'master' into rcv-services 2026-03-04 07:33:20 +00:00
Evgeny c6e3a4d80f add missing exports (#1722)
* add missing exports

* fix dependency
2026-03-04 07:31:46 +00:00
Evgeny Poberezkin aebc01ba58 Merge branch 'master' into rcv-services 2026-03-03 21:16:46 +00:00
Evgeny Poberezkin 0e67647e90 6.5.0.9 2026-03-02 18:14:37 +00:00
EvgenyandEvgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com> f3408d9bb6 explicit exports (#1719)
* explicit exports

* more empty exports

* add exports

* reorder

* use correct ControlProtocol type for xftp router

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-03-02 17:34:01 +00:00
sh 3ecb901e3e xftp-web: fix build and Playwright test failures (#1720) 2026-03-02 16:22:36 +00:00
EvgenyEvgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>shumsh
f6aca47604 xftp: implementation of XFTP client as web page (#1708)
* xftp: implementation of XFTP client as web page (rfc, low level functions)

* protocol, file descriptions, more cryptogrpahy, handshake encoding, etc.

* xftp server changes to support web slients: SNI-based certificate choice, CORS headers, OPTIONS request

* web handshake

* test for xftp web handshake

* xftp-web client functions, fix transmission encoding

* support description "redirect" in agent.ts and cross-platform compatibility tests (Haskell <> TypeScript)

* rfc: web transport

* client transport abstraction

* browser environment

* persistent client sessions

* move rfcs

* web page plan

* improve plan

* webpage implementation (not tested)

* fix test

* fix test 2

* fix test 3

* fixes and page test plan

* allow sending xftp client hello after handshake - for web clients that dont know if established connection exists

* page tests pass

* concurrent and padded hellos in the server

* update TS client to pad hellos

* fix tests

* preview:local

* local preview over https

* fixed https in the test page

* web test cert fixtures

* debug logging in web page and server

* remove debug logging in server/browser, run preview xftp server via cabal run to ensure the latest code is used

* debug logging for page sessions

* add plan

* improve error handling, handle browser reconnections/re-handshake

* fix

* debugging

* opfs fallback

* delete test screenshot

* xftp CLI to support link

* fix encoding for XFTPServerHandshake

* support redirect file descriptions in xftp CLI receive

* refactor CLI redirect

* xftp-web: fixes and multi-server upload (#1714)

* fix: await sodium.ready in crypto/keys.ts (+ digest.ts StateAddress cast)

* multi-server parallel upload, remove pickRandomServer

* fix worker message race: wait for ready signal before posting messages

* suppress vite build warnings: emptyOutDir, externals, chunkSizeWarningLimit

* fix Haskell web tests: use agent+server API, wrap server in array, suppress debug logs

* remove dead APIs: un-export connectXFTP, delete closeXFTP

* fix TypeScript errors in check:web (#1716)

- client.ts: cast globalThis.process to any for browser tsconfig,
  suppress node:http2 import, use any for Buffer/chunks, cast fetch body
- crypto.worker.ts: cast sha512_init() return to StateAddress

* fix: serialize worker message processing to prevent OPFS handle race

async onmessage allows interleaved execution at await points.
When downloadFileRaw fetches chunks from multiple servers in parallel,
concurrent handleDecryptAndStore calls both see downloadWriteHandle
as null and race on createSyncAccessHandle for the same file,
causing intermittent NoModificationAllowedError.

Chain message handlers on a promise queue so each runs to completion
before the next starts.

* xftp-web: prepare for npm publishing (#1715)

* prepare package.json for npm publishing

Remove private flag, add description/license/repository/publishConfig,
rename postinstall to pretest, add prepublishOnly, set files and main.

* stable output filenames in production build

* fix repository url format, expand files array

* embeddable component: scoped CSS, dark mode, i18n, events, share

- worker output to assets/ for single-directory deployment
- scoped all CSS under #app, removed global resets
- dark mode via .dark ancestor class
- progress ring reads colors from CSS custom properties
- i18n via window.__XFTP_I18N__ with t() helper
- configurable mount element via data-xftp-app attribute
- optional hashchange listener (data-no-hashchange)
- completion events: xftp:upload-complete, xftp:download-complete
- enhanced file-too-large error mentioning SimpleX app
- native share button via navigator.share

* deferred init and runtime server configuration

- data-defer-init attribute skips auto-initialization
- window.__XFTP_SERVERS__ overrides baked-in server list

* use relative base path for relocatable build output

* xftp-web: retry resets to default state, use innerHTML for errors

* xftp-web: only enter download mode for valid XFTP URIs in hash

* xftp-web: render UI before WASM is ready

Move sodium.ready await after UI initialization so the upload/download
interface appears instantly. WASM is only needed when user triggers
an actual upload or download. Dispatch xftp:ready event once WASM loads.

* xftp-web: CLS placeholder HTML and embedder CSS selectors

Add placeholder HTML to index.html so the page renders a styled card
before JS executes, preventing layout shift. Use a <template> element
with an inline script to swap to the download placeholder when the URL
hash indicates a file download. Auto-compute CSP SHA-256 hashes for
inline scripts in the vite build plugin.

Change all CSS selectors from #app to :is(#app, [data-xftp-app]) so
styles apply when the widget is embedded with data-xftp-app attribute.

* xftp-web: progress ring overhaul

Rewrite progress ring with smooth lerp animation, green checkmark on
completion, theme reactivity via MutationObserver, and per-phase color
variables (encrypt/upload/download/decrypt).

Show honest per-phase progress: each phase animates 0-100% independently
with a ring color change between phases. Add decrypt progress callback
from the web worker so the decryption phase tracks real chunk processing
instead of showing an indeterminate spinner.

Snap immediately on phase reset (0) and completion (1) to avoid
lingering partial progress. Clean up animation and observers via
destroy() in finally blocks.

* xftp-web: single progress ring for upload, simplify ring color

* xftp-web: single progress ring for download

* feat(xftp-web): granular progress for encrypt/decrypt phases

Add byte-level progress callbacks to encryptFile, decryptChunks,
and sha512Streaming by processing data in 256KB segments. Worker
reports fine-grained progress across all phases (encrypt+hash+write
for upload, read+hash+decrypt for download). Progress ring gains
fillTo method for smooth ease-out animation during minimum display
delays. Encrypt/decrypt phases fill their weighted regions (0-15%
and 85-99%) with real callbacks, with fillTo covering remaining
time when work finishes under the 1s minimum for files >= 100KB.

* rename package

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
Co-authored-by: shum <github.shum@liber.li>
Co-authored-by: sh <37271604+shumvgolove@users.noreply.github.com>
2026-03-02 09:57:46 +00:00
Evgeny Poberezkin 50ae1e1c3e docs: update governance process 2026-02-24 18:05:30 +00:00
EvgenyandEvgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com> 483ac674fb agent: fix possible deadlocks of queue overloading when processing messages (#1713)
* agent: fix possibly deadlocks of queue overloading when processing messages

* simplify

* refactor

* refactor

* refactor 2

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-02-24 16:47:00 +00:00
EvgenyandEvgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com> 3dba9c2b1e rfc: SimpleX Network Consortium governance (#1718)
* rfc: SimpleX Network Consortium governance

* update

* add draft label

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-02-22 18:58:09 +00:00
Evgeny Poberezkin 8fdc0703bc 6.5.0.8 2026-01-28 23:12:19 +00:00
Evgeny Poberezkin d7b90b8415 Merge branch 'stable' 2026-01-28 22:52:33 +00:00
Evgeny Poberezkin 9346b85c3f 6.4.8.0 2026-01-28 21:58:32 +00:00
Evgeny 3c5ec8d9a1 agent: improve error handling (#1707)
* agent: improve error handling

* simplify

* report critical error when subscriber crashes

* fix test
2026-01-28 21:54:41 +00:00
Evgeny 84e8b72ca3 docs: update protocol (#1705) 2026-01-27 21:21:54 +00:00
Evgeny d10e05b796 agent: split creating connection to two steps to prepare connection link in advance (#1704)
* agent: split creating connection to two steps to prepare connection link in advance

* linkEntityId, newOwnerAuth

* simplify
2026-01-27 10:54:13 +00:00
Evgeny 66cc06738e agent: fix tests for short links (#1703) 2026-01-26 00:11:50 +00:00
Evgeny Poberezkin 0f3b8a4a16 docs: update contributing 2026-01-24 18:05:01 +00:00
Evgeny Poberezkin ac825b0ff3 Merge branch 'master' into rcv-services 2026-01-24 14:10:51 +00:00
Evgeny a1596ed234 docs: contributing guide (#1702) 2026-01-24 13:33:17 +00:00
spaced4ndyandEvgeny Poberezkin 89b81d151f agent: getConnShortLinkAsync; implement joinConnectionAsync for contact connections; narrow setConnShortLinkAsync only to Contact connections (#1694)
* agent: getConnShortLinkAsync

* enable all tests

* comment

* comment

* join conn async for contact URI (wip)

* fix test

* remove enableNtfs param

* FixedLinkData

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2026-01-20 08:40:35 +00:00
Evgeny 3e5b654109 agent: support multiple link owners in link data (#1701)
* agent: support multiple link owners in link data

* fix
2026-01-19 22:08:11 +00:00
Evgeny Poberezkin 502d923817 agent: minor fixes 2026-01-17 10:21:25 +00:00
Evgeny Poberezkin d908404143 Merge branch 'master' into rcv-services 2026-01-15 18:31:09 +00:00
spaced4ndyandEvgeny Poberezkin ca26c69937 db: withSavepoint; agent: correctly handle errors in createWithRandomId for postgres (#1693)
* db: withSavepoint; agent: correctly handle errors in createWithRandomId

* comment

* refactor

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2026-01-15 14:59:45 +00:00
Evgeny Poberezkin 58212c421a 6.5.0.7 2026-01-13 20:55:38 +00:00
Evgenyandspaced4ndy 1000107259 agent: clean up old records in batches (#1692)
* agent: clean up old records in batches

* update

* update

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* stabilize tests

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2026-01-13 19:11:46 +00:00
Evgeny 6aadcf1f3f agent: lock rows for concurrent queries in PostgreSQL (#1688)
* agent: lock rows for concurrent queries in PostgreSQL

* fix race conditions in workers

* refactor
2026-01-08 11:09:58 +00:00
spaced4ndy 07604a146f agent: async command to set connection short link (setConnShortLinkAsync) (#1680) 2026-01-07 15:12:52 +00:00
Ritiek Malhotra 4c782d3191 docker: build multi-arch images for x86-64 and arm64 (#1540) 2026-01-07 10:49:21 +00:00
Evgeny c4b687ba64 agent: reset schema after changes (#1690) 2026-01-05 20:37:50 +00:00
Evgeny a7b43b1a3e agent: use strict tables (#1686)
* agent: use strict tables

* migrate existing tables to strict

* test: verify that all tables are strict

* fix column types for device_token and ntf_mode

* fix encodings and column types for ntf_sub_action and ntf_sub_smp_action

* update schema

* remove debug.trace

* log
2026-01-03 17:19:18 +00:00
Evgeny d6df769799 ntf server: additional tests (#1687) 2026-01-02 14:07:13 +00:00
Evgeny db4b27e88a agent: create user with option to enable client service (#1684)
* agent: create user with option to enable client service

* handle HTTP2 errors

* do not catch async exceptions
2025-12-27 09:12:22 +00:00
Evgeny Poberezkin 9e813c26a7 Merge branch 'master' into rcv-services 2025-12-25 21:52:07 +00:00
Evgeny bafdbc1dec smp protocol: fix encoding for SOKS/ENDS responses (#1683) 2025-12-25 13:00:29 +00:00
Evgeny Poberezkin 5f73d1e629 6.5.0.6 2025-12-23 18:00:26 +00:00
Evgeny Poberezkin 70d1b99fb4 Merge branch 'stable' 2025-12-23 14:18:07 +00:00
Evgeny Poberezkin 2ea98db9d8 6.4.7.0 2025-12-23 12:42:46 +00:00
Evgeny bbe1c716e6 xrcp: treat any 127.x.x.x IP address as local (#1682) 2025-12-23 12:42:05 +00:00
Evgeny Poberezkin 77ac452190 6.5.0.5 2025-12-22 21:56:25 +00:00
Evgeny 11ae20ea20 ntf server: use different client certs for each SMP server, remove support for store log (#1681)
* ntf server: remove support for store log

* ntf server: use different client certificates for each SMP server
2025-12-22 07:56:53 +00:00
Evgeny a1277bf6bf agent: remove service queue association when service ID changed, process ENDS event, test migrating to/from service (#1677)
* agent: remove service queue association when service ID changed

* agent: process ENDS event

* agent: send service subscription error event

* agent: test migrating to/from service subscriptions, fixes

* agent: always remove service when disabled, fix service subscriptions
2025-12-19 21:10:12 +00:00
Evgeny Poberezkin c8a72431c0 Merge branch 'master' into rcv-services 2025-12-15 11:57:46 +00:00
Evgeny ea70575275 agent: use server cert hash when selecting server queues to subscribe (#1679) 2025-12-15 11:48:17 +00:00
Evgeny Poberezkin 568500c6bc Merge branch 'master' into rcv-services 2025-12-14 23:14:18 +00:00
Evgeny 49e9ce1649 agent: skip subscribing servers without queues (#1678) 2025-12-14 23:01:21 +00:00
Evgeny f5eb735551 servers: service stats and logging, allow services without option (removed), report errors during service message delivery, remove threads when service subscription ended (#1676)
* smp server: always allow services without option

* smp server: maintain IDs hash in session subscription states

* smp server: service message delivery error handling

* ntf server: log subscription count and hash differences

* smp server: remove delivery threads when service subscription ended/client disconnected
2025-12-14 12:07:29 +00:00
Evgeny Poberezkin 83894074b0 Merge branch 'master' into rcv-services 2025-12-13 19:08:47 +00:00
Evgeny Poberezkin 2ca440dd2d 6.5.0.4 2025-12-10 21:09:33 +00:00
Evgeny Poberezkin 35fe5ac9e4 Merge branch 'master' into rcv-services 2025-12-05 21:30:28 +00:00
Evgenyandspaced4ndy 2ea9a9a143 agent: finalize initial service subscriptions, remove associations on service ID changes (#1672)
* agent: remove service/queue associations when service ID changes

* agent: check that service ID in NEW response matches session ID in transport session

* agent subscription WIP

* test

* comment

* enable tests

* update queries

* agent: option to add SQLite aggregates to DB connection  (#1673)

* agent: add build_relations_vector function to sqlite

* update aggregate

* use static aggregate

* remove relations

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>

* add test, treat BAD_SERVICE as temp error, only remove queue associations on service errors

* add packZipWith for backward compatibility with GHC 8.10.7

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2025-12-05 20:46:48 +00:00
spaced4ndyandEvgeny Poberezkin 92a9579e69 agent: option to add SQLite aggregates to DB connection (#1673)
* agent: add build_relations_vector function to sqlite

* update aggregate

* use static aggregate

* remove relations

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2025-12-04 08:58:14 +00:00
Evgeny Poberezkin ff7bdbcbeb Merge branch 'master' into rcv-services 2025-12-03 17:23:59 +00:00
Evgeny cf9b7e5b6a agent: option to add SQLite functions to DB connection (#1674)
* agent: option to add SQLite functions to DB connection

* add module
2025-12-03 16:10:35 +00:00
Evgeny 38e899957f agent: service subscription events (#1671)
* agent: use server keyhash when loading service record

* agent: process queue/service associations with delayed subscription results

* agent: service subscription events
2025-11-27 21:37:19 +00:00
Evgeny 5e9b164f4e agent: fail when per-connection transport isolation is used with services (#1670) 2025-11-25 23:17:47 +00:00
Evgeny 3ccf854865 servers: maintain xor-hash of all associated queue IDs in PostgreSQL (#1668)
* servers: maintain xor-hash of all associated queue IDs in PostgreSQL (#1615)

* ntf server: maintain xor-hash of all associated queue IDs via PostgreSQL triggers

* smp server: xor hash with triggers

* fix sql and using pgcrypto extension in tests

* track counts and hashes in smp/ntf servers via triggers, smp server stats for service subscription, update SMP protocol to pass expected count and hash in SSUB/NSSUB commands

* agent migrations with functions/triggers

* remove agent triggers

* try tracking service subs in the agent (WIP, does not compile)

* Revert "try tracking service subs in the agent (WIP, does not compile)"

This reverts commit 59e908100d.

* comment

* agent database triggers

* service subscriptions in the client

* test / fix client services

* update schema

* fix postgres migration

* update schema

* move schema test to the end

* use static function with SQLite to avoid dynamic wrapper
2025-11-25 16:55:59 +00:00
Evgeny 1ca4677b28 smp server: messaging services (#1565)
* smp server: refactor message delivery to always respond SOK to subscriptions

* refactor ntf subscribe

* cancel subscription thread and reduce service subscription count when queue is deleted

* subscribe rcv service, deliver sent messages to subscribed service

* subscribe rcv service to messages (TODO delivery on subscription)

* WIP

* efficient initial delivery of messages to subscribed service

* test: delivery to client with service certificate

* test: upgrade/downgrade to/from service subscriptions

* remove service association from agent API, add per-user flag to use the service

* agent client (WIP)

* service certificates in the client

* rfc about drift detection, and SALL to mark end of message delivery

* fix test

* fix test

* add function for postgresql message storage

* update migration
2025-11-07 21:36:28 +00:00
spaced4ndyandEvgeny Poberezkin 3016b929b4 agent: make createConnection and setConnShortLink apis support setting all link data fields (#1665)
* agent: make createConnection and setConnShortLink apis support setting all link data fields

* add functions

* refactor

* refactor

* fix tests

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2025-11-05 07:08:33 +00:00
Evgeny Poberezkin 1ae3e8d0be 6.5.0.3 2025-10-23 08:34:11 +01:00
Evgeny 1329fc726f smp: support client notices (#1659)
* agent: support client notices

* improve

* fix, test

* rename

* cleanup

* send and process notices in more cases

* dont delete

* dont remove notice on other permanent errors

* dont remove notice if there is no notice ID in queue

* add server to error

* allow deleting

* only use notice if key hash matches
2025-10-17 18:34:59 +01:00
spaced4ndyandEvgeny 234aeb81dd agent: add subscription status to connection stats (#1658)
* agent: add subscription status to connection stats

* wip

* conn status

* format

* refactor

* refactor

* m

* shorter

* shorter

---------

Co-authored-by: Evgeny <evgeny@poberezkin.com>
2025-10-13 17:53:28 +01:00
spaced4ndyandEvgeny Poberezkin 1dbc15b2e6 agent: sync connections (#1654)
* agent: sync subscriptions

* remove comment

* add shouldDelete flag

* compare api

* remove instance

* query

* rename

* refactor

* functor

* JSON instances

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2025-10-10 05:41:42 +00:00
Evgeny 80aa56cbcc agent: track which queues need subscribing for iOS NSE (#1657)
* agent: track which queues need subscribing for iOS NSE

* fix down migration

* fix, cleanup
2025-10-09 18:33:27 +01:00
Evgeny 9cda20381f agent: subscribe all connections (#1655)
* agent: subscribe all connections

* query, version

* BoolInt

* add query to errors

* Revert "add query to errors"

This reverts commit 32a1f7fe11.

* fix optional field

* version

* limit number of in-flight subscriptions to 35000
2025-10-09 13:43:48 +01:00
Evgeny 318ddf692a smp server: import/export to/from PostgreSQL with one command, deprecation notice for journal storage (#1653) 2025-10-09 13:31:57 +01:00
Evgeny 11a4859f8e agent: batch processing of subscription results and errors (#1652)
* agent: batch processing of subscription results and errors

* run agent tests with in-memory server storage

* version

* non empty errors

* size
2025-10-07 14:49:38 +01:00
Evgeny a3d1a72eb0 agent: optimize subscriptions memory usage (#1651)
* agent: optimize subscriptions memory usage more (do not store subscribed queues in memory) WIP

* use new session subscriptions data

* version

* remove old data structure

* remove version

* batch deletions

* test TSessionSubs

* comment
2025-10-06 15:21:17 +01:00
Evgeny Poberezkin 17b71cf673 Merge branch 'stable' 2025-10-05 22:03:58 +01:00
Evgeny 779222d1a7 agent: optimize subscriptions memory usage (#1649)
* agent: optimize subscriptions memory usage

* version

* remove typeclass

* disable subscription to test memory usage

* disable delivery and commands

* disable reading connections

* enable subscriptions

* RcvQueueCred

* RcvQueueCred postgres

* fix

* optimize keys

* version

* rename

* comments
2025-10-05 13:35:54 +01:00
Evgeny Poberezkin 538dcb6a4c 6.4.6.0 2025-10-04 14:49:42 +01:00
Evgeny Poberezkin 98115b6db1 6.5.0.2 2025-09-30 12:37:28 +01:00
Evgeny 9854caca33 agent: optimize subscriptions (#1645)
* agent: optimize subscriptions

* simplify

* clean up
2025-09-30 12:34:42 +01:00
spaced4ndy c8b551dcf7 agent: fix file reception error in postgres (#1647) 2025-09-30 08:32:45 +01:00
Evgeny 99f40ae109 support for additional database migrations (#1644) 2025-09-27 11:21:38 +01:00
sh 3a9381b966 ci/scripts: execute git version if chmod fails (#1643) 2025-09-23 09:08:56 +01:00
Evgeny Poberezkin 54a2a6c905 6.5.0.1 2025-09-17 09:04:03 +01:00
Evgeny 7b7616ce7e agent: batch loading of connections with PostgreSQL client (#1639)
* agent: batch loading of connections with PostgreSQL client

* batch more

* optimize getPendingCommandServers

* fix Bool conversion

* enable all tests

* cleanup
2025-09-16 14:28:06 +01:00
sh b020a08ea0 ci: disable aarch64 tests (#1638) 2025-09-15 09:02:34 +01:00
Evgeny Poberezkin 2737e7d602 6.5.0.0.1 2025-09-15 07:52:17 +01:00
Evgeny d3754b36bc smp server: do not use queue cache with PostgreSQL message storage (#1637)
* smp server: do not use queue cache with PostgreSQL message storage

* fix loading queues via notifier IDs
2025-09-15 07:51:27 +01:00
Evgeny Poberezkin 112cd9d5f4 Merge branch 'stable' 2025-09-14 14:42:05 +01:00
Evgeny 8a7991a376 agent: option to set SQLite database backup path (#1636)
* agent: option to set SQLite database backup path

* fix test compilation
2025-09-14 12:41:49 +01:00
Evgeny 6f4d0f5377 Merge pull request #1630 from simplex-chat/db-messages
Feature branch: PostgreSQL message store
2025-09-13 19:24:59 +01:00
Evgeny 9cfdae3494 smp server: uniterruptible mask for DB operations (#1635) 2025-09-13 19:08:07 +01:00
Evgeny Poberezkin 62183699db 6.5.0.0 2025-09-13 11:37:59 +01:00
Evgeny 9be9a76b42 smp server: improve message expiration (#1634)
* smp server: limit by time the queues to export journal messages for

* pass queue/msg thresholds separately

* reset db connection on errors

* Revert "smp server: limit by time the queues to export journal messages for"

This reverts commit d3bc0cba4b.

* fix test compilation

* flag to expire messages

* improve test

* expire messages newer than quota
2025-09-13 11:35:34 +01:00
Evgeny Poberezkin 80f7be6dd7 Merge branch 'master' into db-messages 2025-09-13 09:31:23 +01:00
Evgeny 83721240a4 reset db connection on errors (#1633) 2025-09-13 09:30:01 +01:00
Evgeny 6c66cf367a smp server: set message counts correctly after import (#1632) 2025-09-12 13:08:11 +01:00
Evgeny a137d01c90 smp server: improve counts management and expiration for postgres message store (#1631)
* smp server: improve counts management and expiration for postgres message store

* one update

* fix expiration
2025-09-12 11:39:35 +01:00
Evgeny bac6ea6e91 smp server: store messages in PostgreSQL (#1622)
* smp server: store messages in PostgreSQL

* stored procedures to write and to expire messages

* function to export messages

* move all message functions to PostgreSQL, remove delete trigger

* comments

* import messages to db

* fix message import, add export

* fix export

* fix export

* fix compilation flags

* import messages line by line

* fix server start with database storage

* fix compilation

* comments
2025-09-11 20:22:55 +01:00
Evgeny 0c1030cf02 smp server: faster export (#1626)
* smp server: faster export

* flush

* export messages with PostgreSQL database

* remove flush

* batch message writes

* Revert "batch message writes"

This reverts commit 61fb3c3b9e.

* remove $!

* fast journal export

* another approach

* Revert "smp server: remove dependency of message size on the version (#1627)"

This reverts commit 8fea15245a.

* style

* faster?

* cleanup

* cleanup

* refactor

* refactor

* concurrent read messages

* Revert "concurrent read messages"

This reverts commit 05a32e6491.

* concurrent read/write

* parameter to export to/import from another message log file

* Revert "parameter to export to/import from another message log file"

This reverts commit 4e88b035c4.

* Revert "concurrent read/write"

This reverts commit a8eab1f7e3.
2025-09-10 18:22:58 +01:00
Evgeny Poberezkin 23aff6b0b1 Revert "smp server: remove dependency of message size on the version (#1627)"
This reverts commit 8fea15245a.
2025-09-09 09:56:48 +01:00
sh 3335cd5500 ci: add aarch64 builds (#1624) 2025-09-09 08:55:37 +01:00
spaced4ndyandEvgeny Poberezkin a4f049d8da agent: parameterize withWork, getWorkItem with StoreError; parameterized Binary for SQLite (#1617)
* agent: parameterize withWork StoreError

* getWorkItem

* export

* binary

* remove handleWrkErr AnyStoreError constraint

* put AnyError in AnyStoreError constraint

* move typeclass

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2025-09-08 16:38:08 +01:00
Evgeny 8fea15245a smp server: remove dependency of message size on the version (#1627) 2025-09-08 11:28:42 +01:00
sh 42a2c1fc57 ci: fix git in docker (#1623) 2025-09-08 09:41:20 +01:00
Evgeny Poberezkin 7e98b3103f 6.4.5.2 2025-09-07 15:13:19 +01:00
Evgeny Poberezkin 2a61085f07 backward compatible encoding for missing NetworkError 2025-09-07 15:04:06 +01:00
Evgeny 4386dd8b5a smp server: option to expire messages when new message is sent (now off by default) (#1625)
* smp server: option to expire messages when new message is sent (now off by default)

* fix test
2025-09-07 09:02:32 +01:00
Evgeny Poberezkin 50ddd630be 6.4.5.1 2025-09-03 11:12:20 +01:00
Evgeny cb3250e7b4 servers: better socket leak prevention during TLS handshake, add NetworkError type to better diagnose connection errors (#1619)
* servers: better socket leak prevention during TLS handshake

* log tcp connection errors

* more detailed network error

* log full address

* rename error

* add encodings for NetworkError

* refactor

* comment

* bind

* style

* remove parameters of NETWORK error from encoding
2025-09-02 16:07:37 +01:00
Evgeny Poberezkin 0319addd2b 6.4.5.0 2025-09-01 18:57:09 +01:00
Evgeny 77bf76e1f9 smp server: fix in-memory server not restoring queue/service associations after 2+ restarts (#1618)
* smp server: fix in-memory server not restoring queue/service associations after 2+ restarts

* fix test, do not reuse database
2025-09-01 17:45:31 +01:00
Evgeny beafac1f73 agent: make agent workers usable from other contexts (#1614) 2025-08-29 08:33:55 +01:00
Evgeny Poberezkin a2d777bda0 6.4.4.1 2025-08-23 20:05:10 +01:00
Evgeny e48bedeaf2 servers: fix server pages when source code is not specified, include commit SHA in printed version and in web page. (#1608)
* smp server: fix server pages when source code is not specified

* servers: include git commit in version

* flexible alpn

* fix test

* fix ghc 8.10.7 build
2025-08-23 19:59:00 +01:00
Evgeny Poberezkin a2d35281b2 6.4.4.0 2025-08-18 16:22:24 +01:00
spaced4ndy 46035af9a3 agent: add context to SEMsgNotFound error (#1604) 2025-08-18 10:59:57 +01:00
Evgeny Poberezkin 4b7fc34fe3 enable all tests 2025-08-17 10:40:12 +01:00
Evgeny 96e8b4a146 smp server: wrap all queries in transactions (#1603)
* smp server: wrap all queries in transactions

* fix test

* fix schema test
2025-08-17 10:38:20 +01:00
Evgeny 2cedb66667 agent: chat relay address type for short links (#1602) 2025-08-15 07:24:37 +01:00
Evgeny e345671c76 agent: extend xrcp certificate validity 1 hour in the past, to allow out of sync clocks (#1601) 2025-08-14 22:48:11 +01:00
Evgeny 86fb2cddc5 smp server: support short SimpleX addresses in server information / pages (#1600) 2025-08-12 07:31:32 +01:00
Evgeny Poberezkin 931c533a3d 6.4.3.0 2025-08-05 12:30:28 +01:00
Evgeny 79ba60e3ad agent: update contact request server hosts to match server in short link (#1597) 2025-08-05 12:29:50 +01:00
Evgeny fb477b24d7 smp server: support short link URI as queue identifier in control port commands (#1596) 2025-08-05 08:18:39 +01:00
Evgeny Poberezkin 9f263e8f3e docs: changelog 2025-08-02 09:21:44 +01:00
Evgeny Poberezkin db325cb81f 6.4.2.0 2025-07-31 09:21:11 +01:00
Evgeny b167d01f8a rfc: using the same profile from multiple devices (#1588) 2025-07-30 20:54:03 +01:00
Evgenyandsh f4e7469f96 xftp server: prometheus metrics (#1595)
* xftp server: prometheus metrics

* update

Co-authored-by: sh <37271604+shumvgolove@users.noreply.github.com>

---------

Co-authored-by: sh <37271604+shumvgolove@users.noreply.github.com>
2025-07-30 16:18:49 +01:00
Evgeny 4647d69d4b smp server: do not include previously blocked queues in stats, prevent leak of client threads (#1593) 2025-07-30 11:12:57 +01:00
sh 9ab071d62c ci: fix reproduce-schedule (#1594) 2025-07-30 09:17:28 +01:00
Evgeny Poberezkin f4c09ac51f Merge branch 'stable' 2025-07-29 09:09:36 +01:00
Evgeny Poberezkin 846be50f72 6.4.1.0 2025-07-21 15:22:47 +01:00
Evgeny c0f357d817 servers: control port session improvements (#1591)
* servers: prohibit changing role during control port session

* quota for blocked queues

* allow disabling blocking and quota

* fix test

* fix INI file
2025-07-21 15:19:43 +01:00
Evgeny 40fc09a93d smp server: additional stat counter for ntf credentials created together with the queue (#1589)
* smp server: additional stat counter for ntf credentials created together with the queue

* fix prometheus

* fix test

* fix qSub
2025-07-18 09:26:25 +01:00
Evgeny 2a90a2c552 smp protocol: create notification credentials via NEW command that creates the queue (#1586)
* smp protocol: create notification credentials via NEW command that creates the queue

* create ntf subscription for queues created with ntf credetials

* do not create ntf credentials when switching connection to another queue
2025-07-16 16:46:45 +01:00
sh fc581bf729 github/workflows: fix reproduce schedule (#1587) 2025-07-16 11:42:14 +01:00
Evgeny Poberezkin b6ea025333 docs: update build badge 2025-07-15 10:03:27 +01:00
Evgeny Poberezkin d3e2d9ac5b docs: changelog v6.4 2025-07-13 15:20:26 +01:00
Evgeny Poberezkin 85a7fbf538 6.4.0.12 2025-07-12 14:19:32 +01:00
Evgeny 62733ef4c1 smp server: refactor subscriptions and delivery in order to always response SOK on subscription with an optional message to follow. (#1573)
* smp server: refactor subscriptions and delivery

* metric for time between MSG and ACK

* cleanup

* refactor pattern match for ghc 8.10.7

* time buckets

* split max time metric

* histogram

* fix
2025-07-12 14:18:38 +01:00
Evgeny 99e59b73a3 agent: fix message delivery in case one of the connections has no snd queue for any reason - it could break delivery to all connections (#1585)
* agent: fix message delivery in case one of the connections has no snd queue for any reason - it could break delivery to all connections

* simplify

* comment

* refactor, also postpone failing on ratchet sync send prohibited errors

* postpone failing on connection errors to allow subsequent connections succeed
2025-07-10 10:21:12 +01:00
Evgeny 384a3352cf ntf server: more efficient status update query (#1584) 2025-07-08 23:01:57 +01:00
Evgeny Poberezkin 7a04fdff2e 6.4.0.11.3 2025-07-07 18:50:31 +01:00
Evgeny ba3c75e58c smp server: correctly track if ntf service is subscribed and total subscribed queues count (fixes race condition between NSUB and NSUBS from notification server) (#1583) 2025-07-07 18:49:47 +01:00
Evgeny Poberezkin 1062ccc5c3 6.4.0.11.2 2025-07-07 10:07:54 +01:00
Evgeny 36f05e272e agent: support different timeouts for interactive and background requests (#1582)
* agent: support different timeouts for interactive and background requests

* fix tests

* use one constructor for the first request and for retries
2025-07-07 09:38:52 +01:00
Evgeny Poberezkin 660c4293f0 6.4.0.11.1 2025-07-05 11:35:27 +01:00
spaced4ndyandEvgeny Poberezkin 1b8613d767 build: do not exclude postgres modules for client_library flag (#1579)
* don't exclude postgres modules for client_library flag

* fix build with client and server postgres flag

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2025-07-04 17:31:57 +01:00
Evgeny bfa52c4ba5 docs: update RFC for short links to match the implementation (#1581) 2025-07-03 18:09:02 +01:00
spaced4ndy c5eb66038b agent: allow to accept contact requests after address is deleted (#1580) 2025-07-03 09:29:39 +00:00
Evgeny Poberezkin a46edd60f0 6.4.0.11 2025-06-30 21:50:21 +01:00
Evgeny b4bcfd325b agent: better handling errors during connection handshake retries (#1578)
* agent: handle invitation connection handshake errors

* fix/test retries for connecting via address
2025-06-25 19:06:00 +01:00
Evgeny 976bd3a389 agent: short link encodings (#1577) 2025-06-24 10:06:44 +01:00
Evgeny Poberezkin bf27c846da 6.4.0.10.1 2025-06-20 13:49:17 +01:00
Evgeny 455360205c smp server: prometheus histogram for message confirmation times (ACK) (#1575)
* time buckets

* split max time metric

* histogram

* histogram for confirmed delivery times

* gaugehistogram

* fix created, _ in gauge_histogram

* remove comments

* fix metrics
2025-06-20 13:46:59 +01:00
Evgeny 79c67f2026 agent: JSON encoding for connection mode (#1576) 2025-06-19 11:21:29 +01:00
Evgeny c8928626fc ntf server: do not log subscription status updates on subscribing/unsubscribing (#1574) 2025-06-19 09:15:07 +01:00
Evgeny b47d28a22a smp server: prometheus metrics for delievered and not acknowledged messages (to monitor stuck deliveries) (#1572) 2025-06-17 21:10:09 +01:00
Evgeny c5b7d3c7af agent: use flexible (small/large) string encoding for contact link data, for backward/forward compatibility (#1569)
* agent: use large string encoding for contact link data

* size-dependent user link data encoding

* tag large user data encoding with \255
2025-06-16 11:11:59 +01:00
Evgeny Poberezkin d950012530 6.4.0.10 2025-06-14 21:06:54 +01:00
Evgeny Poberezkin 27b1f48929 Merge branch 'stable' 2025-06-14 21:05:59 +01:00
Evgeny Poberezkin 3d62a383d5 6.4.0.3.1 2025-06-14 14:19:31 +01:00
Evgeny 6ac7101f4f agent: make decoding for short link data forward compatible (#1568) 2025-06-14 14:13:14 +01:00
Evgeny 65cc19842c servers: restore constant time for absent queue/wrong key (regression in 6.4-beta.9) (#1567) 2025-06-14 07:10:51 +01:00
Evgeny 656f290660 agent: fix updating short invitation link data (#1566) 2025-06-13 21:27:46 +01:00
Evgeny Poberezkin 643c3c3b3e 6.4.0.9 2025-06-13 09:48:20 +01:00
Evgeny da37384335 smp server: batch commands (#1559)
* protocol: refactor types and encoding

* clean

* smp server: batch commands (#1560)

* smp server: batch commands verification into one DB transaction

* ghc 8.10.7

* flatten transmission tuples

* diff

* only use batch logic if there is more than one transmission

* func

* reset NTF service when adding notifier

* version

* Revert "smp server: use separate database pool for reading queues and creating service records (#1561)"

This reverts commit 3df2425162.

* version

* Revert "version"

This reverts commit d80a6b74c5.
2025-06-12 23:05:04 +01:00
Evgeny 1658048c2c Revert "smp server: use separate database pool for reading queues and creating service records (#1561)" (#1564)
This reverts commit 3df2425162.
2025-06-12 11:04:43 +01:00
Evgeny 27d38518e1 agent: use PQ keys in contact request data inside link container (but not in contact request link); use PQ keys in invitations sent to contact addresses (#1563)
* agent: use PQ keys in contact request data inside link container (but not in contact request link); use PQ keys in invitations sent to contact addresses

* do not use PQ keys in the link with old address versions
2025-06-11 18:22:37 +01:00
Evgeny cf8088ac6a smp server: do not require ALPN for HTTP to fix Android asset links, do not use port 443 in server-server SMP connections (#1562)
* log alpn

* always use HTTP when SNI is sent, regardless of ALPN

* decide credential based on SNI

* do not use web port in SMP/Ntf servers connecting to SMP servers

* simpler

* refactor

* fix
2025-06-10 22:09:07 +01:00
Evgeny Poberezkin 1e82104224 6.4.0.8 2025-06-09 14:20:25 +01:00
Evgeny 46ff37c362 ntf server: additional statistics (#1558)
* ntf server: additional statistics

* version

* fix stats

* add stats to track notifications without active token

* refactor

* fix stats parser

* version
2025-06-09 14:14:27 +01:00
Evgeny 3df2425162 smp server: use separate database pool for reading queues and creating service records (#1561) 2025-06-09 12:47:35 +01:00
Evgeny 5241f5fe5e rfc: client certificates for servers using SMP protocol as clients (opertors' chat relays, notification servers, service bots) (#1534)
* rfc: client certificates for high volume clients (opertors' chat relays, notification servers, service bots)

* client certificates types (WIP)

* parameterize Transport

* protocol/schema/api changes

* agent API

* rename command

* agent subscriptions return local ClientServiceId to chat

* verify transmissions

* fix receiving client certificates, refactor

* ntf server: remove shared queue for all notification subscriptions (#1543)

* ntf server: remove shared queue for all notification subscriptions

* wait for subscriber with timeout

* safer

* refactor

* log

* remove unused

* WIP service subscriptions and associations, refactor

* process service subscriptions

* rename

* simplify switching subscriptions

* SMP service handshake with additional server handshake response

* notification delivery and STM persistence for services

* smp server: database storage, store log, fix encoding for STORE error, replace String with Text in locks and error

* stats

* more stats

* rename SMP commands

* service subscriptions in ntf server agent (tests fail)

* fix

* refactor

* exports

* subscribe ntf server as service for associated queues

* test ntf service connection, fix SOKS response, fix service associations not removed in STM storage

* INI option to support services

* ntf server: downgrade subscriptions when service is no longer supported, track counts of subscribed queues

* smp protocol: include service certificate fingerprint in the string signed over with entity key (TODO two tests fail)

* fix test

* ntf server prometheus stats, use Int64 in SOKS/ENDS responses (to avoid conversions), additional error status for ntf subscription

* update RFC

* refactor useServiceAuth to avoid ad hoc decisions about which commands use service signatures, and to prohibit service signatures on other commands

* remove duplicate service signature syntax check from checkCredentials, it is checked in verifyTransmission

* service errors, todos

* fix checkCredentials in ntf server, service errors

* refactor service auth

* refactor

* service agent: store returned queue count instead of expected

* refactor serverThread

* refactor serviceSig

* rename

* refactor, rename, test repeat NSUB service association

* respond with error to SUBS

* smp server: export/import service records between database and store log

* comment

* comments

* ghc 8.10.7
2025-06-06 08:03:47 +01:00
Evgeny 8e86c97a13 servers: include supported ALPNs in server transport config (#1557) 2025-06-03 13:35:35 +01:00
Evgeny 90e8c3adf6 smp agent: use static RNG function to avoid dynamic C stub created by Haskell FFI wrapper (#1556) 2025-06-02 20:11:27 +01:00
Evgeny 56851365b1 servers: parameterize SMP agent to optimize subscriptions (#1555)
* servers: parameterize SMP agent to optimize subscriptions

* refactor ClientCommand to avoid additional map in the notification server agent

* rename
2025-06-01 16:14:52 +01:00
Evgeny a9814bb6d3 smp server: parameterize server by store type to avoid boxing/unboxing client handles (#1554)
* smp server: parameterize server by store type to avoid boxing/unboxing client handles

* ghc 8.10.7
2025-05-29 22:17:28 +01:00
Evgeny Poberezkin 3ad8bd15a6 6.4.0.7 2025-05-29 13:14:09 +01:00
spaced4ndy 4c33d8ac43 agent: To and FromField instances for AConnShortLink (#1553) 2025-05-29 10:59:36 +01:00
spaced4ndyandEvgeny Poberezkin a94ca62624 agent: allow changing invitation link data before it is secured (#1552)
* agent: setInvitationShortLink api

* Eq instance

* allow changing link data on server, refactor

* fix

* encodings

* remove link data after connection

* Revert "encodings"

This reverts commit f8e254cca9.

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2025-05-29 10:44:16 +01:00
Evgeny 53b72469b6 refactor types (#1551) 2025-05-27 22:31:02 +01:00
Evgeny Poberezkin f80ed32a06 6.4.0.6 2025-05-25 17:28:39 +01:00
Evgeny 07eaf9157b smp server: allow getting and deleting short links for the old contact queues (#1549)
* smp server: allow getting and deleting short links for the old contact queues

* fix verifaction of legacy contact queues

* test
2025-05-25 17:03:02 +01:00
Evgeny 56ea2fdd56 refactor types for DB entity (#1548) 2025-05-24 18:19:11 +01:00
Evgeny ffecd4a17a parameterize transport by peer type (client/server) (#1545)
* parameterize transport by peer type (client/server)

* LogDebug level when test is retried

* support "flipped" HTTP2, fix test retry to avoid retrying pending tests

* move sync to the end of the tests
2025-05-24 14:34:22 +01:00
Evgeny Poberezkin dae649fb87 6.4.0.5 2025-05-24 14:21:53 +01:00
Evgeny 57a77f75c1 smp server: support adding link data to contact addresses created before July 2024 (#1547) 2025-05-24 14:20:25 +01:00
spaced4ndy 18e73b8aa7 agent: pass CRClientData to setContactShortLink (#1546)
* agent: pass CRClientData to setContactShortLink

* fix

* fix
2025-05-23 18:21:36 +01:00
Evgeny af9ca59e51 smp server: optimize concurrency and memory usage, refactor (#1544)
* smp server: optimize concurrency and memory usage, refactor

* hide clients IntMap

* reduce STM contention

* comment

* version

* correct stats for subscriptions

* version

* comment

* remove subscribed clients from map

* version

* optimze, refactor

* version

* debug test

* enable all tests

* remove test logs

* retry failed tests with debug logging

* increase test timeout

* sync between tests
2025-05-23 12:52:18 +01:00
Evgeny Poberezkin d352d518c2 6.4.0.4 2025-05-21 08:56:01 +01:00
Evgeny f0dc600016 ntf server: remove shared queue for all notification subscriptions (#1543)
* ntf server: remove shared queue for all notification subscriptions

* wait for subscriber with timeout

* safer

* refactor

* log
2025-05-21 08:55:32 +01:00
Evgeny Poberezkin f44ea0a6d8 6.4.0.3 2025-05-15 21:01:49 +01:00
Evgeny f7d31d4c02 Merge pull request #1524 from simplex-chat/ntf-storage
ntf server: PostgreSQL database storage (feature branch)
2025-05-15 21:00:58 +01:00
Evgeny b90e25a3a5 ntf server: fix repeat token registration when it became invalid (regression) (#1539) 2025-05-15 19:56:48 +01:00
Evgeny Poberezkin cf4b9f669d agent: always enable periodic notifications 2025-05-15 15:16:44 +01:00
Evgeny Poberezkin e417d35cce Merge branch 'master' into ntf-storage 2025-05-14 22:20:55 +01:00
Evgeny Poberezkin deaec3cce2 6.4.0.2 2025-05-07 07:24:06 +01:00
Evgeny 7bbd99644a agent: fix query that updates last message timestamps for notification processing (#1536) 2025-05-06 23:19:58 +01:00
Evgeny cb59a449dd agent: return error and message absence differently when getting notification messages (#1535)
* agent: return error and message absence differently when getting notification messages

* fix test

* mapM

* inline nse functions, release lock on error or no message
2025-05-06 16:20:01 +01:00
Evgeny a632eea75b protocol: parsing for UTCTime to stop on "," and ";" (#1532) 2025-05-04 21:11:06 +01:00
Evgeny 3d10c9bf9e agent: handle cases when last message ts is not set for notifications; set last ts for "stale" notifications when messages expired and queue is empty, to prevent repeated processing (#1531)
* agent: handle cases when last message ts is not set for notifications; set last ts for "stale" notifications when messages expired and queue is empty, to prevent repeated processing

* only log errors if they exist

* only set last ts for queue that delivered notification
2025-05-02 11:21:39 +01:00
Evgeny 2f0cdc40af ntf server: fix returned last notifications (#1530) 2025-04-30 14:57:04 +01:00
Evgeny 0a3d014f5d ntf server: priority db pool for notifications and client commands, concurrent resubscriptions, CLI options (#1529)
* ntf server: option to skip specific tokens when importing store log

* logs

* batch later

* subscribe in parallel, bigger queues

* ntf server: test schema and migrations

* optimize subscriptions

* log level

* resubscribe before starting

* better subscribing

* pooled concurrent resubscriptions

* 5

* async resubscribe

* typo

* priority pool

* db batch size
2025-04-30 11:40:32 +01:00
Evgeny 7d0115daec ntf server, agent: send all periodic notifications from one thread, only to old active clients or new clients with periodic notification mode (#1528)
* ntf server, agent: send all periodic notifications from one thread, only to old active clients or new clients with periodic notification mode

* send different type via subscription queues

* option to compact store log on start
2025-04-28 12:18:55 +01:00
Evgeny f024ab1c3f ntf server: prometheus metrics (#1527)
* ntf server: save prometheus stats

* info metrics

* fix test
2025-04-27 13:25:44 +01:00
Evgeny f4bc1f0926 ntf server: skip duplicates when importing tokens and subscriptions (#1526)
* ntf server: skip duplicates when importing tokens and subscriptions

* skip imported last notifications when no token or subscription present

* fix skipped imported notifications count

* all tests

* fix test
2025-04-26 23:37:55 +01:00
Evgeny 42dbb887f7 ntf server: allow retries when creating subscriptions, prohibit subscriptions with the same queue but another notifier key or token (#1525)
* ntf server: allow retries when creating subscriptions, prohibit subscriptions with the same queue but another notifier key or token

* sync files in the test

* refactor
2025-04-25 19:41:18 +01:00
Evgeny 850d2fa423 ntf server: PostgreSQL database storage (#1519)
* ntf server: PostgreSQL database storage

* ntf server: import/export stubs

* ntf server postgres db functions

* some notifications tests pass

* notifications tests pass

* import/export notification store logs

* fix ntf server CLI

* log in parralel

* update subscription statuses using executeMany

* fix import/export

* refactor

* fix queries

* prohibit token_id and subscription_id updates, dont update tokens and subscriptions on conflict, improve server insertion, remove duplicate tokens for import, remove subscriptions without tokens for import

* comment

* remame

* increase test delay
2025-04-25 16:47:39 +01:00
Evgeny 08b84deba4 agent: option to use web port by default for preset servers only (#1523)
* agent: option to use web port by default for preset servers only

* shorten/restore short links in agent, add encodings for SMP web port setting

* decouple preset domains from preset servers for short links

* refactor, rename
2025-04-24 17:11:52 +01:00
379 changed files with 54571 additions and 8587 deletions
+35 -16
View File
@@ -58,11 +58,11 @@ jobs:
# =============================
build:
name: "ubuntu-${{ matrix.os }}, GHC: ${{ matrix.ghc }}"
name: "ubuntu-${{ matrix.os }}-${{ matrix.arch }}, GHC: ${{ matrix.ghc }}"
needs: maybe-release
env:
apps: "smp-server xftp-server ntf-server xftp"
runs-on: ubuntu-${{ matrix.os }}
runs-on: ${{ matrix.runner }}
services:
postgres:
image: postgres:15
@@ -81,16 +81,34 @@ jobs:
matrix:
include:
- os: 22.04
os_underscore: 22_04
arch: x86-64
runner: "ubuntu-22.04"
ghc: "8.10.7"
platform_name: 22_04-8.10.7
should_run: ${{ !(github.ref == 'refs/heads/stable' || startsWith(github.ref, 'refs/tags/v')) }}
- os: 22.04
os_underscore: 22_04
arch: x86-64
runner: "ubuntu-22.04"
ghc: "9.6.3"
platform_name: 22_04-x86-64
should_run: true
- os: 24.04
os_underscore: 24_04
arch: x86-64
runner: "ubuntu-24.04"
ghc: "9.6.3"
should_run: true
- os: 22.04
os_underscore: 22_04
arch: aarch64
runner: "ubuntu-22.04-arm"
ghc: "9.6.3"
should_run: true
- os: 24.04
os_underscore: 24_04
arch: aarch64
runner: "ubuntu-24.04-arm"
ghc: "9.6.3"
platform_name: 24_04-x86-64
should_run: true
steps:
- name: Clone project
@@ -127,11 +145,7 @@ jobs:
context: .
load: true
file: Dockerfile.build
tags: build/${{ matrix.platform_name }}:latest
cache-from: |
type=gha
type=gha,scope=master
cache-to: type=gha,mode=max
tags: build/${{ matrix.os }}:latest
build-args: |
TAG=${{ matrix.os }}
GHC=${{ matrix.ghc }}
@@ -143,23 +157,28 @@ jobs:
path: |
~/.cabal/store
dist-newstyle
key: ${{ matrix.os }}-${{ hashFiles('cabal.project', 'simplexmq.cabal') }}
key: ubuntu-${{ matrix.os }}-${{ matrix.arch }}-ghc${{ matrix.ghc }}-${{ hashFiles('cabal.project', 'simplexmq.cabal') }}
- name: Start container
if: matrix.should_run == true
shell: bash
run: |
docker run -t -d \
--device /dev/fuse \
--cap-add SYS_ADMIN \
--security-opt apparmor:unconfined \
--name builder \
-v ~/.cabal:/root/.cabal \
-v /home/runner/work/_temp:/home/runner/work/_temp \
-v ${{ github.workspace }}:/project \
build/${{ matrix.platform_name }}:latest
build/${{ matrix.os }}:latest
- name: Build smp-server (postgresql) and tests
if: matrix.should_run == true
shell: docker exec -t builder sh -eu {0}
run: |
chmod -fR 777 ~/.cabal ./dist-newstyle || :; git config --global --add safe.directory '*'
cabal clean
cabal update
cabal build --jobs=$(nproc) --enable-tests -fserver_postgres
mkdir -p /out
@@ -181,7 +200,7 @@ jobs:
id: prepare-postgres
shell: bash
run: |
name="smp-server-postgres-ubuntu-${{ matrix.platform_name }}"
name="smp-server-postgres-ubuntu-${{ matrix.os_underscore }}-${{ matrix.arch }}"
docker cp builder:/out/smp-server $name
path="${{ github.workspace }}/$name"
@@ -213,9 +232,9 @@ jobs:
printf 'bins<<EOF\n' > bins.output
printf 'hashes<<EOF\n' > hashes.output
for i in ${{ env.apps }}; do
mv ./out/$i ./$i-ubuntu-${{ matrix.platform_name }}
name="$i-ubuntu-${{ matrix.os_underscore }}-${{ matrix.arch }}"
name="$i-ubuntu-${{ matrix.platform_name }}"
mv ./out/$i ./$name
path="${{ github.workspace }}/$name"
hash="SHA2-256($name)= $(openssl sha256 $path | cut -d' ' -f 2)"
@@ -246,7 +265,7 @@ jobs:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Test
if: matrix.should_run == true
if: matrix.should_run == true && matrix.arch == 'x86-64'
timeout-minutes: 120
shell: bash
env:
+8
View File
@@ -39,9 +39,17 @@ jobs:
type=semver,pattern=v{{major}}.{{minor}}
type=semver,pattern=v{{major}}
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Build and push Docker image
uses: simplex-chat/docker-build-push-action@v6
with:
context: .
platforms: linux/amd64,linux/arm64
push: true
build-args: |
APP=${{ matrix.app }}
+2 -2
View File
@@ -25,7 +25,7 @@ jobs:
- name: Execute reproduce script
run: |
${GITHUB_WORKSPACE}/scripts/reproduce-builds.sh "$TAG"
${GITHUB_WORKSPACE}/scripts/simplexmq-reproduce-builds.sh "$TAG" || :
- name: Check if build has been reproduced
env:
@@ -33,7 +33,7 @@ jobs:
user: ${{ secrets.STATUS_SIMPLEX_WEBHOOK_USER }}
pass: ${{ secrets.STATUS_SIMPLEX_WEBHOOK_PASS }}
run: |
if [ -f "${GITHUB_WORKSPACE}/$TAG/_sha256sums" ]; then
if [ -f "${GITHUB_WORKSPACE}/${TAG}-simplexmq/_sha256sums" ]; then
exit 0
else
curl --proto '=https' --tlsv1.2 -sSf \
+1
View File
@@ -11,3 +11,4 @@ cabal.project.local~
.hpc/
*.tix
.coverage
@@ -0,0 +1,53 @@
# XFTPClientAgent Pattern
## TOC
1. Executive Summary
2. Changes: client.ts
3. Changes: agent.ts
4. Changes: test/browser.test.ts
5. Verification
## Executive Summary
Add `XFTPClientAgent` — a per-server connection pool matching the Haskell pattern. The agent caches `XFTPClient` instances by server URL. All orchestration functions (`uploadFile`, `downloadFile`, `deleteFile`) take `agent` as first parameter and use `getXFTPServerClient(agent, server)` instead of calling `connectXFTP` directly. Connections stay open on success; the caller creates and closes the agent.
`connectXFTP` and `closeXFTP` stay exported (used by `XFTPWebTests.hs` Haskell tests). The `browserClients` hack, per-function `connections: Map`, and `getOrConnect` are deleted.
## Changes: client.ts
**Add** after types section: `XFTPClientAgent` interface, `newXFTPAgent`, `getXFTPServerClient`, `closeXFTPServerClient`, `closeXFTPAgent`.
**Delete**: `browserClients` Map and all `isNode` browser-cache checks in `connectXFTP` and `closeXFTP`.
**Revert `closeXFTP`** to unconditional `c.transport.close()` (browser transport.close() is already a no-op).
`connectXFTP` stays exported (backward compat) but becomes a raw low-level function — no caching.
## Changes: agent.ts
**Imports**: replace `connectXFTP`/`closeXFTP` with `getXFTPServerClient`/`closeXFTPAgent` etc.
**Re-export** from agent.ts: `newXFTPAgent`, `closeXFTPAgent`, `XFTPClientAgent`.
**`uploadFile`**: add `agent: XFTPClientAgent` as first param. Replace `connectXFTP``getXFTPServerClient`. Remove `finally { closeXFTP }`. Pass `agent` to `uploadRedirectDescription`.
**`uploadRedirectDescription`**: change from `(client, server, innerFd)` to `(agent, server, innerFd)`. Get client via `getXFTPServerClient`.
**`downloadFile`**: add `agent` param. Delete local `connections: Map`. Replace `getOrConnect``getXFTPServerClient`. Remove finally cleanup. Pass `agent` to `downloadWithRedirect`.
**`downloadWithRedirect`**: add `agent` param. Same replacements. Remove try/catch cleanup. Recursive call passes `agent`.
**`deleteFile`**: add `agent` param. Same pattern.
**Delete**: `getOrConnect` function entirely.
## Changes: test/browser.test.ts
Create agent before operations, pass to upload/download, close in finally.
## Verification
1. `npx vitest --run` — browser round-trip test passes
2. No remaining `browserClients`, `getOrConnect`, or per-function `connections: Map` locals
3. `connectXFTP` and `closeXFTP` still exported (XFTPWebTests.hs compat)
4. All orchestration functions take `agent` as first param
+60 -2
View File
@@ -1,3 +1,61 @@
# 6.4.4
Servers:
- fix server pages when source code is not specified.
- include commit SHA in printed version and in web page (#1608).
SMP server:
- support short SimpleX addresses in server information page (#1600).
- wrap all queries in transactions (#1603).
SMP agent:
- chat relay address type for short links (#1602).
- extend xrcp certificate validity 1 hour in the past, to allow out of sync clocks (#1601).
# 6.4.3
SMP agent:
- fix some connection errors by updating contact request server hosts to match server in short link (#1597).
SMP server:
- support short link URI as queue identifier in control port commands (#1596).
# 6.4.2
SMP server:
- fix memory leak when connection interrupts straight after client connects.
- do not include repeated queue blocking into stats/quota.
XFTP server:
- prometheus metrics
# 6.4.1
SMP protocol:
- create notification credentials via NEW command that creates the queue (#1586)
SMP server:
- control port session improvements (#1591)
- additional stat counter for ntf credentials created together with the queue (#1589)
# 6.4.0
SMP protocol (server/client):
- support associated queue data and short connection links (see [RFC](./rfcs/2025-03-16-smp-queues.md)).
- service certificates to optimize subscriptions.
SMP agent:
- support retries for interactive connection handshakes.
- use web port 443 by default for preset servers.
- use static RNG function to avoid creating dynamic C stubs when generating sntrup keys (it was detected as Dynamic Code Loading in GrapheneOS).
- different timeouts for interactive and background operations.
Ntf server:
- PostgreSQL storage.
- Prometheus metrics.
- use service certificates.
- fix repeat token registration.
# 6.3.2
Servers:
@@ -18,7 +76,7 @@ Servers:
- update script (simplex-servers-update) downloads scripts from the specified or the latest stable tag.
SMP server:
- support for PostrgreSQL database for queue records for higher traffic servers.
- support for PostgreSQL database for queue records for higher traffic servers.
- fix old clients sending messages to new servers (#1443)
- remove empty journals when opening message queues and expiring idle queues (#1456, #1458).
- additional start options (#1465):
@@ -74,7 +132,7 @@ Servers: more reliable restoring of state.
SMP server: reduced memory usage and faster start.
Notifications: compensate for iOS notifications being droppted by Apple while device is offline (#1378):
Notifications: compensate for iOS notifications being dropped by Apple while device is offline (#1378):
- Ntf server: send multiple SMP notifications in one iOS notification.
- Agent: get multiple messages for one iOS notification.
+6 -6
View File
@@ -1,6 +1,6 @@
# SimpleXMQ
[![GitHub build](https://github.com/simplex-chat/simplexmq/workflows/build/badge.svg)](https://github.com/simplex-chat/simplexmq/actions?query=workflow%3Abuild)
[![GitHub build](https://github.com/simplex-chat/simplexmq/actions/workflows/build.yml/badge.svg)](https://github.com/simplex-chat/simplexmq/actions/workflows/build.yml)
[![GitHub release](https://img.shields.io/github/v/release/simplex-chat/simplexmq)](https://github.com/simplex-chat/simplexmq/releases)
📢 SimpleXMQ v1 is released - with many security, privacy and efficiency improvements, new functionality - see [release notes](https://github.com/simplex-chat/simplexmq/releases/tag/v1.0.0).
@@ -116,7 +116,7 @@ On Linux, you can deploy smp and xftp server using Docker. This will download im
2. Run your Docker container.
- `smp-server`
You must change **your_ip_or_domain**. `-e "pass=password"` is optional variable to password-protect your `smp` server:
```sh
docker run -d \
@@ -129,7 +129,7 @@ On Linux, you can deploy smp and xftp server using Docker. This will download im
```
- `xftp-server`
You must change **your_ip_or_domain** and **maximum_storage**.
```sh
docker run -d \
@@ -187,7 +187,7 @@ On Linux, you can build smp server using Docker.
3. Run your Docker container.
- `smp-server`
You must change **your_ip_or_domain**. `-e "pass=password"` is optional variable to password-protect your `smp` server:
```sh
docker run -d \
@@ -200,7 +200,7 @@ On Linux, you can build smp server using Docker.
```
- `xftp-server`
You must change **your_ip_or_domain** and **maximum_storage**.
```sh
docker run -d \
@@ -247,7 +247,7 @@ On Linux, you can build smp server using Docker.
`xftp-server`
```sh
cabal list-bin exe:xftp-server
cabal list-bin exe:xftp-server
```
- Initialize SMP server with `smp-server init [-l] -n <fqdn>` or `smp-server init [-l] --ip <ip>` - depending on how you initialize it, either FQDN or IP will be used for server's address.
+15
View File
@@ -0,0 +1,15 @@
{-# LANGUAGE TemplateHaskell #-}
module Web.Embedded where
import Data.FileEmbed (embedDir, embedFile)
import Simplex.Messaging.Server.Web (EmbeddedContent (..))
embeddedContent :: EmbeddedContent
embeddedContent =
EmbeddedContent
{ indexHtml = $(embedFile "apps/common/Web/static/index.html"),
linkHtml = $(embedFile "apps/common/Web/static/link.html"),
mediaContent = $(embedDir "apps/common/Web/static/media/"),
wellKnown = $(embedDir "apps/common/Web/static/.well-known/")
}
@@ -105,6 +105,13 @@
class="text-[16px] leading-[26px] tracking-[0.01em] nav-link-text text-black dark:text-white before:bg-black dark:before:bg-white">Server
information</span></a>
</li>
<x-xftpConfig>
<li class="nav-link relative"><a href="/file"
class="flex items-center justify-between gap-2 lg:py-5 whitespace-nowrap"><span
class="text-[16px] leading-[26px] tracking-[0.01em] nav-link-text text-black dark:text-white before:bg-black dark:before:bg-white">File
transfer</span></a>
</li>
</x-xftpConfig>
</ul><a target="_blank" href="https://github.com/simplex-chat/simplex-chat#help-us-with-donations"
class="whitespace-nowrap flex items-center gap-1 self-center text-white dark:text-black text-[16px] font-medium tracking-[0.02em] rounded-[34px] bg-primary-light dark:bg-primary-dark py-3 lg:py-2 px-20 lg:px-5 mb-16 lg:mb-0">Donate</a>
</div>
@@ -223,11 +230,14 @@
<table id="public-info">
<tr class="text-grey-black dark:text-white text-base">
<td>Server version:</td>
<td>${version}</td>
<td>${version}<x-commit> / <a href="${commitSourceCode}/commit/${commit}" target="_blank">${shortCommit}</a></x-commit></td>
</tr>
<tr class="text-grey-black dark:text-white text-base">
<td>Source code:</td>
<td><a href="${sourceCode}" target="_blank">${sourceCode}</a></td>
<td>
<x-sourceCode><a href="${sourceCode}" target="_blank">${sourceCode}</a></x-sourceCode>
<x-noSourceCode>add to ${iniFileName} (required by <a href="https://github.com/simplex-chat/simplexmq/blob/stable/LICENSE" target="_blank">AGPLv3</a>)</x-noSourceCode>
</td>
</tr>
<x-website>
<tr class="text-grey-black dark:text-white text-base">
@@ -314,6 +324,7 @@
<h2 class="text-[30px] mb-[20px] leading-[28px] text-[#606C71] dark:text-white font-bold max-w-[475px]">
Configuration</h2>
<table id="config">
<x-smpConfig>
<tr class="text-grey-black dark:text-white text-base">
<td>Persistence:</td>
<td>${persistence}</td>
@@ -334,6 +345,25 @@
<td>Basic auth enabled:</td>
<td>${basicAuthEnabled}</td>
</tr>
</x-smpConfig>
<x-xftpConfig>
<tr class="text-grey-black dark:text-white text-base">
<td>File expiration:</td>
<td>${fileExpiration}</td>
</tr>
<tr class="text-grey-black dark:text-white text-base">
<td>Stats enabled:</td>
<td>${statsEnabled}</td>
</tr>
<tr class="text-grey-black dark:text-white text-base">
<td>New uploads allowed:</td>
<td>${newUploadsAllowed}</td>
</tr>
<tr class="text-grey-black dark:text-white text-base">
<td>Basic auth enabled:</td>
<td>${basicAuthEnabled}</td>
</tr>
</x-xftpConfig>
</table>
</div>
</div>
@@ -512,6 +512,8 @@
element.innerHTML = 'This is a one-time link of the SimpleX&nbsp;network user'
} else if (url.includes('/c')) {
element.innerHTML = 'This is a public channel address on SimpleX&nbsp;network'
} else if (url.includes('/r')) {
element.innerHTML = 'This is a chat relay address on SimpleX&nbsp;network'
}
}
</script>

Before

Width:  |  Height:  |  Size: 18 KiB

After

Width:  |  Height:  |  Size: 18 KiB

Before

Width:  |  Height:  |  Size: 16 KiB

After

Width:  |  Height:  |  Size: 16 KiB

Before

Width:  |  Height:  |  Size: 289 KiB

After

Width:  |  Height:  |  Size: 289 KiB

Before

Width:  |  Height:  |  Size: 12 KiB

After

Width:  |  Height:  |  Size: 12 KiB

Before

Width:  |  Height:  |  Size: 1.1 KiB

After

Width:  |  Height:  |  Size: 1.1 KiB

Before

Width:  |  Height:  |  Size: 6.3 KiB

After

Width:  |  Height:  |  Size: 6.3 KiB

Before

Width:  |  Height:  |  Size: 7.2 KiB

After

Width:  |  Height:  |  Size: 7.2 KiB

Before

Width:  |  Height:  |  Size: 8.1 KiB

After

Width:  |  Height:  |  Size: 8.1 KiB

Before

Width:  |  Height:  |  Size: 1.1 KiB

After

Width:  |  Height:  |  Size: 1.1 KiB

Before

Width:  |  Height:  |  Size: 1.2 KiB

After

Width:  |  Height:  |  Size: 1.2 KiB

Before

Width:  |  Height:  |  Size: 632 B

After

Width:  |  Height:  |  Size: 632 B

Before

Width:  |  Height:  |  Size: 3.5 KiB

After

Width:  |  Height:  |  Size: 3.5 KiB

Before

Width:  |  Height:  |  Size: 18 KiB

After

Width:  |  Height:  |  Size: 18 KiB

-1
View File
@@ -15,7 +15,6 @@ logCfg = LogConfig {lc_file = Nothing, lc_stderr = True}
main :: IO ()
main = do
setLogLevel LogInfo
cfgPath <- getEnvPath "NTF_SERVER_CFG_PATH" defaultCfgPath
logPath <- getEnvPath "NTF_SERVER_LOG_PATH" defaultLogPath
withGlobalLogging logCfg $ ntfServerCLI cfgPath logPath
+4 -4
View File
@@ -2,8 +2,9 @@ module Main where
import Control.Logger.Simple
import Simplex.Messaging.Server.CLI (getEnvPath)
import Simplex.Messaging.Server.Main
import qualified Static
import Simplex.Messaging.Server.Main (smpServerCLI_)
import Simplex.Messaging.Server.Web (serveStaticFiles, attachStaticFiles)
import SMPWeb (smpGenerateSite)
defaultCfgPath :: FilePath
defaultCfgPath = "/etc/opt/simplex"
@@ -16,7 +17,6 @@ logCfg = LogConfig {lc_file = Nothing, lc_stderr = True}
main :: IO ()
main = do
setLogLevel LogDebug
cfgPath <- getEnvPath "SMP_SERVER_CFG_PATH" defaultCfgPath
logPath <- getEnvPath "SMP_SERVER_LOG_PATH" defaultLogPath
withGlobalLogging logCfg $ smpServerCLI_ Static.generateSite Static.serveStaticFiles Static.attachStaticFiles cfgPath logPath
withGlobalLogging logCfg $ smpServerCLI_ smpGenerateSite serveStaticFiles attachStaticFiles cfgPath logPath
+43
View File
@@ -0,0 +1,43 @@
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE OverloadedStrings #-}
module SMPWeb
( smpGenerateSite,
serverInformation,
) where
import Data.ByteString (ByteString)
import Data.String (fromString)
import Web.Embedded (embeddedContent)
import Simplex.Messaging.Encoding.String (strEncode)
import Simplex.Messaging.Server.Information
import Simplex.Messaging.Server.Main (simplexmqSource)
import qualified Simplex.Messaging.Server.Web as Web
import Simplex.Messaging.Server.Web (render, serverInfoSubsts, timedTTLText)
import Simplex.Messaging.Transport.Client (TransportHost (..))
smpGenerateSite :: ServerInformation -> Maybe TransportHost -> FilePath -> IO ()
smpGenerateSite si onionHost path =
Web.generateSite embeddedContent (serverInformation si onionHost) smpLinkPages path
smpLinkPages :: [String]
smpLinkPages = ["contact", "invitation", "a", "c", "g", "r", "i"]
serverInformation :: ServerInformation -> Maybe TransportHost -> ByteString
serverInformation ServerInformation {config, information} onionHost = render (Web.indexHtml embeddedContent) substs
where
substs = [("smpConfig", Just "y"), ("xftpConfig", Nothing)] <> substConfig <> serverInfoSubsts simplexmqSource information <> [("onionHost", strEncode <$> onionHost), ("iniFileName", Just "smp-server.ini")]
substConfig =
[ ( "persistence",
Just $ case persistence config of
SPMMemoryOnly -> "In-memory only"
SPMQueues -> "Queues"
SPMMessages -> "Queues and messages"
),
("messageExpiration", Just $ maybe "Never" (fromString . timedTTLText) $ messageExpiration config),
("statsEnabled", Just . yesNo $ statsEnabled config),
("newQueuesAllowed", Just . yesNo $ newQueuesAllowed config),
("basicAuthEnabled", Just . yesNo $ basicAuthEnabled config)
]
yesNo True = "Yes"
yesNo False = "No"
-1
View File
@@ -1 +0,0 @@
../link.html
-1
View File
@@ -1 +0,0 @@
../link.html
@@ -1 +0,0 @@
../link.html
-1
View File
@@ -1 +0,0 @@
../link.html
@@ -1 +0,0 @@
../link.html
-244
View File
@@ -1,244 +0,0 @@
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE OverloadedStrings #-}
module Static where
import Control.Logger.Simple
import Control.Monad
import Data.ByteString (ByteString)
import qualified Data.ByteString.Char8 as B
import Data.Char (toUpper)
import Data.IORef (readIORef)
import Data.Maybe (fromMaybe)
import Data.String (fromString)
import Data.Text.Encoding (encodeUtf8)
import Network.Socket (getPeerName)
import Network.Wai (Application, Request (..))
import Network.Wai.Application.Static (StaticSettings (..))
import qualified Network.Wai.Application.Static as S
import qualified Network.Wai.Handler.Warp as W
import qualified Network.Wai.Handler.Warp.Internal as WI
import qualified Network.Wai.Handler.WarpTLS as WT
import Simplex.Messaging.Encoding.String (strEncode)
import Simplex.Messaging.Server (AttachHTTP)
import Simplex.Messaging.Server.Information
import Simplex.Messaging.Server.Main (EmbeddedWebParams (..), WebHttpsParams (..))
import Simplex.Messaging.Transport (simplexMQVersion)
import Simplex.Messaging.Transport.Client (TransportHost (..))
import Simplex.Messaging.Util (tshow)
import Static.Embedded as E
import System.Directory (createDirectoryIfMissing)
import System.FilePath
import UnliftIO.Concurrent (forkFinally)
import UnliftIO.Exception (bracket, finally)
import qualified WaiAppStatic.Types as WAT
serveStaticFiles :: EmbeddedWebParams -> IO ()
serveStaticFiles EmbeddedWebParams {webStaticPath, webHttpPort, webHttpsParams} = do
forM_ webHttpPort $ \port -> flip forkFinally (\e -> logError $ "HTTP server crashed: " <> tshow e) $ do
logInfo $ "Serving static site on port " <> tshow port
W.runSettings (mkSettings port) app
forM_ webHttpsParams $ \WebHttpsParams {port, cert, key} -> flip forkFinally (\e -> logError $ "HTTPS server crashed: " <> tshow e) $ do
logInfo $ "Serving static site on port " <> tshow port <> " (TLS)"
WT.runTLS (WT.tlsSettings cert key) (mkSettings port) app
where
app = staticFiles webStaticPath
mkSettings port = W.setPort port warpSettings
-- | Prepare context and prepare HTTP handler for TLS connections that already passed TLS.handshake and ALPN check.
attachStaticFiles :: FilePath -> (AttachHTTP -> IO ()) -> IO ()
attachStaticFiles path action =
-- Initialize global internal state for http server.
WI.withII warpSettings $ \ii -> do
action $ \socket cxt -> do
-- Initialize internal per-connection resources.
addr <- getPeerName socket
withConnection addr cxt $ \(conn, transport) ->
withTimeout ii conn $ \th ->
-- Run Warp connection handler to process HTTP requests for static files.
WI.serveConnection conn ii th addr transport warpSettings app
where
app = staticFiles path
-- from warp-tls
withConnection socket cxt = bracket (WT.attachConn socket cxt) (terminate . fst)
-- from warp
withTimeout ii conn =
bracket
(WI.registerKillThread (WI.timeoutManager ii) (WI.connClose conn))
WI.cancel
-- shared clean up
terminate conn = WI.connClose conn `finally` (readIORef (WI.connWriteBuffer conn) >>= WI.bufFree)
warpSettings :: W.Settings
warpSettings = W.setGracefulShutdownTimeout (Just 1) W.defaultSettings
staticFiles :: FilePath -> Application
staticFiles root = S.staticApp settings . changeWellKnownPath
where
settings = defSettings {ssListing = Nothing, ssGetMimeType = getMimeType}
defSettings = S.defaultFileServerSettings root
getMimeType f
| WAT.fromPiece (WAT.fileName f) == "apple-app-site-association" = pure "application/json"
| otherwise = (ssGetMimeType defSettings) f
changeWellKnownPath req = case pathInfo req of
".well-known" : rest ->
req
{ pathInfo = "well-known" : rest,
rawPathInfo = "/well-known/" <> B.drop pfxLen (rawPathInfo req)
}
_ -> req
pfxLen = B.length "/.well-known/"
generateSite :: ServerInformation -> Maybe TransportHost -> FilePath -> IO ()
generateSite si onionHost sitePath = do
createDirectoryIfMissing True sitePath
B.writeFile (sitePath </> "index.html") $ serverInformation si onionHost
copyDir "media" E.mediaContent
-- `.well-known` path is re-written in changeWellKnownPath,
-- staticApp does not allow hidden folders.
copyDir "well-known" E.wellKnown
createLinkPage "contact"
createLinkPage "invitation"
createLinkPage "a"
createLinkPage "c"
createLinkPage "g"
createLinkPage "i"
logInfo $ "Generated static site contents at " <> tshow sitePath
where
copyDir dir content = do
createDirectoryIfMissing True $ sitePath </> dir
forM_ content $ \(path, s) -> B.writeFile (sitePath </> dir </> path) s
createLinkPage path = do
createDirectoryIfMissing True $ sitePath </> path
B.writeFile (sitePath </> path </> "index.html") E.linkHtml
serverInformation :: ServerInformation -> Maybe TransportHost -> ByteString
serverInformation ServerInformation {config, information} onionHost = render E.indexHtml substs
where
substs = substConfig <> maybe [] substInfo information <> [("onionHost", strEncode <$> onionHost)]
substConfig =
[ ( "persistence",
Just $ case persistence config of
SPMMemoryOnly -> "In-memory only"
SPMQueues -> "Queues"
SPMMessages -> "Queues and messages"
),
("messageExpiration", Just $ maybe "Never" (fromString . timedTTLText) $ messageExpiration config),
("statsEnabled", Just . yesNo $ statsEnabled config),
("newQueuesAllowed", Just . yesNo $ newQueuesAllowed config),
("basicAuthEnabled", Just . yesNo $ basicAuthEnabled config)
]
yesNo True = "Yes"
yesNo False = "No"
substInfo spi =
concat
[ basic,
maybe [("usageConditions", Nothing), ("usageAmendments", Nothing)] conds (usageConditions spi),
maybe [("operator", Nothing)] operatorE (operator spi),
maybe [("admin", Nothing)] admin (adminContacts spi),
maybe [("complaints", Nothing)] complaints (complaintsContacts spi),
maybe [("hosting", Nothing)] hostingE (hosting spi),
server
]
where
basic =
[ ("sourceCode", Just . encodeUtf8 $ sourceCode spi),
("version", Just $ B.pack simplexMQVersion),
("website", encodeUtf8 <$> website spi)
]
conds ServerConditions {conditions, amendments} =
[ ("usageConditions", Just $ encodeUtf8 conditions),
("usageAmendments", encodeUtf8 <$> amendments)
]
operatorE Entity {name, country} =
[ ("operator", Just ""),
("operatorEntity", Just $ encodeUtf8 name),
("operatorCountry", encodeUtf8 <$> country)
]
admin ServerContactAddress {simplex, email, pgp} =
[ ("admin", Just ""),
("adminSimplex", strEncode <$> simplex),
("adminEmail", encodeUtf8 <$> email),
("adminPGP", encodeUtf8 . pkURI <$> pgp),
("adminPGPFingerprint", encodeUtf8 . pkFingerprint <$> pgp)
]
complaints ServerContactAddress {simplex, email, pgp} =
[ ("complaints", Just ""),
("complaintsSimplex", strEncode <$> simplex),
("complaintsEmail", encodeUtf8 <$> email),
("complaintsPGP", encodeUtf8 . pkURI <$> pgp),
("complaintsPGPFingerprint", encodeUtf8 . pkFingerprint <$> pgp)
]
hostingE Entity {name, country} =
[ ("hosting", Just ""),
("hostingEntity", Just $ encodeUtf8 name),
("hostingCountry", encodeUtf8 <$> country)
]
server =
[ ("serverCountry", encodeUtf8 <$> serverCountry spi),
("hostingType", (\s -> maybe s (\(c, rest) -> toUpper c `B.cons` rest) $ B.uncons s) . strEncode <$> hostingType spi)
]
-- Copy-pasted from simplex-chat Simplex.Chat.Types.Preferences
{-# INLINE timedTTLText #-}
timedTTLText :: (Integral i, Show i) => i -> String
timedTTLText 0 = "0 sec"
timedTTLText ttl = do
let (m', s) = ttl `quotRem` 60
(h', m) = m' `quotRem` 60
(d', h) = h' `quotRem` 24
(mm, d) = d' `quotRem` 30
unwords $
[mms mm | mm /= 0]
<> [ds d | d /= 0]
<> [hs h | h /= 0]
<> [ms m | m /= 0]
<> [ss s | s /= 0]
where
ss s = show s <> " sec"
ms m = show m <> " min"
hs 1 = "1 hour"
hs h = show h <> " hours"
ds 1 = "1 day"
ds 7 = "1 week"
ds 14 = "2 weeks"
ds d = show d <> " days"
mms 1 = "1 month"
mms mm = show mm <> " months"
-- | Rewrite source with provided substitutions
render :: ByteString -> [(ByteString, Maybe ByteString)] -> ByteString
render src = \case
[] -> src
(label, content') : rest -> render (section_ label content' src) rest
-- | Rewrite section content inside @<x-label>...</x-label>@ markers.
-- Markers are always removed when found. Closing marker is mandatory.
-- If content is absent, whole section is removed.
-- Section content is delegated to `item_`. If no sections found, the whole source is delegated.
section_ :: ByteString -> Maybe ByteString -> ByteString -> ByteString
section_ label content' src =
case B.breakSubstring startMarker src of
(_, "") -> item_ label (fromMaybe "" content') src -- no section, just replace items
(before, afterStart') ->
-- found section start, search for end too
case B.breakSubstring endMarker $ B.drop (B.length startMarker) afterStart' of
(_, "") -> error $ "missing section end: " <> show endMarker
(inside, next') ->
let next = B.drop (B.length endMarker) next'
in case content' of
Nothing -> before <> next -- collapse section
Just content -> before <> item_ label content inside <> section_ label content' next
where
startMarker = "<x-" <> label <> ">"
endMarker = "</x-" <> label <> ">"
-- | Replace all occurences of @${label}@ with provided content.
item_ :: ByteString -> ByteString -> ByteString -> ByteString
item_ label content' src =
case B.breakSubstring marker src of
(done, "") -> done
(before, after') -> before <> content' <> item_ label content' (B.drop (B.length marker) after')
where
marker = "${" <> label <> "}"
-18
View File
@@ -1,18 +0,0 @@
{-# LANGUAGE TemplateHaskell #-}
module Static.Embedded where
import Data.FileEmbed (embedDir, embedFile)
import Data.ByteString (ByteString)
indexHtml :: ByteString
indexHtml = $(embedFile "apps/smp-server/static/index.html")
linkHtml :: ByteString
linkHtml = $(embedFile "apps/smp-server/static/link.html")
mediaContent :: [(FilePath, ByteString)]
mediaContent = $(embedDir "apps/smp-server/static/media/")
wellKnown :: [(FilePath, ByteString)]
wellKnown = $(embedDir "apps/smp-server/static/.well-known/")
+4 -2
View File
@@ -1,8 +1,10 @@
module Main where
import Control.Logger.Simple
import Simplex.FileTransfer.Server.Main (xftpServerCLI_)
import Simplex.Messaging.Server.CLI (getEnvPath)
import Simplex.FileTransfer.Server.Main
import Simplex.Messaging.Server.Web (serveStaticFiles)
import XFTPWeb (xftpGenerateSite)
defaultCfgPath :: FilePath
defaultCfgPath = "/etc/opt/simplex-xftp"
@@ -18,4 +20,4 @@ main = do
setLogLevel LogDebug -- change to LogError in production
cfgPath <- getEnvPath "XFTP_SERVER_CFG_PATH" defaultCfgPath
logPath <- getEnvPath "XFTP_SERVER_LOG_PATH" defaultLogPath
withGlobalLogging logCfg $ xftpServerCLI cfgPath logPath
withGlobalLogging logCfg $ xftpServerCLI_ xftpGenerateSite serveStaticFiles cfgPath logPath
+67
View File
@@ -0,0 +1,67 @@
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE TemplateHaskell #-}
module XFTPWeb
( xftpGenerateSite,
xftpServerInformation,
) where
import Control.Monad (forM_)
import qualified Data.ByteString.Char8 as B
import Data.ByteString (ByteString)
import Data.FileEmbed (embedDir, embedFile)
import Data.Maybe (isJust)
import Data.String (fromString)
import Web.Embedded (embeddedContent)
import Simplex.FileTransfer.Server.Env (XFTPServerConfig (..))
import Simplex.Messaging.Encoding.String (strEncode)
import Simplex.Messaging.Server.Expiration (ExpirationConfig (..))
import Simplex.Messaging.Server.Information (ServerPublicInfo)
import Simplex.Messaging.Server.Main (simplexmqSource)
import qualified Simplex.Messaging.Server.Web as Web
import Simplex.Messaging.Server.Web (render, serverInfoSubsts, timedTTLText)
import Simplex.Messaging.Transport.Client (TransportHost (..))
import System.Directory (createDirectoryIfMissing)
import System.FilePath ((</>))
xftpWebContent :: [(FilePath, ByteString)]
xftpWebContent = $(embedDir "apps/xftp-server/static/xftp-web-bundle/")
xftpMediaContent :: [(FilePath, ByteString)]
xftpMediaContent = $(embedDir "apps/xftp-server/static/media/")
xftpFilePageHtml :: ByteString
xftpFilePageHtml = $(embedFile "apps/xftp-server/static/file.html")
xftpGenerateSite :: XFTPServerConfig -> Maybe ServerPublicInfo -> Maybe TransportHost -> FilePath -> IO ()
xftpGenerateSite cfg info onionHost path = do
let substs = xftpSubsts cfg info onionHost
Web.generateSite embeddedContent (render (Web.indexHtml embeddedContent) substs) [] path
let xftpDir = path </> "xftp-web-bundle"
mediaDir = path </> "media"
fileDir = path </> "file"
filePage xftpDir xftpWebContent
filePage mediaDir xftpMediaContent
createDirectoryIfMissing True fileDir
B.writeFile (fileDir </> "index.html") $ render xftpFilePageHtml substs
where
filePage dir content_ = do
createDirectoryIfMissing True dir
forM_ content_ $ \(fp, content) -> B.writeFile (dir </> fp) content
xftpServerInformation :: XFTPServerConfig -> Maybe ServerPublicInfo -> Maybe TransportHost -> ByteString
xftpServerInformation cfg info onionHost = render (Web.indexHtml embeddedContent) (xftpSubsts cfg info onionHost)
xftpSubsts :: XFTPServerConfig -> Maybe ServerPublicInfo -> Maybe TransportHost -> [(ByteString, Maybe ByteString)]
xftpSubsts XFTPServerConfig {fileExpiration, logStatsInterval, allowNewFiles, newFileBasicAuth} information onionHost =
[("smpConfig", Nothing), ("xftpConfig", Just "y")] <> substConfig <> serverInfoSubsts simplexmqSource information <> [("onionHost", strEncode <$> onionHost), ("iniFileName", Just "file-server.ini")]
where
substConfig =
[ ("fileExpiration", Just $ maybe "Never" (fromString . timedTTLText . ttl) fileExpiration),
("statsEnabled", Just . yesNo $ isJust logStatsInterval),
("newUploadsAllowed", Just . yesNo $ allowNewFiles),
("basicAuthEnabled", Just . yesNo $ isJust newFileBasicAuth)
]
yesNo True = "Yes"
yesNo False = "No"
File diff suppressed because one or more lines are too long
@@ -0,0 +1,115 @@
<svg width="440" height="520" viewBox="-20 0 440 520" fill="none" xmlns="http://www.w3.org/2000/svg">
<!-- Sender browser -->
<rect x="120" y="16" width="160" height="56" rx="10" stroke="#70F0F9" stroke-width="1.5"/>
<text x="200" y="40" text-anchor="middle" font-family="system-ui, sans-serif" font-size="13" font-weight="600" fill="#70F0F9">Sender's browser</text>
<text x="200" y="56" text-anchor="middle" font-family="system-ui, sans-serif" font-size="11" fill="rgba(112,240,249,0.7)">encrypts file</text>
<!-- Arrow down from sender to chunks -->
<line x1="200" y1="72" x2="200" y2="120" stroke="#70F0F9" stroke-width="1.5" marker-end="url(#arrowC)"/>
<!-- Chunks row -->
<rect x="112" y="120" width="176" height="40" rx="8" fill="none" stroke="#70F0F9" stroke-width="1" stroke-dasharray="4 3"/>
<text x="200" y="145" text-anchor="middle" font-family="system-ui, sans-serif" font-size="12" fill="#70F0F9">encrypted chunks</text>
<!-- Arrows from chunks to routers -->
<line x1="152" y1="160" x2="80" y2="220" stroke="#70F0F9" stroke-width="1.5" marker-end="url(#arrowC)"/>
<line x1="200" y1="160" x2="200" y2="220" stroke="#70F0F9" stroke-width="1.5" marker-end="url(#arrowC)"/>
<line x1="248" y1="160" x2="320" y2="220" stroke="#70F0F9" stroke-width="1.5" marker-end="url(#arrowC)"/>
<!-- Router 1 (SimpleX) -->
<rect x="20" y="220" width="120" height="56" rx="6" fill="none" stroke="#70F0F9" stroke-width="1.5"/>
<g transform="translate(28, 227)">
<rect width="14" height="4" rx="1" fill="rgba(112,240,249,0.5)"/>
<rect y="6" width="14" height="4" rx="1" fill="rgba(112,240,249,0.5)"/>
<rect y="12" width="14" height="4" rx="1" fill="rgba(112,240,249,0.5)"/>
<circle cx="11" cy="2" r="1" fill="#70F0F9"/>
<circle cx="11" cy="8" r="1" fill="#70F0F9"/>
<circle cx="11" cy="14" r="1" fill="#70F0F9"/>
</g>
<text x="80" y="244" text-anchor="middle" font-family="system-ui, sans-serif" font-size="11" font-weight="600" fill="#70F0F9">SimpleX</text>
<text x="80" y="258" text-anchor="middle" font-family="system-ui, sans-serif" font-size="9" fill="rgba(112,240,249,0.7)">XFTP router</text>
<!-- Router 2 (Flux) -->
<rect x="155" y="220" width="90" height="56" rx="6" fill="none" stroke="#70F0F9" stroke-width="1.5"/>
<g transform="translate(163, 227)">
<rect width="14" height="4" rx="1" fill="rgba(112,240,249,0.5)"/>
<rect y="6" width="14" height="4" rx="1" fill="rgba(112,240,249,0.5)"/>
<rect y="12" width="14" height="4" rx="1" fill="rgba(112,240,249,0.5)"/>
<circle cx="11" cy="2" r="1" fill="#70F0F9"/>
<circle cx="11" cy="8" r="1" fill="#70F0F9"/>
<circle cx="11" cy="14" r="1" fill="#70F0F9"/>
</g>
<text x="200" y="244" text-anchor="middle" font-family="system-ui, sans-serif" font-size="11" font-weight="600" fill="#70F0F9">Flux</text>
<text x="200" y="258" text-anchor="middle" font-family="system-ui, sans-serif" font-size="9" fill="rgba(112,240,249,0.7)">XFTP router</text>
<!-- Router 3 (SimpleX) -->
<rect x="260" y="220" width="120" height="56" rx="6" fill="none" stroke="#70F0F9" stroke-width="1.5"/>
<g transform="translate(268, 227)">
<rect width="14" height="4" rx="1" fill="rgba(112,240,249,0.5)"/>
<rect y="6" width="14" height="4" rx="1" fill="rgba(112,240,249,0.5)"/>
<rect y="12" width="14" height="4" rx="1" fill="rgba(112,240,249,0.5)"/>
<circle cx="11" cy="2" r="1" fill="#70F0F9"/>
<circle cx="11" cy="8" r="1" fill="#70F0F9"/>
<circle cx="11" cy="14" r="1" fill="#70F0F9"/>
</g>
<text x="320" y="244" text-anchor="middle" font-family="system-ui, sans-serif" font-size="11" font-weight="600" fill="#70F0F9">SimpleX</text>
<text x="320" y="258" text-anchor="middle" font-family="system-ui, sans-serif" font-size="9" fill="rgba(112,240,249,0.7)">XFTP router</text>
<!-- Arrows from routers down -->
<line x1="80" y1="276" x2="152" y2="336" stroke="#70F0F9" stroke-width="1.5" marker-end="url(#arrowC)"/>
<line x1="200" y1="276" x2="200" y2="336" stroke="#70F0F9" stroke-width="1.5" marker-end="url(#arrowC)"/>
<line x1="320" y1="276" x2="248" y2="336" stroke="#70F0F9" stroke-width="1.5" marker-end="url(#arrowC)"/>
<!-- Re-encrypt label -->
<text x="330" y="310" text-anchor="start" font-family="system-ui, sans-serif" font-size="10" fill="rgba(112,240,249,0.7)">re-encrypted</text>
<text x="330" y="322" text-anchor="start" font-family="system-ui, sans-serif" font-size="10" fill="rgba(112,240,249,0.7)">per recipient</text>
<!-- Chunks row (download) -->
<rect x="112" y="336" width="176" height="40" rx="8" fill="none" stroke="#70F0F9" stroke-width="1" stroke-dasharray="4 3"/>
<text x="200" y="361" text-anchor="middle" font-family="system-ui, sans-serif" font-size="12" fill="#70F0F9">encrypted chunks</text>
<!-- Arrow down to recipient -->
<line x1="200" y1="376" x2="200" y2="424" stroke="#70F0F9" stroke-width="1.5" marker-end="url(#arrowC)"/>
<!-- Recipient browser -->
<rect x="120" y="424" width="160" height="56" rx="10" stroke="#70F0F9" stroke-width="1.5"/>
<text x="200" y="448" text-anchor="middle" font-family="system-ui, sans-serif" font-size="13" font-weight="600" fill="#70F0F9">Recipient's browser</text>
<text x="200" y="464" text-anchor="middle" font-family="system-ui, sans-serif" font-size="11" fill="rgba(112,240,249,0.7)">decrypts file</text>
<!-- Key path (dashed, side) -->
<path d="M120 44 L8 44 L8 452 L120 452" stroke="#70F0F9" stroke-width="1.5" stroke-dasharray="6 4" fill="none" marker-end="url(#arrowC)"/>
<text x="-6" y="240" text-anchor="middle" font-family="system-ui, sans-serif" font-size="10" fill="#70F0F9" transform="rotate(-90 -6 240)">key in URL fragment - never sent to page server or data router</text>
<!-- Closed padlock: encryption (between sender and chunks) -->
<g transform="translate(192, 88)">
<path d="M4,7 V4 C4,1.2 12,1.2 12,4 V7" stroke="#60a5fa" stroke-width="1.5" fill="none" stroke-linecap="round"/>
<rect x="2" y="7" width="12" height="9" rx="2" fill="#60a5fa"/>
<circle cx="8" cy="12" r="1.2" fill="#0B2A59"/>
</g>
<!-- Open padlock: decryption (between chunks and recipient) -->
<g transform="translate(192, 392)">
<path d="M4,7 V4 C4,1.2 12,1.2 12,4 V2" stroke="#60a5fa" stroke-width="1.5" fill="none" stroke-linecap="round"/>
<rect x="2" y="7" width="12" height="9" rx="2" fill="#60a5fa"/>
<circle cx="8" cy="12" r="1.2" fill="#0B2A59"/>
</g>
<!-- Key icon on dashed line -->
<g transform="translate(8, 410)">
<circle cx="0" cy="0" r="6" stroke="#FBBF24" stroke-width="2" fill="#FBBF24"/>
<circle cx="0" cy="0" r="2" fill="#0B2A59"/>
<line x1="6" y1="0" x2="16" y2="0" stroke="#FBBF24" stroke-width="2"/>
<line x1="14" y1="0" x2="14" y2="4" stroke="#FBBF24" stroke-width="2"/>
<line x1="11" y1="0" x2="11" y2="3.5" stroke="#FBBF24" stroke-width="2"/>
</g>
<!-- Annotation: no shared IDs -->
<text x="200" y="510" text-anchor="middle" font-family="system-ui, sans-serif" font-size="10" fill="rgba(112,240,249,0.7)">Each file fragment uses unique anonymous credentials - no shared identifiers</text>
<defs>
<marker id="arrowC" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="6" markerHeight="6" orient="auto-start-reverse">
<path d="M 0 0 L 10 5 L 0 10 z" fill="#70F0F9"/>
</marker>
</defs>
</svg>

After

Width:  |  Height:  |  Size: 7.2 KiB

@@ -0,0 +1,130 @@
<svg width="440" height="520" viewBox="-20 0 440 520" fill="none" xmlns="http://www.w3.org/2000/svg">
<!-- Sender browser -->
<rect x="120" y="16" width="160" height="56" rx="10" fill="url(#gBox)" stroke="#606C71" stroke-width="1.5"/>
<text x="200" y="40" text-anchor="middle" font-family="system-ui, sans-serif" font-size="13" font-weight="600" fill="#fff">Sender's browser</text>
<text x="200" y="56" text-anchor="middle" font-family="system-ui, sans-serif" font-size="11" fill="rgba(255,255,255,0.8)">encrypts file</text>
<!-- Arrow down from sender to chunks -->
<line x1="200" y1="72" x2="200" y2="120" stroke="#606C71" stroke-width="1.5" marker-end="url(#arrowG)"/>
<!-- Chunks row -->
<rect x="112" y="120" width="176" height="40" rx="8" fill="#f0f7ff" stroke="#0053D0" stroke-width="1" stroke-dasharray="4 3"/>
<text x="200" y="145" text-anchor="middle" font-family="system-ui, sans-serif" font-size="12" fill="#0053D0">encrypted chunks</text>
<!-- Arrows from chunks to routers -->
<line x1="152" y1="160" x2="80" y2="220" stroke="#606C71" stroke-width="1.5" marker-end="url(#arrowG)"/>
<line x1="200" y1="160" x2="200" y2="220" stroke="#606C71" stroke-width="1.5" marker-end="url(#arrowG)"/>
<line x1="248" y1="160" x2="320" y2="220" stroke="#606C71" stroke-width="1.5" marker-end="url(#arrowG)"/>
<!-- Router 1 (SimpleX) -->
<rect x="20" y="220" width="120" height="56" rx="6" fill="#f0f4f8" stroke="#606C71" stroke-width="1.5"/>
<g transform="translate(28, 227)">
<rect width="14" height="4" rx="1" fill="#606C71"/>
<rect y="6" width="14" height="4" rx="1" fill="#606C71"/>
<rect y="12" width="14" height="4" rx="1" fill="#606C71"/>
<circle cx="11" cy="2" r="1" fill="#53C1FF"/>
<circle cx="11" cy="8" r="1" fill="#53C1FF"/>
<circle cx="11" cy="14" r="1" fill="#53C1FF"/>
</g>
<text x="80" y="244" text-anchor="middle" font-family="system-ui, sans-serif" font-size="11" font-weight="600" fill="#3F484B">SimpleX</text>
<text x="80" y="258" text-anchor="middle" font-family="system-ui, sans-serif" font-size="9" fill="#606C71">XFTP router</text>
<!-- Router 2 (Flux) -->
<rect x="155" y="220" width="90" height="56" rx="6" fill="#f0f4f8" stroke="#606C71" stroke-width="1.5"/>
<g transform="translate(163, 227)">
<rect width="14" height="4" rx="1" fill="#606C71"/>
<rect y="6" width="14" height="4" rx="1" fill="#606C71"/>
<rect y="12" width="14" height="4" rx="1" fill="#606C71"/>
<circle cx="11" cy="2" r="1" fill="#53C1FF"/>
<circle cx="11" cy="8" r="1" fill="#53C1FF"/>
<circle cx="11" cy="14" r="1" fill="#53C1FF"/>
</g>
<text x="200" y="244" text-anchor="middle" font-family="system-ui, sans-serif" font-size="11" font-weight="600" fill="#3F484B">Flux</text>
<text x="200" y="258" text-anchor="middle" font-family="system-ui, sans-serif" font-size="9" fill="#606C71">XFTP router</text>
<!-- Router 3 (SimpleX) -->
<rect x="260" y="220" width="120" height="56" rx="6" fill="#f0f4f8" stroke="#606C71" stroke-width="1.5"/>
<g transform="translate(268, 227)">
<rect width="14" height="4" rx="1" fill="#606C71"/>
<rect y="6" width="14" height="4" rx="1" fill="#606C71"/>
<rect y="12" width="14" height="4" rx="1" fill="#606C71"/>
<circle cx="11" cy="2" r="1" fill="#53C1FF"/>
<circle cx="11" cy="8" r="1" fill="#53C1FF"/>
<circle cx="11" cy="14" r="1" fill="#53C1FF"/>
</g>
<text x="320" y="244" text-anchor="middle" font-family="system-ui, sans-serif" font-size="11" font-weight="600" fill="#3F484B">SimpleX</text>
<text x="320" y="258" text-anchor="middle" font-family="system-ui, sans-serif" font-size="9" fill="#606C71">XFTP router</text>
<!-- Arrows from routers down -->
<line x1="80" y1="276" x2="152" y2="336" stroke="#606C71" stroke-width="1.5" marker-end="url(#arrowG)"/>
<line x1="200" y1="276" x2="200" y2="336" stroke="#606C71" stroke-width="1.5" marker-end="url(#arrowG)"/>
<line x1="320" y1="276" x2="248" y2="336" stroke="#606C71" stroke-width="1.5" marker-end="url(#arrowG)"/>
<!-- Re-encrypt label -->
<text x="330" y="310" text-anchor="start" font-family="system-ui, sans-serif" font-size="10" fill="#606C71">re-encrypted</text>
<text x="330" y="322" text-anchor="start" font-family="system-ui, sans-serif" font-size="10" fill="#606C71">per recipient</text>
<!-- Chunks row (download) -->
<rect x="112" y="336" width="176" height="40" rx="8" fill="#f0f7ff" stroke="#0053D0" stroke-width="1" stroke-dasharray="4 3"/>
<text x="200" y="361" text-anchor="middle" font-family="system-ui, sans-serif" font-size="12" fill="#0053D0">encrypted chunks</text>
<!-- Arrow down to recipient -->
<line x1="200" y1="376" x2="200" y2="424" stroke="#606C71" stroke-width="1.5" marker-end="url(#arrowG)"/>
<!-- Recipient browser -->
<rect x="120" y="424" width="160" height="56" rx="10" fill="url(#gBox)" stroke="#606C71" stroke-width="1.5"/>
<text x="200" y="448" text-anchor="middle" font-family="system-ui, sans-serif" font-size="13" font-weight="600" fill="#fff">Recipient's browser</text>
<text x="200" y="464" text-anchor="middle" font-family="system-ui, sans-serif" font-size="11" fill="rgba(255,255,255,0.8)">decrypts file</text>
<!-- Key path (dashed, side) -->
<path d="M120 44 L8 44 L8 452 L120 452" stroke="#0053D0" stroke-width="1.5" stroke-dasharray="6 4" fill="none" marker-end="url(#arrowB)"/>
<text x="-6" y="240" text-anchor="middle" font-family="system-ui, sans-serif" font-size="10" fill="#0053D0" transform="rotate(-90 -6 240)">key in URL fragment - never sent to page server or data router</text>
<!-- Closed padlock: encryption (between sender and chunks) -->
<g transform="translate(192, 88)">
<path d="M4,7 V4 C4,1.2 12,1.2 12,4 V7" stroke="#0053D0" stroke-width="1.5" fill="none" stroke-linecap="round"/>
<rect x="2" y="7" width="12" height="9" rx="2" fill="#0053D0"/>
<circle cx="8" cy="12" r="1.2" fill="#fff"/>
</g>
<!-- Open padlock: decryption (between chunks and recipient) -->
<g transform="translate(192, 392)">
<path d="M4,7 V4 C4,1.2 12,1.2 12,4 V2" stroke="#0053D0" stroke-width="1.5" fill="none" stroke-linecap="round"/>
<rect x="2" y="7" width="12" height="9" rx="2" fill="#0053D0"/>
<circle cx="8" cy="12" r="1.2" fill="#fff"/>
</g>
<!-- Key icon on dashed line -->
<g transform="translate(8, 410)">
<circle cx="0" cy="0" r="6" stroke="#D97706" stroke-width="2" fill="#D97706"/>
<circle cx="0" cy="0" r="2" fill="#fff"/>
<line x1="6" y1="0" x2="16" y2="0" stroke="#D97706" stroke-width="2"/>
<line x1="14" y1="0" x2="14" y2="4" stroke="#D97706" stroke-width="2"/>
<line x1="11" y1="0" x2="11" y2="3.5" stroke="#D97706" stroke-width="2"/>
</g>
<!-- Annotation: no shared IDs -->
<text x="200" y="510" text-anchor="middle" font-family="system-ui, sans-serif" font-size="10" fill="#606C71">Each file fragment uses unique anonymous credentials - no shared identifiers</text>
<defs>
<linearGradient id="gBox" x1="120" y1="16" x2="280" y2="72" gradientUnits="userSpaceOnUse">
<stop stop-color="#0053D0"/>
<stop offset="1" stop-color="#53C1FF"/>
</linearGradient>
<linearGradient id="gSrv1" x1="20" y1="220" x2="140" y2="276" gradientUnits="userSpaceOnUse">
<stop stop-color="#0053D0"/>
<stop offset="1" stop-color="#53C1FF"/>
</linearGradient>
<linearGradient id="gSrv2" x1="155" y1="220" x2="245" y2="276" gradientUnits="userSpaceOnUse">
<stop stop-color="#0053D0"/>
<stop offset="1" stop-color="#53C1FF"/>
</linearGradient>
<marker id="arrowG" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="6" markerHeight="6" orient="auto-start-reverse">
<path d="M 0 0 L 10 5 L 0 10 z" fill="#606C71"/>
</marker>
<marker id="arrowB" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="6" markerHeight="6" orient="auto-start-reverse">
<path d="M 0 0 L 10 5 L 0 10 z" fill="#0053D0"/>
</marker>
</defs>
</svg>

After

Width:  |  Height:  |  Size: 7.8 KiB

File diff suppressed because one or more lines are too long
@@ -0,0 +1,145 @@
#app, [data-xftp-app] {
font-family: system-ui, -apple-system, sans-serif;
color: #333;
width: 100%;
max-width: 480px;
padding: 16px;
box-sizing: border-box;
--xftp-ring-fg: #3b82f6;
}
:is(#app, [data-xftp-app]) .card {
background: #fff;
border-radius: 12px;
padding: 32px 24px;
box-shadow: 0 1px 3px rgba(0,0,0,.1);
text-align: center;
}
:is(#app, [data-xftp-app]) h1 {
font-size: 1.25rem;
font-weight: 600;
margin-bottom: 24px;
}
:is(#app, [data-xftp-app]) .stage { margin-top: 16px; }
/* Drop zone */
:is(#app, [data-xftp-app]) .drop-zone {
border: 2px dashed #ccc;
border-radius: 8px;
padding: 32px 16px;
transition: border-color .15s, background .15s;
}
:is(#app, [data-xftp-app]) .drop-zone.drag-over {
border-color: #3b82f6;
background: #eff6ff;
}
/* Buttons */
:is(#app, [data-xftp-app]) .btn {
display: inline-block;
padding: 10px 24px;
border: none;
border-radius: 6px;
background: #3b82f6;
color: #fff;
font-size: .9rem;
font-weight: 500;
cursor: pointer;
transition: background .15s;
}
:is(#app, [data-xftp-app]) .btn:hover { background: #2563eb; }
:is(#app, [data-xftp-app]) .btn-secondary { background: #6b7280; }
:is(#app, [data-xftp-app]) .btn-secondary:hover { background: #4b5563; }
/* Hints */
:is(#app, [data-xftp-app]) .hint { color: #999; font-size: .85rem; margin-top: 8px; }
:is(#app, [data-xftp-app]) .expiry { margin-top: 12px; }
/* Progress */
:is(#app, [data-xftp-app]) .progress-ring { display: block; margin: 0 auto 12px; }
:is(#app, [data-xftp-app]) #upload-status,
:is(#app, [data-xftp-app]) #dl-status { font-size: .9rem; color: #666; margin-bottom: 12px; }
/* Share link row */
:is(#app, [data-xftp-app]) .link-row {
display: flex;
gap: 8px;
margin-top: 12px;
}
:is(#app, [data-xftp-app]) .link-row input {
flex: 1;
padding: 8px 10px;
border: 1px solid #ccc;
border-radius: 6px;
font-size: .85rem;
background: #f9fafb;
}
/* Upload link */
:is(#app, [data-xftp-app]) .upload-link {
margin-top: 12px;
color: #3b82f6;
font-size: .9rem;
text-decoration: none;
cursor: pointer;
}
:is(#app, [data-xftp-app]) .upload-link:not([hidden]) {
display: inline-block;
}
:is(#app, [data-xftp-app]) .upload-link:hover { text-decoration: underline; }
/* Messages */
:is(#app, [data-xftp-app]) .success { color: #16a34a; font-weight: 600; }
:is(#app, [data-xftp-app]) .error { color: #dc2626; font-weight: 500; margin-bottom: 12px; }
/* Security note */
:is(#app, [data-xftp-app]) .security-note {
margin-top: 20px;
padding: 12px;
background: #f0fdf4;
border-radius: 6px;
font-size: .8rem;
color: #555;
text-align: left;
}
:is(#app, [data-xftp-app]) .security-note p + p { margin-top: 6px; }
:is(#app, [data-xftp-app]) .security-note a { color: #3b82f6; text-decoration: none; }
:is(#app, [data-xftp-app]) .security-note a:hover { text-decoration: underline; }
/* ── Dark mode ─────────────────────────────────── */
.dark :is(#app, [data-xftp-app]) {
color: #e5e7eb;
--xftp-ring-bg: #374151;
--xftp-ring-fg: #60a5fa;
--xftp-ring-text: #e5e7eb;
--xftp-ring-done: #4ade80;
}
.dark :is(#app, [data-xftp-app]) .card {
background: #1f2937;
box-shadow: 0 1px 3px rgba(0,0,0,.4);
}
.dark :is(#app, [data-xftp-app]) .drop-zone { border-color: #4b5563; }
.dark :is(#app, [data-xftp-app]) .drop-zone.drag-over {
border-color: #60a5fa;
background: rgba(59,130,246,.15);
}
.dark :is(#app, [data-xftp-app]) .btn-secondary { background: #4b5563; }
.dark :is(#app, [data-xftp-app]) .btn-secondary:hover { background: #374151; }
.dark :is(#app, [data-xftp-app]) .hint { color: #9ca3af; }
.dark :is(#app, [data-xftp-app]) #upload-status,
.dark :is(#app, [data-xftp-app]) #dl-status { color: #9ca3af; }
.dark :is(#app, [data-xftp-app]) .link-row input {
background: #374151;
border-color: #4b5563;
color: #e5e7eb;
}
.dark :is(#app, [data-xftp-app]) .success { color: #4ade80; }
.dark :is(#app, [data-xftp-app]) .error { color: #f87171; }
.dark :is(#app, [data-xftp-app]) .security-note {
background: rgba(34,197,94,.1);
color: #d1d5db;
}
.dark :is(#app, [data-xftp-app]) .upload-link { color: #60a5fa; }
.dark :is(#app, [data-xftp-app]) .security-note a { color: #60a5fa; }
File diff suppressed because one or more lines are too long
+104
View File
@@ -0,0 +1,104 @@
# Coding and building
This file provides guidance on coding style and approaches and on building the code.
## Code Security
When designing code and planning implementations:
- Apply adversarial thinking, and consider what may happen if one of the communicating parties is malicious.
- Formulate an explicit threat model for each change - who can do which undesirable things and under which circumstances.
## Code Quality Standards
Haskell client and server code serves as system specification, not just implementation — we use type-driven design to reflect the business domain in types. Quality, conciseness, and clarity of Haskell code are critical.
## Code Style, Formatting and Approaches
The project uses **fourmolu** for Haskell code formatting. Configuration is in `fourmolu.yaml`.
**Key formatting rules:**
- 2-space indentation
- Trailing function arrows, commas, and import/export style
- Record brace without space: `{field = value}`
- Single newline between declarations
- Never use unicode symbols
- Inline `let` style with right-aligned `in`
**Format code before committing:**
```bash
# Format a single file
fourmolu -i src/Simplex/Messaging/Protocol.hs
```
Some files that use CPP language extension cannot be formatted as a whole, so individual code fragments need to be formatted.
**Follow existing code patterns:**
- Match the style of surrounding code
- Use qualified imports with short aliases (e.g., `import qualified Data.ByteString.Char8 as B`)
- Use record syntax for types with multiple fields
- Prefer explicit pattern matching over partial functions
**Comments policy:**
- Avoid redundant comments that restate what the code already says
- Only comment on non-obvious design decisions or tricky implementation details
- Function names and type signatures should be self-documenting
- Do not add comments like "wire format encoding" (Encoding class is always wire format) or "check if X" when the function name already says that
- Assume a competent Haskell reader
**Diff and refactoring:**
- Avoid unnecessary changes and code movements
- Never do refactoring unless it substantially reduces cost of solving the current problem, including the cost of refactoring
- Aim to minimize the code changes - do what is minimally required to solve users' problems
**Document and code structure:**
- **Never move existing code or sections around** - add new content at appropriate locations without reorganizing existing structure.
- When adding new sections to documents, continue the existing numbering scheme.
- Minimize diff size - prefer small, targeted changes over reorganization.
**Code analysis and review:**
- Trace data flows end-to-end: from origin, through storage/parameters, to consumption. Flag values that are discarded and reconstructed from partial data (e.g. extracted from a URI missing original fields) — this is usually a bug.
- Read implementations of called functions, not just signatures — if duplication involves a called function, check whether decomposing it resolves the duplication.
- Do not save time on analysis. Read every function in the data flow even when the interface seems clear — wrong assumptions about internals are the main source of missed bugs.
### Haskell Extensions
- `StrictData` enabled by default
- Use STM for safe concurrency
- Assume concurrency in PostgreSQL queries
- Comprehensive warning flags with strict pattern matching
## Build Commands
```bash
# Standard build
cabal build
# Fast build
cabal build --ghc-options -O0
# Build specific executables
cabal build exe:smp-server exe:xftp-server exe:ntf-server exe:xftp
# Build with PostgreSQL server support
cabal build -fserver_postgres
# Client-only library build (no server code)
cabal build -fclient_library
# Find binary location
cabal list-bin exe:smp-server
```
### Cabal Flags
- `swift`: Enable Swift JSON format
- `client_library`: Build without server code
- `client_postgres`: Use PostgreSQL instead of SQLite for agent persistence
- `server_postgres`: PostgreSQL support for server queue/notification store
## External Dependencies
Custom forks specified in `cabal.project`:
- `aeson`, `hs-socks` (SimpleX forks)
- `direct-sqlcipher`, `sqlcipher-simple` (encrypted SQLite)
- `warp`, `warp-tls` (HTTP server)
+105
View File
@@ -0,0 +1,105 @@
# SimpleXMQ repository
This file provides guidance on the project structure to help working with code in this repository.
## Project Overview
SimpleXMQ is a Haskell message broker implementing unidirectional (simplex) queues for privacy-preserving messaging.
Key components:
- **SimpleX Messaging Protocol**: SMP protocol definition and encodings ([code](../src/Simplex/Messaging/Protocol.hs), [transport code](../src/Simplex/Messaging/Transport.hs), [spec](../protocol/simplex-messaging.md)).
- **SMP Server**: Message broker with TLS, in-memory queues, optional persistence ([main code](../src/Simplex/Messaging/Server.hs), [all code files](../src/Simplex/Messaging/Server/), [executable](../apps/smp-server/)). For proxying SMP commands the server uses [lightweight SMP client](../src/Simplex/Messaging/Client/Agent.hs).
- **SMP Client**: Functional API with STM-based message delivery ([code](../src/Simplex/Messaging/Client.hs)).
- **SMP Agent**: High-level duplex connections via multiple simplex queues with E2E encryption ([code](../src/Simplex/Messaging/Agent.hs)). Implements Agent-to-agent protocol ([code](../src/Simplex/Messaging/Agent/Protocol.hs), [spec](../protocol/agent-protocol.md)) via intermediary agent client ([code](../src/Simplex/Messaging/Agent/Client.hs)).
- **XFTP**: SimpleX File Transfer Protocol, server and CLI client ([code](../src/Simplex/FileTransfer/), [spec](../protocol/xftp.md)).
- **XRCP**: SimpleX Remote Control Protocol ([code](`../src/Simplex/RemoteControl/`), [spec](../protocol/xrcp.md)).
- **Notifications**: Push notifications server requires PostgreSQL ([code](../src/Simplex/Messaging/Notifications), [executable](../apps/ntf-server/)). Client protocol is used for clients to communicate with the server ([code](../src/Simplex/Messaging/Notifications/Protocol.hs), [spec](../protocol/push-notifications.md)). For subscribing to SMP notifications the server uses [lightweight SMP client](../src/Simplex/Messaging/Client/Agent.hs).
## Architecture
For general overview see `../protocol/overview-tjr.md`.
SMP Protocol Layers:
```
TLS Transport → SMP Protocol → Agent Protocol → Application protocol
```
XFTP Protocol Layers:
```
TLS Transport (HTTP2 encoding) → XFTP Protocol → Out-of-band file descriptions
```
## Key Patterns
1. **Persistence**: All queue state managed via Software Transactional Memory or via PostgreSQL
- `Simplex.Messaging.Server.MsgStore.STM` - in-memory messages
- `Simplex.Messaging.Server.QueueStore.STM` - in-memory queue state
- `Simplex.Messaging.Server.MsgStore.Postgres` - message storage
- `Simplex.Messaging.Server.QueueStore.Postgres` - queue storage
2. **Append-Only Store Log**: Optional persistence via journal for in-memory storage
- `Simplex.Messaging.Server.StoreLog` - queue creation log
- Compacted on restart
3. **Agent Storage**:
- SQLite (default) or PostgreSQL
- Migrations in `src/Simplex/Messaging/Agent/Store/{SQLite,Postgres}/Migrations/`
4. **Protocol Versioning**: All layers support version negotiation
- `Simplex.Messaging.Version` - version range utilities
5. **Double Ratchet E2E**: Per-connection encryption
- `Simplex.Messaging.Crypto.Ratchet`
- SNTRUP761 post-quantum KEM (`src/Simplex/Messaging/Crypto/SNTRUP761/`)
## Source Layout
```
src/Simplex/
├── Messaging/
│ ├── Agent.hs # Main agent (~210KB)
│ ├── Server.hs # SMP server (~130KB)
│ ├── Client.hs # Client API (~65KB)
│ ├── Protocol.hs # Protocol types (~77KB)
│ ├── Crypto.hs # E2E encryption (~52KB)
│ ├── Transport.hs # Transport encoding over TLS
│ ├── Agent/Store/ # SQLite/Postgres persistence
│ ├── Server/ # Server internals (QueueStore, MsgStore, Control)
│ └── Notifications/ # Push notification system
├── FileTransfer/ # XFTP implementation for file transfers
└── RemoteControl/ # XRCP implementation for device discovery & control
```
## Protocol Documentation
- `protocol/overview-tjr.md`: SMP protocols stack overview
- `protocol/simplex-messaging.md`: SMP protocol spec (v19)
- `protocol/agent-protocol.md`: Agent protocol spec (v7)
- `protocol/xftp.md`: File transfer protocol
- `protocol/xrcp.md`: Remote control protocol
- `rfcs/`: Design RFCs for features
## Testing
```bash
# Run all tests
cabal test --test-show-details=streaming
# Run specific test group (uses HSpec)
cabal test --test-option=--match="/Core tests/Encryption tests/"
# Run single test
cabal test --test-option=--match="/SMP client agent/functional API/"
```
Tests require PostgreSQL running on `localhost:5432` when using `-fserver_postgres` or `-fclient_postgres`.
Test files are in `tests/` with structure:
- `Test.hs`: Main runner
- `AgentTests/`: Agent protocol and connection tests
- `CoreTests/`: Crypto, encoding, storage tests
- `ServerTests.hs`: SMP server tests
- `XFTPServerTests.hs`: File transfer tests
+23
View File
@@ -0,0 +1,23 @@
# Contributing to SimpleX repositories
## Focus on user problems
We do not make code changes to improve code - any change must address a specific user problem or request.
## Discuss the plans as early as possible
Please discuss the problem you want to solve and your detailed implementation plan with the project team prior to contributing, to avoid wasted time and additional changes. Acceptance of your contribution depends on your willingness and ability to iterate the proposed contribution to achieve the required quality level, coding style, test coverage, and alignment with user requirements as they are understood by the project team.
## Follow project structure, coding style and approaches
./PROJECT.md has information about the structure of this `simplexmq` repository.
./CODE.md has details about general requirements common for `simplexmq` and `simplex-chat` repositories.
This files can be used with LLM prompts, e.g. if you use Claude Code you can create CLAUDE.md file in project root importing content from these files:
```markdown
@README.md
@contributing/PROJECT.md
@contributing/CODE.md
```
-23
View File
@@ -1,23 +0,0 @@
common:
corrId - random BS, used as CbNonce
entityId - p2r tlsUniq
# setup
s->p: "proxy", uri, auth?
# unless connected
p->r: "p_handshake"
p<-r: "r_key", tls-signed dh pub
s<-r: "r_key", tls-signed dh pub # reply entityId contains tlsUniq
# working
s ; generate random dh priv, make shared secret
s->p: s2r("forward", random dh pub, SEND command blob)
p->r: p2r("forward", random dh pub, s2r("forward", ...)))
r->c@ "msg", ...
p<-r: p2r("r_res", s2r("ok" / "error", error))
s<-p@ s2r("ok" / "error", error)
# expired
p<-r@ p2r("error", "key expired")
s<-p@ "error", "key expired"
s ; reconnect
@@ -1,8 +1,9 @@
# SMP server message storage
# SMP router message storage
## Problem
Currently SMP servers store all queues in server memory. As the traffic grows, so does the number of undelivered messages. What is worse, Haskell is not avoiding heap fragmentation when messages are allocated and then de-allocated - undelivered messages use ByteString and GC cannot move them around, as they use pinned memory.
Currently SMP routers store all queues in router memory. As the traffic grows, so does the number of undelivered messages. What is worse, Haskell is not avoiding heap fragmentation when messages are allocated and then de-allocated - undelivered messages use ByteString and GC cannot move them around, as they use pinned memory.
## Possible solutions
@@ -10,7 +11,7 @@ Currently SMP servers store all queues in server memory. As the traffic grows, s
Move from ByteString to some other primitive to store messages in memory long term, e.g. ShortByteString, or manage allocation/de-allocation of stored messages manually in some other way.
Pros: the simplest solution that avoids substantial re-engineering of the server.
Pros: the simplest solution that avoids substantial re-engineering of the router.
Cons:
- not a long term solution, as memory growth still has limits.
@@ -22,12 +23,12 @@ Use files or RocksDB to store messages.
Pros:
- much lower memory usage.
- no message loss in case of abnormal server termination (important until clients have delivery redundancy).
- no message loss in case of abnormal router termination (important until clients have delivery redundancy).
- this is a long term solution, and at some point it might need to be done anyway.
Cons:
- substantial re-engineering costs and risks.
- metadata privacy. Currently we only save undelivered messages when server is restarted, with this approach all messages will be stored for some time. this argument is limited, as hosting providers of VMs can make memory snapshots too, on the other hand they are harder to analyze than files. On another hand, with this approach messages will be stored for a shorter time.
- metadata privacy. Currently we only save undelivered messages when router is restarted, with this approach all messages will be stored for some time. this argument is limited, as hosting providers of VMs can make memory snapshots too, on the other hand they are harder to analyze than files. On another hand, with this approach messages will be stored for a shorter time.
#### RocksDB and other key-value stores
@@ -67,7 +68,7 @@ queueLogLine =
%s"write_msg=" digits
```
When queue is first requested by the server:
When queue is first requested by the router:
```c
if queue folder exists:
@@ -87,7 +88,7 @@ nextReadMsg = read_msg
open write_file in AppendMode
```
When message is added to the queue (assumes that queue state is loaded to server memory, if not the previous section will be done first):
When message is added to the queue (assumes that queue state is loaded to router memory, if not the previous section will be done first):
```c
if write_msg > max_queue_messages:
@@ -128,7 +129,7 @@ else
nextReadByte = current position in file
```
When message delivery is acknowledged, the read queue needs to be advanced, and possibly switched to read from the current write_queue:
When message delivery is acknowledged, the read queue needs to be advanced, and possibly switched to read from the current write queue:
```c
if nextReadByte == read_byte:
@@ -162,9 +163,9 @@ Most Linux systems use EXT4 filesystem where the file lookup time scales linearl
So storing all queue folders in one folder won't scale.
To solve this problem we could use recipient queue ID in base64url format not as a folder name, but as a folder path, splitting it to path fragments of some length. The number of fragments can be configurable and migration to a different fragment size can be supported as the number of queues on a given server grows.
To solve this problem we could use recipient queue ID in base64url format not as a folder name, but as a folder path, splitting it to path fragments of some length. The number of fragments can be configurable and migration to a different fragment size can be supported as the number of queues on a given router grows.
Currently, queue ID is 24 bytes random number, thus allowing 2^192 possible queue IDs. If we assume that a server must hold 1b queues, it means that we have ~2^162 possible addresses for each existing queue. 24 bytes in base64 is 32 characters that can be split into say 8 fragments with 4 characters each, so that queue folder path for queue with ID `abcdefghijklmnopqrstuvwxyz012345` would be:
Currently, queue ID is 24 bytes random number, thus allowing 2^192 possible queue IDs. If we assume that a router must hold 1b queues, it means that we have ~2^162 possible addresses for each existing queue. 24 bytes in base64 is 32 characters that can be split into say 8 fragments with 4 characters each, so that queue folder path for queue with ID `abcdefghijklmnopqrstuvwxyz012345` would be:
`/var/opt/simplex/messages/abcd/efgh/ijkl/mnop/qrst/uvwx/yz01/2345`
@@ -174,6 +175,6 @@ So we could use an unequal split of path, two letters each and the last being lo
`/var/opt/simplex/messages/ab/cd/ef/ghijklmnopqrstuvwxyz012345`
The first three levels in this case can have 4096 subfolders each, and it gives 68b possible subfolders (64^2^3), so the last level will be sparse in case of 1b queues on the server. So we could make it 4 levels with 2 letters to never think about it, accounting for a large variance of the random numbers distribution:
The first three levels in this case can have 4096 subfolders each, and it gives 68b possible subfolders (64^2^3), so the last level will be sparse in case of 1b queues on the router. So we could make it 4 levels with 2 letters to never think about it, accounting for a large variance of the random numbers distribution:
`/var/opt/simplex/messages/ab/cd/ef/gh/ijklmnopqrstuvwxyz012345`
@@ -1,6 +1,7 @@
# Sharing protocol ports with HTTPS
Some networks block all ports other than web ports, including port 5223 used for SMP protocol by default. Running SMP servers on a common web port 443 would allow them to work on more networks. The servers would need to provide an HTTPS page for browsers (and probes).
Some networks block all ports other than web ports, including port 5223 used for SMP protocol by default. Running SMP routers on a common web port 443 would allow them to work on more networks. The routers would need to provide an HTTPS page for browsers (and probes).
## Problem
@@ -8,7 +9,7 @@ Browsers and tools rely on system CA bundles instead of certificate pinning.
The crypto parameters used by HTTPS are different from what the protocols use.
Public certificate providers like LetsEncrypt can only sign specific types of keys and Ed25519 isn't one of them.
This means a server should distinguish browser and protocol clients and adjust its behavior to match.
This means a router should distinguish browser and protocol clients and adjust its behavior to match.
## Solution
@@ -16,15 +17,15 @@ This means a server should distinguish browser and protocol clients and adjust i
Since LE certificates are only handed out to domain names, TLS client will be sending the SNI.
However client transports are constructed over connected sockets and the SNI wouldn't be present unless explicitly requested.
When a client sends SNI, then it's a browser and a web credentials should be used.
When a client sends SNI, then it's a browser and web credentials should be used.
Otherwise it's a protocol client to be offered the self-signed ca, cert and key.
When a transport colocated with a HTTPS, its ALPN list should be extended with `h2 http/1.1`.
The browsers will send it, and it should be checked before running transport client.
If HTTP ALPN is detected, then the client connection is served with HTTP `Application` instead (the same "server information" page).
If HTTP ALPN is detected, then the client connection is served with HTTP `Application` instead (the same "router information" page).
If some client connects to server IP, doesn't send SNI and doesn't send ALPN, it will look like a pre-handshake client.
In that case a server will send its handshake first.
If some client connects to router IP, doesn't send SNI and doesn't send ALPN, it will look like a pre-handshake client.
In that case a router will send its handshake first.
This can be mitigated by delaying its handshake and letting the probe to issue its HTTP request.
## Implementation plan
@@ -43,7 +44,7 @@ runServer (tcpPort, ATransport t) = do
else runClient serverSignKey t h `runReaderT` env -- performs serverHandshake etc as usual
```
The web app and server live outside, so `runHttp` has to be provided by the `runSMPServer` caller.
The web app and router live outside, so `runHttp` has to be provided by the `runSMPServer` caller.
Additonally, Warp is using its `InternalInfo` object that's scoped to `withII` bracket.
```haskell
@@ -65,11 +66,9 @@ The implementation relies on a few modification to upstream code:
- `warp`: Only the re-export of `serveConnection` is needed.
Unfortunately the most recent `warp` version can't be used right away due to dependency cascade around `http-5` and `auto-update-2`.
So a fork containing the backported re-export has to be used until the dependencies are refreshed.
### TLS.ServerParams
When a server has port sharing enabled, a new set of TLS params is loaded and combined with transport params:
When a router has port sharing enabled, a new set of TLS params is loaded and combined with transport params:
```haskell
newEnv config = do
@@ -129,7 +128,7 @@ key: /etc/opt/simplex/web.key
# key: /etc/letsencrypt/live/smp.hostname.tld/privkey.pem
```
When `TRANSPORT.port` matches `WEB.https` the transport server becomes shared.
When `TRANSPORT.port` matches `WEB.https` the transport router becomes shared.
Perhaps a more desirable option would be explicit configuration resulting in additional transported to run:
@@ -148,16 +147,16 @@ key: /etc/opt/simplex/web.key
## Caveats
Serving static files and the protocols togother may pose a problem for those who currently use dedicated web servers as they should switch to embedded http handlers.
Serving static files and the protocols together may pose a problem for those who currently use dedicated web servers as they should switch to embedded http handlers.
As before, using embedded HTTP server is increasing attack surface.
Users who want to run everything on a single host will have to add and extra IP address and bind servers to specific IPs instead of 0.0.0.0.
An amalgamated server binary can be provided that would contain both SMP and XFTP servers, where transport will dispatch connections by handshake ALPN.
Users who want to run everything on a single host will have to add an extra IP address and bind routers to specific IPs instead of 0.0.0.0.
An amalgamated router binary can be provided that would contain both SMP and XFTP routers, where transport will dispatch connections by handshake ALPN.
## Alternative: Use transports routable with reverse-proxies
An "industrial" reverse proxy may do the ALPN routing, serving HTTP by itself and delegating `smp` and `xftp` to protocol servers.
Same with the `websockets`.
Since this in effect does TLS termination, the protocol servers will have to rely on credentials from protocol handshakes.
Since this in effect does TLS termination, the protocol routers will have to rely on credentials from protocol handshakes.
@@ -1,8 +1,9 @@
# Expiring messages in journal storage
## Problem
The journal storage servers recently migrated to do not delete delivered or expired messages, they only update pointers to journal file lines. The messages are actually deleted when the whole journal file is deleted (when fully deleted or fully expired).
The journal storage routers recently migrated to do not delete delivered or expired messages, they only update pointers to journal file lines. The messages are actually deleted when the whole journal file is deleted (when fully deleted or fully expired).
The problem is that in case the queue stops receiving the new messages then writing of messages won't switch to the new journal file, and the current journal file containing delivered or expired messages would never be deleted.
+416
View File
@@ -0,0 +1,416 @@
# Fix subQ deadlock: blocking writeTBQueue inside connLock
## Problem
Users report that message reception silently and permanently stops across all connections, with no error alerts. The app appears functional but no messages arrive. Recovery requires restart.
Root cause: a deadlock between worker threads holding `connLock` and the `agentSubscriber` (sole `subQ` reader).
### The deadlock mechanism
`subQ` (`TBQueue ATransmission`, capacity 4096 on mobile / 1024 on desktop) is the single pipeline between the agent layer and the chat layer. The `agentSubscriber` thread (`Commands.hs:4373`) is its **sole reader**.
Three code sites hold `connLock` and call blocking `writeTBQueue subQ` without a fullness check. When `subQ` is full, these block while holding the lock. If `agentSubscriber` simultaneously needs the same `connLock` (via `sendMessagesB_``withConnLocks`), it blocks too — creating a circular wait:
- **Worker**: holds `connLock(X)`, waits for `subQ` space (needs `agentSubscriber` to read)
- **agentSubscriber**: sole `subQ` reader, waits for `connLock(X)` (needs worker to release)
- **Result**: permanent silent deadlock — no exception, no alert, all connections blocked
### Confirmed deadlock scenarios
**Scenario 1**: Delivery worker during queue rotation test
```
Delivery worker: agentSubscriber (sole subQ reader):
withConnLock(X) [2187] readTBQueue subQ → processAgentMessageConn
...DB operations... → sendPendingGroupMessages (on CON/SENT/QCONT)
notify → writeTBQueue subQ [2238] → batchSendConnMessages → deliverMessagesB
[BLOCKED — subQ full] → withAgent sendMessagesB [synchronous]
→ sendMessagesB_ → withConnLocks({..X..}) [1708]
[BLOCKED — connLock(X) held]
```
**Scenario 2**: Async command worker during message ACK with notification
```
Async cmd worker: agentSubscriber (sole subQ reader):
tryWithLock "ICAck" [1930→1824] readTBQueue subQ → processAgentMessageConn
→ withConnLock(X) → sendPendingGroupMessages
→ ack → ackQueueMessage [1899] → sendMessagesB_ → withConnLocks({..X..})
→ sendMsgNtf [2381] [BLOCKED — connLock(X) held]
→ writeTBQueue subQ [2386]
[BLOCKED — subQ full]
```
**Scenario 3**: Synchronous `ackMessage'` API (same mechanism as Scenario 2 but from external API caller)
```
ackMessage' caller: agentSubscriber (sole subQ reader):
withConnLock(X) [2254] → sendMessagesB_ → withConnLocks({..X..})
→ ack → ackQueueMessage [2267] [BLOCKED — connLock(X) held]
→ sendMsgNtf [2381]
→ writeTBQueue subQ [2386]
[BLOCKED — subQ full]
```
### ConnId overlap verified
No guard prevents a connection undergoing queue rotation (AM_QTEST_) or ACK processing from being included in `sendMessagesB_`'s batch. During these operations, the connection has `connStatus == ConnReady`, passing all filters in `memberSendAction`.
### Cascade amplification
Once any single deadlock triggers, `subQ` never drains. ALL other threads that attempt `writeTBQueue subQ` block progressively — their locks are held forever too. The entire threading system freezes within seconds.
### Affected code sites (blocking `writeTBQueue subQ` inside `connLock`)
| Site | File | Lock line | Write line | Events written |
|------|------|-----------|------------|----------------|
| `runSmpQueueMsgDelivery::notify` | Agent.hs | 2187 | 2238 | SWITCH SPCompleted, ERR INTERNAL |
| `runSmpQueueMsgDelivery::internalErr/notifyDel` | Agent.hs | 2187 | 2238 (via notifyDel→notify) | ERR INTERNAL + delMsg |
| `ackQueueMessage::sendMsgNtf` | Agent.hs | 2254 or 1930 | 2386 | MSGNTF |
### Safe patterns that already exist in the codebase
1. **`isFullTBQueue` + pending TVar** (used at `runCommandProcessing` lines 1782-1784/1937, and `runProcessSMP` lines 3027-3029/3216):
```haskell
-- Before processing (e.g. line 1782):
pending <- newTVarIO []
-- During processing — safe notify (e.g. line 1937):
notify cmd =
let t = (corrId, connId, AEvt (sAEntity @e) cmd)
in atomically $ ifM (isFullTBQueue subQ) (modifyTVar' pendingCmds (t :)) (writeTBQueue subQ t)
-- After processing — flush (e.g. line 1784):
mapM_ (atomically . writeTBQueue subQ) . reverse =<< readTVarIO pending
```
2. **`nonBlockingWriteTBQueue`** (used at Client.hs:789, NtfSubSupervisor.hs:507):
```haskell
nonBlockingWriteTBQueue q x = do
sent <- atomically $ tryWriteTBQueue q x
unless sent $ void $ forkIO $ atomically $ writeTBQueue q x
```
Note: `nonBlockingWriteTBQueue` does NOT preserve ordering — the spawned background thread may complete out of order relative to subsequent direct writes from the same calling thread.
### Exhaustive proof: no other deadlock scenarios exist
All 15 `withConnLock` sites in Agent.hs were analyzed. Only 3 write to `subQ`:
| withConnLock site | Writes subQ? | Safe? |
|-------------------|-------------|-------|
| switchConnectionAsync' (899) | No | ✓ |
| setConnShortLinkAsync' (995) | No | ✓ |
| setConnShortLink' (1031) | No | ✓ |
| deleteConnShortLink' (1075) | No | ✓ |
| allowConnection' (1407) | No | ✓ |
| acceptContact' (1417) | No | ✓ |
| sendMessagesB_ (1708, `withConnLocks`) | No | ✓ |
| tryWithLock/runSmpCommand (1930) | Yes (1937) | ✓ — `isFullTBQueue` check |
| tryMoveableWithLock/runSmpCommand (1931) | Yes (1937) | ✓ — `isFullTBQueue` check |
| **runSmpQueueMsgDelivery AM_QTEST_ (2187)** | **Yes (2238)** | **✗ — DEADLOCK** |
| **ackMessage' (2254)** | **Yes (2386)** | **✗ — DEADLOCK** |
| switchConnection' (2298) | No | ✓ |
| abortConnectionSwitch' (2328) | No | ✓ |
| synchronizeRatchet' (2351) | No | ✓ |
| suspendConnection' (2390) | No | ✓ |
| **processSMP (3037)** | Yes (3216) | ✓ — `isFullTBQueue` check |
Note: `processSMP` (line 3037) holds `connLock` and its local `notify` (line 3216) writes to `subQ`, but it uses the safe `isFullTBQueue` pattern. Its `ack` (line 3196) uses `enqueueCmd` (DB-only), NOT `ackQueueMessage`. The actual `ackQueueMessage` runs later from the async command worker via ICAck/ICAckDel.
Other lock pairs checked — no circular dependencies:
- `connLock × DB MVar`: DB never acquires connLock
- `entityLock × connLock`: consistent ordering (entity first in chat, conn in agent)
- `connLock(X) × connLock(Y)`: single agentSubscriber thread, one `withConnLocks` at a time
---
## Deadlock call graph: agentSubscriber → connLock
All deadlock paths require `agentSubscriber` to synchronously acquire `connLock`. Exhaustive analysis shows that **every such path converges on a single agent function**: `sendMessagesB_` → `withConnLocks` (Agent.hs:1708). No other agent API function called synchronously from the agentSubscriber acquires connLock.
Verified (FACT): `ackMessageAsync` → `enqueueCommand` only (no connLock). `toggleConnectionNtfs` → no lock. `deleteConnectionAsync` → `deleteLock` not `connLock`. `joinConnectionAsync` → `withInvLock` not `connLock`.
Also verified (FACT): `Lock = TMVar Text` (Lock.hs:24) is **non-reentrant** — double acquisition on the same thread deadlocks.
### All 22 trigger paths
Every path goes through `deliverMessage`/`deliverMessages`/`deliverMessagesB` → `withAgent sendMessagesB` → `sendMessagesB_` → `withConnLocks`:
| # | Trigger | Chat function | ConnIds locked | Risk |
|---|---------|--------------|----------------|------|
| 1 | Group CON (Invitee) | `introduceToAll` → broadcast XGrpMemNew | **ALL member connIds** | **HIGHEST** |
| 2 | Group MSG XGrpLinkAcpt | `introduceToRemaining` → broadcast | **ALL member connIds** | **HIGHEST** |
| 3 | Group CON (Invitee) | `sendIntroductions` → batch intros to new member | new member connId | Medium |
| 4 | Group CON (Invitee) | `sendHistory` → batch to new member | new member connId | Medium |
| 5 | Group CON | `sendPendingGroupMessages` | member connId | Medium |
| 6 | Group SENT | `sendPendingGroupMessages` | member connId | Medium |
| 7 | Group QCONT | `sendPendingGroupMessages` | member connId | Medium |
| 8 | Group CON (PendingReview) | `introduceToModerators` → to moderators | moderator connIds | Medium |
| 9 | Group CON (PreMember) | `sendXGrpMemCon` → to host | host connId | Low |
| 10 | Group CON (PreMember) | `probeMatchingMemberContact` → probes + hashes | member + N matching connIds | Medium |
| 11 | Direct CON | `probeMatchingMembers` → probes + hashes | contact + N matching connIds | Medium |
| 12 | Direct JOINED | `sendAutoReply` | contact connId | Low |
| 13 | Group JOINED | `sendGroupAutoReply` | member connId | Low |
| 14 | Group INV | `sendXGrpMemInv` → to host | host connId | Low |
| 15 | Group INV (legacy) | `sendGrpInvitation` → to contact | contact connId | Low |
| 16 | Group MSG XGrpMemInv | `xGrpMemInv` → `sendGroupMemberMessage` | re-member connId | Low |
| 17 | Group MSG XGrpMemDel | `forwardToMember` | deleted member connId | Low |
| 18 | Group MSG XGrpLinkMem | `probeMatchingMemberContact` | member + N matching connIds | Medium |
| 19 | Group MSG (dup relay) | `saveGroupRcvMsg` error → `sendDirectMemberMessage` | forwarder connId | Low |
| 20 | SFDONE | `sendFileDescriptions` → to recipients | recipient connIds | Medium |
| 21 | Group MSG XGrpLinkAcpt | `sendHistory` → to accepted member | accepted member connId | Medium |
| 22 | Direct MSG (autoAccept) | `autoAcceptFile` → inline accept reply | contact connId | Low (test-only config) |
### Key observations
1. **Single bottleneck**: All 22 paths converge on `sendMessagesB_` → `withConnLocks` (Agent.hs:1708). The deadlock is between this lock acquisition and any worker thread holding `connLock` + blocking on `writeTBQueue subQ`.
2. **Highest-risk paths** (#1, #2): Broadcasting to ALL group members in `introduceToAll` / `introduceToRemaining` acquires `withConnLocks` on ALL member connIds in a single batch. For large groups, this holds the agentSubscriber thread for a long time, during which subQ fills, which causes worker threads holding connLock on any of those connIds to deadlock.
3. **Medium-risk paths** (#5-7): `sendPendingGroupMessages` fires on every CON/SENT/QCONT. These are frequent and lock the member's connId, which is the SAME connId that a delivery worker or ACK worker may hold while writing to subQ.
---
## Analysis: `withConnLocks` in `sendMessagesB_`
### FACT: the lock protects ratchet encryption state
`sendMessagesB_` (Agent.hs:1708-1713) acquires `withConnLocks` and executes:
1. **`getConn_`** — reads connection metadata, send queues from DB
2. **`setConnPQSupport`** — updates PQ encryption flag per connection
3. **`enqueueMessagesB`** → `enqueueMessageB` → `storeSentMsg_` which calls:
- **`updateSndIds`** (AgentStore.hs:899) — increments `internalSndId` (sequential send counter)
- **`agentRatchetEncryptHeader`** (Agent.hs:3698) — reads current ratchet via `getRatchetForUpdate`, encrypts message header via `rcEncryptHeader`, writes advanced ratchet state via `updateRatchet`
- **`createSndMsg`** + **`createSndMsgDelivery`** — inserts message and delivery records
All operations run within `unsafeWithStore` → `withTransaction` (single DB transaction per batch).
### FACT: the lock CANNOT be removed
Without `withConnLocks`, concurrent `sendMessagesB_` calls targeting the same connection would:
- Read the same ratchet state, both encrypt, one overwrite the other → **ratchet desync** (unrecoverable)
- Get duplicate `internalSndId` values → **message ID collision**
- Race on `setConnPQSupport` → **PQ state inconsistency**
The lock serializes ALL operations on the connection's encryption state. Removing it would introduce data corruption.
Note: `sendMessage` (singular, line 530) uses the same `sendMessagesB_` function — there is no lock-free send path.
### Eliminated strategies
- **Strategy C (remove lock)**: The lock protects ratchet encryption. Removing it causes unrecoverable ratchet desync. Eliminated.
- **Strategy A (async dispatch)**: All 22 chat-layer callers use `deliverMessagesB` return values (delivery IDs, PQ state) synchronously. `forkIO` loses results. Eliminated.
- **Strategy W (isFullTBQueue + pending TVar)**: The existing pattern (lines 1937, 3216) buffers events in a local TVar and flushes after lock release. Between lock release and flush, another thread can acquire the same connLock and write events to subQ — reordering events within the same connection. This trades a visible deadlock for invisible ordering bugs. Eliminated.
- **Strategy O (per-connection overflow queues)**: Bounded overflow queues with "drop when full" were analyzed. Drop consequences are unacceptable at 5 of 6 write sites — CONF, INFO, CON cause permanent connection failure after ACK; INV loses connection invitations; SENT/MERR leave messages stuck forever. Unbounded overflow defeats backpressure. Eliminated.
---
## Solution: move subQ writes outside connLock
### Root cause
The `writeTBQueue subQ` calls at the 3 deadlock sites are inside `connLock` by accident of code structure, not necessity. `connLock` protects ratchet encryption state and DB consistency. The `notify` calls write informational events to `subQ` — they do not modify any state that `connLock` protects.
Moving the writes outside the lock scope eliminates the deadlock: blocking `writeTBQueue subQ` without holding `connLock` is safe — agentSubscriber is free to acquire the lock, process events, and drain `subQ`.
### Why reordering doesn't matter at these sites
The chat layer handlers for the 3 deadlock site events do NOT advance the ratchet:
| Event | Chat handler | Calls sendMessagesB_? |
|-------|-------------|----------------------|
| SWITCH SPCompleted | Creates internal chat item, updates UI | **No** |
| ERR INTERNAL | Logs error to view | **No** |
| MSGNTF | `toView CEvtNtfMessage` → empty output | **No** |
Events that DO trigger ratchet advances (CON, SENT, QCONT → `sendPendingGroupMessages` → `sendMessagesB_`) are all already written OUTSIDE `connLock` in the current code.
Ratchet state lives in the DB, not in subQ events. agentSubscriber processes events sequentially regardless of arrival order. The SENT-before-SWITCH race already exists in the current code (new queue worker writes SENT outside connLock while old queue worker writes SWITCH inside connLock).
### Fix: Site 1 — `runSmpQueueMsgDelivery` AM_QTEST_ (line 2187)
Restructure `withConnLock` to return the event, write outside.
**Current code** (Agent.hs:2187-2214):
```haskell
AM_QTEST_ -> withConnLock c connId "runSmpQueueMsgDelivery AM_QTEST_" $ do
withStore' c $ \db -> setSndQueueStatus db sq Active
SomeConn _ conn <- withStore c (`getConn` connId)
case conn of
DuplexConnection cData' rqs sqs -> do
let addr = qAddress sq
case findQ addr sqs of
Just SndQueue {dbReplaceQueueId = Just replacedId, primary} ->
case removeQP (\sq' -> dbQId sq' == replacedId && not (sameQueue addr sq')) sqs of
Nothing -> internalErr msgId "sent QTEST: queue not found in connection"
Just (sq', sq'' : sqs') -> do
checkSQSwchStatus sq' SSSendingQTEST
atomically $ TM.delete (qAddress sq') $ smpDeliveryWorkers c
withStore' c $ \db -> do
when primary $ setSndQueuePrimary db connId sq
deletePendingMsgs db connId sq'
deleteConnSndQueue db connId sq'
let sqs'' = sq'' :| sqs'
conn' = DuplexConnection cData' rqs sqs''
cStats <- connectionStats c conn'
notify $ SWITCH QDSnd SPCompleted cStats -- DEADLOCK
_ -> internalErr msgId "sent QTEST: ..." -- DEADLOCK (via notifyDel → notify)
_ -> internalErr msgId "sent QTEST: ..." -- DEADLOCK
_ -> internalErr msgId "QTEST sent not in duplex ..." -- DEADLOCK
```
**New code:**
```haskell
AM_QTEST_ -> do
evt_ <- withConnLock c connId "runSmpQueueMsgDelivery AM_QTEST_" $ do
withStore' c $ \db -> setSndQueueStatus db sq Active
SomeConn _ conn <- withStore c (`getConn` connId)
case conn of
DuplexConnection cData' rqs sqs -> do
let addr = qAddress sq
case findQ addr sqs of
Just SndQueue {dbReplaceQueueId = Just replacedId, primary} ->
case removeQP (\sq' -> dbQId sq' == replacedId && not (sameQueue addr sq')) sqs of
Nothing -> pure $ Left "sent QTEST: queue not found in connection"
Just (sq', sq'' : sqs') -> do
checkSQSwchStatus sq' SSSendingQTEST
atomically $ TM.delete (qAddress sq') $ smpDeliveryWorkers c
withStore' c $ \db -> do
when primary $ setSndQueuePrimary db connId sq
deletePendingMsgs db connId sq'
deleteConnSndQueue db connId sq'
let sqs'' = sq'' :| sqs'
conn' = DuplexConnection cData' rqs sqs''
cStats <- connectionStats c conn'
pure $ Right $ SWITCH QDSnd SPCompleted cStats
_ -> pure $ Left "sent QTEST: there is only one queue in connection"
_ -> pure $ Left "sent QTEST: queue not in connection or not replacing another queue"
_ -> pure $ Left "QTEST sent not in duplex connection"
-- subQ write is now OUTSIDE connLock — blocking writeTBQueue is safe
case evt_ of
Right evt -> notify evt
Left err -> internalErr msgId err
```
All DB operations remain inside the lock. Only `notify`/`internalErr` (which write to subQ) move outside. `internalErr` calls `notifyDel` = `notify >> delMsg` — both `notify` (subQ write) and `delMsg` (`deleteSndMsgDelivery`, keyed on unique msgId) are safe outside the lock. The existing double-delete pattern (`delMsg` inside `internalErr` + `delMsgKeep` at line 2216) is preserved.
### Fix: Sites 2 & 3 — `ackQueueMessage::sendMsgNtf` (line 2386)
Change `ackQueueMessage` to return the MSGNTF event instead of writing it to subQ. Callers write to subQ after releasing connLock.
**Current code** (Agent.hs:2371-2386):
```haskell
ackQueueMessage :: AgentClient -> RcvQueue -> SMP.MsgId -> AM ()
ackQueueMessage c rq@RcvQueue {userId, connId, server} srvMsgId = do
atomically $ incSMPServerStat c userId server ackAttempts
tryAllErrors (sendAck c rq srvMsgId) >>= \case
Right _ -> sendMsgNtf ackMsgs
Left (SMP _ SMP.NO_MSG) -> sendMsgNtf ackNoMsgErrs
Left e -> ...
where
sendMsgNtf stat = do
atomically $ incSMPServerStat c userId server stat
whenM (liftIO $ hasGetLock c rq) $ do
atomically $ releaseGetLock c rq
brokerTs_ <- eitherToMaybe <$> tryAllErrors (withStore c $ \db -> getRcvMsgBrokerTs db connId srvMsgId)
atomically $ writeTBQueue (subQ c) ("", connId, AEvt SAEConn $ MSGNTF srvMsgId brokerTs_)
```
**New code** — return `Maybe ATransmission` instead of writing:
```haskell
ackQueueMessage :: AgentClient -> RcvQueue -> SMP.MsgId -> AM (Maybe ATransmission)
ackQueueMessage c rq@RcvQueue {userId, connId, server} srvMsgId = do
atomically $ incSMPServerStat c userId server ackAttempts
tryAllErrors (sendAck c rq srvMsgId) >>= \case
Right _ -> sendMsgNtf ackMsgs
Left (SMP _ SMP.NO_MSG) -> sendMsgNtf ackNoMsgErrs
Left e -> ... >> pure Nothing
where
sendMsgNtf stat = do
atomically $ incSMPServerStat c userId server stat
ifM (liftIO $ hasGetLock c rq)
(do atomically $ releaseGetLock c rq
brokerTs_ <- eitherToMaybe <$> tryAllErrors (withStore c $ \db -> getRcvMsgBrokerTs db connId srvMsgId)
pure $ Just ("", connId, AEvt SAEConn $ MSGNTF srvMsgId brokerTs_))
(pure Nothing)
```
**Caller 1: `ackMessage'`** (Agent.hs:2253-2267) — return event from `withConnLock`, write after:
```haskell
ackMessage' c connId msgId rcptInfo_ = do
t_ <- withConnLock c connId "ackMessage" $ do
SomeConn _ conn <- withStore c (`getConn` connId)
case conn of
DuplexConnection {} -> do
t_ <- ack
sendRcpt conn
del
pure t_
RcvConnection {} -> do
t_ <- ack
del
pure t_
SndConnection {} -> throwE $ CONN SIMPLEX "ackMessage"
ContactConnection {} -> throwE $ CMD PROHIBITED "ackMessage: ContactConnection"
NewConnection _ -> throwE $ CMD PROHIBITED "ackMessage: NewConnection"
-- subQ write is OUTSIDE connLock
case t_ of
Just t -> atomically $ writeTBQueue (subQ c) t
Nothing -> pure ()
```
**Caller 2: `ICAck` / `ICAckDel`** (Agent.hs:1823-1824) — inline `tryWithLock` as `tryCommand` + `withConnLock`, write subQ between the two scopes:
`tryWithLock name = tryCommand . withConnLock c connId name` — by inlining, the subQ write can be placed outside `withConnLock` but inside `tryCommand` (retaining retry/error handling).
```haskell
ICAck rId srvMsgId -> withServer $ \srv ->
tryCommand $ do
t_ <- withConnLock c connId "ICAck" $ ack srv rId srvMsgId
-- subQ write is OUTSIDE connLock — cannot deadlock with agentSubscriber
forM_ t_ $ atomically . writeTBQueue subQ
ICAckDel rId srvMsgId msgId -> withServer $ \srv ->
tryCommand $ do
t_ <- withConnLock c connId "ICAckDel" $ do
t_ <- ack srv rId srvMsgId
withStore' c (\db -> deleteMsg db connId msgId)
pure t_
-- subQ write is OUTSIDE connLock — cannot deadlock with agentSubscriber
forM_ t_ $ atomically . writeTBQueue subQ
```
Where `ack` now returns `AM (Maybe ATransmission)`:
```haskell
ack srv rId srvMsgId = do
rq <- withStore c $ \db -> getRcvQueue db connId srv rId
ackQueueMessage c rq srvMsgId
```
All subQ writes for MSGNTF are now outside connLock. FIFO ordering is preserved — no `nonBlockingWriteTBQueue`, no forked threads. The same thread that held the lock writes to subQ sequentially after releasing it.
### Race analysis
Window between connLock release and subQ write at Site 1:
| Thread | Can acquire connLock(X)? | Writes subQ? | Consequence |
|--------|-------------------------|-------------|-------------|
| agentSubscriber via sendMessagesB_ | Yes | **No** (encrypts only) | No race |
| processSMP for connId X | Yes | Yes (pending flush) | MSG before SWITCH — cosmetic |
| runCommandProcessing for connId X | Yes | Yes (pending flush) | Command response before SWITCH — cosmetic |
| New queue delivery worker | No (SENT outside lock) | Yes | SENT before SWITCH — cosmetic, **already exists in current code** |
All races are cosmetic UI ordering. None affect ratchet state, protocol correctness, or message delivery.
### Summary of changes
| File | Change | Lines affected |
|------|--------|---------------|
| Agent.hs | Restructure AM_QTEST_ to return event from `withConnLock`, write outside | ~2187-2214 |
| Agent.hs | Change `ackQueueMessage` return type to `AM (Maybe ATransmission)`, return event instead of writing | ~2371-2386 |
| Agent.hs | `ackMessage'`: return event from `withConnLock`, write outside | ~2253-2267 |
| Agent.hs | `ICAck`/`ICAckDel`: inline `tryCommand` + `withConnLock`, write subQ between scopes | ~1823-1824 |
| Agent.hs | `ack` helper: propagate new return type | ~1899-1901 |
No new data structures. No new modules. No changes to other write sites (1937, 3216 — already safe). ~25 lines changed total.
+184
View File
@@ -0,0 +1,184 @@
# SMP Server Postgres: Slow Query Analysis
Data from three production servers (A, B, C), ~3.5 day observation window.
## Top queries by total time
| Rank | Query | Server A ms | Server B ms | Server C ms |
|------|-------|-------------|-------------|-------------|
| 1 | getEntityCounts (6 COUNT subqueries) | 1,682,874 | 1,639,325 | 1,619,892 |
| 2 | write_message() | 303,393 | 458,262 | 280,375 |
| 3 | try_del_peek_msg() | 352,912 | 386,036 | 333,877 |
| 4 | expire_old_messages() | 246,034 | 220,003 | 160,232 |
| 5 | UPDATE SET updated_at | 234,146 | 216,911 | 211,480 |
| 6 | INSERT INTO messages | 184,430 | 323,617 | 169,149 |
| 7 | expire batch cursor (array_agg) | 122,739 | 99,061 | 39,975 |
| 8 | Batch recipient_id IN lookups | ~134K | ~79K | ~81K |
| 9 | Batch notifier_id IN lookups | ~143K | ~64K | ~64K |
| 10 | msg_peek (SELECT FROM messages) | ~112K | ~102K | ~126K |
getEntityCounts alone is **45-48%** of total query time on all three servers.
---
## Verified fixes
### 1. getEntityCounts: replace ③④ with SUM(queue_count)
**Query** (`QueueStore/Postgres.hs:160-167`):
```sql
SELECT
(SELECT COUNT(1) FROM msg_queues WHERE deleted_at IS NULL) AS queue_count, -- ①
(SELECT COUNT(1) FROM msg_queues WHERE deleted_at IS NULL AND notifier_id IS NOT NULL) AS notifier_count, -- ②
(SELECT COUNT(1) FROM services WHERE service_role = ?) AS rcv_service_count, -- trivial
(SELECT COUNT(1) FROM services WHERE service_role = ?) AS ntf_service_count, -- trivial
(SELECT COUNT(1) FROM msg_queues WHERE rcv_service_id IS NOT NULL AND deleted_at IS NULL) AS rcv_service_queues_count, -- ③
(SELECT COUNT(1) FROM msg_queues WHERE ntf_service_id IS NOT NULL AND deleted_at IS NULL) AS ntf_service_queues_count -- ④
```
| Server | Calls | Avg ms | Max ms | Total ms |
|--------|-------|--------|--------|----------|
| A | 5,058 | 332.7 | 2,061 | 1,682,874 |
| B | 5,055 | 324.3 | 1,844 | 1,639,325 |
| C | 5,053 | 320.6 | 1,250 | 1,619,892 |
**Problem**: 4 subqueries scan `msg_queues`. Indexes exist for ③
(`idx_msg_queues_rcv_service_id(rcv_service_id, deleted_at)`), ④
(`idx_msg_queues_ntf_service_id(ntf_service_id, deleted_at)`), and potentially ①
(`idx_msg_queues_updated_at_recipient_id(deleted_at, ...)`), but actual query plans
and per-subquery cost are unknown without `EXPLAIN ANALYZE`.
**Fix**: Replace ③ and ④:
```sql
COALESCE((SELECT SUM(queue_count) FROM services WHERE service_role = 'M'), 0) AS rcv_service_queues_count,
COALESCE((SELECT SUM(queue_count) FROM services WHERE service_role = 'N'), 0) AS ntf_service_queues_count
```
**Verification**:
- Trigger logic traced for all transitions (NULL→set, change, soft-delete, physical delete) — correct.
- FK `rcv_service_id REFERENCES services(service_id)` guarantees equivalence.
- `queue_count + p_change` is atomic under READ COMMITTED.
- `update_all_aggregates()` exists as repair mechanism.
**Savings**: Eliminates 2 of 4 msg_queues scans. Exact per-subquery cost unknown — needs `EXPLAIN ANALYZE`.
---
### 2. expire_old_messages: remove trailing COUNTs
At the end of `expire_old_messages` (`Migrations.hs`):
```sql
r_stored_msgs_count := (SELECT COUNT(1) FROM messages);
r_stored_queues := (SELECT COUNT(1) FROM msg_queues WHERE deleted_at IS NULL);
```
| Server | expire avg | r_stored_queues avg | r_stored_msgs avg | COUNTs combined | % of procedure |
|--------|-----------|---------------------|-------------------|-----------------|----------------|
| A | 11,716ms | 695ms | 110ms | 805ms | 6.9% |
| B | 10,476ms | 631ms | 17ms | 648ms | 6.2% |
| C | 7,630ms | 588ms | 53ms | 641ms | 8.4% |
**How used** (`Server.hs:485-488`):
- `storedMsgsCount` → resets `msgCount` stat (also maintained incrementally: +1 on send, -1 on ACK)
- `storedQueues`**only logged** via `printMessageStats`. Same value available from `getEntityCounts`.
**Fix**: Remove both COUNTs. Return only `r_expired_msgs_count`.
**Verification**: All usages traced. `storedQueues` is only logged. `storedMsgsCount` resets
an incrementally-maintained counter — removing means potential drift, corrected on restart.
**Savings**: 641805ms per cycle × 21 cycles = **13.516.9s total** (CSV-verified).
---
### 3. Trigger WHEN clause: skip PL/pgSQL call for non-service updates
**Current** (`Migrations.hs:566-568`):
```sql
CREATE TRIGGER tr_queue_update
AFTER UPDATE ON msg_queues
FOR EACH ROW EXECUTE PROCEDURE on_queue_update();
```
Server C data — ~2.6M updates, only ~110K (4.3%) change service fields:
| UPDATE pattern | Calls | Service fields? |
|----------------|-------|-----------------|
| SET updated_at | ~1,275K | No |
| SET msg_can_write/size/expire (write_message) | ~600K | No |
| SET msg_can_write/size/expire (try_del_*) | ~331K | No |
| SET msg_can_write/size/expire (try_del reset) | ~258K | No |
| SET sender_key | ~17K | No |
| SET msg_can_write/size/expire (delete_expired) | ~3K | No |
| **SET rcv_service_id** | **~101K** | **Yes** |
| **SET deleted_at** | **~10K** | **Yes** |
**Fix**: Add `WHEN` clause — evaluated in C by PostgreSQL, skips function call entirely:
```sql
CREATE TRIGGER tr_queue_update
AFTER UPDATE ON msg_queues
FOR EACH ROW
WHEN (
OLD.deleted_at IS DISTINCT FROM NEW.deleted_at
OR OLD.rcv_service_id IS DISTINCT FROM NEW.rcv_service_id
OR OLD.ntf_service_id IS DISTINCT FROM NEW.ntf_service_id
OR OLD.notifier_id IS DISTINCT FROM NEW.notifier_id
)
EXECUTE PROCEDURE on_queue_update();
```
**Verification**:
- PostgreSQL supports `OLD`/`NEW` in `WHEN` for `AFTER UPDATE` triggers.
- 4 conditions match exactly the fields checked inside `on_queue_update()`.
- Behavioral change: **none**.
**Savings**: ~2.5M PL/pgSQL calls avoided. Per-call overhead estimated ~0.020.05ms.
Total: ~50125s estimated, not measured.
---
## Needs EXPLAIN ANALYZE
### 4. Partial indexes for getEntityCounts ① and ②
After fix #1, subqueries ① and ② remain. ② has no index covering both
`deleted_at IS NULL` and `notifier_id IS NOT NULL`.
```sql
CREATE INDEX idx_msg_queues_live ON msg_queues ((1)) WHERE deleted_at IS NULL;
CREATE INDEX idx_msg_queues_live_notifier ON msg_queues ((1)) WHERE deleted_at IS NULL AND notifier_id IS NOT NULL;
```
**Trade-off**: Write overhead on every INSERT/UPDATE/DELETE. Need `EXPLAIN ANALYZE`
to confirm PostgreSQL uses these for COUNT vs choosing seq scan.
---
## Not problems
- **write_message / try_del_peek_msg** (ranks #2-3): 0.5-0.7ms avg. High total from volume (~600K calls).
Max spikes (490-523ms) are lock contention on `FOR UPDATE` — architectural, not fixable.
- **UPDATE SET updated_at** (rank #5): 0.17ms avg, ~1.3M calls. Already minimal.
Fix #3 eliminates the trigger overhead on these.
- **SET rcv_service_id** (Server C only, 100K calls): CSV shows rows_affected = calls — all
legitimate associations. Haskell guard at `Postgres.hs:487` works correctly.
- **Batch lookups** (ranks #8-9): 1.8-2.0ms avg for ~135 PK probes. Near-optimal.
---
## Summary
| # | Fix | Per-call savings | Calls | Total savings | Verified |
|---|-----|-----------------|-------|---------------|----------|
| 1 | getEntityCounts ③④ → `SUM(queue_count)` | 0158ms (unknown split) | ~5,050 | 0800s | Correctness: yes. Savings: needs EXPLAIN ANALYZE |
| 2 | Remove trailing COUNTs from expire_old_messages | 641805ms | 21 | 13.516.9s | Yes (CSV) |
| 3 | Add WHEN clause to tr_queue_update | ~0.020.05ms | ~2.5M skipped | ~50125s est. | Correctness: yes. Savings: estimated |
| 4 | Partial indexes for ①② | Unknown | ~5,050 | Unknown | Needs EXPLAIN ANALYZE |
+186
View File
@@ -0,0 +1,186 @@
# SMP Server Postgres: Slow Query Analysis (post-reset)
Data from three production servers (A, B, C), ~5.5 hour window after stats reset.
EXPLAIN ANALYZE from a large server (~30M rows in msg_queues).
## Top queries by total time
| Rank | Query | A total ms | B total ms | C total ms |
|------|-------|-----------|-----------|-----------|
| 1 | getEntityCounts (6 COUNT subqueries) | 99,799 | 101,946 | 121,495 |
| 2 | UPDATE SET updated_at | 35,799 | 30,797 | 26,551 |
| 3 | try_del_peek_msg() | 28,903 | 23,147 | 26,482 |
| 4 | write_message() | 23,103 | 18,942 | 20,301 |
| 5 | Batch recipient_id IN lookups | 16,062 | 12,529 | 9,977 |
| 6 | INSERT INTO messages | 14,395 | 11,911 | 12,882 |
| 7 | msg_peek (SELECT FROM messages) | 12,882 | 11,992 | 14,661 |
| 8 | expire_old_messages() | 9,762 | 11,619 | 10,863 |
| 9 | delete_expired_msgs() | 7,801 | 5,456 | 6,256 |
| 10 | expire batch cursor (array_agg) | 4,421 | 5,679 | 4,566 |
Grand totals: A ~292s, B ~317s, C ~288s.
getEntityCounts is **34-42%** of all query time across all three servers.
---
## EXPLAIN ANALYZE results for getEntityCounts
Run on a large server (~30M rows in msg_queues, cold cache):
| Subquery | Time | % of 23.4s | Plan | Rows scanned | Key detail |
|----------|------|-----------|------|-------------|------------|
| ① queue_count | **7,851ms** | **33.5%** | Parallel Seq Scan | 30.6M (97% match) | No useful index |
| ② notifier_count | **6,382ms** | **27.2%** | Parallel Seq Scan | 30.6M (4M match) | No useful index |
| ③ rcv_service_queues | 0.5ms | 0% | Index Only Scan | 0 rows | No rcv services on this server; similar cost to ④ on servers with rcv services |
| ④ ntf_service_queues | **8,914ms** | **38.0%** | Parallel Index Only Scan | 3.7M match | **2.7M heap fetches** |
| Services (③④) | 1.8ms | 0% | Index Only / Bitmap | 0 + 6 rows | Trivial |
| JIT + Planning | 1,341ms | 5.7% | — | — | JIT compilation overhead |
| **Total** | **23,440ms** | | | | |
The CSV averages (302-368ms) reflect warm-cache performance. This EXPLAIN is cold cache
(`shared read=3.4M` vs `shared hit=44K` — 98.7% read from disk).
---
## Verified fixes
### 1. getEntityCounts: replace ③④ with SUM(queue_count)
```sql
-- Current ③ and ④:
(SELECT COUNT(1) FROM msg_queues WHERE rcv_service_id IS NOT NULL AND deleted_at IS NULL) -- ③: 0.5ms
(SELECT COUNT(1) FROM msg_queues WHERE ntf_service_id IS NOT NULL AND deleted_at IS NULL) -- ④: 8,914ms
-- Fix:
COALESCE((SELECT SUM(queue_count) FROM services WHERE service_role = 'M'), 0) -- ~0ms
COALESCE((SELECT SUM(queue_count) FROM services WHERE service_role = 'N'), 0) -- ~0ms
```
**EXPLAIN ANALYZE confirmed**: ③ returns 0 rows (already free). ④ costs 8,914ms due to
Parallel Index Only Scan with 2.7M heap fetches on `idx_msg_queues_ntf_service_id`.
**Verification**:
- Trigger logic traced for all transitions (NULL→set, change, soft-delete, physical delete) — correct.
- FK `rcv_service_id REFERENCES services(service_id)` guarantees equivalence.
- `queue_count + p_change` is atomic under READ COMMITTED.
- `update_all_aggregates()` exists as repair mechanism.
**Savings**: ~8.9s cold cache (38% of query). Warm cache proportionally less but still dominant ④ cost.
---
### 2. getEntityCounts: partial indexes for ① and ②
```sql
CREATE INDEX idx_msg_queues_active ON msg_queues ((1)) WHERE deleted_at IS NULL;
CREATE INDEX idx_msg_queues_active_notifier ON msg_queues ((1)) WHERE deleted_at IS NULL AND notifier_id IS NOT NULL;
```
**EXPLAIN ANALYZE confirmed**: ① does Parallel Seq Scan (7,851ms), ② does Parallel Seq Scan (6,382ms).
No existing index is used for these subqueries despite `idx_msg_queues_updated_at_recipient_id`
having `deleted_at` as first column — PostgreSQL chose seq scan because 97% of rows match.
Partial indexes contain only matching rows, enabling fast index-only COUNT without scanning
the full table.
**Trade-off**: Write overhead on every INSERT/UPDATE/DELETE that changes `deleted_at` or
`notifier_id`. For the ~30M row table with ~300K updates per 5.5h, this is acceptable.
**Savings**: ~14.2s cold cache (61% of query). Combined with fix #1: **23.1s → ~1.3s** (JIT only).
---
### 3. expire_old_messages: remove trailing COUNTs
At the end of `expire_old_messages` (`Migrations.hs`):
```sql
r_stored_msgs_count := (SELECT COUNT(1) FROM messages);
r_stored_queues := (SELECT COUNT(1) FROM msg_queues WHERE deleted_at IS NULL);
```
| Server | expire avg | r_stored_queues | r_stored_msgs | COUNTs combined | % of procedure |
|--------|-----------|-----------------|---------------|-----------------|----------------|
| A | 9,762ms | 535ms | 50ms | 585ms | 6.0% |
| B | 11,619ms | 390ms | 12ms | 402ms | 3.5% |
| C | 10,863ms | 659ms | 25ms | 684ms | 6.3% |
**Usage** (`Server.hs:485-488`):
- `storedMsgsCount` → resets `msgCount` stat (also maintained incrementally: +1 on send, -1 on ACK).
- `storedQueues`**only logged**. Same value available from `getEntityCounts`.
**Fix**: Remove both COUNTs. Return only `r_expired_msgs_count`.
**Verification**: All usages traced. `storedQueues` is only logged. `storedMsgsCount` resets
an incrementally-maintained counter — removing means potential drift, corrected on restart.
**Savings**: 402684ms per cycle (CSV-verified).
---
### 4. Trigger WHEN clause: skip PL/pgSQL call for non-service updates
**Current** (`Migrations.hs:566-568`):
```sql
CREATE TRIGGER tr_queue_update
AFTER UPDATE ON msg_queues
FOR EACH ROW EXECUTE PROCEDURE on_queue_update();
```
Server C data — ~297K updates, only ~670 (0.2%) change service-related fields:
| UPDATE pattern | Calls | Service fields? |
|----------------|-------|-----------------|
| SET updated_at | ~213K | No |
| SET msg_can_write/size/expire (write_message) | ~40K | No |
| SET msg_can_write/size/expire (try_del_*) | ~27K | No |
| SET msg_can_write/size (try_del reset) | ~14K | No |
| SET sender_key | ~2K | No |
| **SET deleted_at** | **~563** | **Yes** |
| **ntf_service_id/notifier_id changes** | **~108** | **Yes** |
**Fix**: Add `WHEN` clause — evaluated in C by PostgreSQL, skips function call entirely:
```sql
CREATE TRIGGER tr_queue_update
AFTER UPDATE ON msg_queues
FOR EACH ROW
WHEN (
OLD.deleted_at IS DISTINCT FROM NEW.deleted_at
OR OLD.rcv_service_id IS DISTINCT FROM NEW.rcv_service_id
OR OLD.ntf_service_id IS DISTINCT FROM NEW.ntf_service_id
OR OLD.notifier_id IS DISTINCT FROM NEW.notifier_id
)
EXECUTE PROCEDURE on_queue_update();
```
**Verification**:
- PostgreSQL supports `OLD`/`NEW` in `WHEN` for `AFTER UPDATE` triggers.
- 4 conditions match exactly the fields checked inside `on_queue_update()`.
- Behavioral change: **none**.
**Savings**: ~296K PL/pgSQL calls avoided (99.8% of trigger fires). Per-call overhead
estimated ~0.020.05ms. Total: ~615s estimated over this 5.5h window.
---
## Not problems
- **write_message / try_del_peek_msg**: 0.5-0.65ms avg. High total from volume. Max spikes are lock contention — architectural.
- **UPDATE SET updated_at**: 0.12-0.17ms avg. Fix #4 eliminates trigger overhead.
- **Batch lookups**: 1.7-2.6ms avg for ~135 PK probes. Near-optimal.
---
## Summary
| # | Fix | Savings | Verified |
|---|-----|---------|----------|
| 1 | getEntityCounts ③④ → `SUM(queue_count)` | ~8.9s/call cold, ④ eliminated (EXPLAIN ANALYZE) | Yes |
| 2 | Partial indexes for getEntityCounts ①② | ~14.2s/call cold, ①② eliminated (EXPLAIN ANALYZE) | Yes — plan confirmed, index benefit to verify after creation |
| 3 | Remove trailing COUNTs from expire_old_messages | 402684ms/cycle (CSV) | Yes |
| 4 | Add WHEN clause to tr_queue_update | ~615s est. over 5.5h (296K calls skipped) | Correctness: yes. Savings: estimated |
Fixes #1 + #2 combined: getEntityCounts **23.4s → ~1.3s cold cache** (94% reduction).
+199
View File
@@ -0,0 +1,199 @@
# SMP Server Postgres: Slow Query Analysis
Data from three production servers (A, B, C) over a multi-day observation window.
## Verified fixes
### 1. getEntityCounts: replace ③④ with SUM(queue_count)
**Query** (`QueueStore/Postgres.hs:160-167`):
```sql
SELECT
(SELECT COUNT(1) FROM msg_queues WHERE deleted_at IS NULL) AS queue_count, -- ① scan
(SELECT COUNT(1) FROM msg_queues WHERE deleted_at IS NULL AND notifier_id IS NOT NULL) AS notifier_count, -- ② scan
(SELECT COUNT(1) FROM services WHERE service_role = ?) AS rcv_service_count, -- trivial (<10 rows)
(SELECT COUNT(1) FROM services WHERE service_role = ?) AS ntf_service_count, -- trivial (<10 rows)
(SELECT COUNT(1) FROM msg_queues WHERE rcv_service_id IS NOT NULL AND deleted_at IS NULL) AS rcv_service_queues_count, -- ③ scan
(SELECT COUNT(1) FROM msg_queues WHERE ntf_service_id IS NOT NULL AND deleted_at IS NULL) AS ntf_service_queues_count -- ④ scan
```
**Performance**: ~315ms avg, ~2s max, ~2500 calls. #1 slow query by total time (~800s).
**Problem**: 4 scans of `msg_queues`. Indexes exist for some subqueries
(`idx_msg_queues_rcv_service_id(rcv_service_id, deleted_at)` for ③,
`idx_msg_queues_ntf_service_id(ntf_service_id, deleted_at)` for ④,
`idx_msg_queues_updated_at_recipient_id(deleted_at, ...)` potentially for ①),
but whether PostgreSQL uses them for COUNT and the actual per-subquery cost is
unknown without `EXPLAIN ANALYZE`.
**Fix**: Replace ③ and ④:
```sql
COALESCE((SELECT SUM(queue_count) FROM services WHERE service_role = 'M'), 0) AS rcv_service_queues_count,
COALESCE((SELECT SUM(queue_count) FROM services WHERE service_role = 'N'), 0) AS ntf_service_queues_count
```
**Verification**:
- Trigger logic traced for all transitions (NULL→set, change, soft-delete, physical delete) — correct.
- FK `rcv_service_id REFERENCES services(service_id)` guarantees every non-NULL value maps to a row.
- `queue_count + p_change` is atomic under READ COMMITTED — concurrent-safe.
- `update_all_aggregates()` exists as repair mechanism.
- `services` table has <10 rows — SUM is O(1) vs full table scan.
**Savings**: Eliminates 2 of 4 msg_queues scans (③④ → trivial SUM on <10 rows).
Exact savings unknown — if ③④ already use indexes efficiently, savings may be modest.
`EXPLAIN ANALYZE` needed to measure actual per-subquery cost.
---
### 2. expire_old_messages: remove trailing COUNTs
**Stored procedure** (`Migrations.hs`), at the end of `expire_old_messages`:
```sql
r_expired_msgs_count := total_deleted;
r_stored_msgs_count := (SELECT COUNT(1) FROM messages); -- 13-114ms per call (CSV)
r_stored_queues := (SELECT COUNT(1) FROM msg_queues WHERE deleted_at IS NULL); -- 544-719ms per call (CSV)
```
**Performance**: 10 calls per observation window. Per-call cost of these COUNTs (from CSV):
| Server | expire_old_messages avg | r_stored_queues avg | r_stored_msgs avg | COUNTs combined | % of procedure |
|---|---|---|---|---|---|
| A | 11,798ms | 719ms | 114ms | 833ms | 7.1% |
| B | 11,242ms | 544ms | 13ms | 557ms | 5.0% |
| C | 7,296ms | 588ms | 67ms | 655ms | 9.0% |
**How results are used** (`Server.hs:485-488`):
```haskell
Right msgStats@MessageStats {storedMsgsCount = stored, expiredMsgsCount = expired} -> do
atomicWriteIORef (msgCount stats) stored -- resets msgCount from storedMsgsCount
atomicModifyIORef'_ (msgExpired stats) (+ expired)
printMessageStats "STORE: messages" msgStats -- logs all three fields
```
- `expiredMsgsCount` — computed incrementally in the loop. **Needed, already cheap.**
- `storedMsgsCount` — used to reset `msgCount` stat. But `msgCount` is also maintained
incrementally (`+1` on send at line 1963, `-1` on ACK at line 1916). The reset corrects drift.
- `storedQueues`**used only for logging**. Same value available from `getEntityCounts`
which runs every ~60s via Prometheus.
**Fix**: Remove both COUNTs from the stored procedure. Return only `r_expired_msgs_count`.
For `storedMsgsCount`: either trust the incremental `msgCount` counter, or query
`SELECT COUNT(1) FROM messages` separately (in parallel, not blocking the procedure).
For `storedQueues`: use the value from the most recent `getEntityCounts` call.
**Verification**: Traced all usages of `MessageStats` fields from `expireOldMessages` in
`Server.hs`. `storedQueues` is only logged. `storedMsgsCount` resets a counter that's already
maintained incrementally — removing the reset means potential drift, but the counter is
corrected on next server restart anyway.
**Savings**: 560-830ms per expiration cycle × 10 cycles = **5.6-8.3s total** over observation window.
---
### 3. Trigger WHEN clause: skip function call for non-service updates
**Current trigger** (`Migrations.hs:566-568`):
```sql
CREATE TRIGGER tr_queue_update
AFTER UPDATE ON msg_queues
FOR EACH ROW EXECUTE PROCEDURE on_queue_update();
```
Fires on **every UPDATE** to `msg_queues`. From Server C data, ~1.2M updates per observation
window, but only ~105K (8.7%) actually change service-related fields:
| UPDATE pattern | Calls | Changes service fields? |
|---|---|---|
| SET updated_at | ~427K | No |
| SET msg_can_write/size/expire (write_message) | ~336K | No |
| SET msg_can_write/size/expire (try_del_*) | ~196K | No |
| SET msg_can_write/size/expire (delete_expired) | ~136K | No |
| SET sender_key | ~8K | No |
| SET status | ~2K | No |
| **SET rcv_service_id** | **~101K** | **Yes** |
| **SET deleted_at** | **~5K** | **Yes** |
The `on_queue_update()` PL/pgSQL function evaluates 8-12 boolean conditions on every call,
then returns without calling `update_aggregates` for 91% of invocations.
**Fix**: Add a `WHEN` clause to the trigger definition. PostgreSQL evaluates `WHEN` in C code
before calling the PL/pgSQL function — no function entry overhead at all:
```sql
CREATE TRIGGER tr_queue_update
AFTER UPDATE ON msg_queues
FOR EACH ROW
WHEN (
OLD.deleted_at IS DISTINCT FROM NEW.deleted_at
OR OLD.rcv_service_id IS DISTINCT FROM NEW.rcv_service_id
OR OLD.ntf_service_id IS DISTINCT FROM NEW.ntf_service_id
OR OLD.notifier_id IS DISTINCT FROM NEW.notifier_id
)
EXECUTE PROCEDURE on_queue_update();
```
**Verification**:
- PostgreSQL supports `OLD`/`NEW` in `WHEN` clauses for `AFTER UPDATE` triggers.
- The 4 conditions match exactly the fields checked inside `on_queue_update()`.
- When WHEN is false, the function is **never called** — zero PL/pgSQL overhead.
- When WHEN is true, the function runs identically to today.
- Behavioral change: **none** — same aggregates updated in same cases.
**Savings**: ~1.1M PL/pgSQL function calls avoided. Each call has fixed overhead
(function entry, OLD/NEW row extraction, condition evaluation, return). Exact savings
need measurement, but function call overhead is non-trivial at this volume.
---
## Fixes that need EXPLAIN ANALYZE
### 4. Partial indexes for getEntityCounts ① and ②
Subqueries ① and ② still scan msg_queues. ① may use
`idx_msg_queues_updated_at_recipient_id(deleted_at, ...)` but ② has no index covering
both `deleted_at IS NULL` and `notifier_id IS NOT NULL`. Actual plans unknown.
Candidate indexes:
```sql
-- For ① queue_count: enables index-only COUNT
CREATE INDEX idx_msg_queues_live ON msg_queues ((1)) WHERE deleted_at IS NULL;
-- For ② notifier_count: enables index-only COUNT
CREATE INDEX idx_msg_queues_live_notifier ON msg_queues ((1)) WHERE deleted_at IS NULL AND notifier_id IS NOT NULL;
```
**Trade-off**: Each index adds write overhead on every INSERT/UPDATE/DELETE touching the
filtered columns. Need `EXPLAIN ANALYZE` to confirm the COUNT actually uses the index
(PostgreSQL may choose seq scan if the partial index covers most rows).
---
## Not fixable (architectural)
- **write_message / try_del_peek_msg max times (490-523ms)**: Lock contention on
`FOR UPDATE` of the same `recipient_id` row. Inherent to concurrent queue access — cannot
use `SKIP LOCKED` because these operations require the lock for correctness.
- **UPDATE msg_queues SET updated_at (~430K calls, 83-90s total, 0.20ms avg)**: Per-call cost
is already minimal. Trigger does zero aggregate work for this pattern (verified — all
IS DISTINCT FROM checks fail, no `update_aggregates` called). Fix #3 eliminates even
the function call overhead.
---
## Summary
| # | Fix | Per-call savings | Calls | Total savings | Verified |
|---|-----|-----------------|-------|---------------|----------|
| 1 | getEntityCounts ③④ → `SUM(queue_count)` | 0158ms (unknown split across 4 subqueries) | ~2,500 | 0395s | Correctness: yes. Savings: needs EXPLAIN ANALYZE |
| 2 | Remove trailing COUNTs from expire_old_messages | 557833ms (CSV-verified) | 10 | 5.68.3s | Yes (CSV verified) |
| 3 | Add WHEN clause to tr_queue_update | ~0.020.05ms (PL/pgSQL entry overhead estimate) | ~1.1M skipped | ~2255s | Correctness: yes. Savings: estimated, not measured |
| 4 | Partial indexes for ①② | Unknown | ~2,500 | Unknown | No — needs EXPLAIN ANALYZE |
+331 -73
View File
@@ -1,4 +1,4 @@
Version 5, 2024-06-22
Version 7, 2025-01-24
# SMP agent protocol - duplex communication over SMP protocol
@@ -6,9 +6,10 @@ Version 5, 2024-06-22
- [Abstract](#abstract)
- [SMP agent](#smp-agent)
- [SMP servers management](#smp-servers-management)
- [SMP routers management](#smp-routers-management)
- [SMP agent protocol scope](#smp-agent-protocol-scope)
- [Duplex connection procedure](#duplex-connection-procedure)
- [Fast duplex connection procedure](#fast-duplex-connection-procedure)
- [Contact addresses](#contact-addresses)
- [Communication between SMP agents](#communication-between-smp-agents)
- [Message syntax](#messages-between-smp-agents)
@@ -20,41 +21,58 @@ Version 5, 2024-06-22
- [Rotating messaging queue](#rotating-messaging-queue)
- [End-to-end encryption](#end-to-end-encryption)
- [Connection link: 1-time invitation and contact address](#connection-link-1-time-invitation-and-contact-address)
- [Appendix A: SMP agent API](#smp-agent-api)
- [Full connection link syntax](#full-connection-link-syntax)
- [Short connection link syntax](#short-connection-link-syntax)
- [Short links](#short-links)
- [Link key derivation](#link-key-derivation)
- [Link data encryption](#link-data-encryption)
- [Short link resolution](#short-link-resolution)
- [Link data management](#link-data-management)
- [Appendix A: SMP agent API](#appendix-a-smp-agent-api)
- [API functions](#api-functions)
- [API events](#api-events)
## Abstract
The purpose of SMP agent protocol is to define the syntax and the semantics of communications between the client and the agent that connects to [SMP](./simplex-messaging.md) servers.
The purpose of SMP agent protocol is to define the syntax and the semantics of communications between the client and the agent that connects to [SMP](./simplex-messaging.md) routers.
It provides:
- API to create and manage bi-directional (duplex) connections between the users of SMP agents consisting of two (or more) separate unidirectional (simplex) SMP queues, abstracting away multiple steps required to establish bi-directional connections and any information about the servers location from the users of the agent protocol.
- API to create and manage bi-directional (duplex) connections between the users of SMP agents consisting of two (or more) separate unidirectional (simplex) SMP queues, abstracting away multiple steps required to establish bi-directional connections and any information about the routers location from the users of the agent protocol.
- management of E2E encryption between SMP agents, generating ephemeral asymmetric keys for each connection.
- SMP command authentication on SMP servers, generating ephemeral keys for each SMP queue.
- TCP/TLS transport handshake with SMP servers.
- SMP command authentication on SMP routers, generating ephemeral keys for each SMP queue.
- TCP/TLS transport handshake with SMP routers.
- validation of message integrity.
SMP agent API provides no security between the agent and the client - it is assumed that the agent is executed in the trusted and secure environment, via the agent library, when the agent logic is included directly into the client application - [SimpleX Chat for terminal](https://github.com/simplex-chat/simplex-chat) uses this approach.
This document describes SMP agent protocol version 7. The version history:
- v1: initial version
- v2: duplex handshake - allows including reply queue(s) in the initial confirmation
- v3: ratchet sync - supports re-negotiating double ratchet encryption
- v4: delivery receipts - supports acknowledging message delivery to the sender
- v5: post-quantum - supports post-quantum key exchange in double ratchet (PQDR)
- v6: sender auth key - supports sender authentication key in confirmations
- v7: ratchet on confirmation - initializes double ratchet during confirmation
## SMP agent
SMP agents communicate with each other via SMP servers using [simplex messaging protocol (SMP)](./simplex-messaging.md) according to the API calls used by the client applications. This protocol is a middle layer in SimpleX protocols (above SMP protocol but below any application level protocol) - it is intended to be used by client-side applications that need secure asynchronous bi-directional communication channels ("connections").
SMP agents communicate with each other via SMP routers using [simplex messaging protocol (SMP)](./simplex-messaging.md) according to the API calls used by the client applications. This protocol is a middle layer in SimpleX protocols (above SMP protocol but below any application level protocol) - it is intended to be used by client-side applications that need secure asynchronous bi-directional communication channels ("connections").
The agent must have a persistent storage to manage the states of known connections and of the client-side information of SMP queues that each connection consists of, and also the buffer of the most recent sent and received messages. The number of the messages that should be stored is implementation specific, depending on the error management approach that the agent implements; at the very least the agent must store the hashes and IDs of the last received and sent messages.
## SMP servers management
## SMP routers management
SMP agent API does not use the addresses of the SMP servers that the agent will use to create and use the connections (excluding the server address in queue URIs used in JOIN command). The list of the servers is a part of the agent configuration and can be dynamically changed by the agent implementation:
SMP agent API does not use the addresses of the SMP routers that the agent will use to create and use the connections (excluding the router address in queue URIs used in JOIN command). The list of the routers is a part of the agent configuration and can be dynamically changed by the agent implementation:
- by the client applications via any API that is outside of scope of this protocol.
- by the agents themselves based on availability and latency of the configured servers.
- by the agents themselves based on availability and latency of the configured routers.
## SMP agent protocol scope
SMP agent protocol has 2 main parts:
- the messages that SMP agents exchange with each other in order to:
- negotiate establishing unidirectional (simplex) encrypted queues on SMP servers.
- negotiate establishing unidirectional (simplex) encrypted queues on SMP routers.
- exchange client messages and delivery notifications, providing sequential message IDs and message integrity (by including the hash of the previous message).
- re-negotiate messaging queues to use and connection e2e encryption.
- the messages that the clients of SMP agents should send out-of-band (as pre-shared "invitation" including queue URIs) to protect [E2E encryption][1] from active attacks ([MITM attacks][2]).
@@ -67,40 +85,40 @@ SMP agent protocol has 2 main parts:
![Duplex connection procedure](./diagrams/duplex-messaging/duplex-creating.svg)
The procedure of establishing a duplex connection is explained on the example of Alice and Bob creating a bi-directional connection consisting of two unidirectional (simplex) queues, using SMP agents (A and B) to facilitate it, and two different SMP servers (which could be the same server). It is shown on the diagram above and has these steps:
The procedure of establishing a duplex connection is explained on the example of Alice and Bob creating a bi-directional connection consisting of two unidirectional (simplex) queues, using SMP agents (A and B) to facilitate it, and two different SMP routers (which could be the same router). It is shown on the diagram above and has these steps:
1. Alice requests the new connection from the SMP agent A using agent `createConnection` api function.
2. Agent A creates an SMP queue on the server (using [SMP protocol](./simplex-messaging.md) `NEW` command) and responds to Alice with the invitation that contains queue information and the encryption keys Bob's agent B should use. The invitation format is described in [Connection link](connection-link-1-time-invitation-and-contact-address).
2. Agent A creates an SMP queue on the router (using [SMP protocol](./simplex-messaging.md) `NEW` command) and responds to Alice with the invitation that contains queue information and the encryption keys Bob's agent B should use. The invitation format is described in [Connection link](connection-link-1-time-invitation-and-contact-address).
3. Alice sends the [connection link](#connection-link-1-time-invitation-and-contact-address) to Bob via any secure channel (out-of-band message) - as a link or as a QR code.
4. Bob uses agent `joinConnection` api function with the connection link as a parameter to agent B to accept the connection.
5. Agent B creates Bob's SMP reply queue with SMP server `NEW` command.
6. Agent B confirms the connection: sends an "SMP confirmation" with SMP server `SEND` command to the SMP queue specified in the connection link - SMP confirmation is an unauthenticated message with an ephemeral key that will be used to authenticate Bob's commands to the queue, as described in SMP protocol, and Bob's info (profile, public key for E2E encryption, and the connection link to this 2nd queue to Agent A - this connection link SHOULD use "simplex" URI scheme). This message is encrypted using key passed in the connection link (or with the derived shared secret, in which case public key for key derivation should be sent in clear text).
6. Alice confirms and continues the connection:
- Agent A receives the SMP confirmation containing Bob's key, reply queue and info as SMP server `MSG`.
5. Agent B creates Bob's SMP reply queue with SMP router `NEW` command.
6. Agent B confirms the connection: sends an "SMP confirmation" with SMP router `SEND` command to the SMP queue specified in the connection link - SMP confirmation is an unauthenticated message with an ephemeral key that will be used to authenticate Bob's commands to the queue, as described in SMP protocol, and Bob's info (profile, public key for E2E encryption, and the connection link to this 2nd queue to Agent A - this connection link SHOULD use "simplex" URI scheme). This message is encrypted using key passed in the connection link (or with the derived shared secret, in which case public key for key derivation should be sent in clear text).
7. Alice confirms and continues the connection:
- Agent A receives the SMP confirmation containing Bob's key, reply queue and info as SMP router `MSG`.
- Agent A notifies Alice sending `CONF` notification with Bob's info.
- Alice allows connection to continue with agent `allowConnection` api function.
- Agent A secures the queue with SMP server `KEY` command.
- Agent A secures the queue with SMP router `KEY` command.
- Agent A sends SMP confirmation with ephemeral sender key, ephemeral public encryption key and profile (but without reply queue).
7. Agent B confirms the connection:
8. Agent B confirms the connection:
- receives the confirmation.
- sends the notification `INFO` with Alice's information to Bob.
- secures SMP queue that it sent to Alice in the first confirmation with SMP `KEY` command .
- sends `HELLO` message via SMP `SEND` command. This confirms that the reply queue is secured and also validates that Agent A secured the first SMP queue
8. Agent A notifies Alice.
9. Agent A notifies Alice.
- receives `HELLO` message from Agent B.
- sends `HELLO` message to Agent B via SMP `SEND` command.
- sends `CON` notification to Alice, confirming that the connection is established.
9. Agent B notifies Bob.
10. Agent B notifies Bob.
- Once Agent B receives `HELLO` from Agent A, it sends to Bob `CON` notification as well.
At this point the duplex connection between Alice and Bob is established, they can use `SEND` command to send messages. The diagram also shows how the connection status changes for both parties, where the first part is the status of the SMP queue to receive messages, and the second part - the status of the queue to send messages.
The most communication happens between the agents and servers, from the point of view of Alice and Bob there are 4 steps (not including notifications):
The most communication happens between the agents and routers, from the point of view of Alice and Bob there are 4 steps (not including notifications):
1. Alice requests a new connection with `createConnection` agent API function and receives the connection link.
2. Alice passes connection link out-of-band to Bob.
3. Bob accepts the connection with `joinConnection` agent API function with the connection link to his agent.
4. Alice accepts the connection with `ACPT` agent API function.
4. Alice accepts the connection with `allowConnection` agent API function.
5. Both parties receive `CON` notification once duplex connection is established.
Clients SHOULD support establishing duplex connection asynchronously (when parties are intermittently offline) by persisting intermediate states and resuming SMP queue subscriptions.
@@ -118,14 +136,14 @@ Faster duplex connection process is possible with the `SKEY` command added in v9
![Fast duplex connection procedure](./diagrams/duplex-messaging/duplex-creating-fast.svg)
1. Alice requests the new connection from the SMP agent A using agent `createConnection` api function
2. Agent A creates an SMP queue on the server (using [SMP protocol](./simplex-messaging.md) `NEW` command with the flag allowing the sender to secure the queue) and responds to Alice with the invitation that contains queue information and the encryption keys Bob's agent B should use. The invitation format is described in [Connection link](connection-link-1-time-invitation-and-contact-address).
2. Agent A creates an SMP queue on the router (using [SMP protocol](./simplex-messaging.md) `NEW` command with the flag allowing the sender to secure the queue) and responds to Alice with the invitation that contains queue information and the encryption keys Bob's agent B should use. The invitation format is described in [Connection link](connection-link-1-time-invitation-and-contact-address).
3. Alice sends the [connection link](connection-link-1-time-invitation-and-contact-address) to Bob via any secure channel (out-of-band message) - as a link or as a QR code. This link contains the flag that the queue can be secured by the sender.
4. Bob uses agent `joinConnection` api function with the connection link as a parameter to agent B to accept the connection.
5. Agent B secures Alice's queue with SMP command `SKEY` - this command can be proxied.
6. Agent B creates Bob's SMP reply queue with SMP server `NEW` command (with the flag allowing the sender to secure the queue).
7. Agent B confirms the connection: sends an "SMP confirmation" with SMP server `SEND` command to the SMP queue specified in the connection link - SMP confirmation is an unauthenticated message with an ephemeral key that will be used to authenticate Bob's commands to the queue, as described in SMP protocol, and Bob's info (profile, public key for E2E encryption, and the connection link to this 2nd queue to Agent A - this connection link SHOULD use "simplex" URI scheme). This message is encrypted using key passed in the connection link (or with the derived shared secret, in which case public key for key derivation should be sent in clear text).
6. Agent B creates Bob's SMP reply queue with SMP router `NEW` command (with the flag allowing the sender to secure the queue).
7. Agent B confirms the connection: sends an "SMP confirmation" with SMP router `SEND` command to the SMP queue specified in the connection link - SMP confirmation is an unauthenticated message with an ephemeral key that will be used to authenticate Bob's commands to the queue, as described in SMP protocol, and Bob's info (profile, public key for E2E encryption, and the connection link to this 2nd queue to Agent A - this connection link SHOULD use "simplex" URI scheme). This message is encrypted using key passed in the connection link (or with the derived shared secret, in which case public key for key derivation should be sent in clear text).
8. Alice confirms the connection:
- Agent A receives the SMP confirmation containing Bob's key, reply queue and info as SMP server `MSG`.
- Agent A receives the SMP confirmation containing Bob's key, reply queue and info as SMP router `MSG`.
- Agent A notifies Alice sending `CONF` notification with Bob's info (that indicates that Agent B already secured the queue).
- Alice allows connection to continue with agent `allowConnection` api function.
- Agent A secures Bob's queue with SMP command `SKEY`.
@@ -140,11 +158,11 @@ Faster duplex connection process is possible with the `SKEY` command added in v9
SMP agents support creating a special type of connection - a contact address - that allows to connect to multiple network users who can send connection requests by sending 1-time connection links to the message queue.
This connection address uses a messaging queue on SMP server to receive invitations to connect - see `agentInvitation` message below. Once connection request is accepted, a new connection is created and the address itself is no longer used to send the messages - deleting this address does not disrupt the connections that were created via it.
This connection address uses a messaging queue on SMP router to receive invitations to connect - see `agentInvitation` message below. Once connection request is accepted, a new connection is created and the address itself is no longer used to send the messages - deleting this address does not disrupt the connections that were created via it.
## Communication between SMP agents
To establish duplex connections and to send messages on behalf of their clients, SMP agents communicate via SMP servers.
To establish duplex connections and to send messages on behalf of their clients, SMP agents communicate via SMP routers.
Agents use SMP message client body (the part of the SMP message after header - see [SMP protocol](./simplex-messaging.md)) to transmit agent client messages and exchange messages between each other.
@@ -152,13 +170,13 @@ These messages are encrypted with per-queue shared secret using NaCL crypto_box
- `agentConfirmation` - used when confirming SMP queues, contains connection information encrypted with double ratchet. This envelope can only contain `agentConnInfo` or `agentConnInfoReply` encrypted with double ratchet.
- `agentMsgEnvelope` - contains different agent messages encrypted with double ratchet, as defined in `agentMessage`.
- `agentInvitation` - sent to SMP queue that is used as contact address, does not use double ratchet.
- `agentRatchetKey` - used to re-negotiate double ratchet encryption - can contain additional information in `agentRatchetKey`.
- `agentRatchetKey` - used to re-negotiate double ratchet encryption - can contain additional information in `agentRatchetInfo`.
```abnf
decryptedSMPClientMessage = agentConfirmation / agentMsgEnvelope / agentInvitation / agentRatchetKey
agentConfirmation = agentVersion %s"C" ("0" / "1" sndE2EEncryptionParams) encConnInfo
agentVersion = 2*2 OCTET
sndE2EEncryptionParams = TODO
sndE2EEncryptionParams = <sender E2E ratchet parameters, see pqdr.md>
encConnInfo = doubleRatchetEncryptedMessage
agentMsgEnvelope = agentVersion %s"M" encAgentMessage
@@ -166,13 +184,25 @@ encAgentMessage = doubleRatchetEncryptedMessage
agentInvitation = agentVersion %s"I" connReqLength connReq connInfo
connReqLength = 2*2 OCTET ; Word16
connReq = *OCTET ; URI text encoding of connection link, length given by connReqLength
connInfo = *OCTET ; opaque connection information (remaining bytes)
agentRatchetKey = agentVersion %s"R" rcvE2EEncryptionParams agentRatchetInfo
rcvE2EEncryptionParams = TODO
agentRatchetKey = agentVersion %s"R" rcvE2EEncryptionParams ratchetKeyInfo
rcvE2EEncryptionParams = <receiver E2E ratchet parameters, see pqdr.md>
ratchetKeyInfo = *OCTET ; additional ratchet renegotiation info (remaining bytes)
doubleRatchetEncryptedMessage = TODO
doubleRatchetEncryptedMessage = <double ratchet encrypted message, see pqdr.md>
```
The maximum size of the encrypted connection info and agent message depend on whether post-quantum key exchange is used:
| Constant | PQ on | PQ off |
|----------|-------|--------|
| `e2eEncConnInfoLength` | 11106 | 14832 |
| `e2eEncAgentMsgLength` | 13618 | 15840 |
The PQ-on sizes are smaller because the ratchet header and reply link include larger PQ keys (SNTRUP761).
This syntax of decrypted SMP client message body is defined by `decryptedAgentMessage` below.
Decrypted SMP message client body can be one of 4 types:
@@ -182,14 +212,15 @@ Decrypted SMP message client body can be one of 4 types:
- `agentMessage` - all other agent messages.
`agentMessage` contains these parts:
- `agentMsgHeader` - agent message header that contains sequential agent message ID for a particular SMP queue, agent timestamp (ISO8601) and the hash of the previous message.
- `agentMsgHeader` - agent message header that contains sequential agent message ID for a particular SMP queue and the hash of the previous message.
- `aMessage` - a command/message to the other SMP agent:
- to confirm the connection (`HELLO`).
- to send and to confirm reception of user messages (`A_MSG`, `A_RCVD`).
- to confirm that the new double ratchet encryption is agreed (`EREADY`).
- to notify another party that it can continue sending messages after queue capacity was exceeded (`A_QCONT`).
- to manage SMP queue rotation (`QADD`, `QKEY`, `QUSE`, `QTEST`).
- `msgPadding` - an optional message padding to make all SMP messages have constant size, to prevent servers from observing the actual message size. The only case the message padding can be absent is when the message has exactly the maximum size, in all other cases the message MUST be padded to a fixed size.
The encoded `agentMessage` is padded to a fixed size by the double ratchet encryption layer (see [ratchet message wire format](./pqdr.md#ratchet-message-wire-format)) to make all SMP messages have constant size, preventing routers from observing the actual message size.
### Messages between SMP agents
@@ -200,9 +231,11 @@ decryptedAgentMessage = agentConnInfo / agentConnInfoReply / agentRatchetInfo /
agentConnInfo = %s"I" connInfo
connInfo = *OCTET
agentConnInfoReply = %s"D" smpQueues connInfo
smpQueues = length 1*newQueueInfo ; NonEmpty list of reply queues
agentRatchetInfo = %s"R" ratchetInfo
ratchetInfo = *OCTET
agentMessage = %s"M" agentMsgHeader aMessage msgPadding
agentMessage = %s"M" agentMsgHeader aMessage
agentMsgHeader = agentMsgId prevMsgHash
agentMsgId = 8*8 OCTET ; Int64
prevMsgHash = shortString
@@ -213,10 +246,13 @@ aMessage = HELLO / A_MSG / A_RCVD / EREADY / A_QCONT /
HELLO = %s"H"
A_MSG = %s"M" userMsgBody
userMsgBody = *OCTET
userMsgBody = *OCTET ; remaining bytes
A_RCVD = %s"V" msgReceipt
A_RCVD = %s"V" msgReceipts
msgReceipts = length 1*msgReceipt ; NonEmpty list
msgReceipt = agentMsgId msgHash rcptLength rcptInfo
msgHash = shortString
rcptInfo = *OCTET ; opaque receipt info, length given by rcptLength (Word16)
EREADY = %s"E" agentMsgId
@@ -224,14 +260,14 @@ A_QCONT = %s"QC" sndQueueAddr
QADD = %s"QA" sndQueues
sndQueues = length 1*(newQueueUri replacedSndQueue)
newQueueUri = clientVRange smpServer senderId dhPublicKey [sndSecure]
newQueueUri = clientVRange smpRouter senderId dhPublicKey [queueMode]
dhPublicKey = length x509encoded
sndSecure = "T"
queueMode = %s"M" / %s"C" ; M - messaging (sender can secure), C - contact
replacedSndQueue = "0" / "1" sndQueueAddr
QKEY = %s"QK" sndQueueKeys
sndQueueKeys = length 1*(newQueueInfo senderKey)
newQueueInfo = version smpServer senderId dhPublicKey [sndSecure]
newQueueInfo = version smpRouter senderId dhPublicKey [queueMode]
senderKey = length x509encoded
QUSE = %s"QU" sndQueuesReady
@@ -241,8 +277,8 @@ primary = %s"T" / %s"F"
QTEST = %s"QT" sndQueueAddrs
sndQueueAddrs = length 1*sndQueueAddr
sndQueueAddr = smpServer senderId
smpServer = hosts port keyHash
sndQueueAddr = smpRouter senderId
smpRouter = hosts port keyHash
hosts = length 1*host
host = shortString
port = shortString
@@ -252,7 +288,6 @@ senderId = shortString
clientVRange = version version
version = 2*2 OCTET
msgPadding = *OCTET
rcptLength = 2*2 OCTET
shortString = length *OCTET
length = 1*1 OCTET
@@ -266,11 +301,11 @@ This message is not used with [fast duplex connection](#fast-duplex-connection-p
#### A_MSG message
This is the agent envelope used to send client messages once the connection is established. This is different from the MSG sent by SMP server to the agent and MSG event from SMP agent to the client that are sent in different contexts.
This is the agent envelope used to send client messages once the connection is established. This is different from the MSG sent by SMP router to the agent and MSG event from SMP agent to the client that are sent in different contexts.
#### A_RCVD message
This message is sent to confirm the client message reception. It includes received message number and message hash.
This message is sent to confirm the client message reception. It includes a list of message receipts, each containing the received message number, message hash and receipt info.
#### EREADY message
@@ -282,7 +317,7 @@ This message is sent to notify the sender client that it can continue sending th
### Rotating messaging queue
SMP agents SHOULD support 4 messages to rotate message reception to another messaging server:
SMP agents SHOULD support 4 messages to rotate message reception to another messaging router:
`QADD`: add the new queue address(es) to the connection - sent by the client that initiates rotation.
`QKEY`: pass sender's key via existing connection (SMP confirmation message will not be used, to avoid the same "race" of the initial key exchange that would create the risk of intercepting the queue for the attacker) - sent by the client accepting the rotation
`QUSE`: instruct the sender to use the new queue with sender's queue ID as parameter. From this point some messages can be sent to both the new queue and the old queue.
@@ -345,31 +380,191 @@ To summarize, the upgrade to DH+KEM secret happens in a sent message that has PQ
Connection links are generated by SMP agent in response to `createConnection` api call, used by another party user with `joinConnection` api, and then another connection link is sent by the agent in `agentConnInfoReply` and used by the first party agent to connect to the reply queue (the second part of the process is invisible to the users).
Connection link syntax:
### Full connection link syntax
```
connectionLink = connectionScheme "/" connLinkType "#/?smp=" smpQueues "&e2e=" e2eEncryption
connectionLink = connectionScheme "/" connLinkType "#/?v=" versionRange "&smp=" smpQueues ["&e2e=" e2eEncryption] ["&data=" clientData]
connLinkType = %s"invitation" / %s"contact"
connectionScheme = (%s"https://" clientAppServer) | %s"simplex:"
connectionScheme = (%s"https://" clientAppServer) / %s"simplex:"
clientAppServer = hostname [ ":" port ]
; client app server, e.g. simplex.chat
e2eEncryption = encryptionScheme ":" publicKey
encryptionScheme = %s"rsa" ; end-to-end encryption and key exchange protocols,
; the current hybrid encryption scheme (RSA-OAEP/AES-256-GCM-SHA256)
; will be replaced with double ratchet protocol and DH key exchange.
publicKey = <base64url X509 SPKI key encoding>
smpQueues = smpQueue [ "," 1*smpQueue ] ; SMP queues for the connection
versionRange = 1*DIGIT / 1*DIGIT "-" 1*DIGIT ; agent version range
e2eEncryption = <e2e encryption parameters for double ratchet>
smpQueues = smpQueue *(";" smpQueue) ; SMP queues for the connection (semicolon-separated)
smpQueue = <URL-encoded queueURI defined in SMP protocol>
clientData = <URL-encoded application-specific data>
```
All parameters are passed via URI hash to avoid sending them to the server (in case "https" scheme is used) - they can be used by the client-side code and processed by the client application. Parameters `smp` and `e2e` can be present in any order, any unknown additional parameters SHOULD be ignored.
All parameters are passed via URI hash to avoid sending them to the router (in case "https" scheme is used) - they can be used by the client-side code and processed by the client application. Parameters can be present in any order, any unknown additional parameters SHOULD be ignored.
`clientAppServer` is not an SMP server - it is a server that shows the instruction on how to download the client app that will connect using this connection link. This server can also host a mobile or desktop app manifest so that this link is opened directly in the app if it is installed on the device.
`clientAppServer` is not an SMP router - it is a server that shows the instruction on how to download the client app that will connect using this connection link. This server can also host a mobile or desktop app manifest so that this link is opened directly in the app if it is installed on the device.
"simplex" URI scheme in `connectionProtocol` can be used instead of client app server, to connect without creating any web traffic. Client apps MUST support this URI scheme.
"simplex" URI scheme in `connectionProtocol` can be used instead of client app router, to connect without creating any web traffic. Client apps MUST support this URI scheme.
See SMP protocol [out-of-band messages](./simplex-messaging.md#out-of-band-messages) for syntax of `queueURI`.
### Short connection link syntax
Short links provide a more compact representation by storing connection data on the router:
```
shortLink = shortLinkScheme "/" linkType "#" [linkId "/"] linkKey ["?" shortLinkParams]
shortLinkScheme = %s"simplex:" / (%s"https://" serverHost)
linkType = %s"i" / contactType ; i - invitation, or contact type
contactType = %s"a" / %s"c" / %s"g" / %s"r" ; a - contact, c - channel, g - group, r - relay
linkId = base64url ; only for invitation links
linkKey = base64url ; SHA3-256 hash of fixed data, used to decrypt link data
shortLinkParams = hostParam ["&" portParam] ["&" keyHashParam]
hostParam = %s"h=" hostList
hostList = host *("," host)
portParam = %s"p=" port
keyHashParam = %s"c=" base64url ; router certificate fingerprint
```
Contact types:
- `a` (CCTContact) - direct contact connection
- `c` (CCTChannel) - channel connection
- `g` (CCTGroup) - group connection
- `r` (CCTRelay) - relay connection
Short links can use either the `simplex:` scheme or `https://` with a router hostname. When using the simplex scheme, router information is included in query parameters.
## Short links
Short links provide a compact representation of connection links by storing encrypted connection data on the SMP router. The link key in the URI fragment (after `#`) is never sent to the router, ensuring the router cannot decrypt the stored connection data.
### Link key derivation
The link key is derived from the fixed link data using SHA3-256 hash function:
```
linkKey = SHA3-256(fixedLinkData)
```
The fixed link data includes:
- Agent version range
- Root public key (Ed25519) for signing
- SMP queue connection request (router, queue IDs, encryption keys)
- Optional link entity ID
For contact links, the link ID and encryption key are derived from the link key using HKDF:
```
(linkId, encryptionKey) = HKDF(info="SimpleXContactLink", key=linkKey, outputLen=56)
; linkId = first 24 bytes, encryptionKey = remaining 32 bytes
```
For invitation links, the link ID is stored separately (usually included in the URI), and only the encryption key is derived:
```
encryptionKey = HKDF(info="SimpleXInvLink", key=linkKey, outputLen=32)
```
### Link data encryption
Link data stored on the router consists of two encrypted parts: fixed data and user data. Both are encrypted using NaCl secret_box (XSalsa20-Poly1305) with the derived encryption key:
```abnf
queueLinkData = encFixedData encUserData
encFixedData = largeString ; encrypted padded(signedFixedData, 2008)
encUserData = largeString ; encrypted padded(signedUserData, 13784)
signedFixedData = signature fixedData
signedUserData = signature userData
signature = length 64*64 OCTET ; Ed25519 signature
fixedData = agentVersionRange rootKey linkConnReq [linkEntityId]
agentVersionRange = version version ; min and max agent protocol version
version = 2*2 OCTET
rootKey = length x509encoded ; Ed25519 public key
linkConnReq = invitationConnReq / contactConnReq ; binary encoding of connection request
invitationConnReq = %s"I" connReqData e2eRatchetParams
contactConnReq = %s"C" connReqData
linkEntityId = shortString
userData = invitationLinkData / contactLinkData
invitationLinkData = %s"I" agentVersionRange userLinkData
contactLinkData = %s"C" agentVersionRange userContactData
userLinkData = shortString / (%xFF largeString) ; opaque application data (e.g., user profile)
; shortString length byte 0x00-0xFE (max 254 bytes); 0xFF is reserved as largeString sentinel
userContactData = direct ownersList relaysList userLinkData
direct = %s"T" / %s"F" ; whether direct connection via connReq is allowed
ownersList = length *ownerAuth
ownerAuth = shortString ; length-prefixed encoding of (ownerId ownerKey authOwnerSig)
ownerId = shortString ; application-specific owner ID (e.g., MemberId)
ownerKey = length x509encoded ; Ed25519 public key
authOwnerSig = length 64*64 OCTET ; Ed25519 signature of (ownerId || ownerKey) by previous owner
relaysList = length *connShortLink ; alternative relay short links
; Binary encoding of connection request (used in linkConnReq)
connReqData = agentVersionRange smpQueueUris clientData
smpQueueUris = length 1*smpQueueUri
clientData = %s"0" / (%s"1" largeString) ; Maybe (Large ByteString)
smpQueueUri = smpClientVersionRange smpServer senderId smpDhPublicKey [queueMode]
smpClientVersionRange = version version ; min and max SMP client versions
smpServer = hosts port serverKeyHash
hosts = length 1*host
host = shortString ; text-encoded hostname or IP address
port = shortString ; text-encoded port number
serverKeyHash = shortString ; CA certificate fingerprint
senderId = shortString ; queue sender ID
smpDhPublicKey = length x509encoded ; X25519 DH public key
queueMode = %s"M" / %s"C" ; messaging or contact (version-dependent trailing field)
e2eRatchetParams = e2eVersionRange e2eDhKey e2eDhKey kemParams
e2eVersionRange = version version ; min and max e2e encryption versions
e2eDhKey = length x509encoded ; X448 DH public key
kemParams = %s"0" / (%s"1" ratchetKEMParams)
ratchetKEMParams = %s"P" kemPublicKey / %s"A" kemCiphertext kemPublicKey
kemPublicKey = largeString ; sntrup761 public key
kemCiphertext = largeString ; sntrup761 ciphertext
; Binary encoding of short link (used in relaysList)
connShortLink = invShortLink / contactShortLink
invShortLink = %s"I" smpServer linkId linkKey
contactShortLink = %s"C" contactConnType smpServer linkKey
contactConnType = %s"A" / %s"C" / %s"G" / %s"R" ; contact / channel / group / relay
linkId = shortString
linkKey = shortString
x509encoded = *OCTET ; DER-encoded X.509 SubjectPublicKeyInfo
largeString = 2*2 OCTET *OCTET ; Word16 length prefix
length = 1*1 OCTET
shortString = length *OCTET
```
The fixed data is signed with the root key and its hash becomes the link key. The user data is signed either with the root key (for invitations) or with an owner key (for contact addresses).
### Short link resolution
When a user receives a short link, the agent resolves it as follows:
1. Extract the link key from the URI fragment
2. Send `LGET` command to the SMP router with the link ID
3. Receive encrypted link data from the router
4. Decrypt the link data using the link key
5. Extract the full connection information (SMP queue URI, encryption keys, profile)
6. Proceed with the standard connection procedure using `joinConnection`
For invitation links, the `LKEY` command is used to set the sender key when getting link data. Repeated `LKEY` would require using the same key.
### Link data management
The recipient who created the queue can manage the short link data:
- **LSET** - Set or update the link data associated with a queue. This is used when creating a short link or updating the user data (e.g., profile changes).
- **LDEL** - Delete the link data from the router. This effectively invalidates the short link.
Short links support different connection modes:
- **invitation** - One-time invitation links that can only be used once
- **contact** - Reusable contact address links that can be used multiple times
For contact addresses, the link data includes additional information about the contact type:
- **contact** - Direct contact connection
- **channel** - Channel connection
- **group** - Group connection
- **relay** - Relay connection
The agent maintains the link data and updates it when connection parameters change, ensuring short links remain valid and reflect current connection information.
## Appendix A: SMP agent API
The exact specification of agent library API and of the events that the agent sends to the client application is out of scope of the protocol specification.
@@ -380,7 +575,7 @@ The list of some of the API functions and events below is supported by the refer
The list of APIs below is not exhaustive and provided for information only. Please consult the source code for more information.
#### Create conection
#### Create connection
`createConnection` api is used to create a connection - it returns the connection link that should be sent out-of-band to another protocol user (the joining party). It should be used by the client of the agent that initiates creating a duplex connection (the initiating party).
@@ -408,13 +603,13 @@ Client can `acceptContact` and `rejectContact`, with `OK` and `ERR` events in ca
#### Send message
`sendMessage` api is always asynchronous. The api call returns message ID, `SENT` event once the message is sent to the server, `MWARN` event in case of temporary delivery failure that can be resolved by the user (e.g., by connecting via Tor or by upgrading the client) and `MERR` in case of permanent delivery failure.
`sendMessage` api is always asynchronous. The api call returns message ID, `SENT` event once the message is sent to the router, `MWARN` event in case of temporary delivery failure that can be resolved by the user (e.g., by connecting via Tor or by upgrading the client) and `MERR` in case of permanent delivery failure.
#### Acknowledge received message
Messages are delivered to the client application via `MSG` event.
Client application must always `ackMessage` to receive the next one - failure to call it in reference implementation will prevent the delivery of subsequent messages until the client reconnects to the server.
Client application must always `ackMessage` to receive the next one - failure to call it in reference implementation will prevent the delivery of subsequent messages until the client reconnects to the router.
This api is also used to acknowledge message delivery to the sending party - that party client application will receive `RCVD` event.
@@ -426,9 +621,17 @@ This api is also used to acknowledge message delivery to the sending party - tha
`getNotificationMessage` is used by push notification subsystem of the client application to receive the message from a specific messaging queue mentioned in the notification. The client application would receive `MSG` and any other events from the agent, and then `MSGNTF` event once the message related to this notification is received.
#### Rotate message queue to another server
#### Set short link data
`switchConnection` api is used to rotate connection queues to another messaging server.
`setConnectionLink` api (`LSET` command) is used to set or update short link data associated with a contact address queue. Returns `LINK` event with the short link URI.
#### Get short link data
`getConnectionLink` api (`LGET` command) is used to retrieve and decrypt the short link data from the router. Returns `LDATA` event with the decrypted link data.
#### Rotate message queue to another router
`switchConnection` api is used to rotate connection queues to another messaging router.
#### Renegotiate e2e encryption
@@ -436,7 +639,7 @@ This api is also used to acknowledge message delivery to the sending party - tha
#### Delete connection
`deleteConnection` api is used to delete connection. In case of asynchronous call, the connection deletion will be confirmed with `DEL_RCVQ` and `DEL_CONN` events.
`deleteConnection` api is used to delete connection. In case of asynchronous call, the connection deletion will be confirmed with `DEL_RCVQS` and `DEL_CONNS` events.
#### Suspend connection
@@ -451,25 +654,80 @@ Agent API uses these events dispatch to notify client application about events r
- `INFO` - information from the party that initiated the connection with `createConnection` sent to the party accepting the connection with `joinConnection`.
- `CON` - notification that connection is established sent to both parties of the connection.
- `END` - notification that connection subscription is terminated when another client subscribed to the same messaging queue.
- `DOWN` - notification that connection server is temporarily unavailable.
- `UP` - notification that the subscriptions made in the current client session are resumed after the server became available.
- `DOWN` - notification that connection router is temporarily unavailable.
- `UP` - notification that the subscriptions made in the current client session are resumed after the router became available.
- `SWITCH` - notification about queue rotation process.
- `RSYNC` - notification about e2e encryption re-negotiation process.
- `SENT` - notification to confirm that the message was delivered to at least one of SMP servers. This notification contains the same message ID as returned to `sendMessage` api. `SENT` notification, depending on network availability, can be sent at any time later, potentially in the next client session.
- `SENT` - notification to confirm that the message was delivered to at least one of SMP routers. This notification contains the same message ID as returned to `sendMessage` api. `SENT` notification, depending on network availability, can be sent at any time later, potentially in the next client session.
- `MWARN` - temporary delivery failure that can be resolved by the user (e.g., by connecting via Tor or by upgrading the client).
- `MERR` - notification about permanent message delivery failure.
- `MERRS` - notification about permanent message delivery failure for multiple messages (e.g., when multiple messages expire).
- `MSG` - sent when agent receives the message from the SMP server.
- `MSG` - sent when agent receives the message from the SMP router.
- `MSGNTF` - sent after agent received and processed the message referenced in the push notification.
- `RCVD` - notification confirming message receipt by another party.
- `QCONT` - notification that the agent continued sending messages after queue capacity was exceeded and recipient received all messages.
- `DEL_RCVQ` - confirmation that message queue was deleted.
- `DEL_CONN` - confirmation that connection was deleted.
- `LINK` - short link URI created or updated for a contact address.
- `LDATA` - decrypted short link data received from the router.
- `DELD` - notification that the connection was deleted.
- `JOINED` - notification that a member joined via a contact address.
- `STAT` - connection statistics event.
- `DEL_RCVQS` - confirmation that receiver message queues were deleted.
- `DEL_CONNS` - confirmation that connections were deleted.
- `OK` - confirmation that asynchronous api call was successful.
- `ERR` - error of asynchronous api call or some other error event.
This list of events is not exhaustive and provided for information only. Please consult the source code for more information.
## Threat model
This threat model complements SimpleX Messaging Protocol [threat model](./security.md#threat-model) with agent-level concerns: duplex connections, end-to-end encryption with [post-quantum double ratchet](./pqdr.md), message integrity, connection establishment and queue rotation. Only additional properties not covered in the SMP threat model are listed below.
#### Additional global assumptions
- The connection link is shared via a trusted out-of-band channel.
- Both agents support post-quantum double ratchet (PQDR).
#### A passive adversary
*cannot:*
- learn the contents of packets, which are additionally encrypted with the double ratchet independently from per-queue encryption.
#### Destination router (chosen by the receiving client application)
*can:*
- correlate queues belonging to the same duplex connection when queue rotation creates a new queue on the same router.
- when both peers of a connection chose the same router, correlate the two directions of the duplex connection.
*cannot:*
- compromise end-to-end encryption even with full access to the per-queue NaCl DH secret.
- correlate queues belonging to the same connection after queue rotation to a different router.
#### An attacker who obtained a client application's (decrypted) database
*can:*
- learn the full communication graph: all communication peers, associated router addresses, and queue identifiers.
*cannot:*
- decrypt future messages once the client application resumes communication and the double ratchet completes a new ratchet step, provided PQDR is active.
#### A communication peer
*can:*
- send malformed agent messages that may affect the client application processing them.
- skip message IDs, causing the recipient to generate and store excessive intermediate ratchet keys.
- prevent double ratchet advancement by not sending messages, delaying break-in recovery.
*cannot:*
- disrupt packet delivery in other queues.
#### An attacker who obtained a connection link
*can:*
- learn the initiating party's chosen router address and public keys.
*cannot:*
- use the link after the intended recipient has completed the connection.
[1]: https://en.wikipedia.org/wiki/End-to-end_encryption
[2]: https://en.wikipedia.org/wiki/Man-in-the-middle_attack
[3]: https://tools.ietf.org/html/rfc5234
+86 -227
View File
@@ -1,4 +1,4 @@
Revision 2, 2024-06-22
Revision 4, 2026-03-09
Evgeny Poberezkin
@@ -8,16 +8,17 @@ Evgeny Poberezkin
- [Introduction](#introduction)
- [What is SimpleX](#what-is-simplex)
- [Network model](#network-model)
- [Applications](#applications)
- [SimpleX objectives](#simplex-objectives)
- [In Comparison](#in-comparison)
- [Technical Details](#technical-details)
- [Trust in Servers](#trust-in-servers)
- [Client -> Server Communication](#client---server-communication)
- [Trust in Routers](#trust-in-routers)
- [Client -> Router Communication](#client---router-communication)
- [2-hop Onion Message Routing](#2-hop-onion-message-routing)
- [SimpleX Messaging Protocol](#simplex-messaging-protocol)
- [SimpleX Agents](#simplex-agents)
- [Encryption Primitives Used](#encryption-primitives-used)
- [Threat model](#threat-model)
- [Security](#security)
- [Acknowledgements](#acknowledgements)
@@ -27,27 +28,27 @@ Evgeny Poberezkin
SimpleX as a whole is a platform upon which applications can be built. [SimpleX Chat](https://github.com/simplex-chat/simplex-chat) is one such application that also serves as an example and reference application.
- [SimpleX Messaging Protocol](./simplex-messaging.md) (SMP) is a protocol to send messages in one direction to a recipient, relying on a server in-between. The messages are delivered via uni-directional queues created by recipients.
- SMP protocol allows to send message via a SMP server playing proxy role using 2-hop onion routing (referred to as "private routing" in messaging clients) to protect transport information of the sender (IP address and session) from the server chosen (and possibly controlled) by the recipient.
- [SimpleX Messaging Protocol](./simplex-messaging.md) (SMP) is a protocol to send messages in one direction to a recipient, relying on a router in-between. The messages are delivered via uni-directional queues created by recipients.
- SMP protocol allows to send message via a SMP router playing proxy role using 2-hop onion routing (referred to as "private routing" in messaging clients) to protect transport information of the sender (IP address and session) from the router chosen (and possibly controlled) by the recipient.
- SMP runs over a transport protocol (shown below as TLS) that provides integrity, server authentication, confidentiality, and transport channel binding.
- A SimpleX Server is one of those servers.
- A SimpleX router is one of those routers.
- The SimpleX Network is the term used for the collective of SimpleX Servers that facilitate SMP.
- The SimpleX Network is the term used for the collective of SimpleX routers that facilitate SMP.
- SimpleX Client libraries speak SMP to SimpleX Servers and provide a low-level API not generally intended to be used by applications.
- SimpleX Client libraries speak SMP to SimpleX routers and provide a low-level API not generally intended to be used by applications.
- SimpleX Agents interface with SimpleX Clients to provide a more high-level API intended to be used by applications. Typically they are embedded as libraries, but can also be abstracted into local services.
- SimpleX Agents communicate with other agents inside e2e encrypted envelopes provided by SMP protocol - the syntax and semantics of the messages exchanged by the agent are defined by [SMP agent protocol](./agent-protocol.md)
*Diagram showing the SimpleX Chat app, with logical layers of the chat application interfacing with a SimpleX Agent library, which in turn interfaces with a SimpleX Client library. The Client library in turn speaks the Messaging Protocol to a SimpleX Server.*
*Diagram showing the SimpleX Chat app, with logical layers of the chat application interfacing with a SimpleX Agent library, which in turn interfaces with a SimpleX Client library. The Client library in turn speaks the Messaging Protocol to a SimpleX router.*
```
User's Computer Internet Third-Party Server
User's Computer Internet Third-Party Router
------------------ | ---------------------- | -------------------------
| |
SimpleX Chat | |
@@ -57,11 +58,43 @@ SimpleX as a whole is a platform upon which applications can be built. [SimpleX
+----------------+ | |
| SimpleX Agent | | |
+----------------+ -------------- TLS ---------------- +----------------+
| SimpleX Client | ------ SimpleX Messaging Protocol ------> | SimpleX Server |
| SimpleX Client | ------ SimpleX Messaging Protocol ------> | SimpleX router |
+----------------+ ----------------------------------- +----------------+
| |
```
#### Network model
SimpleX is a general-purpose packet routing network built on top of the Internet. Network endpoints — end-user devices, automated services, AI-enabled applications, IoT devices — exchange data packets through SimpleX network nodes (SMP routers), which accept, buffer, and deliver packets. Each router operates independently and can be operated by any party on standard computing hardware.
SimpleX routers use resource-based addressing: each address identifies a resource on a router, similar to how the World Wide Web addresses resources via URLs. Internet routers, by comparison, use endpoint-based addressing, where IP addresses identify destination devices. Because of this design, SimpleX network participants do not need globally unique addresses to communicate.
SimpleX network has two resource-based addressing schemes:
- *Messaging queues* ([SMP](./simplex-messaging.md)). A queue is a unidirectional, ordered sequence of fixed-size data packets (16,384 bytes each). Each queue has a resource address on a specific router, gated by cryptographic credentials that separately authorize sending and receiving.
- *Data packets* ([XFTP](./xftp.md)). A data packet is an individually addressed block in one of the standard sizes. Each packet has a unique resource address on a specific router, gated by cryptographic credentials. Data packet addressing is more efficient for delivery of larger payloads than queues.
Packet delivery follows a two-router path. The sending endpoint submits a packet to a first router, which forwards it to a second router, where the receiving endpoint retrieves it. The sending endpoint's IP address is known only to the first router; the receiving endpoint's IP address is known only to the second router. See [2-hop Onion Message Routing](#2-hop-onion-message-routing) for details.
Routers buffer packets between submission and retrieval — from seconds to days, enabling asynchronous delivery when endpoints are online at different times. Packets are removed after delivery or after a configured expiration period.
#### Applications
Applications currently using SimpleX network:
- **SimpleX Chat** — a peer-to-peer messenger using SimpleX network as a transport layer, in the same way that communication applications use WebRTC, Tor, i2p, or Nym. All communication logic — contacts, conversations, groups, message formats, end-to-end encryption — runs on endpoint devices.
- **IoT devices** — using the SimpleX queue protocol directly for sensor data collection and device control.
- **AI-based services** — automated services built on the SimpleX Chat application core.
- **Secure monitoring and control systems** — applications for equipment monitoring and control, including robotics, using the network for command delivery and telemetry collection.
[SimpleGo](https://simplego.dev), developed by an independent organization, is a microcontroller-based device running a SimpleX Chat-compatible messenger directly on a microcontroller without a general-purpose operating system. Running over 20 days on a single battery charge, it demonstrates the energy efficiency of resource-based addressing: the device receives packets without continuous polling. A microcontroller-based router implementation that functions simultaneously as a WiFi router is also in development.
#### SimpleX objectives
1. Provide messaging infrastructure for distributed applications. This infrastructure needs to have the following qualities:
@@ -70,7 +103,7 @@ SimpleX as a whole is a platform upon which applications can be built. [SimpleX
- Privacy: protect against traffic correlation attacks to determine the contacts that the users communicate with.
- Reliability: the messages should be delivered even if some participating network servers or receiving clients fail, with at least once delivery guarantee.
- Reliability: the messages should be delivered even if some participating network routers or receiving clients fail, with "at least once" delivery guarantee.
- Integrity: the messages sent in one direction are ordered in a way that sender and recipient agree on; the recipient can detect when a message was removed or changed.
@@ -78,63 +111,63 @@ SimpleX as a whole is a platform upon which applications can be built. [SimpleX
- Low latency: the delay introduced by the network should not be higher than 100ms-1s in addition to the underlying TCP network latency.
2. Provide better communication security and privacy than the alternative instant messaging solutions. In particular SimpleX provides better privacy of metadata (who talks to whom and when) and better security against active network attackers and malicious servers.
2. Provide better communication security and privacy than the alternative instant messaging solutions. In particular SimpleX provides better privacy of metadata (who talks to whom and when) and better security against active network attackers and malicious routers.
3. Balance user experience with privacy requirements, prioritizing experience of mobile device users.
#### In Comparison
SimpleX network has a design similar to P2P networks, but unlike most P2P networks it consists of clients and servers without depending on any centralized component.
SimpleX network has a design similar to P2P networks, but unlike most P2P networks it consists of clients and routers without depending on any centralized component.
In comparison to more traditional messaging applications (e.g. WhatsApp, Signal, Telegram) the key differences of SimpleX network are:
- participants do not need to have globally unique addresses to communicate, instead they use redundant unidirectional (simplex) messaging queues, with a separate set of queues for each contact.
- connection requests are passed out-of-band, non-optionally protecting key exchange against man-in-the-middle attack.
- simple message queues provided by network servers are used by the clients to create more complex communication scenarios, such as duplex one-to-one communication, transmitting files, group communication without central servers, and content/communication channels.
- simple message queues provided by network routers are used by the clients to create more complex communication scenarios, such as duplex one-to-one communication, transmitting files, group communication without central routers, and content/communication channels.
- servers do not store any user information (no user profiles or contacts, or messages once they are delivered), and primarily use in-memory persistence.
- routers do not store any user information (no user profiles or contacts, or messages once they are delivered), and primarily use in-memory persistence.
- users can change servers with minimal disruption - even after an in-use server disappears, simply by changing the configuration on which servers the new queues are created.
- users can change routers with minimal disruption - even after an in-use router disappears, simply by changing the configuration on which routers the new queues are created.
## Technical Details
#### Trust in Servers
#### Trust in Routers
Clients communicate directly with servers (but not with other clients) using SimpleX Messaging Protocol (SMP) running over some transport protocol that provides integrity, server authentication, confidentiality, and transport channel binding. By default, we assume this transport protocol is TLS.
Clients communicate directly with routers (but not with other clients) using SimpleX Messaging Protocol (SMP) running over some transport protocol that provides integrity, server authentication, confidentiality, and transport channel binding. By default, we assume this transport protocol is TLS.
Users use multiple servers, and choose where to receive their messages. Accordingly, they send messages to their communication partners' chosen servers either directly, if this is a known/trusted server, or via another SMP server providing proxy functionality to protect IP address and session of the sender.
Users use multiple routers, and choose where to receive their messages. Accordingly, they send messages to their communication partners' chosen routers either directly, if this is a known/trusted router, or via another SMP router providing proxy functionality to protect IP address and session of the sender.
Although end-to-end encryption is always present, users place a degree of trust in servers they connect to. This trust decision is very similar to a user's choice of email provider; however the trust placed in a SimpleX server is significantly less. Notably, there is no re-used identifier or credential between queues on the same (or different) servers. While a user *may* re-use a transport connection to fetch messages from multiple queues, or connect to a server from the same IP address, both are choices a user may opt into to break the promise of un-correlatable queues.
Although end-to-end encryption is always present, users place a degree of trust in routers they connect to. This trust decision is very similar to a user's choice of email provider; however the trust placed in a SimpleX router is significantly less. Notably, there is no re-used identifier or credential between queues on the same (or different) routers. While a user *may* re-use a transport connection to fetch messages from multiple queues, or connect to a router from the same IP address, both are choices a user may opt into to break the promise of un-correlatable queues.
Users may trust a server because:
Users may trust a router because:
- They deploy and control the servers themselves from the available open-source code. This has the trade-offs of strong trust in the server but limited metadata obfuscation to a passive network observer. Techniques such as noise traffic, traffic mixing (incurring latency), and using an onion routing transport protocol can mitigate that.
- They deploy and control the routers themselves from the available open-source code. This has the trade-offs of strong trust in the router but limited metadata obfuscation to a passive network observer. Techniques such as noise traffic, traffic mixing (incurring latency), and using an onion routing transport protocol can mitigate that.
- They use servers from a trusted commercial provider. The more clients the provider has, the less metadata about the communication times is leaked to the network observers.
- They use routers from a trusted commercial provider. The more clients the provider has, the less metadata about the communication times is leaked to the network observers.
By default, servers do not retain access logs, and permanently delete messages and queues when requested. Messages persist only in memory until they cross a threshold of time, typically on the order of days.[0] There is still a risk that a server maliciously records all queues and messages (even though encrypted) sent via the same transport connection to gain a partial knowledge of the users communications graph and other meta-data.
By default, routers do not retain access logs, and permanently delete messages and queues when requested. Messages persist in memory or in a database until they cross a threshold of time, typically on the order of days.[0] There is still a risk that a router maliciously records all queues and messages (even though encrypted) sent via the same transport connection to gain a partial knowledge of the user's communications graph and other meta-data.
SimpleX supports measures (managed transparently to the user at the agent level) to mitigate the trust placed in servers. These include rotating the queues in use between users, noise traffic, supporting overlay networks such as Tor, and isolating traffic to different queues to different transport connections (and Tor circuits, if Tor is used).
SimpleX supports measures (managed transparently to the user at the agent level) to mitigate the trust placed in routers. These include rotating the queues in use between users, noise traffic, supporting overlay networks such as Tor, and isolating traffic to different queues to different transport connections (and Tor circuits, if Tor is used).
[0] While configurable by servers, a minimum value is enforced by the default software. SimpleX Agents can provide redundant routing over queues to mitigate against message loss.
[0] While configurable by routers, a minimum value is enforced by the default software. SimpleX Agents can provide redundant routing over queues to mitigate against message loss.
#### Client -> Server Communication
#### Client -> Router Communication
Utilizing TLS grants the SimpleX Messaging Protocol (SMP) server authentication and metadata protection to a passive network observer. But SMP does not rely on the transport protocol for message confidentiality or client authentication. The SMP protocol itself provides end-to-end confidentiality, authentication, and integrity of messages between communicating parties.
Servers have long-lived, self-signed, offline certificates whose hash is pre-shared with clients over secure channels - either provided with the client library or provided in the secure introduction between clients, as part of the server address. The offline certificate signs an online certificate used in the transport protocol handshake. [0]
Routers have long-lived, self-signed, offline certificates whose hash is pre-shared with clients over secure channels - either provided with the client library or provided in the secure introduction between clients, as part of the router address. The offline certificate signs an online certificate used in the transport protocol handshake. [0]
If the transport protocol's confidentiality is broken, incoming and outgoing messages to the server cannot be correlated by message contents. Additionally, because of encryption at the SMP layer, impersonating the server is not sufficient to pass (and therefore correlate) a message from a sender to recipient - the only attack possible is to drop the messages. Only by additionally *compromising* the server can one pass and correlate messages.
If the transport protocol's confidentiality is broken, incoming and outgoing messages to the router cannot be correlated by message contents. Additionally, because of encryption at the SMP layer, impersonating the router is not sufficient to pass (and therefore correlate) a message from a sender to recipient - the only attack possible is to drop the messages. Only by additionally *compromising* the router can one pass and correlate messages.
It's important to note that the SMP protocol does not do server authentication. Instead we rely upon the fact that an attacker who tricks the transport protocol into authenticating the server incorrectly cannot do anything with the SMP messages except drop them.
It's important to note that the SMP protocol does not do server authentication. Instead we rely upon the fact that an attacker who tricks the transport protocol into authenticating the router incorrectly cannot do anything with the SMP messages except drop them.
After the connection is established, the client sends blocks of a fixed size 16KB, and the server replies with the blocks of the same size to reduce metadata observable to a network adversary. The protocol has been designed to make traffic correlation attacks difficult, adapting ideas from Tor, remailers, and more general onion and mix networks. It does not try to replace Tor though - SimpleX servers can be deployed as onion services and SimpleX clients can communicate with servers over Tor to further improve participants privacy.
After the connection is established, the client sends blocks of a fixed size 16KB, and the router replies with the blocks of the same size to reduce metadata observable to a network adversary. The protocol has been designed to make traffic correlation attacks difficult, adapting ideas from Tor, remailers, and more general onion and mix networks. It does not try to replace Tor though - SimpleX routers can be deployed as onion services and SimpleX clients can communicate with routers over Tor to further improve participants privacy.
By using fixed-size blocks, oversized for the expected content, the vast majority of traffic is uniform in nature. When enough traffic is transiting a server simultaneously, the server acts as a low-latency mix node. We can't rely on this behavior to make a security claim, but we have engineered to take advantage of it when we can. As mentioned, this holds true even if the transport connection is compromised.
By using fixed-size blocks, oversized for the expected content, the vast majority of traffic is uniform in nature. When enough traffic is transiting a router simultaneously, the router acts as a low-latency mix node. We can't rely on this behavior to make a security claim, but we have engineered to take advantage of it when we can. As mentioned, this holds true even if the transport connection is compromised.
The protocol does not protect against attacks targeted at particular users with known identities - e.g., if the attacker wants to prove that two known users are communicating, they can achieve it by observing their local traffic. At the same time, it substantially complicates large-scale traffic correlation, making determining the real user identities much less effective.
@@ -143,39 +176,39 @@ The protocol does not protect against attacks targeted at particular users with
#### 2-hop Onion Message Routing
As SimpleX Messaging Protocol servers providing messaging queues are chosen by the recipients, in case senders connect to these servers directly the server owners (who potentially can be the recipients themselves) can learn senders' IP addresses (if Tor is not used) and which other queues on the same server are accessed by the user in the same transport connection (even if Tor is used).
As SimpleX Messaging Protocol routers providing messaging queues are chosen by the recipients, in case senders connect to these routers directly the router owners (who potentially can be the recipients themselves) can learn senders' IP addresses (if Tor is not used) and which other queues on the same router are accessed by the user in the same transport connection (even if Tor is used).
While the clients support isolating the messages sent to different queues into different transport connections (and Tor circuits), this is not practical, as it consumes additional traffic and system resources.
To mitigate this problem SimpleX Messaging Protocol servers support 2-hop onion message routing when the SMP server chosen by the sender forwards the messages to the servers chosen by the recipients, thus protecting both the senders IP addresses and sessions, even if connection isolation and Tor are not used.
To mitigate this problem SimpleX Messaging Protocol routers support 2-hop onion message routing when the SMP router chosen by the sender forwards the messages to the routers chosen by the recipients, thus protecting both the senders IP addresses and sessions, even if connection isolation and Tor are not used.
The design of 2-hop onion message routing prevents these potential attacks:
- MITM by proxy (SMP server that forwards the messages).
- MITM by proxy (SMP router that forwards the messages).
- Identification by the proxy which and how many queues the sender sends messages to (as messages are additionally e2e encrypted between the sender and the destination SMP server).
- Identification by the proxy which and how many queues the sender sends messages to (as messages are additionally e2e encrypted between the sender and the destination SMP router).
- Correlation of messages sent to different queues via the same user session (as random correlation IDs and keys are used for each message).
See more details about 2-hop onion message routing design in [SimpleX Messaging Protocol](./simplex-messaging.md#proxying-sender-commands)
Also see [Threat model](#threat-model)
Also see [Security](./security.md)
#### SimpleX Messaging Protocol
SMP is initialized with an in-person or out-of-band introduction message, where Alice provides Bob with details of a server (including IP address or host name, port, and hash of the long-lived offline certificate), a queue ID, and Alice's public keys to agree e2e encryption. These introductions are similar to the PANDA key-exchange, in that if observed, the adversary can race to establish the communication channel instead of the intended participant. [0]
SMP is initialized with an in-person or out-of-band introduction message, where Alice provides Bob with details of a router (including IP address or host name, port, and hash of the long-lived offline certificate), a queue ID, and Alice's public keys to agree e2e encryption. These introductions are similar to the PANDA key-exchange, in that if observed, the adversary can race to establish the communication channel instead of the intended participant. [0]
Because queues are uni-directional, Bob provides an identically-formatted introduction message to Alice over Alice's now-established receiving queue.
When setting up a queue, the server will create separate sender and recipient queue IDs (provided to Alice during set-up and Bob during initial connection). Additionally, during set-up Alice will perform a DH exchange with the server to agree upon a shared secret. This secret will be used to re-encrypt Bob's incoming message before Alice receives it, creating the anti-correlation property earlier-described should the transport encryption be compromised.
When setting up a queue, the router will create separate sender and recipient queue IDs (provided to Alice during set-up and Bob during initial connection). Additionally, during set-up Alice will perform a DH exchange with the router to agree upon a shared secret. This secret will be used to re-encrypt Bob's incoming message before Alice receives it, creating the anti-correlation property earlier-described should the transport encryption be compromised.
[0] Users can additionally create public 'contact queues' that are only used to receive connection requests.
[0] Users can additionally create public 'contact queues' that are only used to receive connection requests.
#### SimpleX Agents
SimpleX agents provide higher-level operations compared to SimpleX Clients, who are primarily concerned with creating queues and communicating with servers using SMP. Agent operations include:
SimpleX agents provide higher-level operations compared to SimpleX Clients, who are primarily concerned with creating queues and communicating with routers using SMP. Agent operations include:
- Managing sets of bi-directional, redundant queues for communication partners
@@ -186,195 +219,21 @@ SimpleX agents provide higher-level operations compared to SimpleX Clients, who
- Noise traffic
#### Encryption Primitives Used
## Security
- Ed25519 or Curve25519 to authorize/verify commands to SMP servers (authorization algorithm is set via client/server configuration).
- Curve25519 for DH exchange to agree:
- the shared secret between server and recipient (to encrypt message bodies - it avoids shared cipher-text in sender and recipient traffic)
- the shared secret between sender and recipient (to encrypt messages end-to-end in each queue - it avoids shared cipher-text in redundant queues).
- [NaCl crypto_box](https://nacl.cr.yp.to/box.html) encryption scheme (curve25519xsalsa20poly1305) for message body encryption between server and recipient and for E2E per-queue encryption.
- SHA256 to validate server offline certificates.
- [double ratchet](https://signal.org/docs/specifications/doubleratchet/) protocol for end-to-end message encryption between the agents:
- Curve448 keys to agree shared secrets required for double ratchet initialization (using [X3DH](https://signal.org/docs/specifications/x3dh/) key agreement with 2 ephemeral keys for each side),
- AES-GCM AEAD cipher,
- SHA512-based HKDF for key derivation.
For encryption primitives, threat model, and detailed security analysis, see [Security](./security.md).
SimpleX provides these security properties:
## Threat Model
- **End-to-end encryption** with forward secrecy via double ratchet protocol, with optional post-quantum protection.
#### Global Assumptions
- **No shared identifiers** across connections — contacts cannot prove they communicate with the same user.
- A user protects their local database and key material.
- The user's application is authentic, and no local malware is running.
- The cryptographic primitives in use are not broken.
- A user's choice of servers is not directly tied to their identity or otherwise represents distinguishing information about the user.
- The user's client uses 2-hop onion message routing.
- **Sender deniability** — neither routers nor recipients can cryptographically prove message origin.
#### A passive adversary able to monitor the traffic of one user
- **Transport metadata protection** — fixed-size blocks, 2-hop onion routing, and connection isolation frustrate traffic correlation.
*can:*
- identify that and when a user is using SimpleX.
- determine which servers the user receives the messages from.
- observe how much traffic is being sent, and make guesses as to its purpose.
*cannot:*
- see who sends messages to the user and who the user sends the messages to.
- determine the servers used by users' contacts.
#### A passive adversary able to monitor a set of senders and recipients
*can:*
- identify who and when is using SimpleX.
- learn which SimpleX Messaging Protocol servers are used as receive queues for which users.
- learn when messages are sent and received.
- perform traffic correlation attacks against senders and recipients and correlate senders and recipients within the monitored set, frustrated by the number of users on the servers.
- observe how much traffic is being sent, and make guesses as to its purpose
*cannot, even in case of a compromised transport protocol:*
- perform traffic correlation attacks with any increase in efficiency over a non-compromised transport protocol
#### SimpleX Messaging Protocol server
*can:*
- learn when a queue recipient is online
- know how many messages are sent via the queue (although some may be noise or not content messages).
- learn which messages would trigger notifications even if a user does not use [push notifications](./push-notifications.md).
- perform the correlation of the queue used to receive messages (matching multiple queues to a single user) via either a re-used transport connection, user's IP Address, or connection timing regularities.
- learn a recipient's IP address, track them through other IP addresses they use to access the same queue, and infer information (e.g. employer) based on the IP addresses, as long as Tor is not used.
- drop all future messages inserted into a queue, detectable only over other, redundant queues.
- lie about the state of a queue to the recipient and/or to the sender (e.g. suspended or deleted when it is not).
- spam a user with invalid messages.
*cannot:*
- undetectably add, duplicate, or corrupt individual messages.
- undetectably drop individual messages, so long as a subsequent message is delivered.
- learn the contents or type of messages.
- distinguish noise messages from regular messages except via timing regularities.
- compromise the users' end-to-end encryption with an active attack.
- learn a sender's IP address, track them through other IP addresses they use to access the same queue, and infer information (e.g. employer) based on the IP addresses, even if Tor is not used (provided messages are sent via proxy SMP server).
- perform senders' queue correlation (matching multiple queues to a single sender) via either a re-used transport connection, user's IP Address, or connection timing regularities, unless it has additional information from the proxy SMP server (provided messages are sent via proxy SMP server).
#### SimpleX Messaging Protocol server that proxies the messages to another SMP server
*can:*
- learn a sender's IP address, as long as Tor is not used.
- learn when a sender with a given IP address is online.
- know how many messages are sent from a given IP address and to a given destination SMP server.
- drop all messages from a given IP address or to a given destination server.
- unless destination SMP server detects repeated public DH keys of senders, replay messages to a destination server within a single session, causing either duplicate message delivery (which will be detected and ignored by the receiving clients), or, when receiving client is not connected to SMP server, exhausting capacity of destination queues used within the session.
*cannot:*
- perform queue correlation (matching multiple queues to a single user), unless it has additional information from the destination SMP server.
- undetectably add, duplicate, or corrupt individual messages.
- undetectably drop individual messages, so long as a subsequent message is delivered.
- learn the contents or type of messages.
- learn which messages would trigger notifications.
- learn the destination queues of messages.
- distinguish noise messages from regular messages except via timing regularities.
- compromise the user's end-to-end encryption with another user via an active attack.
- compromise the user's end-to-end encryption with the destination SMP servers via an active attack.
#### An attacker who obtained Alice's (decrypted) chat database
*can:*
- see the history of all messages exchanged by Alice with her communication partners.
- see shared profiles of contacts and groups.
- surreptitiously receive new messages sent to Alice via existing queues; until communication queues are rotated or the Double-Ratchet advances forward.
- prevent Alice from receiving all new messages sent to her - either surreptitiously by emptying the queues regularly or overtly by deleting them.
- send messages from the user to their contacts; recipients will detect it as soon as the user sends the next message, because the previous message hash wont match (and potentially wont be able to decrypt them in case they dont keep the previous ratchet keys).
*cannot:*
- impersonate a sender and send messages to the user whose database was stolen. Doing so requires also compromising the server (to place the message in the queue, that is possible until the Double-Ratchet advances forward) or the user's device at a subsequent time (to place the message in the database).
- undetectably communicate at the same time as Alice with her contacts. Doing so would result in the contact getting different messages with repeated IDs.
- undetectably monitor message queues in realtime without alerting the user they are doing so, as a second subscription request unsubscribes the first and notifies the second.
#### A users contact
*can:*
- spam the user with messages.
- forever retain messages from the user.
*cannot:*
- cryptographically prove to a third-party that a message came from a user (assuming the users device is not seized).
- prove that two contacts they have is the same user.
- cannot collaborate with another of the user's contacts to confirm they are communicating with the same user.
#### An attacker who observes Alice showing an introduction message to Bob
*can:*
- Impersonate Bob to Alice.
*cannot:*
- Impersonate Alice to Bob.
#### An attacker with Internet access
*can:*
- Denial of Service SimpleX messaging servers.
- spam a user's public “contact queue” with connection requests.
*cannot:*
- send messages to a user who they are not connected with.
- enumerate queues on a SimpleX server.
- **Out-of-band key exchange** — connection requests passed outside the network protect against MITM attacks.
## Acknowledgements
+82 -5
View File
@@ -13,6 +13,11 @@ Version 1, 2024-06-22
- [Initialization](#initialization)
- [Encrypting messages](#encrypting-messages)
- [Decrypting messages](#decrypting-messages)
- [Ratchet message wire format](#ratchet-message-wire-format)
- [Encrypted ratchet message](#encrypted-ratchet-message)
- [Encrypted message header](#encrypted-message-header)
- [Plaintext message header](#plaintext-message-header)
- [KEM state machine](#kem-state-machine)
- [Implementation considerations](#implementation-considerations)
- [Chosen KEM algorithm](#chosen-kem-algorithm)
- [Summary](#summary)
@@ -71,11 +76,10 @@ def RatchetInitAlicePQ2HE(state, SK, bob_dh_public_key, shared_hka, shared_nhkb,
// below added for post-quantum KEM
state.PQRs = GENERATE_PQKEM()
state.PQRr = bob_pq_kem_encapsulation_key
state.PQRss = random // shared secret for KEM
state.PQRct = PQKEM-ENC(state.PQRr, state.PQRss) // encapsulated additional shared secret
state.PQRct, state.PQRss = PQKEM-ENC(state.PQRr) // encapsulate: generates shared secret and ciphertext
// above added for KEM
// the next line augments DH key agreement with PQ shared secret
state.RK, state.CKs, state.NHKs = KDF_RK_HE(SK, DH(state.DHRs, state.DHRr) || state.PQRss)
state.RK, state.CKs, state.NHKs = KDF_RK_HE(SK, DH(state.DHRs, state.DHRr) || state.PQRss)
state.CKr = None
state.Ns = 0
state.Nr = 0
@@ -176,8 +180,7 @@ def DHRatchetPQ2HE(state, header):
state.DHRs = GENERATE_DH()
// below is added for KEM
state.PQRs = GENERATE_PQKEM() // generate new PQ key pair
state.PQRss = random // shared secret for KEM
state.PQRct = PQKEM-ENC(state.PQRr, state.PQRss) // encapsulated additional shared secret KEM #1
state.PQRct, state.PQRss = PQKEM-ENC(state.PQRr) // encapsulate: generates shared secret and ciphertext KEM #1
// above is added for KEM
// use new shared secret with sending ratchet
state.RK, state.CKs, state.NHKs = KDF_RK_HE(state.RK, DH(state.DHRs, state.DHRr) || state.PQRss)
@@ -191,6 +194,80 @@ Other than augmenting DH key agreements with the shared secrets from KEM, the ab
It is worth noting that while DH agreements work as ping-pong, when the new received DH key is used for both DH agreements (and only the sent DH key is updated for the second DH key agreement), PQ KEM agreements in the proposed scheme work as a "parallel ping-pong", with two balls in play all the time (two KEM agreements run in parallel).
## Ratchet message wire format
The pseudocode above describes the algorithm. This section specifies the actual binary encoding used in SimpleX implementation with Curve448 DH keys, sntrup761 KEM and AES-256-GCM AEAD.
The ratchet-encrypted message has three encoding layers, from outermost to innermost:
1. **Encrypted ratchet message** — the complete ratchet message envelope, referenced as an opaque encrypted body in [agent protocol](./agent-protocol.md).
2. **Encrypted message header** — the encrypted header within the ratchet message, used as associated data for message body encryption.
3. **Plaintext message header** — the DH and KEM ratchet keys and counters.
### Encrypted ratchet message
The outer envelope contains the encrypted header (used as associated data for body authentication), the body authentication tag, and the encrypted message body.
The message body is encrypted with AES-256-GCM using the message key derived from the sending chain key (`KDF_CK`). The associated data for body encryption is the concatenation of the ratchet associated data and the encoded encrypted header.
```abnf
encRatchetMessage = versionedLength encMessageHeader msgAuthTag encMsgBody
; encMessageHeader is used as associated data for body decryption: AD = rcAD || encMessageHeader
msgAuthTag = 16*16 OCTET ; AES-256-GCM authentication tag for the message body
encMsgBody = *OCTET ; AES-256-GCM encrypted padded message body (remaining bytes)
```
### Encrypted message header
The encrypted header wraps the current ratchet e2e encryption version, an initialization vector, an authentication tag, and the encrypted padded header body.
The header body is encrypted with AES-256-GCM using the header key (`HKs`). The associated data for header encryption is the ratchet associated data. The header is padded before encryption to a fixed size to prevent leaking information about the KEM state.
```abnf
encMessageHeader = currentVersion headerIV headerAuthTag versionedLength encHeaderBody
currentVersion = 2*2 OCTET ; Word16, current ratchet e2e encryption version
headerIV = 16*16 OCTET ; AES-256 initialization vector for header encryption
headerAuthTag = 16*16 OCTET ; AES-256-GCM authentication tag for the header
encHeaderBody = *OCTET ; AES-256-GCM encrypted padded header (see plaintext format below)
```
`versionedLength` uses a 2-byte length prefix (Word16) when the current e2e version supports PQ encryption, or a 1-byte length prefix otherwise. The parser distinguishes the two encodings by peeking at the first byte: values below 32 indicate a 2-byte prefix (as the header is always at least 69 bytes).
```abnf
versionedLength = largeLength / length ; 2-byte for PQ versions, 1-byte for pre-PQ versions
```
The padded header sizes before encryption are: 2310 bytes when PQ is supported, 88 bytes when PQ is not supported. Padding uses a 2-byte big-endian length prefix followed by the plaintext header and `#` fill bytes.
### Plaintext message header
```abnf
msgHeader = maxVersion dhPublicKey [kemParams] prevMsgCount msgCount
maxVersion = 2*2 OCTET ; Word16, max supported e2e encryption version
dhPublicKey = length x509encoded ; Curve448 public DH ratchet key
kemParams = noKEM / proposedKEM / acceptedKEM
; present only when current ratchet version >= pqRatchetE2EEncryptVersion
noKEM = %x30 ; "0" - no KEM parameters
proposedKEM = %x31 %s"P" kemEncapsulationKey ; KEM proposed, not yet accepted
acceptedKEM = %x31 %s"A" kemCiphertext kemEncapsulationKey ; KEM accepted
kemEncapsulationKey = largeLength 1158*1158 OCTET ; sntrup761 encapsulation key
kemCiphertext = largeLength 1039*1039 OCTET ; sntrup761 ciphertext
prevMsgCount = 4*4 OCTET ; Word32, number of messages in previous sending chain
msgCount = 4*4 OCTET ; Word32, message number in current sending chain
length = 1*1 OCTET
largeLength = 2*2 OCTET ; Word16
```
### KEM state machine
PQ encryption can be enabled or disabled during a connection's lifetime. The KEM parameters in the header reflect three states:
- **No KEM** (`noKEM`): PQ encryption is not active. The header contains only the DH key, as in the original double ratchet.
- **Proposed** (`proposedKEM`): One party generated a KEM key pair and includes the encapsulation key in the header, proposing PQ encryption. No ciphertext is included because the other party has not yet sent its encapsulation key.
- **Accepted** (`acceptedKEM`): The party received the other's encapsulation key, performed encapsulation (KEM #1), and includes both the ciphertext and its own new encapsulation key (for KEM #2). This is the steady state for active PQ encryption.
The transition from Proposed to Accepted happens when a party receives a message containing KEM parameters (either Proposed or Accepted) and responds with its own Accepted parameters. Once both parties are in Accepted state, the double PQ KEM augmentation described in the algorithm above operates in each DH ratchet step.
## Implementation considerations for SimpleX Messaging Protocol
As SimpleX Messaging Protocol pads messages to a fixed size, using 16kb transport blocks, the size increase introduced by this scheme can be compensated for by using ZSTD encryption of JSON bodies and image previews encoded as base64. While there may be some rare cases of random texts that would fail to compress, in all real scenarios it would not cause the message size reduction.
+73 -51
View File
@@ -1,14 +1,19 @@
Version 2, 2024-06-22
Version 3, 2025-01-24
# Overview of push notifications for SimpleX Messaging Servers
# Overview of push notifications for SimpleX Messaging Routers
This document describes Notification Router protocol version 3. Version history:
- v1: initial version
- v2: authenticated commands, command batching
- v3: detailed invalid token reason
## Table of contents
- [Introduction](#introduction)
- [Participating servers](#participating-servers)
- [Participating routers](#participating-routers)
- [Register device token to receive push notifications](#register-device-token-to-receive-push-notifications)
- [Subscribe to connection notifications](#subscribe-to-connection-notifications)
- [SimpleX Notification Server protocol](#simplex-notification-server-protocol)
- [SimpleX Notification Router protocol](#simplex-notification-router-protocol)
- [Register new notification token](#register-new-notification-token)
- [Verify notification token](#verify-notification-token)
- [Check notification token status](#check-notification-token-status)
@@ -23,35 +28,35 @@ Version 2, 2024-06-22
## Introduction
SimpleX Messaging servers already operate as push servers and deliver the messages to subscribed clients as soon as they are sent to the servers.
SimpleX Messaging routers already operate as push routers and deliver the messages to subscribed clients as soon as they are sent to the routers.
The reason for push notifications is to support instant message notifications on iOS that does not allow background services.
## Participating servers
## Participating routers
The diagram below shows which servers participate in message notification delivery.
The diagram below shows which routers participate in message notification delivery.
While push provider (e.g., APN) can learn how many notifications are delivered to the user, it cannot access message content, even encrypted, or any message metadata - the notifications are e2e encrypted between SimpleX Notification Server and the user's device.
While push provider (e.g., APN) can learn how many notifications are delivered to the user, it cannot access message content, even encrypted, or any message metadata - the notifications are e2e encrypted between SimpleX Notification Router and the user's device.
```
User's iOS device Internet Servers
User's iOS device Internet Routers
--------------------- . ------------------------ . -----------------------------
. .
. . can be self-hosted now
+--------------+ . . +----------------+
| SimpleX Chat | -------------- TLS --------------- | SimpleX |
| client |------> SimpleX Messaging Protocol (SMP) ------> | Messaging |
+--------------+ ---------------------------------- | Server |
+--------------+ ---------------------------------- | Router |
^ | . . +----------------+
| | . . . . . | . . .
| | . . | V |
| | . . |SMP| TLS
| | . . | | | SimpleX
| | . . . . . V . . . NTF Server
| | . . . . . V . . . NTF Router
| | . . +----------------------------------+
| | . . | +---------------+ |
| | -------------- TLS --------------- | | SimpleX | can be |
| |-----------> Notification Server Protocol -----> | | Notifications | self-hosted |
| |-----------> Notification Router Protocol -----> | | Notifications | self-hosted |
| ---------------------------------- | | Subscriber | in the future |
| . . | +---------------+ |
| . . | | |
@@ -59,7 +64,7 @@ While push provider (e.g., APN) can learn how many notifications are delivered t
| . . | +---------------+ |
| . . | | SimpleX | |
| . . | | Push | |
| . . | | Server | |
| . . | | Router | |
| . . | +---------------+ |
| . . +----------------------------------+
| . . . . . | . . .
@@ -85,25 +90,28 @@ This diagram shows the process of subscription to notifications, notification de
![Subscribe to notifications](./diagrams/notifications/subscription.svg)
## SimpleX Notification Server protocol
## SimpleX Notification Router protocol
To manage notification subscriptions to SMP servers, SimpleX Notification Server provides an RPC protocol with a similar design to SimpleX Messaging Protocol server.
To manage notification subscriptions to SMP routers, SimpleX Notification Router provides an RPC protocol with a similar design to SimpleX Messaging Protocol router.
This protocol sends requests and responses in a fixed size blocks of 512 bytes over TLS, uses the same [syntax of protocol transmissions](./simplex-messaging.md#smp-transmission-and-transport-block-structure) as SMP protocol, and has the same transport [handshake syntax](./simplex-messaging.md#transport-handshake) (except the server certificate is not included in the handshake).
This protocol sends requests and responses in a fixed size blocks of 512 bytes over TLS, uses the same [syntax of protocol transmissions](./simplex-messaging.md#smp-transmission-and-transport-block-structure) as SMP protocol, and has the same transport [handshake syntax](./simplex-messaging.md#transport-handshake) (except the router certificate is not included in the handshake).
The client and router use ALPN extension with `ntf/1` protocol name to agree handshake version.
Protocol commands have this syntax:
```
ntfServerTransmission =
ntfServerCmd = newTokenCmd / verifyTokenCmd / checkTokenCmd /
```abnf
ntfRouterTransmission = authorization corrId entityId ntfRouterCmd
; same transmission structure as SMP, see simplex-messaging.md
ntfRouterCmd = newTokenCmd / verifyTokenCmd / checkTokenCmd /
replaceTokenCmd / deleteTokenCmd / cronCmd /
newSubCmd / checkSubCmd / deleteSubCmd
newSubCmd / checkSubCmd / deleteSubCmd / pingCmd
```
### Register new notification token
This command should be used after the client app obtains a token from push notifications provider to register the token with the server.
This command should be used after the client app obtains a token from push notifications provider to register the token with the router.
Having received this command the server will deliver a test notification via the push provider to validate that the client has this token.
Having received this command the router will deliver a test notification via the push provider to validate that the client has this token.
The command syntax:
@@ -111,23 +119,24 @@ The command syntax:
newTokenCmd = %s"TNEW" SP newToken
newToken = %s"T" deviceToken authPubKey clientDhPubKey
deviceToken = pushProvider tokenString
pushProvider = apnsDev / apnsProd / apnsNull
pushProvider = apnsDev / apnsProd / apnsTest / apnsNull
apnsDev = "AD" ; APNS token for development environment
apnsProd = "AP" ; APNS token for production environment
apnsNull = "AN" ; token that does not trigger any notification delivery - used for server testing
apnsTest = "AT" ; APNS token for test environment (mock server)
apnsNull = "AN" ; token that does not trigger any notification delivery - used for router testing
tokenString = shortString
authPubKey = length x509encoded ; Ed25519 key used to verify clients commands
clientDhPubKey = length x509encoded ; X25519 key to agree e2e encryption between the server and client
clientDhPubKey = length x509encoded ; X25519 key to agree e2e encryption between the router and client
shortString = length *OCTET
length = 1*1 OCTET
```
The server response syntax:
The router response syntax:
```abnf
tokenIdResp = %s"IDTKN" SP entityId serverDhPubKey
tokenIdResp = %s"IDTKN" SP entityId routerDhPubKey
entityId = shortString
serverDhPubKey = length x509encoded ; X25519 key to agree e2e encryption between the server and client
routerDhPubKey = length x509encoded ; X25519 key to agree e2e encryption between the router and client
```
### Verify notification token
@@ -159,7 +168,9 @@ The response to this command:
```abnf
tokenStatusResp = %s"TKN" SP tokenStatus
tokenStatus = %s"NEW" / %s"REGISTERED" / %s"INVALID" / %s"CONFIRMED" / %s"ACTIVE" / %s"EXPIRED"
tokenStatus = %s"NEW" / %s"REGISTERED" / tokenInvalid / %s"CONFIRMED" / %s"ACTIVE" / %s"EXPIRED"
tokenInvalid = %s"INVALID" ["," invalidReason] ; optional reason added in v3
invalidReason = %s"BAD" / %s"TOPIC" / %s"EXPIRED" / %s"UNREGISTERED"
```
### Replace notification token
@@ -200,8 +211,8 @@ After this command all message notification subscriptions will be removed and no
This command enables or disables periodic notifications sent to the client device irrespective of message notifications.
This is useful for two reasons:
- it provides better privacy from notification server, as while the server learns the device token, it doesn't learn anything else about user communications.
- it allows to receive messages when notifications were dropped by push provider, e.g. while the device was offline, or lost by notification server, e.g. while it was restarting.
- it provides better privacy from notification router, as while the router learns the device token, it doesn't learn anything else about user communications.
- it allows to receive messages when notifications were dropped by push provider, e.g. while the device was offline, or lost by notification router, e.g. while it was restarting.
The command syntax:
@@ -214,18 +225,18 @@ The interval for periodic notifications is set in minutes, with the minimum of 2
### Create SMP message notification subscription
This command makes notification server subscribe to message notifications from SMP server and to deliver them to push provider:
This command makes notification router subscribe to message notifications from SMP router and to deliver them to push provider:
```abnf
newSubCmd = %s"SNEW" newSub
newSub = %s "S" tokenId smpServer notifierId notifierKey
newSubCmd = %s"SNEW" SP newSub
newSub = %s"S" tokenId smpRouter notifierId notifierKey
tokenId = shortString ; returned in response to `TNEW` command
smpServer = smpServer = hosts port fingerprint
smpRouter = hosts port fingerprint
hosts = length 1*host
host = shortString
port = shortString
fingerprint = shortString
notifierId = shortString ; returned by SMP server in response to `NKEY` SMP command
notifierId = shortString ; returned by SMP router in response to `NKEY` SMP command
notifierKey = length x509encoded ; private key used to authorize requests to subscribe to message notifications
```
@@ -247,10 +258,10 @@ The response:
```abnf
subStatusResp = %s"SUB" SP subStatus
subStatus = %s"NEW" / %s"PENDING" / ; e.g., after SMP server disconnect/timeout while ntf server is retrying to connect
%s"ACTIVE" / %s"INACTIVE" / %s"END" / ; if another server subscribed to notifications
%s"AUTH" / subErrStatus
subErrStatus = %s"ERR" SP shortString
subStatus = %s"NEW" / %s"PENDING" / ; e.g., after SMP router disconnect/timeout while ntf router is retrying to connect
%s"ACTIVE" / %s"INACTIVE" / %s"END" / ; if another router subscribed to notifications
%s"AUTH" / %s"DELETED" / %s"SERVICE" / subErrStatus
subErrStatus = %s"ERR" SP *OCTET
```
### Delete notification subscription
@@ -265,6 +276,17 @@ The response to this command is `okResp` or `errorResp`.
After this command no more message notifications will be sent from this queue.
### Keep-alive command
To keep the transport connection alive the clients should use `PING` command:
```abnf
pingCmd = %s"PING"
pongResp = %s"PONG"
```
This command is sent unsigned and without entity ID.
### Error responses
All commands can return error response:
@@ -277,7 +299,7 @@ Where `errorType` has the same syntax as in [SimpleX Messaging Protocol](./simpl
## Threat Model
This threat model compliments SimpleX Messaging Protocol [threat model](./overview-tjr.md#threat-model)
This threat model compliments SimpleX Messaging Protocol [threat model](./security.md#threat-model)
#### A passive adversary able to monitor the traffic of one user
@@ -287,21 +309,21 @@ This threat model compliments SimpleX Messaging Protocol [threat model](./overvi
*cannot:*
- determine which servers a user subscribed to the notifications from.
- determine which routers a user subscribed to the notifications from.
#### A passive adversary able to monitor a set of senders and recipients
*can:*
- perform more efficient traffic correlation attacks against senders and recipients and correlate senders and recipients within the monitored set, frustrated by the number of users on the servers.
- perform more efficient traffic correlation attacks against senders and recipients and correlate senders and recipients within the monitored set, frustrated by the number of users on the routers.
#### SimpleX Messaging Protocol server
#### SimpleX Messaging Protocol router
*can:*
- learn which messages trigger push notifications.
- learn IP address of SimpleX notification servers used by the user.
- learn IP address of SimpleX notification routers used by the user.
- drop message notifications.
@@ -313,13 +335,13 @@ This threat model compliments SimpleX Messaging Protocol [threat model](./overvi
- learn which queues belong to the same users with any additional efficiency compared with not using push notifications.
#### SimpleX Notification Server subscribed to message notifications
#### SimpleX Notification Router subscribed to message notifications
*can:*
- learn a user device token.
- learn how many messaging queues and servers a user receives messages from.
- learn how many messaging queues and routers a user receives messages from.
- learn how many message notifications are delivered to the user from each queue.
@@ -339,7 +361,7 @@ This threat model compliments SimpleX Messaging Protocol [threat model](./overvi
- add, duplicate, or corrupt individual messages that will be shown to the user.
#### SimpleX Notification Server subscribed ONLY to periodic notifications
#### SimpleX Notification Router subscribed ONLY to periodic notifications
*can:*
@@ -351,7 +373,7 @@ This threat model compliments SimpleX Messaging Protocol [threat model](./overvi
*cannot:*
- learn how many messaging queues and servers a user receives messages from.
- learn how many messaging queues and routers a user receives messages from.
- learn how many message notifications are delivered to the user from each queue.
@@ -383,7 +405,7 @@ This threat model compliments SimpleX Messaging Protocol [threat model](./overvi
*cannot:*
- learn which SimpleX Messaging Protocol servers are used by a user (notifications are e2e encrypted).
- learn which SimpleX Messaging Protocol routers are used by a user (notifications are e2e encrypted).
- learn which or how many messaging queues a user receives notifications from.
@@ -395,4 +417,4 @@ This threat model compliments SimpleX Messaging Protocol [threat model](./overvi
- register notification token not present on attacker's device.
- enumerate tokens or subscriptions on a SimpleX Notification Server.
- enumerate tokens or subscriptions on a SimpleX Notification Router.
+215
View File
@@ -0,0 +1,215 @@
Revision 1, 2026-03-09
# SimpleX Network: Security
This document describes the cryptographic primitives and threat model for the SimpleX network. For a general introduction, see [SimpleX: messaging and application platform](./overview-tjr.md).
## Table of contents
- [Encryption primitives](#encryption-primitives)
- [Threat model](#threat-model)
- [Global Assumptions](#global-assumptions)
- [A passive adversary able to monitor the traffic of one user](#a-passive-adversary-able-to-monitor-the-traffic-of-one-user)
- [A passive adversary able to monitor a set of senders and recipients](#a-passive-adversary-able-to-monitor-a-set-of-senders-and-recipients)
- [SimpleX Messaging Protocol router](#simplex-messaging-protocol-router)
- [SimpleX Messaging Protocol router that proxies the messages to another SMP router](#simplex-messaging-protocol-router-that-proxies-the-messages-to-another-smp-router)
- [An attacker who obtained Alice's (decrypted) chat database](#an-attacker-who-obtained-alices-decrypted-chat-database)
- [A user's contact](#a-users-contact)
- [An attacker who observes Alice showing an introduction message to Bob](#an-attacker-who-observes-alice-showing-an-introduction-message-to-bob)
- [An attacker with Internet access](#an-attacker-with-internet-access)
## Encryption primitives
- **Router command authorization**: X25519 DH-based authenticated encryption (SMP v7+), providing sender deniability. Ed25519 signatures used for recipient commands and notifier commands.
- **Per-queue key agreement**: Curve25519 DH exchange to agree:
- the shared secret between router and recipient (to encrypt message bodies — avoids shared ciphertext in sender and recipient traffic),
- the shared secret between sender and recipient (to encrypt messages end-to-end in each queue — avoids shared ciphertext in redundant queues).
- **SMP-layer encryption**: [NaCl crypto_box](https://nacl.cr.yp.to/box.html) (curve25519xsalsa20poly1305) for message body encryption between router and recipient, and for e2e per-queue encryption.
- **Certificate validation**: SHA256 to validate router offline certificates.
- **End-to-end encryption**: [Double ratchet](https://signal.org/docs/specifications/doubleratchet/) protocol:
- Curve448 keys for shared secret agreement via [X3DH](https://signal.org/docs/specifications/x3dh/) with 2 ephemeral keys per side,
- optional [SNTRUP761](https://ntruprime.cr.yp.to/) post-quantum KEM running in parallel with the DH ratchet (see [PQDR](./pqdr.md)), providing post-quantum forward secrecy,
- AES-GCM AEAD cipher,
- SHA512-based HKDF for key derivation.
## Threat Model
### Global Assumptions
- A user protects their local database and key material.
- The user's application is authentic, and no local malware is running.
- The cryptographic primitives in use are not broken.
- A user's choice of routers is not directly tied to their identity or otherwise represents distinguishing information about the user.
- The user's client uses 2-hop onion message routing.
### A passive adversary able to monitor the traffic of one user
*can:*
- identify that and when a user is using SimpleX.
- determine which routers the user receives messages from.
- observe how much traffic is being sent, and make guesses as to its purpose.
*cannot:*
- see who sends messages to the user and who the user sends messages to.
- determine the routers used by users' contacts.
### A passive adversary able to monitor a set of senders and recipients
*can:*
- identify who and when is using SimpleX.
- learn which SimpleX Messaging Protocol routers are used as receive queues for which users.
- learn when messages are sent and received.
- perform traffic correlation attacks against senders and recipients and correlate senders and recipients within the monitored set, frustrated by the number of users on the routers.
- observe how much traffic is being sent, and make guesses as to its purpose.
*cannot, even in case of a compromised transport protocol:*
- perform traffic correlation attacks with any increase in efficiency over a non-compromised transport protocol.
### SimpleX Messaging Protocol router
*can:*
- learn when a queue recipient is online.
- know how many messages are sent via the queue (although some may be noise or not content messages).
- learn which messages would trigger notifications even if a user does not use [push notifications](./push-notifications.md).
- perform the correlation of the queue used to receive messages (matching multiple queues to a single user) via either a re-used transport connection, user's IP Address, or connection timing regularities.
- learn a recipient's IP address, track them through other IP addresses they use to access the same queue, and infer information (e.g. employer) based on the IP addresses, as long as Tor is not used.
- drop all future messages inserted into a queue, detectable only over other, redundant queues.
- lie about the state of a queue to the recipient and/or to the sender (e.g. suspended or deleted when it is not).
- spam a user with invalid messages.
*cannot:*
- undetectably add, duplicate, or corrupt individual messages.
- undetectably drop individual messages, so long as a subsequent message is delivered.
- learn the contents or type of messages.
- distinguish noise messages from regular messages except via timing regularities.
- compromise the users' end-to-end encryption with an active attack.
- learn a sender's IP address, track them through other IP addresses they use to access the same queue, and infer information (e.g. employer) based on the IP addresses, even if Tor is not used (provided messages are sent via proxy SMP router).
- perform senders' queue correlation (matching multiple queues to a single sender) via either a re-used transport connection, user's IP Address, or connection timing regularities, unless it has additional information from the proxy SMP router (provided messages are sent via proxy SMP router).
### SimpleX Messaging Protocol router that proxies the messages to another SMP router
*can:*
- learn a sender's IP address, as long as Tor is not used.
- learn when a sender with a given IP address is online.
- know how many messages are sent from a given IP address and to a given destination SMP router.
- drop all messages from a given IP address or to a given destination router.
- unless destination SMP router detects repeated public DH keys of senders, replay messages to a destination router within a single session, causing either duplicate message delivery (which will be detected and ignored by the receiving clients), or, when receiving client is not connected to SMP router, exhausting capacity of destination queues used within the session.
*cannot:*
- perform queue correlation (matching multiple queues to a single user), unless it has additional information from the destination SMP router.
- undetectably add, duplicate, or corrupt individual messages.
- undetectably drop individual messages, so long as a subsequent message is delivered.
- learn the contents or type of messages.
- learn which messages would trigger notifications.
- learn the destination queues of messages.
- distinguish noise messages from regular messages except via timing regularities.
- compromise the user's end-to-end encryption with another user via an active attack.
- compromise the user's end-to-end encryption with the destination SMP routers via an active attack.
### An attacker who obtained Alice's (decrypted) chat database
*can:*
- see the history of all messages exchanged by Alice with her communication partners.
- see shared profiles of contacts and groups.
- surreptitiously receive new messages sent to Alice via existing queues; until communication queues are rotated or the Double-Ratchet advances forward.
- prevent Alice from receiving all new messages sent to her - either surreptitiously by emptying the queues regularly or overtly by deleting them.
- send messages from the user to their contacts; recipients will detect it as soon as the user sends the next message, because the previous message hash won't match (and potentially won't be able to decrypt them in case they don't keep the previous ratchet keys).
*cannot:*
- impersonate a sender and send messages to the user whose database was stolen. Doing so requires also compromising the router (to place the message in the queue, that is possible until the Double-Ratchet advances forward) or the user's device at a subsequent time (to place the message in the database).
- undetectably communicate at the same time as Alice with her contacts. Doing so would result in the contact getting different messages with repeated IDs.
- undetectably monitor message queues in realtime without alerting the user they are doing so, as a second subscription request unsubscribes the first and notifies the first.
### A user's contact
*can:*
- spam the user with messages.
- forever retain messages from the user.
*cannot:*
- cryptographically prove to a third-party that a message came from a user (assuming the user's device is not seized).
- prove that two contacts they have is the same user.
- cannot collaborate with another of the user's contacts to confirm they are communicating with the same user.
### An attacker who observes Alice showing an introduction message to Bob
*can:*
- Impersonate Bob to Alice.
*cannot:*
- Impersonate Alice to Bob.
### An attacker with Internet access
*can:*
- Denial of Service SimpleX messaging routers.
- spam a user's public "contact queue" with connection requests.
*cannot:*
- send messages to a user who they are not connected with.
- enumerate queues on a SimpleX router.
File diff suppressed because it is too large Load Diff
+199 -157
View File
@@ -1,4 +1,4 @@
Version 2, 2024-06-22
Version 3, 2025-01-24
# SimpleX File Transfer Protocol
@@ -11,12 +11,12 @@ Version 2, 2024-06-22
- [XFTP procedure](#xftp-procedure)
- [File description](#file-description)
- [URIs syntax](#uris-syntax)
- [XFTP server URI](#xftp-server-uri)
- [XFTP router URI](#xftp-router-uri)
- [File description URI](#file-description-URI)
- [XFTP qualities and features](#xftp-qualities-and-features)
- [Cryptographic algorithms](#cryptographic-algorithms)
- [File chunk IDs](#file-chunk-ids)
- [Server security requirements](#server-security-requirements)
- [Data packet IDs](#data-packet-ids)
- [Router security requirements](#router-security-requirements)
- [Transport protocol](#transport-protocol)
- [TLS ALPN](#tls-alpn)
- [Connection handshake](#connection-handshake)
@@ -26,13 +26,14 @@ Version 2, 2024-06-22
- [Command authentication](#command-authentication)
- [Keep-alive command](#keep-alive-command)
- [File sender commands](#file-sender-commands)
- [Register new file chunk](#register-new-file-chunk)
- [Add file chunk recipients](#add-file-chunk-recipients)
- [Upload file chunk](#upload-file-chunk)
- [Delete file chunk](#delete-file-chunk)
- [Register new data packet](#register-new-data-packet)
- [Add data packet recipients](#add-data-packet-recipients)
- [Upload data packet](#upload-data-packet)
- [Delete data packet](#delete-data-packet)
- [File recipient commands](#file-recipient-commands)
- [Download file chunk](#download-file-chunk)
- [Acknowledge file chunk download](#acknowledge-file-chunk-download)
- [Download data packet](#download-data-packet)
- [Acknowledge data packet download](#acknowledge-data-packet-download)
- [Error responses](#error-responses)
- [Threat model](#threat-model)
## Abstract
@@ -45,23 +46,31 @@ It is designed as a application level protocol to solve the problem of secure an
## Introduction
The objective of SimpleX File Transfer Protocol (XFTP) is to facilitate the secure and private unidirectional transfer of files from senders to recipients via persistent file chunks stored by the xftp server.
The objective of SimpleX File Transfer Protocol (XFTP) is to facilitate the secure and private unidirectional transfer of files from senders to recipients via persistent data packets stored by the xftp router.
XFTP is implemented as an application level protocol on top of HTTP2 and TLS.
The protocol describes the set of commands that senders and recipients can send to XFTP servers to create, upload, download and delete file chunks of several pre-defined sizes. XFTP servers SHOULD support chunks of 4 sizes: 64KB, 256KB, 1MB and 4MB (1KB = 1024 bytes, 1MB = 1024KB).
This document describes XFTP protocol version 3. The version history:
The protocol is designed with the focus on meta-data privacy and security. While using TLS, the protocol does not rely on TLS security by using additional encryption to achieve that there are no identifiers or ciphertext in common in received and sent server traffic, frustrating traffic correlation even if TLS is compromised.
- v1: initial version
- v2: authenticated commands - added basic auth support for commands
- v3: blocked files - added BLOCKED error type for policy violations
XFTP does not use any form of participants' identities. It relies on out-of-band passing of "file description" - a human-readable YAML document with the list of file chunk locations, hashes and necessary cryptographic keys.
The protocol describes the set of commands that senders and recipients can send to XFTP routers to create, upload, download and delete data packets of several pre-defined sizes. XFTP routers SHOULD support packets of 4 sizes: 64KB, 256KB, 1MB and 4MB (1KB = 1024 bytes, 1MB = 1024KB).
The protocol is designed with the focus on meta-data privacy and security. While using TLS, the protocol does not rely on TLS security by using additional encryption to achieve that there are no identifiers or ciphertext in common in received and sent router traffic, frustrating traffic correlation even if TLS is compromised.
XFTP does not use any form of participants' identities. It relies on out-of-band passing of "file description" - a human-readable YAML document with the list of data packet locations, hashes and necessary cryptographic keys.
> **Note:** While this protocol was originally designed for file transfer, it handles generic addressed data packets. File-specific semantics (splitting files into packets, assembly, naming) are application-level concerns defined in the [agent protocol](./agent-protocol.md).
## XFTP Model
The XFTP model has three communication participants: the recipient, the file server (XFTP server) that is chosen and, possibly, controlled by the sender, and the sender.
The XFTP model has three communication participants: the recipient, the XFTP router that is chosen and, possibly, controlled by the sender, and the sender.
XFTP server allows uploading fixed size file chunks, with or without basic authentication. The same party that can be the sender of one file chunk can be the recipient of another, without exposing it to the server.
XFTP router allows uploading fixed size data packets, with or without basic authentication. The same party that can be the sender of one data packet can be the recipient of another, without exposing it to the router.
Each file chunk allows multiple recipients, each recipient can download the same chunk multiple times. It allows depending on the threat model use the same recipient credentials for multiple parties, thus reducing server ability to understand the number of intended recipients (but server can still track IP addresses to determine it), or use one unique set of credentials for each recipient, frustrating traffic correlation on the assumption of compromised TLS. In the latter case, senders can create a larger number of recipient credentials to hide the actual number of intended recipients from the servers (which is what SimpleX clients do).
Each data packet allows multiple recipients, each recipient can download the same packet multiple times. It allows depending on the threat model use the same recipient credentials for multiple parties, thus reducing router ability to understand the number of intended recipients (but router can still track IP addresses to determine it), or use one unique set of credentials for each recipient, frustrating traffic correlation on the assumption of compromised TLS. In the latter case, senders can create a larger number of recipient credentials to hide the actual number of intended recipients from the routers (which is what SimpleX clients do).
```
Sender Internet XFTP relays Internet Recipient
@@ -69,7 +78,7 @@ Each file chunk allows multiple recipients, each recipient can download the same
| | | |
| | (can be self-hosted) | |
| | +---------+ | |
chunk 1 ----- HTTP2 over TLS ------ | XFTP | ---- HTTP2 / TLS ----- chunk 1
packet 1 ----- HTTP2 over TLS ------ | XFTP | ---- HTTP2 / TLS ----- packet 1
|---> SimpleX File Transfer Protocol (XFTP) --> | Relay | ---> XFTP ------------->|
| --------------------------- +---------+ ---------------------- |
| | | | | |
@@ -83,21 +92,21 @@ file ---> | XFTP | ------> XFTP ----> | Relay | --->
| | | +---------+ | | |
| ------- HTTP2 / TLS ------- | XFTP | ---- HTTP2 / TLS ---- |
|-------------> XFTP ----> | Relay | ---> XFTP ------------->|
chunk N --------------------------- +---------+ --------------------- chunk N
| | (store file chunks) | |
packet N --------------------------- +---------+ --------------------- packet N
| | (store data packets) | |
| | | |
| | | |
```
When sender client uploads a file chunk, it has to register it first with one sender ID and multiple recipient IDs, and one random unique key per ID to authenticate sender and recipients, and also provide its size and hash that will be validated when chunk is uploaded.
When sender client uploads a data packet, it has to register it first with one sender ID and multiple recipient IDs, and one random unique key per ID to authenticate sender and recipients, and also provide its size and hash that will be validated when packet is uploaded.
To send the actual file, the sender client MUST pad it and encrypt it with a random symmetric key and distribute chunks of fixed sized across multiple XFTP servers. Information about chunk locations, keys, hashes and required keys is passed to the recipients as "[file description](#file-description)" out-of-band.
To send the actual file, the sender client MUST pad it and encrypt it with a random symmetric key and distribute packets of fixed sized across multiple XFTP routers. Information about packet locations, keys, hashes and required keys is passed to the recipients as "[file description](#file-description)" out-of-band.
Creating, uploading, downloading and deleting file chunks requires sending commands to the XFTP server - they are described in detail in [XFTP commands](#xftp-commands) section.
Creating, uploading, downloading and deleting data packets requires sending commands to the XFTP router - they are described in detail in [XFTP commands](#xftp-commands) section.
## Persistence model
Server stores file chunk records in memory, with optional adding to append-only log, to allow restoring them on server restart. File chunk bodies can be stored as files or as objects in any object store (e.g. S3).
Router stores data packet records in memory, with optional adding to append-only log, to allow restoring them on router restart. Data packet bodies can be stored as files or as objects in any object store (e.g. S3).
## XFTP procedure
@@ -107,28 +116,28 @@ To send the file, the sender will:
1) Prepare file
- compute its SHA512 digest.
- prepend header with the name and pad the file to match the whole number of chunks in size. It is RECOMMENDED to use 2 of 4 allowed chunk sizes, to balance upload size and metadata privacy.
- prepend header with the name and pad the file to match the whole number of packets in size. It is RECOMMENDED to use 2 of 4 allowed packet sizes, to balance upload size and metadata privacy.
- encrypt it with a randomly chosen symmetric key and IV (e.g., using NaCL secret_box).
- split into allowed size chunks.
- split into allowed size packets.
- generate per-recipient keys. It is recommended that the sending client generates more per-recipient keys than the actual number of recipients, rounding up to a power of 2, to conceal the actual number of intended recipients.
2) Upload file chunks
- register each chunk record with randomly chosen one or more (for redundancy) XFTP server(s).
2) Upload data packets
- register each packet record with randomly chosen one or more (for redundancy) XFTP router(s).
- optionally request additional recipient IDs, if required number of recipient keys didn't fit into register request.
- upload each chunk to chosen server(s).
- upload each packet to chosen router(s).
3) Prepare file descriptions, one per recipient.
The sending client combines addresses of all chunks and other information into "file description", different for each file recipient, that will include:
The sending client combines addresses of all packets and other information into "file description", different for each file recipient, that will include:
- an encryption key used to encrypt/decrypt the full file (the same for all recipients).
- file SHA512 digest to validate download.
- list of chunk descriptions; information for each chunk:
- private Ed25519 key to sign commands for file transfer server.
- chunk address (server host and chunk ID).
- chunk sha512 digest.
- list of packet descriptions; information for each packet:
- private Ed25519 key to sign commands for file transfer router.
- packet address (router host and packet ID).
- packet sha256 digest.
To reduce the size of file description, chunks are grouped by the server host.
To reduce the size of file description, packets are grouped by the router host.
4) Send file description(s) to the recipient(s) out-of-band, via pre-existing secure and authenticated channel. E.g., SimpleX clients send it as messages via SMP protocol, but it can be done via any other channel.
@@ -138,16 +147,16 @@ To reduce the size of file description, chunks are grouped by the server host.
Having received the description, the recipient will:
1) Download all chunks.
1) Download all packets.
The receiving client can fall back to secondary servers, if necessary:
- if the server is not available.
- if the chunk is not present on the server (ERR AUTH response).
- if the hash of the downloaded file chunk does not match the description.
The receiving client can fall back to secondary routers, if necessary:
- if the router is not available.
- if the packet is not present on the router (ERR AUTH response).
- if the hash of the downloaded data packet does not match the description.
Optionally recipient can acknowledge file chunk reception to delete file ID from server for this recipient.
Optionally recipient can acknowledge data packet reception to delete file ID from router for this recipient.
2) Combine the chunks into a file.
2) Combine the packets into a file.
3) Decrypt the file using the key in file description.
@@ -163,35 +172,35 @@ Optionally recipient can acknowledge file chunk reception to delete file ID from
It includes these fields:
- `party` - "sender" or "recipient". Sender's file description is required to delete the file.
- `size` - padded file size equal to total size of all chunks, see `fileSize` syntax below.
- `size` - padded file size equal to total size of all packets, see `fileSize` syntax below.
- `digest` - SHA512 hash of encrypted file, base64url encoded string.
- `key` - symmetric encryption key to decrypt the file, base64url encoded string.
- `nonce` - nonce to decrypt the file, base64url encoded string.
- `chunkSize` - default chunk size, see `fileSize` syntax below.
- `replicas` - the array of file chunk replicas descriptions.
- `chunkSize` - default packet size, see `fileSize` syntax below.
- `replicas` - the array of data packet replicas descriptions.
- `redirect` - optional property for redirect information indicating that the file is itself a description to another file, allowing to use file description as a short URI.
Each replica description is an object with 2 fields:
- `chunks` - and array of chunk replica descriptions stored on one server.
- `server` - [server address](#xftp-server-uri) where the chunks can be downloaded from.
- `chunks` - an array of packet replica descriptions stored on one server.
- `server` - [router address](#xftp-router-uri) where the packets can be downloaded from.
Each server replica description is a string with this syntax:
Each router replica description is a string with this syntax:
```abnf
chunkReplica = chunkNo ":" replicaId ":" replicaKey [":" chunkDigest [":" chunkSize]]
chunkNo = 1*DIGIT
; a sequential 1-based chunk number in the original file.
packetReplica = packetNo ":" replicaId ":" replicaKey [":" packetDigest [":" packetSize]]
packetNo = 1*DIGIT
; a sequential 1-based packet number in the original file.
replicaId = base64url
; server-assigned random chunk replica ID.
; router-assigned random packet replica ID.
replicaKey = base64url
; sender-generated random key to receive (or to delete, in case of sender's file description) the chunk replica.
chunkDigest = base64url
; chunk digest that MUST be specified for the first replica of each chunk,
; sender-generated random key to receive (or to delete, in case of sender's file description) the packet replica.
packetDigest = base64url
; packet digest that MUST be specified for the first replica of each packet,
; and SHOULD be omitted (or be the same) on the subsequent replicas
chunkSize = fileSize
packetSize = fileSize
fileSize = sizeInBytes / sizeInUnits
; chunk size SHOULD only be specified on the first replica and only if it is different from default chunk size
; packet size SHOULD only be specified on the first replica and only if it is different from default packet size
sizeInBytes = 1*DIGIT
sizeInUnits = 1*DIGIT sizeUnit
sizeUnit = %s"kb" / %s"mb" / %s"gb"
@@ -204,28 +213,28 @@ Optional redirect information has two fields:
## URIs syntax
### XFTP server URI
### XFTP router URI
The XFTP server address is a URI with the following syntax:
The XFTP router address is a URI with the following syntax:
```abnf
xftpServerURI = %s"xftp://" xftpServer
xftpServer = serverIdentity [":" basicAuth] "@" srvHost [":" port]
xftpRouterURI = %s"xftp://" xftpRouter
xftpRouter = routerIdentity [":" basicAuth] "@" srvHost [":" port]
srvHost = <hostname> ; RFC1123, RFC5891
port = 1*DIGIT
serverIdentity = base64url
routerIdentity = base64url
basicAuth = base64url
```
### File description URI
This file description URI can be generated by the client application to share a small file description as a QR code or as a link. Practically, to be able to scan a QR code it should be under 1000 characters, so only file descriptions with 1-2 chunks can be used in this case. This is supported with `redirect` property when file description leads to a file which in itself is a larger file description to another file - akin to URL shortener.
This file description URI can be generated by the client application to share a small file description as a QR code or as a link. Practically, to be able to scan a QR code it should be under 1000 characters, so only file descriptions with 1-2 packets can be used in this case. This is supported with `redirect` property when file description leads to a file which in itself is a larger file description to another file - akin to URL shortener.
File description URI syntax:
```abnf
fileDescriptionURI = serviceScheme "/file" "#/?desc=" description [ "&data=" userData ]
serviceScheme = (%s"https://" clientAppServer) | %s"simplex:"
serviceScheme = (%s"https://" clientAppServer) / %s"simplex:"
clientAppServer = hostname [ ":" port ]
; client app server, e.g. simplex.chat
description = <URI-escaped YAML file description>
@@ -240,50 +249,50 @@ clientAppServer is not a server the client connects to - it is a server that sho
XFTP stands for SimpleX File Transfer Protocol. Its design is based on the same ideas and has some of the qualities of SimpleX Messaging Protocol:
- recipient cannot see sender's IP address, as the file fragments (chunks) are temporarily stored on multiple XFTP relays.
- recipient cannot see sender's IP address, as the file fragments (packets) are temporarily stored on multiple XFTP relays.
- file can be sent asynchronously, without requiring the sender to be online for file to be received.
- there is no network of peers that can observe this transfer - sender chooses which XFTP relays to use, and can self-host their own.
- XFTP relays do not have any file metadata - they only see individual chunks, with access to each chunk authorized with anonymous credentials (using Edwards curve cryptographic signature) that are random per chunk.
- chunks have one of the sizes allowed by the servers - 64KB, 256KB, 1MB and 4MB chunks, so sending a large file looks indistinguishable from sending many small files to XFTP server. If the same transport connection is reused, server would only know that chunks are sent by the same user.
- each chunk can be downloaded by multiple recipients, but each recipient uses their own key and chunk ID to authorize access, and the chunk is encrypted by a different key agreed via ephemeral DH keys (NaCl crypto_box (SalsaX20Poly1305 authenticated encryption scheme ) with shared secret derived from Curve25519 key exchange) on the way from the server to each recipient. XFTP protocol as a result has the same quality as SMP protocol - there are no identifiers and ciphertext in common between sent and received traffic inside TLS connection, so even if TLS is compromised, it complicates traffic correlation attacks.
- XFTP protocol supports redundancy - each file chunk can be sent via multiple relays, and the recipient can choose the one that is available. Current implementation of XFTP protocol in SimpleX Chat does not support redundancy though.
- XFTP relays do not have any file metadata - they only see individual packets, with access to each packet authorized with anonymous credentials (using Edwards curve cryptographic signature) that are random per packet.
- packets have one of the sizes allowed by the routers - 64KB, 256KB, 1MB and 4MB packets, so sending a large file looks indistinguishable from sending many small files to XFTP router. If the same transport connection is reused, router would only know that packets are sent by the same user.
- each packet can be downloaded by multiple recipients, but each recipient uses their own key and packet ID to authorize access, and the packet is encrypted by a different key agreed via ephemeral DH keys (NaCl crypto_box (SalsaX20Poly1305 authenticated encryption scheme ) with shared secret derived from Curve25519 key exchange) on the way from the router to each recipient. XFTP protocol as a result has the same quality as SMP protocol - there are no identifiers and ciphertext in common between sent and received traffic inside TLS connection, so even if TLS is compromised, it complicates traffic correlation attacks.
- XFTP protocol supports redundancy - each data packet can be sent via multiple relays, and the recipient can choose the one that is available. Current implementation of XFTP protocol in SimpleX Chat does not support redundancy though.
- the file as a whole is encrypted with a random symmetric key using NaCl secret_box.
## Cryptographic algorithms
Clients must cryptographically authorize XFTP commands, see [Command authentication](#command-authentication).
To authorize/verify transmissions clients and servers MUST use either signature algorithm Ed25519 algorithm defined in RFC8709 or using deniable authentication scheme based on NaCL crypto_box (see Simplex Messaging Protocol).
To authorize/verify transmissions clients and routers MUST use either signature algorithm Ed25519 algorithm defined in RFC8709 or using deniable authentication scheme based on NaCL crypto_box (see Simplex Messaging Protocol).
To encrypt/decrypt file chunk bodies delivered to the recipients, servers/clients MUST use NaCL crypto_box.
To encrypt/decrypt data packet bodies delivered to the recipients, routers/clients MUST use NaCL crypto_box.
Clients MUST encrypt file chunk bodies sent via XFTP servers using use NaCL crypto_box.
Clients MUST encrypt data packet bodies sent via XFTP routers using use NaCL crypto_box.
## File chunk IDs
## Data packet IDs
XFTP servers MUST generate a separate new set of IDs for each new chunk - for the sender (that uploads the chunk) and for each intended recipient. It is REQUIRED that:
XFTP routers MUST generate a separate new set of IDs for each new packet - for the sender (that uploads the packet) and for each intended recipient. It is REQUIRED that:
- These IDs are different and unique within the server.
- These IDs are different and unique within the router.
- Based on random bytes generated with cryptographically strong pseudo-random number generator.
## Server security requirements
## Router security requirements
XFTP server implementations MUST NOT create, store or send to any other servers:
XFTP router implementations MUST NOT create, store or send to any other routers:
- Logs of the client commands and transport connections in the production environment.
- History of retrieved files.
- Snapshots of the database they use to store file chunks (instead clients can manage redundancy by creating chunk replicas using more than one XFTP server). In-memory persistence is recommended for file chunks records.
- Snapshots of the database they use to store data packets (instead clients can manage redundancy by creating packet replicas using more than one XFTP router). In-memory persistence is recommended for data packets records.
- Any other information that may compromise privacy or [forward secrecy][4] of communication between clients using XFTP servers.
- Any other information that may compromise privacy or [forward secrecy][4] of communication between clients using XFTP routers.
## Transport protocol
- binary-encoded commands sent as fixed-size padded block in the body of HTTP2 POST request, similar to SMP and notifications server protocol transmission encodings.
- binary-encoded commands sent as fixed-size padded block in the body of HTTP2 POST request, similar to SMP and notifications router protocol transmission encodings.
- HTTP2 POST with a fixed size padded block body for file upload and download.
Block size - 4096 bytes (it would fit ~120 Ed25519 recipient keys).
Block size - 16384 bytes (it would fit ~350 Ed25519 recipient keys).
The reasons to use HTTP2:
@@ -299,40 +308,41 @@ The reason not to use URI segments / HTTP verbs / REST semantics is to have cons
### ALPN to agree handshake version
Client and server use [ALPN extension][18] of TLS to agree handshake version.
Client and router use [ALPN extension][18] of TLS to agree handshake version.
Server SHOULD send `xftp/1` protocol name and the client should confirm this name in order to use the current protocol version. This is added to allow support of older clients without breaking backward compatibility and to extend or modify handshake syntax.
Router SHOULD send `xftp/1` protocol name and the client should confirm this name in order to use the current protocol version. This is added to allow support of older clients without breaking backward compatibility and to extend or modify handshake syntax.
If the client does not confirm this protocol name, the server would fall back to v1 of XFTP protocol.
If the client does not confirm this protocol name, the router would fall back to v1 of XFTP protocol.
### Transport handshake
When a client and a server agree on handshake version using ALPN extension, they should proceed with XFTP handshake.
When a client and a router agree on handshake version using ALPN extension, they should proceed with XFTP handshake.
As with SMP, a client doesn't reveal its version range to avoid version fingerprinting. Unlike SMP, XFTP runs a HTTP2 protocol over TLS and the server can't just send its handshake right away. So a session handshake is driven by client-sent requests:
As with SMP, a client doesn't reveal its version range to avoid version fingerprinting. Unlike SMP, XFTP runs a HTTP2 protocol over TLS and the router can't just send its handshake right away. So a session handshake is driven by client-sent requests:
1. To pass initiative to the server, the client sends a request with empty body.
2. Server responds with its `paddedServerHello` block.
1. To pass initiative to the router, the client sends a request with empty body.
2. Router responds with its `paddedRouterHello` block.
3. Clients sends a request containing `paddedClientHello` block,
4. Server sends an empty response, finalizing the handshake.
4. Router sends an empty response, finalizing the handshake.
Once TLS handshake is complete, client and server will exchange blocks of fixed size (16384 bytes).
Once TLS handshake is complete, client and router will exchange blocks of fixed size (16384 bytes).
```abnf
paddedServerHello = <padded(serverHello, 16384)>
serverHello = xftpVersionRange sessionIdentifier serverCert signedServerKey ignoredPart
paddedRouterHello = <padded(routerHello, 16384)>
routerHello = xftpVersionRange sessionIdentifier routerCerts signedRouterKey ignoredPart
xftpVersionRange = minXftpVersion maxXftpVersion
minXftpVersion = xftpVersion
maxXftpVersion = xftpVersion
sessionIdentifier = shortString
; unique session identifier derived from transport connection handshake
serverCert = originalLength <x509encoded>
signedServerKey = originalLength <x509encoded> ; signed by server certificate
routerCerts = length 1*routerCert ; NonEmpty list of certificates in chain
routerCert = originalLength <x509encoded>
signedRouterKey = originalLength <x509encoded> ; signed by router certificate
paddedClientHello = <padded(clientHello, 16384)>
clientHello = xftpVersion keyHash ignoredPart
; chosen XFTP protocol version - must be the maximum supported version
; within the range offered by the server
; within the range offered by the router
xftpVersion = 2*2OCTET ; Word16 version number
keyHash = shortString
@@ -342,47 +352,47 @@ originalLength = 2*2OCTET
ignoredPart = *OCTET
```
In XFTP v2 the handshake is only used for version negotiation, but `serverCert` and `signedServerKey` must be validated by the client.
In XFTP v2 the handshake is only used for version negotiation, but `routerCert` and `signedRouterKey` must be validated by the client.
`keyHash` is the CA fingerprint used by client to validate TLS certificate chain and is checked by a server against its own key.
`keyHash` is the CA fingerprint used by client to validate TLS certificate chain and is checked by a router against its own key.
`ignoredPart` in handshake allows to add additional parameters in handshake without changing protocol version - the client and servers must ignore any extra bytes within the original block length.
`ignoredPart` in handshake allows to add additional parameters in handshake without changing protocol version - the client and routers must ignore any extra bytes within the original block length.
For TLS transport client should assert that `sessionIdentifier` is equal to `tls-unique` channel binding defined in [RFC 5929][14] (TLS Finished message struct); we pass it in `serverHello` block to allow communication over some other transport protocol (possibly, with another channel binding).
For TLS transport client should assert that `sessionIdentifier` is equal to `tls-unique` channel binding defined in [RFC 5929][14] (TLS Finished message struct); we pass it in `routerHello` block to allow communication over some other transport protocol (possibly, with another channel binding).
### Requests and responses
- File sender:
- create file chunk record.
- create data packet record.
- Parameters:
- Ed25519 key for subsequent sender commands and Ed25519 keys for commands of each recipient.
- chunk size.
- packet size.
- Response:
- chunk ID for the sender and different IDs for all recipients.
- add recipients to file chunk
- packet ID for the sender and different IDs for all recipients.
- add recipients to data packet
- Parameters:
- sender's chunk ID
- sender's packet ID
- Ed25519 keys for commands of each recipient.
- Response:
- chunk IDs for new recipients.
- upload file chunk.
- delete file chunk (invalidates all recipient IDs).
- packet IDs for new recipients.
- upload data packet.
- delete data packet (invalidates all recipient IDs).
- File recipient:
- download file chunk:
- chunk ID
- DH key for additional encryption of the chunk.
- command should be signed with the key passed by the sender when creating chunk record.
- delete file chunk ID (only for one recipient): signed with the same key.
- download data packet:
- packet ID
- DH key for additional encryption of the packet.
- command should be signed with the key passed by the sender when creating packet record.
- delete data packet ID (only for one recipient): signed with the same key.
## XFTP commands
Commands syntax below is provided using ABNF with case-sensitive strings extension.
```abnf
xftpCommand = ping / senderCommand / recipientCmd / serverMsg
xftpCommand = ping / senderCommand / recipientCmd / routerMsg
senderCommand = register / add / put / delete
recipientCmd = get / ack
serverMsg = pong / sndIds / rcvIds / ok / file
routerMsg = pong / sndIds / rcvIds / ok / file / error
```
The syntax of specific commands and responses is defined below.
@@ -393,11 +403,11 @@ Commands are made via HTTP2 requests, responses to commands are correlated as HT
### Command authentication
XFTP servers must authenticate all transmissions (excluding `ping`) by verifying the client signatures. Command signature should be generated by applying the algorithm specified for the file to the `signed` block of the transmission, using the key associated with the file chunk ID (recipient's or sender's depending on which file chunk ID is used).
XFTP routers must authenticate all transmissions (excluding `ping`) by verifying the client signatures. Command signature should be generated by applying the algorithm specified for the file to the `signed` block of the transmission, using the key associated with the data packet ID (recipient's or sender's depending on which data packet ID is used).
### Keep-alive command
To keep the transport connection alive and to generate noise traffic the clients should use `ping` command to which the server responds with `pong` response. This command should be sent unsigned and without file chunk ID.
To keep the transport connection alive and to generate noise traffic the clients should use `ping` command to which the router responds with `pong` response. This command should be sent unsigned and without data packet ID.
```abnf
ping = %s"PING"
@@ -405,21 +415,19 @@ ping = %s"PING"
This command is always sent unsigned.
data FileResponse = ... | FRPong | ...
```abnf
pong = %s"PONG"
```
### File sender commands
Sending any of the commands in this section (other than `register`, that is sent without file chunk ID) is only allowed with sender's ID.
Sending any of the commands in this section (other than `register`, that is sent without data packet ID) is only allowed with sender's ID. The `register` command must be signed (using `sndKey` included in `fileInfo` for verification) but must NOT include a data packet ID.
#### Register new file chunk
#### Register new data packet
This command is sent by the sender to the XFTP server to register a new file chunk.
This command is sent by the sender to the XFTP router to register a new data packet.
Servers SHOULD support basic auth with this command, to allow only server owners and trusted users to create file chunks on the servers.
Routers SHOULD support basic auth with this command, to allow only router owners and trusted users to create data packets on the routers.
The syntax is:
@@ -427,7 +435,7 @@ The syntax is:
register = %s"FNEW " fileInfo rcvPublicAuthKeys basicAuth
fileInfo = sndKey size digest
sndKey = length x509encoded
size = 1*DIGIT
size = 4*4 OCTET ; Word32 big-endian
digest = length *OCTET
rcvPublicAuthKeys = length 1*rcvPublicAuthKey
rcvPublicAuthKey = length x509encoded
@@ -438,7 +446,7 @@ x509encoded = <binary X509 key encoding>
length = 1*1 OCTET
```
If the file chunk is registered successfully, the server must send `sndIds` response with the sender's and recipients' file chunk IDs:
If the data packet is registered successfully, the router must send `sndIds` response with the sender's and recipients' data packet IDs:
```abnf
sndIds = %s"SIDS " senderId recipientIds
@@ -447,9 +455,9 @@ recipientIds = length 1*recipientId
recipientId = length *OCTET
```
#### Add file chunk recipients
#### Add data packet recipients
This command is sent by the sender to the XFTP server to add additional recipient keys to the file chunk record, in case number of keys requested by client didn't fit into `register` command. The syntax is:
This command is sent by the sender to the XFTP router to add additional recipient keys to the data packet record, in case number of keys requested by client didn't fit into `register` command. The syntax is:
```abnf
add = %s"FADD " rcvPublicAuthKeys
@@ -457,7 +465,7 @@ rcvPublicAuthKeys = length 1*rcvPublicAuthKey
rcvPublicAuthKey = length x509encoded
```
If additional keys were added successfully, the server must send `rcvIds` response with the added recipients' file chunk IDs:
If additional keys were added successfully, the router must send `rcvIds` response with the added recipients' data packet IDs:
```abnf
rcvIds = %s"RIDS " recipientIds
@@ -465,66 +473,100 @@ recipientIds = length 1*recipientId
recipientId = length *OCTET
```
#### Upload file chunk
#### Upload data packet
This command is sent by the sender to the XFTP server to upload file chunk body to server. The syntax is:
This command is sent by the sender to the XFTP router to upload data packet body to router. The syntax is:
```abnf
put = %s"FPUT"
```
Chunk body is streamed via HTTP2 request.
Packet body is streamed via HTTP2 request.
If file chunk body was successfully received, the server must send `ok` response.
If data packet body was successfully received, the router must send `ok` response.
```abnf
ok = %s"OK"
```
#### Delete file chunk
#### Delete data packet
This command is sent by the sender to the XFTP server to delete file chunk from the server. The syntax is:
This command is sent by the sender to the XFTP router to delete data packet from the router. The syntax is:
```abnf
delete = %s"FDEL"
```
Server should delete file chunk record, invalidating all recipient IDs, and delete file body from file storage. If file chunk was successfully deleted, the server must send `ok` response.
Router should delete data packet record, invalidating all recipient IDs, and delete file body from file storage. If data packet was successfully deleted, the router must send `ok` response.
### File recipient commands
Sending any of the commands in this section is only allowed with recipient's ID.
#### Download file chunk
#### Download data packet
This command is sent by the recipient to the XFTP server to download file chunk body from the server. The syntax is:
This command is sent by the recipient to the XFTP router to download data packet body from the router. The syntax is:
```abnf
get = %s"FGET " rDhKey
rDhKey = length x509encoded
```
If requested file is successfully located, the server must send `file` response. File chunk body is sent as HTTP2 response body.
If requested file is successfully located, the router must send `file` response. Data packet body is sent as HTTP2 response body.
```abnf
file = %s"FILE " sDhKey cbNonce
sDhKey = length x509encoded
cbNonce = <nonce used in NaCl crypto_box encryption scheme>
cbNonce = 24*24 OCTET ; NaCl crypto_box nonce
```
Chunk is additionally encrypted on the way from the server to the recipient using a key agreed via ephemeral DH keys `rDhKey` and `sDhKey`, so there is no ciphertext in common between sent and received traffic inside TLS connection, in order to complicate traffic correlation attacks, if TLS is compromised.
Packet is additionally encrypted on the way from the router to the recipient using a key agreed via ephemeral DH keys `rDhKey` and `sDhKey`, so there is no ciphertext in common between sent and received traffic inside TLS connection, in order to complicate traffic correlation attacks, if TLS is compromised.
#### Acknowledge file chunk download
#### Acknowledge data packet download
This command is sent by the recipient to the XFTP server to acknowledge file reception, deleting file ID from server for this recipient. The syntax is:
This command is sent by the recipient to the XFTP router to acknowledge file reception, deleting file ID from router for this recipient. The syntax is:
```abnf
ack = %s"FACK"
```
If file recipient ID is successfully deleted, the server must send `ok` response.
If file recipient ID is successfully deleted, the router must send `ok` response.
In current implementation of XFTP protocol in SimpleX Chat clients don't use FACK command. Files are automatically expired on servers after configured time interval.
In current implementation of XFTP protocol in SimpleX Chat clients don't use FACK command. Files are automatically expired on routers after configured time interval.
### Error responses
The router responds with `ERR` followed by the error type:
```abnf
error = %s"ERR " errorType
errorType = %s"BLOCK" / %s"SESSION" / %s"HANDSHAKE" /
%s"CMD" SP cmdError / %s"AUTH" / %s"BLOCKED" SP blockingInfo /
%s"SIZE" / %s"QUOTA" / %s"DIGEST" / %s"CRYPTO" /
%s"NO_FILE" / %s"HAS_FILE" / %s"FILE_IO" /
%s"TIMEOUT" / %s"INTERNAL"
cmdError = %s"UNKNOWN" / %s"SYNTAX" / %s"PROHIBITED" / %s"NO_AUTH" / %s"HAS_AUTH" / %s"NO_ENTITY"
blockingInfo = %s"reason=" blockingReason ["," %s"notice=" jsonNotice]
blockingReason = %s"spam" / %s"content"
jsonNotice = *OCTET ; JSON-encoded notice object
```
Error types:
- `BLOCK` - incorrect block format, encoding or signature size.
- `SESSION` - incorrect session ID (TLS Finished message / tls-unique binding).
- `HANDSHAKE` - incorrect handshake command.
- `CMD` - command syntax errors (UNKNOWN, SYNTAX, PROHIBITED, NO_AUTH, HAS_AUTH, NO_ENTITY).
- `AUTH` - command authorization error - bad signature or non-existing data packet.
- `BLOCKED` - data packet was blocked due to policy violation (added in v3). Contains blocking reason and optional notice.
- `SIZE` - incorrect file size.
- `QUOTA` - storage quota exceeded.
- `DIGEST` - incorrect file digest.
- `CRYPTO` - file encryption/decryption failed.
- `NO_FILE` - no expected file body in request/response or no file on the router.
- `HAS_FILE` - unexpected file body.
- `FILE_IO` - file IO error.
- `TIMEOUT` - file sending or receiving timeout.
- `INTERNAL` - internal router error.
## Threat model
@@ -533,7 +575,7 @@ In current implementation of XFTP protocol in SimpleX Chat clients don't use FAC
- A user protects their local database and key material.
- The user's application is authentic, and no local malware is running.
- The cryptographic primitives in use are not broken.
- A user's choice of servers is not directly tied to their identity or otherwise represents distinguishing information about the user.
- A user's choice of routers is not directly tied to their identity or otherwise represents distinguishing information about the user.
#### A passive adversary able to monitor the traffic of one user
@@ -541,7 +583,7 @@ In current implementation of XFTP protocol in SimpleX Chat clients don't use FAC
- identify that and when a user is sending files over XFTP protocol.
- determine which servers the user sends/receives files to/from.
- determine which routers the user sends/receives files to/from.
- observe how much traffic is being sent, and make guesses as to its purpose.
@@ -553,11 +595,11 @@ In current implementation of XFTP protocol in SimpleX Chat clients don't use FAC
*can:*
- learn which XFTP servers are used to send and receive files for which users.
- learn which XFTP routers are used to send and receive files for which users.
- learn when files are sent and received.
- perform traffic correlation attacks against senders and recipients and correlate senders and recipients within the monitored set, frustrated by the number of users on the servers.
- perform traffic correlation attacks against senders and recipients and correlate senders and recipients within the monitored set, frustrated by the number of users on the routers.
- observe how much traffic is being sent, and make guesses as to its purpose.
@@ -567,31 +609,31 @@ In current implementation of XFTP protocol in SimpleX Chat clients don't use FAC
- perform traffic correlation attacks.
#### XFTP server
#### XFTP router
*can:*
- learn when file senders and recipients are online.
- know how many file chunks and chunk sizes are sent via the server.
- know how many data packets and packet sizes are sent via the router.
- perform the correlation of the file chunks as belonging to one file via either a re-used transport connection, user's IP address, or connection timing regularities.
- perform the correlation of the data packets as belonging to one file via either a re-used transport connection, user's IP address, or connection timing regularities.
- learn file senders' and recipients' IP addresses, and infer information (e.g. employer) based on the IP addresses, as long as Tor is not used.
- delete file chunks, preventing file delivery, as long as redundant delivery is not used.
- delete data packets, preventing file delivery, as long as redundant delivery is not used.
- lie about the state of a file chunk to the recipient and/or to the sender (e.g. deleted when it is not).
- lie about the state of a data packet to the recipient and/or to the sender (e.g. deleted when it is not).
- refuse deleting the file when instructed by the sender.
*cannot:*
- undetectably corrupt file chunks.
- undetectably corrupt data packets.
- learn the contents, name or the exact size of sent files.
- learn approximate size of sent files, as long as more than one server is used to send file chunks.
- learn approximate size of sent files, as long as more than one router is used to send data packets.
- compromise the users' end-to-end encryption of files with an active attack.
@@ -603,7 +645,7 @@ In current implementation of XFTP protocol in SimpleX Chat clients don't use FAC
- receive all files sent and received by Alice that did not expire yet, as long as information about these files was not removed from the database.
- prevent Alice's contacts from receiving the files she sent by deleting all or some of the file chunks from XFTP servers.
- prevent Alice's contacts from receiving the files she sent by deleting all or some of the data packets from XFTP routers.
#### A user's contact
@@ -625,10 +667,10 @@ In current implementation of XFTP protocol in SimpleX Chat clients don't use FAC
*can:*
- Denial of Service XFTP servers.
- Denial of Service XFTP routers.
*cannot:*
- send files to a user who they are not connected with.
- enumerate file chunks on an XFTP server.
- enumerate data packets on an XFTP router.
+13 -19
View File
@@ -10,7 +10,7 @@ Version 1, 2024-06-22
- [Session invitation](#session-invitation)
- [Establishing TLS connection](#establishing-tls-connection)
- [Session verification and protocol negotiation](#session-verification-and-protocol-negotiation)
- [Controller/host session operation](#сontrollerhost-session-operation)
- [Controller/host session operation](#controllerhost-session-operation)
- [Key agreement for announcement packet and for session](#key-agreement-for-announcement-packet-and-for-session)
- [Threat model](#threat-model)
@@ -104,12 +104,11 @@ Multicast session announcement is a binary encoded packet with this syntax:
```abnf
sessionAddressPacket = dhPubKey nonce encrypted(unpaddedSize sessionAddress packetPad)
dhPubKey = length x509encoded ; same as announced
nonce = length *OCTET
sessionAddress = largeLength sessionAddressUri ; as above
nonce = 24*24 OCTET ; NaCl 192-bit nonce, no length prefix
sessionAddress = sessionAddressUri ; length given by unpaddedSize
length = 1*1 OCTET ; for binary data up to 255 bytes
largeLength = 2*2 OCTET ; for binary data up to 65535 bytes
packetPad = <pad packet size to 1450 bytes> ; possibly, we may need to move KEM agreement one step later,
; with encapsulation key in HELLO block and KEM ciphertext in reply to HELLO.
packetPad = <pad invitation content to 900 bytes before encryption>
```
### Establishing TLS connection
@@ -143,7 +142,7 @@ hostHello = %s"HELLO " dhPubKey nonce encrypted(unpaddedSize hostHelloJSON hello
unpaddedSize = largeLength
dhPubKey = length x509encoded
pad = <pad block size to 16384 bytes>
helloPad = <pad hello size to 12888 bytes>
helloPad = <pad hello size to 12288 bytes>
largeLength = 2*2 OCTET
```
@@ -157,10 +156,7 @@ The controller decrypts (including the first session) and validates the received
{
"definitions": {
"version": {
"type": "string",
"metadata": {
"format": "[0-9]+"
}
"type": "uint16"
},
"base64url": {
"type": "string",
@@ -172,9 +168,7 @@ The controller decrypts (including the first session) and validates the received
"properties": {
"v": {"ref": "version"},
"ca": {"ref": "base64url"},
"kem": {"ref": "base64url"}
},
"optionalProperties": {
"kem": {"ref": "base64url"},
"app": {"properties": {}, "additionalProperties": true}
},
"additionalProperties": true
@@ -190,7 +184,7 @@ ctrlHello = %s"HELLO " kemCiphertext encrypted(unpaddedSize ctrlHelloJSON helloP
unpaddedSize = largeLength
kemCiphertext = largeLength *OCTET
pad = <pad block size to 16384 bytes>
helloPad = <pad hello size to 12888 bytes>
helloPad = <pad hello size to 12288 bytes>
largeLength = 2*2 OCTET
ctrlError = %s"ERROR " nonce encrypted(unpaddedSize ctrlErrorMessage helloPad) pad
@@ -206,7 +200,7 @@ JTD schema for the encrypted part of controller HELLO block `ctrlHelloJSON`:
}
```
Controller `hello` block and all subsequent protocol messages are encrypted with the chain keys derived from the hybrid key (see key exchange below) - that is why conntroller hello block does not include nonce. That provides forward secrecy within the XRCP session. Receiving this `hello` block allows host to compute the same hybrid keys and to derive the same chain keys.
Controller `hello` block and all subsequent protocol messages are encrypted with the chain keys derived from the hybrid key (see key exchange below) - that is why controller hello block does not include nonce. That provides forward secrecy within the XRCP session. Receiving this `hello` block allows host to compute the same hybrid keys and to derive the same chain keys.
Once the controller replies HELLO to the valid host HELLO block, it should stop accepting new TCP connections.
@@ -261,7 +255,7 @@ kemCiphertext(1) = enc(kemSecret(1), kemEncKey(1))
kemSecret(1) = dec(kemCiphertext(1), kemDecKey(1))
// multicast announcement for session n
announcementSecret(n) = sha256(dhSecret(n'))
announcementSecret(n) = dhSecret(n')
dhSecret(n') = dh(hostHelloDhKey(n - 1), controllerDhKey(n))
// session n
@@ -277,11 +271,11 @@ If controller fails to store the new host DH key after receiving HELLO block, th
To decrypt a multicast announcement, the host should try to decrypt it using the keys of all known (paired) remote controllers.
Once kemSecret is agreed for the session, it is used to derive two chain keys, to receive and to send messages:
Once sessionSecret is agreed for the session, it is used to derive two chain keys, to receive and to send messages:
```
host: sndKey, rcvKey = HKDF(kemSecret, "SimpleXSbChainInit", 64)
controller: rcvKey, sndKey = HKDF(kemSecret, "SimpleXSbChainInit", 64)
controller: sndKey, rcvKey = HKDF(sessionSecret, "SimpleXSbChainInit", 64)
host: rcvKey, sndKey = HKDF(sessionSecret, "SimpleXSbChainInit", 64)
```
where HKDF is based on SHA512, with empty salt.
+2 -2
View File
@@ -3,9 +3,9 @@
## Problem
When sending an SMP confirmation a network timeout can lead to the following race condition:
- server receives the confirmation while the joining party fails to receive the server's response;
- router receives the confirmation while the joining party fails to receive the router's response;
- joining party deletes the connection together with credentials sent in the confirmation for securing the queue;
- initiating party will receive the confirmation from the server and secure the queue;
- initiating party will receive the confirmation from the router and secure the queue;
- on subsequent attempt to join via the same invitation link initiating party will generate new credentials and fail authorization.
This renders the joining party permanently unable to join via that invitation link and complete the connection.
+2 -2
View File
@@ -3,12 +3,12 @@
## Problem
iOS notifications may fail to deliver for several reasons, but there are two important reasons that we could address:
- when notification server is not subscribed to SMP server(s), the notifications can be dropped - it can happen because either notification server restarts or becuase SMP server restarted and some messages are received before notification server resubscribed. We lose approximately 3% of notifications because of this reason.
- when notification router is not subscribed to SMP router(s), the notifications can be dropped - it can happen because either notification router restarts or becuase SMP router restarted and some messages are received before notification router resubscribed. We lose approximately 3% of notifications because of this reason.
- when user device is offline or has low power condition, Apple does not deliver notification, but puts them to storage. If while the notification is in storage a new one arrives it would overwrite the previous notification. If it was the message to the same message queue, the client will download messages anyway, up to a limit, but if the message was to another queue, it will not be delivered until the app is opened. Apple delivers about 88% of notifications that should be delivered (not accounting for uninstalled apps), the rest is replaced with the newer notifications.
## Solution
The first problem can be solved by preserving notifications for a limited time (say 1 hour) in case there is no subscription to notification from notification server. At the very least, they can be preserved in SMP server memory but can also be stored to a file on restart, similar to messages, and be delivered when notification server resubscribes. It is sufficient to store one notification per messaging queue.
The first problem can be solved by preserving notifications for a limited time (say 1 hour) in case there is no subscription to notification from notification router. At the very least, they can be preserved in SMP router memory but can also be stored to a file on restart, similar to messages, and be delivered when notification router resubscribes. It is sufficient to store one notification per messaging queue.
The second problem is both more damaging and more complex to solve. The solution could be to always deliver several last notifications to different queues in one packet (Apple allows up to ~4-5kb notification size, and we are sending packets of fixed size 512 bytes, so we could fit up to 8-10 of them in each notification).
+12 -12
View File
@@ -8,7 +8,7 @@ See [Short invitation links](./2024-06-21-short-links.md).
2) clients only delete queue records based on some user action, pending connections do not expire.
While part 2 should be improved in the client, indefinite storage of queue records becomes a much bigger issue if each of them would result in a permanent storage of 4-16kb blob in server memory, without server-side expiration for short invitation links.
While part 2 should be improved in the client, indefinite storage of queue records becomes a much bigger issue if each of them would result in a permanent storage of 4-16kb blob in router memory, without router-side expiration for short invitation links.
## Possible solutions
@@ -16,15 +16,15 @@ While part 2 should be improved in the client, indefinite storage of queue recor
The problem with this approach is that contact addresses are also unsecured queues, and they should not be expired.
We could set really large expiration time, and require that clients "update" the unsecured queues they need at least every 1-2 years, but it would not solve the problem of storing a large number of blobs in the server memory for unused/abandoned 1-time invitations.
We could set really large expiration time, and require that clients "update" the unsecured queues they need at least every 1-2 years, but it would not solve the problem of storing a large number of blobs in the router memory for unused/abandoned 1-time invitations.
2) Do not store blobs in memory / append-only log, and instead use something like RocksDB. While it may be a correct long term solution, it may be not expedient enough at the current POC stage for this feature. Also, the lack of expiration is wrong in any case and would indefinitely grow server storage.
2) Do not store blobs in memory / append-only log, and instead use something like RocksDB. While it may be a correct long term solution, it may be not expedient enough at the current POC stage for this feature. Also, the lack of expiration is wrong in any case and would indefinitely grow router storage.
3) Add flag allowing the server to differentiate permanent queues used as contact addresses, also using different blob sizes for them. In this case, messaging queues will be expired if not secured after 3 weeks, and contact address queues would be expired if not "updated" by the owner within 2 years.
3) Add flag allowing the router to differentiate permanent queues used as contact addresses, also using different blob sizes for them. In this case, messaging queues will be expired if not secured after 3 weeks, and contact address queues would be expired if not "updated" by the owner within 2 years.
Probably all three solutions need to be used, to avoid creating a non-expiring blob storage in memory, as in case too many of such blobs are created it would not be possible to differentiate between real users and resource exhaustion attacks, and unlike with messages, they won't be expiring too.
Servers already can differentiate messaging queues and contact address queues, if they want to:
Routers already can differentiate messaging queues and contact address queues, if they want to:
- with the old 4-message handshake, the confirmation message on a normal queue was different, and also KEY command was eventually used.
- with the fast 2-message handshake, while the confirmation message has the same syntax, and the differences are inside encrypted envelope, the client still uses SKEY command.
- in both cases, the usual messaging queues are secured, and contact addresses are not, so this difference is visible in the storage as well (although it is not easy to differentiate between abandoned 1-time invitations and contact addresses).
@@ -33,7 +33,7 @@ Differentiating these queues can also allow different message retention times -
## Proposed solution
1. Add queue updated_at date into queue records. While it adds some metadata, it seems necessary to manage retention and quality of service. It will not include exact time, only date, and the time of creation will be replaced by the time of any update - queue secured, a message is sent, or queue owner subscribes to the queue. To avoid the need to update store log on every message this information can be appended to store log on server termination. Or given that only one update per day is needed it may be ok to make these updates as they happen (temporarily making the sequence and time of these events available in storage).
1. Add queue updated_at date into queue records. While it adds some metadata, it seems necessary to manage retention and quality of service. It will not include exact time, only date, and the time of creation will be replaced by the time of any update - queue secured, a message is sent, or queue owner subscribes to the queue. To avoid the need to update store log on every message this information can be appended to store log on router termination. Or given that only one update per day is needed it may be ok to make these updates as they happen (temporarily making the sequence and time of these events available in storage).
2. Add flag to indicate the queue usage - messaging queue or queue for contact address connection requests. This would result in different queue size and different retention policy for queue and its messages. We already have "sender can secure flag" which is, effectively, this flag - contact address queues are never secured. So this does not increase stored metadata in any way.
@@ -41,11 +41,11 @@ Differentiating these queues can also allow different message retention times -
This is a design considerations and a concept, not a design yet.
Instead of implementing a generic blob storage that can be used as an attack vector, and adds additional failure point (another server storing blob that is necessary to connect to the queue on the current server), but instead adds an extended queue information blobs, most of which could be dropped without the loss of connectivity, so that the attack can be mitigated by deleting these blobs without users losing the ability to connect, as long as the queue and minimal extended information is retained.
Instead of implementing a generic blob storage that can be used as an attack vector, and adds additional failure point (another router storing blob that is necessary to connect to the queue on the current router), but instead adds an extended queue information blobs, most of which could be dropped without the loss of connectivity, so that the attack can be mitigated by deleting these blobs without users losing the ability to connect, as long as the queue and minimal extended information is retained.
So, to make the connection there need to be these elements:
- queue server and queue ID - mandatory part, that can be included in short link
- queue router and queue ID - mandatory part, that can be included in short link
- SMP key - mandatory part for all queues. We are considering initializing ratchets earlier for contact addresses, and include ratchet keys and pre-keys into queue data as well, but it is out of scope here.
- Ratchet keys - mandatory part for 1-time invitation that won't fit in short link.
- PQ key - optional part that can be stored with addresses if ratchet keys are added and with 1-time invitations.
@@ -56,8 +56,8 @@ So rather that storing one blob with a large address inside it, not associated w
Also, we need the address shared with the sender (party accepting the connection) to be short. We could use a similar approach that was proposed for data blobs, using a single random seed per queues to derive multiple keys and IDs from it. For example:
1. The queue owner:
- generates Ed25529 key pair `(sk, spk)` and X25519 key pair `(dhk, dhpk)` to use with the server, same as now sent in NEW command.
- generates queue recipient ID (this ID can still be server-generated).
- generates Ed25529 key pair `(sk, spk)` and X25519 key pair `(dhk, dhpk)` to use with the router, same as now sent in NEW command.
- generates queue recipient ID (this ID can still be router-generated).
- generates X25519 key pair `(k, pk)` to use with the accepting party.
- derives from `k`:
- sender ID.
@@ -73,9 +73,9 @@ The algorithm used to derive key and ID from `k` needs to be cryptographically s
So, coupling blob storage with messaging queues has these pros/cons:
Cons:
- no additional layer of privacy - the server used for connection is visible in the link, even after the blobs are removed from the server.
- no additional layer of privacy - the router used for connection is visible in the link, even after the blobs are removed from the router.
Pros:
- no additional point of failure in the connection process - the same server will be used to retrieve necessary blobs as for connection.
- no additional point of failure in the connection process - the same router will be used to retrieve necessary blobs as for connection.
- queue blobs of messaging blobs will be automatically removed once the queue is secured or expired, without additional request from the recipient - reducing the storage and the time these blobs are available.
- queue blobs for contact addresses will be structured and some of the large blobs can be removed in case of resource exhaustion attack (and recreated by the client if needed), with the only downside that PQ handshake will be postponed (which is the case now) and profile will not be available at a point of connection.
+8 -8
View File
@@ -2,25 +2,25 @@
## Problem
Our current handshake protocol is open to this attack: whoever observes the link exchange, knows on which server connection is being made, and if the traffic on this server is observed, then it can confirm communication between parties. Further, even with the [last proposal](./2024-09-09-smp-blobs.md#possible-privacy-improvement), having real-time access to the server data allows to establish the exact messaging queue that is used to send messages.
Our current handshake protocol is open to this attack: whoever observes the link exchange, knows on which router connection is being made, and if the traffic on this router is observed, then it can confirm communication between parties. Further, even with the [last proposal](./2024-09-09-smp-blobs.md#possible-privacy-improvement), having real-time access to the router data allows to establish the exact messaging queue that is used to send messages.
## Solution
We could make the initial link exchange more private by making it harder for any observer to discover which server will be used for messaging by hiding this information from the server that hosts the initial link.
We could make the initial link exchange more private by making it harder for any observer to discover which router will be used for messaging by hiding this information from the router that hosts the initial link.
Preliminary, the protocol could be the following:
1. Connection initiator stores 224-256 bytes of encrypted connection link on a rendezvous server (link contains server host and linkId on another messaging server, not a rendezvous one).
1. Connection initiator stores 224-256 bytes of encrypted connection link on a rendezvous router (link contains router host and linkId on another messaging router, not a rendezvous one).
2. Rendezvous server adds these links to buckets, up to 64 links per bucket. Bucket ID is the timestamp when the bucket was created + a sequential bucket number, in case more than one bucket is created per second.
2. Rendezvous router adds these links to buckets, up to 64 links per bucket. Bucket ID is the timestamp when the bucket was created + a sequential bucket number, in case more than one bucket is created per second.
3. The server responds to the link creator with a bucket ID where this link was added. That bucket ID is its timestamp + a number prevents server "fingerprinting" clients and using say one bucket for each client. If timestamp is different or a bucket number within this timestamp is too large, the client can refuse to use it, depending on the client settings.
3. The router responds to the link creator with a bucket ID where this link was added. That bucket ID is its timestamp + a number prevents router "fingerprinting" clients and using say one bucket for each client. If timestamp is different or a bucket number within this timestamp is too large, the client can refuse to use it, depending on the client settings.
4. The initiating party will pass to the accepting party the rendezvous server host, the hash of this bucket ID (bucket link) and the passphrase to derive the key from. The initiating party has an option to pass a link and passphrase via two channels - in which case the link will only contain the bucket ID.
4. The initiating party will pass to the accepting party the rendezvous router host, the hash of this bucket ID (bucket link) and the passphrase to derive the key from. The initiating party has an option to pass a link and passphrase via two channels - in which case the link will only contain the bucket ID.
5. The accepting party would then request the bucket via its ID hash (the server would store hashes to be able to look up - hash is used to prevent showing time in the link) and attempt to decrypt all contained links using the provided key.
5. The accepting party would then request the bucket via its ID hash (the router would store hashes to be able to look up - hash is used to prevent showing time in the link) and attempt to decrypt all contained links using the provided key.
The accepting party then will continue the connection via the decrypted link.
This obviously does not protect accepting party from the initiating party, if it can choose rendezvous server it controls. It also does not protect from the malicious rendezvous server that would collaborate with link observers. I think reunion doesnt protect from it too.
This obviously does not protect accepting party from the initiating party, if it can choose rendezvous router it controls. It also does not protect from the malicious rendezvous router that would collaborate with link observers. I think reunion doesnt protect from it too.
But it does protect connection from whoever observes the link, particularly if this link only contains the bucket and the key is passed separately, via some other channel.
+9 -9
View File
@@ -2,7 +2,7 @@
## Problem
For iOS notifications to be delivered the client has to create credentials for notification subscription on SMP server using NKEY command and after that create a subscription on notification server using SNEW command. These two commands are sent in sequence, after the connections are created, and for it to happen the client needs to be online and in foreground.
For iOS notifications to be delivered the client has to create credentials for notification subscription on SMP router using NKEY command and after that create a subscription on notification router using SNEW command. These two commands are sent in sequence, after the connections are created, and for it to happen the client needs to be online and in foreground.
iOS users tend to close the app when it is not used, and iOS has very limited permissions for background activities, so these notification subscriptions are created with a substantial delay, and notifications do not work.
@@ -12,19 +12,19 @@ This problem is distinct from and probably more common than other problems affec
1. When the new connection is created, the client already knows if it needs to create notification subscription or not, based on the conversation setting (e.g., if the group is muted, the client will not create notification subscription as well.). We should extend NEW command to avoid the need to send additional NKEY command with an option to create notification subscription at the point where connection is created. NDEL would still be used to disable this notification, and NKEY will be used to re-enable it.
2. In the same way we stopped using SDEL command (NDEL sends notification DELD to subscribed notification server) to delete notificaiton subscriptions from notification server, we should delegate creating notification subscription on notification server to SMP servers. Clients could use keys agreed with ntf server for e2e encryption and for command authorization to encrypt and sign instruction to create notification subscription that will be forwarded to notification server using protocol similar to SMP proxies. This will avoid the need for clients to separately contact notification servers that won't happen until they are online.
2. In the same way we stopped using SDEL command (NDEL sends notification DELD to subscribed notification router) to delete notificaiton subscriptions from notification router, we should delegate creating notification subscription on notification router to SMP routers. Clients could use keys agreed with ntf router for e2e encryption and for command authorization to encrypt and sign instruction to create notification subscription that will be forwarded to notification router using protocol similar to SMP proxies. This will avoid the need for clients to separately contact notification routers that won't happen until they are online.
3. Instead of making Ntf server trust DELD notifications, we could send deletion instructions signed by the client, which will only fail to send in case notification server is down (and they won't be sent later after server restart).
3. Instead of making Ntf router trust DELD notifications, we could send deletion instructions signed by the client, which will only fail to send in case notification router is down (and they won't be sent later after router restart).
Cons:
- If SMP servers were to retain in the storage the information about which notification server is used for which queue, it would reduce metadata privacy. While currently it is not an issue, as all notification servers are known and operated by us, once there are other client apps, this can be used for app users fingerprinting, which would act as a deterrence from using new apps but only if app users use servers of operators who are different from the app provider. To mitigate it, we could only store it in server memory and include notification instruction in subscription commands (SUB) and include notification subscription status in SUB responses. We don't need to mitigate the problem of server being able to store this information, as messaging servers can observe which notification servers connect to them anyway.
- If SMP server is restarted before the subscription request is forwared to the notification server, then it will have to be forwarded again, once the client subscribes. The problem here is that if the client is offline, it will neither subscribe to the queue to send notification subscription request, nor receive notifications from this queue. Storing notification server and subscription request would mitigate that, as in this case we could send all pending requests on server start, without depending on client subscriptions.
- "Small" agent will need to support connections to ntf servers and manage workers that retry sending pending subscription requests.
- Until the client learns the public keys of notification server, it will not be able to decrypt notifications. It potentially can be mitigated by using the public key of the server returned when token is created, in this way different client keys (per-queue) will be combined with the same ntf server key (per-token).
- If SMP routers were to retain in the storage the information about which notification router is used for which queue, it would reduce metadata privacy. While currently it is not an issue, as all notification routers are known and operated by us, once there are other client apps, this can be used for app users fingerprinting, which would act as a deterrence from using new apps but only if app users use routers of operators who are different from the app provider. To mitigate it, we could only store it in router memory and include notification instruction in subscription commands (SUB) and include notification subscription status in SUB responses. We don't need to mitigate the problem of router being able to store this information, as messaging routers can observe which notification routers connect to them anyway.
- If SMP router is restarted before the subscription request is forwared to the notification router, then it will have to be forwarded again, once the client subscribes. The problem here is that if the client is offline, it will neither subscribe to the queue to send notification subscription request, nor receive notifications from this queue. Storing notification router and subscription request would mitigate that, as in this case we could send all pending requests on router start, without depending on client subscriptions.
- "Small" agent will need to support connections to ntf routers and manage workers that retry sending pending subscription requests.
- Until the client learns the public keys of notification router, it will not be able to decrypt notifications. It potentially can be mitigated by using the public key of the router returned when token is created, in this way different client keys (per-queue) will be combined with the same ntf router key (per-token).
## Implementation details
1. NEW and NKEY commands will need to be extended to include notification subscription request. As the notifier ID needs to be sent to notification server, this notifier ID will have to be client-generated and supplied as part of NEW command.
1. NEW and NKEY commands will need to be extended to include notification subscription request. As the notifier ID needs to be sent to notification router, this notifier ID will have to be client-generated and supplied as part of NEW command.
now:
@@ -46,4 +46,4 @@ NKEY :: NtfPublicAuthKey -> RcvNtfPublicDhKey -> Maybe NtfServerRequest -> Comma
-- NotifierID is passed in entity ID field of the transmission
```
2. Notification server will need to support an additional command to receive "proxied" subscription commands, `SFWD`, that would include `NtfServerRequest`. This command can include both `SNEW` and `SDEL` commands.
2. Notification router will need to support an additional command to receive "proxied" subscription commands, `SFWD`, that would include `NtfServerRequest`. This command can include both `SNEW` and `SDEL` commands.
+1 -1
View File
@@ -5,7 +5,7 @@ This document evolves the design proposed [here](./2024-09-09-smp-blobs.md).
## Problems
In addition to problems in the first doc, we have these issues with in-memory queue record storage:
- many queues are idle or rarely used, but they are loaded to memory, and currently just loading all queues uses 20gb RAM on each server, and takes 10 min to process, increasing downtimes during restarts.
- many queues are idle or rarely used, but they are loaded to memory, and currently just loading all queues uses 20gb RAM on each router, and takes 10 min to process, increasing downtimes during restarts.
- adding blobs to memory would make this problem much worse.
## Proposed solution
-304
View File
@@ -1,304 +0,0 @@
# Protocol changes for creating and connecting to SMP queues
## Problems
This change is related to these problems:
- differentiating queue retention time,
- supporting MITM-resistant short connection links,
- improving notifications.
This RFC is based on the previous discussions about short links, blob storage and notifications ([1](./2024-06-21-short-links.md), [2](./2024-09-09-smp-blobs.md), [3](./2024-11-25-queue-blobs-2.md), [4](./2024-09-25-ios-notifications-2.md)).
SMP protocol supports two types of queues - queues to communicate over and queues to send invitations. While SMP protocol was originally "unaware" of these queue types, it could differentiate it by message flow, and with the recent addition of SKEY command to allow securing the queue by the sender this difference became persistent.
Simply designating queue types would allow to use this information to decide for how long to retain queues, and potentially extending it:
- unsecured 1-time invitation queues with sndSecure (support of securing by sender) - e.g., 3 months.
- contact address queues without sndSecure - e.g., 3 years without activity.
- Possibly, "queues" that prohibit messages and used only as blob storage - they would be used to store group profiles and superpeer addresses for the group.
This proposal also combines NEW and NKEY command to streamline notifications in preparation to reworking of the notifications protocol.
## Design objectives
We want to achieve these objectives:
1. no possibility to provide incorrect SenderId inside link data (e.g. from another queue).
2. link data cannot be accessed by the server unless it has the link.
3. prevent MITM attack by the server, including the server that obtained the link.
4. prevent changing of connection request by the user (to prevent MITM via break-in attack in the originating client).
5. for one-time links, prevent accessing link data by link observers who did not compromise the server.
6. allow changing the user-defined part of link data.
7. avoid changing the link when user-defined part of link data changes, while preventing MITM attack by the server on user-defined part, even if it has the link.
8. retain the quality that it is impossible to check the existense of secured queue from having any of its temporary visible IDs (sender ID and link ID in 1-time invitations) - it requires that these IDs remain server-generated (contrary to the previous RFCs).
To achieve these objectives the queue data must have immutable part and mutable part.
Immutable part would include:
- full conection request (the current long link with all keys, including PQ keys). This includes SenderId that must match server response.
- public signature key to verify mutable part of link data.
Signed mutable part would inlcude:
- any links to chat relays that should be contacted instead of this queue (not in this RFC), but would allow delegating group connections and contact request connections to prevent spam, hiding online presense, etc.
- and user-defined data - user profile or group profile.
The link itself should include both the key and auth tag from the encryption of immutable part. Accessing one-time link data should require providing sender key and signing the command (`LKEY`).
## Solution
Current NEW and NKEY commands:
```haskell
NEW :: RcvPublicAuthKey -> RcvPublicDhKey -> Maybe BasicAuth -> SubscriptionMode -> SenderCanSecure -> Command Recipient
NKEY :: NtfPublicAuthKey -> RcvNtfPublicDhKey -> Command Recipient
-- | Queue IDs and keys, returned in IDS response
data QueueIdsKeys = QIK
{ rcvId :: RecipientId,
sndId :: SenderId,
rcvPublicDhKey :: RcvPublicDhKey,
sndSecure :: SenderCanSecure
}
```
Proposed NEW command replaces SenderCanSecure with QueueMode, adds link data, and combines NKEY command:
```haskell
NEW :: NewQueueRequest -> Command Recipient
data NewQueueReq = NewQueueReq
{ rcvAuthKey :: RcvPublicAuthKey,
rcvDhKey :: RcvPublicDhKey,
auth_ :: Maybe BasicAuth,
subMode :: SubscriptionMode,
queueData :: Maybe QueueReqData,
ntfCreds :: Maybe NewNtfCreds
}
-- Replaces NKEY command
-- This avoids additional command required from the client to enable notifications.
-- Further changes would move NotifierId generation to the client, and including a signed and encrypted command to be forwarded by SMP server to notification server.
data NtfRequest = NtfRequest NtfPublicAuthKey RcvNtfPublicDhKey
-- QRMessaging implies that sender can secure the queue.
-- LinkId is not used with QRMessaging, to prevent the possibility of checking when connection is established by re-using the same link ID when creating another queue the creating would have to fail if it is used.
-- LinkId is required with QRContact, to have shorter link - it will be derived from the link_uri. And in this case we do not need to prevent checks that this queue exists.
data QueueReqData = QRMessaging (Maybe QueueLinkData) | QRContact (Maybe (LinkId, QueueLinkData))
-- SenderId should be computed client-side as the first 24 bytes of sha3-384(correlation_id),
-- The server must verify it and reject if it is not.
type QueueLinkData = (SenderId, EncImmutableDataBytes, EncUserDataBytes)
type EncImmutableDataBytes = ByteString
type EncUserDataBytes = ByteString
-- We need to use binary encoding for AConnectionRequestUri to reduce its size
-- connReq including the full link allows connection redundancy.
-- The clients would reject changed immutable data (based on auth tag in the link) and
-- AConnectionRequestUri where SenderId of the queue does not match.
data ImmutableLinkData = ImmutableLinkData
{ signature :: SignatureEd25519, -- signature of the remaining part of immutable data
connReq :: AConnectionRequestUri,
sigKey :: PublicKeyEd25519
}
-- This part of link data can also include any relays, but possibly we need a separate blob for it
data UserLinkData = UserLinkData
{ signature :: SignatureEd25519, -- signs the remaining part of the data
userData :: ByteString -- the max size needs to be estimated, but it is likely to be ~ 14kb
}
-- | Updated queue IDs and keys, returned in IDS response
data QueueIdsKeys = QIK
{ rcvId :: RecipientId, -- server-generated
sndId :: SenderId, -- server-generated
rcvPublicDhKey :: RcvPublicDhKey,
sndSecure :: SenderCanSecure, -- possibly, can be removed? or implied?
linkId :: Maybe LinkId, -- server-generated
serverNtfCreds :: Maybe ServerNtfCreds -- currently returned in NID response
}
data ServerNtfCreds = ServerNtfCreds NotifierId RcvNtfPublicDhKey -- NotifierId is server-generated.
```
In addition to that we add the command allowing to update and also to retrieve and, optionally, secure the queue and get link data in one request, to have only one request:
```haskell
-- This command allows to set all data or to update mutlable part of contact address queue.
-- This command should fail on queues that support sndSecure and also on new queues created with QRMessaging.
-- This should fail if LinkId or immutable part of data is changed with the update, but will succeed if only mutable part is updated, so it can be retried.
-- Entity ID is RecipientId.
-- The response to this command is `OK`.
LSET :: LinkId -> QueueLinkData -> Command Recipient
-- Delete should link and associated data
-- Entity ID is RecipientId
LDEL :: Command Recipient
-- To be used with 1-time links.
-- Sender's key provided on the first request prevents observers from undetectably accessing 1-time link data.
-- If queue mode is QRContact (and queue does NOT allow sndSecure) the command will fail, same as SKEY.
-- Once queue is secured, the key must be the same in subsequent requests - to allow retries in case of network failures, and to prevent passive attacks.
-- The difference with securing queues is that queues allow sending unsecured messages to queues that allow sndSecure (for backwards compatibility), and 1-time links will NOT allow retrieving link data without securing the queue at the same time, preventing undetected access by observers.
-- Entity ID is LinkId
LKEY :: SndPublicAuthKey -> Command Sender
-- If queue mode is QRMessaging the command will fail.
-- Entity ID is LinkId
LGET :: Command Sender
-- Response to LGET, LSKEY and LSGET
-- Entity ID is the same as in the command
LNK :: SenderId -> QueueLinkData -> BrokerMsg
```
To both include sender_id into the full link before the server response, and to prevent "oracle attack" when a failure to create the queue with the supplied `sender_id` can be used as a proof of queue existense, it is proposed that `sender_id` is computed client-side as the first 24 bytes of 48 in `sha3-384(correlation_id)` and validated server-side, where `corelation_id` is the transmission correlation ID.
To allow retries and to avoid regenerating all queue data, NEW command must be idempotent, and `correlation_id` must be preserved in command for queue creation, so that the same `correlation_id` and all other data is used in retries. `correlation_id` should be removed after queue creation success.
To allow retries, every time the command is sent a new random `correlation_id` and new `sender_id` / `link_id` should be used on each attempt, because other IDs would be generated randomly on the server, and in case the previous command succeeded on the server but failed to be communicated to the client, the retry will fail if the same ID is used.
Alternative solutions considered and rejected:
- additional request to save queue data, after `sender_id` is returned by the server. The scenarios that require short links are interactive - creating user addresses and 1-time invitations - so making two requests instead of one would make the UX worse.
- include empty sender_id in the immutable data and have it replaced by the accepting party with `sender_id` received in `LINK` response - both a weird design, and might create possibility for some attacks via server, especially for contact addresses.
- making NEW commands idempotent. Doing it would require generating all IDs client-side, not only `sender_id`. It increases complexity, and it is not really necessary as the only scenarios when retries are needed are async NEW commands, that do not require short links. For future short links of chat relays the retries are much less likely, as chat relays will have good network connections.
## Algorithm to prepare and to interpret queue link data.
For contact addresses this approach follows the design proposed in [Short links](./2024-06-21-short-links.md) RFC - when link id is derived from the same random binary as key. For 1-time invitations link ID is independent and server-generated, to prevent existense checks.
**Prepare queue link data**
- the queue owner generates a random 256 bit `link_key` that will be used in the link URI.
- for 1-time links: crypto_box key and 2 nonces to encrypt link data are derived from link_uri using HKDF: `cb_key <> nonce1 <> nonce2 = HKDF(link_key, 80 bytes)` (nonce1 is used for immutable and nonce2 for user-defined parts).
- for contact address links: key and 2 nonces and linkId will be derived: `link_id <> cb_key <> nonce1 <> nonce2 = HKDF(link_key, 104 bytes)`
- both parts of link data are encrypted with crypto_box, and included into `NEW` or `LNEW` commands.
**Retrieving queue link data**
- the sender uses `LinkId` from URI (or derived from URI) as entity ID to retrieve link data.
- for one time links the sender must authorize the request to retrieve the data, the key is provided with the first request, preventing undetected access by link observers.
- having received the link data, the client can now decrypt it using secret_box.
## Improved algorithm to prepare and to interpret queue link data.
This scheme reduces the size of the binary in the link from 48 bytes (72 in case of 1-time links) to 32 bytes (56 bytes for 1-time links).
For immutable data.
1. `link_key = SHA3-256(immutable_data)` - used as part of link, and to encrypt content.
2. HKDF:
1) contact address: `(link_id, key) = HKDF(link_key, 56 bytes)`.
2) 1-time invitation: `key = HKDF(link_key, 32 bytes)`, `link-id` - server-generated.
3.
3. Random `nonce1` (for immutable data), to be stored with the link data.
4. Encrypt: `(ct1, tag1) = secret_box(immutable_data, key, nonce1)`.
5. Store: `(nonce1, ct1, tag1)` stored as immutable link data.
For mutable user data:
1. Random `nonce2` and the same key are used.
2. Sign `user_data` with key included in `immutable_data`.
3. Encrypt: `(ct2, tag2) = secret_box(signed_used_data, key, nonce2)`.
4. Store: `(nonce2, ct2, tag2)`
Link recipient:
1. Receives `link_key` in the link, for 1-time invitations also `link_id`.
2. HKDF:
1) contact address: `(link_id, key) = HKDF(link_key, 56 bytes)`.
2) 1-time invitation: `key = HKDF(link_key, 32 bytes)`.
3. Retrieves via `link_id`: `(nonce1, ct1, tag1)` and `(nonce2, ct2, tag2)`.
4. Decrypt: `immutable_data = decrypt (nonce1, ct1, tag1)`.
5. Verify: `SHA3-256(immutable_data) == link_key`, abort if not.
6. Decrypt: `signed_used_data = decrypt(nonce2, ct2, tag2)`
7. Verify signature with key in immutable data.
While using content hash as encryption key is unconventional, it is not completely unheard of - e.g., it is used in convergent encryption (although in our case using random nonce makes it not convergent, but other use cases suggest that this approach preserves encryption security). It is particularly acceptable for our use case, as `immutable_data` contains mostly random keys.
## Threat model
**Compromised SMP server**
can:
- delete link data.
- hide link selectively from some requests.
cannot:
- undetectably replace link data, even if they have the link (objective 3).
- access unencrypted link data, whether it was or was not accessed by the accepting party, provided it has no link (objective 2).
- observe IP addresses of the users accessing link data, if private routing is used.
**Passive observer who observed short link**:
can:
- access original unencrypted link data for contact address links.
cannot:
- undetectably access observed 1-time link data, accessing the link would make the link inaccessible to the sender (objective 5).
- undetectbly check the existense of messaging queue or 1-time link (objective 8).
- replace or delete the link data.
**Queue owner who did not comprmise the server**:
cannot:
- redirect connecting user to another queue, on the same or on another server (objective 1).
- replace connection request in the link (objective 4).
## Correlation of design objectives with design elements
1. The presense of `SenderId` in `LINK` response from the server.
2. Encryption of link data with crypto_box.
3. Auth tag in the link prevents server modification of immutable part of link data. Signature verification key in immutable part, and signing of mutable part prevents server modification of mutable part of link data.
4. No server command to change immutable part of link data once it's set.
5. 1-time link data can only be accessed with `LKEY` command, that while allows retries to mitigate network failures, will require the same key for retries.
6. `LSET` command.
7. The link only includes auth tag for immutable part, mutable part includes signature.
8. Temporarily public IDs (SenderId and LinkId for 1-time invitations) are generated server-side, and cannot be provided by the clients when creating the queues to check if these IDs are free.
## Syntax for short links
The proposed syntax:
```abnf
shortConnectionLink = %s"https://" smpServerHost "/" linkUri [ "?" param *( "&" param ) ]
smpServerHost = <hostname> ; RFC1123, RFC5891
linkUri = %s"i#" serverInfo oneTimeLinkBytes / %s"c#" serverInfo contactLinkBytes
oneTimeLinkBytes = <base64url(linkId | linkKey)> ; 56 bytes / 75 base64 encoded characters
contactLinkBytes = <base64url(linkKey)> ; 32 bytes / 43 base64 encoded characters
; linkId - 96 bits/24 bytes
; linkKey - 256 bits/32 bytes
serverInfo = [fingerprint "@" [hostnames "/"]] ; not needed for preset servers, required otherwise - the clients must refuse to connect if they don't have fingerprint in the code.
fingerprint = <base64url(server offline certificate fingerprint)>
hostnames = "h=" <hostname> *( "," <hostname> ) ; additional hostnames, e.g. onion
```
To have shorter links fingerpring and additional server hostnames do not need to be specified for preconfigured servers, even if they are disabled - they can be used from the client code. Any user defined servers will require including additional hosts and server fingerprint.
Example one-time link for preset server (103 characters):
```
https://smp12.simplex.im/i#abcdefghij0123456789abcdefghij0123456789abcdefghij0123456789abcdefghij01234
```
Example contact link for preset server (71 characters):
```
https://smp12.simplex.im/c#abcdefghij0123456789abcdefghij0123456789abc
```
Example contact link for user-defined server (with fingerprint, but without onion hostname - 115 characters):
```
https://smp1.example.com/c#0YuTwO05YJWS8rkjn9eLJDjQhFKvIYd8d4xG8X1blIU@abcdefghij0123456789abcdefghij0123456789abc
```
Example contact link for user-defined server (with fingerprint ant onion hostname - 178 characters):
```
https://smp1.example.com/c#0YuTwO05YJWS8rkjn9eLJDjQhFKvIYd8d4xG8X1blIU@beccx4yfxxbvyhqypaavemqurytl6hozr47wfc7uuecacjqdvwpw2xid.onion/abcdefghij0123456789abcdefghij0123456789abc
```
For the links to work in the browser the servers must provide server pages.
+21 -21
View File
@@ -6,65 +6,65 @@ iOS notifications have these problems:
- iOS notification service crashes exceeding memory limit. This is being addressed by changes in GHC RTS.
- there is a large number of connections, because each member in a group requires individual connection. This will improve with chat relays when each group would require 2-3 connections.
- some notification may be not shown if notification with reply/mention is skipped, and instead some other message is delivered, which may be muted. This would not improve without some changes, as notifications may be skipped anyway.
- client devices delay communication with ntf server because it is done in background, and by that time the app may be suspended.
- notification server represents a bottleneck, as it has to be owned by the app vendor, and the current design when ntf server subscribes to notifications scales very badly.
- client devices delay communication with ntf router because it is done in background, and by that time the app may be suspended.
- notification router represents a bottleneck, as it has to be owned by the app vendor, and the current design when ntf router subscribes to notifications scales very badly.
This RFC is based on the previous [RFC related to notifications](./2024-09-25-ios-notifications-2.md).
## Solution
As notification server has to know client token and currently it associates subscriptions with this token anyway, we are not gaining any privacy and security by using per-subscription keys - both authorization and encryption keys of notification subscription can be dropped.
As notification router has to know client token and currently it associates subscriptions with this token anyway, we are not gaining any privacy and security by using per-subscription keys - both authorization and encryption keys of notification subscription can be dropped.
We still need to store the list of queue IDs associated with the token on the notification server, but we do not need any per-queue keys on the notification server, and we don't need subscriptions - it's effectively a simple set of IDs, with no other information.
We still need to store the list of queue IDs associated with the token on the notification router, but we do not need any per-queue keys on the notification router, and we don't need subscriptions - it's effectively a simple set of IDs, with no other information.
In this case, when queue is created the client would supply notifier ID - it has to be derived from correlation ID, to prevent existense check (see previous RFC). As we also supply sender ID, instead of deriving it as sha3-192 of correlation ID, they both can be derived as sha3-384 and split to two IDs - 24 bytes each.
The notification server will maintain a rotating list of server keys with the latest key communicated to the client every time the token is registered and checked. The keys would expire after, say, 1 week or 1 month, and removed from notification server on expiration.
The notification router will maintain a rotating list of router keys with the latest key communicated to the client every time the token is registered and checked. The keys would expire after, say, 1 week or 1 month, and removed from notification router on expiration.
The packet containing association between notifier queue ID and token will be crypto_box encrypted using key agreement between identified notification server master key and an ephemeral per packet (effectively, per-queue) client-key.
The packet containing association between notifier queue ID and token will be crypto_box encrypted using key agreement between identified notification router master key and an ephemeral per packet (effectively, per-queue) client-key.
Deleting the queue may also include encrypted packet that would verify that the client deleted the queue.
Instead of notification server subscribing to the notifications creating a lot of traffic for the queues without messages, the SMP server would push notifications via NTF server connection (whether via NTF or via SMP protocol). This could be used as a mechanism to migrate existing queues when with the next subscription the notification server would communicate it's address to SMP server and this association would be stored together with the queue.
Instead of notification router subscribing to the notifications creating a lot of traffic for the queues without messages, the SMP router would push notifications via NTF router connection (whether via NTF or via SMP protocol). This could be used as a mechanism to migrate existing queues when with the next subscription the notification router would communicate it's address to SMP router and this association would be stored together with the queue.
## Protocol design
Additional/changed SMP commands:
```haskell
-- register notification server
-- should be signed with server key
-- register notification router
-- should be signed with router key
NSRV :: NtfServerCreds -> Command NtfServer
-- response
NSID :: NtfServerId -> BrokerMsg
-- to communicate which server is responsible for the queue
-- to communicate which router is responsible for the queue
-- should be signed with queue key
NSUB :: Maybe NtfServerId -> Command Notifier
-- subscribe to notificaions from all queues associated with the server
-- should be signed with server key
-- subscribe to notificaions from all queues associated with the router
-- should be signed with router key
-- entity ID - NtfServerId
NSSUB :: Command NtfServer
data NtfServerCreds = NtfServerCreds
{ server :: NtfServer,
-- NTF server certificate chain that should match fingerpring in address
-- NTF router certificate chain that should match fingerpring in address
cert :: X.CertificateChain,
-- server autorizatio key to sign server subscription requests
-- router autorizatio key to sign router subscription requests
authKey :: X.SignedExact X.PubKey
}
-- entity ID is recipient ID
NSKEY :: NtfSubscription -> Command Recipient
NSKEY :: NtfSubscription -> Command Recipient
data NtfSubscription = NtfSubscription
-- key to encrypt notifications e2e with the client
{ ntfPubDbKey :: RcvNtfPublicDhKey,
ntfServer :: NtfServer,
-- should be linked to correlation ID to prevent existense check
-- the ID sent to notification server could be its hash?
-- the ID sent to notification router could be its hash?
ntfId :: NotifierId,
encNtfTokenAssoc :: EncDataBytes
}
@@ -77,12 +77,12 @@ data NtfTokenAssoc = NtfTokenAssoc
}
```
SMP server will need to maintain the list of Ntf servers and their credentials, and when NSSUB arrives to make only one subscription. When message arrives it would deliver notification to the correct connection via queue / ntf server association.
SMP router will need to maintain the list of Ntf routers and their credentials, and when NSSUB arrives to make only one subscription. When message arrives it would deliver notification to the correct connection via queue / ntf router association.
Ntf server needs to maintain three indices to the same data:
Ntf router needs to maintain three indices to the same data:
- `(smpServer, queueId) -> tokenId` - to deliver notification to the correct token
- `tokenId -> [smpServer -> [queueId]]` - to remove all queues when token is removed, and to store/update these associations effficiently - store log may have one compact line per token (after compacting), or per token/server combination.
- `[smpServer]` - array of SMP servers to subscribe to.
- `tokenId -> [smpServer -> [queueId]]` - to remove all queues when token is removed, and to store/update these associations effficiently - store log may have one compact line per token (after compacting), or per token/router combination.
- `[smpServer]` - array of SMP routers to subscribe to.
## Mention notifications
@@ -90,4 +90,4 @@ Currently we are marking messages with T (true) for messages that require notifi
The proposal is to:
- add additional values to this metadata, e.g. 2 (priority) and 3 (high priority) (and T/F could be sent as 0/1 respectively) - that is, to deliver notifications even if notifications are generally disabled (they can still be further filtered by the client).
- instead of deleting notification credentials when notifications are disabled - which is costly - communicate to SMP server the change of notificaion priority level, e.g. the client could set minimal notification priority to deliver notifications, where 0 would mean disabling it completely, 1 enable for all, 2 for priority 2+, 3 for priority 3. The downside here is that it could be used for timing correlation of queues in the group, but it already can be used on bulk deletions of ntf credentials for these queues and when sending messages.
- instead of deleting notification credentials when notifications are disabled - which is costly - communicate to SMP router the change of notificaion priority level, e.g. the client could set minimal notification priority to deliver notifications, where 0 would mean disabling it completely, 1 enable for all, 2 for priority 2+, 3 for priority 3. The downside here is that it could be used for timing correlation of queues in the group, but it already can be used on bulk deletions of ntf credentials for these queues and when sending messages.
+9 -9
View File
@@ -35,18 +35,18 @@ This could possibly be evolved into the requirement to have a direct connection
3. Allow "joint management" of SMP queues.
SMP servers can support multiple recipients for contact queues:\
SMP routers can support multiple recipients for contact queues:\
- subscription would be possible to the "subscriber recipient".
- all other changes (update data, change subscriber recipient, add or remove recipients) would require multiple recipient signatures on SMP command in line with n-of-m multisig rules, that the command sender would have to collect out-of-band (from SMP protocol point of view).
Pros: allows joint ownership, and protects from losing access to master owner device.
Cons:
- complicates queue abstraction with approach that is not needed for most queues.
- still retains the server as a single point of failure.
- still retains the router as a single point of failure.
4. Introduce "group" as a new type of entity managed by SMP servers.
4. Introduce "group" as a new type of entity managed by SMP routers.
SMP servers would provide a separate set of commands for managing group records that would include in an encrypted container:
SMP routers would provide a separate set of commands for managing group records that would include in an encrypted container:
- the group profile
- the list of chat relay links
- the list of owner member IDs with their public keys
@@ -54,7 +54,7 @@ SMP servers would provide a separate set of commands for managing group records
- alternative group entity locations
- possibly, a globally unique group identity (as the hash of the initial/seed group data).
While the server domain would be used as the hostname in group link, it may contain alternative hosts (not just hostnames of the same server), both in the link and in the group record data.
While the router domain would be used as the hostname in group link, it may contain alternative hosts (not just hostnames of the same router), both in the link and in the group record data.
Pros: separates additional complexity to where it is needed, allowing reliability and redundancy for group ownership.
Cons: complexity, coupling between SMP and chat protocol.
@@ -86,7 +86,7 @@ Cons:
- if no messages are accepted, this is not even a queue.
- no way to directly contact owners (maybe it is not a downside, as for relays there would be a communication channel anyway as part of the group).
Option 2 looks more simple and attractive, implementing server broadcast for SMP seems unnecessary, as while it could have been used for simple groups, it does not solve such problems as spam and pre-moderation anyway - it requires a higher level protocol.
Option 2 looks more simple and attractive, implementing router broadcast for SMP seems unnecessary, as while it could have been used for simple groups, it does not solve such problems as spam and pre-moderation anyway - it requires a higher level protocol.
The command to update owner keys would be `RKEY` with the list of keys, and we can make `NEW` accept multiple keys too, although the use case here is less clear.
@@ -96,7 +96,7 @@ Option 1: Use the same keys in SMP as when signing queue data.
Option 2: Use different keys.
The value here could be that the server could validate these signatures too, and also maintain the chain of key changes. While tempting, it is probably unnecessary, and this chain of ownership is better to be maintained on chat relay level, as there are no size constraints on the size of this chain. Also, it is better for metadata privacy to not couple transport and chat protocol keys.
The value here could be that the router could validate these signatures too, and also maintain the chain of key changes. While tempting, it is probably unnecessary, and this chain of ownership is better to be maintained on chat relay level, as there are no size constraints on the size of this chain. Also, it is better for metadata privacy to not couple transport and chat protocol keys.
We still need to bind the mutable data updates to the "genesis" signature key (the one included in the immutable data).
@@ -147,12 +147,12 @@ The size of the OwnerInfo record encoding is:
~189 bytes, so we should practically limit the number of owners to say 8 - 1 original + 7 addiitonal. Original creator could use a different key as a "genesis" key, to conceal creator identity from other members, and it needs to include the record with memberId anyway.
The structure is simplified, and it does not allow arbitrary ownership changes. Its purpose is not to comprehensively manage ownership changes - while it is possible with a generic blockchain, it seems not appropriate at this stage, - but rather to ensure access continuity and that the server cannot modify the data (although nothing prevents the server from removing the data completely or from serving the previous version of the data).
The structure is simplified, and it does not allow arbitrary ownership changes. Its purpose is not to comprehensively manage ownership changes - while it is possible with a generic blockchain, it seems not appropriate at this stage, - but rather to ensure access continuity and that the router cannot modify the data (although nothing prevents the router from removing the data completely or from serving the previous version of the data).
For example it would only allow any given owner to remove subsequenty added owners, preserving the group link and identity, but it won't allow removing owners that signed this owner authorization. So owners are not equal, with the creator having the highest rank and being able to remove all additional owners, and owners authorise by creator can remove all other owners but themselves and creator, and so on - they have to maintain the chain that authorized themselves, at least. We could explicitely include owner rank into OwnerInfo, or we could require that they are sorted by rank, or the rank can be simply derived from signatures.
When additional owners want to be added to the group, they would have to provide any of the current owners:
- the key for SMP commands authorization - this will be passed to SMP server together with other keys. There could be either RKEY to pass all keys (some risk to miss some, or of race conditions), or RADD/RGET/RDEL to add and remove recipient keys, which has no risk of race conditions.
- the key for SMP commands authorization - this will be passed to SMP router together with other keys. There could be either RKEY to pass all keys (some risk to miss some, or of race conditions), or RADD/RGET/RDEL to add and remove recipient keys, which has no risk of race conditions.
- the signature of the immutable data by their member key included in their profile.
- the current owner would then include their member key into the queue data, and update it with LSET command. In any case there should be some simple consensus protocol between owners for owner changes, and it has to be maintained as a blockchain by owners and by chat relays, as otherwise it may lead to race conditions with LSET command.
+104
View File
@@ -0,0 +1,104 @@
# Using the same profile from multiple devices
## Problem
Double Ratchet algorithm makes it hard to send/receive messages sent to the user from different devices, as each message changes the state of Double Ratchet keys, and these state changes must be strictly sequential and they cannot be reversed (although skipping is possible).
Traditional approach for multi-device converts each direct conversation into a group, where each device participates as a member. Likewise, for group conversations each device also participates as a member. While these members *look* as if they are the same user to others, a very simple client app modification may show device ID for each message, and the communication peers, both in direct chats and in groups would know how many devices a user has and which device the user sent the message from. In addition to that, with this approach communication peers can send different messages to different devices (it can be prevented by provider who would request that only message key is encrypted with DR, while the encrypted message is the same) or withheld from some devices (it cannot be prevented by provider, as it cannot add key to the communication in case it is missing, and cannot withhold the message completely too). These opens various vectors for targeted attacks, e.g.:
- tracking movements of the user: once each devices is identified as "desk" and "phone" it would allow to know where the user is at a given time.
- manipulating information by sending messages to one device (to have proof it was sent) and withholding from others, or sending different messages if the protocol allows it.
In addition to that, the specific implementation of this approach in Signal compromises break-in recovery property (aka post-compromise security) of Double-Ratchet algorithm, making its design ineffective - the only reason to have the second ratchet in DR algorithm is to provide break-in recovery, without it a much simpler design with a single ratchet is sufficient. See [this paper](https://eprint.iacr.org/2021/626.pdf) for details.
While this limitation can be addressed with notifications when a new device is added and per-device keys, we still find the remaining attack vectors on user security and privacy to be unacceptable, and opening unsuspecting users to various criminal actions - and it is wrong to say that would only affect security conscious users, and most people would not be affected by these risks. Allowing potential criminals in groups to know which device you are currently using is a real risk for all users.
Another approach was offered by Threema that is ["mediator" router](https://threema.com/en/blog/md-architectural-overview) where the state of encryption ratchets is stored router-side. While it protects the user from their communication peers, it increases required level of trust to the routers, and in case of SimpleX network it would expose the knowledge of who communicates to whom. So while the idea of router-side storage of encryption state is promising, it has to be per-connection, to retain "no-accounts" property of SimpleX messaging network.
Also see [FAQ](https://simplex.chat/faq/#why-cant-i-use-the-same-profile-on-different-devices) and [this issue](https://github.com/simplex-chat/simplex-chat/issues/444#issuecomment-3066968358).
## Proposed solution
One of the ideas presented in FAQ - to store the state of Double Ratchet algorithm in the encrypted container on the router seems promising. The RFC develops this idea.
### Considerations for the design
1. The largest ratchet state size with the current implementation is less than 8kb (which is achieved when both sides shared PQ keys and ciphertexts), so while it cannot fit in the same transport blocks together with sent and received messages, it would fit in one transport block.
2. Protocol commands and events may be changed (even if at the cost of slightly reducing message size) can fit the hash of the ratchet state (32 bytes sha256 would be sufficient), so that the client can determine whether it has the most recent ratchet state or if it needs to retrieve the latest copy. Message size reduction won't affect the users because we use compression, and there is a substantial reserve.
3. Client commands that modify ratchet state would include the hash of the previous ratchet state so that the router can reject or ignore the command in case the previous ratchet state is different or in case command is repeated in case of lost response).
4. The client does not need to retrieve message state for each encryption and decryption operation - it can "speculatively" use the ratchet state it has, and receive correct ratchet state in the "error" response after attempting encryption based on incorrect ratchet state.
## Proposed protocol design
Ratchet state will be stored on the same router that stores message queue, as part of message queue record. 8kb is a sufficient size for this blob (the actual max size is 7800 bytes). The router would also store the hashes of the current and, possibly, the previous ratchet states (TBC).
While ratchet is used for duplex connection, the connection still has primary queue, and with redundancy the same ratchet state can be stored on all secondary queues.
Ratchet state will be encrypted using secret_box - a symmetric encryption scheme, so PQ-resistant. If ratchet state is stored on more than one router, it has to be encrypted with a different key for each router.
Questions: how to rotate the key used to store ratchet? Should key used to encrypt ratchet rotate at the same time when queue is rotated? The latter is a logical option, as it prevents additional complexity and solves the problem anyway. A possible option is to have "ratchet version" that will be used to advance the key used to encrypt ratchet via HKDF.
Security considerations: the scheme may reduce break-in recovery to the points queues are rotated, unless there is some randomness mixed-in into the key derivation (the key used to encrypt ratchet state). But including randomness would defeat the purpose, as other devices wouldn't be able to access the ratchets. Another approach would be to have each device use its own key for encryption, and encrypt to all keys of all devices (or to encrypt key, to avoid size increase). Having multiple encryptions would show how many devices use the queue, but routers already can observe it, so it is a better tradeoff. Another idea would be to rotate the key used to authorize queue commands - we already support multiple recipient keys, and it can be used for multi-device scenario. That would partially mitigate break-in attacks as the attacker who obtained the key from ratchet state would be able to decrypt it, but won't be able to decrypt it (the attacker collusion with the router is not mitigated). Yet another idea would be for each party (device) to share its private (or encapsulation) key and to have a symmetric key (used to encrypt the ratchet state) encrypted (encapsulated) separately for each device. This would reduce the size of the stored data to `ratchet size` + `encrypted key size` * N, so even in case of PQ encryption (e.g. sntrup) the size required to store the ratchet would be under transport block size, while limiting it to say 4-8 devices, which is sufficient.
To participate in multi-device scheme the devices would join the usual group that will be used to share public (encapsulation) device keys and to communicate updates to conversations that were received by the currently "active" device. "Active" means the device that received or sent and processed the message, and while only one device can receive messages from a given queue, device "active" state may be determined per queue, allowing concurrent usage.
The scheme must be resilient to state updates being lost, and in case of direct messages it would result in some messages not being shown (or shown as skipped), while conversation preference and profile updates can be re-requested from peers, while the current profile of the user would become the latest. Likewise, for groups state updates ca be requested from super-peers or for decentralized groups - from owners. Maintaining chat state consistency is an important consideration, but is not a focus of this RFC - the focus is managing message delivery and DR encryption for multiple devices. Other multi-device schemes have the same issues with state consistency. Partially, the profile state consistency can be improved by using a single shared queue (or set of queues) to store user's profile and chat preferences to synchronize profile updates asynchronously between the devices.
## The protocol to send the message
`rsi` - ratchet state on device `i`.
`enc(rs)` - current authoritative ratchet state on the router.
`pt` and `ct` - plaintext and ciphertext messages.
Encryption is a state transition function ratchetEnc: `(ct, rs') = ratchetEnc(pt, rs)`.
1. Device encrypts the message using the stored ratchet state: `(ct, rsi') = ratchetEnc(pt, rsi)`
2. Device sends modified encrypted ratchet state and the hash of the previous encrypted state to the router that stores the queue: `RSET (hash(enc(rsi)), enc(rsi'))`.
3. If the hash of the previous state matches state stored on the router (`hash(enc(rsi)) == hash(enc(rs))`), the router updates the state and responds with `ratchet_ok` (that may include the current state or it's hash, for validation). If the hash is different, the router responds with `bad_ratchet(enc(rs))` message that includes the correct ratchet state. These updates must be atomic. In this case device has to update the local ratchet state (provided it can decrypt it), and repeat encryption attempt. If device cannot decrypt the provided ratchet state, it means that the connection is disrupted (possibly, device is removed from device group, but missed the notifications).
4. After successful state update in primary receiving queue, the device would update it in secondary receiving queues.
5. Device sends encrypted message as usual, via proxy that must be different both from the router that stores the ratchet and from the destination router.
6. Device broadcasts sent message and new ratchet state to other devices in the device group.
This protocol is simple, and it minimizes requests when sending the message to one additional request to update ratchet state in most cases, only requiring two requests when device state was not updated via device group prior to message sending attempt.
## The protocol to receive the message
Decryption is also a state transition function: `(pt, rs') = ratchetDec(ct, rs)`
1. Router sends the message to the device (can be in response to SUB or ACK commands, or with active subscription). Pushed message would include the hash of the currently stored ratchet state: `hash(enc(rs))`.
2. If device has the ratchet state with the same hash (`hash(enc(rs)) == hash(enc(rsi))`), it decrypts the message: `(pt, rsi') = ratchetDec(ct, rsi)`.
3. If device has ratchet state with a different hash, it requests ratchet from the router with additional protocol command `RGET` with response `RCHT (enc(rs))` and updates the local state.
4. Device decrypts the message `(pt, rsi') = ratchetDec(ct, rsi)` and processes it as usual.
5. Device sends acknowledgement to the router as usual, but now it includes the new ratchet state and the hash of the previous state: `ACK msgId (hash(enc(rsi)), enc(rsi'))`
6. The router compares ratchet state with stored state hash, and in case it matches it processes `ACK` and responds with `OK` as usual (or `NO_MSG` in case msgId is incorrect, also as usual - it would happen in repeated ACK requests). If ratchet state hash does not match, the router would respond with `bad_ratchet(enc(rs))` - which means that the message was already processed by another device and ratchet was advanced. This is a complex scenario, as the client has to either revert the change from message processing or somehow combine the change with the updates communicated via device group (as a side note, device group can simply re-broadcast messages, not state updates, but it will result in state divergence between devices when different messages are lost).
Unlike sending messages, this flow does not require any additional requests in most cases, only requiring requesting message state reconciliation when the same message was received and processed by more than one client, but it does not require re-acknowledgement.
## Challenges
This is an idea of the design rather than the actual design, as it requires more thinking about:
- how to handle concurrent ratchet state updates,
- "active" status transitions per queue,
- avoiding concurrent subscriptions to queues from multiple devices,
- state updates and synchronization between devices,
- handling skipped messages,
- costs to update ratchets in bulk send scenario - this scheme would substantially increase costs of preparing large broadcasts, and it makes this scheme not acceptable for chat relays. Which means that "profile" on desktop used as chat relay won't be synched to other devices.
- etc.
## Advantages
The communication peers won't know how many devices the user has, and which device was used to send the message. Also, the communication peers won't be able to send different messages to different user's devices, or to withhold messages from some devices.
+101
View File
@@ -0,0 +1,101 @@
# Detecting and fixing state with service subscriptions
## Problem
While service certificates and subscriptions hugely decrease startup time and delivery delays on router restarts, they introduce the risk of losing subscriptions in case of state drifts. They also do not provide efficient mechanism for validating that the list of subscribed queues is in sync.
How can the state drift happen?
There are several possibilities:
- lost broker response would make the broker consider that the queue is associated, but the client won't know it, and will have to re-associate. While in itself it is not a problem, as it'll be resolved, it would make drift detected more frequently (regardless of the detection logic used). That service certificates are used on clients with good connection would make it less likely though.
- router state restored from the backup, in case of some failure. Nothing can be done to recover lost queues, but we may restore lost service associations.
- queue blocking or removal by router operator because of policy violation.
- router downgrade (when it loses all service associations) with subsequent upgrade - the client would think queues are associated, while they are not, and won't receive any messages at all in this scenario.
- any other router-side error or logic error.
In addition to the possibility of the drift, we simply need to have confidence that service subscriptions work as intended, without skipping queues. We ignored this consideration for notifications, as the tolerance to lost notifications is higher, but we can't ignore it for messages.
## Solution
Previously considered approach of sending NIL to all queues without messages is very expensive for traffic (most queues don't have messages), and it is also very expensive to detect and validate drift in the client because of asynchronous / concurrent events.
We cannot read all queues into memory, and we cannot aggregate all responses in memory, and we cannot create database writes on every single service subscription to say 1m queues (a realistic number), as it simply won't work well even at the current scale.
An approach of having an efficient way to detect drift, but load the full list of IDs when drift is detected, also won't work well, as drifts may be common, so we need both efficient way to detect there is diff and also to reconcile it.
### Drift detection
Both client and router would maintain the number of associated queues and the "symmetric" hash over the set of queue IDs. The requirements for this hash algorithm are:
- not cryptographically strong, to be fast.
- 128 bits to minimize collisions over the large set of millions of queues.
- symmetric - the result should not depend on ID order.
- allows fast additions and removals.
In this way, every time association is added or removed (including queue marked as deleted), both peers would recompute this hash in the same transaction.
The client would suspend sending and processing any other commands on the router and the queues of this router until SOKS response is received from this router, to prevent drift. It can be achieved with per-router semaphores/locks in memory. UI clients need to become responsive sooner than these responses are received, but we do not service certificates on UI clients, and chat relays may prevent operations on router queues until SOKS response is received.
SOKS response would include both the count of associated queues (as now) and the hash over all associated queue IDs (to be added). If both count and hash match, the client will not do anything. If either does not match the client would perform full sync (see below).
There is a value from doing the same in notification router as well to detect and "fix" drifts.
The algorithm to compute hashes can be the following.
1. Compute hash of each queue ID using xxHash3_128 ([xxhash-ffi](https://hackage.haskell.org/package/xxhash-ffi) library). They don't need to be stored or loaded at once, initially, it can be done with streaming if it is detected on start that there is no pre-computed hash.
2. Combine hashes using XOR. XOR is both commutative and associative, so it would produce the same aggregate hash irrespective of the ID order.
3. Adding queue ID to pre-computed hash requires a single XOR with ID hash: `new_aggregate = aggregate XOR hash(queue_id)`.
4. Removing queue ID from pre-computed hash also requires the same XOR (XOR is involutory, it undoes itself): `new_aggregate = aggregate XOR hash(queue_id)`.
These hashes need to be computed per user/router in the client and per service certificate in the router - on startup both have to validate and compute them once if necessary.
There can be also a start-up option to recompute hashe(s) to detect and fix any errors.
This is all rather simple and would help detecting drifts.
### Synchronization when drift is detected
The assumption here is that in most cases drifts are rare, and isolated to few IDs (e.g., this is the case with notification router).
But the algorithm should be resilient to losing all associations, and it should not be substantially worse than simply restoring all associations or loading all IDs.
We have `c_n` and `c_hash` for client-side count and hash of queue IDs and `s_n` and `s_hash` for router-side, which are returned in SOKS response to SUBS command.
1. If `c_n /= s_n || c_hash /= s_hash`, the client must perform sync.
2. If `abs(c_n - s_n) / max(c_n, s_n) > 0.5`, the client will request the full list of queues (more than half of the queues are different), and will perform diff with the queues it has. While performing the diff the client will continue block operations with this user/router.
3. Otherwise would perform some algorithm for determining the difference between queue IDs between client and router. This algorithm can be made efficient (`O(log N)`) by relying on efficient sorting of IDs and database loading of ranges, via computing and communicating hashes of ranges, and performing a binary search on ranges, with batching to optimize network traffic.
This algorithm is similar to Merkle tree reconcilliation, but it is optimized for database reading of ordered ranges, and for our 16kb block size to minimize network requests.
The algorithm:
1. The client would request all ranges from the router.
2. The router would compute hashes for N ranges of IDs and send them to the client. Each range would include start_id, optional end_id (for single ID ranges) and XOR-hash of the range. N is determined based on the block size and the range size.
3. The client would perform the same computation for the same ranges, and compare them with the returned ranges from the router, while detecting any gaps between ranges and missing range boundaries.
4. If more than half of the ranges don't match, the client would request the full list. Otherwise it would repeat the same algorithm for each mismatched range and for gaps.
It can be further optimized by merging adjacent ranges and by batching all range requests, it is quite simple.
Once the client determines the list of missing and extra queues it can:
- create associations (via SUB) for missing queues,
- request removal of association (a new command, e.g. BUS) for extra queues on the router.
The pseudocode for the algorightm:
For the router to return all ranges or subranges of requested range:
```haskell
getSubRanges :: Maybe (RecipientId, RecipientId) -> [(RecipientId, Maybe RecipientId, Hash)]
getSubRanges range_ = do
((min_id, max_id), s_n) <- case range_ of
Nothing -> getAssociatedQueueRange -- with the certificate in the client session.
Just range -> (range,) <$> getAssociatedQueueCount range
if
| s_n <= max_N -> reply_with_single_queue_ranges
| otherwise -> do
let range_size = s_n `div` max_N
read_all_ranges -- in a recursive loop, with max_id, range_hash and next_min_id in each step
reply_ranges
```
We don't need to implement this synchronization logic right now, so not including client logic here, it's sufficient to implement drift detection, and the action to fix the drift would be to disable and to re-enable certificates via some command-line parameter of CLI.
+150
View File
@@ -0,0 +1,150 @@
# SimpleX Network Protocol Specifications — Governance and Evolution (draft)
## Why this document exists
SimpleX Network protocol specifications must evolve as the network grows. This document defines how specifications change, who governs those changes, and how the history of changes is preserved.
### Lessons from the web: why ratcheted governance matters
The web's governance history demonstrates both the necessity of consortium governance and the dangers of getting the transition wrong.
[Tim Berners-Lee invented the web in 1991](https://home.cern/science/computing/birth-web/short-history-web). [Netscape took over in 1994](https://en.wikipedia.org/wiki/Netscape_Navigator), driving rapid innovation as a single company — SSL, cookies, JavaScript, and the features that made the web commercially viable. In 1994, [W3C was founded](https://www.w3.org/about/history/) as a consortium hosted across multiple independent institutions (MIT in the US, INRIA/ERCIM in Europe, Keio University in Japan, later Beihang University in China) to govern web standards.
The transition from company-led innovation to consortium governance was abrupt rather than gradual. Netscape's decline (accelerated by the [browser wars](https://en.wikipedia.org/wiki/Browser_wars) and [AOL acquisition](https://cybercultural.com/p/1999-the-fall-of-netscape-and-the-rise-of-mozilla/)) transferred control to a standards body that prioritized process over progress. The result was [a lost decade of web stagnation](https://eev.ee/blog/2020/02/01/old-css-new-css/): CSS 2.0 shipped in 1998; CSS 2.1 didn't reach Candidate Recommendation until 2004 and wasn't finalized until 2011. W3C pursued XHTML and rejected proposed enhancements to HTML, until frustrated engineers from Apple, Mozilla, and Opera formed [WHATWG in 2004](https://en.wikipedia.org/wiki/WHATWG) to build HTML5 outside W3C's process. The abrupt governance transition, without a mechanism to balance community guarantees against the imperative to continue evolving the product at pace, dramatically slowed web evolution at the time it was needed most.
Then in 2023, [W3C restructured from a multi-host consortium into a single 501(c)(3) nonprofit entity](https://www.w3.org/press-releases/2023/w3c-le-launched/) — W3C Inc, incorporated in the US. The previous structure distributed governance across four independent university hosts in different countries, making capture by any single entity structurally difficult. The new structure concentrates governance in a single legal entity with a board of directors. While presented as modernization, this effectively ended the decentralized consortium model that had protected web standards for nearly three decades.
### The governance double ratchet
SimpleX follows the same Netscape-to-consortium evolution path, but with two ratchets designed to prevent both failure modes — stagnation from premature governance transfer, and capture from governance centralization:
- **Licensing ratchet**: all contributed IP is licensed under AGPLv3 (software) and Creative Commons (documentation), perpetually and irrevocably. What is licensed cannot be unlicensed. If a Party transfers Licensed IP, the licensing obligations transfer with it.
- **Governance ratchet**: power can be given to the SimpleX Network Consortium, but never taken back. The Consortium Agreement requires majority decision of all Governing Parties for changes to the agreement itself, IP policy, and admission or removal of parties.
The ratcheted transition is historically proven to be necessary. It allows the company to continue driving rapid product innovation (as Netscape did for the web) while incrementally and irreversibly transferring governance to the consortium, without the abrupt handover that stalled web evolution or the centralization that later undermined it.
### Specification governance via the Consortium Agreement
The SimpleX Network Consortium Agreement (being deployed in 2026) establishes two levels of intellectual property governance: **Licensed IP** (all contributed protocol specifications, software, and documentation, licensed perpetually and irrevocably) and **Core IP** (the subset essential to the network, requiring consortium governance to change). The distinction between these levels and how they map to the RFC process is described in [Standard vs Core specifications](#standard-vs-core-specifications) below.
## Specification change process: protocol specifications and RFCs
Protocol knowledge lives in two places:
### `protocol/` — Consolidated specifications
Each file is a complete, self-contained description of a protocol as it exists today. Like consolidated legislation in the UK legal system: the full current law in one document, not a patchwork of amendments.
Consolidated specifications are maintained on every code change that affects protocol behavior. With LLMs, the cost of maintaining consolidated documents collapses — reworking prose to incorporate a new RFC is now inexpensive relative to the value of a single authoritative document per protocol.
Implementers read `protocol/`. They should never need to reconstruct current behavior from a base spec plus a chain of RFCs.
### `rfcs/` — Protocol evolution commits
Each RFC describes a single change to a protocol specification. RFCs are the atomic unit of protocol evolution — analogous to commits in version control, or amending acts in legislation.
An RFC is not part of the protocol specification. It becomes part of the specification only when embedded into the consolidated `protocol/` document. The RFC itself remains as a permanent historical record of what changed, when, and why.
## RFC lifecycle
```
┌——> done/ ——> standard/
draft (root) ——>──┤
└——> rejected/
```
### Draft — `rfcs/*.md`
A proposal for a protocol change. Not yet implemented. Active proposals live in the `rfcs/` root directory.
Named by proposal date: `YYYY-MM-DD-topic.md`.
A draft may be rejected if the proposal is considered but not accepted for implementation.
### Done — `rfcs/done/`
Implemented in code. The protocol change described by this RFC exists in the codebase, but the RFC has not yet been verified against the actual implementation (code may have diverged from the proposal during implementation).
### Standard — `rfcs/standard/`
Verified against the actual implementation and synchronized with code. The RFC accurately describes what was implemented. This is a permanent historical record — standard RFCs are never modified or removed.
On promotion to standard, the RFC is:
1. Renamed from proposal date to standardization date: `YYYY-MM-DD-topic.md` (new date, same topic slug)
2. Updated with a document history header capturing the full lifecycle
3. Embedded into the corresponding `protocol/` consolidated specification
The `protocol/` document references embedded RFCs by name (e.g., "Private message routing added by RFC 2023-09-12-second-relays, standardized 2026-XX-XX"), similar to UK legislation citing the amending act for each clause.
Protocol version numbers make it clear which RFCs are included in which protocol revision — no separate tracking is needed.
### Rejected — `rfcs/rejected/`
Draft proposals that were considered but not accepted for implementation. Only drafts move to rejected — once an RFC is implemented (done/), it proceeds to standard/ after verification. Preserved for historical record of design decisions.
### Document history header
Every RFC in `standard/` carries a history header:
```
---
Proposed: YYYY-MM-DD
Implemented: YYYY-MM-DD
Standardized: YYYY-MM-DD
Protocol: simplex-messaging v9 (or whichever protocol this amends)
---
```
## Governance
SimpleX Network follows the Netscape-to-W3C evolution path, with ratcheted rather than abrupt transitions:
| Phase | Period | Governance | Development process |
|-------|--------|-----------|-------------------|
| Protocol invented | 2020 | Two people | Prototype developed |
| SimpleX Chat Ltd | 2022 | One company | Product-first: code leads, specs follow |
| SimpleX Network Consortium | 2026 | Agreement of SimpleX Chat Ltd and non-profit entities | Product-first for standard; standards-first for core |
| Decentralized governance | Future | TBD (DAO research ongoing) | Standards-first |
### Current: product-first development
SimpleX protocols currently follow a product-first development process: requirements drive code, code drives specification. RFCs are written as design proposals before implementation, but implementation details are figured out in code. Consolidated protocol specifications in `protocol/` are then amended to match the implementation.
This process is governed by SimpleX Chat Ltd as the IP Holding Party under the Consortium Agreement.
Any Specification Author (as defined in the Consortium Agreement) may propose RFCs. Acceptance and standardization decisions are made by SimpleX Chat Ltd during the current product-first phase.
### Standard vs Core specifications
The distinction between standard and core maps directly to the two levels of IP governance in the Consortium Agreement, and reflects the difference between product-first and standards-first development:
**Standard** — Licensed IP, not yet under consortium governance. Governed by the company.
All contributed protocol specifications are Licensed IP under the Consortium Agreement. Standard specifications follow product-first development: the company can evolve them with product needs, and they must be maintained on every code change that affects protocol behavior.
Standard specifications live in `rfcs/standard/` and `protocol/`.
**Core** — Governed IP, governed by the consortium.
A subset of standard specifications will be designated as Core IP under the Consortium Agreement. Core specifications will follow standards-first development: specification changes must be agreed via Governing Decision before code changes.
This is a legally binding commitment. Once Licensed IP is included in Core IP, the company that owns the code cannot unilaterally change it — even though they own the code, the Consortium Agreement requires a Governing Decision for any change to Core IP. This protects the fundamental properties of the network (privacy, security, decentralization) from unilateral modification by any single party.
The designation of specific specifications as Core IP is itself a Governing Decision that requires Consortium vote. The transition will happen incrementally as protocols stabilize — the governance ratchet ensures that each designation is irreversible.
The exact mechanism for distinguishing core from standard within the RFC and protocol folder structure is TBD — it will be decided as the first protocols are designated as Core IP.
### Future: standards-first development
As more protocols are designated as Core IP, development naturally transitions to a standards-first process for a growing portion of the protocol suite. The governance ratchet ensures this transition is gradual and irreversible — each protocol that becomes core gains the protection of consortium governance permanently, while remaining standard protocols continue to evolve at product pace.
## Current state
| Location | Contents | Count |
|----------|----------|-------|
| `protocol/` | Consolidated specs (SMP v19, Agent v7, XFTP v3, XRCP v1, NTF v3, PQDR v1) | 6 specs + overview |
| `rfcs/` root | Active draft proposals | 10 |
| `rfcs/done/` | Implemented, not yet verified | 1 (+10 sub-RFCs) |
| `rfcs/standard/` | Verified against implementation | 31 |
| `rfcs/rejected/` | Draft proposals not accepted | 7 |
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,154 @@
# XFTP Router: SNI, CORS, and Web Support
Implementation details for Phase 3 of `rfcs/2026-01-30-send-file-page.md` (sections 6.1-6.4).
## 1. Overview
The XFTP router is extended to support web browser clients by:
1. **SNI-based TLS certificate switching** — Present a CA-issued web certificate (e.g., Let's Encrypt) to browsers, while continuing to present the self-signed XFTP identity certificate to native XFTP clients.
2. **CORS headers** — Add CORS response headers on SNI connections so browsers allow cross-origin XFTP requests.
3. **Configuration**`[WEB]` INI section for HTTPS cert/key paths; opt-in (commented out by default).
Web handshake (challenge-response identity proof, §6.3 of parent RFC) is not yet implemented and will be added separately.
## 2. SNI Certificate Switching
### 2.1 Reusing the SMP Pattern
The SMP router already implements SNI-based certificate switching via `TLSServerCredential` and `runTransportServerState_` (see `rfcs/2024-09-15-shared-port.md`). The XFTP router applies the same pattern with one key difference: both native and web XFTP clients use HTTP/2 transport, whereas SMP switches between raw SMP protocol and HTTP entirely.
### 2.2 Approach
When `httpServerCreds` is configured, the XFTP router bypasses `runHTTP2Server` and uses `runTransportServerState_` directly to obtain the per-connection `sniUsed` flag. It then sets up HTTP/2 manually on each TLS connection using `withHTTP2` (same internals as `runHTTP2ServerWith_`). The `sniUsed` flag is captured in the closure and shared by all HTTP/2 requests on that connection.
When `httpServerCreds` is absent, the existing `runHTTP2Server` path is unchanged.
```
Native client (no SNI) ──TLS──> XFTP identity cert ──HTTP/2──> processRequest (no CORS)
Browser client (SNI) ──TLS──> Web CA cert ──HTTP/2──> processRequest (+ CORS)
```
### 2.3 Certificate Chain
The web certificate file (e.g., `web.crt`) must contain the full chain: leaf certificate followed by the signing CA certificate. `loadServerCredential` uses `T.credentialLoadX509Chain` which reads all PEM blocks from the file.
The client validates the chain by comparing `idCert` fingerprint (the CA cert, second in the 2-cert chain) against the known `keyHash`. This is the same validation as for XFTP identity certificates — the CA that signed the web cert must match the XFTP router's identity.
## 3. CORS Support
### 3.1 Design
CORS headers are only added when both conditions are true:
- `addCORSHeaders` is `True` in `TransportServerConfig` (set in XFTP `Main.hs`)
- `sniUsed` is `True` for the current TLS connection
This ensures native clients never see CORS headers.
### 3.2 Response Headers
All POST responses on SNI connections include:
```
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: *
```
### 3.3 OPTIONS Preflight
OPTIONS requests are intercepted at the HTTP/2 dispatch level, before `processRequest`. This is necessary because `processRequest` rejects bodies that don't match `xftpBlockSize`.
Preflight response:
```
HTTP/2 200
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: POST, OPTIONS
Access-Control-Allow-Headers: *
Access-Control-Max-Age: 86400
```
### 3.4 Security
`Access-Control-Allow-Origin: *` is safe because:
- All XFTP commands require Ed25519 authentication (per-packet keys from file description).
- No cookies or browser credentials are involved.
- File content is end-to-end encrypted.
## 4. Configuration
### 4.1 INI Template
```ini
[WEB]
# cert: /etc/opt/simplex-xftp/web.crt
# key: /etc/opt/simplex-xftp/web.key
```
Commented out by default — web support is opt-in.
### 4.2 Behavior
- `[WEB]` section not configured: silently ignored, router operates normally for native clients only.
- `[WEB]` section configured with valid cert/key paths: SNI + CORS enabled.
- `[WEB]` section configured with missing cert files: warning + continue (non-fatal, unlike SMP router where it is fatal).
## 5. Files Modified
### 5.1 `src/Simplex/Messaging/Transport/Server.hs`
Added `addCORSHeaders :: Bool` field to `TransportServerConfig`. Updated `mkTransportServerConfig` to accept the new parameter. All existing SMP call sites pass `False`.
### 5.2 `src/Simplex/Messaging/Transport/HTTP2/Server.hs`
- Extracted `expireInactiveClient` from `runHTTP2ServerWith_`'s `where` clause to a module-level function.
- Parameterized `runHTTP2ServerWith_`: setup type changed from `((TLS p -> IO ()) -> a)` to `(((Bool, TLS p) -> IO ()) -> a)`, callback from `HTTP2ServerFunc` to `Bool -> HTTP2ServerFunc`. The `Bool` is the per-connection `sniUsed` flag, threaded through `H.run` to the callback.
- Extended `runHTTP2Server` with `Maybe T.Credential` parameter for SNI web certificate. Its setup uses `runTransportServerState_` with `TLSServerCredential`, which naturally provides `(sniUsed, tls)` pairs matching the new `runHTTP2ServerWith_` setup type.
- Adapted `runHTTP2ServerWith` (client-side HTTP/2, no SNI): wraps its setup to inject `(False, tls)` and its callback with `const`.
- Updated `getHTTP2Server` (test helper) to pass `Nothing` for httpCreds.
### 5.3 `src/Simplex/FileTransfer/Server/Env.hs`
- Added `httpCredentials :: Maybe ServerCredentials` to `XFTPServerConfig`.
- Added `httpServerCreds :: Maybe T.Credential` to `XFTPEnv`.
- `newXFTPServerEnv` loads HTTP credentials when configured.
### 5.4 `src/Simplex/FileTransfer/Server/Main.hs`
- Added `[WEB]` section to INI template.
- Added `httpCredentials` parsing from INI `[WEB]` section (`cert` and `key` fields).
- Set `addCORSHeaders = isJust httpCredentials_` in transport config (conditional on web cert presence).
### 5.5 `src/Simplex/FileTransfer/Server.hs`
Core server changes:
- `runServer` calls `runHTTP2Server` with `httpCreds_` and a `\sniUsed -> handleRequest (sniUsed && addCORSHeaders transportConfig)` callback. TLS params are `defaultSupportedParamsHTTPS` when web creds present, `defaultSupportedParams` otherwise. SNI routing, HTTP/2 setup, and client expiration are handled inside `runHTTP2Server`.
- `XFTPTransportRequest` carries `addCORS :: Bool` field, threaded through to `sendXFTPResponse`.
- `sendXFTPResponse` conditionally includes CORS headers based on `addCORS`.
- OPTIONS requests on SNI connections return CORS preflight headers before reaching `processRequest`.
- Helper functions: `corsHeaders` (response headers), `corsPreflightHeaders` (preflight headers).
### 5.6 `tests/XFTPClient.hs`
- Added `httpCredentials = Nothing` to `testXFTPServerConfig`.
- Added `testXFTPServerConfigSNI` with web cert config and `addCORSHeaders = True`.
- Added `withXFTPServerSNI` helper.
### 5.7 `tests/XFTPServerTests.hs`
Added SNI and CORS tests as a subsection within `xftpServerTests` (6 tests):
1. **SNI cert selection** — Connect with SNI + `h2` ALPN, verify RSA web certificate is presented.
2. **Non-SNI cert selection** — Connect without SNI + `xftp/1` ALPN, verify Ed448 XFTP certificate is presented.
3. **CORS headers** — SNI POST request includes `Access-Control-Allow-Origin: *` and `Access-Control-Expose-Headers: *`.
4. **OPTIONS preflight** — SNI OPTIONS request returns all CORS preflight headers.
5. **No CORS without SNI** — Non-SNI POST request has no CORS headers.
6. **Data packet delivery** — Full XFTP data packet upload/download through SNI-enabled router verifying no regression.
## 6. Remaining Work
- **Web handshake** (§6.3 of parent RFC): Challenge-response identity proof for SNI connections. The router detects web clients via the `sniUsed` flag and expects a 32-byte challenge in the first POST body (non-empty, unlike standard handshake). Response includes full cert chain + signature over `(challenge ++ sessionId)`.
- **Static page serving** (§6.5 of parent RFC): Optional serving of the web page HTML/JS bundle on GET requests.
@@ -0,0 +1,246 @@
# Web Handshake — Challenge-Response Identity Proof
RFC §6.3: Router proves XFTP identity to web clients independently of TLS CA infrastructure.
## 1. Protocol
**Standard handshake** (unchanged):
```
Client → empty POST → Server
Server → padded {vRange, sessionId, authPubKey, Nothing} → Client
Client → padded {version, keyHash, Nothing} → Server
Server → empty → Client
```
**Web handshake** (SNI connection, non-empty hello):
```
Client → padded {32 random bytes} → Server
Server → padded {vRange, sessionId, authPubKey, Just sigBytes} → Client
sigBytes = signatureBytes(sign(identityLeafKey, challenge <> sessionId))
Client validates:
1. chainIdCaCerts(authPubKey.certChain) → CCValid {leafCert, idCert}
2. SHA-256(idCert) == keyHash (server identity)
3. verify(leafCert.pubKey, sigBytes, challenge <> sessionId) (challenge-response)
4. verify(leafCert.pubKey, signedPubKey.signature, signedPubKey.objectDer) (DH key auth)
Client → padded {version, keyHash, Just challenge} → Server
Server verifies: echoed challenge == stored challenge from step 1
Server → empty → Client
```
**Detection**: `sniUsed` per-connection flag. Non-empty hello allowed only when `sniUsed`. Empty hello with SNI → standard handshake.
**Why both steps 3 and 4**: Native clients verify `signedPubKey` using the TLS peer certificate (`serverKey` from `getServerVerifyKey`), which is the XFTP identity cert in non-SNI connections — TLS provides this binding. Web clients cannot access TLS peer certificate data (browser API limitation; TLS presents the web CA cert but provides no API to extract it). So web clients must verify at the application layer using `authPubKey.certChain`, which always contains the XFTP identity chain regardless of which cert TLS used. Step 3 proves the router holds its identity key *right now* (freshness via random challenge). Step 4 proves the DH session key was signed by the identity key holder (prevents MITM key substitution). Together they give web clients some assurance native clients get from TLS, except channel binding for commands.
## 2. Type Changes — `src/Simplex/FileTransfer/Transport.hs`
### `XFTPServerHandshake` (line 114)
Add field: `webIdentityProof :: Maybe ByteString` — raw Ed448 signature bytes (114 bytes), or `Nothing` for standard handshake. No record needed — the cert chain is already in `authPubKey.certChain`.
### `Encoding XFTPServerHandshake` (line 136)
- `smpEncode`: append `smpEncode webIdentityProof`
- `smpP`: `Tail compat`, if non-empty `eitherToMaybe $ smpDecode compat`
Backward compat: old clients ignore via `Tail _compat`; new client + old server → empty compat → `Nothing`.
### `XFTPClientHandshake` (line 121)
Add field: `webChallenge :: Maybe ByteString`
### `Encoding XFTPClientHandshake` (line 128)
Same `Tail compat` pattern as server handshake.
### Export list
Both types use `(..)` export — new fields auto-exported.
## 3. Router Changes — `src/Simplex/FileTransfer/Server.hs`
### `XFTPTransportRequest` (line 88)
Add field: `sniUsed :: SNICredentialUsed` (`Bool` from `Transport.Server`). Add import.
### `Handshake` (line 117)
`HandshakeSent C.PrivateKeyX25519``HandshakeSent C.PrivateKeyX25519 (Maybe ByteString)` — stores 32-byte web challenge or `Nothing`.
### `runServer` handler (line 145161)
- Pass `sniUsed` into request construction (line 154)
- SNI-first routing: when `sniUsed`, always route to `xftpServerHandshakeV1` (web ALPN `h2` would otherwise fall to `_` catch-all)
### `xftpServerHandshakeV1` (line 162)
- Destructure `sniUsed` from request
- Match `HandshakeSent pk challenge_``processClientHandshake pk challenge_`
### `processHello` (line 171)
- Branch `(sniUsed, B.null bodyHead)`:
- `(_, True)` → standard: `challenge_ = Nothing`
- `(True, False)` → web: unpad, verify 32 bytes, `challenge_ = Just`
- `(False, False)``throwE HANDSHAKE`
- Store: `HandshakeSent pk challenge_`
- Compute: `webIdentityProof = C.signatureBytes . C.sign serverSignKey . (<> sessionId) <$> challenge_`
- Construct `XFTPServerHandshake` with `webIdentityProof`
### `processClientHandshake` (line 183)
- Accept `challenge_` parameter
- Decode `webChallenge` from `XFTPClientHandshake`
- Add: `unless (challenge_ == webChallenge) $ throwE HANDSHAKE`
(standard: both `Nothing` → passes)
## 4. Native Client — `src/Simplex/FileTransfer/Client.hs`
### `xftpClientHandshakeV1` (line 142)
Add `webChallenge = Nothing` in `sendClientHandshake` call.
No other changes — parser handles new fields via `Tail`, native client ignores `webIdentityProof`.
## 5. TypeScript Changes (DONE except Ed448)
Sections 5.1 and 5.2 are implemented. Section 5.3 needs Ed448 support.
## 10. Ed448 Support via `@noble/curves`
**Problem**: Production servers use Ed448 certificates (default). `identity.ts` only supports Ed25519 via libsodium. libsodium has no Ed448 support and never will.
**Solution**: Add `@noble/curves` dependency for Ed448 verification only. All other crypto stays with libsodium.
### 10.1 `xftp-web/package.json` — Add dependency
```json
"dependencies": {
"libsodium-wrappers-sumo": "^0.7.13",
"@noble/curves": "^1.9.7"
}
```
Use v1.x (supports both CJS and ESM). v2.x is ESM-only with `.js` extension requirement.
### 10.2 `xftp-web/src/crypto/keys.ts` — Ed448 DER constants and decode
Add Ed448 SPKI DER prefix (12 bytes, same prefix length as Ed25519):
```
30 43 30 05 06 03 2b 65 71 03 3a 00
```
| Property | Ed25519 | Ed448 |
|----------|---------|-------|
| OID | `2b 65 70` | `2b 65 71` |
| SPKI prefix | `30 2a ...` | `30 43 ...` |
| Raw key size | 32 bytes | 57 bytes |
| SPKI total | 44 bytes | 69 bytes |
| Signature size | 64 bytes | 114 bytes |
New functions:
- `decodePubKeyEd448(der: Uint8Array): Uint8Array` — 69 bytes → 57 bytes raw
- `encodePubKeyEd448(raw: Uint8Array): Uint8Array` — 57 bytes → 69 bytes DER
- `verifyEd448(publicKey: Uint8Array, sig: Uint8Array, msg: Uint8Array): boolean` — uses `ed448.verify(sig, msg, publicKey)` from `@noble/curves/ed448`
Note: `@noble/curves` parameter order is `(signature, message, publicKey)`, not `(publicKey, signature, message)`.
### 10.3 `xftp-web/src/crypto/identity.ts` — Algorithm-agnostic verification
Replace `extractCertEd25519Key` + hardcoded Ed25519 `verify` with algorithm detection:
1. `extractCertPublicKeyInfo(certDer)` → SPKI DER (already exists, works for any algorithm)
2. Detect algorithm from SPKI: byte at offset 8 is `0x70` (Ed25519) or `0x71` (Ed448)
3. Extract raw key with appropriate decoder
4. Verify signatures with appropriate function
```typescript
type CertKeyAlgorithm = 'ed25519' | 'ed448'
function detectKeyAlgorithm(spki: Uint8Array): CertKeyAlgorithm {
if (spki.length === 44 && spki[8] === 0x70) return 'ed25519'
if (spki.length === 69 && spki[8] === 0x71) return 'ed448'
throw new Error("unsupported certificate key algorithm")
}
```
`verifyIdentityProof` changes:
- Extract SPKI from leaf cert
- Detect algorithm → choose `decodePubKeyEd25519`/`decodePubKeyEd448` and `verify`/`verifyEd448`
- Both challenge signature and DH key signature use the same leaf key + algorithm
Remove `extractCertEd25519Key` (replaced by generic path). Keep `extractCertPublicKeyInfo` (already generic).
### 10.4 `xftp-web/src/protocol/handshake.ts` — Comment update
`SignedKey.signature` comment: "raw Ed25519 signature bytes (64 bytes)" → "raw signature bytes (Ed25519: 64, Ed448: 114)"
### 10.5 Tests — `tests/XFTPWebTests.hs`
**Integration test**: Switch from `withXFTPServerEd25519SNI` (Ed25519 fixtures) to `withXFTPServerSNI` (default Ed448 fixtures). Update fingerprint source from `tests/fixtures/ed25519/ca.crt` to the default `tests/fixtures/ca.crt`.
Optionally add a second integration test with Ed25519 to cover both paths, or rely on existing unit tests for Ed25519 coverage.
### 10.6 Implementation order
1. `npm install @noble/curves` in `xftp-web/`
2. `keys.ts` — Ed448 constants, decode, encode, verifyEd448
3. `identity.ts` — algorithm detection, generic verification
4. `handshake.ts` — comment fix
5. `XFTPWebTests.hs` — switch integration test to Ed448
6. Build TS + run all tests
## 6. Haskell Integration Test — `tests/XFTPServerTests.hs`
Add `testWebHandshake` to "XFTP SNI and CORS" describe block.
1. `withXFTPServerSNI` — server with web credentials
2. Connect with SNI + `h2` ALPN
3. Send padded 32-byte challenge
4. Decode `XFTPServerHandshake`, assert `webIdentityProof` is `Just`
5. `chainIdCaCerts` on `authPubKey.certChain``CCValid {leafCert, idCert}`
6. Verify `SHA-256(idCert) == keyHash`
7. Extract `leafCert` public key, verify challenge signature
8. Verify `signedPubKey` signature using `leafCert` key (DH key auth)
9. Send `XFTPClientHandshake` with `webChallenge = Just challenge`
10. Assert empty response
Imports: `XFTPServerHandshake (..)`, `XFTPClientHandshake (..)`, `ChainCertificates (..)`, `chainIdCaCerts`.
## 7. TS Tests — `tests/XFTPWebTests.hs`
### Unit tests
- **`decodeServerHandshake` with proof**: Haskell-encode with `Just sigBytes`, TS-decode, verify bytes match.
- **`encodeClientHandshake` with challenge**: TS-encode, compare with Haskell-encoded.
- **`chainIdCaCerts`**: 2/3/4-cert chains return correct positions.
- **`caFingerprint` (fixed)**: matches `sha256(idCert)` for 2 and 3-cert chains.
### Integration test
Node.js inline script against `withXFTPServerSNI`:
1. Connect with SNI via `http2.connect`
2. Send padded challenge, decode `XFTPServerHandshake` with TS
3. `verifyIdentityProof` — full chain validation + challenge sig + DH key sig
4. Send client handshake with echoed challenge
5. Assert empty response
## 8. Implementation Order
1. `Transport.hs``Maybe` fields + encoding instances
2. `Server.hs``sniUsed`, challenge in `Handshake`, `processHello`, `processClientHandshake`, SNI routing
3. `Client.hs``webChallenge = Nothing`
4. Build: `cabal build --ghc-options -O0`
5. Run existing SNI/CORS tests
6. `XFTPServerTests.hs``testWebHandshake`
7. `handshake.ts` — types, decoding, `chainIdCaCerts`, fix `caFingerprint`
8. `crypto/identity.ts` — Node.js verification functions
9. `XFTPWebTests.hs` — unit + integration tests
10. Build TS + run all tests
## 9. Verification
```bash
cd xftp-web && npm install && npm run build && cd ..
cabal test --ghc-options=-O0 --test-option='--match=/XFTP/XFTP server/XFTP SNI and CORS/' --test-show-details=streaming
cabal test --ghc-options=-O0 --test-option='--match=/XFTP Web Client/' --test-show-details=streaming
```
@@ -0,0 +1,208 @@
# Plan: Browser ↔ Haskell File Transfer Tests
## Table of Contents
1. Goal
2. Current State
3. Implementation
4. Success Criteria
5. Files
6. Order
## 1. Goal
Run browser upload/download tests in headless Chromium via Vitest, proving fetch-based transport works in real browser environment.
## 2. Current State
- `client.ts`: Transport abstraction done — http2 for Node, fetch for browser ✓
- `agent.ts`: Uses `node:crypto` (randomBytes) and `node:zlib` (deflateRawSync/inflateRawSync) — **won't run in browser**
- `XFTPWebTests.hs`: Cross-language tests exist (Haskell calls TS via Node.js) ✓
## 3. Implementation
### 3.1 Make agent.ts isomorphic
| Current (Node.js only) | Isomorphic replacement |
|------------------------|------------------------|
| `import crypto from "node:crypto"` | Remove import |
| `import zlib from "node:zlib"` | `import pako from "pako"` |
| `crypto.randomBytes(32)` | `crypto.getRandomValues(new Uint8Array(32))` |
| `zlib.deflateRawSync(buf)` | `pako.deflateRaw(buf)` |
| `zlib.inflateRawSync(buf)` | `pako.inflateRaw(buf)` |
Note: `crypto.getRandomValues` available in both browser and Node.js (globalThis.crypto).
### 3.2 Vitest browser mode setup
`package.json` additions:
```json
"devDependencies": {
"vitest": "^3.0.0",
"@vitest/browser": "^3.0.0",
"playwright": "^1.50.0",
"@types/pako": "^2.0.3"
},
"dependencies": {
"pako": "^2.1.0"
}
```
`vitest.config.ts`:
```typescript
import {defineConfig} from 'vitest/config'
import {readFileSync} from 'fs'
import {createHash} from 'crypto'
// Compute fingerprint from ca.crt (same as Haskell's loadFileFingerprint)
const caCert = readFileSync('../tests/fixtures/ca.crt')
const fingerprint = createHash('sha256').update(caCert).digest('base64url')
const serverAddr = `xftp://${fingerprint}@localhost:7000`
export default defineConfig({
define: {
'import.meta.env.XFTP_SERVER': JSON.stringify(serverAddr)
},
test: {
browser: {
enabled: true,
provider: 'playwright',
instances: [{browser: 'chromium'}],
headless: true,
providerOptions: {
launch: {ignoreHTTPSErrors: true}
}
},
globalSetup: './test/globalSetup.ts'
}
})
```
### 3.3 Server startup
`test/globalSetup.ts`:
```typescript
import {spawn, ChildProcess} from 'child_process'
import {resolve, join} from 'path'
import {mkdtempSync, writeFileSync, copyFileSync} from 'fs'
import {tmpdir} from 'os'
let server: ChildProcess | null = null
export async function setup() {
const fixtures = resolve(__dirname, '../../tests/fixtures')
// Create temp directories
const cfgDir = mkdtempSync(join(tmpdir(), 'xftp-cfg-'))
const logDir = mkdtempSync(join(tmpdir(), 'xftp-log-'))
const filesDir = mkdtempSync(join(tmpdir(), 'xftp-files-'))
// Copy certificates to cfgDir (xftp-server expects ca.crt, server.key, server.crt there)
copyFileSync(join(fixtures, 'ca.crt'), join(cfgDir, 'ca.crt'))
copyFileSync(join(fixtures, 'server.key'), join(cfgDir, 'server.key'))
copyFileSync(join(fixtures, 'server.crt'), join(cfgDir, 'server.crt'))
// Write INI config file
const iniContent = `[STORE_LOG]
enable: off
[TRANSPORT]
host: localhost
port: 7000
[FILES]
path: ${filesDir}
[WEB]
cert: ${join(fixtures, 'web.crt')}
key: ${join(fixtures, 'web.key')}
`
writeFileSync(join(cfgDir, 'file-server.ini'), iniContent)
// Spawn xftp-server with env vars
server = spawn('cabal', ['exec', 'xftp-server', '--', 'start'], {
env: {
...process.env,
XFTP_SERVER_CFG_PATH: cfgDir,
XFTP_SERVER_LOG_PATH: logDir
},
stdio: ['ignore', 'pipe', 'pipe']
})
// Wait for "Listening on port 7000..."
await waitForServerReady(server)
}
export async function teardown() {
server?.kill('SIGTERM')
await new Promise(r => setTimeout(r, 500))
}
function waitForServerReady(proc: ChildProcess): Promise<void> {
return new Promise((resolve, reject) => {
const timeout = setTimeout(() => reject(new Error('Server start timeout')), 15000)
proc.stdout?.on('data', (data: Buffer) => {
if (data.toString().includes('Listening on port')) {
clearTimeout(timeout)
resolve()
}
})
proc.stderr?.on('data', (data: Buffer) => {
console.error('[xftp-server]', data.toString())
})
proc.on('error', reject)
proc.on('exit', (code) => {
clearTimeout(timeout)
if (code !== 0) reject(new Error(`Server exited with code ${code}`))
})
})
}
```
Server env vars (from `apps/xftp-server/Main.hs` + `getEnvPath`):
- `XFTP_SERVER_CFG_PATH` — directory containing `file-server.ini` and certs (`ca.crt`, `server.key`, `server.crt`)
- `XFTP_SERVER_LOG_PATH` — directory for logs
### 3.4 Browser test
`test/browser.test.ts`:
```typescript
import {test, expect} from 'vitest'
import {encryptFileForUpload, uploadFile, downloadFile} from '../src/agent.js'
import {parseXFTPServer} from '../src/protocol/address.js'
const server = parseXFTPServer(import.meta.env.XFTP_SERVER)
test('browser upload + download round-trip', async () => {
const data = new Uint8Array(50000)
crypto.getRandomValues(data)
const encrypted = encryptFileForUpload(data, 'test.bin')
const {rcvDescription} = await uploadFile(server, encrypted)
const {content} = await downloadFile(rcvDescription)
expect(content).toEqual(data)
})
```
## 4. Success Criteria
1. `npm run build` — agent.ts compiles without node: imports
2. `cabal test --test-option='--match=/XFTP Web Client/'` — existing Node.js tests still pass
3. `npm run test:browser` — browser round-trip test passes in headless Chromium
## 5. Files to Create/Modify
**Modify:**
- `xftp-web/package.json` — add vitest, @vitest/browser, playwright, pako, @types/pako
- `xftp-web/src/agent.ts` — replace node:crypto, node:zlib with isomorphic alternatives
**Create:**
- `xftp-web/vitest.config.ts` — browser mode config
- `xftp-web/test/globalSetup.ts` — xftp-server lifecycle
- `xftp-web/test/browser.test.ts` — browser round-trip test
## 6. Order of Implementation
1. **Add pako dependency**`npm install pako @types/pako`
2. **Make agent.ts isomorphic** — replace node:crypto, node:zlib
3. **Verify Node.js tests pass**`cabal test --test-option='--match=/XFTP Web Client/'`
4. **Set up Vitest** — add devDeps, create vitest.config.ts
5. **Create globalSetup.ts** — write INI config, spawn xftp-server
6. **Write browser test** — upload + download round-trip
7. **Verify browser test passes**`npm run test:browser`
@@ -0,0 +1,921 @@
# Browser Transport & Web Worker Architecture
## TOC
1. Executive Summary
2. Transport: fetch() API
3. Architecture: Environment Abstraction
4. Web Worker Implementation
5. OPFS Implementation
6. Implementation Plan
7. Testing Strategy
## 1. Executive Summary
Adapt `client.ts` from `node:http2` to `fetch()` API for isomorphic Node.js/browser support. Add environment abstraction layer so the same upload/download pipeline works with or without Web Workers and with or without OPFS. In browsers, crypto runs in a Web Worker to keep UI responsive; in Node.js tests, crypto runs directly.
**Key architectural constraint:** Existing crypto functions (`encryptFile`, `decryptChunks`, etc.) remain unchanged. The abstraction layer wraps them, choosing execution context (direct vs Worker) and storage (memory vs OPFS) based on environment.
**Scope:**
- Replace `node:http2` with `fetch()` in `client.ts`
- Add `CryptoBackend` abstraction with three implementations
- Create Web Worker that calls existing crypto functions
- Add OPFS storage for large files in browser
**Out of scope:** Web page UI (Phase 5 in main RFC).
## 2. Transport: fetch() API
### 2.1 Current State
`client.ts` uses `node:http2`:
```typescript
import http2 from "node:http2"
const session = http2.connect(url)
const stream = session.request({':method': 'POST', ':path': '/'})
stream.write(commandBlock)
stream.end(chunkData)
```
### 2.2 Target State
Isomorphic `fetch()` (Node.js 18+ and browsers):
```typescript
const response = await fetch(url, {
method: 'POST',
body: concatStreams(commandBlock, chunkData),
duplex: 'half', // Required for streaming request body
})
const reader = response.body!.getReader()
```
### 2.3 Key Differences
| Aspect | node:http2 | fetch() |
|--------|-----------|---------|
| Session management | Explicit `session.connect()` / `session.close()` | Per-request (HTTP/2 connection reuse is automatic) |
| Streaming upload | `stream.write()` chunks | `ReadableStream` body + `duplex: 'half'` |
| Streaming download | `stream.on('data')` | `response.body.getReader()` |
| Connection pooling | Manual | Automatic per origin |
### 2.4 API Changes
```typescript
// Before (node:http2)
export interface XFTPClient {
session: http2.ClientHttp2Session
thParams: THParams
server: XFTPServer
}
// After (fetch)
export interface XFTPClient {
baseUrl: string // "https://host:port"
thParams: THParams
server: XFTPServer
}
```
`connectXFTP()` performs handshake via fetch, returns `XFTPClient` with `baseUrl`.
Subsequent commands use `fetch(client.baseUrl, ...)`.
### 2.5 Handshake via fetch()
**TLS session binding:** Multiple fetch() requests to the same origin reuse the HTTP/2 connection, which means they share the same TLS session. The server's `sessionId` (derived from TLS channel binding) remains consistent across the handshake round-trips and subsequent commands.
```typescript
async function connectXFTP(server: XFTPServer): Promise<XFTPClient> {
const baseUrl = `https://${server.host}:${server.port}`
// Round-trip 1: challenge → server handshake + identity proof
const challenge = crypto.getRandomValues(new Uint8Array(32))
const req1 = pad(encodeWebClientHello(challenge), xftpBlockSize)
const resp1 = await fetch(baseUrl, {method: 'POST', body: req1})
const reader = resp1.body!.getReader()
const serverBlock = await readExactly(reader, xftpBlockSize)
const serverHs = decodeServerHandshake(unPad(serverBlock))
const proofBody = await readRemaining(reader)
verifyIdentityProof(server.keyHash, challenge, serverHs.sessionId, proofBody)
// Round-trip 2: client handshake → server ack
const clientHs = encodeClientHandshake({xftpVersion: 3, keyHash: server.keyHash})
const req2 = pad(clientHs, xftpBlockSize)
await fetch(baseUrl, {method: 'POST', body: req2})
return {baseUrl, thParams: {sessionId: serverHs.sessionId, ...}, server}
}
```
### 2.6 Command Execution
```typescript
async function sendXFTPCommand(
client: XFTPClient,
key: Uint8Array,
entityId: Uint8Array,
cmd: Uint8Array,
chunkData?: Uint8Array
): Promise<{response: Uint8Array, body?: ReadableStream}> {
const block = xftpEncodeAuthTransmission(client.thParams, key, entityId, cmd)
const reqBody = chunkData
? concatBytes(block, chunkData)
: block
const resp = await fetch(client.baseUrl, {
method: 'POST',
body: reqBody,
duplex: 'half',
})
const reader = resp.body!.getReader()
const responseBlock = await readExactly(reader, xftpBlockSize)
const parsed = xftpDecodeTransmission(responseBlock)
// For FGET: remaining body is encrypted chunk
const hasMore = await peekReader(reader)
return {
response: parsed,
body: hasMore ? wrapAsStream(reader) : undefined
}
}
```
## 3. Architecture: Environment Abstraction
### 3.1 Core Principle
**Existing crypto functions remain unchanged.** The functions `encryptFile()`, `decryptChunks()`, `sha512()`, etc. in `crypto/file.ts` and `crypto/digest.ts` are pure computation — they take input bytes and produce output bytes. They have no knowledge of Workers, OPFS, or execution context.
The abstraction layer sits between `agent.ts` (upload/download orchestration) and these crypto functions:
```
┌─────────────────────────────────────────────────────────────────────┐
│ agent.ts (upload/download orchestration) │
│ - Unchanged logic: encrypt → chunk → upload → build description │
│ - Calls CryptoBackend interface, not crypto functions directly │
├─────────────────────────────────────────────────────────────────────┤
│ CryptoBackend interface (env.ts) │
│ - Abstract interface for encrypt/decrypt/readChunk/writeChunk │
│ - Factory function selects implementation based on environment │
├──────────────┬──────────────────────┬───────────────────────────────┤
│ DirectMemory │ WorkerMemory │ WorkerOPFS │
│ Backend │ Backend │ Backend │
│ (Node.js) │ (Browser, ≤50MB) │ (Browser, >50MB) │
├──────────────┼──────────────────────┼───────────────────────────────┤
│ Calls crypto │ Posts to Worker, │ Posts to Worker, │
│ functions │ Worker calls crypto │ Worker calls crypto, │
│ directly │ functions, returns │ streams through OPFS │
│ │ via postMessage │ │
├──────────────┴──────────────────────┴───────────────────────────────┤
│ crypto/file.ts, crypto/digest.ts (unchanged) │
│ - encryptFile(), decryptChunks(), sha512(), etc. │
│ - Pure functions, no environment dependencies │
└─────────────────────────────────────────────────────────────────────┘
```
### 3.2 CryptoBackend Interface
```typescript
// env.ts
export interface CryptoBackend {
// Encrypt file, store result (in memory or OPFS depending on backend)
encrypt(
data: Uint8Array,
fileName: string,
onProgress?: (done: number, total: number) => void
): Promise<EncryptResult>
// Decrypt from stored encrypted data
decrypt(
key: Uint8Array,
nonce: Uint8Array,
size: number,
onProgress?: (done: number, total: number) => void
): Promise<DecryptResult>
// Read chunk from stored encrypted data (for upload)
readChunk(offset: number, size: number): Promise<Uint8Array>
// Write chunk to storage (for download, before decrypt)
writeChunk(data: Uint8Array, offset: number): Promise<void>
// Clean up temporary storage
cleanup(): Promise<void>
}
export interface EncryptResult {
digest: Uint8Array // SHA-512 of encrypted data
key: Uint8Array // Generated encryption key
nonce: Uint8Array // Generated nonce
chunkSizes: number[] // Chunk sizes for upload
totalSize: number // Total encrypted size
}
export interface DecryptResult {
header: FileHeader // Extracted file header (fileName, etc.)
content: Uint8Array // Decrypted file content
}
```
### 3.3 Backend Implementations
**DirectMemoryBackend** (Node.js):
```typescript
class DirectMemoryBackend implements CryptoBackend {
private encryptedData: Uint8Array | null = null
async encrypt(data: Uint8Array, fileName: string, onProgress?): Promise<EncryptResult> {
const key = randomBytes(32)
const nonce = randomBytes(24)
// Call existing crypto function directly
this.encryptedData = encryptFile(data, fileName, key, nonce, onProgress)
const digest = sha512(this.encryptedData)
const chunkSizes = prepareChunkSizes(this.encryptedData.length)
return { digest, key, nonce, chunkSizes, totalSize: this.encryptedData.length }
}
async decrypt(key, nonce, size, onProgress): Promise<DecryptResult> {
// Call existing crypto function directly
return decryptChunks([this.encryptedData!], key, nonce, size, onProgress)
}
async readChunk(offset: number, size: number): Promise<Uint8Array> {
return this.encryptedData!.slice(offset, offset + size)
}
async writeChunk(data: Uint8Array, offset: number): Promise<void> {
if (!this.encryptedData) this.encryptedData = new Uint8Array(offset + data.length)
this.encryptedData.set(data, offset)
}
async cleanup(): Promise<void> {
this.encryptedData = null
}
}
```
**WorkerMemoryBackend** and **WorkerOPFSBackend** are similar but post messages to a Web Worker instead of calling crypto directly. The Worker then calls the same `encryptFile()`, `decryptChunks()` functions. See §4 for Worker implementation details.
### 3.4 Factory Function
```typescript
// env.ts
export function createCryptoBackend(fileSize: number): CryptoBackend {
const hasWorker = typeof Worker !== 'undefined'
const hasOPFS = typeof navigator?.storage?.getDirectory !== 'undefined'
const isLargeFile = fileSize > 50 * 1024 * 1024
if (hasWorker && hasOPFS && isLargeFile) {
return new WorkerOPFSBackend() // Browser + large file
} else if (hasWorker) {
return new WorkerMemoryBackend() // Browser + small file
} else {
return new DirectMemoryBackend() // Node.js
}
}
```
### 3.5 Usage in agent.ts
```typescript
// agent.ts - upload orchestration (simplified)
export async function uploadFile(
server: XFTPServer,
fileData: Uint8Array,
fileName: string,
onProgress?: ProgressCallback
): Promise<string> {
// Create backend based on environment
const backend = createCryptoBackend(fileData.length)
try {
// Encrypt (runs in Worker in browser, directly in Node)
const enc = await backend.encrypt(fileData, fileName, onProgress)
// Upload chunks (same code regardless of backend)
const client = await connectXFTP(server)
const sentChunks = []
let offset = 0
for (const size of enc.chunkSizes) {
const chunk = await backend.readChunk(offset, size)
const sent = await uploadChunk(client, chunk, enc.digest)
sentChunks.push(sent)
offset += size
}
// Build description and URI
const fd = buildFileDescription(enc, sentChunks)
return encodeFileDescriptionURI(fd)
} finally {
await backend.cleanup()
}
}
```
The key point: `uploadFile()` logic is identical regardless of whether crypto runs in a Worker or directly. The `CryptoBackend` abstraction hides that detail.
### 3.6 Why This Matters for Testing
- **Layer 1 tests** (per-function): Call `encryptFile()`, `decryptChunks()` directly via Node — unchanged
- **Layer 2 tests** (full flow): Call `uploadFile()`, `downloadFile()` in Node — uses `DirectMemoryBackend`, same code path as browser except for Worker
- **Layer 3 tests** (browser): Call `uploadFile()`, `downloadFile()` in Playwright — uses `WorkerMemoryBackend` or `WorkerOPFSBackend`
All three layers exercise the same crypto functions. The only difference is execution context.
## 4. Web Worker Implementation
### 4.1 Why Web Worker
File encryption (XSalsa20-Poly1305) is sequential and CPU-bound:
- 100 MB file ≈ 1-2 seconds of continuous computation
- Running on main thread blocks UI (no progress updates, frozen page)
- Chunking into async microtasks adds complexity and still causes jank
Web Worker runs crypto in parallel thread. Main thread stays responsive.
### 4.2 Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ Main Thread │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │
│ │ UI (upload/ │ │ Progress │ │ Network (fetch) │ │
│ │ download) │ │ display │ │ │ │
│ └──────┬──────┘ └──────▲──────┘ └──────────▲──────────┘ │
│ │ │ │ │
│ │ postMessage │ progress │ encrypted │
│ ▼ │ events │ chunks │
├─────────────────────────────────────────────────────────────┤
│ Web Worker │
│ ┌─────────────────────────────────────────────────────────┐│
│ │ Crypto Pipeline ││
│ │ - encryptFile() with progress callbacks ││
│ │ - decryptChunks() with progress callbacks ││
│ │ - OPFS read/write for temp storage ││
│ └─────────────────────────────────────────────────────────┘│
└─────────────────────────────────────────────────────────────┘
```
### 4.3 Message Protocol
**Main → Worker:**
```typescript
type WorkerRequest =
// Encrypt file, store result in OPFS (large) or memory (small)
| {type: 'encrypt', file: File, fileName: string, useOPFS: boolean}
// Read encrypted chunk from OPFS for upload
| {type: 'readChunk', offset: number, size: number}
// Write downloaded chunk to OPFS for later decryption
| {type: 'writeChunk', data: ArrayBuffer, offset: number}
// Decrypt from OPFS or provided chunks
| {type: 'decrypt', key: Uint8Array, nonce: Uint8Array, size: number, chunks?: ArrayBuffer[]}
// Delete OPFS temp files
| {type: 'cleanup'}
| {type: 'cancel'}
```
**Worker → Main:**
```typescript
type WorkerResponse =
| {type: 'progress', phase: 'encrypt' | 'decrypt', done: number, total: number}
// For OPFS: encData is empty, data lives in OPFS temp file
| {type: 'encrypted', encData: ArrayBuffer | null, digest: Uint8Array, key: Uint8Array, nonce: Uint8Array, chunkSizes: number[]}
| {type: 'chunk', data: ArrayBuffer} // Response to readChunk
| {type: 'chunkWritten'} // Response to writeChunk
| {type: 'decrypted', header: FileHeader, content: ArrayBuffer}
| {type: 'cleaned'} // Response to cleanup
| {type: 'error', message: string}
```
### 4.4 Worker Implementation
```typescript
// crypto.worker.ts
import {encryptFile, encryptFileStreaming, decryptChunks, decryptFromOPFS} from './crypto/file.js'
import {sha512} from './crypto/digest.js'
import {prepareChunkSizes} from './protocol/chunks.js'
let opfsHandle: FileSystemSyncAccessHandle | null = null
self.onmessage = async (e: MessageEvent<WorkerRequest>) => {
const req = e.data
if (req.type === 'encrypt') {
const key = crypto.getRandomValues(new Uint8Array(32))
const nonce = crypto.getRandomValues(new Uint8Array(24))
if (req.useOPFS) {
// Large file: stream through OPFS to avoid memory pressure
const root = await navigator.storage.getDirectory()
const fileHandle = await root.getFileHandle('encrypted-temp', {create: true})
opfsHandle = await fileHandle.createSyncAccessHandle()
// Stream encrypt: read 64KB from File, encrypt, write to OPFS
const digest = await encryptFileStreaming(
req.file,
req.fileName,
key,
nonce,
opfsHandle,
(done, total) => self.postMessage({type: 'progress', phase: 'encrypt', done, total})
)
const encSize = opfsHandle.getSize()
const chunkSizes = prepareChunkSizes(encSize)
self.postMessage({
type: 'encrypted',
encData: null, // Data in OPFS, not memory
digest, key, nonce, chunkSizes
})
} else {
// Small file: in-memory is fine
const source = new Uint8Array(await req.file.arrayBuffer())
const encData = encryptFile(source, req.fileName, key, nonce, (done, total) => {
self.postMessage({type: 'progress', phase: 'encrypt', done, total})
})
const digest = sha512(encData)
const chunkSizes = prepareChunkSizes(encData.length)
self.postMessage({
type: 'encrypted',
encData: encData.buffer,
digest, key, nonce, chunkSizes
}, [encData.buffer])
}
}
if (req.type === 'readChunk') {
// Read chunk from OPFS for upload
const chunk = new Uint8Array(req.size)
opfsHandle!.read(chunk, {at: req.offset})
self.postMessage({type: 'chunk', data: chunk.buffer}, [chunk.buffer])
}
if (req.type === 'writeChunk') {
// Write downloaded chunk to OPFS
if (!opfsHandle) {
const root = await navigator.storage.getDirectory()
const fileHandle = await root.getFileHandle('download-temp', {create: true})
opfsHandle = await fileHandle.createSyncAccessHandle()
}
opfsHandle.write(new Uint8Array(req.data), {at: req.offset})
self.postMessage({type: 'chunkWritten'})
}
if (req.type === 'decrypt') {
let result
if (req.chunks) {
// Small file: chunks provided in memory
const chunks = req.chunks.map(b => new Uint8Array(b))
result = decryptChunks(chunks, req.key, req.nonce, req.size, (done, total) => {
self.postMessage({type: 'progress', phase: 'decrypt', done, total})
})
} else {
// Large file: read from OPFS
result = decryptFromOPFS(opfsHandle!, req.key, req.nonce, req.size, (done, total) => {
self.postMessage({type: 'progress', phase: 'decrypt', done, total})
})
}
self.postMessage({
type: 'decrypted',
header: result.header,
content: result.content.buffer
}, [result.content.buffer])
}
if (req.type === 'cleanup') {
if (opfsHandle) {
opfsHandle.close()
opfsHandle = null
}
const root = await navigator.storage.getDirectory()
try { await root.removeEntry('encrypted-temp') } catch {}
try { await root.removeEntry('download-temp') } catch {}
self.postMessage({type: 'cleaned'})
}
}
```
### 4.5 Main Thread Wrapper
```typescript
// crypto-worker.ts (main thread)
export class CryptoWorker {
private worker: Worker
private pending: Map<string, {resolve: Function, reject: Function}> = new Map()
private onProgress?: (done: number, total: number) => void
constructor() {
this.worker = new Worker(new URL('./crypto.worker.js', import.meta.url), {type: 'module'})
this.worker.onmessage = (e) => this.handleMessage(e.data)
}
async encrypt(file: File, onProgress?: (done: number, total: number) => void): Promise<EncryptedFileInfo> {
const useOPFS = file.size > 50 * 1024 * 1024 // 50 MB threshold
return new Promise((resolve, reject) => {
this.pending.set('encrypt', {resolve, reject})
this.onProgress = onProgress
this.worker.postMessage({type: 'encrypt', file, fileName: file.name, useOPFS})
})
}
async decrypt(
chunks: Uint8Array[],
key: Uint8Array,
nonce: Uint8Array,
size: number,
onProgress?: (done: number, total: number) => void
): Promise<DownloadResult> {
return new Promise((resolve, reject) => {
this.pending.set('decrypt', {resolve, reject})
this.onProgress = onProgress
this.worker.postMessage({
type: 'decrypt',
chunks: chunks.map(c => c.buffer),
key, nonce, size
}, chunks.map(c => c.buffer))
})
}
private handleMessage(msg: WorkerResponse) {
if (msg.type === 'progress') {
this.onProgress?.(msg.done, msg.total)
} else if (msg.type === 'encrypted') {
this.pending.get('encrypt')?.resolve({
encData: msg.encData ? new Uint8Array(msg.encData) : null, // null when using OPFS
digest: msg.digest,
key: msg.key,
nonce: msg.nonce,
chunkSizes: msg.chunkSizes
})
} else if (msg.type === 'decrypted') {
this.pending.get('decrypt')?.resolve({
header: msg.header,
content: new Uint8Array(msg.content)
})
} else if (msg.type === 'error') {
// Reject all pending
for (const p of this.pending.values()) p.reject(new Error(msg.message))
}
}
}
```
## 5. OPFS Implementation
### 5.1 Purpose
For files approaching 100 MB, holding encrypted data in memory while uploading creates memory pressure. OPFS provides temporary file storage:
- Write encrypted data to OPFS as it's generated
- Read chunks from OPFS for upload
- Delete after upload completes
### 5.2 When to Use
- Files > 50 MB: Use OPFS
- Files ≤ 50 MB: In-memory (simpler, no OPFS overhead)
Threshold is configurable.
### 5.3 OPFS API
```typescript
// In Web Worker (synchronous API for performance)
const root = await navigator.storage.getDirectory()
const fileHandle = await root.getFileHandle('encrypted-temp', {create: true})
const accessHandle = await fileHandle.createSyncAccessHandle()
// Write encrypted chunks as they're generated
accessHandle.write(encryptedChunk, {at: offset})
// Read chunk for upload
const chunk = new Uint8Array(chunkSize)
accessHandle.read(chunk, {at: chunkOffset})
// Cleanup
accessHandle.close()
await root.removeEntry('encrypted-temp')
```
### 5.4 Upload Flow with OPFS
```
1. Main: user drops file
2. Main → Worker: {type: 'encrypt', file}
3. Worker:
- Create OPFS temp file
- Encrypt 64KB at a time, write to OPFS
- Post progress every 64KB
- Compute digest
- Return {digest, key, nonce, chunkSizes} (data stays in OPFS)
4. Main: for each chunk:
- Main → Worker: {type: 'readChunk', offset, size}
- Worker: read from OPFS, return chunk
- Main: upload chunk via fetch()
5. Main → Worker: {type: 'cleanup'}
6. Worker: delete OPFS temp file
```
### 5.5 Download Flow with OPFS
```
1. Main: parse URL, get FileDescription
2. Main: for each chunk:
- Download via fetch()
- Main → Worker: {type: 'writeChunk', data, offset}
- Worker: write to OPFS temp file
3. Main → Worker: {type: 'decrypt', key, nonce, size}
4. Worker:
- Read from OPFS
- Decrypt, verify auth tag
- Return {header, content}
5. Main: trigger browser download
6. Main → Worker: {type: 'cleanup'}
```
## 6. Implementation Plan
### 6.1 Phase A: fetch() Transport
**Goal:** Replace `node:http2` with `fetch()` in `client.ts`. All existing Node.js tests pass.
1. Rewrite `connectXFTP()` to use fetch() for handshake
2. Rewrite `sendXFTPCommand()` to use fetch()
3. Update `createXFTPChunk`, `uploadXFTPChunk`, `downloadXFTPChunk`, etc.
4. Remove `node:http2` import
5. Run existing Haskell integration tests — must pass
**Files:** `client.ts`
### 6.2 Phase B: Environment Abstraction + Web Worker
**Goal:** Add `CryptoBackend` abstraction (§3) so the same code works in Node (direct) and browser (Worker).
1. Create `env.ts` with `CryptoBackend` interface and `createCryptoBackend()` factory (as specified in §3)
2. Implement `DirectMemoryBackend` for Node.js
3. Create `crypto.worker.ts` that imports and calls existing crypto functions
4. Implement `WorkerMemoryBackend` for browser
5. Update `agent.ts` to use `createCryptoBackend()` instead of direct crypto calls
6. Existing tests pass (now using `DirectMemoryBackend`)
**Files:** `env.ts`, `crypto.worker.ts`, `agent.ts`
### 6.3 Phase C: OPFS Backend
**Goal:** Large files (>50 MB) use OPFS for temp storage in browser.
1. Implement `WorkerOPFSBackend` — uses OPFS sync API in worker
2. Add OPFS helpers in worker: read/write to temp file
3. Factory function now returns `WorkerOPFSBackend` for large files
4. Same `agent.ts` code works — only backend implementation differs
**Files:** `env.ts`, `crypto.worker.ts`
### 6.4 Phase D: Browser Testing
**Goal:** Verify everything works in real browsers.
1. Create minimal test HTML page
2. Test upload flow in Chrome, Firefox, Safari
3. Test download flow
4. Test progress reporting
5. Test cancellation
6. Test error handling (network failure, invalid file)
## 7. Testing Strategy
### 7.1 Test Layers
The `CryptoBackend` abstraction (§3) enables testing at multiple levels without code duplication:
```
┌─────────────────────────────────────────────────────────────────┐
│ Layer 3: Browser Integration (Playwright) │
│ - Web Worker message passing │
│ - OPFS read/write │
│ - Progress UI updates │
│ - Real browser fetch() with CORS │
├─────────────────────────────────────────────────────────────────┤
│ Layer 2: Full Flow (Haskell-driven, Node.js) │
│ - fetch() transport against real xftp-server │
│ - Upload: encrypt → chunk → upload → build description │
│ - Download: parse → download → verify → decrypt │
│ - Cross-language: TS upload ↔ Haskell download (and vice versa) │
├─────────────────────────────────────────────────────────────────┤
│ Layer 1: Per-Function (Haskell-driven, Node.js) │
│ - 172 existing tests │
│ - Byte-identical output vs Haskell functions │
└─────────────────────────────────────────────────────────────────┘
```
### 7.2 Layer 1: Per-Function Tests (Existing)
Existing Haskell-driven tests in `XFTPWebTests.hs`. Each test calls a TypeScript function via Node and compares output with Haskell.
```bash
cabal test --ghc-options -O0 --test-option='--match=/XFTP Web Client/'
```
All 172 tests must pass. No changes needed for browser transport work.
### 7.3 Layer 2: Full Flow Tests (Node.js + fetch)
Haskell-driven integration tests using Node.js native fetch(). These test the complete upload/download flow without Worker/OPFS.
```haskell
-- XFTPWebTests.hs (extends existing test file)
it "fetch transport: upload and download round-trip" $ do
withXFTPServer testXFTPServerConfigSNI $ \server -> do
-- TypeScript uploads via fetch(), returns URI
uri <- jsOut $ callTS "src/agent" "uploadFileTest" serverAddrHex <> testFileHex
-- TypeScript downloads via fetch()
content <- jsOut $ callTS "src/agent" "downloadFileTest" uriHex
content `shouldBe` testFileContent
it "fetch transport: TS upload, Haskell download" $ do
withXFTPServer testXFTPServerConfigSNI $ \server -> do
uri <- jsOut $ callTS "src/agent" "uploadFileTest" serverAddrHex <> testFileHex
-- Haskell agent downloads using existing xftp CLI pattern
outPath <- withAgent 1 agentCfg initAgentServers testDB $ \a -> do
rfId <- xftpReceiveFile' a 1 uri Nothing
waitRfDone a
content <- B.readFile outPath
content `shouldBe` testFileContent
```
**What this tests:**
- fetch() handshake (challenge-response, TLS session binding)
- fetch() command execution (FNEW, FPUT, FGET, FACK)
- Streaming request/response bodies
- Full encrypt → upload → download → decrypt flow
**What this doesn't test:**
- Web Worker message passing
- OPFS storage
- Browser-specific fetch() behavior (CORS preflight, etc.)
### 7.4 Layer 3: Browser Integration Tests (Playwright)
Playwright tests run in real browsers, testing browser-specific functionality.
**Test infrastructure:**
```
xftp-web/
├── test/
│ ├── browser.test.ts # Playwright test file
│ └── test-server.ts # Spawns xftp-server for tests
└── test-page/
├── index.html # Minimal test UI
└── test-harness.ts # Exposes test functions to window
```
**Running browser tests:**
```bash
cd xftp-web
npm run test:browser # Spawns xftp-server, runs Playwright
```
**Test cases:**
```typescript
// test/browser.test.ts
import { test, expect } from '@playwright/test'
import { spawn } from 'child_process'
let serverProcess: ChildProcess
test.beforeAll(async () => {
// Spawn xftp-server with SNI cert for browser TLS
serverProcess = spawn('xftp-server', ['start', '-c', 'test-config.ini'])
await waitForServer()
})
test.afterAll(async () => {
serverProcess.kill()
})
test('small file upload/download (in-memory)', async ({ page }) => {
await page.goto('/test-page/')
const result = await page.evaluate(async () => {
const data = new Uint8Array(1024 * 1024) // 1 MB
crypto.getRandomValues(data)
const file = new File([data], 'small.bin')
const uri = await window.xftp.uploadFile(file)
const downloaded = await window.xftp.downloadFile(uri)
return {
uploadedSize: data.length,
downloadedSize: downloaded.length,
match: arraysEqual(data, downloaded),
usedOPFS: window.xftp.lastUploadUsedOPFS
}
})
expect(result.match).toBe(true)
expect(result.usedOPFS).toBe(false) // Small file, no OPFS
})
test('large file upload/download (OPFS)', async ({ page }) => {
await page.goto('/test-page/')
const result = await page.evaluate(async () => {
const data = new Uint8Array(60 * 1024 * 1024) // 60 MB
crypto.getRandomValues(data)
const file = new File([data], 'large.bin')
const uri = await window.xftp.uploadFile(file)
const downloaded = await window.xftp.downloadFile(uri)
return {
match: arraysEqual(data, downloaded),
usedOPFS: window.xftp.lastUploadUsedOPFS
}
})
expect(result.match).toBe(true)
expect(result.usedOPFS).toBe(true) // Large file, used OPFS
})
test('progress events fire during upload', async ({ page }) => {
await page.goto('/test-page/')
const progressEvents = await page.evaluate(async () => {
const events: number[] = []
const data = new Uint8Array(10 * 1024 * 1024) // 10 MB
const file = new File([data], 'progress.bin')
await window.xftp.uploadFile(file, (done, total) => {
events.push(done / total)
})
return events
})
expect(progressEvents.length).toBeGreaterThan(1)
expect(progressEvents[progressEvents.length - 1]).toBe(1) // 100% at end
})
test('Web Worker keeps UI responsive', async ({ page }) => {
await page.goto('/test-page/')
// Start upload and measure main thread responsiveness
const result = await page.evaluate(async () => {
const data = new Uint8Array(50 * 1024 * 1024) // 50 MB
const file = new File([data], 'responsive.bin')
let frameCount = 0
let uploadDone = false
// Count animation frames during upload
function countFrames() {
frameCount++
if (!uploadDone) requestAnimationFrame(countFrames)
}
requestAnimationFrame(countFrames)
const start = performance.now()
await window.xftp.uploadFile(file)
uploadDone = true
const elapsed = performance.now() - start
// If main thread was blocked, frameCount would be very low
const expectedFrames = (elapsed / 1000) * 30 // ~30 fps minimum
return { frameCount, expectedFrames, elapsed }
})
// Should maintain reasonable frame rate (Worker offloaded crypto)
expect(result.frameCount).toBeGreaterThan(result.expectedFrames * 0.5)
})
```
### 7.5 Cross-Browser Matrix
| Browser | fetch streaming | Web Worker | OPFS sync | Status |
|---------|----------------|------------|-----------|--------|
| Chrome 105+ | ✓ | ✓ | ✓ | Primary target |
| Firefox 111+ | ✓ | ✓ | ✓ | Supported |
| Safari 16.4+ | ✓ | ✓ | ✓ | Supported |
| Edge 105+ | ✓ | ✓ | ✓ | Supported (Chromium) |
Playwright tests run against Chrome by default. CI can run against all browsers.
### 7.6 Test Execution Summary
| Phase | Test Layer | Command | What's Verified |
|-------|-----------|---------|-----------------|
| A | Layer 1 + 2 | `cabal test --test-option='--match=/XFTP Web Client/'` | fetch() transport, full flow |
| B | Layer 3 | `npm run test:browser` | Worker message passing, progress |
| C | Layer 3 | `npm run test:browser` | OPFS storage for large files |
| D | Layer 3 | `npm run test:browser -- --project=firefox,webkit` | Cross-browser |
@@ -0,0 +1,772 @@
# Send File Web Page — Implementation Plan
## TOC
1. Executive Summary
2. Architecture
3. CryptoBackend & Web Worker
4. Server Configuration
5. Page Structure & UI
6. Upload Flow
7. Download Flow
8. Build & Dev Setup
9. agent.ts Changes
10. Testing
11. Files
12. Implementation Order
## 1. Executive Summary
Build a static web page for browser-based XFTP file transfer (Phase 5 of master RFC). The page supports upload (drag-drop → encrypt → upload → shareable link) and download (open link → download → decrypt → save). Crypto runs in a Web Worker; large files use OPFS temp storage.
Two build variants:
- **Local**: single test server at `localhost:7000` (development/testing)
- **Production**: 12 preset XFTP routers (6 SimpleX + 6 Flux)
Uses Vite for bundling (already a dependency via vitest). No CSS framework — plain CSS per RFC spec.
## 2. Architecture
```
xftp-web/
├── src/ # Library (existing, targeted changes)
│ ├── agent.ts # Modified: uploadFile readChunk, downloadFileRaw
│ ├── client.ts # Modified: downloadXFTPChunkRaw
│ ├── crypto/ # Unchanged
│ ├── download.ts # Unchanged
│ └── protocol/
│ └── description.ts # Fix: SHA-256 → SHA-512 comment on digest field
├── web/ # Web page (new)
│ ├── index.html # Entry point (CSP meta tag)
│ ├── main.ts # Router + sodium.ready init
│ ├── upload.ts # Upload UI + orchestration
│ ├── download.ts # Download UI + orchestration
│ ├── progress.ts # Circular progress canvas component
│ ├── servers.ts # Server list (build-time configured, imports servers.json)
│ ├── servers.json # Preset server addresses (shared with vite.config.ts)
│ ├── crypto-backend.ts # CryptoBackend interface + WorkerBackend
│ ├── crypto.worker.ts # Web Worker: encrypt/decrypt/OPFS
│ └── style.css # Minimal styling
├── vite.config.ts # Page build config (new)
├── tsconfig.web.json # IDE/CI type-check for web/ (new)
├── tsconfig.worker.json # IDE/CI type-check for worker (new)
├── playwright.config.ts # Page E2E test config (new)
├── vitest.config.ts # Test config (existing)
├── .gitignore # Existing (add dist-web/)
└── test/ # Tests (existing + new page test)
```
Data flow:
```
┌───────────────────────────────────────────┐
│ Main Thread │
│ │
│ Upload: upload.ts ──► agent.ts ──► fetch()│
│ Download: download.ts ──► agent.ts ──► fetch()
│ │ │
│ postMessage HTTP/2 │
│ ▼ ▼
│ ┌─────────────────┐ ┌──────────┐│
│ │ Web Worker │ │ XFTP ││
│ │ crypto.worker.ts │ │ Server ││
│ │ ┌─────────────┐ │ └──────────┘│
│ │ │ OPFS temp │ │ │
│ │ └─────────────┘ │ │
│ └─────────────────┘ │
└───────────────────────────────────────────┘
```
Both upload and download use `agent.ts` for orchestration (connection pooling, parallel chunk transfers, redirect handling). Upload uses a `readChunk` callback for Worker data access. Download uses an `onRawChunk` callback to route raw encrypted chunks to the Worker for decryption (see §7.2). ACK is the caller's responsibility — `downloadFileRaw` returns the resolved `FileDescription` without ACKing, so the caller can verify integrity before acknowledging.
## 3. CryptoBackend & Web Worker
### 3.1 Interface
```typescript
// crypto-backend.ts
export interface CryptoBackend {
// Upload: encrypt file, store encrypted data in OPFS
encrypt(data: Uint8Array, fileName: string,
onProgress?: (done: number, total: number) => void
): Promise<EncryptResult>
// Upload: read encrypted chunk from OPFS (called by agent.ts via readChunk callback)
readChunk(offset: number, size: number): Promise<Uint8Array>
// Download: transit-decrypt raw chunk and store in OPFS
decryptAndStoreChunk(
dhSecret: Uint8Array, nonce: Uint8Array,
body: Uint8Array, digest: Uint8Array, chunkNo: number
): Promise<void>
// Download: verify digest + file-level decrypt all stored chunks
// Only needs size/digest/key/nonce — not the full FileDescription (avoids sending private keys to Worker)
verifyAndDecrypt(params: {size: number, digest: Uint8Array, key: Uint8Array, nonce: Uint8Array}
): Promise<{header: FileHeader, content: Uint8Array}>
cleanup(): Promise<void>
}
// Structurally identical to EncryptedFileMetadata from agent.ts (§9.1).
// Kept separate to avoid crypto-backend.ts importing from agent.ts
// (which would pull in node:http2 via client.ts, breaking Worker bundling).
// TypeScript structural typing makes them assignment-compatible.
export interface EncryptResult {
digest: Uint8Array
key: Uint8Array
nonce: Uint8Array
chunkSizes: number[]
}
```
### 3.2 Factory
```typescript
export function createCryptoBackend(): CryptoBackend {
if (typeof Worker === 'undefined') {
throw new Error('Web Workers required — update your browser')
}
return new WorkerBackend()
}
```
The Worker always uses OPFS for temp storage (single code path — no memory/disk branching). OPFS I/O overhead is negligible relative to crypto and network time. Each Worker session creates a unique directory in OPFS root named `session-<Date.now()>-<crypto.randomUUID()>`, containing `upload.bin` and `download.bin` as needed. `cleanup()` deletes the entire session directory. On Worker startup (before processing messages), sweep OPFS root and delete any `session-*` directories whose embedded timestamp (parsed from the name) is older than 1 hour — this handles stale files from crashed tabs. The OPFS API does not expose directory timestamps, so the name-encoded timestamp is the only reliable mechanism. This prevents cross-tab collisions and unbounded OPFS growth.
### 3.3 Worker message protocol
Every request carries a numeric `id`. Responses carry the same `id`. WorkerBackend maintains a `Map<number, {resolve, reject}>` to match responses to pending promises.
Main → Worker (fields marked `†` are Transferable — arrive as `ArrayBuffer` in Worker, must be wrapped with `new Uint8Array(...)` before use):
- `{id: number, type: 'encrypt', data†: ArrayBuffer, fileName: string}` — encrypt file, store in OPFS
- `{id: number, type: 'readChunk', offset: number, size: number}` — read encrypted chunk from OPFS
- `{id: number, type: 'decryptAndStoreChunk', dhSecret: Uint8Array, nonce: Uint8Array, body†: ArrayBuffer, chunkDigest: Uint8Array, chunkNo: number}` — transit-decrypt + store in OPFS. `chunkDigest` is the per-chunk SHA-256 digest (verified by `decryptReceivedChunk`). Distinct from the file-level SHA-512 digest in `verifyAndDecrypt`.
- `{id: number, type: 'verifyAndDecrypt', size: number, digest: Uint8Array, key: Uint8Array, nonce: Uint8Array}` — verify digest + file-level decrypt all chunks. Only the four fields needed for verification/decryption are sent — not the full `FileDescription`, which contains private replica keys that the Worker doesn't need.
- `{id: number, type: 'cleanup'}` — delete OPFS temp files
Worker → Main (fields marked `†` are Transferable):
- `{id: number, type: 'progress', done: number, total: number}` — encryption/decryption progress (fire-and-forget, no promise)
- `{id: number, type: 'encrypted', digest: Uint8Array, key: Uint8Array, nonce: Uint8Array, chunkSizes: number[]}` — all fields structured-cloned (not transferred)
- `{id: number, type: 'chunk', data†: ArrayBuffer}` — readChunk response
- `{id: number, type: 'stored'}` — decryptAndStore acknowledgment
- `{id: number, type: 'decrypted', header: FileHeader, content†: ArrayBuffer}` — verifyAndDecrypt response
- `{id: number, type: 'cleaned'}`
- `{id: number, type: 'error', message: string}` — rejects the pending promise for this `id`
All messages carrying large `ArrayBuffer` payloads use `postMessage(msg, [transferables])` to transfer ownership instead of structured-clone copying. Only `ArrayBuffer` can be transferred — `Uint8Array`, `number[]`, and other types are always structured-cloned. This applies to: `encrypt` request (`data`), `readChunk` response (`data`), `decryptAndStoreChunk` request (`body`), and `verifyAndDecrypt` response (`content`). The `WorkerBackend` implementation must ensure the transferred `ArrayBuffer` covers the full `Uint8Array` — if `byteOffset !== 0` or `byteLength !== buffer.byteLength`, slice first: `data.buffer.slice(data.byteOffset, data.byteOffset + data.byteLength)`. This is required for `decryptAndStore` request bodies: `sendXFTPCommand` returns `body = fullResp.subarray(XFTP_BLOCK_SIZE)`, which has `byteOffset = XFTP_BLOCK_SIZE`. Other payloads are full-buffer views (§6 step 3 creates `new Uint8Array(await file.arrayBuffer())`; Worker responses allocate fresh buffers) but `WorkerBackend` should guard unconditionally.
### 3.4 Worker internals
**Imports:** The Worker imports directly from `libsodium-wrappers-sumo` (for `await sodium.ready`), `src/crypto/file.js` (`encryptFile`, `encodeFileHeader`, `decryptChunks`), `src/crypto/digest.js` (`sha512`), `src/protocol/chunks.js` (`prepareChunkSizes`, `fileSizeLen`, `authTagSize`), `src/protocol/encoding.js` (`concatBytes`), and `src/download.js` (`decryptReceivedChunk`). `download.js` directly imports `src/protocol/client.js` (for `decryptTransportChunk`). These transitively pull in `src/crypto/secretbox.js`, `src/crypto/keys.js`, and `src/crypto/padding.js`. None of these import `src/agent.ts` or `src/client.ts` — those pull in `node:http2` via dynamic import which would break Worker bundling. Vite tree-shakes the transitive deps automatically. Note: `download.js``protocol/client.js``crypto/keys.js` transitively pulls in `@noble/curves` (~50-80KB). This is unavoidable since `decryptTransportChunk` needs `dh` from `keys.js`. If Worker bundle size becomes a concern, `decryptReceivedChunk` could be refactored out of `download.js` into a separate module that doesn't import `protocol/client.js`.
**ArrayBuffer → Uint8Array conversion:** All Transferable fields arrive in the Worker as `ArrayBuffer`. The Worker's message handler must wrap them before passing to library functions: `new Uint8Array(msg.data)` for encrypt, `new Uint8Array(msg.body)` for decryptAndStore. Non-transferred fields (`dhSecret`, `nonce`, `digest`, `chunkSizes`) arrive as their original types (`Uint8Array` / `number[]`) via structured clone.
The Worker's encrypt handler calls the same functions as `encryptFileForUpload` in agent.ts (key/nonce generation → `encryptFile``sha512``prepareChunkSizes`). This is not reimplementation — it's calling the same library functions from a different entry point.
**Libsodium init:** Both the Worker and the main thread must `await sodium.ready` before calling any crypto functions that use libsodium. The Worker does this once on startup before processing messages. The main thread needs it before `connectXFTP` (which uses libsodium via `verifyIdentityProof`) and before `downloadXFTPChunkRaw` (which uses libsodium via `generateX25519KeyPair` + `dh`). In practice, `main.ts` calls `await sodium.ready` at page load, before any XFTP calls.
Encrypt (mirrors `encryptFileForUpload` in agent.ts):
1. Generate key (32B) + nonce (24B) via `crypto.getRandomValues`
2. `fileHdr = encodeFileHeader({fileName, fileExtra: null})`
3. `fileSize = BigInt(fileHdr.length + source.length)`
4. `payloadSize = Number(fileSize) + fileSizeLen + authTagSize`
5. `chunkSizes = prepareChunkSizes(payloadSize)`
6. `encSize = BigInt(chunkSizes.reduce((a, b) => a + b, 0))`
7. `encData = encryptFile(source, fileHdr, key, nonce, fileSize, encSize)`
8. `digest = sha512(encData)` — note: the `digest` field comment in `FileDescription` in `description.ts` says "SHA-256" but the actual hash is SHA-512 everywhere (`sha512` in agent.ts and download.ts). Fix the comment during implementation.
9. Open OPFS upload file via `createSyncAccessHandle`, write `encData`, flush, close handle. Null out `encData` reference.
10. Reopen the same OPFS file with `createSyncAccessHandle` as a persistent read handle (stored on the Worker module scope). This handle is used by all subsequent `readChunk` calls and closed on `cleanup`.
11. Post back `{digest, key, nonce, chunkSizes}` (no encData transfer — data stays in OPFS)
readChunk:
- Use the persistent read handle: `handle.read(buf, {at: offset})` → return slice as transferable ArrayBuffer. OPFS allows only one `FileSystemSyncAccessHandle` per file; the persistent handle avoids per-call open/close overhead.
decryptAndStoreChunk (removes transport encryption only — stored data is still file-level encrypted):
1. `decryptReceivedChunk(dhSecret, nonce, new Uint8Array(body), chunkDigest)` → transit-decrypted chunk data (still file-level encrypted — only the transport layer is removed). Argument order matches signature `(dhSecret, cbNonce, encData, expectedDigest)` from download.ts. `body` arrives as `ArrayBuffer` via Transferable and must be wrapped; `dhSecret`, `nonce`, `chunkDigest` arrive as `Uint8Array` via structured clone.
2. On first call, open the OPFS download temp file via `createSyncAccessHandle` and store as a persistent write handle. Record `{chunkNo, size: decrypted.length}` in an in-memory `chunkMeta: Map<number, {offset: number, size: number}>` — offset is the running sum of sizes for chunks stored so far (chunks may arrive out of order with `concurrency > 1`, so offset is assigned as `currentFileOffset`, then `currentFileOffset += size`)
3. Write decrypted chunk to the persistent handle at the recorded offset
verifyAndDecrypt (mirrors size/digest checks in agent.ts `downloadFile`):
1. Close the persistent download write handle (flush first), then reopen as a read handle. Read each chunk from OPFS into a `Uint8Array[]` array, ordered by `chunkNo`: for each entry in `chunkMeta` sorted by `chunkNo`, `handle.read(buf, {at: offset})` with the recorded offset and size
2. Concatenate for verification: `combined = concatBytes(...chunks)`
3. Verify total size: `combined.length === params.size`
4. Verify SHA-512 digest: `sha512(combined)` matches `params.digest`
5. Decrypt: `decryptChunks(BigInt(params.size), chunks, params.key, params.nonce)``params.size` is the encrypted file size (`fd.size` = `sum(chunkSizes)` = `decryptChunks`' first param `encSize`). Called directly instead of via `processDownloadedFile` (which expects a full `FileDescription`). Pass the original `chunks` array (not `combined`), as `decryptChunks` handles concatenation internally.
6. Delete OPFS download temp file
7. Return `{header, content}` via transferable ArrayBuffer
### 3.5 Browser requirements
The page requires a modern browser with Web Worker and OPFS support:
- Chrome 102+, Firefox 114+, Safari 15.2+ (Workers + OPFS + ES module Workers — Firefox added module Worker support in 114)
- If Worker or OPFS is unavailable, the page shows an error message rather than falling back silently.
No `DirectBackend` is needed — the page is browser-only, and tests run in vitest browser mode (real Chromium). The existing library tests (`test/browser.test.ts`) test the crypto/upload/download pipeline directly without Workers.
## 4. Server Configuration
### 4.1 Server lists
`web/servers.json` — single source of truth for preset server addresses (imported by both `servers.ts` and `vite.config.ts`):
```json
{
"simplex": [
"xftp://da1aH3nOT-9G8lV7bWamhxpDYdJ1xmW7j3JpGaDR5Ug=@xftp1.simplex.im",
"xftp://5vog2Imy1ExJB_7zDZrkV1KDWi96jYFyy9CL6fndBVw=@xftp2.simplex.im",
"xftp://PYa32DdYNFWi0uZZOprWQoQpIk5qyjRJ3EF7bVpbsn8=@xftp3.simplex.im",
"xftp://k_GgQl40UZVV0Y4BX9ZTyMVqX5ZewcLW0waQIl7AYDE=@xftp4.simplex.im",
"xftp://-bIo6o8wuVc4wpZkZD3tH-rCeYaeER_0lz1ffQcSJDs=@xftp5.simplex.im",
"xftp://6nSvtY9pJn6PXWTAIMNl95E1Kk1vD7FM2TeOA64CFLg=@xftp6.simplex.im"
],
"flux": [
"xftp://92Sctlc09vHl_nAqF2min88zKyjdYJ9mgxRCJns5K2U=@xftp1.simplexonflux.com",
"xftp://YBXy4f5zU1CEhnbbCzVWTNVNsaETcAGmYqGNxHntiE8=@xftp2.simplexonflux.com",
"xftp://ARQO74ZSvv2OrulRF3CdgwPz_AMy27r0phtLSq5b664=@xftp3.simplexonflux.com",
"xftp://ub2jmAa9U0uQCy90O-fSUNaYCj6sdhl49Jh3VpNXP58=@xftp4.simplexonflux.com",
"xftp://Rh19D5e4Eez37DEE9hAlXDB3gZa1BdFYJTPgJWPO9OI=@xftp5.simplexonflux.com",
"xftp://0AznwoyfX8Od9T_acp1QeeKtxUi676IBIiQjXVwbdyU=@xftp6.simplexonflux.com"
]
}
```
`web/servers.ts`:
```typescript
import {parseXFTPServer, type XFTPServer} from '../src/protocol/address.js'
import presets from './servers.json'
declare const __XFTP_SERVERS__: string[]
const serverAddresses: string[] = typeof __XFTP_SERVERS__ !== 'undefined'
? __XFTP_SERVERS__
: [...presets.simplex, ...presets.flux]
export function getServers(): XFTPServer[] {
return serverAddresses.map(parseXFTPServer)
}
export function pickRandomServer(servers: XFTPServer[]): XFTPServer {
return servers[Math.floor(Math.random() * servers.length)]
}
```
### 4.2 Build-time injection
`vite.config.ts` defines `__XFTP_SERVERS__`:
- `mode === 'local'`: `["xftp://<test-fingerprint>@localhost:7000"]`
- `mode === 'production'`: not defined → falls through to hardcoded list
### 4.3 Assumption
Production XFTP routers must have `[WEB]` section configured with a CA-signed certificate for browser TLS. Without this, browsers will reject the self-signed XFTP identity cert. The local test router uses `tests/fixtures/` certs which Chromium accepts via `ignoreHTTPSErrors`.
## 5. Page Structure & UI
### 5.1 Routing
`main.ts` checks `window.location.hash` once on page load:
- Hash present → download mode
- Hash absent → upload mode
No `hashchange` listener — the shareable link opens in a new tab. Simple page-load routing.
### 5.2 Upload UI states
1. **Landing**: Drag-drop zone centered, file picker button, size limit note
2. **Uploading**: Circular progress (canvas), percentage, cancel button
3. **Complete**: Shareable link (input + copy button), "Install SimpleX" CTA
4. **Error**: Error message + retry button. On server-unreachable, auto-retry with exponential backoff (1s, 2s, 4s, up to 3 attempts) before showing the error state.
### 5.3 Download UI states
1. **Ready**: Approximate file size displayed (encrypted size from `fd.size` or `fd.redirect.size` — see §7 step 2; file name is unavailable — it's inside the encrypted content), download button
2. **Downloading**: Circular progress, percentage
3. **Complete**: Browser save dialog triggered automatically
4. **Error**: Error message (expired, corrupted, unreachable)
### 5.4 Security summary (RFC §7.4)
Both upload-complete and download-ready states display a brief non-technical security summary:
- Files are encrypted in the browser before upload — the server never sees file contents.
- The link contains the decryption key in the hash fragment, which the browser never sends to any server.
- For maximum security, use the SimpleX app.
### 5.5 File expiry
Display on upload-complete state: "Files are typically available for 48 hours." This is an approximation — actual expiry depends on each XFTP router's `[STORE_LOG]` retention configuration. The 48-hour figure matches the current preset router defaults.
### 5.6 Styling
Plain CSS, no framework. White background, centered content, responsive. Circular progress via `<canvas>` (arc drawing, percentage text in center).
File size limit: 100MB. Displayed on upload page.
### 5.7 CSP
`index.html` includes a `<meta>` Content-Security-Policy tag with a build-time placeholder:
```html
<meta http-equiv="Content-Security-Policy"
content="default-src 'self'; worker-src 'self' blob:; style-src 'self' 'unsafe-inline'; connect-src __CSP_CONNECT_SRC__;">
```
Vite's `transformIndexHtml` hook (in `vite.config.ts`) replaces `__CSP_CONNECT_SRC__` at build time with origins derived from the server list:
- Local mode: `https://localhost:7000`
- Production: `https://xftp1.simplex.im:443 https://xftp2.simplex.im:443 ...` (all 12 servers)
## 6. Upload Flow
`web/upload.ts`:
1. User drops/picks file → `File` object
2. Validate `file.size <= 100 * 1024 * 1024` — show error if exceeded
3. Read file: `new Uint8Array(await file.arrayBuffer())` — note: after `backend.encrypt()` transfers the buffer to the Worker, `fileData` is detached (zero-length). Peak memory is ~2× file size (main thread holds original until transfer, Worker holds encrypted copy before OPFS write). Acceptable for the 100MB limit; do not raise the limit without considering memory implications.
4. Create `CryptoBackend` via factory
5. Create `XFTPClientAgent`
6. `backend.encrypt(fileData, file.name, onProgress)``EncryptResult`
- Encryption progress shown on canvas (Worker posts progress messages)
7. Pick one random server from configured list (V1: all chunks to same server)
8. Call `uploadFile(agent, server, metadata, {onProgress, readChunk: (off, sz) => backend.readChunk(off, sz)})`:
- `metadata` = `{digest, key, nonce, chunkSizes}` from EncryptResult
- Network progress shown on canvas
- Returns `{rcvDescription, sndDescription, uri}`
9. Construct full URL: `window.location.origin + window.location.pathname + '#' + uri`
10. Display link, copy button
11. Cleanup: `backend.cleanup()`, `closeXFTPAgent(agent)`
**Cancel:** User can abort via cancel button. Sets an `AbortController` signal that:
- Sends `{type: 'cleanup'}` to Worker
- Closes the XFTPClientAgent (drops HTTP/2 connections)
- Resets UI to landing state
## 7. Download Flow
`web/download.ts`:
1. Parse `window.location.hash.slice(1)``decodeDescriptionURI(fragment)``FileDescription`
2. Display file size (`fd.size` bytes, formatted human-readable). Note: `fd.size` is the encrypted size (slightly larger than plaintext due to padding + auth tag). The plaintext size is not available until decryption — display it as an approximate file size. If `fd.redirect !== null`, size comes from `fd.redirect.size` (which is the inner encrypted size).
3. User clicks "Download"
4. Create `CryptoBackend` and `XFTPClientAgent`
5. Call `downloadFileRaw(agent, fd, onRawChunk, {onProgress, concurrency: 3})`:
- `onRawChunk` forwards each raw chunk to the Worker: `backend.decryptAndStoreChunk(raw.dhSecret, raw.nonce, raw.body, raw.digest, raw.chunkNo)`
- `downloadFileRaw` handles redirect resolution internally (see §7.1), parallel downloads, and connection pooling
- Returns the resolved `FileDescription` (inner fd for redirect case, original fd otherwise)
6. `backend.verifyAndDecrypt({size: resolvedFd.size, digest: resolvedFd.digest, key: resolvedFd.key, nonce: resolvedFd.nonce})``{header, content}`
- Verifies size + SHA-512 digest + file-level decryption inside Worker. Only the four needed fields are sent — private replica keys stay on the main thread.
7. ACK: `ackFileChunks(agent, resolvedFd)` — best-effort, after verification succeeds
8. Sanitize `header.fileName` before use: strip path separators (`/`, `\`), replace null/control characters (U+0000-U+001F, U+007F), strip Unicode bidi override characters (U+202A-U+202E, U+2066-U+2069 — prevents `doc.pdf.exe` appearing as `doc.exe.pdf`), limit length to 255 chars. The filename is user-controlled (set by the uploader) and arrives via decrypted content. Then trigger browser save: `new Blob([content])``<a download="${sanitizedName}">` click
9. Cleanup: `backend.cleanup()`, `closeXFTPAgent(agent)`
### 7.1 Redirect handling
Handled inside `downloadFileRaw` in agent.ts — the web page doesn't see it. When `fd.redirect !== null`:
1. Download redirect chunks via `downloadXFTPChunkRaw` (parallel, same as regular chunks)
2. Transit-decrypt + verify + file-level decrypt on main thread (redirect data is always small — a few KB of YAML, so main thread decryption is fine)
3. Parse YAML → inner `FileDescription`, validate against `fd.redirect.{size, digest}`
4. ACK redirect chunks (best-effort)
5. Continue downloading inner description's chunks, calling `onRawChunk` for each
### 7.2 Architecture note: download refactoring
Both upload and download use `agent.ts` for orchestration. The key difference is where the crypto/network split happens:
- **Upload**: agent.ts reads encrypted chunks from the Worker via `readChunk` callback, sends them over the network.
- **Download**: agent.ts receives raw encrypted responses from the network via `downloadXFTPChunkRaw` (DH key exchange + network only, no decryption), passes them to the web page via `onRawChunk` callback, which routes them to the Worker for transit decryption.
This split keeps all expensive crypto off the main thread. Transit decryption uses a custom JS Salsa20 implementation (`xorKeystream` in secretbox.ts) that would block the UI for ~50-200ms on a 4MB chunk. File-level decryption (`decryptChunks`) is similarly expensive. Both happen in the Worker.
The cheap operations stay on the main thread: DH key exchange (`generateX25519KeyPair` + `dh` — ~1ms via libsodium WASM), XFTP command encoding/decoding, connection management.
## 8. Build & Dev Setup
### 8.1 vite.config.ts (new, separate from vitest.config.ts)
```typescript
import {defineConfig, type Plugin} from 'vite'
import {readFileSync} from 'fs'
import {createHash} from 'crypto'
import presets from './web/servers.json'
function parseHost(addr: string): string {
const m = addr.match(/@(.+)$/)
if (!m) throw new Error('bad server address: ' + addr)
const host = m[1].split(',')[0]
return host.includes(':') ? host : host + ':443'
}
function cspPlugin(servers: string[]): Plugin {
const origins = servers.map(s => 'https://' + parseHost(s)).join(' ')
return {
name: 'csp-connect-src',
transformIndexHtml: {
order: 'pre',
handler(html, ctx) {
if (ctx.server) {
// Dev mode: remove CSP meta tag entirely — Vite HMR needs inline scripts
return html.replace(/<meta\s[^>]*?Content-Security-Policy[\s\S]*?>/i, '')
}
return html.replace('__CSP_CONNECT_SRC__', origins)
}
}
}
}
export default defineConfig(({mode}) => {
const define: Record<string, string> = {}
let servers: string[]
if (mode === 'local') {
const pem = readFileSync('../tests/fixtures/ca.crt', 'utf-8')
const der = Buffer.from(pem.replace(/-----[^-]+-----/g, '').replace(/\s/g, ''), 'base64')
const fp = createHash('sha256').update(der).digest('base64')
.replace(/\+/g, '-').replace(/\//g, '_')
servers = [`xftp://${fp}@localhost:7000`]
define['__XFTP_SERVERS__'] = JSON.stringify(servers)
} else {
servers = [...presets.simplex, ...presets.flux]
}
return {
root: 'web',
build: {outDir: '../dist-web'},
define,
worker: {format: 'es'},
plugins: [cspPlugin(servers)],
}
})
```
### 8.2 package.json scripts
```json
"dev": "vite --mode local",
"build:local": "vite build --mode local",
"build:prod": "vite build --mode production",
"preview": "vite preview",
"check:web": "tsc -p tsconfig.web.json --noEmit && tsc -p tsconfig.worker.json --noEmit"
```
Note: `check:web` type-checks `src/` twice (once per config) — acceptable for this small library.
Add `vite` as an explicit devDependency (`^6.0.0` — matching the version vitest 3.x depends on transitively). Relying on transitive resolution is fragile across package managers.
### 8.3 TypeScript configuration
The existing `tsconfig.json` has `rootDir: "src"` and `include: ["src/**/*.ts"]` — this is for library compilation only (output to `dist/`). Vite handles `web/` TypeScript compilation independently via esbuild, so the main tsconfig is unchanged. `web/*.ts` files import from `../src/*.js` using relative paths.
Add two tsconfigs for `web/` type-checking — split by environment to avoid type pollution between DOM and WebWorker globals:
`tsconfig.web.json` — main-thread files (DOM globals: `document`, `window`, etc.):
```json
{
"extends": "./tsconfig.json",
"compilerOptions": {
"rootDir": ".",
"noEmit": true,
"types": [],
"moduleResolution": "bundler",
"lib": ["ES2022", "DOM"]
},
"include": ["web/**/*.ts", "src/**/*.ts"],
"exclude": ["web/crypto.worker.ts"]
}
```
`tsconfig.worker.json` — Worker file (`self`, `FileSystemSyncAccessHandle`, etc.):
```json
{
"extends": "./tsconfig.json",
"compilerOptions": {
"rootDir": ".",
"noEmit": true,
"types": [],
"moduleResolution": "bundler",
"lib": ["ES2022", "WebWorker"]
},
"include": ["web/crypto.worker.ts", "src/**/*.ts"]
}
```
Both configs set `"types": []` to prevent auto-inclusion of `@types/node` and `"moduleResolution": "bundler"` for Vite-compatible resolution (JSON imports, `.js` extension mapping). The base config's `"moduleResolution": "node"` would cause false type errors on `import ... from './servers.json'`. Both override `@types/node`, which would pollute DOM/WebWorker environments with Node.js globals (`process`, `Buffer`, etc.). This means `src/client.ts`'s dynamic `import("node:http2")` will produce a type error in these configs. This is acceptable — `src/client.ts` provides `createNodeTransport` which is never used in browser code (Vite tree-shakes it out), and full `src/` type-checking is handled by the base `tsconfig.json`. If the error is distracting, add `src/client.ts` to both configs' `exclude` arrays.
Both extend the library tsconfig (inheriting `strict`, `module`, etc.) and include `src/**/*.ts` so imports from `../src/*.js` resolve. `"noEmit": true` means they're only used for type-checking — Vite handles actual compilation. The inherited `"exclude": ["node_modules", "dist", "test"]` intentionally excludes `test/` — test files are type-checked by their own vitest/playwright configs, not by `check:web`.
### 8.4 Dev workflow
`npm run dev` → Vite dev server at `localhost:5173`, configured for local test server. Start `xftp-server` on port 7000 separately (or via the existing globalSetup).
Note: The CSP meta tag's `default-src 'self'` blocks Vite's injected HMR inline scripts in dev mode. The `cspPlugin` handles this by removing the entire CSP `<meta>` tag in serve mode (dev server), so HMR works without restrictions. Production builds always have the correct CSP.
## 9. Library Changes (agent.ts + client.ts)
Changes to support the web page: upload `readChunk` callback, download `onRawChunk` callback with parallel chunk downloads.
### 9.1 Type changes
Split the existing `EncryptedFileInfo` (which currently has `encData`, `digest`, `key`, `nonce`, `chunkSizes` as direct fields) into a metadata-only base and an extension:
```typescript
// Metadata-only variant (no encData — data lives in Worker/OPFS)
export interface EncryptedFileMetadata {
digest: Uint8Array
key: Uint8Array
nonce: Uint8Array
chunkSizes: number[]
}
// Full variant (existing, extends metadata with data)
export interface EncryptedFileInfo extends EncryptedFileMetadata {
encData: Uint8Array
}
```
### 9.2 uploadFile signature change
Replace positional optional params with an options bag. Add optional `readChunk`. When provided, `encrypted.encData` is not accessed.
```typescript
export interface UploadOptions {
onProgress?: (uploaded: number, total: number) => void
redirectThreshold?: number
readChunk?: (offset: number, size: number) => Promise<Uint8Array>
}
export async function uploadFile(
agent: XFTPClientAgent,
server: XFTPServer,
encrypted: EncryptedFileMetadata,
options?: UploadOptions
): Promise<UploadResult>
```
Inside `uploadFile`:
- Chunk read: if `options?.readChunk` is provided, use it. Otherwise, verify `'encData' in encrypted` at runtime (throws `"uploadFile: readChunk required when encData is absent"` if missing), then use `(off, sz) => Promise.resolve((encrypted as EncryptedFileInfo).encData.subarray(off, off + sz))`. This guards against calling `uploadFile` with `EncryptedFileMetadata` but no `readChunk`. For each chunk, call `readChunk(offset, size)` once and use the returned `Uint8Array` for both `getChunkDigest(chunkData)` and `uploadXFTPChunk(..., chunkData)` — do not call `readChunk` twice per chunk.
- Progress total: `const total = encrypted.chunkSizes.reduce((a, b) => a + b, 0)` — replaces `encrypted.encData.length` (line 129) since `EncryptedFileMetadata` has no `encData`. The values are identical: `encData.length === sum(chunkSizes)`.
- `buildDescription` parameter type: change from `EncryptedFileInfo` to `EncryptedFileMetadata` — it only accesses `chunkSizes`, `digest`, `key`, `nonce` (not `encData`).
`uploadRedirectDescription` (internal) is unchanged — redirect descriptions are always small and created in-memory by `encryptFileForUpload`.
### 9.3 Backward compatibility
The signature change from positional params `(agent, server, encrypted, onProgress?, redirectThreshold?)` to `(agent, server, encrypted, options?)` is a breaking change for callers that pass `onProgress` or `redirectThreshold`. In practice, the only callers are the browser test (which passes no options — no change needed) and the web page (new code). `EncryptedFileInfo` extends `EncryptedFileMetadata`, so existing callers that pass `EncryptedFileInfo` work without change.
### 9.4 client.ts: downloadXFTPChunkRaw
Split `downloadXFTPChunk` at the network/crypto boundary. The new function does DH key exchange and network I/O but skips transit decryption:
```typescript
export interface RawChunkResponse {
dhSecret: Uint8Array
nonce: Uint8Array
body: Uint8Array
}
export async function downloadXFTPChunkRaw(
c: XFTPClient, rpKey: Uint8Array, fId: Uint8Array
): Promise<RawChunkResponse> {
const {publicKey, privateKey} = generateX25519KeyPair()
const cmd = encodeFGET(encodePubKeyX25519(publicKey))
const {response, body} = await sendXFTPCommand(c, rpKey, fId, cmd)
if (response.type !== "FRFile") throw new Error("unexpected response: " + response.type)
const dhSecret = dh(response.rcvDhKey, privateKey)
return {dhSecret, nonce: response.nonce, body}
}
```
`RawChunkResponse` contains only what client.ts produces (DH secret, nonce, encrypted body). The chunk metadata (`chunkNo`, `digest`) is added by agent.ts when constructing `RawDownloadedChunk` (see §9.5).
The existing `downloadXFTPChunk` is refactored to call `downloadXFTPChunkRaw` + `decryptReceivedChunk`:
```typescript
export async function downloadXFTPChunk(
c: XFTPClient, rpKey: Uint8Array, fId: Uint8Array, digest?: Uint8Array
): Promise<Uint8Array> {
const {dhSecret, nonce, body} = await downloadXFTPChunkRaw(c, rpKey, fId)
return decryptReceivedChunk(dhSecret, nonce, body, digest ?? null)
}
```
### 9.5 agent.ts: downloadFileRaw, ackFileChunks, RawDownloadedChunk
New type combining client.ts's `RawChunkResponse` with chunk metadata from agent.ts:
```typescript
export interface RawDownloadedChunk {
chunkNo: number
dhSecret: Uint8Array
nonce: Uint8Array
body: Uint8Array
digest: Uint8Array
}
```
New function providing download orchestration with a raw chunk callback. Handles connection pooling, parallel downloads, redirect resolution, and progress. Does **not** ACK — the caller ACKs after verification.
```typescript
export interface DownloadRawOptions {
onProgress?: (downloaded: number, total: number) => void
concurrency?: number // max parallel chunk downloads, default 1
}
export async function downloadFileRaw(
agent: XFTPClientAgent,
fd: FileDescription,
onRawChunk: (chunk: RawDownloadedChunk) => Promise<void>,
options?: DownloadRawOptions
): Promise<FileDescription>
```
Returns the resolved `FileDescription` — for redirect files this is the inner fd, for non-redirect files this is the original fd. The caller uses this for verification and ACK.
Internal structure:
1. Validate `fd` via `validateFileDescription` (may double-validate if caller already validated via `decodeDescriptionURI` — harmless)
2. If `fd.redirect !== null`: resolve redirect on main thread (redirect data is small):
a. Download redirect chunks via `downloadXFTPChunk` (not raw — main thread decryption is fine for a few KB)
b. Verify size + digest, `processDownloadedFile` → YAML bytes
c. Parse inner `FileDescription`, validate against `fd.redirect.{size, digest}`
d. ACK redirect chunks (best-effort — redirect chunks are small and separate from the file chunks)
e. Replace `fd` with inner description
3. Pre-connect: call `getXFTPServerClient(agent, server)` for each unique server before launching concurrent workers. This ensures the client connection exists in the agent's map, avoiding a race condition where multiple concurrent workers all see the client as missing and each call `connectXFTP` independently (leaking all but the last connection). Known limitation: if a connection drops mid-download and multiple workers attempt reconnection simultaneously, the same TOCTOU race reappears. This is a pre-existing issue in `getXFTPServerClient`; a proper fix (per-key connection promise) is out of scope for this plan but should be tracked for follow-up.
4. Download file chunks in parallel (concurrency-limited via sliding window):
- Create a queue of chunk indices `[0, 1, ..., N-1]`. Launch `min(concurrency, N)` async workers, each pulling the next index from the queue until empty. Each worker loops: pull index → derive key → `getXFTPServerClient``downloadXFTPChunkRaw``await onRawChunk(...)` → update progress → next index. `await Promise.all(workers)` to wait for completion.
- For each chunk: derive key (`decodePrivKeyEd25519``ed25519KeyPairFromSeed`), get client (`getXFTPServerClient`), call `downloadXFTPChunkRaw`, `await onRawChunk(...)` with result + `chunkNo` + `chunk.digest`
- Each concurrency slot awaits its `onRawChunk` before starting the next download on that slot. With `concurrency > 1`, multiple `onRawChunk` calls may be in-flight concurrently (one per slot). The Worker handles this correctly — messages are queued and processed sequentially.
- Update progress after each chunk: `downloaded += chunk.chunkSize; onProgress?.(downloaded, resolvedFd.size)` — both values use encrypted sizes for consistency
5. Return the resolved `fd`
New helper for ACKing after verification:
```typescript
export async function ackFileChunks(
agent: XFTPClientAgent, fd: FileDescription
): Promise<void> {
for (const chunk of fd.chunks) {
const replica = chunk.replicas[0]
if (!replica) continue
try {
const client = await getXFTPServerClient(agent, parseXFTPServer(replica.server))
const seed = decodePrivKeyEd25519(replica.replicaKey)
const kp = ed25519KeyPairFromSeed(seed)
await ackXFTPChunk(client, kp.privateKey, replica.replicaId)
} catch (_) {}
}
}
```
The existing `downloadFile` is refactored to use `downloadFileRaw` internally:
```typescript
export async function downloadFile(
agent: XFTPClientAgent,
fd: FileDescription,
onProgress?: (downloaded: number, total: number) => void
): Promise<DownloadResult> {
const chunks: Uint8Array[] = []
const resolvedFd = await downloadFileRaw(agent, fd, async (raw) => {
chunks[raw.chunkNo - 1] = decryptReceivedChunk(
raw.dhSecret, raw.nonce, raw.body, raw.digest
)
}, {onProgress})
// verify + file-level decrypt using resolvedFd (inner fd for redirect case)
const combined = chunks.length === 1 ? chunks[0] : concatBytes(...chunks)
if (combined.length !== resolvedFd.size) throw new Error("downloadFile: file size mismatch")
const digest = sha512(combined)
if (!digestEqual(digest, resolvedFd.digest)) throw new Error("downloadFile: file digest mismatch")
// processDownloadedFile re-concatenates chunks internally — this mirrors the
// existing downloadFile pattern (verify on concatenated data, then pass chunks
// array to decryptChunks which concatenates again). Acceptable overhead for
// correctness: verification must happen on transit-decrypted data before
// file-level decryption transforms it.
const result = processDownloadedFile(resolvedFd, chunks)
await ackFileChunks(agent, resolvedFd)
return result
}
```
Existing callers retain serial behavior (`concurrency` defaults to 1). The web page opts into parallelism by passing `concurrency: 3`. The browser test (`test/browser.test.ts`) continues to work unchanged. The chunks array is initialized empty (`[]`) and populated by sparse index assignment (`chunks[raw.chunkNo - 1] = ...`), so it correctly handles both redirect and non-redirect cases regardless of the outer fd's chunk count. `digestEqual` is an existing module-private helper in agent.ts (line 327) that performs constant-time byte comparison.
### 9.6 Backward compatibility (download)
`downloadFile` signature is unchanged — existing callers are unaffected. The refactoring adds `downloadFileRaw`, `ackFileChunks`, and `RawDownloadedChunk` as new exports from agent.ts, and `downloadXFTPChunkRaw` + `RawChunkResponse` as new exports from client.ts.
## 10. Testing
### 10.1 Existing tests (unchanged)
- `npm run test:browser` — vitest browser round-trip (library-level)
- `cabal test --test-option='--match=/XFTP Web Client/'` — Haskell per-function tests
### 10.2 New: page E2E test
Add `test/page.spec.ts` using `@playwright/test` (not vitest browser mode — vitest tests run IN the browser and can't control page navigation; Playwright tests run in Node.js and control the browser). Add `@playwright/test` as a devDependency.
Add `playwright.config.ts` at the project root (`xftp-web/`):
- `webServer: { command: 'vite build --mode local && vite preview', url: 'http://localhost:4173', reuseExistingServer: !process.env.CI }` — the `url` property tells Playwright to wait until the preview server is ready before running tests
- `use.ignoreHTTPSErrors: true` (test server uses self-signed cert)
- `use.launchOptions: { args: ['--ignore-certificate-errors'] }` — required because Playwright's `ignoreHTTPSErrors` only affects page navigation, not `fetch()` calls from in-page JavaScript. Without this flag, the page's `createBrowserTransport` fetch to `https://localhost:7000` would fail TLS validation.
- `globalSetup`: `'./test/globalSetup.ts'` (starts xftp-server, shared with vitest)
```typescript
import {test, expect} from '@playwright/test'
test('page upload + download round-trip', async ({page}) => {
await page.goto(PAGE_URL)
// Set file input via page.setInputFiles()
// Wait for upload link to appear: page.waitForSelector('[data-testid="share-link"]')
// Extract hash from link text
// Navigate to PAGE_URL + '#' + hash
// Wait for download complete state
// Verify file was offered for save (check download event)
})
```
Add script: `"test:page": "playwright test test/page.spec.ts"`
This tests the real bundle including Worker loading, OPFS, and CSP. The existing `test/browser.test.ts` continues to test the library-level pipeline (vitest browser mode, no Workers).
### 10.3 Manual testing
`npm run dev` → open `localhost:5173` in browser → drag file → get link → open link in new tab → download. Requires xftp-server running on port 7000 (local mode).
## 11. Files
**Create:**
- `xftp-web/web/index.html` — page entry point (includes CSP meta tag)
- `xftp-web/web/main.ts` — router + libsodium init
- `xftp-web/web/upload.ts` — upload UI + orchestration
- `xftp-web/web/download.ts` — download UI + orchestration
- `xftp-web/web/progress.ts` — circular progress canvas component
- `xftp-web/web/servers.json` — preset server addresses (shared by servers.ts and vite.config.ts)
- `xftp-web/web/servers.ts` — server configuration (imports servers.json)
- `xftp-web/web/crypto-backend.ts` — CryptoBackend interface + WorkerBackend + factory
- `xftp-web/web/crypto.worker.ts` — Web Worker implementation
- `xftp-web/web/style.css` — styles
- `xftp-web/vite.config.ts` — page build config (CSP generation, server list)
- `xftp-web/tsconfig.web.json` — IDE/CI type-checking for `web/` main-thread files (DOM)
- `xftp-web/tsconfig.worker.json` — IDE/CI type-checking for `web/crypto.worker.ts` (WebWorker)
- `xftp-web/playwright.config.ts` — Playwright E2E test config (webServer, globalSetup)
- `xftp-web/test/page.spec.ts` — page E2E test (Playwright)
**Modify:**
- `xftp-web/src/agent.ts` — add `EncryptedFileMetadata` type, `uploadFile` options bag with `readChunk`, `downloadFileRaw` with `onRawChunk` callback + parallel downloads, `ackFileChunks`, `RawDownloadedChunk` type, refactor `downloadFile` on top of `downloadFileRaw`, add `import {decryptReceivedChunk} from "./download.js"` (needed by refactored `downloadFile`)
- `xftp-web/src/client.ts` — add `downloadXFTPChunkRaw`, `RawChunkResponse` type, refactor `downloadXFTPChunk` to use raw variant
- `xftp-web/package.json` — add dev/build/check:web/test:page scripts, add `vite` + `@playwright/test` devDeps
- `xftp-web/src/protocol/description.ts` — fix stale "SHA-256" comment on `FileDescription.digest` to "SHA-512"
- `xftp-web/.gitignore` — add `dist-web/`
## 12. Implementation Order
1. **Library refactoring**`client.ts`: add `downloadXFTPChunkRaw`; `agent.ts`: add `downloadFileRaw` + parallel downloads, `uploadFile` options bag with `readChunk`; refactor existing `downloadFile` on top of `downloadFileRaw`. Run existing tests to verify no regressions.
2. **Vite config + HTML shell**`vite.config.ts`, `index.html`, `main.ts`, verify dev server works
3. **Server config**`servers.ts` with both local and production server lists
4. **CryptoBackend + Worker** — interface, WorkerBackend, Worker implementation, OPFS logic
5. **Upload flow**`upload.ts` with drag-drop, encrypt via Worker, upload via agent, show link
6. **Download flow**`download.ts` with URL parsing, download via agent `downloadFileRaw`, Worker decrypt, browser save
7. **Progress component**`progress.ts` canvas drawing
8. **Styling**`style.css`
9. **Testing** — page E2E test, manual browser verification
10. **Build scripts**`build:local`, `build:prod` in package.json
@@ -0,0 +1,53 @@
# XFTPClientAgent Pattern
## TOC
1. Executive Summary
2. Changes: client.ts
3. Changes: agent.ts
4. Changes: test/browser.test.ts
5. Verification
## Executive Summary
Add `XFTPClientAgent` — a per-server connection pool matching the Haskell pattern. The agent caches `XFTPClient` instances by server URL. All orchestration functions (`uploadFile`, `downloadFile`, `deleteFile`) take `agent` as first parameter and use `getXFTPServerClient(agent, server)` instead of calling `connectXFTP` directly. Connections stay open on success; the caller creates and closes the agent.
`connectXFTP` and `closeXFTP` stay exported (used by `XFTPWebTests.hs` Haskell tests). The `browserClients` hack, per-function `connections: Map`, and `getOrConnect` are deleted.
## Changes: client.ts
**Add** after types section: `XFTPClientAgent` interface, `newXFTPAgent`, `getXFTPServerClient`, `closeXFTPServerClient`, `closeXFTPAgent`.
**Delete**: `browserClients` Map and all `isNode` browser-cache checks in `connectXFTP` and `closeXFTP`.
**Revert `closeXFTP`** to unconditional `c.transport.close()` (browser transport.close() is already a no-op).
`connectXFTP` stays exported (backward compat) but becomes a raw low-level function — no caching.
## Changes: agent.ts
**Imports**: replace `connectXFTP`/`closeXFTP` with `getXFTPServerClient`/`closeXFTPAgent` etc.
**Re-export** from agent.ts: `newXFTPAgent`, `closeXFTPAgent`, `XFTPClientAgent`.
**`uploadFile`**: add `agent: XFTPClientAgent` as first param. Replace `connectXFTP``getXFTPServerClient`. Remove `finally { closeXFTP }`. Pass `agent` to `uploadRedirectDescription`.
**`uploadRedirectDescription`**: change from `(client, server, innerFd)` to `(agent, server, innerFd)`. Get client via `getXFTPServerClient`.
**`downloadFile`**: add `agent` param. Delete local `connections: Map`. Replace `getOrConnect``getXFTPServerClient`. Remove finally cleanup. Pass `agent` to `downloadWithRedirect`.
**`downloadWithRedirect`**: add `agent` param. Same replacements. Remove try/catch cleanup. Recursive call passes `agent`.
**`deleteFile`**: add `agent` param. Same pattern.
**Delete**: `getOrConnect` function entirely.
## Changes: test/browser.test.ts
Create agent before operations, pass to upload/download, close in finally.
## Verification
1. `npx vitest --run` — browser round-trip test passes
2. No remaining `browserClients`, `getOrConnect`, or per-function `connections: Map` locals
3. `connectXFTP` and `closeXFTP` still exported (XFTPWebTests.hs compat)
4. All orchestration functions take `agent` as first param
@@ -0,0 +1,859 @@
# XFTP Web Page E2E Tests Plan
## Table of Contents
1. [Executive Summary](#1-executive-summary)
2. [Test Infrastructure](#2-test-infrastructure)
3. [Test Infrastructure - Page Objects](#3-test-infrastructure---page-objects)
4. [Upload Flow Tests](#4-upload-flow-tests)
5. [Download Flow Tests](#5-download-flow-tests)
6. [Edge Cases](#6-edge-cases)
7. [Implementation Order](#7-implementation-order)
8. [Test Utilities](#8-test-utilities)
---
## 1. Executive Summary
This document specifies comprehensive Playwright E2E tests for the XFTP web page. The existing test (`page.spec.ts`) performs a basic upload/download round-trip. This plan extends coverage to:
- **Upload flow**: File selection (picker + drag-drop), validation, progress, cancellation, link sharing, error handling
- **Download flow**: Invalid link handling, download button, progress, file save, error states
- **Edge cases**: Boundary file sizes, special characters, network failures, multi-packet files with redirect, UI information display
**Key constraints**:
- Tests run against a local XFTP router (started via `globalSetup.ts`)
- Server port is dynamic (read from `/tmp/xftp-test-server.port`)
- Browser uses `--ignore-certificate-errors` for self-signed certs
- OPFS and Web Workers are required (Chromium supports both)
**Test file location**: `/code/simplexmq/xftp-web/test/page.spec.ts`
**Architecture**: Tests use the Page Object Model pattern to encapsulate UI interactions, making tests read as domain-specific scenarios rather than raw Playwright API calls.
---
## 2. Test Infrastructure
### 2.1 Current Setup
```
xftp-web/
├── playwright.config.ts # Playwright config (webServer, globalSetup)
├── test/
│ ├── globalSetup.ts # Starts xftp-server, writes port to PORT_FILE
│ ├── page.spec.ts # E2E tests (to be extended)
│ └── pages/ # Page Objects (new)
│ ├── UploadPage.ts
│ └── DownloadPage.ts
```
### 2.2 Prerequisites
- `globalSetup.ts` starts the XFTP router and writes port to `PORT_FILE`
- Tests must read the port dynamically: `readFileSync(PORT_FILE, 'utf-8').trim()`
- Vite builds and serves the page at `http://localhost:4173`
---
## 3. Test Infrastructure - Page Objects
Page Objects encapsulate page-specific selectors and actions, providing a clean API for tests. This follows the standard Page Object Model pattern used in simplex-chat and most professional test suites.
### 3.1 UploadPage
```typescript
// test/pages/UploadPage.ts
import {Page, Locator, expect} from '@playwright/test'
export class UploadPage {
readonly page: Page
readonly dropZone: Locator
readonly fileInput: Locator
readonly progressStage: Locator
readonly progressCanvas: Locator
readonly statusText: Locator
readonly cancelButton: Locator
readonly completeStage: Locator
readonly shareLink: Locator
readonly copyButton: Locator
readonly errorStage: Locator
readonly errorMessage: Locator
readonly retryButton: Locator
readonly expiryNote: Locator
readonly securityNote: Locator
constructor(page: Page) {
this.page = page
this.dropZone = page.locator('#drop-zone')
this.fileInput = page.locator('#file-input')
this.progressStage = page.locator('#upload-progress')
this.progressCanvas = page.locator('#progress-container canvas')
this.statusText = page.locator('#upload-status')
this.cancelButton = page.locator('#cancel-btn')
this.completeStage = page.locator('#upload-complete')
this.shareLink = page.locator('[data-testid="share-link"]')
this.copyButton = page.locator('#copy-btn')
this.errorStage = page.locator('#upload-error')
this.errorMessage = page.locator('#error-msg')
this.retryButton = page.locator('#retry-btn')
this.expiryNote = page.locator('.expiry')
this.securityNote = page.locator('.security-note')
}
async goto() {
await this.page.goto('http://localhost:4173')
}
async selectFile(name: string, content: Buffer, mimeType = 'application/octet-stream') {
await this.fileInput.setInputFiles({name, mimeType, buffer: content})
}
async selectTextFile(name: string, content: string) {
await this.selectFile(name, Buffer.from(content, 'utf-8'), 'text/plain')
}
async selectLargeFile(name: string, sizeBytes: number) {
// Create large file in browser to avoid memory issues in test process
await this.page.evaluate(({name, size}) => {
const input = document.getElementById('file-input') as HTMLInputElement
const buffer = new ArrayBuffer(size)
new Uint8Array(buffer).fill(0x55)
const file = new File([buffer], name, {type: 'application/octet-stream'})
const dt = new DataTransfer()
dt.items.add(file)
input.files = dt.files
input.dispatchEvent(new Event('change', {bubbles: true}))
}, {name, size: sizeBytes})
}
async dragDropFile(name: string, content: Buffer) {
// Drag-drop uses same file input handler internally
await this.selectFile(name, content)
}
async waitForEncrypting(timeout = 10_000) {
await expect(this.statusText).toContainText('Encrypting', {timeout})
}
async waitForUploading(timeout = 30_000) {
await expect(this.statusText).toContainText('Uploading', {timeout})
}
async waitForShareLink(timeout = 60_000): Promise<string> {
await expect(this.shareLink).toBeVisible({timeout})
return await this.shareLink.inputValue()
}
async clickCopy() {
await this.copyButton.click()
await expect(this.copyButton).toContainText('Copied!')
}
async clickCancel() {
await this.cancelButton.click()
}
async clickRetry() {
await this.retryButton.click()
}
async expectError(messagePattern: string | RegExp) {
await expect(this.errorStage).toBeVisible()
await expect(this.errorMessage).toContainText(messagePattern)
}
async expectDropZoneVisible() {
await expect(this.dropZone).toBeVisible()
}
async expectProgressVisible() {
await expect(this.progressStage).toBeVisible()
await expect(this.progressCanvas).toBeVisible()
}
async expectCompleteWithExpiry() {
await expect(this.completeStage).toBeVisible()
await expect(this.expiryNote).toContainText('48 hours')
}
async expectSecurityNote() {
await expect(this.securityNote).toBeVisible()
await expect(this.securityNote).toContainText('encrypted')
}
getHashFromLink(url: string): string {
return new URL(url).hash
}
}
```
### 3.2 DownloadPage
```typescript
// test/pages/DownloadPage.ts
import {Page, Locator, expect, Download} from '@playwright/test'
export class DownloadPage {
readonly page: Page
readonly readyStage: Locator
readonly downloadButton: Locator
readonly progressStage: Locator
readonly progressCanvas: Locator
readonly statusText: Locator
readonly errorStage: Locator
readonly errorMessage: Locator
readonly retryButton: Locator
readonly securityNote: Locator
constructor(page: Page) {
this.page = page
this.readyStage = page.locator('#dl-ready')
this.downloadButton = page.locator('#dl-btn')
this.progressStage = page.locator('#dl-progress')
this.progressCanvas = page.locator('#dl-progress-container canvas')
this.statusText = page.locator('#dl-status')
this.errorStage = page.locator('#dl-error')
this.errorMessage = page.locator('#dl-error-msg')
this.retryButton = page.locator('#dl-retry-btn')
this.securityNote = page.locator('.security-note')
}
async goto(hash: string) {
await this.page.goto(`http://localhost:4173${hash}`)
}
async gotoWithLink(fullUrl: string) {
const hash = new URL(fullUrl).hash
await this.goto(hash)
}
async expectFileReady() {
await expect(this.readyStage).toBeVisible()
await expect(this.downloadButton).toBeVisible()
}
async expectFileSizeDisplayed() {
await expect(this.readyStage).toContainText(/\d+(?:\.\d+)?\s*(?:KB|MB|B)/)
}
async clickDownload(): Promise<Download> {
const downloadPromise = this.page.waitForEvent('download')
await this.downloadButton.click()
return downloadPromise
}
async waitForDownloading(timeout = 30_000) {
await expect(this.statusText).toContainText('Downloading', {timeout})
}
async waitForDecrypting(timeout = 30_000) {
await expect(this.statusText).toContainText('Decrypting', {timeout})
}
async expectProgressVisible() {
await expect(this.progressStage).toBeVisible()
await expect(this.progressCanvas).toBeVisible()
}
async expectInitialError(messagePattern: string | RegExp) {
// For malformed links - error shown in card without #dl-error stage
await expect(this.page.locator('.card .error')).toBeVisible()
await expect(this.page.locator('.card .error')).toContainText(messagePattern)
}
async expectRuntimeError(messagePattern: string | RegExp) {
// For runtime download errors - uses #dl-error stage
await expect(this.errorStage).toBeVisible()
await expect(this.errorMessage).toContainText(messagePattern)
}
async expectSecurityNote() {
await expect(this.securityNote).toBeVisible()
await expect(this.securityNote).toContainText('encrypted')
}
}
```
### 3.3 Test Fixtures
```typescript
// test/fixtures.ts
import {test as base} from '@playwright/test'
import {UploadPage} from './pages/UploadPage'
import {DownloadPage} from './pages/DownloadPage'
import {readFileSync} from 'fs'
// Extend Playwright test with page objects
export const test = base.extend<{
uploadPage: UploadPage
downloadPage: DownloadPage
}>({
uploadPage: async ({page}, use) => {
const uploadPage = new UploadPage(page)
await uploadPage.goto()
await use(uploadPage)
},
downloadPage: async ({page}, use) => {
await use(new DownloadPage(page))
},
})
export {expect} from '@playwright/test'
// Test data helpers
export function createTestContent(size: number, fill = 0x41): Buffer {
return Buffer.alloc(size, fill)
}
export function createTextContent(text: string): Buffer {
return Buffer.from(text, 'utf-8')
}
export function uniqueFileName(base: string, ext = 'txt'): string {
return `${base}-${Date.now()}.${ext}`
}
```
---
## 4. Upload Flow Tests
### 4.1 File Selection - File Picker Button
**Test ID**: `upload-file-picker`
```typescript
test('upload via file picker button', async ({uploadPage}) => {
await uploadPage.expectDropZoneVisible()
await uploadPage.selectTextFile('picker-test.txt', 'test content ' + Date.now())
await uploadPage.waitForEncrypting()
await uploadPage.waitForUploading()
const link = await uploadPage.waitForShareLink()
expect(link).toMatch(/^http:\/\/localhost:\d+\/#/)
})
```
### 4.2 File Selection - Drag and Drop
**Test ID**: `upload-drag-drop`
```typescript
test('upload via drag and drop', async ({uploadPage}) => {
await uploadPage.dragDropFile('dragdrop-test.txt', createTextContent('drag drop test'))
await uploadPage.expectProgressVisible()
const link = await uploadPage.waitForShareLink()
expect(link).toContain('#')
})
```
### 4.3 File Size Validation - Too Large
**Test ID**: `upload-file-too-large`
```typescript
test('upload rejects file over 100MB', async ({uploadPage}) => {
await uploadPage.selectLargeFile('large.bin', 100 * 1024 * 1024 + 1)
await uploadPage.expectError('too large')
await uploadPage.expectError('100 MB')
})
```
### 4.4 File Size Validation - Empty File
**Test ID**: `upload-file-empty`
```typescript
test('upload rejects empty file', async ({uploadPage}) => {
await uploadPage.selectFile('empty.txt', Buffer.alloc(0))
await uploadPage.expectError('empty')
})
```
### 4.5 Progress Display
**Test ID**: `upload-progress-display`
```typescript
test('upload shows progress during encryption and upload', async ({uploadPage}) => {
await uploadPage.selectFile('progress-test.bin', createTestContent(500 * 1024))
await uploadPage.expectProgressVisible()
await uploadPage.waitForEncrypting()
await uploadPage.waitForUploading()
await uploadPage.waitForShareLink()
})
```
### 4.6 Cancel Button
**Test ID**: `upload-cancel`
```typescript
test('cancel button aborts upload and returns to landing', async ({uploadPage}) => {
await uploadPage.selectFile('cancel-test.bin', createTestContent(1024 * 1024))
await uploadPage.expectProgressVisible()
await uploadPage.clickCancel()
await uploadPage.expectDropZoneVisible()
await expect(uploadPage.shareLink).toBeHidden()
})
```
### 4.7 Share Link Display and Copy
**Test ID**: `upload-share-link-copy`
```typescript
test('share link copy button works', async ({uploadPage, context}) => {
await context.grantPermissions(['clipboard-read', 'clipboard-write'])
await uploadPage.selectTextFile('copy-test.txt', 'copy test content')
const link = await uploadPage.waitForShareLink()
await uploadPage.clickCopy()
// Verify clipboard (may fail in headless)
try {
const clipboardText = await uploadPage.page.evaluate(() => navigator.clipboard.readText())
expect(clipboardText).toBe(link)
} catch {
// Clipboard API may not be available
}
})
```
### 4.8 Error Handling and Retry
**Test ID**: `upload-error-retry`
```typescript
test('error state shows retry button', async ({uploadPage}) => {
await uploadPage.selectFile('error-test.txt', Buffer.alloc(0))
await uploadPage.expectError('empty')
await expect(uploadPage.retryButton).toBeVisible()
})
```
---
## 5. Download Flow Tests
### 5.1 Invalid Link Handling - Malformed Hash
**Test ID**: `download-invalid-hash-malformed`
```typescript
test('download shows error for malformed hash', async ({downloadPage}) => {
await downloadPage.goto('#not-valid-base64!!!')
await downloadPage.expectInitialError(/[Ii]nvalid|corrupted/)
await expect(downloadPage.downloadButton).not.toBeVisible()
})
```
### 5.2 Invalid Link Handling - Valid Base64 but Invalid Structure
**Test ID**: `download-invalid-hash-structure`
```typescript
test('download shows error for invalid structure', async ({downloadPage}) => {
await downloadPage.goto('#AAAA')
await downloadPage.expectInitialError(/[Ii]nvalid|corrupted/)
})
```
### 5.3 Download Button Click
**Test ID**: `download-button-click`
```typescript
test('download button initiates download', async ({uploadPage, downloadPage}) => {
// Upload first
await uploadPage.selectTextFile('dl-btn-test.txt', 'download test content')
const link = await uploadPage.waitForShareLink()
// Navigate to download
await downloadPage.gotoWithLink(link)
await downloadPage.expectFileReady()
// Click download
const download = await downloadPage.clickDownload()
expect(download.suggestedFilename()).toBe('dl-btn-test.txt')
})
```
### 5.4 Progress Display
**Test ID**: `download-progress-display`
```typescript
test('download shows progress', async ({uploadPage, downloadPage}) => {
await uploadPage.selectFile('dl-progress.bin', createTestContent(500 * 1024))
const link = await uploadPage.waitForShareLink()
await downloadPage.gotoWithLink(link)
const downloadPromise = downloadPage.clickDownload()
await downloadPage.expectProgressVisible()
await downloadPage.waitForDownloading()
await downloadPromise
})
```
### 5.5 File Save Verification
**Test ID**: `download-file-save`
```typescript
test('downloaded file content matches upload', async ({uploadPage, downloadPage}) => {
const content = 'verification content ' + Date.now()
const fileName = 'verify.txt'
await uploadPage.selectTextFile(fileName, content)
const link = await uploadPage.waitForShareLink()
await downloadPage.gotoWithLink(link)
const download = await downloadPage.clickDownload()
expect(download.suggestedFilename()).toBe(fileName)
const path = await download.path()
if (path) {
const downloadedContent = (await import('fs')).readFileSync(path, 'utf-8')
expect(downloadedContent).toBe(content)
}
})
```
---
## 6. Edge Cases
### 6.1 Very Small Files
**Test ID**: `edge-small-file`
```typescript
test('upload and download 1-byte file', async ({uploadPage, downloadPage}) => {
await uploadPage.selectFile('tiny.bin', Buffer.from([0x42]))
const link = await uploadPage.waitForShareLink()
await downloadPage.gotoWithLink(link)
const download = await downloadPage.clickDownload()
expect(download.suggestedFilename()).toBe('tiny.bin')
const path = await download.path()
if (path) {
const content = (await import('fs')).readFileSync(path)
expect(content.length).toBe(1)
expect(content[0]).toBe(0x42)
}
})
```
### 6.2 Files Near 100MB Limit
**Test ID**: `edge-near-limit`
```typescript
test.slow()
test('upload file at exactly 100MB', async ({uploadPage}) => {
await uploadPage.selectLargeFile('exactly-100mb.bin', 100 * 1024 * 1024)
// Should succeed (not show error)
await expect(uploadPage.errorStage).toBeHidden({timeout: 5000})
await uploadPage.expectProgressVisible()
// Wait for completion (may take a while)
await uploadPage.waitForShareLink(300_000)
})
```
### 6.3 Special Characters in Filename
**Test ID**: `edge-special-chars-filename`
```typescript
test('upload and download file with unicode filename', async ({uploadPage, downloadPage}) => {
const fileName = 'test-\u4e2d\u6587-\u0420\u0443\u0441\u0441\u043a\u0438\u0439.txt'
await uploadPage.selectTextFile(fileName, 'unicode filename test')
const link = await uploadPage.waitForShareLink()
await downloadPage.gotoWithLink(link)
const download = await downloadPage.clickDownload()
expect(download.suggestedFilename()).toBe(fileName)
})
test('upload and download file with spaces', async ({uploadPage, downloadPage}) => {
const fileName = 'my document (final) v2.txt'
await uploadPage.selectTextFile(fileName, 'spaces test')
const link = await uploadPage.waitForShareLink()
await downloadPage.gotoWithLink(link)
const download = await downloadPage.clickDownload()
expect(download.suggestedFilename()).toBe(fileName)
})
test('filename with path separators is sanitized', async ({uploadPage, downloadPage}) => {
await uploadPage.selectTextFile('../../../etc/passwd', 'path traversal test')
const link = await uploadPage.waitForShareLink()
await downloadPage.gotoWithLink(link)
const download = await downloadPage.clickDownload()
expect(download.suggestedFilename()).not.toContain('/')
expect(download.suggestedFilename()).not.toContain('\\')
})
```
### 6.4 Network Errors (Mocked)
**Test ID**: `edge-network-error`
```typescript
test('upload handles network error gracefully', async ({uploadPage}) => {
// Intercept and abort POST requests
await uploadPage.page.route('**/localhost:*', route => {
if (route.request().method() === 'POST') {
route.abort('failed')
} else {
route.continue()
}
})
await uploadPage.selectTextFile('network-error.txt', 'network error test')
await uploadPage.expectError(/.+/) // Any error message
})
```
### 6.5 Binary File Content Integrity
**Test ID**: `edge-binary-content`
```typescript
test('binary file with all byte values', async ({uploadPage, downloadPage}) => {
// Create buffer with all 256 byte values
const buffer = Buffer.alloc(256)
for (let i = 0; i < 256; i++) buffer[i] = i
await uploadPage.selectFile('all-bytes.bin', buffer)
const link = await uploadPage.waitForShareLink()
await downloadPage.gotoWithLink(link)
const download = await downloadPage.clickDownload()
const path = await download.path()
if (path) {
const content = (await import('fs')).readFileSync(path)
expect(content.length).toBe(256)
for (let i = 0; i < 256; i++) {
expect(content[i]).toBe(i)
}
}
})
```
### 6.6 Multiple Concurrent Downloads
**Test ID**: `edge-concurrent-downloads`
```typescript
test('concurrent downloads from same link', async ({browser}) => {
const context = await browser.newContext({ignoreHTTPSErrors: true})
const page1 = await context.newPage()
const upload = new UploadPage(page1)
await upload.goto()
await upload.selectTextFile('concurrent.txt', 'concurrent download test')
const link = await upload.waitForShareLink()
const hash = upload.getHashFromLink(link)
// Open two tabs and download concurrently
const page2 = await context.newPage()
const page3 = await context.newPage()
const dl2 = new DownloadPage(page2)
const dl3 = new DownloadPage(page3)
await dl2.goto(hash)
await dl3.goto(hash)
const [download2, download3] = await Promise.all([
dl2.clickDownload(),
dl3.clickDownload()
])
expect(download2.suggestedFilename()).toBe('concurrent.txt')
expect(download3.suggestedFilename()).toBe('concurrent.txt')
await context.close()
})
```
### 6.7 Redirect File Handling (Multi-packet)
**Test ID**: `edge-redirect-file`
```typescript
test.slow()
test('upload and download multi-chunk file with redirect', async ({uploadPage, downloadPage}) => {
// Use ~5MB file to get multiple chunks
await uploadPage.selectLargeFile('multi-chunk.bin', 5 * 1024 * 1024)
const link = await uploadPage.waitForShareLink(120_000)
await downloadPage.gotoWithLink(link)
const download = await downloadPage.clickDownload()
expect(download.suggestedFilename()).toBe('multi-chunk.bin')
const path = await download.path()
if (path) {
const stat = (await import('fs')).statSync(path)
expect(stat.size).toBe(5 * 1024 * 1024)
}
})
```
### 6.8 UI Information Display
**Test ID**: `edge-ui-info`
```typescript
test('upload complete shows expiry and security note', async ({uploadPage}) => {
await uploadPage.selectTextFile('ui-test.txt', 'ui test')
await uploadPage.waitForShareLink()
await uploadPage.expectCompleteWithExpiry()
await uploadPage.expectSecurityNote()
})
test('download page shows file size and security note', async ({uploadPage, downloadPage}) => {
await uploadPage.selectFile('size-test.bin', createTestContent(1024))
const link = await uploadPage.waitForShareLink()
await downloadPage.gotoWithLink(link)
await downloadPage.expectFileSizeDisplayed()
await downloadPage.expectSecurityNote()
})
```
---
## 7. Implementation Order
### Phase 1: Core Infrastructure (Priority: High)
1. Create `test/pages/UploadPage.ts` with Page Object
2. Create `test/pages/DownloadPage.ts` with Page Object
3. Create `test/fixtures.ts` with extended test function
4. Refactor existing test to use Page Objects
### Phase 2: Core Happy Path (Priority: High)
5. `upload-file-picker` - Basic upload via file picker
6. `download-button-click` - Basic download
7. `download-file-save` - Content verification
### Phase 3: Validation (Priority: High)
8. `upload-file-too-large` - Size validation
9. `upload-file-empty` - Empty file validation
10. `download-invalid-hash-malformed` - Invalid link handling
11. `download-invalid-hash-structure` - Invalid structure handling
### Phase 4: Progress and Cancel (Priority: Medium)
12. `upload-progress-display` - Progress visibility
13. `upload-cancel` - Cancel functionality
14. `download-progress-display` - Download progress
### Phase 5: Link Sharing (Priority: Medium)
15. `upload-share-link-copy` - Copy button functionality
16. `upload-drag-drop` - Drag-drop upload
### Phase 6: Edge Cases (Priority: Low)
17. `edge-small-file` - 1-byte file
18. `edge-special-chars-filename` - Unicode/special characters
19. `edge-binary-content` - Binary content integrity
20. `edge-near-limit` - 100MB file (slow test)
21. `edge-network-error` - Network error handling
### Phase 7: Error Recovery and Advanced (Priority: Low)
22. `upload-error-retry` - Retry after error
23. `edge-concurrent-downloads` - Concurrent access
24. `edge-redirect-file` - Multi-packet file with redirect (slow)
25. `edge-ui-info` - Expiry message, security notes
---
## 8. Test Utilities
### 8.1 Shared Test Setup
```typescript
// test/page.spec.ts
import {test, expect, createTestContent, createTextContent, uniqueFileName} from './fixtures'
test.describe('Upload Flow', () => {
test('upload via file picker', async ({uploadPage}) => {
// Tests use uploadPage fixture which navigates automatically
})
})
test.describe('Download Flow', () => {
test('download works', async ({uploadPage, downloadPage}) => {
// Both pages available via fixtures
})
})
test.describe('Edge Cases', () => {
// Edge case tests
})
```
### 8.2 File Structure
```
xftp-web/test/
├── fixtures.ts # Playwright fixtures with page objects
├── pages/
│ ├── UploadPage.ts # Upload page object
│ └── DownloadPage.ts # Download page object
├── page.spec.ts # All E2E tests
└── globalSetup.ts # Server startup (existing)
```
---
## Appendix: Test Matrix
| Test ID | Category | Priority | Estimated Time | Dependencies |
|---------|----------|----------|----------------|--------------|
| upload-file-picker | Upload | High | 30s | - |
| upload-drag-drop | Upload | Medium | 30s | - |
| upload-file-too-large | Upload | High | 5s | - |
| upload-file-empty | Upload | High | 5s | - |
| upload-progress-display | Upload | Medium | 45s | - |
| upload-cancel | Upload | Medium | 30s | - |
| upload-share-link-copy | Upload | Medium | 30s | - |
| upload-error-retry | Upload | Low | 30s | - |
| download-invalid-hash-malformed | Download | High | 5s | - |
| download-invalid-hash-structure | Download | High | 5s | - |
| download-button-click | Download | High | 45s | upload |
| download-progress-display | Download | Medium | 60s | upload |
| download-file-save | Download | High | 45s | upload |
| edge-small-file | Edge | Low | 30s | - |
| edge-near-limit | Edge | Low | 300s | - |
| edge-special-chars-filename | Edge | Low | 30s | - |
| edge-network-error | Edge | Low | 45s | - |
| edge-binary-content | Edge | Low | 30s | - |
| edge-concurrent-downloads | Edge | Low | 60s | upload |
| edge-redirect-file | Edge | Low | 120s | - |
| edge-ui-info | Edge | Low | 60s | upload |
**Total estimated time**: ~18 minutes (excluding 100MB and 5MB tests)
@@ -0,0 +1,221 @@
# XFTP Web Hello Header — Session Re-handshake for Browser Connection Reuse
## 1. Problem Statement
Browser HTTP/2 connection pooling reuses TLS connections across page navigations (same origin = same connection pool). The XFTP router maintains per-TLS-connection session state in `TMap SessionId Handshake` keyed by `tlsUniq tls`. When a browser navigates from the upload page to the download page (or reloads), the new page sends a fresh ClientHello on the reused HTTP/2 connection. The server is already in `HandshakeAccepted` state for that connection, so it routes the request to `processRequest`, which expects a 16384-byte command block but receives a 34-byte ClientHello → `ERR BLOCK`.
**Root cause**: The router cannot distinguish a ClientHello from a command on an already-handshaked connection because both arrive on the same HTTP/2 connection (same `tlsUniq`), and there is no content-level discriminator (ClientHello is unpadded, but the router never gets to parse it — the size check in `processRequest` rejects it first).
**Browser limitation**: `fetch()` provides zero control over HTTP/2 connection pooling. There is no browser API to force a new connection or detect connection reuse before a request is sent.
## 2. Solution Summary
Add an HTTP header `xftp-web-hello` to web ClientHello requests. When the router sees this header on an already-handshaked connection (`HandshakeAccepted` state), it re-runs `processHello` **reusing the existing session keys** (same X25519 key pair from the original handshake). The client then completes the normal handshake flow (sends ClientHandshake, receives ack) and proceeds with commands.
Key properties:
- Router reuses existing `serverPrivKey` — no new key material generated on re-handshake, so `thAuth` remains consistent with any in-flight commands on concurrent HTTP/2 streams.
- Header is only checked when `sniUsed` is true (web/browser connections). Native XFTP clients are unaffected.
- CORS preflight already allows all headers (`Access-Control-Allow-Headers: *`).
- Web clients always send this header on ClientHello — it's harmless on first connection (`Nothing` state) and enables re-handshake on reused connections (`HandshakeAccepted` state).
## 3. Detailed Technical Design
### 3.1 Router change: parameterize `processHello` (`src/Simplex/FileTransfer/Server.hs`)
The entire router change is parameterizing the existing `processHello` with `Maybe C.PrivateKeyX25519`. Zero new functions.
#### Current code (lines 165-191):
```haskell
xftpServerHandshakeV1 chain serverSignKey sessions
XFTPTransportRequest {thParams = thParams0@THandleParams {sessionId}, reqBody = HTTP2Body {bodyHead}, sendResponse, sniUsed, addCORS} = do
s <- atomically $ TM.lookup sessionId sessions
r <- runExceptT $ case s of
Nothing -> processHello
Just (HandshakeSent pk) -> processClientHandshake pk
Just (HandshakeAccepted thParams) -> pure $ Just thParams
either sendError pure r
where
processHello = do
challenge_ <-
if
| B.null bodyHead -> pure Nothing
| sniUsed -> do
XFTPClientHello {webChallenge} <- liftHS $ smpDecode bodyHead
pure webChallenge
| otherwise -> throwE HANDSHAKE
(k, pk) <- atomically . C.generateKeyPair =<< asks random
atomically $ TM.insert sessionId (HandshakeSent pk) sessions
-- ...build and send ServerHandshake...
pure Nothing
```
#### After (diff is ~10 lines):
```haskell
xftpServerHandshakeV1 chain serverSignKey sessions
XFTPTransportRequest {thParams = thParams0@THandleParams {sessionId}, request, reqBody = HTTP2Body {bodyHead}, sendResponse, sniUsed, addCORS} = do
-- ^^^^^^^ bind request
s <- atomically $ TM.lookup sessionId sessions
r <- runExceptT $ case s of
Nothing -> processHello Nothing
Just (HandshakeSent pk) -> processClientHandshake pk
Just (HandshakeAccepted thParams)
| webHello -> processHello (serverPrivKey <$> thAuth thParams)
| otherwise -> pure $ Just thParams
either sendError pure r
where
webHello = sniUsed && any (\(t, _) -> tokenKey t == "xftp-web-hello") (fst $ H.requestHeaders request)
processHello pk_ = do
challenge_ <-
if
| B.null bodyHead -> pure Nothing
| sniUsed -> do
XFTPClientHello {webChallenge} <- liftHS $ smpDecode bodyHead
pure webChallenge
| otherwise -> throwE HANDSHAKE
(k, pk) <- maybe
(atomically . C.generateKeyPair =<< asks random)
(\pk -> pure (C.publicKey pk, pk))
pk_
atomically $ TM.insert sessionId (HandshakeSent pk) sessions
-- ...rest unchanged...
pure Nothing
```
#### What changes:
1. **Bind `request`** in the `XFTPTransportRequest` pattern (+1 field)
2. **Add `webHello`** binding in `where` clause (1 line) — checks header only when `sniUsed`
3. **Add `pk_` parameter** to `processHello` (change signature)
4. **Replace key generation** with `maybe` that generates fresh keys when `pk_ = Nothing`, or derives public from existing private when `pk_ = Just pk` (3 lines replace 1 line)
5. **Add guard** in `HandshakeAccepted` branch (2 lines replace 1 line)
6. **Call site** `Nothing -> processHello Nothing` (+1 word)
7. **One import** added: `Network.HPACK.Token (tokenKey)`
#### Imports to add:
```haskell
import Network.HPACK.Token (tokenKey)
```
`OverloadedStrings` (already enabled in Server.hs) provides the `IsString` instance for `CI ByteString`, so `tokenKey t == "xftp-web-hello"` works without importing `Data.CaseInsensitive`. Verified on Hackage: `requestHeaders :: Request -> HeaderTable`, `tokenKey :: Token -> CI ByteString`.
### 3.2 Re-handshake flow
When `webHello` is true in `HandshakeAccepted` state:
1. `processHello (serverPrivKey <$> thAuth thParams)` is called with `Just pk` (existing private key)
2. `(k, pk) <- pure (C.publicKey pk, pk)` — reuses same key pair, no generation
3. `TM.insert sessionId (HandshakeSent pk) sessions` — transitions state back to `HandshakeSent` with same `pk`
4. Server sends `ServerHandshake` response (same format as initial handshake)
5. Client sends `ClientHandshake` on next stream → enters `Just (HandshakeSent pk) -> processClientHandshake pk` → normal flow
6. `processClientHandshake` stores `HandshakeAccepted thParams` with same `serverPrivKey = pk`
### 3.3 Web client change (`xftp-web/src/client.ts`)
Add optional `headers?` parameter to `Transport.post()`, thread it through `fetch()` and `session.request()`, and pass `{"xftp-web-hello": "1"}` in the ClientHello call in `connectXFTP`.
### 3.4 What does NOT change
- **CORS**: Already has `Access-Control-Allow-Headers: *` (Server.hs:106).
- **Native Haskell client**: Uses `[]` headers. No header = existing behavior.
- **Protocol wire format**: ClientHello, ServerHandshake, ClientHandshake, commands — all unchanged.
- **`processRequest`**, **`processClientHandshake`**, **`sendError`**, **`encodeXftp`** — unchanged.
### 3.5 Haskell test (`tests/XFTPServerTests.hs`)
Add `testWebReHandshake` next to the existing `testWebHandshake` (line 504). It reuses the same SNI + HTTP/2 setup pattern, performs a full handshake, then sends a second ClientHello with the `xftp-web-hello` header on the same connection and verifies the router responds with a valid ServerHandshake (same `sessionId`), then completes the second handshake.
```haskell
-- Register in xftpServerTests (after line 86):
it "should re-handshake on same connection with xftp-web-hello header" testWebReHandshake
-- Test (after testWebHandshake):
testWebReHandshake :: Expectation
testWebReHandshake =
withXFTPServerSNI $ \_ -> do
Fingerprint fp <- loadFileFingerprint "tests/fixtures/ca.crt"
let keyHash = C.KeyHash fp
cfg = defaultTransportClientConfig {clientALPN = Just ["h2"], useSNI = True}
runTLSTransportClient defaultSupportedParamsHTTPS Nothing cfg Nothing "localhost" xftpTestPort (Just keyHash) $ \(tls :: TLS 'TClient) -> do
let h2cfg = HC.defaultHTTP2ClientConfig {HC.bodyHeadSize = 65536}
h2 <- either (error . show) pure =<< HC.attachHTTP2Client h2cfg (THDomainName "localhost") xftpTestPort mempty 65536 tls
g <- C.newRandom
-- First handshake (same as testWebHandshake)
challenge1 <- atomically $ C.randomBytes 32 g
let helloReq1 = H2.requestBuilder "POST" "/" [] $ byteString (smpEncode (XFTPClientHello {webChallenge = Just challenge1}))
resp1 <- either (error . show) pure =<< HC.sendRequest h2 helloReq1 (Just 5000000)
shs1 <- either error pure $ smpDecode =<< C.unPad (bodyHead (HC.respBody resp1))
let XFTPServerHandshake {sessionId = sid1} = shs1
clientHsPadded <- either (error . show) pure $ C.pad (smpEncode (XFTPClientHandshake {xftpVersion = VersionXFTP 1, keyHash})) xftpBlockSize
resp1b <- either (error . show) pure =<< HC.sendRequest h2 (H2.requestBuilder "POST" "/" [] $ byteString clientHsPadded) (Just 5000000)
B.length (bodyHead (HC.respBody resp1b)) `shouldBe` 0
-- Second handshake on same connection with xftp-web-hello header
challenge2 <- atomically $ C.randomBytes 32 g
let helloReq2 = H2.requestBuilder "POST" "/" [("xftp-web-hello", "1")] $ byteString (smpEncode (XFTPClientHello {webChallenge = Just challenge2}))
resp2 <- either (error . show) pure =<< HC.sendRequest h2 helloReq2 (Just 5000000)
shs2 <- either error pure $ smpDecode =<< C.unPad (bodyHead (HC.respBody resp2))
let XFTPServerHandshake {sessionId = sid2} = shs2
sid2 `shouldBe` sid1 -- same TLS connection → same sessionId
-- Complete second handshake
resp2b <- either (error . show) pure =<< HC.sendRequest h2 (H2.requestBuilder "POST" "/" [] $ byteString clientHsPadded) (Just 5000000)
B.length (bodyHead (HC.respBody resp2b)) `shouldBe` 0
```
The only difference from `testWebHandshake`: the second `helloReq2` passes `[("xftp-web-hello", "1")]` instead of `[]`. The test verifies:
1. Server responds with `ServerHandshake` (not `ERR BLOCK`)
2. Same `sessionId` (same TLS connection)
3. Second `ClientHandshake` completes with empty ACK
## 4. Implementation Plan
### Step 1: Router — parameterize `processHello`
Apply the diff from Section 3.1 to `src/Simplex/FileTransfer/Server.hs`.
### Step 2: Test — add `testWebReHandshake`
Add the test from Section 3.5 to `tests/XFTPServerTests.hs`.
### Step 3: Client — add `xftp-web-hello` header
Add optional `headers?` to `Transport.post()`, pass `{"xftp-web-hello": "1"}` on ClientHello in `connectXFTP`.
### Step 4: Test
Run Haskell tests (`cabal test`) and E2E Playwright tests (`npx playwright test` in `xftp-web/`).
## 5. Race Condition Analysis
### Single-tab navigation (the common case)
1. Upload page completes, all fetch() requests finish
2. Browser navigates to download page (or reloads)
3. All upload-page fetches are aborted on page unload
4. Download page sends ClientHello with `xftp-web-hello` header
5. Server is in `HandshakeAccepted``processHello (Just pk)``HandshakeSent pk` (same key)
6. No concurrent streams → no race
**Safe.**
### Multi-tab (edge case)
Tab A (upload) and Tab B (download) share the same HTTP/2 connection.
1. Tab A has active command streams (e.g., FPUT upload in progress)
2. Tab B sends ClientHello with header
3. Server reads `HandshakeAccepted` atomically for both streams
4. Tab A's stream already has its `thParams` snapshot → proceeds with `processRequest` using old `thParams`
5. Tab B's stream triggers `processHello (Just pk)` → stores `HandshakeSent pk` (same pk!)
6. Tab A's in-progress FPUT continues with snapshot `thParams` → completes normally (same `serverPrivKey`)
7. Tab A's NEXT command reads `HandshakeSent` from TMap → enters `processClientHandshake` → fails (command body ≠ ClientHandshake format) → HANDSHAKE error
**Tab A's in-flight commands succeed. Tab A's subsequent commands fail with HANDSHAKE error.** This is the inherent multi-tab problem — unavoidable with per-connection session state and HTTP/2 connection sharing. The failure is clean (HANDSHAKE error, not silent corruption).
## 6. Security Considerations
- **No new key material**: Re-handshake reuses existing `serverPrivKey`. No opportunity for key confusion or downgrade.
- **Identity re-verification**: Router re-signs the web challenge with its long-term signing key. Client verifies identity again.
- **Header cannot escalate privileges**: The header only triggers re-handshake (which the router was already capable of doing on first connection). It does not bypass any authentication.
- **Timing**: Re-handshake takes the same code path as initial handshake, so timing side-channels are unchanged.
@@ -0,0 +1,948 @@
# XFTP Web Error Handling and Connection Resilience
## 1. Problem Statement
The XFTP web client is fundamentally fragile: any transient error (browser opening a new HTTP/2 connection, network hiccup, router restart) causes an unrecoverable failure with a cryptic error message. There is no retry logic, no fetch timeout, no error categorization, and the upload uses a single router instead of distributing data packets across preset routers. This makes the app frustrating — it works most of the time but fails unpredictably, which is worse than being completely broken.
### Confirmed root cause (from diagnostic logs)
When the browser opens a new HTTP/2 connection mid-operation, the new connection has a different TLS SessionId with no handshake state in the router's `TMap SessionId Handshake`. The router's `Nothing` branch in `xftpServerHandshakeV1` (Server.hs:169) unconditionally calls `processHello`, which tries to decode the command body as `XFTPClientHello`, fails, and sends a raw padded "HANDSHAKE" error string. The client cannot parse this as a proper transmission (first byte 'H' = 72 is read as batch count), producing `"expected batch count 1, got 72"`.
Router log confirming the SessionId change:
```
DEBUG dispatch: Accepted+command sessId="ZSo1GGETgIvjbB7CWHbvGPpbMjx_b2IlC1eTI6aKfqc="
...20 successful commands...
DEBUG dispatch: Nothing sessId="mJC7Sck9xxW5UsXoPGoUWduuHghSVgf6CnD6ZC6SBhU=" webHello=False
```
### Why re-handshake is required (cannot be made optional)
1. **SessionId is baked into signed command data.** `encodeAuthTransmission` signs `concat(encode(sessionId), tInner)` with Ed25519. Router's `tDecodeServer` (Protocol.hs:2242) verifies `sessId == sessionId`. New connection = different sessionId = signature mismatch.
2. **Router generates per-session DH keys.** `processHello` creates fresh X25519 keypair stored in `HandshakeSent`. For SMP browser clients (future), `verifyCmdAuth` (Protocol.hs:1322) requires the matching `serverPrivKey` from `thAuth`.
3. **This applies to both XFTP and future SMP browser clients** — the session management approach is the same.
### Why multiple preset routers cannot work
Upload (`agent.ts:105-157`) takes a single `server: XFTPServer` parameter and uploads ALL data packets to it. `web/upload.ts:133` calls `pickRandomServer(servers)` which selects ONE random router from all presets. The multi-router preset configuration is pointless — only one router is ever used per upload. The design intent (RFC section 11.6: "upload in parallel to 8 randomly selected routers") is not implemented. This must be fixed in Phase 2 (section 3.7).
## 2. Solution Summary
### Phase 1: Error handling and connection resilience
1. **Router: strict dispatch for allowed protocol combinations** — reject all invalid combinations
2. **Client: automatic retry with re-handshake** on SESSION/HANDSHAKE errors
3. **Client: fetch timeout** with configurable duration
4. **UI: error categorization and retry** — auto-retry temporary, human-readable permanent
5. **Client: connection state with Promise-based lock and per-router queues**`ServerConnection` with `client: Promise<XFTPClient>` + `queue: Promise<void>`
6. **Client: fix cache key** — include keyHash
### Phase 2: Multi-router upload (after Phase 1)
7. **Multi-router upload with router selection and failover** — distribute data packets across routers, retry FNEW on different router if one fails
## 3. Detailed Technical Design
### 3.1 Router: strict dispatch for allowed protocol combinations
**Principle:** Everything not explicitly done by existing Haskell/TS clients is prohibited. It is better to fail on impossible combinations than to be permissive — permissiveness complicates debugging and creates attack vectors via unexpected behaviors.
**Allowed behaviors by client type:**
| Client | SNI | webHello header | Hello body | When |
|--------|-----|----------------|------------|------|
| Haskell | No | No | Empty | New connection only |
| Web | Yes | Yes | Non-empty (XFTPClientHello) | New OR existing connection |
**Minimal surgical change.** The existing dispatch (Server.hs:169-189) already correctly handles `HandshakeSent` and `HandshakeAccepted` — their guards cover all valid and invalid combinations. The ONLY missing case is `Nothing` + web client sending a command on a stale session.
`processHello` (Server.hs:194-217) already internally routes: `B.null bodyHead` → Haskell hello, `sniUsed` → web hello decode, else → HANDSHAKE. For stale web sessions, it currently tries to decode a command body as `XFTPClientHello`, fails, and throws HANDSHAKE. The fix: detect this case BEFORE calling processHello and throw SESSION instead, so the client knows to re-handshake (not that its hello was malformed).
**Change: add one guard to `Nothing` branch, remove debug logging.**
```haskell
-- Before (1 line):
Nothing -> processHello Nothing
-- After (3 lines):
Nothing
| sniUsed && not webHello -> throwE SESSION -- web command on stale session
| otherwise -> processHello Nothing -- normal hello (web or Haskell)
```
`throwE SESSION` is caught by `either sendError pure r` (line 190). `sendError` pads `smpEncode SESSION` = `"SESSION"` (Transport.hs:298) to `xftpBlockSize`. The client's padded error detection (section 3.2) catches this as a retriable error and triggers re-handshake. SESSION is a valid `XFTPErrorType` constructor (Transport.hs:225) — no new helpers needed.
**All other branches remain unchanged.** `HandshakeSent` guards (`webHello` → processHello, `otherwise` → processClientHandshake with body size check inside) are correct. `HandshakeAccepted` guards (`webHello`, `webHandshake`, `otherwise` → command) are correct.
### 3.2 Client: automatic retry with re-handshake
**Location:** `sendXFTPCommand` in `client.ts`
**Design:** Retry loop inside `sendXFTPCommand`. Maximum 3 attempts. On retriable error, close old client, re-handshake, retry.
**Error classification:**
| Error | Type | Retriable? | Human-readable message |
|-------|------|-----------|----------------------|
| Padded "HANDSHAKE" | Temporary | Yes (auto) | "Connection interrupted, reconnecting..." |
| Padded "SESSION" | Temporary | Yes (auto) | "Session expired, reconnecting..." |
| `FRErr SESSION` | Temporary | Yes (auto) | "Session expired, reconnecting..." |
| `FRErr HANDSHAKE` | Temporary | Yes (auto) | "Connection interrupted, reconnecting..." |
| `fetch()` TypeError | Temporary | Yes (auto) | "Network error, retrying..." |
| AbortError (timeout) | Temporary | Yes (auto) | "Router timeout, retrying..." |
| `FRErr AUTH` | Permanent | No | "File is invalid, expired, or has been removed" |
| `FRErr NO_FILE` | Permanent | No | "File not found — it may have expired" |
| `FRErr SIZE` | Permanent | No | "File size exceeds router limit" |
| `FRErr QUOTA` | Permanent | No | "Router storage quota exceeded" |
| `FRErr BLOCKED` | Permanent | No | "File has been blocked by router" |
| `FRErr DIGEST` | Permanent | No | "File integrity check failed" |
| `FRErr INTERNAL` | Permanent | No | "Router internal error" |
| `CMD *` | Permanent | No | "Protocol error" |
**Retry behavior:**
- Auto-retry up to 3 times for temporary errors, transparent to user
- After 3 failures: show human-readable error with diagnosis, offer manual retry button
- Permanent errors: show human-readable error immediately, NO manual retry button (user can reload page)
**Implementation:**
```typescript
async function sendXFTPCommand(
agent: XFTPClientAgent,
server: XFTPServer,
privateKey: Uint8Array,
entityId: Uint8Array,
cmdBytes: Uint8Array,
chunkData?: Uint8Array,
maxRetries: number = 3
): Promise<{response: FileResponse, body: Uint8Array}> {
let clientP = getXFTPServerClient(agent, server)
let client = await clientP
for (let attempt = 1; attempt <= maxRetries; attempt++) {
try {
return await sendXFTPCommandOnce(client, privateKey, entityId, cmdBytes, chunkData)
} catch (e) {
if (!isRetriable(e)) {
// Permanent error (AUTH, NO_FILE, etc.) — connection is fine, don't touch it
throw categorizeError(e)
}
if (attempt === maxRetries) {
// Retriable error exhausted — connection is bad, remove stale promise
removeStaleConnection(agent, server, clientP)
throw categorizeError(e)
}
clientP = reconnectClient(agent, server)
client = await clientP
}
}
throw new Error("unreachable")
}
```
**`sendXFTPCommandOnce`** — renamed from current `sendXFTPCommand`. Two changes:
1. **Padded error detection** (before `decodeTransmission`):
```typescript
// After getting respBlock, before decodeTransmission:
const raw = blockUnpad(respBlock)
if (raw.length < 20) {
const text = new TextDecoder().decode(raw)
if (/^[A-Z_]+$/.test(text)) {
throw new XFTPRetriableError(text) // "HANDSHAKE" or "SESSION"
}
}
```
2. **FRErr classification** (replaces current unconditional throw):
```typescript
// After decodeResponse, instead of throw new Error("Router error: " + err.type):
if (response.type === "FRErr") {
const err = response.err
if (err.type === "SESSION" || err.type === "HANDSHAKE") {
throw new XFTPRetriableError(err.type)
}
throw new XFTPPermanentError(err.type, humanReadableMessage(err))
}
```
### 3.3 Client: fetch timeout
**Location:** `createBrowserTransport` and `createNodeTransport` in `client.ts`
**Design:** `AbortController` with configurable timeout on every `fetch()`.
```typescript
interface TransportConfig {
timeoutMs: number // default 30000, lower for tests
}
function createBrowserTransport(baseUrl: string, config: TransportConfig): Transport {
return {
async post(body: Uint8Array, headers?: Record<string, string>): Promise<Uint8Array> {
const controller = new AbortController()
const timer = setTimeout(() => controller.abort(), config.timeoutMs)
try {
const resp = await fetch(effectiveUrl, {
method: "POST", headers, body,
signal: controller.signal
})
if (!resp.ok) throw new Error(`Server request failed: ${resp.status}`)
return new Uint8Array(await resp.arrayBuffer())
} finally {
clearTimeout(timer)
}
},
close() {}
}
}
```
For Node.js transport, use `setTimeout` on the HTTP/2 request stream.
Default: 30s for production, 5s for tests. Threaded through `connectXFTP``createTransport`.
### 3.4 UI: error categorization and retry
**Behavior (Option D):**
- **Temporary errors:** Auto-retry loop (3 attempts). After 3 failures, show human-readable diagnosis with manual retry button. Diagnosis examples: "Router timeout — the router may be temporarily unavailable", "Connection interrupted — your network may be unstable".
- **Permanent errors:** Show human-readable error immediately, NO retry button. User can reload page if they want to retry. Examples: "File is invalid, expired, or has been removed" (AUTH), "File not found" (NO_FILE).
**Current UI retry buttons:**
- `upload.ts:73-75` — retry calls `startUpload(pendingFile)` from scratch
- `download.ts:60` — retry calls `startDownload()` from scratch
**Improvement:** Track uploaded/downloaded data packet indices. On manual retry, skip completed data packets:
```typescript
// Upload: track which data packets completed
const completedChunks: Set<number> = new Set()
for (let i = 0; i < specs.length; i++) {
if (completedChunks.has(i)) continue
// ... create + upload data packet
completedChunks.add(i)
}
// Download: already naturally resumable — each data packet is independent
```
### 3.5 Client: connection state with Promise-based lock and per-router queues
**Design:** Each router gets a `ServerConnection` record containing a `Promise<XFTPClient>` (the connection lock) and a `Promise<void>` (the sequential command queue). The `XFTPClientAgent` maps router keys to these records.
The promise IS the lock — every consumer awaits the same promise. When reconnect is needed, the promise is replaced atomically.
```typescript
interface ServerConnection {
client: Promise<XFTPClient> // resolves to connected client; replaced on reconnect
queue: Promise<void> // tail of sequential command chain
}
interface XFTPClientAgent {
connections: Map<string, ServerConnection>
}
function newXFTPAgent(): XFTPClientAgent {
return {connections: new Map()}
}
```
**Connection lifecycle — `getXFTPServerClient` and `reconnectClient`:**
```typescript
function getXFTPServerClient(agent: XFTPClientAgent, server: XFTPServer): Promise<XFTPClient> {
const key = formatXFTPServer(server)
let conn = agent.connections.get(key)
if (!conn) {
const p = connectXFTP(server)
conn = {client: p, queue: Promise.resolve()}
agent.connections.set(key, conn)
// On connection failure, remove from map so next call retries
p.catch(() => {
const cur = agent.connections.get(key)
if (cur && cur.client === p) agent.connections.delete(key)
})
}
return conn.client
}
function reconnectClient(agent: XFTPClientAgent, server: XFTPServer): Promise<XFTPClient> {
const key = formatXFTPServer(server)
const old = agent.connections.get(key)
// Close old client (fire-and-forget)
old?.client.then(c => c.transport.close(), () => {})
// Replace with new connection promise — all concurrent callers will await this
// Queue survives reconnect — pending operations stay ordered
const p = connectXFTP(server)
const conn: ServerConnection = {client: p, queue: old?.queue ?? Promise.resolve()}
agent.connections.set(key, conn)
p.catch(() => {
const cur = agent.connections.get(key)
if (cur && cur.client === p) agent.connections.delete(key)
})
return p
}
function closeXFTPServerClient(agent: XFTPClientAgent, server: XFTPServer): void {
const key = formatXFTPServer(server)
const conn = agent.connections.get(key)
if (conn) {
agent.connections.delete(key)
conn.client.then(c => c.transport.close(), () => {})
}
}
function closeXFTPAgent(agent: XFTPClientAgent): void {
for (const conn of agent.connections.values()) {
conn.client.then(c => c.transport.close(), () => {})
}
agent.connections.clear()
}
```
**Precise semantics:**
1. `getXFTPServerClient(agent, server)` — returns existing `conn.client` promise if present, otherwise creates a new `ServerConnection` with fresh connection and empty queue
2. When error detected, first caller calls `reconnectClient` which replaces `conn.client` with a new connection promise. The queue is preserved across reconnect.
3. All concurrent callers awaiting the OLD promise receive the error
4. They then call `getXFTPServerClient` which returns the NEW promise
5. If reconnection fails, auto-cleanup (`p.catch(() => delete)`) removes the entry so the next caller starts fresh
**Stale error cleanup rule:** When a caller exhausts retries for a retriable error, it removes the failed entry from the map (only if no concurrent caller has already replaced it via `reconnectClient`). This prevents the next caller from receiving a stale rejected promise. Permanent errors (AUTH, NO_FILE, etc.) do NOT remove the connection — the transport is fine, only the command failed.
```typescript
function removeStaleConnection(
agent: XFTPClientAgent, server: XFTPServer, failedP: Promise<XFTPClient>
): void {
const key = formatXFTPServer(server)
const conn = agent.connections.get(key)
// Only remove if current promise is the one that failed — not if already replaced by reconnect
if (conn && conn.client === failedP) {
agent.connections.delete(key)
failedP.then(c => c.transport.close(), () => {})
}
}
```
**Per-router sequential queue:** `queue` is a `Promise<void>` — the tail of the sequential operation chain. Each new operation `.then()`s onto it. It's `void` because callers hold their own typed promises; the queue only tracks completion order:
```typescript
async function enqueueCommand<T>(
agent: XFTPClientAgent,
server: XFTPServer,
fn: () => Promise<T> // no client param — fn uses command wrappers (agent+server)
): Promise<T> {
const key = formatXFTPServer(server)
// Ensure connection exists (with auto-cleanup on failure)
await getXFTPServerClient(agent, server)
const conn = agent.connections.get(key)! // guaranteed to exist after getXFTPServerClient
// Chain onto the queue — fn runs after previous operation completes
let resolve_: (v: T) => void, reject_: (e: any) => void
const result = new Promise<T>((res, rej) => { resolve_ = res; reject_ = rej })
conn.queue = conn.queue.then(
() => fn().then(resolve_!, reject_!),
() => fn().then(resolve_!, reject_!)
).then(() => {}, () => {}) // swallow errors in the chain
return result
}
```
Commands to the same router execute one at a time via the queue. Commands to different routers execute concurrently because each has its own queue. `enqueueCommand` provides sequencing; `sendXFTPCommand` (called inside `fn` via command wrappers) provides retry. They compose as: `enqueueCommand` sequences calls to wrappers that internally use `sendXFTPCommand`.
**Download change:** Group data packets by router, process each router's data packets sequentially, routers in parallel. Uses `for` loop for per-router sequencing (same pattern as Stage 2 upload). `enqueueCommand` is available for cases where different callers target the same router.
```typescript
const byServer = new Map<string, FileChunk[]>()
for (const chunk of resolvedFd.chunks) {
const srv = chunk.replicas[0]?.server ?? ""
if (!byServer.has(srv)) byServer.set(srv, [])
byServer.get(srv)!.push(chunk)
}
await Promise.all([...byServer.entries()].map(async ([srv, chunks]) => {
const server = parseXFTPServer(srv)
for (const chunk of chunks) {
const seed = decodePrivKeyEd25519(chunk.replicas[0].replicaKey)
const kp = ed25519KeyPairFromSeed(seed)
const raw = await downloadXFTPChunkRaw(agent, server, kp.privateKey, chunk.replicas[0].replicaId)
await onRawChunk({chunkNo: chunk.chunkNo, dhSecret: raw.dhSecret, nonce: raw.nonce, body: raw.body, digest: chunk.digest})
downloaded += chunk.chunkSize
onProgress?.(downloaded, resolvedFd.size)
}
}))
```
### 3.6 Fix cache key
**Bug:** `getXFTPServerClient` (client.ts:110) uses `"https://" + server.host + ":" + server.port` as cache key, ignoring `keyHash`. Two routers with same host:port but different keyHash share a cached connection, bypassing identity verification.
**Fix:** Use `formatXFTPServer(server)` as cache key (includes keyHash). Already available in `protocol/address.ts:52-54`.
```typescript
// Before:
const key = "https://" + server.host + ":" + server.port
// After:
const key = formatXFTPServer(server)
```
Note: With the redesign in 3.5, the cache key fix is inherent — the `connections` Map uses `formatXFTPServer(server)` everywhere.
### 3.7 Phase 2: Multi-router upload with router selection and failover
**Problem:** Current upload (`agent.ts:105-157`) takes a single `server: XFTPServer` and uploads ALL data packets to it. The 12 preset routers (6 SimpleX + 6 Flux) are pointless — only one is ever used.
**Design goal:** Distribute data packets across routers. Retry FNEW on a different router if one fails. Once working routers are found, prefer them (heuristic: router unlikely to fail mid-process, more likely to be broken initially due to maintenance/downtime).
**Reference implementation:** Haskell `Agent.hs:457-486` (`createChunk` / `createWithNextSrv`) + `Client.hs:2335-2385` (`getNextServer_` / `withNextSrv`).
#### Haskell algorithm summary
Two-stage architecture:
1. **Allocate stage (serial per file in Haskell):** For each data packet, call FNEW on a randomly-selected router. If FNEW fails, pick a different router and retry. Track tried hosts to avoid retrying the same router. After all data packets are assigned to routers, spawn one upload worker per router.
2. **Upload stage (parallel per router):** Each router worker uploads its assigned data packets sequentially (FPUT). On FPUT failure, retry on the same router with backoff (because the data packet replica already exists on that router). No router failover for FPUT.
Router selection constraints (hierarchical, `getNextServer_` Client.hs:2335-2350):
1. Prefer routers from unused operators (operator diversity)
2. Prefer routers with unused hosts (host diversity)
3. Random pick from the most-constrained candidate set
4. If all exhausted, reset tried set and start over
#### Web client adaptation
The web client doesn't have operators or a database. Simplified algorithm with two stages:
**Stage 1 — Allocate:** Create data packet records on routers (FNEW). Unlike Haskell which is serial here, web FNEW runs concurrently within a concurrency limit. FNEW is a small command — concurrent FNEW on the same connection is not a problem, and concurrent FNEW across routers improves upload startup time.
**Stage 2 — Upload:** Upload data packet content (FPUT). Parallel across routers, sequential per router (reuses per-router queues from 3.5). FPUT retries on the same router with backoff — no router rotation because the data packet replica already exists on that router. Stage 2 reads data packet content by offset (via `readChunk`), so `SentChunk` must be extended with `chunkOffset: number` (from ChunkSpec).
```typescript
interface UploadState {
untriedServers: XFTPServer[] // routers not yet attempted — initially all routers
workingServers: XFTPServer[] // routers that succeeded FNEW
}
const MAX_FNEW_ATTEMPTS = 5 // per data packet: try up to 5 different routers
async function uploadFile(
agent: XFTPClientAgent,
allServers: XFTPServer[],
encrypted: EncryptedFileMetadata,
options?: UploadOptions
): Promise<UploadResult> {
const state: UploadState = {untriedServers: [...allServers], workingServers: []}
const specs = prepareChunkSpecs(encrypted.chunkSizes)
const concurrency = options?.concurrency ?? 4
// Stage 1: Allocate — concurrent FNEW within concurrency limit
const sentChunks: SentChunk[] = new Array(specs.length)
const queue = specs.map((spec, i) => ({spec, chunkNo: i + 1, index: i}))
let idx = 0
async function allocateWorker() {
while (idx < queue.length) {
const item = queue[idx++]
const {server, chunk} = await createChunkWithFailover(
agent, allServers, state, concurrency, item.spec, item.chunkNo
)
sentChunks[item.index] = chunk
}
}
const allocateWorkers = Array.from(
{length: Math.min(concurrency, queue.length)},
() => allocateWorker()
)
await Promise.all(allocateWorkers)
// Stage 2: Upload — parallel across routers, sequential per router
// readChunk reads from the encrypted file by offset (same as Phase 1 uploadFile)
let uploaded = 0
const total = encrypted.chunkSizes.reduce((a, b) => a + b, 0)
const byServer = groupBy(sentChunks, c => formatXFTPServer(c.server))
await Promise.all([...byServer.entries()].map(async ([srvKey, chunks]) => {
for (const chunk of chunks) {
const chunkData = await readChunk(chunk.chunkOffset, chunk.chunkSize)
await uploadXFTPChunk(agent, chunk.server, chunk.senderKey, chunk.senderId, chunkData)
uploaded += chunk.chunkSize
options?.onProgress?.(uploaded, total)
}
}))
return buildDescriptions(encrypted, sentChunks)
}
```
**`createChunkWithFailover`** — router selection with per-data-packet retry limit:
```typescript
async function createChunkWithFailover(
agent: XFTPClientAgent,
allServers: XFTPServer[],
state: UploadState,
concurrency: number,
spec: ChunkSpec,
chunkNo: number
): Promise<{server: XFTPServer, chunk: SentChunk}> {
const maxAttempts = Math.min(allServers.length, MAX_FNEW_ATTEMPTS)
for (let attempt = 0; attempt < maxAttempts; attempt++) {
const server = pickServer(allServers, state, concurrency)
try {
const chunk = await createAndPrepareChunk(agent, server, spec, chunkNo)
// Success — add to working set (if not already there)
if (!state.workingServers.some(s => formatXFTPServer(s) === formatXFTPServer(server))) {
state.workingServers.push(server)
}
return {server, chunk}
} catch (e) {
// Remove from working if it was there
state.workingServers = state.workingServers.filter(
s => formatXFTPServer(s) !== formatXFTPServer(server)
)
if (attempt === maxAttempts - 1) throw e
}
}
throw new Error("unreachable")
}
```
**`pickServer`** — two-list selection:
```typescript
function pickServer(
allServers: XFTPServer[],
state: UploadState,
concurrency: number
): XFTPServer {
// Once enough working routers found, only use those
if (state.workingServers.length >= concurrency) {
return randomPick(state.workingServers)
}
// Still exploring — pick from untried
if (state.untriedServers.length > 0) {
const idx = Math.floor(Math.random() * state.untriedServers.length)
return state.untriedServers.splice(idx, 1)[0] // remove from untried
}
// All tried — reset untried to non-working routers and retry
state.untriedServers = allServers.filter(
s => !state.workingServers.some(w => formatXFTPServer(w) === formatXFTPServer(s))
)
if (state.untriedServers.length > 0) {
const idx = Math.floor(Math.random() * state.untriedServers.length)
return state.untriedServers.splice(idx, 1)[0]
}
// Every router is working — pick any working
return randomPick(state.workingServers)
}
```
**Algorithm:** Two lists — `untriedServers` (initially all) and `workingServers` (initially empty). When `workingServers.length < concurrency`, pick from `untriedServers` (removing on pick). On FNEW success, add to `workingServers`. On FNEW failure, router is already removed from `untriedServers`; remove from `workingServers` if present. When `untriedServers` is empty, reset it to all non-working routers. Once `workingServers.length >= concurrency`, pick randomly only from `workingServers`.
**Termination condition:** Each data packet tries at most `min(routerCount, 5)` different routers. If all attempts fail, the data packet fails and the upload fails with the last error. Rationale: if 5 out of 12 routers are down, something systemic is wrong and continuing is unlikely to help. Timeouts count as failures — the timed-out router is removed from working and a different router is picked next.
**Key differences from Haskell:**
- No operator concept — just host diversity via random selection
- No database — state tracked in-memory during upload
- FNEW runs concurrently (Haskell is serial) — improves startup time
- FNEW is cheap and retried with router rotation; FPUT retries on same router
**Download changes (also Phase 2):** Default concurrency should be 4 (matching Haskell). Download already groups by router in 3.5. If `replicas[0]` download fails, try `replicas[1]`, `replicas[2]`, etc. (fallback across replicas).
## 4. Implementation Plan
### Phase 1: Error handling and connection resilience
Steps are ordered by dependency and should be implemented one by one.
#### Step 1: Fix cache key (3.6)
- Change cache key to `formatXFTPServer(server)` in `getXFTPServerClient` and `closeXFTPServerClient`
- Add import for `formatXFTPServer`
- Run existing tests to verify no regression
#### Step 2: Typed error detection for padded router errors (3.2 client-side)
- Add `XFTPRetriableError` class
- In `sendXFTPCommand`, detect padded error strings before `decodeTransmission`
- Classify `FRErr` responses as retriable or permanent with human-readable messages
- Run existing tests
#### Step 3: Fetch timeout (3.3)
- Add `TransportConfig` with `timeoutMs`
- Thread config through `createTransport``connectXFTP` → command wrappers
- Add `AbortController` to browser `fetch()` and `setTimeout` to Node.js HTTP/2
- Add vitest test: timeout triggers after configured duration
- Run existing tests
#### Step 4: Connection state with Promise-based lock and per-router queues (3.5)
- Introduce `ServerConnection` record: `{client: Promise<XFTPClient>, queue: Promise<void>}`
- Replace `XFTPClientAgent.clients: Map<string, XFTPClient>` with `connections: Map<string, ServerConnection>`
- Implement `reconnectClient` — replaces `conn.client` with new promise, preserves queue
- Implement `enqueueCommand` — chains operation onto router's queue
- Implement `removeStaleConnection` — removes entry only if current promise is the failed one
- Auto-cleanup: `p.catch(() => delete)` removes failed connections so next caller starts fresh
- Adapt `closeXFTPServerClient` and `closeXFTPAgent`
- Add vitest tests:
- Concurrent calls to same router produce single connection
- Failed promise is cleaned up, next caller gets fresh connection
#### Step 5: Automatic retry in sendXFTPCommand (3.2)
- Add retry loop with reconnect
- Change `sendXFTPCommand` signature: takes `agent + server` instead of `client`; export it (needed by tests and by agent.ts callers)
- Rename current `sendXFTPCommand``sendXFTPCommandOnce` (private); add padded error detection + FRErr classification (throw `XFTPRetriableError` for SESSION/HANDSHAKE, `XFTPPermanentError` for AUTH/NO_FILE/etc.)
- All command wrappers (`createXFTPChunk`, `uploadXFTPChunk`, etc.) pass agent + server
- Update agent.ts call sites: remove `getXFTPServerClient` calls before command wrappers (in `uploadFile`, `uploadRedirectDescription`, `downloadFileRaw`, `resolveRedirect`, `deleteFile`)
- Max 3 retries for retriable errors, immediate throw for permanent
- On retriable error: call `reconnectClient` and retry. On retriable error exhausted: call `removeStaleConnection` to clean up. On permanent error: throw immediately without touching connection
- Add vitest tests:
- Router started with delay → first attempt fails, retry succeeds
- 3 retries exhausted → error propagates with human-readable message
- Non-retriable error (AUTH) → no retry, immediate failure
#### Step 6: Router-side stale session handling (3.1)
- Add one guard to `Nothing` branch: `sniUsed && not webHello -> throwE SESSION`
- Remove debug `hPutStrLn stderr` lines (all 6 occurrences in dispatch)
- All other branches unchanged
- Run Haskell tests + Playwright tests
#### Step 7: Download with per-router grouping
- Modify `downloadFileRaw` to group data packets by router, sequential within each router (`for` loop), parallel across routers (`Promise.all`)
- Add vitest test: concurrent downloads from different routers run in parallel
#### Step 8: UI error improvements (3.4)
- Temporary errors: auto-retry loop (3 attempts), then show human-readable diagnosis + manual retry button
- Permanent errors: show human-readable error, NO retry button
- Manual retry resumes from last successful data packet (not full restart)
#### Step 9: Remove debug logging
- Remove all `console.log('[DEBUG ...]')` and `hPutStrLn stderr "DEBUG ..."` lines
- Keep `console.error('[XFTP] ...')` error logging
### Phase 2: Multi-router upload
Implement after Phase 1 is complete and tested.
#### Step 10: Multi-router upload with failover (3.7)
- Extend `SentChunk` with `chunkOffset: number` (from ChunkSpec) and `server: XFTPServer` (assigned during allocate) — Stage 2 reads data by offset and groups data packets by router
- Change `uploadFile` signature: takes `allServers: XFTPServer[]` instead of single `server`
- Implement `UploadState` with `untriedServers` and `workingServers`
- Implement `createChunkWithFailover` and `pickServer`: two-list selection (untried → working once enough found), max `min(routerCount, 5)` attempts per data packet
- Allocate stage: concurrent FNEW within concurrency limit (default 4)
- Upload stage: parallel across routers, sequential per router (reuse queue from Step 7)
- Update `web/upload.ts`: pass `getServers()` instead of `pickRandomServer(getServers())`
- Update description building: each data packet references its actual router
- Add vitest tests:
- File split across N routers (verify different routers in description)
- One router down → data packets redistributed to others
- All routers down → error after exhausting 5 attempts per data packet
#### Step 11: Download concurrency and replica fallback
- Change default download concurrency from 1 to 4
- If `replicas[0]` download fails, try `replicas[1]`, `replicas[2]`, etc.
- Uses per-router queues from Step 7
## 5. Testing Plan
### Principle
Prefer low-level vitest tests over Playwright E2E. Each new function gets one focused test. Pure functions tested without mocks; connection management tested with mock `connectXFTP`; router behavior tested with real router. Total: 13 tests across 4 files.
Tests A-C run in browser context (`@vitest/browser` with Chromium headless), configured in `vitest.config.ts`. Test D (integration) requires a separate Node.js vitest config since it uses `node:http2`. Existing `globalSetup.ts` provides a real XFTP router for integration tests.
### Test file A: `test/errors.test.ts` — pure, no router
Tests error classification and padded error detection (Steps 2, 5).
**T1. `isRetriable` classifies errors correctly**
```typescript
// Retriable:
expect(isRetriable(new XFTPRetriableError("SESSION"))).toBe(true)
expect(isRetriable(new XFTPRetriableError("HANDSHAKE"))).toBe(true)
expect(isRetriable(new TypeError("fetch failed"))).toBe(true) // network error
expect(isRetriable(Object.assign(new Error(), {name: "AbortError"}))).toBe(true) // timeout
// Not retriable:
expect(isRetriable(new XFTPPermanentError("AUTH", "..."))).toBe(false)
expect(isRetriable(new XFTPPermanentError("NO_FILE", "..."))).toBe(false)
expect(isRetriable(new XFTPPermanentError("INTERNAL", "..."))).toBe(false)
```
**T2. `categorizeError` produces human-readable messages**
```typescript
// categorizeError receives thrown errors (from sendXFTPCommandOnce or transport)
const e = categorizeError(new XFTPPermanentError("AUTH", "File is invalid, expired, or has been removed"))
expect(e.message).toContain("expired")
// Verify every permanent error type maps to a non-empty human-readable message
for (const errType of ["AUTH", "NO_FILE", "SIZE", "QUOTA", "BLOCKED", "DIGEST", "INTERNAL"]) {
expect(humanReadableMessage({type: errType}).length).toBeGreaterThan(0)
}
// Retriable errors also get human-readable messages after exhaustion
const re = categorizeError(new XFTPRetriableError("SESSION"))
expect(re.message).toContain("expired") // "Session expired, reconnecting..."
```
**T3. Padded error detection extracts error string from padded block**
```typescript
import {blockPad, blockUnpad} from '../src/protocol/transmission.js'
// Simulate router sending padded "SESSION"
const padded = blockPad(new TextEncoder().encode("SESSION"))
const raw = blockUnpad(padded)
expect(raw.length).toBeLessThan(20)
expect(new TextDecoder().decode(raw)).toBe("SESSION")
// Normal transmission block (batch count + large-encoded data) is NOT a short string
const sessionId = new Uint8Array(32) // dummy
const normalBlock = encodeTransmission(sessionId, new Uint8Array(0), new Uint8Array(0), encodePING())
const normalRaw = blockUnpad(normalBlock)
expect(normalRaw.length).toBeGreaterThan(20) // not mistaken for padded error
```
### Test file B: `test/connection.test.ts` — mock connectXFTP, no router
Tests connection management functions (Steps 4, 5). Uses `vi.mock` to replace `connectXFTP` with a controllable promise factory.
**T4. `getXFTPServerClient` coalesces concurrent calls**
```typescript
// Mock connectXFTP to return a deferred promise
const {promise, resolve} = promiseWithResolvers<XFTPClient>()
vi.mocked(connectXFTP).mockReturnValueOnce(promise)
const agent = newXFTPAgent()
const p1 = getXFTPServerClient(agent, server)
const p2 = getXFTPServerClient(agent, server)
expect(p1).toBe(p2) // same promise, single connection
resolve(mockClient)
expect(await p1).toBe(mockClient)
```
**T5. `getXFTPServerClient` auto-cleans failed connections**
```typescript
vi.mocked(connectXFTP).mockReturnValueOnce(Promise.reject(new Error("down")))
const agent = newXFTPAgent()
const p1 = getXFTPServerClient(agent, server)
await expect(p1).rejects.toThrow("down")
// After microtask, entry is removed
await new Promise(r => setTimeout(r, 0))
expect(agent.connections.has(formatXFTPServer(server))).toBe(false)
// Next call creates fresh connection
vi.mocked(connectXFTP).mockReturnValueOnce(Promise.resolve(mockClient))
const p2 = getXFTPServerClient(agent, server)
expect(p2).not.toBe(p1)
```
**T6. `removeStaleConnection` respects promise identity**
```typescript
const agent = newXFTPAgent()
const p1 = Promise.resolve(mockClient)
agent.connections.set(key, {client: p1, queue: Promise.resolve()})
// Replace with reconnect
const p2 = Promise.resolve(mockClient2)
agent.connections.set(key, {client: p2, queue: Promise.resolve()})
// removeStaleConnection with old promise does NOT remove new entry
removeStaleConnection(agent, server, p1)
expect(agent.connections.has(key)).toBe(true)
expect(agent.connections.get(key)!.client).toBe(p2)
// removeStaleConnection with current promise removes it
removeStaleConnection(agent, server, p2)
expect(agent.connections.has(key)).toBe(false)
```
**T7. `reconnectClient` replaces promise but preserves queue**
```typescript
const agent = newXFTPAgent()
const origQueue = Promise.resolve()
agent.connections.set(key, {client: Promise.resolve(mockClient), queue: origQueue})
vi.mocked(connectXFTP).mockReturnValueOnce(Promise.resolve(mockClient2))
reconnectClient(agent, server)
const conn = agent.connections.get(key)!
expect(await conn.client).toBe(mockClient2) // new client
expect(conn.queue).toBe(origQueue) // queue preserved
```
**T8. Retry loop: retriable error triggers reconnect, permanent error does not**
Mock approach: `vi.mock('../src/client.js')` to mock `connectXFTP` (exported). `reconnectClient` is not exported — its behavior is controlled indirectly via `connectXFTP` mock (it calls `connectXFTP` internally). Verify retry count via `connectXFTP` call count. Note: vitest module mocking may need adjustment depending on ESM transform behavior — if intra-module calls bypass the mock, extract `connectXFTP` to a separate module or use dependency injection for testing.
```typescript
// Script: first connectXFTP returns client whose post throws retriable,
// second connectXFTP (from reconnect) returns client whose post succeeds
vi.mocked(connectXFTP)
.mockResolvedValueOnce({
...mockClient,
transport: { post: async () => { throw new XFTPRetriableError("SESSION") }, close: () => {} }
})
.mockResolvedValueOnce({
...mockClient,
transport: { post: async () => okResponseBlock, close: () => {} }
})
const agent = newXFTPAgent()
const result = await sendXFTPCommand(agent, server, dummyKey, dummyId, encodePING())
expect(result.response.type).toBe("FROk")
expect(vi.mocked(connectXFTP)).toHaveBeenCalledTimes(2) // initial + 1 reconnect
// Reset — all 3 retries exhausted: connectXFTP called 3 times (initial + 2 reconnects)
vi.mocked(connectXFTP).mockClear()
vi.mocked(connectXFTP).mockResolvedValue({
...mockClient,
transport: { post: async () => { throw new XFTPRetriableError("SESSION") }, close: () => {} }
})
const agent2 = newXFTPAgent()
await expect(sendXFTPCommand(agent2, server, dummyKey, dummyId, encodePING()))
.rejects.toThrow(/reconnecting|expired/)
expect(vi.mocked(connectXFTP)).toHaveBeenCalledTimes(3) // initial + 2 reconnects
// Reset — permanent error: connectXFTP called once (initial only, no reconnect)
vi.mocked(connectXFTP).mockClear()
vi.mocked(connectXFTP).mockResolvedValue({
...mockClient,
transport: { post: async () => authErrorBlock, close: () => {} }
})
const agent3 = newXFTPAgent()
await expect(sendXFTPCommand(agent3, server, dummyKey, dummyId, encodePING()))
.rejects.toThrow(/expired/)
expect(vi.mocked(connectXFTP)).toHaveBeenCalledTimes(1) // initial only, no reconnect
```
### Test file C: `test/server-selection.test.ts` — pure, no router
Tests `pickServer` state machine (Step 10). Determinism: seed `Math.random` or test invariants not specific picks.
**T9. `pickServer` picks from untried when working < concurrency**
```typescript
const servers = [s1, s2, s3, s4, s5]
const state: UploadState = {untriedServers: [...servers], workingServers: []}
const picked = pickServer(servers, state, 4)
// picked is from untried, and was removed from untried
expect(state.untriedServers.length).toBe(4)
expect(state.untriedServers).not.toContainEqual(picked)
```
**T10. `pickServer` picks only from working when working >= concurrency**
```typescript
const state: UploadState = {
untriedServers: [s5], // still has untried
workingServers: [s1, s2, s3, s4]
}
const picked = pickServer(servers, state, 4)
// Must pick from working, NOT from untried
expect([s1, s2, s3, s4]).toContainEqual(picked)
expect(state.untriedServers.length).toBe(1) // untried unchanged
```
**T11. `pickServer` resets untried when exhausted**
```typescript
const state: UploadState = {
untriedServers: [], // all tried
workingServers: [s1, s2] // only 2 working, concurrency=4
}
const picked = pickServer(servers, state, 4)
// Should have reset untried to non-working routers and picked from them
expect([s3, s4, s5]).toContainEqual(picked)
expect(state.untriedServers.length).toBe(2) // 3 non-working minus 1 picked
```
### Test file D: `test/integration.test.ts` — real router, Node.js mode
Requires separate vitest config with `browser: {enabled: false}` since these tests use `node:http2` directly. Alternatively, add `test/vitest.node.config.ts` that includes only `test/integration.test.ts` and runs in Node.js.
**T12. Stale session returns padded SESSION error (requires Step 6)**
```typescript
import http2 from 'node:http2'
// Connect and handshake normally via the client
const client = await connectXFTP(server)
// Create a raw HTTP/2 session (new TLS SessionId, no handshake state on router)
const session = http2.connect(client.baseUrl, {rejectUnauthorized: false})
// Build a dummy command block using the old client's sessionId.
// Content doesn't matter — router detects stale session before parsing command.
const dummyKey = new Uint8Array(64) // Ed25519 private key (dummy)
const dummyId = new Uint8Array(24) // entity ID (dummy)
const cmdBlock = encodeAuthTransmission(client.sessionId, new Uint8Array(0), dummyId, encodePING(), dummyKey)
const resp = await new Promise<Uint8Array>((resolve, reject) => {
const req = session.request({":method": "POST", ":path": "/"})
const chunks: Buffer[] = []
req.on("data", (c: Buffer) => chunks.push(c))
req.on("end", () => resolve(new Uint8Array(Buffer.concat(chunks))))
req.on("error", reject)
req.end(Buffer.from(cmdBlock))
})
// Router should return padded "SESSION" (not crash, not "HANDSHAKE")
const raw = blockUnpad(resp.subarray(0, XFTP_BLOCK_SIZE))
expect(new TextDecoder().decode(raw)).toBe("SESSION")
session.close()
closeXFTP(client)
```
**T13. Fetch timeout fires within configured duration**
```typescript
// connectXFTP with 1ms timeout — handshake requires multiple round trips,
// so even on localhost it will exceed 1ms and trigger abort
await expect(
connectXFTP(server, {timeoutMs: 1})
).rejects.toThrow(/abort|timeout/i)
```
### What existing tests already cover (no new tests needed)
| Behavior | Covered by |
|----------|-----------|
| Cache key fix (Step 1) | Existing round-trip test — uses `formatXFTPServer` after refactor |
| Basic upload/download | 24 Playwright tests + 1 vitest browser test |
| File size limits, unicode filenames | Playwright edge case tests |
| Router startup/teardown | `globalSetup.ts` / `globalTeardown.ts` |
| Handshake + identity verification | `connectXFTP` in existing round-trip test |
### Test ordering
Tests must be added alongside their implementation step:
- **Step 2**: Add T1, T2, T3 (test/errors.test.ts)
- **Step 3**: Add T13 (test/integration.test.ts) — requires Node.js vitest config
- **Step 4**: Add T4, T5, T6, T7 (test/connection.test.ts)
- **Step 5**: Add T8 (test/connection.test.ts)
- **Step 6**: Add T12 (test/integration.test.ts) — requires router change + Node.js vitest config
- **Step 10**: Add T9, T10, T11 (test/server-selection.test.ts)
## 6. Context for Implementation Sessions
### Files to re-read on session start
**TypeScript (xftp-web/src/):**
- `client.ts``XFTPClient`, `XFTPClientAgent`, `getXFTPServerClient`, `closeXFTPServerClient`, `connectXFTP`, `sendXFTPCommand`, `createBrowserTransport`, `createNodeTransport`, all command wrappers
- `agent.ts``uploadFile`, `downloadFileRaw`, `downloadFile`, `resolveRedirect`, `encryptFileForUpload`
- `protocol/transmission.ts``encodeAuthTransmission`, `decodeTransmission`, `blockPad`, `blockUnpad`
- `protocol/commands.ts``XFTPErrorType`, `FileResponse`, `decodeResponse`, `decodeXFTPError`
- `protocol/handshake.ts``decodeServerHandshake` (padded error detection heuristic)
- `protocol/address.ts``XFTPServer`, `parseXFTPServer`, `formatXFTPServer`
- `web/upload.ts` — UI error handling, retry button
- `web/download.ts` — UI error handling, retry button
- `web/servers.ts``getServers`, `pickRandomServer`
**TypeScript (xftp-web/test/):**
- `browser.test.ts` — vitest Node.js test template (uses real Haskell router)
- `globalSetup.ts` — router startup, config generation, port file
- `page.spec.ts` — Playwright page tests
**Haskell (reference for multi-router):**
- `src/Simplex/FileTransfer/Agent.hs``createChunk` (lines 457-486, allocate stage), `runXFTPSndPrepareWorker` (lines 391-430, serial allocate in Haskell), `runXFTPSndWorker` (lines 494-548, per-router upload worker)
- `src/Simplex/Messaging/Agent/Client.hs``getNextServer_` (lines 2335-2350), `withNextSrv` (lines 2366-2385), `pickServer` (lines 2309-2314)
**Haskell (router):**
- `src/Simplex/FileTransfer/Server.hs``xftpServerHandshakeV1` (lines 165-244), `processRequest` (lines 403-435)
- `src/Simplex/Messaging/Protocol.hs``tDecodeServer` (lines 2239-2265) — sessionId verification at line 2242
### Key design constraints
1. `tDecodeServer` (Protocol.hs:2242) verifies `sessId == sessionId` — commands signed with old sessionId WILL fail on new connection
2. Router generates per-session DH key in `processHello` (Server.hs:207) — cannot be shared across sessions
3. `fetch()` provides zero control over HTTP/2 connection reuse — browser decides
4. `xftp-web-hello` header is only checked in dispatch (Server.hs:192), NOT inside `processHello`
5. Handshake-phase errors are raw padded strings; command-phase errors are proper ERR transmissions
6. Ed25519 signature verification (`TASignature` path, Protocol.hs:1314) does NOT use `thAuth` — but SMP will
7. Reconnect must re-handshake to get new sessionId AND new router DH key
8. The new `throwE SESSION` guard (Step 6) sends a raw padded "SESSION" string — no sessionId framing. Client detects this via padded error heuristic (section 3.2), not via sessionId mismatch
9. FNEW is cheap (creates data packet record on router) — retry with different router on failure
10. FPUT retries on same router (data packet replica already exists there) — close connection + backoff
## 7. Plan Maintenance
This plan must be updated as implementation proceeds:
- Mark completed steps with date
- Record any deviations from the plan with rationale
- Add new issues discovered during implementation
- Update file references if code moves
@@ -0,0 +1,327 @@
# CLI-Web Link Compatibility
## Problem
CLI and web clients are isolated: CLI outputs `.xftp` description files, web outputs
`https://host/#<encoded>` links. A file uploaded via one cannot be downloaded via the other.
## Solution Summary
Make CLI produce and consume web-compatible links so that:
- CLI `send` always outputs a web link (in addition to `.xftp` files)
- CLI `recv` accepts a web link URL as input (alternative to `.xftp` file path)
- Browser can download files uploaded by CLI and vice versa
The web page host is derived from the XFTP router address - the router that hosts the file
also hosts the download page. Making XFTP routers actually serve the web page is a separate
concern (not covered here), but the link format anticipates it.
The YAML file description format is already identical between CLI and web.
The only gap is the URI encoding layer: DEFLATE-raw compression + base64url + URL structure.
## Current State
### Web link format
```
https://<xftp-server-host>/#<base64url(deflateRaw(YAML))>
```
Encoding chain (agent.ts:64-68):
1. `encodeFileDescription(fd)` -> YAML string
2. `TextEncoder.encode(yaml)` -> bytes
3. `pako.deflateRaw(bytes)` -> compressed
4. `base64urlEncode(compressed)` -> URI fragment (no `#`)
For multi-packet files exceeding ~400 chars in URI, a redirect description is uploaded:
the real file description is encrypted, uploaded as a separate XFTP file, and a smaller
"redirect" description (pointing to it) is put in the URI.
### CLI file format
```
xftp send FILE -> writes rcv1.xftp (raw YAML), snd.xftp.private
xftp recv FILE.xftp -> reads raw YAML from file
```
No URI support. No compression. No redirect descriptions.
### Existing Haskell `FileDescriptionURI`
`Description.hs:243-266` defines a `simplex:/file#/?desc=<URL-encoded raw YAML>` format.
This is the SimpleX Chat app format - NOT the web page format. It uses URL-encoded raw YAML
(no DEFLATE compression), and has a different URL structure.
## Detailed Tech Design
### 1. File Header (Filename) Compatibility
The filename is carried **inside the encrypted file data**, not in the file description YAML.
Both CLI and web use the same `FileHeader` structure and binary encoding - full interop.
#### FileHeader type
Haskell (`Types.hs:36-46`):
```haskell
data FileHeader = FileHeader { fileName :: Text, fileExtra :: Maybe Text }
instance Encoding FileHeader where
smpEncode FileHeader {fileName, fileExtra} = smpEncode (fileName, fileExtra)
```
TypeScript (`crypto/file.ts:11-24`):
```typescript
interface FileHeader { fileName: string; fileExtra: string | null }
function encodeFileHeader(hdr: FileHeader): Uint8Array {
return concatBytes(encodeString(hdr.fileName), encodeMaybe(encodeString, hdr.fileExtra))
}
```
Both produce identical binary: `[1-byte UTF-8 length][fileName bytes]['0']` (for null fileExtra).
Max filename: 255 UTF-8 bytes (1-byte length prefix).
#### Encrypted file structure
Both CLI and web produce the same encrypted stream:
```
XSalsa20-Poly1305 encrypted:
[8-byte Int64 fileSize] [FileHeader] [file content] ['#' padding]
+ [16-byte auth tag]
Where fileSize = len(FileHeader) + len(file content)
```
The 8-byte length prefix and padding are handled identically:
- Haskell: `Crypto.hs:43-56` (`encryptFile`) / `Crypto.hs:81-87` (`decryptFirstChunk`)
- TypeScript: `crypto/file.ts:51-70` (`encryptFile`) / `crypto/file.ts:81-94` (`decryptChunks`)
On decryption, `unPadLazy`/`splitLen` strips the 8-byte length prefix, then `parseFileHeader`
extracts the filename from the remaining decrypted bytes (up to 1024 bytes examined, both sides).
#### CLI upload: sets real filename (ok)
`Client/Main.hs:246-247,273`:
```haskell
let (_, fileNameStr) = splitFileName filePath
fileName = T.pack fileNameStr
...
fileHdr = smpEncode FileHeader {fileName, fileExtra = Nothing}
```
Extracts the actual filename from the path and embeds it in the encrypted header.
#### CLI download: uses filename from header (ok)
`Crypto.hs:62-66` (single data packet) / `Crypto.hs:72-74` (multi-packet):
```haskell
(FileHeader {fileName}, rest) <- parseFileHeader decryptedContent
destFile <- withExceptT FTCEFileIOError $ getDestFile fileName
```
`Client/Main.hs:435-441` (`getFilePath`):
- If output dir specified: saves to `<dir>/<fileName>`
- If no dir: saves to `~/Downloads/<fileName>`
The filename from the decrypted header determines the output file name.
#### Web upload: sets real filename (ok)
`upload.ts:121` -> `agent.ts:86`:
```typescript
const fileHdr = encodeFileHeader({fileName, fileExtra: null})
```
Where `fileName` comes from `file.name` (browser File API).
#### Web download: uses filename from header (ok)
`download.ts:97,102`:
```typescript
const fileName = sanitizeFileName(header.fileName)
a.download = encodeURIComponent(fileName)
```
The web client additionally sanitizes the filename (strips path separators, control chars,
bidi overrides, limits to 255 chars).
#### Web redirect description: empty filename (correct)
`agent.ts:193`: `encryptFileForUpload(yamlBytes, "")` - redirect descriptions use empty filename
because they are internal artifacts, not user files. This is handled correctly on both sides:
the redirect content is decrypted and parsed as YAML, not saved as a file.
#### Cross-client interop: fully compatible (ok)
| Scenario | Filename flow | Status |
|----------|--------------|--------|
| CLI upload -> CLI download | `splitFileName` -> header -> `getDestFile` | Works |
| Web upload -> Web download | `File.name` -> header -> `sanitizeFileName` | Works |
| CLI upload -> Web download | `splitFileName` -> header -> `sanitizeFileName` | **Compatible** |
| Web upload -> CLI download | `File.name` -> header -> `getDestFile` | **Compatible** |
The binary encoding is identical (smpEncode). No changes needed for filename interop.
The CLI should consider adding filename sanitization similar to the web client for safety.
### 2. Web Link Host Derivation
The web page URL domain comes from the XFTP router address, not from a CLI flag:
- **Non-redirected description**: use the router host of the first data packet's first replica.
E.g., `xftp://abc=@xftp1.simplex.im` -> `https://xftp1.simplex.im/#<encoded>`
- **Redirected description**: use the router host of the redirect data packet (the outer description's
data packet that stores the encrypted inner description).
The router address format is `xftp://<keyhash>@<host>[,<host2>,...][:<port>]`.
The web link uses `https://<host>` (port 443 implied).
This means the CLI does not need a `--web-url` flag - the router address fully determines
the link. The XFTP router serving the web page is a separate deployment concern.
### 3. Web URI Encoding/Decoding in Haskell
Add two functions (new module or in `Description.hs`):
```haskell
-- Encode file description as web URI fragment (no leading #)
encodeWebURI :: FileDescription 'FRecipient -> ByteString
-- 1. Y.encode . encodeFileDescription -> YAML bytes
-- 2. deflateRaw (raw DEFLATE, no zlib/gzip header) via zlib package
-- 3. base64url encode (with padding, matching Data.ByteString.Base64.URL)
-- Decode web URI fragment (no leading #) to file description
decodeWebURI :: ByteString -> Either String (ValidFileDescription 'FRecipient)
-- 1. base64url decode
-- 2. inflateRaw (raw DEFLATE decompress)
-- 3. Y.decodeEither' -> YAMLFileDescription -> FileDescription
-- 4. validateFileDescription
-- Build full web link from file description
-- Extracts router host from first data packet replica (or redirect data packet)
fileWebLink :: FileDescription 'FRecipient -> (String, ByteString)
-- Returns (webHost, uriFragment)
-- Caller assembles: "https://" <> webHost <> "/#" <> uriFragment
```
**Dependency**: Add `zlib` to `simplexmq.cabal` (for raw DEFLATE).
The codebase already has `zstd` for message compression - `zlib` is standard and small.
The `zlib` Haskell package provides `Codec.Compression.Zlib.Raw` for raw DEFLATE
(no header/trailer), matching `pako.deflateRaw()` / `pako.inflateRaw()`.
### 4. Redirect Description Support
The CLI currently does NOT create redirect descriptions. For single-router single-recipient
uploads, most file descriptions fit in a reasonable URI even for multi-packet files. But for
large files (many data packets x long router hostnames), the URI can exceed practical limits.
**Approach**: Match the web client threshold.
- After encoding the URI, if `length > 400` and data packets > 1, upload a redirect description.
- The redirect upload uses the same XFTP upload flow: encrypt YAML -> upload as file -> create
outer description pointing to it.
- This matches `agent.ts:152-155` exactly.
- The redirect data packet's router becomes the web link host.
For CLI download from a redirect URI, the existing `cliReceiveFile` needs extension:
- After decoding the file description, check `redirect` field.
- If present: download and decrypt the redirect data packets first to get the inner description,
then download the actual file using the inner description.
- The web client already does this (`resolveRedirect` in agent.ts:320-346).
### 5. CLI Command Changes
#### `xftp send` - always output web link
```
xftp send FILE [DIR] [-n COUNT] [-s SERVERS]
```
- Upload file as usual
- Generate web link: `https://<server-host>/#<encodeWebURI(rcvDescription)>`
- If URI exceeds threshold, upload redirect description first
- Print web link to stdout (in addition to `.xftp` file paths)
- Only generates link for the first recipient (web links are single-recipient)
**Output change**:
```
Sender file description: ./file.xftp/snd.xftp.private
Pass file descriptions to the recipient(s):
./file.xftp/rcv1.xftp
Web link:
https://xftp1.simplex.im/#eJy0VduO2zYQ...
```
#### `xftp recv` - accept URL as input
```
xftp recv <FILE.xftp | URL> [DIR]
```
- If input starts with `http://` or `https://`, extract hash fragment after `#`
- Decode: base64url -> inflateRaw -> YAML -> FileDescription
- Resolve redirect if present
- Download and decrypt as usual
The URL must be quoted on the command line (`"https://...#..."`) because `#` is a shell
comment character when unquoted.
Implementation: modify `receiveP` parser to accept URL, add `decodeWebURI` path in
`cliReceiveFile` alongside existing `getFileDescription'`.
### 6. YAML Format Compatibility
Already identical. The web `description.ts` explicitly matches Haskell `Data.Yaml` output:
- Same field names (alphabetical key order)
- Same base64url encoding for binary fields (with `=` padding)
- Same server replica colon-delimited format: `chunkNo:replicaId:replicaKey[:digest][:chunkSize]`
- Same size encoding (`kb`/`mb`/`gb` suffixes)
- Same redirect structure
**Verification**: The Playwright test suite already tests upload->download round-trips.
Adding a cross-client test (CLI upload -> web download, or web upload -> CLI download) would
validate interop end-to-end.
### 7. Router Compatibility
No router changes needed. Both clients use the same XFTP protocol (FGET, FPUT, FNEW, FACK, FDEL).
The web client adds `xftp-web-hello: 1` header for the hello handshake, but the actual file
operations are identical wire-format.
The only consideration: CLI uses native HTTP/2 (via `http2` Haskell package), web uses
browser `fetch()` API over HTTP/2. Both produce identical XFTP protocol frames.
**Note**: Making XFTP routers actually serve the web download page at `https://<host>/` is a
separate deployment/infrastructure task. This plan only establishes the link format convention
so that links are ready to work once servers serve the page.
## Implementation Plan
### Phase 1: Web URI codec in Haskell
1. Add `zlib` dependency to `simplexmq.cabal`
2. Add `encodeWebURI` / `decodeWebURI` / `fileWebLink` to `Simplex.FileTransfer.Description`
(or a new `Simplex.FileTransfer.Description.WebURI` module)
3. `fileWebLink` extracts host from first data packet's first replica router address
4. Add unit tests: encode a known FileDescription, verify output matches web client encoding
5. Add round-trip test: encode -> decode -> compare
### Phase 2: CLI `recv` accepts URL
1. Modify `ReceiveOptions` to accept `Either FilePath WebURL` for `fileDescription`
2. In `cliReceiveFile`: if URL, extract fragment after `#`, call `decodeWebURI`
3. Add redirect resolution: if `redirect /= Nothing`, download redirect data packets,
decrypt, parse inner description, then proceed with download
4. Test: upload via web page -> copy link -> `xftp recv <link>`
### Phase 3: CLI `send` outputs web link
1. After upload, call `fileWebLink` to get (host, fragment)
2. If fragment exceeds threshold, upload redirect description first, rebuild link
3. Print `https://<host>/#<fragment>` to stdout
4. Test: `xftp send FILE` -> open link in browser -> download
### Phase 4: Cross-client integration test
1. Add test: CLI send -> extract link from stdout -> Playwright browser download -> verify
2. Add test: Playwright browser upload -> extract link -> CLI recv -> verify
3. These can be shell-script or Haskell test-suite tests that spawn both clients
@@ -1,3 +1,10 @@
---
Proposed: 2022-07-22
Implemented: ~2022-08
Standardized: 2026-03-09
Protocol: simplex-messaging
---
# Accessing SMP servers via Tor
## Problem
@@ -1,3 +1,12 @@
---
Proposed: 2021-01-26
Implemented: ~2022
Standardized: 2026-03-09
Protocol: simplex-messaging v1, evolved through v7
---
> **Implementation note:** All cryptographic primitives changed from this proposal. Transport: TLS 1.2/1.3 replaced the custom RSA handshake. E2E: Double ratchet with AES-GCM replaced per-message RSA-OAEP encryption. Auth: Ed25519/X25519 DH-based authenticated encryption (SMP v7) replaced RSA-PSS signatures. The transmission format (signature CRLF signed) was implemented as proposed.
# SMP agent: cryptography
3 main directions of work to enable basic level of security for communication via SMP agents and servers at the current stage of the project:
@@ -1,3 +1,10 @@
---
Proposed: 2022-06-13
Implemented: ~2022-06
Standardized: 2026-03-09
Protocol: agent-protocol
---
# DB access and processing messages for iOS notification service extension
## Problem

Some files were not shown because too many files have changed in this diff Show More