Compare commits

...
42 Commits
Author SHA1 Message Date
Evgeny Poberezkin 5884d2cf03 ntf server: use multiple APNS clients for each provider 2026-08-10 16:59:20 +01:00
Evgeny Poberezkin 27a37387be 7.0.1.0 2026-07-31 15:20:41 +01:00
sh d65d790a20 ntf server: shard push workers per notification token (#1840)
* ntf server: shard push workers per notification token

* ntf server: inline push worker shard calculation
2026-07-31 15:19:43 +01:00
shandEvgeny Poberezkin 7d0820dd44 smp server: do not create messaging queues in SMP proxy to prevent deadlock in processing (#1839)
* smp: fix proxy message queue memory leak

* smp server: do not create messaging queues in SMP proxy to prevent deadlock in processing

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2026-07-30 16:33:27 +01:00
Evgeny Poberezkin efaad8e734 7.0.0.6 2026-07-25 11:58:17 +01:00
Evgeny 1b4dcfe63e agent: refactor type for private keys in double ratchet (#1830) 2026-07-19 18:27:55 +01:00
Evgeny Poberezkin f7e8ed52bf 7.0.0.5 2026-07-18 06:35:31 +01:00
sh 399c5fe8c6 agent: pass optional SMP server to prepareConnectionLink (#1771) 2026-07-14 09:04:19 +01:00
sh 43e46dd8cc smp-server: fix service subscription memory leak (#1827)
Service subscription counters (totalServiceSubs, serviceSubsCount,
ntfServiceSubsCount) are TVar (Int64, IdsHash). modifyTVar' only forces
the pair to WHNF, so `n +/- n'` and `idsHash <> idsHash'` stay
unevaluated and accumulate an unbounded thunk chain under subscription
and delivery churn (the IdsHash chain also retains a bytestring per
update) - a space leak proportional to the number of updates.

Force both components in addServiceSubs/subtractServiceSubs. Verified
with the load bench: svc churn drops from +5.4 KiB/iter (linear) to flat.
2026-07-10 09:00:40 +01:00
Evgeny Poberezkin 551de8039f 7.0.0.4 2026-07-06 07:34:52 +01:00
EvgenyandEvgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com> a45d764eaa agent: fix rare race conditions in async API (#1792)
* agent: fix rare race conditions in async API

* split async accept too

* fix, reduce diff

* composition

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-07-03 15:24:43 +01:00
Evgeny Poberezkin 836254a4c6 types: rename name types 2026-07-02 12:58:43 +01:00
EvgenyandEvgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com> 93925b257c types: instance for contact connection type (#1822)
Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-07-01 17:37:45 +01:00
Evgeny Poberezkin 6ef38a6ee7 7.0.0.3 2026-06-30 23:24:49 +01:00
shEvgenyEvgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
209f7826cb smp-server: support namespaces (#1784)
* smp-server: namespaces resolver scaffolding

* smp-server: Names resolver hardening + cleanup

* smp-server: fuse parallel dispatchers

* smp-server: JSON wire format for NameRecord + Names.hs restructure

* smp-server: redact RpcAuth in Show

* smp-server: JSON wire fixups + spec rewrite + small cleanups

* plan: prepend implementation-diverged banner

* move SimplexName into shared module

* smp-server: name + contract whitelist on RSLV

* smp-server: address audit findings (canonical JSON, INI guards, SSRF, TLD case, shutdown)

* smp-server: round 2 audit fixes (label case, response cap, ipv6 link-local)

* smp-server: round 3 audit fixes (SSRF coverage, drop noop closeManager, CSV order)

* smp-server: round 4 audit fixes (0X-hex host, expanded IPv6 forms, pingEndpoint timeout)

* smp-server: hardcode TldRegistries (drop registry_tld_* INI keys)

* smp-server: round 6 audit fixes (IPv6 SSRF, redirects, ASCII labels)

- Reject IPv6 aliases of 169.254.169.254 (IPv4-compatible / IPv4-mapped /
  6to4 / NAT64) via numeric range check on parsed IPv6.
- Disable HTTP redirects on the Eth RPC request.
- Restrict SimplexName labels to ASCII (Cyrillic/Greek/full-width otherwise
  hash to different on-chain records and diverge from UTS-46 registrars).
- pingEndpoint: only JsonRpcErr means "reachable"; transport/decode failures
  fail startup. boundedIniInt: readMaybe over partial read.
- Add 127.0.0.0/8 and 0.0.0.0 to isLoopback.
- Replace hand-rolled hex helpers with Data.ByteArray.Encoding; raise
  managerConnCount to match rpcMaxConcurrency; hex Show for NameOwner.
- Fuse parallel http/https when into unless+case; drop reverse/re-reverse
  in mkDomain TLDWeb; first AbiInvariantViolated; Nothing <$ decodeAddress;
  forM_ (eitherToMaybe ...); >>= chain in NameOwner FromJSON.
- Drop dead imports/exports/pragmas and two restating comments.
- Tests: factor unsafeOwner/unsafeLink, addr1/2/3, testNamesConfig; add
  non-ASCII label rejection coverage.

* namespace: bound parser input to 253 bytes (DoS defense)

The bare-name fallback and bareDomain parser would otherwise consume
arbitrarily many non-space bytes via takeWhile1 before any validation
or length check. A crafted multi-megabyte token would be decoded as
UTF-8 and re-parsed in full before being rejected.

Introduce `boundedNonSpace` (scan with 253-byte cap) at the two
takeWhile1 sites. Inputs longer than 253 bytes leave residue that
parseOnly's implicit endOfInput rejects, so the parser fails fast
without ever allocating the full input.

The bound is the DNS full-domain limit, chosen for being a familiar
ceiling generous enough to cover any realistic SimpleX name (longest
plausible @user.subdomain.simplex stays well under 100 bytes). No
per-label cap — SimpleX names don't go through DNS label resolution
and there's no semantic reason to constrain individual labels.

* namespace: switch to Python HTTP resolver + agent plumbing (#1796)

* namespace: relax resolver_endpoint validation (path prefix, http without auth)

validateUrl gains two operator-friendly relaxations and a regression test:

- Allow a path prefix (e.g. https://gw.example.com:443/snrc) for a resolver
  behind a reverse-proxy sub-path; /resolve/<name> and /health are appended
  (HttpResolver already strips one trailing slash, so root and sub-path
  behave identically). Query/fragment/userinfo stay rejected.

- Off-loopback, reject only http WITH resolver_auth (the Authorization header
  would travel in cleartext). http without auth is now allowed (no secret to
  leak; resolver data is public — also lets dev setups reach a host resolver
  via http://host.docker.internal). https is always allowed, with or without
  auth. Plain http has no response integrity; intended for trusted/local
  networks only.

Exports validateUrl and adds validateUrlSpec (11 cases) to SMPNamesTests.

* namespace: NameRecord links as arrays (multi-link, cap 5)

* namespace: distinct RSLV error responses

RSLV collapsed every non-hit (no resolver, malformed name, not found,
backing-store failure) to ERR AUTH, so a client iterating its configured
servers could not tell "this router has no resolver, try the next" from
"name not registered, stop", and a transient backend error read as an
authoritative miss.

Names capability is runtime config, orthogonal to the linear SMP version
(a future v21 router without [NAMES] must still advertise v21), so it is
signalled by a command-time error like allowSMPProxy, not by the version
range:

  no resolver configured -> ERR CMD PROHIBITED  (client skips, tries next)
  backing-store failure   -> ERR INTERNAL        (transient: retry/surface)
  not found / malformed   -> ERR AUTH            (authoritative "no such name")

Update the protocol spec error table and add agent tests for the
no-resolver (CMD PROHIBITED) and backend-failure (INTERNAL) paths.

* refactor(names): server role + one error type

Addresses epoberezkin's review (PR #1784). Name resolution becomes a
server role like proxy; the agent owns resolution + server selection;
one error type flows through the whole stack.

- ServerRoles gains `names`; UserServers gains `nameSrvs` (opt-in list);
  resolveSimplexName drops the explicit server arg and picks a
  names-capable server via getNextServer.
- RSLV carries SimplexNameDomain (was RslvRequest): no JSON on the wire,
  contract dropped, name validated at parse (invalid -> CMD SYNTAX).
- Version check moves from the encoder to Client.hs (no ERR to server).
- ErrorType.NAME {nameErr :: NameErrorType} (+ AgentErrorType.NAME),
  wire- and JSON-encoded; resolver errors surface with diagnostics.
  Success response renamed NAME -> RNAME to free the collision.
- NameOwner -> EthAddress (record selector); NameRecord derives FromJSON
  and gains field-ordered Encoding; per-field caps removed.
- Remove newEnvWithNames / runSMPServerBlockingWithNames test seams;
  stub resolver folded into ServerConfig.namesResolverCall_.

* test(server): update stats backup line count

NameResolverStatsData adds 6 lines to the server stats backup (the
"rslvStats:" header plus the reqs/succ/notFound/resolverErrs/disabled
fields), so testRestoreMessages' expected stats-backup line count is
95 -> 101.

* feat(names): public-namespace resolution via RSLV/RNAME

SNRC names resolver role: RSLV command -> HTTP resolver -> RNAME record.
Agent owns server selection (ServerRoles.names); NAME error family; async,
concurrency-bounded resolution; length-prefixed extensible wire; spec.

* remove comments

Co-authored-by: Evgeny <evgeny@poberezkin.com>

* simplify

* move tests name

* simplify: text addresses, Tail JSON, drop admitRslv

* fix

* remove spaghetti

* reduce diff

* async again, refactor

* different threads limit for name resolutions

* remove comment

* FromField instance for SimplexNameInfo

* remove comments

* unStrJSON

* add sameConnShortLink

* remove scheme prefix

* remove unused import

* remove connecttarget tests

* remove comment

* comment

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-06-30 22:54:55 +01:00
sh be58967a86 smp-server: fix subscriptions memory leak (#1820)
* docs: plan for service subscriptions memory leak

* fix: remove leaked service delivery subscriptions

The CSAEndServiceSub handler decremented subscription counters but did
not remove the per-queue delivery Sub from the service client's
subscriptions map. Over queue churn a long-lived service connection
accumulated orphaned Sub entries until disconnect, leaking memory.

Mirror CSAEndSub via endServiceQueueSub (reusing endSub) so the entry
is removed and its delivery thread cancelled.

Add a regression test with white-box access to the server Env via
runSMPServerBlocking_; verified failing without the fix.

* test: remove service subs leak regression test

Remove testServiceSubsRemovedOnQueueDelete and the test-only Env
exposure (runSMPServerBlocking_, withSmpServerConfigEnvOn,
serviceSubsMapSize), leaving only the server fix.

* refactor: simplify unsubPrev with applicative

Express the cancel-if-present logic as sequence_ (unsub_ <*> s_).
2026-06-30 15:38:45 +00:00
shandEvgeny Poberezkin c9ebf72e80 smp: fix proxy reconnection to relay after restart (#1806)
* tests: add SMP proxy relay reconnection tests

Reproduces the proxy failing to reconnect to a destination relay when the
sender disconnects mid-connection (empty session var left in smpClients).

* fix: bracket session var creation to drop it on interrupt

getSessVar inserts an empty session var that the connect path then fills with
putTMVar. If the connecting thread is killed by an async exception before that
fill (a proxy worker on client disconnect, an agent worker on cancel), the empty
var was left in the map forever and every later request for that server blocked
on it until timing out (permanent PCEResponseTimeout).

Wrap get-or-create with withGetSessVar (bracketOnError) at the call sites, so the
cleanup is established where the var is created and covers the whole connect: on
interrupt before fill the still-empty var is dropped and the next request
reconnects. This closes the window between getSessVar and the fill that a handler
installed inside the connect function cannot cover.

* test: cover session var leak on interrupted connect

UtilTests: tryAllErrors rethrows ThreadKilled/StackOverflow (the mechanism
that skips putTMVar). SMPProxyTests: agent client reconnection after a
cancelled connect, plus a control proving the stalling relay alone does not
cause the failure; refine the relay reconnection tests.

* refactor

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2026-06-29 10:49:00 +00:00
sh 2dff11a808 resolver: cleanup (#1817)
* resolver: cleanup

* resolver: update .testing registry address
2026-06-23 16:30:44 +01:00
Evgeny Poberezkin 98391fd677 7.0.0.2 2026-06-21 13:28:46 +01:00
Evgeny Poberezkin d32a25c988 Merge branch 'stable' 2026-06-21 13:28:16 +01:00
EvgenyandPaul Bottinelli b2bdade380 fix: ignore pending XFTP files in storage accounting (#1814)
* fix: ignore pending XFTP files in storage accounting

* style

---------

Co-authored-by: Paul Bottinelli <paul.bottinelli@trailofbits.com>
2026-06-21 13:27:28 +01:00
Evgeny Poberezkin 92598c2ddb 6.5.5.0 2026-06-21 13:08:27 +01:00
Samy 84724bc03e crypto: validate BBS proof parameters (#1810) 2026-06-21 13:06:30 +01:00
Paul Bottinelli 91cb297e9e fix: disable web in cloud scripts without certs (#1804) 2026-06-21 12:52:08 +01:00
74a86043cc lib: parse bracketed IPv6 server addresses (#1807)
* Parse bracketed IPv6 server hosts

* lib: parse service-scheme and invitation hosts via TransportHost

* correct encoding

* encoding

---------

Co-authored-by: Paul Bottinelli <paul.bottinelli@trailofbits.com>
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2026-06-21 12:18:34 +01:00
Evgeny 958de3bfca library: limit decompressed size (#1815) 2026-06-21 12:11:48 +01:00
EvgenyandPaul Bottinelli 45b21ec1db Reject duplicate STM short link updates (#1813)
Co-authored-by: Paul Bottinelli <paul.bottinelli@trailofbits.com>
2026-06-21 09:42:35 +01:00
EvgenyEvgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>Paul Bottinellish
aca1d9a462 crypto: sntrup length validation (#1811)
* Validate SNTRUP761 KEM input lengths

* crypto: BBS scheme for anonymous credentials with multiple presentations (#1794)

* crypto: BBS scheme for anonymous credentials with multiple presentations

* verify

* add files to sources

* more files

* more files, use cabal 3.0

* fix path

* extensions

* switch libbbs to fork

* return either from keygen

* use only secret key to sign

* improve FFI

* simplify

* update libbbs to support iOS

* add commoncrypto flag

* bump libbbs

* reject input of wrong length

* ci: get submodules

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>

* 6.5.4.0

* core: add getentropy shim for windows build (#1809)

* simplify

---------

Co-authored-by: Paul Bottinelli <paul.bottinelli@trailofbits.com>
Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
Co-authored-by: sh <37271604+shumvgolove@users.noreply.github.com>
2026-06-21 08:03:20 +01:00
brenziandsh 056314396d SNRC name resolver (#1795)
* add REST API to resolve SNRC

* fix unset fields

* support multi-TLD deployments

* update for mainnet tests

* haskell-friendly fieldnames

* add subname hint

* resolver: dockerize

* support multiple fallback links for splx contact and channels

* add test

* change url separator to semicolon

---------

Co-authored-by: sh <github.shum@liber.li>
2026-06-20 10:09:56 +01:00
Evgeny Poberezkin df6c53f830 7.0.0.1 2026-06-18 14:33:37 +01:00
Evgeny Poberezkin 220371cec1 Merge branch 'stable' 2026-06-18 14:29:31 +01:00
sh 44898bf7f6 core: add getentropy shim for windows build (#1809) 2026-06-18 14:28:55 +01:00
Evgeny Poberezkin 8e0b8de529 7.0.0.0 2026-06-17 17:14:36 +01:00
shandPaul Bottinelli db3e98f13a ntf-server: add push provider policy (#1808)
* Disable APNS test provider in production

* refactor(ntf): extract guardPushProvider for test-provider guard

* test(ntf): fix APNS test provider test compilation

* ntf server: use ifM for push provider guard (review)

---------

Co-authored-by: Paul Bottinelli <paul.bottinelli@trailofbits.com>
2026-06-17 09:14:38 +01:00
shandPaul Bottinelli 8a1b5608bf xftp-cli: add deprecation notice (#1799)
Co-authored-by: Paul Bottinelli <paul.bottinelli@trailofbits.com>
2026-06-16 10:07:13 +01:00
Evgeny Poberezkin e250a9ec9d Merge branch 'stable' 2026-06-16 06:50:14 +01:00
Evgeny Poberezkin 376d6a261a 6.5.4.0 2026-06-15 22:26:45 +01:00
EvgenyandEvgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com> 9f9b6c8e88 crypto: BBS scheme for anonymous credentials with multiple presentations (#1794)
* crypto: BBS scheme for anonymous credentials with multiple presentations

* verify

* add files to sources

* more files

* more files, use cabal 3.0

* fix path

* extensions

* switch libbbs to fork

* return either from keygen

* use only secret key to sign

* improve FFI

* simplify

* update libbbs to support iOS

* add commoncrypto flag

* bump libbbs

* reject input of wrong length

* ci: get submodules

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-06-15 09:44:11 +01:00
shandPaul Bottinelli 24e464926e scripts: fix check in simplex-servers-update (#1797)
Co-authored-by: Paul Bottinelli <paul.bottinelli@trailofbits.com>
2026-06-06 09:05:24 +01:00
shandPaul Bottinelli 7d3cfa56d3 xftp-web: remove debug logs (#1798)
Co-authored-by: Paul Bottinelli <paul.bottinelli@trailofbits.com>
2026-06-06 09:04:26 +01:00
sh 53bc0fe663 scripts: add docker-compose resolver setup (#1793) 2026-06-02 10:24:56 +01:00
Evgeny Poberezkin b981dcb70b 6.5.3.0 2026-06-01 13:17:47 +01:00
93 changed files with 5096 additions and 537 deletions
+4
View File
@@ -24,6 +24,8 @@ jobs:
- name: Clone project
if: startsWith(github.ref, 'refs/tags/v')
uses: actions/checkout@v3
with:
submodules: recursive
- name: Build changelog
id: build_changelog
@@ -114,6 +116,8 @@ jobs:
- name: Clone project
if: matrix.should_run == true
uses: actions/checkout@v3
with:
submodules: recursive
- name: Set up Docker Buildx
if: matrix.should_run == true
+2
View File
@@ -20,6 +20,8 @@ jobs:
steps:
- name: Clone project
uses: actions/checkout@v4
with:
submodules: recursive
- name: Log in to Docker Hub
uses: simplex-chat/docker-login-action@v3
+2
View File
@@ -11,6 +11,8 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@v3
with:
submodules: recursive
- name: Get latest release
shell: bash
+6
View File
@@ -0,0 +1,6 @@
[submodule "cbits/libbbs"]
path = cbits/libbbs
url = https://github.com/simplex-chat/libbbs.git
[submodule "cbits/blst"]
path = cbits/blst
url = https://github.com/supranational/blst.git
@@ -619,7 +619,6 @@ async function handleEncrypt(id, data, fileName) {
const digest = sha512Streaming([encData], (done) => {
self.postMessage({ id, type: "progress", done: source.length + done, total });
}, encDataLen);
console.log(`[WORKER-DBG] encrypt: encData.len=${encData.length} digest=${_whex(digest, 64)} chunkSizes=[${chunkSizes.join(",")}]`);
const dir = await getSessionDir();
const fileHandle = await dir.getFileHandle("upload.bin", { create: true });
const writeHandle = await fileHandle.createSyncAccessHandle();
@@ -643,9 +642,7 @@ function handleReadChunk(id, offset, size) {
}
async function handleDecryptAndStore(id, dhSecret, nonce, body, chunkDigest, chunkNo) {
const bodyArr = new Uint8Array(body);
console.log(`[WORKER-DBG] store chunk=${chunkNo} body.len=${bodyArr.length} nonce=${_whex(nonce, 24)} dhSecret=${_whex(dhSecret)} digest=${_whex(chunkDigest, 32)} body[0..8]=${_whex(bodyArr)} body[-8..]=${_whex(bodyArr.slice(-8))}`);
const decrypted = decryptReceivedChunk(dhSecret, nonce, bodyArr, chunkDigest);
console.log(`[WORKER-DBG] decrypted chunk=${chunkNo} len=${decrypted.length} [0..8]=${_whex(decrypted)} [-8..]=${_whex(decrypted.slice(-8))}`);
if (useMemory) {
memoryChunks.set(chunkNo, decrypted);
self.postMessage({ id, type: "stored" });
@@ -660,7 +657,6 @@ async function handleDecryptAndStore(id, dhSecret, nonce, body, chunkDigest, chu
currentDownloadOffset += decrypted.length;
chunkMeta.set(chunkNo, { offset, size: decrypted.length });
const written = downloadWriteHandle.write(decrypted, { at: offset });
console.log(`[WORKER-DBG] OPFS write chunk=${chunkNo} offset=${offset} size=${decrypted.length} written=${written}`);
if (written !== decrypted.length) {
console.warn(`[WORKER] OPFS write failed chunk=${chunkNo}: ${written}/${decrypted.length}, falling back to in-memory storage`);
for (const [cn, meta] of chunkMeta.entries()) {
@@ -684,23 +680,16 @@ async function handleDecryptAndStore(id, dhSecret, nonce, body, chunkDigest, chu
return;
}
downloadWriteHandle.flush();
const verifyBuf = new Uint8Array(Math.min(8, decrypted.length));
downloadWriteHandle.read(verifyBuf, { at: offset });
const verifyEnd = new Uint8Array(Math.min(8, decrypted.length));
downloadWriteHandle.read(verifyEnd, { at: offset + decrypted.length - verifyEnd.length });
console.log(`[WORKER-DBG] OPFS verify chunk=${chunkNo} readBack[0..8]=${_whex(verifyBuf)} readBack[-8..]=${_whex(verifyEnd)} expected[0..8]=${_whex(decrypted)} expected[-8..]=${_whex(decrypted.slice(-8))}`);
self.postMessage({ id, type: "stored" });
}
async function handleVerifyAndDecrypt(id, size, digest, key, nonce) {
console.log(`[WORKER-DBG] verify: expectedSize=${size} expectedDigest=${_whex(digest, 64)} useMemory=${useMemory} chunkMeta.size=${chunkMeta.size} memoryChunks.size=${memoryChunks.size}`);
const chunks = [];
let totalSize = 0;
const total = size * 3;
let done = 0;
if (useMemory) {
const sorted = [...memoryChunks.entries()].sort((a, b) => a[0] - b[0]);
for (const [chunkNo, data] of sorted) {
console.log(`[WORKER-DBG] verify memory chunk=${chunkNo} size=${data.length}`);
for (const [, data] of sorted) {
chunks.push(data);
totalSize += data.length;
done += data.length;
@@ -715,12 +704,10 @@ async function handleVerifyAndDecrypt(id, size, digest, key, nonce) {
const dir = await getSessionDir();
const fileHandle = await dir.getFileHandle("download.bin");
const readHandle = await fileHandle.createSyncAccessHandle();
console.log(`[WORKER-DBG] verify: OPFS file size=${readHandle.getSize()}`);
const sortedEntries = [...chunkMeta.entries()].sort((a, b) => a[0] - b[0]);
for (const [chunkNo, meta] of sortedEntries) {
for (const [, meta] of sortedEntries) {
const buf = new Uint8Array(meta.size);
const bytesRead = readHandle.read(buf, { at: meta.offset });
console.log(`[WORKER-DBG] verify read chunk=${chunkNo} offset=${meta.offset} size=${meta.size} bytesRead=${bytesRead} [0..8]=${_whex(buf)} [-8..]=${_whex(buf.slice(-8))}`);
readHandle.read(buf, { at: meta.offset });
chunks.push(buf);
totalSize += meta.size;
done += meta.size;
@@ -745,20 +732,9 @@ async function handleVerifyAndDecrypt(id, size, digest, key, nonce) {
}
const actualDigest = r.crypto_hash_sha512_final(state);
if (!digestEqual(actualDigest, digest)) {
console.error(`[WORKER-DBG] DIGEST MISMATCH: expected=${_whex(digest, 64)} actual=${_whex(actualDigest, 64)} chunks=${chunks.length} totalSize=${totalSize}`);
const state2 = r.crypto_hash_sha512_init();
for (let i = 0; i < chunks.length; i++) {
const chunk = chunks[i];
for (let off = 0; off < chunk.length; off += hashSEG) {
r.crypto_hash_sha512_update(state2, chunk.subarray(off, Math.min(off + hashSEG, chunk.length)));
}
const chunkDigest = sha512Streaming([chunk]);
console.error(`[WORKER-DBG] chunk[${i}] size=${chunk.length} sha512=${_whex(chunkDigest, 32)}… [0..8]=${_whex(chunk)} [-8..]=${_whex(chunk.slice(-8))}`);
}
self.postMessage({ id, type: "error", message: "File digest mismatch" });
return;
}
console.log(`[WORKER-DBG] verify: digest OK`);
const result = decryptChunks(BigInt(size), chunks, key, nonce, (d) => {
self.postMessage({ id, type: "progress", done: size * 2 + d, total });
});
@@ -334,11 +334,6 @@ class WorkerBackend {
const nonceCopy = new Uint8Array(nonce);
const digestCopy = new Uint8Array(digest);
const buf = this.toTransferable(body);
const hex = (b, n = 8) => {
const u = b instanceof ArrayBuffer ? new Uint8Array(b) : b;
return Array.from(u.slice(0, n)).map((x) => x.toString(16).padStart(2, "0")).join("");
};
console.log(`[BACKEND-DBG] chunk=${chunkNo} body.len=${body.length} body.byteOff=${body.byteOffset} buf.byteLen=${buf.byteLength} nonce=${hex(nonceCopy, 24)} dhSecret=${hex(dhSecretCopy)} digest=${hex(digestCopy, 32)} buf[0..8]=${hex(buf)} body[-8..]=${hex(body.slice(-8))}`);
await this.send(
{ type: "decryptAndStoreChunk", dhSecret: dhSecretCopy, nonce: nonceCopy, body: buf, chunkDigest: digestCopy, chunkNo },
[buf]
@@ -10913,14 +10908,12 @@ async function sendXFTPCommandOnce(client, privateKey, entityId, cmdBytes, chunk
const block = encodeAuthTransmission(client.sessionId, corrId, entityId, cmdBytes, privateKey);
const reqBody = chunkData ? concatBytes$1(block, chunkData) : block;
const fullResp = await client.transport.post(reqBody);
console.log(`[XFTP-DBG] sendOnce: fullResp.length=${fullResp.length} entityId=${_hex(entityId)} cmdTag=${cmdBytes[0]}`);
if (fullResp.length < XFTP_BLOCK_SIZE) {
console.error("[XFTP] Response too short: %d bytes (expected >= %d)", fullResp.length, XFTP_BLOCK_SIZE);
throw new Error("Server response too short");
}
const respBlock = fullResp.subarray(0, XFTP_BLOCK_SIZE);
const body = fullResp.subarray(XFTP_BLOCK_SIZE);
console.log(`[XFTP-DBG] sendOnce: body.length=${body.length} body.byteOffset=${body.byteOffset} body.buffer.byteLength=${body.buffer.byteLength}`);
const raw = blockUnpad(respBlock);
if (raw.length < 20) {
const text = new TextDecoder().decode(raw);
@@ -10939,18 +10932,13 @@ async function sendXFTPCommandOnce(client, privateKey, entityId, cmdBytes, chunk
}
return { response, body };
}
function _hex(b, n = 8) {
return Array.from(b.slice(0, n)).map((x) => x.toString(16).padStart(2, "0")).join("");
}
async function sendXFTPCommand(agent, server, privateKey, entityId, cmdBytes, chunkData, maxRetries = 3) {
let clientP = getXFTPServerClient(agent, server);
let client = await clientP;
for (let attempt = 1; attempt <= maxRetries; attempt++) {
try {
if (attempt > 1) console.log(`[XFTP-DBG] sendCmd: retry attempt=${attempt}/${maxRetries}`);
return await sendXFTPCommandOnce(client, privateKey, entityId, cmdBytes, chunkData);
} catch (e) {
console.log(`[XFTP-DBG] sendCmd: attempt=${attempt} failed: ${e instanceof Error ? e.message : String(e)} retriable=${isRetriable(e)}`);
if (!isRetriable(e)) {
throw categorizeError(e);
}
@@ -10979,7 +10967,6 @@ async function downloadXFTPChunkRaw(agent, server, rpKey, fId) {
const { response, body } = await sendXFTPCommand(agent, server, rpKey, fId, cmd);
if (response.type !== "FRFile") throw new Error("unexpected response: " + response.type);
const dhSecret = dh(response.rcvDhKey, privateKey);
console.log(`[XFTP-DBG] dlChunkRaw: body.length=${body.length} nonce=${_hex(response.nonce, 24)} dhSecret=${_hex(dhSecret)} body[0..8]=${_hex(body)} body[-8..]=${_hex(body.slice(-8))}`);
return { dhSecret, nonce: response.nonce, body };
}
async function downloadXFTPChunk(agent, server, rpKey, fId, digest) {
@@ -11012,7 +10999,6 @@ function encryptFileForUpload(source, fileName) {
const encSize = BigInt(chunkSizes.reduce((a, b) => a + b, 0));
const encData = encryptFile(source, fileHdr, key, nonce, fileSize, encSize);
const digest = sha512Streaming([encData]);
console.log(`[AGENT-DBG] encrypt: encData.len=${encData.length} digest=${_dbgHex(digest, 64)} chunkSizes=[${chunkSizes.join(",")}]`);
return { encData, digest, key, nonce, chunkSizes };
}
const DEFAULT_REDIRECT_THRESHOLD = 400;
@@ -11169,9 +11155,7 @@ async function downloadFileRaw(agent, fd, onRawChunk, options) {
if (err) throw new Error("downloadFileRaw: " + err);
const { onProgress} = options ?? {};
if (fd.redirect !== null) {
console.log(`[AGENT-DBG] resolving redirect: outer size=${fd.size} chunks=${fd.chunks.length}`);
fd = await resolveRedirect(agent, fd);
console.log(`[AGENT-DBG] resolved: size=${fd.size} chunks=${fd.chunks.length} digest=${Array.from(fd.digest.slice(0, 16)).map((x) => x.toString(16).padStart(2, "0")).join("")}`);
}
const resolvedFd = fd;
let downloaded = 0;
@@ -11189,7 +11173,6 @@ async function downloadFileRaw(agent, fd, onRawChunk, options) {
const seed = decodePrivKeyEd25519(replica.replicaKey);
const kp = ed25519KeyPairFromSeed(seed);
const raw = await downloadXFTPChunkRaw(agent, server, kp.privateKey, replica.replicaId);
console.log(`[AGENT-DBG] chunk=${chunk.chunkNo} body.len=${raw.body.length} expectedChunkSize=${chunk.chunkSize} digest=${_dbgHex(chunk.digest, 32)} body.byteOffset=${raw.body.byteOffset} body.buffer.byteLength=${raw.body.buffer.byteLength}`);
await onRawChunk({
chunkNo: chunk.chunkNo,
dhSecret: raw.dhSecret,
Submodule
+1
Submodule cbits/blst added at db3defd0d5
+24
View File
@@ -0,0 +1,24 @@
// SPDX-License-Identifier: Apache-2.0
// getentropy() shim for Windows, where it is absent from the CRT.
// Follows the POSIX contract: fills `buffer` with `length` random bytes
// (length must not exceed 256), returns 0 on success or -1 with errno set.
#ifdef _WIN32
#include <errno.h>
#include <stddef.h>
#include <windows.h>
#include <bcrypt.h>
int getentropy(void *buffer, size_t length) {
if (length > 256) {
errno = EIO;
return -1;
}
NTSTATUS status = BCryptGenRandom(NULL, (PUCHAR)buffer, (ULONG)length,
BCRYPT_USE_SYSTEM_PREFERRED_RNG);
if (!BCRYPT_SUCCESS(status)) {
errno = EIO;
return -1;
}
return 0;
}
#endif
Submodule
+1
Submodule cbits/libbbs added at 59a0f4bf32
+126
View File
@@ -0,0 +1,126 @@
# BBS+ Bindings for simplexmq
Haskell FFI bindings to libbbs for BBS+ signatures. General-purpose - the module knows nothing about specific applications.
## How BBS+ works
BBS+ signs a fixed list of N messages. Each message is an arbitrary byte array. The signer signs all N messages at once with one signature.
The holder of the signature can then generate a proof that selectively discloses some messages and hides others. The verifier learns the disclosed messages and confirms they were signed by the signer, but learns nothing about the hidden messages. Different proofs from the same signature are unlinkable.
Key constraint: the total number of messages N is fixed at signing time. The verifier must know N. A proof generated from a 3-message signature cannot be verified as a 2-message proof.
## Types
```haskell
newtype BBSSecretKey = BBSSecretKey ByteString -- 32 bytes
newtype BBSPublicKey = BBSPublicKey ByteString -- 96 bytes (BLS12-381 G2 point)
newtype BBSSignature = BBSSignature ByteString -- 80 bytes
newtype BBSProof = BBSProof ByteString -- 272 + 32 * numUndisclosed bytes
newtype BBSHeader = BBSHeader ByteString -- always-disclosed context (e.g. protocol identifier)
newtype BBSPresHeader = BBSPresHeader ByteString -- random nonce for proof unlinkability
```
All newtypes get StrEncoding (base64url), ToJSON/FromJSON (via strToJSON/strParseJSON), Eq, Show.
## Functions
```haskell
bbsKeyGen :: IO (Either String BBSKeyPair) -- BBSKeyPair = (BBSPublicKey, BBSSecretKey)
-- pk is derived from sk internally, so it is not a parameter
bbsSign
:: BBSSecretKey
-> BBSHeader -- always-disclosed context
-> [ByteString] -- all N messages
-> IO (Either String BBSSignature)
-- C order: pk, signature, header, presentation_header, disclosed_indexes, messages
bbsProofGen
:: BBSPublicKey
-> BBSSignature
-> BBSHeader -- must match what was signed
-> BBSPresHeader -- random nonce bound into the proof
-> [Int] -- disclosed indexes (0-based)
-> [ByteString] -- all N messages (needed internally, hidden ones not revealed in proof)
-> IO (Either String BBSProof)
-- C order: pk, proof, header, presentation_header, disclosed_indexes, n, messages
bbsProofVerify
:: BBSPublicKey
-> BBSProof
-> BBSHeader -- must match what was signed
-> BBSPresHeader -- must match what was used in bbsProofGen
-> [Int] -- disclosed indexes
-> Int -- total message count N
-> [ByteString] -- disclosed messages only
-> IO Bool
```
## How applications use it
An application defines:
- A message layout: which index means what
- Which indexes are disclosed vs hidden
- How to encode application values as ByteString messages
### Badge example (in simplex-chat, not in this module)
Message layout (always 3 messages):
- Index 0: master secret (32 random bytes) - HIDDEN
- Index 1: expiry (UTF-8 encoded timestamp string) - DISCLOSED
- Index 2: badge type (UTF-8 encoded, e.g. "supporter") - DISCLOSED
Signing (v2, on the server):
```
bbsSign sk header [ms, encodeUtf8 "2026-07-31", encodeUtf8 "supporter"]
```
Proof generation (v2, on the client):
```
bbsProofGen pk sig header presHeader [1, 2] [ms, encodeUtf8 "2026-07-31", encodeUtf8 "supporter"]
```
Proof verification (v1, on the recipient):
```
bbsProofVerify pk proof header presHeader 3 [1, 2] [encodeUtf8 "2026-07-31", encodeUtf8 "supporter"]
```
The recipient only sees the proof, presentationHeader, expiry string, and badge type string. They verify these were signed by the server (pk is hardcoded). They never see the master secret.
Expiry is always present as a string. Monthly badges use a date like `"2026-07-31"`, lifetime badges use `"lifetime"`. BBS+ doesn't interpret the bytes - expiry semantics are the application's responsibility. This keeps the message count fixed at 3 for all badge types.
## libbbs C API mapping
```c
int bbs_keygen_full(ciphersuite, sk, pk)
int bbs_sign(ciphersuite, sk, pk, signature, header, header_len, n, messages, message_lens)
int bbs_proof_gen(ciphersuite, pk, signature, proof, header, header_len, presentation_header, presentation_header_len, disclosed_indexes, disclosed_indexes_len, n, messages, message_lens)
int bbs_proof_verify(ciphersuite, pk, proof, proof_len, header, header_len, presentation_header, presentation_header_len, disclosed_indexes, disclosed_indexes_len, n, messages, message_lens)
```
We use `bbs_sha256_ciphersuite`. The header parameter is exposed in all Haskell functions - the application decides what to put there. Tests use `"SimpleX"` as header.
The `presentation_header` parameter is what we call `presentationHeader`.
In `bbs_proof_verify`, the `n` parameter is the total number of messages (not the number of disclosed messages). The `messages` array contains only the disclosed messages, and `disclosed_indexes` maps each to its position in the original message list.
## Build
Submodules in cbits/:
- `cbits/libbbs` - https://github.com/Fraunhofer-AISEC/libbbs
- `cbits/blst` - https://github.com/supranational/blst (libbbs dependency)
C sources in cabal: `cbits/blst/src/server.c`, `cbits/blst/build/assembly.S`, libbbs source files.
Include dirs: `cbits/blst/bindings/`, `cbits/blst/src/`, `cbits/libbbs/include/`, `cbits/libbbs/src/`.
C flags: `-D__BLST_PORTABLE__` for cross-CPU-generation compatibility.
## Tests
- Keygen produces keys of correct size
- Sign + proofGen + proofVerify roundtrip succeeds
- Tampered proof fails verification
- Tampered disclosed message fails verification
- Wrong public key fails verification
- Two proofs from same credential with different nonces both verify
- Proof size matches expected (272 + 32 * numUndisclosed)
@@ -0,0 +1,57 @@
## Root cause: orphaned `Sub` entries in the service client's `subscriptions` map
**The leak is service-specific and was introduced by PR #1667 "messaging services" (`f0b7a4be`).** A long-lived messaging-service connection accumulates per-queue `Sub` records in its `Client.subscriptions` map that are **never removed** when the associated queues are deleted or unassociated — only the counter is decremented. Over normal queue churn the map grows monotonically for the entire lifetime of the service connection.
### The proof — an asymmetry between two handlers in `serverThread`
Both individual queue subscriptions and service subscriptions store a `Sub` per queue in `Client.subscriptions` (= `clientSubs` for the SMP subscriber thread, wired at `Server.hs:189`). When a queue ends/is deleted, the two paths diverge:
**Individual subscriber — entry IS removed** (`Server.hs:332`, `346`):
```haskell
CSAEndSub qId -> atomically (endSub c qId) >>= a unsub_ -- :332
...
endSub c qId = TM.lookupDelete qId (clientSubs c) >>= (removeWhenNoSubs c $>) -- :346
```
**Service subscriber — entry is NOT removed** (`Server.hs:336-340`):
```haskell
CSAEndServiceSub qId -> atomically $ do
modifyTVar' (clientServiceSubs c) decrease -- decrements serviceSubsCount
modifyTVar' totalServiceSubs decrease -- decrements global count
where decrease = subtractServiceSubs (1, queueIdHash qId)
-- never touches (clientSubs c) — the Sub for qId stays forever
```
### Where the orphaned entries are added (both new in this PR)
- `Server.hs:1860-1862` — on service subscribe (`SSUB`), one `Sub` inserted per queue that has a pending message.
- `Server.hs:2039-2043` (`newServiceDeliverySub`) — on **every** `SEND` to a service-associated queue with no existing sub, a `Sub` is inserted into the service client's `subscriptions`. After delivery the thread state resets to `NoSub` (`:2069`) but the map entry remains as a "already delivering" marker (`:1856-1859`).
### Why they leak
The only places the service client's `subscriptions` map is cleared are:
- `clientDisconnected``swapTVar subscriptions M.empty` (`Server.hs:1097`) — only on disconnect.
- `CSADecreaseSubs``swapTVar (clientSubs c) M.empty` (`Server.hs:343`) — only on full service takeover by another connection.
- `delQueueAndMsgs``TM.lookupDelete entId $ subscriptions clnt` (`Server.hs:2164`) — but `clnt` here is **the recipient deleting its own queue, not the service client**. The service's entry for that queue is reached only via the `CSDeleted → endServiceSub → CSAEndServiceSub` path (`Server.hs:306, 313, 336`), which decrements the counter but leaves the map entry.
**Concrete scenario (fully traced):** Service `S` subscribes (`SSUB`) and stays connected for days. Recipient `R` owns service-associated queue `Q`. A `SEND` to `Q` inserts a `Sub` into `S.subscriptions[Q]` (`:2042`). `R` later deletes `Q``delQueueAndMsgs` runs on `R`'s connection, removes `Q` from `R.subscriptions`, decrements counters, enqueues `CSDeleted Q (Just S)` (`:2167`) → `serverThread` runs `CSAEndServiceSub Q` for `S` (`:336`), decrementing `S.serviceSubsCount` but **leaving `S.subscriptions[Q]` in place**. Net: one orphaned `Sub` (record + 2 TVars) per service-associated queue ever deleted/unassociated, never reclaimed until `S` disconnects. The logical counter `serviceSubsCount` correctly drops, so the map size diverges from the counter — making the leak invisible to the existing service-sub metric.
### Verdict
This is a deterministic, static-provable memory leak — no production logging needed to confirm the existence; the asymmetry between `CSAEndSub` (removes) and `CSAEndServiceSub` (doesn't) is the smoking gun. It is specific to messaging-service certificate clients, which is exactly the population added by the services/certificate PR.
### Secondary findings (lower impact, same PR area, not the primary cause)
- **`forkClient` register-after-fork race** (`Server.hs:1356-1359`): if the forked action's `finally` delete (`:1358`) runs before the parent's `IM.insert` (`:1359`), a `Weak ThreadId` of a dead thread is left in `endThreads` until disconnect. Pre-existing, tiny per-entry, but exercised far more by the PR's higher END/DELD volume.
- **Wrong-client counter decrement** (`Server.hs:2166`): `delQueueAndMsgs` decrements `serviceSubsCount` of the *deleting* client, not the service; harmless for non-service deleters (floored at 0) but corrupts accounting if a service deletes its own queue.
---
### Recommended fix (mirror `endSub` in the service path)
Make `CSAEndServiceSub` also delete the per-queue `Sub` and cancel its delivery thread, exactly as `CSAEndSub`/`endSub` do for individual subscribers. Roughly:
```haskell
CSAEndServiceSub qId -> do
s_ <- atomically $ do
modifyTVar' (clientServiceSubs c) decrease
modifyTVar' totalServiceSubs decrease
TM.lookupDelete qId (clientSubs c) <* removeWhenNoSubs c
forM_ unsub_ $ \unsub -> mapM_ unsub s_
where decrease = subtractServiceSubs (1, queueIdHash qId)
```
+455
View File
@@ -0,0 +1,455 @@
# Server: SMP support for public namespaces
> **⚠ Implementation diverged from this plan.** Six audit rounds reshaped the
> original design. **The shipped code differs in several load-bearing ways:**
>
> - **Wire format**: `NameRecord` is now JSON (aeson), not the custom binary
> ABNF this plan documents. See `protocol/simplex-messaging.md` §Resolver
> commands and `src/Simplex/Messaging/Protocol.hs` ToJSON/FromJSON instances.
> - **No cache**: the TTL + FIFO + byte-cap cache, in-flight coalescing,
> `psqueues` dep, and `cache_*` INI keys are all gone. Every RSLV becomes
> one `eth_call` bounded by `rpcMaxConcurrency` + `rpcTimeoutMs`. See
> `src/Simplex/Messaging/Server/Names.hs`.
> - **No `allow_dangerous_colocation` flag**: the proxy co-location guard
> was demoted to a startup `logWarn` (the flag was always-on because
> `[PROXY]` has no enable toggle).
> - **Module shape**: `Names/Resolver.hs` was merged into `Names.hs`; only
> `Names/Eth/RPC.hs` and `Names/Eth/SNRC.hs` remain as separate modules.
> - **Test list**: of the 15 specs listed below, ~7 shipped; the rest were
> either superseded by the cache removal (CacheSpec) or deferred
> (ForwardedRslvSpec, MockRpcSpec, StartupGuardSpec, UrlValidationSpec,
> EipChecksumSpec).
>
> Sources of truth: `CHANGELOG.md` (release notes),
> `protocol/simplex-messaging.md` §Resolver commands (wire format),
> `src/Simplex/Messaging/Server/Names*.hs` (implementation). This file is
> retained as historical context; do not treat it as a specification.
Implementation plan for Part 2 of [RFC 2026-05-21-public-namespaces](https://github.com/simplex-chat/simplex-chat/blob/ep/namespace/docs/rfcs/2026-05-21-public-namespaces.md). Adds a forwarded-only `RSLV <lookup_key>` SMP command that returns `NAME <NameRecord>` read from the SNRC contract via a Reth+Nimbus JSON-RPC endpoint. Smp-server becomes name-capable by `[NAMES] enable: on`.
Out of scope: `Simplex.Messaging.Client` API, agent-side resolution flow, `ServerRoles.names` in the agent, default-router list, reverse resolution, multicoin/text records, state proofs.
## Architecture
```mermaid
sequenceDiagram
participant C as Client
participant P as Proxy (storage role)
participant N as Name server (names role)
participant E as Ethereum endpoint<br/>(Reth+Nimbus)
C ->> P: PFWD(enc(RSLV key))
P ->> N: RFWD(enc(RSLV key))
note over N: verifyTransmission True →<br/>vc SResolver (RSLV _) → VRVerified
N ->> N: cache lookup
alt cache miss
N ->> E: eth_call(SNRC, namehash(key))
E -->> N: ABI bytes
note over N: ABI decode + zero-owner check + cache insert
end
N -->> P: RFWD(enc(NAME rec | ERR AUTH))
P -->> C: PRES(enc(NAME rec | ERR AUTH))
```
RSLV is **forwarded-only** — direct RSLV is rejected `CMD PROHIBITED`. This preserves the RFC's two-server resolution: the name server sees the lookup key but never the client's IP, session, or identity.
## Protocol
Shared library: `src/Simplex/Messaging/Protocol.hs` and `src/Simplex/Messaging/Transport.hs`.
**Version.** `Transport.hs:226`: `namesSMPVersion = VersionSMP 20`. Bump `currentClientSMPRelayVersion`, `currentServerSMPRelayVersion`, `proxiedSMPRelayVersion` to 20. Pre-v20 binaries lack the `RSLV_` tag; v20 binaries with sessions negotiated at v < 20 reject `RSLV_` at the parameter parser. The proxied-version bump 18 → 20 is safe (v19's `RecipientService`/`NotifierService` aren't in the forwarded whitelist; v18's `BLOCKED info` is already version-branched at `Protocol.hs:1943`).
**Party kind.** Append `Resolver` to `Party` (line 335); add `SResolver` (line 349), `TestEquality` clause (line 361), `PartyI Resolver` (line 394). `queueParty SResolver = Nothing` (falls through line 412). `partyClientRole SResolver = Nothing`.
**`RSLV` command.**
```haskell
RSLV :: LookupKey -> Command Resolver
newtype LookupKey = LookupKey ByteString
instance Encoding LookupKey where
smpEncode (LookupKey s) = smpEncode s
smpP = do
n <- lenP
when (n > 64) $ fail "LookupKey too large"
LookupKey <$> A.take n
```
Name-syntax validation is client-side per RFC; the server treats the key as opaque bytes. Tag `"RSLV"`, version guard inside `protocolP v (CT SResolver RSLV_)`: `| v >= namesSMPVersion -> Cmd SResolver . RSLV <$> _smpP`.
**Testnet/mainnet selector**: how the `#testnet:name` namespace appears in `LookupKey` bytes is determined by the SNRC contract (Part 1) — confirm with Part 1 before merging.
**`NAME` response.**
```haskell
NAME :: NameRecord -> BrokerMsg
```
Tag `"NAME"`. Symmetric version guards on encode (in `encodeProtocol v`) and decode (in `protocolP v NAME_`): `| v >= namesSMPVersion -> ...`. `NameRecord` has **no `Encoding` typeclass instance** — the typeclass cannot version-branch. Use top-level helpers `nameRecBytes :: VersionSMP -> NameRecord -> ByteString` and `parseNameRec :: VersionSMP -> Parser NameRecord`, mirroring the `IDS QIK` precedent at `Protocol.hs:19121979`.
**`NameRecord` schema and wire layout.**
```haskell
data NameRecord = NameRecord
{ nrDisplayName :: Text -- ≤255 bytes UTF-8
, nrOwner :: NameOwner -- 20 raw bytes
, nrChannelLinks :: [NameLink]
, nrContactLinks :: [NameLink]
, nrAdminAddress :: Maybe Text
, nrAdminEmail :: Maybe Text
, nrExpiry :: Int64 -- Unix seconds, ≥ 0
, nrIsTest :: Bool
}
newtype NameOwner = NameOwner ByteString -- bare ctor NOT exported; smart ctor enforces length 20
newtype NameLink = NameLink Text -- bare ctor NOT exported; smart ctor enforces ≤1024 bytes
unNameOwner :: NameOwner -> ByteString
unNameOwner (NameOwner bs) = bs
unNameLink :: NameLink -> Text
unNameLink (NameLink t) = t
```
Field additions are gated by future SMP version bumps (matching the `IDS QIK` precedent at `Protocol.hs:19121979`) — no separate record-version field.
| Field | Encoding | Max bytes |
|---|---|---|
| `nrDisplayName` | 1-byte length prefix + UTF-8 | 1 + 255 |
| `nrOwner` | 20 raw bytes, no prefix | 20 |
| `nrChannelLinks`, `nrContactLinks` | 1-byte count + per-element (Word16 BE len + UTF-8); combined cap **8 entries** across both lists | 1 + Σ(2 + ≤1024) |
| `nrAdminAddress`, `nrAdminEmail` | `'0'` or `'1'` + (1-byte length + UTF-8 if `'1'`) | 1 + 1 + 255 |
| `nrExpiry` | two big-endian `Word32` | 8 |
| `nrIsTest` | `'T'` or `'F'` | 1 |
`Encoding NameLink` reads the Word16 length **before** `A.take` allocates — going through the existing `Large` wrapper allows up to 65 535 bytes per element. There is no `Encoding [a]` instance — use `smpEncodeList` / `smpListP` / a bounded variant:
```haskell
smpListPUpTo :: Encoding a => Int -> Parser [a]
smpListPUpTo cap = do
n <- lenP
when (n > cap) $ fail "list too long"
A.count n smpP
parseNameRec _v = do
nrDisplayName <- smpP
nrOwner <- smpP
nrChannelLinks <- smpListPUpTo 8
nrContactLinks <- smpListPUpTo (8 - length nrChannelLinks)
nrAdminAddress <- smpP
nrAdminEmail <- smpP
nrExpiry <- smpP
when (nrExpiry < 0) $ fail "expiry must be non-negative"
nrIsTest <- smpP
pure NameRecord{..}
```
Both list parsers fail at the count step before allocating; the second inherits the residual budget. Canonical encoding by construction: every primitive has exactly one valid byte form — two name servers reading the same SNRC state produce byte-identical responses.
**Wire-size budget.** `paddedProxiedTLength = 16226` is the plaintext input to `cbEncrypt` (`Server.hs:2117`); `pad` reserves 2 bytes → framed transmission ≤ 16 224 bytes. Combined-link cap 8 yields max payload ≈ 9 050 bytes — generous margin.
**Error semantics.** A single wire code: `ERR AUTH`. Per RFC, this collapses every failure (name not found, malformed key, names disabled, RPC unreachable, decode error, timeout). Resolver internally distinguishes the cause for stats only.
**Forwarded-only access.** Direct RSLV is rejected with `CMD PROHIBITED`. The shape of `THAuthServer` alone cannot discriminate direct from forwarded (`Transport.hs:852` sets `sessSecret' = Just _` for every v6+ direct client too). An explicit `forwarded :: Bool` flag is threaded through `verifyTransmission` (see below).
## Server changes
All edits in `src/Simplex/Messaging/Server.hs`.
**`forwarded :: Bool` plumbing.** Three signatures change:
- `verifyTransmission :: Bool -> ...` (line 1233) — direct path passes `False` (lines 11521153), forwarded path passes `True` (line 2129).
- `verifyLoadedQueue :: Bool -> ...` (line 1238) — receives the flag from `verifyTransmission` (lines 1235, 1240).
- `verifyQueueTransmission :: Bool -> ...` (line 1244) — receives and uses the flag.
New `vc` clauses inside `verifyQueueTransmission`:
```haskell
vc SResolver (RSLV _) | forwarded = VRVerified Nothing
| otherwise = VRFailed (CMD PROHIBITED)
vc SResolver _ = VRFailed (CMD PROHIBITED) -- defensive catch-all
```
**Forwarded whitelist** (`Server.hs:2132`):
```haskell
Cmd SResolver (RSLV _) -> True
```
**`processCommand` branch** (alongside line 1481):
```haskell
Cmd SResolver (RSLV (LookupKey key)) -> do
st <- asks (rslvStats . serverStats)
incStat (rslvReqs st)
asks namesEnv >>= \case
Nothing -> incStat (rslvDisabled st) $> response (corrId, NoEntity, ERR AUTH)
Just nenv -> liftIO (resolveName nenv key) >>= \case
Right rec -> incStat (rslvSucc st) $> response (corrId, NoEntity, NAME rec)
Left NotFound -> incStat (rslvNotFound st) $> response (corrId, NoEntity, ERR AUTH)
Left _ -> incStat (rslvEthErrs st) $> response (corrId, NoEntity, ERR AUTH)
```
**Shutdown.** Add `closeNamesEnv :: NamesEnv -> IO ()` calling `closeManager`. Wire into `closeServer` (`Server.hs:247`):
```haskell
closeServer = do
asks (smpAgent . proxyAgent) >>= liftIO . closeSMPClientAgent
asks namesEnv >>= liftIO . mapM_ closeNamesEnv
```
In-flight `resolveName` calls during shutdown receive `ConnectionClosed``EthHttpErr` → masked-leader cleanup runs → waiters unblock with `ERR AUTH`.
**`incStat` relocation.** Defined at `Server.hs:2220`, currently unexported. Move to `Server/Stats.hs` (one-line transplant + export) so `Resolver.hs` can use it.
**Co-located proxy warning.** `newEnv` logs a startup warning whenever `allowSMPProxy = True` and `namesConfig = Just _`. RSLV is the first slow forwarded command; on a proxy host it can serialise other forwarded commands on the same proxy-relay session up to `rpcTimeoutMs` per cache miss. The warning is not a hard refusal because `[PROXY]` has no `enable: on/off` toggle — proxy is always on for every smp-server. `forkForwardedCmd` async dispatch is the longer-term fix, tracked as a follow-up; once the proxy role is gateable per-server, the warning can be tightened back to a refusal.
## Resolver subtree
New module tree at `src/Simplex/Messaging/Server/Names/`:
| Module | Contents |
|---|---|
| `Names.hs` | Façade — re-exports `NamesConfig`, `NamesEnv`, `ResolveError`, `resolveName`, `newNamesEnv`, `closeNamesEnv`. |
| `Names/Resolver.hs` | All types + cache + in-flight + `resolveName`. Helpers exported directly (no `.Internal` per codebase convention). **Test seam**: `NamesEnv` holds `ethCall` as a function value, so tests construct stubs via `newNamesEnvWith`. |
| `Names/Eth/RPC.hs` | `EthRpcEnv`; `ethCallReal` via `http-client` + `withResponse` + `brReadSome rpcMaxResponseBytes`. JSON-RPC error / HTTP error split. `rpcMaxConcurrency` semaphore. `Authorization` header from `rpcAuth`. |
| `Names/Eth/SNRC.hs` | `EthAddress`, Keccak-256 namehash via `crypton`'s `Crypto.Hash.Algorithms.Keccak_256` (mirroring `Crypto.hs:10231025` for SHA3), hand-rolled bounded Solidity ABI codec, `getRecord` with zero-owner detection. **Ethereum's Keccak ≠ NIST SHA3-256.** |
**ABI codec invariants**, enforced before any allocation: `offset + 32 ≤ buf.length`; `offset + 32 + length ≤ buf.length`; `offset ≥ headEnd` (no backward jumps); every length ≤ per-field cap; `string[]` outer length × 32 ≤ buf.length; recursion depth ≤ 2; `uint256 → Int64` rejects if any high 24 bytes non-zero; UTF-8 via `decodeUtf8'` returns `EthDecodeErr`.
**Zero-owner → `NotFound`**: ENS-style resolvers return zeroed records for non-existent names. After ABI decode, if `nrOwner == NameOwner (B.replicate 20 0)` return `Left NotFound`.
**Errors.**
```haskell
data ResolveError = NotFound | EthHttpErr | EthRpcErr { rpcCode :: Int, rpcMessage :: Text }
| EthDecodeErr | TimedOut
```
All collapse to `ERR AUTH`. `EthRpcErr` carries JSON-RPC `error` object — method-not-found (SNRC not deployed at `snrc_address`) is logged immediately on the first error after a recent success: `logError "NAMES: JSON-RPC error from endpoint — check snrc_address: <code> <message>"`. No automatic retry.
**Cache.** TTL + FIFO eviction. `TVar (OrdPSQ LookupKey Word64 NameRecord, Int)` — priority = monotonic-ns at insert; the `Int` is running byte count. `cacheLookup` is one STM transaction (read, expiry-check, expired-delete-with-byte-decrement). `cacheInsert` is one STM transaction: while `size > cacheMaxEntries` OR `bytes + sizeOf(rec) > cacheMaxBytes`, `minView` to drop oldest, then `insert`. Byte counter prevents `100 000 × 9 KB ≈ 900 MB` worst-case blow-up.
**Request coalescing** (async-exception safe via `E.mask`):
```haskell
resolveName env bs = do
let k = LookupKey bs
now <- getMonotonicTimeNSec
atomically (cacheLookup env k now) >>= \case
Just rec -> incStat (rslvCacheHits ...) $> Right rec
Nothing -> do
incStat (rslvCacheMiss ...)
ticket <- atomically $ TM.lookup k (inflight env) >>= \case
Just mv -> pure (Waiter mv)
Nothing -> newEmptyTMVar >>= \mv -> TM.insert k mv (inflight env) $> Leader mv
case ticket of
Waiter mv -> atomically (readTMVar mv)
Leader mv -> E.mask $ \restore -> do
r <- restore (fetchOnceTimed env bs)
`E.catch` \(e :: E.SomeException) -> pure (Left (mapEthErr e))
atomically $ putTMVar mv r >> TM.delete k (inflight env)
case r of Right rec -> atomically (cacheInsert env k now rec); Left _ -> pure ()
pure r
fetchOnceTimed env bs =
System.Timeout.timeout (rpcTimeoutMs (config env) * 1000) (fetchOnce env bs) >>= \case
Just r -> pure r
Nothing -> pure (Left TimedOut)
```
`E.mask` ensures `putTMVar + TM.delete` runs even on async exception; `fetchOnceTimed` runs under `restore` so it remains interruptible. Waiters always see a value; the in-flight TMap entry is always removed.
`fetchOnce`, `mapEthErr`, `scrubUrl`, `cacheLookup`, `cacheInsert` are internal to `Resolver.hs`. `getMonotonicTimeNSec` from `GHC.Clock` — first monotonic-clock use in the codebase; clock-jump safe.
**STM contention.** Cache hits are read-only `readTVar` — STM scales. Cache writes under sustained miss traffic can retry; `CacheSpec` asserts < 5% retry at 4 readers + 1 writer @ 1k RPS. If observed higher, swap `TVar` for `IORef` + `atomicModifyIORef'`.
**Multicoin and text records** are not in `NameRecord`. If Part 1 contract returns them from `getRecord`, extend `NameRecord` and the wire-size budget. **Confirm with Part 1 author before implementing `Eth/SNRC.hs`.**
## Configuration
`ServerConfig` (`Env/STM.hs:142`) gains one field `namesConfig :: Maybe NamesConfig`. `Env` (`Env/STM.hs:261`) gains `namesEnv :: Maybe NamesEnv`. `newEnv` constructs it after `proxyAgent` (line 605) with the co-location guard.
```haskell
data NamesConfig = NamesConfig
{ ethereumEndpoint :: Text -- http(s), no userinfo, explicit port required
, snrcAddress :: NameOwner -- 20 bytes
, rpcAuth :: Maybe RpcAuth -- required when https & non-loopback host
, cacheSeconds :: Int -- 300
, cacheMaxEntries :: Int -- 100000
, cacheMaxBytes :: Int -- 67108864 (64 MB)
, rpcTimeoutMs :: Int -- 3000
, rpcMaxResponseBytes :: Int -- 262144 (256 KB)
, rpcMaxConcurrency :: Int -- 8
}
data RpcAuth = AuthBearer Text | AuthBasic Text Text
```
INI parsing in `Server/Main.hs`:
- `validateUrl` (using new `network-uri` dep): accepts only http(s), non-empty host, **explicit port** (rejects `http://localhost` defaulting to 80 while Reth is on 8545), no userinfo, no query/fragment. Rejects `https://...` without `rpc_auth` when host is non-loopback. On rejection: `logError` + `exitFailure`.
- `parseEthAddr`: accepts `0x[0-9a-fA-F]{40}` and the same without `0x`. Mixed-case → verify EIP-55 checksum and reject mismatch (catches typos).
- `parseRpcAuth`: reads optional `rpc_auth` key; format `bearer <token>` or `basic <user>:<pass>`.
- `scrubUrl`: strips userinfo from all log lines mentioning the endpoint, including inside `mapEthErr`.
- Transition-aware error logging: log immediately on first error after a recent success, then at most hourly while persisting + summary at every stats reset.
Default INI template (`Server/Main/Init.hs`, after `[PROXY]`):
```
[NAMES]
# Public-namespace resolution (SNRC on Ethereum).
# Requires an Ethereum JSON-RPC endpoint (Reth+Nimbus). See deployment guide.
# Cannot be combined with [PROXY] enable: on by default — see allow_dangerous_colocation.
# Restart required to change settings.
enable: off
# Same-host:
# ethereum_endpoint: http://127.0.0.1:8545
# Central Reth via Caddy:
# ethereum_endpoint: https://eth.simplex.chat:443
# rpc_auth: basic <username>:<password>
# snrc_address: 0x0000000000000000000000000000000000000000
# cache_seconds: 300
# cache_max_entries: 100000
# cache_max_bytes: 67108864
# rpc_timeout_ms: 3000
# rpc_max_response_bytes: 262144
# rpc_max_concurrency: 8
# allow_dangerous_colocation: off
```
Upgrade from a pre-v6.6 INI: missing `[NAMES]` section → disabled. No operator action required.
## Operator deployment
Two supported topologies. smp-server is agnostic — only `ethereum_endpoint` changes.
**Topology A (same-host)**: smp-server, Caddy (optional), Reth, Nimbus all on one box. `ethereum_endpoint: http://127.0.0.1:8545`.
**Topology B (central Reth, N smp-server hosts — recommended for fleets)**: one operator runs one eth host with Reth+Nimbus behind Caddy on public HTTPS. Each smp-server has its own credential.
```mermaid
flowchart LR
subgraph eth-host
Caddy["Caddy<br/>(public :443, basic auth)"]
Reth["Reth<br/>(127.0.0.1:8545)"]
Nimbus["Nimbus"]
Caddy --> Reth
Nimbus -- Engine API (jwt.hex) --> Reth
end
subgraph smp-host-1
S1["smp-server #1"]
end
subgraph smp-host-N
SN["smp-server #N"]
end
S1 -- HTTPS + Authorization --> Caddy
SN -- HTTPS + Authorization --> Caddy
Reth <-- Ethereum p2p --> internet
Nimbus <-- beacon sync --> internet
```
Sharing one Reth across **multiple operators** is **not** supported — collapses the RFC's two-server resolution privacy.
**Reth + Nimbus**: Reth (execution layer) holds Ethereum state on ~260 GB pruned NVMe; Nimbus (consensus light client) follows beacon-chain headers. Paired via Engine API on `127.0.0.1:8551` with a shared `jwt.hex`. Recommended Reth flags:
```bash
reth node \
--http.addr 127.0.0.1 \
--http.api eth \ # only eth namespace
--rpc.gascap 50000000 \ # cap gas per eth_call
--rpc.max-response-size 5242880 \ # 5 MB
--http.corsdomain none \
--authrpc.jwtsecret /opt/eth/jwt.hex \
--authrpc.addr 127.0.0.1 --authrpc.port 8551
```
**Caddy + Let's Encrypt + Basic auth** (Topology B):
```caddy
eth.simplex.chat {
basicauth {
smp-server-1 $2a$14$<bcrypt-hash-1>
smp-server-2 $2a$14$<bcrypt-hash-2>
}
log { format filter { wrap json; fields { request>headers>Authorization delete } } }
reverse_proxy 127.0.0.1:8545
}
```
Caddy auto-fetches Let's Encrypt cert. Each smp-server has its own credential; revoking one = delete the line. `Authorization` stripped from access logs. Port 80 needed for the ACME HTTP-01 challenge (use TLS-ALPN-01 or DNS-01 to drop it). The threat being defended against is DoS (SNRC state is public); mTLS would be overkill. WireGuard/Tailscale are alternative network-layer approaches — both compatible with the plan.
**Capacity.** One Reth+Nimbus box handles a realistic operator fleet by 101000× margin. Per-smp-server peak RSLV ≈ 1700 RPS (pessimistic); cache hit rate ≥ 95% → ~85 RPS cache miss per smp-server; 10 smp-servers → ~850 RPS aggregate cache miss reaching Reth; Reth `eth_call` throughput on warm NVMe ≈ 1k10k RPS. Sizing: 8 vCPU, 32 GB RAM, 1 TB NVMe is comfortable. Scale-out path: more Reth+Nimbus pairs, smp-servers round-robin or shard.
## Implementation
**Order**:
1. Protocol: party/SParty/PartyI, RSLV+tag, NAME+tag, NameRecord + helpers, version constants in `Transport.hs`.
2. `verifyTransmission`/`verifyLoadedQueue`/`verifyQueueTransmission` `forwarded :: Bool` flag + `vc SResolver` clauses.
3. Forwarded whitelist + `processCommand` branch + `incStat` move to `Stats.hs`.
4. Env plumbing: `Server/Env/STM.hs`, `Server/Main.hs` INI parse, `Server/Main/Init.hs` template.
5. Resolver subtree: `Eth/SNRC.hs``Eth/RPC.hs``Resolver.hs`.
6. `NameResolverStats` sub-record + CSV log + Prometheus `names =` block.
7. Replace stub in (3) with real `resolveName`.
8. Tests.
9. `protocol/simplex-messaging.md`: header version line 1 (`19 → 20`), sentence at line 86, version-history list (lines 93105) v20 entry, TOC (lines 2568) "Resolver commands" subsection, new section with ABNF + byte layout + error semantics, "Router security requirements" paragraph about names-role outbound HTTP, cross-ref `Transport.hs:226`.
10. `CHANGELOG.md`: v6.6 entry.
**Cabal** (`simplexmq.cabal`): bump `version: 6.6.0.0`. Add to `if !flag(client_library)` block: `http-client >=0.7 && <0.8`, `http-client-tls >=0.3 && <0.4`, `network-uri >=2.6 && <2.7`, `psqueues >=0.2.7 && <0.3`. Expose 4 new `Server.Names.*` modules in the same block. `crypton` already provides `Keccak_256`.
**Files changed**:
| File | Change |
|---|---|
| `Protocol.hs` | Resolver party + RSLV/NAME tags + version guards; `NameRecord` + newtypes + smart ctors; `nameRecBytes`/`parseNameRec`/`smpListPUpTo` helpers (no Encoding NameRecord instance); `LookupKey` parser-side cap |
| `Transport.hs` | `namesSMPVersion = 20`; bump current/proxied SMP versions |
| `Server.hs` | Thread `forwarded :: Bool`; `vc SResolver` clauses; whitelist (2132); Resolver branch in `processCommand` (1481); `closeServer` calls `closeNamesEnv`; CSV log (579618); **remove** local `incStat` |
| `Server/Env/STM.hs` | `namesConfig` field; `namesEnv` field; `newEnv` constructs `NamesEnv` with co-location guard |
| `Server/Main.hs` | `[NAMES]` parse: `validateUrl`/`parseEthAddr`/`parseRpcAuth`; `scrubUrl` in logs |
| `Server/Main/Init.hs` | `[NAMES]` block in default INI |
| `Server/Stats.hs` | `incStat` moved here + exported; `NameResolverStats` sub-record + helpers; `rslvStats` field |
| `Server/Prometheus.hs` | `names =` metric block |
| `Server/Names.hs` (new) | Façade re-exports |
| `Server/Names/Resolver.hs` (new) | All resolver types + cache + coalescing + `fetchOnceTimed` + `newNamesEnv[With]` + `closeNamesEnv` |
| `Server/Names/Eth/RPC.hs` (new) | `EthRpcEnv`, `ethCallReal` with bounded body + concurrency semaphore + `Authorization` header |
| `Server/Names/Eth/SNRC.hs` (new) | `EthAddress`, Keccak namehash, bounded ABI (8 invariants), `getRecord` with zero-owner detection |
| `simplexmq.cabal` | Bump `6.6.0.0`; 4 new deps + 4 new modules in `if !flag(client_library)` block |
| `protocol/simplex-messaging.md` | Header version, version-history v20 entry, new "Resolver commands" section |
| `CHANGELOG.md` | v6.6 entry |
## Testing
`tests/SMPNamesTests/` registered in `tests/Test.hs:112151`. Build only when `client_library = False`.
1. **ProtocolEncodingSpec**`nameRecBytes``parseNameRec` round-trip; oversized fields rejected at parse; combined-list cap 8 enforced; negative `nrExpiry` rejected; canonical encoding byte-stable.
2. **MaxSizeSpec** — max `NameRecord` encodes ≤ ~9 KB; `encodeTransmission v ≤ paddedProxiedTLength - 2`; `cbEncrypt` succeeds.
3. **CommandTagSpec**`"RSLV"`/`"NAME"` parse; v < 20 sessions reject `RSLV_` at parameter parser.
4. **ForwardedGateSpec** — direct RSLV → `CMD PROHIBITED`; forwarded RSLV reaches handler.
5. **ForwardedRslvSpec** — RSLV wrapped in PFWD reaches the handler end-to-end. **Test infra cost**: first protocol-level PFWD test; budget for `runProxiedSmpCommand` helper performing `PRXY`/`PKEY`/`PFWD` manually.
6. **CacheSpec** — hit avoids RPC; TTL expiry forces re-fetch; bytes cap evicts before entries cap on large records; concurrent same-key callers issue one RPC; leader exception → all waiters get `Left _`, TMap entry removed; leader async-cancel → cleanup STM still runs.
7. **AbiSpec** — encode/decode against pinned fixtures (`tests/fixtures/snrc/`); QuickCheck fuzz on random buffers ≤ `rpcMaxResponseBytes` must never crash.
8. **NamehashSpec** — Keccak-256 reference vectors; assert Keccak ≠ SHA3-256.
9. **MockRpcSpec** — fake HTTP server; missing → `EthHttpErr`; slow → `TimedOut`; multi-GB body truncated → `EthDecodeErr`. `rpcAuth = AuthBasic` sends correct header.
10. **Uint256OverflowSpec**`expiry > Int64.maxBound``EthDecodeErr`.
11. **ZeroOwnerSpec**`owner = 0x000...000``NotFound`.
12. **StartupGuardSpec**`allowSMPProxy + names.enable` aborts; `allow_dangerous_colocation = on` starts with warning.
13. **UrlValidationSpec** — userinfo/scheme/host/port edge cases; rejects `https://` without `rpc_auth` for non-loopback.
14. **EipChecksumSpec**`parseEthAddr` accepts lower/upper; verifies mixed-case checksum; rejects typos.
15. **AbiBoundsSpec** — each of 8 ABI invariants triggers `EthDecodeErr` without crash/allocation blow-up.
Integration against real Reth+Nimbus mainnet deferred to ops.
## Threat model, scope, coordination
| Actor | Can | Cannot |
|---|---|---|
| Name server | See lookup-key bytes; see query timing; see Eth endpoint URL (operator-self) | See client IP/session; correlate clients across queries |
| Compromised Eth endpoint | Poison this server's cache for one TTL window; see every lookup key the server queries | Bypass two-server agreement (client-side, out of scope) |
| Adversarial client (high-rate unique keys) | Cache-thrash DoS; fill `Manager` connection pool up to `managerConnCount = 8` | Bypass `rpcMaxResponseBytes` or `fetchOnceTimed` |
| Adversarial proxy (slow inner RSLVs) | Block other forwarded commands on that proxy connection up to `rpcTimeoutMs` per miss | Affect other proxy connections |
| Operator with footgun config (https no auth, public Eth RPC) | (rejected at startup, or operator-acknowledged data leak) | — |
Mitigations: caching + coalescing + `rpcTimeoutMs` + `rpcMaxResponseBytes` + `rpcMaxConcurrency`; co-location refused at startup; URL validation; Caddy + auth in front of Reth; Reth's own gas/size caps. Timing side-channels (cache-hit vs miss latency) not mitigated — flagged for post-MVP. State proofs deferred to post-MVP per RFC.
**Cross-repo coordination.** The `simplex-chat` `ep/namespace` branch currently contains only the RFC commit — no agent-side wire-format code yet. This plan's wire format is validated only by simplexmq's own tests until a matching agent PR lands (structurally weak — encoder/decoder bugs are mutually consistent with themselves). Coordinate with the agent-side implementer **before merging** on: exact `NameRecord` field order and types; `LookupKey` namespace-prefix convention; error-code semantics; Part 1 SNRC contract `getRecord` ABI surface.
+122 -2
View File
@@ -1,4 +1,4 @@
Version 19, 2025-01-24
Version 20, 2026-05-25
# Simplex Messaging Protocol (SMP)
@@ -67,6 +67,9 @@ Version 19, 2025-01-24
- [Queue deleted notification](#queue-deleted-notification)
- [Error responses](#error-responses)
- [OK response](#ok-response)
- [Resolver commands](#resolver-commands)
- [Resolve name command](#resolve-name-command)
- [Name record response](#name-record-response)
- [Transport connection with the SMP router](#transport-connection-with-the-SMP-router)
- [General transport protocol considerations](#general-transport-protocol-considerations)
- [TLS transport encryption](#tls-transport-encryption)
@@ -83,7 +86,7 @@ It's designed with the focus on communication security and integrity, under the
It is designed as a low level protocol for other application protocols to solve the problem of secure and private message transmission, making [MITM attack][1] very difficult at any part of the message transmission system.
This document describes SMP protocol version 19. Versions 1-5 are discontinued. The version history:
This document describes SMP protocol version 20. Versions 1-5 are discontinued. The version history:
- v1: binary protocol encoding
- v2: message flags (used to control notifications)
@@ -103,6 +106,7 @@ This document describes SMP protocol version 19. Versions 1-5 are discontinued.
- v17: create notification credentials with NEW command
- v18: support client notices in BLOCKED error
- v19: service subscriptions to messages (SUBS, NSUBS, SOKS, ENDS, ALLS commands)
- v20: public namespaces resolver (RSLV command, RNAME response) — direct or forwarded via PFWD
## Introduction
@@ -424,6 +428,8 @@ Simplex messaging router implementations MUST NOT create, store or send to any o
- Any other information that may compromise privacy or [forward secrecy][4] of communication between clients using simplex messaging routers (the routers cannot compromise forward secrecy of any application layer protocol, such as double ratchet).
Routers with the names role make outbound HTTP calls to a backing resolver service (the reference implementation is `scripts/resolver/snrc-resolve.py`, which in turn makes JSON-RPC calls to an Ethereum endpoint) to read `NameRecord` data; the lookup key reaches that resolver and its upstream RPC endpoint. Operators MUST run both the resolver process and its upstream RPC endpoint themselves (loopback Reth + Nimbus, or a self-hosted central deployment) — sharing them across multiple operators collapses the two-server privacy property because the resolver / RPC operator would see every lookup key across all of them. The names role and the SMP-proxy role MUST NOT be enabled on the same router by default: a client forwarding `RSLV` through a proxy that is also the names router would expose both its connection and the lookup key to one operator, collapsing the two-server privacy property. (Resolution itself runs on a forked thread, so a slow `RSLV` does not serialise other forwarded commands on the session.)
## Message delivery notifications
Supporting message delivery while the client mobile app is not running requires sending push notifications with the device token. All alternative mechanisms for background message delivery are unreliable, particularly on iOS platform.
@@ -1422,6 +1428,120 @@ When the command is successfully executed by the router, it should respond with
ok = %s"OK"
```
### Resolver commands
Resolver commands implement public-namespace name resolution on the names-role
router. A names router translates an opaque lookup key (such as `alice` or
`alice.simplex.eth`) into a `NameRecord` carrying the channel and contact links
the named party publishes.
**Direct or forwarded.** RSLV is an unauthenticated command accepted both
directly from a transport client and inside a `PFWD` block via the SMP proxy;
the client chooses. Forwarded delivery preserves the two-server privacy property
of the resolver design: the names router sees the lookup key but never the
client IP, session, or identity, while the proxy router sees the client
connection but cannot read the encrypted lookup key inside the forwarded
transmission. Direct delivery is simpler but exposes the client's connection to
the names router, so clients SHOULD prefer the forwarded path when proxying is
available.
**Backing store.** This protocol does not prescribe where the names router
reads `NameRecord` from. The reference implementation forwards each RSLV to a
companion REST resolver process (`scripts/resolver/snrc-resolve.py`) that
queries the SNRC contract on Ethereum; alternative backings (different chains,
DHT, etc.) are valid as long as they expose the documented HTTP shape (`GET
/resolve/<name>` returning a `NameRecord` on 200, 404 / 400 for unknown names
or TLDs, 502 for upstream RPC failures) or substitute a different transport
while still returning a `NameRecord` matching the encoding below.
#### Resolve name command
The `RSLV` command carries the canonical fully-qualified name directly as the
payload (not JSON):
```abnf
rslv = %s"RSLV" SP domain ; domain = canonical name as non-space bytes, consuming the remainder of the transmission
```
`domain` is the UTF-8 canonical fully-qualified name with the TLD always
explicit (e.g. `privacy.simplex`, `test.testing`, `example.com`), bounded to
253 bytes.
**Server-side validation.** The names router parses `domain` as a
fully-qualified name (TLD required — bare labels are rejected) and forwards it
to the configured backing resolver, which is the source of truth for which
on-chain registry maps to each TLD.
The names router responds with either an `RNAME` response carrying the resolved
record, or an `ERR NAME` error whose subcode a client iterating across several
configured servers can act on distinctly:
| Response | Condition | Client action |
|---|---|---|
| `RNAME` | record resolved | use it |
| `ERR NAME NOT_FOUND` | name not registered, unknown TLD, or malformed name | authoritative "no such name" — stop |
| `ERR NAME NO_RESOLVER` | this router has no resolver (names role not enabled) | skip this server, try the next |
| `ERR NAME RESOLVER <detail>` | transient failure: backing resolver error (upstream 5xx, transport, timeout, decode) | transient — retry or surface, do not treat as "not found" |
A client SHOULD NOT broadcast a `name` to further servers after a name-capable
router has answered (`NOT_FOUND` or `RESOLVER`), since that router has already
seen the lookup key; `NO_RESOLVER` discloses nothing about the name beyond the
fact that this router cannot resolve, so iterating past it is safe.
#### Name record response
The `RNAME` response carries a JSON-encoded record as the payload:
```abnf
rname = %s"RNAME" SP json-bytes ; json-bytes consumes the remainder of the transmission
```
`json-bytes` MUST be a UTF-8 JSON object with the following schema:
| Field | JSON type | Constraints |
|---|---|---|
| `name` | string | ≤ 255 bytes UTF-8 |
| `nickname` | string | ≤ 255 bytes UTF-8; senders MUST emit the empty string `""` when unset |
| `website` | string | ≤ 255 bytes UTF-8; same empty-string-when-unset rule |
| `location` | string | ≤ 255 bytes UTF-8; same empty-string-when-unset rule |
| `simplexContact` | array of strings | each a SimpleX contact link (primary first); empty array `[]` when unset |
| `simplexChannel` | array of strings | each a SimpleX channel link (primary first); empty array `[]` when unset |
| `eth` | string or null | ≤ 255 bytes UTF-8; senders MUST emit `null` when unset; receivers MUST also accept absent keys as unset |
| `btc` | string or null | ≤ 255 bytes UTF-8; same null / absent rules |
| `xmr` | string or null | ≤ 255 bytes UTF-8; same null / absent rules |
| `dot` | string or null | ≤ 255 bytes UTF-8; same null / absent rules |
| `owner` | string | `"0x"` followed by 40 lowercase hex characters (20 raw bytes) |
| `resolver` | string | `"0x"` followed by 40 lowercase hex characters; the resolver contract address that produced the record |
Text fields (`nickname`, `website`, `location`) use the empty string `""` as
the "unset" sentinel: a backing resolver with no value for the field MUST emit
an empty string, not JSON `null` and not an absent key. Link fields
(`simplexContact`, `simplexChannel`) are arrays, primary link first, and use the
empty array `[]` when unset. Coin fields (`eth`, `btc`, `xmr`, `dot`) use JSON
`null` as the "unset" sentinel and MAY also be absent from the object entirely.
The backing resolver filters records that are expired or otherwise unavailable
(the names router then returns `ERR NAME NOT_FOUND` to the client), so the wire
format carries no expiry field. Testnet-vs-mainnet status is derived from the
queried TLD rather than an in-record flag.
Receivers MUST tolerate extra unknown fields (forward-compatibility for future
field additions). Adding a required field is a breaking change requiring an
SMP version bump.
**Field order is not significant.** Receivers parse JSON by key name, so object
key order, insignificant whitespace, and number formatting carry no meaning;
records are interpreted by decoded value, never compared byte-for-byte. Peers
MUST NOT rely on a byte-canonical form — a different resolver or server may emit
the same record with different key order or spacing. This order-independence is
what makes the format forward-compatible (see the unknown-field rule above).
**Wire-size budget.** The names router caps the resolver response it will
accept (`resolver_max_response_bytes`, ≤ 16000 bytes, the default) so the
re-encoded `RNAME` stays within the SMP proxied transmission budget of 16224
bytes; a response over the cap is rejected as `ERR NAME RESOLVER`. The link
arrays are bounded by this overall budget rather than a fixed per-field count.
## Transport connection with the SMP router
### General transport protocol considerations
+2 -2
View File
@@ -237,11 +237,11 @@ checks() {
exit 1
fi
mkdir -p "$path_conf_info" "$path_tmp_bin"
check_versions
check_distro
mkdir -p $path_conf_info $path_tmp_bin
return 0
}
+22
View File
@@ -0,0 +1,22 @@
# ============================================================================
# Required settings — the stack will not start without these.
# ============================================================================
# Ethereum network: mainnet (the SNRC `.testing` contracts live on mainnet)
# or holesky (test). Mainnet full sync needs ~1 day and ~1.2 TB NVMe.
NETWORK=mainnet
# Beacon checkpoint-sync URL — used ONCE on first sync. Must expose the heavy
# /eth/v2/debug/beacon/states/finalized endpoint (generic beacon APIs do not;
# use a dedicated checkpoint provider). List: https://eth-clients.github.io/checkpoint-sync-endpoints/
# mainnet: https://mainnet-checkpoint-sync.attestant.io (also beaconstate.info, sync-mainnet.beaconcha.in)
# holesky: https://checkpoint-sync.holesky.ethpandaops.io
TRUSTED_NODE_URL=https://mainnet-checkpoint-sync.attestant.io
# ============================================================================
# Optional overrides — sensible defaults are baked into docker-compose.yml,
# so leave these commented unless you need to change them.
# ============================================================================
# Nimbus NAT (default: any). For a stable public node set an explicit IP:
# NAT=extip:1.2.3.4 # your public IPv4: curl -s ifconfig.me
+146
View File
@@ -0,0 +1,146 @@
# Self-hosted SNRC stack
One `docker compose up` runs the self-hosted SimpleX Namespace (SNRC) backend
against **Ethereum mainnet** (where the `.testing` contracts live):
| # | Component | What it does |
|---|---|---|
| 1 | **reth + nimbus** | self-hosted Ethereum node (`--minimal` — enough for the resolver's `eth_call` at chain head) |
| 2 | **resolver** | the REST resolver the smp-server's `[NAMES]` role queries (`snrc-resolve.py`) |
## Requirements
- **Docker** + Compose v2.
- **≥ 300 GB NVMe SSD** for `reth --minimal` (~260 GB on mainnet; TLC, not QLC
— QLC stalls during sync) + **32 GB RAM**, fast multi-core CPU.
- **~1 day** for the initial reth sync. The resolver returns errors until reth
has caught up — that's expected.
- Firewall: open p2p ports `30303` (tcp/udp) and `9000` (tcp/udp).
## 1. Configure
Edit `.env` — the defaults work as-is; override only if needed:
```sh
NETWORK=mainnet # default
TRUSTED_NODE_URL=https://mainnet-checkpoint-sync.attestant.io # default
```
Everything else (NAT) has a working default baked into `docker-compose.yml`;
uncomment the hints in `.env` only to override.
## 2. Run
```sh
cd scripts/resolver
docker compose up -d
docker compose logs -f reth resolver
```
`depends_on` handles ordering automatically (start node → start resolver).
## 3. Wait for the node to sync
```sh
docker compose logs --tail=20 reth
```
This is the long pole (~1 day on mainnet). Until reth is synced the resolver
returns `502`.
## Verify
Run these once the stack is up (the node-dependent ones pass after sync):
**1. reth is reachable and reporting a block:**
```sh
curl -s -X POST http://127.0.0.1:8545 \
-H 'content-type: application/json' \
-d '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}' | jq
```
**2. resolver is healthy:**
```sh
curl -s http://127.0.0.1:8000/health | jq
# → {"ok": true, "rpc": "http://reth:8545", "registries": {"testing": "0x…", "simplex": ""}}
```
**3. resolver resolves a live name** (`foobar.testing` is a populated test name):
```sh
curl -s http://127.0.0.1:8000/resolve/foobar.testing | jq
# → {"name":"foobar.testing","nickname":"Foo","simplexContact":["https://smp16.simplex.im/a#…"], … }
```
**Wire your smp-server:** in its `[NAMES]` section set
`resolver_endpoint: http://127.0.0.1:8000` (no auth needed for loopback).
## Ports (all loopback unless noted)
| Service | Host | Purpose |
|---|---|---|
| reth JSON-RPC | `127.0.0.1:8545` | smp-server RPC |
| reth p2p | `:30303` tcp/udp | Ethereum sync (open on firewall) |
| nimbus p2p | `:9000` tcp/udp | beacon sync (open on firewall) |
| nimbus REST | `127.0.0.1:5052` | beacon API |
| **resolver** | `127.0.0.1:8000` | SNRC REST (`/resolve`, `/health`) |
## Caveats
- **All images track `:latest`** (reth, nimbus) — you get upstream fixes on each
`docker compose pull`; re-run the verify checks after pulling.
- All ports bind to loopback; expose only what you put behind a TLS reverse proxy.
## Teardown
```sh
docker compose down # stop, keep all state
docker compose down -v # also wipe volumes → full re-sync
```
`down -v` wipes the chain data (full re-sync on the next `up`).
---
## Resolver API reference
The resolver (`snrc-resolve.py`, host `127.0.0.1:8000`) is also runnable
standalone for local dev (no Docker), via [`uv`](https://docs.astral.sh/uv/):
```sh
uv run scripts/resolver/service/snrc-resolve.py # defaults to local reth + mainnet .testing
```
### Response shape
```jsonc
{
"name": "foobar.testing",
"nickname": "Foo", "website": "https://foo.bar", "location": "",
"simplexContact": ["https://smp16.simplex.im/a#…", "https://smp11…"], // primary first, fallbacks after
"simplexChannel": [],
"eth": null, "btc": "bc1q…", "xmr": "4ANz…", "dot": "139G…",
"owner": "0xd83b…", "resolver": "0x80fa…"
}
```
`simplexContact`/`simplexChannel` are arrays (a name can advertise multiple SMP
servers; clients try them in order). On-chain they're a single comma-separated
text record; the resolver splits/trims/drops-empties. Address encodings are
canonical per chain (EIP-55 / bech32 / SS58 / Monero-base58). Subnames work
identically (`bar.foobar.testing`).
### Status codes
| Status | Meaning |
|---|---|
| 200 | resolved |
| 400 | TLD not configured, or not a fully-qualified name |
| 404 | name has no resolver set on the registry |
| 502 | upstream RPC error / reth not synced |
### Configuring registries
Defaults to mainnet `.testing` (`0x03f438…`); `.simplex` is unset until
deployed. Override per TLD via env on the `resolver` service in
`docker-compose.yml` (`SNRC_REGISTRY_TESTING` / `SNRC_REGISTRY_SIMPLEX`), or as
env vars for the standalone script.
+160
View File
@@ -0,0 +1,160 @@
services:
# One-shot setup (runs as root): generates /jwt/jwt.hex and chowns the
# nimbus-data volume to UID 1000 (the user Nimbus runs as inside its image).
# Without this chown Nimbus gets "Permission denied" on its data dir
# because docker creates fresh named volumes owned by root.
init:
image: alpine:latest
volumes:
- jwt:/jwt
- nimbus-data:/nimbus-data
command: >
sh -c '
set -e;
if [ ! -f /jwt/jwt.hex ]; then
apk add --no-cache openssl >/dev/null;
openssl rand -hex 32 | tr -d "\n" > /jwt/jwt.hex;
chmod 644 /jwt/jwt.hex;
echo "Generated /jwt/jwt.hex";
else
echo "jwt.hex already exists";
fi;
chown 1000:1000 /nimbus-data;
echo "Chowned /nimbus-data to 1000:1000";
'
restart: "no"
# One-shot: fetches a recent finalised checkpoint into the Nimbus data dir
# using the trustedNodeSync subcommand. Skipped if the data dir is already
# initialised, so subsequent compose-ups are no-ops.
nimbus-checkpoint-sync:
image: statusim/nimbus-eth2:multiarch-latest
depends_on:
init:
condition: service_completed_successfully
volumes:
- nimbus-data:/home/user/nimbus-eth2/build/data
entrypoint:
- sh
- -c
- |
if [ -d /home/user/nimbus-eth2/build/data/${NETWORK}/db ]; then
echo "Nimbus data dir already initialised — skipping checkpoint sync";
exit 0;
fi;
/home/user/nimbus-eth2/build/nimbus_beacon_node trustedNodeSync \
--network=${NETWORK} \
--data-dir=/home/user/nimbus-eth2/build/data/${NETWORK} \
--trusted-node-url=${TRUSTED_NODE_URL} \
--backfill=false
restart: "no"
# One-shot: downloads a pre-synced snapshot from snapshots.reth.rs into the
# Reth data dir. Turns a multi-day from-scratch sync into a ~hour download.
# Skipped if the data dir is already initialised — re-runs are no-ops.
# Privacy note: snapshots.reth.rs sees this download (operator existence).
# Subsequent eth_call traffic stays local.
reth-snapshot-init:
image: ghcr.io/paradigmxyz/reth:latest
depends_on:
init:
condition: service_completed_successfully
volumes:
- reth-data:/data
entrypoint:
- sh
- -c
- |
if [ -f /data/.snapshot-done ] || [ -d /data/db ]; then
echo "Reth data already initialised — skipping snapshot download";
exit 0;
fi;
echo "Downloading Reth ${NETWORK} --minimal snapshot...";
reth download --datadir /data --chain ${NETWORK} --minimal && \
touch /data/.snapshot-done && \
echo "Snapshot download complete"
restart: "no"
reth:
image: ghcr.io/paradigmxyz/reth:latest
depends_on:
reth-snapshot-init:
condition: service_completed_successfully
volumes:
- reth-data:/data
- jwt:/jwt:ro
ports:
# JSON-RPC for smp-server. Bound to loopback — put Caddy in front for remote access.
- "127.0.0.1:8545:8545"
# p2p (Ethereum network). Open these on your firewall for sync.
- "30303:30303/tcp"
- "30303:30303/udp"
command: >
node
--datadir /data
--chain ${NETWORK}
--minimal
--authrpc.jwtsecret /jwt/jwt.hex
--authrpc.addr 0.0.0.0 --authrpc.port 8551
--http
--http.addr 0.0.0.0 --http.port 8545
--http.api eth,net
--rpc.gascap 50000000
--port 30303
--discovery.port 30303
restart: unless-stopped
nimbus:
image: statusim/nimbus-eth2:multiarch-latest
depends_on:
nimbus-checkpoint-sync:
condition: service_completed_successfully
volumes:
- nimbus-data:/home/user/nimbus-eth2/build/data
- jwt:/jwt:ro
ports:
- "9000:9000/tcp"
- "9000:9000/udp"
- "127.0.0.1:5052:5052"
command: >
--network=${NETWORK}
--data-dir=/home/user/nimbus-eth2/build/data/${NETWORK}
--el=http://reth:8551
--jwt-secret=/jwt/jwt.hex
--non-interactive
--rest --rest-address=0.0.0.0 --rest-port=5052
--nat=${NAT:-any}
restart: unless-stopped
# SNRC REST resolver. Talks to reth on the compose-internal network,
# exposes /resolve and /health on 127.0.0.1:8000 by default. The
# smp-server points its [NAMES] resolver_endpoint at this URL.
# To change the host port, edit the LEFT side of the port mapping below.
resolver:
build:
context: ./service
dockerfile: Dockerfile
depends_on:
# reth's `service_started` is sufficient — the resolver tolerates
# eth_call failures gracefully (returns 502 with the error body), so
# starting before reth has finished snapshot replay just yields a few
# 502s until the chain is queryable. The upstream reth image doesn't
# ship a HEALTHCHECK, so we can't gate on healthy.
reth:
condition: service_started
environment:
SNRC_RPC: http://reth:8545
SNRC_BIND: 0.0.0.0
# Registry addresses cascade through the script's own defaults
# (mainnet `.testing`; `.simplex` unconfigured). Set explicitly here
# only if you're deploying against a different network or contract.
# SNRC_REGISTRY_TESTING: 0x...
# SNRC_REGISTRY_SIMPLEX: 0x...
ports:
- "127.0.0.1:8000:8000"
restart: unless-stopped
volumes:
reth-data:
nimbus-data:
jwt:
+48
View File
@@ -0,0 +1,48 @@
# syntax=docker/dockerfile:1.7
# ---------- builder ----------
# Use the official uv image (Astral) on top of a slim Python base.
# uv resolves and installs the lockfile-free pyproject.toml in seconds and
# produces a portable .venv we can copy into the runtime stage.
FROM ghcr.io/astral-sh/uv:python3.13-bookworm-slim AS builder
ENV UV_LINK_MODE=copy \
UV_COMPILE_BYTECODE=1 \
UV_PYTHON_DOWNLOADS=never \
UV_NO_PROGRESS=1
WORKDIR /app
# Install deps first (separate layer) — script edits won't bust this cache.
COPY pyproject.toml ./
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --no-dev --no-install-project
# Script is added after the dep layer for cache friendliness.
COPY snrc-resolve.py ./
# ---------- runtime ----------
# Slim runtime — only the venv + script. No uv, no apt.
FROM python:3.13-slim AS runtime
ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1 \
PATH="/app/.venv/bin:$PATH"
# Non-root user (matches resolver privacy posture: it has no need for root).
RUN groupadd --system --gid 10001 snrc && \
useradd --system --uid 10001 --gid snrc --no-create-home --shell /usr/sbin/nologin snrc
WORKDIR /app
COPY --from=builder --chown=snrc:snrc /app /app
USER snrc:snrc
EXPOSE 8000
# Liveness check hits the script's own /health route. ThreadingHTTPServer is
# fast enough that 3s is generous for a localhost probe; restart if it stops
# responding entirely.
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
CMD ["python", "-c", "import urllib.request, sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=3).status == 200 else 1)"]
ENTRYPOINT ["python", "snrc-resolve.py"]
+13
View File
@@ -0,0 +1,13 @@
[project]
name = "snrc-resolve"
version = "0.1.0"
description = "SimpleX Namespace (SNRC) resolver — REST API over ENS-shaped Ethereum registries"
readme = "README.md"
requires-python = ">=3.11"
license = "AGPL-3.0-only"
dependencies = [
"eth-hash[pycryptodome]>=0.7",
]
[tool.uv]
package = false
+517
View File
@@ -0,0 +1,517 @@
#!/usr/bin/env python3
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "eth-hash[pycryptodome]>=0.7",
# ]
# ///
"""SimpleX Namespace (SNRC) resolver — REST API.
Resolves names like `alice.testing` / `bob.simplex` against the SNRC
deployment on Ethereum mainnet (or any compatible ENS-shaped registry)
and returns a flat JSON document with these fields:
name, nickname, website, location,
simplexContact, simplexChannel, -- list[str], primary first
eth, btc, xmr, dot,
owner, resolver
`simplexContact` and `simplexChannel` are arrays so a name can advertise
multiple SMP servers for redundancy. Clients SHOULD try the URLs in the
order returned. The on-chain text record stores them as a single
`LINK_SEPARATOR` (`;`)-joined string; this resolver splits and trims into a list.
All keys are valid Haskell record-field identifiers (lowercase initial,
no dots), so consumers can derive aeson FromJSON instances directly
without a key-rewriting layer.
Usage:
./snrc-resolve.py # serve on :8000
curl -s http://127.0.0.1:8000/resolve/foobar.testing | jq .
curl -s http://127.0.0.1:8000/health
Environment:
SNRC_RPC JSON-RPC endpoint (default: http://127.0.0.1:8545)
SNRC_REGISTRY_TESTING ENSRegistry for the .testing deployment
(default: mainnet,
0x58fc46996d975c57883564648bda5206d1a0102b)
SNRC_REGISTRY_SIMPLEX ENSRegistry for the .simplex deployment
(default: empty — TLD not yet deployed)
SNRC_PORT Listen port (default: 8000)
SNRC_BIND Bind address (default: 0.0.0.0)
Each TLD is a separate SNRC deployment with its own ENSRegistry; the
resolver dispatches by the queried name's rightmost label.
Dependencies are declared inline (PEP 723) at the top of this file. Run with:
uv run snrc-resolve.py # uv resolves & caches deps; one-line setup
python snrc-resolve.py # if eth-hash[pycryptodome] is already installed
Addresses are returned in each chain's canonical presentation:
eth EIP-55 mixed-case checksummed hex (e.g. 0xEa65A0…1572)
btc bech32(m) for segwit/taproot, base58check for P2PKH/P2SH
(e.g. bc1q… / 1A1zP1…)
dot SS58 with Polkadot network prefix 0 (e.g. 15oF4u…)
xmr Monero base58 (e.g. 4Aux5y…)
Unrecognised payloads fall back to `0x`-prefixed raw hex.
"""
import hashlib
import json
import os
import sys
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from urllib.parse import unquote, urlparse
from urllib.request import Request, urlopen
from eth_hash.auto import keccak
RPC = os.environ.get("SNRC_RPC", "http://127.0.0.1:8545")
BIND = os.environ.get("SNRC_BIND", "0.0.0.0")
PORT = int(os.environ.get("SNRC_PORT", "8000"))
# Each TLD is its own SNRC deployment with its own ENSRegistry. Dispatch
# happens on the rightmost label of the queried name. Empty / unset means
# "not deployed" — requests for that TLD return 400 with a clear error.
# `... or "..."` makes the script's defaults the single source of truth:
# unset AND empty-string both fall through to the literal. docker-compose
# can therefore pass `SNRC_REGISTRY_TESTING=${SNRC_REGISTRY_TESTING:-}`
# without duplicating the registry address.
REGISTRIES = {
"testing": os.environ.get("SNRC_REGISTRY_TESTING", "")
or "0x58fc46996d975c57883564648bda5206d1a0102b", # mainnet .testing
"simplex": os.environ.get("SNRC_REGISTRY_SIMPLEX", ""), # not deployed yet
}
# SLIP-44 coin types (https://github.com/satoshilabs/slips/blob/master/slip-0044.md)
COIN_ETH = 60
COIN_BTC = 0
COIN_XMR = 128
COIN_DOT = 354
ZERO_ADDR = "0x0000000000000000000000000000000000000000"
# ---------- RPC + ABI helpers (mirrors ens-lookup.py shape) ----------
def rpc(method, params):
body = json.dumps(
{"jsonrpc": "2.0", "method": method, "params": params, "id": 1}
).encode()
# Set a non-default User-Agent; Cloudflare-fronted public RPCs (drpc,
# publicnode, etc.) reject `Python-urllib/3.x` with 403.
req = Request(
RPC,
data=body,
headers={
"Content-Type": "application/json",
"User-Agent": "snrc-resolve/1.0",
},
)
res = json.loads(urlopen(req, timeout=15).read())
if "error" in res:
raise RuntimeError(res["error"])
return res["result"]
def namehash(name: str) -> bytes:
node = b"\x00" * 32
if name:
for label in reversed(name.split(".")):
node = keccak(node + keccak(label.encode()))
return node
def selector(signature: str) -> str:
return "0x" + keccak(signature.encode())[:4].hex()
def eth_call(to: str, data: str) -> str:
return rpc("eth_call", [{"to": to, "data": data}, "latest"])
def decode_address(hex_data: str) -> str:
return "0x" + hex_data[-40:]
def decode_bytes(hex_data: str) -> bytes:
raw = bytes.fromhex(hex_data[2:] if hex_data.startswith("0x") else hex_data)
if len(raw) < 64:
return b""
length = int.from_bytes(raw[32:64], "big")
return raw[64:64 + length]
def encode_text_call(node: bytes, key: str) -> str:
sel = selector("text(bytes32,string)")
head = node.hex() + (0x40).to_bytes(32, "big").hex()
key_bytes = key.encode()
body = len(key_bytes).to_bytes(32, "big").hex() + key_bytes.hex()
body += "00" * ((-len(key_bytes)) % 32)
return sel + head + body
def text(resolver: str, node: bytes, key: str) -> str:
raw = decode_bytes(eth_call(resolver, encode_text_call(node, key)))
return raw.decode("utf-8", errors="replace") if raw else ""
def encode_addr_multicoin_call(node: bytes, coin_type: int) -> str:
"""ENSIP-9 addr(bytes32 node, uint256 coinType) — both static, no offsets."""
return (
selector("addr(bytes32,uint256)")
+ node.hex()
+ coin_type.to_bytes(32, "big").hex()
)
def addr_multicoin(resolver: str, node: bytes, coin_type: int):
"""Read ENSIP-9 raw bytes for `coinType`, then encode to that chain's
canonical presentation form. Falls back to `0x`-prefixed hex if the
payload doesn't match any recognised on-chain shape. Returns None when
the record is unset."""
try:
raw = decode_bytes(eth_call(resolver, encode_addr_multicoin_call(node, coin_type)))
except RuntimeError:
return None
if not raw:
return None
# An all-zero payload is the ENS convention for "unset" — many tools
# write 20 zero bytes for coinType=60 instead of clearing the slot.
# Treat it as null so the response doesn't surface a zero address.
if raw == b"\x00" * len(raw):
return None
encoder = COIN_ENCODERS.get(coin_type)
if encoder is None:
return "0x" + raw.hex()
try:
return encoder(raw) or ("0x" + raw.hex())
except Exception:
return "0x" + raw.hex()
# ---------- Coin-specific address encoders ----------
# Each takes raw bytes as stored under ENSIP-9 and returns the canonical
# user-facing string for that chain (EIP-55 for ETH, bech32/base58check
# for BTC, SS58 for DOT, Monero-base58 for XMR). All stdlib + eth_hash.
B58_ALPHA = "123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz"
def _b58_encode(b: bytes) -> str:
n = int.from_bytes(b, "big")
out = ""
while n:
n, r = divmod(n, 58)
out = B58_ALPHA[r] + out
# leading zero bytes → leading '1's
pad = len(b) - len(b.lstrip(b"\x00"))
return "1" * pad + out
def _b58check_encode(payload: bytes) -> str:
"""Base58Check used by BTC legacy/P2SH: payload + dSHA256(payload)[:4]."""
chk = hashlib.sha256(hashlib.sha256(payload).digest()).digest()[:4]
return _b58_encode(payload + chk)
# ---- Bech32 / Bech32m (BIP-173 / BIP-350) ----
_BECH32_CHARSET = "qpzry9x8gf2tvdw0s3jn54khce6mua7l"
_BECH32_GEN = [0x3B6A57B2, 0x26508E6D, 0x1EA119FA, 0x3D4233DD, 0x2A1462B3]
def _bech32_polymod(values):
chk = 1
for v in values:
b = chk >> 25
chk = ((chk & 0x1FFFFFF) << 5) ^ v
for i in range(5):
if (b >> i) & 1:
chk ^= _BECH32_GEN[i]
return chk
def _bech32_hrp_expand(hrp):
return [ord(c) >> 5 for c in hrp] + [0] + [ord(c) & 31 for c in hrp]
def _bech32_create_checksum(hrp, data, spec):
const = 1 if spec == "bech32" else 0x2BC830A3 # bech32m
values = _bech32_hrp_expand(hrp) + data + [0] * 6
polymod = _bech32_polymod(values) ^ const
return [(polymod >> 5 * (5 - i)) & 31 for i in range(6)]
def _bech32_encode(hrp, data, spec):
combined = data + _bech32_create_checksum(hrp, data, spec)
return hrp + "1" + "".join(_BECH32_CHARSET[d] for d in combined)
def _convertbits(data, frombits, tobits, pad=True):
acc = 0
bits = 0
ret = []
maxv = (1 << tobits) - 1
max_acc = (1 << (frombits + tobits - 1)) - 1
for value in data:
if value < 0 or (value >> frombits):
return None
acc = ((acc << frombits) | value) & max_acc
bits += frombits
while bits >= tobits:
bits -= tobits
ret.append((acc >> bits) & maxv)
if pad and bits:
ret.append((acc << (tobits - bits)) & maxv)
elif not pad and (bits >= frombits or ((acc << (tobits - bits)) & maxv)):
return None
return ret
def _segwit_encode(hrp: str, witver: int, witprog: bytes) -> str:
spec = "bech32" if witver == 0 else "bech32m"
data = [witver] + _convertbits(list(witprog), 8, 5)
return _bech32_encode(hrp, data, spec)
# ---- BTC scriptPubKey → address ----
# ENSIP-9 stores the raw output script. Dispatch by length + opcode prefix.
def _btc_encode(raw: bytes) -> str | None:
hrp = "bc" # mainnet
if len(raw) == 25 and raw[:3] == b"\x76\xa9\x14" and raw[23:25] == b"\x88\xac":
return _b58check_encode(b"\x00" + raw[3:23]) # P2PKH
if len(raw) == 23 and raw[:2] == b"\xa9\x14" and raw[22:23] == b"\x87":
return _b58check_encode(b"\x05" + raw[2:22]) # P2SH
if len(raw) == 22 and raw[:2] == b"\x00\x14":
return _segwit_encode(hrp, 0, raw[2:22]) # P2WPKH
if len(raw) == 34 and raw[:2] == b"\x00\x20":
return _segwit_encode(hrp, 0, raw[2:34]) # P2WSH
if len(raw) == 34 and raw[:2] == b"\x51\x20":
return _segwit_encode(hrp, 1, raw[2:34]) # P2TR
return None
# ---- Polkadot SS58 ----
# Per SS58 spec: base58( prefix_byte + pubkey + blake2b-512("SS58PRE" + body)[:2] )
# Polkadot mainnet uses network prefix 0 (single byte); Kusama uses 2.
_SS58_PRE = b"SS58PRE"
def _ss58_encode(pubkey: bytes, network_prefix: int = 0) -> str:
if len(pubkey) != 32:
return None
body = bytes([network_prefix]) + pubkey
checksum = hashlib.blake2b(_SS58_PRE + body, digest_size=64).digest()[:2]
return _b58_encode(body + checksum)
def _dot_encode(raw: bytes) -> str | None:
return _ss58_encode(raw, network_prefix=0)
# ---- Monero base58 ----
# Monero base58 encodes in 8-byte blocks; each full block → 11 chars, partial
# block sizes per fixed table. Alphabet is identical to Bitcoin's.
_XMR_BLOCK_SIZES = [0, 2, 3, 5, 6, 7, 9, 10, 11]
def _xmr_encode(raw: bytes) -> str:
out = []
for i in range(0, len(raw), 8):
chunk = raw[i:i + 8]
n = int.from_bytes(chunk, "big")
width = 11 if len(chunk) == 8 else _XMR_BLOCK_SIZES[len(chunk)]
block = []
for _ in range(width):
n, r = divmod(n, 58)
block.append(B58_ALPHA[r])
out.append("".join(reversed(block)))
return "".join(out)
# ---- ETH EIP-55 mixed-case checksum ----
def _eth_encode(raw: bytes) -> str | None:
if len(raw) != 20:
return None
hex_addr = raw.hex()
hash_hex = keccak(hex_addr.encode()).hex()
return "0x" + "".join(
c.upper() if c.isalpha() and int(hash_hex[i], 16) >= 8 else c
for i, c in enumerate(hex_addr)
)
COIN_ENCODERS = {
COIN_ETH: _eth_encode,
COIN_BTC: _btc_encode,
COIN_XMR: _xmr_encode,
COIN_DOT: _dot_encode,
}
# ---------- Resolution logic ----------
# Text-record keys we read from the resolver. Surfaced under the response
# field names listed in the docstring above. `name` and `description` are
# common ENS fallbacks for a human-readable nickname.
TEXT_KEYS = [
"name",
"nickname",
"description",
"url",
"location",
"simplex.contact",
"simplex.channel",
]
# Separator that joins the SMP-server URL list inside a simplex.contact /
# simplex.channel text record. MUST match SIMPLEX_LINK_SEPARATOR in the dApp
# (ens-app-v3 src/constants/simplex.ts) — the two sides decode the same record.
LINK_SEPARATOR = ";"
def split_links(value: str) -> list:
"""Split a separator-joined text record into an ordered list of entries.
Trims whitespace around each element and drops empties so trailing
separators, doubled separators, and all-whitespace inputs all yield clean
output. Single-value records yield a 1-element list; empty inputs
yield `[]`. Used for `simplex.contact` / `simplex.channel`, which
store one-or-more SMP-server URLs as a single `LINK_SEPARATOR`-joined string.
"""
return [item.strip() for item in value.split(LINK_SEPARATOR) if item.strip()]
def resolve(name: str):
tld = name.rsplit(".", 1)[-1]
registry = REGISTRIES.get(tld)
if not registry:
configured = [k for k, v in REGISTRIES.items() if v]
return 400, {
"name": name,
"error": f"TLD '{tld}' is not configured on this resolver",
"configured_tlds": configured,
}
node = namehash(name)
node_hex = node.hex()
resolver_raw = eth_call(registry, selector("resolver(bytes32)") + node_hex)
resolver_addr = decode_address(resolver_raw)
if resolver_addr == ZERO_ADDR:
return 404, {"name": name, "error": "no resolver set for this name"}
owner_raw = eth_call(registry, selector("owner(bytes32)") + node_hex)
owner = decode_address(owner_raw)
texts = {}
for k in TEXT_KEYS:
try:
v = text(resolver_addr, node, k)
except RuntimeError:
v = ""
if v:
texts[k] = v
# The user-facing "nickname" prefers an explicit `nickname` record,
# falls back to `name`, then `description` (ENSIP-5 convention).
nickname = texts.get("nickname") or texts.get("name") or texts.get("description") or ""
# Keys chosen to be valid Haskell record-field identifiers (lowercase
# initial, no dots) so consumers can derive aeson FromJSON instances
# without a key-rewriting layer. On-chain text-record names still
# use the ENSIP-5 dot convention (e.g. "simplex.contact") — only the
# resolver's JSON surface camelCases them.
return 200, {
"name": name,
"nickname": nickname,
"website": texts.get("url", ""),
"location": texts.get("location", ""),
"simplexContact": split_links(texts.get("simplex.contact", "")),
"simplexChannel": split_links(texts.get("simplex.channel", "")),
"eth": addr_multicoin(resolver_addr, node, COIN_ETH),
"btc": addr_multicoin(resolver_addr, node, COIN_BTC),
"xmr": addr_multicoin(resolver_addr, node, COIN_XMR),
"dot": addr_multicoin(resolver_addr, node, COIN_DOT),
"owner": owner,
"resolver": resolver_addr,
}
# ---------- HTTP layer ----------
class Handler(BaseHTTPRequestHandler):
def do_GET(self): # noqa: N802 - http.server contract
path = urlparse(self.path).path
parts = [unquote(p) for p in path.split("/") if p]
if parts == ["health"]:
self._respond(
200,
{"ok": True, "rpc": RPC, "registries": REGISTRIES},
)
return
if len(parts) == 2 and parts[0] == "resolve":
name = parts[1].strip().lower()
if not name or "." not in name:
self._respond(
400,
{
"error": "expected fully-qualified name, e.g. /resolve/alice.testing",
"got": name,
},
)
return
try:
status, body = resolve(name)
except Exception as e: # surface upstream errors as 502
status, body = 502, {"name": name, "error": f"{type(e).__name__}: {e}"}
self._respond(status, body)
return
self._respond(
404,
{"error": "not found", "routes": ["/health", "/resolve/<name>"]},
)
def _respond(self, status: int, body: dict):
data = json.dumps(body, indent=2).encode()
self.send_response(status)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(data)))
self.end_headers()
self.wfile.write(data)
def log_message(self, fmt, *args):
# Quiet the default per-request access log; route to stderr in one line.
sys.stderr.write(f"{self.address_string()} - {fmt % args}\n")
def main():
server = ThreadingHTTPServer((BIND, PORT), Handler)
sys.stderr.write(
f"snrc-resolve listening on {BIND}:{PORT}\n"
f" RPC = {RPC}\n"
f" Registries:\n"
)
for tld, addr in REGISTRIES.items():
sys.stderr.write(f" .{tld:<8s} = {addr or '(not configured)'}\n")
sys.stderr.write(" GET /resolve/<name> GET /health\n")
try:
server.serve_forever()
except KeyboardInterrupt:
sys.stderr.write("\nshutting down\n")
server.server_close()
if __name__ == "__main__":
main()
@@ -0,0 +1,86 @@
#!/usr/bin/env python3
"""Unit tests for snrc-resolve helpers.
Run with `python3 -m unittest scripts/resolver/service/test_snrc_resolve.py`.
"""
import importlib.util
import os
import unittest
# snrc-resolve.py has a hyphen, so import it via importlib instead of `import`.
_HERE = os.path.dirname(os.path.abspath(__file__))
_SPEC = importlib.util.spec_from_file_location(
"snrc_resolve", os.path.join(_HERE, "snrc-resolve.py")
)
snrc = importlib.util.module_from_spec(_SPEC)
_SPEC.loader.exec_module(snrc)
class SplitLinksTests(unittest.TestCase):
"""`split_links` decodes the multi-URL convention for simplex.contact /
simplex.channel text records. Reuses the same rule the dApp's
`parseSimplexUrls` uses (separator `;`), so the two sides round-trip
cleanly."""
def test_empty_string_yields_empty_list(self):
self.assertEqual(snrc.split_links(""), [])
def test_whitespace_only_yields_empty_list(self):
self.assertEqual(snrc.split_links(" "), [])
self.assertEqual(snrc.split_links(" ; ; "), [])
def test_single_url_yields_singleton_list(self):
self.assertEqual(
snrc.split_links("https://smp16.simplex.im/a#H1"),
["https://smp16.simplex.im/a#H1"],
)
def test_two_urls_split_on_separator(self):
self.assertEqual(
snrc.split_links(
"https://smp16.simplex.im/a#H1;https://smp19.simplex.im/a#H1"
),
[
"https://smp16.simplex.im/a#H1",
"https://smp19.simplex.im/a#H1",
],
)
def test_whitespace_around_separators_is_trimmed(self):
self.assertEqual(
snrc.split_links(
" https://smp16.simplex.im/a#H1 ;\thttps://smp19.simplex.im/a#H1 "
),
[
"https://smp16.simplex.im/a#H1",
"https://smp19.simplex.im/a#H1",
],
)
def test_trailing_separator_does_not_produce_empty_entry(self):
self.assertEqual(
snrc.split_links("https://smp16.simplex.im/a#H1;"),
["https://smp16.simplex.im/a#H1"],
)
def test_doubled_separator_does_not_produce_empty_entry(self):
self.assertEqual(
snrc.split_links(
"https://smp16.simplex.im/a#H1;;https://smp19.simplex.im/a#H1"
),
[
"https://smp16.simplex.im/a#H1",
"https://smp19.simplex.im/a#H1",
],
)
def test_order_is_preserved(self):
self.assertEqual(
snrc.split_links("c;a;b"),
["c", "a", "b"],
)
if __name__ == "__main__":
unittest.main()
+168
View File
@@ -0,0 +1,168 @@
#!/usr/bin/env python3
"""Resolve an ENS name via local Reth (the same shape SNRC will use).
Usage:
./ens-lookup.py # defaults to simplexchat.eth
./ens-lookup.py vitalik.eth
./ens-lookup.py corevo.eth
Requires: pip install --break-system-packages 'eth-hash[pycryptodome]'
"""
import base64
import json
import sys
from urllib.request import Request, urlopen
from eth_hash.auto import keccak
RPC = "http://127.0.0.1:8545"
# ENS Registry (current, post-2020 migration)
ENS_REGISTRY = "0x00000000000C2E074eC69A0dFb2997BA6C7d2e1e"
def rpc(method, params):
body = json.dumps({"jsonrpc": "2.0", "method": method, "params": params, "id": 1}).encode()
req = Request(RPC, data=body, headers={"Content-Type": "application/json"})
res = json.loads(urlopen(req, timeout=15).read())
if "error" in res:
raise RuntimeError(res["error"])
return res["result"]
def namehash(name: str) -> bytes:
"""ENS namehash — recursive keccak256 over reversed labels."""
node = b"\x00" * 32
if name:
for label in reversed(name.split(".")):
node = keccak(node + keccak(label.encode()))
return node
def selector(signature: str) -> str:
return "0x" + keccak(signature.encode())[:4].hex()
def eth_call(to: str, data: str) -> str:
return rpc("eth_call", [{"to": to, "data": data}, "latest"])
def decode_address(hex_data: str) -> str:
return "0x" + hex_data[-40:]
def decode_bytes(hex_data: str) -> bytes:
raw = bytes.fromhex(hex_data[2:] if hex_data.startswith("0x") else hex_data)
if len(raw) < 64:
return b""
length = int.from_bytes(raw[32:64], "big")
return raw[64:64 + length]
def encode_text_call(node: bytes, key: str) -> str:
"""ABI-encode text(bytes32 node, string key). String arg is dynamic:
offset (=0x40) + length + right-padded data."""
sel = selector("text(bytes32,string)")
head = node.hex() + (0x40).to_bytes(32, "big").hex()
key_bytes = key.encode()
body = len(key_bytes).to_bytes(32, "big").hex() + key_bytes.hex()
# right-pad to 32-byte boundary
pad = (-len(key_bytes)) % 32
body += "00" * pad
return sel + head + body
def text(resolver: str, node: bytes, key: str) -> str:
raw = decode_bytes(eth_call(resolver, encode_text_call(node, key)))
return raw.decode("utf-8", errors="replace") if raw else ""
# Common ENS text keys (ENSIP-5). Resolvers may return empty for any of these.
TEXT_KEYS = [
"url",
"avatar",
"description",
"email",
"notice",
"keywords",
"com.twitter",
"com.github",
"com.discord",
"org.telegram",
"io.keybase",
"xyz.farcaster",
]
def decode_contenthash(raw: bytes) -> str:
"""ENS contenthash → human-readable URI (best-effort)."""
if not raw:
return "(empty)"
# Multicodec prefixes:
# 0xe301 = ipfs-ns + dag-pb (CIDv0/v1)
# 0xe501 = ipns-ns
# 0xe40101701b... = swarm
if raw[:2] == b"\xe3\x01":
cid_bytes = raw[2:]
# Base32 lowercase + 'b' prefix per CIDv1 spec
b32 = base64.b32encode(cid_bytes).decode().lower().rstrip("=")
return f"ipfs://b{b32}"
if raw[:2] == b"\xe5\x01":
cid_bytes = raw[2:]
b32 = base64.b32encode(cid_bytes).decode().lower().rstrip("=")
return f"ipns://b{b32}"
return "0x" + raw.hex()
def main():
name = sys.argv[1] if len(sys.argv) > 1 else "simplexchat.eth"
print(f" name: {name}")
node = namehash(name)
print(f" namehash: 0x{node.hex()}")
# 1. Ask the registry which resolver is responsible for this name
resolver_data = selector("resolver(bytes32)") + node.hex()
resolver_raw = eth_call(ENS_REGISTRY, resolver_data)
resolver = decode_address(resolver_raw)
print(f" resolver: {resolver}")
if resolver == "0x0000000000000000000000000000000000000000":
print(" → no resolver set for this name")
return
node_hex = node.hex()
# 2. Ask the resolver for the address
try:
addr = decode_address(eth_call(resolver, selector("addr(bytes32)") + node_hex))
print(f" address: {addr}")
except Exception as e:
print(f" address: (error: {e})")
# 3. Ask the resolver for the content hash (IPFS pointer)
try:
ch = decode_bytes(eth_call(resolver, selector("contenthash(bytes32)") + node_hex))
print(f" contenthash: {decode_contenthash(ch)}")
except Exception as e:
print(f" contenthash: (not supported: {e})")
# 4. Owner from the registry
try:
owner = decode_address(eth_call(ENS_REGISTRY, selector("owner(bytes32)") + node_hex))
print(f" owner: {owner}")
except Exception as e:
print(f" owner: (error: {e})")
# 5. Text records (EIP-634). Print only the non-empty ones.
print(" text records:")
for key in TEXT_KEYS:
try:
v = text(resolver, node, key)
if v:
print(f" {key:<16s} {v}")
except Exception as e:
print(f" {key:<16s} (error: {e})")
if __name__ == "__main__":
main()
+246
View File
@@ -0,0 +1,246 @@
#!/usr/bin/env python3
"""Sync progress for the Reth + Nimbus stack.
Usage:
./progress.py # continuous (Ctrl-C to exit, auto-exits when synced)
./progress.py --once # single snapshot
Requires Nimbus REST port exposed at 127.0.0.1:5052 (add --rest flag in compose).
"""
import json
import sys
import time
from collections import deque
from datetime import timedelta
from urllib.error import URLError
from urllib.request import Request, urlopen
RETH = "http://127.0.0.1:8545"
NIMBUS = "http://127.0.0.1:5052"
INTERVAL = 5
WINDOW = 60
BAR_W = 40
# ANSI helpers
def c(s, code): return f"\033[{code}m{s}\033[0m"
GREEN, YELLOW, RED, DIM, BOLD = "32", "33", "31", "2;37", "1"
def rpc(method):
body = json.dumps({"jsonrpc": "2.0", "method": method, "params": [], "id": 1}).encode()
req = Request(RETH, data=body, headers={"Content-Type": "application/json"})
return json.loads(urlopen(req, timeout=5).read())["result"]
def get_reth():
try:
r = rpc("eth_syncing")
try:
peers = int(rpc("net_peerCount"), 16)
except Exception:
peers = -1 # net namespace not exposed
if r is False:
head = int(rpc("eth_blockNumber"), 16)
return {"state": "synced", "current": head, "target": head, "peers": peers,
"stage": None, "stages": {}, "err": None}
current = int(r["currentBlock"], 16)
highest = int(r["highestBlock"], 16)
# Build stage map (name -> block).
stages = {s["name"]: int(s["block"], 16) for s in r.get("stages", [])}
active_stages = {k: v for k, v in stages.items() if v > 0}
# Headers download phase: nothing has progressed yet.
if current == 0 and highest == 0 and not active_stages:
return {"state": "headers", "current": 0, "target": 0, "peers": peers,
"stage": "Headers", "stages": stages, "err": None}
# Derive progress from the stages pipeline.
# Bottleneck (rate-limiting stage) = stage with lowest non-zero block.
# Target = leading stage block (typically Headers = chain tip).
# Reth's top-level currentBlock/highestBlock are unreliable during initial
# sync (often 0 until execution stage runs), so prefer stages-derived values.
if active_stages:
bottleneck = min(active_stages, key=active_stages.get)
stage_current = active_stages[bottleneck]
stage_target = max(stages.values()) if stages else 0
# Trust the stages-derived values if highest is unset or stages tip is higher.
if highest <= 0 or stage_target > highest:
current = stage_current
highest = stage_target
elif current <= 0:
current = stage_current
else:
bottleneck = None
return {"state": "syncing", "current": current, "target": highest,
"peers": peers, "stage": bottleneck, "stages": stages, "err": None}
except URLError as e:
return {"state": "down", "current": 0, "target": 0, "peers": 0,
"stage": None, "stages": {}, "err": str(e.reason)}
except Exception as e:
return {"state": "error", "current": 0, "target": 0, "peers": 0,
"stage": None, "stages": {}, "err": str(e)}
def get_nimbus():
try:
d = json.loads(urlopen(f"{NIMBUS}/eth/v1/node/syncing", timeout=5).read())["data"]
peers_d = json.loads(urlopen(f"{NIMBUS}/eth/v1/node/peer_count", timeout=5).read())["data"]
head = int(d["head_slot"])
dist = int(d["sync_distance"])
peers = int(peers_d.get("connected", "0"))
return {"state": "synced" if not d["is_syncing"] else "syncing",
"current": head, "target": head + dist, "peers": peers,
"optimistic": bool(d.get("is_optimistic", False)),
"el_offline": bool(d.get("el_offline", False)),
"err": None}
except URLError as e:
return {"state": "down", "current": 0, "target": 0, "peers": 0,
"optimistic": False, "el_offline": False, "err": str(e.reason)}
except Exception as e:
return {"state": "error", "current": 0, "target": 0, "peers": 0,
"optimistic": False, "el_offline": False, "err": str(e)}
def format_num(n): return f"{n:,}"
def format_eta(seconds):
if seconds is None: return "?"
if seconds < 0: return "?"
if seconds < 60: return f"{int(seconds)}s"
if seconds < 3600:
return f"{int(seconds // 60)}m {int(seconds % 60)}s"
if seconds < 86400:
return f"{int(seconds // 3600)}h {int((seconds % 3600) // 60)}m"
return f"{int(seconds // 86400)}d {int((seconds % 86400) // 3600)}h"
def rate_per_sec(history):
if len(history) < 2: return None
t0, c0 = history[0]
t1, c1 = history[-1]
if t1 <= t0: return None
return (c1 - c0) / (t1 - t0)
def eta_seconds(history, target):
r = rate_per_sec(history)
if r is None or r <= 0: return None
remaining = target - history[-1][1]
if remaining <= 0: return 0
return remaining / r
def progress_bar(pct):
pct = max(0.0, min(100.0, pct))
filled = int(pct / 100 * BAR_W)
return c("" * filled, GREEN) + c("" * (BAR_W - filled), DIM)
def peers_label(peers):
if peers < 0:
return c("· peers unknown (enable net namespace)", DIM)
return c(f"· {peers} peers", DIM)
def stages_summary(stages):
"""One-line view: stages that have progressed, with their block numbers."""
if not stages:
return ""
advanced = [(n, b) for n, b in stages.items() if b > 0]
if not advanced:
return c(" stages: all 0 (headers downloading)", DIM)
advanced.sort(key=lambda kv: kv[1], reverse=True)
parts = [f"{n}={format_num(b)}" for n, b in advanced[:4]]
return c(" stages: " + ", ".join(parts), DIM)
def render_one(name, x, hist):
state = x["state"]
peers = x.get("peers", 0)
extras = []
if name == "Nimbus":
if x.get("optimistic"):
extras.append(c("(optimistic head — Reth not yet verifying)", YELLOW))
if x.get("el_offline"):
extras.append(c("⚠ EL OFFLINE", RED))
if state == "synced":
out = [f" {c(name, BOLD):<14s} {c('✓ synced', GREEN)} {c(format_num(x['current']), BOLD)} {peers_label(peers)}"]
elif state == "headers":
out = [
f" {c(name, BOLD):<14s} {c('⧗ headers', YELLOW)} {c('downloading initial chain', DIM)} {peers_label(peers)}",
f" {c('(per-block progress unavailable until headers validated — see docker logs)', DIM)}",
]
elif state == "syncing" and x["target"] <= 0:
out = [f" {c(name, BOLD):<14s} {c('⧗ syncing', YELLOW)} {c('waiting for fork-choice', DIM)} {peers_label(peers)}"]
elif state == "syncing":
pct = x["current"] / x["target"] * 100
r = rate_per_sec(hist)
eta = eta_seconds(hist, x["target"])
rate_s = f"{format_num(int(r))} /s" if r and r > 0 else c("stalled", RED)
eta_s = format_eta(eta) if eta is not None else "?"
stage = x.get("stage")
stage_s = c(f"[{stage}]", DIM) if stage else ""
out = [
f" {c(name, BOLD):<14s} {c('⧗ syncing', YELLOW)} {format_num(x['current'])} / {format_num(x['target'])} {stage_s} {peers_label(peers)}",
f" {progress_bar(pct)} {c(f'{pct:6.2f}%', BOLD)}",
f" {c(rate_s, DIM)} ETA {c(eta_s, BOLD)}",
]
else:
out = [
f" {c(name, BOLD):<14s} {c('' + state, RED)}",
f" {c(x.get('err') or '', DIM)}",
]
# Reth-only: stages summary
if name == "Reth" and x.get("stages"):
out.append(f" {stages_summary(x['stages'])}")
for e in extras:
out.append(f" {e}")
return out
def render(reth, nimbus, reth_hist, nim_hist):
print("\033[2J\033[H", end="")
width = 64
title = f"Reth + Nimbus sync"
ts = time.strftime("%H:%M:%S")
print()
print(f" {c(title, BOLD)} {c(ts, DIM)}")
print(f" {c('' * width, DIM)}")
print()
for line in render_one("Reth", reth, reth_hist):
print(line)
print()
for line in render_one("Nimbus", nimbus, nim_hist):
print(line)
print()
win_s = (len(reth_hist) - 1) * INTERVAL if len(reth_hist) > 1 else 0
print(f" {c(f'window {win_s}s · refresh {INTERVAL}s · Ctrl-C to exit', DIM)}")
print()
def main():
once = "--once" in sys.argv
reth_hist = deque(maxlen=WINDOW)
nim_hist = deque(maxlen=WINDOW)
try:
while True:
r = get_reth()
n = get_nimbus()
now = time.time()
if r["target"] > 0 or r["state"] == "syncing":
reth_hist.append((now, r["current"]))
if n["target"] > 0 or n["state"] == "syncing":
nim_hist.append((now, n["current"]))
render(r, n, reth_hist, nim_hist)
if once:
break
if r["state"] == "synced" and n["state"] == "synced":
print(f" {c('✓ all synced.', GREEN)}\n")
break
time.sleep(INTERVAL)
except KeyboardInterrupt:
print()
if __name__ == "__main__":
main()
@@ -22,7 +22,7 @@ smp-server --version
# Initialize server
ip_address=$(curl ifconfig.me)
smp-server init -l --ip $ip_address
smp-server init -l --disable-web --ip $ip_address
# Server fingerprint
fingerprint=$(cat /etc/opt/simplex/fingerprint)
@@ -12,6 +12,11 @@ Check SMP server status with: systemctl status smp-server
To keep this server secure, the UFW firewall is enabled.
All ports are BLOCKED except 22 (SSH), 443 (HTTPS), 5223 (SMP server).
Embedded HTTPS web is disabled because this image does not provision
/etc/opt/simplex/web.crt or /etc/opt/simplex/web.key. To enable it, provision
those files, uncomment WEB https/cert/key in /etc/opt/simplex/smp-server.ini,
and restart smp-server.
********************************************************************************
To stop seeing this message delete line - bash /opt/simplex/on_login.sh - from /root/.bashrc
EOF
+8
View File
@@ -75,6 +75,9 @@ init_opts=()
[[ $ENABLE_STORE_LOG == "on" ]] && init_opts+=(-l)
# This script does not provision /etc/opt/simplex/web.crt or web.key.
init_opts+=(--disable-web)
ip_address=$(curl ifconfig.me)
init_opts+=(--ip $ip_address)
@@ -111,6 +114,11 @@ Check SMP server status with: systemctl status smp-server
To keep this server secure, the UFW firewall is enabled.
All ports are BLOCKED except 22 (SSH), 443 (HTTPS), 5223 (SMP server).
Embedded HTTPS web is disabled because this script does not provision
/etc/opt/simplex/web.crt or /etc/opt/simplex/web.key. To enable it, provision
those files, uncomment WEB https/cert/key in /etc/opt/simplex/smp-server.ini,
and restart smp-server.
********************************************************************************
To stop seeing this message delete line - bash /opt/simplex/on_login.sh - from /root/.bashrc
EOF2
+51 -3
View File
@@ -1,7 +1,7 @@
cabal-version: 1.12
cabal-version: 3.0
name: simplexmq
version: 6.5.2.0
version: 7.0.1.0
synopsis: SimpleXMQ message broker
description: This package includes <./docs/Simplex-Messaging-Server.html server>,
<./docs/Simplex-Messaging-Client.html client> and
@@ -16,7 +16,7 @@ homepage: https://github.com/simplex-chat/simplexmq#readme
author: simplex.chat
maintainer: chat@simplex.chat
copyright: 2020-2022 simplex.chat
license: AGPL-3
license: AGPL-3.0-only
license-file: LICENSE
build-type: Simple
extra-source-files:
@@ -24,6 +24,13 @@ extra-source-files:
CHANGELOG.md
cbits/sha512.h
cbits/sntrup761.h
cbits/blst/**/*.c
cbits/blst/**/*.h
cbits/blst/**/*.s
cbits/blst/**/*.S
cbits/blst/**/*.asm
cbits/libbbs/**/*.c
cbits/libbbs/**/*.h
apps/common/Web/static/index.html
apps/common/Web/static/link.html
apps/common/Web/static/media/apk_icon.png
@@ -82,6 +89,11 @@ flag server_postgres
manual: True
default: False
flag commoncrypto
description: On Apple platforms, use SecRandomCopyBytes (Security.framework) for libbbs randomness. getentropy is a non-public symbol on iOS and triggers App Store rejection (ITMS-90338).
manual: True
default: False
library
exposed-modules:
Simplex.FileTransfer.Agent
@@ -122,6 +134,7 @@ library
Simplex.Messaging.Crypto.File
Simplex.Messaging.Crypto.Lazy
Simplex.Messaging.Crypto.Ratchet
Simplex.Messaging.Crypto.BBS
Simplex.Messaging.Crypto.SNTRUP761
Simplex.Messaging.Crypto.SNTRUP761.Bindings
Simplex.Messaging.Crypto.SNTRUP761.Bindings.Defines
@@ -130,6 +143,7 @@ library
Simplex.Messaging.Crypto.ShortLink
Simplex.Messaging.Encoding
Simplex.Messaging.Encoding.String
Simplex.Messaging.Names.Record
Simplex.Messaging.Notifications.Client
Simplex.Messaging.Notifications.Protocol
Simplex.Messaging.Notifications.Transport
@@ -141,6 +155,7 @@ library
Simplex.Messaging.Server.QueueStore.Postgres.Config
Simplex.Messaging.Server.QueueStore.QueueInfo
Simplex.Messaging.ServiceScheme
Simplex.Messaging.SimplexName
Simplex.Messaging.Session
Simplex.Messaging.SystemTime
Simplex.Messaging.TMap
@@ -261,6 +276,8 @@ library
Simplex.Messaging.Server.MsgStore.Journal.SharedLock
Simplex.Messaging.Server.MsgStore.STM
Simplex.Messaging.Server.MsgStore.Types
Simplex.Messaging.Server.Names
Simplex.Messaging.Server.Names.HttpResolver
Simplex.Messaging.Server.NtfStore
Simplex.Messaging.Server.Prometheus
Simplex.Messaging.Server.QueueStore
@@ -300,11 +317,33 @@ library
ghc-options: -Weverything -Wno-missing-exported-signatures -Wno-missing-import-lists -Wno-missed-specialisations -Wno-all-missed-specialisations -Wno-unsafe -Wno-safe -Wno-missing-local-signatures -Wno-missing-kind-signatures -Wno-missing-deriving-strategies -Wno-monomorphism-restriction -Wno-prepositive-qualified-module -Wno-implicit-prelude -Wno-missing-safe-haskell-mode -Wno-missing-export-lists -Wno-partial-fields -Wcompat -Werror=incomplete-record-updates -Werror=incomplete-patterns -Werror=incomplete-uni-patterns -Werror=missing-home-modules -Werror=missing-methods -Werror=tabs -Wredundant-constraints -Wincomplete-record-updates -Wunused-type-patterns -O2
include-dirs:
cbits
cbits/blst/bindings
cbits/blst/src
cbits/libbbs/include
cbits/libbbs/src
cc-options: -D__BLST_PORTABLE__
if flag(commoncrypto)
cc-options: -DBBS_CRYPTO_CC
frameworks: Security
c-sources:
cbits/sha512.c
cbits/sntrup761.c
cbits/blst/src/server.c
cbits/libbbs/src/bbs.c
cbits/libbbs/src/bbs_ciphersuites.c
cbits/libbbs/src/bbs_util.c
cbits/libbbs/src/compat-string.c
cbits/libbbs/src/sha256.c
cbits/libbbs/src/shake256.c
asm-sources:
cbits/blst/build/assembly.S
extra-libraries:
crypto
if os(windows)
c-sources:
cbits/getentropy_win.c
extra-libraries:
bcrypt
build-depends:
aeson ==2.2.*
, asn1-encoding ==0.9.*
@@ -355,7 +394,10 @@ library
build-depends:
case-insensitive ==1.2.*
, hashable ==1.4.*
, http-client >=0.7 && <0.8
, http-client-tls >=0.3 && <0.4
, ini ==0.4.1
, network-uri >=2.6 && <2.7
, optparse-applicative >=0.15 && <0.17
, process ==1.6.*
, temporary ==1.3.*
@@ -489,6 +531,7 @@ test-suite simplexmq-test
AgentTests.EqInstances
AgentTests.FunctionalAPITests
AgentTests.MigrationTests
AgentTests.ResolveNameTests
AgentTests.ServerChoice
AgentTests.ShortLinkTests
CLITests
@@ -505,9 +548,12 @@ test-suite simplexmq-test
CoreTests.VersionRangeTests
FileDescriptionTests
RemoteControl
NamesResolverServer
RSLVTests
ServerTests
SMPAgentClient
SMPClient
SMPNamesTests
SMPProxyTests
Util
XFTPAgent
@@ -588,6 +634,8 @@ test-suite simplexmq-test
, unliftio
, unliftio-core
, unordered-containers
, wai
, warp
, yaml
default-language: Haskell2010
if flag(server_postgres)
+13 -2
View File
@@ -14,6 +14,7 @@ module Simplex.FileTransfer.Client.Main
( SendOptions (..),
CLIError (..),
xftpClientCLI,
xftpClientDeprecationNotice,
cliSendFile,
cliSendFileOpts,
encodeWebURI,
@@ -69,7 +70,6 @@ import Simplex.Messaging.Encoding
import Simplex.Messaging.Encoding.String (StrEncoding (..))
import Simplex.Messaging.Parsers (parseAll)
import Simplex.Messaging.Protocol (ProtoServerWithAuth (..), ProtocolServer (..), SenderId, SndPrivateAuthKey, XFTPServer, XFTPServerWithAuth)
import Simplex.Messaging.Server.CLI (getCliCommand')
import Simplex.Messaging.Util (groupAllOn, ifM, tshow, whenM)
import System.Exit (exitFailure)
import System.FilePath (splitFileName, (</>))
@@ -81,6 +81,10 @@ import UnliftIO.Directory
xftpClientVersion :: String
xftpClientVersion = "1.0.1"
xftpClientDeprecationNotice :: String
xftpClientDeprecationNotice =
"WARNING: the standalone xftp CLI is experimental and deprecated."
newtype CLIError = CLIError String
deriving (Eq, Show, Exception)
@@ -223,7 +227,13 @@ logCfg = LogConfig {lc_file = Nothing, lc_stderr = True}
xftpClientCLI :: IO ()
xftpClientCLI =
getCliCommand' cliCommandP clientVersion >>= \case
customExecParser
(prefs showHelpOnEmpty)
( info
(helper <*> versionOption <*> cliCommandP)
(header (clientVersion <> "\n" <> xftpClientDeprecationNotice) <> fullDesc)
)
>>= \case
SendFile opts -> runLogE opts $ cliSendFile opts
ReceiveFile opts -> runLogE opts $ cliReceiveFile opts
DeleteFile opts -> runLogE opts $ cliDeleteFile opts
@@ -231,6 +241,7 @@ xftpClientCLI =
RandomFile opts -> cliRandomFile opts
where
clientVersion = "SimpleX XFTP client v" <> xftpClientVersion
versionOption = infoOption clientVersion (long "version" <> short 'v' <> help "Show version")
runLogE :: HasField "verbose" a Bool => a -> ExceptT CLIError IO () -> IO ()
runLogE opts a
+6 -4
View File
@@ -22,11 +22,11 @@ where
import Data.Kind (Type)
import Control.Concurrent.STM
import Control.Monad (forM, void)
import Control.Monad
import qualified Data.Attoparsec.ByteString.Char8 as A
import Data.Int (Int64)
import qualified Data.Map.Strict as M
import Data.Maybe (catMaybes)
import Data.Maybe (catMaybes, isJust)
import Data.Set (Set)
import qualified Data.Set as S
import Data.Word (Word32)
@@ -175,8 +175,10 @@ instance FileStoreClass STMFileStore where
pure $ Just (sId, path, size)
else pure Nothing
getUsedStorage STMFileStore {files} =
M.foldl' (\acc FileRec {fileInfo = FileInfo {size}} -> acc + fromIntegral size) 0 <$> readTVarIO files
getUsedStorage STMFileStore {files} = foldM addSize 0 =<< readTVarIO files
where
addSize acc FileRec {fileInfo = FileInfo {size}, filePath} =
ifM (isJust <$> readTVarIO filePath) (pure $! acc + fromIntegral size) (pure acc)
getFileCount STMFileStore {files} = M.size <$> readTVarIO files
@@ -164,7 +164,7 @@ instance FileStoreClass PostgresFileStore where
getUsedStorage st =
withTransaction (dbStore st) $ \db -> do
[Only total] <- DB.query_ db "SELECT COALESCE(SUM(file_size::BIGINT), 0)::BIGINT FROM files"
[Only total] <- DB.query_ db "SELECT COALESCE(SUM(file_size::BIGINT), 0)::BIGINT FROM files WHERE file_path IS NOT NULL"
pure total
getFileCount st =
+63 -57
View File
@@ -49,6 +49,7 @@ module Simplex.Messaging.Agent
deleteUser,
setUserService,
connRequestPQSupport,
prepareConnectionToCreate,
createConnectionAsync,
setConnShortLinkAsync,
getConnShortLinkAsync,
@@ -65,6 +66,7 @@ module Simplex.Messaging.Agent
setConnShortLink,
deleteConnShortLink,
getConnShortLink,
resolveSimplexName,
getConnLinkPrivKey,
deleteLocalInvShortLink,
changeConnectionUser,
@@ -216,6 +218,7 @@ import Simplex.Messaging.Protocol
ErrorType (AUTH),
MsgBody,
MsgFlags (..),
NameRecord,
NtfServer,
ProtoServerWithAuth (..),
ProtocolServer (..),
@@ -356,9 +359,14 @@ setUserService :: AgentClient -> UserId -> Bool -> AE ()
setUserService c = withAgentEnv c .: setUserService' c
{-# INLINE setUserService #-}
-- | Create SMP agent connection (NEW command) asynchronously, synchronous response is new connection id
createConnectionAsync :: ConnectionModeI c => AgentClient -> UserId -> ACorrId -> Bool -> SConnectionMode c -> CR.InitialKeys -> SubscriptionMode -> AE ConnId
createConnectionAsync c userId aCorrId enableNtfs = withAgentEnv c .:. newConnAsync c userId aCorrId enableNtfs
-- | Create SMP agent connection without queue (to be used with createConnectionAsync).
prepareConnectionToCreate :: AgentClient -> UserId -> Bool -> SConnectionMode c -> PQSupport -> AE ConnId
prepareConnectionToCreate c userId enableNtfs = withAgentEnv c .: newConnNoQueues c userId enableNtfs
{-# INLINE prepareConnectionToCreate #-}
-- | Enqueue NEW command for a prepared connection.
createConnectionAsync :: ConnectionModeI c => AgentClient -> ACorrId -> ConnId -> Bool -> SConnectionMode c -> CR.InitialKeys -> SubscriptionMode -> AE ()
createConnectionAsync c aCorrId connId enableNtfs = withAgentEnv c .:. newConnAsync c aCorrId connId enableNtfs
{-# INLINE createConnectionAsync #-}
-- | Create or update user's contact connection short link (LSET command) asynchronously, no synchronous response
@@ -371,10 +379,9 @@ getConnShortLinkAsync :: AgentClient -> UserId -> ACorrId -> Maybe ConnId -> Con
getConnShortLinkAsync c = withAgentEnv c .:: getConnShortLinkAsync' c
{-# INLINE getConnShortLinkAsync #-}
-- | Join SMP agent connection (JOIN command) asynchronously, synchronous response is new connection id.
-- If connId is provided (for contact URIs), it updates the existing connection record created by getConnShortLinkAsync.
joinConnectionAsync :: AgentClient -> UserId -> ACorrId -> Maybe ConnId -> Bool -> ConnectionRequestUri c -> ConnInfo -> PQSupport -> SubscriptionMode -> AE ConnId
joinConnectionAsync c userId aCorrId connId_ enableNtfs = withAgentEnv c .:: joinConnAsync c userId aCorrId connId_ enableNtfs
-- | Enqueue JOIN command for a prepared connection.
joinConnectionAsync :: AgentClient -> ACorrId -> Bool -> ConnId -> Bool -> ConnectionRequestUri c -> ConnInfo -> PQSupport -> SubscriptionMode -> AE ()
joinConnectionAsync c aCorrId updateConn connId enableNtfs = withAgentEnv c .:: joinConnAsync c aCorrId updateConn connId enableNtfs
{-# INLINE joinConnectionAsync #-}
-- | Allow connection to continue after CONF notification (LET command), no synchronous response
@@ -382,9 +389,9 @@ allowConnectionAsync :: AgentClient -> ACorrId -> ConnId -> ConfirmationId -> Co
allowConnectionAsync c = withAgentEnv c .:: allowConnectionAsync' c
{-# INLINE allowConnectionAsync #-}
-- | Accept contact after REQ notification (ACPT command) asynchronously, synchronous response is new connection id
acceptContactAsync :: AgentClient -> UserId -> ACorrId -> Bool -> ConfirmationId -> ConnInfo -> PQSupport -> SubscriptionMode -> AE ConnId
acceptContactAsync c userId aCorrId enableNtfs = withAgentEnv c .:: acceptContactAsync' c userId aCorrId enableNtfs
-- | Accept contact after REQ notification (ACPT command) asynchronously, for a prepared connection.
acceptContactAsync :: AgentClient -> ACorrId -> ConnId -> Bool -> ConfirmationId -> ConnInfo -> PQSupport -> SubscriptionMode -> AE ()
acceptContactAsync c aCorrId connId enableNtfs = withAgentEnv c .:: acceptContactAsync' c aCorrId connId enableNtfs
{-# INLINE acceptContactAsync #-}
-- | Acknowledge message (ACK command) asynchronously, no synchronous response
@@ -416,8 +423,9 @@ createConnection c nm userId enableNtfs checkNotices = withAgentEnv c .::. newCo
-- Caller provides root signing key pair and link entity ID.
-- Returns the created link and internal params.
-- The link address is fully determined at this point.
prepareConnectionLink :: AgentClient -> UserId -> C.KeyPairEd25519 -> ByteString -> Bool -> Maybe CRClientData -> AE (CreatedConnLink 'CMContact, PreparedLinkParams)
prepareConnectionLink c userId rootKey linkEntityId checkNotices = withAgentEnv c . prepareConnectionLink' c userId rootKey linkEntityId checkNotices
prepareConnectionLink :: AgentClient -> UserId -> C.KeyPairEd25519 -> ByteString -> Bool -> Maybe CRClientData -> Maybe SMPServerWithAuth -> AE (CreatedConnLink 'CMContact, PreparedLinkParams)
prepareConnectionLink c userId rootKey linkEntityId checkNotices clientData srv_ =
withAgentEnv c $ prepareConnectionLink' c userId rootKey linkEntityId checkNotices clientData srv_
{-# INLINE prepareConnectionLink #-}
-- | Create connection for prepared link (single network call).
@@ -440,6 +448,13 @@ getConnShortLink :: AgentClient -> NetworkRequestMode -> UserId -> ConnShortLink
getConnShortLink c = withAgentEnv c .:. getConnShortLink' c
{-# INLINE getConnShortLink #-}
-- | Resolve a SimpleX name (PFWD RSLV). The agent owns server selection: it
-- picks a names-capable server (ServerRoles.names) from the user's nameSrvs, so
-- chat clients just pass the parsed domain.
resolveSimplexName :: AgentClient -> NetworkRequestMode -> UserId -> SimplexDomain -> AE NameRecord
resolveSimplexName c nm userId domain = withAgentEnv c $ resolveSimplexName' c nm userId domain
{-# INLINE resolveSimplexName #-}
getConnLinkPrivKey :: AgentClient -> ConnId -> AE (Maybe C.PrivateKeyEd25519)
getConnLinkPrivKey c = withAgentEnv c . getConnLinkPrivKey' c
{-# INLINE getConnLinkPrivKey #-}
@@ -837,11 +852,10 @@ setUserService' c userId enable = do
unless ok $ throwE $ CMD PROHIBITED "setUserService"
when (changed && not enable) $ withStore' c (`deleteClientServices` userId)
newConnAsync :: ConnectionModeI c => AgentClient -> UserId -> ACorrId -> Bool -> SConnectionMode c -> CR.InitialKeys -> SubscriptionMode -> AM ConnId
newConnAsync c userId corrId enableNtfs cMode pqInitKeys subMode = do
connId <- newConnNoQueues c userId enableNtfs cMode (CR.connPQEncryption pqInitKeys)
newConnAsync :: ConnectionModeI c => AgentClient -> ACorrId -> ConnId -> Bool -> SConnectionMode c -> CR.InitialKeys -> SubscriptionMode -> AM ()
newConnAsync c corrId connId enableNtfs cMode pqInitKeys subMode =
enqueueCommand c corrId connId Nothing $ AClientCommand $ NEW enableNtfs (ACM cMode) pqInitKeys subMode
pure connId
{-# INLINE newConnAsync #-}
newConnNoQueues :: AgentClient -> UserId -> Bool -> SConnectionMode c -> PQSupport -> AM ConnId
newConnNoQueues c userId enableNtfs cMode pqSupport = do
@@ -852,34 +866,21 @@ newConnNoQueues c userId enableNtfs cMode pqSupport = do
-- TODO [short links] TBC, but probably we will need async join for contact addresses as the contact will be created after user confirming the connection,
-- and join should retry, the same as 1-time invitation joins.
joinConnAsync :: AgentClient -> UserId -> ACorrId -> Maybe ConnId -> Bool -> ConnectionRequestUri c -> ConnInfo -> PQSupport -> SubscriptionMode -> AM ConnId
joinConnAsync c userId corrId connId_ enableNtfs cReqUri@CRInvitationUri {} cInfo pqSup subMode = do
when (isJust connId_) $ throwE $ CMD PROHIBITED "joinConnAsync: connId not allowed for invitation URI"
withInvLock c (strEncode cReqUri) "joinConnAsync" $ do
joinConnAsync :: AgentClient -> ACorrId -> Bool -> ConnId -> Bool -> ConnectionRequestUri c -> ConnInfo -> PQSupport -> SubscriptionMode -> AM ()
joinConnAsync c corrId updateConn connId enableNtfs cReqUri@CRInvitationUri {} cInfo pqSup subMode = do
when updateConn $ throwE $ CMD PROHIBITED "joinConnAsync: updateConn not allowed for invitation URI"
withInvLock c (strEncode cReqUri) "joinConnAsync" $
lift (compatibleInvitationUri cReqUri) >>= \case
Just (_, Compatible (CR.E2ERatchetParams v _ _ _), Compatible connAgentVersion) -> do
g <- asks random
let pqSupport = pqSup `CR.pqSupportAnd` versionPQSupport_ connAgentVersion (Just v)
cData = ConnData {userId, connId = "", connAgentVersion, enableNtfs, lastExternalSndId = 0, deleted = False, ratchetSyncState = RSOk, pqSupport}
connId <- withStore c $ \db -> createNewConn db g cData SCMInvitation
enqueueCommand c corrId connId Nothing $ AClientCommand $ JOIN enableNtfs (ACR sConnectionMode cReqUri) pqSupport subMode cInfo
pure connId
Nothing -> throwE $ AGENT A_VERSION
joinConnAsync c userId corrId connId_ enableNtfs cReqUri@(CRContactUri _) cInfo pqSup subMode = do
joinConnAsync c corrId updateConn connId enableNtfs cReqUri@(CRContactUri _) cInfo pqSup subMode =
lift (compatibleContactUri cReqUri) >>= \case
Just (_, Compatible connAgentVersion) -> do
let pqSupport = pqSup `CR.pqSupportAnd` versionPQSupport_ connAgentVersion Nothing
connId <- case connId_ of
Just cId -> do
-- update connection record created by getConnShortLinkAsync
withStore' c $ \db -> updateNewConnJoin db cId connAgentVersion pqSupport enableNtfs
pure cId
Nothing -> do
g <- asks random
let cData = ConnData {userId, connId = "", connAgentVersion, enableNtfs, lastExternalSndId = 0, deleted = False, ratchetSyncState = RSOk, pqSupport}
withStore c $ \db -> createNewConn db g cData SCMInvitation
when updateConn $ withStore' c $ \db -> updateNewConnJoin db connId connAgentVersion pqSupport enableNtfs
enqueueCommand c corrId connId Nothing $ AClientCommand $ JOIN enableNtfs (ACR sConnectionMode cReqUri) pqSupport subMode cInfo
pure connId
Nothing -> throwE $ AGENT A_VERSION
allowConnectionAsync' :: AgentClient -> ACorrId -> ConnId -> ConfirmationId -> ConnInfo -> AM ()
@@ -895,11 +896,11 @@ allowConnectionAsync' c corrId connId confId ownConnInfo =
-- and also it can't be triggered by user concurrently several times in a row. It could be improved similarly to
-- `acceptContact` by creating a new map for invitation locks and taking lock here, and removing `unacceptInvitation`
-- while marking invitation as accepted inside "lock level transaction" after successful `joinConnAsync`.
acceptContactAsync' :: AgentClient -> UserId -> ACorrId -> Bool -> InvitationId -> ConnInfo -> PQSupport -> SubscriptionMode -> AM ConnId
acceptContactAsync' c userId corrId enableNtfs invId ownConnInfo pqSupport subMode = do
acceptContactAsync' :: AgentClient -> ACorrId -> ConnId -> Bool -> InvitationId -> ConnInfo -> PQSupport -> SubscriptionMode -> AM ()
acceptContactAsync' c corrId connId enableNtfs invId ownConnInfo pqSupport subMode = do
Invitation {connReq} <- withStore c $ \db -> getInvitation db "acceptContactAsync'" invId
withStore' c $ \db -> acceptInvitation db invId ownConnInfo
joinConnAsync c userId corrId Nothing enableNtfs connReq ownConnInfo pqSupport subMode `catchAllErrors` \err -> do
joinConnAsync c corrId False connId enableNtfs connReq ownConnInfo pqSupport subMode `catchAllErrors` \err -> do
withStore' c (`unacceptInvitation` invId)
throwE err
@@ -965,10 +966,10 @@ newConn c nm userId enableNtfs checkNotices cMode linkData_ clientData pqInitKey
-- | Prepare connection link for contact mode (no network, no database).
-- Caller provides root signing key pair and link entity ID.
prepareConnectionLink' :: AgentClient -> UserId -> C.KeyPairEd25519 -> ByteString -> Bool -> Maybe CRClientData -> AM (CreatedConnLink 'CMContact, PreparedLinkParams)
prepareConnectionLink' c userId rootKey@(_, plpRootPrivKey) linkEntityId checkNotices clientData = do
prepareConnectionLink' :: AgentClient -> UserId -> C.KeyPairEd25519 -> ByteString -> Bool -> Maybe CRClientData -> Maybe SMPServerWithAuth -> AM (CreatedConnLink 'CMContact, PreparedLinkParams)
prepareConnectionLink' c userId rootKey@(_, plpRootPrivKey) linkEntityId checkNotices clientData srv_ = do
g <- asks random
plpSrvWithAuth@(ProtoServerWithAuth srv _) <- getSMPServer c userId
plpSrvWithAuth@(ProtoServerWithAuth srv _) <- maybe (getSMPServer c userId) pure srv_
when checkNotices $ checkClientNotices c plpSrvWithAuth
AgentConfig {smpClientVRange, smpAgentVRange} <- asks config
plpNonce@(C.CbNonce corrId) <- atomically $ C.randomCbNonce g
@@ -1182,6 +1183,11 @@ getConnShortLink' c nm userId = \case
deleteLocalInvShortLink' :: AgentClient -> ConnShortLink 'CMInvitation -> AM ()
deleteLocalInvShortLink' c (CSLInvitation _ srv linkId _) = withStore' c $ \db -> deleteInvShortLink db srv linkId
resolveSimplexName' :: AgentClient -> NetworkRequestMode -> UserId -> SimplexDomain -> AM NameRecord
resolveSimplexName' c nm userId domain = do
resolverSrv <- getNextNameServer c userId
resolveName c nm userId resolverSrv domain
changeConnectionUser' :: AgentClient -> UserId -> ConnId -> UserId -> AM ()
changeConnectionUser' c oldUserId connId newUserId = do
SomeConn _ conn <- withStore c (`getConn` connId)
@@ -1219,8 +1225,8 @@ newRcvConnSrv c nm userId connId enableNtfs cMode userLinkData_ clientData pqIni
SCMInvitation -> do
g <- asks random
let pqEnc = CR.initialPQEncryption (isJust userLinkData_) pqInitKeys
(pk1, pk2, pKem, e2eRcvParams) <- liftIO $ CR.generateRcvE2EParams g (maxVersion e2eEncryptVRange) pqEnc
withStore' c $ \db -> createRatchetX3dhKeys db connId pk1 pk2 pKem
(pks, e2eRcvParams) <- liftIO $ CR.generateRcvE2EParams g (maxVersion e2eEncryptVRange) pqEnc
withStore' c $ \db -> createRatchetX3dhKeys db connId pks
pure $ CRInvitationUri crData $ toVersionRangeT e2eRcvParams e2eEncryptVRange
prepareLinkData :: UserConnLinkData c -> C.PublicKeyX25519 -> AM (C.CbNonce, SMPQueueUri, ConnectionRequestUri c, ClntQueueReqData)
prepareLinkData userLinkData e2eDhKey = do
@@ -1341,9 +1347,9 @@ startJoinInvitation c userId connId sq_ enableNtfs cReqUri pqSup =
Nothing -> throwE $ AGENT A_VERSION
where
createRatchet_ db g maxSupported pqSupport e2eRcvParams@(CR.E2ERatchetParams v _ rcDHRr kem_) = do
(pk1, pk2, pKem, e2eSndParams) <- liftIO $ CR.generateSndE2EParams g v (CR.replyKEM_ v kem_ pqSupport)
(pks, e2eSndParams) <- liftIO $ CR.generateSndE2EParams g v (CR.replyKEM_ v kem_ pqSupport)
(_, rcDHRs) <- atomically $ C.generateKeyPair g
rcParams <- liftEitherWith (SEAgentError . cryptoError) $ CR.pqX3dhSnd pk1 pk2 pKem e2eRcvParams
rcParams <- liftEitherWith (SEAgentError . cryptoError) $ CR.pqX3dhSnd pks e2eRcvParams
let rcVs = CR.RatchetVersions {current = v, maxSupported}
rc = CR.initSndRatchet rcVs rcDHRr rcDHRs rcParams
liftIO $ createSndRatchet db connId rc e2eSndParams
@@ -1420,8 +1426,8 @@ joinConnSrv c nm userId connId enableNtfs cReqUri@CRContactUri {} cInfo pqSup su
Left e -> do
nonBlockingWriteTBQueue (subQ c) ("", connId, AEvt SAEConn (ERR $ INTERNAL $ "no rcv ratchet " <> show e))
let pqEnc = CR.initialPQEncryption False pqInitKeys
(pk1, pk2, pKem, e2eRcvParams) <- liftIO $ CR.generateRcvE2EParams g (maxVersion e2eVR) pqEnc
createRatchetX3dhKeys db connId pk1 pk2 pKem
(pks, e2eRcvParams) <- liftIO $ CR.generateRcvE2EParams g (maxVersion e2eVR) pqEnc
createRatchetX3dhKeys db connId pks
pure e2eRcvParams
let cReq = CRInvitationUri crData $ toVersionRangeT e2eRcvParams e2eVR
pure $ CCLink cReq Nothing
@@ -2469,11 +2475,11 @@ synchronizeRatchet' c connId pqSupport' force = withConnLock c connId "synchroni
let cData' = cData {pqSupport = pqSupport'} :: ConnData
AgentConfig {e2eEncryptVRange} <- asks config
g <- asks random
(pk1, pk2, pKem, e2eParams) <- liftIO $ CR.generateRcvE2EParams g (maxVersion e2eEncryptVRange) pqSupport'
(pks, e2eParams) <- liftIO $ CR.generateRcvE2EParams g (maxVersion e2eEncryptVRange) pqSupport'
enqueueRatchetKeyMsgs c sqs e2eParams
withStore' c $ \db -> do
setConnRatchetSync db connId RSStarted
setRatchetX3dhKeys db connId pk1 pk2 pKem
setRatchetX3dhKeys db connId pks
let cData'' = cData' {ratchetSyncState = RSStarted} :: ConnData
conn' = DuplexConnection cData'' rqs sqs
connectionStats c conn'
@@ -3404,8 +3410,8 @@ processSMPTransmissions c@AgentClient {subQ} (tSess@(userId, srv, _), THandlePar
-- party initiating connection
(RcvConnection _ _, Just (CR.AE2ERatchetParams _ e2eSndParams@(CR.E2ERatchetParams e2eVersion _ _ _))) -> do
unless (e2eVersion `isCompatible` e2eEncryptVRange) (throwE $ AGENT A_VERSION)
(pk1, rcDHRs, pKem) <- withStore c (`getRatchetX3dhKeys` connId)
rcParams <- liftError cryptoError $ CR.pqX3dhRcv pk1 rcDHRs pKem e2eSndParams
pks@(_, rcDHRs, _) <- withStore c (`getRatchetX3dhKeys` connId)
rcParams <- liftError cryptoError $ CR.pqX3dhRcv pks e2eSndParams
let rcVs = CR.RatchetVersions {current = e2eVersion, maxSupported = maxVersion e2eEncryptVRange}
pqSupport' = pqSupport `CR.pqSupportAnd` versionPQSupport_ agentVersion (Just e2eVersion)
rc = CR.initRcvRatchet rcVs rcDHRs rcParams pqSupport'
@@ -3646,7 +3652,7 @@ processSMPTransmissions c@AgentClient {subQ} (tSess@(userId, srv, _), THandlePar
exists <- checkRatchetKeyHashExists db connId rkHashRcv
unless exists $ addProcessedRatchetKeyHash db connId rkHashRcv
pure exists
getSendRatchetKeys :: AM (C.PrivateKeyX448, C.PrivateKeyX448, Maybe CR.RcvPrivRKEMParams)
getSendRatchetKeys :: AM (CR.RcvE2EPrivRatchetParams 'C.X448)
getSendRatchetKeys = case rss of
RSOk -> sendReplyKey -- receiving client
RSAllowed -> sendReplyKey
@@ -3662,9 +3668,9 @@ processSMPTransmissions c@AgentClient {subQ} (tSess@(userId, srv, _), THandlePar
where
sendReplyKey = do
g <- asks random
(pk1, pk2, pKem, e2eParams) <- liftIO $ CR.generateRcvE2EParams g e2eVersion pqSupport
(pks, e2eParams) <- liftIO $ CR.generateRcvE2EParams g e2eVersion pqSupport
enqueueRatchetKeyMsgs c sqs e2eParams
pure (pk1, pk2, pKem)
pure pks
notifyRatchetSyncError = do
let cData'' = cData' {ratchetSyncState = RSRequired} :: ConnData
conn'' = updateConnection cData'' conn'
@@ -3683,14 +3689,14 @@ processSMPTransmissions c@AgentClient {subQ} (tSess@(userId, srv, _), THandlePar
createRatchet db connId rc
-- compare public keys `k1` in AgentRatchetKey messages sent by self and other party
-- to determine ratchet initilization ordering
initRatchet :: CR.RatchetVersions -> (C.PrivateKeyX448, C.PrivateKeyX448, Maybe CR.RcvPrivRKEMParams) -> AM ()
initRatchet :: CR.RatchetVersions -> CR.RcvE2EPrivRatchetParams 'C.X448 -> AM ()
initRatchet rcVs (pk1, pk2, pKem)
| rkHash (C.publicKey pk1) (C.publicKey pk2) <= rkHashRcv = do
rcParams <- liftError cryptoError $ CR.pqX3dhRcv pk1 pk2 pKem e2eOtherPartyParams
rcParams <- liftError cryptoError $ CR.pqX3dhRcv (pk1, pk2, pKem) e2eOtherPartyParams
recreateRatchet $ CR.initRcvRatchet rcVs pk2 rcParams pqSupport
| otherwise = do
(_, rcDHRs) <- atomically . C.generateKeyPair =<< asks random
rcParams <- liftEitherWith cryptoError $ CR.pqX3dhSnd pk1 pk2 (CR.APRKP CR.SRKSProposed <$> pKem) e2eOtherPartyParams
rcParams <- liftEitherWith cryptoError $ CR.pqX3dhSnd (pk1, pk2, CR.APRKP CR.SRKSProposed <$> pKem) e2eOtherPartyParams
recreateRatchet $ CR.initSndRatchet rcVs k2Rcv rcDHRs rcParams
void . enqueueMessages' c cData' sqs SMP.MsgFlags {notification = True} $ EREADY lastExternalSndId
+37 -23
View File
@@ -68,6 +68,8 @@ module Simplex.Messaging.Agent.Client
deleteQueueLink,
secureGetQueueLink,
getQueueLink,
resolveName,
getNextNameServer,
enableQueueNotifications,
EnableQueueNtfReq (..),
enableQueuesNtfs,
@@ -267,6 +269,7 @@ import Simplex.Messaging.Protocol
NetworkError (..),
MsgFlags (..),
MsgId,
NameRecord,
NtfServer,
NtfServerWithAuth,
ProtoServer,
@@ -674,8 +677,7 @@ getSMPServerClient :: AgentClient -> NetworkRequestMode -> SMPTransportSession -
getSMPServerClient c@AgentClient {active, smpClients, workerSeq} nm tSess = do
unlessM (readTVarIO active) $ throwE INACTIVE
ts <- liftIO getCurrentTime
atomically (getSessVar workerSeq tSess smpClients ts)
>>= either newClient (waitForProtocolClient c nm tSess smpClients)
withGetSessVar workerSeq tSess smpClients ts newClient (waitForProtocolClient c nm tSess smpClients)
where
newClient v = do
prs <- liftIO TM.emptyIO
@@ -686,29 +688,25 @@ getSMPProxyClient c@AgentClient {active, smpClients, smpProxiedRelays, workerSeq
unlessM (readTVarIO active) $ throwE INACTIVE
proxySrv <- maybe (getNextServer c userId proxySrvs [destSrv]) pure proxySrv_
ts <- liftIO getCurrentTime
atomically (getClientVar proxySrv ts) >>= \(tSess, auth, v) ->
either (newProxyClient tSess auth ts) (waitForProxyClient tSess auth) v
(tSess, auth) <- atomically $ proxiedTransportSession proxySrv
withGetSessVar workerSeq tSess smpClients ts (newProxyClient tSess auth ts) (waitForProxyClient tSess auth)
where
getClientVar :: SMPServerWithAuth -> UTCTime -> STM (SMPTransportSession, Maybe SMP.BasicAuth, Either SMPClientVar SMPClientVar)
getClientVar proxySrv ts = do
proxiedTransportSession :: SMPServerWithAuth -> STM (SMPTransportSession, Maybe SMP.BasicAuth)
proxiedTransportSession proxySrv = do
ProtoServerWithAuth srv auth <- TM.lookup destSess smpProxiedRelays >>= maybe (TM.insert destSess proxySrv smpProxiedRelays $> proxySrv) pure
let tSess = (userId, srv, qId)
(tSess,auth,) <$> getSessVar workerSeq tSess smpClients ts
pure ((userId, srv, qId), auth)
newProxyClient :: SMPTransportSession -> Maybe SMP.BasicAuth -> UTCTime -> SMPClientVar -> AM (SMPConnectedClient, Either AgentErrorType ProxiedRelay)
newProxyClient tSess auth ts v = do
prs <- liftIO TM.emptyIO
-- we do not need to check if it is a new proxied relay session,
-- as the client is just created and there are no sessions yet
rv <- atomically $ either id id <$> getSessVar workerSeq destSrv prs ts
clnt <- smpConnectClient c nm tSess prs v
(clnt,) <$> newProxiedRelay clnt auth rv
-- the relay var is always new (the client is just created and has no sessions yet)
sess <- withGetSessVar workerSeq destSrv prs ts (newProxiedRelay clnt auth) (waitForProxiedRelay tSess)
pure (clnt, sess)
waitForProxyClient :: SMPTransportSession -> Maybe SMP.BasicAuth -> SMPClientVar -> AM (SMPConnectedClient, Either AgentErrorType ProxiedRelay)
waitForProxyClient tSess auth v = do
clnt@(SMPConnectedClient _ prs) <- waitForProtocolClient c nm tSess smpClients v
ts <- liftIO getCurrentTime
sess <-
atomically (getSessVar workerSeq destSrv prs ts)
>>= either (newProxiedRelay clnt auth) (waitForProxiedRelay tSess)
sess <- withGetSessVar workerSeq destSrv prs ts (newProxiedRelay clnt auth) (waitForProxiedRelay tSess)
pure (clnt, sess)
newProxiedRelay :: SMPConnectedClient -> Maybe SMP.BasicAuth -> ProxiedRelayVar -> AM (Either AgentErrorType ProxiedRelay)
newProxiedRelay (SMPConnectedClient smp prs) proxyAuth rv =
@@ -846,10 +844,7 @@ getNtfServerClient :: AgentClient -> NetworkRequestMode -> NtfTransportSession -
getNtfServerClient c@AgentClient {active, ntfClients, workerSeq, proxySessTs, presetDomains} nm tSess@(_, srv, _) = do
unlessM (readTVarIO active) $ throwE INACTIVE
ts <- liftIO getCurrentTime
atomically (getSessVar workerSeq tSess ntfClients ts)
>>= either
(newProtocolClient c tSess ntfClients connectClient)
(waitForProtocolClient c nm tSess ntfClients)
withGetSessVar workerSeq tSess ntfClients ts (newProtocolClient c tSess ntfClients connectClient) (waitForProtocolClient c nm tSess ntfClients)
where
connectClient :: NtfClientVar -> AM NtfClient
connectClient v = do
@@ -870,10 +865,7 @@ getXFTPServerClient :: AgentClient -> XFTPTransportSession -> AM XFTPClient
getXFTPServerClient c@AgentClient {active, xftpClients, workerSeq, proxySessTs, presetDomains} tSess@(_, srv, _) = do
unlessM (readTVarIO active) $ throwE INACTIVE
ts <- liftIO getCurrentTime
atomically (getSessVar workerSeq tSess xftpClients ts)
>>= either
(newProtocolClient c tSess xftpClients connectClient)
(waitForProtocolClient c NRMBackground tSess xftpClients)
withGetSessVar workerSeq tSess xftpClients ts (newProtocolClient c tSess xftpClients connectClient) (waitForProtocolClient c NRMBackground tSess xftpClients)
where
connectClient :: XFTPClientVar -> AM XFTPClient
connectClient v = do
@@ -1990,6 +1982,28 @@ getQueueLink c nm userId server lnkId =
getViaProxy smp proxySess = proxyGetSMPQueueLink smp nm proxySess lnkId
getDirectly smp = getSMPQueueLink smp nm lnkId
-- | Resolve a public-namespace name. Prefers PFWD (hides client IP from the
-- resolver) and falls back to a direct send when the proxy is unavailable
-- (faster but exposes the client IP). Mode selection is delegated to
-- `sendOrProxySMPCommand`, which honours the network config (SPMNever etc.).
resolveName :: AgentClient -> NetworkRequestMode -> UserId -> SMPServer -> SimplexDomain -> AM NameRecord
resolveName c nm userId server domain =
snd <$> sendOrProxySMPCommand c nm userId server "" "RSLV" NoEntity resolveViaProxy resolveDirectly
where
resolveViaProxy smp proxySess = proxyResolveName smp nm proxySess domain
resolveDirectly smp = directResolveName smp nm domain
-- | Pick a names-capable server for the user (the agent owns server selection,
-- accounting for the names role). nameSrvs is opt-in (a plain list); empty means
-- no server resolves names - a declared agent error, never a fallback.
getNextNameServer :: AgentClient -> UserId -> AM SMPServer
getNextNameServer c userId =
liftIO (TM.lookupIO userId (userServers c :: TMap UserId (UserServers 'PSMP))) >>= \case
Just UserServers {nameSrvs} -> case L.nonEmpty nameSrvs of
Just srvs -> protoServer <$> pickServer srvs
Nothing -> throwE NO_NAME_SERVERS
Nothing -> throwE $ INTERNAL "unknown userId - no user servers"
enableQueueNotifications :: AgentClient -> RcvQueue -> SMP.NtfPublicAuthKey -> SMP.RcvNtfPublicDhKey -> AM (SMP.NotifierId, SMP.RcvNtfPublicDhKey)
enableQueueNotifications c rq@RcvQueue {rcvId, rcvPrivateKey} notifierKey rcvNtfPublicDhKey =
withSMPClient c NRMBackground rq "NKEY <nkey>" $ \smp ->
+8 -3
View File
@@ -105,12 +105,13 @@ data ServerCfg p = ServerCfg
data ServerRoles = ServerRoles
{ storage :: Bool,
proxy :: Bool
proxy :: Bool,
names :: Bool
}
deriving (Show)
allRoles :: ServerRoles
allRoles = ServerRoles True True
allRoles = ServerRoles True True True
presetServerCfg :: Bool -> ServerRoles -> Maybe OperatorId -> ProtoServerWithAuth p -> ServerCfg p
presetServerCfg enabled roles operator server =
@@ -119,6 +120,9 @@ presetServerCfg enabled roles operator server =
data UserServers p = UserServers
{ storageSrvs :: NonEmpty (Maybe OperatorId, ProtoServerWithAuth p),
proxySrvs :: NonEmpty (Maybe OperatorId, ProtoServerWithAuth p),
-- name resolution is opt-in: a plain list (NOT NonEmpty, no fallback-to-all).
-- Empty = no servers resolve names = a clean agent error, never falls back.
nameSrvs :: [(Maybe OperatorId, ProtoServerWithAuth p)],
knownHosts :: Set TransportHost
}
@@ -126,9 +130,10 @@ type OperatorId = Int64
-- This function sets all servers as enabled in case all passed servers are disabled.
mkUserServers :: NonEmpty (ServerCfg p) -> UserServers p
mkUserServers srvs = UserServers {storageSrvs = filterSrvs storage, proxySrvs = filterSrvs proxy, knownHosts}
mkUserServers srvs = UserServers {storageSrvs = filterSrvs storage, proxySrvs = filterSrvs proxy, nameSrvs, knownHosts}
where
filterSrvs role = L.map (\ServerCfg {operator, server} -> (operator, server)) $ fromMaybe srvs $ L.nonEmpty $ L.filter (\ServerCfg {enabled, roles} -> enabled && role roles) srvs
nameSrvs = map (\ServerCfg {operator, server} -> (operator, server)) $ L.filter (\ServerCfg {enabled, roles} -> enabled && names roles) srvs
knownHosts = S.unions $ L.map (\ServerCfg {server = ProtoServerWithAuth srv _} -> serverHosts srv) srvs
serverHosts :: ProtocolServer p -> Set TransportHost
+23 -79
View File
@@ -123,7 +123,7 @@ module Simplex.Messaging.Agent.Protocol
ConnectionLink (..),
AConnectionLink (..),
SimplexNameInfo (..),
SimplexNameDomain (..),
SimplexDomain (..),
SimplexTLD (..),
SimplexNameType (..),
ConnShortLink (..),
@@ -137,7 +137,9 @@ module Simplex.Messaging.Agent.Protocol
validateOwners,
validateLinkOwners,
sameConnReqContact,
sameConnShortLink,
sameShortLinkContact,
sameShortLinkInv,
simplexChat,
connReqUriP',
simplexConnReqUri,
@@ -195,11 +197,10 @@ import qualified Data.Aeson.TH as J
import qualified Data.Aeson.Types as JT
import Data.Attoparsec.ByteString.Char8 (Parser)
import qualified Data.Attoparsec.ByteString.Char8 as A
import qualified Data.Attoparsec.Text as AT
import qualified Data.ByteString.Base64.URL as B64
import Data.ByteString.Char8 (ByteString)
import qualified Data.ByteString.Char8 as B
import Data.Char (isAlpha, isDigit, toLower, toUpper)
import Data.Char (toLower, toUpper)
import Data.Foldable (find)
import Data.Functor (($>))
import Data.Int (Int64)
@@ -237,6 +238,7 @@ import Simplex.Messaging.Crypto.Ratchet
)
import Simplex.Messaging.Encoding
import Simplex.Messaging.Encoding.String
import Simplex.Messaging.SimplexName (SimplexDomain (..), SimplexNameInfo (..), SimplexNameType (..), SimplexTLD (..), fullDomainName, shortNameInfoStr)
import Simplex.Messaging.Parsers
import Simplex.Messaging.Protocol
( AProtocolType,
@@ -1531,75 +1533,6 @@ instance (Typeable c, ConnectionModeI c) => FromField (ConnShortLink c) where fr
data ContactConnType = CCTContact | CCTChannel | CCTGroup | CCTRelay deriving (Eq, Show)
data SimplexNameInfo = SimplexNameInfo
{ nameType :: SimplexNameType,
nameDomain :: SimplexNameDomain
}
deriving (Eq, Show)
data SimplexNameDomain = SimplexNameDomain
{ nameTLD :: SimplexTLD,
domain :: Text,
subDomain :: [Text] -- parent to child: ["b", "a"] for a.b.domain.simplex
}
deriving (Eq, Show)
data SimplexTLD = TLDSimplex | TLDTesting | TLDWeb
deriving (Eq, Show)
data SimplexNameType = NTPublicGroup | NTContact
deriving (Eq, Show)
instance StrEncoding SimplexNameType where
strEncode = \case
NTPublicGroup -> "#"
NTContact -> "@"
strP = A.char '#' $> NTPublicGroup <|> A.char '@' $> NTContact
nameLabelP :: AT.Parser Text
nameLabelP = T.intercalate "-" <$> AT.takeWhile1 (\c -> isNameLetter c || isDigit c) `AT.sepBy1` AT.char '-'
where
isNameLetter c = isAlpha c && not (c >= '\x00c0' && c <= '\x024f')
instance StrEncoding SimplexNameInfo where
strEncode SimplexNameInfo {nameType, nameDomain} =
"simplex:/name" <> strEncode nameType <> strEncode nameDomain
strP = optional "simplex:/name" *> ((strP >>= infoP) <|> infoP NTPublicGroup)
where
infoP NTPublicGroup = SimplexNameInfo NTPublicGroup <$> (strP <|> bareName)
infoP NTContact = SimplexNameInfo NTContact <$> strP
bareName = parseBare . safeDecodeUtf8 <$?> A.takeWhile1 (not . A.isSpace)
parseBare s = (\name -> SimplexNameDomain TLDSimplex name []) <$> AT.parseOnly (nameLabelP <* AT.endOfInput) s
instance StrEncoding SimplexNameDomain where
strEncode = encodeUtf8 . fullDomainName
strP = parseDomain . safeDecodeUtf8 <$?> A.takeWhile1 (not . A.isSpace)
where
parseDomain s = AT.parseOnly (nameLabelP `AT.sepBy1` AT.char '.' <* AT.endOfInput) s >>= mkDomain
mkDomain labels = case reverse labels of
[] -> Left "empty name"
[_] -> Left "domain requires TLD"
"simplex" : name : sub -> Right $ SimplexNameDomain TLDSimplex name sub
"testing" : name : sub -> Right $ SimplexNameDomain TLDTesting name sub
_ -> Right $ SimplexNameDomain TLDWeb (T.intercalate "." labels) []
fullDomainName :: SimplexNameDomain -> Text
fullDomainName SimplexNameDomain {nameTLD, domain, subDomain} = T.intercalate "." (reverse subDomain ++ [domain] ++ tld')
where
tld' = case nameTLD of
TLDSimplex -> ["simplex"]
TLDTesting -> ["testing"]
TLDWeb -> []
shortNameInfoStr :: SimplexNameInfo -> Text
shortNameInfoStr = \case
SimplexNameInfo {nameType = NTPublicGroup, nameDomain = SimplexNameDomain {nameTLD = TLDSimplex, domain, subDomain = []}} -> "#" <> domain
info -> pfx <> fullDomainName (nameDomain info)
where
pfx = case nameType info of
NTPublicGroup -> "#"
NTContact -> "@"
data AConnShortLink = forall m. ConnectionModeI m => ACSL (SConnectionMode m) (ConnShortLink m)
instance Eq AConnShortLink where
@@ -1758,6 +1691,10 @@ ctTypeChar = \case
CCTRelay -> 'R'
{-# INLINE ctTypeChar #-}
instance StrEncoding ContactConnType where
strEncode = B.singleton . ctTypeChar
strP = A.anyChar >>= ctTypeP
-- the servers passed to this function should be all preset servers, not servers configured by the user.
shortenShortLink :: NonEmpty SMPServer -> ConnShortLink m -> ConnShortLink m
shortenShortLink presetSrvs = \case
@@ -1801,10 +1738,21 @@ sameConnReqContact (CRContactUri ConnReqUriData {crSmpQueues = qs}) (CRContactUr
where
same (q, q') = sameQAddress (qAddress q) (qAddress q')
sameConnShortLink :: AConnShortLink -> AConnShortLink -> Bool
sameConnShortLink (ACSL m sl) (ACSL m' sl') = case testEquality m m' of
Just Refl -> case sl of
CSLContact {} -> sameShortLinkContact sl sl'
CSLInvitation {} -> sameShortLinkInv sl sl'
Nothing -> False
sameShortLinkContact :: ConnShortLink 'CMContact -> ConnShortLink 'CMContact -> Bool
sameShortLinkContact (CSLContact _ ct srv k) (CSLContact _ ct' srv' k') =
ct == ct' && sameSrvAddr srv srv' && k == k'
sameShortLinkInv :: ConnShortLink 'CMInvitation -> ConnShortLink 'CMInvitation -> Bool
sameShortLinkInv (CSLInvitation _ srv lnkId k) (CSLInvitation _ srv' lnkId' k') =
sameSrvAddr srv srv' && lnkId == lnkId' && k == k'
checkConnMode :: forall t m m'. (ConnectionModeI m, ConnectionModeI m') => t m' -> Either String (t m)
checkConnMode c = case testEquality (sConnectionMode @m) (sConnectionMode @m') of
Just Refl -> Right c
@@ -2067,6 +2015,9 @@ data AgentErrorType
XFTP {serverAddress :: String, xftpErr :: XFTPErrorType}
| -- | XFTP agent errors
FILE {fileErr :: FileErrorType}
| -- | no name-resolving servers configured for the user (agent-origin).
-- Server-origin name errors arrive forwarded as SMP _ (NAME ...) instead.
NO_NAME_SERVERS
| -- | SMP proxy errors
PROXY {proxyServer :: String, relayServer :: String, proxyErr :: ProxyClientError}
| -- | XRCP protocol errors forwarded to agent clients
@@ -2278,10 +2229,3 @@ instance ToJSON ACreatedConnLink where
toEncoding (ACCL _ ccLink) = toEncoding ccLink
toJSON (ACCL _ ccLink) = toJSON ccLink
$(J.deriveJSON (enumJSON $ dropPrefix "TLD") ''SimplexTLD)
$(J.deriveJSON (enumJSON $ dropPrefix "NT") ''SimplexNameType)
$(J.deriveJSON defaultJSON ''SimplexNameDomain)
$(J.deriveJSON defaultJSON ''SimplexNameInfo)
@@ -1359,11 +1359,11 @@ deleteSndMsgsExpired db ttl limit = do
|]
(cutoffTs, limit)
createRatchetX3dhKeys :: DB.Connection -> ConnId -> C.PrivateKeyX448 -> C.PrivateKeyX448 -> Maybe CR.RcvPrivRKEMParams -> IO ()
createRatchetX3dhKeys db connId x3dhPrivKey1 x3dhPrivKey2 pqPrivKem =
createRatchetX3dhKeys :: DB.Connection -> ConnId -> CR.RcvE2EPrivRatchetParams 'C.X448 -> IO ()
createRatchetX3dhKeys db connId (x3dhPrivKey1, x3dhPrivKey2, pqPrivKem) =
DB.execute db "INSERT INTO ratchets (conn_id, x3dh_priv_key_1, x3dh_priv_key_2, pq_priv_kem) VALUES (?, ?, ?, ?)" (connId, x3dhPrivKey1, x3dhPrivKey2, pqPrivKem)
getRatchetX3dhKeys :: DB.Connection -> ConnId -> IO (Either StoreError (C.PrivateKeyX448, C.PrivateKeyX448, Maybe CR.RcvPrivRKEMParams))
getRatchetX3dhKeys :: DB.Connection -> ConnId -> IO (Either StoreError (CR.RcvE2EPrivRatchetParams 'C.X448))
getRatchetX3dhKeys db connId =
firstRow' keys SEX3dhKeysNotFound $
DB.query db "SELECT x3dh_priv_key_1, x3dh_priv_key_2, pq_priv_kem FROM ratchets WHERE conn_id = ?" (Only connId)
@@ -1373,8 +1373,8 @@ getRatchetX3dhKeys db connId =
_ -> Left SEX3dhKeysNotFound
-- used to remember new keys when starting ratchet re-synchronization
setRatchetX3dhKeys :: DB.Connection -> ConnId -> C.PrivateKeyX448 -> C.PrivateKeyX448 -> Maybe CR.RcvPrivRKEMParams -> IO ()
setRatchetX3dhKeys db connId x3dhPrivKey1 x3dhPrivKey2 pqPrivKem =
setRatchetX3dhKeys :: DB.Connection -> ConnId -> CR.RcvE2EPrivRatchetParams 'C.X448 -> IO ()
setRatchetX3dhKeys db connId (x3dhPrivKey1, x3dhPrivKey2, pqPrivKem) =
DB.execute
db
[sql|
@@ -50,7 +50,6 @@ import Data.Bits (xor)
import Data.ByteArray (ScrubbedBytes)
import qualified Data.ByteArray as BA
import Data.ByteString (ByteString)
import qualified Data.ByteString as B
import Data.Functor (($>))
import Data.IORef
import Data.Maybe (fromMaybe)
+30
View File
@@ -73,6 +73,8 @@ module Simplex.Messaging.Client
deleteSMPQueues,
connectSMPProxiedRelay,
proxySMPMessage,
proxyResolveName,
directResolveName,
forwardSMPTransmission,
getSMPQueueInfo,
sendProtocolCommand,
@@ -164,6 +166,7 @@ import Simplex.Messaging.Parsers (defaultJSON, dropPrefix, enumJSON, sumTypeJSON
import Simplex.Messaging.Protocol
import Simplex.Messaging.Protocol.Types
import Simplex.Messaging.Server.QueueStore.QueueInfo
import Simplex.Messaging.SimplexName (SimplexDomain)
import Simplex.Messaging.TMap (TMap)
import qualified Simplex.Messaging.TMap as TM
import Simplex.Messaging.Transport
@@ -1046,6 +1049,33 @@ sendSMPMessage c nm spKey sId flags msg =
proxySMPMessage :: SMPClient -> NetworkRequestMode -> ProxiedRelay -> Maybe SndPrivateAuthKey -> SenderId -> MsgFlags -> MsgBody -> ExceptT SMPClientError IO (Either ProxyClientError ())
proxySMPMessage c nm proxiedRelay spKey sId flags msg = proxyOKSMPCommand c nm proxiedRelay spKey sId (SEND flags msg)
-- | Resolve a public-namespace name via PFWD. Preferred path - hides the
-- client IP from the resolver. Mirrors `proxySMPMessage`'s shape; routes
-- through `proxySMPCommand` and pattern-matches the expected RNAME response.
-- Version-gated on the destination relay (mirrors `connectSMPProxiedRelay`):
-- the client never sends RSLV to a relay that predates names support.
proxyResolveName :: SMPClient -> NetworkRequestMode -> ProxiedRelay -> SimplexDomain -> ExceptT SMPClientError IO (Either ProxyClientError NameRecord)
proxyResolveName c nm proxiedRelay name
| prVersion proxiedRelay >= namesSMPVersion =
proxySMPCommand c nm proxiedRelay Nothing NoEntity (RSLV name) >>= \case
Right (RNAME nr) -> pure $ Right nr
Right r -> throwE $ unexpectedResponse r
Left e -> pure $ Left e
| otherwise = throwE $ PCETransportError TEVersion
-- | Direct (non-PFWD) name resolution. Exposes the client IP to the resolver;
-- callers that want anonymity should use `proxyResolveName` via the standard
-- proxy fallback in the agent. RSLV requires no entity ID or authorization
-- (see `noAuthCmd` in Protocol.hs). Version-gated on the session here, not the
-- encoder, so an old server never receives RSLV.
directResolveName :: SMPClient -> NetworkRequestMode -> SimplexDomain -> ExceptT SMPClientError IO NameRecord
directResolveName c nm name
| thVersion (thParams c) >= namesSMPVersion =
sendProtocolCommand c nm Nothing NoEntity (Cmd SResolver (RSLV name)) >>= \case
RNAME nr -> pure nr
r -> throwE $ unexpectedResponse r
| otherwise = throwE $ PCETransportError TEVersion
-- | Acknowledge message delivery (server deletes the message).
--
-- https://github.com/simplex-chat/simplexmq/blob/master/protocol/simplex-messaging.md#acknowledge-message-delivery
+7 -9
View File
@@ -107,7 +107,7 @@ data SMPClientAgentConfig = SMPClientAgentConfig
{ smpCfg :: ProtocolClientConfig SMPVersion,
reconnectInterval :: RetryInterval,
persistErrorInterval :: NominalDiffTime,
msgQSize :: Natural,
msgQSize :: Maybe Natural,
agentQSize :: Natural,
agentSubsBatchSize :: Int,
ownServerDomains :: [ByteString]
@@ -124,7 +124,7 @@ defaultSMPClientAgentConfig =
maxInterval = 10 * second
},
persistErrorInterval = 30, -- seconds
msgQSize = 2048,
msgQSize = Just 2048,
agentQSize = 2048,
agentSubsBatchSize = 1360,
ownServerDomains = []
@@ -138,7 +138,7 @@ data SMPClientAgent p = SMPClientAgent
dbService :: Maybe DBService,
active :: TVar Bool,
startedAt :: UTCTime,
msgQ :: TBQueue (ServerTransmissionBatch SMPVersion ErrorType BrokerMsg),
msgQ :: Maybe (TBQueue (ServerTransmissionBatch SMPVersion ErrorType BrokerMsg)),
agentQ :: TBQueue SMPClientAgentEvent,
randomDrg :: TVar ChaChaDRG,
smpClients :: TMap SMPServer SMPClientVar,
@@ -162,7 +162,8 @@ newSMPClientAgent :: SParty p -> SMPClientAgentConfig -> Maybe DBService -> TVar
newSMPClientAgent agentParty agentCfg@SMPClientAgentConfig {msgQSize, agentQSize} dbService randomDrg = do
active <- newTVarIO True
startedAt <- getCurrentTime
msgQ <- newTBQueueIO msgQSize
-- Only subscribing agents receive server transmissions, should not be created until processed to prevent deadlock.
msgQ <- mapM newTBQueueIO msgQSize
agentQ <- newTBQueueIO agentQSize
smpClients <- TM.emptyIO
smpSessions <- TM.emptyIO
@@ -205,11 +206,8 @@ getSMPServerClient' ca srv = snd <$> getSMPServerClient'' ca srv
getSMPServerClient'' :: SMPClientAgent p -> SMPServer -> ExceptT SMPClientError IO (OwnServer, SMPClient)
getSMPServerClient'' ca@SMPClientAgent {agentCfg, smpClients, smpSessions, workerSeq} srv = do
ts <- liftIO getCurrentTime
atomically (getClientVar ts) >>= either (ExceptT . newSMPClient) waitForSMPClient
withGetSessVar workerSeq srv smpClients ts (ExceptT . newSMPClient) waitForSMPClient
where
getClientVar :: UTCTime -> STM (Either SMPClientVar SMPClientVar)
getClientVar = getSessVar workerSeq srv smpClients
waitForSMPClient :: SMPClientVar -> ExceptT SMPClientError IO (OwnServer, SMPClient)
waitForSMPClient v = do
let ProtocolClientConfig {networkConfig = NetworkConfig {tcpConnectTimeout}} = smpCfg agentCfg
@@ -267,7 +265,7 @@ connectClient ca@SMPClientAgent {agentCfg, dbService, smpClients, smpSessions, m
Nothing -> getClient cfg
where
cfg = smpCfg agentCfg
getClient cfg' = getProtocolClient randomDrg NRMBackground (1, srv, Nothing) cfg' [] (Just msgQ) startedAt clientDisconnected
getClient cfg' = getProtocolClient randomDrg NRMBackground (1, srv, Nothing) cfg' [] msgQ startedAt clientDisconnected
clientDisconnected :: SMPClient -> IO ()
clientDisconnected smp = do
+7 -3
View File
@@ -8,6 +8,7 @@ module Simplex.Messaging.Compression
compress1,
decompress1,
limitDecompress1,
limitDecompress',
decompressedSize,
) where
@@ -57,9 +58,12 @@ decompress1 = \case
limitDecompress1 :: Int -> Compressed -> Either String ByteString
limitDecompress1 limit = \case
Passthrough bs -> Right bs
Compressed (Large bs) -> case Z1.decompressedSize bs of
Just sz | sz <= limit -> decompress_ bs
_ -> Left $ "compressed size not specified or exceeds " <> show limit
Compressed (Large bs) -> limitDecompress' limit bs
limitDecompress' :: Int -> ByteString -> Either String ByteString
limitDecompress' limit bs = case Z1.decompressedSize bs of
Just sz | sz <= limit -> decompress_ bs
_ -> Left $ "compressed size not specified or exceeds " <> show limit
decompress_ :: ByteString -> Either String ByteString
decompress_ bs = case Z1.decompress bs of
+302
View File
@@ -0,0 +1,302 @@
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE DerivingStrategies #-}
{-# LANGUAGE DerivingVia #-}
{-# LANGUAGE ForeignFunctionInterface #-}
{-# LANGUAGE GeneralizedNewtypeDeriving #-}
{-# LANGUAGE KindSignatures #-}
{-# LANGUAGE ScopedTypeVariables #-}
-- | FFI bindings to libbbs (BBS+ signatures over BLS12-381, SHA-256 suite).
-- Values parsed from untrusted input are length-validated; see FixedBS and the
-- BBSProof StrEncoding instance.
module Simplex.Messaging.Crypto.BBS
( BBSSecretKey (..),
BBSPublicKey (..),
BBSKeyPair,
BBSSignature (..),
BBSProof (..),
BBSHeader (..),
BBSPresHeader (..),
bbsKeyGen,
bbsPublicKey,
bbsSign,
bbsVerify,
bbsProofGen,
bbsProofVerify,
) where
import Data.Aeson (FromJSON, ToJSON)
import Data.ByteString (ByteString)
import qualified Data.ByteString as B
import qualified Data.ByteString.Unsafe as BU
import Data.Proxy (Proxy (..))
import Foreign
import Foreign.C
import GHC.TypeLits (KnownNat, KnownSymbol, Nat, Symbol, natVal, symbolVal)
import Simplex.Messaging.Encoding.String
import System.IO.Unsafe (unsafePerformIO)
-- Note: the data constructors below are unchecked escape hatches for trusted,
-- internally-produced values (e.g. keygen output). Any value parsed from
-- untrusted input (StrEncoding / FromJSON) is length-validated — see FixedBS
-- and the BBSProof StrEncoding instance.
newtype BBSSecretKey = BBSSecretKey ByteString
deriving newtype (Eq, Show)
deriving (StrEncoding) via (FixedBS "BBSSecretKey" 32)
deriving (ToJSON, FromJSON) via (StrJSON "BBSSecretKey" BBSSecretKey)
newtype BBSPublicKey = BBSPublicKey ByteString
deriving newtype (Eq, Show)
deriving (StrEncoding) via (FixedBS "BBSPublicKey" 96)
deriving (ToJSON, FromJSON) via (StrJSON "BBSPublicKey" BBSPublicKey)
type BBSKeyPair = (BBSPublicKey, BBSSecretKey)
newtype BBSSignature = BBSSignature ByteString
deriving newtype (Eq, Show)
deriving (StrEncoding) via (FixedBS "BBSSignature" 80)
deriving (ToJSON, FromJSON) via (StrJSON "BBSSignature" BBSSignature)
newtype BBSProof = BBSProof ByteString
deriving newtype (Eq, Show)
deriving (ToJSON, FromJSON) via (StrJSON "BBSProof" BBSProof)
newtype BBSHeader = BBSHeader ByteString
deriving newtype (Eq, Show, StrEncoding)
deriving (ToJSON, FromJSON) via (StrJSON "BBSHeader" BBSHeader)
newtype BBSPresHeader = BBSPresHeader ByteString
deriving newtype (Eq, Show, StrEncoding)
deriving (ToJSON, FromJSON) via (StrJSON "BBSPresHeader" BBSPresHeader)
-- | A ByteString validated to be exactly @n@ bytes when parsed via StrEncoding
-- (and the JSON derived from it). Local to BBS, where every key/signature is a
-- fixed size; @name@ appears in the decode error only.
newtype FixedBS (name :: Symbol) (n :: Nat) = FixedBS ByteString
instance forall name n. (KnownSymbol name, KnownNat n) => StrEncoding (FixedBS name n) where
strEncode (FixedBS bs) = strEncode bs
strP = do
bs <- base64urlP
let n = fromIntegral (natVal (Proxy :: Proxy n))
if B.length bs == n
then pure (FixedBS bs)
else fail $ symbolVal (Proxy :: Proxy name) <> ": expected " <> show n <> " bytes, got " <> show (B.length bs)
-- Constants
bbsSkLen, bbsPkLen, bbsSigLen, bbsProofBaseLen, bbsProofUdElemLen :: Int
bbsSkLen = 32
bbsPkLen = 96
bbsSigLen = 80
bbsProofBaseLen = 272
bbsProofUdElemLen = 32
bbsProofLen :: Int -> Int
bbsProofLen numUndisclosed = bbsProofBaseLen + numUndisclosed * bbsProofUdElemLen
-- | A proof is @bbsProofBaseLen + 32 * numUndisclosed@ bytes; reject anything else.
instance StrEncoding BBSProof where
strEncode (BBSProof bs) = strEncode bs
strP = do
bs <- base64urlP
let len = B.length bs
if len >= bbsProofBaseLen && (len - bbsProofBaseLen) `mod` bbsProofUdElemLen == 0
then pure (BBSProof bs)
else fail $ "BBS: invalid proof length " <> show len
-- FFI
data BBS_Ciphersuite
foreign import ccall "bbs_keygen_full"
c_bbs_keygen_full :: Ptr BBS_Ciphersuite -> Ptr Word8 -> Ptr Word8 -> IO CInt
foreign import ccall "bbs_sk_to_pk"
c_bbs_sk_to_pk :: Ptr BBS_Ciphersuite -> Ptr Word8 -> Ptr Word8 -> IO CInt
foreign import ccall "bbs_sign"
c_bbs_sign ::
Ptr BBS_Ciphersuite ->
Ptr Word8 -> Ptr Word8 -> Ptr Word8 ->
Ptr Word8 -> CSize ->
CSize -> Ptr (Ptr Word8) -> Ptr CSize ->
IO CInt
foreign import ccall "bbs_verify"
c_bbs_verify ::
Ptr BBS_Ciphersuite ->
Ptr Word8 -> Ptr Word8 ->
Ptr Word8 -> CSize ->
CSize -> Ptr (Ptr Word8) -> Ptr CSize ->
IO CInt
foreign import ccall "bbs_proof_gen"
c_bbs_proof_gen ::
Ptr BBS_Ciphersuite ->
Ptr Word8 -> Ptr Word8 -> Ptr Word8 ->
Ptr Word8 -> CSize ->
Ptr Word8 -> CSize ->
Ptr CSize -> CSize ->
CSize -> Ptr (Ptr Word8) -> Ptr CSize ->
IO CInt
foreign import ccall "bbs_proof_verify"
c_bbs_proof_verify ::
Ptr BBS_Ciphersuite ->
Ptr Word8 ->
Ptr Word8 -> CSize ->
Ptr Word8 -> CSize ->
Ptr Word8 -> CSize ->
Ptr CSize -> CSize ->
CSize -> Ptr (Ptr Word8) -> Ptr CSize ->
IO CInt
foreign import ccall "&bbs_sha256_ciphersuite"
c_bbs_sha256_ciphersuite :: Ptr (Ptr BBS_Ciphersuite)
-- The ciphersuite is a static const pointer in libbbs; read it once.
ciphersuite :: Ptr BBS_Ciphersuite
ciphersuite = unsafePerformIO $ peek c_bbs_sha256_ciphersuite
{-# NOINLINE ciphersuite #-}
-- Helpers
withBS :: ByteString -> (Ptr Word8 -> CSize -> IO a) -> IO a
withBS bs f = BU.unsafeUseAsCStringLen bs $ \(p, l) -> f (castPtr p) (fromIntegral l)
packPtr :: Ptr Word8 -> Int -> IO ByteString
packPtr ptr len = B.packCStringLen (castPtr ptr, len)
-- Marshals a list of messages into parallel pointer/length arrays. Each
-- ByteString is held alive (via nested unsafeUseAsCStringLen) until @f@ returns,
-- so the C callee never sees a pointer to freed memory.
withMessages :: [ByteString] -> (Ptr (Ptr Word8) -> Ptr CSize -> CSize -> IO a) -> IO a
withMessages msgs f = go msgs []
where
go [] acc =
let cstrs = reverse acc
in withArray (map fst cstrs) $ \msgsPtr ->
withArray (map snd cstrs) $ \lensPtr ->
f msgsPtr lensPtr (fromIntegral $ length cstrs)
go (m : ms) acc =
BU.unsafeUseAsCStringLen m $ \(p, l) ->
go ms ((castPtr p :: Ptr Word8, fromIntegral l :: CSize) : acc)
withIndexes :: [Int] -> (Ptr CSize -> CSize -> IO a) -> IO a
withIndexes idxs f =
withArrayLen (map fromIntegral idxs :: [CSize]) $ \n ptr -> f ptr (fromIntegral n)
-- libbbs expects disclosed indexes strictly ascending and in [0, total). This
-- both matches the spec and guarantees the output-buffer size we compute matches
-- what libbbs writes (no out-of-bounds write from a bad index list).
ascendingInRange :: [Int] -> Int -> Bool
ascendingInRange idxs total =
all (\i -> i >= 0 && i < total) idxs && and (zipWith (<) idxs (drop 1 idxs))
-- Public API
bbsKeyGen :: IO (Either String BBSKeyPair)
bbsKeyGen =
allocaBytes bbsSkLen $ \skPtr ->
allocaBytes bbsPkLen $ \pkPtr -> do
rc <- c_bbs_keygen_full ciphersuite skPtr pkPtr
if rc /= 0
then pure $ Left "bbsKeyGen failed"
else do
sk <- packPtr skPtr bbsSkLen
pk <- packPtr pkPtr bbsPkLen
pure $ Right (BBSPublicKey pk, BBSSecretKey sk)
bbsPublicKey :: BBSSecretKey -> IO (Either String BBSPublicKey)
bbsPublicKey (BBSSecretKey sk) =
allocaBytes bbsPkLen $ \pkPtr ->
withBS sk $ \skPtr _ -> do
rc <- c_bbs_sk_to_pk ciphersuite skPtr pkPtr
if rc /= 0
then pure $ Left "bbsPublicKey failed"
else Right . BBSPublicKey <$> packPtr pkPtr bbsPkLen
bbsSign ::
BBSSecretKey ->
BBSHeader ->
[ByteString] ->
IO (Either String BBSSignature)
bbsSign secret@(BBSSecretKey sk) (BBSHeader header) msgs =
bbsPublicKey secret >>= either (pure . Left) sign'
where
sign' (BBSPublicKey pk) =
allocaBytes bbsSigLen $ \sigPtr ->
withBS sk $ \skPtr _ ->
withBS pk $ \pkPtr _ ->
withBS header $ \hdrPtr hdrLen ->
withMessages msgs $ \msgsPtr lensPtr n -> do
rc <- c_bbs_sign ciphersuite skPtr pkPtr sigPtr hdrPtr hdrLen n msgsPtr lensPtr
if rc /= 0
then pure $ Left "bbsSign failed"
else Right . BBSSignature <$> packPtr sigPtr bbsSigLen
bbsVerify ::
BBSPublicKey ->
BBSSignature ->
BBSHeader ->
[ByteString] ->
IO Bool
bbsVerify (BBSPublicKey pk) (BBSSignature sig) (BBSHeader header) msgs =
withBS pk $ \pkPtr _ ->
withBS sig $ \sigPtr _ ->
withBS header $ \hdrPtr hdrLen ->
withMessages msgs $ \msgsPtr lensPtr n -> do
rc <- c_bbs_verify ciphersuite pkPtr sigPtr hdrPtr hdrLen n msgsPtr lensPtr
pure (rc == 0)
bbsProofGen ::
BBSPublicKey ->
BBSSignature ->
BBSHeader ->
BBSPresHeader ->
[Int] ->
[ByteString] ->
IO (Either String BBSProof)
bbsProofGen (BBSPublicKey pk) (BBSSignature sig) (BBSHeader header) (BBSPresHeader ph) disclosedIdxs msgs
| not (ascendingInRange disclosedIdxs (length msgs)) = pure $ Left "bbsProofGen: invalid disclosed indexes"
| otherwise =
allocaBytes proofSz $ \proofPtr ->
withBS pk $ \pkPtr _ ->
withBS sig $ \sigPtr _ ->
withBS header $ \hdrPtr hdrLen ->
withBS ph $ \phPtr phLen ->
withIndexes disclosedIdxs $ \idxsPtr idxsLen ->
withMessages msgs $ \msgsPtr lensPtr n -> do
rc <- c_bbs_proof_gen ciphersuite pkPtr sigPtr proofPtr hdrPtr hdrLen phPtr phLen idxsPtr idxsLen n msgsPtr lensPtr
if rc /= 0
then pure $ Left "bbsProofGen failed"
else Right . BBSProof <$> packPtr proofPtr proofSz
where
numUndisclosed = length msgs - length disclosedIdxs
proofSz = bbsProofLen numUndisclosed
bbsProofVerify ::
BBSPublicKey ->
BBSProof ->
BBSHeader ->
BBSPresHeader ->
[Int] ->
Int ->
[ByteString] ->
IO Bool
bbsProofVerify (BBSPublicKey pk) (BBSProof proof) (BBSHeader header) (BBSPresHeader ph) disclosedIdxs numMessages disclosedMsgs
| numMessages < 0 = pure False
| length disclosedIdxs /= length disclosedMsgs = pure False
| not (ascendingInRange disclosedIdxs numMessages) = pure False
| B.length proof /= bbsProofLen (numMessages - length disclosedIdxs) = pure False
| otherwise =
withBS pk $ \pkPtr _ ->
withBS proof $ \proofPtr proofLen ->
withBS header $ \hdrPtr hdrLen ->
withBS ph $ \phPtr phLen ->
withIndexes disclosedIdxs $ \idxsPtr idxsLen ->
withMessages disclosedMsgs $ \msgsPtr lensPtr _ -> do
rc <- c_bbs_proof_verify ciphersuite pkPtr proofPtr proofLen hdrPtr hdrLen phPtr phLen idxsPtr idxsLen (fromIntegral numMessages) msgsPtr lensPtr
pure (rc == 0)
+20 -13
View File
@@ -45,6 +45,7 @@ module Simplex.Messaging.Crypto.Ratchet
AE2ERatchetParams (..),
E2ERatchetParamsUri (..),
E2ERatchetParams (..),
RcvE2EPrivRatchetParams,
VersionE2E,
VersionRangeE2E,
pattern VersionE2E,
@@ -403,24 +404,30 @@ instance RatchetKEMStateI s => ToField (PrivRKEMParams s) where toField = toFiel
instance (Typeable s, RatchetKEMStateI s) => FromField (PrivRKEMParams s) where fromField = blobFieldDecoder smpDecode
type E2EPrivRatchetParams s a = (PrivateKey a, PrivateKey a, Maybe (PrivRKEMParams s))\
type RcvE2EPrivRatchetParams a = E2EPrivRatchetParams 'RKSProposed a
type AE2EPrivRatchetParams a = (PrivateKey a, PrivateKey a, Maybe APrivRKEMParams)
data UseKEM (s :: RatchetKEMState) where
ProposeKEM :: UseKEM 'RKSProposed
AcceptKEM :: KEMPublicKey -> UseKEM 'RKSAccepted
data AUseKEM = forall s. RatchetKEMStateI s => AUseKEM (SRatchetKEMState s) (UseKEM s)
mkRcvE2ERatchetParams :: VersionE2E -> (PrivateKey a, PrivateKey a, Maybe RcvPrivRKEMParams) -> RcvE2ERatchetParams a
mkRcvE2ERatchetParams :: VersionE2E -> RcvE2EPrivRatchetParams a -> RcvE2ERatchetParams a
mkRcvE2ERatchetParams v (pk1, pk2, pKem) = E2ERatchetParams v (publicKey pk1) (publicKey pk2) (mkKem <$> pKem)
where
mkKem :: RcvPrivRKEMParams -> RcvRKEMParams
mkKem (PrivateRKParamsProposed (k, _)) = RKParamsProposed k
generateE2EParams :: forall s a. (AlgorithmI a, DhAlgorithm a) => TVar ChaChaDRG -> VersionE2E -> Maybe (UseKEM s) -> IO (PrivateKey a, PrivateKey a, Maybe (PrivRKEMParams s), E2ERatchetParams s a)
generateE2EParams :: forall s a. (AlgorithmI a, DhAlgorithm a) => TVar ChaChaDRG -> VersionE2E -> Maybe (UseKEM s) -> IO (E2EPrivRatchetParams s a, E2ERatchetParams s a)
generateE2EParams g v useKEM_ = do
(k1, pk1) <- atomically $ generateKeyPair g
(k2, pk2) <- atomically $ generateKeyPair g
kems <- kemParams
pure (pk1, pk2, snd <$> kems, E2ERatchetParams v k1 k2 (fst <$> kems))
pure ((pk1, pk2, snd <$> kems), E2ERatchetParams v k1 k2 (fst <$> kems))
where
kemParams :: IO (Maybe (RKEMParams s, PrivRKEMParams s))
kemParams = case useKEM_ of
@@ -436,7 +443,7 @@ generateE2EParams g v useKEM_ = do
_ -> pure Nothing
-- used by party initiating connection, Bob in double-ratchet spec
generateRcvE2EParams :: (AlgorithmI a, DhAlgorithm a) => TVar ChaChaDRG -> VersionE2E -> PQSupport -> IO (PrivateKey a, PrivateKey a, Maybe (PrivRKEMParams 'RKSProposed), E2ERatchetParams 'RKSProposed a)
generateRcvE2EParams :: (AlgorithmI a, DhAlgorithm a) => TVar ChaChaDRG -> VersionE2E -> PQSupport -> IO (RcvE2EPrivRatchetParams a, RcvE2ERatchetParams a)
generateRcvE2EParams g v = generateE2EParams g v . proposeKEM_
where
proposeKEM_ :: PQSupport -> Maybe (UseKEM 'RKSProposed)
@@ -445,14 +452,14 @@ generateRcvE2EParams g v = generateE2EParams g v . proposeKEM_
PQSupportOff -> Nothing
-- used by party accepting connection, Alice in double-ratchet spec
generateSndE2EParams :: forall a. (AlgorithmI a, DhAlgorithm a) => TVar ChaChaDRG -> VersionE2E -> Maybe AUseKEM -> IO (PrivateKey a, PrivateKey a, Maybe APrivRKEMParams, AE2ERatchetParams a)
generateSndE2EParams :: forall a. (AlgorithmI a, DhAlgorithm a) => TVar ChaChaDRG -> VersionE2E -> Maybe AUseKEM -> IO (AE2EPrivRatchetParams a, AE2ERatchetParams a)
generateSndE2EParams g v = \case
Nothing -> do
(pk1, pk2, _, e2eParams) <- generateE2EParams g v Nothing
pure (pk1, pk2, Nothing, AE2ERatchetParams SRKSProposed e2eParams)
((pk1, pk2, _), e2eParams) <- generateE2EParams g v Nothing
pure ((pk1, pk2, Nothing), AE2ERatchetParams SRKSProposed e2eParams)
Just (AUseKEM s useKEM) -> do
(pk1, pk2, pKem, e2eParams) <- generateE2EParams g v (Just useKEM)
pure (pk1, pk2, APRKP s <$> pKem, AE2ERatchetParams s e2eParams)
((pk1, pk2, pKem), e2eParams) <- generateE2EParams g v (Just useKEM)
pure ((pk1, pk2, APRKP s <$> pKem), AE2ERatchetParams s e2eParams)
data RatchetInitParams = RatchetInitParams
{ assocData :: Str,
@@ -464,9 +471,9 @@ data RatchetInitParams = RatchetInitParams
deriving (Show)
-- this is used by the peer joining the connection
pqX3dhSnd :: DhAlgorithm a => PrivateKey a -> PrivateKey a -> Maybe APrivRKEMParams -> E2ERatchetParams 'RKSProposed a -> Either CryptoError (RatchetInitParams, Maybe KEMKeyPair)
pqX3dhSnd :: DhAlgorithm a => AE2EPrivRatchetParams a -> E2ERatchetParams 'RKSProposed a -> Either CryptoError (RatchetInitParams, Maybe KEMKeyPair)
-- 3. replied 2. received
pqX3dhSnd spk1 spk2 spKem_ (E2ERatchetParams v rk1 rk2 rKem_) = do
pqX3dhSnd (spk1, spk2, spKem_) (E2ERatchetParams v rk1 rk2 rKem_) = do
(ks_, kem_) <- sndPq
let initParams = pqX3dh (publicKey spk1, rk1) (dh' rk1 spk2) (dh' rk2 spk1) (dh' rk2 spk2) kem_
pure (initParams, ks_)
@@ -480,9 +487,9 @@ pqX3dhSnd spk1 spk2 spKem_ (E2ERatchetParams v rk1 rk2 rKem_) = do
_ -> Right (Nothing, Nothing)
-- this is used by the peer that created new connection, after receiving the reply
pqX3dhRcv :: forall s a. (RatchetKEMStateI s, DhAlgorithm a) => PrivateKey a -> PrivateKey a -> Maybe (PrivRKEMParams 'RKSProposed) -> E2ERatchetParams s a -> ExceptT CryptoError IO (RatchetInitParams, Maybe KEMKeyPair)
pqX3dhRcv :: forall s a. (RatchetKEMStateI s, DhAlgorithm a) => RcvE2EPrivRatchetParams a -> E2ERatchetParams s a -> ExceptT CryptoError IO (RatchetInitParams, Maybe KEMKeyPair)
-- 1. sent 4. received in reply
pqX3dhRcv rpk1 rpk2 rpKem_ (E2ERatchetParams v sk1 sk2 sKem_) = do
pqX3dhRcv (rpk1, rpk2, rpKem_) (E2ERatchetParams v sk1 sk2 sKem_) = do
kem_ <- rcvPq
let initParams = pqX3dh (sk1, publicKey rpk1) (dh' sk2 rpk1) (dh' sk1 rpk2) (dh' sk2 rpk2) (snd <$> kem_)
pure (initParams, fst <$> kem_)
@@ -1,11 +1,14 @@
{-# LANGUAGE CPP #-}
{-# LANGUAGE TypeApplications #-}
{-# LANGUAGE PatternSynonyms #-}
module Simplex.Messaging.Crypto.SNTRUP761.Bindings
( KEMPublicKey (..),
( KEMPublicKey,
KEMSecretKey,
KEMCiphertext (..),
KEMSharedKey (..),
KEMCiphertext,
KEMSharedKey,
pattern KEMPublicKey,
pattern KEMSharedKey,
KEMKeyPair,
sntrup761Keypair,
sntrup761Enc,
@@ -25,8 +28,9 @@ import Simplex.Messaging.Crypto.SNTRUP761.Bindings.FFI
import Simplex.Messaging.Crypto.SNTRUP761.Bindings.RNG (rngFuncPtr, withDRG)
import Simplex.Messaging.Encoding
import Simplex.Messaging.Encoding.String
import Simplex.Messaging.Util ((<$?>))
newtype KEMPublicKey = KEMPublicKey ByteString
newtype KEMPublicKey = KEMPublicKey_ ByteString
deriving (Eq, Show)
newtype KEMSecretKey = KEMSecretKey ScrubbedBytes
@@ -35,18 +39,24 @@ newtype KEMSecretKey = KEMSecretKey ScrubbedBytes
newtype KEMCiphertext = KEMCiphertext ByteString
deriving (Eq, Show)
newtype KEMSharedKey = KEMSharedKey ScrubbedBytes
newtype KEMSharedKey = KEMSharedKey_ ScrubbedBytes
deriving (Eq, Show)
unsafeRevealKEMSharedKey :: KEMSharedKey -> String
unsafeRevealKEMSharedKey (KEMSharedKey scrubbed) = show (BA.convert scrubbed :: ByteString)
{-# DEPRECATED unsafeRevealKEMSharedKey "unsafeRevealKEMSharedKey left in code" #-}
pattern KEMPublicKey :: ByteString -> KEMPublicKey
pattern KEMPublicKey s <- KEMPublicKey_ s
pattern KEMSharedKey :: ScrubbedBytes -> KEMSharedKey
pattern KEMSharedKey s <- KEMSharedKey_ s
{-# COMPLETE KEMPublicKey #-}
{-# COMPLETE KEMSharedKey #-}
type KEMKeyPair = (KEMPublicKey, KEMSecretKey)
sntrup761Keypair :: TVar ChaChaDRG -> IO KEMKeyPair
sntrup761Keypair drg =
bimap KEMPublicKey KEMSecretKey
bimap KEMPublicKey_ KEMSecretKey
<$> BA.allocRet
c_SNTRUP761_SECRETKEY_SIZE
( \skPtr ->
@@ -57,7 +67,7 @@ sntrup761Keypair drg =
sntrup761Enc :: TVar ChaChaDRG -> KEMPublicKey -> IO (KEMCiphertext, KEMSharedKey)
sntrup761Enc drg (KEMPublicKey pk) =
BA.withByteArray pk $ \pkPtr ->
bimap KEMCiphertext KEMSharedKey
bimap KEMCiphertext KEMSharedKey_
<$> BA.allocRet
c_SNTRUP761_SIZE
( \kPtr ->
@@ -69,40 +79,47 @@ sntrup761Dec :: KEMCiphertext -> KEMSecretKey -> IO KEMSharedKey
sntrup761Dec (KEMCiphertext c) (KEMSecretKey sk) =
BA.withByteArray sk $ \skPtr ->
BA.withByteArray c $ \cPtr ->
KEMSharedKey
KEMSharedKey_
<$> BA.alloc c_SNTRUP761_SIZE (\kPtr -> c_sntrup761_dec kPtr cPtr skPtr)
parseKey :: BA.ByteArrayAccess bs => (bs -> key) -> String -> Int -> bs -> Either String key
parseKey kCon name expected s
| len == expected = Right $ kCon s
| otherwise = Left $ name <> " must be " <> show expected <> " bytes, got " <> show len
where
len = BA.length s
instance Encoding KEMSecretKey where
smpEncode (KEMSecretKey c) = smpEncode . Large $ BA.convert c
smpP = KEMSecretKey . BA.convert . unLarge <$> smpP
smpP = parseKey KEMSecretKey "SNTRUP761 secret key" c_SNTRUP761_SECRETKEY_SIZE . BA.convert . unLarge <$?> smpP
instance StrEncoding KEMSecretKey where
strEncode (KEMSecretKey pk) = strEncode (BA.convert pk :: ByteString)
strP = KEMSecretKey . BA.convert <$> strP @ByteString
strP = parseKey KEMSecretKey "SNTRUP761 secret key" c_SNTRUP761_SECRETKEY_SIZE . BA.convert <$?> strP @ByteString
instance Encoding KEMPublicKey where
smpEncode (KEMPublicKey pk) = smpEncode . Large $ BA.convert pk
smpP = KEMPublicKey . BA.convert . unLarge <$> smpP
smpP = parseKey KEMPublicKey_ "SNTRUP761 public key" c_SNTRUP761_PUBLICKEY_SIZE . unLarge <$?> smpP
instance StrEncoding KEMPublicKey where
strEncode (KEMPublicKey pk) = strEncode (BA.convert pk :: ByteString)
strP = KEMPublicKey . BA.convert <$> strP @ByteString
strP = parseKey KEMPublicKey_ "SNTRUP761 public key" c_SNTRUP761_PUBLICKEY_SIZE <$?> strP @ByteString
instance Encoding KEMCiphertext where
smpEncode (KEMCiphertext c) = smpEncode . Large $ BA.convert c
smpP = KEMCiphertext . BA.convert . unLarge <$> smpP
smpP = parseKey KEMCiphertext "SNTRUP761 ciphertext" c_SNTRUP761_CIPHERTEXT_SIZE . unLarge <$?> smpP
instance Encoding KEMSharedKey where
smpEncode (KEMSharedKey c) = smpEncode (BA.convert c :: ByteString)
smpP = KEMSharedKey . BA.convert <$> smpP @ByteString
smpP = KEMSharedKey_ . BA.convert <$> smpP @ByteString
instance StrEncoding KEMCiphertext where
strEncode (KEMCiphertext pk) = strEncode (BA.convert pk :: ByteString)
strP = KEMCiphertext . BA.convert <$> strP @ByteString
strP = parseKey KEMCiphertext "SNTRUP761 ciphertext" c_SNTRUP761_CIPHERTEXT_SIZE <$?> strP @ByteString
instance StrEncoding KEMSharedKey where
strEncode (KEMSharedKey pk) = strEncode (BA.convert pk :: ByteString)
strP = KEMSharedKey . BA.convert <$> strP @ByteString
strP = KEMSharedKey_ . BA.convert <$> strP @ByteString
instance ToJSON KEMSecretKey where
toJSON = strToJSON
@@ -130,9 +147,9 @@ instance ToField KEMSharedKey where
instance FromField KEMSharedKey where
#if defined(dbPostgres)
fromField f dat = KEMSharedKey . BA.convert @ByteString <$> fromField f dat
fromField f dat = KEMSharedKey_ . BA.convert @ByteString <$> fromField f dat
#else
fromField f = KEMSharedKey . BA.convert @ByteString <$> fromField f
fromField f = KEMSharedKey_ . BA.convert @ByteString <$> fromField f
#endif
instance ToJSON KEMSharedKey where
+24
View File
@@ -1,10 +1,14 @@
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE FlexibleInstances #-}
{-# LANGUAGE KindSignatures #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE ScopedTypeVariables #-}
module Simplex.Messaging.Encoding.String
( TextEncoding (..),
StrEncoding (..),
Str (..),
StrJSON (..),
strP_,
_strP,
strToJSON,
@@ -34,6 +38,7 @@ import Data.Int (Int64)
import Data.IntSet (IntSet)
import qualified Data.IntSet as IS
import qualified Data.List.NonEmpty as L
import Data.Proxy (Proxy (..))
import Data.Set (Set)
import qualified Data.Set as S
import Data.Text (Text)
@@ -44,6 +49,7 @@ import Data.Time.Format.ISO8601
import Data.Word (Word16, Word32)
import qualified Data.X509 as X
import qualified Data.X509.Validation as XV
import GHC.TypeLits (KnownSymbol, Symbol, symbolVal)
import Simplex.Messaging.Encoding
import Simplex.Messaging.Parsers (parseAll)
import Simplex.Messaging.Util (bshow, safeDecodeUtf8, (<$?>))
@@ -247,3 +253,21 @@ textToEncoding = JE.text . textEncode
textParseJSON :: TextEncoding a => String -> J.Value -> JT.Parser a
textParseJSON name = J.withText name $ maybe (fail name) pure . textDecode
-- | Derives ToJSON/FromJSON from the wrapped type's own StrEncoding (a base64url
-- string), so any validation that StrEncoding performs (e.g. length) also applies
-- to JSON parsing. The @name@ symbol is the parse error label. The type parameter
-- @a@ is essential: it makes parseJSON resolve at the wrapped type rather than at
-- ByteString. Use via DerivingVia, e.g.:
--
-- > newtype Key = Key ByteString
-- > deriving (ToJSON, FromJSON) via (StrJSON "Key" Key)
newtype StrJSON (name :: Symbol) a = StrJSON {unStrJSON :: a}
deriving (Eq, Show)
instance StrEncoding a => ToJSON (StrJSON name a) where
toJSON (StrJSON a) = strToJSON a
toEncoding (StrJSON a) = strToJEncoding a
instance forall name a. (KnownSymbol name, StrEncoding a) => FromJSON (StrJSON name a) where
parseJSON = fmap StrJSON . strParseJSON (symbolVal (Proxy :: Proxy name))
+46
View File
@@ -0,0 +1,46 @@
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE StrictData #-}
{-# LANGUAGE TemplateHaskell #-}
module Simplex.Messaging.Names.Record
( NameRecord (..),
)
where
import qualified Data.Aeson as J
import qualified Data.Aeson.TH as JQ
import Data.Text (Text)
import Simplex.Messaging.Parsers (defaultJSON, dropPrefix)
-- | Resolved name record returned by the names role. JSON keys match the
-- resolver REST output; both FromJSON (resolver -> server) and ToJSON
-- (server diagnostics) are TH-derived from one Options value, so the Haskell
-- type IS the schema. Text fields use the empty string as the "unset"
-- sentinel; coin fields use JSON null. simplexContact / simplexChannel are
-- arrays of links (primary first, empty when unset) so a name can advertise
-- fallback SMP servers. owner / resolver are 0x-hex Ethereum addresses, kept
-- verbatim as text (the resolver is the source of truth for their validity).
-- The only size bound is the SMP transport block (enforced by the framing).
data NameRecord = NameRecord
{ nrName :: Text,
nrNickname :: Text,
nrWebsite :: Text,
nrLocation :: Text,
nrSimplexContact :: [Text],
nrSimplexChannel :: [Text],
nrEth :: Maybe Text,
nrBtc :: Maybe Text,
nrXmr :: Maybe Text,
nrDot :: Maybe Text,
nrOwner :: Text,
nrResolver :: Text -- resolver address (0x hex) that produced the record
}
deriving (Eq, Show)
-- omitNothingFields False so absent coin fields surface as JSON null (matches
-- the resolver output for unset coins).
$( JQ.deriveJSON
defaultJSON {J.omitNothingFields = False, J.fieldLabelModifier = dropPrefix "nr"}
''NameRecord
)
+38 -18
View File
@@ -34,6 +34,7 @@ import Data.ByteString.Char8 (ByteString)
import qualified Data.ByteString.Char8 as B
import Data.Either (partitionEithers)
import Data.Functor (($>))
import Data.Hashable (hash)
import Data.IORef
import Data.Int (Int64)
import qualified Data.IntSet as IS
@@ -526,10 +527,10 @@ subscribeNtfs NtfSubscriber {smpSubscribers, subscriberSeq, smpAgent = ca} st sm
subscribeQueuesNtfs ca smpServer' [sub]
ntfSubscriber :: NtfSubscriber -> M ()
ntfSubscriber NtfSubscriber {smpAgent = ca@SMPClientAgent {msgQ, agentQ}} =
ntfSubscriber NtfSubscriber {smpAgent = ca@SMPClientAgent {msgQ = msgQ_, agentQ}} =
race_ receiveSMP receiveAgent
where
receiveSMP = do
receiveSMP = forM_ msgQ_ $ \msgQ -> do
st <- asks store
ps <- asks pushServer
stats <- asks serverStats
@@ -640,20 +641,38 @@ showServer' :: SMPServer -> Text
showServer' = decodeLatin1 . strEncode . host
pushNotification :: NtfPushServer -> Maybe T.Text -> OwnServer -> NtfTknRec -> PushNotification -> M ()
pushNotification s srvHost_ isOwn tkn@NtfTknRec {token = DeviceToken pp _} ntf = do
q <- getOrCreatePushWorker s (srvHost_, pp) isOwn
atomically $ writeTBQueue q (tkn, ntf)
pushNotification s srvHost_ isOwn tkn@NtfTknRec {token = token@(DeviceToken pp _)} ntf =
ifM
(pushProviderAllowed token)
(getOrCreatePushWorker s (srvHost_, pp, pushTokenShardId 8 tkn) isOwn >>= atomically . (`writeTBQueue` (tkn, ntf)))
(logWarn "skipping disabled APNS test push provider")
getOrCreatePushWorker :: NtfPushServer -> (Maybe T.Text, PushProvider) -> OwnServer -> M (TBQueue (NtfTknRec, PushNotification))
getOrCreatePushWorker s@NtfPushServer {pushWorkers, pushWorkerSeq, pushQSize} key@(srvHost_, _) isOwn = do
pushTokenShardId :: Int -> NtfTknRec -> Int
pushTokenShardId pushWorkersPerServer NtfTknRec {ntfTknId} =
hash (unEntityId ntfTknId) `mod` pushWorkersPerServer
pushProviderAllowed :: DeviceToken -> M Bool
pushProviderAllowed (DeviceToken PPApnsTest _) = asks (allowTestPushProvider . config)
pushProviderAllowed _ = pure True
guardPushProvider :: DeviceToken -> M NtfResponse -> M NtfResponse
guardPushProvider token action =
ifM
(pushProviderAllowed token)
action
(pure $ NRErr $ CMD SMP.PROHIBITED)
getOrCreatePushWorker :: NtfPushServer -> (Maybe T.Text, PushProvider, Int) -> OwnServer -> M (TBQueue (NtfTknRec, PushNotification))
getOrCreatePushWorker s@NtfPushServer {pushWorkers, pushWorkerSeq, pushQSize} key@(srvHost_, _, _) isOwn = do
ts <- liftIO getCurrentTime
atomically (getSessVar pushWorkerSeq key pushWorkers ts) >>= \case
Left v -> do
withGetSessVar' pushWorkerSeq key pushWorkers ts createWorker existingWorker
where
createWorker v = do
q <- liftIO $ newTBQueueIO pushQSize
tId <- mkWeakThreadId =<< forkIO (runPushWorker s srvHost_ isOwn q)
atomically $ putTMVar (sessionVar v) PushWorker {workerQ = q, workerThreadId = tId}
pure q
Right v -> workerQ <$> atomically (readTMVar $ sessionVar v)
existingWorker v = workerQ <$> atomically (readTMVar $ sessionVar v)
runPushWorker :: NtfPushServer -> Maybe T.Text -> OwnServer -> TBQueue (NtfTknRec, PushNotification) -> M ()
runPushWorker s srvHost_ isOwn q = forever $ do
@@ -690,7 +709,7 @@ runPushWorker s srvHost_ isOwn q = forever $ do
| otherwise = liftIO $ logError "bad notification token status"
deliverNotification :: NtfPostgresStore -> PushProvider -> NtfTknRec -> PushNotification -> IO (Either PushProviderError ())
deliverNotification st pp tkn@NtfTknRec {ntfTknId} ntf' = do
(deliver, clientVar) <- getPushClient s pp
(deliver, clientVar) <- getPushClient s pp tokenShardId
runExceptT (deliver tkn ntf') >>= \case
Right _ -> pure $ Right ()
Left e -> case e of
@@ -703,11 +722,12 @@ runPushWorker s srvHost_ isOwn q = forever $ do
err e
PPPermanentError -> err e
where
tokenShardId = pushTokenShardId 8 tkn
retryDeliver :: PushClientVar -> Text -> IO (Either PushProviderError ())
retryDeliver oldVar reason = do
logWarn $ "retrying push (" <> tshow pp <> ", " <> tshow ntfTknId <> "): " <> reason
atomically $ removeSessVar oldVar pp (pushClients s)
(deliver, _) <- getPushClient s pp
atomically $ removeSessVar oldVar (pp, tokenShardId) (pushClients s)
(deliver, _) <- getPushClient s pp tokenShardId
runExceptT (deliver tkn ntf') >>= \case
Right _ -> pure $ Right ()
Left e -> case e of
@@ -717,7 +737,7 @@ runPushWorker s srvHost_ isOwn q = forever $ do
_ -> err e
err e = logError ("Push provider error (" <> tshow pp <> ", " <> tshow ntfTknId <> "): " <> tshow e) $> Left e
pushWorkersQLength :: TMap (Maybe T.Text, PushProvider) PushWorkerVar -> IO Natural
pushWorkersQLength :: TMap (Maybe T.Text, PushProvider, Int) PushWorkerVar -> IO Natural
pushWorkersQLength workers = do
ws <- readTVarIO workers
foldM addQLength 0 ws
@@ -834,7 +854,7 @@ client NtfServerClient {rcvQ, sndQ} ns@NtfSubscriber {smpAgent = ca} ps =
where
processCommand :: NtfRequest -> M (Transmission NtfResponse)
processCommand = \case
NtfReqNew corrId (ANE SToken newTkn@(NewNtfTkn token _ dhPubKey)) -> (corrId,NoEntity,) <$> do
NtfReqNew corrId (ANE SToken newTkn@(NewNtfTkn token _ dhPubKey)) -> (corrId,NoEntity,) <$> guardPushProvider token (do
logDebug "TNEW - new token"
(srvDhPubKey, srvDhPrivKey) <- atomically . C.generateKeyPair =<< asks random
let dhSecret = C.dh' dhPubKey srvDhPrivKey
@@ -846,10 +866,10 @@ client NtfServerClient {rcvQ, sndQ} ns@NtfSubscriber {smpAgent = ca} ps =
pushNotification ps Nothing False tkn $ PNVerification regCode
incNtfStatT token ntfVrfQueued
incNtfStatT token tknCreated
pure $ NRTknId tknId srvDhPubKey
pure $ NRTknId tknId srvDhPubKey)
NtfReqCmd SToken (NtfTkn tkn@NtfTknRec {token, ntfTknId, tknStatus, tknRegCode, tknDhSecret, tknDhPrivKey}) (corrId, tknId, cmd) -> do
(corrId,tknId,) <$> case cmd of
TNEW (NewNtfTkn _ _ dhPubKey) -> do
TNEW (NewNtfTkn _ _ dhPubKey) -> guardPushProvider token $ do
logDebug "TNEW - registered token"
let dhSecret = C.dh' dhPubKey tknDhPrivKey
-- it is required that DH secret is the same, to avoid failed verifications if notification is delaying
@@ -872,7 +892,7 @@ client NtfServerClient {rcvQ, sndQ} ns@NtfSubscriber {smpAgent = ca} ps =
TCHK -> do
logDebug "TCHK"
pure $ NRTkn tknStatus
TRPL token' -> do
TRPL token' -> guardPushProvider token' $ do
logDebug "TRPL - replace token"
regCode <- getRegCode
let tkn' = tkn {token = token', tknStatus = NTRegistered, tknRegCode = regCode}
@@ -30,11 +30,9 @@ module Simplex.Messaging.Notifications.Server.Env
import Control.Concurrent (ThreadId)
import qualified Control.Exception as E
import Control.Logger.Simple
import Control.Monad
import Control.Monad.Except
import Control.Monad.Trans.Except
import Crypto.Random
import Data.Functor (($>))
import Data.Int (Int64)
import Simplex.Messaging.Agent.RetryInterval
import Data.List.NonEmpty (NonEmpty)
@@ -66,7 +64,6 @@ import Simplex.Messaging.Transport.Credentials (genCredentials, tlsCredentials)
import Simplex.Messaging.Transport.Server (AddHTTP, ServerCredentials, TransportServerConfig, loadFingerprint, loadServerCredential)
import Simplex.Messaging.Util (liftEitherWith, tshow)
import Simplex.Messaging.Util ()
import System.Exit (exitFailure)
import System.Mem.Weak (Weak)
import UnliftIO.STM
@@ -81,6 +78,7 @@ data NtfServerConfig = NtfServerConfig
pushQSize :: Natural,
smpAgentCfg :: SMPClientAgentConfig,
apnsConfig :: APNSPushClientConfig,
allowTestPushProvider :: Bool,
subsBatchSize :: Int,
inactiveClientExpiration :: Maybe ExpirationConfig,
dbStoreConfig :: PostgresStoreCfg,
@@ -173,10 +171,10 @@ data SMPSubscriber = SMPSubscriber
}
data NtfPushServer = NtfPushServer
{ pushWorkers :: TMap (Maybe T.Text, PushProvider) PushWorkerVar,
{ pushWorkers :: TMap (Maybe T.Text, PushProvider, Int) PushWorkerVar, -- Int is the worker shard
pushWorkerSeq :: TVar Int,
pushQSize :: Natural,
pushClients :: TMap PushProvider PushClientVar,
pushClients :: TMap (PushProvider, Int) PushClientVar,
pushClientSeq :: TVar Int,
apnsConfig :: APNSPushClientConfig
}
@@ -202,11 +200,11 @@ newNtfPushServer pushQSize apnsConfig = do
-- | Single-flight access to the per-provider push client with bounded retry.
-- The returned PushClientVar is the handle retryDeliver passes to removeSessVar to evict
-- this specific instance before re-fetching.
getPushClient :: NtfPushServer -> PushProvider -> IO (PushProviderClient, PushClientVar)
getPushClient s@NtfPushServer {apnsConfig = APNSPushClientConfig {reconnectInterval}} pp =
getPushClient :: NtfPushServer -> PushProvider -> Int -> IO (PushProviderClient, PushClientVar)
getPushClient s@NtfPushServer {apnsConfig = APNSPushClientConfig {reconnectInterval}} pp tokenShardId =
withRetryIntervalCount reconnectInterval $ \n _delay loop -> do
ts <- getCurrentTime
E.try (atomically (getSessVar (pushClientSeq s) pp (pushClients s) ts) >>= either (newPushClient s pp) waitForPushClient) >>= \case
E.try (atomically (getSessVar (pushClientSeq s) (pp, tokenShardId) (pushClients s) ts) >>= either (newPushClient s pp tokenShardId) waitForPushClient) >>= \case
Right result -> pure result
Left e
| n < 2 -> do
@@ -214,15 +212,15 @@ getPushClient s@NtfPushServer {apnsConfig = APNSPushClientConfig {reconnectInter
loop
| otherwise -> E.throwIO e
newPushClient :: NtfPushServer -> PushProvider -> PushClientVar -> IO (PushProviderClient, PushClientVar)
newPushClient NtfPushServer {pushClients, apnsConfig} pp v = do
newPushClient :: NtfPushServer -> PushProvider -> Int -> PushClientVar -> IO (PushProviderClient, PushClientVar)
newPushClient NtfPushServer {pushClients, apnsConfig} pp tokenShardId v = do
r <- E.try $ case apnsProviderHost pp of
Nothing -> pure $ \_ _ -> pure ()
Just host -> apnsPushProviderClient <$> createAPNSPushClient host apnsConfig
atomically $ do
putTMVar (sessionVar v) r
case r of
Left _ -> removeSessVar v pp pushClients
Left _ -> removeSessVar v (pp, tokenShardId) pushClients
Right _ -> pure ()
either E.throwIO (\c -> pure (c, v)) r
@@ -193,6 +193,7 @@ ntfServerCLI cfgPath logPath =
persistErrorInterval = 0 -- seconds
},
apnsConfig = defaultAPNSPushClientConfig,
allowTestPushProvider = False,
subsBatchSize = 900,
inactiveClientExpiration =
settingIsOn "INACTIVE_CLIENTS" "disconnect" ini
+64 -3
View File
@@ -80,6 +80,7 @@ module Simplex.Messaging.Protocol
ErrorType (..),
CommandError (..),
ProxyError (..),
NameErrorType (..),
BrokerErrorType (..),
NetworkError (..),
BlockingInfo (..),
@@ -163,6 +164,7 @@ module Simplex.Messaging.Protocol
EncTransmission (..),
FwdResponse (..),
FwdTransmission (..),
NameRecord (..),
MsgFlags (..),
initialSMPClientVersion,
currentSMPClientVersion,
@@ -263,10 +265,12 @@ import Simplex.Messaging.Agent.Store.DB (Binary (..), FromField (..), ToField (.
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Encoding
import Simplex.Messaging.Encoding.String
import Simplex.Messaging.Names.Record (NameRecord (..))
import Simplex.Messaging.Parsers
import Simplex.Messaging.Protocol.Types
import Simplex.Messaging.Server.QueueStore.QueueInfo
import Simplex.Messaging.ServiceScheme
import Simplex.Messaging.SimplexName (SimplexDomain)
import Simplex.Messaging.Transport
import Simplex.Messaging.Transport.Client (TransportHost, TransportHosts (..))
import Simplex.Messaging.Util (bshow, eitherToMaybe, safeDecodeUtf8, (<$?>))
@@ -343,6 +347,7 @@ data Party
| LinkClient
| ProxiedClient
| ProxyService
| Resolver
deriving (Show)
-- | Singleton types for SMP protocol clients
@@ -357,6 +362,7 @@ data SParty :: Party -> Type where
SSenderLink :: SParty LinkClient
SProxiedClient :: SParty ProxiedClient
SProxyService :: SParty ProxyService
SResolver :: SParty Resolver
instance TestEquality SParty where
testEquality SCreator SCreator = Just Refl
@@ -369,6 +375,7 @@ instance TestEquality SParty where
testEquality SSenderLink SSenderLink = Just Refl
testEquality SProxiedClient SProxiedClient = Just Refl
testEquality SProxyService SProxyService = Just Refl
testEquality SResolver SResolver = Just Refl
testEquality _ _ = Nothing
deriving instance Show (SParty p)
@@ -395,6 +402,8 @@ instance PartyI ProxiedClient where sParty = SProxiedClient
instance PartyI ProxyService where sParty = SProxyService
instance PartyI Resolver where sParty = SResolver
-- command parties that can read queues
type family QueueParty (p :: Party) :: Constraint where
QueueParty Recipient = ()
@@ -473,6 +482,7 @@ partyClientRole = \case
SSenderLink -> Just SRMessaging
SProxiedClient -> Just SRMessaging
SProxyService -> Just SRProxy
SResolver -> Just SRMessaging
{-# INLINE partyClientRole #-}
partyServiceRole :: ServiceParty p => SParty p -> SMPServiceRole
@@ -597,6 +607,8 @@ data Command (p :: Party) where
-- - entity ID: empty
-- - corrId: unique correlation ID between proxy and relay, also used as a nonce to encrypt forwarded transmission
RFWD :: EncFwdTransmission -> Command ProxyService -- use CorrId as CbNonce, proxy to relay
-- Resolve SimpleX name.
RSLV :: SimplexDomain -> Command Resolver
deriving instance Show (Command p)
@@ -732,6 +744,8 @@ data BrokerMsg where
OK :: BrokerMsg
ERR :: ErrorType -> BrokerMsg
PONG :: BrokerMsg
-- Resolved SimpleX name.
RNAME :: NameRecord -> BrokerMsg
deriving (Eq, Show)
data RcvMessage = RcvMessage
@@ -942,6 +956,7 @@ data CommandTag (p :: Party) where
RFWD_ :: CommandTag ProxyService
NSUB_ :: CommandTag Notifier
NSUBS_ :: CommandTag NotifierService
RSLV_ :: CommandTag Resolver
data CmdTag = forall p. PartyI p => CT (SParty p) (CommandTag p)
@@ -968,6 +983,7 @@ data BrokerMsgTag
| OK_
| ERR_
| PONG_
| RNAME_
deriving (Show)
class ProtocolMsgTag t where
@@ -1004,6 +1020,7 @@ instance PartyI p => Encoding (CommandTag p) where
RFWD_ -> "RFWD"
NSUB_ -> "NSUB"
NSUBS_ -> "NSUBS"
RSLV_ -> "RSLV"
smpP = messageTagP
instance ProtocolMsgTag CmdTag where
@@ -1032,6 +1049,7 @@ instance ProtocolMsgTag CmdTag where
"RFWD" -> Just $ CT SProxyService RFWD_
"NSUB" -> Just $ CT SNotifier NSUB_
"NSUBS" -> Just $ CT SNotifierService NSUBS_
"RSLV" -> Just $ CT SResolver RSLV_
_ -> Nothing
instance Encoding CmdTag where
@@ -1061,6 +1079,7 @@ instance Encoding BrokerMsgTag where
OK_ -> "OK"
ERR_ -> "ERR"
PONG_ -> "PONG"
RNAME_ -> "RNAME"
smpP = messageTagP
instance ProtocolMsgTag BrokerMsgTag where
@@ -1083,6 +1102,7 @@ instance ProtocolMsgTag BrokerMsgTag where
"OK" -> Just OK_
"ERR" -> Just ERR_
"PONG" -> Just PONG_
"RNAME" -> Just RNAME_
_ -> Nothing
-- | SMP message body format
@@ -1526,11 +1546,14 @@ queueIdHash = IdsHash . C.md5Hash . unEntityId
{-# INLINE queueIdHash #-}
addServiceSubs :: (Int64, IdsHash) -> (Int64, IdsHash) -> (Int64, IdsHash)
addServiceSubs (n', idsHash') (n, idsHash) = (n + n', idsHash <> idsHash')
addServiceSubs (n', idsHash') (n, idsHash) =
let !n'' = n + n'
!h = idsHash <> idsHash'
in (n'', h)
subtractServiceSubs :: (Int64, IdsHash) -> (Int64, IdsHash) -> (Int64, IdsHash)
subtractServiceSubs (n', idsHash') (n, idsHash)
| n > n' = (n - n', idsHash <> idsHash') -- concat is a reversible xor: (x `xor` y) `xor` y == x
| n > n' = let !n'' = n - n'; !h = idsHash <> idsHash' in (n'', h) -- concat is a reversible xor: (x `xor` y) `xor` y == x
| otherwise = (0, mempty)
data ProtocolErrorType = PECmdSyntax | PECmdUnknown | PESession | PEBlock
@@ -1565,10 +1588,22 @@ data ErrorType
EXPIRED
| -- | internal server error
INTERNAL
| -- | name resolution error
NAME {nameErr :: NameErrorType}
| -- | used internally, never returned by the server (to be removed)
DUPLICATE_ -- not part of SMP protocol, used internally
deriving (Eq, Show)
-- | Name resolution error
data NameErrorType
= -- | the names role / resolver is not configured on this server
NO_RESOLVER
| -- | the name is not registered (resolver returned not-found)
NOT_FOUND
| -- | backing resolver/RPC failure - contains the diagnostic detail
RESOLVER {resolverErr :: Text}
deriving (Eq, Show)
instance StrEncoding ErrorType where
strEncode = \case
BLOCK -> "BLOCK"
@@ -1585,6 +1620,7 @@ instance StrEncoding ErrorType where
LARGE_MSG -> "LARGE_MSG"
EXPIRED -> "EXPIRED"
INTERNAL -> "INTERNAL"
NAME e -> "NAME " <> strEncode e
DUPLICATE_ -> "DUPLICATE_"
strP =
A.choice
@@ -1592,6 +1628,7 @@ instance StrEncoding ErrorType where
"SESSION" $> SESSION,
"CMD " *> (CMD <$> parseRead1),
"PROXY " *> (PROXY <$> strP),
"NAME " *> (NAME <$> strP),
"AUTH" $> AUTH,
"BLOCKED " *> strP,
"SERVICE" $> SERVICE,
@@ -1792,6 +1829,7 @@ instance PartyI p => ProtocolEncoding SMPVersion ErrorType (Command p) where
PRXY host auth_ -> e (PRXY_, ' ', host, auth_)
PFWD fwdV pubKey (EncTransmission s) -> e (PFWD_, ' ', fwdV, pubKey, Tail s)
RFWD (EncFwdTransmission s) -> e (RFWD_, ' ', Tail s)
RSLV d -> e (RSLV_, ' ', d)
where
e :: Encoding a => a -> ByteString
e = smpEncode
@@ -1816,6 +1854,7 @@ instance PartyI p => ProtocolEncoding SMPVersion ErrorType (Command p) where
PRXY {} -> noAuthCmd
PFWD {} -> entityCmd
RFWD _ -> noAuthCmd
RSLV _ -> noAuthCmd
SUB -> serviceCmd
NSUB -> serviceCmd
-- other client commands must have both signature and queue ID
@@ -1899,6 +1938,7 @@ instance ProtocolEncoding SMPVersion ErrorType Cmd where
CT SNotifierService NSUBS_
| v >= rcvServiceSMPVersion -> Cmd SNotifierService <$> (NSUBS <$> _smpP <*> smpP)
| otherwise -> pure $ Cmd SNotifierService $ NSUBS (-1) mempty
CT SResolver RSLV_ -> Cmd SResolver . RSLV <$> _smpP <* A.takeByteString
fromProtocolError = fromProtocolError @SMPVersion @ErrorType @BrokerMsg
{-# INLINE fromProtocolError #-}
@@ -1945,6 +1985,7 @@ instance ProtocolEncoding SMPVersion ErrorType BrokerMsg where
| v < clientNoticesSMPVersion -> BLOCKED info {notice = Nothing}
_ -> err
PONG -> e PONG_
RNAME rec -> e (RNAME_, ' ', Tail $ LB.toStrict $ J.encode rec)
where
e :: Encoding a => a -> ByteString
e = smpEncode
@@ -1992,6 +2033,7 @@ instance ProtocolEncoding SMPVersion ErrorType BrokerMsg where
OK_ -> pure OK
ERR_ -> ERR <$> _smpP
PONG_ -> pure PONG
RNAME_ -> fmap RNAME . J.eitherDecodeStrict . unTail <$?> _smpP
where
serviceRespP resp
| v >= rcvServiceSMPVersion = resp <$> _smpP <*> smpP
@@ -2014,6 +2056,7 @@ instance ProtocolEncoding SMPVersion ErrorType BrokerMsg where
PKEY {} -> noEntityMsg
RRES _ -> noEntityMsg
ALLS -> noEntityMsg
RNAME _ -> noEntityMsg
-- other broker responses must have queue ID
_
| B.null entId -> Left $ CMD NO_ENTITY
@@ -2056,6 +2099,7 @@ instance Encoding ErrorType where
NO_MSG -> "NO_MSG"
LARGE_MSG -> "LARGE_MSG"
INTERNAL -> "INTERNAL"
NAME err -> "NAME " <> smpEncode err
DUPLICATE_ -> "DUPLICATE_"
smpP =
@@ -2074,9 +2118,26 @@ instance Encoding ErrorType where
"NO_MSG" -> pure NO_MSG
"LARGE_MSG" -> pure LARGE_MSG
"INTERNAL" -> pure INTERNAL
"NAME" -> NAME <$> _smpP
"DUPLICATE_" -> pure DUPLICATE_
_ -> fail "bad ErrorType"
instance Encoding NameErrorType where
smpEncode = \case
NO_RESOLVER -> "NO_RESOLVER"
NOT_FOUND -> "NOT_FOUND"
RESOLVER e -> "RESOLVER " <> encodeUtf8 e
smpP =
A.takeTill (== ' ') >>= \case
"NO_RESOLVER" -> pure NO_RESOLVER
"NOT_FOUND" -> pure NOT_FOUND
"RESOLVER" -> RESOLVER . safeDecodeUtf8 <$> (A.space *> A.takeByteString)
_ -> fail "bad NameErrorType"
instance StrEncoding NameErrorType where
strEncode = smpEncode
strP = smpP
instance Encoding CommandError where
smpEncode e = case e of
UNKNOWN -> "UNKNOWN"
@@ -2376,4 +2437,4 @@ $(J.deriveJSON (sumTypeJSON id) ''BrokerErrorType)
$(J.deriveJSON defaultJSON ''BlockingInfo)
-- run deriveJSON in one TH splice to allow mutual instance
$(concat <$> mapM @[] (J.deriveJSON (sumTypeJSON id)) [''ProxyError, ''ErrorType])
$(concat <$> mapM @[] (J.deriveJSON (sumTypeJSON id)) [''ProxyError, ''NameErrorType, ''ErrorType])
+90 -49
View File
@@ -65,7 +65,7 @@ import Data.Constraint (Dict (..))
import Data.Dynamic (toDyn)
import Data.Either (fromRight, partitionEithers)
import Data.Foldable (foldrM)
import Data.Functor (($>))
import Data.Functor (($>), (<&>))
import Data.IORef
import Data.Int (Int64)
import qualified Data.IntMap.Strict as IM
@@ -103,11 +103,13 @@ import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Encoding
import Simplex.Messaging.Encoding.String
import Simplex.Messaging.Protocol
import Simplex.Messaging.SimplexName (SimplexDomain)
import Simplex.Messaging.Server.Control
import Simplex.Messaging.Server.Env.STM as Env
import Simplex.Messaging.Server.Expiration
import Simplex.Messaging.Server.MsgStore
import Simplex.Messaging.Server.MsgStore.Journal (JournalMsgStore, JournalQueue (..), getJournalQueueMessages)
import Simplex.Messaging.Server.Names (NamesEnv, closeNamesEnv, resolveName)
import Simplex.Messaging.Server.MsgStore.STM
import Simplex.Messaging.Server.MsgStore.Types
import Simplex.Messaging.Server.NtfStore
@@ -245,7 +247,9 @@ smpServer started cfg@ServerConfig {transports, transportConfig = tCfg, startOpt
saveServerStats
closeServer :: M s ()
closeServer = asks (smpAgent . proxyAgent) >>= liftIO . closeSMPClientAgent
closeServer = do
asks (smpAgent . proxyAgent) >>= liftIO . closeSMPClientAgent
asks namesEnv >>= liftIO . mapM_ closeNamesEnv
serverThread ::
forall sub. String ->
@@ -329,21 +333,25 @@ smpServer started cfg@ServerConfig {transports, transportConfig = tCfg, startOpt
endPreviousSubscriptions = mapM_ $ \(c, subAction, evt) -> do
atomically $ modifyTVar' pendingEvents $ IM.alter (Just . maybe [evt] (evt <|)) (clientId c)
case subAction of
CSAEndSub qId -> atomically (endSub c qId) >>= a unsub_
where
a (Just unsub) (Just s) = unsub s
a _ _ = pure ()
CSAEndServiceSub qId -> atomically $ do
modifyTVar' (clientServiceSubs c) decrease
modifyTVar' totalServiceSubs decrease
where
decrease = subtractServiceSubs (1, queueIdHash qId)
CSAEndSub qId -> atomically (endSub c qId) >>= unsubPrev
-- like endSub, also removes the delivery subscription from the service client's subscriptions map,
-- otherwise the map retains entries for queues unassociated/deleted while the service stays connected.
CSAEndServiceSub qId -> atomically (endServiceQueueSub c qId) >>= unsubPrev
CSADecreaseSubs changedSubs -> do
atomically $ modifyTVar' totalServiceSubs $ subtractServiceSubs changedSubs
forM_ unsub_ $ \unsub -> atomically (swapTVar (clientSubs c) M.empty) >>= mapM_ unsub
where
unsubPrev :: Maybe sub -> IO ()
unsubPrev s_ = sequence_ (unsub_ <*> s_)
endSub :: Client s -> QueueId -> STM (Maybe sub)
endSub c qId = TM.lookupDelete qId (clientSubs c) >>= (removeWhenNoSubs c $>)
endServiceQueueSub :: Client s -> QueueId -> STM (Maybe sub)
endServiceQueueSub c qId = do
modifyTVar' (clientServiceSubs c) decrease
modifyTVar' totalServiceSubs decrease
endSub c qId
where
decrease = subtractServiceSubs (1, queueIdHash qId)
-- remove client from server's subscribed cients
removeWhenNoSubs c = do
noClientSubs <- null <$> readTVar (clientSubs c)
@@ -513,7 +521,7 @@ smpServer started cfg@ServerConfig {transports, transportConfig = tCfg, startOpt
initialDelay <- (startAt -) . fromIntegral . (`div` 1000000_000000) . diffTimeToPicoseconds . utctDayTime <$> liftIO getCurrentTime
liftIO $ putStrLn $ "server stats log enabled: " <> statsFilePath
liftIO $ threadDelay' $ 1000000 * (initialDelay + if initialDelay < 0 then 86400 else 0)
ss@ServerStats {fromTime, qCreated, qSecured, qDeletedAll, qDeletedAllB, qDeletedNew, qDeletedSecured, qSub, qSubAllB, qSubAuth, qSubDuplicate, qSubProhibited, qSubEnd, qSubEndB, ntfCreated, ntfDeleted, ntfDeletedB, ntfSub, ntfSubB, ntfSubAuth, ntfSubDuplicate, msgSent, msgSentAuth, msgSentQuota, msgSentLarge, msgRecv, msgRecvGet, msgGet, msgGetNoMsg, msgGetAuth, msgGetDuplicate, msgGetProhibited, msgExpired, activeQueues, msgSentNtf, msgRecvNtf, activeQueuesNtf, qCount, msgCount, ntfCount, pRelays, pRelaysOwn, pMsgFwds, pMsgFwdsOwn, pMsgFwdsRecv, rcvServices, ntfServices}
ss@ServerStats {fromTime, qCreated, qSecured, qDeletedAll, qDeletedAllB, qDeletedNew, qDeletedSecured, qSub, qSubAllB, qSubAuth, qSubDuplicate, qSubProhibited, qSubEnd, qSubEndB, ntfCreated, ntfDeleted, ntfDeletedB, ntfSub, ntfSubB, ntfSubAuth, ntfSubDuplicate, msgSent, msgSentAuth, msgSentQuota, msgSentLarge, msgRecv, msgRecvGet, msgGet, msgGetNoMsg, msgGetAuth, msgGetDuplicate, msgGetProhibited, msgExpired, activeQueues, msgSentNtf, msgRecvNtf, activeQueuesNtf, qCount, msgCount, ntfCount, pRelays, pRelaysOwn, pMsgFwds, pMsgFwdsOwn, pMsgFwdsRecv, rcvServices, ntfServices, rslvStats}
<- asks serverStats
st <- asks msgStore
EntityCounts {queueCount, notifierCount, rcvServiceCount, ntfServiceCount, rcvServiceQueuesCount, ntfServiceQueuesCount} <-
@@ -576,6 +584,7 @@ smpServer started cfg@ServerConfig {transports, transportConfig = tCfg, startOpt
qCount' <- readIORef qCount
msgCount' <- readIORef msgCount
ntfCount' <- readIORef ntfCount
rslvStats' <- getResetNameResolverStatsData rslvStats
T.hPutStrLn h $
T.intercalate
","
@@ -649,6 +658,7 @@ smpServer started cfg@ServerConfig {transports, transportConfig = tCfg, startOpt
]
<> showServiceStats rcvServices'
<> showServiceStats ntfServices'
<> showNameResolverStats rslvStats'
)
liftIO $ threadDelay' interval
where
@@ -656,6 +666,8 @@ smpServer started cfg@ServerConfig {transports, transportConfig = tCfg, startOpt
map tshow [_pRequests, _pSuccesses, _pErrorsConnect, _pErrorsCompat, _pErrorsOther]
showServiceStats ServiceStatsData {_srvAssocNew, _srvAssocDuplicate, _srvAssocUpdated, _srvAssocRemoved, _srvSubCount, _srvSubDuplicate, _srvSubQueues, _srvSubEnd} =
map tshow [_srvAssocNew, _srvAssocDuplicate, _srvAssocUpdated, _srvAssocRemoved, _srvSubCount, _srvSubDuplicate, _srvSubQueues, _srvSubEnd]
showNameResolverStats NameResolverStatsData {_rslvReqs, _rslvSucc, _rslvNotFound, _rslvResolverErrs, _rslvDisabled} =
map tshow [_rslvReqs, _rslvSucc, _rslvNotFound, _rslvResolverErrs, _rslvDisabled]
prometheusMetricsThread_ :: ServerConfig s -> [M s ()]
prometheusMetricsThread_ ServerConfig {prometheusInterval = Just interval, prometheusMetricsFile} =
@@ -1262,6 +1274,7 @@ verifyQueueTransmission service thAuth (tAuth, authorized, (corrId, entId, comma
vc SNotifierService NSUBS {} = verifyServiceCmd
vc SProxiedClient _ = VRVerified Nothing
vc SProxyService (RFWD _) = VRVerified Nothing
vc SResolver (RSLV _) = VRVerified Nothing
checkRole = case (service, partyClientRole p) of
(Just THClientService {serviceRole}, Just role) -> serviceRole == role
_ -> True
@@ -1459,19 +1472,40 @@ client
Nothing -> inc False pRequests >> inc False pErrorsConnect $> Just (ERR $ PROXY NO_SESSION)
where
forkProxiedCmd :: M s BrokerMsg -> M s (Maybe BrokerMsg)
forkProxiedCmd cmdAction = do
bracket_ wait signal . forkClient clnt (B.unpack $ "client $" <> encode sessionId <> " proxy") $ do
-- commands MUST be processed under a reasonable timeout or the client would halt
cmdAction >>= \t -> atomically $ writeTBQueue sndQ ([(corrId, EntityId sessId, t)], [])
pure Nothing
where
wait = do
ServerConfig {serverClientConcurrency} <- asks config
atomically $ do
used <- readTVar procThreads
when (used >= serverClientConcurrency) retry
writeTVar procThreads $! used + 1
signal = atomically $ modifyTVar' procThreads (\t -> t - 1)
forkProxiedCmd = forkCmd serverClientConcurrency corrId (EntityId sessId)
-- Run a slow command on a thread
forkCmd :: (ServerConfig s -> Int) -> CorrId -> EntityId -> M s BrokerMsg -> M s (Maybe a)
forkCmd concurrency corrId entId cmdAction = do
bracket_ wait signal . forkClient clnt (B.unpack $ "client $" <> encode sessionId <> " cmd") $
-- commands MUST be processed under a reasonable timeout or the client would halt
cmdAction >>= \t -> atomically $ writeTBQueue sndQ ([(corrId, entId, t)], [])
pure Nothing
where
wait = do
limit <- asks (concurrency . config)
atomically $ do
used <- readTVar procThreads
when (used >= limit) retry
writeTVar procThreads $! used + 1
signal = atomically $ modifyTVar' procThreads (\t -> t - 1)
rslvNamesEnv :: M s (Maybe NamesEnv)
rslvNamesEnv = do
st <- asks (rslvStats . serverStats)
incStat (rslvReqs st)
asks namesEnv >>= \case
Nothing -> incStat (rslvDisabled st) $> Nothing
Just nenv -> pure (Just nenv)
-- Runs on a forked thread so RSLV does not block other commands;
-- concurrency is limited by serverResolverConcurrency in forkCmd.
resolveNameMsg :: NamesEnv -> SimplexDomain -> M s BrokerMsg
resolveNameMsg nenv d = do
st <- asks (rslvStats . serverStats)
(selector, msg) <-
liftIO (resolveName nenv d) <&> \case
Right rec -> (rslvSucc, RNAME rec)
Left e@NOT_FOUND -> (rslvNotFound, ERR $ NAME e)
Left e -> (rslvResolverErrs, ERR $ NAME e)
incStat (selector st) $> msg
transportErr :: TransportError -> ErrorType
transportErr = PROXY . BROKER . TRANSPORT
mkIncProxyStats :: MonadIO m => ProxyStats -> ProxyStats -> OwnServer -> (ProxyStats -> IORef Int) -> m ()
@@ -1485,7 +1519,10 @@ client
SKEY k -> withQueue $ \q qr -> checkMode QMMessaging qr $ secureQueue_ q k
SEND flags msgBody -> response <$> withQueue_ False err (sendMessage flags msgBody)
Cmd SIdleClient PING -> pure $ response (corrId, NoEntity, PONG)
Cmd SProxyService (RFWD encBlock) -> response . (corrId,NoEntity,) <$> processForwardedCommand encBlock
Cmd SProxyService (RFWD encBlock) -> (response . (corrId, NoEntity,) =<<) <$> processForwardedCommand encBlock
Cmd SResolver (RSLV d) -> rslvNamesEnv >>= \case
Nothing -> pure $ response (corrId, NoEntity, ERR (NAME NO_RESOLVER))
Just nenv -> forkCmd serverResolverConcurrency corrId NoEntity (resolveNameMsg nenv d)
Cmd SSenderLink command -> case command of
LKEY k -> withQueue $ \q qr -> checkMode QMMessaging qr $ secureQueue_ q k $>> getQueueLink_ q qr
LGET -> withQueue $ \q qr -> checkContact qr $ getQueueLink_ q qr
@@ -2083,8 +2120,8 @@ client
encNMsgMeta = C.cbEncrypt rcvNtfDhSecret ntfNonce (smpEncode msgMeta) 128
pure $ MsgNtf {ntfMsgId = msgId, ntfTs = msgTs, ntfNonce, ntfEncMeta = fromRight "" encNMsgMeta}
processForwardedCommand :: EncFwdTransmission -> M s BrokerMsg
processForwardedCommand (EncFwdTransmission s) = fmap (either ERR RRES) . runExceptT $ do
processForwardedCommand :: EncFwdTransmission -> M s (Maybe BrokerMsg)
processForwardedCommand (EncFwdTransmission s) = fmap (either (Just . ERR) id) . runExceptT $ do
THAuthServer {serverPrivKey, sessSecret'} <- maybe (throwE $ transportErr TENoServerAuth) pure (thAuth thParams')
sessSecret <- maybe (throwE $ transportErr TENoServerAuth) pure sessSecret'
let proxyNonce = C.cbNonce $ bs corrId
@@ -2099,28 +2136,31 @@ client
t :| [] -> pure $ tDecodeServer clntTHParams t
_ -> throwE BLOCK
let clntThAuth = Just $ THAuthServer {serverPrivKey, peerClientService = Nothing, sessSecret' = Just clientSecret}
-- process forwarded command
r <-
lift (rejectOrVerify clntThAuth t') >>= \case
Left r -> pure r
-- rejectOrVerify filters allowed commands, no need to repeat it here.
-- INTERNAL is used because processCommand never returns Nothing for sender commands (could be extracted for better types).
-- `fst` removes empty message that is only returned for `SUB` command
Right t''@(_, (corrId', entId', _)) -> maybe (corrId', entId', ERR INTERNAL) fst <$> lift (processCommand Nothing fwdVersion (Right (M.empty, M.empty, M.empty)) t'')
-- encode response
r' <- case batchTransmissions clntTHParams [Right (Nothing, encodeTransmission clntTHParams r)] of
[] -> throwE INTERNAL -- at least 1 item is guaranteed from NonEmpty/Right
TBError _ _ : _ -> throwE BLOCK
TBTransmission b' _ : _ -> pure b'
TBTransmissions b' _ _ : _ -> pure b'
-- encrypt to client
r2 <- liftEitherWith (const BLOCK) $ EncResponse <$> C.cbEncrypt clientSecret (C.reverseNonce clientNonce) r' paddedProxiedTLength
-- encrypt to proxy
let fr = FwdResponse {fwdCorrId, fwdResponse = r2}
r3 = EncFwdResponse $ C.cbEncryptNoPad sessSecret (C.reverseNonce proxyNonce) (smpEncode fr)
encodeResp r = do
r' <- case batchTransmissions clntTHParams [Right (Nothing, encodeTransmission clntTHParams r)] of
[] -> throwE INTERNAL -- at least 1 item is guaranteed from NonEmpty/Right
TBError _ _ : _ -> throwE BLOCK
TBTransmission b' _ : _ -> pure b'
TBTransmissions b' _ _ : _ -> pure b'
r2 <- liftEitherWith (const BLOCK) $ EncResponse <$> C.cbEncrypt clientSecret (C.reverseNonce clientNonce) r' paddedProxiedTLength
let fr = FwdResponse {fwdCorrId, fwdResponse = r2}
pure $ RRES $ EncFwdResponse $ C.cbEncryptNoPad sessSecret (C.reverseNonce proxyNonce) (smpEncode fr)
-- the inner response, or Nothing if forked (RSLV).
r_ <- lift (rejectOrVerify clntThAuth t') >>= \case
-- rejectOrVerify filters allowed commands, no need to repeat it here.
Left r -> pure $ Just r
Right t''@(_, (corrId', entId', cmd')) -> case cmd' of
Cmd SResolver (RSLV d) -> lift $ rslvNamesEnv >>= \case
Nothing -> pure $ Just (corrId', entId', ERR (NAME NO_RESOLVER))
Just nenv -> forkCmd serverResolverConcurrency corrId NoEntity $ do
msg <- resolveNameMsg nenv d
either ERR id <$> runExceptT (encodeResp (corrId', entId', msg))
-- INTERNAL because processCommand never returns Nothing for sender commands;
-- `fst` drops the empty message only returned for SUB.
_ -> Just . maybe (corrId', entId', ERR INTERNAL) fst <$> lift (processCommand Nothing fwdVersion (Right (M.empty, M.empty, M.empty)) t'')
stats <- asks serverStats
incStat $ pMsgFwdsRecv stats
pure r3
traverse encodeResp r_
where
rejectOrVerify :: Maybe (THandleAuth 'TServer) -> SignedTransmissionOrError ErrorType Cmd -> M s (VerifiedTransmissionOrError s)
rejectOrVerify clntThAuth = \case
@@ -2134,6 +2174,7 @@ client
Cmd SSender (SKEY _) -> True
Cmd SSenderLink (LKEY _) -> True
Cmd SSenderLink LGET -> True
Cmd SResolver (RSLV _) -> True
_ -> False
verified = \case
VRVerified q -> Right (q, t'')
@@ -2468,4 +2509,4 @@ restoreServerStats msgStats_ ntfStats = asks (serverStatsBackupFile . config) >>
logNote $ "error restoring server stats: " <> T.pack e
liftIO exitFailure
compareCounts name statsCnt storeCnt =
when (statsCnt /= storeCnt) $ logWarn $ name <> " count differs: stats: " <> tshow statsCnt <> ", store: " <> tshow storeCnt
when (statsCnt /= storeCnt) $ logWarn $ name <> " count differs: stats: " <> tshow statsCnt <> ", store: " <> tshow storeCnt
+28 -4
View File
@@ -67,6 +67,7 @@ module Simplex.Messaging.Server.Env.STM
defaultNtfExpiration,
defaultInactiveClientExpiration,
defaultProxyClientConcurrency,
defaultNameResolverConcurrency,
defaultMaxJournalMsgCount,
defaultMaxJournalStateLines,
defaultIdleQueueInterval,
@@ -115,6 +116,7 @@ import Simplex.Messaging.Server.Information
import Simplex.Messaging.Server.MsgStore.Journal
import Simplex.Messaging.Server.MsgStore.STM
import Simplex.Messaging.Server.MsgStore.Types
import Simplex.Messaging.Server.Names (NamesConfig (..), NamesEnv, newNamesEnv, pingEndpoint)
import Simplex.Messaging.Server.NtfStore
import Simplex.Messaging.Server.QueueStore
import Simplex.Messaging.Server.QueueStore.Postgres.Config
@@ -128,7 +130,7 @@ import Simplex.Messaging.TMap (TMap)
import qualified Simplex.Messaging.TMap as TM
import Simplex.Messaging.Transport (ASrvTransport, SMPVersion, THandleParams, TransportPeer (..), VersionRangeSMP)
import Simplex.Messaging.Transport.Server
import Simplex.Messaging.Util (ifM, whenM, ($>>=))
import Simplex.Messaging.Util (ifM, tshow, whenM, ($>>=))
import System.Directory (doesFileExist)
import System.Exit (exitFailure)
import System.IO (IOMode (..))
@@ -197,6 +199,13 @@ data ServerConfig s = ServerConfig
smpAgentCfg :: SMPClientAgentConfig,
allowSMPProxy :: Bool, -- auth is the same with `newQueueBasicAuth`
serverClientConcurrency :: Int,
-- | max concurrent name resolutions per connection, enforced in forkCmd.
-- Much higher than serverClientConcurrency: forwarded RSLVs from many clients
-- aggregate over a single proxy->relay connection (only servers send proxied
-- requests), so bounding them by the per-client limit would throttle unduly.
serverResolverConcurrency :: Int,
-- | public-namespace resolver config; Nothing disables the names role
namesConfig :: Maybe NamesConfig,
-- | server public information
information :: Maybe ServerPublicInfo,
startOptions :: StartOptions
@@ -243,6 +252,9 @@ defaultInactiveClientExpiration =
defaultProxyClientConcurrency :: Int
defaultProxyClientConcurrency = 32
defaultNameResolverConcurrency :: Int
defaultNameResolverConcurrency = 1000
journalMsgStoreDepth :: Int
journalMsgStoreDepth = 5
@@ -272,7 +284,8 @@ data Env s = Env
serverStats :: ServerStats,
sockets :: TVar [(ServiceName, SocketState)],
clientSeq :: TVar ClientId,
proxyAgent :: ProxyAgent -- senders served on this proxy
proxyAgent :: ProxyAgent, -- senders served on this proxy
namesEnv :: Maybe NamesEnv -- public-namespace resolver, present when [NAMES] enable: on
}
msgStore :: Env s -> s
@@ -558,7 +571,7 @@ newProhibitedSub = do
return Sub {subThread = ProhibitSub, delivered}
newEnv :: ServerConfig s -> IO (Env s)
newEnv config@ServerConfig {smpCredentials, httpCredentials, serverStoreCfg, smpAgentCfg, information, messageExpiration, idleQueueInterval, msgQueueQuota, maxJournalMsgCount, maxJournalStateLines} = do
newEnv config@ServerConfig {smpCredentials, httpCredentials, serverStoreCfg, smpAgentCfg, information, messageExpiration, idleQueueInterval, msgQueueQuota, maxJournalMsgCount, maxJournalStateLines, namesConfig} = do
serverActive <- newTVarIO True
server <- newServer
msgStore_ <- case serverStoreCfg of
@@ -603,6 +616,16 @@ newEnv config@ServerConfig {smpCredentials, httpCredentials, serverStoreCfg, smp
sockets <- newTVarIO []
clientSeq <- newTVarIO 0
proxyAgent <- newSMPProxyAgent smpAgentCfg random
namesEnv <- forM namesConfig $ \nc -> do
logInfo $ "[NAMES] resolver enabled, endpoint=" <> T.pack (resolverEndpoint nc)
env <- newNamesEnv nc
-- Probe the endpoint at startup. Don't exitFailure: a flapping network or a
-- resolver host coming up minutes after smp-server should not block the
-- server. Log so operators can spot it.
pingEndpoint env >>= \case
Right _ -> logInfo "[NAMES] endpoint probe ok"
Left e -> logWarn $ "[NAMES] endpoint probe failed (server will still start, RSLV will return ERR (NAME ...) until reachable): " <> tshow e
pure env
pure
Env
{ serverActive,
@@ -618,7 +641,8 @@ newEnv config@ServerConfig {smpCredentials, httpCredentials, serverStoreCfg, smp
serverStats,
sockets,
clientSeq,
proxyAgent
proxyAgent,
namesEnv
}
where
loadStoreLog :: StoreQueueClass q => (RecipientId -> QueueRec -> IO q) -> FilePath -> STMQueueStore q -> IO ()
+65 -1
View File
@@ -34,6 +34,7 @@ module Simplex.Messaging.Server.Main
simplexmqSource,
serverPublicInfo,
validCountryValue,
validateUrl,
printSourceCode,
cliCommandP,
strParse,
@@ -50,7 +51,7 @@ import Data.Char (isAlpha, isAscii, toUpper)
import Data.Either (fromRight)
import Data.Functor (($>))
import Data.Ini (Ini, lookupValue, readIniFile)
import Data.List (find, isPrefixOf)
import Data.List (dropWhileEnd, find, isPrefixOf)
import qualified Data.List.NonEmpty as L
import Data.Maybe (fromMaybe, isJust, isNothing)
import Data.Text (Text)
@@ -66,6 +67,7 @@ import Simplex.Messaging.Client.Agent (SMPClientAgentConfig (..), defaultSMPClie
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Encoding.String
import Simplex.Messaging.Parsers (parseAll)
import Network.URI (URI (..), URIAuth (..), parseAbsoluteURI)
import Simplex.Messaging.Protocol (BasicAuth (..), ProtoServerWithAuth (ProtoServerWithAuth), pattern SMPServer)
import Simplex.Messaging.Server (AttachHTTP, exportMessages, importMessages, printMessageStats, runSMPServer)
import Simplex.Messaging.Server.CLI
@@ -76,6 +78,7 @@ import Simplex.Messaging.Server.Main.Init
import Simplex.Messaging.Server.Web (EmbeddedWebParams (..), WebHttpsParams (..))
import Simplex.Messaging.Server.MsgStore.Journal (JournalMsgStore (..), QStoreCfg (..), stmQueueStore)
import Simplex.Messaging.Server.MsgStore.Types (MsgStoreClass (..), SQSType (..), SMSType (..), newMsgStore)
import Simplex.Messaging.Server.Names (NamesConfig (..), RpcAuth (..))
import Simplex.Messaging.Server.QueueStore.Postgres.Config
import Simplex.Messaging.Server.StoreLog.ReadWrite (readQueueStore)
import Simplex.Messaging.Transport (supportedProxyClientSMPRelayVRange, alpnSupportedSMPHandshakes, supportedServerSMPRelayVRange)
@@ -601,10 +604,13 @@ smpServerCLI_ generateSite serveStaticFiles attachStaticFiles cfgPath logPath =
}
},
ownServerDomains = either (const []) textToOwnServers $ lookupValue "PROXY" "own_server_domains" ini,
msgQSize = Nothing, -- to prevent accumulation of late responses, and deadlocks in SMP proxy
persistErrorInterval = 30 -- seconds
},
allowSMPProxy = True,
serverClientConcurrency = readIniDefault defaultProxyClientConcurrency "PROXY" "client_concurrency" ini,
serverResolverConcurrency = readIniDefault defaultNameResolverConcurrency "NAMES" "resolver_concurrency" ini,
namesConfig = readNamesConfig ini,
information = serverPublicInfo ini,
startOptions
}
@@ -796,6 +802,64 @@ validCountryValue field s
| length s == 2 && all (\c -> isAscii c && isAlpha c) s = Right $ T.pack $ map toUpper s
| otherwise = Left $ "Use ISO3166 2-letter code for " <> field
readNamesConfig :: Ini -> Maybe NamesConfig
readNamesConfig ini
| not enabled = Nothing
| otherwise =
let resolverAuth_ = either (error . ("[NAMES] resolver_auth: " <>)) Just . parseRpcAuth =<< eitherToMaybe (lookupValue "NAMES" "resolver_auth" ini)
endpoint = requiredText "resolver_endpoint"
in Just
NamesConfig
{ resolverEndpoint = either (error . ("[NAMES] resolver_endpoint: " <>)) id (validateUrl endpoint resolverAuth_),
resolverAuth = resolverAuth_,
resolverTimeoutMs = boundedIniInt 3000 100 60000 "resolver_timeout_ms",
resolverMaxResponseBytes = boundedIniInt 16000 1024 16000 "resolver_max_response_bytes"
}
where
enabled = fromMaybe False (iniOnOff "NAMES" "enable" ini)
requiredText key =
either (error . (("[NAMES] " <> T.unpack key <> " is required: ") <>)) id $
lookupValue "NAMES" key ini
boundedIniInt def floor_ ceiling_ key = case lookupValue "NAMES" key ini of
Left _ -> def
Right raw -> case readMaybe (T.unpack (T.strip raw)) of
Nothing ->
error $ "[NAMES] " <> T.unpack key <> ": not an integer (got " <> show raw <> ")"
Just n
| n >= floor_ && n <= ceiling_ -> n
| otherwise ->
error $ "[NAMES] " <> T.unpack key <> " must be in [" <> show floor_ <> ".." <> show ceiling_ <> "] (got " <> show n <> ")"
-- | Validate the resolver_endpoint URL: it must be an absolute http(s) URL with a host.
-- http + resolver_auth to a non-loopback host is rejected.
validateUrl :: Text -> Maybe RpcAuth -> Either String String
validateUrl url auth_ = do
let s = T.unpack url
uri <- maybe (Left "not an absolute URI") Right $ parseAbsoluteURI s
let scheme = uriScheme uri
unless (scheme == "http:" || scheme == "https:") $ Left "scheme must be http or https"
ua <- maybe (Left "missing host") Right (uriAuthority uri)
let host = uriRegName ua
when (null host) $ Left "empty host"
unless (null (uriUserInfo ua)) $ Left "userinfo (user:pass@) not allowed; put credentials in resolver_auth"
when (scheme == "http:" && isJust auth_ && not (isLoopback host)) $
Left "http with resolver_auth on a non-loopback host not allowed (the Authorization header would travel in cleartext); use https, or drop resolver_auth"
Right (dropWhileEnd (== '/') s)
where
isLoopback h = h == "localhost" || h == "127.0.0.1" || h == "[::1]" || h == "0.0.0.0"
-- | Parse an rpc_auth INI value. Scheme keyword is case-insensitive so
-- "Bearer <token>" / "BEARER <token>" (Caddy / RFC 7235 convention) work
-- as well as the lowercase form.
parseRpcAuth :: Text -> Either String RpcAuth
parseRpcAuth t = case T.words t of
[scheme, tok] | T.toLower scheme == "bearer" -> Right $ AuthBearer tok
[scheme, up] | T.toLower scheme == "basic" -> case T.breakOn ":" up of
(u, rest)
| not (T.null u) && ":" `T.isPrefixOf` rest -> Right $ AuthBasic u (T.drop 1 rest)
_ -> Left "basic auth expects user:password"
_ -> Left "expected `bearer <token>` or `basic <user>:<pass>`"
printSourceCode :: Maybe Text -> IO ()
printSourceCode = \case
Just sourceCode -> T.putStrLn $ "Server source code: " <> sourceCode
+16
View File
@@ -154,6 +154,22 @@ iniFileContent cfgPath logPath opts host basicAuth controlPortPwds =
\# socks_mode = onion\n\n\
\# Limit number of threads a client can spawn to process proxy commands in parrallel.\n"
<> ("# client_concurrency = " <> tshow defaultProxyClientConcurrency)
<> "\n\n\
\[NAMES]\n\
\# Public-namespace resolution via the snrc-resolve.py REST resolver.\n\
\# Operator runs the resolver alongside smp-server (default port 8000)\n\
\# with its own Ethereum JSON-RPC endpoint configured in resolver.toml.\n\
\enable: off\n\
\# Same-host:\n\
\# resolver_endpoint: http://127.0.0.1:8000\n\
\# Resolver behind TLS reverse proxy:\n\
\# resolver_endpoint: https://names.simplex.chat:443\n\
\# resolver_auth: basic <username>:<password>\n\
\# resolver_timeout_ms: 3000\n\
\# resolver_max_response_bytes: 16000\n\
\# Max concurrent name resolutions per connection (forwarded RSLVs from many\n\
\# clients share one proxy connection, so this is much higher than PROXY client_concurrency).\n"
<> ("# resolver_concurrency = " <> tshow defaultNameResolverConcurrency)
<> "\n\n\
\[INACTIVE_CLIENTS]\n\
\# TTL and interval to check inactive clients\n\
+84
View File
@@ -0,0 +1,84 @@
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE StrictData #-}
module Simplex.Messaging.Server.Names
( NamesConfig (..),
RpcAuth (..),
NamesEnv (..),
newNamesEnv,
closeNamesEnv,
pingEndpoint,
resolveName,
)
where
import qualified Control.Exception as E
import Control.Logger.Simple (logError)
import Data.Bifunctor (first)
import Data.Maybe (fromMaybe)
import qualified Data.Text as T
import Simplex.Messaging.Protocol (NameErrorType (..), NameRecord)
import Simplex.Messaging.Server.Names.HttpResolver
( ResolverEnv,
ResolverError (..),
RpcAuth (..),
closeResolverEnv,
healthHttp,
newResolverEnv,
resolveHttp,
)
import Simplex.Messaging.SimplexName (SimplexDomain, fullDomainName)
import System.Timeout (timeout)
data NamesConfig = NamesConfig
{ resolverEndpoint :: String,
resolverAuth :: Maybe RpcAuth,
resolverTimeoutMs :: Int,
resolverMaxResponseBytes :: Int
}
deriving (Show)
data NamesEnv = NamesEnv
{ config :: NamesConfig,
resolverEnv :: ResolverEnv
}
newNamesEnv :: NamesConfig -> IO NamesEnv
newNamesEnv config = do
resolverEnv <- newResolverEnv (resolverEndpoint config) (resolverAuth config) (resolverTimeoutMs config) (resolverMaxResponseBytes config)
pure NamesEnv {config, resolverEnv}
closeNamesEnv :: NamesEnv -> IO ()
closeNamesEnv NamesEnv {resolverEnv} = closeResolverEnv resolverEnv
pingEndpoint :: NamesEnv -> IO (Either ResolverError ())
pingEndpoint NamesEnv {resolverEnv, config} =
fromMaybe (Left ResolverTimeout) <$> timeout (resolverTimeoutMs config * 1000) (healthHttp resolverEnv)
resolveName :: NamesEnv -> SimplexDomain -> IO (Either NameErrorType NameRecord)
resolveName env d = do
r <- E.try (timeout (resolverTimeoutMs (config env) * 1000) (fetch env d))
case r of
Right result -> pure (fromMaybe (Left (RESOLVER "timeout")) result)
Left e
| Just (_ :: E.SomeAsyncException) <- E.fromException e -> E.throwIO e
| otherwise -> do
logError $ "[NAMES] resolver fetch raised " <> T.pack (E.displayException e)
pure (Left (RESOLVER "resolver error"))
fetch :: NamesEnv -> SimplexDomain -> IO (Either NameErrorType NameRecord)
fetch NamesEnv {resolverEnv} d =
first mapResolverError <$> resolveHttp resolverEnv (fullDomainName d)
mapResolverError :: ResolverError -> NameErrorType
mapResolverError = \case
HttpStatusErr 404 -> NOT_FOUND
HttpStatusErr 400 -> NOT_FOUND
HttpStatusErr code -> RESOLVER ("HTTP " <> T.pack (show code))
HttpFailure _ -> RESOLVER "transport failure"
BodyTooLarge -> RESOLVER "response too large"
InvalidJson _ -> RESOLVER "invalid response"
ResolverTimeout -> RESOLVER "timeout"
@@ -0,0 +1,144 @@
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE StrictData #-}
-- | HTTP transport for the public-namespace resolver.
--
-- The Python REST resolver (see scripts/resolver/snrc-resolve.py) exposes
--
-- GET /resolve/<name> -> 200 with a NameRecord JSON document
-- 404 / 400 for unknown names / TLDs
-- 502 for upstream RPC failures
-- GET /health -> 200 when the resolver process is ready
--
-- Boundary properties:
-- * Response body read with `brReadSome maxResponseBytes` — adversarial
-- endpoints cannot exhaust memory with multi-GB bodies.
-- * `redirectCount = 0` — a compromised resolver cannot bounce credentials
-- to a private-IP target (SSRF amplification on top of the URL validation
-- performed at config load in Server.Main.validateUrl).
-- * Authorization header attached only when configured.
module Simplex.Messaging.Server.Names.HttpResolver
( RpcAuth (..),
ResolverEnv,
ResolverError (..),
newResolverEnv,
closeResolverEnv,
resolveHttp,
healthHttp,
)
where
import qualified Control.Exception as E
import qualified Data.Aeson as J
import Data.Bifunctor (first)
import qualified Data.ByteArray.Encoding as BAE
import Data.ByteString.Char8 (ByteString)
import qualified Data.ByteString.Char8 as B
import qualified Data.ByteString.Lazy as BL
import Data.Text (Text)
import Data.Text.Encoding (encodeUtf8)
import Network.HTTP.Client
( HttpException,
Manager,
ManagerSettings (..),
brReadSome,
parseRequest,
redirectCount,
requestHeaders,
responseBody,
responseStatus,
responseTimeoutMicro,
withResponse,
)
import qualified Network.HTTP.Client as HC
import Network.HTTP.Client.TLS (tlsManagerSettings)
import qualified Network.HTTP.Types as HT
import Network.HTTP.Types.URI (urlEncode)
import Simplex.Messaging.Names.Record (NameRecord)
data RpcAuth = AuthBearer Text | AuthBasic Text Text
-- | Redacts the bearer token / basic-auth password so an accidental
-- `show` / `tshow` on NamesConfig never lands secrets in logs.
instance Show RpcAuth where
show (AuthBearer _) = "AuthBearer <redacted>"
show (AuthBasic u _) = "AuthBasic " <> show u <> " <redacted>"
data ResolverEnv = ResolverEnv
{ manager :: Manager,
baseUrl :: String,
authHdr :: [HT.Header],
timeoutMicro :: Int,
maxResponseBytes :: Int
}
data ResolverError
= HttpFailure HttpException
| HttpStatusErr Int
| BodyTooLarge
| InvalidJson String
| ResolverTimeout
deriving (Show)
newResolverEnv :: String -> Maybe RpcAuth -> Int -> Int -> IO ResolverEnv
newResolverEnv baseUrl auth_ timeoutMs maxResponseBytes = do
manager <- HC.newManager tlsManagerSettings {managerConnCount = 10}
pure
ResolverEnv
{ manager,
baseUrl,
authHdr = maybe [] (pure . authHeader) auth_,
timeoutMicro = timeoutMs * 1000,
maxResponseBytes
}
-- | http-client's `closeManager` is a deprecated no-op since 0.5; the
-- manager is released by the GC finalizer on its internal state. Hook kept
-- as a future-cleanup seam.
closeResolverEnv :: ResolverEnv -> IO ()
closeResolverEnv _ = pure ()
authHeader :: RpcAuth -> HT.Header
authHeader = \case
AuthBearer tok -> ("Authorization", "Bearer " <> encodeUtf8 tok)
AuthBasic u p ->
let encoded = BAE.convertToBase BAE.Base64 (encodeUtf8 u <> ":" <> encodeUtf8 p) :: ByteString
in ("Authorization", "Basic " <> encoded)
-- | GET <baseUrl>/resolve/<percent-encoded name>, decoding the 200 body
-- directly into a NameRecord in one pass (no intermediate Aeson Value). The
-- name is percent-encoded (every non-unreserved byte per RFC 3986): the
-- resolver expects raw labels, so slashes/punctuation must not alter the path.
resolveHttp :: ResolverEnv -> Text -> IO (Either ResolverError NameRecord)
resolveHttp env name =
(>>= first InvalidJson . J.eitherDecodeStrict . BL.toStrict)
<$> httpGet env ("/resolve/" <> B.unpack (urlEncode True (encodeUtf8 name)))
-- | GET <baseUrl>/health; success = reachable with status < 400. The body is
-- size-capped but NOT decoded — the probe only checks reachability.
healthHttp :: ResolverEnv -> IO (Either ResolverError ())
healthHttp env = (() <$) <$> httpGet env "/health"
-- | GET <baseUrl><path>, returning the response body bytes on status < 400
-- within the size cap. Redirects are disabled and Authorization is attached
-- only when configured.
httpGet :: ResolverEnv -> String -> IO (Either ResolverError BL.ByteString)
httpGet ResolverEnv {manager, baseUrl, authHdr, timeoutMicro, maxResponseBytes} path = do
req0 <- parseRequest (baseUrl <> path)
let req =
req0
{ redirectCount = 0,
requestHeaders = ("Accept", "application/json") : authHdr,
HC.responseTimeout = responseTimeoutMicro timeoutMicro
}
result <- E.try $ withResponse req manager $ \res -> do
let status = HT.statusCode (responseStatus res)
if status >= 400
then pure (Left (HttpStatusErr status))
else do
bs <- brReadSome (responseBody res) (maxResponseBytes + 1)
pure $ if BL.length bs > fromIntegral maxResponseBytes then Left BodyTooLarge else Right bs
pure (either (Left . HttpFailure) id result)
+28 -2
View File
@@ -59,7 +59,7 @@ data RTSubscriberMetrics = RTSubscriberMetrics
{-# FOURMOLU_DISABLE\n#-}
prometheusMetrics :: ServerMetrics -> RealTimeMetrics -> UTCTime -> Text
prometheusMetrics sm rtm ts =
time <> queues <> subscriptions <> messages <> ntfMessages <> ntfs <> relays <> services <> info
time <> queues <> subscriptions <> messages <> ntfMessages <> ntfs <> relays <> services <> names <> info
where
ServerMetrics {statsData, activeQueueCounts = ps, activeNtfCounts = psNtf, entityCounts, rtsOptions} = sm
RealTimeMetrics
@@ -128,7 +128,8 @@ prometheusMetrics sm rtm ts =
_rcvServicesSubDuplicate,
_qCount,
_msgCount,
_ntfCount
_ntfCount,
_rslvStats
} = statsData
time =
"# Recorded at: " <> T.pack (iso8601Show ts) <> "\n\
@@ -459,6 +460,31 @@ prometheusMetrics sm rtm ts =
\# TYPE simplex_smp_" <> pfx <> "_services_sub_fewer_total gauge\n\
\simplex_smp_" <> pfx <> "_services_sub_fewer_total " <> mshow (_srvSubFewerTotal ss) <> "\n# " <> pfx <> ".srvSubFewerTotal\n\
\\n"
names =
let NameResolverStatsData {_rslvReqs, _rslvSucc, _rslvNotFound, _rslvResolverErrs, _rslvDisabled} = _rslvStats
in "# Names\n\
\# -----\n\
\\n\
\# HELP simplex_smp_names_reqs Total RSLV requests forwarded to this server.\n\
\# TYPE simplex_smp_names_reqs counter\n\
\simplex_smp_names_reqs " <> mshow _rslvReqs <> "\n# rslvReqs\n\
\\n\
\# HELP simplex_smp_names_success NameRecord successfully resolved and returned.\n\
\# TYPE simplex_smp_names_success counter\n\
\simplex_smp_names_success " <> mshow _rslvSucc <> "\n# rslvSucc\n\
\\n\
\# HELP simplex_smp_names_not_found Name not registered (resolver returned 404 / 400).\n\
\# TYPE simplex_smp_names_not_found counter\n\
\simplex_smp_names_not_found " <> mshow _rslvNotFound <> "\n# rslvNotFound\n\
\\n\
\# HELP simplex_smp_names_resolver_errs Resolver backend errors (HTTP 5xx, transport, decode, or timeout).\n\
\# TYPE simplex_smp_names_resolver_errs counter\n\
\simplex_smp_names_resolver_errs " <> mshow _rslvResolverErrs <> "\n# rslvResolverErrs\n\
\\n\
\# HELP simplex_smp_names_disabled RSLV requests rejected because no resolver is configured (names role off).\n\
\# TYPE simplex_smp_names_disabled counter\n\
\simplex_smp_names_disabled " <> mshow _rslvDisabled <> "\n# rslvDisabled\n\
\\n"
info =
"# Info\n\
\# ----\n\
@@ -178,10 +178,13 @@ instance StoreQueueClass q => QueueStoreClass q (STMQueueStore q) where
_ -> pure $ Left AUTH
where
addLink = do
let !q' = q {queueData = Just (lnkId, d)}
writeTVar qr $ Just q'
TM.insert lnkId rId $ links st
pure $ Right ()
TM.lookup lnkId (links st) >>= \case
Just rId' | rId' /= rId -> pure $ Left AUTH
_ -> do
let !q' = q {queueData = Just (lnkId, d)}
writeTVar qr $ Just q'
TM.insert lnkId rId $ links st
pure $ Right ()
deleteQueueLinkData :: STMQueueStore q -> q -> IO (Either ErrorType ())
deleteQueueLinkData st sq =
+110 -6
View File
@@ -39,6 +39,13 @@ module Simplex.Messaging.Server.Stats
setServiceStats,
emptyTimeBuckets,
updateTimeBuckets,
NameResolverStats (..),
NameResolverStatsData (..),
newNameResolverStats,
newNameResolverStatsData,
getNameResolverStatsData,
getResetNameResolverStatsData,
setNameResolverStats,
) where
import Control.Applicative (optional, (<|>))
@@ -123,7 +130,8 @@ data ServerStats = ServerStats
rcvServicesSubDuplicate :: IORef Int,
qCount :: IORef Int,
msgCount :: IORef Int,
ntfCount :: IORef Int
ntfCount :: IORef Int,
rslvStats :: NameResolverStats
}
data ServerStatsData = ServerStatsData
@@ -184,7 +192,8 @@ data ServerStatsData = ServerStatsData
_rcvServicesSubDuplicate :: Int,
_qCount :: Int,
_msgCount :: Int,
_ntfCount :: Int
_ntfCount :: Int,
_rslvStats :: NameResolverStatsData
}
deriving (Show)
@@ -248,6 +257,7 @@ newServerStats ts = do
qCount <- newIORef 0
msgCount <- newIORef 0
ntfCount <- newIORef 0
rslvStats <- newNameResolverStats
pure
ServerStats
{ fromTime,
@@ -307,7 +317,8 @@ newServerStats ts = do
rcvServicesSubDuplicate,
qCount,
msgCount,
ntfCount
ntfCount,
rslvStats
}
getServerStatsData :: ServerStats -> IO ServerStatsData
@@ -370,6 +381,7 @@ getServerStatsData s = do
_qCount <- readIORef $ qCount s
_msgCount <- readIORef $ msgCount s
_ntfCount <- readIORef $ ntfCount s
_rslvStats <- getNameResolverStatsData $ rslvStats s
pure
ServerStatsData
{ _fromTime,
@@ -429,7 +441,8 @@ getServerStatsData s = do
_rcvServicesSubDuplicate,
_qCount,
_msgCount,
_ntfCount
_ntfCount,
_rslvStats
}
-- this function is not thread safe, it is used on server start only
@@ -493,6 +506,7 @@ setServerStats s d = do
writeIORef (qCount s) $! _qCount d
writeIORef (msgCount s) $! _msgCount d
writeIORef (ntfCount s) $! _ntfCount d
setNameResolverStats (rslvStats s) $! _rslvStats d
instance StrEncoding ServerStatsData where
strEncode d =
@@ -557,7 +571,9 @@ instance StrEncoding ServerStatsData where
"rcvServices:",
strEncode (_rcvServices d),
"ntfServices:",
strEncode (_ntfServices d)
strEncode (_ntfServices d),
"rslvStats:",
strEncode (_rslvStats d)
]
strP = do
_fromTime <- "fromTime=" *> strP <* A.endOfLine
@@ -628,6 +644,10 @@ instance StrEncoding ServerStatsData where
_pMsgFwdsRecv <- opt "pMsgFwdsRecv="
_rcvServices <- serviceStatsP "rcvServices:"
_ntfServices <- serviceStatsP "ntfServices:"
_rslvStats <-
optional ("rslvStats:" <* A.endOfLine) >>= \case
Just _ -> strP <* optional A.endOfLine
_ -> pure newNameResolverStatsData
pure
ServerStatsData
{ _fromTime,
@@ -687,7 +707,8 @@ instance StrEncoding ServerStatsData where
_rcvServicesSubDuplicate = 0,
_qCount,
_msgCount = 0,
_ntfCount = 0
_ntfCount = 0,
_rslvStats
}
where
opt s = A.string s *> strP <* A.endOfLine <|> pure 0
@@ -862,6 +883,89 @@ instance StrEncoding ProxyStatsData where
_pErrorsOther <- "errorsOther=" *> strP
pure ProxyStatsData {_pRequests, _pSuccesses, _pErrorsConnect, _pErrorsCompat, _pErrorsOther}
data NameResolverStats = NameResolverStats
{ rslvReqs :: IORef Int,
rslvSucc :: IORef Int,
rslvNotFound :: IORef Int,
rslvResolverErrs :: IORef Int,
rslvDisabled :: IORef Int
}
newNameResolverStats :: IO NameResolverStats
newNameResolverStats = do
rslvReqs <- newIORef 0
rslvSucc <- newIORef 0
rslvNotFound <- newIORef 0
rslvResolverErrs <- newIORef 0
rslvDisabled <- newIORef 0
pure NameResolverStats {rslvReqs, rslvSucc, rslvNotFound, rslvResolverErrs, rslvDisabled}
data NameResolverStatsData = NameResolverStatsData
{ _rslvReqs :: Int,
_rslvSucc :: Int,
_rslvNotFound :: Int,
_rslvResolverErrs :: Int,
_rslvDisabled :: Int
}
deriving (Show)
newNameResolverStatsData :: NameResolverStatsData
newNameResolverStatsData =
NameResolverStatsData
{ _rslvReqs = 0,
_rslvSucc = 0,
_rslvNotFound = 0,
_rslvResolverErrs = 0,
_rslvDisabled = 0
}
getNameResolverStatsData :: NameResolverStats -> IO NameResolverStatsData
getNameResolverStatsData s = do
_rslvReqs <- readIORef $ rslvReqs s
_rslvSucc <- readIORef $ rslvSucc s
_rslvNotFound <- readIORef $ rslvNotFound s
_rslvResolverErrs <- readIORef $ rslvResolverErrs s
_rslvDisabled <- readIORef $ rslvDisabled s
pure NameResolverStatsData {_rslvReqs, _rslvSucc, _rslvNotFound, _rslvResolverErrs, _rslvDisabled}
getResetNameResolverStatsData :: NameResolverStats -> IO NameResolverStatsData
getResetNameResolverStatsData s = do
_rslvReqs <- atomicSwapIORef (rslvReqs s) 0
_rslvSucc <- atomicSwapIORef (rslvSucc s) 0
_rslvNotFound <- atomicSwapIORef (rslvNotFound s) 0
_rslvResolverErrs <- atomicSwapIORef (rslvResolverErrs s) 0
_rslvDisabled <- atomicSwapIORef (rslvDisabled s) 0
pure NameResolverStatsData {_rslvReqs, _rslvSucc, _rslvNotFound, _rslvResolverErrs, _rslvDisabled}
-- not thread safe; used on server start only
setNameResolverStats :: NameResolverStats -> NameResolverStatsData -> IO ()
setNameResolverStats s d = do
writeIORef (rslvReqs s) $! _rslvReqs d
writeIORef (rslvSucc s) $! _rslvSucc d
writeIORef (rslvNotFound s) $! _rslvNotFound d
writeIORef (rslvResolverErrs s) $! _rslvResolverErrs d
writeIORef (rslvDisabled s) $! _rslvDisabled d
instance StrEncoding NameResolverStatsData where
strEncode NameResolverStatsData {_rslvReqs, _rslvSucc, _rslvNotFound, _rslvResolverErrs, _rslvDisabled} =
"reqs="
<> strEncode _rslvReqs
<> "\nsucc="
<> strEncode _rslvSucc
<> "\nnotFound="
<> strEncode _rslvNotFound
<> "\nresolverErrs="
<> strEncode _rslvResolverErrs
<> "\ndisabled="
<> strEncode _rslvDisabled
strP = do
_rslvReqs <- "reqs=" *> strP <* A.endOfLine
_rslvSucc <- "succ=" *> strP <* A.endOfLine
_rslvNotFound <- "notFound=" *> strP <* A.endOfLine
_rslvResolverErrs <- "resolverErrs=" *> strP <* A.endOfLine
_rslvDisabled <- "disabled=" *> strP
pure NameResolverStatsData {_rslvReqs, _rslvSucc, _rslvNotFound, _rslvResolverErrs, _rslvDisabled}
data ServiceStats = ServiceStats
{ srvAssocNew :: IORef Int,
srvAssocDuplicate :: IORef Int,
+11 -5
View File
@@ -11,8 +11,9 @@ import Control.Applicative ((<|>))
import qualified Data.Attoparsec.ByteString.Char8 as A
import qualified Data.ByteString.Char8 as B
import Data.Functor (($>))
import Network.Socket (HostName, ServiceName)
import Network.Socket (ServiceName)
import Simplex.Messaging.Encoding.String (StrEncoding (..))
import Simplex.Messaging.Transport.Client (TransportHost (..))
data ServiceScheme = SSSimplex | SSAppServer SrvLoc
deriving (Eq, Show)
@@ -25,14 +26,19 @@ instance StrEncoding ServiceScheme where
"simplex:" $> SSSimplex
<|> "https://" *> (SSAppServer <$> strP)
data SrvLoc = SrvLoc HostName ServiceName
data SrvLoc = SrvLoc TransportHost ServiceName
deriving (Eq, Ord, Show)
instance StrEncoding SrvLoc where
strEncode (SrvLoc host port) = B.pack $ host <> if null port then "" else ':' : port
strP = SrvLoc <$> host <*> (port <|> pure "")
strEncode (SrvLoc host port)
| null port = strEncode host
| otherwise = h <> B.pack (':' : port)
where
h = case host of
THIPv6 _ -> ('[' `B.cons` strEncode host) `B.snoc` ']'
_ -> strEncode host
strP = SrvLoc <$> strP <*> (port <|> pure "")
where
host = B.unpack <$> A.takeWhile1 (A.notInClass ":#,;/ ")
port = show <$> (A.char ':' *> (A.decimal :: A.Parser Int))
simplexChat :: ServiceScheme
+33 -1
View File
@@ -6,14 +6,20 @@ module Simplex.Messaging.Session
( SessionVar (..),
getSessVar,
removeSessVar,
withGetSessVar,
withGetSessVar',
tryReadSessVar,
) where
import Control.Concurrent.STM
import Control.Monad.Except (ExceptT (..), runExceptT)
import Control.Monad.IO.Class (liftIO)
import Control.Monad.IO.Unlift (MonadUnliftIO)
import Data.Time (UTCTime)
import Simplex.Messaging.TMap (TMap)
import qualified Simplex.Messaging.TMap as TM
import Simplex.Messaging.Util (($>>=))
import Simplex.Messaging.Util (whenM, ($>>=))
import UnliftIO.Exception (bracketOnError)
data SessionVar a = SessionVar
{ sessionVar :: TMVar a,
@@ -38,5 +44,31 @@ removeSessVar v sessKey vs =
Just v' | sessionVarId v == sessionVarId v' -> TM.delete sessKey vs
_ -> pure ()
-- | Get or create a session var and route to onNew (newly created) or onExisting. The new-var
-- branch is bracketed from the point of creation: if it is interrupted before filling the var
-- (e.g. an async exception during connect), the still-empty var is dropped from the map so the
-- next request creates a fresh session instead of blocking on a var that will never be filled.
-- A thrown ExceptT error is a normal result (the var keeps the error it was filled with) - only
-- an interrupting exception drops the empty var.
withGetSessVar ::
(Ord k, MonadUnliftIO m) =>
TVar Int -> k -> TMap k (SessionVar a) -> UTCTime ->
(SessionVar a -> ExceptT e m b) -> (SessionVar a -> ExceptT e m b) -> ExceptT e m b
withGetSessVar sessSeq sessKey vs ts onNew onExisting =
ExceptT $ withGetSessVar' sessSeq sessKey vs ts (runExceptT . onNew) (runExceptT . onExisting)
-- | withGetSessVar for actions in the underlying monad (without ExceptT).
withGetSessVar' ::
(Ord k, MonadUnliftIO m) =>
TVar Int -> k -> TMap k (SessionVar a) -> UTCTime ->
(SessionVar a -> m b) -> (SessionVar a -> m b) -> m b
withGetSessVar' sessSeq sessKey vs ts onNew onExisting =
bracketOnError
(liftIO $ atomically $ getSessVar sessSeq sessKey vs ts)
(either (liftIO . atomically . dropEmptySessVar) (\_ -> pure ()))
(either onNew onExisting)
where
dropEmptySessVar v = whenM (isEmptyTMVar $ sessionVar v) $ removeSessVar v sessKey vs
tryReadSessVar :: Ord k => k -> TMap k (SessionVar a) -> STM (Maybe a)
tryReadSessVar sessKey vs = TM.lookup sessKey vs $>>= (tryReadTMVar . sessionVar)
+137
View File
@@ -0,0 +1,137 @@
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE StrictData #-}
{-# LANGUAGE TemplateHaskell #-}
module Simplex.Messaging.SimplexName
( SimplexNameInfo (..),
SimplexDomain (..),
SimplexTLD (..),
SimplexNameType (..),
fullDomainName,
shortNameInfoStr,
)
where
import Control.Applicative (optional, (<|>))
import qualified Data.Aeson.TH as J
import qualified Data.Attoparsec.ByteString.Char8 as A
import qualified Data.Attoparsec.Text as AT
import Data.ByteString.Char8 (ByteString)
import qualified Data.ByteString.Char8 as B
import Data.Char (isDigit)
import Data.Functor (($>))
import Data.Text (Text)
import qualified Data.Text as T
import Data.Text.Encoding (decodeLatin1, encodeUtf8)
import Simplex.Messaging.Agent.Store.DB (FromField (..), ToField (..), fromTextField_)
import Simplex.Messaging.Encoding (Encoding (..))
import Simplex.Messaging.Encoding.String
import Simplex.Messaging.Parsers (defaultJSON, dropPrefix, enumJSON)
import Simplex.Messaging.Util (eitherToMaybe, safeDecodeUtf8, (<$?>))
data SimplexNameInfo = SimplexNameInfo
{ nameType :: SimplexNameType,
nameDomain :: SimplexDomain
}
deriving (Eq, Show)
data SimplexDomain = SimplexDomain
{ nameTLD :: SimplexTLD,
domain :: Text,
subDomain :: [Text] -- parent to child: ["b", "a"] for a.b.domain.simplex
}
deriving (Eq, Show)
data SimplexTLD = TLDSimplex | TLDTesting | TLDWeb
deriving (Eq, Show)
data SimplexNameType = NTPublicGroup | NTContact
deriving (Eq, Show)
instance StrEncoding SimplexNameType where
strEncode = \case
NTPublicGroup -> "#"
NTContact -> "@"
strP = A.char '#' $> NTPublicGroup <|> A.char '@' $> NTContact
nameLabelP :: AT.Parser Text
nameLabelP = do
label <- T.intercalate "-" <$> AT.takeWhile1 (\c -> isNameLetter c || isDigit c) `AT.sepBy1` AT.char '-'
-- DNS label limit: each dot-separated component is at most 63 bytes (labels
-- are ASCII, so character count == byte count)
if T.length label > 63 then fail "name label exceeds 63 bytes" else pure label
where
-- ASCII letters only. SNRC contracts hash byte sequences via keccak; ENS
-- uses UTS-46 + Punycode for IDN, which we do not implement. Admitting
-- Cyrillic / Greek / etc. via Data.Char.isAlpha would (a) make namehash
-- diverge from any IDN-aware registrar and (b) allow homograph spoofing
-- (Cyrillic а vs ASCII a hash to different on-chain records).
isNameLetter c = c >= 'a' && c <= 'z' || c >= 'A' && c <= 'Z'
-- | Cap the name at 253 bytes (DNS full-domain limit)
boundedNonSpace :: A.Parser ByteString
boundedNonSpace = do
bs <- A.scan (0 :: Int) $ \i c ->
if i <= 253 && not (A.isSpace c) then Just (i + 1) else Nothing
if B.null bs
then fail "expected non-empty name token"
else if B.length bs > 253 then fail "name exceeds 253 bytes" else pure bs
instance StrEncoding SimplexNameInfo where
strEncode SimplexNameInfo {nameType, nameDomain} =
strEncode nameType <> strEncode nameDomain
strP = optional "simplex:/name" *> ((strP >>= infoP) <|> infoP NTPublicGroup)
where
infoP NTPublicGroup = SimplexNameInfo NTPublicGroup <$> (strP <|> bareName)
infoP NTContact = SimplexNameInfo NTContact <$> strP
bareName = parseBare . safeDecodeUtf8 <$?> boundedNonSpace
parseBare s = (\name -> SimplexDomain TLDSimplex (T.toLower name) []) <$> AT.parseOnly (nameLabelP <* AT.endOfInput) s
instance StrEncoding SimplexDomain where
strEncode = encodeUtf8 . fullDomainName
strP = parseDomain . safeDecodeUtf8 <$?> boundedNonSpace
where
parseDomain s = AT.parseOnly (nameLabelP `AT.sepBy1` AT.char '.' <* AT.endOfInput) s >>= mkDomain
mkDomain labels = case reverse lowered of
[] -> Left "empty name"
[_] -> Left "domain requires TLD"
"simplex" : name : sub -> Right (SimplexDomain TLDSimplex name sub)
"testing" : name : sub -> Right (SimplexDomain TLDTesting name sub)
_ -> Right (SimplexDomain TLDWeb (T.intercalate "." lowered) [])
where
lowered = map T.toLower labels
instance Encoding SimplexDomain where
smpEncode = strEncode
smpP = strP
fullDomainName :: SimplexDomain -> Text
fullDomainName SimplexDomain {nameTLD, domain, subDomain} = T.intercalate "." (reverse subDomain ++ [domain] ++ tld')
where
tld' = case nameTLD of
TLDSimplex -> ["simplex"]
TLDTesting -> ["testing"]
TLDWeb -> []
shortNameInfoStr :: SimplexNameInfo -> Text
shortNameInfoStr = \case
SimplexNameInfo {nameType = NTPublicGroup, nameDomain = SimplexDomain {nameTLD = TLDSimplex, domain, subDomain = []}} -> "#" <> domain
info -> pfx <> fullDomainName (nameDomain info)
where
pfx = case nameType info of
NTPublicGroup -> "#"
NTContact -> "@"
instance ToField SimplexDomain where toField = toField . decodeLatin1 . strEncode
instance FromField SimplexDomain where fromField = fromTextField_ (eitherToMaybe . strDecode . encodeUtf8)
$(J.deriveJSON (enumJSON $ dropPrefix "TLD") ''SimplexTLD)
$(J.deriveJSON (enumJSON $ dropPrefix "NT") ''SimplexNameType)
$(J.deriveJSON defaultJSON ''SimplexDomain)
$(J.deriveJSON defaultJSON ''SimplexNameInfo)
+14 -8
View File
@@ -57,6 +57,7 @@ module Simplex.Messaging.Transport
newNtfCredsSMPVersion,
clientNoticesSMPVersion,
rcvServiceSMPVersion,
namesSMPVersion,
simplexMQVersion,
smpBlockSize,
TransportConfig (..),
@@ -223,6 +224,9 @@ clientNoticesSMPVersion = VersionSMP 18
rcvServiceSMPVersion :: VersionSMP
rcvServiceSMPVersion = VersionSMP 19
namesSMPVersion :: VersionSMP
namesSMPVersion = VersionSMP 20
minClientSMPRelayVersion :: VersionSMP
minClientSMPRelayVersion = VersionSMP 6
@@ -230,21 +234,23 @@ minServerSMPRelayVersion :: VersionSMP
minServerSMPRelayVersion = VersionSMP 6
currentClientSMPRelayVersion :: VersionSMP
currentClientSMPRelayVersion = VersionSMP 19
currentClientSMPRelayVersion = VersionSMP 20
legacyServerSMPRelayVersion :: VersionSMP
legacyServerSMPRelayVersion = VersionSMP 6
currentServerSMPRelayVersion :: VersionSMP
currentServerSMPRelayVersion = VersionSMP 19
currentServerSMPRelayVersion = VersionSMP 20
-- Max SMP protocol version to be used in e2e encrypted
-- connection between client and server, as defined by SMP proxy.
-- SMP proxy sets it to lower than its current version
-- to prevent client version fingerprinting by the
-- destination relays when clients upgrade at different times.
-- Max SMP protocol version to be used in e2e encrypted connection between
-- client and server, as defined by SMP proxy. Normally set below the current
-- version to prevent client version fingerprinting by the destination relays
-- when clients upgrade at different times. Pinned to the current version (20)
-- for this release because proxied name resolution is gated on namesSMPVersion
-- (20), so the one-version anti-fingerprinting buffer does not apply yet; it
-- reappears once the current version advances past 20.
proxiedSMPRelayVersion :: VersionSMP
proxiedSMPRelayVersion = VersionSMP 18
proxiedSMPRelayVersion = VersionSMP 20
-- minimal supported protocol version is 6
-- TODO remove code that supports sending commands without batching
+1 -1
View File
@@ -89,7 +89,7 @@ instance StrEncoding TransportHost where
[ THIPv4 <$> ((,,,) <$> ipNum <*> ipNum <*> ipNum <*> A.decimal),
maybe (Left "bad IPv6") (Right . THIPv6 . fromIPv6w) . readMaybe . B.unpack <$?> ipv6StrP,
THOnionHost <$> ((<>) <$> A.takeWhile (\c -> isAsciiLower c || isDigit c) <*> A.string ".onion"),
THDomainName . B.unpack <$> (notOnion <$?> A.takeWhile1 (A.notInClass ":#,;/ \n\r\t"))
THDomainName . B.unpack <$> (notOnion <$?> A.takeWhile1 (A.notInClass ":#,;/ \n\r\t[]"))
]
where
ipNum = validIP <$?> (A.decimal <* A.char '.')
+1 -1
View File
@@ -81,7 +81,7 @@ instance StrEncoding RCInvitation where
_ <- A.string "xrcp:/"
ca <- strP
_ <- A.char '@'
host <- A.takeWhile (/= ':') >>= either fail pure . strDecode . urlDecode True
host <- strP
_ <- A.char ':'
port <- strP
_ <- A.string "#/?"
+2
View File
@@ -12,6 +12,7 @@ import AgentTests.ConnectionRequestTests
import AgentTests.DoubleRatchetTests (doubleRatchetTests)
import AgentTests.FunctionalAPITests (functionalAPITests)
import AgentTests.MigrationTests (migrationTests)
import AgentTests.ResolveNameTests (resolveNameTests)
import AgentTests.ServerChoice (serverChoiceTests)
import AgentTests.ShortLinkTests (shortLinkTests)
import Simplex.Messaging.Server.Env.STM (AStoreType (..))
@@ -37,6 +38,7 @@ agentCoreTests = do
describe "Connection request" connectionRequestTests
describe "Double ratchet tests" doubleRatchetTests
describe "Short link tests" shortLinkTests
describe "resolve names" resolveNameTests
agentTests :: (ASrvTransport, AStoreType) -> Spec
agentTests ps = do
+44 -44
View File
@@ -381,19 +381,19 @@ testX3dh :: forall a. (AlgorithmI a, DhAlgorithm a) => C.SAlgorithm a -> IO ()
testX3dh _ = do
g <- C.newRandom
let v = max pqRatchetE2EEncryptVersion currentE2EEncryptVersion
(pkBob1, pkBob2, Nothing, AE2ERatchetParams _ e2eBob) <- liftIO $ generateSndE2EParams @a g v Nothing
(pkAlice1, pkAlice2, Nothing, e2eAlice) <- liftIO $ generateRcvE2EParams @a g v PQSupportOff
let paramsBob = pqX3dhSnd pkBob1 pkBob2 Nothing e2eAlice
paramsAlice <- runExceptT $ pqX3dhRcv pkAlice1 pkAlice2 Nothing e2eBob
(pksBob@(_, _, Nothing), AE2ERatchetParams _ e2eBob) <- liftIO $ generateSndE2EParams @a g v Nothing
(pksAlice@(_, _, Nothing), e2eAlice) <- liftIO $ generateRcvE2EParams @a g v PQSupportOff
let paramsBob = pqX3dhSnd pksBob e2eAlice
paramsAlice <- runExceptT $ pqX3dhRcv pksAlice e2eBob
paramsAlice `shouldBe` paramsBob
testX3dhV1 :: forall a. (AlgorithmI a, DhAlgorithm a) => C.SAlgorithm a -> IO ()
testX3dhV1 _ = do
g <- C.newRandom
(pkBob1, pkBob2, Nothing, AE2ERatchetParams _ e2eBob) <- liftIO $ generateSndE2EParams @a g (VersionE2E 1) Nothing
(pkAlice1, pkAlice2, Nothing, e2eAlice) <- liftIO $ generateRcvE2EParams @a g (VersionE2E 1) PQSupportOff
let paramsBob = pqX3dhSnd pkBob1 pkBob2 Nothing e2eAlice
paramsAlice <- runExceptT $ pqX3dhRcv pkAlice1 pkAlice2 Nothing e2eBob
(pksBob@(_, _, Nothing), AE2ERatchetParams _ e2eBob) <- liftIO $ generateSndE2EParams @a g (VersionE2E 1) Nothing
(pksAlice@(_, _, Nothing), e2eAlice) <- liftIO $ generateRcvE2EParams @a g (VersionE2E 1) PQSupportOff
let paramsBob = pqX3dhSnd pksBob e2eAlice
paramsAlice <- runExceptT $ pqX3dhRcv pksAlice e2eBob
paramsAlice `shouldBe` paramsBob
testPqX3dhProposeInReply :: forall a. (AlgorithmI a, DhAlgorithm a) => C.SAlgorithm a -> IO ()
@@ -401,11 +401,11 @@ testPqX3dhProposeInReply _ = do
g <- C.newRandom
let v = max pqRatchetE2EEncryptVersion currentE2EEncryptVersion
-- initiate (no KEM)
(pkAlice1, pkAlice2, Nothing, e2eAlice) <- liftIO $ generateRcvE2EParams @a g v PQSupportOff
(pksAlice@(_, _, Nothing), e2eAlice) <- liftIO $ generateRcvE2EParams @a g v PQSupportOff
-- propose KEM in reply
(pkBob1, pkBob2, pKemBob_@(Just _), AE2ERatchetParams _ e2eBob) <- liftIO $ generateSndE2EParams @a g v (Just $ AUseKEM SRKSProposed ProposeKEM)
Right paramsBob <- pure $ pqX3dhSnd pkBob1 pkBob2 pKemBob_ e2eAlice
Right paramsAlice <- runExceptT $ pqX3dhRcv pkAlice1 pkAlice2 Nothing e2eBob
(pksBob@(_, _, Just _), AE2ERatchetParams _ e2eBob) <- liftIO $ generateSndE2EParams @a g v (Just $ AUseKEM SRKSProposed ProposeKEM)
Right paramsBob <- pure $ pqX3dhSnd pksBob e2eAlice
Right paramsAlice <- runExceptT $ pqX3dhRcv pksAlice e2eBob
paramsAlice `compatibleRatchets` paramsBob
testPqX3dhProposeAccept :: forall a. (AlgorithmI a, DhAlgorithm a) => C.SAlgorithm a -> IO ()
@@ -413,12 +413,12 @@ testPqX3dhProposeAccept _ = do
g <- C.newRandom
let v = max pqRatchetE2EEncryptVersion currentE2EEncryptVersion
-- initiate (propose KEM)
(pkAlice1, pkAlice2, pKemAlice_@(Just _), e2eAlice) <- liftIO $ generateRcvE2EParams @a g v PQSupportOn
(pksAlice@(_, _, Just _), e2eAlice) <- liftIO $ generateRcvE2EParams @a g v PQSupportOn
E2ERatchetParams _ _ _ (Just (RKParamsProposed aliceKem)) <- pure e2eAlice
-- accept KEM
(pkBob1, pkBob2, pKemBob_@(Just _), AE2ERatchetParams _ e2eBob) <- liftIO $ generateSndE2EParams @a g v (Just $ AUseKEM SRKSAccepted $ AcceptKEM aliceKem)
Right paramsBob <- pure $ pqX3dhSnd pkBob1 pkBob2 pKemBob_ e2eAlice
Right paramsAlice <- runExceptT $ pqX3dhRcv pkAlice1 pkAlice2 pKemAlice_ e2eBob
(pksBob@(_, _, Just _), AE2ERatchetParams _ e2eBob) <- liftIO $ generateSndE2EParams @a g v (Just $ AUseKEM SRKSAccepted $ AcceptKEM aliceKem)
Right paramsBob <- pure $ pqX3dhSnd pksBob e2eAlice
Right paramsAlice <- runExceptT $ pqX3dhRcv pksAlice e2eBob
paramsAlice `compatibleRatchets` paramsBob
testPqX3dhProposeReject :: forall a. (AlgorithmI a, DhAlgorithm a) => C.SAlgorithm a -> IO ()
@@ -426,12 +426,12 @@ testPqX3dhProposeReject _ = do
g <- C.newRandom
let v = max pqRatchetE2EEncryptVersion currentE2EEncryptVersion
-- initiate (propose KEM)
(pkAlice1, pkAlice2, pKemAlice_@(Just _), e2eAlice) <- liftIO $ generateRcvE2EParams @a g v PQSupportOn
(pksAlice@(_, _, Just _), e2eAlice) <- liftIO $ generateRcvE2EParams @a g v PQSupportOn
E2ERatchetParams _ _ _ (Just (RKParamsProposed _)) <- pure e2eAlice
-- reject KEM
(pkBob1, pkBob2, Nothing, AE2ERatchetParams _ e2eBob) <- liftIO $ generateSndE2EParams @a g v Nothing
Right paramsBob <- pure $ pqX3dhSnd pkBob1 pkBob2 Nothing e2eAlice
Right paramsAlice <- runExceptT $ pqX3dhRcv pkAlice1 pkAlice2 pKemAlice_ e2eBob
(pksBob@(_, _, Nothing), AE2ERatchetParams _ e2eBob) <- liftIO $ generateSndE2EParams @a g v Nothing
Right paramsBob <- pure $ pqX3dhSnd pksBob e2eAlice
Right paramsAlice <- runExceptT $ pqX3dhRcv pksAlice e2eBob
paramsAlice `compatibleRatchets` paramsBob
testPqX3dhAcceptWithoutProposalError :: forall a. (AlgorithmI a, DhAlgorithm a) => C.SAlgorithm a -> IO ()
@@ -439,26 +439,26 @@ testPqX3dhAcceptWithoutProposalError _ = do
g <- C.newRandom
let v = max pqRatchetE2EEncryptVersion currentE2EEncryptVersion
-- initiate (no KEM)
(pkAlice1, pkAlice2, Nothing, e2eAlice) <- liftIO $ generateRcvE2EParams @a g v PQSupportOff
(pksAlice@(_, _, Nothing), e2eAlice) <- liftIO $ generateRcvE2EParams @a g v PQSupportOff
E2ERatchetParams _ _ _ Nothing <- pure e2eAlice
-- incorrectly accept KEM
-- we don't have key in proposal, so we just generate it
(k, _) <- sntrup761Keypair g
(pkBob1, pkBob2, pKemBob_@(Just _), AE2ERatchetParams _ e2eBob) <- liftIO $ generateSndE2EParams @a g v (Just $ AUseKEM SRKSAccepted $ AcceptKEM k)
pqX3dhSnd pkBob1 pkBob2 pKemBob_ e2eAlice `shouldBe` Left C.CERatchetKEMState
runExceptT (pqX3dhRcv pkAlice1 pkAlice2 Nothing e2eBob) `shouldReturn` Left C.CERatchetKEMState
(pksBob@(_, _, Just _), AE2ERatchetParams _ e2eBob) <- liftIO $ generateSndE2EParams @a g v (Just $ AUseKEM SRKSAccepted $ AcceptKEM k)
pqX3dhSnd pksBob e2eAlice `shouldBe` Left C.CERatchetKEMState
runExceptT (pqX3dhRcv pksAlice e2eBob) `shouldReturn` Left C.CERatchetKEMState
testPqX3dhProposeAgain :: forall a. (AlgorithmI a, DhAlgorithm a) => C.SAlgorithm a -> IO ()
testPqX3dhProposeAgain _ = do
g <- C.newRandom
let v = max pqRatchetE2EEncryptVersion currentE2EEncryptVersion
-- initiate (propose KEM)
(pkAlice1, pkAlice2, pKemAlice_@(Just _), e2eAlice) <- liftIO $ generateRcvE2EParams @a g v PQSupportOn
(pksAlice@(_, _, Just _), e2eAlice) <- liftIO $ generateRcvE2EParams @a g v PQSupportOn
E2ERatchetParams _ _ _ (Just (RKParamsProposed _)) <- pure e2eAlice
-- propose KEM again in reply - this is not an error
(pkBob1, pkBob2, pKemBob_@(Just _), AE2ERatchetParams _ e2eBob) <- liftIO $ generateSndE2EParams @a g v (Just $ AUseKEM SRKSProposed ProposeKEM)
Right paramsBob <- pure $ pqX3dhSnd pkBob1 pkBob2 pKemBob_ e2eAlice
Right paramsAlice <- runExceptT $ pqX3dhRcv pkAlice1 pkAlice2 pKemAlice_ e2eBob
(pksBob@(_, _, Just _), AE2ERatchetParams _ e2eBob) <- liftIO $ generateSndE2EParams @a g v (Just $ AUseKEM SRKSProposed ProposeKEM)
Right paramsBob <- pure $ pqX3dhSnd pksBob e2eAlice
Right paramsAlice <- runExceptT $ pqX3dhRcv pksAlice e2eBob
paramsAlice `compatibleRatchets` paramsBob
compatibleRatchets :: (RatchetInitParams, x) -> (RatchetInitParams, x) -> Expectation
@@ -515,10 +515,10 @@ initRatchets :: (AlgorithmI a, DhAlgorithm a) => IO (Ratchet a, Ratchet a, Encry
initRatchets = do
g <- C.newRandom
let v = max pqRatchetE2EEncryptVersion currentE2EEncryptVersion
(pkBob1, pkBob2, _pKemParams@Nothing, AE2ERatchetParams _ e2eBob) <- liftIO $ generateSndE2EParams g v Nothing
(pkAlice1, pkAlice2, _pKem@Nothing, e2eAlice) <- liftIO $ generateRcvE2EParams g v PQSupportOff
Right paramsBob <- pure $ pqX3dhSnd pkBob1 pkBob2 Nothing e2eAlice
Right paramsAlice <- runExceptT $ pqX3dhRcv pkAlice1 pkAlice2 Nothing e2eBob
(pksBob@(_, _, Nothing), AE2ERatchetParams _ e2eBob) <- liftIO $ generateSndE2EParams g v Nothing
(pksAlice@(_, pkAlice2, Nothing), e2eAlice) <- liftIO $ generateRcvE2EParams g v PQSupportOff
Right paramsBob <- pure $ pqX3dhSnd pksBob e2eAlice
Right paramsAlice <- runExceptT $ pqX3dhRcv pksAlice e2eBob
(_, pkBob3) <- atomically $ C.generateKeyPair g
let vs = testRatchetVersions
bob = initSndRatchet vs (C.publicKey pkAlice2) pkBob3 paramsBob
@@ -530,12 +530,12 @@ initRatchetsKEMProposed = do
g <- C.newRandom
let v = max pqRatchetE2EEncryptVersion currentE2EEncryptVersion
-- initiate (no KEM)
(pkAlice1, pkAlice2, Nothing, e2eAlice) <- liftIO $ generateRcvE2EParams g v PQSupportOff
(pksAlice@(_, pkAlice2, Nothing), e2eAlice) <- liftIO $ generateRcvE2EParams g v PQSupportOff
-- propose KEM in reply
let useKem = AUseKEM SRKSProposed ProposeKEM
(pkBob1, pkBob2, pKemParams_@(Just _), AE2ERatchetParams _ e2eBob) <- liftIO $ generateSndE2EParams g v (Just useKem)
Right paramsBob <- pure $ pqX3dhSnd pkBob1 pkBob2 pKemParams_ e2eAlice
Right paramsAlice <- runExceptT $ pqX3dhRcv pkAlice1 pkAlice2 Nothing e2eBob
(pksBob@(_, _, Just _), AE2ERatchetParams _ e2eBob) <- liftIO $ generateSndE2EParams g v (Just useKem)
Right paramsBob <- pure $ pqX3dhSnd pksBob e2eAlice
Right paramsAlice <- runExceptT $ pqX3dhRcv pksAlice e2eBob
(_, pkBob3) <- atomically $ C.generateKeyPair g
let vs = testRatchetVersions
bob = initSndRatchet vs (C.publicKey pkAlice2) pkBob3 paramsBob
@@ -547,13 +547,13 @@ initRatchetsKEMAccepted = do
g <- C.newRandom
let v = max pqRatchetE2EEncryptVersion currentE2EEncryptVersion
-- initiate (propose)
(pkAlice1, pkAlice2, pKem_@(Just _), e2eAlice) <- liftIO $ generateRcvE2EParams g v PQSupportOn
(pksAlice@(_, pkAlice2, Just _), e2eAlice) <- liftIO $ generateRcvE2EParams g v PQSupportOn
E2ERatchetParams _ _ _ (Just (RKParamsProposed aliceKem)) <- pure e2eAlice
-- accept
let useKem = AUseKEM SRKSAccepted (AcceptKEM aliceKem)
(pkBob1, pkBob2, pKemParams_@(Just _), AE2ERatchetParams _ e2eBob) <- liftIO $ generateSndE2EParams g v (Just useKem)
Right paramsBob <- pure $ pqX3dhSnd pkBob1 pkBob2 pKemParams_ e2eAlice
Right paramsAlice <- runExceptT $ pqX3dhRcv pkAlice1 pkAlice2 pKem_ e2eBob
(pksBob@(_, _, Just _), AE2ERatchetParams _ e2eBob) <- liftIO $ generateSndE2EParams g v (Just useKem)
Right paramsBob <- pure $ pqX3dhSnd pksBob e2eAlice
Right paramsAlice <- runExceptT $ pqX3dhRcv pksAlice e2eBob
(_, pkBob3) <- atomically $ C.generateKeyPair g
let vs = testRatchetVersions
bob = initSndRatchet vs (C.publicKey pkAlice2) pkBob3 paramsBob
@@ -565,12 +565,12 @@ initRatchetsKEMProposedAgain = do
g <- C.newRandom
let v = max pqRatchetE2EEncryptVersion currentE2EEncryptVersion
-- initiate (propose KEM)
(pkAlice1, pkAlice2, pKem_@(Just _), e2eAlice) <- liftIO $ generateRcvE2EParams g v PQSupportOn
(pksAlice@(_, pkAlice2, Just _), e2eAlice) <- liftIO $ generateRcvE2EParams g v PQSupportOn
-- propose KEM again in reply
let useKem = AUseKEM SRKSProposed ProposeKEM
(pkBob1, pkBob2, pKemParams_@(Just _), AE2ERatchetParams _ e2eBob) <- liftIO $ generateSndE2EParams g v (Just useKem)
Right paramsBob <- pure $ pqX3dhSnd pkBob1 pkBob2 pKemParams_ e2eAlice
Right paramsAlice <- runExceptT $ pqX3dhRcv pkAlice1 pkAlice2 pKem_ e2eBob
(pksBob@(_, _, Just _), AE2ERatchetParams _ e2eBob) <- liftIO $ generateSndE2EParams g v (Just useKem)
Right paramsBob <- pure $ pqX3dhSnd pksBob e2eAlice
Right paramsAlice <- runExceptT $ pqX3dhRcv pksAlice e2eBob
(_, pkBob3) <- atomically $ C.generateKeyPair g
let vs = testRatchetVersions
bob = initSndRatchet vs (C.publicKey pkAlice2) pkBob3 paramsBob
+23 -12
View File
@@ -1425,7 +1425,8 @@ testInvitationShortLinkAsync viaProxy a b = do
connReq' `shouldBe` connReq
linkUserData connData' `shouldBe` userData
runRight $ do
aId <- A.joinConnectionAsync b 1 "123" Nothing True connReq "bob's connInfo" PQSupportOn SMSubscribe
aId <- A.prepareConnectionToJoin b 1 True connReq PQSupportOn
A.joinConnectionAsync b "123" False aId True connReq "bob's connInfo" PQSupportOn SMSubscribe
get b =##> \case ("123", c, JOINED sndSecure) -> c == aId && sndSecure; _ -> False
("", _, CONF confId _ "bob's connInfo") <- get a
allowConnection a bId confId "alice's connInfo"
@@ -1662,7 +1663,7 @@ testPrepareCreateConnectionLink ps = withSmpServer ps $ withAgentClients2 $ \a b
linkEntId <- atomically $ C.randomBytes 32 g
runRight $ do
(ccLink@(CCLink connReq (Just shortLink)), preparedParams) <-
A.prepareConnectionLink a 1 rootKey linkEntId True Nothing
A.prepareConnectionLink a 1 rootKey linkEntId True Nothing Nothing
liftIO $ strDecode (strEncode shortLink) `shouldBe` Right shortLink
_ <- A.createConnectionForLink a NRMInteractive 1 True ccLink preparedParams userLinkData CR.IKPQOn SMSubscribe
(FixedLinkData {linkConnReq = connReq', linkEntityId}, ContactLinkData _ userCtData') <- getConnShortLink b 1 shortLink
@@ -2685,10 +2686,12 @@ receiveMsg c cId msgId msg = do
testAsyncCommands :: SndQueueSecured -> AgentClient -> AgentClient -> AgentMsgId -> IO ()
testAsyncCommands sqSecured alice bob baseId =
runRight_ $ do
bobId <- createConnectionAsync alice 1 "1" True SCMInvitation IKPQOn SMSubscribe
bobId <- prepareConnectionToCreate alice 1 True SCMInvitation PQSupportOn
createConnectionAsync alice "1" bobId True SCMInvitation IKPQOn SMSubscribe
("1", bobId', INV (ACR _ qInfo)) <- get alice
liftIO $ bobId' `shouldBe` bobId
aliceId <- joinConnectionAsync bob 1 "2" Nothing True qInfo "bob's connInfo" PQSupportOn SMSubscribe
aliceId <- prepareConnectionToJoin bob 1 True qInfo PQSupportOn
joinConnectionAsync bob "2" False aliceId True qInfo "bob's connInfo" PQSupportOn SMSubscribe
("2", aliceId', JOINED sqSecured') <- get bob
liftIO $ do
aliceId' `shouldBe` aliceId
@@ -2779,8 +2782,8 @@ testGetConnShortLinkAsync ps = withAgentClients2 $ \alice bob ->
liftIO $ qInfo' `shouldBe` qInfo
liftIO $ userCtData' `shouldBe` userCtData
-- join connection async using connId from getConnShortLinkAsync
aliceId <- joinConnectionAsync bob 1 "2" (Just newId) True qInfo' "bob's connInfo" PQSupportOn SMSubscribe
liftIO $ aliceId `shouldBe` newId
joinConnectionAsync bob "2" True newId True qInfo' "bob's connInfo" PQSupportOn SMSubscribe
let aliceId = newId
("2", aliceId', JOINED False) <- get bob
liftIO $ aliceId' `shouldBe` aliceId
-- complete connection
@@ -2796,7 +2799,10 @@ testGetConnShortLinkAsync ps = withAgentClients2 $ \alice bob ->
testAsyncCommandsRestore :: (ASrvTransport, AStoreType) -> IO ()
testAsyncCommandsRestore ps = do
alice <- getSMPAgentClient' 1 agentCfg initAgentServers testDB
bobId <- runRight $ createConnectionAsync alice 1 "1" True SCMInvitation IKPQOn SMSubscribe
bobId <- runRight $ do
connId <- prepareConnectionToCreate alice 1 True SCMInvitation PQSupportOn
createConnectionAsync alice "1" connId True SCMInvitation IKPQOn SMSubscribe
pure connId
liftIO $ noMessages alice "alice doesn't receive INV because server is down"
disposeAgentClient alice
withAgent 2 agentCfg initAgentServers testDB $ \alice' ->
@@ -2812,7 +2818,8 @@ testAcceptContactAsync sqSecured alice bob baseId =
(aliceId, sqSecuredJoin) <- joinConnection bob 1 True qInfo "bob's connInfo" SMSubscribe
liftIO $ sqSecuredJoin `shouldBe` False -- joining via contact address connection
("", _, REQ invId _ "bob's connInfo") <- get alice
bobId <- acceptContactAsync alice 1 "1" True invId "alice's connInfo" PQSupportOn SMSubscribe
bobId <- prepareConnectionToAccept alice 1 True invId PQSupportOn
acceptContactAsync alice "1" bobId True invId "alice's connInfo" PQSupportOn SMSubscribe
get alice =##> \case ("1", c, JOINED sqSecured') -> c == bobId && sqSecured' == sqSecured; _ -> False
("", _, CONF confId _ "alice's connInfo") <- get bob
allowConnection bob aliceId confId "bob's connInfo"
@@ -3083,10 +3090,12 @@ testJoinConnectionAsyncReplyErrorV8 ps@(t, ASType qsType _) = do
withAgent 1 cfg' initAgentServers testDB $ \a ->
withAgent 2 cfg' initAgentServersSrv2 testDB2 $ \b -> do
(aId, bId) <- withSmpServerStoreLogOn ps testPort $ \_ -> runRight $ do
bId <- createConnectionAsync a 1 "1" True SCMInvitation IKPQOn SMSubscribe
bId <- prepareConnectionToCreate a 1 True SCMInvitation PQSupportOn
createConnectionAsync a "1" bId True SCMInvitation IKPQOn SMSubscribe
("1", bId', INV (ACR _ qInfo)) <- get a
liftIO $ bId' `shouldBe` bId
aId <- joinConnectionAsync b 1 "2" Nothing True qInfo "bob's connInfo" PQSupportOn SMSubscribe
aId <- prepareConnectionToJoin b 1 True qInfo PQSupportOn
joinConnectionAsync b "2" False aId True qInfo "bob's connInfo" PQSupportOn SMSubscribe
liftIO $ threadDelay 500000
ConnectionStats {rcvQueuesInfo = [], sndQueuesInfo = [SndQueueInfo {}]} <- getConnectionServers b aId
pure (aId, bId)
@@ -3128,10 +3137,12 @@ testJoinConnectionAsyncReplyError ps@(t, ASType qsType _) = do
withAgent 1 agentCfg initAgentServers testDB $ \a ->
withAgent 2 agentCfg initAgentServersSrv2 testDB2 $ \b -> do
(aId, bId) <- withSmpServerStoreLogOn ps testPort $ \_ -> runRight $ do
bId <- createConnectionAsync a 1 "1" True SCMInvitation IKPQOn SMSubscribe
bId <- prepareConnectionToCreate a 1 True SCMInvitation PQSupportOn
createConnectionAsync a "1" bId True SCMInvitation IKPQOn SMSubscribe
("1", bId', INV (ACR _ qInfo)) <- get a
liftIO $ bId' `shouldBe` bId
aId <- joinConnectionAsync b 1 "2" Nothing True qInfo "bob's connInfo" PQSupportOn SMSubscribe
aId <- prepareConnectionToJoin b 1 True qInfo PQSupportOn
joinConnectionAsync b "2" False aId True qInfo "bob's connInfo" PQSupportOn SMSubscribe
liftIO $ threadDelay 500000
ConnectionStats {rcvQueuesInfo = [], sndQueuesInfo = [SndQueueInfo {}]} <- getConnectionServers b aId
pure (aId, bId)
+152
View File
@@ -0,0 +1,152 @@
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE DuplicateRecordFields #-}
{-# LANGUAGE GADTs #-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE OverloadedLists #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE PatternSynonyms #-}
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE TypeApplications #-}
{-# OPTIONS_GHC -fno-warn-ambiguous-fields #-}
module AgentTests.ResolveNameTests (resolveNameTests) where
import AgentTests.FunctionalAPITests (withAgent)
import Control.Monad.Except (runExceptT)
import qualified Data.Aeson as J
import qualified Data.ByteString.Lazy as LB
import Data.List (isInfixOf)
import Network.HTTP.Types (Status, status200, status404, status502)
import NamesResolverServer (memCfg, memCfg2, memProxyCfg, withNames)
import qualified NamesResolverServer as NRS
import SMPAgentClient
import SMPClient
import SMPNamesTests (testNameRecord)
import Simplex.Messaging.Agent (resolveSimplexName)
import Simplex.Messaging.Agent.Client (AgentClient)
import Simplex.Messaging.Agent.Env.SQLite (InitialAgentServers (..), ServerCfg, ServerRoles (..), presetServerCfg)
import Simplex.Messaging.Agent.Protocol (AgentErrorType (..))
import Simplex.Messaging.Client (SMPProxyFallback (..), SMPProxyMode (..), pattern NRMInteractive)
import Simplex.Messaging.Protocol (SMPServer)
import qualified Simplex.Messaging.Protocol as SMP
import Simplex.Messaging.SimplexName (SimplexDomain (..), SimplexTLD (..))
import Simplex.Messaging.Transport
import Test.Hspec hiding (fit, it)
import Util (it)
nameSrvCfg :: SMPServer -> ServerCfg 'SMP.PSMP
nameSrvCfg = presetServerCfg True ServerRoles {storage = True, proxy = False, names = True} (Just 1) . SMP.noAuthSrv
proxySrvCfg :: SMPServer -> ServerCfg 'SMP.PSMP
proxySrvCfg = presetServerCfg True ServerRoles {storage = True, proxy = True, names = False} (Just 1) . SMP.noAuthSrv
oneSrv :: ServerCfg 'SMP.PSMP -> InitialAgentServers
oneSrv cfg_ = (initAgentServersProxy_ SPMNever SPFProhibit) {smp = [(1, [cfg_])]}
withDirectResolver :: (Status, LB.ByteString) -> (AgentClient -> IO a) -> IO a
withDirectResolver (st, body) k =
NRS.withResolverServer (NRS.resolveResp st body) $ \port _ ->
withSmpServerConfigOn (transport @TLS) (withNames port memCfg) testPort $ \_ ->
withAgent 1 agentCfg (oneSrv (nameSrvCfg testSMPServer)) testDB k
withProxyAndResolver :: (Status, LB.ByteString) -> (AgentClient -> IO a) -> IO a
withProxyAndResolver (st, body) k =
NRS.withResolverServer (NRS.resolveResp st body) $ \port _ ->
withSmpServerConfigOn (transport @TLS) memProxyCfg testPort $ \_ ->
withSmpServerConfigOn (transport @TLS) (withNames port memCfg2) testPort2 $ \_ ->
withAgent 1 agentCfg proxyServers testDB k
where
-- only testSMPServer2 (the resolver) has the names role; testSMPServer is the proxy
proxyServers = (initAgentServersProxy_ SPMAlways SPFProhibit) {smp = [(1, [proxySrvCfg testSMPServer, nameSrvCfg testSMPServer2])]}
withNoResolver :: (AgentClient -> IO a) -> IO a
withNoResolver k =
withSmpServerConfigOn (transport @TLS) memCfg testPort $ \_ ->
withAgent 1 agentCfg (oneSrv (nameSrvCfg testSMPServer)) testDB k
withNoNameServers :: (AgentClient -> IO a) -> IO a
withNoNameServers k = withAgent 1 agentCfg (oneSrv (proxySrvCfg testSMPServer)) testDB k
resolveNameTests :: Spec
resolveNameTests = do
describe "direct path (SPMNever)" $
it "404 propagates as SMP host (NAME NOT_FOUND)" testDirectNotFound
describe "proxy path (SPMAlways)" $
it "404 from resolver propagates via proxy as SMP <proxyHost> (NAME NOT_FOUND)" testProxyNotFound
describe "TLDTesting path" $
it "NAME NOT_FOUND for TLDTesting too" testTestingTldNotFound
describe "TLDWeb path" $
it "NAME NOT_FOUND for TLDWeb too" testWebTldNotFound
describe "no resolver configured" $
it "answers NAME NO_RESOLVER" testNoResolver
describe "no names servers (names role off everywhere)" $
it "fails agent-side with NO_NAME_SERVERS" testNoNameServers
describe "backing resolver failure" $
it "surfaces as SMP host (NAME (RESOLVER ..))" testBackendError
describe "success path" $
it "returns NameRecord" testDirectSuccess
testDirectNotFound :: HasCallStack => IO ()
testDirectNotFound =
withDirectResolver (status404, "{}") $ \c -> do
r <- runExceptT $ resolveSimplexName c NRMInteractive 1 (SimplexDomain TLDSimplex "alice" [])
case r of
Left (SMP _ (SMP.NAME SMP.NOT_FOUND)) -> pure ()
_ -> expectationFailure $ "expected Left (SMP _ (NAME NOT_FOUND)), got: " <> show r
testProxyNotFound :: HasCallStack => IO ()
testProxyNotFound =
withProxyAndResolver (status404, "{}") $ \c -> do
r <- runExceptT $ resolveSimplexName c NRMInteractive 1 (SimplexDomain TLDSimplex "alice" [])
case r of
Left (SMP host (SMP.NAME SMP.NOT_FOUND)) | testPort `isInfixOf` host -> pure ()
_ -> expectationFailure $ "expected Left (SMP <proxyHost:" <> testPort <> "> (NAME NOT_FOUND)), got: " <> show r
testTestingTldNotFound :: HasCallStack => IO ()
testTestingTldNotFound =
withDirectResolver (status404, "{}") $ \c -> do
r <- runExceptT $ resolveSimplexName c NRMInteractive 1 (SimplexDomain TLDTesting "bob" [])
case r of
Left (SMP _ (SMP.NAME SMP.NOT_FOUND)) -> pure ()
_ -> expectationFailure $ "expected Left (SMP _ (NAME NOT_FOUND)), got: " <> show r
testWebTldNotFound :: HasCallStack => IO ()
testWebTldNotFound =
withDirectResolver (status404, "{}") $ \c -> do
r <- runExceptT $ resolveSimplexName c NRMInteractive 1 (SimplexDomain TLDWeb "example.com" [])
case r of
Left (SMP _ (SMP.NAME SMP.NOT_FOUND)) -> pure ()
_ -> expectationFailure $ "expected Left (SMP _ (NAME NOT_FOUND)), got: " <> show r
testNoResolver :: HasCallStack => IO ()
testNoResolver =
withNoResolver $ \c -> do
r <- runExceptT $ resolveSimplexName c NRMInteractive 1 (SimplexDomain TLDSimplex "alice" [])
case r of
Left (SMP _ (SMP.NAME SMP.NO_RESOLVER)) -> pure ()
_ -> expectationFailure $ "expected Left (SMP _ (NAME NO_RESOLVER)), got: " <> show r
testNoNameServers :: HasCallStack => IO ()
testNoNameServers =
withNoNameServers $ \c -> do
r <- runExceptT $ resolveSimplexName c NRMInteractive 1 (SimplexDomain TLDSimplex "alice" [])
case r of
Left NO_NAME_SERVERS -> pure ()
_ -> expectationFailure $ "expected Left NO_NAME_SERVERS, got: " <> show r
testBackendError :: HasCallStack => IO ()
testBackendError =
withDirectResolver (status502, "{}") $ \c -> do
r <- runExceptT $ resolveSimplexName c NRMInteractive 1 (SimplexDomain TLDSimplex "alice" [])
case r of
Left (SMP _ (SMP.NAME (SMP.RESOLVER _))) -> pure ()
_ -> expectationFailure $ "expected Left (SMP _ (NAME (RESOLVER ..))), got: " <> show r
testDirectSuccess :: HasCallStack => IO ()
testDirectSuccess =
withDirectResolver (status200, J.encode testNameRecord) $ \c -> do
r <- runExceptT $ resolveSimplexName c NRMInteractive 1 (SimplexDomain TLDSimplex "alice" [])
case r of
Right nr -> nr `shouldBe` testNameRecord
_ -> expectationFailure $ "expected Right NameRecord, got: " <> show r
+2 -2
View File
@@ -52,10 +52,10 @@ testSMPServers =
]
storageOnly :: ServerRoles
storageOnly = ServerRoles {storage = True, proxy = False}
storageOnly = ServerRoles {storage = True, proxy = False, names = False}
proxyOnly :: ServerRoles
proxyOnly = ServerRoles {storage = False, proxy = True}
proxyOnly = ServerRoles {storage = False, proxy = True, names = False}
initServers :: InitialAgentServers
initServers =
+19 -1
View File
@@ -13,7 +13,7 @@ import qualified Crypto.PubKey.RSA as RSA
import qualified Data.ByteString.Lazy as BL
import qualified Data.HashMap.Strict as HM
import Data.Ini (Ini (..), lookupValue, readIniFile, writeIniFile)
import Data.List (isPrefixOf)
import Data.List (isInfixOf, isPrefixOf)
import qualified Data.Text as T
import qualified Data.X509 as X
import qualified Data.X509.File as XF
@@ -85,6 +85,7 @@ cliTests = do
it "no store log, no password" $ smpServerTest False False
it "with store log, no password" $ smpServerTest True False
it "static files" smpServerTestStatic
it "cloud scripts disable embedded web without certificates" smpCloudScriptsDisableWeb
#if defined(dbServerPostgres)
around_ (postgressBracket ntfTestServerDBConnectInfo) $ before_ (createNtfSchema ntfTestServerDBConnectInfo ntfTestStoreDBOpts) $
describe "Ntf server CLI" $ do
@@ -200,6 +201,23 @@ smpServerTestStatic = do
let X.CertificateChain cc = tlsPeerCert tls
in map (X.signedObject . X.getSigned) cc
smpCloudScriptsDisableWeb :: HasCallStack => IO ()
smpCloudScriptsDisableWeb = do
linode <- readFile "scripts/smp-server-linode.sh"
digitalOceanInit <-
readFile "scripts/smp-server-digitalocean-droplet/files/opt/simplex/initialize_server.sh"
digitalOceanLogin <-
readFile "scripts/smp-server-digitalocean-droplet/files/opt/simplex/on_login.sh"
linode `shouldSatisfy` ("init_opts+=(--disable-web)" `isInfixOf`)
linode `shouldSatisfy` ("web.crt" `isInfixOf`)
linode `shouldSatisfy` ("web.key" `isInfixOf`)
linode `shouldSatisfy` ("uncomment WEB https/cert/key" `isInfixOf`)
digitalOceanInit
`shouldSatisfy` ("smp-server init -l --disable-web --ip $ip_address" `isInfixOf`)
digitalOceanLogin `shouldSatisfy` ("web.crt" `isInfixOf`)
digitalOceanLogin `shouldSatisfy` ("web.key" `isInfixOf`)
digitalOceanLogin `shouldSatisfy` ("uncomment WEB https/cert/key" `isInfixOf`)
#if defined(dbServerPostgres)
createNtfSchema :: PSQL.ConnectInfo -> DBOpts -> IO ()
createNtfSchema connInfo DBOpts {schema} = do
+176 -3
View File
@@ -1,15 +1,17 @@
{-# LANGUAGE GADTs #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE TypeApplications #-}
{-# OPTIONS_GHC -Wno-orphans #-}
module CoreTests.CryptoTests (cryptoTests) where
import Control.Concurrent.STM
import Control.Monad.Except
import qualified Data.Aeson as J
import qualified Data.ByteString.Char8 as B
import qualified Data.ByteString.Lazy.Char8 as LB
import Data.Either (isRight)
import Data.Either (isLeft, isRight)
import Data.Int (Int64)
import qualified Data.Text as T
import Data.Text.Encoding (encodeUtf8)
@@ -22,11 +24,15 @@ import qualified Data.X509.Validation as XV
import qualified SMPClient
import qualified Simplex.Messaging.Crypto as C
import qualified Simplex.Messaging.Crypto.Lazy as LC
import Simplex.Messaging.Crypto.BBS
import Simplex.Messaging.Crypto.SNTRUP761.Bindings
import Simplex.Messaging.Crypto.SNTRUP761.Bindings.Defines
import Simplex.Messaging.Encoding (Large (..), smpDecode, smpEncode)
import Simplex.Messaging.Encoding.String (strDecode, strEncode)
import Simplex.Messaging.Transport.Client
import Test.Hspec hiding (fit, it)
import Test.Hspec.QuickCheck (modifyMaxSuccess)
import Test.QuickCheck
import Test.QuickCheck hiding (Large)
import Util
cryptoTests :: Spec
@@ -99,8 +105,20 @@ cryptoTests = do
describe "X448" $ testEncoding C.SX448
describe "X509 chains" $ do
it "should validate certificates" testValidateX509
describe "sntrup761" $
describe "sntrup761" $ do
it "should enc/dec key" testSNTRUP761
it "should reject malformed KEM encodings" testSNTRUP761RejectsMalformedEncodings
describe "BBS+" $ do
it "should sign and verify" testBBSSignVerify
it "should derive public key from secret key" testBBSPublicKeyDerivation
it "should generate and verify proof" testBBSProofRoundtrip
it "should reject tampered proof" testBBSTamperedProof
it "should reject wrong disclosed message" testBBSWrongMessage
it "should reject wrong public key" testBBSWrongKey
it "should reject invalid proof parameters" testBBSInvalidProofParams
it "should produce unlinkable proofs" testBBSUnlinkable
it "should produce proof of expected size" testBBSProofSize
it "should roundtrip JSON and reject wrong-length input" testBBSJSON
instance Eq C.APublicKey where
C.APublicKey a k == C.APublicKey a' k' = case testEquality a a' of
@@ -271,3 +289,158 @@ testSNTRUP761 = do
(c, KEMSharedKey k) <- sntrup761Enc drg pk
KEMSharedKey k' <- sntrup761Dec c sk
k' `shouldBe` k
testSNTRUP761RejectsMalformedEncodings :: IO ()
testSNTRUP761RejectsMalformedEncodings = do
smpDecode @KEMPublicKey (smpEncode $ Large shortPublicKey) `shouldSatisfy` isLeft
strDecode @KEMPublicKey (strEncode shortPublicKey) `shouldSatisfy` isLeft
smpDecode @KEMPublicKey (smpEncode $ Large validPublicKey) `shouldSatisfy` isRight
strDecode @KEMPublicKey (strEncode validPublicKey) `shouldSatisfy` isRight
smpDecode @KEMCiphertext (smpEncode $ Large shortCiphertext) `shouldSatisfy` isLeft
strDecode @KEMCiphertext (strEncode shortCiphertext) `shouldSatisfy` isLeft
smpDecode @KEMCiphertext (smpEncode $ Large validCiphertext) `shouldSatisfy` isRight
strDecode @KEMCiphertext (strEncode validCiphertext) `shouldSatisfy` isRight
smpDecode @KEMSecretKey (smpEncode $ Large shortSecretKey) `shouldSatisfy` isLeft
strDecode @KEMSecretKey (strEncode shortSecretKey) `shouldSatisfy` isLeft
shortPublicKey :: B.ByteString
shortPublicKey = B.replicate (c_SNTRUP761_PUBLICKEY_SIZE - 1) 'p'
validPublicKey :: B.ByteString
validPublicKey = B.replicate c_SNTRUP761_PUBLICKEY_SIZE 'p'
shortCiphertext :: B.ByteString
shortCiphertext = B.replicate (c_SNTRUP761_CIPHERTEXT_SIZE - 1) 'c'
validCiphertext :: B.ByteString
validCiphertext = B.replicate c_SNTRUP761_CIPHERTEXT_SIZE 'c'
shortSecretKey :: B.ByteString
shortSecretKey = B.replicate (c_SNTRUP761_SECRETKEY_SIZE - 1) 's'
-- BBS+ tests
bbsHeader :: BBSHeader
bbsHeader = BBSHeader "SimpleX"
bbsMessages :: [B.ByteString]
bbsMessages = ["secret_master_key", "2026-07-31", "supporter"]
bbsDisclosedIdxs :: [Int]
bbsDisclosedIdxs = [1, 2]
bbsDisclosedMsgs :: [B.ByteString]
bbsDisclosedMsgs = ["2026-07-31", "supporter"]
testBBSSignVerify :: IO ()
testBBSSignVerify = do
Right (pk, sk) <- bbsKeyGen
let BBSSecretKey skBs = sk
BBSPublicKey pkBs = pk
B.length skBs `shouldBe` 32
B.length pkBs `shouldBe` 96
Right sig <- bbsSign sk bbsHeader bbsMessages
let BBSSignature sigBs = sig
B.length sigBs `shouldBe` 80
bbsVerify pk sig bbsHeader bbsMessages >>= (`shouldBe` True)
bbsVerify pk sig bbsHeader ["wrong", "2026-07-31", "supporter"] >>= (`shouldBe` False)
testBBSPublicKeyDerivation :: IO ()
testBBSPublicKeyDerivation = do
Right (pk, sk) <- bbsKeyGen
-- the public key derived from the secret key matches the one keygen returned
bbsPublicKey sk >>= (`shouldBe` Right pk)
testBBSProofRoundtrip :: IO ()
testBBSProofRoundtrip = do
Right (pk, sk) <- bbsKeyGen
Right sig <- bbsSign sk bbsHeader bbsMessages
let ph = BBSPresHeader "test-nonce-1"
Right proof <- bbsProofGen pk sig bbsHeader ph bbsDisclosedIdxs bbsMessages
result <- bbsProofVerify pk proof bbsHeader ph bbsDisclosedIdxs 3 bbsDisclosedMsgs
result `shouldBe` True
testBBSTamperedProof :: IO ()
testBBSTamperedProof = do
Right (pk, sk) <- bbsKeyGen
Right sig <- bbsSign sk bbsHeader bbsMessages
let ph = BBSPresHeader "test-nonce-2"
Right (BBSProof proofBs) <- bbsProofGen pk sig bbsHeader ph bbsDisclosedIdxs bbsMessages
let tampered = BBSProof $ B.take 10 proofBs <> "\xff" <> B.drop 11 proofBs
result <- bbsProofVerify pk tampered bbsHeader ph bbsDisclosedIdxs 3 bbsDisclosedMsgs
result `shouldBe` False
testBBSWrongMessage :: IO ()
testBBSWrongMessage = do
Right (pk, sk) <- bbsKeyGen
Right sig <- bbsSign sk bbsHeader bbsMessages
let ph = BBSPresHeader "test-nonce-3"
Right proof <- bbsProofGen pk sig bbsHeader ph bbsDisclosedIdxs bbsMessages
result <- bbsProofVerify pk proof bbsHeader ph bbsDisclosedIdxs 3 ["2026-07-31", "business"]
result `shouldBe` False
testBBSWrongKey :: IO ()
testBBSWrongKey = do
Right (pk, sk) <- bbsKeyGen
Right (pk2, _) <- bbsKeyGen
Right sig <- bbsSign sk bbsHeader bbsMessages
let ph = BBSPresHeader "test-nonce-4"
Right proof <- bbsProofGen pk sig bbsHeader ph bbsDisclosedIdxs bbsMessages
result <- bbsProofVerify pk2 proof bbsHeader ph bbsDisclosedIdxs 3 bbsDisclosedMsgs
result `shouldBe` False
testBBSInvalidProofParams :: IO ()
testBBSInvalidProofParams = do
Right (pk, sk) <- bbsKeyGen
Right sig <- bbsSign sk bbsHeader bbsMessages
let ph = BBSPresHeader "test-nonce-invalid"
Right proof <- bbsProofGen pk sig bbsHeader ph bbsDisclosedIdxs bbsMessages
bbsProofGen pk sig bbsHeader ph [2, 1] bbsMessages
`shouldReturn` Left "bbsProofGen: invalid disclosed indexes"
bbsProofGen pk sig bbsHeader ph [1, 1] bbsMessages
`shouldReturn` Left "bbsProofGen: invalid disclosed indexes"
bbsProofGen pk sig bbsHeader ph [3] bbsMessages
`shouldReturn` Left "bbsProofGen: invalid disclosed indexes"
bbsProofVerify pk proof bbsHeader ph bbsDisclosedIdxs 3 ["2026-07-31"]
>>= (`shouldBe` False)
bbsProofVerify pk proof bbsHeader ph [2, 1] 3 bbsDisclosedMsgs
>>= (`shouldBe` False)
bbsProofVerify pk proof bbsHeader ph bbsDisclosedIdxs 4 bbsDisclosedMsgs
>>= (`shouldBe` False)
bbsProofVerify pk proof bbsHeader ph [] (-1) []
>>= (`shouldBe` False)
testBBSUnlinkable :: IO ()
testBBSUnlinkable = do
Right (pk, sk) <- bbsKeyGen
Right sig <- bbsSign sk bbsHeader bbsMessages
let ph1 = BBSPresHeader "nonce-contact-1"
ph2 = BBSPresHeader "nonce-contact-2"
Right (BBSProof proof1) <- bbsProofGen pk sig bbsHeader ph1 bbsDisclosedIdxs bbsMessages
Right (BBSProof proof2) <- bbsProofGen pk sig bbsHeader ph2 bbsDisclosedIdxs bbsMessages
proof1 `shouldNotBe` proof2
bbsProofVerify pk (BBSProof proof1) bbsHeader ph1 bbsDisclosedIdxs 3 bbsDisclosedMsgs >>= (`shouldBe` True)
bbsProofVerify pk (BBSProof proof2) bbsHeader ph2 bbsDisclosedIdxs 3 bbsDisclosedMsgs >>= (`shouldBe` True)
testBBSProofSize :: IO ()
testBBSProofSize = do
Right (pk, sk) <- bbsKeyGen
Right sig <- bbsSign sk bbsHeader bbsMessages
let ph = BBSPresHeader "test-nonce-size"
Right (BBSProof proofBs) <- bbsProofGen pk sig bbsHeader ph bbsDisclosedIdxs bbsMessages
B.length proofBs `shouldBe` 304 -- 272 + 32 * 1 undisclosed
testBBSJSON :: IO ()
testBBSJSON = do
Right (pk, sk) <- bbsKeyGen
Right sig <- bbsSign sk bbsHeader bbsMessages
let ph = BBSPresHeader "json-nonce"
Right proof <- bbsProofGen pk sig bbsHeader ph bbsDisclosedIdxs bbsMessages
-- valid values roundtrip through JSON
J.decode (J.encode sk) `shouldBe` Just sk
J.decode (J.encode pk) `shouldBe` Just pk
J.decode (J.encode sig) `shouldBe` Just sig
J.decode (J.encode proof) `shouldBe` Just proof
-- FromJSON must reject wrong-length input (regression: StrJSON length validation)
(J.decode (J.encode (BBSSecretKey (B.replicate 16 '\0'))) :: Maybe BBSSecretKey) `shouldBe` Nothing
(J.decode (J.encode (BBSSignature (B.replicate 10 '\0'))) :: Maybe BBSSignature) `shouldBe` Nothing
+29
View File
@@ -10,11 +10,14 @@ import Data.ByteString.Char8 (ByteString)
import qualified Data.ByteString.Char8 as B
import Data.ByteString.Internal (w2c)
import Data.Int (Int64)
import Data.List.NonEmpty (NonEmpty (..))
import Data.Time.Clock.System (SystemTime (..), getSystemTime, utcToSystemTime)
import Data.Time.ISO8601 (parseISO8601)
import Simplex.Messaging.Encoding
import Simplex.Messaging.Encoding.String
import Simplex.Messaging.Parsers (parseAll)
import Simplex.Messaging.Protocol (ProtocolServer (..), XFTPServer)
import Simplex.Messaging.ServiceScheme (ServiceScheme (..), SrvLoc (..))
import Simplex.Messaging.Transport.Client (TransportHost (..))
import Test.Hspec hiding (fit, it)
import Test.Hspec.QuickCheck (modifyMaxSuccess)
@@ -67,7 +70,29 @@ encodingTests = modifyMaxSuccess (const 1000) $ do
THDomainName "192.256.0.1" #==# "192.256.0.1"
THDomainName "192.168.0.-1" #==# "192.168.0.-1"
shouldNotParse @TransportHost "192.168.0.0.1" "endOfInput"
-- brackets are reserved for IPv6 literals
shouldReject @TransportHost "[simplex.chat]"
shouldReject @TransportHost "[smp.simplex.im]"
describe "Encoding service locations" $ do
it "should parse bracketed IPv6 host with port" $ do
strDecode @ServiceScheme "https://[2001:db8::1]:8443"
`shouldBe` Right (SSAppServer $ SrvLoc "2001:db8::1" "8443")
strEncode (SSAppServer $ SrvLoc "2001:db8::1" "8443")
`shouldBe` "https://[2001:db8::1]:8443"
it "should reject bracketed non-IPv6 host" $
shouldReject @ServiceScheme "https://[simplex.chat]:8443"
describe "Encoding protocol servers" $ do
it "should parse bracketed IPv6 server host with port" $
case strDecode @XFTPServer "xftp://1234-w==@[2001:db8::1]:443" of
Left err -> expectationFailure err
Right (ProtocolServer _ parsedHost parsedPort _) -> do
parsedHost `shouldBe` (ipv6Host :| [])
parsedPort `shouldBe` "443"
it "should reject bracketed non-IPv6 server host" $
shouldReject @XFTPServer "xftp://1234-w==@[simplex.chat]:443"
where
ipv6Host :: TransportHost
ipv6Host = either error id $ strDecode "2001:db8::1"
testSystemTime :: SystemTime -> Expectation
testSystemTime t = do
smpEncode t `shouldBe` smpEncode (systemSeconds t)
@@ -78,3 +103,7 @@ encodingTests = modifyMaxSuccess (const 1000) $ do
strDecode s `shouldBe` Right x
shouldNotParse :: forall s. (StrEncoding s, Eq s, Show s) => ByteString -> String -> Expectation
shouldNotParse s err = strDecode s `shouldBe` (Left err :: Either String s)
shouldReject :: forall s. (StrEncoding s, Show s) => ByteString -> Expectation
shouldReject s = case strDecode s :: Either String s of
Left _ -> pure ()
Right a -> expectationFailure $ "expected parse failure, got " <> show a
+7
View File
@@ -50,6 +50,13 @@ utilTests = do
runExceptT (tryAllErrors throwTestException) `shouldReturn` Right (Left (TestException "user error (error)"))
it "should return no errors as Right" $
runExceptT (tryAllErrors noErrors) `shouldReturn` Right (Right "no errors")
-- tryAllErrors rethrows asynchronous exceptions (it uses UnliftIO.catch). Any recovery placed
-- after `tryAllErrors action` - e.g. putTMVar to fill a SessionVar - is therefore SKIPPED when
-- the thread is killed mid-action. Unlike tryAllOwnErrors, it also rethrows the overflow exceptions.
it "should rethrow ThreadKilled" $
runExceptT (tryAllErrors $ throwAsync ThreadKilled) `shouldThrow` (\e -> e == ThreadKilled)
it "should rethrow StackOverflow" $
runExceptT (tryAllErrors $ throwAsync StackOverflow) `shouldThrow` (\e -> e == StackOverflow)
describe "catchAllErrors" $ do
it "should catch ExceptT error" $
runExceptT (throwTestError `catchAllErrors` handleCatch) `shouldReturn` Right "caught TestError \"error\""
+36 -17
View File
@@ -23,18 +23,21 @@ import Util
import XFTPClient (testXFTPPostgresCfg)
xftpStoreTests :: Spec
xftpStoreTests = describe "PostgresFileStore operations" $ do
it "should add and get file by sender" testAddGetFileSender
it "should add and get file by recipient" testAddGetFileRecipient
it "should reject duplicate file" testDuplicateFile
it "should return AUTH for nonexistent file" testGetNonexistent
it "should set file path with IS NULL guard" testSetFilePath
it "should reject duplicate recipient" testDuplicateRecipient
it "should delete file and cascade recipients" testDeleteFileCascade
it "should block file and update status" testBlockFile
it "should ack file reception" testAckFile
it "should return expired files with limit" testExpiredFiles
it "should compute used storage and file count" testStorageAndCount
xftpStoreTests = do
describe "STMFileStore operations" $
it "should compute committed used storage and file count" testSTMStorageAndCount
describe "PostgresFileStore operations" $ do
it "should add and get file by sender" testAddGetFileSender
it "should add and get file by recipient" testAddGetFileRecipient
it "should reject duplicate file" testDuplicateFile
it "should return AUTH for nonexistent file" testGetNonexistent
it "should set file path with IS NULL guard" testSetFilePath
it "should reject duplicate recipient" testDuplicateRecipient
it "should delete file and cascade recipients" testDeleteFileCascade
it "should block file and update status" testBlockFile
it "should ack file reception" testAckFile
it "should return expired files with limit" testExpiredFiles
it "should compute committed used storage and file count" testStorageAndCount
xftpMigrationTests :: Spec
xftpMigrationTests = describe "XFTP migration round-trip" $ do
@@ -201,16 +204,32 @@ testExpiredFiles = withPgStore $ \st -> do
testStorageAndCount :: Expectation
testStorageAndCount = withPgStore $ \st -> do
testStorageAndCountForStore st
testSTMStorageAndCount :: Expectation
testSTMStorageAndCount = do
st <- newFileStore () :: IO STMFileStore
testStorageAndCountForStore st
closeFileStore st
testStorageAndCountForStore :: FileStoreClass s => s -> Expectation
testStorageAndCountForStore st = do
g <- C.newRandom
(sndKey, _) <- atomically $ C.generateAuthKeyPair C.SEd25519 g
getUsedStorage st `shouldReturn` 0
getFileCount st `shouldReturn` 0
let fileInfo = testFileInfo sndKey
addFile st (EntityId "file_a__________") fileInfo testCreatedAt EntityActive `shouldReturn` Right ()
addFile st (EntityId "file_b__________") fileInfo testCreatedAt EntityActive `shouldReturn` Right ()
let fileInfoA = testFileInfo sndKey
fileInfoB = fileInfoA {size = 64000}
fileA = EntityId "file_a__________"
fileB = EntityId "file_b__________"
addFile st fileA fileInfoA testCreatedAt EntityActive `shouldReturn` Right ()
addFile st fileB fileInfoB testCreatedAt EntityActive `shouldReturn` Right ()
getFileCount st `shouldReturn` 2
used <- getUsedStorage st
used `shouldBe` 256000 -- 128000 * 2
getUsedStorage st `shouldReturn` 0
setFilePath st fileA "/tmp/file_a" `shouldReturn` Right ()
getUsedStorage st `shouldReturn` 128000
setFilePath st fileB "/tmp/file_b" `shouldReturn` Right ()
getUsedStorage st `shouldReturn` 192000
-- Migration round-trip test
+81
View File
@@ -0,0 +1,81 @@
{-# LANGUAGE DuplicateRecordFields #-}
{-# LANGUAGE GADTs #-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE OverloadedStrings #-}
{-# OPTIONS_GHC -fno-warn-ambiguous-fields #-}
-- | Name resolver mock
module NamesResolverServer
( withResolverServer,
withResolverServerDelayed,
resolveResp,
testNamesConfig,
memCfg,
memProxyCfg,
memCfg2,
withNames,
)
where
import Control.Concurrent (threadDelay)
import Control.Monad (when)
import qualified Data.ByteString.Lazy as LB
import Data.IORef (IORef, atomicModifyIORef', newIORef)
import Data.Text (Text)
import Network.HTTP.Types (Status, hContentType, notFound404, ok200)
import Network.Wai (Application, pathInfo, responseLBS)
import qualified Network.Wai.Handler.Warp as Warp
import SMPClient (AServerConfig (..), cfgMS, proxyCfgMS, testStoreLogFile2, testStoreMsgsFile2, updateCfg)
import Simplex.Messaging.Server.Env.STM (AStoreType (..), ServerConfig (..), ServerStoreCfg (..), StorePaths (..))
import Simplex.Messaging.Server.MsgStore.Types (SMSType (..), SQSType (..))
import Simplex.Messaging.Server.Names (NamesConfig (..))
-- | Run an action with a local HTTP resolver on a free port.
withResolverServer :: ([Text] -> (Status, LB.ByteString)) -> (Int -> IORef [[Text]] -> IO a) -> IO a
withResolverServer = withResolverServerDelayed 0
withResolverServerDelayed :: Int -> ([Text] -> (Status, LB.ByteString)) -> (Int -> IORef [[Text]] -> IO a) -> IO a
withResolverServerDelayed delayMs handler action = do
reqs <- newIORef []
Warp.withApplication (pure (app reqs)) $ \port -> action port reqs
where
app :: IORef [[Text]] -> Application
app reqs req send = do
atomicModifyIORef' reqs $ \rs -> (rs <> [pathInfo req], ())
when (delayMs > 0) $ threadDelay (delayMs * 1000)
let (st, body) = handler (pathInfo req)
send $ responseLBS st [(hContentType, "application/json")] body
resolveResp :: Status -> LB.ByteString -> [Text] -> (Status, LB.ByteString)
resolveResp st body = \case
["health"] -> (ok200, "{}")
("resolve" : _) -> (st, body)
_ -> (notFound404, "{}")
testNamesConfig :: Int -> NamesConfig
testNamesConfig port =
NamesConfig
{ resolverEndpoint = "http://127.0.0.1:" <> show port,
resolverAuth = Nothing,
resolverTimeoutMs = 1000,
resolverMaxResponseBytes = 65536
}
memCfg :: AServerConfig
memCfg = cfgMS (ASType SQSMemory SMSMemory)
memProxyCfg :: AServerConfig
memProxyCfg = proxyCfgMS (ASType SQSMemory SMSMemory)
memCfg2 :: AServerConfig
memCfg2 = case memCfg of
ASrvCfg qt mt c -> ASrvCfg qt mt c {serverStoreCfg = newStoreCfg (serverStoreCfg c)}
where
newStoreCfg :: ServerStoreCfg s -> ServerStoreCfg s
newStoreCfg = \case
SSCMemory _ -> SSCMemory (Just StorePaths {storeLogFile = testStoreLogFile2, storeMsgsFile = Just testStoreMsgsFile2})
other -> other
withNames :: Int -> AServerConfig -> AServerConfig
withNames port c = updateCfg c $ \cfg_ -> cfg_ {namesConfig = Just (testNamesConfig port)}
+1
View File
@@ -141,6 +141,7 @@ ntfServerCfg =
{ apnsPort = apnsTestPort,
caStoreFile = "tests/fixtures/ca.crt"
},
allowTestPushProvider = True,
subsBatchSize = 900,
inactiveClientExpiration = Just defaultInactiveClientExpiration,
dbStoreConfig = ntfTestDBCfg,
+16
View File
@@ -39,6 +39,7 @@ import qualified Simplex.Messaging.Agent.Protocol as AP
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Encoding
import Simplex.Messaging.Notifications.Protocol
import Simplex.Messaging.Notifications.Server.Env (NtfServerConfig (..))
import Simplex.Messaging.Notifications.Server.Push.APNS
import Simplex.Messaging.Notifications.Transport (THandleNTF)
import Simplex.Messaging.Parsers (parse, parseAll)
@@ -52,6 +53,8 @@ import Util
ntfServerTests :: (ASrvTransport, AStoreType) -> Spec
ntfServerTests ps@(t, _) = do
describe "Notifications server protocol syntax" $ ntfSyntaxTests t
describe "Push provider policy" $ do
it "rejects APNS test provider unless enabled" $ testApnsTestProviderRejected t
describe "Notification subscriptions (NKEY)" $ testNotificationSubscription ps createNtfQueueNKEY
describe "Notification subscriptions (NEW with ntf creds)" $ testNotificationSubscription ps createNtfQueueNEW
describe "Retried notification subscription" $ testRetriedNtfSubscription ps
@@ -72,6 +75,19 @@ ntfSyntaxTests (ATransport t) = do
Expectation
command >#> response = withAPNSMockServer $ \_ -> ntfServerTest t command `shouldReturn` response
testApnsTestProviderRejected :: ASrvTransport -> Expectation
testApnsTestProviderRejected (ATransport (t :: TProxy c 'TServer)) = do
g <- C.newRandom
(tknPub, tknKey) <- atomically $ C.generateAuthKeyPair C.SEd25519 g
(dhPub, _dhPriv :: C.PrivateKeyX25519) <- atomically $ C.generateKeyPair g
let tkn = DeviceToken PPApnsTest "abcd"
ntfCfg = ntfServerCfg {allowTestPushProvider = False, transports = [(ntfTestPort, ATransport t, False)]}
withNtfServerCfg ntfCfg $ \_ ->
testNtfClient $ \(nh :: THandleNTF c 'TClient) -> do
RespNtf "1" NoEntity (NRErr (CMD PROHIBITED)) <-
signSendRecvNtf nh tknKey ("1", NoEntity, TNEW $ NewNtfTkn tkn tknPub dhPub)
pure ()
pattern RespNtf :: CorrId -> QueueId -> NtfResponse -> Transmission (Either ErrorType NtfResponse)
pattern RespNtf corrId queueId command <- (corrId, queueId, Right command)
+159
View File
@@ -0,0 +1,159 @@
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE DuplicateRecordFields #-}
{-# LANGUAGE GADTs #-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE PatternSynonyms #-}
{-# LANGUAGE ScopedTypeVariables #-}
{-# LANGUAGE TypeApplications #-}
{-# OPTIONS_GHC -fno-warn-ambiguous-fields #-}
module RSLVTests (rslvTests) where
import Control.Monad.Trans.Except (ExceptT, runExceptT)
import qualified Data.Aeson as J
import qualified Data.ByteString.Char8 as B
import qualified Data.ByteString.Lazy as LB
import Data.List.NonEmpty (NonEmpty (..))
import Data.Text (Text)
import Data.Text.Encoding (encodeUtf8)
import Data.Time.Clock (getCurrentTime)
import Network.HTTP.Types (Status, status200, status404, status502)
import NamesResolverServer (memCfg, memCfg2, memProxyCfg, withNames)
import qualified NamesResolverServer as NRS
import SMPClient
import Simplex.Messaging.Client
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Encoding.String (strDecode)
import SMPNamesTests (testNameRecord)
import Simplex.Messaging.Protocol
( BrokerMsg (..),
Cmd (..),
Command (..),
CorrId (..),
ErrorType (..),
NameErrorType (..),
SParty (..),
Transmission,
TransmissionForAuth (..),
encodeTransmissionForAuth,
pattern SMPServer,
tGetClient,
tPut,
)
import qualified Simplex.Messaging.Protocol as SMP
import Simplex.Messaging.SimplexName (SimplexDomain)
import Simplex.Messaging.Transport
import Simplex.Messaging.Version (mkVersionRange)
import Test.Hspec hiding (fit, it)
import Util (it)
domain :: Text -> SimplexDomain
domain = either error id . strDecode . encodeUtf8
withResolverServer :: (Status, LB.ByteString) -> IO a -> IO a
withResolverServer (st, body) runTest =
NRS.withResolverServer (NRS.resolveResp st body) $ \port _ ->
withSmpServerConfigOn (transport @TLS) (withNames port memCfg) testPort (const runTest)
withProxyAndResolver :: (Status, LB.ByteString) -> IO a -> IO a
withProxyAndResolver (st, body) runTest =
NRS.withResolverServer (NRS.resolveResp st body) $ \port _ ->
withSmpServerConfigOn (transport @TLS) memProxyCfg testPort $ \_ ->
withSmpServerConfigOn (transport @TLS) (withNames port memCfg2) testPort2 (const runTest)
sendRslv :: Transport c => THandleSMP c 'TClient -> B.ByteString -> SimplexDomain -> IO (Transmission (Either ErrorType BrokerMsg))
sendRslv h@THandle {params} corrId d = do
let TransmissionForAuth {tToSend} = encodeTransmissionForAuth params (CorrId corrId, NoEntity, Cmd SResolver (RSLV d))
[Right ()] <- tPut h (Right (Nothing, tToSend) :| [])
r :| _ <- tGetClient h
pure r
rslvTests :: Spec
rslvTests = do
describe "RSLV direct (non-forwarded)" $ do
it "resolver replies 404 -> NAME NOT_FOUND (reached, not CMD PROHIBITED)" testRslvBackendNotFound
it "resolver replies 502 -> NAME (RESOLVER ..)" testRslvBackendHttpErr
it "no names config -> NAME NO_RESOLVER" testRslvDisabled
it "refuses to send RSLV on a session below namesSMPVersion" testRslvVersion
describe "RSLV forwarded (PFWD)" $ do
it "PFWD-wrapped RSLV reaches resolver via proxy (PCEProtocolError (NAME NOT_FOUND))" testRslvForwarded
it "PFWD-wrapped RSLV success returns RNAME (record JSON frames over the proxy)" testRslvForwardedSuccess
describe "RSLV success path (RNAME response)" $ do
it "returns RNAME with NameRecord" testRslvSuccess
testRslvBackendNotFound :: IO ()
testRslvBackendNotFound =
withResolverServer (status404, "{}") $
testSMPClient @TLS $ \h -> do
(corrId, _entId, resp) <- sendRslv h "rs01" (domain "ghost.simplex")
corrId `shouldBe` CorrId "rs01"
resp `shouldBe` Right (ERR (NAME NOT_FOUND))
testRslvBackendHttpErr :: IO ()
testRslvBackendHttpErr =
withResolverServer (status502, "{}") $
testSMPClient @TLS $ \h -> do
(_, _, resp) <- sendRslv h "rs05" (domain "alice.simplex")
resp `shouldBe` Right (ERR (NAME (RESOLVER "HTTP 502")))
testRslvDisabled :: IO ()
testRslvDisabled =
withSmpServerConfigOn (transport @TLS) memCfg testPort $ const $
testSMPClient @TLS $ \h -> do
(_, _, resp) <- sendRslv h "rs06" (domain "alice.simplex")
resp `shouldBe` Right (ERR (NAME NO_RESOLVER))
testRslvVersion :: IO ()
testRslvVersion =
withResolverServer (status200, J.encode testNameRecord) $ do
g <- C.newRandom
ts <- getCurrentTime
let srv = SMPServer testHost testPort testKeyHash
oldCfg = defaultSMPClientConfig {serverVRange = mkVersionRange minServerSMPRelayVersion rcvServiceSMPVersion}
pcE <- getProtocolClient g NRMInteractive (1, srv, Nothing) oldCfg [] Nothing ts (\_ -> pure ())
pc <- either (fail . show) pure pcE
r <- runExceptT (directResolveName pc NRMInteractive (domain "alice.simplex"))
case r of
Left (PCETransportError TEVersion) -> pure ()
_ -> expectationFailure $ "expected Left (PCETransportError TEVersion), got: " <> show r
forwardedResolveAlice :: IO (Either SMPClientError (Either ProxyClientError SMP.NameRecord))
forwardedResolveAlice = do
g <- C.newRandom
ts <- getCurrentTime
let proxyServ = SMPServer testHost testPort testKeyHash
relayServ = SMPServer testHost2 testPort2 testKeyHash
cfg' = defaultSMPClientConfig {serverVRange = mkVersionRange minServerSMPRelayVersion currentClientSMPRelayVersion}
pcE <- getProtocolClient g NRMInteractive (1, proxyServ, Nothing) cfg' [] Nothing ts (\_ -> pure ())
pc <- either (fail . show) pure pcE
sess <- runExceptT' (connectSMPProxiedRelay pc NRMInteractive relayServ Nothing)
runExceptT (proxyResolveName pc NRMInteractive sess (domain "alice.simplex"))
testRslvForwarded :: IO ()
testRslvForwarded =
withProxyAndResolver (status404, "{}") $
forwardedResolveAlice >>= \r -> case r of
Left (PCEProtocolError (SMP.NAME SMP.NOT_FOUND)) -> pure ()
_ -> expectationFailure $ "expected Left (PCEProtocolError (NAME NOT_FOUND)), got: " <> show r
testRslvForwardedSuccess :: IO ()
testRslvForwardedSuccess =
withProxyAndResolver (status200, J.encode testNameRecord) $
forwardedResolveAlice >>= \r -> case r of
Right (Right nr) -> nr `shouldBe` testNameRecord
_ -> expectationFailure $ "expected Right (Right NameRecord), got: " <> show r
testRslvSuccess :: IO ()
testRslvSuccess =
withResolverServer (status200, J.encode testNameRecord) $
testSMPClient @TLS $ \h -> do
(corrId, _entId, resp) <- sendRslv h "rs07" (domain "alice.simplex")
corrId `shouldBe` CorrId "rs07"
case resp of
Right (RNAME nr) -> nr `shouldBe` testNameRecord
_ -> expectationFailure $ "expected Right (RNAME ..), got: " <> show resp
runExceptT' :: Show e => ExceptT e IO a -> IO a
runExceptT' a = runExceptT a >>= either (fail . show) pure
+65 -1
View File
@@ -1,7 +1,9 @@
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE DuplicateRecordFields #-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE TypeApplications #-}
module RemoteControl where
@@ -9,15 +11,23 @@ import AgentTests.FunctionalAPITests (runRight)
import Control.Logger.Simple
import Crypto.Random (ChaChaDRG)
import qualified Data.Aeson as J
import qualified Data.ByteString.Char8 as B
import qualified Data.ByteString.Lazy.Char8 as LB
import Data.List (stripPrefix)
import Data.List.NonEmpty (NonEmpty (..))
import Data.Time.Clock.System (SystemTime (..))
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Encoding.String (StrEncoding (..))
import Simplex.Messaging.Transport (TSbChainKeys (..))
import Simplex.Messaging.Transport.Client (TransportHost)
import qualified Simplex.RemoteControl.Client as HC (RCHostClient (action))
import qualified Simplex.RemoteControl.Client as RC
import Simplex.RemoteControl.Discovery (mkLastLocalHost, preferAddress)
import Simplex.RemoteControl.Invitation (RCSignedInvitation, verifySignedInvitation)
import Simplex.RemoteControl.Invitation
( RCInvitation (..),
RCSignedInvitation,
verifySignedInvitation,
)
import Simplex.RemoteControl.Types
import Test.Hspec hiding (fit, it)
import UnliftIO
@@ -27,6 +37,9 @@ import Util
remoteControlTests :: Spec
remoteControlTests = do
describe "preferred bindings should go first" testPreferAddress
describe "Invitation parsing" $ do
it "should parse bracketed IPv6 host with port" testInvitationBracketedIPv6Host
it "should reject bracketed non-IPv6 host" testInvitationBracketedNonIPv6HostRejected
describe "New controller/host pairing" $ do
it "should connect to new pairing" testNewPairing
it "should connect to existing pairing" testExistingPairing
@@ -65,6 +78,57 @@ testPreferAddress = do
addrsDups = "10.20.30.40" `on` "eth1" : addrs'
ifaceDups = "10.20.30.41" `on` "eth0" : addrs'
testInvitationBracketedIPv6Host :: IO ()
testInvitationBracketedIPv6Host = do
invitation <- testIPv6Invitation
let bracketedUri =
B.pack . replaceFirst "@2001:db8::1:" "@[2001:db8::1]:" . B.unpack $
strEncode invitation
expectedHost = either error id (strDecode "2001:db8::1") :: TransportHost
case strDecode bracketedUri of
Left err -> expectationFailure err
Right RCInvitation {host, port} -> do
host `shouldBe` expectedHost
port `shouldBe` 5223
testInvitationBracketedNonIPv6HostRejected :: IO ()
testInvitationBracketedNonIPv6HostRejected = do
invitation <- testIPv6Invitation
let bracketedUri =
B.pack . replaceFirst "@2001:db8::1:" "@[simplex.chat]:" . B.unpack $
strEncode invitation
case strDecode bracketedUri :: Either String RCInvitation of
Left _ -> pure ()
Right _ -> expectationFailure "expected parse failure for bracketed non-IPv6 host"
replaceFirst :: String -> String -> String -> String
replaceFirst needle replacement = go
where
go [] = []
go input@(c : cs) =
case stripPrefix needle input of
Just rest -> replacement <> rest
Nothing -> c : go cs
testIPv6Invitation :: IO RCInvitation
testIPv6Invitation = do
drg <- C.newRandom
(skey, _) <- atomically $ C.generateKeyPair @'C.Ed25519 drg
(idkey, _) <- atomically $ C.generateKeyPair @'C.Ed25519 drg
(dh, _) <- atomically $ C.generateKeyPair @'C.X25519 drg
pure
RCInvitation
{ ca = C.KeyHash "test-ca",
host = either error id $ strDecode "2001:db8::1",
port = 5223,
v = supportedRCPVRange,
app = J.String "app",
ts = MkSystemTime 0 0,
skey,
idkey,
dh
}
testNewPairing :: IO ()
testNewPairing = do
drg <- C.newRandom
+2 -2
View File
@@ -116,7 +116,7 @@ userServers :: NonEmpty (ProtocolServer p) -> Map UserId (NonEmpty (ServerCfg p)
userServers = userServers' . L.map noAuthSrv
userServers' :: NonEmpty (ProtoServerWithAuth p) -> Map UserId (NonEmpty (ServerCfg p))
userServers' srvs = M.fromList [(1, L.map (presetServerCfg True (ServerRoles True True) (Just 1)) srvs)]
userServers' srvs = M.fromList [(1, L.map (presetServerCfg True (ServerRoles True True True) (Just 1)) srvs)]
noAuthSrvCfg :: ProtocolServer p -> ServerCfg p
noAuthSrvCfg = presetServerCfg True (ServerRoles True True) (Just 1) . noAuthSrv
noAuthSrvCfg = presetServerCfg True (ServerRoles True True True) (Just 1) . noAuthSrv
+23 -2
View File
@@ -25,7 +25,7 @@ import Network.Socket
import qualified Network.TLS as TLS
import Simplex.Messaging.Agent.Store.Postgres.Options (DBOpts (..))
import Simplex.Messaging.Agent.Store.Shared (MigrationConfirmation (..))
import Simplex.Messaging.Client (ProtocolClientConfig (..), chooseTransportHost, defaultNetworkConfig)
import Simplex.Messaging.Client (NetworkConfig (..), NetworkTimeout (..), ProtocolClientConfig (..), chooseTransportHost, defaultNetworkConfig)
import Simplex.Messaging.Client.Agent (SMPClientAgentConfig (..), defaultSMPClientAgentConfig)
import qualified Simplex.Messaging.Crypto as C
import Simplex.Messaging.Encoding
@@ -275,9 +275,11 @@ cfgMS msType = withStoreCfg (testServerStoreConfig msType) $ \serverStoreCfg ->
smpServerVRange = supportedServerSMPRelayVRange,
transportConfig = mkTransportServerConfig True (Just alpnSupportedSMPHandshakes) True,
controlPort = Nothing,
smpAgentCfg = defaultSMPClientAgentConfig {persistErrorInterval = 1}, -- seconds
smpAgentCfg = defaultSMPClientAgentConfig {persistErrorInterval = 1, msgQSize = Nothing}, -- seconds
allowSMPProxy = False,
serverClientConcurrency = 2,
serverResolverConcurrency = defaultNameResolverConcurrency,
namesConfig = Nothing,
information = Nothing,
startOptions = defaultStartOptions
}
@@ -339,6 +341,16 @@ proxyCfgJ2QS = \case
SQSMemory -> journalCfg (proxyCfgMS $ ASType SQSMemory SMSJournal) testStoreLogFile2 testStoreMsgsDir2
SQSPostgres -> journalCfgDB (proxyCfgMS $ ASType SQSPostgres SMSJournal) testStoreDBOpts2 testStoreMsgsDir2
-- Proxy config with a short relay-connection timeout, to bound how long a failing
-- proxy->relay connection attempt blocks in the relay reconnection tests.
proxyCfgShortTimeout :: AServerConfig
proxyCfgShortTimeout =
updateCfg proxyCfg $ \cfg' ->
let aCfg = smpAgentCfg cfg'
cCfg = smpCfg aCfg
nt = NetworkTimeout {backgroundTimeout = 4_000000, interactiveTimeout = 4_000000}
in cfg' {smpAgentCfg = aCfg {smpCfg = cCfg {networkConfig = (networkConfig cCfg) {tcpConnectTimeout = nt}}}}
proxyVRangeV8 :: VersionRangeSMP
proxyVRangeV8 = mkVersionRange minServerSMPRelayVersion sendingProxySMPVersion
@@ -383,6 +395,15 @@ serverBracket process afterProcess f = do
Nothing -> error $ "server did not " <> s
_ -> pure ()
-- A TCP server that accepts connections but never performs a TLS handshake, so a client
-- connecting to it stays blocked in the TLS handshake until its connection timeout.
withStallingServerOn :: HasCallStack => ServiceName -> IO a -> IO a
withStallingServerOn port action =
serverBracket
(\started -> runLocalTCPServer started port (\_ -> threadDelay maxBound))
(pure ())
(const action)
withSmpServerOn :: HasCallStack => (ASrvTransport, AStoreType) -> ServiceName -> IO a -> IO a
withSmpServerOn ps port' = withSmpServerThreadOn ps port' . const
+251
View File
@@ -0,0 +1,251 @@
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE ScopedTypeVariables #-}
module SMPNamesTests (smpNamesTests, testNameRecord) where
import qualified Data.Aeson as J
import qualified Data.ByteString.Char8 as B
import qualified Data.ByteString.Lazy as LB
import Data.Either (isLeft, isRight)
import Data.IORef (readIORef)
import Data.List (sort)
import qualified Data.Text as T
import Data.Text.Encoding (encodeUtf8)
import Network.HTTP.Types (status200, status400, status404, status500, status502)
import NamesResolverServer (resolveResp, testNamesConfig, withResolverServer, withResolverServerDelayed)
import Simplex.Messaging.Encoding (smpDecode, smpEncode)
import Simplex.Messaging.Encoding.String (strDecode)
import Simplex.Messaging.Protocol (ErrorType (..), NameErrorType (..), NameRecord (..))
import Simplex.Messaging.Server.Main (validateUrl)
import Simplex.Messaging.Server.Names
( NamesConfig (..),
RpcAuth (..),
newNamesEnv,
pingEndpoint,
resolveName,
)
import Simplex.Messaging.Server.Names.HttpResolver (ResolverError (..))
import Simplex.Messaging.SimplexName (SimplexDomain (..), SimplexTLD (..))
import Test.Hspec
testNameRecord :: NameRecord
testNameRecord =
NameRecord
{ nrName = "alice.simplex",
nrNickname = "Alice",
nrWebsite = "https://alice.example",
nrLocation = "Earth",
nrSimplexContact = ["simplex:/contact/abc#xyz"],
nrSimplexChannel = [],
nrEth = Just "0x0000000000000000000000000000000000000001",
nrBtc = Nothing,
nrXmr = Nothing,
nrDot = Nothing,
nrOwner = "0x0101010101010101010101010101010101010101",
nrResolver = "0x0202020202020202020202020202020202020202"
}
smpNamesTests :: Spec
smpNamesTests = do
describe "NameRecord JSON (Protocol)" nameRecordEncodingSpec
describe "ErrorType NAME wire encoding" errorWireSpec
describe "Name parsing (SimplexDomain)" parseNameSpec
describe "HTTP resolver" resolverSpec
describe "Resolver health probe" healthSpec
describe "resolver_endpoint validation" validateUrlSpec
nameRecordEncodingSpec :: Spec
nameRecordEncodingSpec = do
it "round-trips JSON encode / decode" $
J.eitherDecodeStrict (LB.toStrict (J.encode testNameRecord)) `shouldBe` Right testNameRecord
it "emits keys in spec-documented order (resolver shape)" $ do
let bytes = LB.toStrict (J.encode testNameRecord)
offset k = B.length (fst (B.breakSubstring k bytes))
offsets =
map
offset
[ "name",
"nickname",
"website",
"location",
"simplexContact",
"simplexChannel",
"eth",
"btc",
"xmr",
"dot",
"owner",
"resolver"
]
offsets `shouldBe` sort offsets
it "emits unset coin fields as null (not absent)" $ do
let bytes = LB.toStrict (J.encode testNameRecord)
B.isInfixOf "\"btc\":null" bytes `shouldBe` True
B.isInfixOf "\"xmr\":null" bytes `shouldBe` True
B.isInfixOf "\"dot\":null" bytes `shouldBe` True
it "emits unset link fields as empty arrays (not null)" $ do
let bytes = LB.toStrict (J.encode testNameRecord)
B.isInfixOf "\"simplexChannel\":[]" bytes `shouldBe` True
B.isInfixOf "\"simplexChannel\":null" bytes `shouldBe` False
errorWireSpec :: Spec
errorWireSpec =
it "ErrorType NAME family round-trips smpEncode / smpDecode" $ do
smpDecode (smpEncode (NAME NO_RESOLVER)) `shouldBe` Right (NAME NO_RESOLVER)
smpDecode (smpEncode (NAME NOT_FOUND)) `shouldBe` Right (NAME NOT_FOUND)
-- RESOLVER detail may contain spaces - must survive the round-trip
smpDecode (smpEncode (NAME (RESOLVER "HTTP 502"))) `shouldBe` Right (NAME (RESOLVER "HTTP 502"))
parseNameSpec :: Spec
parseNameSpec = do
it "accepts a valid simplex-TLD name" $
case parseN "privacy.simplex" of
Right d -> do
nameTLD d `shouldBe` TLDSimplex
domain d `shouldBe` "privacy"
Left e -> expectationFailure ("expected Right, got Left " <> e)
it "normalises case across labels (Alice.SIMPLEX = alice.simplex)" $
parseN "alice.simplex" `shouldBe` parseN "Alice.SIMPLEX"
it "accepts a testing-TLD name" $
case parseN "bob.testing" of
Right d -> nameTLD d `shouldBe` TLDTesting
Left e -> expectationFailure ("expected Right, got Left " <> e)
it "accepts a TLDWeb name (server forwards to resolver, which will likely 404/400)" $
parseN "example.com" `shouldSatisfy` isRight
it "rejects a bare (no-TLD) name" $
parseN "privacy" `shouldSatisfy` isLeft
it "rejects non-ASCII labels (homograph attacks)" $
parseN "\1072lice.simplex" `shouldSatisfy` isLeft
it "rejects oversized inputs (>253 bytes)" $
parseN (T.replicate 254 "a" <> ".simplex") `shouldSatisfy` isLeft
it "rejects a label longer than 63 bytes (DNS label limit)" $
parseN (T.replicate 64 "a" <> ".simplex") `shouldSatisfy` isLeft
it "accepts a label of exactly 63 bytes" $
parseN (T.replicate 63 "a" <> ".simplex") `shouldSatisfy` isRight
where
parseN :: T.Text -> Either String SimplexDomain
parseN = strDecode . encodeUtf8
resolverSpec :: Spec
resolverSpec = do
it "returns NameRecord on 200 OK" $
withResolverServer (resolveResp status200 (J.encode testNameRecord)) $ \port _ -> do
env <- newNamesEnv (testNamesConfig port)
resolveName env aliceDomain `shouldReturn` Right testNameRecord
it "returns NOT_FOUND on 404" $
withResolverServer (resolveResp status404 "{}") $ \port _ -> do
env <- newNamesEnv (testNamesConfig port)
resolveName env aliceDomain `shouldReturn` Left NOT_FOUND
it "returns NOT_FOUND on 400 (unknown TLD)" $
withResolverServer (resolveResp status400 "{}") $ \port _ -> do
env <- newNamesEnv (testNamesConfig port)
resolveName env aliceDomain `shouldReturn` Left NOT_FOUND
it "returns RESOLVER on 502 (upstream failure)" $
withResolverServer (resolveResp status502 "{}") $ \port _ -> do
env <- newNamesEnv (testNamesConfig port)
resolveName env aliceDomain `shouldReturn` Left (RESOLVER "HTTP 502")
it "returns RESOLVER when the body exceeds the response cap" $
withResolverServer (resolveResp status200 (LB.fromStrict (B.replicate 500 'x'))) $ \port _ -> do
env <- newNamesEnv (testNamesConfig port) {resolverMaxResponseBytes = 100}
resolveName env aliceDomain `shouldReturn` Left (RESOLVER "response too large")
it "returns RESOLVER on malformed JSON from the resolver" $
withResolverServer (resolveResp status200 "this is not json") $ \port _ -> do
env <- newNamesEnv (testNamesConfig port)
resolveName env aliceDomain `shouldReturn` Left (RESOLVER "invalid response")
it "returns RESOLVER when JSON parses but isn't a NameRecord shape" $
withResolverServer (resolveResp status200 "{}") $ \port _ -> do
env <- newNamesEnv (testNamesConfig port)
resolveName env aliceDomain `shouldReturn` Left (RESOLVER "invalid response")
it "returns RESOLVER (timeout) when the resolver is slower than resolverTimeoutMs" $
withResolverServerDelayed 1500 (resolveResp status200 (J.encode testNameRecord)) $ \port _ -> do
env <- newNamesEnv (testNamesConfig port) {resolverTimeoutMs = 300}
resolveName env aliceDomain `shouldReturn` Left (RESOLVER "timeout")
it "sends one HTTP request per lookup (no cache)" $
withResolverServer (resolveResp status200 (J.encode testNameRecord)) $ \port reqs -> do
env <- newNamesEnv (testNamesConfig port)
_ <- resolveName env aliceDomain
_ <- resolveName env aliceDomain
readIORef reqs >>= \rs -> length rs `shouldBe` 2
it "addresses the resolver with the full canonical domain name" $
withResolverServer (resolveResp status200 (J.encode testNameRecord)) $ \port reqs -> do
env <- newNamesEnv (testNamesConfig port)
_ <- resolveName env aliceDomain
readIORef reqs `shouldReturn` [["resolve", "alice.simplex"]]
where
aliceDomain = SimplexDomain {nameTLD = TLDSimplex, domain = "alice", subDomain = []}
healthSpec :: Spec
healthSpec = do
it "pingEndpoint succeeds on a 200 OK /health response" $
withResolverServer (resolveResp status200 "{}") $ \port _ -> do
env <- newNamesEnv (testNamesConfig port)
pingEndpoint env >>= \case
Right () -> pure ()
Left e -> expectationFailure $ "expected Right (), got Left " <> show e
it "pingEndpoint fails on a 500 /health response" $
withResolverServer healthFails $ \port _ -> do
env <- newNamesEnv (testNamesConfig port)
pingEndpoint env >>= \case
Left (HttpStatusErr 500) -> pure ()
r -> expectationFailure $ "expected Left (HttpStatusErr 500), got " <> show r
it "pingEndpoint queries /health" $
withResolverServer (resolveResp status200 "{}") $ \port reqs -> do
env <- newNamesEnv (testNamesConfig port)
_ <- pingEndpoint env
readIORef reqs `shouldReturn` [["health"]]
where
healthFails = \case
["health"] -> (status500, "{}")
_ -> (status404, "{}")
validateUrlSpec :: Spec
validateUrlSpec = do
it "accepts an https URL with a path prefix" $
validateUrl "https://gw.example.com:443/snrc" Nothing `shouldSatisfy` isRight
it "accepts an http URL" $
validateUrl "http://127.0.0.1:8000" Nothing `shouldSatisfy` isRight
it "accepts a URL without an explicit port" $
validateUrl "https://gw.example.com/snrc" Nothing `shouldSatisfy` isRight
it "rejects a relative / non-absolute URI" $
validateUrl "gw.example.com/snrc" Nothing `shouldSatisfy` isLeft
it "rejects a non-http(s) scheme" $
validateUrl "ftp://gw.example.com:21" Nothing `shouldSatisfy` isLeft
it "rejects an empty host" $
validateUrl "http://" Nothing `shouldSatisfy` isLeft
it "accepts https with auth (Authorization is TLS-protected)" $
validateUrl "https://gw.example.com" (Just auth) `shouldSatisfy` isRight
it "accepts loopback http with auth (no cleartext exposure)" $
validateUrl "http://localhost:8000" (Just auth) `shouldSatisfy` isRight
it "rejects non-loopback http with auth (cleartext credential leak)" $
validateUrl "http://gw.example.com:8000" (Just auth) `shouldSatisfy` isLeft
it "rejects URL-embedded userinfo (credentials belong in resolver_auth)" $
validateUrl "https://user:pass@gw.example.com" Nothing `shouldSatisfy` isLeft
it "rejects http+auth to a 127.-prefixed non-loopback host (not real loopback)" $
validateUrl "http://127.evil.com:8000" (Just auth) `shouldSatisfy` isLeft
where
auth = AuthBasic "user" "pass"
+91
View File
@@ -58,6 +58,14 @@ smpProxyTests = do
describe "server configuration" $ do
it "refuses proxy handshake unless enabled" testNoProxy
it "checks basic auth in proxy requests" testProxyAuth
describe "relay reconnection" $ do
it "recovers when unresponsive relay restarts (control, no disconnect)" $ \_ ->
testProxyRecoversWithoutDisconnect
it "reconnects to relay after sender disconnects mid-connection" $ \_ ->
testProxyReconnectAfterRelayRestart
describe "agent client reconnection" $ do
it "reconnects after a connect is cancelled mid-flight" $ \_ ->
testAgentClientReconnectAfterCancel
describe "proxy requests" $ do
describe "bad relay URIs" $ do
xit "host not resolved" todo
@@ -447,6 +455,89 @@ testProxyAuth msType = do
where
proxyCfgAuth = updateCfg (proxyCfgMS msType) $ \cfg_ -> cfg_ {newQueueBasicAuth = Just "correct"}
-- Connect a sender client to the proxy and request a relay session to testSMPServer2 (PRXY).
-- On success the reply is PKEY; otherwise it is the proxy error for the relay connection.
requestRelaySession :: IO (Either SMP.ErrorType SMP.BrokerMsg)
requestRelaySession =
testSMPClient_ "localhost" testPort proxyVRangeV8 Nothing $ \(th :: THandleSMP TLS 'TClient) ->
(\(_, _, reply) -> reply) <$> sendRecv th (Nothing, "1", NoEntity, SMP.PRXY testSMPServer2 Nothing)
-- Shared "phase 2" of the reconnection tests: start a healthy relay, confirm it is reachable
-- directly (PING, not via the proxy) so a proxy failure can only mean the proxy didn't reconnect,
-- let any stored connection error expire, then require the proxy to establish the session (PKEY).
requireProxyReconnect :: IO ()
requireProxyReconnect =
withSmpServerConfigOn (transport @TLS) proxyCfgJ2 testPort2 $ \_ -> do
testSMPClient_ "127.0.0.1" testPort2 proxyVRangeV8 Nothing $ \(th :: THandleSMP TLS 'TClient) -> do
(_, _, reply) <- sendRecv th (Nothing, "0", NoEntity, SMP.PING)
reply `shouldBe` Right SMP.PONG
threadDelay 1500000 -- > persistErrorInterval (1s), so the stored connection error has expired
requestRelaySession >>= \case
Right SMP.PKEY {} -> pure ()
reply -> expectationFailure $ "proxy failed to reach the healthy relay; expected PKEY, got: " <> show reply
-- Control: same stalling relay and proxy config as the bug test, but the sender stays connected.
-- The connect fails by timing out (storing a Left error that self-heals via persistErrorInterval),
-- so once a healthy relay is running the proxy reconnects. This proves the stalling relay alone
-- does not cause the permanent failure - only the mid-connection disconnect does.
testProxyRecoversWithoutDisconnect :: IO ()
testProxyRecoversWithoutDisconnect =
withSmpServerConfigOn (transport @TLS) proxyCfgShortTimeout testPort $ \_ -> do
withStallingServerOn testPort2 $
requestRelaySession >>= \case
Right (SMP.ERR (SMP.PROXY (SMP.BROKER _))) -> pure ()
reply -> expectationFailure $ "expected a proxy broker error from the unresponsive relay, got: " <> show reply
requireProxyReconnect
-- Reproduces the production bug: an SMP proxy permanently fails to reconnect to a destination
-- relay after the relay restarts (logs: repeated PCEResponseTimeout).
--
-- A PRXY request makes the proxy worker (forked via forkClient, registered in the sender's
-- endThreads) insert an empty SessionVar into smpClients and then block in connectClient. If the
-- sender disconnects while that connect is in flight, clientDisconnected kills the worker;
-- clientHandlers re-throws the async exception, so the SessionVar is never filled. Nothing removes
-- an empty SessionVar, so every later request waits the connection timeout on it - PROXY (BROKER
-- TIMEOUT) - forever, even once the relay is healthy again.
--
-- The stalling relay (accepts TCP, never completes TLS) holds the connect open long enough to
-- interleave the disconnect. Phase 2 (requireProxyReconnect) is identical to the control above;
-- the only difference is this disconnect.
testProxyReconnectAfterRelayRestart :: IO ()
testProxyReconnectAfterRelayRestart =
withSmpServerConfigOn (transport @TLS) proxyCfgShortTimeout testPort $ \_ -> do
-- disconnect the sender 1s into the 4s connect to the stalling relay, killing the in-flight worker
withStallingServerOn testPort2 $
race_ (threadDelay 1000000) requestRelaySession
requireProxyReconnect
-- Bug B (same root cause as the proxy, in the messaging agent): getSMPServerClient inserts an
-- empty SessionVar into smpClients, then connects inside newProtocolClient's tryAllErrors, which
-- rethrows async exceptions. If the connecting thread is cancelled mid-connect, putTMVar is
-- skipped and the empty var is left in smpClients, so every later connection to that server times
-- out on it. Phase 1 cancels a connect to a stalling relay; phase 2 requires a fresh connect to a
-- healthy relay to succeed.
testAgentClientReconnectAfterCancel :: IO ()
testAgentClientReconnectAfterCancel =
withAgent 1 agentCfg agentServersLeak testDB $ \a -> do
withStallingServerOn testPort2 $ do
t <- async $ runExceptT $ A.createConnection a NRMInteractive 1 True True SCMInvitation Nothing Nothing CR.IKPQOn SMSubscribe
threadDelay 1000000 -- let the connect to the stalling relay start, then kill it mid-flight
cancel t
withSmpServerConfigOn (transport @TLS) cfgJ2 testPort2 $ \_ -> do
testSMPClient_ "127.0.0.1" testPort2 proxyVRangeV8 Nothing $ \(th :: THandleSMP TLS 'TClient) -> do
(_, _, reply) <- sendRecv th (Nothing, "0", NoEntity, SMP.PING)
reply `shouldBe` Right SMP.PONG -- the relay is up and reachable, so a timeout can only be the poisoned var
r <- timeout 8000000 $ runExceptT $ A.createConnection a NRMInteractive 1 True True SCMInvitation Nothing Nothing CR.IKPQOn SMSubscribe
case r of
Just (Right _) -> pure ()
_ -> expectationFailure $ "agent failed to connect after a cancelled connect; got: " <> show r
where
agentServersLeak =
initAgentServers
{ smp = userServers [testSMPServer2],
netCfg = (netCfg initAgentServers) {tcpConnectTimeout = NetworkTimeout 4000000 4000000}
}
todo :: AStoreType -> IO ()
todo _ = fail "TODO"
+50 -3
View File
@@ -110,6 +110,7 @@ serverTests = do
describe "Short links" $ do
testInvQueueLinkData
testContactQueueLinkData
testDuplicateQueueLinkData
pattern Resp :: CorrId -> QueueId -> BrokerMsg -> Transmission (Either ErrorType BrokerMsg)
pattern Resp corrId queueId command <- (corrId, queueId, Right command)
@@ -1100,7 +1101,7 @@ testRestoreMessages =
pure ()
rId <- readTVarIO recipientId
logSize testStoreLogFile `shouldReturn` 2
logSize testServerStatsBackupFile `shouldReturn` 95
logSize testServerStatsBackupFile `shouldReturn` 101
Right stats1 <- strDecode <$> B.readFile testServerStatsBackupFile
checkStats stats1 [rId] 5 1
withSmpServerConfigOn at cfg' testPort . runTest t $ \h -> do
@@ -1116,7 +1117,7 @@ testRestoreMessages =
logSize testStoreLogFile `shouldReturn` (if compacting then 1 else 2)
-- the last message is not removed because it was not ACK'd
-- logSize testStoreMsgsFile `shouldReturn` 3
logSize testServerStatsBackupFile `shouldReturn` 95
logSize testServerStatsBackupFile `shouldReturn` 101
Right stats2 <- strDecode <$> B.readFile testServerStatsBackupFile
checkStats stats2 [rId] 5 3
@@ -1134,7 +1135,7 @@ testRestoreMessages =
pure ()
logSize testStoreLogFile `shouldReturn` (if compacting then 1 else 2)
removeFile testStoreLogFile
logSize testServerStatsBackupFile `shouldReturn` 95
logSize testServerStatsBackupFile `shouldReturn` 101
Right stats3 <- strDecode <$> B.readFile testServerStatsBackupFile
checkStats stats3 [rId] 5 5
removeFileIfExists testStoreMsgsFile
@@ -1720,6 +1721,52 @@ testContactQueueLinkData =
Resp "12" lnkId3 (ERR AUTH) <- sendRecv s ("", "12", lnkId, LGET)
lnkId3 `shouldBe` lnkId
testDuplicateQueueLinkData :: SpecWith (ASrvTransport, AStoreType)
testDuplicateQueueLinkData =
it "rejects attaching an existing short link to another queue" $ \(ATransport t, msType) ->
smpTest2 t msType $ \r s -> do
g <- C.newRandom
(victimPub, victimKey) <- atomically $ C.generateAuthKeyPair C.SEd25519 g
(victimDhPub, _victimDhPriv :: C.PrivateKeyX25519) <- atomically $ C.generateKeyPair g
C.CbNonce corrId <- atomically $ C.randomCbNonce g
lnkId <- EntityId <$> atomically (C.randomBytes 24 g)
let victimSId = EntityId $ B.take 24 $ C.sha3_384 corrId
victimLD = (EncDataBytes "fixed data", EncDataBytes "victim user data")
victimQRD = QRContact $ Just (lnkId, (victimSId, victimLD))
victimReq = NEW (NewQueueReq victimPub victimDhPub Nothing SMSubscribe (Just victimQRD) Nothing)
Resp
_
NoEntity
(IDS QIK {sndId = victimSId', queueMode = Just QMContact, linkId = Just lnkId'}) <-
signSendRecv r victimKey (corrId, NoEntity, victimReq)
lnkId' `shouldBe` lnkId
victimSId' `shouldBe` victimSId
Resp "1" lnkId1 (LNK victimSId1 victimLD1) <- sendRecv s ("", "1", lnkId, LGET)
lnkId1 `shouldBe` lnkId
victimSId1 `shouldBe` victimSId
victimLD1 `shouldBe` victimLD
(attackerPub, attackerKey) <- atomically $ C.generateAuthKeyPair C.SEd25519 g
(attackerDhPub, _attackerDhPriv :: C.PrivateKeyX25519) <- atomically $ C.generateKeyPair g
let attackerReq =
NEW (NewQueueReq attackerPub attackerDhPub Nothing SMSubscribe (Just $ QRContact Nothing) Nothing)
Resp
"2"
NoEntity
(IDS QIK {rcvId = attackerRId, queueMode = Just QMContact, linkId = Nothing}) <-
signSendRecv r attackerKey ("2", NoEntity, attackerReq)
let attackerLD = (EncDataBytes "fixed data", EncDataBytes "attacker user data")
Resp "3" attackerRId' (ERR _) <-
signSendRecv r attackerKey ("3", attackerRId, LSET lnkId attackerLD)
attackerRId' `shouldBe` attackerRId
Resp "4" lnkId2 (LNK victimSId2 victimLD2) <- sendRecv s ("", "4", lnkId, LGET)
lnkId2 `shouldBe` lnkId
victimSId2 `shouldBe` victimSId
victimLD2 `shouldBe` victimLD
samplePubKey :: C.APublicVerifyKey
samplePubKey = C.APublicVerifyKey C.SEd25519 "MCowBQYDK2VwAyEAfAOflyvbJv1fszgzkQ6buiZJVgSpQWsucXq7U6zjMgY="
+4
View File
@@ -21,7 +21,9 @@ import CoreTests.VersionRangeTests
import FileDescriptionTests (fileDescriptionTests)
import GHC.IO.Exception (IOException (..))
import qualified GHC.IO.Exception as IOException
import RSLVTests (rslvTests)
import RemoteControl (remoteControlTests)
import SMPNamesTests (smpNamesTests)
import SMPProxyTests (smpProxyTests)
import ServerTests
import Simplex.Messaging.Server.Env.STM (AStoreType (..))
@@ -97,6 +99,8 @@ main = do
#endif
describe "TSessionSubs tests" tSessionSubsTests
describe "Util tests" utilTests
describe "Names resolver tests" smpNamesTests
describe "RSLV functional API tests" rslvTests
describe "Agent core tests" agentCoreTests
#if defined(dbServerPostgres)
around_ (postgressBracket testServerDBConnectInfo) $
+16 -3
View File
@@ -1,14 +1,19 @@
module XFTPCLI (xftpCLIFileTests, xftpCLI, senderFiles, recipientFiles, testBracket) where
import Control.Exception (bracket_)
import Control.Exception (bracket_, try)
import qualified Data.ByteString as LB
import Data.List (isInfixOf, isPrefixOf, isSuffixOf)
import Simplex.FileTransfer.Client.Main (prepareChunkSizes, xftpClientCLI)
import Simplex.FileTransfer.Client.Main
( prepareChunkSizes,
xftpClientCLI,
xftpClientDeprecationNotice,
)
import Simplex.FileTransfer.Description (kb, mb)
import System.Directory (createDirectoryIfMissing, getFileSize, listDirectory, removeDirectoryRecursive)
import System.Environment (withArgs)
import System.Exit (ExitCode (ExitSuccess))
import System.FilePath ((</>))
import System.IO.Silently (capture_)
import System.IO.Silently (capture, capture_)
import Test.Hspec hiding (fit, it)
import Util
import Simplex.FileTransfer.Server.Env (AFStoreType)
@@ -16,6 +21,8 @@ import XFTPClient (cfgFS, cfgFS2, withXFTPServer, withXFTPServerConfigOn, testXF
xftpCLIFileTests :: SpecWith AFStoreType
xftpCLIFileTests = around_ testBracket $ do
it "shows experimental deprecation notice in help" $ \_ ->
testXFTPCLIHelpDeprecationNotice
it "should send and receive file" $ withXFTPServer testXFTPCLISendReceive_
it "should send and receive file with 2 servers" $ \fsType ->
withXFTPServerConfigOn (cfgFS fsType) $ \_ -> withXFTPServerConfigOn (cfgFS2 fsType) $ \_ -> testXFTPCLISendReceive2servers_
@@ -40,6 +47,12 @@ recipientFiles = "tests/tmp/xftp-recipient-files"
xftpCLI :: [String] -> IO [String]
xftpCLI params = lines <$> capture_ (withArgs params xftpClientCLI)
testXFTPCLIHelpDeprecationNotice :: IO ()
testXFTPCLIHelpDeprecationNotice = do
(output, result) <- capture $ try $ withArgs ["--help"] xftpClientCLI
result `shouldBe` (Left ExitSuccess :: Either ExitCode ())
unwords (words output) `shouldSatisfy` (xftpClientDeprecationNotice `isInfixOf`)
testXFTPCLISendReceive_ :: IO ()
testXFTPCLISendReceive_ = do
let filePath = senderFiles </> "testfile"
+1 -1
View File
@@ -9,7 +9,6 @@
module XFTPClient where
import Control.Concurrent (ThreadId, threadDelay)
import Control.Monad (void)
import Data.String (fromString)
import Data.Time.Clock (getCurrentTime)
import Network.Socket (ServiceName)
@@ -25,6 +24,7 @@ import Simplex.Messaging.Transport.HTTP2 (httpALPN)
import Simplex.Messaging.Transport.Server
import Test.Hspec hiding (fit, it)
#if defined(dbServerPostgres)
import Control.Monad (void)
import qualified Database.PostgreSQL.Simple as PSQL
import Database.PostgreSQL.Simple (ConnectInfo (..), defaultConnectInfo)
import Simplex.FileTransfer.Server.Store.Postgres.Config (PostgresFileStoreCfg (..), defaultXFTPDBOpts)
+18 -1
View File
@@ -14,7 +14,7 @@
module XFTPWebTests (xftpWebTests) where
import Control.Concurrent (forkIO, newEmptyMVar, putMVar, takeMVar)
import Control.Monad (replicateM, when)
import Control.Monad (forM_, replicateM, when)
import Crypto.Error (throwCryptoError)
import qualified Crypto.PubKey.Curve25519 as X25519
import qualified Crypto.PubKey.Ed25519 as Ed25519
@@ -170,6 +170,7 @@ jsOut expr = "process.stdout.write(Buffer.from(" <> expr <> "));"
xftpWebTests :: IO () -> Spec
xftpWebTests dbCleanup = do
xftpWebSourceHygieneTests
distExists <- runIO $ doesDirectoryExist (xftpWebDir <> "/dist")
if distExists
then do
@@ -193,6 +194,22 @@ xftpWebTests dbCleanup = do
it "skipped (run 'cd xftp-web && npm install && npm run build' first)" $
pendingWith "TS project not compiled"
xftpWebSourceHygieneTests :: Spec
xftpWebSourceHygieneTests = describe "source hygiene" $
it "does not commit XFTP web debug logs that expose secrets or plaintext" $ do
let files =
[ "xftp-web/src/client.ts",
"xftp-web/src/agent.ts",
"xftp-web/web/crypto-backend.ts",
"xftp-web/web/crypto.worker.ts",
"apps/xftp-server/static/xftp-web-bundle/index.js",
"apps/xftp-server/static/xftp-web-bundle/crypto.worker.js"
]
markers = ["XFTP-DBG", "AGENT-DBG", "BACKEND-DBG", "WORKER-DBG"]
forM_ files $ \path -> do
contents <- B.readFile path
mapM_ (\marker -> contents `shouldNotSatisfy` B.isInfixOf marker) markers
-- ── protocol/encoding ──────────────────────────────────────────────
tsEncodingTests :: Spec
-8
View File
@@ -90,7 +90,6 @@ export function encryptFileForUpload(source: Uint8Array, fileName: string): Encr
const encSize = BigInt(chunkSizes.reduce((a, b) => a + b, 0))
const encData = encryptFile(source, fileHdr, key, nonce, fileSize, encSize)
const digest = sha512Streaming([encData])
console.log(`[AGENT-DBG] encrypt: encData.len=${encData.length} digest=${_dbgHex(digest, 64)} chunkSizes=[${chunkSizes.join(',')}]`)
return {encData, digest, key, nonce, chunkSizes}
}
@@ -280,9 +279,7 @@ export async function downloadFileRaw(
const {onProgress, concurrency = 1} = options ?? {}
// Resolve redirect on main thread (redirect data is small)
if (fd.redirect !== null) {
console.log(`[AGENT-DBG] resolving redirect: outer size=${fd.size} chunks=${fd.chunks.length}`)
fd = await resolveRedirect(agent, fd)
console.log(`[AGENT-DBG] resolved: size=${fd.size} chunks=${fd.chunks.length} digest=${Array.from(fd.digest.slice(0, 16)).map(x => x.toString(16).padStart(2, '0')).join('')}`)
}
const resolvedFd = fd
// Group chunks by server, sequential within each server, parallel across servers
@@ -301,7 +298,6 @@ export async function downloadFileRaw(
const seed = decodePrivKeyEd25519(replica.replicaKey)
const kp = ed25519KeyPairFromSeed(seed)
const raw = await downloadXFTPChunkRaw(agent, server, kp.privateKey, replica.replicaId)
console.log(`[AGENT-DBG] chunk=${chunk.chunkNo} body.len=${raw.body.length} expectedChunkSize=${chunk.chunkSize} digest=${_dbgHex(chunk.digest, 32)} body.byteOffset=${raw.body.byteOffset} body.buffer.byteLength=${raw.body.buffer.byteLength}`)
await onRawChunk({
chunkNo: chunk.chunkNo,
dhSecret: raw.dhSecret,
@@ -377,10 +373,6 @@ export async function deleteFile(agent: XFTPClientAgent, sndDescription: FileDes
// -- Internal
function _dbgHex(b: Uint8Array, n = 8): string {
return Array.from(b.slice(0, n)).map(x => x.toString(16).padStart(2, '0')).join('')
}
function digestEqual(a: Uint8Array, b: Uint8Array): boolean {
if (a.length !== b.length) return false
let diff = 0
-10
View File
@@ -311,14 +311,12 @@ async function sendXFTPCommandOnce(
const block = encodeAuthTransmission(client.sessionId, corrId, entityId, cmdBytes, privateKey)
const reqBody = chunkData ? concatBytes(block, chunkData) : block
const fullResp = await client.transport.post(reqBody)
console.log(`[XFTP-DBG] sendOnce: fullResp.length=${fullResp.length} entityId=${_hex(entityId)} cmdTag=${cmdBytes[0]}`)
if (fullResp.length < XFTP_BLOCK_SIZE) {
console.error('[XFTP] Response too short: %d bytes (expected >= %d)', fullResp.length, XFTP_BLOCK_SIZE)
throw new Error("Server response too short")
}
const respBlock = fullResp.subarray(0, XFTP_BLOCK_SIZE)
const body = fullResp.subarray(XFTP_BLOCK_SIZE)
console.log(`[XFTP-DBG] sendOnce: body.length=${body.length} body.byteOffset=${body.byteOffset} body.buffer.byteLength=${body.buffer.byteLength}`)
// Detect padded error strings (HANDSHAKE, SESSION) before decodeTransmission
const raw = blockUnpad(respBlock)
if (raw.length < 20) {
@@ -339,10 +337,6 @@ async function sendXFTPCommandOnce(
return {response, body}
}
function _hex(b: Uint8Array, n = 8): string {
return Array.from(b.slice(0, n)).map(x => x.toString(16).padStart(2, '0')).join('')
}
// -- Send command (with retry + reconnect)
export async function sendXFTPCommand(
@@ -358,10 +352,8 @@ export async function sendXFTPCommand(
let client = await clientP
for (let attempt = 1; attempt <= maxRetries; attempt++) {
try {
if (attempt > 1) console.log(`[XFTP-DBG] sendCmd: retry attempt=${attempt}/${maxRetries}`)
return await sendXFTPCommandOnce(client, privateKey, entityId, cmdBytes, chunkData)
} catch (e) {
console.log(`[XFTP-DBG] sendCmd: attempt=${attempt} failed: ${e instanceof Error ? e.message : String(e)} retriable=${isRetriable(e)}`)
if (!isRetriable(e)) {
throw categorizeError(e)
}
@@ -416,7 +408,6 @@ export async function downloadXFTPChunkRaw(
const {response, body} = await sendXFTPCommand(agent, server, rpKey, fId, cmd)
if (response.type !== "FRFile") throw new Error("unexpected response: " + response.type)
const dhSecret = dh(response.rcvDhKey, privateKey)
console.log(`[XFTP-DBG] dlChunkRaw: body.length=${body.length} nonce=${_hex(response.nonce, 24)} dhSecret=${_hex(dhSecret)} body[0..8]=${_hex(body)} body[-8..]=${_hex(body.slice(-8))}`)
return {dhSecret, nonce: response.nonce, body}
}
@@ -444,4 +435,3 @@ export async function pingXFTP(agent: XFTPClientAgent, server: XFTPServer): Prom
const response = decodeResponse(command)
if (response.type !== "FRPong") throw new Error("unexpected response: " + response.type)
}
+17 -1
View File
@@ -35,6 +35,22 @@ export function parseXFTPServer(address: string): XFTPServer {
const hostPart = m[2]
// Take the first host (before any comma), then split port from that
const firstHost = hostPart.split(',')[0]
return {keyHash, ...parseHostPort(firstHost)}
}
function parseHostPort(firstHost: string): Pick<XFTPServer, "host" | "port"> {
if (firstHost.length === 0) throw new Error("parseXFTPServer: missing host")
if (firstHost.startsWith('[')) {
const bracketEnd = firstHost.indexOf(']')
if (bracketEnd < 0) throw new Error("parseXFTPServer: invalid bracketed host")
const host = firstHost.substring(0, bracketEnd + 1)
const rest = firstHost.substring(bracketEnd + 1)
if (rest.length === 0) return {host, port: "443"}
if (!rest.startsWith(':')) throw new Error("parseXFTPServer: invalid bracketed host")
const port = rest.substring(1)
if (port.length === 0) throw new Error("parseXFTPServer: missing port")
return {host, port}
}
const colonIdx = firstHost.lastIndexOf(':')
let host: string
let port: string
@@ -45,7 +61,7 @@ export function parseXFTPServer(address: string): XFTPServer {
host = firstHost
port = "443"
}
return {keyHash, host, port}
return {host, port}
}
// Format an XFTPServer back to its URI string representation.
+21
View File
@@ -0,0 +1,21 @@
import {expect, test} from 'vitest'
import {formatXFTPServer, parseXFTPServer, serverOrigin} from '../src/protocol/address.js'
const keyHash = 'LcJUMfVhwD8yxjAiSaDzzGF3-kLG4Uh0Fl_ZIjrRwjI='
test('parseXFTPServer supports bracketed IPv6 hosts with ports', () => {
const server = parseXFTPServer(`xftp://${keyHash}@[2001:db8::1]:8443,example.com`)
expect(server.host).toBe('[2001:db8::1]')
expect(server.port).toBe('8443')
expect(serverOrigin(server)).toBe('https://[2001:db8::1]:8443')
expect(formatXFTPServer(server)).toBe(`xftp://${keyHash}@[2001:db8::1]:8443`)
})
test('parseXFTPServer uses the default port for bracketed IPv6 hosts', () => {
const server = parseXFTPServer(`xftp://${keyHash}@[2001:db8::1]`)
expect(server.host).toBe('[2001:db8::1]')
expect(server.port).toBe('443')
expect(serverOrigin(server)).toBe('https://[2001:db8::1]')
})
-5
View File
@@ -108,11 +108,6 @@ class WorkerBackend implements CryptoBackend {
const nonceCopy = new Uint8Array(nonce)
const digestCopy = new Uint8Array(digest)
const buf = this.toTransferable(body)
const hex = (b: Uint8Array | ArrayBuffer, n = 8) => {
const u = b instanceof ArrayBuffer ? new Uint8Array(b) : b
return Array.from(u.slice(0, n)).map(x => x.toString(16).padStart(2, '0')).join('')
}
console.log(`[BACKEND-DBG] chunk=${chunkNo} body.len=${body.length} body.byteOff=${body.byteOffset} buf.byteLen=${buf.byteLength} nonce=${hex(nonceCopy, 24)} dhSecret=${hex(dhSecretCopy)} digest=${hex(digestCopy, 32)} buf[0..8]=${hex(buf)} body[-8..]=${hex(body.slice(-8))}`)
await this.send(
{type: 'decryptAndStoreChunk', dhSecret: dhSecretCopy, nonce: nonceCopy, body: buf, chunkDigest: digestCopy, chunkNo},
[buf]
+3 -34
View File
@@ -59,7 +59,6 @@ async function handleEncrypt(id: number, data: ArrayBuffer, fileName: string) {
const digest = sha512Streaming([encData], (done) => {
self.postMessage({id, type: 'progress', done: source.length + done, total})
}, encDataLen)
console.log(`[WORKER-DBG] encrypt: encData.len=${encData.length} digest=${_whex(digest, 64)} chunkSizes=[${chunkSizes.join(',')}]`)
// Write to OPFS
const dir = await getSessionDir()
@@ -93,9 +92,7 @@ async function handleDecryptAndStore(
body: ArrayBuffer, chunkDigest: Uint8Array, chunkNo: number
) {
const bodyArr = new Uint8Array(body)
console.log(`[WORKER-DBG] store chunk=${chunkNo} body.len=${bodyArr.length} nonce=${_whex(nonce, 24)} dhSecret=${_whex(dhSecret)} digest=${_whex(chunkDigest, 32)} body[0..8]=${_whex(bodyArr)} body[-8..]=${_whex(bodyArr.slice(-8))}`)
const decrypted = decryptReceivedChunk(dhSecret, nonce, bodyArr, chunkDigest)
console.log(`[WORKER-DBG] decrypted chunk=${chunkNo} len=${decrypted.length} [0..8]=${_whex(decrypted)} [-8..]=${_whex(decrypted.slice(-8))}`)
if (useMemory) {
memoryChunks.set(chunkNo, decrypted)
@@ -113,7 +110,6 @@ async function handleDecryptAndStore(
currentDownloadOffset += decrypted.length
chunkMeta.set(chunkNo, {offset, size: decrypted.length})
const written = downloadWriteHandle.write(decrypted, {at: offset})
console.log(`[WORKER-DBG] OPFS write chunk=${chunkNo} offset=${offset} size=${decrypted.length} written=${written}`)
if (written !== decrypted.length) {
console.warn(`[WORKER] OPFS write failed chunk=${chunkNo}: ${written}/${decrypted.length}, falling back to in-memory storage`)
@@ -140,21 +136,12 @@ async function handleDecryptAndStore(
downloadWriteHandle.flush()
// Verify: read back and compare first/last 8 bytes
const verifyBuf = new Uint8Array(Math.min(8, decrypted.length))
downloadWriteHandle.read(verifyBuf, {at: offset})
const verifyEnd = new Uint8Array(Math.min(8, decrypted.length))
downloadWriteHandle.read(verifyEnd, {at: offset + decrypted.length - verifyEnd.length})
console.log(`[WORKER-DBG] OPFS verify chunk=${chunkNo} readBack[0..8]=${_whex(verifyBuf)} readBack[-8..]=${_whex(verifyEnd)} expected[0..8]=${_whex(decrypted)} expected[-8..]=${_whex(decrypted.slice(-8))}`)
self.postMessage({id, type: 'stored'})
}
async function handleVerifyAndDecrypt(
id: number, size: number, digest: Uint8Array, key: Uint8Array, nonce: Uint8Array
) {
console.log(`[WORKER-DBG] verify: expectedSize=${size} expectedDigest=${_whex(digest, 64)} useMemory=${useMemory} chunkMeta.size=${chunkMeta.size} memoryChunks.size=${memoryChunks.size}`)
// Read chunks — from memory (fallback) or OPFS
const chunks: Uint8Array[] = []
let totalSize = 0
@@ -162,8 +149,7 @@ async function handleVerifyAndDecrypt(
let done = 0
if (useMemory) {
const sorted = [...memoryChunks.entries()].sort((a, b) => a[0] - b[0])
for (const [chunkNo, data] of sorted) {
console.log(`[WORKER-DBG] verify memory chunk=${chunkNo} size=${data.length}`)
for (const [, data] of sorted) {
chunks.push(data)
totalSize += data.length
done += data.length
@@ -179,12 +165,10 @@ async function handleVerifyAndDecrypt(
const dir = await getSessionDir()
const fileHandle = await dir.getFileHandle('download.bin')
const readHandle = await fileHandle.createSyncAccessHandle()
console.log(`[WORKER-DBG] verify: OPFS file size=${readHandle.getSize()}`)
const sortedEntries = [...chunkMeta.entries()].sort((a, b) => a[0] - b[0])
for (const [chunkNo, meta] of sortedEntries) {
for (const [, meta] of sortedEntries) {
const buf = new Uint8Array(meta.size)
const bytesRead = readHandle.read(buf, {at: meta.offset})
console.log(`[WORKER-DBG] verify read chunk=${chunkNo} offset=${meta.offset} size=${meta.size} bytesRead=${bytesRead} [0..8]=${_whex(buf)} [-8..]=${_whex(buf.slice(-8))}`)
readHandle.read(buf, {at: meta.offset})
chunks.push(buf)
totalSize += meta.size
done += meta.size
@@ -212,20 +196,9 @@ async function handleVerifyAndDecrypt(
}
const actualDigest = sodium.crypto_hash_sha512_final(state)
if (!digestEqual(actualDigest, digest)) {
console.error(`[WORKER-DBG] DIGEST MISMATCH: expected=${_whex(digest, 64)} actual=${_whex(actualDigest, 64)} chunks=${chunks.length} totalSize=${totalSize}`)
const state2 = sodium.crypto_hash_sha512_init() as unknown as import('libsodium-wrappers').StateAddress
for (let i = 0; i < chunks.length; i++) {
const chunk = chunks[i]
for (let off = 0; off < chunk.length; off += hashSEG) {
sodium.crypto_hash_sha512_update(state2, chunk.subarray(off, Math.min(off + hashSEG, chunk.length)))
}
const chunkDigest = sha512Streaming([chunk])
console.error(`[WORKER-DBG] chunk[${i}] size=${chunk.length} sha512=${_whex(chunkDigest, 32)}… [0..8]=${_whex(chunk)} [-8..]=${_whex(chunk.slice(-8))}`)
}
self.postMessage({id, type: 'error', message: 'File digest mismatch'})
return
}
console.log(`[WORKER-DBG] verify: digest OK`)
// File-level decrypt with byte-level progress
const result = decryptChunks(BigInt(size), chunks, key, nonce, (d) => {
@@ -312,10 +285,6 @@ self.onmessage = (e: MessageEvent) => {
// ── Helpers ─────────────────────────────────────────────────────
function _whex(b: Uint8Array, n = 8): string {
return Array.from(b.slice(0, n)).map(x => x.toString(16).padStart(2, '0')).join('')
}
function digestEqual(a: Uint8Array, b: Uint8Array): boolean {
if (a.length !== b.length) return false
let diff = 0