Commit Graph
866 Commits
Author SHA1 Message Date
Evgeny @ SimpleX Chat aa471cd570 smp server: deliver later messages after SUB while a delivery waits (#2006)
* smp server: deliver later messages after SUB while a delivery waits

* smp server: reset delivery thread state only for the subscribed client
2026-10-30 22:29:16 +00:00
Evgeny @ SimpleX Chat 2376adaf98 tests: name the test in retry lines (#1994) 2026-10-26 21:54:20 +00:00
Evgeny @ SimpleX Chat 195df0d094 xftp server: release storage of interrupted uploads (#1977)
* xftp server: release storage of interrupted uploads

* xftp server: release storage of interrupted file deletion
2026-10-26 11:12:25 +00:00
Evgeny @ SimpleX Chat 3818284007 xftp: close connection after failed handshake (#1974) 2026-10-25 13:53:56 +00:00
Evgeny @ SimpleX Chat bba4cb5874 ntf server: continue notifications when SMP service becomes unavailable (#1976) 2026-10-25 09:24:11 +00:00
Evgeny @ SimpleX Chat 14122174c9 smp server: keep queues blocked with notice after restart and import (#1971) 2026-10-25 09:10:45 +00:00
Evgeny @ SimpleX Chat 31172c4251 clients: close interrupted connection attempts (#1967)
* client: kill transport thread on interrupt

* tests: close cancelled HTTP/2 connection attempt, stalling listener
2026-10-25 09:10:09 +00:00
Evgeny @ SimpleX Chat 568cd2c68c xftp: prohibit YAML aliases, limit file description replicas (#1965)
* xftp: limit description replicas, no YAML aliases

* inline limits

* xftp: derive file description chunk limit from maximum file size

* xftp: allow descriptions of 64 GB files at redundancy 4
2026-10-25 07:32:32 +00:00
Evgeny @ SimpleX Chat 11673c2450 agent, server: one thread per worker; close sockets in reader threads (#1972)
## Agent

- Workers: a worker started by concurrent calls runs in one thread. `runWorkerAsync` put back a stale "not running" value after another call had taken the running one, and a worker restart wrote the same value, so a second thread executed the same commands (for example, a confirmation was sent twice and `CON` was emitted twice), or a caller blocked forever.
- Delivery receipts that fail with AUTH are deleted without an error event.
- Notification credentials are stored per receive queue rather than per connection.
- The protocol client fails pending requests when its connection ends.
- XFTP: a failure to remove a received file's temporary data is logged and no longer fails the worker; server test files have random names.

## Server

- An inactive SMP client session ends without its connection being closed by the expiration thread.
- An inactive HTTP2 session is ended by killing its thread rather than closing its TLS connection from another thread.
- Static files: the timeout handler kills the connection thread and leaves the socket close to that thread.

## Dependencies

- warp: simplex-chat/wai@63667cb5 (simplex-chat/wai#1). The timeout manager kills the connection thread without closing its socket.

## CI and tests

- Tests run on a RAM disk on Linux.
- Test HTTP2 clients are closed before their TLS connections; test temporary directories are cleared between suites.
2026-10-25 07:24:36 +00:00
Evgeny @ SimpleX Chat 14b98672be lib: parse IPv6 addresses with IPv4 suffix (#1969)
* transport: IPv6 with IPv4 suffix, ntf server: decode servers per row

* keep domain name parser

* ntf server: decode servers as on master, tests: IPv6 hosts after restart
2026-10-25 05:38:32 +00:00
Evgeny @ SimpleX Chat 5b79311ac8 agent: report duplicate connection when joining a used one-time link (#1970)
* agent: check connection when adding send queue

* simplify createSndConn

* agent: check confirmed send queue before adding to connection

* agent: report duplicate connection when joining a used one-time link
2026-10-25 04:50:10 +00:00
Evgeny @ SimpleX Chat fd58a2eedf transport, crypto: certificate chain, keys and DH secret (#1964) 2026-10-24 17:46:55 +01:00
sh 8286303d0f smp-server: fix notification store key leak (#1907)
* smp-server: remove expired notification store keys

* smp-server: remove delivered notification keys

* tests: generate MsgId for ntf store test

* smp-server: keep ntf store comments in place

* smp-server: strict count of expired notifications
2026-10-23 09:55:12 +01:00
EvgenyandEvgeny @ SimpleX Chat f795c44b64 ntf server: one db connection to find subscription (#1956)
* ntf server: one db connection to find subscription

* pass connection to getNtfToken_

* getNtfToken_' with connection

* getNtfToken' with connection, inline

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-10-23 08:36:10 +01:00
EvgenyandEvgeny @ SimpleX Chat 7b2403671d server: retry accept, cache static files (#1962)
Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-10-21 23:32:15 +01:00
EvgenyandEvgeny @ SimpleX Chat 69d55c387a smp server: check correlation ID of forwarded transmission (#1954)
* smp server: check correlation ID of forwarded transmission

* simpler

* change error

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-10-21 14:35:16 +01:00
EvgenyandEvgeny @ SimpleX Chat fa0d1be801 agent: known host, smp protocol: parse empty messages (#1953)
Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-10-21 08:24:49 +01:00
sh 4b8d31a476 smp-server: remove deleted queues from store maps (#1947) 2026-10-16 09:46:09 +01:00
shandEvgeny @ SimpleX Chat e91711c7d7 smp-server: remove journal store leftovers (#1946)
* smp-server: remove postgres queue cache

* smp-server: remove queue locks

* smp-server: simplify message store class

* smp-server: remove unused functions and exports

* tests: remove journal leftovers

* agent: remove unused lock exports

* fix

* fixes

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-10-15 19:36:45 +01:00
f1f079044c smp-server: remove journal message store (#1919)
Co-authored-by: sh <github.shum@liber.li>
Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-10-15 17:37:49 +01:00
EvgenyandEvgeny @ SimpleX Chat 9d83c0378e tests: execute in parallel, agent: fix race condition with files and when switching queue (#1932)
* tests: execute in parallel

* fix test

* fix postgresql build

* fix test

* fix test

* fix some tests

* fix tests

* fix tests

* fix test

* update schema

* fix test

* update schemas

* diff

* fix race

* fixes

* update schemas

* fix tests

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-10-15 14:10:04 +01:00
sh 1d39da91c5 smp-server: enforce command concurrency limit (#1944)
* smp-server: hold command slot until completion

* smp-server: command slot wait timeout and stats

* smp-server: clarify command slot comments

* smp-server: simplify command slot fork

* smp-server: limit direct RSLV per client

* smp-server: distinct error for RSLV slot timeout

* smp-server: client command slot wait timeout

* smp-server: reply to forwarded RSLV slot timeout

* smp-server: raise default concurrency limits

* smp-server: simplify command slot wait
2026-10-15 13:58:07 +04:00
EvgenyandEvgeny @ SimpleX Chat b5e8a4b8cc fix tests on ghc 8.10.7 (#1939)
* fix tests on ghc 8.10.7

* fix test

* fix schema dump test

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-10-11 20:36:51 +01:00
EvgenyandEvgeny @ SimpleX Chat 3db4029bb9 ci: build cache (#1938)
* ci: build cache

* update build job

* rename steps, cache all builds, patch timestamps for ghc 8.10.7

* refactor, save O0 caches in master and stable builds

* update build to include toolchain fingerprint

* do not save cache on exact match

* remove caching docker from prs

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-10-11 11:52:24 +01:00
EvgenyandEvgeny @ SimpleX Chat 8a80727fa5 smp server: delayed AUTH responses (#1937)
* smp server: delayed AUTH responses

* plan

* update

* simplify

* simplify

* stat when sent without delay

* update schema

* round up response time to the nearest delay multiple

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-10-11 07:37:33 +01:00
EvgenyandEvgeny @ SimpleX Chat a2f985dc68 agent: prepare invitation link, create ratchet when connection is prepared (#1880)
* agent: prepare invitation link

* simplify

* refactor

* fix

* change min SMP version to 15

* remove dead code for SMP version <= 15

* update

* diff

* create ratchet when connection is prepared

* createRatchetIfNotExists

* clean up, only check client notices for contacts, fix orphan record

* simplify

* refactor

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-10-10 23:15:19 +01:00
EvgenyandEvgeny @ SimpleX Chat c18e0297c4 agent: migration to add index to read queues (#1934)
* agent: migration to add index to read queues

* update postgres schema

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-10-08 17:07:55 +01:00
EvgenyandEvgeny @ SimpleX Chat e24d537eaa agent: fix race when switching reception to the new queue (#1933)
* agent: fix race when switching reception to the new queue

* return bool

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-10-08 16:03:06 +01:00
sh 7754f8624e smp-server: copy entity IDs out of received block (#1906) 2026-10-08 15:28:38 +01:00
EvgenyandEvgeny @ SimpleX Chat 8d81f775b1 smp protocol: require fixed size corr ID when used as nonce, and min size msg ID to use as nonce (#1925)
* smp protocol: require fixed size corr ID when used as nonce, and min size msg ID to use as nonce

* simpler

* fixes

* randomMsgId

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-10-08 09:32:11 +01:00
EvgenyandEvgeny @ SimpleX Chat c0a1b7bd2d agent: fail database transaction instantly when database is closed, prevent race conditions in command processing, fix race when uploading files (#1885)
* agent: fail database transaction instantly when database is closed

* update direct-sqlcipher

* update direct-sqlcipher, fixes

* autoincrement command ID

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-10-06 13:01:55 +01:00
EvgenyandEvgeny @ SimpleX Chat 4ab5376a6e smp: reduce min supported version to 15 (short links support, 3/30/2025) (#1931)
Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-10-05 11:31:57 +01:00
Evgeny aaffb5b07d Merge pull request #1922 from simplex-chat/ratchet
Improve double ratchet
2026-10-05 08:07:04 +01:00
EvgenyandEvgeny @ SimpleX Chat 65893a359a agent: resolve potential race conditions in double ratchet (#1923)
* agent: resolve potential race conditions in double ratchet

* update

* improve

* diff

* diff

* diff

* simpler

* more races

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-10-04 22:28:50 +01:00
EvgenyandEvgeny @ SimpleX Chat fcc5670ae2 agent: keep ratchet hashes for connection lifetime, up to some cap, to prevent replay of expired ratchets (#1893)
* agent: keep ratchet hashes for connection lifetime, to prevent replay of expired ratchets

* cap

* simpler

* simpler2

* split

* one query

* simplify

* performance

* migration

* fix schema

* comment

* improve

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-10-04 14:43:21 +01:00
Evgenyandshum 18440c0d17 smp-server: check queue is not secured in link store (#1917)
Co-authored-by: shum <github.shum@liber.li>
2026-10-04 13:41:48 +01:00
EvgenyandEvgeny @ SimpleX Chat 64c5d19fc4 agent: aggregate cap on skipped message keys (#1903)
* agent: aggregate cap on skipped message keys

* migration

* update migration

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-10-04 10:18:41 +01:00
EvgenyandEvgeny @ SimpleX Chat e11dfb985d consider IP address protected only if it is used for the chosen host (#1895)
* consider IP address protected only if it is used for the chosen host

* simplify

* simplify

* simplify

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-10-03 22:45:44 +01:00
Evgenyandshum 76248dd767 sntrup761: bound KeyGen retries on R3 inversion (#1918)
Co-authored-by: shum <github.shum@liber.li>
2026-10-03 17:01:36 +01:00
brenzi 9c641fed91 reject e2e params with kem_ct and without kem_key (#1901) 2026-10-03 15:10:42 +01:00
EvgenyandEvgeny @ SimpleX Chat 24036368f8 xrcp: limit multicast validity time window (#1899)
Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-10-03 14:51:48 +01:00
EvgenyandEvgeny @ SimpleX Chat ead8f10f26 smp protocol: bind agreed version of forwarded command (#1915)
* smp protocol: bind agreed version of forwarded command

* refactor

* pass proxied relay version range via config

* simplify

* refactor

* packZipWith

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-10-03 08:56:19 +01:00
sh 234b5cd856 smp-server: rework message expiration (#1873)
* smp server: rework message expiration

* smp server: paginate message expiration sweep
2026-10-01 09:08:16 +01:00
EvgenyandEvgeny @ SimpleX Chat 0e1ecdcff6 xftp server: validate paths in store log (#1894)
Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-09-30 09:36:45 +01:00
EvgenyandEvgeny @ SimpleX Chat 551a79d99b smp server: validate client version in forwarded command (#1890)
Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-09-29 19:52:15 +01:00
EvgenyandEvgeny @ SimpleX Chat 875ba7d53f agent: make QUOTA error temporary for async commands (#1881)
* agent: make QUOTA error temporary for async commands

* retry commands on quota

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-09-25 18:13:49 +01:00
EvgenyandEvgeny @ SimpleX Chat 35f1b145a7 xftp: fix race when sending the file (#1879)
* xftp: fix race when sending the file

* simpler test

* fix tests

* test

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-09-25 11:59:06 +01:00
EvgenyandEvgeny @ SimpleX Chat 5294b7d8b7 agent: PQ rachet security code, store AD and pqAD in database, bulk reading (#1874)
* agent: PQ rachet security code, store AD and pqAD in database, bulk reading

* rename to verify code

* diff

* refactor

* query

* reduce transactions

* request binding

* simplify

* fix query and security code derivation

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-09-23 21:21:09 +01:00
ea43df2349 extend SMP protocol to support name availability queries by labelhash (#1863)
* implement resolving 2LD names by labelhash

* tiny test addition

* simplify language

* report expiry and availability correctly

* compare block instead of wall clock

* simplify language

* simplify language

* map resolver 410 to NAME NOT_FOUND (a lapsed name is an answer, not a failure)

* split error into a machine-readable code and a human message

* resolver: reduce comments

* resolver: reduce comments, remove REVIEW.html

* extend SMP protocol to support name availability queries with accurate and meaningful replies

* review fixes

* more review fixes

* docs shortening and other review fixes

* add catch all for furture variants

* adversarial review (against simplex-chat) fix

* next iteration fixes

* adapt house style

* eth_call guard and cache constants

* revert NAVL command and wrap it all into RSLV

* doc fixes

* Update src/Simplex/Messaging/Server/Names.hs

Co-authored-by: Evgeny <evgeny@poberezkin.com>

* Update src/Simplex/Messaging/Protocol.hs

Co-authored-by: Evgeny <evgeny@poberezkin.com>

* Update src/Simplex/Messaging/Protocol.hs

Co-authored-by: Evgeny <evgeny@poberezkin.com>

* Update src/Simplex/Messaging/Protocol.hs

Co-authored-by: Evgeny <evgeny@poberezkin.com>

* Update src/Simplex/Messaging/Protocol.hs

Co-authored-by: Evgeny <evgeny@poberezkin.com>

* protocol types refactoring

* next iteration on types only

* rentPrices map

* claude answering ep review. to be continued...

* separate labelhash and plaintext names cleanly

* align implementation with latest type changes to test against client

* fix adversarial review findings

* trim diff

* align resolver with contract changes for names v2

* fix reverse compatibility with .testing mainnet

* fix more robustly

* NameQuery simplification

* revert drive-by refactoring

* implement full type change and introduce resolver endpoint versioning

* fix stale docs

* derive NameRegistration JSON the same way on every build

  sumTypeJSON switches to the _owsf form on swift builds, but this JSON is
  the RNAME payload and the resolver's HTTP contract, so a swift client and
  a Linux relay would disagree on every field. taggedObjectJSON is what it
  already resolves to everywhere else.

  The label modifier keeps the reservedReason_ collision escape out of the
  API, as AgentWorkersDetails does: both arms now say reservedReason.

* fix resolver boundary: status before body, cover /v2, drop dead field

  httpGet read the response body before checking the status, so an oversized
  error page surfaced as a transient "response too large" instead of the
  authoritative status. Reverting it to its previous shape restores that and
  removes the status test both callers had been re-deriving.

  registration() had no tests at all, though it is the endpoint SMP v22
  consumes. RegistrationV2Tests covers the three answer shapes, the error
  paths and the exact key set of each, which is the wire contract.

  auctionUntil was always None with no consumer. The spec claimed a reason
  word travels unchanged; a resolver can only send a word it has, and SNRC's
  registry records a number.

* fix review findings

* resolver errors say what went wrong, not "no such name"

  /v2/resolve answers 200, 400 or 502, and an unregistered name is
  NRAvailable, so no status means "not registered". Mapping 400/404/410 to
  NOT_FOUND made a misconfigured relay deny every name, and hid a relay
  upgraded ahead of its resolver. All three now surface as RESOLVER.

  rslvNotFound would have gone dead, so it counts what its name says: an
  availability answer the encoder downgrades for a session below v22. The
  wire is unchanged.

  A hashed query the registrar cannot name is refused with 502 rather than
  answered with a record named "unknown", which the client rejects anyway.
  NRRUnknown is capped to 32 printable characters again, as the spec says.
  A registered name that is also reserved no longer offers a date it will
  never free up on. rentPrices is registrationPrices throughout, and
  yearPriceUSD is USDCents rather than a bare Int64.

* fix regressions found reviewing the last two commits

  resolveNameMsg read the version off thParams', which on the PFWD path is
  the proxy's session, not the client's. It takes the version as an argument
  now, so each call site passes its own — the forwarded one uses fwdVersion.

  reservedReasonOf matched the reason words before capping, so "internal
  review" became NRRUnknown "internal", which encodes back as NRRInternal.
  Capping precedes the match, so what is kept encodes to what it decoded.

  Four agent tests still pinned NAME NOT_FOUND from a 404 stub, and two spec
  statements still described the old mapping. A registrar that does not
  record labels cannot answer a hashed query, which the resolver README now
  says.

* docs sweep

* simplify encoding

* drop resolver caching (#1866)

* rename

* remove trailing_ filter

* fix resolver v2 for subnames (#1867)

* fix resolver v2 for subnames

* simplify doc

* resolver should report response truth freshness (#1868)

* first shot at reporting freshness

* fix review findings

* renaming

---------

Co-authored-by: sh <github.shum@liber.li>
Co-authored-by: Evgeny <evgeny@poberezkin.com>
2026-09-16 09:58:44 +01:00
EvgenyandEvgeny @ SimpleX Chat 0b97e025ce xftp server: disable ack command (#1871)
* xftp server: disable ack

* fix tests

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-09-12 10:06:09 +01:00