Commit Graph
2156 Commits
Author SHA1 Message Date
shum 706211dc24 bench: add server-only memory and load phases 2026-09-30 12:03:25 +00:00
shum 2722006e30 bench: let finalizers run before measuring 2026-09-30 12:01:44 +00:00
shum a6214163b4 bench: fix msgqfill after msgQ became optional 2026-09-30 12:01:44 +00:00
shum 6b9a5895df Merge remote-tracking branch 'origin/master' into sh/fix-leak
# Conflicts:
#	src/Simplex/Messaging/Client.hs
#	tests/RSLVTests.hs
2026-09-30 09:37:34 +00:00
EvgenyandEvgeny @ SimpleX Chat b4c288c597 docs: clarify security model (#1902)
Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-09-30 10:18:52 +01:00
EvgenyandEvgeny @ SimpleX Chat 0e1ecdcff6 xftp server: validate paths in store log (#1894)
Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-09-30 09:36:45 +01:00
EvgenyandEvgeny @ SimpleX Chat c1776dc5a4 server: improve control port auth (#1898)
Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-09-30 09:04:28 +01:00
EvgenyandEvgeny @ SimpleX Chat e2df2e1b7c docs: update SMP protocol (#1900)
Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-09-30 08:10:35 +01:00
EvgenyandEvgeny @ SimpleX Chat 267e2e0727 agent: fix ratchet - dont save if body auth failed (#1892)
Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-09-29 21:32:21 +01:00
EvgenyandEvgeny @ SimpleX Chat 551a79d99b smp server: validate client version in forwarded command (#1890)
Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-09-29 19:52:15 +01:00
EvgenyandEvgeny @ SimpleX Chat 972e50e768 docs: update XRCP doc to make code verification optional for known connections (#1891)
Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-09-29 18:08:54 +01:00
Evgeny Poberezkin 5eecd99bc2 7.1.0.9 2026-09-26 12:52:57 +01:00
69eb28b3ab agent: share server value across subscription rows of server-keyed queries (#1877)
* agent: share server value across subscription rows of server-keyed queries

* reduce fields in query

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-09-26 12:05:20 +01:00
EvgenyandEvgeny @ SimpleX Chat 875ba7d53f agent: make QUOTA error temporary for async commands (#1881)
* agent: make QUOTA error temporary for async commands

* retry commands on quota

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-09-25 18:13:49 +01:00
EvgenyandEvgeny @ SimpleX Chat 35f1b145a7 xftp: fix race when sending the file (#1879)
* xftp: fix race when sending the file

* simpler test

* fix tests

* test

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-09-25 11:59:06 +01:00
sh c7163afa3a ci: cancel stale runs (#1878) 2026-09-24 15:33:22 +01:00
EvgenyandEvgeny @ SimpleX Chat 5294b7d8b7 agent: PQ rachet security code, store AD and pqAD in database, bulk reading (#1874)
* agent: PQ rachet security code, store AD and pqAD in database, bulk reading

* rename to verify code

* diff

* refactor

* query

* reduce transactions

* request binding

* simplify

* fix query and security code derivation

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-09-23 21:21:09 +01:00
Simplex Operations 900c45ffae 7.1.0.8 v7.1.0-beta.3 2026-09-19 08:04:24 +00:00
sh 7056ae22cc smp-server: reduce Prometheus and expiration DB load (#1872)
* smp server: add prometheus scan indexes

* smp server: estimate queue count in metrics

* docs: add smp server db load report

* smp server: reduce msg_queues write load

* docs: document batch-2 fixes and operator actions
2026-09-19 09:02:04 +01:00
Simplex Operations c6ef2876f1 7.1.0.7 2026-09-19 07:17:34 +00:00
ea43df2349 extend SMP protocol to support name availability queries by labelhash (#1863)
* implement resolving 2LD names by labelhash

* tiny test addition

* simplify language

* report expiry and availability correctly

* compare block instead of wall clock

* simplify language

* simplify language

* map resolver 410 to NAME NOT_FOUND (a lapsed name is an answer, not a failure)

* split error into a machine-readable code and a human message

* resolver: reduce comments

* resolver: reduce comments, remove REVIEW.html

* extend SMP protocol to support name availability queries with accurate and meaningful replies

* review fixes

* more review fixes

* docs shortening and other review fixes

* add catch all for furture variants

* adversarial review (against simplex-chat) fix

* next iteration fixes

* adapt house style

* eth_call guard and cache constants

* revert NAVL command and wrap it all into RSLV

* doc fixes

* Update src/Simplex/Messaging/Server/Names.hs

Co-authored-by: Evgeny <evgeny@poberezkin.com>

* Update src/Simplex/Messaging/Protocol.hs

Co-authored-by: Evgeny <evgeny@poberezkin.com>

* Update src/Simplex/Messaging/Protocol.hs

Co-authored-by: Evgeny <evgeny@poberezkin.com>

* Update src/Simplex/Messaging/Protocol.hs

Co-authored-by: Evgeny <evgeny@poberezkin.com>

* Update src/Simplex/Messaging/Protocol.hs

Co-authored-by: Evgeny <evgeny@poberezkin.com>

* protocol types refactoring

* next iteration on types only

* rentPrices map

* claude answering ep review. to be continued...

* separate labelhash and plaintext names cleanly

* align implementation with latest type changes to test against client

* fix adversarial review findings

* trim diff

* align resolver with contract changes for names v2

* fix reverse compatibility with .testing mainnet

* fix more robustly

* NameQuery simplification

* revert drive-by refactoring

* implement full type change and introduce resolver endpoint versioning

* fix stale docs

* derive NameRegistration JSON the same way on every build

  sumTypeJSON switches to the _owsf form on swift builds, but this JSON is
  the RNAME payload and the resolver's HTTP contract, so a swift client and
  a Linux relay would disagree on every field. taggedObjectJSON is what it
  already resolves to everywhere else.

  The label modifier keeps the reservedReason_ collision escape out of the
  API, as AgentWorkersDetails does: both arms now say reservedReason.

* fix resolver boundary: status before body, cover /v2, drop dead field

  httpGet read the response body before checking the status, so an oversized
  error page surfaced as a transient "response too large" instead of the
  authoritative status. Reverting it to its previous shape restores that and
  removes the status test both callers had been re-deriving.

  registration() had no tests at all, though it is the endpoint SMP v22
  consumes. RegistrationV2Tests covers the three answer shapes, the error
  paths and the exact key set of each, which is the wire contract.

  auctionUntil was always None with no consumer. The spec claimed a reason
  word travels unchanged; a resolver can only send a word it has, and SNRC's
  registry records a number.

* fix review findings

* resolver errors say what went wrong, not "no such name"

  /v2/resolve answers 200, 400 or 502, and an unregistered name is
  NRAvailable, so no status means "not registered". Mapping 400/404/410 to
  NOT_FOUND made a misconfigured relay deny every name, and hid a relay
  upgraded ahead of its resolver. All three now surface as RESOLVER.

  rslvNotFound would have gone dead, so it counts what its name says: an
  availability answer the encoder downgrades for a session below v22. The
  wire is unchanged.

  A hashed query the registrar cannot name is refused with 502 rather than
  answered with a record named "unknown", which the client rejects anyway.
  NRRUnknown is capped to 32 printable characters again, as the spec says.
  A registered name that is also reserved no longer offers a date it will
  never free up on. rentPrices is registrationPrices throughout, and
  yearPriceUSD is USDCents rather than a bare Int64.

* fix regressions found reviewing the last two commits

  resolveNameMsg read the version off thParams', which on the PFWD path is
  the proxy's session, not the client's. It takes the version as an argument
  now, so each call site passes its own — the forwarded one uses fwdVersion.

  reservedReasonOf matched the reason words before capping, so "internal
  review" became NRRUnknown "internal", which encodes back as NRRInternal.
  Capping precedes the match, so what is kept encodes to what it decoded.

  Four agent tests still pinned NAME NOT_FOUND from a 404 stub, and two spec
  statements still described the old mapping. A registrar that does not
  record labels cannot answer a hashed query, which the resolver README now
  says.

* docs sweep

* simplify encoding

* drop resolver caching (#1866)

* rename

* remove trailing_ filter

* fix resolver v2 for subnames (#1867)

* fix resolver v2 for subnames

* simplify doc

* resolver should report response truth freshness (#1868)

* first shot at reporting freshness

* fix review findings

* renaming

---------

Co-authored-by: sh <github.shum@liber.li>
Co-authored-by: Evgeny <evgeny@poberezkin.com>
2026-09-16 09:58:44 +01:00
Evgeny Poberezkin 7f0218e1d5 7.1.0.6 v7.1.0-beta.2 2026-09-12 10:34:03 +01:00
EvgenyandEvgeny @ SimpleX Chat 0b97e025ce xftp server: disable ack command (#1871)
* xftp server: disable ack

* fix tests

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-09-12 10:06:09 +01:00
EvgenyandEvgeny @ SimpleX Chat a00e225d74 xftp: hash of the shared part of file description (#1865)
Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-09-12 08:30:13 +01:00
shum 85d74352cf docs: add stuck relay session finding 2026-09-09 14:44:06 +00:00
shum 15606bfff7 docs: restructure leak findings with tables 2026-09-09 14:34:48 +00:00
shum 5297427733 docs: mark proxy msgQ leak fixed upstream 2026-09-09 14:01:39 +00:00
shum 7309340821 docs: add postgres backend findings 2026-09-09 12:35:44 +00:00
shum 1a2145a04f docs: add RSLV fan-out to leak findings 2026-09-09 12:35:44 +00:00
shum c3c4fa7604 tests: reproduce RSLV fan-out, proxy stuck session 2026-09-09 12:35:44 +00:00
shum 588966097a smp server: log relay host on proxy forward error 2026-09-09 12:35:44 +00:00
sh ee97ffcab8 docs: drop non-findings sections from leak report 2026-09-09 12:35:44 +00:00
sh 068b929917 docs: use plainer wording in leak findings 2026-09-09 12:35:44 +00:00
sh 3dd259de68 docs: condense leak findings report 2026-09-09 12:35:44 +00:00
sh 1c77a88b5f tests: add proxy msgQ retention bench and counter 2026-09-09 12:35:44 +00:00
sh 5f9e8c16c0 docs: correct endThreads race mechanism and reachability 2026-09-09 12:31:15 +00:00
sh 0b60c8eea5 tests: measure concurrency cap and fork race reachability 2026-09-09 12:31:15 +00:00
sh f513751005 tests: verify socket accounting via control port 2026-09-09 12:31:15 +00:00
sh 8d2da160bd tests: drop phase for already-fixed session var leak 2026-09-09 12:31:15 +00:00
sh db4b0dbb6b tests: add batched subscribe leak phase 2026-09-09 12:31:15 +00:00
sh 1d80f8c5ed tests: measure when proxy leak is bounded vs unbounded 2026-09-09 12:31:15 +00:00
sh fc464a9a7f docs: add ntf server exposure and socket stats bug 2026-09-09 12:31:15 +00:00
sh c5f8ef47b4 tests: sweep proxy latency in memory leak bench 2026-09-09 12:31:15 +00:00
sh 1f7557a30d docs: add smp-server memory leak findings 2026-09-09 12:31:15 +00:00
sh 689ff8901f tests: add proxy and TLS memory leak bench phases 2026-09-09 12:31:15 +00:00
sh 2e97f493ec tests: add latency transport for smp-server bench 2026-09-09 12:31:15 +00:00
sh 0a91f1bbc8 smp-server: add server port to leak diagnostics 2026-09-09 12:31:15 +00:00
sh 77699740f4 smp-server: fix notification store key retention leak
deleteExpiredNtfs trimmed each notifier's message list but never removed
the outer NtfStore map key, so one empty entry per notifier queue that
ever received a notification was retained forever (grows with the active
notifier set, never shrinks).

Remove the outer key when its list becomes empty, and make storeNtf fully
atomic so it cannot race the removal and write a notification to an
orphaned TVar. Verified with the load bench (ntfexp): after expiry
ntfStore_keys drops from the queue count to 0 instead of staying flat.
2026-09-09 12:31:15 +00:00
sh dcfb2921cf tests: add smp-server memory leak load bench
Standalone smp-mem-bench executable that starts an in-process SMP server
and drives churn workloads, reporting GHC live-heap residency per
checkpoint after a forced major GC. Store selectable via BENCHSTORE
(pgmsg/pgjournal/journal).

Phases: plain, svc, svcrace, ntf, conc, svcsubs, getp, link, and leak
repros - stuck (delivery threads blocked forever on a full sndQ),
certchurn (serviceLocks/services grow per distinct service certificate),
and ntfexp (NtfStore keys retained after notifications expire).
2026-09-09 12:31:15 +00:00
sh 9f1aab636b smp-server: add leak diagnostics logging
Add an exception-guarded periodic thread that logs a single greppable
"LEAKDIAG" line censusing every growable in-memory structure: live
threads, per-client endThreads and subscriptions (by SubThread state),
subscriber maps, ntf store, store entity/loaded counts, and proxy agent
maps with in-flight sentCommands. Interval via SMP_LEAKDIAG_SEC
(default 60), no RTS flags required.

Adds pClientSentCommandsCount and getAgentLeakStats accessors.
2026-09-09 12:31:15 +00:00