deleteExpiredNtfs trimmed each notifier's message list but never removed
the outer NtfStore map key, so one empty entry per notifier queue that
ever received a notification was retained forever (grows with the active
notifier set, never shrinks).
Remove the outer key when its list becomes empty, and make storeNtf fully
atomic so it cannot race the removal and write a notification to an
orphaned TVar. Verified with the load bench (ntfexp): after expiry
ntfStore_keys drops from the queue count to 0 instead of staying flat.
Standalone smp-mem-bench executable that starts an in-process SMP server
and drives churn workloads, reporting GHC live-heap residency per
checkpoint after a forced major GC. Store selectable via BENCHSTORE
(pgmsg/pgjournal/journal).
Phases: plain, svc, svcrace, ntf, conc, svcsubs, getp, link, and leak
repros - stuck (delivery threads blocked forever on a full sndQ),
certchurn (serviceLocks/services grow per distinct service certificate),
and ntfexp (NtfStore keys retained after notifications expire).
Add an exception-guarded periodic thread that logs a single greppable
"LEAKDIAG" line censusing every growable in-memory structure: live
threads, per-client endThreads and subscriptions (by SubThread state),
subscriber maps, ntf store, store entity/loaded counts, and proxy agent
maps with in-flight sentCommands. Interval via SMP_LEAKDIAG_SEC
(default 60), no RTS flags required.
Adds pClientSentCommandsCount and getAgentLeakStats accessors.
* feat: add server public information handling in XFTP protocol
* test
---------
Co-authored-by: sh <github.shum@liber.li>
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
* agent: fast queue rotation does not requiring the current server to be online
* update plan
* update plan and rfc
* corrections
* update plan
* implementation
* test
* refactor
* test switching notificaitons
* rename
* fix, refactor
* fix possible crash
* simplify
* refactor
* simplify
* refactor
* simplify
* order, avoid re-reading connection
---------
Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
* agent: report PRXY broker errors as proxy-to-relay errors
The proxy returns PROXY BROKER errors only when it fails to connect to the
destination relay, but for PRXY they were mapped to SMP <proxy> (PROXY ...),
losing the relay address, so the clients reported them as errors of the
connection to the forwarding server. Map them to PROXY {proxyServer,
relayServer, ...}, the same shape PFWD errors already use.
* Apply suggestion from @epoberezkin
* agent: keep ProxyProtocolError for PRXY broker errors
Reverts 826a2377.
ProxyResponseError wraps PCEResponseError, a response that failed to
parse, and both clients match protocolError when rendering this error,
so the connect alert falls through to the raw error dump - and released
clients cannot render it at all. temporaryAgentError and serverHostError
also match ProxyProtocolError only, so the failure stops being retried
and proxy fallback no longer engages.
---------
Co-authored-by: Evgeny <evgeny@poberezkin.com>
* smp-server: add serverInfoBytes to THandleParams and related functions
* server information
* fix test
---------
Co-authored-by: Ed Asriyan <service.github@asriyan.me>
* servers: drop support of versions before 01/2025
* fix agent tests
* fix ntf tests
* fix test
* remove code for non-batched transmissions
* fix batching test
* fix ntf
* comment
Service subscription counters (totalServiceSubs, serviceSubsCount,
ntfServiceSubsCount) are TVar (Int64, IdsHash). modifyTVar' only forces
the pair to WHNF, so `n +/- n'` and `idsHash <> idsHash'` stay
unevaluated and accumulate an unbounded thunk chain under subscription
and delivery churn (the IdsHash chain also retains a bytestring per
update) - a space leak proportional to the number of updates.
Force both components in addServiceSubs/subtractServiceSubs. Verified
with the load bench: svc churn drops from +5.4 KiB/iter (linear) to flat.
sh