* tests: service subscription counts after queue deletion
* smp server: decrease service subscription count on DEL and SUBS
* tests: service subscription count after the client service changes
* smp server: on DEL, decrease the count of the queue's own service only
* tests: reproduce SUB and SEND race in one block
* smp server: move subscriber registration out of serverThread
* smp server: deliver messages sent while the recipient subscribes
* smp server: keep subscriber registration in smpServer, pass it to client
* tests: receive SUB and SEND responses in order
* smp server: register subscription of a new queue directly
* smp server: name server subscribers srvSubs in runClient
* tests: sign SUB with authTransmission, keep test helpers unchanged
* smp server: remove do and mask_ from subscribeNewQueue
* smp server: send the first message once when SEND delivers it during SUB
* smp server: plan with SUB and SEND sequences before and after the change
* smp server: one sequence diagram with added and removed steps in the plan
* smp server: process all subscription changes in the client thread, remove subQ and serverThread
* smp server: send the first message once after SUBS, keep service counts exact
* smp server: name the reader monad in the type of setService
* plans: correct the diagrams and decisions of subscription processing
* plans: diagrams of a duplicate SUB and of SUBS, corrected decisions
* plans: duplicate SUB with SEND, service counts before and after the change
* tests: switch subscription with SUB and SEND in one block
* docs: remove the outdated SMP server design diagram
* smp server: move processClientSub back to the place of serverThread, passed to client threads as a parameter
* smp server: pass the pair of subscription processors to client, applied to the client where called
* smp server: show subscribeQueue and sharedSubscribeQueue as comments at their master places, temporary [skip ci]
* smp server: remove the temporary comments, the tree equals 966701dd [skip ci]
* tests: cover the changed subscription paths with concurrent, blocked, service and notifier scenarios
* tests: cover SMP server subscription changes
* smp server: remove the subscription branch for a disconnected client
* core: reject empty server key hash and null byte in host
* core: reject notification subscription to a server without key hash
* lib: exclude null byte in host with the other excluded characters
* tests: name the subscription server address test by the rejected values
* agent: continue XFTP uploads started in foreground while suspending
* tests: verify XFTP upload stage of agent suspension
* docs: plan for XFTP uploads while suspending
* docs: revise plan for XFTP uploads while suspending
* docs: revise plan wording for XFTP uploads while suspending
* agent: exclude ended and stopped XFTP uploads from suspension wait
* tests: verify suspension wait for ended and stopped XFTP uploads
* docs: plan decisions for ended XFTP uploads while suspending
* agent: simplify tryBeginAgentOperation with orElse
* agent: rename upload wait functions by the conditions they check
* tests: verify database check and deleted rows at suspension start
* docs: plan residual case, conditions and names for XFTP suspension
* docs: plan tests for the remaining XFTP suspension paths
* tests: verify the remaining XFTP suspension paths
* docs: update plan line references and the query error test
* tests: foreground agent before checking the query error result
* docs: plan line references after merging master and stage diagrams
* agent: one check of the begun database operation when removing deleted or failed counted files
* agent: suspendAgent returns before the database query of counted files
* tests: closed database store tests excluded from PostgreSQL agent builds
* tests: suspension timeout test waits for the timeout and bounds SUSPENDED
* tests: one setup helper for suspension tests with a small sent file, one for the bounded SUSPENDED wait
* tests: suspendAgent returns while the agent database is in use
* docs: plan states the forked removal at suspension start, uncounted redirects, offline files and the changed tests
* docs: plan links the simplex-chat pull request
* agent: suspend timeout starts when suspendAgent returns, not after the database query of counted files
* docs: plan states the separate removal thread and the TTL cleanup interval
* agent: acquire connection locks in order when sending
* tests: enable OverloadedStrings for lock acquisition tests
* agent: acquire connection locks with explicit recursion
* agent: test version 2 ratchets with previous header encoding
* agent: deliver messages between 7.0 and 7.1 apps in old connections
* agent: test version 2 ratchets with v7.0 encoding on master code
* agent: decode headers in version 2 ratchet tests without attoparsec
* agent: deliver messages to and from 7.0 apps in old connections
* agent: choose the header format by its length prefix
* xftp: test decoding handshake of servers before v6.5.0
* xftp: send and receive files through servers before v6.5.0
* xftp: refuse server handshake with invalid signature
* names: an available name carries its bona fide community terms
A bona fide community name can be registered only by someone who
accepts the community terms its name policy keeps. NRAvailable gains
bonaFideTerms_: the terms' text when it fits the answer, else the 0x
hex keccak256 of the text; Nothing for any other name. Nothing is
omitted from the JSON, so a plain name's answer is unchanged, and an
answer without the field decodes as a plain name.
Claude-Session: https://claude.ai/code/session_014JqabRiGc38JjH4KrE5LEU
* names: the bona fide terms travel as the zstd frame the name policy keeps
The name policy keeps the community terms' text compressed in one zstd
frame, and a buyer accepts the keccak256 of that frame. An available
bona fide name carries the frame when it fits the answer, else its hash.
ZstdTerms is base64url in JSON, as other bytes are.
Claude-Session: https://claude.ai/code/session_014JqabRiGc38JjH4KrE5LEU
* names: every available name carries the terms its buyer accepts
Every registration accepts terms, not only a bona fide name's. The name
policy keeps the general name terms and the community terms, each as a
zstd frame. NRAvailable's nameTerms is NTName with the general terms, or
NTBonaFide with a bona fide name's community terms, which include them;
either carries the frame when it fits the answer, else its hash. It is
absent only from a router before v24.
Claude-Session: https://claude.ai/code/session_014JqabRiGc38JjH4KrE5LEU
* names: NTBonaFide is NTCommunity
Claude-Session: https://claude.ai/code/session_014JqabRiGc38JjH4KrE5LEU
* names: terms are the policy's link to a file, and the file when it fits
* names: general and community terms, as the name policy now names them
---------
Co-authored-by: brenzi-agent <346870783+brenzi-agent@users.noreply.github.com>
* agent, ntf server: add lookupOrInsertIO, use it for locks and stats
* agent, server: use lookupOrInsertIO for map entries
* agent: use lookupOrInsertIO for server stats, keep GET lock in one transaction
* smp server: update getSubscription comment for lookupOrInsertIO result
* agent, server: keep original function names, reduce diff
* agent, server: simplify incServerStat and getMessage
* agent, ntf server: stat under whenPendingSub, lookupOrInsertIO in addPendingSubs
* smp server, session: use lookupOrInsertIO directly, drop the newSub alias
* agent, server: fromEither, remove unused incXFTPServerStat'
* util: hide UnliftIO fromEither where Util fromEither is used
* agent, server: use either id id, remove fromEither
* agent, server: keep isPendingSub, reword service delivery comment
## Agent
- Workers: a worker started by concurrent calls runs in one thread. `runWorkerAsync` put back a stale "not running" value after another call had taken the running one, and a worker restart wrote the same value, so a second thread executed the same commands (for example, a confirmation was sent twice and `CON` was emitted twice), or a caller blocked forever.
- Delivery receipts that fail with AUTH are deleted without an error event.
- Notification credentials are stored per receive queue rather than per connection.
- The protocol client fails pending requests when its connection ends.
- XFTP: a failure to remove a received file's temporary data is logged and no longer fails the worker; server test files have random names.
## Server
- An inactive SMP client session ends without its connection being closed by the expiration thread.
- An inactive HTTP2 session is ended by killing its thread rather than closing its TLS connection from another thread.
- Static files: the timeout handler kills the connection thread and leaves the socket close to that thread.
## Dependencies
- warp: simplex-chat/wai@63667cb5 (simplex-chat/wai#1). The timeout manager kills the connection thread without closing its socket.
## CI and tests
- Tests run on a RAM disk on Linux.
- Test HTTP2 clients are closed before their TLS connections; test temporary directories are cleared between suites.
* transport: IPv6 with IPv4 suffix, ntf server: decode servers per row
* keep domain name parser
* ntf server: decode servers as on master, tests: IPv6 hosts after restart
* agent: check connection when adding send queue
* simplify createSndConn
* agent: check confirmed send queue before adding to connection
* agent: report duplicate connection when joining a used one-time link
* smp-server: remove expired notification store keys
* smp-server: remove delivered notification keys
* tests: generate MsgId for ntf store test
* smp-server: keep ntf store comments in place
* smp-server: strict count of expired notifications
* ntf server: one db connection to find subscription
* pass connection to getNtfToken_
* getNtfToken_' with connection
* getNtfToken' with connection, inline
---------
Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
* smp server: delayed AUTH responses
* plan
* update
* simplify
* simplify
* stat when sent without delay
* update schema
* round up response time to the nearest delay multiple
---------
Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
* smp protocol: require fixed size corr ID when used as nonce, and min size msg ID to use as nonce
* simpler
* fixes
* randomMsgId
---------
Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>