* smp: deliver service subscription to correct client
* tests: more resilient to concurrency
* optimize PostgreSQL query
* fix service re-association after server "downgrade"
* correctly handle service removed from server (and ID changed)
* remove unused
---------
Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
* agent: remove service queue association when service ID changed
* agent: process ENDS event
* agent: send service subscription error event
* agent: test migrating to/from service subscriptions, fixes
* agent: always remove service when disabled, fix service subscriptions
* agent: remove service/queue associations when service ID changes
* agent: check that service ID in NEW response matches session ID in transport session
* agent subscription WIP
* test
* comment
* enable tests
* update queries
* agent: option to add SQLite aggregates to DB connection (#1673)
* agent: add build_relations_vector function to sqlite
* update aggregate
* use static aggregate
* remove relations
---------
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
* add test, treat BAD_SERVICE as temp error, only remove queue associations on service errors
* add packZipWith for backward compatibility with GHC 8.10.7
---------
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
* agent: use server keyhash when loading service record
* agent: process queue/service associations with delayed subscription results
* agent: service subscription events
* servers: maintain xor-hash of all associated queue IDs in PostgreSQL (#1615)
* ntf server: maintain xor-hash of all associated queue IDs via PostgreSQL triggers
* smp server: xor hash with triggers
* fix sql and using pgcrypto extension in tests
* track counts and hashes in smp/ntf servers via triggers, smp server stats for service subscription, update SMP protocol to pass expected count and hash in SSUB/NSSUB commands
* agent migrations with functions/triggers
* remove agent triggers
* try tracking service subs in the agent (WIP, does not compile)
* Revert "try tracking service subs in the agent (WIP, does not compile)"
This reverts commit 59e908100d.
* comment
* agent database triggers
* service subscriptions in the client
* test / fix client services
* update schema
* fix postgres migration
* update schema
* move schema test to the end
* use static function with SQLite to avoid dynamic wrapper
* smp server: refactor message delivery to always respond SOK to subscriptions
* refactor ntf subscribe
* cancel subscription thread and reduce service subscription count when queue is deleted
* subscribe rcv service, deliver sent messages to subscribed service
* subscribe rcv service to messages (TODO delivery on subscription)
* WIP
* efficient initial delivery of messages to subscribed service
* test: delivery to client with service certificate
* test: upgrade/downgrade to/from service subscriptions
* remove service association from agent API, add per-user flag to use the service
* agent client (WIP)
* service certificates in the client
* rfc about drift detection, and SALL to mark end of message delivery
* fix test
* fix test
* add function for postgresql message storage
* update migration
* agent: make createConnection and setConnShortLink apis support setting all link data fields
* add functions
* refactor
* refactor
* fix tests
---------
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
* agent: support client notices
* improve
* fix, test
* rename
* cleanup
* send and process notices in more cases
* dont delete
* dont remove notice on other permanent errors
* dont remove notice if there is no notice ID in queue
* add server to error
* allow deleting
* only use notice if key hash matches
* agent: subscribe all connections
* query, version
* BoolInt
* add query to errors
* Revert "add query to errors"
This reverts commit 32a1f7fe11.
* fix optional field
* version
* limit number of in-flight subscriptions to 35000
* agent: optimize subscriptions memory usage more (do not store subscribed queues in memory) WIP
* use new session subscriptions data
* version
* remove old data structure
* remove version
* batch deletions
* test TSessionSubs
* comment
* smp server: store messages in PostgreSQL
* stored procedures to write and to expire messages
* function to export messages
* move all message functions to PostgreSQL, remove delete trigger
* comments
* import messages to db
* fix message import, add export
* fix export
* fix export
* fix compilation flags
* import messages line by line
* fix server start with database storage
* fix compilation
* comments
* agent: use PQ keys in contact request data inside link container (but not in contact request link); use PQ keys in invitations sent to contact addresses
* do not use PQ keys in the link with old address versions
* rfc: client certificates for high volume clients (opertors' chat relays, notification servers, service bots)
* client certificates types (WIP)
* parameterize Transport
* protocol/schema/api changes
* agent API
* rename command
* agent subscriptions return local ClientServiceId to chat
* verify transmissions
* fix receiving client certificates, refactor
* ntf server: remove shared queue for all notification subscriptions (#1543)
* ntf server: remove shared queue for all notification subscriptions
* wait for subscriber with timeout
* safer
* refactor
* log
* remove unused
* WIP service subscriptions and associations, refactor
* process service subscriptions
* rename
* simplify switching subscriptions
* SMP service handshake with additional server handshake response
* notification delivery and STM persistence for services
* smp server: database storage, store log, fix encoding for STORE error, replace String with Text in locks and error
* stats
* more stats
* rename SMP commands
* service subscriptions in ntf server agent (tests fail)
* fix
* refactor
* exports
* subscribe ntf server as service for associated queues
* test ntf service connection, fix SOKS response, fix service associations not removed in STM storage
* INI option to support services
* ntf server: downgrade subscriptions when service is no longer supported, track counts of subscribed queues
* smp protocol: include service certificate fingerprint in the string signed over with entity key (TODO two tests fail)
* fix test
* ntf server prometheus stats, use Int64 in SOKS/ENDS responses (to avoid conversions), additional error status for ntf subscription
* update RFC
* refactor useServiceAuth to avoid ad hoc decisions about which commands use service signatures, and to prohibit service signatures on other commands
* remove duplicate service signature syntax check from checkCredentials, it is checked in verifyTransmission
* service errors, todos
* fix checkCredentials in ntf server, service errors
* refactor service auth
* refactor
* service agent: store returned queue count instead of expected
* refactor serverThread
* refactor serviceSig
* rename
* refactor, rename, test repeat NSUB service association
* respond with error to SUBS
* smp server: export/import service records between database and store log
* comment
* comments
* ghc 8.10.7
* agent: setInvitationShortLink api
* Eq instance
* allow changing link data on server, refactor
* fix
* encodings
* remove link data after connection
* Revert "encodings"
This reverts commit f8e254cca9.
---------
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
* parameterize transport by peer type (client/server)
* LogDebug level when test is retried
* support "flipped" HTTP2, fix test retry to avoid retrying pending tests
* move sync to the end of the tests
* agent: return error and message absence differently when getting notification messages
* fix test
* mapM
* inline nse functions, release lock on error or no message
* agent: handle cases when last message ts is not set for notifications; set last ts for "stale" notifications when messages expired and queue is empty, to prevent repeated processing
* only log errors if they exist
* only set last ts for queue that delivered notification
* smp server: short links and owners for channels
* types
* support mutliple rcv keys
* fix down migration, test/create server schema dump
* reduce schema dump
* parameterize type for link data by connection type
* return full connection link data
* test version
* change short link encoding
* test: print pg_dump output
* server pages, link encoding
* fix connection request when queue data and sender ID are created for old servers
* test, change pattern
* ci: install postgresql tools in runner (#1507)
* ci: install postgresql tools in runner
* ci: docker shell abort on error
* fix pattern for ghc 8.10.7
* patch ConnReqUriData SMP encoding to preserve queue mode after decoding
* test for RKEY
* fix/test store log with RKEY
---------
Co-authored-by: sh <37271604+shumvgolove@users.noreply.github.com>
Evgeny