* servers: maintain xor-hash of all associated queue IDs in PostgreSQL (#1615)
* ntf server: maintain xor-hash of all associated queue IDs via PostgreSQL triggers
* smp server: xor hash with triggers
* fix sql and using pgcrypto extension in tests
* track counts and hashes in smp/ntf servers via triggers, smp server stats for service subscription, update SMP protocol to pass expected count and hash in SSUB/NSSUB commands
* agent migrations with functions/triggers
* remove agent triggers
* try tracking service subs in the agent (WIP, does not compile)
* Revert "try tracking service subs in the agent (WIP, does not compile)"
This reverts commit 59e908100d.
* comment
* agent database triggers
* service subscriptions in the client
* test / fix client services
* update schema
* fix postgres migration
* update schema
* move schema test to the end
* use static function with SQLite to avoid dynamic wrapper
* agent: support client notices
* improve
* fix, test
* rename
* cleanup
* send and process notices in more cases
* dont delete
* dont remove notice on other permanent errors
* dont remove notice if there is no notice ID in queue
* add server to error
* allow deleting
* only use notice if key hash matches
* smp server: fix server pages when source code is not specified
* servers: include git commit in version
* flexible alpn
* fix test
* fix ghc 8.10.7 build
* don't exclude postgres modules for client_library flag
* fix build with client and server postgres flag
---------
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
* protocol: refactor types and encoding
* clean
* smp server: batch commands (#1560)
* smp server: batch commands verification into one DB transaction
* ghc 8.10.7
* flatten transmission tuples
* diff
* only use batch logic if there is more than one transmission
* func
* reset NTF service when adding notifier
* version
* Revert "smp server: use separate database pool for reading queues and creating service records (#1561)"
This reverts commit 3df2425162.
* version
* Revert "version"
This reverts commit d80a6b74c5.
* log alpn
* always use HTTP when SNI is sent, regardless of ALPN
* decide credential based on SNI
* do not use web port in SMP/Ntf servers connecting to SMP servers
* simpler
* refactor
* fix
* rfc: client certificates for high volume clients (opertors' chat relays, notification servers, service bots)
* client certificates types (WIP)
* parameterize Transport
* protocol/schema/api changes
* agent API
* rename command
* agent subscriptions return local ClientServiceId to chat
* verify transmissions
* fix receiving client certificates, refactor
* ntf server: remove shared queue for all notification subscriptions (#1543)
* ntf server: remove shared queue for all notification subscriptions
* wait for subscriber with timeout
* safer
* refactor
* log
* remove unused
* WIP service subscriptions and associations, refactor
* process service subscriptions
* rename
* simplify switching subscriptions
* SMP service handshake with additional server handshake response
* notification delivery and STM persistence for services
* smp server: database storage, store log, fix encoding for STORE error, replace String with Text in locks and error
* stats
* more stats
* rename SMP commands
* service subscriptions in ntf server agent (tests fail)
* fix
* refactor
* exports
* subscribe ntf server as service for associated queues
* test ntf service connection, fix SOKS response, fix service associations not removed in STM storage
* INI option to support services
* ntf server: downgrade subscriptions when service is no longer supported, track counts of subscribed queues
* smp protocol: include service certificate fingerprint in the string signed over with entity key (TODO two tests fail)
* fix test
* ntf server prometheus stats, use Int64 in SOKS/ENDS responses (to avoid conversions), additional error status for ntf subscription
* update RFC
* refactor useServiceAuth to avoid ad hoc decisions about which commands use service signatures, and to prohibit service signatures on other commands
* remove duplicate service signature syntax check from checkCredentials, it is checked in verifyTransmission
* service errors, todos
* fix checkCredentials in ntf server, service errors
* refactor service auth
* refactor
* service agent: store returned queue count instead of expected
* refactor serverThread
* refactor serviceSig
* rename
* refactor, rename, test repeat NSUB service association
* respond with error to SUBS
* smp server: export/import service records between database and store log
* comment
* comments
* ghc 8.10.7
* parameterize transport by peer type (client/server)
* LogDebug level when test is retried
* support "flipped" HTTP2, fix test retry to avoid retrying pending tests
* move sync to the end of the tests
* ntf server, agent: send all periodic notifications from one thread, only to old active clients or new clients with periodic notification mode
* send different type via subscription queues
* option to compact store log on start
* ntf server: skip duplicates when importing tokens and subscriptions
* skip imported last notifications when no token or subscription present
* fix skipped imported notifications count
* all tests
* fix test
* ntf server: allow retries when creating subscriptions, prohibit subscriptions with the same queue but another notifier key or token
* sync files in the test
* refactor
* ntf server: record token invalidation reason, add date of the last token activity
* update time
* rename
* optional
* include token ID in delivery error
* version
* protocol version
* fix, log error
* ntf: get messages for multiple last notifications (#1352)
* ntf: separate get ntf conns api (#1379)
* ntf: separate get ntf conns api
* nonempty
* update
* update
* remove single get api
* fix test
* refactor
* refactor
* ntf: batch get connections (#1387)
* ntf: batch get apis
* works
* fix
* fix
---------
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
* smp server: remove STM function from MsgStore
* polymorphic MsgStore
* jourmal storage for messages (WIP)
* more journal, test setup
* writeMsg
* test
* tryDelMsg
* delMsgQueue
* remove MsgStoreClass instance of existential wrapper for Msg stores
* store config
* extract common logic out of store instances
* add store type to config
* open journals, cache last message, tests pass
* CLI commands
* refactor import/export messages
* cli commands to import/export journal message store
* export journal without draining, import/export tests
* journal command
* import/export progress
* better progress info
* only log queue state once when importing
* logs
* handle IO errors in journal store, return as STORE error
* recover from state file errors
* fix message files after crash
* fix messages folder
* ntf server: control port
* version .405
* control
* use own_server_domains from INI file
* fix subs by server in control port
* bigger queues
* ntf server: only print subscriptions per own server when they are > 0
* fix tests
* revert version change
* dont import listThreads in ghc 8.10.7
* smp-server: Allow serving HTTPS and transport on the same port
* update rfc
* servers: refactor TLS credentials
* provide server credentials in SNI hook
* determine TLS server params dynamically, when starting the server
* remove alpn from TransportServerConfig to decide it dynamically where server is started
* diff
* combine HTTP and SMP on the shared port
* Update to SockAddr
* Fix params and web.https parser
* Switch fork urls
* WIP: add smpServerTestStatic test
* Update warp-tls repo
* shared connection tests
* cleanup
* Add protocol tests
* rename cert file, enable both ports and web by default
* terminate with message on missing credentials
* test cert file
* client option to use port 443 as default SMP port
* use SNI in non-SMP clients
* supported
* remove TODO
* advice
* fix test build
* Add RSA-4096 check for web creds, fix test
* Remove directory listing from static app
* message
* messages
* update log tests
---------
Co-authored-by: IC Rainbow <aenor.realm@gmail.com>
* servers: refactor TLS credentials
* provide server credentials in SNI hook
* determine TLS server params dynamically, when starting the server
* remove alpn from TransportServerConfig to decide it dynamically where server is started
* agent: support socks proxy without isolate-by-auth, with and without credentials
* add unit tests
* make xftp use correct SOCKS credentials
* rename
* support ipv6 in brackets, test parsing
* constant
* textToHostMode
* space
Evgeny Poberezkin