* protocol: refactor types and encoding
* clean
* smp server: batch commands (#1560)
* smp server: batch commands verification into one DB transaction
* ghc 8.10.7
* flatten transmission tuples
* diff
* only use batch logic if there is more than one transmission
* func
* reset NTF service when adding notifier
* version
* Revert "smp server: use separate database pool for reading queues and creating service records (#1561)"
This reverts commit 3df2425162.
* version
* Revert "version"
This reverts commit d80a6b74c5.
* rfc: client certificates for high volume clients (opertors' chat relays, notification servers, service bots)
* client certificates types (WIP)
* parameterize Transport
* protocol/schema/api changes
* agent API
* rename command
* agent subscriptions return local ClientServiceId to chat
* verify transmissions
* fix receiving client certificates, refactor
* ntf server: remove shared queue for all notification subscriptions (#1543)
* ntf server: remove shared queue for all notification subscriptions
* wait for subscriber with timeout
* safer
* refactor
* log
* remove unused
* WIP service subscriptions and associations, refactor
* process service subscriptions
* rename
* simplify switching subscriptions
* SMP service handshake with additional server handshake response
* notification delivery and STM persistence for services
* smp server: database storage, store log, fix encoding for STORE error, replace String with Text in locks and error
* stats
* more stats
* rename SMP commands
* service subscriptions in ntf server agent (tests fail)
* fix
* refactor
* exports
* subscribe ntf server as service for associated queues
* test ntf service connection, fix SOKS response, fix service associations not removed in STM storage
* INI option to support services
* ntf server: downgrade subscriptions when service is no longer supported, track counts of subscribed queues
* smp protocol: include service certificate fingerprint in the string signed over with entity key (TODO two tests fail)
* fix test
* ntf server prometheus stats, use Int64 in SOKS/ENDS responses (to avoid conversions), additional error status for ntf subscription
* update RFC
* refactor useServiceAuth to avoid ad hoc decisions about which commands use service signatures, and to prohibit service signatures on other commands
* remove duplicate service signature syntax check from checkCredentials, it is checked in verifyTransmission
* service errors, todos
* fix checkCredentials in ntf server, service errors
* refactor service auth
* refactor
* service agent: store returned queue count instead of expected
* refactor serverThread
* refactor serviceSig
* rename
* refactor, rename, test repeat NSUB service association
* respond with error to SUBS
* smp server: export/import service records between database and store log
* comment
* comments
* ghc 8.10.7
* agent: option to use web port by default for preset servers only
* shorten/restore short links in agent, add encodings for SMP web port setting
* decouple preset domains from preset servers for short links
* refactor, rename
* ntf server: control port
* version .405
* control
* use own_server_domains from INI file
* fix subs by server in control port
* bigger queues
* ntf server: only print subscriptions per own server when they are > 0
* fix tests
* revert version change
* dont import listThreads in ghc 8.10.7
* smp-server: Allow serving HTTPS and transport on the same port
* update rfc
* servers: refactor TLS credentials
* provide server credentials in SNI hook
* determine TLS server params dynamically, when starting the server
* remove alpn from TransportServerConfig to decide it dynamically where server is started
* diff
* combine HTTP and SMP on the shared port
* Update to SockAddr
* Fix params and web.https parser
* Switch fork urls
* WIP: add smpServerTestStatic test
* Update warp-tls repo
* shared connection tests
* cleanup
* Add protocol tests
* rename cert file, enable both ports and web by default
* terminate with message on missing credentials
* test cert file
* client option to use port 443 as default SMP port
* use SNI in non-SMP clients
* supported
* remove TODO
* advice
* fix test build
* Add RSA-4096 check for web creds, fix test
* Remove directory listing from static app
* message
* messages
* update log tests
---------
Co-authored-by: IC Rainbow <aenor.realm@gmail.com>
* agent: transport isolation mode "Session" (default) to use new SOCKS credentials when client restarts or SOCKS proxy configuration changes
* fix test
* ntf server: only mark subscriptions as pending if the disconnected client is current
* add sessionId to subscribed queue
* add sessionId to subscriptions in ntf server agent
* fix
* agent: aggregate multiple expired subscription responses into a single UP event
* clean up
* refactor processing of expired responses
* refactor
* refactor 2
* refactor unexpectedResponse
* smp: use session vars for reconnecting small agent
* process errors
* split session and protocol functions
* add active flag to agent
* actually invoke agent shutdown
* close proxy agent too
* restore stopping ntf subscribers
* agent: do not mark subscriptions on expired sessions as active, do mark delayed subscriptions as active on the same session, SUBOK response in the next SMP protocol version
* client: prevent sub actions from zombie sessions (#1122)
* client: prevent sub actions from zombie sessions
* error handling
* add AERR to pass background errors to client
* switch to activeClientSession
* put closeClient under activeClientSession
* rename
* remove AERR, do not skip processing
* move check and state update to one transaction
* catch extra UPs
* fix
* check queue is still pending before making it active
---------
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
* do not forward agent error
* revert not expiring sending subs
* fixes
* track subscription responses better
* add pending connection
* Revert "revert not expiring sending subs"
This reverts commit 4310a69391.
* do not expire sending commands
* rename
* fix race
* function
---------
Co-authored-by: Alexander Bondarenko <486682+dpwiz@users.noreply.github.com>
* SMP proxy: low level client and server implementation
* SMP proxy: server implementation (#1098)
* wip
* PRXY command
* progress
* SMP Proxy: client-level implementation (#1101)
* buildable
* encode messages
* update pkey
* fix queue types
* wrap SEND in proxy lookup
* WIP proxy client
* WIP
* post-rebase fixes
* encode something with something
* cleanup
* update
* fix nonce/corrId in batchingTests
* WIP: dig into createSMPProxySession
* agent
* test progress
* pass the test
* parameterize transport handle with transport peer to include server certificate (#1100)
* parameterize transport handle with transport peer to include server certificate
* include server certificate into THandle
* load server chain and sign key
* fix key type
* fix for 8.10
---------
Co-authored-by: Alexander Bondarenko <486682+dpwiz@users.noreply.github.com>
Co-authored-by: IC Rainbow <aenor.realm@gmail.com>
* cleanup
* add 2-server test
* remove subsumed test
* checkCredentials for BrokerMsg
* skip batching tests
* remove userId param
* remove agent changes
---------
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
---------
Co-authored-by: Alexander Bondarenko <486682+dpwiz@users.noreply.github.com>
* remove unused type
* icrease test timeout
* reduce transport block
* envelope sizes
* don't fork unless have proxied commands to process
---------
Co-authored-by: Alexander Bondarenko <486682+dpwiz@users.noreply.github.com>
Co-authored-by: IC Rainbow <aenor.realm@gmail.com>
* smp: command authorization
* fix encoding, most tests
* remove old tests
* authorize via crypto_box
* extract authenticator to Crypto module
* make TransmissionAuth Maybe
* rfc
* support authenticators in NTF protocol, test matrix (no backwards compatibility yet from new clients to old servers)
* fix/add tests, add version config to "small" agent
* separate client and server versions for SMP protocol
* test batching SMP v7
* do not send session ID in each transmission
* refactor auth verification in the server, split tests
* server "warm up" fixes timing test
* uncomment SUB timing test
* comments, disable two timing tests
* rename version
* increase auth timing test failure threshold
* use different algorithms to authorize snd/rcv commands, use random correlation ID
* transport: fetch and store server certificate (#985)
* THandleParams (WIP, does not compile)
* transport: fetch and store server certificate
* smp: add getOnlinePubKey example to smpClientHandshake
* add server certs and sign authPub
* cleanup
* update
* style
* load server certs from test fixtures
* sign ntf authPubKey
* fix onServerCertificate
* increase delay before sending messages
* require certificate with key in SMP server handshake
---------
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
* remove dhSecret from THandle
* remove v8, merge all changes to one version
* parameterize THandle
* rfc: transmission ecnryption
* Revert "parameterize THandle"
This reverts commit 75adfc94fb.
* use batch syntax for ntf server commands
* separate encodeTransmission when there is no key
* typo
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
* rename
* diff
---------
Co-authored-by: Alexander Bondarenko <486682+dpwiz@users.noreply.github.com>
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
* ntf server: better batching and logging
* reduce batch delay for ntf server
* comments
* 5.1.3, ntf 1.4.2
* more logging
* more logging
* split large batches, more logging
* remove some logs
* execute asynchronous commands for correct users
* pass transport session to message processing to avoid race condition
* account for server changes when determining unused servers
* enable one test
* enable all tests, remove log
* store SMP client version per queue
* use versioned format for queue addresses (not compatible with v1)
* multiple server hosts in queue URI/data, versioned encoding (cross-version tests fail)
* choose server host based on socksProxy setting
* test works
* multiple server addresses mostly work
* add onion hosts for provided servers
* update SMP hosts
