* smp server: queue store typeclass
* parameterize JournalMsgStore
* typeclass for queue store
* postgres WIP
* compiles, passes tests
* remove StoreType
* split migrations
* progress
* addQueueRec
* reduce type spaghetti
* remove addQueue from typeclass definition
* getQueue
* test postgres storage in SMP server
* fix schema
* comment
* import queues to postgresql
* import queues to postgresql
* log
* fix test
* counts
* ci: test smp server with postgres backend (#1463)
* ci: test smp server with postgres backend
* postgres service
* attempt
* attempt
* empty
* empty
* PGHOST attempt
* PGHOST + softlink attempt
* only softlink attempt
* working attempt (PGHOST)
* remove env var
* empty
* do not start server without DB schema, do not import when schema exists
* export database
* enable all tests, disable two tests
* option for migration confirmation
* comments
---------
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
* agent: store shared message body only once (when it is the same across messages when batching)
* rename
* refactor
* refactor
* save bodies and messages in single transaction
* comment
* comment
* comment
* box
* mapME
* box
* ValueOrRef
* remove instances
* refactor
* comments
* test
* refactor
* mapAccumLM compatibility with ghc 8.10.7
---------
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
* smp server: combine queue and message store into one class (WIP)
* keep deleted queue tombstones to prevent race conditions and errors when restoring
* move store log from server to store implementations
* STMQueueStore type class
* fix store closed when messages expired, handle store writing errors
* types
* version
* fix recovery from missing write journal, tests
* version
* smp server: remove STM function from MsgStore
* polymorphic MsgStore
* jourmal storage for messages (WIP)
* more journal, test setup
* writeMsg
* test
* tryDelMsg
* delMsgQueue
* remove MsgStoreClass instance of existential wrapper for Msg stores
* store config
* extract common logic out of store instances
* add store type to config
* open journals, cache last message, tests pass
* CLI commands
* refactor import/export messages
* cli commands to import/export journal message store
* export journal without draining, import/export tests
* journal command
* import/export progress
* better progress info
* only log queue state once when importing
* logs
* handle IO errors in journal store, return as STORE error
* recover from state file errors
* fix message files after crash
* fix messages folder
* smp server: fewer map updates on re-subscriptions
* temp version
* replace Client with ClientId in queues
* version
* version
* comments
* reduce threads when sending ENDs
* revert version
* put DRG state to IORef, split STM transaction of sending notification (#1288)
* put DRG state to IORef, split STM transaction of sending notification
* remove comment
* remove comment
* add comment
* revert version
* newtype for server entity IDs, fix TRcvQueues
* Revert "put DRG state to IORef, split STM transaction of sending notification (#1288)"
This reverts commit 517933d189.
* logServer
* server: add control port commands for clients and ghc threads (#835)
* Add stats-rts control query
With supporting ghc-options that would provide the data.
* Add CPSkip command
Allows spamming empty lines a few times to clean up the view.
* server: Add CP commands to enumerate clients and threads
* style
---------
Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
* use base64 encoding for session ID
* fromMaybe
* whitespace
---------
Co-authored-by: Alexander Bondarenko <486682+dpwiz@users.noreply.github.com>
* agent: catch IO errors correctly in MonadError
* correction
* correction
* utils
* agentFinally to catch IO exceptions in ExceptT
* rename
* remove, inline
* rename utils
* utils unit test
* test to show catch and finally problems
* tryAllErrors
* enable all tests
* ntf server: better batching and logging
* reduce batch delay for ntf server
* comments
* 5.1.3, ntf 1.4.2
* more logging
* more logging
* split large batches, more logging
* remove some logs
* add optional auxiliary data field to ConnectionRequestUri
* remove import
* fix, test
* fix StrEncoding Char
* data only in sync command, type
* fixing
* queryParamStr
* safeDecodeUtf8
Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
* batch server commands in one transport block
* subscribe to multiple queues using batched commands
* agent method to subscribe to multiple queues using batched commands
* refactor
* test for batched subscriptions
* delete part of connections in batched test
* add resubscribeConnections
* remove comment
* update SMP protocol doc
* support stopping and resuming agent (#385)
* export agentDbPath
* support fully closing and resuming agent
* whitespace
* clean up
* support message flags visible to SMP server to control notifications (and for any future extensions)
* simplify message flags encoding
* GET command
* primary tokens
* support repeat token registration (TODO fix tests)
* fix notifications tests
* fix/test repeat/new registrations of the same token
* re-register token when subsequent ntf command fails with AUTH error (e.g. when server is re-started)
* cancel periodic notifications when token is deleted on the server
* debug failing test on CI
* fix notification test in CI
* debug CI test
* add delay in notificaitons test after server restart
* separate skipped messages from ratchet
* return diff for skipped messages instead of the whole state (tests fail)
* fix ratchet tests
* JSON encoding/decoding for ratchet state
* schema for ratchets
* split MonadUnliftIO instance for ExceptT to a separate file
* update StrEncoding instance for Str
* ratchet store methods
* updateRatchet store method
* move E2E ratchet params to Ratchet module
* x3dh key agreement for double ratchet
* test/fix x3dh, use x3dh for ratchets initialization
* store/get x3dh keys, save ratchet of fixed X448 type
* double-ratchet encryption integration (tests fail)
* fix double ratchet
* fix padding and message length
* remove unused code for "activations"
* remove comment
* add version checks for forward/backward compatibility
* split loading ratchet and skipped message keys
* remove unused encoding instances for Algorithm types
* update ratchet initialization params
* make KeyHash non-optional, StrEncoding class
* change server URI format in agent config, refactor with StrEncoding
* refactor Crypto using checkAlgorithm
* refactor parsing connection requests
* prepare to validate CA fingerprint sent in client handshake
* KeyHash check in handshake
* rename type to CliCommand
* server validates keyhash sent by the client
* validate -a option when parsing
* more of StrEncoding
* SMP commands for notifications (LSTN, NTFY) with separate queue IDs and keys
* rename Notifier types
* remove notify key and id from NEW and IDS commands (TODO add other commands)
* fix StoreLog serialization
* add commands for managing notifications
* add notification subscribers to server state, add notifier ID and key to store log
* add notifier ID and key to the queue
* refactor END notification to work for both types of subscriptions, deliver message notification (NMSG)
* process NSUB command - subscribe to message notifications
* test for message notifications
* fix SMP client function for NSUB command
* fix parse/serialize NID command
* refactor use ifM
* check duplicate notifier ID only against other notifier IDs
* refactor getQueue
* test notifier ID and key with store log
* Update src/Simplex/Messaging/Client.hs
Co-authored-by: Efim Poberezkin <8711996+efim-poberezkin@users.noreply.github.com>
* Update src/Simplex/Messaging/Server.hs
Co-authored-by: Efim Poberezkin <8711996+efim-poberezkin@users.noreply.github.com>
* store log: s/NOTIFY/NOTIFIER/
Co-authored-by: Efim Poberezkin <8711996+efim-poberezkin@users.noreply.github.com>
Evgeny