Evgeny Poberezkin
a7a467196f
smp: command authorization
2024-02-05 09:07:27 +00:00
Alexander Bondarenko
f7cdec2f08
fix: support multiple notification servers in configuration ( #971 )
...
* tests: add trpl-keeps-server check
* add smp server switch check
* add connection test and fix withNtfServer
* Update src/Simplex/Messaging/Agent/NtfSubSupervisor.hs
Co-authored-by: Evgeny Poberezkin <e.poberezkin@me.com >
* use ntfServer from token
* rename
---------
Co-authored-by: Evgeny Poberezkin <e.poberezkin@me.com >
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com >
2024-01-19 19:42:28 +00:00
Evgeny Poberezkin
baf2c47065
agent: expire messages failed after quota exceeded after 7 days, and expire multiple messages at once ( #973 )
...
* agent: expire messages failed after quota exceeded after 7 days, and expire multiple messages at once
* fix, test
* refactor
* catch in loop
2024-01-19 17:21:25 +00:00
Alexander Bondarenko
40fa34c2d5
tests: test multiple Ntf servers ( #966 )
...
* tests: add ntf case with multiple Ntf servers
* simplify test
* fix for master
* add server switch test
* add server switch test
* add message test for ntf server switch
2024-01-17 19:47:47 +00:00
Alexander Bondarenko
eb41abfb8f
tests: fix random order in async commands case ( #969 )
2024-01-17 11:48:41 +00:00
Evgeny Poberezkin
b547f34cc0
Revert "Revert "agent: do not create user record in new databases ( #957 )" ( #963 )" ( #964 )
...
* Revert "Revert "agent: do not create user record in new databases (#957 )" (#963 )"
This reverts commit aee9088417
2024-01-17 11:01:04 +00:00
Evgeny Poberezkin
00c4ff4a21
agent: fix race condition in subscription worker (results in stuck delivery), better error handling in batch operations ( #962 )
...
* debugging test: join connection when reply queue creation fails
* more logs
* more logs
* more logs, fixed bug
* cleanup
* catch all errors
* simplify
* comment
* remove client ID counter
2024-01-16 10:29:35 +00:00
Evgeny Poberezkin
aee9088417
Revert "agent: do not create user record in new databases ( #957 )" ( #963 )
...
This reverts commit 9b9a0bd0df
2024-01-14 22:28:28 +00:00
Alexander Bondarenko
68f5e189a6
smp: check for progress when resubscribing ( #956 )
...
* smp: check for progress when resubscribing
* add allowClose to reconnectServer to distinguish entry points
* resolve error todo
* make reconnect async per-session
* remove allowClose
* deregister reconnecter when it finishes
* signal/react more work explicitly
* fix restart condition
* wait for reconnecter to finish
* remove redundant reconnect locks
* rename getClientVar for expanded scope
* formatting
* remove withPending
* move pending check to tryReconnectSMPClient loop
* combine pending check and slot release transactions
* actually reserve the slot for async
* simplify
* refactor
* refactor
* use removeClientVar
* rename
* refactor
* test
* reduce MonadError scope
---------
Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com >
2024-01-12 19:29:18 +00:00
Evgeny Poberezkin
9b9a0bd0df
agent: do not create user record in new databases ( #957 )
...
* agent: do not create user record in new databases
* update
2024-01-12 16:57:57 +00:00
Evgeny Poberezkin
ca527b4d6c
use Builder for batching commands ( #953 )
...
* use Builder for batching commands
* refactor
* refactor
* increase timeout in test
* do not materialize builder
* use concat
* debug test
* debug2
* delays
* more delay, less worker delay
* remove logs
* reduce delay
* tests
* sized builder
* builder
* comment
* remove unsafe
* style
* remove fromIntegral
2024-01-08 22:54:27 +00:00
spaced4ndy
e43e4860b9
xftp: file workers cycle through pending files based on retries count (limit number of iterations per work item to prevent stuck delivery) ( #930 )
...
* xftp: test file reception - shouldn't get stuck if file is deleted on server
* comment
* expiration test
* approach
* wip
* sort by retries in other works, revert some diff
* revert diff
* modify tests
* refactor
* refactor
* remove prints
* apply to other workers
* remove import
* comment
* refactor
* revert queue size
* fix test
* rename
* comment, correct number of retries
---------
Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com >
2023-12-26 22:20:25 +00:00
Evgeny Poberezkin
577e3cf14d
agent: add server msg IDs to agent logs, disable 2 tests ( #934 )
...
* agent: race when using agent via TCP
* remove logs
* enable tests
* comments
* use different databases for different clients
* enable all tests
2023-12-26 17:50:39 +00:00
Evgeny Poberezkin
7ddeca50e4
agent: mark work items failed ( #931 )
...
* agent: mark work items failed (WIP)
* add tests, created_at
* getWorkItem for snd and rcv files
* fix
* tests
* fix
* tests
* test
* tests
* rename
* fix,refactor
* add indexes
* update schema
* do not try to get more work when resuming an existing worker
---------
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
2023-12-26 17:00:42 +04:00
Evgeny Poberezkin
2489333c87
agent: use DB to queue async commands and messages ( #929 )
...
* agent: use DB to queue async commands and messages
* fix message delivery, async commands dont work
* fix async commands
* remove comment
* rename
* comment
* balance agent operation
* empty lines
* balance another agent operation
* diff
* worker
* qAddress
* fix incorrect queue ID in v1
* types for queues
* fix test
* update index
2023-12-23 12:59:53 +00:00
Evgeny Poberezkin
1c2604f6a3
agent: background mode for agent NSE ( #924 )
2023-12-22 08:59:07 +00:00
Evgeny Poberezkin
13a60d1d39
use ChaChaDRG as the source of randomness ( #920 )
...
* use ChaChaDRG as the source of randomness
* remove functions using entropy directly
* comment
2023-12-21 00:12:08 +00:00
Alexander Bondarenko
7c27357eb2
server: stop inactive clients without subscriptions, additional monitoring via control port ( #901 )
...
* fix thread labelling points
* add subscription thread labels
* add explicit sections for sockets/sessions/disconnects
* adjust label threads wrt. next blocking op
* WIP: emit event on server finish
So the event log wouldn't be cut short giving
false impression of unterminated sessions.
* add handshake timeout
* hack server socket stats
* trace closeTLS exceptions
* fix build
* clean up disconnectTransport
* make disconnectTransport terminating
Thus, participating in racyAny_ around it.
* trach snd/rcv time separately, log clients as CSV
* add direct client counting/stats
* add csv dump for socket threads
* guard socket threads dump from older GHC
* cut socket thread listing into socket-threads command
* store client creation time for stats
* wrap tls setup in timeout and labels
* server: expire clients without subscriptions
* fix check for subscriptions
* remove excess tracing
* move hardcoded timeouts to configs
* cleanup
* fix tests
* wrap direct queue waits in timeout
* rewrite old TCP wrappers with their SocketState variants
* add testInactiveWithSubs
---------
Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com >
2023-12-19 15:50:45 +00:00
Evgeny Poberezkin
560dc55312
agent: notify about polled message processing (for iOS notifications) ( #908 )
...
* agent: notify about polled message processing (for iOS notifications)
* optionally keep key and support re-opening database
* exports
* test that cannot reopen when created with keepKey: false
* set max number of messages to receive for a notification to 3
2023-12-11 13:03:53 +00:00
Evgeny Poberezkin
248144f3de
do not use previous KEM shared secret in multicast packets and in host HELLO ( #888 )
...
* do not use previous KEM shared secret in multicast packets and in host HELLO
* simplify
2023-11-08 12:06:10 +00:00
Evgeny Poberezkin
4660ee9cff
agent: revert to TVar ( #883 )
2023-11-01 09:15:51 +00:00
Evgeny Poberezkin
b5f733d2db
agent: use IORef for DRG ( #882 )
2023-10-31 23:52:13 +00:00
Evgeny Poberezkin
cf8b9c12ff
remove test dependencies ( #861 )
...
* remove test dependencies
* .hlint.yaml
2023-10-22 10:50:36 +01:00
Evgeny Poberezkin
9d12d76078
add fourmolu ( #868 )
...
* add fourmolu
* linebreak
* simplify
* comment
2023-10-22 09:20:14 +01:00
spaced4ndy
80905c0822
test concurrent send and receipts ( #866 )
2023-10-19 20:08:58 +04:00
spaced4ndy
1ad69cf74f
agent: always parse connection request uri as simplex:/ ( #863 )
2023-10-16 18:25:56 +04:00
Evgeny Poberezkin
fda1284ae4
agent: support closing and re-opening store ( #855 )
...
* agent: support closing and re-opening store
* add closed flag, tests
2023-09-27 12:50:24 +01:00
Evgeny Poberezkin
887ccbcf6c
Merge branch 'master' into master-ghc9
2023-09-10 18:00:33 +01:00
Evgeny Poberezkin
84ce037867
extend SMP protocol to allow creating new queues without subscriptions ( #839 )
...
* Trace auto-subs flag
* Replace Bools with SubscriptionMode
* Handle SMOnlyCreate
* Wire remaining todos
* Update tests and fix
* Bump protocol level
* Apply suggestions from code review
Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com >
* Scrub needs_sub from agent DB
* Scrub a few more needSubs from the agent api
* change API, fix test
* agent: do not subscribe to queue when creating reply queue
* fix encoding
* WIP: SMOnlyCreate test
* Add SM guard for confirmQueue
Allows the test case to pump the allowConnection
reply without getting PROHIBITED.
* Remove tracing
* add noMessages, remove unnecessary getConnectionMessage from test
* add sending messages to the test
---------
Co-authored-by: IC Rainbow <aenor.realm@gmail.com >
Co-authored-by: Alexander Bondarenko <486682+dpwiz@users.noreply.github.com >
2023-09-10 17:07:19 +01:00
Moritz Angermann
002f36dde0
5.4.0.0: use ghc 9.6.2 ( #791 )
...
* Add 9.6 compat
* compile with GHC9.6.2: dependencies, imports, code
* refactor typeclasses
* refactor record dot
* update cabal version
* update github actions
* update direct-sqlcipher
* 5.4.0.0
* update cabal.project
---------
Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com >
2023-08-24 20:22:02 +01:00
Evgeny Poberezkin
b001b748db
agent: rename functions, remove unused code, stabilize tests ( #830 )
...
* agent: rename functions, stabilize tests
* test pattern match
2023-08-24 19:52:58 +01:00
Evgeny Poberezkin
d09adb5cd6
agent: wait for db (Revert Revert db error busy treatments ( #796 ) ( #808 )) ( #828 )
...
* Revert "Revert "db error busy treatments (#796 )" (#808 )"
This reverts commit 82aec2cd8f
2023-08-21 09:06:51 +01:00
Evgeny Poberezkin
b329eb9a86
agent: fix async JOIN ( #827 )
...
* agent: fix async JOIN
* refactor
2023-08-20 18:45:16 +01:00
spaced4ndy
e586bef57a
agent: add indexes to improve slow queries performance ( #823 )
2023-08-16 10:29:09 +04:00
Evgeny Poberezkin
e2065ab352
agent: track slow SQL queries ( #822 )
...
* agent: track slow SQL queries
* add executeMany
* reduce threshold for slow queries to 50ms
2023-08-12 18:18:10 +01:00
Evgeny Poberezkin
82aec2cd8f
Revert "db error busy treatments ( #796 )" ( #808 )
...
This reverts commit 1afcefa5e7
2023-08-09 12:24:03 +01:00
Evgeny Poberezkin
58cb2855d2
agent: delivery receipts ( #752 )
...
* rfc: delivery receipts
* update doc
* update rfc
* implementation plan, types, schema
* migration, update types
* update types
* rename migration
* export MsgReceiptStatus, JSON encoding
* update rfc, schema
* correction
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
* skeleton of the implementation
* more implementation (some tests fail)
* more code, 1 test fails
* fix encoding
* refactor
* refactor
* test, fix
* only send receipts in v3+, test
* flip condition
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
* flip condition
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
* agent version 4 required to send receipts
* fix test
---------
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
2023-07-13 22:33:48 +01:00
spaced4ndy
1afcefa5e7
db error busy treatments ( #796 )
2023-07-11 20:19:20 +04:00
spaced4ndy
f2657f9c0b
add crypto error to RSYNC event ( #794 )
2023-07-05 18:14:51 +04:00
spaced4ndy
0e1562deae
increase connection agent version according to received messages; add version and flags to connection stats, bump currentSMPAgentVersion ( #789 )
2023-06-30 20:19:10 +04:00
spaced4ndy
8be2505fa0
ratchet re-synchronization ( #774 )
...
* ratchet re-synchronization rfc wip
* additions
* additions, types
* fix tests
* re-sync implementation wip
* re-sync implementation 1st rev.
* test wip
* test passes
* doc
* wording
* improve doc schema
* single agreed state
* refactor (1 state variable)
* allowed -> required
* prohibit enqueue
* enqueue
* send via multiple queues
* test with server offline
* clarify errors
* rename
* more tests
* refactor
* rename AgentRKey
* rename AM_CONN_RATCHET_KEY
* more tests
* rename
* write encoded AgentRatchetInfo to AgentRatchetKey info
* move withConnLock
* refactor qDuplex
* re-create ratchet on receiving second key
* invert condition
* refactor
* simplify
---------
Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com >
2023-06-30 14:17:08 +04:00
spaced4ndy
a000419bd7
restore confirmation after client restart ( #783 )
2023-06-29 10:10:34 +04:00
spaced4ndy
c8a8e2c297
differentiate agent INACTIVE error, treat as temporary ( #784 )
2023-06-29 10:00:53 +04:00
Evgeny Poberezkin
4a927d1ae2
ntf server: better batching and logging ( #780 )
...
* ntf server: better batching and logging
* reduce batch delay for ntf server
* comments
* 5.1.3, ntf 1.4.2
* more logging
* more logging
* split large batches, more logging
* remove some logs
2023-06-26 20:14:35 +01:00
Evgeny Poberezkin
acea477aab
batch subscriptions for ntf server ( #778 )
...
* return updated ConnectionStats from switchConnectionAsync (#777 )
* batch ntf server subscriptions to SMP servers
* refactor
* fix filtering queues in reconnect
* test, remove repeated grouping
* enable all tests
* reduce subscriptions count
---------
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
2023-06-25 08:27:50 +01:00
spaced4ndy
c31668ce28
return updated ConnectionStats from switchConnectionAsync ( #777 )
2023-06-19 14:51:14 +04:00
spaced4ndy
2efe1496d2
rename "stop switch" -> "abort switch", add canAbortSwitch to rcv queue info ( #775 )
2023-06-16 13:41:01 +04:00
spaced4ndy
011e1109e9
connection switch - track status, stop switch command ( #764 )
...
* connection switch improvements
* stop in transaction
* refactor
* checks in transaction
* withSwitchingRQ in transaction
* withSwitchedSQ in transaction
* refactor
* refactor
* inverse some cases
* reset snd switch progress on repeat qadd
* remove unused functions
* connection info type
* correction
* SPFinalizing phase, correctly update connection in event
* refactor
* tests
* refactor switch conn lock taking, take lock in switch stop, remove ineligible test
* comment
* corrections
* corrections
* corrections
* refactor
* remove checks in transactions, remove some statuses
* remove diff
* refactor
* refactor tests
* update connection in memory
* revert diff
* corrections
* more tests
* remove statuses, change switching queue lookup
* async queue deletion
* refactor
* lock, phases
* test status
* refactor
---------
Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com >
2023-06-15 11:15:46 +04:00
spaced4ndy
fbf1afc9f5
run a single batch test to avoid hanging on server disconnect ( #773 )
2023-06-13 19:36:47 +04:00
spaced4ndy
b747080db3
add more message delivery tests ( #763 )
2023-06-02 18:00:24 +04:00
