dandri/simplexmq
1
0
Fork 0
mirror of https://github.com/simplex-chat/simplexmq.git synced 2026-04-24 14:26:26 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,247 Commits 93 Branches 173 Tags
fa848decd2922cd00a3fccf491cd7a3d72c2ee24
Commit Graph

101 Commits

Author SHA1 Message Date
Evgeny Poberezkin
baf2c47065 agent: expire messages failed after quota exceeded after 7 days, and expire multiple messages at once (#973) 
* agent: expire messages failed after quota exceeded after 7 days, and expire multiple messages at once

* fix, test

* refactor

* catch in loop
 2024-01-19 17:21:25 +00:00
Alexander Bondarenko
eb41abfb8f tests: fix random order in async commands case (#969) 2024-01-17 11:48:41 +00:00
Evgeny Poberezkin
b547f34cc0 Revert "Revert "agent: do not create user record in new databases (#957)" (#963)" (#964) 
* Revert "Revert "agent: do not create user record in new databases (#957)" (#963)"

This reverts commit aee9088417.

* tests

* remove logging
 2024-01-17 11:01:04 +00:00
Evgeny Poberezkin
00c4ff4a21 agent: fix race condition in subscription worker (results in stuck delivery), better error handling in batch operations (#962) 
* debugging test: join connection when reply queue creation fails

* more logs

* more logs

* more logs, fixed bug

* cleanup

* catch all errors

* simplify

* comment

* remove client ID counter
 2024-01-16 10:29:35 +00:00
Evgeny Poberezkin
aee9088417 Revert "agent: do not create user record in new databases (#957)" (#963) 
This reverts commit 9b9a0bd0df.
 2024-01-14 22:28:28 +00:00
Alexander Bondarenko
68f5e189a6 smp: check for progress when resubscribing (#956) 
* smp: check for progress when resubscribing

* add allowClose to reconnectServer to distinguish entry points

* resolve error todo

* make reconnect async per-session

* remove allowClose

* deregister reconnecter when it finishes

* signal/react more work explicitly

* fix restart condition

* wait for reconnecter to finish

* remove redundant reconnect locks

* rename getClientVar for expanded scope

* formatting

* remove withPending

* move pending check to tryReconnectSMPClient loop

* combine pending check and slot release transactions

* actually reserve the slot for async

* simplify

* refactor

* refactor

* use removeClientVar

* rename

* refactor

* test

* reduce MonadError scope

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
 2024-01-12 19:29:18 +00:00
Evgeny Poberezkin
9b9a0bd0df agent: do not create user record in new databases (#957) 
* agent: do not create user record in new databases

* update
 2024-01-12 16:57:57 +00:00
Evgeny Poberezkin
577e3cf14d agent: add server msg IDs to agent logs, disable 2 tests (#934) 
* agent: race when using agent via TCP

* remove logs

* enable tests

* comments

* use different databases for different clients

* enable all tests
 2023-12-26 17:50:39 +00:00
Evgeny Poberezkin
7ddeca50e4 agent: mark work items failed (#931) 
* agent: mark work items failed (WIP)

* add tests, created_at

* getWorkItem for snd and rcv files

* fix

* tests

* fix

* tests

* test

* tests

* rename

* fix,refactor

* add indexes

* update schema

* do not try to get more work when resuming an existing worker

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
 2023-12-26 17:00:42 +04:00
Evgeny Poberezkin
2489333c87 agent: use DB to queue async commands and messages (#929) 
* agent: use DB to queue async commands and messages

* fix message delivery, async commands dont work

* fix async commands

* remove comment

* rename

* comment

* balance agent operation

* empty lines

* balance another agent operation

* diff

* worker

* qAddress

* fix incorrect queue ID in v1

* types for queues

* fix test

* update index
 2023-12-23 12:59:53 +00:00
Evgeny Poberezkin
1c2604f6a3 agent: background mode for agent NSE (#924) 2023-12-22 08:59:07 +00:00
Alexander Bondarenko
7c27357eb2 server: stop inactive clients without subscriptions, additional monitoring via control port (#901) 
* fix thread labelling points

* add subscription thread labels

* add explicit sections for sockets/sessions/disconnects

* adjust label threads wrt. next blocking op

* WIP: emit event on server finish

So the event log wouldn't be cut short giving
false impression of unterminated sessions.

* add handshake timeout

* hack server socket stats

* trace closeTLS exceptions

* fix build

* clean up disconnectTransport

* make disconnectTransport terminating

Thus, participating in racyAny_ around it.

* trach snd/rcv time separately, log clients as CSV

* add direct client counting/stats

* add csv dump for socket threads

* guard socket threads dump from older GHC

* cut socket thread listing into socket-threads command

* store client creation time for stats

* wrap tls setup in timeout and labels

* server: expire clients without subscriptions

* fix check for subscriptions

* remove excess tracing

* move hardcoded timeouts to configs

* cleanup

* fix tests

* wrap direct queue waits in timeout

* rewrite old TCP wrappers with their SocketState variants

* add testInactiveWithSubs

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
 2023-12-19 15:50:45 +00:00
Evgeny Poberezkin
560dc55312 agent: notify about polled message processing (for iOS notifications) (#908) 
* agent: notify about polled message processing (for iOS notifications)

* optionally keep key and support re-opening database

* exports

* test that cannot reopen when created with keepKey: false

* set max number of messages to receive for a notification to 3
 2023-12-11 13:03:53 +00:00
Evgeny Poberezkin
248144f3de do not use previous KEM shared secret in multicast packets and in host HELLO (#888) 
* do not use previous KEM shared secret in multicast packets and in host HELLO

* simplify
 2023-11-08 12:06:10 +00:00
Evgeny Poberezkin
cf8b9c12ff remove test dependencies (#861) 
* remove test dependencies

* .hlint.yaml
 2023-10-22 10:50:36 +01:00
Evgeny Poberezkin
9d12d76078 add fourmolu (#868) 
* add fourmolu

* linebreak

* simplify

* comment
 2023-10-22 09:20:14 +01:00
spaced4ndy
80905c0822 test concurrent send and receipts (#866) 2023-10-19 20:08:58 +04:00
Evgeny Poberezkin
887ccbcf6c Merge branch 'master' into master-ghc9 2023-09-10 18:00:33 +01:00
Evgeny Poberezkin
84ce037867 extend SMP protocol to allow creating new queues without subscriptions (#839) 
* Trace auto-subs flag

* Replace Bools with SubscriptionMode

* Handle SMOnlyCreate

* Wire remaining todos

* Update tests and fix

* Bump protocol level

* Apply suggestions from code review

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>

* Scrub needs_sub from agent DB

* Scrub a few more needSubs from the agent api

* change API, fix test

* agent: do not subscribe to queue when creating reply queue

* fix encoding

* WIP: SMOnlyCreate test

* Add SM guard for confirmQueue

Allows the test case to pump the allowConnection
reply without getting PROHIBITED.

* Remove tracing

* add noMessages, remove unnecessary getConnectionMessage from test

* add sending messages to the test

---------

Co-authored-by: IC Rainbow <aenor.realm@gmail.com>
Co-authored-by: Alexander Bondarenko <486682+dpwiz@users.noreply.github.com>
 2023-09-10 17:07:19 +01:00
Moritz Angermann
002f36dde0 5.4.0.0: use ghc 9.6.2 (#791) 
* Add 9.6 compat

* compile with GHC9.6.2: dependencies, imports, code

* refactor typeclasses

* refactor record dot

* update cabal version

* update github actions

* update direct-sqlcipher

* 5.4.0.0

* update cabal.project

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
 2023-08-24 20:22:02 +01:00
Evgeny Poberezkin
b001b748db agent: rename functions, remove unused code, stabilize tests (#830) 
* agent: rename functions, stabilize tests

* test pattern match
 2023-08-24 19:52:58 +01:00
Evgeny Poberezkin
b329eb9a86 agent: fix async JOIN (#827) 
* agent: fix async JOIN

* refactor
 2023-08-20 18:45:16 +01:00
Evgeny Poberezkin
58cb2855d2 agent: delivery receipts (#752) 
* rfc: delivery receipts

* update doc

* update rfc

* implementation plan, types, schema

* migration, update types

* update types

* rename migration

* export MsgReceiptStatus, JSON encoding

* update rfc, schema

* correction

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* skeleton of the implementation

* more implementation (some tests fail)

* more code, 1 test fails

* fix encoding

* refactor

* refactor

* test, fix

* only send receipts in v3+, test

* flip condition

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* flip condition

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* agent version 4 required to send receipts

* fix test

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
 2023-07-13 22:33:48 +01:00
spaced4ndy
f2657f9c0b add crypto error to RSYNC event (#794) 2023-07-05 18:14:51 +04:00
spaced4ndy
0e1562deae increase connection agent version according to received messages; add version and flags to connection stats, bump currentSMPAgentVersion (#789) 2023-06-30 20:19:10 +04:00
spaced4ndy
8be2505fa0 ratchet re-synchronization (#774) 
* ratchet re-synchronization rfc wip

* additions

* additions, types

* fix tests

* re-sync implementation wip

* re-sync implementation 1st rev.

* test wip

* test passes

* doc

* wording

* improve doc schema

* single agreed state

* refactor (1 state variable)

* allowed -> required

* prohibit enqueue

* enqueue

* send via multiple queues

* test with server offline

* clarify errors

* rename

* more tests

* refactor

* rename AgentRKey

* rename AM_CONN_RATCHET_KEY

* more tests

* rename

* write encoded AgentRatchetInfo to AgentRatchetKey info

* move withConnLock

* refactor qDuplex

* re-create ratchet on receiving second key

* invert condition

* refactor

* simplify

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
 2023-06-30 14:17:08 +04:00
spaced4ndy
a000419bd7 restore confirmation after client restart (#783) 2023-06-29 10:10:34 +04:00
spaced4ndy
c8a8e2c297 differentiate agent INACTIVE error, treat as temporary (#784) 2023-06-29 10:00:53 +04:00
Evgeny Poberezkin
4a927d1ae2 ntf server: better batching and logging (#780) 
* ntf server: better batching and logging

* reduce batch delay for ntf server

* comments

* 5.1.3, ntf 1.4.2

* more logging

* more logging

* split large batches, more logging

* remove some logs
 2023-06-26 20:14:35 +01:00
spaced4ndy
c31668ce28 return updated ConnectionStats from switchConnectionAsync (#777) 2023-06-19 14:51:14 +04:00
spaced4ndy
2efe1496d2 rename "stop switch" -> "abort switch", add canAbortSwitch to rcv queue info (#775) 2023-06-16 13:41:01 +04:00
spaced4ndy
011e1109e9 connection switch - track status, stop switch command (#764) 
* connection switch improvements

* stop in transaction

* refactor

* checks in transaction

* withSwitchingRQ in transaction

* withSwitchedSQ in transaction

* refactor

* refactor

* inverse some cases

* reset snd switch progress on repeat qadd

* remove unused functions

* connection info type

* correction

* SPFinalizing phase, correctly update connection in event

* refactor

* tests

* refactor switch conn lock taking, take lock in switch stop, remove ineligible test

* comment

* corrections

* corrections

* corrections

* refactor

* remove checks in transactions, remove some statuses

* remove diff

* refactor

* refactor tests

* update connection in memory

* revert diff

* corrections

* more tests

* remove statuses, change switching queue lookup

* async queue deletion

* refactor

* lock, phases

* test status

* refactor

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
 2023-06-15 11:15:46 +04:00
spaced4ndy
fbf1afc9f5 run a single batch test to avoid hanging on server disconnect (#773) 2023-06-13 19:36:47 +04:00
spaced4ndy
b747080db3 add more message delivery tests (#763) 2023-06-02 18:00:24 +04:00
Evgeny Poberezkin
ed0f8c79a2 xftp: wait for agent in foreground in XFTP workers without delaying suspension (#721) 
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
 2023-04-13 20:04:38 +04:00
Evgeny Poberezkin
9f8db13553 xftp: agent API to set and test servers (#704) 
* xftp: agent API to set and test servers

* ProtocolTestStep

* update agent API for XFTP servers

* ci: update ubuntu versions

* disable test hanging on ubuntu
 2023-04-05 20:37:03 +01:00
Evgeny Poberezkin
6a665a0833 support down migrations (#696) 
* support down migrations

* test

* update

* add down migrations and tests

* get current migrations

* corrections

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* update tests

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
 2023-03-27 12:44:53 +01:00
Evgeny Poberezkin
ddc2da8fe4 xftp: agent API for xftp commands and events (#684) 
* xftp: agent API for xftp commands and events

* fix tests

* fix tests 2

* xftp: update agent send api

* update API to make temp path optional

* revert tmp path changes (fixes send)

---------

Co-authored-by: spacedandy <8711996+spaced4ndy@users.noreply.github.com>
 2023-03-13 09:04:23 +00:00
spaced4ndy
c1266d55a4 xftp: experimental send api wip (#682) 2023-03-10 20:43:10 +04:00
Evgeny Poberezkin
8fde8e1344 xftp: agent command entities (#676) 
* xftp: agent command entities

* progress event

* parameterize agent command by entity

* refactor

* Eq instance for APartyCmdTag
 2023-03-10 09:10:52 +00:00
Evgeny Poberezkin
15637e545b Merge branch 'master' into xftp 2023-02-17 21:40:35 +00:00
spaced4ndy
c0dcf283eb server_key_hash fields (#643) 
* server_key_hash fields

* test

* refactor

* fix

* order

* use sync command in test

* refactor

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
 2023-02-17 21:24:32 +00:00
Stanislav Dmitrenko
bccef0ba47 files: server and client spike - basic upload/download (#591) 
* Files: main, env, stats, storeLog

* Better + transport

* Executable

* Env

* Update Client.hs, Server.hs, and 4 more files...

* Answer on request

* Delay

* Temp file

* Bypass cert check

* update package.yml, rename

* update store log

* extend HTTP2 transport

* refactor caStore

* HTTP2 body

* update server stats

* file server/client framework

* verify server commands

* process FNEW command, CLI test works

* simple XFTP server test (fails)

* fix test, refactor

* upload chunk works

* receive file chunk in the client

* remove transport handshake

* typo

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* fix names

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
 2023-02-13 13:36:02 +00:00
Evgeny Poberezkin
5d8febc353 avoid possible race conditions when cancelled clients/asyncs can be removed after the new ones are added (so that the new are removed as well) (#627) 2023-01-30 17:46:06 +00:00
Evgeny Poberezkin
7fb48930f6 increase database transaction timeout, add test for user deletion (#622) 
* increase database transaction timeout, add test for user deletion

* more test logs
 2023-01-25 20:58:03 +00:00
Evgeny Poberezkin
b59669a65e refactor functions operating on single connections using batch functions (#620) 
* refactor functions operating on single connections using batch functions

* update tests
 2023-01-24 13:14:13 +00:00
Evgeny Poberezkin
2ccef1690b events sent by the agent when queue, connection and user are deleted asynchronously (#619) 
* events sent by the agent when queue, connection and user are deleted asynchronously

* update test

* more tests

* add user tests

* queries

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>
 2023-01-24 11:12:07 +00:00
Evgeny Poberezkin
d4fc638478 async batch connection deletion (#617) 
* async batch connection deletion

* delete user record with connections, async connection deletion

* updates

* update query
 2023-01-21 13:41:16 +00:00
Evgeny Poberezkin
a854b5f2b2 batched queue deletion (#611) 
* batched queue deletion

* only re-connect client on subscribe errors, not in all batch commands

* add test, fix

* correction

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>

* refactor

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>
 2023-01-19 16:53:28 +00:00
JRoberts
42df6a421d add missing fkey indexes (#610) 2023-01-17 16:07:13 +04:00