dandri/simplexmq
1
0
Fork 0
mirror of https://github.com/simplex-chat/simplexmq.git synced 2026-04-03 04:56:27 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,363 Commits 95 Branches 171 Tags
ep/proxy-client-server
Commit Graph

249 Commits

Author SHA1 Message Date
Evgeny Poberezkin
b741b65700 use online status in network information (#1094) 
* use online status in network information

* test

* refactor

* flip condition

* fix
 2024-04-12 22:22:24 +01:00
spaced4ndy
3b1dd6dff9 agent: notify about available quota (#1087) 
* agent: notify about available quota

* rename

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
 2024-04-11 09:38:12 +01:00
Evgeny Poberezkin
2e44ec5083 fix message expiration test (#1092) 2024-04-11 08:52:15 +01:00
Evgeny Poberezkin
8fa9ed6317 wait for user network availability (#1085) 
* ghc-options

* wait for user network availability

* test

* update

* comment

* refactor

* slow config

* line

* waitForUserNetwork in xftp and ntf workers

* refactor

* refactor with registerDelay

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
 2024-04-10 21:50:05 +01:00
Evgeny Poberezkin
1219446996 dispose agent instances, fix tests, (#1089) 
* dispose agent instances in tests

* fix quota test

* tests: fix tests with -threaded (#1088)

* fix some tests

* match RTS opts with apps

* less verbose rts stats

* enable sqlite extended error codes

* clean up

* unfocus

* remove extendedErrorCode

It's actually setExtendedResultCodes, which isn't yet available.

* diff

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>

* fix switch test

* fix

---------

Co-authored-by: Alexander Bondarenko <486682+dpwiz@users.noreply.github.com>
 2024-04-10 19:34:02 +01:00
Evgeny Poberezkin
84b8c8417b revert base64 change (#1081) 2024-04-03 09:27:36 +01:00
Evgeny Poberezkin
6ded721daa remove monad typeclasses to reduce overhead (#1077) 
* remove monad typeclasses to reduce overhead

* remove unliftIO

* StrictData

* inline

* optional agent port

* avoid MonadUnliftIO instance (#1078)

* avoid MonadUnliftIO instance

* simpler liftError'

* rename

* narrow down instance

* revert

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>

* logServer

---------

Co-authored-by: Alexander Bondarenko <486682+dpwiz@users.noreply.github.com>
 2024-03-31 20:50:35 +01:00
Alexander Bondarenko
ee90ea6a69 replace base64-bytestring with base64 (#1065) 
* replace base64-bytestring with base64

* minify

* use bytestring-0.10 compatible fork

PR pending...

* bump base64 fork with text compat

* move compat details to modules

* switch repo

* add back module

* cleanup

* minify

* clean imports

* rename

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
 2024-03-27 23:35:09 +00:00
Evgeny Poberezkin
b32259d048 notify about critical error on agent crash (#1062) 
* notify about critical error on agent crash

* waitUntilActive

* disposeAgent

* fix
 2024-03-21 14:54:57 +00:00
Evgeny Poberezkin
851ed2d02e pqdr: more tests, pass e2e version to rcEncrypt from config (#1040) 
* pqdr: more tests, pass e2e version to rcEncrypt from config

* fix
 2024-03-10 13:29:03 +00:00
Evgeny Poberezkin
054b6edb14 pqdr: clean up (#1039) 2024-03-10 11:27:31 +00:00
Alexander Bondarenko
8cdd49b912 core: restore Eq instances (#1038) 
* core: restore Eq instances

* remove duplicates from tests
 2024-03-08 13:43:33 +00:00
Evgeny Poberezkin
8ff4c628b5 pqdr: make envelope sizes dependent on version, test enabling PQ (#1035) 2024-03-08 08:28:15 +00:00
Evgeny Poberezkin
07fa75ec49 pqdr: agent api to confirm PQ encryption support during connection handshake, fix incorrect PQ support (#1032) 
* pqdr: agent api to confirm PQ encryption support during connection handshake

* fix CONF, tests

* fix REQ, tests

* remove unused
 2024-03-07 08:35:40 +00:00
Evgeny Poberezkin
4ffb6a348a pqdr: use different newtypes for supporting and enabling PQ encryption in connections (#1031) 
* pqdr: use different newtypes for supporting and enabling PQ encryption in connections

* rename field, fix test

* refactor
 2024-03-06 21:28:03 +00:00
Evgeny Poberezkin
b435a4dacb envelope sizes dependent on PQ encryption (#1028) 
* envelope sizes dependent on PQ encryption (WIP)

* add "supported" flag to ratchets, update this flag on ratchet resync

* change connection PQ status on sendMessage

* comment, fix

* refactor
 2024-03-06 16:38:30 +00:00
Evgeny Poberezkin
b050cf5027 double ratchet versioning for post-quantum encryption (#1025) 
* correctly parse new Ratchet fields when omitted

* rfc: migrating connection versions to pqdr

* update rfc

* WIP (dont commit)

* rename versions

* update ratchet version based on PQ encryption feature flag

* remove duplicate function

* synchronize ratchet, fix tests, refactor

* comments

* test

* pattern
 2024-03-05 17:07:15 +00:00
Evgeny Poberezkin
ba1bfaa5aa Merge branch 'master' into pq 2024-03-04 20:13:01 +00:00
Evgeny Poberezkin
dd2bd11584 parameterize version scopes with phantom types (#1026) 
* parameterize version scopes with phantom types

* move Version to another module

* parens
 2024-03-04 19:06:51 +00:00
Evgeny Poberezkin
e06e22328f agent: quantum-resistant double ratchet encryption (#939) 
* doc

* diff

* ratchet header

* types

* ratchet step with PQ KEM, message header with KEM

* comment

* update types, remove Eq instances, store KEM keys to database

* pqx3dh

* PQ double ratchet test

* pqdr tests pass

* fix most tests

* refactor

* allow KEM proposals from both sides

* test names

* agent API parameters to use PQ KEM

* initialize ratchet state for enabling KEM

* fix/test KEM state machine to support disabling/enabling via messages

* more tests

* diff

* diff2

* refactor

* refactor

* refactor

* refactor

* remove Maybe

* rename

* add PQ encryption status to CON, MID and MSG events and sendMessage API results

* different PQ parameter when creating connection

* rename/reorganize types for PQ encryption modes

* rename

* fix testWaitDeliveryTimeout

* rename

* rename2

* ghc8107

* rename

* increase timeouts for concurrent send/receive test

* enable all tests

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
 2024-03-03 19:40:49 +00:00
spaced4ndy
30fd4065d9 rename delete waiting delivery tests (#1022) 2024-03-03 12:56:54 +04:00
Evgeny Poberezkin
ce78646c7f refactor creating connection record (#1021) 2024-03-02 18:27:51 +00:00
spaced4ndy
294d7ec8dd agent: delay connection deletion to finish delivery of pending messages (#1015) 
* agent: delay connection deletion to finish delivery of pending messages (wip)

* fixes, test

* notify, test

* add tests

* comment

* add test

* timeout

* test timeout

* up

* more tests

* rename

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
 2024-02-29 18:08:58 +00:00
Evgeny Poberezkin
155831ae36 tests: test notification decryption 2024-02-18 09:30:21 +00:00
Evgeny Poberezkin
51be2fea20 agent: test NTF server (#1000) 2024-02-17 14:55:40 +00:00
Evgeny Poberezkin
abf956d425 fix tests for new protocol versions (#998) 
* fix basic auth tests

* fix batching and syntax tests

* fix test matrix
 2024-02-16 20:21:15 +00:00
Evgeny Poberezkin
c179073260 remove support for old versions (#990) 
* remove support for old versions (WIP)

* fix

* updates

* use version var
 2024-02-16 13:28:50 +00:00
Evgeny Poberezkin
416f1b1721 smp: command authorization (#982) 
* smp: command authorization

* fix encoding, most tests

* remove old tests

* authorize via crypto_box

* extract authenticator to Crypto module

* make TransmissionAuth Maybe

* rfc

* support authenticators in NTF protocol, test matrix (no backwards compatibility yet from new clients to old servers)

* fix/add tests, add version config to "small" agent

* separate client and server versions for SMP protocol

* test batching SMP v7

* do not send session ID in each transmission

* refactor auth verification in the server, split tests

* server "warm up" fixes timing test

* uncomment SUB timing test

* comments, disable two timing tests

* rename version

* increase auth timing test failure threshold

* use different algorithms to authorize snd/rcv commands, use random correlation ID

* transport: fetch and store server certificate (#985)

* THandleParams (WIP, does not compile)

* transport: fetch and store server certificate

* smp: add getOnlinePubKey example to smpClientHandshake

* add server certs and sign authPub

* cleanup

* update

* style

* load server certs from test fixtures

* sign ntf authPubKey

* fix onServerCertificate

* increase delay before sending messages

* require certificate with key in SMP server handshake

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>

* remove dhSecret from THandle

* remove v8, merge all changes to one version

* parameterize THandle

* rfc: transmission ecnryption

* Revert "parameterize THandle"

This reverts commit 75adfc94fb.

* use batch syntax for ntf server commands

* separate encodeTransmission when there is no key

* typo

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* rename

* diff

---------

Co-authored-by: Alexander Bondarenko <486682+dpwiz@users.noreply.github.com>
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
 2024-02-16 11:45:54 +00:00
Alexander Bondarenko
2f7a288280 xftp: add sending and receiving via URI-encoded redirects (#968) 
* xftp: add URI encoding for FileDescription

* tweak URI

* allow smaller blocks

* draft xftpReceiveFileFollow' and xftpSendFilePublic'

* add sending with redirect

* allow 64k chunks

* add migrations with redirect fields

* add test case

* fix deadlock

* revert CLI code

* WIP: working send/receive via URI

* fix field ambiguity

* cleanup

* update agent db schema

* update minimal chunk size

* add rfc

* apply suggestions from code review

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>

* add createRcvFileRedirect

* extract Simplex.Messaging.ServiceScheme and reuse for files

* update db schema

* check size/digest on receive complete

* cleanup

* use SIZE/DIGEST errors for redirects too

* split digest/size errors from redirect checks

* fix redirect error encoding

* rename RedirectMeta to RedirectFileInfo

* use query encoding for file URI

* group maybe fields under RcvFileRedirect

* add extras field

* update rfc

* add extras encoding and no-redirect tests

* fix toStrict for old ghc

* extra client data in file descr URI

* remove decoded yaml file

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
 2024-02-13 14:08:49 +00:00
Alexander Bondarenko
f7cdec2f08 fix: support multiple notification servers in configuration (#971) 
* tests: add trpl-keeps-server check

* add smp server switch check

* add connection test and fix withNtfServer

* Update src/Simplex/Messaging/Agent/NtfSubSupervisor.hs

Co-authored-by: Evgeny Poberezkin <e.poberezkin@me.com>

* use ntfServer from token

* rename

---------

Co-authored-by: Evgeny Poberezkin <e.poberezkin@me.com>
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
 2024-01-19 19:42:28 +00:00
Evgeny Poberezkin
baf2c47065 agent: expire messages failed after quota exceeded after 7 days, and expire multiple messages at once (#973) 
* agent: expire messages failed after quota exceeded after 7 days, and expire multiple messages at once

* fix, test

* refactor

* catch in loop
 2024-01-19 17:21:25 +00:00
Alexander Bondarenko
40fa34c2d5 tests: test multiple Ntf servers (#966) 
* tests: add ntf case with multiple Ntf servers

* simplify test

* fix for master

* add server switch test

* add server switch test

* add message test for ntf server switch
 2024-01-17 19:47:47 +00:00
Alexander Bondarenko
eb41abfb8f tests: fix random order in async commands case (#969) 2024-01-17 11:48:41 +00:00
Evgeny Poberezkin
b547f34cc0 Revert "Revert "agent: do not create user record in new databases (#957)" (#963)" (#964) 
* Revert "Revert "agent: do not create user record in new databases (#957)" (#963)"

This reverts commit aee9088417.

* tests

* remove logging
 2024-01-17 11:01:04 +00:00
Evgeny Poberezkin
00c4ff4a21 agent: fix race condition in subscription worker (results in stuck delivery), better error handling in batch operations (#962) 
* debugging test: join connection when reply queue creation fails

* more logs

* more logs

* more logs, fixed bug

* cleanup

* catch all errors

* simplify

* comment

* remove client ID counter
 2024-01-16 10:29:35 +00:00
Evgeny Poberezkin
aee9088417 Revert "agent: do not create user record in new databases (#957)" (#963) 
This reverts commit 9b9a0bd0df.
 2024-01-14 22:28:28 +00:00
Alexander Bondarenko
68f5e189a6 smp: check for progress when resubscribing (#956) 
* smp: check for progress when resubscribing

* add allowClose to reconnectServer to distinguish entry points

* resolve error todo

* make reconnect async per-session

* remove allowClose

* deregister reconnecter when it finishes

* signal/react more work explicitly

* fix restart condition

* wait for reconnecter to finish

* remove redundant reconnect locks

* rename getClientVar for expanded scope

* formatting

* remove withPending

* move pending check to tryReconnectSMPClient loop

* combine pending check and slot release transactions

* actually reserve the slot for async

* simplify

* refactor

* refactor

* use removeClientVar

* rename

* refactor

* test

* reduce MonadError scope

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
 2024-01-12 19:29:18 +00:00
Evgeny Poberezkin
9b9a0bd0df agent: do not create user record in new databases (#957) 
* agent: do not create user record in new databases

* update
 2024-01-12 16:57:57 +00:00
Evgeny Poberezkin
ca527b4d6c use Builder for batching commands (#953) 
* use Builder for batching commands

* refactor

* refactor

* increase timeout in test

* do not materialize builder

* use concat

* debug test

* debug2

* delays

* more delay, less worker delay

* remove logs

* reduce delay

* tests

* sized builder

* builder

* comment

* remove unsafe

* style

* remove fromIntegral
 2024-01-08 22:54:27 +00:00
spaced4ndy
e43e4860b9 xftp: file workers cycle through pending files based on retries count (limit number of iterations per work item to prevent stuck delivery) (#930) 
* xftp: test file reception - shouldn't get stuck if file is deleted on server

* comment

* expiration test

* approach

* wip

* sort by retries in other works, revert some diff

* revert diff

* modify tests

* refactor

* refactor

* remove prints

* apply to other workers

* remove import

* comment

* refactor

* revert queue size

* fix test

* rename

* comment, correct number of retries

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
 2023-12-26 22:20:25 +00:00
Evgeny Poberezkin
577e3cf14d agent: add server msg IDs to agent logs, disable 2 tests (#934) 
* agent: race when using agent via TCP

* remove logs

* enable tests

* comments

* use different databases for different clients

* enable all tests
 2023-12-26 17:50:39 +00:00
Evgeny Poberezkin
7ddeca50e4 agent: mark work items failed (#931) 
* agent: mark work items failed (WIP)

* add tests, created_at

* getWorkItem for snd and rcv files

* fix

* tests

* fix

* tests

* test

* tests

* rename

* fix,refactor

* add indexes

* update schema

* do not try to get more work when resuming an existing worker

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
 2023-12-26 17:00:42 +04:00
Evgeny Poberezkin
2489333c87 agent: use DB to queue async commands and messages (#929) 
* agent: use DB to queue async commands and messages

* fix message delivery, async commands dont work

* fix async commands

* remove comment

* rename

* comment

* balance agent operation

* empty lines

* balance another agent operation

* diff

* worker

* qAddress

* fix incorrect queue ID in v1

* types for queues

* fix test

* update index
 2023-12-23 12:59:53 +00:00
Evgeny Poberezkin
1c2604f6a3 agent: background mode for agent NSE (#924) 2023-12-22 08:59:07 +00:00
Evgeny Poberezkin
13a60d1d39 use ChaChaDRG as the source of randomness (#920) 
* use ChaChaDRG as the source of randomness

* remove functions using entropy directly

* comment
 2023-12-21 00:12:08 +00:00
Alexander Bondarenko
7c27357eb2 server: stop inactive clients without subscriptions, additional monitoring via control port (#901) 
* fix thread labelling points

* add subscription thread labels

* add explicit sections for sockets/sessions/disconnects

* adjust label threads wrt. next blocking op

* WIP: emit event on server finish

So the event log wouldn't be cut short giving
false impression of unterminated sessions.

* add handshake timeout

* hack server socket stats

* trace closeTLS exceptions

* fix build

* clean up disconnectTransport

* make disconnectTransport terminating

Thus, participating in racyAny_ around it.

* trach snd/rcv time separately, log clients as CSV

* add direct client counting/stats

* add csv dump for socket threads

* guard socket threads dump from older GHC

* cut socket thread listing into socket-threads command

* store client creation time for stats

* wrap tls setup in timeout and labels

* server: expire clients without subscriptions

* fix check for subscriptions

* remove excess tracing

* move hardcoded timeouts to configs

* cleanup

* fix tests

* wrap direct queue waits in timeout

* rewrite old TCP wrappers with their SocketState variants

* add testInactiveWithSubs

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
 2023-12-19 15:50:45 +00:00
Evgeny Poberezkin
560dc55312 agent: notify about polled message processing (for iOS notifications) (#908) 
* agent: notify about polled message processing (for iOS notifications)

* optionally keep key and support re-opening database

* exports

* test that cannot reopen when created with keepKey: false

* set max number of messages to receive for a notification to 3
 2023-12-11 13:03:53 +00:00
Evgeny Poberezkin
248144f3de do not use previous KEM shared secret in multicast packets and in host HELLO (#888) 
* do not use previous KEM shared secret in multicast packets and in host HELLO

* simplify
 2023-11-08 12:06:10 +00:00
Evgeny Poberezkin
4660ee9cff agent: revert to TVar (#883) 2023-11-01 09:15:51 +00:00
Evgeny Poberezkin
b5f733d2db agent: use IORef for DRG (#882) 2023-10-31 23:52:13 +00:00