Evgeny Poberezkin
5cafd9d5c4
server: more efficient responses to batch subscriptions ( #1137 )
...
* server: more efficient responses to batch subscriptions
* comments
* comment
* enable tests
* LogError
2024-05-09 09:20:57 +01:00
Alexander Bondarenko
7a07076277
transport: require ALPN for extended handshakes ( #1134 )
...
* transport: require ALPN for extended handshakes
* fix 8.10 build
* rename
---------
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com >
2024-05-08 00:06:09 +01:00
Evgeny Poberezkin
ee8e4067b0
agent: prepare connection record before joining to prevent race conditions ( #1128 )
...
* agent: prepare connection record before joining to prevent race conditions
* prepare connection for contact address as well
* clean up
2024-05-05 12:12:19 +01:00
Evgeny Poberezkin
fe28e02be7
agent: make version independent of PQ enqryption support ( #1114 )
...
* agent: make version independent of PQ enqryption support
* remove comment
2024-04-22 13:40:24 +01:00
Evgeny Poberezkin
b741b65700
use online status in network information ( #1094 )
...
* use online status in network information
* test
* refactor
* flip condition
* fix
2024-04-12 22:22:24 +01:00
spaced4ndy
3b1dd6dff9
agent: notify about available quota ( #1087 )
...
* agent: notify about available quota
* rename
---------
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com >
2024-04-11 09:38:12 +01:00
Evgeny Poberezkin
2e44ec5083
fix message expiration test ( #1092 )
2024-04-11 08:52:15 +01:00
Evgeny Poberezkin
8fa9ed6317
wait for user network availability ( #1085 )
...
* ghc-options
* wait for user network availability
* test
* update
* comment
* refactor
* slow config
* line
* waitForUserNetwork in xftp and ntf workers
* refactor
* refactor with registerDelay
---------
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
2024-04-10 21:50:05 +01:00
Evgeny Poberezkin
1219446996
dispose agent instances, fix tests, ( #1089 )
...
* dispose agent instances in tests
* fix quota test
* tests: fix tests with -threaded (#1088 )
* fix some tests
* match RTS opts with apps
* less verbose rts stats
* enable sqlite extended error codes
* clean up
* unfocus
* remove extendedErrorCode
It's actually setExtendedResultCodes, which isn't yet available.
* diff
---------
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com >
* fix switch test
* fix
---------
Co-authored-by: Alexander Bondarenko <486682+dpwiz@users.noreply.github.com >
2024-04-10 19:34:02 +01:00
Evgeny Poberezkin
84b8c8417b
revert base64 change ( #1081 )
2024-04-03 09:27:36 +01:00
Evgeny Poberezkin
6ded721daa
remove monad typeclasses to reduce overhead ( #1077 )
...
* remove monad typeclasses to reduce overhead
* remove unliftIO
* StrictData
* inline
* optional agent port
* avoid MonadUnliftIO instance (#1078 )
* avoid MonadUnliftIO instance
* simpler liftError'
* rename
* narrow down instance
* revert
---------
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com >
* logServer
---------
Co-authored-by: Alexander Bondarenko <486682+dpwiz@users.noreply.github.com >
2024-03-31 20:50:35 +01:00
Alexander Bondarenko
ee90ea6a69
replace base64-bytestring with base64 ( #1065 )
...
* replace base64-bytestring with base64
* minify
* use bytestring-0.10 compatible fork
PR pending...
* bump base64 fork with text compat
* move compat details to modules
* switch repo
* add back module
* cleanup
* minify
* clean imports
* rename
---------
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com >
2024-03-27 23:35:09 +00:00
Evgeny Poberezkin
b32259d048
notify about critical error on agent crash ( #1062 )
...
* notify about critical error on agent crash
* waitUntilActive
* disposeAgent
* fix
2024-03-21 14:54:57 +00:00
Evgeny Poberezkin
851ed2d02e
pqdr: more tests, pass e2e version to rcEncrypt from config ( #1040 )
...
* pqdr: more tests, pass e2e version to rcEncrypt from config
* fix
2024-03-10 13:29:03 +00:00
Evgeny Poberezkin
054b6edb14
pqdr: clean up ( #1039 )
2024-03-10 11:27:31 +00:00
Alexander Bondarenko
8cdd49b912
core: restore Eq instances ( #1038 )
...
* core: restore Eq instances
* remove duplicates from tests
2024-03-08 13:43:33 +00:00
Evgeny Poberezkin
8ff4c628b5
pqdr: make envelope sizes dependent on version, test enabling PQ ( #1035 )
2024-03-08 08:28:15 +00:00
Evgeny Poberezkin
07fa75ec49
pqdr: agent api to confirm PQ encryption support during connection handshake, fix incorrect PQ support ( #1032 )
...
* pqdr: agent api to confirm PQ encryption support during connection handshake
* fix CONF, tests
* fix REQ, tests
* remove unused
2024-03-07 08:35:40 +00:00
Evgeny Poberezkin
4ffb6a348a
pqdr: use different newtypes for supporting and enabling PQ encryption in connections ( #1031 )
...
* pqdr: use different newtypes for supporting and enabling PQ encryption in connections
* rename field, fix test
* refactor
2024-03-06 21:28:03 +00:00
Evgeny Poberezkin
b435a4dacb
envelope sizes dependent on PQ encryption ( #1028 )
...
* envelope sizes dependent on PQ encryption (WIP)
* add "supported" flag to ratchets, update this flag on ratchet resync
* change connection PQ status on sendMessage
* comment, fix
* refactor
2024-03-06 16:38:30 +00:00
Evgeny Poberezkin
b050cf5027
double ratchet versioning for post-quantum encryption ( #1025 )
...
* correctly parse new Ratchet fields when omitted
* rfc: migrating connection versions to pqdr
* update rfc
* WIP (dont commit)
* rename versions
* update ratchet version based on PQ encryption feature flag
* remove duplicate function
* synchronize ratchet, fix tests, refactor
* comments
* test
* pattern
2024-03-05 17:07:15 +00:00
Evgeny Poberezkin
ba1bfaa5aa
Merge branch 'master' into pq
2024-03-04 20:13:01 +00:00
Evgeny Poberezkin
dd2bd11584
parameterize version scopes with phantom types ( #1026 )
...
* parameterize version scopes with phantom types
* move Version to another module
* parens
2024-03-04 19:06:51 +00:00
Evgeny Poberezkin
e06e22328f
agent: quantum-resistant double ratchet encryption ( #939 )
...
* doc
* diff
* ratchet header
* types
* ratchet step with PQ KEM, message header with KEM
* comment
* update types, remove Eq instances, store KEM keys to database
* pqx3dh
* PQ double ratchet test
* pqdr tests pass
* fix most tests
* refactor
* allow KEM proposals from both sides
* test names
* agent API parameters to use PQ KEM
* initialize ratchet state for enabling KEM
* fix/test KEM state machine to support disabling/enabling via messages
* more tests
* diff
* diff2
* refactor
* refactor
* refactor
* refactor
* remove Maybe
* rename
* add PQ encryption status to CON, MID and MSG events and sendMessage API results
* different PQ parameter when creating connection
* rename/reorganize types for PQ encryption modes
* rename
* fix testWaitDeliveryTimeout
* rename
* rename2
* ghc8107
* rename
* increase timeouts for concurrent send/receive test
* enable all tests
---------
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
2024-03-03 19:40:49 +00:00
spaced4ndy
30fd4065d9
rename delete waiting delivery tests ( #1022 )
2024-03-03 12:56:54 +04:00
Evgeny Poberezkin
ce78646c7f
refactor creating connection record ( #1021 )
2024-03-02 18:27:51 +00:00
spaced4ndy
294d7ec8dd
agent: delay connection deletion to finish delivery of pending messages ( #1015 )
...
* agent: delay connection deletion to finish delivery of pending messages (wip)
* fixes, test
* notify, test
* add tests
* comment
* add test
* timeout
* test timeout
* up
* more tests
* rename
---------
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com >
2024-02-29 18:08:58 +00:00
Evgeny Poberezkin
155831ae36
tests: test notification decryption
2024-02-18 09:30:21 +00:00
Evgeny Poberezkin
51be2fea20
agent: test NTF server ( #1000 )
2024-02-17 14:55:40 +00:00
Evgeny Poberezkin
abf956d425
fix tests for new protocol versions ( #998 )
...
* fix basic auth tests
* fix batching and syntax tests
* fix test matrix
2024-02-16 20:21:15 +00:00
Evgeny Poberezkin
c179073260
remove support for old versions ( #990 )
...
* remove support for old versions (WIP)
* fix
* updates
* use version var
2024-02-16 13:28:50 +00:00
Evgeny Poberezkin
416f1b1721
smp: command authorization ( #982 )
...
* smp: command authorization
* fix encoding, most tests
* remove old tests
* authorize via crypto_box
* extract authenticator to Crypto module
* make TransmissionAuth Maybe
* rfc
* support authenticators in NTF protocol, test matrix (no backwards compatibility yet from new clients to old servers)
* fix/add tests, add version config to "small" agent
* separate client and server versions for SMP protocol
* test batching SMP v7
* do not send session ID in each transmission
* refactor auth verification in the server, split tests
* server "warm up" fixes timing test
* uncomment SUB timing test
* comments, disable two timing tests
* rename version
* increase auth timing test failure threshold
* use different algorithms to authorize snd/rcv commands, use random correlation ID
* transport: fetch and store server certificate (#985 )
* THandleParams (WIP, does not compile)
* transport: fetch and store server certificate
* smp: add getOnlinePubKey example to smpClientHandshake
* add server certs and sign authPub
* cleanup
* update
* style
* load server certs from test fixtures
* sign ntf authPubKey
* fix onServerCertificate
* increase delay before sending messages
* require certificate with key in SMP server handshake
---------
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com >
* remove dhSecret from THandle
* remove v8, merge all changes to one version
* parameterize THandle
* rfc: transmission ecnryption
* Revert "parameterize THandle"
This reverts commit 75adfc94fb8711996+spaced4ndy@users.noreply.github.com >
* rename
* diff
---------
Co-authored-by: Alexander Bondarenko <486682+dpwiz@users.noreply.github.com >
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
2024-02-16 11:45:54 +00:00
Alexander Bondarenko
2f7a288280
xftp: add sending and receiving via URI-encoded redirects ( #968 )
...
* xftp: add URI encoding for FileDescription
* tweak URI
* allow smaller blocks
* draft xftpReceiveFileFollow' and xftpSendFilePublic'
* add sending with redirect
* allow 64k chunks
* add migrations with redirect fields
* add test case
* fix deadlock
* revert CLI code
* WIP: working send/receive via URI
* fix field ambiguity
* cleanup
* update agent db schema
* update minimal chunk size
* add rfc
* apply suggestions from code review
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com >
* add createRcvFileRedirect
* extract Simplex.Messaging.ServiceScheme and reuse for files
* update db schema
* check size/digest on receive complete
* cleanup
* use SIZE/DIGEST errors for redirects too
* split digest/size errors from redirect checks
* fix redirect error encoding
* rename RedirectMeta to RedirectFileInfo
* use query encoding for file URI
* group maybe fields under RcvFileRedirect
* add extras field
* update rfc
* add extras encoding and no-redirect tests
* fix toStrict for old ghc
* extra client data in file descr URI
* remove decoded yaml file
---------
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com >
2024-02-13 14:08:49 +00:00
Alexander Bondarenko
f7cdec2f08
fix: support multiple notification servers in configuration ( #971 )
...
* tests: add trpl-keeps-server check
* add smp server switch check
* add connection test and fix withNtfServer
* Update src/Simplex/Messaging/Agent/NtfSubSupervisor.hs
Co-authored-by: Evgeny Poberezkin <e.poberezkin@me.com >
* use ntfServer from token
* rename
---------
Co-authored-by: Evgeny Poberezkin <e.poberezkin@me.com >
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com >
2024-01-19 19:42:28 +00:00
Evgeny Poberezkin
baf2c47065
agent: expire messages failed after quota exceeded after 7 days, and expire multiple messages at once ( #973 )
...
* agent: expire messages failed after quota exceeded after 7 days, and expire multiple messages at once
* fix, test
* refactor
* catch in loop
2024-01-19 17:21:25 +00:00
Alexander Bondarenko
40fa34c2d5
tests: test multiple Ntf servers ( #966 )
...
* tests: add ntf case with multiple Ntf servers
* simplify test
* fix for master
* add server switch test
* add server switch test
* add message test for ntf server switch
2024-01-17 19:47:47 +00:00
Alexander Bondarenko
eb41abfb8f
tests: fix random order in async commands case ( #969 )
2024-01-17 11:48:41 +00:00
Evgeny Poberezkin
b547f34cc0
Revert "Revert "agent: do not create user record in new databases ( #957 )" ( #963 )" ( #964 )
...
* Revert "Revert "agent: do not create user record in new databases (#957 )" (#963 )"
This reverts commit aee9088417
2024-01-17 11:01:04 +00:00
Evgeny Poberezkin
00c4ff4a21
agent: fix race condition in subscription worker (results in stuck delivery), better error handling in batch operations ( #962 )
...
* debugging test: join connection when reply queue creation fails
* more logs
* more logs
* more logs, fixed bug
* cleanup
* catch all errors
* simplify
* comment
* remove client ID counter
2024-01-16 10:29:35 +00:00
Evgeny Poberezkin
aee9088417
Revert "agent: do not create user record in new databases ( #957 )" ( #963 )
...
This reverts commit 9b9a0bd0df
2024-01-14 22:28:28 +00:00
Alexander Bondarenko
68f5e189a6
smp: check for progress when resubscribing ( #956 )
...
* smp: check for progress when resubscribing
* add allowClose to reconnectServer to distinguish entry points
* resolve error todo
* make reconnect async per-session
* remove allowClose
* deregister reconnecter when it finishes
* signal/react more work explicitly
* fix restart condition
* wait for reconnecter to finish
* remove redundant reconnect locks
* rename getClientVar for expanded scope
* formatting
* remove withPending
* move pending check to tryReconnectSMPClient loop
* combine pending check and slot release transactions
* actually reserve the slot for async
* simplify
* refactor
* refactor
* use removeClientVar
* rename
* refactor
* test
* reduce MonadError scope
---------
Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com >
2024-01-12 19:29:18 +00:00
Evgeny Poberezkin
9b9a0bd0df
agent: do not create user record in new databases ( #957 )
...
* agent: do not create user record in new databases
* update
2024-01-12 16:57:57 +00:00
Evgeny Poberezkin
ca527b4d6c
use Builder for batching commands ( #953 )
...
* use Builder for batching commands
* refactor
* refactor
* increase timeout in test
* do not materialize builder
* use concat
* debug test
* debug2
* delays
* more delay, less worker delay
* remove logs
* reduce delay
* tests
* sized builder
* builder
* comment
* remove unsafe
* style
* remove fromIntegral
2024-01-08 22:54:27 +00:00
spaced4ndy
e43e4860b9
xftp: file workers cycle through pending files based on retries count (limit number of iterations per work item to prevent stuck delivery) ( #930 )
...
* xftp: test file reception - shouldn't get stuck if file is deleted on server
* comment
* expiration test
* approach
* wip
* sort by retries in other works, revert some diff
* revert diff
* modify tests
* refactor
* refactor
* remove prints
* apply to other workers
* remove import
* comment
* refactor
* revert queue size
* fix test
* rename
* comment, correct number of retries
---------
Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com >
2023-12-26 22:20:25 +00:00
Evgeny Poberezkin
577e3cf14d
agent: add server msg IDs to agent logs, disable 2 tests ( #934 )
...
* agent: race when using agent via TCP
* remove logs
* enable tests
* comments
* use different databases for different clients
* enable all tests
2023-12-26 17:50:39 +00:00
Evgeny Poberezkin
7ddeca50e4
agent: mark work items failed ( #931 )
...
* agent: mark work items failed (WIP)
* add tests, created_at
* getWorkItem for snd and rcv files
* fix
* tests
* fix
* tests
* test
* tests
* rename
* fix,refactor
* add indexes
* update schema
* do not try to get more work when resuming an existing worker
---------
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
2023-12-26 17:00:42 +04:00
Evgeny Poberezkin
2489333c87
agent: use DB to queue async commands and messages ( #929 )
...
* agent: use DB to queue async commands and messages
* fix message delivery, async commands dont work
* fix async commands
* remove comment
* rename
* comment
* balance agent operation
* empty lines
* balance another agent operation
* diff
* worker
* qAddress
* fix incorrect queue ID in v1
* types for queues
* fix test
* update index
2023-12-23 12:59:53 +00:00
Evgeny Poberezkin
1c2604f6a3
agent: background mode for agent NSE ( #924 )
2023-12-22 08:59:07 +00:00
Evgeny Poberezkin
13a60d1d39
use ChaChaDRG as the source of randomness ( #920 )
...
* use ChaChaDRG as the source of randomness
* remove functions using entropy directly
* comment
2023-12-21 00:12:08 +00:00
Alexander Bondarenko
7c27357eb2
server: stop inactive clients without subscriptions, additional monitoring via control port ( #901 )
...
* fix thread labelling points
* add subscription thread labels
* add explicit sections for sockets/sessions/disconnects
* adjust label threads wrt. next blocking op
* WIP: emit event on server finish
So the event log wouldn't be cut short giving
false impression of unterminated sessions.
* add handshake timeout
* hack server socket stats
* trace closeTLS exceptions
* fix build
* clean up disconnectTransport
* make disconnectTransport terminating
Thus, participating in racyAny_ around it.
* trach snd/rcv time separately, log clients as CSV
* add direct client counting/stats
* add csv dump for socket threads
* guard socket threads dump from older GHC
* cut socket thread listing into socket-threads command
* store client creation time for stats
* wrap tls setup in timeout and labels
* server: expire clients without subscriptions
* fix check for subscriptions
* remove excess tracing
* move hardcoded timeouts to configs
* cleanup
* fix tests
* wrap direct queue waits in timeout
* rewrite old TCP wrappers with their SocketState variants
* add testInactiveWithSubs
---------
Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com >
2023-12-19 15:50:45 +00:00
