51be2fea20
agent: test NTF server ( #1000 )
2024-02-17 14:55:40 +00:00
f6acc5c240
remove test chunk after XFTP server test ( #999 )
2024-02-16 23:24:57 +00:00
194a7bb58e
agent: remove redirect entries when deleting rcv files ( #995 )
...
* agent: remove redirect entries when deleting rcv files
* update schema dump
* use rcv_file primary key for redirect lookup
* allow multiple redirects
2024-02-16 19:49:50 +04:00
c179073260
remove support for old versions ( #990 )
...
* remove support for old versions (WIP)
* fix
* updates
* use version var
2024-02-16 13:28:50 +00:00
416f1b1721
smp: command authorization ( #982 )
...
* smp: command authorization
* fix encoding, most tests
* remove old tests
* authorize via crypto_box
* extract authenticator to Crypto module
* make TransmissionAuth Maybe
* rfc
* support authenticators in NTF protocol, test matrix (no backwards compatibility yet from new clients to old servers)
* fix/add tests, add version config to "small" agent
* separate client and server versions for SMP protocol
* test batching SMP v7
* do not send session ID in each transmission
* refactor auth verification in the server, split tests
* server "warm up" fixes timing test
* uncomment SUB timing test
* comments, disable two timing tests
* rename version
* increase auth timing test failure threshold
* use different algorithms to authorize snd/rcv commands, use random correlation ID
* transport: fetch and store server certificate (#985 )
* THandleParams (WIP, does not compile)
* transport: fetch and store server certificate
* smp: add getOnlinePubKey example to smpClientHandshake
* add server certs and sign authPub
* cleanup
* update
* style
* load server certs from test fixtures
* sign ntf authPubKey
* fix onServerCertificate
* increase delay before sending messages
* require certificate with key in SMP server handshake
---------
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com >
* remove dhSecret from THandle
* remove v8, merge all changes to one version
* parameterize THandle
* rfc: transmission ecnryption
* Revert "parameterize THandle"
This reverts commit 75adfc94fb8711996+spaced4ndy@users.noreply.github.com >
* rename
* diff
---------
Co-authored-by: Alexander Bondarenko <486682+dpwiz@users.noreply.github.com >
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
2024-02-16 11:45:54 +00:00
004597c764
agent: add index to file redirect migration ( #988 )
2024-02-14 18:47:39 +04:00
2f7a288280
xftp: add sending and receiving via URI-encoded redirects ( #968 )
...
* xftp: add URI encoding for FileDescription
* tweak URI
* allow smaller blocks
* draft xftpReceiveFileFollow' and xftpSendFilePublic'
* add sending with redirect
* allow 64k chunks
* add migrations with redirect fields
* add test case
* fix deadlock
* revert CLI code
* WIP: working send/receive via URI
* fix field ambiguity
* cleanup
* update agent db schema
* update minimal chunk size
* add rfc
* apply suggestions from code review
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com >
* add createRcvFileRedirect
* extract Simplex.Messaging.ServiceScheme and reuse for files
* update db schema
* check size/digest on receive complete
* cleanup
* use SIZE/DIGEST errors for redirects too
* split digest/size errors from redirect checks
* fix redirect error encoding
* rename RedirectMeta to RedirectFileInfo
* use query encoding for file URI
* group maybe fields under RcvFileRedirect
* add extras field
* update rfc
* add extras encoding and no-redirect tests
* fix toStrict for old ghc
* extra client data in file descr URI
* remove decoded yaml file
---------
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com >
2024-02-13 14:08:49 +00:00
2ae1c9f79d
agent: only retry connecting SMP clients when there are pending subscriptions ( #981 )
...
* agent: only retry connecting SMP clients when there are pending subscriptions
* fix
* remove retry on creating clients
* simplify
2024-02-01 16:17:37 +00:00
24b84106a6
core: add activity tests to workers details ( #980 )
...
* core: add activity tests to workers details
* remove names
---------
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com >
2024-02-01 11:02:31 +00:00
15bc027f23
core: fix reconnection bugs ( #979 )
...
* core: fix reconnection bugs
* untangle newProtocolClient
* refactor
* report busy clientVar error
* log error
* comments
---------
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com >
2024-02-01 10:55:50 +00:00
fd4eeb36db
agent: optimize expired messages query ( #976 )
...
* agent: optimize expired messages query
* schema
* fix query
* fix
* typo
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
* fix
* refactor
* comment
* refactor2
---------
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
2024-01-22 14:04:57 +00:00
f7cdec2f08
fix: support multiple notification servers in configuration ( #971 )
...
* tests: add trpl-keeps-server check
* add smp server switch check
* add connection test and fix withNtfServer
* Update src/Simplex/Messaging/Agent/NtfSubSupervisor.hs
Co-authored-by: Evgeny Poberezkin <e.poberezkin@me.com >
* use ntfServer from token
* rename
---------
Co-authored-by: Evgeny Poberezkin <e.poberezkin@me.com >
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com >
2024-01-19 19:42:28 +00:00
baf2c47065
agent: expire messages failed after quota exceeded after 7 days, and expire multiple messages at once ( #973 )
...
* agent: expire messages failed after quota exceeded after 7 days, and expire multiple messages at once
* fix, test
* refactor
* catch in loop
2024-01-19 17:21:25 +00:00
b547f34cc0
Revert "Revert "agent: do not create user record in new databases ( #957 )" ( #963 )" ( #964 )
...
* Revert "Revert "agent: do not create user record in new databases (#957 )" (#963 )"
This reverts commit aee9088417
2024-01-17 11:01:04 +00:00
a0b35cec4f
agent: fix potential race when good client can be removed instead of bad for the same transport session ( #967 )
...
* agent: fix potential race when good client can be removed instead of bad for the same transport session
* tryAgentError
* case
2024-01-16 13:45:51 +00:00
00c4ff4a21
agent: fix race condition in subscription worker (results in stuck delivery), better error handling in batch operations ( #962 )
...
* debugging test: join connection when reply queue creation fails
* more logs
* more logs
* more logs, fixed bug
* cleanup
* catch all errors
* simplify
* comment
* remove client ID counter
2024-01-16 10:29:35 +00:00
aee9088417
Revert "agent: do not create user record in new databases ( #957 )" ( #963 )
...
This reverts commit 9b9a0bd0df
2024-01-14 22:28:28 +00:00
cd4329f2de
agent: collect worker stats ( #959 )
...
* agent: collect worker stats
* add more workers
* process asyncCmdWorkers as a Map
2024-01-12 23:38:35 +00:00
68f5e189a6
smp: check for progress when resubscribing ( #956 )
...
* smp: check for progress when resubscribing
* add allowClose to reconnectServer to distinguish entry points
* resolve error todo
* make reconnect async per-session
* remove allowClose
* deregister reconnecter when it finishes
* signal/react more work explicitly
* fix restart condition
* wait for reconnecter to finish
* remove redundant reconnect locks
* rename getClientVar for expanded scope
* formatting
* remove withPending
* move pending check to tryReconnectSMPClient loop
* combine pending check and slot release transactions
* actually reserve the slot for async
* simplify
* refactor
* refactor
* use removeClientVar
* rename
* refactor
* test
* reduce MonadError scope
---------
Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com >
2024-01-12 19:29:18 +00:00
9b9a0bd0df
agent: do not create user record in new databases ( #957 )
...
* agent: do not create user record in new databases
* update
2024-01-12 16:57:57 +00:00
fa794d7878
agent: send errors on subscription timeouts ( #937 )
...
* agent: debug server reconnections
* update lock only if it has the same name already
* report subscription timeouts
* clean up
2024-01-05 17:36:57 +00:00
2488cf1e3a
agent: restart worker when it terminated ( #948 )
...
* agent: restart worker when it terminated
* comment
* remove writeTMVar
* remove writeTMVar 2
* remove writeTMVar 3
2024-01-05 14:22:26 +00:00
22e1932372
xftp, ntf: refactor, reuse workers abstractions ( #935 )
...
* xftp: refactor, reuse workers abstractions (wip, test fails)
* rename
* refactor (tests pass)
* refactor2
* linebreaks
* do
* reuse worker abstractions for notifications
---------
Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com >
2023-12-27 13:02:08 +04:00
e43e4860b9
xftp: file workers cycle through pending files based on retries count (limit number of iterations per work item to prevent stuck delivery) ( #930 )
...
* xftp: test file reception - shouldn't get stuck if file is deleted on server
* comment
* expiration test
* approach
* wip
* sort by retries in other works, revert some diff
* revert diff
* modify tests
* refactor
* refactor
* remove prints
* apply to other workers
* remove import
* comment
* refactor
* revert queue size
* fix test
* rename
* comment, correct number of retries
---------
Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com >
2023-12-26 22:20:25 +00:00
577e3cf14d
agent: add server msg IDs to agent logs, disable 2 tests ( #934 )
...
* agent: race when using agent via TCP
* remove logs
* enable tests
* comments
* use different databases for different clients
* enable all tests
2023-12-26 17:50:39 +00:00
7ddeca50e4
agent: mark work items failed ( #931 )
...
* agent: mark work items failed (WIP)
* add tests, created_at
* getWorkItem for snd and rcv files
* fix
* tests
* fix
* tests
* test
* tests
* rename
* fix,refactor
* add indexes
* update schema
* do not try to get more work when resuming an existing worker
---------
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
2023-12-26 17:00:42 +04:00
1e15d56e92
5.5.0.1: fix migration
2023-12-23 16:06:11 +00:00
2489333c87
agent: use DB to queue async commands and messages ( #929 )
...
* agent: use DB to queue async commands and messages
* fix message delivery, async commands dont work
* fix async commands
* remove comment
* rename
* comment
* balance agent operation
* empty lines
* balance another agent operation
* diff
* worker
* qAddress
* fix incorrect queue ID in v1
* types for queues
* fix test
* update index
2023-12-23 12:59:53 +00:00
fa457d1c25
agent: avoid race condition between worker and supervisor when getting work ( #928 )
2023-12-22 11:12:36 +00:00
1c2604f6a3
agent: background mode for agent NSE ( #924 )
2023-12-22 08:59:07 +00:00
13a60d1d39
use ChaChaDRG as the source of randomness ( #920 )
...
* use ChaChaDRG as the source of randomness
* remove functions using entropy directly
* comment
2023-12-21 00:12:08 +00:00
8c250ebe19
agent: batch sending messages ( #922 )
...
* agent: batch sending messages (attempt 4)
* handle errors in batch sending
* batch attempt 5 (#923 )
* attempt 5
* remove IORefs
* add liftA2 for 8.10 compat
* remove db-related zipping
* traversable
---------
Co-authored-by: IC Rainbow <aenor.realm@gmail.com >
* s/mapE/bindRight/
* name
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
* comment
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
* remove unused funcs
---------
Co-authored-by: IC Rainbow <aenor.realm@gmail.com >
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
2023-12-19 23:01:34 +00:00
fb113ff008
agent: include queries longer than 5ms into slow queries (was 50ms)
2023-12-12 15:34:49 +00:00
560dc55312
agent: notify about polled message processing (for iOS notifications) ( #908 )
...
* agent: notify about polled message processing (for iOS notifications)
* optionally keep key and support re-opening database
* exports
* test that cannot reopen when created with keepKey: false
* set max number of messages to receive for a notification to 3
2023-12-11 13:03:53 +00:00
a860936072
agent: do not account for delivery of stored messages for agent suspension ( #916 )
2023-12-05 23:14:33 +00:00
6bffcc8503
compatibility with GHC 8.10.7 ( #906 )
...
* build with ghc-8.10
* add 8.10 to matrix
* cleanup
* stats
* remove getField
* agent
* server
* remote
* space
---------
Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com >
2023-11-29 12:53:06 +00:00
317c68a860
agent: fix leak in AgentOpState counters
2023-11-21 14:53:53 +02:00
40ba94ce72
remote: add multicast discovery w/ encrypted announce ( #895 )
...
* Implement multicast discovery
* replace rcConnectMulticast with explicit discoverRCCtrl
* add multicast source/invitation host check
* remove JSON encoding for multicast invitations
* add specific error for announcing "new" controllers
* rename
* set size, rename
---------
Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com >
2023-11-17 17:56:14 +00:00
3b348a463c
Merge branch 'master' into xrcp
2023-11-14 14:36:21 +00:00
7aae6f3cbe
agent: fix case of ratchet synchronization getting stuck in endless loop ( #892 )
2023-11-14 17:55:20 +04:00
6a2e6b040e
additional lazy crypto for XRCP ( #890 )
2023-11-10 11:16:43 +00:00
1a8dfb4cbe
agent: xrcp implementation for connection via link (no multicast) ( #884 )
...
* WIP: start working on connection invites
startSession/OOB is broken now - the port isn't coming from the actualy assigned one.
* Add invite types
* clean old invite-related types
* apply renames
* Move SessionKeys from Invitation
* Stub host-side keys and handle
* move keys and handles to Types
* add Simplex.RemoteControl.Client
* more keys
* progress
* crypto for sessions
* progress to multicast
* multicast crypto
* add RC TLS server
* agent api for remote control
* WIP: tls client
* fix test
* update encoding to include nonce
* add TODO
* update
* Use network-info to find TransportHost
* request and submit tls client certificate
* WIP: add missing bits for testing RC client
* RCEncryptedHello encoding
* add block encoding
* refactor
* validate known host certificate
* remove some spaghetti
* functional API to host/ctrl clients
* refactor connectRCCtrl_
* refactor connectRCHost
* question
* add type
* fix RC session
* update doc
* update doc 2
* add block on confirmation
* remove unused parameter
* export CtrlSessKeys
* export
* fix parsing
* move test of xrcp handshake
* move KEM to HELLO step
* fix JSON
* type
---------
Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com >
2023-11-07 13:05:18 +00:00
4660ee9cff
agent: revert to TVar ( #883 )
2023-11-01 09:15:51 +00:00
b5f733d2db
agent: use IORef for DRG ( #882 )
2023-10-31 23:52:13 +00:00
511d793b92
derive JSON instances with TH ( #869 )
...
* derive JSON instances with TH
* defaultJSON
* remove ToJSON
2023-10-26 09:20:30 +01:00
d920a2504b
agent: check confirmed snd queue doesn't exist before replacing ( #871 )
2023-10-25 09:48:54 +04:00
55a6157880
agent: change invLocks key type to ByteString
2023-10-24 17:20:39 +04:00
73d7f84ee3
agent: take invitation lock on join ( #870 )
2023-10-24 16:55:57 +04:00
cf8b9c12ff
remove test dependencies ( #861 )
...
* remove test dependencies
* .hlint.yaml
2023-10-22 10:50:36 +01:00
9d12d76078
add fourmolu ( #868 )
...
* add fourmolu
* linebreak
* simplify
* comment
2023-10-22 09:20:14 +01:00
Evgeny Poberezkin