Evgeny
72c2ddcf57
agent: move migrations ( #1459 )
2025-02-18 23:39:29 +00:00
spaced4ndy and Evgeny Poberezkin
0d8a1a2879
agent: encrypt messages on delivery ( #1446 )
...
* agent: save message body once (plan, schema)
* split
* new type
* bs
* encrypt on delivery
* schema
* fix test
* check pad size
* rename
---------
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com >
2025-02-14 12:35:18 +00:00
Evgeny
a58d3540ad
agent: refactor migrations ( #1452 )
...
* agent: refactor migrations
* fix postgres
2025-02-07 17:17:53 +00:00
Evgeny
992b42e922
agent: option to enable/disable vacuum after SQLite migration ( #1429 )
2024-12-28 21:06:42 +00:00
spaced4ndy
b7a8df40c4
Merge branch 'master' into postgres
2024-12-25 12:15:41 +04:00
spaced4ndy and Evgeny Poberezkin
bf28902327
agent: reuse ratchet on repeat join ( #1426 )
...
* agent: reuse ratchet on repeat join
* check status
* update
---------
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com >
2024-12-25 00:07:45 +00:00
spaced4ndy
64149599de
postgres: db interfaces wip (sqlite passes) ( #1419 )
2024-12-12 17:42:58 +04:00
spaced4ndy and Evgeny Poberezkin
a8471eed5b
ntf: get messages for multiple last notifications ( #1378 )
...
* ntf: get messages for multiple last notifications (#1352 )
* ntf: separate get ntf conns api (#1379 )
* ntf: separate get ntf conns api
* nonempty
* update
* update
* remove single get api
* fix test
* refactor
* refactor
* ntf: batch get connections (#1387 )
* ntf: batch get apis
* works
* fix
* fix
---------
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com >
2024-10-25 15:39:41 +04:00
spaced4ndy
243548631a
ntf: repeat failed token deletions ( #1346 )
2024-10-01 16:30:14 +04:00
Evgeny Poberezkin
aa60bd6770
use strict Maps, fix stats for sent END batches
2024-08-26 19:35:57 +01:00
spaced4ndy and Evgeny Poberezkin
ae8e1c5e9a
agent: servers stats improvements, fixes ( #1208 )
...
* agent: reset stats startedAt time in memory
* getAgentSubsSummary
* change sub counting
* ack statistics
* add import
* instance
* Revert "instance"
This reverts commit 1f63740d56ef72df8014evgeny@poberezkin.com >
2024-07-03 12:36:15 +01:00
Evgeny Poberezkin
5c839b40c9
Merge branch 'master' into fast-connection
2024-07-01 21:14:32 +01:00
spaced4ndy
a50e2e74a5
agent: disable saving stats ( #1214 )
...
* agent: disable stat saving
* disable migration
* schema
* disable restore
2024-07-01 21:06:38 +01:00
Evgeny Poberezkin
9ee684b0f4
rfc: faster handshake protocol ( #1203 )
...
* rfc: faster handshake protocol
* update
* 1 message
* SKEY
* use SKEY for both parties
* test
* update doc
* NEW command parameter
* add k=s param to queue URI
* fix
* add sndSecure field to queues
* make sender key non-optional in SndQueue (WIP, tests fail)
* fast handshake sometimes works (many tests fail)
* correctly handle SKEY retries, avoiding to re-generate the keys
* handle SKEY retries during async connection
* fix most tests (1 test fails)
* remove do
* fix contact requests encoding/tests
* export
* fix: ignore duplicate confirmations, fixes testBatchedPendingMessages
* do not store sndSecure in store log if it is false to allow server downgrade
* add connection invitation encoding tests
2024-06-30 08:36:24 +01:00
spaced4ndy and Evgeny Poberezkin
c788692687
agent: servers summary types, api ( #1202 )
...
* agent: servers summary types, api [wip]
* encoding
* export
* Revert "export"
This reverts commit cd9f315fe8f876fbd41826c368d82a4c9e3753b56f656440532769109+epoberezkin@users.noreply.github.com >
2024-06-25 09:42:59 +04:00
spaced4ndy
2e5433676e
xftp: check proxy before downloading from unknown server ( #1102 )
...
* xftp: check proxy before downloading from unknown server
* corrections
* remove import
* tests
* comment
* tests
* don't wrap into internal error
* fix tests
2024-05-20 14:14:04 +01:00
Evgeny Poberezkin and spaced4ndy
e06e22328f
agent: quantum-resistant double ratchet encryption ( #939 )
...
* doc
* diff
* ratchet header
* types
* ratchet step with PQ KEM, message header with KEM
* comment
* update types, remove Eq instances, store KEM keys to database
* pqx3dh
* PQ double ratchet test
* pqdr tests pass
* fix most tests
* refactor
* allow KEM proposals from both sides
* test names
* agent API parameters to use PQ KEM
* initialize ratchet state for enabling KEM
* fix/test KEM state machine to support disabling/enabling via messages
* more tests
* diff
* diff2
* refactor
* refactor
* refactor
* refactor
* remove Maybe
* rename
* add PQ encryption status to CON, MID and MSG events and sendMessage API results
* different PQ parameter when creating connection
* rename/reorganize types for PQ encryption modes
* rename
* fix testWaitDeliveryTimeout
* rename
* rename2
* ghc8107
* rename
* increase timeouts for concurrent send/receive test
* enable all tests
---------
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
2024-03-03 19:40:49 +00:00
spaced4ndy and Evgeny Poberezkin
294d7ec8dd
agent: delay connection deletion to finish delivery of pending messages ( #1015 )
...
* agent: delay connection deletion to finish delivery of pending messages (wip)
* fixes, test
* notify, test
* add tests
* comment
* add test
* timeout
* test timeout
* up
* more tests
* rename
---------
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com >
2024-02-29 18:08:58 +00:00
Alexander Bondarenko and Evgeny Poberezkin
2f7a288280
xftp: add sending and receiving via URI-encoded redirects ( #968 )
...
* xftp: add URI encoding for FileDescription
* tweak URI
* allow smaller blocks
* draft xftpReceiveFileFollow' and xftpSendFilePublic'
* add sending with redirect
* allow 64k chunks
* add migrations with redirect fields
* add test case
* fix deadlock
* revert CLI code
* WIP: working send/receive via URI
* fix field ambiguity
* cleanup
* update agent db schema
* update minimal chunk size
* add rfc
* apply suggestions from code review
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com >
* add createRcvFileRedirect
* extract Simplex.Messaging.ServiceScheme and reuse for files
* update db schema
* check size/digest on receive complete
* cleanup
* use SIZE/DIGEST errors for redirects too
* split digest/size errors from redirect checks
* fix redirect error encoding
* rename RedirectMeta to RedirectFileInfo
* use query encoding for file URI
* group maybe fields under RcvFileRedirect
* add extras field
* update rfc
* add extras encoding and no-redirect tests
* fix toStrict for old ghc
* extra client data in file descr URI
* remove decoded yaml file
---------
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com >
2024-02-13 14:08:49 +00:00
Evgeny Poberezkin and spaced4ndy
fd4eeb36db
agent: optimize expired messages query ( #976 )
...
* agent: optimize expired messages query
* schema
* fix query
* fix
* typo
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
* fix
* refactor
* comment
* refactor2
---------
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
2024-01-22 14:04:57 +00:00
Evgeny Poberezkin
b547f34cc0
Revert "Revert "agent: do not create user record in new databases ( #957 )" ( #963 )" ( #964 )
...
* Revert "Revert "agent: do not create user record in new databases (#957 )" (#963 )"
This reverts commit aee9088417
2024-01-17 11:01:04 +00:00
Evgeny Poberezkin
aee9088417
Revert "agent: do not create user record in new databases ( #957 )" ( #963 )
...
This reverts commit 9b9a0bd0df
2024-01-14 22:28:28 +00:00
Evgeny Poberezkin
9b9a0bd0df
agent: do not create user record in new databases ( #957 )
...
* agent: do not create user record in new databases
* update
2024-01-12 16:57:57 +00:00
Evgeny Poberezkin and spaced4ndy
7ddeca50e4
agent: mark work items failed ( #931 )
...
* agent: mark work items failed (WIP)
* add tests, created_at
* getWorkItem for snd and rcv files
* fix
* tests
* fix
* tests
* test
* tests
* rename
* fix,refactor
* add indexes
* update schema
* do not try to get more work when resuming an existing worker
---------
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
2023-12-26 17:00:42 +04:00
Evgeny Poberezkin
2489333c87
agent: use DB to queue async commands and messages ( #929 )
...
* agent: use DB to queue async commands and messages
* fix message delivery, async commands dont work
* fix async commands
* remove comment
* rename
* comment
* balance agent operation
* empty lines
* balance another agent operation
* diff
* worker
* qAddress
* fix incorrect queue ID in v1
* types for queues
* fix test
* update index
2023-12-23 12:59:53 +00:00
Evgeny Poberezkin
511d793b92
derive JSON instances with TH ( #869 )
...
* derive JSON instances with TH
* defaultJSON
* remove ToJSON
2023-10-26 09:20:30 +01:00
Evgeny Poberezkin
9d12d76078
add fourmolu ( #868 )
...
* add fourmolu
* linebreak
* simplify
* comment
2023-10-22 09:20:14 +01:00
Evgeny Poberezkin
5dc3d739b2
agent: support encrypted local files ( #837 )
...
* agent: support encrypted local files
* migration, update store, api
* tests, fix
* use CF.plain
2023-08-31 22:43:58 +01:00
Evgeny Poberezkin
d09adb5cd6
agent: wait for db (Revert Revert db error busy treatments ( #796 ) ( #808 )) ( #828 )
...
* Revert "Revert "db error busy treatments (#796 )" (#808 )"
This reverts commit 82aec2cd8f
2023-08-21 09:06:51 +01:00
spaced4ndy
e586bef57a
agent: add indexes to improve slow queries performance ( #823 )
2023-08-16 10:29:09 +04:00
Evgeny Poberezkin
82aec2cd8f
Revert "db error busy treatments ( #796 )" ( #808 )
...
This reverts commit 1afcefa5e7
2023-08-09 12:24:03 +01:00
spaced4ndy
e81f3b5529
add indexes ( #809 )
2023-07-22 13:20:49 +01:00
spaced4ndy
7e2b309450
migration to delete expired messages ( #801 is suspected to have caused temporary traffic increase due to flood of non expired messages) ( #806 )
2023-07-20 16:15:33 +04:00
Evgeny Poberezkin and spaced4ndy
58cb2855d2
agent: delivery receipts ( #752 )
...
* rfc: delivery receipts
* update doc
* update rfc
* implementation plan, types, schema
* migration, update types
* update types
* rename migration
* export MsgReceiptStatus, JSON encoding
* update rfc, schema
* correction
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
* skeleton of the implementation
* more implementation (some tests fail)
* more code, 1 test fails
* fix encoding
* refactor
* refactor
* test, fix
* only send receipts in v3+, test
* flip condition
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
* flip condition
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
* agent version 4 required to send receipts
* fix test
---------
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
2023-07-13 22:33:48 +01:00
spaced4ndy
1afcefa5e7
db error busy treatments ( #796 )
2023-07-11 20:19:20 +04:00
spaced4ndy and Evgeny Poberezkin
8be2505fa0
ratchet re-synchronization ( #774 )
...
* ratchet re-synchronization rfc wip
* additions
* additions, types
* fix tests
* re-sync implementation wip
* re-sync implementation 1st rev.
* test wip
* test passes
* doc
* wording
* improve doc schema
* single agreed state
* refactor (1 state variable)
* allowed -> required
* prohibit enqueue
* enqueue
* send via multiple queues
* test with server offline
* clarify errors
* rename
* more tests
* refactor
* rename AgentRKey
* rename AM_CONN_RATCHET_KEY
* more tests
* rename
* write encoded AgentRatchetInfo to AgentRatchetKey info
* move withConnLock
* refactor qDuplex
* re-create ratchet on receiving second key
* invert condition
* refactor
* simplify
---------
Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com >
2023-06-30 14:17:08 +04:00
spaced4ndy and Evgeny Poberezkin
011e1109e9
connection switch - track status, stop switch command ( #764 )
...
* connection switch improvements
* stop in transaction
* refactor
* checks in transaction
* withSwitchingRQ in transaction
* withSwitchedSQ in transaction
* refactor
* refactor
* inverse some cases
* reset snd switch progress on repeat qadd
* remove unused functions
* connection info type
* correction
* SPFinalizing phase, correctly update connection in event
* refactor
* tests
* refactor switch conn lock taking, take lock in switch stop, remove ineligible test
* comment
* corrections
* corrections
* corrections
* refactor
* remove checks in transactions, remove some statuses
* remove diff
* refactor
* refactor tests
* update connection in memory
* revert diff
* corrections
* more tests
* remove statuses, change switching queue lookup
* async queue deletion
* refactor
* lock, phases
* test status
* refactor
---------
Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com >
2023-06-15 11:15:46 +04:00
spaced4ndy
6c6f22051d
check previous encrypted message hashes for duplicates on decryption errors ( #756 )
2023-05-17 15:42:57 +04:00
spaced4ndy
ce64c91d5a
add indexes for reading pending files ( #751 )
2023-05-10 15:17:26 +04:00
spaced4ndy and Evgeny Poberezkin
8954f39425
agent: vacuum on migrations ( #745 )
...
* agent: vacuum on migrations
* vacuum after all migrations
* style
---------
Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com >
2023-05-04 17:26:05 +04:00
spaced4ndy
d8e60ecfdb
xftp: recoverable send - spike ( #707 )
2023-04-07 11:19:50 +04:00
Evgeny Poberezkin and spaced4ndy
6a665a0833
support down migrations ( #696 )
...
* support down migrations
* test
* update
* add down migrations and tests
* get current migrations
* corrections
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
* update tests
---------
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
2023-03-27 12:44:53 +01:00
Evgeny Poberezkin
d41c2bec2a
store retry intervals to resume from the same interval on restart ( #693 )
...
* store retry intervals to resume from the same interval on restart
* add migration
* updates
2023-03-22 08:42:56 +00:00
spaced4ndy and Evgeny Poberezkin
cf147397a4
xftp: agent receive file ( #637 )
...
* xftp: agent receive file draft
* receiveFile more stubs, changes to types, schema
* cabal file
* comments
* xftp_server_id
* schema changes, get client, local worker/action
* agent env, save file description, adjust schema
* client stubs
* download chunk wip, store, schema, types
* remove commented code
* read file description, schema, types
* check received, decrypt
* remove pure
* todo
* add XFTP to agent client
* add user id
* agent test
* tests
* rename supervisor into agent
---------
Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com >
2023-03-03 18:50:16 +04:00
spaced4ndy and Evgeny Poberezkin
c0dcf283eb
server_key_hash fields ( #643 )
...
* server_key_hash fields
* test
* refactor
* fix
* order
* use sync command in test
* refactor
---------
Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com >
2023-02-17 21:24:32 +00:00
Evgeny Poberezkin
d4fc638478
async batch connection deletion ( #617 )
...
* async batch connection deletion
* delete user record with connections, async connection deletion
* updates
* update query
2023-01-21 13:41:16 +00:00
JRoberts
42df6a421d
add missing fkey indexes ( #610 )
2023-01-17 16:07:13 +04:00
Evgeny Poberezkin and JRoberts
f4ad3a983e
support users in agent to isolate traffic of different users ( #598 )
...
* users table, isolate traffic sessions by users or by queues
* remove extra indices
* corrections
Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com >
Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com >
2023-01-11 17:47:20 +04:00
Evgeny Poberezkin and JRoberts
eb5c1c78cb
connection queue redundancy and rotation ( #521 )
...
* rfc: queue rotation
* update rfc
* messages for queue rotation
* allow multiple subscribed queues per connection in Agent/Client.hs
* refactor
* fix module name
* allow multiple queues in duplex connection type
* update commands
* add indices
* addConnectionRcvQueue
* switch connection to another queue (WIP)
* update schema/protocol
* switching queue works, but sending messages after the switch fails
* messages are delivered after rotation
* use connection-scoped queue ID
* rename queue records fields
* refactor using SMPQueue class/instances
* simplify queries
* QKEY: check queue is not secured, refactor
* update rfc
* mark queue as primary in QUSE
* queue rotation errors
* fix async ack
* fix async ACK to send OK
* correction
Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com >
* use SWCH command
* rename
* take into account only active queue subscription when determining connection result if at least one queue is active
* remove comment
* only enable notifications for connections with enableNtfs = True
* async test (WIP)
* async queue rotation test
* simplify combining results
* test with 2 servers
* fix unused subscribeConnection
* switch to cabal build
* increase build timeout
* increase delay in async test
* skip queue rotation tests
* build matrix
* step name
* use ubuntu-18.04 in build matrix
* enable rotation tests
Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com >
2022-10-29 18:57:01 +01:00
Evgeny Poberezkin
29332a5e9f
lints ( #539 )
...
* lints
* revert
* add import
* remove StrictData
2022-10-01 11:51:52 +01:00
Evgeny