dandri/simplexmq
1
0
Fork 0
mirror of https://github.com/simplex-chat/simplexmq.git synced 2026-04-04 15:36:40 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,060 Commits 94 Branches 171 Tags
e5cdec58fcbd2a6add00a56f89855be28aa4538c
Commit Graph

79 Commits

Author SHA1 Message Date
Evgeny Poberezkin
58cb2855d2 agent: delivery receipts (#752) 
* rfc: delivery receipts

* update doc

* update rfc

* implementation plan, types, schema

* migration, update types

* update types

* rename migration

* export MsgReceiptStatus, JSON encoding

* update rfc, schema

* correction

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* skeleton of the implementation

* more implementation (some tests fail)

* more code, 1 test fails

* fix encoding

* refactor

* refactor

* test, fix

* only send receipts in v3+, test

* flip condition

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* flip condition

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* agent version 4 required to send receipts

* fix test

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
 2023-07-13 22:33:48 +01:00
spaced4ndy
f2657f9c0b add crypto error to RSYNC event (#794) 2023-07-05 18:14:51 +04:00
spaced4ndy
0e1562deae increase connection agent version according to received messages; add version and flags to connection stats, bump currentSMPAgentVersion (#789) 2023-06-30 20:19:10 +04:00
spaced4ndy
8be2505fa0 ratchet re-synchronization (#774) 
* ratchet re-synchronization rfc wip

* additions

* additions, types

* fix tests

* re-sync implementation wip

* re-sync implementation 1st rev.

* test wip

* test passes

* doc

* wording

* improve doc schema

* single agreed state

* refactor (1 state variable)

* allowed -> required

* prohibit enqueue

* enqueue

* send via multiple queues

* test with server offline

* clarify errors

* rename

* more tests

* refactor

* rename AgentRKey

* rename AM_CONN_RATCHET_KEY

* more tests

* rename

* write encoded AgentRatchetInfo to AgentRatchetKey info

* move withConnLock

* refactor qDuplex

* re-create ratchet on receiving second key

* invert condition

* refactor

* simplify

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
 2023-06-30 14:17:08 +04:00
spaced4ndy
a000419bd7 restore confirmation after client restart (#783) 2023-06-29 10:10:34 +04:00
spaced4ndy
c8a8e2c297 differentiate agent INACTIVE error, treat as temporary (#784) 2023-06-29 10:00:53 +04:00
Evgeny Poberezkin
4a927d1ae2 ntf server: better batching and logging (#780) 
* ntf server: better batching and logging

* reduce batch delay for ntf server

* comments

* 5.1.3, ntf 1.4.2

* more logging

* more logging

* split large batches, more logging

* remove some logs
 2023-06-26 20:14:35 +01:00
spaced4ndy
c31668ce28 return updated ConnectionStats from switchConnectionAsync (#777) 2023-06-19 14:51:14 +04:00
spaced4ndy
2efe1496d2 rename "stop switch" -> "abort switch", add canAbortSwitch to rcv queue info (#775) 2023-06-16 13:41:01 +04:00
spaced4ndy
011e1109e9 connection switch - track status, stop switch command (#764) 
* connection switch improvements

* stop in transaction

* refactor

* checks in transaction

* withSwitchingRQ in transaction

* withSwitchedSQ in transaction

* refactor

* refactor

* inverse some cases

* reset snd switch progress on repeat qadd

* remove unused functions

* connection info type

* correction

* SPFinalizing phase, correctly update connection in event

* refactor

* tests

* refactor switch conn lock taking, take lock in switch stop, remove ineligible test

* comment

* corrections

* corrections

* corrections

* refactor

* remove checks in transactions, remove some statuses

* remove diff

* refactor

* refactor tests

* update connection in memory

* revert diff

* corrections

* more tests

* remove statuses, change switching queue lookup

* async queue deletion

* refactor

* lock, phases

* test status

* refactor

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
 2023-06-15 11:15:46 +04:00
spaced4ndy
fbf1afc9f5 run a single batch test to avoid hanging on server disconnect (#773) 2023-06-13 19:36:47 +04:00
spaced4ndy
b747080db3 add more message delivery tests (#763) 2023-06-02 18:00:24 +04:00
Evgeny Poberezkin
ed0f8c79a2 xftp: wait for agent in foreground in XFTP workers without delaying suspension (#721) 
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
 2023-04-13 20:04:38 +04:00
Evgeny Poberezkin
9f8db13553 xftp: agent API to set and test servers (#704) 
* xftp: agent API to set and test servers

* ProtocolTestStep

* update agent API for XFTP servers

* ci: update ubuntu versions

* disable test hanging on ubuntu
 2023-04-05 20:37:03 +01:00
Evgeny Poberezkin
6a665a0833 support down migrations (#696) 
* support down migrations

* test

* update

* add down migrations and tests

* get current migrations

* corrections

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* update tests

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
 2023-03-27 12:44:53 +01:00
Evgeny Poberezkin
ddc2da8fe4 xftp: agent API for xftp commands and events (#684) 
* xftp: agent API for xftp commands and events

* fix tests

* fix tests 2

* xftp: update agent send api

* update API to make temp path optional

* revert tmp path changes (fixes send)

---------

Co-authored-by: spacedandy <8711996+spaced4ndy@users.noreply.github.com>
 2023-03-13 09:04:23 +00:00
spaced4ndy
c1266d55a4 xftp: experimental send api wip (#682) 2023-03-10 20:43:10 +04:00
Evgeny Poberezkin
8fde8e1344 xftp: agent command entities (#676) 
* xftp: agent command entities

* progress event

* parameterize agent command by entity

* refactor

* Eq instance for APartyCmdTag
 2023-03-10 09:10:52 +00:00
Evgeny Poberezkin
15637e545b Merge branch 'master' into xftp 2023-02-17 21:40:35 +00:00
spaced4ndy
c0dcf283eb server_key_hash fields (#643) 
* server_key_hash fields

* test

* refactor

* fix

* order

* use sync command in test

* refactor

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
 2023-02-17 21:24:32 +00:00
Stanislav Dmitrenko
bccef0ba47 files: server and client spike - basic upload/download (#591) 
* Files: main, env, stats, storeLog

* Better + transport

* Executable

* Env

* Update Client.hs, Server.hs, and 4 more files...

* Answer on request

* Delay

* Temp file

* Bypass cert check

* update package.yml, rename

* update store log

* extend HTTP2 transport

* refactor caStore

* HTTP2 body

* update server stats

* file server/client framework

* verify server commands

* process FNEW command, CLI test works

* simple XFTP server test (fails)

* fix test, refactor

* upload chunk works

* receive file chunk in the client

* remove transport handshake

* typo

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* fix names

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
 2023-02-13 13:36:02 +00:00
Evgeny Poberezkin
5d8febc353 avoid possible race conditions when cancelled clients/asyncs can be removed after the new ones are added (so that the new are removed as well) (#627) 2023-01-30 17:46:06 +00:00
Evgeny Poberezkin
7fb48930f6 increase database transaction timeout, add test for user deletion (#622) 
* increase database transaction timeout, add test for user deletion

* more test logs
 2023-01-25 20:58:03 +00:00
Evgeny Poberezkin
b59669a65e refactor functions operating on single connections using batch functions (#620) 
* refactor functions operating on single connections using batch functions

* update tests
 2023-01-24 13:14:13 +00:00
Evgeny Poberezkin
2ccef1690b events sent by the agent when queue, connection and user are deleted asynchronously (#619) 
* events sent by the agent when queue, connection and user are deleted asynchronously

* update test

* more tests

* add user tests

* queries

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>
 2023-01-24 11:12:07 +00:00
Evgeny Poberezkin
d4fc638478 async batch connection deletion (#617) 
* async batch connection deletion

* delete user record with connections, async connection deletion

* updates

* update query
 2023-01-21 13:41:16 +00:00
Evgeny Poberezkin
a854b5f2b2 batched queue deletion (#611) 
* batched queue deletion

* only re-connect client on subscribe errors, not in all batch commands

* add test, fix

* correction

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>

* refactor

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>
 2023-01-19 16:53:28 +00:00
JRoberts
42df6a421d add missing fkey indexes (#610) 2023-01-17 16:07:13 +04:00
Evgeny Poberezkin
324e01300d execute asynchronous commands for correct users, include session in messages from SMP clients to agent (#608) 
* execute asynchronous commands for correct users

* pass transport session to message processing to avoid race condition

* account for server changes when determining unused servers

* enable one test

* enable all tests, remove log
 2023-01-16 11:06:19 +00:00
Evgeny Poberezkin
b549910c17 support switching session mode, transport session now uses connection ID (#607) 
* support switching session mode, transport session now uses connection ID

* refactor
 2023-01-16 08:51:41 +00:00
Evgeny Poberezkin
abee1fa70b Merge branch 'master' into users 2023-01-14 18:43:09 +00:00
Evgeny Poberezkin
701d06ba01 refactor tests to improve errors (#606) 
* refactor tests to improve errors

* fix test descriptions

* fix test
 2023-01-14 18:28:37 +00:00
Evgeny Poberezkin
f4ad3a983e support users in agent to isolate traffic of different users (#598) 
* users table, isolate traffic sessions by users or by queues

* remove extra indices

* corrections

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>
 2023-01-11 17:47:20 +04:00
Evgeny Poberezkin
fb21d9836e agent: api to get hash of ratchet associated data for connection verification (#577) 
* agent: api to get hash of ratchet associated data for connection verification

* enable all tests

* correction

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>
 2022-12-09 09:41:12 +00:00
Evgeny Poberezkin
e4842f4f47 agent: include server address string into BROKER errors (#575) 
* agent: include server address string into BROKER errors

* eol

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>
 2022-11-30 08:25:07 +00:00
Evgeny Poberezkin
f3b6ed4db0 agent API for SMP server test (#563) 
* agent API for SMP server test

* add JSON encoding
 2022-11-15 07:50:46 +00:00
Evgeny Poberezkin
e281efdcb8 server: SMP basic auth (#561) 
* server: SMP basic auth

* update ini default

* rfc, types

* tests

* update INI file

* typo

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>

* refactor

* update INI options, log new queue creation mode on start (on/off/requires auth)

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>
 2022-11-12 22:29:36 +00:00
JRoberts
d9a0e78b04 add optional client data field to ConnectionRequestUri (#551) 
* add optional auxiliary data field to ConnectionRequestUri

* remove import

* fix, test

* fix StrEncoding Char

* data only in sync command, type

* fixing

* queryParamStr

* safeDecodeUtf8

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
 2022-11-01 13:42:44 +04:00
Evgeny Poberezkin
4beaafa35a add queue directions to SWITCH events (#553) 2022-11-01 08:44:32 +00:00
Evgeny Poberezkin
8d9816809f simplify queue rotation protocol (#550) 
* simplify queue rotation protocol

* use simplified rotation protocol, update tests

* simplify schema

* delete all connection queues

* refactor

* switch notifications to the new queue, tests

* remove TODO

* refactor

* rfc correction

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>

* remove duplicate set active

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>
 2022-10-31 13:33:28 +04:00
Evgeny Poberezkin
eb5c1c78cb connection queue redundancy and rotation (#521) 
* rfc: queue rotation

* update rfc

* messages for queue rotation

* allow multiple subscribed queues per connection in Agent/Client.hs

* refactor

* fix module name

* allow multiple queues in duplex connection type

* update commands

* add indices

* addConnectionRcvQueue

* switch connection to another queue (WIP)

* update schema/protocol

* switching queue works, but sending messages after the switch fails

* messages are delivered after rotation

* use connection-scoped queue ID

* rename queue records fields

* refactor using SMPQueue class/instances

* simplify queries

* QKEY: check queue is not secured, refactor

* update rfc

* mark queue as primary in QUSE

* queue rotation errors

* fix async ack

* fix async ACK to send OK

* correction

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>

* use SWCH command

* rename

* take into account only active queue subscription when determining connection result if at least one queue is active

* remove comment

* only enable notifications for connections with enableNtfs = True

* async test (WIP)

* async queue rotation test

* simplify combining results

* test with 2 servers

* fix unused subscribeConnection

* switch to cabal build

* increase build timeout

* increase delay in async test

* skip queue rotation tests

* build matrix

* step name

* use ubuntu-18.04 in build matrix

* enable rotation tests

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>
 2022-10-29 18:57:01 +01:00
Evgeny Poberezkin
d06bb2b9c5 use KDF in X3DH (#548) 
* use KDF in X3DH

* configure ratchet version via agent config, test matrix

* add comment
 2022-10-20 11:03:19 +01:00
Evgeny Poberezkin
f43e5d6833 refactor test matrix (#549) 2022-10-20 10:13:33 +01:00
JRoberts
f97c1a7712 agent: acceptContactAsync functional api (#545) 2022-10-13 19:46:32 +04:00
Evgeny Poberezkin
10e0e58ec3 improve concurrency by not holding connection lock while async command retries (#544) 
* improve concurrency by not holding connection lock while async command retries

* fix

* debug test
 2022-10-09 18:00:42 +01:00
JRoberts
a2eea4f18f add deleteConnectionAsync functional api (#538) 2022-09-29 21:11:36 +04:00
Evgeny Poberezkin
413aad5139 allow passing agent store instead of path and key (#535) 2022-09-23 18:45:00 +01:00
Evgeny Poberezkin
bab6a1577b correlation IDs and command tags for async commands (#519) 
* agent protocol command tags

* store/send async command correlation IDs

* fix, refactor

* delete failed command
 2022-09-10 16:33:15 +01:00
JRoberts
882d2b123b agent async commands (#513) 
* wip

* wip

* wip

* wip

* don't delete conn on join failure in async mode

* test restart

* remove unused pragma

* simplify restore

* rename variable

* refactor

* refactor

* split

* remove error

* prohibit async join via contact request

* refactor subscribeConnections

* simplify

* resum command processing on subscriptions

* unfocus tests

* schema

* remove do

* rename variable

* corrections

* move code

* refactor getPendingCommands

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
 2022-09-08 20:27:56 +04:00
Evgeny Poberezkin
212a9bc4ab enable/disable connection notifications (#501) 
* enable/disable connection notifications

* filter connections when intializing/disabling subscriptions

* fix

* fix 2

* add delay

* increase test delay

* increase test delay

* fix

* increase test delays

* increase test delays
 2022-08-19 15:23:57 +01:00