Commit Graph

100 Commits

Author SHA1 Message Date
spaced4ndy
3b1dd6dff9 agent: notify about available quota (#1087)
* agent: notify about available quota

* rename

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2024-04-11 09:38:12 +01:00
Evgeny Poberezkin
1219446996 dispose agent instances, fix tests, (#1089)
* dispose agent instances in tests

* fix quota test

* tests: fix tests with -threaded (#1088)

* fix some tests

* match RTS opts with apps

* less verbose rts stats

* enable sqlite extended error codes

* clean up

* unfocus

* remove extendedErrorCode

It's actually setExtendedResultCodes, which isn't yet available.

* diff

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>

* fix switch test

* fix

---------

Co-authored-by: Alexander Bondarenko <486682+dpwiz@users.noreply.github.com>
2024-04-10 19:34:02 +01:00
Evgeny Poberezkin
07fa75ec49 pqdr: agent api to confirm PQ encryption support during connection handshake, fix incorrect PQ support (#1032)
* pqdr: agent api to confirm PQ encryption support during connection handshake

* fix CONF, tests

* fix REQ, tests

* remove unused
2024-03-07 08:35:40 +00:00
Evgeny Poberezkin
4ffb6a348a pqdr: use different newtypes for supporting and enabling PQ encryption in connections (#1031)
* pqdr: use different newtypes for supporting and enabling PQ encryption in connections

* rename field, fix test

* refactor
2024-03-06 21:28:03 +00:00
Evgeny Poberezkin
b050cf5027 double ratchet versioning for post-quantum encryption (#1025)
* correctly parse new Ratchet fields when omitted

* rfc: migrating connection versions to pqdr

* update rfc

* WIP (dont commit)

* rename versions

* update ratchet version based on PQ encryption feature flag

* remove duplicate function

* synchronize ratchet, fix tests, refactor

* comments

* test

* pattern
2024-03-05 17:07:15 +00:00
Evgeny Poberezkin
e06e22328f agent: quantum-resistant double ratchet encryption (#939)
* doc

* diff

* ratchet header

* types

* ratchet step with PQ KEM, message header with KEM

* comment

* update types, remove Eq instances, store KEM keys to database

* pqx3dh

* PQ double ratchet test

* pqdr tests pass

* fix most tests

* refactor

* allow KEM proposals from both sides

* test names

* agent API parameters to use PQ KEM

* initialize ratchet state for enabling KEM

* fix/test KEM state machine to support disabling/enabling via messages

* more tests

* diff

* diff2

* refactor

* refactor

* refactor

* refactor

* remove Maybe

* rename

* add PQ encryption status to CON, MID and MSG events and sendMessage API results

* different PQ parameter when creating connection

* rename/reorganize types for PQ encryption modes

* rename

* fix testWaitDeliveryTimeout

* rename

* rename2

* ghc8107

* rename

* increase timeouts for concurrent send/receive test

* enable all tests

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2024-03-03 19:40:49 +00:00
Evgeny Poberezkin
c179073260 remove support for old versions (#990)
* remove support for old versions (WIP)

* fix

* updates

* use version var
2024-02-16 13:28:50 +00:00
Evgeny Poberezkin
2ae1c9f79d agent: only retry connecting SMP clients when there are pending subscriptions (#981)
* agent: only retry connecting SMP clients when there are pending subscriptions

* fix

* remove retry on creating clients

* simplify
2024-02-01 16:17:37 +00:00
Evgeny Poberezkin
577e3cf14d agent: add server msg IDs to agent logs, disable 2 tests (#934)
* agent: race when using agent via TCP

* remove logs

* enable tests

* comments

* use different databases for different clients

* enable all tests
2023-12-26 17:50:39 +00:00
Evgeny Poberezkin
7ddeca50e4 agent: mark work items failed (#931)
* agent: mark work items failed (WIP)

* add tests, created_at

* getWorkItem for snd and rcv files

* fix

* tests

* fix

* tests

* test

* tests

* rename

* fix,refactor

* add indexes

* update schema

* do not try to get more work when resuming an existing worker

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2023-12-26 17:00:42 +04:00
Evgeny Poberezkin
9d12d76078 add fourmolu (#868)
* add fourmolu

* linebreak

* simplify

* comment
2023-10-22 09:20:14 +01:00
Evgeny Poberezkin
84ce037867 extend SMP protocol to allow creating new queues without subscriptions (#839)
* Trace auto-subs flag

* Replace Bools with SubscriptionMode

* Handle SMOnlyCreate

* Wire remaining todos

* Update tests and fix

* Bump protocol level

* Apply suggestions from code review

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>

* Scrub needs_sub from agent DB

* Scrub a few more needSubs from the agent api

* change API, fix test

* agent: do not subscribe to queue when creating reply queue

* fix encoding

* WIP: SMOnlyCreate test

* Add SM guard for confirmQueue

Allows the test case to pump the allowConnection
reply without getting PROHIBITED.

* Remove tracing

* add noMessages, remove unnecessary getConnectionMessage from test

* add sending messages to the test

---------

Co-authored-by: IC Rainbow <aenor.realm@gmail.com>
Co-authored-by: Alexander Bondarenko <486682+dpwiz@users.noreply.github.com>
2023-09-10 17:07:19 +01:00
spaced4ndy
e586bef57a agent: add indexes to improve slow queries performance (#823) 2023-08-16 10:29:09 +04:00
Evgeny Poberezkin
6a665a0833 support down migrations (#696)
* support down migrations

* test

* update

* add down migrations and tests

* get current migrations

* corrections

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* update tests

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2023-03-27 12:44:53 +01:00
Evgeny Poberezkin
ddc2da8fe4 xftp: agent API for xftp commands and events (#684)
* xftp: agent API for xftp commands and events

* fix tests

* fix tests 2

* xftp: update agent send api

* update API to make temp path optional

* revert tmp path changes (fixes send)

---------

Co-authored-by: spacedandy <8711996+spaced4ndy@users.noreply.github.com>
2023-03-13 09:04:23 +00:00
Evgeny Poberezkin
8fde8e1344 xftp: agent command entities (#676)
* xftp: agent command entities

* progress event

* parameterize agent command by entity

* refactor

* Eq instance for APartyCmdTag
2023-03-10 09:10:52 +00:00
Evgeny Poberezkin
bbf9e28696 xftp: add server to client encryption layer with cryptobox (#646)
* xftp: add server to client encryption layer with cryptobox

* workarounds for http2

* update http2 commit

* remove TODOs
2023-02-22 09:03:18 +00:00
Evgeny Poberezkin
324e01300d execute asynchronous commands for correct users, include session in messages from SMP clients to agent (#608)
* execute asynchronous commands for correct users

* pass transport session to message processing to avoid race condition

* account for server changes when determining unused servers

* enable one test

* enable all tests, remove log
2023-01-16 11:06:19 +00:00
Evgeny Poberezkin
b549910c17 support switching session mode, transport session now uses connection ID (#607)
* support switching session mode, transport session now uses connection ID

* refactor
2023-01-16 08:51:41 +00:00
Evgeny Poberezkin
058e3ac55e send/process "quota exceeded" message from SMP server when sender gets ERR QUOTA (#585)
* send "quota exceeded" message from SMP server when sender gets ERR QUOTA (ignored in the agent for now)

* send msg quota to the recipient to indicate that sender got ERR QUOTA, test

* switch between slow/fast retry intervals (tests do not pass yet)

* send QCONT message, refactor RetryInterval, test

* refactor

* remove comment

* remove space

* unit test for withRetryLock2

* refactor
2023-01-04 14:10:13 +00:00
Evgeny Poberezkin
e4842f4f47 agent: include server address string into BROKER errors (#575)
* agent: include server address string into BROKER errors

* eol

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>
2022-11-30 08:25:07 +00:00
Evgeny Poberezkin
413aad5139 allow passing agent store instead of path and key (#535) 2022-09-23 18:45:00 +01:00
Evgeny Poberezkin
0a9b6e4ab4 allow repeated secure with the same key (KEY command), store suspended queues in store log (#534)
* allow repeated secure with the same key (KEY command), store suspended queues in store log

* fix test
2022-09-22 13:09:39 +01:00
JRoberts
882d2b123b agent async commands (#513)
* wip

* wip

* wip

* wip

* don't delete conn on join failure in async mode

* test restart

* remove unused pragma

* simplify restore

* rename variable

* refactor

* refactor

* split

* remove error

* prohibit async join via contact request

* refactor subscribeConnections

* simplify

* resum command processing on subscriptions

* unfocus tests

* schema

* remove do

* rename variable

* corrections

* move code

* refactor getPendingCommands

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2022-09-08 20:27:56 +04:00
Evgeny Poberezkin
02bba01c16 send host events when server hosts are connected and disconnected (#496) 2022-08-13 11:57:36 +01:00
Evgeny Poberezkin
d810db4eed send servers in agent events (#478)
* send servers in agent events

* remove some changes

* command/function to get connection servers

* getConnectionServers return type
2022-07-20 13:56:43 +01:00
Evgeny Poberezkin
9c1b43791c SMP v3: encrypt message timestamp and flags together with the body between server and recipient (#457)
* SMP v3: encrypt message timestamp and flags together with the body between server and recipient

* v3 tests

* update protocol doc

* add test for max size message

* delay in notifications test

* simplify v3

* encrypt server message to the recipient when sent

* refactor

* exit on error restoring the messages

* refactor, increase test timeout

* style

* add prints to the test

* remove error from unsafeMaxLenBS

* update protocol

* lint, improve test, change func param
2022-07-05 21:08:05 +01:00
Evgeny Poberezkin
c1348aa54f SMP connection handshake v2 (#390)
* SMP connection handshake v2

* hadshake v2 [mostly] works

* all tests pass, some race conditions remain

* fix build

* fix race conditions, send CON after all HELLOs in duplexHandshake mode

* add comments

* comment

* add comments
2022-06-09 13:47:07 +01:00
JRoberts
bfb556c860 ntf: test notification subscription (#389) 2022-06-07 19:14:51 +04:00
Evgeny Poberezkin
4b3d04bd27 support message flags visible to SMP server to control notifications (and for any future extensions) (#386)
* support stopping and resuming agent  (#385)

* export agentDbPath

* support fully closing and resuming agent

* whitespace

* clean up

* support message flags visible to SMP server to control notifications (and for any future extensions)

* simplify message flags encoding

* GET command
2022-06-06 12:59:45 +01:00
Evgeny Poberezkin
c74f4d729b create/verify agent schema during tests (#374)
* create/verify agent schema during tests

* add --indent to .schema, enable all tests

* remove -threaded from test
2022-05-24 19:28:40 +01:00
Evgeny Poberezkin
c6dde772b4 batch all connections in DOWN/UP agent messages (#363) 2022-04-25 08:26:52 +01:00
Evgeny Poberezkin
7a19ab224b make sending confirmation asynchronous (#327)
* make sending confirmation asynchronous

* confirm first queue synchronously (on JOIN, and delete connection on failure), second queue asynchronously (from REPLY msg)

* test to retry JOIN when the first attempt fails

* process confirmation delivery errors
2022-03-02 15:52:45 +00:00
Efim Poberezkin
c380c79560 make new smp client attempt to reconnect on network error (#315)
* make new smp client attempt to reconnect on network error

* reconnect async wip

* remove comments

* trigger tryReconnectClient

* add subscription before trying to take smp client

* move atomically modify asyncClients

* remove logs

* corrections

* don't put error into new smp client TMVar

* UnliftIO async

* re-work

* test for connection to the server that was initially down

* fix tests

* additionally test that bob receives the message after re-connect

* make test independent of event order

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2022-02-10 14:48:11 +00:00
Evgeny Poberezkin
6fe3bfa980 JSON encoding of types used in simplex-chat (#311)
* JSON encoding of types used in simplex-chat

* add field names for JSON encodings, encode all error sum-types as objects (to allow extension)
2022-01-26 20:18:41 +00:00
Efim Poberezkin
d48da40f7e add IP and FQDN to online certificate (#278) 2022-01-11 16:35:06 +04:00
Efim Poberezkin
5e57c5dfd7 rename remaining occurrences of connection alias into connection id (#265) 2022-01-07 15:37:40 +04:00
Efim Poberezkin
d87ff42cf5 start with internal message id 1; fix id ordering in tests (#264) 2022-01-07 14:52:37 +04:00
Evgeny Poberezkin
a5a00f3d02 Merge branch 'stable' into ep/message-delivery 2022-01-06 16:38:22 +00:00
Evgeny Poberezkin
10c62e7fa2 fix message delivery when quota exceeded - making it concurrent per queue, not per server (#260)
* test: HELLO blocking message delivery on the same server

* test: quota exceeded in one queue should not block delivery in other queues

* fix test to work in stable branch

* simplify pending message delivery (#202)

* simplify pending message delivery (WIP)

* refactor

* fix concurrent message delivery

* remove type synonym
2022-01-06 08:56:19 +00:00
Evgeny Poberezkin
6f1d9db8ec test: quota exceeded in one queue should not block delivery in other queues 2022-01-05 21:48:40 +00:00
Evgeny Poberezkin
3c923a3dc0 test: HELLO blocking message delivery on the same server 2022-01-05 21:12:31 +00:00
Evgeny Poberezkin
488398df9f change message envelopes and encoding, unify message delivery (#252)
* types and encodings for double ratchet integration

* upgrade stack resolver

* type classes for version agreement, encode/decode connection request links and E2E params with versioning

* encode/decode client parameters (version and DH key) in SMP queue URI using query string parameters

* restore support of the current SMP queue URI format

* update AMessage to only send queues in REPLY message (not the full connection request)

* new agent message evnvelopes (tests fail)

* new message envelopes - tests pass

* store fully encrypted messages before sending

* unify message delivery via DB queue (excluding confirmation and invitation)

* remove activateSecuredQueue

* linter hints

* remove comment

* export order

* save rachet-encrypted message, not per-queue encrypted

* delete message after it is accepted by the server, reduce message delivery interval for the tests

Co-authored-by: Efim Poberezkin <8711996+efim-poberezkin@users.noreply.github.com>
2022-01-05 19:52:37 +00:00
Evgeny Poberezkin
f3523bbba9 make KeyHash non-optional, verify KeyHash in SMP handshake, use StrEncoding class (#250)
* make KeyHash non-optional, StrEncoding class

* change server URI format in agent config, refactor with StrEncoding

* refactor Crypto using checkAlgorithm

* refactor parsing connection requests

* prepare to validate CA fingerprint sent in client handshake

* KeyHash check in handshake

* rename type to CliCommand

* server validates keyhash sent by the client

* validate -a option when parsing

* more of StrEncoding
2022-01-02 22:24:43 +00:00
Evgeny Poberezkin
73cad5a6c4 simple per-queue e2e encryption with NaCl crypto_box (#242)
* simple per-queue e2e encryption with NaCl crypto_box

* add e2e keys and DH secrets to schema

* agree and save shared DH secret per queue (not used yet)

* protocol changes for uniform padding and message part lengths

* correct message structure diagrams

* make per-queue E2E encryption non-optional

* refactor crypto keys

* use NaCl crypto_box for per-queue E2E encryption, remove RSA keys from queues

* remove RSA support

* merge migration with E2E DH keys

* clean up

* remove unused methods

* parsing/serializing agent messages

* remove sender timestamp from DB and code

* clean up

* slean up

* s/SMPConfMsg/SMPConfirmation/

* serializeAgentMessage = serializeClientMessage . agentToClientMsg

* simplify error handling

* update protocol docs
2021-12-29 14:27:10 +00:00
Evgeny Poberezkin
51a9750891 double ratchet algorithm implementation (#236)
* started double ratchet implementation

* initialize ratchets

* started ratchet encryption

* ratchet encryption

* simplify / narrow down Ratchet type

* double ratchet decryption "framework"

* advance receive ratched on skipped messages

* more ratchet decryption

* double ratchet encrypt/decrypt (TODO tests)

* double ratchet tests

* double ratchet tests

* use ratchet AD in header encryption, use header and ratchet AD as AD in message encryption

* change ratchet message error, remove Show instances

* Update tests/AgentTests/DoubleRatchetTests.hs

Co-authored-by: Efim Poberezkin <8711996+efim-poberezkin@users.noreply.github.com>

* Update tests/AgentTests/DoubleRatchetTests.hs

Co-authored-by: Efim Poberezkin <8711996+efim-poberezkin@users.noreply.github.com>

* Update tests/AgentTests/DoubleRatchetTests.hs

Co-authored-by: Efim Poberezkin <8711996+efim-poberezkin@users.noreply.github.com>

* Update tests/AgentTests/DoubleRatchetTests.hs

Co-authored-by: Efim Poberezkin <8711996+efim-poberezkin@users.noreply.github.com>

* Update tests/AgentTests/DoubleRatchetTests.hs

Co-authored-by: Efim Poberezkin <8711996+efim-poberezkin@users.noreply.github.com>

* Update src/Simplex/Messaging/Crypto/Ratchet.hs

Co-authored-by: Efim Poberezkin <8711996+efim-poberezkin@users.noreply.github.com>

* test in the same ratchet step

* merge tests

* Update src/Simplex/Messaging/Crypto/Ratchet.hs

* Update src/Simplex/Messaging/Crypto/Ratchet.hs

* remove HMAC comment

Co-authored-by: Efim Poberezkin <8711996+efim-poberezkin@users.noreply.github.com>
2021-12-25 17:13:53 +00:00
Efim Poberezkin
0c866105d2 chain of two certificates - offline (identity) and online; switch certificates to v3 (#238) 2021-12-23 21:20:41 +04:00
Efim Poberezkin
96cf4d288c base64url encode fingerprint in agent test (#237) 2021-12-23 13:04:56 +04:00
Efim Poberezkin
e2cd370513 certificate validation on client side; check stored fingerprint on server start-up; non-optional fingerprint parsing (#234, closes #155)
Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2021-12-22 23:24:58 +04:00
Evgeny Poberezkin
f640e71f82 remove BlockArguments extension (#220) 2021-12-10 12:21:54 +00:00