Commit Graph
123 Commits
Author SHA1 Message Date
Evgeny Poberezkin 4ffb6a348a pqdr: use different newtypes for supporting and enabling PQ encryption in connections (#1031)
* pqdr: use different newtypes for supporting and enabling PQ encryption in connections

* rename field, fix test

* refactor
2024-03-06 21:28:03 +00:00
Evgeny Poberezkin ba1bfaa5aa Merge branch 'master' into pq 2024-03-04 20:13:01 +00:00
Evgeny Poberezkin dd2bd11584 parameterize version scopes with phantom types (#1026)
* parameterize version scopes with phantom types

* move Version to another module

* parens
2024-03-04 19:06:51 +00:00
Evgeny Poberezkinandspaced4ndy e06e22328f agent: quantum-resistant double ratchet encryption (#939)
* doc

* diff

* ratchet header

* types

* ratchet step with PQ KEM, message header with KEM

* comment

* update types, remove Eq instances, store KEM keys to database

* pqx3dh

* PQ double ratchet test

* pqdr tests pass

* fix most tests

* refactor

* allow KEM proposals from both sides

* test names

* agent API parameters to use PQ KEM

* initialize ratchet state for enabling KEM

* fix/test KEM state machine to support disabling/enabling via messages

* more tests

* diff

* diff2

* refactor

* refactor

* refactor

* refactor

* remove Maybe

* rename

* add PQ encryption status to CON, MID and MSG events and sendMessage API results

* different PQ parameter when creating connection

* rename/reorganize types for PQ encryption modes

* rename

* fix testWaitDeliveryTimeout

* rename

* rename2

* ghc8107

* rename

* increase timeouts for concurrent send/receive test

* enable all tests

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2024-03-03 19:40:49 +00:00
Evgeny Poberezkin c179073260 remove support for old versions (#990)
* remove support for old versions (WIP)

* fix

* updates

* use version var
2024-02-16 13:28:50 +00:00
416f1b1721 smp: command authorization (#982)
* smp: command authorization

* fix encoding, most tests

* remove old tests

* authorize via crypto_box

* extract authenticator to Crypto module

* make TransmissionAuth Maybe

* rfc

* support authenticators in NTF protocol, test matrix (no backwards compatibility yet from new clients to old servers)

* fix/add tests, add version config to "small" agent

* separate client and server versions for SMP protocol

* test batching SMP v7

* do not send session ID in each transmission

* refactor auth verification in the server, split tests

* server "warm up" fixes timing test

* uncomment SUB timing test

* comments, disable two timing tests

* rename version

* increase auth timing test failure threshold

* use different algorithms to authorize snd/rcv commands, use random correlation ID

* transport: fetch and store server certificate (#985)

* THandleParams (WIP, does not compile)

* transport: fetch and store server certificate

* smp: add getOnlinePubKey example to smpClientHandshake

* add server certs and sign authPub

* cleanup

* update

* style

* load server certs from test fixtures

* sign ntf authPubKey

* fix onServerCertificate

* increase delay before sending messages

* require certificate with key in SMP server handshake

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>

* remove dhSecret from THandle

* remove v8, merge all changes to one version

* parameterize THandle

* rfc: transmission ecnryption

* Revert "parameterize THandle"

This reverts commit 75adfc94fb.

* use batch syntax for ntf server commands

* separate encodeTransmission when there is no key

* typo

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* rename

* diff

---------

Co-authored-by: Alexander Bondarenko <486682+dpwiz@users.noreply.github.com>
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2024-02-16 11:45:54 +00:00
Evgeny Poberezkinandspaced4ndy 7ddeca50e4 agent: mark work items failed (#931)
* agent: mark work items failed (WIP)

* add tests, created_at

* getWorkItem for snd and rcv files

* fix

* tests

* fix

* tests

* test

* tests

* rename

* fix,refactor

* add indexes

* update schema

* do not try to get more work when resuming an existing worker

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2023-12-26 17:00:42 +04:00
Evgeny Poberezkin 2489333c87 agent: use DB to queue async commands and messages (#929)
* agent: use DB to queue async commands and messages

* fix message delivery, async commands dont work

* fix async commands

* remove comment

* rename

* comment

* balance agent operation

* empty lines

* balance another agent operation

* diff

* worker

* qAddress

* fix incorrect queue ID in v1

* types for queues

* fix test

* update index
2023-12-23 12:59:53 +00:00
Alexander BondarenkoandEvgeny Poberezkin 6bffcc8503 compatibility with GHC 8.10.7 (#906)
* build with ghc-8.10

* add 8.10 to matrix

* cleanup

* stats

* remove getField

* agent

* server

* remote

* space

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-11-29 12:53:06 +00:00
spaced4ndy d920a2504b agent: check confirmed snd queue doesn't exist before replacing (#871) 2023-10-25 09:48:54 +04:00
Moritz AngermannandEvgeny Poberezkin 002f36dde0 5.4.0.0: use ghc 9.6.2 (#791)
* Add 9.6 compat

* compile with GHC9.6.2: dependencies, imports, code

* refactor typeclasses

* refactor record dot

* update cabal version

* update github actions

* update direct-sqlcipher

* 5.4.0.0

* update cabal.project

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-08-24 20:22:02 +01:00
Evgeny Poberezkinandspaced4ndy 58cb2855d2 agent: delivery receipts (#752)
* rfc: delivery receipts

* update doc

* update rfc

* implementation plan, types, schema

* migration, update types

* update types

* rename migration

* export MsgReceiptStatus, JSON encoding

* update rfc, schema

* correction

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* skeleton of the implementation

* more implementation (some tests fail)

* more code, 1 test fails

* fix encoding

* refactor

* refactor

* test, fix

* only send receipts in v3+, test

* flip condition

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* flip condition

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* agent version 4 required to send receipts

* fix test

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2023-07-13 22:33:48 +01:00
spaced4ndy 0e1562deae increase connection agent version according to received messages; add version and flags to connection stats, bump currentSMPAgentVersion (#789) 2023-06-30 20:19:10 +04:00
spaced4ndyandEvgeny Poberezkin 8be2505fa0 ratchet re-synchronization (#774)
* ratchet re-synchronization rfc wip

* additions

* additions, types

* fix tests

* re-sync implementation wip

* re-sync implementation 1st rev.

* test wip

* test passes

* doc

* wording

* improve doc schema

* single agreed state

* refactor (1 state variable)

* allowed -> required

* prohibit enqueue

* enqueue

* send via multiple queues

* test with server offline

* clarify errors

* rename

* more tests

* refactor

* rename AgentRKey

* rename AM_CONN_RATCHET_KEY

* more tests

* rename

* write encoded AgentRatchetInfo to AgentRatchetKey info

* move withConnLock

* refactor qDuplex

* re-create ratchet on receiving second key

* invert condition

* refactor

* simplify

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-06-30 14:17:08 +04:00
spaced4ndy 2efe1496d2 rename "stop switch" -> "abort switch", add canAbortSwitch to rcv queue info (#775) 2023-06-16 13:41:01 +04:00
spaced4ndyandEvgeny Poberezkin 011e1109e9 connection switch - track status, stop switch command (#764)
* connection switch improvements

* stop in transaction

* refactor

* checks in transaction

* withSwitchingRQ in transaction

* withSwitchedSQ in transaction

* refactor

* refactor

* inverse some cases

* reset snd switch progress on repeat qadd

* remove unused functions

* connection info type

* correction

* SPFinalizing phase, correctly update connection in event

* refactor

* tests

* refactor switch conn lock taking, take lock in switch stop, remove ineligible test

* comment

* corrections

* corrections

* corrections

* refactor

* remove checks in transactions, remove some statuses

* remove diff

* refactor

* refactor tests

* update connection in memory

* revert diff

* corrections

* more tests

* remove statuses, change switching queue lookup

* async queue deletion

* refactor

* lock, phases

* test status

* refactor

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-06-15 11:15:46 +04:00
spaced4ndy 6c6f22051d check previous encrypted message hashes for duplicates on decryption errors (#756) 2023-05-17 15:42:57 +04:00
spaced4ndy e346a81e46 xftp: delete snd files on servers (#722) 2023-04-13 17:46:43 +04:00
Evgeny Poberezkin d41c2bec2a store retry intervals to resume from the same interval on restart (#693)
* store retry intervals to resume from the same interval on restart

* add migration

* updates
2023-03-22 08:42:56 +00:00
Evgeny Poberezkinandspacedandy ddc2da8fe4 xftp: agent API for xftp commands and events (#684)
* xftp: agent API for xftp commands and events

* fix tests

* fix tests 2

* xftp: update agent send api

* update API to make temp path optional

* revert tmp path changes (fixes send)

---------

Co-authored-by: spacedandy <8711996+spaced4ndy@users.noreply.github.com>
2023-03-13 09:04:23 +00:00
Evgeny Poberezkin 8fde8e1344 xftp: agent command entities (#676)
* xftp: agent command entities

* progress event

* parameterize agent command by entity

* refactor

* Eq instance for APartyCmdTag
2023-03-10 09:10:52 +00:00
spaced4ndyandEvgeny Poberezkin cf147397a4 xftp: agent receive file (#637)
* xftp: agent receive file draft

* receiveFile more stubs, changes to types, schema

* cabal file

* comments

* xftp_server_id

* schema changes, get client, local worker/action

* agent env, save file description, adjust schema

* client stubs

* download chunk wip, store, schema, types

* remove commented code

* read file description, schema, types

* check received, decrypt

* remove pure

* todo

* add XFTP to agent client

* add user id

* agent test

* tests

* rename supervisor into agent

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-03-03 18:50:16 +04:00
spaced4ndyandEvgeny Poberezkin c0dcf283eb server_key_hash fields (#643)
* server_key_hash fields

* test

* refactor

* fix

* order

* use sync command in test

* refactor

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-02-17 21:24:32 +00:00
Evgeny Poberezkin d4fc638478 async batch connection deletion (#617)
* async batch connection deletion

* delete user record with connections, async connection deletion

* updates

* update query
2023-01-21 13:41:16 +00:00
Evgeny Poberezkin 324e01300d execute asynchronous commands for correct users, include session in messages from SMP clients to agent (#608)
* execute asynchronous commands for correct users

* pass transport session to message processing to avoid race condition

* account for server changes when determining unused servers

* enable one test

* enable all tests, remove log
2023-01-16 11:06:19 +00:00
Evgeny PoberezkinandJRoberts f4ad3a983e support users in agent to isolate traffic of different users (#598)
* users table, isolate traffic sessions by users or by queues

* remove extra indices

* corrections

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>
2023-01-11 17:47:20 +04:00
Evgeny Poberezkin 1d3a604988 refactor SMP client, NTF server store (#580)
* refactor SMP client, NTF server store

* remove undefined
2022-12-26 18:54:28 +00:00
Evgeny PoberezkinandJRoberts 8d9816809f simplify queue rotation protocol (#550)
* simplify queue rotation protocol

* use simplified rotation protocol, update tests

* simplify schema

* delete all connection queues

* refactor

* switch notifications to the new queue, tests

* remove TODO

* refactor

* rfc correction

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>

* remove duplicate set active

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>
2022-10-31 13:33:28 +04:00
Evgeny PoberezkinandJRoberts eb5c1c78cb connection queue redundancy and rotation (#521)
* rfc: queue rotation

* update rfc

* messages for queue rotation

* allow multiple subscribed queues per connection in Agent/Client.hs

* refactor

* fix module name

* allow multiple queues in duplex connection type

* update commands

* add indices

* addConnectionRcvQueue

* switch connection to another queue (WIP)

* update schema/protocol

* switching queue works, but sending messages after the switch fails

* messages are delivered after rotation

* use connection-scoped queue ID

* rename queue records fields

* refactor using SMPQueue class/instances

* simplify queries

* QKEY: check queue is not secured, refactor

* update rfc

* mark queue as primary in QUSE

* queue rotation errors

* fix async ack

* fix async ACK to send OK

* correction

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>

* use SWCH command

* rename

* take into account only active queue subscription when determining connection result if at least one queue is active

* remove comment

* only enable notifications for connections with enableNtfs = True

* async test (WIP)

* async queue rotation test

* simplify combining results

* test with 2 servers

* fix unused subscribeConnection

* switch to cabal build

* increase build timeout

* increase delay in async test

* skip queue rotation tests

* build matrix

* step name

* use ubuntu-18.04 in build matrix

* enable rotation tests

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>
2022-10-29 18:57:01 +01:00
Evgeny Poberezkin 29332a5e9f lints (#539)
* lints

* revert

* add import

* remove StrictData
2022-10-01 11:51:52 +01:00
Evgeny Poberezkin 41d3c14157 secure queue asynchronously (#532)
* secure queue asynchronously

* delete commands with internal logic errors
2022-09-20 15:11:35 +01:00
Evgeny Poberezkin d4962daf11 internal async commands (#530)
* internal async commands

* rename

* remove GADT from AgentCommand
2022-09-18 14:02:20 +01:00
Evgeny Poberezkin e3e05d474d wrap agent command for async command processing, to allow extension with internal commands (#528)
* wrap agent command for async command processing, to allow extension with internal commands

* change encoding for constructor
2022-09-17 14:08:49 +01:00
JRobertsandEvgeny Poberezkin 882d2b123b agent async commands (#513)
* wip

* wip

* wip

* wip

* don't delete conn on join failure in async mode

* test restart

* remove unused pragma

* simplify restore

* rename variable

* refactor

* refactor

* split

* remove error

* prohibit async join via contact request

* refactor subscribeConnections

* simplify

* resum command processing on subscriptions

* unfocus tests

* schema

* remove do

* rename variable

* corrections

* move code

* refactor getPendingCommands

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2022-09-08 20:27:56 +04:00
Evgeny Poberezkin 212a9bc4ab enable/disable connection notifications (#501)
* enable/disable connection notifications

* filter connections when intializing/disabling subscriptions

* fix

* fix 2

* add delay

* increase test delay

* increase test delay

* fix

* increase test delays

* increase test delays
2022-08-19 15:23:57 +01:00
Evgeny Poberezkin 68138c08d2 support multiple server hostnames per queue (#494)
* store SMP client version per queue

* use versioned format for queue addresses (not compatible with v1)

* multiple server hosts in queue URI/data, versioned encoding (cross-version tests fail)

* choose server host based on socksProxy setting

* test works

* multiple server addresses mostly work

* add onion hosts for provided servers

* update SMP hosts
2022-08-12 11:32:04 +01:00
JRoberts 7a72488275 ntf: use separate key to encrypt NMsgMeta (#410)
* ntf: use separate key to encrypt NMsgMeta

* key negotiation

* save key on server, use for encryption

* refactor?

* store error

* NtfQueueCreds

* server - NtfCreds

* comment, rename

* fix type

* ClientNtfCreds

* encoding
2022-06-20 17:16:26 +04:00
Evgeny Poberezkin d1db7d6f79 withStore runs in a single transaction (#408)
* remove store typeclass

* make withStore run single transaction

* combine 2 store calls

* refactor

* add exports

* fix catch

* correction
2022-06-18 18:01:04 +01:00
JRobertsandEvgeny Poberezkin ec70429106 ntf: getNotificationMessage' (#406)
* ntf: getNotificationMessage'

* refactor

* ntfTs

* fix parser

* length

* fix test

* fix encryption, test

* remove test focus

* getActiveNtfToken

* rename

* Update src/Simplex/Messaging/Protocol.hs

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>

* getNtfRcvQueue

* refactor

* don't fail

* Update src/Simplex/Messaging/Notifications/Protocol.hs

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>

* Update src/Simplex/Messaging/Notifications/Server.hs

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>

* Update src/Simplex/Messaging/Server.hs

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>

* import

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2022-06-18 18:21:35 +04:00
JRobertsandEvgeny Poberezkin 3336f987c3 ntf: SCHK, cancel actions on errors, wait until action ts before executing (#402)
* SCHK

* NSACheck

* updateSubFutureTs

* dont execute too early

* fix diffTimeToMillis

* rename function

* errors

* nested status

* increase ntfSubCheckInterval

* adjust import

* Update src/Simplex/Messaging/Agent/NtfSubSupervisor.hs

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>

* Update src/Simplex/Messaging/Agent/NtfSubSupervisor.hs

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>

* refactor rescheduleAction

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2022-06-16 16:11:46 +04:00
JRoberts f7bd8a3982 ntf: agent ntf workers implementation, store methods (#394) 2022-06-14 14:34:18 +04:00
Evgeny Poberezkin 62485b9367 re-deliver duplicate message if it was not acknowledged by the user, ignore otherwise (#397)
* re-deliver duplicate message if it was not acknowledged by the user, ignore otherwise

* delete message after ACK
2022-06-13 08:27:08 +01:00
JRobertsandEvgeny Poberezkin 9c12b4edc0 ntf: agent spike, NtfSupervisor (#392)
Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2022-06-10 16:35:06 +04:00
Evgeny Poberezkin c1348aa54f SMP connection handshake v2 (#390)
* SMP connection handshake v2

* hadshake v2 [mostly] works

* all tests pass, some race conditions remain

* fix build

* fix race conditions, send CON after all HELLOs in duplexHandshake mode

* add comments

* comment

* add comments
2022-06-09 13:47:07 +01:00
Evgeny Poberezkin 60294521f4 add msgId to ACK to avoid the risks of losing messages with concurrent delivery (in app/NSE) (#387)
* add msgId to ACK to avoid the risks of losing messages with concurrent delivery (in app/NSE)

* update ACK to only remove message and update stats if msgId matches

* add tests, fix

* rename sameMsgId/msgDeleted
2022-06-07 10:18:40 +01:00
Evgeny Poberezkin 4b3d04bd27 support message flags visible to SMP server to control notifications (and for any future extensions) (#386)
* support stopping and resuming agent  (#385)

* export agentDbPath

* support fully closing and resuming agent

* whitespace

* clean up

* support message flags visible to SMP server to control notifications (and for any future extensions)

* simplify message flags encoding

* GET command
2022-06-06 12:59:45 +01:00
Evgeny Poberezkin a2204c9b84 remove comments, add EOL 2022-04-21 17:15:41 +01:00
Evgeny Poberezkin 9d8a9c4fe4 interval notifications (TCRN command) (#352)
* notifications: periodic notifications

* agent: allow repeat token registrations, delete old tokens from notification server (e.g., when database is moved to another device)

* decrypt token verification code in the agent

* check token status, send TCRN on registration if it was enabled

* fix http2/apns response handling for error responses (also, APNS seems not to send content-length header?)
2022-04-17 10:34:51 +01:00
Evgeny Poberezkin 45ddecc4b8 APNS/HTTP2 push client for notifications server (#350)
* APN push client (WIP)

* APNS push client works

* TODO comments

* comment

* send notification and process the response

* config

* e2e encryption for notification parameter

* read apns key filename and ID from env

* connect APNS client to server, fix notification JSON encoding to use dash case

* do not connect to APNS on start to pass CI tests

* fix CI test

* remove redundant import
2022-04-14 11:33:17 +01:00
Evgeny Poberezkin f577fcdacf agent schema/methods/types/store methods for notifications tokens (#348)
* agent schema/methods/types/store methods for notifications tokens

* register notification token on the server

* agent commands for notification tokens

* refactor initial servers from AgentConfig

* agent store functions for notification tokens

* server STM store methods for tokens

* fix protocol client for ntfs (use generic handshake), minimal server and agent tests

* server command to verify ntf token
2022-04-08 08:47:04 +01:00