Commit Graph

73 Commits

Author SHA1 Message Date
Evgeny Poberezkin d908404143 Merge branch 'master' into rcv-services 2026-01-15 18:31:09 +00:00
Evgeny a7b43b1a3e agent: use strict tables (#1686)
* agent: use strict tables

* migrate existing tables to strict

* test: verify that all tables are strict

* fix column types for device_token and ntf_mode

* fix encodings and column types for ntf_sub_action and ntf_sub_smp_action

* update schema

* remove debug.trace

* log
2026-01-03 17:19:18 +00:00
Evgeny 3ccf854865 servers: maintain xor-hash of all associated queue IDs in PostgreSQL (#1668)
* servers: maintain xor-hash of all associated queue IDs in PostgreSQL (#1615)

* ntf server: maintain xor-hash of all associated queue IDs via PostgreSQL triggers

* smp server: xor hash with triggers

* fix sql and using pgcrypto extension in tests

* track counts and hashes in smp/ntf servers via triggers, smp server stats for service subscription, update SMP protocol to pass expected count and hash in SSUB/NSSUB commands

* agent migrations with functions/triggers

* remove agent triggers

* try tracking service subs in the agent (WIP, does not compile)

* Revert "try tracking service subs in the agent (WIP, does not compile)"

This reverts commit 59e908100d.

* comment

* agent database triggers

* service subscriptions in the client

* test / fix client services

* update schema

* fix postgres migration

* update schema

* move schema test to the end

* use static function with SQLite to avoid dynamic wrapper
2025-11-25 16:55:59 +00:00
Evgeny 11a4859f8e agent: batch processing of subscription results and errors (#1652)
* agent: batch processing of subscription results and errors

* run agent tests with in-memory server storage

* version

* non empty errors

* size
2025-10-07 14:49:38 +01:00
Evgeny beafac1f73 agent: make agent workers usable from other contexts (#1614) 2025-08-29 08:33:55 +01:00
Evgeny 2a90a2c552 smp protocol: create notification credentials via NEW command that creates the queue (#1586)
* smp protocol: create notification credentials via NEW command that creates the queue

* create ntf subscription for queues created with ntf credetials

* do not create ntf credentials when switching connection to another queue
2025-07-16 16:46:45 +01:00
Evgeny 36f05e272e agent: support different timeouts for interactive and background requests (#1582)
* agent: support different timeouts for interactive and background requests

* fix tests

* use one constructor for the first request and for retries
2025-07-07 09:38:52 +01:00
spaced4ndy 64149599de postgres: db interfaces wip (sqlite passes) (#1419) 2024-12-12 17:42:58 +04:00
spaced4ndy 243548631a ntf: repeat failed token deletions (#1346) 2024-10-01 16:30:14 +04:00
spaced4ndy 9cf6c97137 ntf: batch ntf worker command processing (#1337)
* ntf: batch ntf worker command processing

* remove comment

* change batch size

* wip

* catch

* refactor

* refactor

* batch check

* refactor

* reschedule

* increase ntfSubCheckInterval

* first check interval

* check more statuses

* refactor, remove foldr'

* refactor 2

* refactor client

* refactor 3

* ntf server: improve support for batched commands (#1340)

* re-create ntf subscriptions on NTF AUTH errors

* name

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2024-09-29 18:50:38 +01:00
spaced4ndy 41e1b2bd39 ntf: use common function for differentiating temporary errors (#1333) 2024-09-25 18:40:59 +01:00
spaced4ndy 81fcdf8ac9 ntf: batch smp worker command processing (#1331)
* ntf: refactor smp worker (separate command processing into steps)

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2024-09-25 15:00:06 +04:00
spaced4ndy bef11e4cbe agent: ntf errs (#1316)
* agent: ntf errs

* refactor

* errs

* refactor

* simplify

* inline

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2024-09-20 20:08:42 +04:00
spaced4ndy ea67b34ba7 ntf: batch supervisor commands (#1313) 2024-09-13 15:26:57 +04:00
spaced4ndy e247f691cb agent: do not reuse notifier credentials when re-creating notification subscription; on deleting ntf sub delete directly on smp server skipping ntf server (#1311)
* agent: change notifier credentials when re-creating notification subscription

* skip ndel

* enable tests

* fix race

* Revert "fix race"

This reverts commit ed9b18e8a7.

* delete record

* rename

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2024-09-12 14:02:51 +04:00
Evgeny 7762b4d364 reduce the number of STM transactions (#1255)
* reduce the number of STM transactions

* reduce STM more

* refactor

* remove more
2024-08-09 16:38:46 +01:00
Evgeny f4d1a33ed8 agent: retry loop that resumes subscriptions as soon as agent is moved to foreground, suspend retry loops while agent is suspended (#1249)
* agent: retry loop that resumes subscriptions as soon as agent is moved to foreground, suspend retry loops while agent is suspended

* reset retry enterval when moving to foreground

* account for network state too

* simplify

* typo

* simplify
2024-08-05 15:09:46 +01:00
Evgeny Poberezkin 051bf38bc7 agent: remove delays in notification processing, add notification stats (#1235)
* agent: remove delays in notification processing, add notification stats

* do not replace token after failed check

* more stats

* refactor

* fix

* backwards compatible JSON parsing

* retry deleting subscription on temporary error

* remove attempts to get multiple notification messages

* fix JSON decoding to be backwards compatible
2024-07-22 15:42:34 +01:00
Evgeny Poberezkin 3d605310ed agent: remove protocol encodings and agent TCP server (#1189)
* rfc: remove agent protocol encodings

* agent: remove protocol encodings and agent TCP server

* update

* remove unused code

* remove

* move tests

* add delay to tests

* stabilize test

* test

* more delays

* reduce delays

* enable all tests

* delays

* style
2024-06-05 14:34:40 +01:00
Alexander Bondarenko 984394d906 core: remove MonadUnliftIO ExceptT orphans (#1169) 2024-05-23 16:44:00 +01:00
Evgeny Poberezkin 33f6d2f1da agent: optimize waiting for user network to avoid contention for state updates from multiple threads (#1155)
* haskell magic

* update

* agent: optimize waiting for user network with TChan

* clean up

* Int64

* use TVar

* cleanup

* fix

* testing

* update
2024-05-17 15:12:05 +01:00
Evgeny Poberezkin 8fa9ed6317 wait for user network availability (#1085)
* ghc-options

* wait for user network availability

* test

* update

* comment

* refactor

* slow config

* line

* waitForUserNetwork in xftp and ntf workers

* refactor

* refactor with registerDelay

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2024-04-10 21:50:05 +01:00
Evgeny Poberezkin 6ded721daa remove monad typeclasses to reduce overhead (#1077)
* remove monad typeclasses to reduce overhead

* remove unliftIO

* StrictData

* inline

* optional agent port

* avoid MonadUnliftIO instance (#1078)

* avoid MonadUnliftIO instance

* simpler liftError'

* rename

* narrow down instance

* revert

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>

* logServer

---------

Co-authored-by: Alexander Bondarenko <486682+dpwiz@users.noreply.github.com>
2024-03-31 20:50:35 +01:00
Evgeny Poberezkin 416f1b1721 smp: command authorization (#982)
* smp: command authorization

* fix encoding, most tests

* remove old tests

* authorize via crypto_box

* extract authenticator to Crypto module

* make TransmissionAuth Maybe

* rfc

* support authenticators in NTF protocol, test matrix (no backwards compatibility yet from new clients to old servers)

* fix/add tests, add version config to "small" agent

* separate client and server versions for SMP protocol

* test batching SMP v7

* do not send session ID in each transmission

* refactor auth verification in the server, split tests

* server "warm up" fixes timing test

* uncomment SUB timing test

* comments, disable two timing tests

* rename version

* increase auth timing test failure threshold

* use different algorithms to authorize snd/rcv commands, use random correlation ID

* transport: fetch and store server certificate (#985)

* THandleParams (WIP, does not compile)

* transport: fetch and store server certificate

* smp: add getOnlinePubKey example to smpClientHandshake

* add server certs and sign authPub

* cleanup

* update

* style

* load server certs from test fixtures

* sign ntf authPubKey

* fix onServerCertificate

* increase delay before sending messages

* require certificate with key in SMP server handshake

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>

* remove dhSecret from THandle

* remove v8, merge all changes to one version

* parameterize THandle

* rfc: transmission ecnryption

* Revert "parameterize THandle"

This reverts commit 75adfc94fb.

* use batch syntax for ntf server commands

* separate encodeTransmission when there is no key

* typo

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* rename

* diff

---------

Co-authored-by: Alexander Bondarenko <486682+dpwiz@users.noreply.github.com>
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2024-02-16 11:45:54 +00:00
Alexander Bondarenko f7cdec2f08 fix: support multiple notification servers in configuration (#971)
* tests: add trpl-keeps-server check

* add smp server switch check

* add connection test and fix withNtfServer

* Update src/Simplex/Messaging/Agent/NtfSubSupervisor.hs

Co-authored-by: Evgeny Poberezkin <e.poberezkin@me.com>

* use ntfServer from token

* rename

---------

Co-authored-by: Evgeny Poberezkin <e.poberezkin@me.com>
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2024-01-19 19:42:28 +00:00
Evgeny Poberezkin 2488cf1e3a agent: restart worker when it terminated (#948)
* agent: restart worker when it terminated

* comment

* remove writeTMVar

* remove writeTMVar 2

* remove writeTMVar 3
2024-01-05 14:22:26 +00:00
spaced4ndy 22e1932372 xftp, ntf: refactor, reuse workers abstractions (#935)
* xftp: refactor, reuse workers abstractions (wip, test fails)

* rename

* refactor (tests pass)

* refactor2

* linebreaks

* do

* reuse worker abstractions for notifications

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-12-27 13:02:08 +04:00
Evgeny Poberezkin 2489333c87 agent: use DB to queue async commands and messages (#929)
* agent: use DB to queue async commands and messages

* fix message delivery, async commands dont work

* fix async commands

* remove comment

* rename

* comment

* balance agent operation

* empty lines

* balance another agent operation

* diff

* worker

* qAddress

* fix incorrect queue ID in v1

* types for queues

* fix test

* update index
2023-12-23 12:59:53 +00:00
Evgeny Poberezkin fa457d1c25 agent: avoid race condition between worker and supervisor when getting work (#928) 2023-12-22 11:12:36 +00:00
Evgeny Poberezkin 13a60d1d39 use ChaChaDRG as the source of randomness (#920)
* use ChaChaDRG as the source of randomness

* remove functions using entropy directly

* comment
2023-12-21 00:12:08 +00:00
Evgeny Poberezkin 9d12d76078 add fourmolu (#868)
* add fourmolu

* linebreak

* simplify

* comment
2023-10-22 09:20:14 +01:00
Moritz Angermann 002f36dde0 5.4.0.0: use ghc 9.6.2 (#791)
* Add 9.6 compat

* compile with GHC9.6.2: dependencies, imports, code

* refactor typeclasses

* refactor record dot

* update cabal version

* update github actions

* update direct-sqlcipher

* 5.4.0.0

* update cabal.project

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-08-24 20:22:02 +01:00
Evgeny Poberezkin f76a5ca5b6 agent: catch IO errors correctly in MonadError (#795)
* agent: catch IO errors correctly in MonadError

* correction

* correction

* utils

* agentFinally to catch IO exceptions in ExceptT

* rename

* remove, inline

* rename utils

* utils unit test

* test to show catch and finally problems

* tryAllErrors

* enable all tests
2023-07-09 18:04:45 +01:00
spaced4ndy d693868bc0 change implementation of time diff calculation (#754) 2023-05-15 20:26:18 +04:00
spaced4ndy bdff274f56 use Int64 for time calculations (#706) 2023-04-04 14:51:13 +04:00
spaced4ndy 0df7733125 xftp: relative file paths, worker management (#694) 2023-03-22 15:34:25 +04:00
Evgeny Poberezkin 02eb1e9822 add stateTVar imports for compatibility with older hackage index in nix builds 2023-03-18 07:57:25 +00:00
Evgeny Poberezkin e33d5b5184 use different constraint for non-error functions (#690) 2023-03-17 16:01:24 +04:00
spaced4ndy 9255d48b69 xftp: move delay to replica (#688) 2023-03-16 18:12:38 +04:00
Evgeny Poberezkin 8fde8e1344 xftp: agent command entities (#676)
* xftp: agent command entities

* progress event

* parameterize agent command by entity

* refactor

* Eq instance for APartyCmdTag
2023-03-10 09:10:52 +00:00
Evgeny Poberezkin 2f7c115207 retryInterval functions pass current interval to the action, to resume with the same interval on restart (improve battery consumption) (#670) 2023-03-03 14:12:36 +00:00
spaced4ndy c0dcf283eb server_key_hash fields (#643)
* server_key_hash fields

* test

* refactor

* fix

* order

* use sync command in test

* refactor

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-02-17 21:24:32 +00:00
Evgeny Poberezkin 61e0c346df simplify C.sign, Crypto tests, triage TODOs (#592)
* simplify C.sign, Crypto tests, triage TODOs

* update condition

* remove todos

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>
2023-01-06 15:34:05 +00:00
Evgeny Poberezkin e4842f4f47 agent: include server address string into BROKER errors (#575)
* agent: include server address string into BROKER errors

* eol

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>
2022-11-30 08:25:07 +00:00
Evgeny Poberezkin 8d9816809f simplify queue rotation protocol (#550)
* simplify queue rotation protocol

* use simplified rotation protocol, update tests

* simplify schema

* delete all connection queues

* refactor

* switch notifications to the new queue, tests

* remove TODO

* refactor

* rfc correction

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>

* remove duplicate set active

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>
2022-10-31 13:33:28 +04:00
Evgeny Poberezkin eb5c1c78cb connection queue redundancy and rotation (#521)
* rfc: queue rotation

* update rfc

* messages for queue rotation

* allow multiple subscribed queues per connection in Agent/Client.hs

* refactor

* fix module name

* allow multiple queues in duplex connection type

* update commands

* add indices

* addConnectionRcvQueue

* switch connection to another queue (WIP)

* update schema/protocol

* switching queue works, but sending messages after the switch fails

* messages are delivered after rotation

* use connection-scoped queue ID

* rename queue records fields

* refactor using SMPQueue class/instances

* simplify queries

* QKEY: check queue is not secured, refactor

* update rfc

* mark queue as primary in QUSE

* queue rotation errors

* fix async ack

* fix async ACK to send OK

* correction

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>

* use SWCH command

* rename

* take into account only active queue subscription when determining connection result if at least one queue is active

* remove comment

* only enable notifications for connections with enableNtfs = True

* async test (WIP)

* async queue rotation test

* simplify combining results

* test with 2 servers

* fix unused subscribeConnection

* switch to cabal build

* increase build timeout

* increase delay in async test

* skip queue rotation tests

* build matrix

* step name

* use ubuntu-18.04 in build matrix

* enable rotation tests

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>
2022-10-29 18:57:01 +01:00
Evgeny Poberezkin 13a9eee0cf cancel threads asynchronously (#527) 2022-09-17 00:18:58 +01:00
Evgeny Poberezkin 959ae34f16 fix agent not stopping and race conditions (#526)
* fix agent not stopping and race conditions

* simplify
2022-09-16 13:42:13 +01:00
Evgeny Poberezkin 1ce63bee44 create notification subscriptions in batch subscribe (#477)
* create notification subscriptions in batch subscribe

* refactor

* refactor
2022-07-19 09:30:17 +01:00
Evgeny Poberezkin abd694bd5a parameterize protocol server by scheme (#459) 2022-07-05 21:56:14 +01:00