Commit Graph

746 Commits

Author SHA1 Message Date
Evgeny @ SimpleX Chat
c8f2edc242 spec for agent protocol 2026-03-13 07:35:15 +00:00
Evgeny Poberezkin
1d3057985c Merge branch 'master' into rcv-services 2026-03-04 07:33:20 +00:00
Evgeny
c6e3a4d80f add missing exports (#1722)
* add missing exports

* fix dependency
2026-03-04 07:31:46 +00:00
Evgeny Poberezkin
aebc01ba58 Merge branch 'master' into rcv-services 2026-03-03 21:16:46 +00:00
Evgeny
f3408d9bb6 explicit exports (#1719)
* explicit exports

* more empty exports

* add exports

* reorder

* use correct ControlProtocol type for xftp router

---------

Co-authored-by: Evgeny @ SimpleX Chat <259188159+evgeny-simplex@users.noreply.github.com>
2026-03-02 17:34:01 +00:00
Evgeny Poberezkin
d7b90b8415 Merge branch 'stable' 2026-01-28 22:52:33 +00:00
Evgeny
3c5ec8d9a1 agent: improve error handling (#1707)
* agent: improve error handling

* simplify

* report critical error when subscriber crashes

* fix test
2026-01-28 21:54:41 +00:00
Evgeny
d10e05b796 agent: split creating connection to two steps to prepare connection link in advance (#1704)
* agent: split creating connection to two steps to prepare connection link in advance

* linkEntityId, newOwnerAuth

* simplify
2026-01-27 10:54:13 +00:00
Evgeny
66cc06738e agent: fix tests for short links (#1703) 2026-01-26 00:11:50 +00:00
Evgeny Poberezkin
ac825b0ff3 Merge branch 'master' into rcv-services 2026-01-24 14:10:51 +00:00
spaced4ndy
89b81d151f agent: getConnShortLinkAsync; implement joinConnectionAsync for contact connections; narrow setConnShortLinkAsync only to Contact connections (#1694)
* agent: getConnShortLinkAsync

* enable all tests

* comment

* comment

* join conn async for contact URI (wip)

* fix test

* remove enableNtfs param

* FixedLinkData

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2026-01-20 08:40:35 +00:00
Evgeny
3e5b654109 agent: support multiple link owners in link data (#1701)
* agent: support multiple link owners in link data

* fix
2026-01-19 22:08:11 +00:00
Evgeny Poberezkin
502d923817 agent: minor fixes 2026-01-17 10:21:25 +00:00
Evgeny Poberezkin
d908404143 Merge branch 'master' into rcv-services 2026-01-15 18:31:09 +00:00
spaced4ndy
ca26c69937 db: withSavepoint; agent: correctly handle errors in createWithRandomId for postgres (#1693)
* db: withSavepoint; agent: correctly handle errors in createWithRandomId

* comment

* refactor

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2026-01-15 14:59:45 +00:00
Evgeny
1000107259 agent: clean up old records in batches (#1692)
* agent: clean up old records in batches

* update

* update

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* stabilize tests

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2026-01-13 19:11:46 +00:00
Evgeny
6aadcf1f3f agent: lock rows for concurrent queries in PostgreSQL (#1688)
* agent: lock rows for concurrent queries in PostgreSQL

* fix race conditions in workers

* refactor
2026-01-08 11:09:58 +00:00
spaced4ndy
07604a146f agent: async command to set connection short link (setConnShortLinkAsync) (#1680) 2026-01-07 15:12:52 +00:00
Evgeny
c4b687ba64 agent: reset schema after changes (#1690) 2026-01-05 20:37:50 +00:00
Evgeny
a7b43b1a3e agent: use strict tables (#1686)
* agent: use strict tables

* migrate existing tables to strict

* test: verify that all tables are strict

* fix column types for device_token and ntf_mode

* fix encodings and column types for ntf_sub_action and ntf_sub_smp_action

* update schema

* remove debug.trace

* log
2026-01-03 17:19:18 +00:00
Evgeny
11ae20ea20 ntf server: use different client certs for each SMP server, remove support for store log (#1681)
* ntf server: remove support for store log

* ntf server: use different client certificates for each SMP server
2025-12-22 07:56:53 +00:00
Evgeny
a1277bf6bf agent: remove service queue association when service ID changed, process ENDS event, test migrating to/from service (#1677)
* agent: remove service queue association when service ID changed

* agent: process ENDS event

* agent: send service subscription error event

* agent: test migrating to/from service subscriptions, fixes

* agent: always remove service when disabled, fix service subscriptions
2025-12-19 21:10:12 +00:00
Evgeny Poberezkin
c8a72431c0 Merge branch 'master' into rcv-services 2025-12-15 11:57:46 +00:00
Evgeny
ea70575275 agent: use server cert hash when selecting server queues to subscribe (#1679) 2025-12-15 11:48:17 +00:00
Evgeny
2ea9a9a143 agent: finalize initial service subscriptions, remove associations on service ID changes (#1672)
* agent: remove service/queue associations when service ID changes

* agent: check that service ID in NEW response matches session ID in transport session

* agent subscription WIP

* test

* comment

* enable tests

* update queries

* agent: option to add SQLite aggregates to DB connection  (#1673)

* agent: add build_relations_vector function to sqlite

* update aggregate

* use static aggregate

* remove relations

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>

* add test, treat BAD_SERVICE as temp error, only remove queue associations on service errors

* add packZipWith for backward compatibility with GHC 8.10.7

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2025-12-05 20:46:48 +00:00
spaced4ndy
92a9579e69 agent: option to add SQLite aggregates to DB connection (#1673)
* agent: add build_relations_vector function to sqlite

* update aggregate

* use static aggregate

* remove relations

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2025-12-04 08:58:14 +00:00
Evgeny Poberezkin
ff7bdbcbeb Merge branch 'master' into rcv-services 2025-12-03 17:23:59 +00:00
Evgeny
cf9b7e5b6a agent: option to add SQLite functions to DB connection (#1674)
* agent: option to add SQLite functions to DB connection

* add module
2025-12-03 16:10:35 +00:00
Evgeny
38e899957f agent: service subscription events (#1671)
* agent: use server keyhash when loading service record

* agent: process queue/service associations with delayed subscription results

* agent: service subscription events
2025-11-27 21:37:19 +00:00
Evgeny
5e9b164f4e agent: fail when per-connection transport isolation is used with services (#1670) 2025-11-25 23:17:47 +00:00
Evgeny
3ccf854865 servers: maintain xor-hash of all associated queue IDs in PostgreSQL (#1668)
* servers: maintain xor-hash of all associated queue IDs in PostgreSQL (#1615)

* ntf server: maintain xor-hash of all associated queue IDs via PostgreSQL triggers

* smp server: xor hash with triggers

* fix sql and using pgcrypto extension in tests

* track counts and hashes in smp/ntf servers via triggers, smp server stats for service subscription, update SMP protocol to pass expected count and hash in SSUB/NSSUB commands

* agent migrations with functions/triggers

* remove agent triggers

* try tracking service subs in the agent (WIP, does not compile)

* Revert "try tracking service subs in the agent (WIP, does not compile)"

This reverts commit 59e908100d.

* comment

* agent database triggers

* service subscriptions in the client

* test / fix client services

* update schema

* fix postgres migration

* update schema

* move schema test to the end

* use static function with SQLite to avoid dynamic wrapper
2025-11-25 16:55:59 +00:00
Evgeny
1ca4677b28 smp server: messaging services (#1565)
* smp server: refactor message delivery to always respond SOK to subscriptions

* refactor ntf subscribe

* cancel subscription thread and reduce service subscription count when queue is deleted

* subscribe rcv service, deliver sent messages to subscribed service

* subscribe rcv service to messages (TODO delivery on subscription)

* WIP

* efficient initial delivery of messages to subscribed service

* test: delivery to client with service certificate

* test: upgrade/downgrade to/from service subscriptions

* remove service association from agent API, add per-user flag to use the service

* agent client (WIP)

* service certificates in the client

* rfc about drift detection, and SALL to mark end of message delivery

* fix test

* fix test

* add function for postgresql message storage

* update migration
2025-11-07 21:36:28 +00:00
spaced4ndy
3016b929b4 agent: make createConnection and setConnShortLink apis support setting all link data fields (#1665)
* agent: make createConnection and setConnShortLink apis support setting all link data fields

* add functions

* refactor

* refactor

* fix tests

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2025-11-05 07:08:33 +00:00
Evgeny
1329fc726f smp: support client notices (#1659)
* agent: support client notices

* improve

* fix, test

* rename

* cleanup

* send and process notices in more cases

* dont delete

* dont remove notice on other permanent errors

* dont remove notice if there is no notice ID in queue

* add server to error

* allow deleting

* only use notice if key hash matches
2025-10-17 18:34:59 +01:00
spaced4ndy
234aeb81dd agent: add subscription status to connection stats (#1658)
* agent: add subscription status to connection stats

* wip

* conn status

* format

* refactor

* refactor

* m

* shorter

* shorter

---------

Co-authored-by: Evgeny <evgeny@poberezkin.com>
2025-10-13 17:53:28 +01:00
spaced4ndy
1dbc15b2e6 agent: sync connections (#1654)
* agent: sync subscriptions

* remove comment

* add shouldDelete flag

* compare api

* remove instance

* query

* rename

* refactor

* functor

* JSON instances

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2025-10-10 05:41:42 +00:00
Evgeny
80aa56cbcc agent: track which queues need subscribing for iOS NSE (#1657)
* agent: track which queues need subscribing for iOS NSE

* fix down migration

* fix, cleanup
2025-10-09 18:33:27 +01:00
Evgeny
9cda20381f agent: subscribe all connections (#1655)
* agent: subscribe all connections

* query, version

* BoolInt

* add query to errors

* Revert "add query to errors"

This reverts commit 32a1f7fe11.

* fix optional field

* version

* limit number of in-flight subscriptions to 35000
2025-10-09 13:43:48 +01:00
Evgeny
11a4859f8e agent: batch processing of subscription results and errors (#1652)
* agent: batch processing of subscription results and errors

* run agent tests with in-memory server storage

* version

* non empty errors

* size
2025-10-07 14:49:38 +01:00
Evgeny
a3d1a72eb0 agent: optimize subscriptions memory usage (#1651)
* agent: optimize subscriptions memory usage more (do not store subscribed queues in memory) WIP

* use new session subscriptions data

* version

* remove old data structure

* remove version

* batch deletions

* test TSessionSubs

* comment
2025-10-06 15:21:17 +01:00
Evgeny
779222d1a7 agent: optimize subscriptions memory usage (#1649)
* agent: optimize subscriptions memory usage

* version

* remove typeclass

* disable subscription to test memory usage

* disable delivery and commands

* disable reading connections

* enable subscriptions

* RcvQueueCred

* RcvQueueCred postgres

* fix

* optimize keys

* version

* rename

* comments
2025-10-05 13:35:54 +01:00
spaced4ndy
c8b551dcf7 agent: fix file reception error in postgres (#1647) 2025-09-30 08:32:45 +01:00
Evgeny
99f40ae109 support for additional database migrations (#1644) 2025-09-27 11:21:38 +01:00
Evgeny
7b7616ce7e agent: batch loading of connections with PostgreSQL client (#1639)
* agent: batch loading of connections with PostgreSQL client

* batch more

* optimize getPendingCommandServers

* fix Bool conversion

* enable all tests

* cleanup
2025-09-16 14:28:06 +01:00
Evgeny Poberezkin
112cd9d5f4 Merge branch 'stable' 2025-09-14 14:42:05 +01:00
Evgeny
8a7991a376 agent: option to set SQLite database backup path (#1636)
* agent: option to set SQLite database backup path

* fix test compilation
2025-09-14 12:41:49 +01:00
Evgeny Poberezkin
80f7be6dd7 Merge branch 'master' into db-messages 2025-09-13 09:31:23 +01:00
Evgeny
83721240a4 reset db connection on errors (#1633) 2025-09-13 09:30:01 +01:00
Evgeny
bac6ea6e91 smp server: store messages in PostgreSQL (#1622)
* smp server: store messages in PostgreSQL

* stored procedures to write and to expire messages

* function to export messages

* move all message functions to PostgreSQL, remove delete trigger

* comments

* import messages to db

* fix message import, add export

* fix export

* fix export

* fix compilation flags

* import messages line by line

* fix server start with database storage

* fix compilation

* comments
2025-09-11 20:22:55 +01:00
spaced4ndy
a4f049d8da agent: parameterize withWork, getWorkItem with StoreError; parameterized Binary for SQLite (#1617)
* agent: parameterize withWork StoreError

* getWorkItem

* export

* binary

* remove handleWrkErr AnyStoreError constraint

* put AnyError in AnyStoreError constraint

* move typeclass

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2025-09-08 16:38:08 +01:00