Commit Graph

1973 Commits

Author SHA1 Message Date
Evgeny Poberezkin c8a72431c0 Merge branch 'master' into rcv-services 2025-12-15 11:57:46 +00:00
Evgeny ea70575275 agent: use server cert hash when selecting server queues to subscribe (#1679) 2025-12-15 11:48:17 +00:00
Evgeny Poberezkin 568500c6bc Merge branch 'master' into rcv-services 2025-12-14 23:14:18 +00:00
Evgeny 49e9ce1649 agent: skip subscribing servers without queues (#1678) 2025-12-14 23:01:21 +00:00
Evgeny f5eb735551 servers: service stats and logging, allow services without option (removed), report errors during service message delivery, remove threads when service subscription ended (#1676)
* smp server: always allow services without option

* smp server: maintain IDs hash in session subscription states

* smp server: service message delivery error handling

* ntf server: log subscription count and hash differences

* smp server: remove delivery threads when service subscription ended/client disconnected
2025-12-14 12:07:29 +00:00
Evgeny Poberezkin 83894074b0 Merge branch 'master' into rcv-services 2025-12-13 19:08:47 +00:00
Evgeny Poberezkin 2ca440dd2d 6.5.0.4 2025-12-10 21:09:33 +00:00
Evgeny Poberezkin 35fe5ac9e4 Merge branch 'master' into rcv-services 2025-12-05 21:30:28 +00:00
Evgeny 2ea9a9a143 agent: finalize initial service subscriptions, remove associations on service ID changes (#1672)
* agent: remove service/queue associations when service ID changes

* agent: check that service ID in NEW response matches session ID in transport session

* agent subscription WIP

* test

* comment

* enable tests

* update queries

* agent: option to add SQLite aggregates to DB connection  (#1673)

* agent: add build_relations_vector function to sqlite

* update aggregate

* use static aggregate

* remove relations

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>

* add test, treat BAD_SERVICE as temp error, only remove queue associations on service errors

* add packZipWith for backward compatibility with GHC 8.10.7

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2025-12-05 20:46:48 +00:00
spaced4ndy 92a9579e69 agent: option to add SQLite aggregates to DB connection (#1673)
* agent: add build_relations_vector function to sqlite

* update aggregate

* use static aggregate

* remove relations

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2025-12-04 08:58:14 +00:00
Evgeny Poberezkin ff7bdbcbeb Merge branch 'master' into rcv-services 2025-12-03 17:23:59 +00:00
Evgeny cf9b7e5b6a agent: option to add SQLite functions to DB connection (#1674)
* agent: option to add SQLite functions to DB connection

* add module
2025-12-03 16:10:35 +00:00
Evgeny 38e899957f agent: service subscription events (#1671)
* agent: use server keyhash when loading service record

* agent: process queue/service associations with delayed subscription results

* agent: service subscription events
2025-11-27 21:37:19 +00:00
Evgeny 5e9b164f4e agent: fail when per-connection transport isolation is used with services (#1670) 2025-11-25 23:17:47 +00:00
Evgeny 3ccf854865 servers: maintain xor-hash of all associated queue IDs in PostgreSQL (#1668)
* servers: maintain xor-hash of all associated queue IDs in PostgreSQL (#1615)

* ntf server: maintain xor-hash of all associated queue IDs via PostgreSQL triggers

* smp server: xor hash with triggers

* fix sql and using pgcrypto extension in tests

* track counts and hashes in smp/ntf servers via triggers, smp server stats for service subscription, update SMP protocol to pass expected count and hash in SSUB/NSSUB commands

* agent migrations with functions/triggers

* remove agent triggers

* try tracking service subs in the agent (WIP, does not compile)

* Revert "try tracking service subs in the agent (WIP, does not compile)"

This reverts commit 59e908100d.

* comment

* agent database triggers

* service subscriptions in the client

* test / fix client services

* update schema

* fix postgres migration

* update schema

* move schema test to the end

* use static function with SQLite to avoid dynamic wrapper
2025-11-25 16:55:59 +00:00
Evgeny 1ca4677b28 smp server: messaging services (#1565)
* smp server: refactor message delivery to always respond SOK to subscriptions

* refactor ntf subscribe

* cancel subscription thread and reduce service subscription count when queue is deleted

* subscribe rcv service, deliver sent messages to subscribed service

* subscribe rcv service to messages (TODO delivery on subscription)

* WIP

* efficient initial delivery of messages to subscribed service

* test: delivery to client with service certificate

* test: upgrade/downgrade to/from service subscriptions

* remove service association from agent API, add per-user flag to use the service

* agent client (WIP)

* service certificates in the client

* rfc about drift detection, and SALL to mark end of message delivery

* fix test

* fix test

* add function for postgresql message storage

* update migration
2025-11-07 21:36:28 +00:00
spaced4ndy 3016b929b4 agent: make createConnection and setConnShortLink apis support setting all link data fields (#1665)
* agent: make createConnection and setConnShortLink apis support setting all link data fields

* add functions

* refactor

* refactor

* fix tests

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2025-11-05 07:08:33 +00:00
Evgeny Poberezkin 1ae3e8d0be 6.5.0.3 v6.5.0-beta.3 2025-10-23 08:34:11 +01:00
Evgeny 1329fc726f smp: support client notices (#1659)
* agent: support client notices

* improve

* fix, test

* rename

* cleanup

* send and process notices in more cases

* dont delete

* dont remove notice on other permanent errors

* dont remove notice if there is no notice ID in queue

* add server to error

* allow deleting

* only use notice if key hash matches
2025-10-17 18:34:59 +01:00
spaced4ndy 234aeb81dd agent: add subscription status to connection stats (#1658)
* agent: add subscription status to connection stats

* wip

* conn status

* format

* refactor

* refactor

* m

* shorter

* shorter

---------

Co-authored-by: Evgeny <evgeny@poberezkin.com>
2025-10-13 17:53:28 +01:00
spaced4ndy 1dbc15b2e6 agent: sync connections (#1654)
* agent: sync subscriptions

* remove comment

* add shouldDelete flag

* compare api

* remove instance

* query

* rename

* refactor

* functor

* JSON instances

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2025-10-10 05:41:42 +00:00
Evgeny 80aa56cbcc agent: track which queues need subscribing for iOS NSE (#1657)
* agent: track which queues need subscribing for iOS NSE

* fix down migration

* fix, cleanup
2025-10-09 18:33:27 +01:00
Evgeny 9cda20381f agent: subscribe all connections (#1655)
* agent: subscribe all connections

* query, version

* BoolInt

* add query to errors

* Revert "add query to errors"

This reverts commit 32a1f7fe11.

* fix optional field

* version

* limit number of in-flight subscriptions to 35000
2025-10-09 13:43:48 +01:00
Evgeny 318ddf692a smp server: import/export to/from PostgreSQL with one command, deprecation notice for journal storage (#1653) 2025-10-09 13:31:57 +01:00
Evgeny 11a4859f8e agent: batch processing of subscription results and errors (#1652)
* agent: batch processing of subscription results and errors

* run agent tests with in-memory server storage

* version

* non empty errors

* size
2025-10-07 14:49:38 +01:00
Evgeny a3d1a72eb0 agent: optimize subscriptions memory usage (#1651)
* agent: optimize subscriptions memory usage more (do not store subscribed queues in memory) WIP

* use new session subscriptions data

* version

* remove old data structure

* remove version

* batch deletions

* test TSessionSubs

* comment
2025-10-06 15:21:17 +01:00
Evgeny Poberezkin 17b71cf673 Merge branch 'stable' 2025-10-05 22:03:58 +01:00
Evgeny 779222d1a7 agent: optimize subscriptions memory usage (#1649)
* agent: optimize subscriptions memory usage

* version

* remove typeclass

* disable subscription to test memory usage

* disable delivery and commands

* disable reading connections

* enable subscriptions

* RcvQueueCred

* RcvQueueCred postgres

* fix

* optimize keys

* version

* rename

* comments
2025-10-05 13:35:54 +01:00
Evgeny Poberezkin 538dcb6a4c 6.4.6.0 2025-10-04 14:49:42 +01:00
Evgeny Poberezkin 98115b6db1 6.5.0.2 v6.5.0-beta.2 2025-09-30 12:37:28 +01:00
Evgeny 9854caca33 agent: optimize subscriptions (#1645)
* agent: optimize subscriptions

* simplify

* clean up
2025-09-30 12:34:42 +01:00
spaced4ndy c8b551dcf7 agent: fix file reception error in postgres (#1647) 2025-09-30 08:32:45 +01:00
Evgeny 99f40ae109 support for additional database migrations (#1644) 2025-09-27 11:21:38 +01:00
sh 3a9381b966 ci/scripts: execute git version if chmod fails (#1643) 2025-09-23 09:08:56 +01:00
Evgeny Poberezkin 54a2a6c905 6.5.0.1 v6.5.0-beta.1 2025-09-17 09:04:03 +01:00
Evgeny 7b7616ce7e agent: batch loading of connections with PostgreSQL client (#1639)
* agent: batch loading of connections with PostgreSQL client

* batch more

* optimize getPendingCommandServers

* fix Bool conversion

* enable all tests

* cleanup
2025-09-16 14:28:06 +01:00
sh b020a08ea0 ci: disable aarch64 tests (#1638) 2025-09-15 09:02:34 +01:00
Evgeny Poberezkin 2737e7d602 6.5.0.0.1 2025-09-15 07:52:17 +01:00
Evgeny d3754b36bc smp server: do not use queue cache with PostgreSQL message storage (#1637)
* smp server: do not use queue cache with PostgreSQL message storage

* fix loading queues via notifier IDs
2025-09-15 07:51:27 +01:00
Evgeny Poberezkin 112cd9d5f4 Merge branch 'stable' 2025-09-14 14:42:05 +01:00
Evgeny 8a7991a376 agent: option to set SQLite database backup path (#1636)
* agent: option to set SQLite database backup path

* fix test compilation
2025-09-14 12:41:49 +01:00
Evgeny 6f4d0f5377 Merge pull request #1630 from simplex-chat/db-messages
Feature branch: PostgreSQL message store
2025-09-13 19:24:59 +01:00
Evgeny 9cfdae3494 smp server: uniterruptible mask for DB operations (#1635) 2025-09-13 19:08:07 +01:00
Evgeny Poberezkin 62183699db 6.5.0.0 2025-09-13 11:37:59 +01:00
Evgeny 9be9a76b42 smp server: improve message expiration (#1634)
* smp server: limit by time the queues to export journal messages for

* pass queue/msg thresholds separately

* reset db connection on errors

* Revert "smp server: limit by time the queues to export journal messages for"

This reverts commit d3bc0cba4b.

* fix test compilation

* flag to expire messages

* improve test

* expire messages newer than quota
2025-09-13 11:35:34 +01:00
Evgeny Poberezkin 80f7be6dd7 Merge branch 'master' into db-messages 2025-09-13 09:31:23 +01:00
Evgeny 83721240a4 reset db connection on errors (#1633) 2025-09-13 09:30:01 +01:00
Evgeny 6c66cf367a smp server: set message counts correctly after import (#1632) 2025-09-12 13:08:11 +01:00
Evgeny a137d01c90 smp server: improve counts management and expiration for postgres message store (#1631)
* smp server: improve counts management and expiration for postgres message store

* one update

* fix expiration
2025-09-12 11:39:35 +01:00
Evgeny bac6ea6e91 smp server: store messages in PostgreSQL (#1622)
* smp server: store messages in PostgreSQL

* stored procedures to write and to expire messages

* function to export messages

* move all message functions to PostgreSQL, remove delete trigger

* comments

* import messages to db

* fix message import, add export

* fix export

* fix export

* fix compilation flags

* import messages line by line

* fix server start with database storage

* fix compilation

* comments
2025-09-11 20:22:55 +01:00