Commit Graph

1866 Commits

Author SHA1 Message Date
Evgeny Poberezkin
d80a6b74c5 version 2025-06-12 11:10:28 +01:00
Evgeny Poberezkin
52efce8e00 Merge branch 'master' into ep/protocol-transmission 2025-06-12 11:09:43 +01:00
Evgeny
1658048c2c Revert "smp server: use separate database pool for reading queues and creating service records (#1561)" (#1564)
This reverts commit 3df2425162.
2025-06-12 11:04:43 +01:00
Evgeny Poberezkin
e93a546ce0 Revert "smp server: use separate database pool for reading queues and creating service records (#1561)"
This reverts commit 3df2425162.
2025-06-11 21:44:01 +01:00
Evgeny Poberezkin
82c31e3085 Merge branch 'master' into ep/protocol-transmission 2025-06-11 21:43:41 +01:00
Evgeny
27d38518e1 agent: use PQ keys in contact request data inside link container (but not in contact request link); use PQ keys in invitations sent to contact addresses (#1563)
* agent: use PQ keys in contact request data inside link container (but not in contact request link); use PQ keys in invitations sent to contact addresses

* do not use PQ keys in the link with old address versions
2025-06-11 18:22:37 +01:00
Evgeny
cf8088ac6a smp server: do not require ALPN for HTTP to fix Android asset links, do not use port 443 in server-server SMP connections (#1562)
* log alpn

* always use HTTP when SNI is sent, regardless of ALPN

* decide credential based on SNI

* do not use web port in SMP/Ntf servers connecting to SMP servers

* simpler

* refactor

* fix
2025-06-10 22:09:07 +01:00
Evgeny Poberezkin
1e82104224 6.4.0.8 v6.4.0-beta.8 2025-06-09 14:20:25 +01:00
Evgeny
46ff37c362 ntf server: additional statistics (#1558)
* ntf server: additional statistics

* version

* fix stats

* add stats to track notifications without active token

* refactor

* fix stats parser

* version
2025-06-09 14:14:27 +01:00
Evgeny
3df2425162 smp server: use separate database pool for reading queues and creating service records (#1561) 2025-06-09 12:47:35 +01:00
Evgeny Poberezkin
9faac515e7 version 2025-06-09 08:11:35 +01:00
Evgeny Poberezkin
6b12d8d040 reset NTF service when adding notifier 2025-06-08 23:17:42 +01:00
Evgeny Poberezkin
d3f229a666 func 2025-06-08 23:15:08 +01:00
Evgeny Poberezkin
3948980000 only use batch logic if there is more than one transmission 2025-06-08 22:30:00 +01:00
Evgeny Poberezkin
f25a5a864b diff 2025-06-08 22:19:05 +01:00
Evgeny
4a405a94bb smp server: batch commands (#1560)
* smp server: batch commands verification into one DB transaction

* ghc 8.10.7

* flatten transmission tuples
2025-06-08 21:14:56 +01:00
Evgeny Poberezkin
6886d78509 clean 2025-06-08 18:46:25 +01:00
Evgeny Poberezkin
97f45f8284 protocol: refactor types and encoding 2025-06-07 08:10:31 +01:00
Evgeny
5241f5fe5e rfc: client certificates for servers using SMP protocol as clients (opertors' chat relays, notification servers, service bots) (#1534)
* rfc: client certificates for high volume clients (opertors' chat relays, notification servers, service bots)

* client certificates types (WIP)

* parameterize Transport

* protocol/schema/api changes

* agent API

* rename command

* agent subscriptions return local ClientServiceId to chat

* verify transmissions

* fix receiving client certificates, refactor

* ntf server: remove shared queue for all notification subscriptions (#1543)

* ntf server: remove shared queue for all notification subscriptions

* wait for subscriber with timeout

* safer

* refactor

* log

* remove unused

* WIP service subscriptions and associations, refactor

* process service subscriptions

* rename

* simplify switching subscriptions

* SMP service handshake with additional server handshake response

* notification delivery and STM persistence for services

* smp server: database storage, store log, fix encoding for STORE error, replace String with Text in locks and error

* stats

* more stats

* rename SMP commands

* service subscriptions in ntf server agent (tests fail)

* fix

* refactor

* exports

* subscribe ntf server as service for associated queues

* test ntf service connection, fix SOKS response, fix service associations not removed in STM storage

* INI option to support services

* ntf server: downgrade subscriptions when service is no longer supported, track counts of subscribed queues

* smp protocol: include service certificate fingerprint in the string signed over with entity key (TODO two tests fail)

* fix test

* ntf server prometheus stats, use Int64 in SOKS/ENDS responses (to avoid conversions), additional error status for ntf subscription

* update RFC

* refactor useServiceAuth to avoid ad hoc decisions about which commands use service signatures, and to prohibit service signatures on other commands

* remove duplicate service signature syntax check from checkCredentials, it is checked in verifyTransmission

* service errors, todos

* fix checkCredentials in ntf server, service errors

* refactor service auth

* refactor

* service agent: store returned queue count instead of expected

* refactor serverThread

* refactor serviceSig

* rename

* refactor, rename, test repeat NSUB service association

* respond with error to SUBS

* smp server: export/import service records between database and store log

* comment

* comments

* ghc 8.10.7
2025-06-06 08:03:47 +01:00
Evgeny
8e86c97a13 servers: include supported ALPNs in server transport config (#1557) 2025-06-03 13:35:35 +01:00
Evgeny
90e8c3adf6 smp agent: use static RNG function to avoid dynamic C stub created by Haskell FFI wrapper (#1556) 2025-06-02 20:11:27 +01:00
Evgeny
56851365b1 servers: parameterize SMP agent to optimize subscriptions (#1555)
* servers: parameterize SMP agent to optimize subscriptions

* refactor ClientCommand to avoid additional map in the notification server agent

* rename
2025-06-01 16:14:52 +01:00
Evgeny
a9814bb6d3 smp server: parameterize server by store type to avoid boxing/unboxing client handles (#1554)
* smp server: parameterize server by store type to avoid boxing/unboxing client handles

* ghc 8.10.7
2025-05-29 22:17:28 +01:00
Evgeny Poberezkin
3ad8bd15a6 6.4.0.7 v6.4.0-beta.7 2025-05-29 13:14:09 +01:00
spaced4ndy
4c33d8ac43 agent: To and FromField instances for AConnShortLink (#1553) 2025-05-29 10:59:36 +01:00
spaced4ndy
a94ca62624 agent: allow changing invitation link data before it is secured (#1552)
* agent: setInvitationShortLink api

* Eq instance

* allow changing link data on server, refactor

* fix

* encodings

* remove link data after connection

* Revert "encodings"

This reverts commit f8e254cca9.

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2025-05-29 10:44:16 +01:00
Evgeny
53b72469b6 refactor types (#1551) 2025-05-27 22:31:02 +01:00
Evgeny Poberezkin
f80ed32a06 6.4.0.6 v6.4.0-beta.6 2025-05-25 17:28:39 +01:00
Evgeny
07eaf9157b smp server: allow getting and deleting short links for the old contact queues (#1549)
* smp server: allow getting and deleting short links for the old contact queues

* fix verifaction of legacy contact queues

* test
2025-05-25 17:03:02 +01:00
Evgeny
56ea2fdd56 refactor types for DB entity (#1548) 2025-05-24 18:19:11 +01:00
Evgeny
ffecd4a17a parameterize transport by peer type (client/server) (#1545)
* parameterize transport by peer type (client/server)

* LogDebug level when test is retried

* support "flipped" HTTP2, fix test retry to avoid retrying pending tests

* move sync to the end of the tests
2025-05-24 14:34:22 +01:00
Evgeny Poberezkin
dae649fb87 6.4.0.5 v6.4.0-beta.5 2025-05-24 14:21:53 +01:00
Evgeny
57a77f75c1 smp server: support adding link data to contact addresses created before July 2024 (#1547) 2025-05-24 14:20:25 +01:00
spaced4ndy
18e73b8aa7 agent: pass CRClientData to setContactShortLink (#1546)
* agent: pass CRClientData to setContactShortLink

* fix

* fix
2025-05-23 18:21:36 +01:00
Evgeny
af9ca59e51 smp server: optimize concurrency and memory usage, refactor (#1544)
* smp server: optimize concurrency and memory usage, refactor

* hide clients IntMap

* reduce STM contention

* comment

* version

* correct stats for subscriptions

* version

* comment

* remove subscribed clients from map

* version

* optimze, refactor

* version

* debug test

* enable all tests

* remove test logs

* retry failed tests with debug logging

* increase test timeout

* sync between tests
2025-05-23 12:52:18 +01:00
Evgeny Poberezkin
d352d518c2 6.4.0.4 v6.4.0-beta.4 2025-05-21 08:56:01 +01:00
Evgeny
f0dc600016 ntf server: remove shared queue for all notification subscriptions (#1543)
* ntf server: remove shared queue for all notification subscriptions

* wait for subscriber with timeout

* safer

* refactor

* log
2025-05-21 08:55:32 +01:00
Evgeny Poberezkin
f44ea0a6d8 6.4.0.3 v6.4.0-beta.3 2025-05-15 21:01:49 +01:00
Evgeny
f7d31d4c02 Merge pull request #1524 from simplex-chat/ntf-storage
ntf server: PostgreSQL database storage (feature branch)
2025-05-15 21:00:58 +01:00
Evgeny
b90e25a3a5 ntf server: fix repeat token registration when it became invalid (regression) (#1539) 2025-05-15 19:56:48 +01:00
Evgeny Poberezkin
cf4b9f669d agent: always enable periodic notifications 2025-05-15 15:16:44 +01:00
Evgeny Poberezkin
e417d35cce Merge branch 'master' into ntf-storage 2025-05-14 22:20:55 +01:00
Evgeny Poberezkin
deaec3cce2 6.4.0.2 2025-05-07 07:24:06 +01:00
Evgeny
7bbd99644a agent: fix query that updates last message timestamps for notification processing (#1536) 2025-05-06 23:19:58 +01:00
Evgeny
cb59a449dd agent: return error and message absence differently when getting notification messages (#1535)
* agent: return error and message absence differently when getting notification messages

* fix test

* mapM

* inline nse functions, release lock on error or no message
2025-05-06 16:20:01 +01:00
Evgeny
a632eea75b protocol: parsing for UTCTime to stop on "," and ";" (#1532) 2025-05-04 21:11:06 +01:00
Evgeny
3d10c9bf9e agent: handle cases when last message ts is not set for notifications; set last ts for "stale" notifications when messages expired and queue is empty, to prevent repeated processing (#1531)
* agent: handle cases when last message ts is not set for notifications; set last ts for "stale" notifications when messages expired and queue is empty, to prevent repeated processing

* only log errors if they exist

* only set last ts for queue that delivered notification
2025-05-02 11:21:39 +01:00
Evgeny
2f0cdc40af ntf server: fix returned last notifications (#1530) 2025-04-30 14:57:04 +01:00
Evgeny
0a3d014f5d ntf server: priority db pool for notifications and client commands, concurrent resubscriptions, CLI options (#1529)
* ntf server: option to skip specific tokens when importing store log

* logs

* batch later

* subscribe in parallel, bigger queues

* ntf server: test schema and migrations

* optimize subscriptions

* log level

* resubscribe before starting

* better subscribing

* pooled concurrent resubscriptions

* 5

* async resubscribe

* typo

* priority pool

* db batch size
2025-04-30 11:40:32 +01:00
Evgeny
7d0115daec ntf server, agent: send all periodic notifications from one thread, only to old active clients or new clients with periodic notification mode (#1528)
* ntf server, agent: send all periodic notifications from one thread, only to old active clients or new clients with periodic notification mode

* send different type via subscription queues

* option to compact store log on start
2025-04-28 12:18:55 +01:00