Commit Graph
9 Commits
Author SHA1 Message Date
Evgeny 9be9a76b42 smp server: improve message expiration (#1634)
* smp server: limit by time the queues to export journal messages for

* pass queue/msg thresholds separately

* reset db connection on errors

* Revert "smp server: limit by time the queues to export journal messages for"

This reverts commit d3bc0cba4b.

* fix test compilation

* flag to expire messages

* improve test

* expire messages newer than quota
2025-09-13 11:35:34 +01:00
Evgeny a137d01c90 smp server: improve counts management and expiration for postgres message store (#1631)
* smp server: improve counts management and expiration for postgres message store

* one update

* fix expiration
2025-09-12 11:39:35 +01:00
Evgeny bac6ea6e91 smp server: store messages in PostgreSQL (#1622)
* smp server: store messages in PostgreSQL

* stored procedures to write and to expire messages

* function to export messages

* move all message functions to PostgreSQL, remove delete trigger

* comments

* import messages to db

* fix message import, add export

* fix export

* fix export

* fix compilation flags

* import messages line by line

* fix server start with database storage

* fix compilation

* comments
2025-09-11 20:22:55 +01:00
Evgeny 5241f5fe5e rfc: client certificates for servers using SMP protocol as clients (opertors' chat relays, notification servers, service bots) (#1534)
* rfc: client certificates for high volume clients (opertors' chat relays, notification servers, service bots)

* client certificates types (WIP)

* parameterize Transport

* protocol/schema/api changes

* agent API

* rename command

* agent subscriptions return local ClientServiceId to chat

* verify transmissions

* fix receiving client certificates, refactor

* ntf server: remove shared queue for all notification subscriptions (#1543)

* ntf server: remove shared queue for all notification subscriptions

* wait for subscriber with timeout

* safer

* refactor

* log

* remove unused

* WIP service subscriptions and associations, refactor

* process service subscriptions

* rename

* simplify switching subscriptions

* SMP service handshake with additional server handshake response

* notification delivery and STM persistence for services

* smp server: database storage, store log, fix encoding for STORE error, replace String with Text in locks and error

* stats

* more stats

* rename SMP commands

* service subscriptions in ntf server agent (tests fail)

* fix

* refactor

* exports

* subscribe ntf server as service for associated queues

* test ntf service connection, fix SOKS response, fix service associations not removed in STM storage

* INI option to support services

* ntf server: downgrade subscriptions when service is no longer supported, track counts of subscribed queues

* smp protocol: include service certificate fingerprint in the string signed over with entity key (TODO two tests fail)

* fix test

* ntf server prometheus stats, use Int64 in SOKS/ENDS responses (to avoid conversions), additional error status for ntf subscription

* update RFC

* refactor useServiceAuth to avoid ad hoc decisions about which commands use service signatures, and to prohibit service signatures on other commands

* remove duplicate service signature syntax check from checkCredentials, it is checked in verifyTransmission

* service errors, todos

* fix checkCredentials in ntf server, service errors

* refactor service auth

* refactor

* service agent: store returned queue count instead of expected

* refactor serverThread

* refactor serviceSig

* rename

* refactor, rename, test repeat NSUB service association

* respond with error to SUBS

* smp server: export/import service records between database and store log

* comment

* comments

* ghc 8.10.7
2025-06-06 08:03:47 +01:00
Evgenyandsh b7a9542213 smp server: short links and owners for channels (#1506)
* smp server: short links and owners for channels

* types

* support mutliple rcv keys

* fix down migration, test/create server schema dump

* reduce schema dump

* parameterize type for link data by connection type

* return full connection link data

* test version

* change short link encoding

* test: print pg_dump output

* server pages, link encoding

* fix connection request when queue data and sender ID are created for old servers

* test, change pattern

* ci: install postgresql tools in runner (#1507)

* ci: install postgresql tools in runner

* ci: docker shell abort on error

* fix pattern for ghc 8.10.7

* patch ConnReqUriData SMP encoding to preserve queue mode after decoding

* test for RKEY

* fix/test store log with RKEY

---------

Co-authored-by: sh <37271604+shumvgolove@users.noreply.github.com>
2025-04-10 19:09:47 +01:00
Evgeny b83d897650 smp protocol: short links and other changes from RFC (#1489)
* smp protocol: short links types and other changes from RFC

* add fields for queue link ID and data

* create queue and ntf credentials with NEW command

* all tests

* simplfiy types, update rfc

* update rfc

* include SenderId in NEW request in case queue data is sent

* store queue data and generate link ID if needed

* update rfc

* agent API and types

* SMP commands and persistence for short links

* SMP client functions for short links

* agent client functions for short links

* create rcv queue with short link (TODO secret_box)

* encryption and encoding for link data, postgres client migration

* test creating short link

* get link and data, tests

* comments

* type signature
2025-03-26 17:26:27 +00:00
Evgeny 614fa2b163 smp server: reduce queue expiration/idle intervals, skip expiring very old queues (#1488)
* smp server: reduce idle queue interval and queue expiration interval

* only expire recent queues (not very old)

* fix

* version
2025-03-20 08:57:47 +00:00
Evgeny 7b42aaa132 smp server: expire messages in postgres database, mark queues as deleted, combine tables (#1471)
* smp server: expire messages in postgres database

* tty

* fail if nothing updated in db

* remove old deleted queues

* index

* fix tests
2025-03-10 09:31:50 +00:00
Evgenyandspaced4ndy 4dc40bd795 smp server: PostgreSQL queue store (#1448)
* smp server: queue store typeclass

* parameterize JournalMsgStore

* typeclass for queue store

* postgres WIP

* compiles, passes tests

* remove StoreType

* split migrations

* progress

* addQueueRec

* reduce type spaghetti

* remove addQueue from typeclass definition

* getQueue

* test postgres storage in SMP server

* fix schema

* comment

* import queues to postgresql

* import queues to postgresql

* log

* fix test

* counts

* ci: test smp server with postgres backend (#1463)

* ci: test smp server with postgres backend

* postgres service

* attempt

* attempt

* empty

* empty

* PGHOST attempt

* PGHOST + softlink attempt

* only softlink attempt

* working attempt (PGHOST)

* remove env var

* empty

* do not start server without DB schema, do not import when schema exists

* export database

* enable all tests, disable two tests

* option for migration confirmation

* comments

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2025-02-24 15:47:27 +00:00