Commit Graph

1762 Commits

Author SHA1 Message Date
Evgeny
fdf8bd7ee2 smp server: mask database store operations from async exception for state consistency (#1481)
* smp server: mask database store operations from async exception for state consistency

* remove unused, comments
2025-03-13 13:57:46 +00:00
Evgeny
019a32a623 smp server: optionally append store log with postgres storage (without loading and compacting, for debugging during migration) (#1480)
* smp server: optionally maintain store log with postgres storage (without loading and compacting, for debugging during migration)

* refactor

* remove comment
2025-03-13 10:25:56 +00:00
Evgeny
d44f09d111 tests: pass AStoreType to agent test as a parameter (#1479)
* tests: run agent tests with PostgreSQL SMP servers

* agent tests with postgres database

* enable tests

* fix store log tests

* fix test
2025-03-12 21:47:44 +00:00
Evgeny
6b5de2c51b smp server: exception on invalid migration, check queue storage settings on server start (#1478)
* smp server: check queue storage settings on server start

* fix incorrect postgres migration not throwing exception
2025-03-10 17:28:26 +00:00
shum
9410fb6f16 docker: add libpq dep 2025-03-10 10:03:44 +00:00
Evgeny
7b42aaa132 smp server: expire messages in postgres database, mark queues as deleted, combine tables (#1471)
* smp server: expire messages in postgres database

* tty

* fail if nothing updated in db

* remove old deleted queues

* index

* fix tests
2025-03-10 09:31:50 +00:00
Evgeny Poberezkin
e4b9aa9746 Merge branch 'master' into postgres 2025-03-09 21:58:39 +00:00
sh
6e505f5c0b scripts: add script to reproduce binaries locally (#1477) 2025-03-08 20:51:20 +00:00
Evgeny Poberezkin
a491a1d878 6.3.0.8 v6.3.0 2025-03-07 14:30:00 +00:00
sh
36f5539b9a ci: introduce reproducible builds (#1476)
* ci: introduce reproducible builds

* ci: return 20.04

* smp server: increase timing test threshold

* ci: test outside docker

* ci: fix test step

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2025-03-07 14:15:18 +00:00
Evgeny
1a2afe8bfd agent: fix JSON encoding for protocol errors to be compatible with iOS (#1475) 2025-03-06 08:02:27 +00:00
Evgeny Poberezkin
9fece9ce3d 6.3.0.7 v6.3.0-beta.7 2025-03-02 22:45:07 +00:00
Evgeny Poberezkin
2e2ede5968 Merge branch 'master' into postgres 2025-03-02 22:35:28 +00:00
Evgeny
205d4ead1c smp server: remove store log backups when server starts (#1472) 2025-02-27 07:39:05 +00:00
Evgeny
80a070a8ea smp server: update STM cache only after PostgreSQL update (#1470)
* refactor journal store

* update postgres store methods to update STM cache only after DB update

* todos

* drop

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* fix GHC 8.10.7

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2025-02-26 11:43:08 +00:00
Evgeny
172540984c agent, smp server: PostgreSQL connection pool (#1468)
* agent, smp server: PostgreSQL connection pool

* simplify, create all connections before start

* rename

* remove imports
2025-02-25 15:33:54 +00:00
Evgeny
4dc40bd795 smp server: PostgreSQL queue store (#1448)
* smp server: queue store typeclass

* parameterize JournalMsgStore

* typeclass for queue store

* postgres WIP

* compiles, passes tests

* remove StoreType

* split migrations

* progress

* addQueueRec

* reduce type spaghetti

* remove addQueue from typeclass definition

* getQueue

* test postgres storage in SMP server

* fix schema

* comment

* import queues to postgresql

* import queues to postgresql

* log

* fix test

* counts

* ci: test smp server with postgres backend (#1463)

* ci: test smp server with postgres backend

* postgres service

* attempt

* attempt

* empty

* empty

* PGHOST attempt

* PGHOST + softlink attempt

* only softlink attempt

* working attempt (PGHOST)

* remove env var

* empty

* do not start server without DB schema, do not import when schema exists

* export database

* enable all tests, disable two tests

* option for migration confirmation

* comments

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2025-02-24 15:47:27 +00:00
Evgeny Poberezkin
f9d7b1eebc 6.3.0.6 v6.3.0-beta.6 2025-02-23 19:42:16 +00:00
Evgeny
ffbc733d58 smp server: remove duplicate progress log (#1466) 2025-02-23 19:40:49 +00:00
Evgeny
2286726d72 smp server: start options maintenance and skip-warnings (#1465)
* smp server: start options `maintenance` and `skip-warnings`

* ignore invalid parsing of the last lines

* parsingErr

* fix
2025-02-22 19:26:03 +00:00
spaced4ndy
1b8110a332 xftp server: restore file status from log (#1461)
* xftp server: restore file blocking info from log

* fix parse

* rework

* update

* rename
2025-02-21 22:43:56 +00:00
sh
dad7e1b60c simplex-servers-update: download scripts from tag (#1457)
* simplex-servers-update: download scripts from tag

* set remote version when re-executing the update script

* safeguard variables

* additional checks
2025-02-20 09:26:03 +00:00
Evgeny
72c2ddcf57 agent: move migrations (#1459) 2025-02-18 23:39:29 +00:00
Evgeny
a75e138965 smp server: remove empty queues journals when expiring messages of idle queues (#1458)
* smp server: remove empty queues journals when expiring messages of idle queues

* remove unnecessary update

* ci: update action

* rename
2025-02-18 20:04:58 +00:00
Evgeny
fa319d798a smp server: remove empty journals when opening message queue (#1456)
* smp server: remove empty journals when opening message queue

* update, do not backup state

* test

* version

* do not close queue state when queue is opened for writing

* comment

* quota = 4

* refactor openMsgQueue to prevent extra state backups

* use interval in config

* version, expire backups after 5 min

* refactor

* test
2025-02-17 23:11:34 +00:00
Evgeny Poberezkin
c192339af9 6.3.0.5 v6.3.0-beta.5 2025-02-15 15:36:44 +00:00
Evgeny
fa67d128d1 agent: fix deleting messages after delivery to avoid deleting shared message bodies (#1455)
* agent: fix deleting messages after delivery to avoid deleting shared message bodies

* fix, comments

* rename

* comment
2025-02-15 12:40:10 +00:00
Evgeny
55ff581655 SMP client: dont block on writing to sending queues (#1454)
* SMP client: dont block on writing to sending queues

* only fork if full
2025-02-15 11:31:05 +00:00
sh
560b257af7 scripts: simplex-servers-update menu to update only server binaries etc. (#1445)
* scripts/simplex-servers-update: refactor

* simplex-servers-update: don't check empty variable

* simplex-servers-update: minor decoration change

* simplex-servers-update: add new line when reexecuting script

* simplex-servers-update: refactor menus

* simplex-servers-update: fix re-executing

* simplex-servers-update: fix menus

* simplex-servers-update: do not rerended menu every time

* simplex-servers-update: update menus logic

* simplex-servers-update: ask permission to update only if versions differ

* simplex-servers-update: skip empty variable

* simplex-servers-update: include while loop

* simplex-servers-update: update re-executing skip logic

* simplex-servers-update: ignore empty

* simplex-servers-update: s/while/if

* simplex-servers-update: ignore empty variable

* simplex-servers-update: update skipping scripts

* simplex-servers-update: minor fixes

* simplex-servers-update: add spaces

* simplex-servers-update: safeguard files and add cleanup logic

* simplex-servers-update: update menus

* simplex-servers-update: fix binary check

* simplex-servers-update: adjust erasing lines

* simplex-servers-update: simplify bin update logic

* simplex-servers-update: fix binary path

* simplex-servers-update: format errors

* scripts/main/simplex-servers-update: initial refactor2

* simplex-servers-update: incr update

* simplex-servers-update: initial dynamic menu

* simplex-servers-update: misc fixes

* simplex-servers-update: fix VER

* simplex-servers-update: add VER check

* simplex-servers-update: abort early if there's no options available

* simplex-servers-update: add new helper function and update_misc

* simplex-servers-update: pass selection

* simplex-servers-update: non-interactive usage

* simplex-servers-update: add help menu

* simplex-servers-update: misc fixes and new binary update logic

* simplex-servers-update: local path check

* simplex-servers-update: fix change check

* simplex-servers-update: OLF -> OLD

* simplex-servers-update: always return code from functions

* simplex-servers-update: new menu flow

* simplex-servers-update: fixes

* simplex-servers-update: download function + fixes

* simplex-servers-update: s/bins/binaries

* simplex-servers-update: download message

* simplex-servers-update: cleanup
2025-02-15 10:31:19 +00:00
spaced4ndy
7ac80bffcb agent: store shared message body only once (when it is the same across messages when batching) (#1453)
* agent: store shared message body only once (when it is the same across messages when batching)

* rename

* refactor

* refactor

* save bodies and messages in single transaction

* comment

* comment

* comment

* box

* mapME

* box

* ValueOrRef

* remove instances

* refactor

* comments

* test

* refactor

* mapAccumLM compatibility with ghc 8.10.7

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2025-02-14 18:01:40 +00:00
spaced4ndy
0d8a1a2879 agent: encrypt messages on delivery (#1446)
* agent: save message body once (plan, schema)

* split

* new type

* bs

* encrypt on delivery

* schema

* fix test

* check pad size

* rename

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2025-02-14 12:35:18 +00:00
Evgeny Poberezkin
bd97cb0449 6.3.0.4 v6.3.0-beta.4 2025-02-10 15:12:54 +00:00
Evgeny
a58d3540ad agent: refactor migrations (#1452)
* agent: refactor migrations

* fix postgres
2025-02-07 17:17:53 +00:00
Evgeny
5dbe6337ea ntf server: additional statistics, new invalid token reasons (#1451)
* agent: check ntf token status on registration

* remove check

* update on check

* refactor

* version

* fix

* ntf server: additional statistics

* swap

* version

* more stats

* test, verify invalid

* rename

* exclude test token from stats

* increase delay

* handle invalid token in retry, more reasons

* focus tests

* disable new tests in CI

* fix

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2025-02-07 12:19:11 +00:00
spaced4ndy
b633f89c1a agent: check ntf token status on registration (#1450)
* agent: check ntf token status on registration

* remove check

* update on check

* refactor

* version

* fix

* test, verify invalid

* rename

* increase delay

* disable new tests in CI

* fix

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2025-02-07 11:36:29 +00:00
Evgeny
944a22a2fb ntf server: record token invalidation reason, add date of the last token activity (#1449)
* ntf server: record token invalidation reason, add date of the last token activity

* update time

* rename

* optional

* include token ID in delivery error

* version

* protocol version

* fix, log error
2025-02-06 17:11:35 +00:00
Evgeny
ce24f83b64 refactor STM queues (#1447) 2025-02-05 12:04:27 +00:00
Evgeny Poberezkin
45373e7f1f 6.3.0.3 v6.3.0-beta.3 2025-01-31 13:01:03 +00:00
sh
efe71cd598 docker: refactor (#1438)
* docker: refactor

* github/docker: bump actions and adjust smp ports
2025-01-30 09:22:13 +00:00
Evgeny
ae41717b9b smp server: use origin client version when processing proxied command, fixes old client sending to new server (#1443)
* smp server: pass origin client version when processing proxied command, fixes old client sending to new server

* version

* version
2025-01-28 22:04:46 +00:00
Evgeny
e78ab60c97 build: fix postgres (#1444) 2025-01-28 22:02:41 +00:00
Evgeny
b3c8358a43 agent: combine connection deletion events (#1442) 2025-01-25 08:52:40 +00:00
sh
817f5e1737 scripts/systemd: update services (#1440) 2025-01-24 13:35:33 +00:00
Evgeny
eda9e36c82 agent: track queries (#1439) 2025-01-24 10:31:50 +00:00
Evgeny Poberezkin
2318975375 6.3.0.2 v6.3.0-beta.2 2025-01-20 16:38:14 +00:00
Evgeny
298666380f servers: handle accept error ECONNABORTED (#1437) 2025-01-20 16:32:42 +00:00
Evgeny
6a9075141f xftp server: use recipient ID in control port to delete and block files, smp server: fix version negotiation (#1434)
* xftp server: use recipient ID in control port to delete and block files

* cap smp proxy agent version at 10

* version

* fix prometheus

* fix

* remove old version support

* log connection parameter on error

* tests

* log sent command tag

* log error and client version

* cap proxy version for previous destination server

* comment, test

* remove logging tag

* remove logs

* version

* SMP version 14

* version

* remove comments

* version
2025-01-20 13:45:49 +00:00
spaced4ndy
488c7082f3 agent: store interface (#1436) 2025-01-20 17:02:39 +04:00
spaced4ndy
fdde9863cd agent: add reopenStore function for postgres; run notification tests with postgres (#1435) 2025-01-17 16:27:37 +04:00
Evgeny Poberezkin
9404a3ab63 xftp server: block stats 2025-01-14 10:46:27 +00:00