Evgeny Poberezkin
aace3fd2fb
6.3.1.0
2025-03-22 13:51:55 +00:00
sh
2e67ed9c4c
ci/scripts: refactor reproducible builds ( #1490 )
...
* ci: build cache, remove 20.04, minor refactor
* scripts: update build reproducibility script
* script: add postgresql reproducibility
* script: no cache when building image
* scripts: force symlinks
* scripts: also download prebuilt postgresql binary
* scripts: add missing app var
2025-03-22 13:27:11 +00:00
Evgeny
614fa2b163
smp server: reduce queue expiration/idle intervals, skip expiring very old queues ( #1488 )
...
* smp server: reduce idle queue interval and queue expiration interval
* only expire recent queues (not very old)
* fix
* version
2025-03-20 08:57:47 +00:00
Evgeny
903e96bdfa
rfc: smp queue protocol changes ( #1485 )
...
* rfc: smp queue protocol changes
* questions
* update
* update
* correction
2025-03-19 10:47:57 +00:00
Evgeny
5c0adcbbff
Merge pull request #1467 from simplex-chat/postgres
...
SMP server PostgreSQL storage - feature branch (merge, dont squash)
2025-03-18 16:48:25 +00:00
sh
b2f16eeff4
ci: seperate builds (pg/standard) ( #1486 )
2025-03-18 09:42:53 +00:00
Evgeny
6db79808aa
smp server: use COPY to import store log to postgres db, improve concurrency and error handling ( #1487 )
...
* smp server: use COPY to import store log to postgres db
* compact queues when importing to postgres
* mempty
* version
* handle errors while expiring, mask async exceptions while getting queue
* whitespace
* version
2025-03-18 09:40:22 +00:00
Evgeny
f4b55bfc0c
smp server: CLI option to compact store log with PostgreSQL database ( #1484 )
...
* smp server: CLI option to compact store log with PostgreSQL database
* version
* fix test
2025-03-16 18:47:02 +00:00
Evgeny
fe64d42db1
smp server: split postgres support to a separate executable, to not require postgres library in the main binary ( #1482 )
...
* smp server: split postgres support to a separate executable, to not require postgres library in the main binary
* comments
* enable server_postgres flag by default, add CPP option to test
* refactor
* change default for server_postgres to False
* diff
2025-03-16 11:37:25 +00:00
Evgeny
1b5a9f3b0c
smp server: do not cache all queues from database while processing expirations ( #1483 )
...
* smp server: expire only active queues
* version
* do not cache all queues while processing expirations
* refactor
* foldWithOptions_
* version
* use shared lock when expiring all queues
* use TMVar
* comment
* rename
* remove fold options
* do not create locks in the Map for temporarily loaded queues
* fix
* revert version
2025-03-15 20:11:30 +00:00
Evgeny
fdf8bd7ee2
smp server: mask database store operations from async exception for state consistency ( #1481 )
...
* smp server: mask database store operations from async exception for state consistency
* remove unused, comments
2025-03-13 13:57:46 +00:00
Evgeny
019a32a623
smp server: optionally append store log with postgres storage (without loading and compacting, for debugging during migration) ( #1480 )
...
* smp server: optionally maintain store log with postgres storage (without loading and compacting, for debugging during migration)
* refactor
* remove comment
2025-03-13 10:25:56 +00:00
Evgeny
d44f09d111
tests: pass AStoreType to agent test as a parameter ( #1479 )
...
* tests: run agent tests with PostgreSQL SMP servers
* agent tests with postgres database
* enable tests
* fix store log tests
* fix test
2025-03-12 21:47:44 +00:00
Evgeny
6b5de2c51b
smp server: exception on invalid migration, check queue storage settings on server start ( #1478 )
...
* smp server: check queue storage settings on server start
* fix incorrect postgres migration not throwing exception
2025-03-10 17:28:26 +00:00
shum
9410fb6f16
docker: add libpq dep
2025-03-10 10:03:44 +00:00
Evgeny
7b42aaa132
smp server: expire messages in postgres database, mark queues as deleted, combine tables ( #1471 )
...
* smp server: expire messages in postgres database
* tty
* fail if nothing updated in db
* remove old deleted queues
* index
* fix tests
2025-03-10 09:31:50 +00:00
Evgeny Poberezkin
e4b9aa9746
Merge branch 'master' into postgres
2025-03-09 21:58:39 +00:00
Evgeny Poberezkin
aa26a55df4
Merge branch 'master' into stable
2025-03-09 21:53:03 +00:00
sh
6e505f5c0b
scripts: add script to reproduce binaries locally ( #1477 )
2025-03-08 20:51:20 +00:00
Evgeny Poberezkin
a491a1d878
6.3.0.8
2025-03-07 14:30:00 +00:00
sh
36f5539b9a
ci: introduce reproducible builds ( #1476 )
...
* ci: introduce reproducible builds
* ci: return 20.04
* smp server: increase timing test threshold
* ci: test outside docker
* ci: fix test step
---------
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com >
2025-03-07 14:15:18 +00:00
Evgeny
1a2afe8bfd
agent: fix JSON encoding for protocol errors to be compatible with iOS ( #1475 )
2025-03-06 08:02:27 +00:00
Evgeny Poberezkin
9fece9ce3d
6.3.0.7
2025-03-02 22:45:07 +00:00
Evgeny Poberezkin
2e2ede5968
Merge branch 'master' into postgres
2025-03-02 22:35:28 +00:00
Evgeny
205d4ead1c
smp server: remove store log backups when server starts ( #1472 )
2025-02-27 07:39:05 +00:00
Evgeny
80a070a8ea
smp server: update STM cache only after PostgreSQL update ( #1470 )
...
* refactor journal store
* update postgres store methods to update STM cache only after DB update
* todos
* drop
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
* fix GHC 8.10.7
---------
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
2025-02-26 11:43:08 +00:00
Evgeny
172540984c
agent, smp server: PostgreSQL connection pool ( #1468 )
...
* agent, smp server: PostgreSQL connection pool
* simplify, create all connections before start
* rename
* remove imports
2025-02-25 15:33:54 +00:00
Evgeny
4dc40bd795
smp server: PostgreSQL queue store ( #1448 )
...
* smp server: queue store typeclass
* parameterize JournalMsgStore
* typeclass for queue store
* postgres WIP
* compiles, passes tests
* remove StoreType
* split migrations
* progress
* addQueueRec
* reduce type spaghetti
* remove addQueue from typeclass definition
* getQueue
* test postgres storage in SMP server
* fix schema
* comment
* import queues to postgresql
* import queues to postgresql
* log
* fix test
* counts
* ci: test smp server with postgres backend (#1463 )
* ci: test smp server with postgres backend
* postgres service
* attempt
* attempt
* empty
* empty
* PGHOST attempt
* PGHOST + softlink attempt
* only softlink attempt
* working attempt (PGHOST)
* remove env var
* empty
* do not start server without DB schema, do not import when schema exists
* export database
* enable all tests, disable two tests
* option for migration confirmation
* comments
---------
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
2025-02-24 15:47:27 +00:00
Evgeny Poberezkin
f9d7b1eebc
6.3.0.6
2025-02-23 19:42:16 +00:00
Evgeny
ffbc733d58
smp server: remove duplicate progress log ( #1466 )
2025-02-23 19:40:49 +00:00
Evgeny
2286726d72
smp server: start options maintenance and skip-warnings ( #1465 )
...
* smp server: start options `maintenance` and `skip-warnings`
* ignore invalid parsing of the last lines
* parsingErr
* fix
2025-02-22 19:26:03 +00:00
spaced4ndy
1b8110a332
xftp server: restore file status from log ( #1461 )
...
* xftp server: restore file blocking info from log
* fix parse
* rework
* update
* rename
2025-02-21 22:43:56 +00:00
sh
dad7e1b60c
simplex-servers-update: download scripts from tag ( #1457 )
...
* simplex-servers-update: download scripts from tag
* set remote version when re-executing the update script
* safeguard variables
* additional checks
2025-02-20 09:26:03 +00:00
Evgeny
72c2ddcf57
agent: move migrations ( #1459 )
2025-02-18 23:39:29 +00:00
Evgeny
a75e138965
smp server: remove empty queues journals when expiring messages of idle queues ( #1458 )
...
* smp server: remove empty queues journals when expiring messages of idle queues
* remove unnecessary update
* ci: update action
* rename
2025-02-18 20:04:58 +00:00
Evgeny
fa319d798a
smp server: remove empty journals when opening message queue ( #1456 )
...
* smp server: remove empty journals when opening message queue
* update, do not backup state
* test
* version
* do not close queue state when queue is opened for writing
* comment
* quota = 4
* refactor openMsgQueue to prevent extra state backups
* use interval in config
* version, expire backups after 5 min
* refactor
* test
2025-02-17 23:11:34 +00:00
Evgeny Poberezkin
c192339af9
6.3.0.5
2025-02-15 15:36:44 +00:00
Evgeny
fa67d128d1
agent: fix deleting messages after delivery to avoid deleting shared message bodies ( #1455 )
...
* agent: fix deleting messages after delivery to avoid deleting shared message bodies
* fix, comments
* rename
* comment
2025-02-15 12:40:10 +00:00
Evgeny
55ff581655
SMP client: dont block on writing to sending queues ( #1454 )
...
* SMP client: dont block on writing to sending queues
* only fork if full
2025-02-15 11:31:05 +00:00
sh
560b257af7
scripts: simplex-servers-update menu to update only server binaries etc. ( #1445 )
...
* scripts/simplex-servers-update: refactor
* simplex-servers-update: don't check empty variable
* simplex-servers-update: minor decoration change
* simplex-servers-update: add new line when reexecuting script
* simplex-servers-update: refactor menus
* simplex-servers-update: fix re-executing
* simplex-servers-update: fix menus
* simplex-servers-update: do not rerended menu every time
* simplex-servers-update: update menus logic
* simplex-servers-update: ask permission to update only if versions differ
* simplex-servers-update: skip empty variable
* simplex-servers-update: include while loop
* simplex-servers-update: update re-executing skip logic
* simplex-servers-update: ignore empty
* simplex-servers-update: s/while/if
* simplex-servers-update: ignore empty variable
* simplex-servers-update: update skipping scripts
* simplex-servers-update: minor fixes
* simplex-servers-update: add spaces
* simplex-servers-update: safeguard files and add cleanup logic
* simplex-servers-update: update menus
* simplex-servers-update: fix binary check
* simplex-servers-update: adjust erasing lines
* simplex-servers-update: simplify bin update logic
* simplex-servers-update: fix binary path
* simplex-servers-update: format errors
* scripts/main/simplex-servers-update: initial refactor2
* simplex-servers-update: incr update
* simplex-servers-update: initial dynamic menu
* simplex-servers-update: misc fixes
* simplex-servers-update: fix VER
* simplex-servers-update: add VER check
* simplex-servers-update: abort early if there's no options available
* simplex-servers-update: add new helper function and update_misc
* simplex-servers-update: pass selection
* simplex-servers-update: non-interactive usage
* simplex-servers-update: add help menu
* simplex-servers-update: misc fixes and new binary update logic
* simplex-servers-update: local path check
* simplex-servers-update: fix change check
* simplex-servers-update: OLF -> OLD
* simplex-servers-update: always return code from functions
* simplex-servers-update: new menu flow
* simplex-servers-update: fixes
* simplex-servers-update: download function + fixes
* simplex-servers-update: s/bins/binaries
* simplex-servers-update: download message
* simplex-servers-update: cleanup
2025-02-15 10:31:19 +00:00
spaced4ndy
7ac80bffcb
agent: store shared message body only once (when it is the same across messages when batching) ( #1453 )
...
* agent: store shared message body only once (when it is the same across messages when batching)
* rename
* refactor
* refactor
* save bodies and messages in single transaction
* comment
* comment
* comment
* box
* mapME
* box
* ValueOrRef
* remove instances
* refactor
* comments
* test
* refactor
* mapAccumLM compatibility with ghc 8.10.7
---------
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com >
2025-02-14 18:01:40 +00:00
spaced4ndy
0d8a1a2879
agent: encrypt messages on delivery ( #1446 )
...
* agent: save message body once (plan, schema)
* split
* new type
* bs
* encrypt on delivery
* schema
* fix test
* check pad size
* rename
---------
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com >
2025-02-14 12:35:18 +00:00
Evgeny Poberezkin
bd97cb0449
6.3.0.4
2025-02-10 15:12:54 +00:00
Evgeny
a58d3540ad
agent: refactor migrations ( #1452 )
...
* agent: refactor migrations
* fix postgres
2025-02-07 17:17:53 +00:00
Evgeny
5dbe6337ea
ntf server: additional statistics, new invalid token reasons ( #1451 )
...
* agent: check ntf token status on registration
* remove check
* update on check
* refactor
* version
* fix
* ntf server: additional statistics
* swap
* version
* more stats
* test, verify invalid
* rename
* exclude test token from stats
* increase delay
* handle invalid token in retry, more reasons
* focus tests
* disable new tests in CI
* fix
---------
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
2025-02-07 12:19:11 +00:00
spaced4ndy
b633f89c1a
agent: check ntf token status on registration ( #1450 )
...
* agent: check ntf token status on registration
* remove check
* update on check
* refactor
* version
* fix
* test, verify invalid
* rename
* increase delay
* disable new tests in CI
* fix
---------
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com >
2025-02-07 11:36:29 +00:00
Evgeny
944a22a2fb
ntf server: record token invalidation reason, add date of the last token activity ( #1449 )
...
* ntf server: record token invalidation reason, add date of the last token activity
* update time
* rename
* optional
* include token ID in delivery error
* version
* protocol version
* fix, log error
2025-02-06 17:11:35 +00:00
Evgeny
ce24f83b64
refactor STM queues ( #1447 )
2025-02-05 12:04:27 +00:00
Evgeny Poberezkin
45373e7f1f
6.3.0.3
2025-01-31 13:01:03 +00:00
sh
efe71cd598
docker: refactor ( #1438 )
...
* docker: refactor
* github/docker: bump actions and adjust smp ports
2025-01-30 09:22:13 +00:00
