Commit Graph

1069 Commits

Author SHA1 Message Date
Evgeny Poberezkin
82aec2cd8f Revert "db error busy treatments (#796)" (#808)
This reverts commit 1afcefa5e7.
2023-08-09 12:24:03 +01:00
Evgeny Poberezkin
fdbfe0e8d1 5.3.0.1 v5.3.0 2023-07-22 13:21:53 +01:00
spaced4ndy
e81f3b5529 add indexes (#809) 2023-07-22 13:20:49 +01:00
Evgeny Poberezkin
6314bb1706 agent: partially fix message queries, debug message times (#810)
* agent: partially fix message queries, debug message times

* fix more joins
2023-07-22 13:00:25 +01:00
Evgeny Poberezkin
c54be32135 Revert "agent: expire messages on quota errors (#801)" (#807)
This reverts commit 3fee468051.
2023-07-22 09:16:06 +01:00
spaced4ndy
7e2b309450 migration to delete expired messages (#801 is suspected to have caused temporary traffic increase due to flood of non expired messages) (#806) 2023-07-20 16:15:33 +04:00
spaced4ndy
4ed40fa5d5 enable a test in CI ("should resume sending file after restart") (#800) 2023-07-17 20:33:25 +04:00
Evgeny Poberezkin
de95119ca6 5.3.0.0 v5.3.0-beta.0 2023-07-15 13:35:09 +01:00
Evgeny Poberezkin
4fae7dcaee server: control port (#804)
* server: control port

* do not remove messages when saving via control port

* remove unused record fields

* fix tests
2023-07-15 13:33:00 +01:00
Evgeny Poberezkin
d989d11478 server: refactor restoring store log (#803) 2023-07-15 10:37:08 +01:00
Evgeny Poberezkin
1901e96ecc server: make main SMP server queues unbounded (#802) 2023-07-14 21:07:45 +01:00
Evgeny Poberezkin
3fee468051 agent: expire messages on quota errors (#801) 2023-07-14 18:13:54 +01:00
Evgeny Poberezkin
58cb2855d2 agent: delivery receipts (#752)
* rfc: delivery receipts

* update doc

* update rfc

* implementation plan, types, schema

* migration, update types

* update types

* rename migration

* export MsgReceiptStatus, JSON encoding

* update rfc, schema

* correction

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* skeleton of the implementation

* more implementation (some tests fail)

* more code, 1 test fails

* fix encoding

* refactor

* refactor

* test, fix

* only send receipts in v3+, test

* flip condition

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* flip condition

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* agent version 4 required to send receipts

* fix test

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2023-07-13 22:33:48 +01:00
Evgeny Poberezkin
745a144e0c 5.2.1 (ntf 1.5.1, xftp 1.0.1) (#799) v5.2.1 2023-07-13 19:23:06 +01:00
Evgeny Poberezkin
4c6c436e7f resubscribe in batches in Client.Agent (#798)
* resubscribe in batches in Client.Agent

* fix test
2023-07-12 07:35:38 +01:00
Evgeny Poberezkin
b61e3b5f95 server: disable transport timeout (#793) 2023-07-11 19:43:31 +01:00
Evgeny Poberezkin
58dbc197ce increase TCP timeout for clients (#797) 2023-07-11 18:54:14 +01:00
spaced4ndy
1afcefa5e7 db error busy treatments (#796) 2023-07-11 20:19:20 +04:00
Evgeny Poberezkin
532cd2f39c fix allFinally 2023-07-09 21:36:23 +01:00
Evgeny Poberezkin
2f5c646e55 test with catchError/tryError 2023-07-09 18:38:44 +01:00
Evgeny Poberezkin
f76a5ca5b6 agent: catch IO errors correctly in MonadError (#795)
* agent: catch IO errors correctly in MonadError

* correction

* correction

* utils

* agentFinally to catch IO exceptions in ExceptT

* rename

* remove, inline

* rename utils

* utils unit test

* test to show catch and finally problems

* tryAllErrors

* enable all tests
2023-07-09 18:04:45 +01:00
spaced4ndy
f2657f9c0b add crypto error to RSYNC event (#794) 2023-07-05 18:14:51 +04:00
Evgeny Poberezkin
fe22d9b299 agent api to reconnect all servers (#792) 2023-07-04 22:54:45 +01:00
Evgeny Poberezkin
75fe28a8a6 lint 2023-07-03 17:43:56 +01:00
Evgeny Poberezkin
54dc8d42e7 5.2.0, ntf server 1.5.0 (#790) v5.2.0 2023-07-03 13:32:58 +01:00
spaced4ndy
0e1562deae increase connection agent version according to received messages; add version and flags to connection stats, bump currentSMPAgentVersion (#789) 2023-06-30 20:19:10 +04:00
Evgeny Poberezkin
94540a2c71 servers: add TCP timeouts to avoid memory leaks (#776)
* servers: add TCP timeouts to avoid memory leaks

* fix tests

* only use RecvTimeOut

* servers: simple timeout for TCP transport

* revert dependency change

* simplify

* simplify

* simplify 2
2023-06-30 16:22:01 +01:00
Evgeny Poberezkin
16367fcb3b make timeouts for batched functions dependent on the number of batches (#786)
* make timeouts for batched functions dependent on the number of batches

* fix

* refactor

* refactor

* change import

* refactor

* rename function

* rename

* refactor

* remove space
2023-06-30 12:26:09 +01:00
spaced4ndy
8be2505fa0 ratchet re-synchronization (#774)
* ratchet re-synchronization rfc wip

* additions

* additions, types

* fix tests

* re-sync implementation wip

* re-sync implementation 1st rev.

* test wip

* test passes

* doc

* wording

* improve doc schema

* single agreed state

* refactor (1 state variable)

* allowed -> required

* prohibit enqueue

* enqueue

* send via multiple queues

* test with server offline

* clarify errors

* rename

* more tests

* refactor

* rename AgentRKey

* rename AM_CONN_RATCHET_KEY

* more tests

* rename

* write encoded AgentRatchetInfo to AgentRatchetKey info

* move withConnLock

* refactor qDuplex

* re-create ratchet on receiving second key

* invert condition

* refactor

* simplify

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-06-30 14:17:08 +04:00
spaced4ndy
a000419bd7 restore confirmation after client restart (#783) 2023-06-29 10:10:34 +04:00
spaced4ndy
c8a8e2c297 differentiate agent INACTIVE error, treat as temporary (#784) 2023-06-29 10:00:53 +04:00
Evgeny Poberezkin
f7d038ef20 agent: more batched commands (#782)
* agent: more batched commands

* layout
2023-06-28 19:50:11 +01:00
Evgeny Poberezkin
4a927d1ae2 ntf server: better batching and logging (#780)
* ntf server: better batching and logging

* reduce batch delay for ntf server

* comments

* 5.1.3, ntf 1.4.2

* more logging

* more logging

* split large batches, more logging

* remove some logs
v5.1.3
2023-06-26 20:14:35 +01:00
spaced4ndy
3a74558e84 fix connection switch status db encoding (#781) 2023-06-26 21:03:46 +04:00
Evgeny Poberezkin
e3fff98b1b 5.1.2, ntf server 1.4.1 v5.1.2 2023-06-25 09:31:55 +01:00
Evgeny Poberezkin
acea477aab batch subscriptions for ntf server (#778)
* return updated ConnectionStats from switchConnectionAsync (#777)

* batch ntf server subscriptions to SMP servers

* refactor

* fix filtering queues in reconnect

* test, remove repeated grouping

* enable all tests

* reduce subscriptions count

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2023-06-25 08:27:50 +01:00
spaced4ndy
c31668ce28 return updated ConnectionStats from switchConnectionAsync (#777) 2023-06-19 14:51:14 +04:00
spaced4ndy
2efe1496d2 rename "stop switch" -> "abort switch", add canAbortSwitch to rcv queue info (#775) 2023-06-16 13:41:01 +04:00
spaced4ndy
011e1109e9 connection switch - track status, stop switch command (#764)
* connection switch improvements

* stop in transaction

* refactor

* checks in transaction

* withSwitchingRQ in transaction

* withSwitchedSQ in transaction

* refactor

* refactor

* inverse some cases

* reset snd switch progress on repeat qadd

* remove unused functions

* connection info type

* correction

* SPFinalizing phase, correctly update connection in event

* refactor

* tests

* refactor switch conn lock taking, take lock in switch stop, remove ineligible test

* comment

* corrections

* corrections

* corrections

* refactor

* remove checks in transactions, remove some statuses

* remove diff

* refactor

* refactor tests

* update connection in memory

* revert diff

* corrections

* more tests

* remove statuses, change switching queue lookup

* async queue deletion

* refactor

* lock, phases

* test status

* refactor

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-06-15 11:15:46 +04:00
spaced4ndy
fbf1afc9f5 run a single batch test to avoid hanging on server disconnect (#773) 2023-06-13 19:36:47 +04:00
sh
76e3863cbb install: simplify systemd service and add new helper stopscript (#771)
* install: simplify systemd service and add new helper stopscript

* README: update hash
2023-06-05 13:01:16 +04:00
sh
1c2fbdb2c5 install: some more fixes (#770) 2023-06-05 11:12:09 +04:00
sh
1822296481 install.sh: fixes (#765)
* install.sh: fixes

* change back to stable
2023-06-05 10:27:02 +04:00
spaced4ndy
b747080db3 add more message delivery tests (#763) 2023-06-02 18:00:24 +04:00
tcely
20f7b538e5 Docker entrypoint fixes (#753)
* docker entrypoint: DRY store log backup

Don't Repeat Yourself

Set a variable for the full path to the source file.

Create the backup path by appending an extension from `date` output.

Also fixed quoting and switched to an `if` block.

Clean up block level variables.

* docker entrypoint: use ISO 8601 format

The previous format discards information about the local time zone.

* docker entrypoint: always use UTC

Now the format always ends in +00:00, and we can ignore that part again.

* docker entrypoint: provide the date format again

* docker entrypoint: remove time zone from the date format

* docker entrypoint: use an unambiguous date format

Present a leading zero before the month:
YYYY-0MM-DD

Both YYYY-MM-DD and YYYY-DD-MM are used by people and can be confusing in the beginning of the month.

* docker entrypoint: use appropriate quoting

Without avoiding field splitting a password containing a space changes the number of arguments being set.

* docker entrypoint: use explicit braces for custom variables

Make the intentions clear and don't assume the user knows the special cases for when variables won't be extended.

Example:
word=animal
words=mistake
echo "$words vs $word vs ${word}s"

* docker entrypoint-xftp-server: braces and quoting

* docker entrypoint-xftp-server: backup block

* docker entrypoints: explain date format in a comment

* switched from long to short option to date for POSIX

* docker entrypoint-smp-server: explain date format

* docker entrypoint-xftp-server: explain date format

* docker entrypoints: further explain format

I fixed the case to match the date format letters.

Also, use words to explain since I don't want everyone to need to read about date formats to understand.

* docker entrypoints: only quote letters

I was either going to quote the dashes too or stop quoting the colons.

Having less quotes was more readable.

* Revert "docker entrypoint: use an unambiguous date format"

This reverts commit ba2a93bad9.

* docker entrypoints: remove %F

* docker entrypoint-smp-server: remove %F

Used part of the explicit ISO 8601 format,
provided by the coreutils date invocation guide.

* docker entrypoint-xftp-server: remove %F

Used part of the explicit ISO 8601 format,
provided by the coreutils date invocation guide.
2023-05-29 14:12:23 +03:00
Evgeny Poberezkin
89caf55729 agent: ack message in case of decryption error (#759) 2023-05-27 18:28:14 +01:00
Evgeny Poberezkin
c08cfb460b 5.1.1 v5.1.1 2023-05-27 15:54:40 +01:00
Evgeny Poberezkin
03eca19d38 server: expire messages when restoring them (#758)
* server: expire messages when restoring them

* add test

* specify constructor

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* improve test

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2023-05-27 15:52:49 +01:00
Evgeny Poberezkin
a5a3a2cbad server: increase server main queue size 2023-05-27 15:18:53 +01:00
spaced4ndy
4a3c5abf32 add with withTransaction version with logging (#757) 2023-05-26 20:33:48 +04:00