Commit Graph

56 Commits

Author SHA1 Message Date
spaced4ndy
22e1932372 xftp, ntf: refactor, reuse workers abstractions (#935)
* xftp: refactor, reuse workers abstractions (wip, test fails)

* rename

* refactor (tests pass)

* refactor2

* linebreaks

* do

* reuse worker abstractions for notifications

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-12-27 13:02:08 +04:00
spaced4ndy
e43e4860b9 xftp: file workers cycle through pending files based on retries count (limit number of iterations per work item to prevent stuck delivery) (#930)
* xftp: test file reception - shouldn't get stuck if file is deleted on server

* comment

* expiration test

* approach

* wip

* sort by retries in other works, revert some diff

* revert diff

* modify tests

* refactor

* refactor

* remove prints

* apply to other workers

* remove import

* comment

* refactor

* revert queue size

* fix test

* rename

* comment, correct number of retries

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-12-26 22:20:25 +00:00
Evgeny Poberezkin
2489333c87 agent: use DB to queue async commands and messages (#929)
* agent: use DB to queue async commands and messages

* fix message delivery, async commands dont work

* fix async commands

* remove comment

* rename

* comment

* balance agent operation

* empty lines

* balance another agent operation

* diff

* worker

* qAddress

* fix incorrect queue ID in v1

* types for queues

* fix test

* update index
2023-12-23 12:59:53 +00:00
Evgeny Poberezkin
fa457d1c25 agent: avoid race condition between worker and supervisor when getting work (#928) 2023-12-22 11:12:36 +00:00
Evgeny Poberezkin
13a60d1d39 use ChaChaDRG as the source of randomness (#920)
* use ChaChaDRG as the source of randomness

* remove functions using entropy directly

* comment
2023-12-21 00:12:08 +00:00
Alexander Bondarenko
6bffcc8503 compatibility with GHC 8.10.7 (#906)
* build with ghc-8.10

* add 8.10 to matrix

* cleanup

* stats

* remove getField

* agent

* server

* remote

* space

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-11-29 12:53:06 +00:00
Evgeny Poberezkin
b5f733d2db agent: use IORef for DRG (#882) 2023-10-31 23:52:13 +00:00
Evgeny Poberezkin
9d12d76078 add fourmolu (#868)
* add fourmolu

* linebreak

* simplify

* comment
2023-10-22 09:20:14 +01:00
Evgeny Poberezkin
8d47f69083 agent: fix creating empty file for XFTP to close file (#851) 2023-09-21 11:57:00 +01:00
Evgeny Poberezkin
887ccbcf6c Merge branch 'master' into master-ghc9 2023-09-10 18:00:33 +01:00
Evgeny Poberezkin
5dc3d739b2 agent: support encrypted local files (#837)
* agent: support encrypted local files

* migration, update store, api

* tests, fix

* use CF.plain
2023-08-31 22:43:58 +01:00
Moritz Angermann
002f36dde0 5.4.0.0: use ghc 9.6.2 (#791)
* Add 9.6 compat

* compile with GHC9.6.2: dependencies, imports, code

* refactor typeclasses

* refactor record dot

* update cabal version

* update github actions

* update direct-sqlcipher

* 5.4.0.0

* update cabal.project

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-08-24 20:22:02 +01:00
Evgeny Poberezkin
b001b748db agent: rename functions, remove unused code, stabilize tests (#830)
* agent: rename functions, stabilize tests

* test pattern match
2023-08-24 19:52:58 +01:00
Evgeny Poberezkin
ec268e2006 agent: wait while agent is in foreground to continue xftp operation (#831) 2023-08-24 19:11:44 +01:00
Evgeny Poberezkin
f76a5ca5b6 agent: catch IO errors correctly in MonadError (#795)
* agent: catch IO errors correctly in MonadError

* correction

* correction

* utils

* agentFinally to catch IO exceptions in ExceptT

* rename

* remove, inline

* rename utils

* utils unit test

* test to show catch and finally problems

* tryAllErrors

* enable all tests
2023-07-09 18:04:45 +01:00
spaced4ndy
32a64b994e remove agentOperationBracket from file transfer (#755) 2023-05-15 20:23:18 +04:00
spaced4ndy
af3f70829d xftp: remove user id from internal deletion apis (#739) 2023-04-25 12:48:10 +04:00
spaced4ndy
ff8197b87b xftp: check encrypted file exists when uploading (#738) 2023-04-24 19:45:30 +04:00
spaced4ndy
9f0b9a83d6 xftp: use chunk digest as entity id for transport session (#725) 2023-04-14 13:01:14 +04:00
Evgeny Poberezkin
27cfc4c62b move assertForeground (#724) 2023-04-14 12:52:18 +04:00
Evgeny Poberezkin
ed0f8c79a2 xftp: wait for agent in foreground in XFTP workers without delaying suspension (#721)
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2023-04-13 20:04:38 +04:00
spaced4ndy
e346a81e46 xftp: delete snd files on servers (#722) 2023-04-13 17:46:43 +04:00
spaced4ndy
e6e17ec637 xftp: api to delete snd files internally, cleanup snd files, tests (#719) 2023-04-12 14:50:40 +04:00
spaced4ndy
d35bd8a954 xftp: restore snd files, expire snd files in agent (#718) 2023-04-11 22:00:09 +04:00
spaced4ndy
d1774e5b56 xftp: send progress events (#717) 2023-04-11 19:36:51 +04:00
spaced4ndy
d0cd34989d xftp: choose server when creating chunk, retrying with different servers (#714) 2023-04-10 19:13:03 +04:00
Evgeny Poberezkin
d838be773b xftp: agent client functions to upload file and add recipients (#713)
* xftp: agent client functions to upload file and add recipients

* re-read replicas
2023-04-07 19:34:51 +04:00
Evgeny Poberezkin
2edc7529f9 xftp: client function to create new chunk (#712)
* xftp: client function to create new chunk

* remove chunk digest update

* rename
2023-04-07 14:50:38 +01:00
spaced4ndy
6d9af2ec17 xftp: use fs path when encrypting file and uploading chunks (#711) 2023-04-07 16:08:31 +04:00
spaced4ndy
3533929905 xftp: transform SndFile to recipient descriptions; non optional chunk digest (#710) 2023-04-07 14:29:07 +04:00
spaced4ndy
d8e60ecfdb xftp: recoverable send - spike (#707) 2023-04-07 11:19:50 +04:00
Evgeny Poberezkin
9f8db13553 xftp: agent API to set and test servers (#704)
* xftp: agent API to set and test servers

* ProtocolTestStep

* update agent API for XFTP servers

* ci: update ubuntu versions

* disable test hanging on ubuntu
2023-04-05 20:37:03 +01:00
spaced4ndy
bdff274f56 use Int64 for time calculations (#706) 2023-04-04 14:51:13 +04:00
spaced4ndy
0f23b4ab5c xftp: re-create output file if decryption restarts (#701) 2023-03-30 16:27:47 +04:00
spaced4ndy
e27c4f7b81 xftp: notify about errors on rcv files retry; process snd files errors (#700) 2023-03-30 16:26:33 +04:00
spaced4ndy
b5869cf169 xftp: agent to expire rcv files older than 2 days (#699) 2023-03-28 15:23:08 +04:00
Evgeny Poberezkin
c5eb65fec8 xftp: send events for file sending/receiving progress (#697) 2023-03-27 11:44:15 +01:00
spaced4ndy
0df7733125 xftp: relative file paths, worker management (#694) 2023-03-22 15:34:25 +04:00
spaced4ndy
2bec353eae xftp: add command to delete rcv file; agent manages save path (#692) 2023-03-20 20:08:38 +04:00
Evgeny Poberezkin
40da7e76dd import stateTVar 2023-03-18 13:57:17 +00:00
spaced4ndy
2b1a02d7d4 xftp: reconnect XFTP client on replica retry (#689)
* xftp: re-create XFTP client on replica retry

* closeXFTPSessionClient

* refactor

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-03-17 09:50:49 +00:00
spaced4ndy
9255d48b69 xftp: move delay to replica (#688) 2023-03-16 18:12:38 +04:00
Evgeny Poberezkin
a0eb53b891 xftp: use agent servers in experimental send, refactor decryption (#686)
* xftp: use agent servers in experimental send, refactor decryption

* fix/enable xftp send test test

* allow any order
2023-03-14 22:56:19 +00:00
spaced4ndy
db120b6d2e xftp: pass save path to agent (#685) 2023-03-14 19:16:25 +04:00
Evgeny Poberezkin
ddc2da8fe4 xftp: agent API for xftp commands and events (#684)
* xftp: agent API for xftp commands and events

* fix tests

* fix tests 2

* xftp: update agent send api

* update API to make temp path optional

* revert tmp path changes (fixes send)

---------

Co-authored-by: spacedandy <8711996+spaced4ndy@users.noreply.github.com>
2023-03-13 09:04:23 +00:00
Evgeny Poberezkin
3fe1f3c822 rename FileParty constructors 2023-03-11 10:05:07 +00:00
spaced4ndy
cf1cd75a15 xftp: fix experimental send api description paths (#683) 2023-03-10 21:06:09 +04:00
spaced4ndy
c1266d55a4 xftp: experimental send api wip (#682) 2023-03-10 20:43:10 +04:00
spaced4ndy
cf8f78f518 xftp: add random bytestring entity id for rcv files (#681) 2023-03-10 20:32:37 +04:00
spaced4ndy
cc2066473c xftp: cleanup complete & error files tmp paths; remove save path before decrypting if exists (#677) 2023-03-10 14:26:34 +04:00