Evgeny Poberezkin
40da7e76dd
import stateTVar
2023-03-18 13:57:17 +00:00
Evgeny Poberezkin
2e02ba86fd
add stateTVar import
2023-03-18 13:38:20 +00:00
spaced4ndy
2b1a02d7d4
xftp: reconnect XFTP client on replica retry ( #689 )
...
* xftp: re-create XFTP client on replica retry
* closeXFTPSessionClient
* refactor
---------
Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com >
2023-03-17 09:50:49 +00:00
spaced4ndy
9255d48b69
xftp: move delay to replica ( #688 )
2023-03-16 18:12:38 +04:00
Evgeny Poberezkin
bab689099f
xftp: streaming file encryption/decryption to avoid memory spikes ( #687 )
...
* xftp: streaming file decryption to avoid memory spikes
* refactor, enable tests
* streaming encryption
* refactor
2023-03-16 13:57:21 +00:00
Evgeny Poberezkin
a0eb53b891
xftp: use agent servers in experimental send, refactor decryption ( #686 )
...
* xftp: use agent servers in experimental send, refactor decryption
* fix/enable xftp send test test
* allow any order
2023-03-14 22:56:19 +00:00
spaced4ndy
db120b6d2e
xftp: pass save path to agent ( #685 )
2023-03-14 19:16:25 +04:00
Evgeny Poberezkin
ddc2da8fe4
xftp: agent API for xftp commands and events ( #684 )
...
* xftp: agent API for xftp commands and events
* fix tests
* fix tests 2
* xftp: update agent send api
* update API to make temp path optional
* revert tmp path changes (fixes send)
---------
Co-authored-by: spacedandy <8711996+spaced4ndy@users.noreply.github.com >
2023-03-13 09:04:23 +00:00
Evgeny Poberezkin
3fe1f3c822
rename FileParty constructors
2023-03-11 10:05:07 +00:00
spaced4ndy
cf1cd75a15
xftp: fix experimental send api description paths ( #683 )
2023-03-10 21:06:09 +04:00
spaced4ndy
c1266d55a4
xftp: experimental send api wip ( #682 )
2023-03-10 20:43:10 +04:00
spaced4ndy
cf8f78f518
xftp: add random bytestring entity id for rcv files ( #681 )
2023-03-10 20:32:37 +04:00
spaced4ndy
cc2066473c
xftp: cleanup complete & error files tmp paths; remove save path before decrypting if exists ( #677 )
2023-03-10 14:26:34 +04:00
Evgeny Poberezkin
8fde8e1344
xftp: agent command entities ( #676 )
...
* xftp: agent command entities
* progress event
* parameterize agent command by entity
* refactor
* Eq instance for APartyCmdTag
2023-03-10 09:10:52 +00:00
spaced4ndy
deec963de8
xftp: agent error handling ( #675 )
...
- chunk download doesn't loop on permanent errors
- decryption errors are considered permanent - local worker doesn't retry
- update replica retries; to do - consider use for this field, or remove it
- rcv file Error status - to prevent repeat reads of chunks for download, files for decryption; also plan to use it for filtering on cleanup
- error string saved in separate field for debugging (not part of status type)
- agent event for rcv file errors
2023-03-09 15:32:17 +04:00
Evgeny Poberezkin
552759018e
xftp: server types for any protocol ( #672 )
2023-03-09 09:54:31 +00:00
spaced4ndy
6d52150672
xftp: agent send file design wip ( #674 )
2023-03-08 14:30:37 +04:00
spaced4ndy
e9f949f117
xftp: resume file reception and decryption on agent restart ( #673 )
2023-03-07 13:29:28 +04:00
spaced4ndy
a0c86158d9
xftp agent: update chunk delay ( #671 )
2023-03-03 20:39:15 +04:00
spaced4ndy
cf147397a4
xftp: agent receive file ( #637 )
...
* xftp: agent receive file draft
* receiveFile more stubs, changes to types, schema
* cabal file
* comments
* xftp_server_id
* schema changes, get client, local worker/action
* agent env, save file description, adjust schema
* client stubs
* download chunk wip, store, schema, types
* remove commented code
* read file description, schema, types
* check received, decrypt
* remove pure
* todo
* add XFTP to agent client
* add user id
* agent test
* tests
* rename supervisor into agent
---------
Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com >
2023-03-03 18:50:16 +04:00
Evgeny Poberezkin
6ed4dd1515
update http2 library
2023-03-03 11:00:10 +00:00
Evgeny Poberezkin
914240108d
xftp: update http2 to remove workarounds with padding and delay ( #667 )
2023-03-02 16:00:46 +00:00
Evgeny Poberezkin
94de01a06f
xftp: use SHA256 digest for file chunks to reduce file description size ( #666 )
...
* xftp: use SHA256 digest for file chunks to reduce file description size
* enable all tests
2023-03-01 14:36:55 +00:00
Evgeny Poberezkin
faec32dc5e
xftp: constrain supported file size to 1gb
2023-03-01 13:50:08 +00:00
Evgeny Poberezkin
6227215c3f
xftp: remove file description that cannot be used ( #665 )
...
* xftp: remove file description that cannot be used
* show sender file description first
* change order
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
---------
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
2023-03-01 13:08:59 +00:00
Evgeny Poberezkin
3d3eb335e8
xftp: download timeout, reconnect CLI on timeouts, verbose logging ( #664 )
...
* xftp: download timeout, reconnect CLI on timeouts, verbose logging
* typo
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
---------
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
2023-03-01 07:47:27 +00:00
Evgeny Poberezkin
249bcc7bb3
update cGet to not throw exception if returned string is shorter (trying to fix HTTP2), fix test ( #663 )
2023-02-28 19:16:35 +00:00
Evgeny Poberezkin
524c1de4de
xftp: server stats ( #661 )
2023-02-28 08:59:28 +00:00
Evgeny Poberezkin
875a4bac5e
xftp cli: use 3 chunk sizes (256kb, 1mb, 4mb) ( #660 )
...
* xftp cli: use 3 chunk sizes (256kb, 1mb, 4mb)
* hardcode default servers
* disable rand command
* hide rand command instead of removing
2023-02-27 21:15:22 +00:00
Evgeny Poberezkin
2f15ce2662
xftp: add quota param to server CLI, restrict chunk sizes ( #659 )
...
* xftp: add quota param to server CLI
* only allow certain file sizes, fix tests
2023-02-27 18:01:18 +00:00
Evgeny Poberezkin
781f8e0000
xftp: increase timeouts for file uploads, show progress ( #658 )
...
* xftp: increase timeouts for file uploads, show progress
* refactor
* use default port 443
* typo
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
* upload/download to/from chosen servers concurrently
---------
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
2023-02-27 13:42:15 +00:00
spaced4ndy
1b4b184517
xftp: server - retry on duplicate ids ( #655 )
...
* xftp: server - retry on duplicate ids (with commented monad code)
* remove commented code, refactor
* replace traverse with mapM
* refactor
* refactor
* refactor
* refactor
* refactor
---------
Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com >
2023-02-25 17:55:58 +00:00
Evgeny Poberezkin
8c8a984c1c
xftp: add basic auth ( #653 )
2023-02-24 17:40:26 +04:00
spaced4ndy
202922bceb
xftp: server store log ( #652 )
2023-02-24 15:21:07 +04:00
Evgeny Poberezkin
3ab5e9d110
xftp: expire files on the server, track/limit used storage ( #651 )
...
* xftp: expire files on the server
* track/limit used storage
* support storage quota and disabling queue creation in CLI parameters
* fix ini file
* correction
---------
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
2023-02-24 11:09:51 +04:00
Evgeny Poberezkin
4ce4fa3423
xftp: write secret_box auth tag to the end of the file, for efficiency of ecryption/decryption ( #650 )
...
* xftp: write secret_box auth tag to the end of the file, for efficiency of ecryption/decryption
* comments
2023-02-23 18:28:20 +00:00
spaced4ndy
fbf0b8b175
xftp: cli - delete file command, acknowledge received chunks ( #649 )
2023-02-23 16:54:50 +04:00
spaced4ndy
1f394b76a6
xftp: sender file description ( #647 )
...
* xftp: sender file description
* tests
* check, info, tests
* test
* refactor
* refactor, rename sender file
---------
Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com >
2023-02-22 17:16:49 +04:00
Evgeny Poberezkin
97b8b37325
xftp: FDEL and FACK commands ( #648 )
...
* xftp: FDEL and FACK commands
* correction
* more tests
---------
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com >
2023-02-22 12:32:30 +00:00
Evgeny Poberezkin
bbf9e28696
xftp: add server to client encryption layer with cryptobox ( #646 )
...
* xftp: add server to client encryption layer with cryptobox
* workarounds for http2
* update http2 commit
* remove TODOs
2023-02-22 09:03:18 +00:00
Evgeny Poberezkin
c970674452
faster lazy secret_box
2023-02-19 10:54:54 +00:00
Evgeny Poberezkin
4a9b5412db
xftp: refactor
2023-02-18 23:21:59 +00:00
Evgeny Poberezkin
1ecef81b4d
xftp: validate digests of file chunks in client and server
2023-02-18 22:44:44 +00:00
Evgeny Poberezkin
2bf8cba4fd
xftp: download chunks to separate files
2023-02-18 20:56:46 +00:00
Evgeny Poberezkin
628169adb4
xftp: only receive correct number of bytes
2023-02-18 19:47:17 +00:00
Evgeny Poberezkin
bd0cc25083
XFTP error type
2023-02-17 22:27:36 +00:00
Evgeny Poberezkin
acdf4f41c5
XFTP: CLI tests, output command results, info command ( #642 )
...
* XFTP: CLI tests, output command results
* info command, test
* fix test
2023-02-17 11:38:43 +00:00
spaced4ndy
a3d9ee213d
xftp: cli - option to pass servers, choose servers randomly ( #641 )
2023-02-16 21:06:23 +04:00
spaced4ndy
cf7346d48b
xftp: cli - verify file digest ( #640 )
2023-02-16 21:01:41 +04:00
spaced4ndy
4e8657097c
xftp: cli client draft ( #635 )
...
* xftp: cli client draft
* more stubs
* compiles
* hash, app
* options parsers, random
* tmp
* xftp CLI client agent, simplify CLI command syntax
* only allow argument as a second parameter
* pivot signature draft
* receive file
* pivot sent chunks to recipients
* encryptFile - temp, chunks, specs
* send (upload) file and save file descriptions
* refactor, remove encrypted file
* save file size in description as string
* include filename inside padded encrypted file
* call chunk uploads concurrently, using queueing in HTTP2 as library client does not support concurrent streaming uploads
* download file (does not work yet)
* add digests to sent chunks
* fix recv - save file using AppendMode
* encrypt/decrypt sent file with secretbox
* remove print
* fix file description parsing in tests
* fix test
---------
Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com >
2023-02-16 14:06:03 +04:00
