Commit Graph
981 Commits
Author SHA1 Message Date
spaced4ndyandGitHub d8e60ecfdb xftp: recoverable send - spike (#707) 2023-04-07 11:19:50 +04:00
Evgeny PoberezkinandGitHub 9f8db13553 xftp: agent API to set and test servers (#704)
* xftp: agent API to set and test servers

* ProtocolTestStep

* update agent API for XFTP servers

* ci: update ubuntu versions

* disable test hanging on ubuntu
2023-04-05 20:37:03 +01:00
Evgeny Poberezkin 511a97c5d0 update http2 2023-04-05 09:41:13 +01:00
Evgeny PoberezkinandGitHub 44f0dd39f3 ci: disable 2 tests on linux only, switch to ubuntu 20 and 22 (#705)
* ci: disable 2 tests on linux only, switch to ubuntu 20 and 22

* fix platform name

* keep ubuntu 22 binaries

* Revert "keep ubuntu 22 binaries"

This reverts commit a1bbb12870.

* skip 1 more test

* skip 1 more test

* log os

* log os

* unconditionally skip test

* skip 1 more test in CI

* fix tests
2023-04-04 12:19:38 +01:00
spaced4ndyandGitHub bdff274f56 use Int64 for time calculations (#706) 2023-04-04 14:51:13 +04:00
Evgeny PoberezkinandGitHub 83364052b1 xftp: implement server command to add file recipients (#702) 2023-04-03 09:33:12 +01:00
Evgeny Poberezkin c6033e51a0 update http2 2023-04-01 16:43:41 +01:00
spaced4ndyandGitHub 0f23b4ab5c xftp: re-create output file if decryption restarts (#701) 2023-03-30 16:27:47 +04:00
spaced4ndyandGitHub e27c4f7b81 xftp: notify about errors on rcv files retry; process snd files errors (#700) 2023-03-30 16:26:33 +04:00
spaced4ndyandGitHub b5869cf169 xftp: agent to expire rcv files older than 2 days (#699) 2023-03-28 15:23:08 +04:00
spaced4ndyandGitHub 6b03876ef1 catch errors in cleanup manager (#698) 2023-03-27 18:55:20 +04:00
6a665a0833 support down migrations (#696)
* support down migrations

* test

* update

* add down migrations and tests

* get current migrations

* corrections

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* update tests

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2023-03-27 12:44:53 +01:00
Evgeny PoberezkinandGitHub c5eb65fec8 xftp: send events for file sending/receiving progress (#697) 2023-03-27 11:44:15 +01:00
Tristan de CacquerayandGitHub 5743069021 Remove duplicated serverIdentity field (#491) 2023-03-26 12:37:31 +01:00
spaced4ndyandGitHub 0df7733125 xftp: relative file paths, worker management (#694) 2023-03-22 15:34:25 +04:00
Evgeny PoberezkinandGitHub d41c2bec2a store retry intervals to resume from the same interval on restart (#693)
* store retry intervals to resume from the same interval on restart

* add migration

* updates
2023-03-22 08:42:56 +00:00
spaced4ndyandGitHub 7e265e5c59 Merge pull request #584 from simplex-chat/xftp
XFTP
2023-03-20 20:33:01 +04:00
spaced4ndyandGitHub 2bec353eae xftp: add command to delete rcv file; agent manages save path (#692) 2023-03-20 20:08:38 +04:00
Evgeny Poberezkin 40da7e76dd import stateTVar 2023-03-18 13:57:17 +00:00
Evgeny Poberezkin 2e02ba86fd add stateTVar import 2023-03-18 13:38:20 +00:00
Evgeny Poberezkin 02eb1e9822 add stateTVar imports for compatibility with older hackage index in nix builds 2023-03-18 07:57:25 +00:00
Evgeny Poberezkin cd48c33c0d Merge branch 'master' into xftp 2023-03-17 13:30:52 +00:00
Evgeny PoberezkinandGitHub 5258fe9b7e add sha512 (#691)
* add sha512

* update export title
2023-03-17 13:01:49 +00:00
Evgeny PoberezkinandGitHub e33d5b5184 use different constraint for non-error functions (#690) 2023-03-17 16:01:24 +04:00
2b1a02d7d4 xftp: reconnect XFTP client on replica retry (#689)
* xftp: re-create XFTP client on replica retry

* closeXFTPSessionClient

* refactor

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-03-17 09:50:49 +00:00
spaced4ndyandGitHub 9255d48b69 xftp: move delay to replica (#688) 2023-03-16 18:12:38 +04:00
Evgeny PoberezkinandGitHub bab689099f xftp: streaming file encryption/decryption to avoid memory spikes (#687)
* xftp: streaming file decryption to avoid memory spikes

* refactor, enable tests

* streaming encryption

* refactor
2023-03-16 13:57:21 +00:00
Evgeny PoberezkinandGitHub a0eb53b891 xftp: use agent servers in experimental send, refactor decryption (#686)
* xftp: use agent servers in experimental send, refactor decryption

* fix/enable xftp send test test

* allow any order
2023-03-14 22:56:19 +00:00
spaced4ndyandGitHub db120b6d2e xftp: pass save path to agent (#685) 2023-03-14 19:16:25 +04:00
64f5d63d2e Support for building ARM docker images (#679)
* add llvm to build stage

running docker build on Apple Sillicon would result in this error:

#14 49.39 <no location info>: error:
#14 49.39     Warning: Couldn't figure out LLVM version!
#14 49.39              Make sure you have installed LLVM between [9 and 13)

I assume it can’t find llvm/clang?

* install using unattended helper script

the current haskell is hardcoded to x86, so switch to using haskell’s official helper script which supports arm

also need to bump up setting the path, since ghcup is no longer in /usr/bin

reference: https://www.haskell.org/ghcup/guide/#continuous-integration

* add numa headers/libs

docker build (specifically haskell install step) would fail with :

#7 107.2 utils/ghc-cabal/dist-install/build/tmp/ghc-cabal: error while loading shared libraries: libnuma.so.1: cannot open shared object file: No such file or directory

I think this is because numa isn’t installed on arm instances of focal by default, but it is in later versions

anyway this fixes that

* also add numa to final

The image builds at this point! But it fails to boot on arm - again with a numa error.

This fixes that.

* don’t specify llvm version

---------

Co-authored-by: +shyfire131 <shyfire131@shyfire131.net>
2023-03-13 20:00:34 +00:00
ddc2da8fe4 xftp: agent API for xftp commands and events (#684)
* xftp: agent API for xftp commands and events

* fix tests

* fix tests 2

* xftp: update agent send api

* update API to make temp path optional

* revert tmp path changes (fixes send)

---------

Co-authored-by: spacedandy <8711996+spaced4ndy@users.noreply.github.com>
2023-03-13 09:04:23 +00:00
Evgeny Poberezkin 3fe1f3c822 rename FileParty constructors 2023-03-11 10:05:07 +00:00
Evgeny Poberezkin 448b674fb1 5.0.0-beta.5 v5.0.0-beta.5 2023-03-10 17:37:38 +00:00
spaced4ndyandGitHub cf1cd75a15 xftp: fix experimental send api description paths (#683) 2023-03-10 21:06:09 +04:00
Evgeny PoberezkinandGitHub 40164ff21f update http2 to 4.1 (#680) 2023-03-10 16:59:26 +00:00
spaced4ndyandGitHub c1266d55a4 xftp: experimental send api wip (#682) 2023-03-10 20:43:10 +04:00
spaced4ndyandGitHub cf8f78f518 xftp: add random bytestring entity id for rcv files (#681) 2023-03-10 20:32:37 +04:00
Evgeny Poberezkin accfee5a52 5.0.0-beta.4 v5.0.0-beta.4 2023-03-10 10:32:05 +00:00
spaced4ndyandGitHub cc2066473c xftp: cleanup complete & error files tmp paths; remove save path before decrypting if exists (#677) 2023-03-10 14:26:34 +04:00
Evgeny PoberezkinandGitHub 8fde8e1344 xftp: agent command entities (#676)
* xftp: agent command entities

* progress event

* parameterize agent command by entity

* refactor

* Eq instance for APartyCmdTag
2023-03-10 09:10:52 +00:00
spaced4ndyandGitHub deec963de8 xftp: agent error handling (#675)
- chunk download doesn't loop on permanent errors
- decryption errors are considered permanent - local worker doesn't retry
- update replica retries; to do - consider use for this field, or remove it
- rcv file Error status - to prevent repeat reads of chunks for download, files for decryption; also plan to use it for filtering on cleanup
- error string saved in separate field for debugging (not part of status type)
- agent event for rcv file errors
2023-03-09 15:32:17 +04:00
Evgeny PoberezkinandGitHub 552759018e xftp: server types for any protocol (#672) 2023-03-09 09:54:31 +00:00
spaced4ndyandGitHub 6d52150672 xftp: agent send file design wip (#674) 2023-03-08 14:30:37 +04:00
spaced4ndyandGitHub e9f949f117 xftp: resume file reception and decryption on agent restart (#673) 2023-03-07 13:29:28 +04:00
spaced4ndyandGitHub a0c86158d9 xftp agent: update chunk delay (#671) 2023-03-03 20:39:15 +04:00
spacedandy cc38deb65d Merge branch 'master' into xftp 2023-03-03 19:16:12 +04:00
cf147397a4 xftp: agent receive file (#637)
* xftp: agent receive file draft

* receiveFile more stubs, changes to types, schema

* cabal file

* comments

* xftp_server_id

* schema changes, get client, local worker/action

* agent env, save file description, adjust schema

* client stubs

* download chunk wip, store, schema, types

* remove commented code

* read file description, schema, types

* check received, decrypt

* remove pure

* todo

* add XFTP to agent client

* add user id

* agent test

* tests

* rename supervisor into agent

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-03-03 18:50:16 +04:00
Evgeny PoberezkinandGitHub 2f7c115207 retryInterval functions pass current interval to the action, to resume with the same interval on restart (improve battery consumption) (#670) 2023-03-03 14:12:36 +00:00
Evgeny Poberezkin 6ed4dd1515 update http2 library 2023-03-03 11:00:10 +00:00
Evgeny PoberezkinandGitHub 914240108d xftp: update http2 to remove workarounds with padding and delay (#667) 2023-03-02 16:00:46 +00:00