dandri/simplexmq
1
0
Fork 0
mirror of https://github.com/simplex-chat/simplexmq.git synced 2026-04-25 18:32:21 +00:00
Code Issues Packages Projects Releases Wiki Activity
942 Commits 93 Branches 173 Tags
8fde8e1344699cdcdc67709595c9285cd06bbef3
Commit Graph

608 Commits

Author SHA1 Message Date
Evgeny Poberezkin
8fde8e1344 xftp: agent command entities (#676) 
* xftp: agent command entities

* progress event

* parameterize agent command by entity

* refactor

* Eq instance for APartyCmdTag
 2023-03-10 09:10:52 +00:00
spaced4ndy
deec963de8 xftp: agent error handling (#675) 
- chunk download doesn't loop on permanent errors
- decryption errors are considered permanent - local worker doesn't retry
- update replica retries; to do - consider use for this field, or remove it
- rcv file Error status - to prevent repeat reads of chunks for download, files for decryption; also plan to use it for filtering on cleanup
- error string saved in separate field for debugging (not part of status type)
- agent event for rcv file errors
 2023-03-09 15:32:17 +04:00
Evgeny Poberezkin
552759018e xftp: server types for any protocol (#672) 2023-03-09 09:54:31 +00:00
spaced4ndy
6d52150672 xftp: agent send file design wip (#674) 2023-03-08 14:30:37 +04:00
spaced4ndy
e9f949f117 xftp: resume file reception and decryption on agent restart (#673) 2023-03-07 13:29:28 +04:00
spaced4ndy
a0c86158d9 xftp agent: update chunk delay (#671) 2023-03-03 20:39:15 +04:00
spacedandy
cc38deb65d Merge branch 'master' into xftp 2023-03-03 19:16:12 +04:00
spaced4ndy
cf147397a4 xftp: agent receive file (#637) 
* xftp: agent receive file draft

* receiveFile more stubs, changes to types, schema

* cabal file

* comments

* xftp_server_id

* schema changes, get client, local worker/action

* agent env, save file description, adjust schema

* client stubs

* download chunk wip, store, schema, types

* remove commented code

* read file description, schema, types

* check received, decrypt

* remove pure

* todo

* add XFTP to agent client

* add user id

* agent test

* tests

* rename supervisor into agent

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
 2023-03-03 18:50:16 +04:00
Evgeny Poberezkin
2f7c115207 retryInterval functions pass current interval to the action, to resume with the same interval on restart (improve battery consumption) (#670) 2023-03-03 14:12:36 +00:00
Evgeny Poberezkin
94de01a06f xftp: use SHA256 digest for file chunks to reduce file description size (#666) 
* xftp: use SHA256 digest for file chunks to reduce file description size

* enable all tests
 2023-03-01 14:36:55 +00:00
Evgeny Poberezkin
249bcc7bb3 update cGet to not throw exception if returned string is shorter (trying to fix HTTP2), fix test (#663) 2023-02-28 19:16:35 +00:00
Evgeny Poberezkin
781f8e0000 xftp: increase timeouts for file uploads, show progress (#658) 
* xftp: increase timeouts for file uploads, show progress

* refactor

* use default port 443

* typo

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* upload/download to/from chosen servers concurrently

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
 2023-02-27 13:42:15 +00:00
Evgeny Poberezkin
50c9c95287 Merge branch 'master' into xftp 2023-02-25 18:36:48 +00:00
Evgeny Poberezkin
e4aad7583f unpadded AES-GCM encryption now requires 12 bytes IV (#656) 
* unpadded AES-GCM encryption now requires 12 bytes IV

* update

* simplify AuthTag encoding
 2023-02-25 16:22:10 +00:00
Evgeny Poberezkin
8c8a984c1c xftp: add basic auth (#653) 2023-02-24 17:40:26 +04:00
spaced4ndy
202922bceb xftp: server store log (#652) 2023-02-24 15:21:07 +04:00
Evgeny Poberezkin
3ab5e9d110 xftp: expire files on the server, track/limit used storage (#651) 
* xftp: expire files on the server

* track/limit used storage

* support storage quota and disabling queue creation in CLI parameters

* fix ini file

* correction

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
 2023-02-24 11:09:51 +04:00
Evgeny Poberezkin
4ce4fa3423 xftp: write secret_box auth tag to the end of the file, for efficiency of ecryption/decryption (#650) 
* xftp: write secret_box auth tag to the end of the file, for efficiency of ecryption/decryption

* comments
 2023-02-23 18:28:20 +00:00
spaced4ndy
1f394b76a6 xftp: sender file description (#647) 
* xftp: sender file description

* tests

* check, info, tests

* test

* refactor

* refactor, rename sender file

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
 2023-02-22 17:16:49 +04:00
Evgeny Poberezkin
bbf9e28696 xftp: add server to client encryption layer with cryptobox (#646) 
* xftp: add server to client encryption layer with cryptobox

* workarounds for http2

* update http2 commit

* remove TODOs
 2023-02-22 09:03:18 +00:00
Evgeny Poberezkin
cd1adc8307 Merge branch 'master' into xftp 2023-02-20 09:37:25 +00:00
spaced4ndy
733c9374bf fix comment (#645) 2023-02-20 11:35:01 +04:00
Evgeny Poberezkin
a8121fc8ad add unpadded AES encryption 2023-02-19 12:14:48 +00:00
Evgeny Poberezkin
c970674452 faster lazy secret_box 2023-02-19 10:54:54 +00:00
Evgeny Poberezkin
fcef7deec0 Merge branch 'master' into xftp 2023-02-18 18:46:28 +00:00
Evgeny Poberezkin
6a48b735d1 increase timeout for QUOTA errors 2023-02-18 18:09:58 +00:00
Evgeny Poberezkin
15637e545b Merge branch 'master' into xftp 2023-02-17 21:40:35 +00:00
spaced4ndy
c0dcf283eb server_key_hash fields (#643) 
* server_key_hash fields

* test

* refactor

* fix

* order

* use sync command in test

* refactor

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
 2023-02-17 21:24:32 +00:00
Evgeny Poberezkin
2ddfb044fc parameterize protocol by error type (#644) 2023-02-17 20:46:01 +00:00
spaced4ndy
4e8657097c xftp: cli client draft (#635) 
* xftp: cli client draft

* more stubs

* compiles

* hash, app

* options parsers, random

* tmp

* xftp CLI client agent, simplify CLI command syntax

* only allow argument as a second parameter

* pivot signature draft

* receive file

* pivot sent chunks to recipients

* encryptFile - temp, chunks, specs

* send (upload) file and save file descriptions

* refactor, remove encrypted file

* save file size in description as string

* include filename inside padded encrypted file

* call chunk uploads concurrently, using queueing in HTTP2 as library client does not support concurrent streaming uploads

* download file (does not work yet)

* add digests to sent chunks

* fix recv - save file using AppendMode

* encrypt/decrypt sent file with secretbox

* remove print

* fix file description parsing in tests

* fix test

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
 2023-02-16 14:06:03 +04:00
Evgeny Poberezkin
d8f81b5cdb Merge branch 'master' into xftp 2023-02-15 22:02:19 +00:00
Evgeny Poberezkin
2ae3100bed lazy pad/unpad, secretbox encrypt/decrypt (#639) 2023-02-15 22:01:33 +00:00
Evgeny Poberezkin
88c6b0d663 Merge branch 'master' into xftp 2023-02-15 12:51:34 +00:00
Evgeny Poberezkin
8659d4de05 NaCL secretbox based on cryptonite (#636) 2023-02-15 12:51:08 +00:00
Stanislav Dmitrenko
bccef0ba47 files: server and client spike - basic upload/download (#591) 
* Files: main, env, stats, storeLog

* Better + transport

* Executable

* Env

* Update Client.hs, Server.hs, and 4 more files...

* Answer on request

* Delay

* Temp file

* Bypass cert check

* update package.yml, rename

* update store log

* extend HTTP2 transport

* refactor caStore

* HTTP2 body

* update server stats

* file server/client framework

* verify server commands

* process FNEW command, CLI test works

* simple XFTP server test (fails)

* fix test, refactor

* upload chunk works

* receive file chunk in the client

* remove transport handshake

* typo

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* fix names

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
 2023-02-13 13:36:02 +00:00
Evgeny Poberezkin
d7a008985b Merge branch 'xftp' of github.com:simplex-chat/simplexmq into xftp 2023-02-12 15:53:48 +00:00
spaced4ndy
9775861b6f files: file description parsing & serialization, tests (#633) 
* files: file description parsing & serialization, tests

* chunk list

* processFileDescription

* name

* different values

* undo chunk objects to strings

* parser, serializer

* use Either as monad

* refactor

* remove comments

* fix 2 tests

* fix test

* enable all tests

* remove comment

* move

---------

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>
Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
 2023-02-12 15:53:37 +00:00
Evgeny Poberezkin
4d333a90fa Merge branch 'master' into xftp 2023-02-10 19:07:12 +00:00
Evgeny Poberezkin
eb1723acce refactor transport encoding, make client IO error "temporary" (#634) 
* file server/client framework

* refactor

* make IOError temporary

* fix test
 2023-02-10 19:06:28 +00:00
Evgeny Poberezkin
eff501ddcc Merge branch 'master' into xftp 2023-02-09 23:10:46 +00:00
Evgeny Poberezkin
b342b1dc59 extend HTTP2 transport (#632) 
* extend HTTP2 transport

* refactor caStore

* HTTP2 body

* enable test

* remove maxBodySize
 2023-02-09 23:09:24 +00:00
JRoberts
1523c00bcc files: file description types (#631) 2023-02-09 21:22:58 +04:00
Evgeny Poberezkin
360e55a48f Merge branch 'master' into xftp 2023-02-08 20:57:24 +00:00
Evgeny Poberezkin
ee2a764f93 agent: more strict parsing of TransportHost, tests (#629) 2023-02-04 22:59:28 +00:00
Evgeny Poberezkin
cfe995325a agent: prevent batch deletions from failing on one connection sql error (#628) 
* agent: prevent error reading one connection failing batched subscription

* prevent batch deletions from failing on one connection sql error

* rename
 2023-02-04 20:46:45 +00:00
Evgeny Poberezkin
5d8febc353 avoid possible race conditions when cancelled clients/asyncs can be removed after the new ones are added (so that the new are removed as well) (#627) 2023-01-30 17:46:06 +00:00
Evgeny Poberezkin
6858f74897 Merge branch 'master' into xftp 2023-01-27 18:00:11 +00:00
Evgeny Poberezkin
7fb48930f6 increase database transaction timeout, add test for user deletion (#622) 
* increase database transaction timeout, add test for user deletion

* more test logs
 2023-01-25 20:58:03 +00:00
JRoberts
ff038b492c fix DEL_RCVQ queue parameter type (#621) 2023-01-24 15:59:32 +00:00
Evgeny Poberezkin
b59669a65e refactor functions operating on single connections using batch functions (#620) 
* refactor functions operating on single connections using batch functions

* update tests
 2023-01-24 13:14:13 +00:00