Commit Graph

50 Commits

Author SHA1 Message Date
Alexander Bondarenko
246a0d10c2 xftp: raise internal upload limit to 5gb (#1020)
* xftp: raise internal upload limit to 5gb

* extract hard limit from agent
2024-03-02 18:46:05 +00:00
Evgeny Poberezkin
416f1b1721 smp: command authorization (#982)
* smp: command authorization

* fix encoding, most tests

* remove old tests

* authorize via crypto_box

* extract authenticator to Crypto module

* make TransmissionAuth Maybe

* rfc

* support authenticators in NTF protocol, test matrix (no backwards compatibility yet from new clients to old servers)

* fix/add tests, add version config to "small" agent

* separate client and server versions for SMP protocol

* test batching SMP v7

* do not send session ID in each transmission

* refactor auth verification in the server, split tests

* server "warm up" fixes timing test

* uncomment SUB timing test

* comments, disable two timing tests

* rename version

* increase auth timing test failure threshold

* use different algorithms to authorize snd/rcv commands, use random correlation ID

* transport: fetch and store server certificate (#985)

* THandleParams (WIP, does not compile)

* transport: fetch and store server certificate

* smp: add getOnlinePubKey example to smpClientHandshake

* add server certs and sign authPub

* cleanup

* update

* style

* load server certs from test fixtures

* sign ntf authPubKey

* fix onServerCertificate

* increase delay before sending messages

* require certificate with key in SMP server handshake

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>

* remove dhSecret from THandle

* remove v8, merge all changes to one version

* parameterize THandle

* rfc: transmission ecnryption

* Revert "parameterize THandle"

This reverts commit 75adfc94fb.

* use batch syntax for ntf server commands

* separate encodeTransmission when there is no key

* typo

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* rename

* diff

---------

Co-authored-by: Alexander Bondarenko <486682+dpwiz@users.noreply.github.com>
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2024-02-16 11:45:54 +00:00
Alexander Bondarenko
2f7a288280 xftp: add sending and receiving via URI-encoded redirects (#968)
* xftp: add URI encoding for FileDescription

* tweak URI

* allow smaller blocks

* draft xftpReceiveFileFollow' and xftpSendFilePublic'

* add sending with redirect

* allow 64k chunks

* add migrations with redirect fields

* add test case

* fix deadlock

* revert CLI code

* WIP: working send/receive via URI

* fix field ambiguity

* cleanup

* update agent db schema

* update minimal chunk size

* add rfc

* apply suggestions from code review

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>

* add createRcvFileRedirect

* extract Simplex.Messaging.ServiceScheme and reuse for files

* update db schema

* check size/digest on receive complete

* cleanup

* use SIZE/DIGEST errors for redirects too

* split digest/size errors from redirect checks

* fix redirect error encoding

* rename RedirectMeta to RedirectFileInfo

* use query encoding for file URI

* group maybe fields under RcvFileRedirect

* add extras field

* update rfc

* add extras encoding and no-redirect tests

* fix toStrict for old ghc

* extra client data in file descr URI

* remove decoded yaml file

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2024-02-13 14:08:49 +00:00
Evgeny Poberezkin
13a60d1d39 use ChaChaDRG as the source of randomness (#920)
* use ChaChaDRG as the source of randomness

* remove functions using entropy directly

* comment
2023-12-21 00:12:08 +00:00
Alexander Bondarenko
6bffcc8503 compatibility with GHC 8.10.7 (#906)
* build with ghc-8.10

* add 8.10 to matrix

* cleanup

* stats

* remove getField

* agent

* server

* remote

* space

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-11-29 12:53:06 +00:00
Evgeny Poberezkin
9d12d76078 add fourmolu (#868)
* add fourmolu

* linebreak

* simplify

* comment
2023-10-22 09:20:14 +01:00
Evgeny Poberezkin
887ccbcf6c Merge branch 'master' into master-ghc9 2023-09-10 18:00:33 +01:00
Evgeny Poberezkin
504465f8e2 xftp: add 64kb file chunk (#842)
* xftp: add 64kb file chunk

* disable chunk size 64kb in the agent

* revert rename
2023-09-08 13:31:11 +01:00
Evgeny Poberezkin
5dc3d739b2 agent: support encrypted local files (#837)
* agent: support encrypted local files

* migration, update store, api

* tests, fix

* use CF.plain
2023-08-31 22:43:58 +01:00
Moritz Angermann
002f36dde0 5.4.0.0: use ghc 9.6.2 (#791)
* Add 9.6 compat

* compile with GHC9.6.2: dependencies, imports, code

* refactor typeclasses

* refactor record dot

* update cabal version

* update github actions

* update direct-sqlcipher

* 5.4.0.0

* update cabal.project

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-08-24 20:22:02 +01:00
Evgeny Poberezkin
b001b748db agent: rename functions, remove unused code, stabilize tests (#830)
* agent: rename functions, stabilize tests

* test pattern match
2023-08-24 19:52:58 +01:00
Evgeny Poberezkin
745a144e0c 5.2.1 (ntf 1.5.1, xftp 1.0.1) (#799) 2023-07-13 19:23:06 +01:00
Evgeny Poberezkin
4a927d1ae2 ntf server: better batching and logging (#780)
* ntf server: better batching and logging

* reduce batch delay for ntf server

* comments

* 5.1.3, ntf 1.4.2

* more logging

* more logging

* split large batches, more logging

* remove some logs
2023-06-26 20:14:35 +01:00
spaced4ndy
5e39c47975 Merge pull request #709 from simplex-chat/xftp-send
xftp: recoverable send
2023-04-13 20:13:04 +04:00
Evgeny Poberezkin
2edc7529f9 xftp: client function to create new chunk (#712)
* xftp: client function to create new chunk

* remove chunk digest update

* rename
2023-04-07 14:50:38 +01:00
spaced4ndy
3533929905 xftp: transform SndFile to recipient descriptions; non optional chunk digest (#710) 2023-04-07 14:29:07 +04:00
spaced4ndy
d8e60ecfdb xftp: recoverable send - spike (#707) 2023-04-07 11:19:50 +04:00
Evgeny Poberezkin
b1489138ea xftp: fix file reception on 32 bit platforms (#708) 2023-04-06 20:40:52 +01:00
Evgeny Poberezkin
c5eb65fec8 xftp: send events for file sending/receiving progress (#697) 2023-03-27 11:44:15 +01:00
Evgeny Poberezkin
2e02ba86fd add stateTVar import 2023-03-18 13:38:20 +00:00
spaced4ndy
9255d48b69 xftp: move delay to replica (#688) 2023-03-16 18:12:38 +04:00
Evgeny Poberezkin
bab689099f xftp: streaming file encryption/decryption to avoid memory spikes (#687)
* xftp: streaming file decryption to avoid memory spikes

* refactor, enable tests

* streaming encryption

* refactor
2023-03-16 13:57:21 +00:00
Evgeny Poberezkin
a0eb53b891 xftp: use agent servers in experimental send, refactor decryption (#686)
* xftp: use agent servers in experimental send, refactor decryption

* fix/enable xftp send test test

* allow any order
2023-03-14 22:56:19 +00:00
Evgeny Poberezkin
ddc2da8fe4 xftp: agent API for xftp commands and events (#684)
* xftp: agent API for xftp commands and events

* fix tests

* fix tests 2

* xftp: update agent send api

* update API to make temp path optional

* revert tmp path changes (fixes send)

---------

Co-authored-by: spacedandy <8711996+spaced4ndy@users.noreply.github.com>
2023-03-13 09:04:23 +00:00
Evgeny Poberezkin
3fe1f3c822 rename FileParty constructors 2023-03-11 10:05:07 +00:00
spaced4ndy
c1266d55a4 xftp: experimental send api wip (#682) 2023-03-10 20:43:10 +04:00
Evgeny Poberezkin
552759018e xftp: server types for any protocol (#672) 2023-03-09 09:54:31 +00:00
spaced4ndy
cf147397a4 xftp: agent receive file (#637)
* xftp: agent receive file draft

* receiveFile more stubs, changes to types, schema

* cabal file

* comments

* xftp_server_id

* schema changes, get client, local worker/action

* agent env, save file description, adjust schema

* client stubs

* download chunk wip, store, schema, types

* remove commented code

* read file description, schema, types

* check received, decrypt

* remove pure

* todo

* add XFTP to agent client

* add user id

* agent test

* tests

* rename supervisor into agent

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-03-03 18:50:16 +04:00
Evgeny Poberezkin
94de01a06f xftp: use SHA256 digest for file chunks to reduce file description size (#666)
* xftp: use SHA256 digest for file chunks to reduce file description size

* enable all tests
2023-03-01 14:36:55 +00:00
Evgeny Poberezkin
faec32dc5e xftp: constrain supported file size to 1gb 2023-03-01 13:50:08 +00:00
Evgeny Poberezkin
6227215c3f xftp: remove file description that cannot be used (#665)
* xftp: remove file description that cannot be used

* show sender file description first

* change order

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2023-03-01 13:08:59 +00:00
Evgeny Poberezkin
3d3eb335e8 xftp: download timeout, reconnect CLI on timeouts, verbose logging (#664)
* xftp: download timeout, reconnect CLI on timeouts, verbose logging

* typo

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2023-03-01 07:47:27 +00:00
Evgeny Poberezkin
875a4bac5e xftp cli: use 3 chunk sizes (256kb, 1mb, 4mb) (#660)
* xftp cli: use 3 chunk sizes (256kb, 1mb, 4mb)

* hardcode default servers

* disable rand command

* hide rand command instead of removing
2023-02-27 21:15:22 +00:00
Evgeny Poberezkin
2f15ce2662 xftp: add quota param to server CLI, restrict chunk sizes (#659)
* xftp: add quota param to server CLI

* only allow certain file sizes, fix tests
2023-02-27 18:01:18 +00:00
Evgeny Poberezkin
781f8e0000 xftp: increase timeouts for file uploads, show progress (#658)
* xftp: increase timeouts for file uploads, show progress

* refactor

* use default port 443

* typo

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* upload/download to/from chosen servers concurrently

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2023-02-27 13:42:15 +00:00
Evgeny Poberezkin
8c8a984c1c xftp: add basic auth (#653) 2023-02-24 17:40:26 +04:00
Evgeny Poberezkin
4ce4fa3423 xftp: write secret_box auth tag to the end of the file, for efficiency of ecryption/decryption (#650)
* xftp: write secret_box auth tag to the end of the file, for efficiency of ecryption/decryption

* comments
2023-02-23 18:28:20 +00:00
spaced4ndy
fbf0b8b175 xftp: cli - delete file command, acknowledge received chunks (#649) 2023-02-23 16:54:50 +04:00
spaced4ndy
1f394b76a6 xftp: sender file description (#647)
* xftp: sender file description

* tests

* check, info, tests

* test

* refactor

* refactor, rename sender file

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-02-22 17:16:49 +04:00
Evgeny Poberezkin
bbf9e28696 xftp: add server to client encryption layer with cryptobox (#646)
* xftp: add server to client encryption layer with cryptobox

* workarounds for http2

* update http2 commit

* remove TODOs
2023-02-22 09:03:18 +00:00
Evgeny Poberezkin
c970674452 faster lazy secret_box 2023-02-19 10:54:54 +00:00
Evgeny Poberezkin
4a9b5412db xftp: refactor 2023-02-18 23:21:59 +00:00
Evgeny Poberezkin
1ecef81b4d xftp: validate digests of file chunks in client and server 2023-02-18 22:44:44 +00:00
Evgeny Poberezkin
2bf8cba4fd xftp: download chunks to separate files 2023-02-18 20:56:46 +00:00
Evgeny Poberezkin
bd0cc25083 XFTP error type 2023-02-17 22:27:36 +00:00
Evgeny Poberezkin
acdf4f41c5 XFTP: CLI tests, output command results, info command (#642)
* XFTP: CLI tests, output command results

* info command, test

* fix test
2023-02-17 11:38:43 +00:00
spaced4ndy
a3d9ee213d xftp: cli - option to pass servers, choose servers randomly (#641) 2023-02-16 21:06:23 +04:00
spaced4ndy
cf7346d48b xftp: cli - verify file digest (#640) 2023-02-16 21:01:41 +04:00
spaced4ndy
4e8657097c xftp: cli client draft (#635)
* xftp: cli client draft

* more stubs

* compiles

* hash, app

* options parsers, random

* tmp

* xftp CLI client agent, simplify CLI command syntax

* only allow argument as a second parameter

* pivot signature draft

* receive file

* pivot sent chunks to recipients

* encryptFile - temp, chunks, specs

* send (upload) file and save file descriptions

* refactor, remove encrypted file

* save file size in description as string

* include filename inside padded encrypted file

* call chunk uploads concurrently, using queueing in HTTP2 as library client does not support concurrent streaming uploads

* download file (does not work yet)

* add digests to sent chunks

* fix recv - save file using AppendMode

* encrypt/decrypt sent file with secretbox

* remove print

* fix file description parsing in tests

* fix test

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-02-16 14:06:03 +04:00
Evgeny Poberezkin
1d78e35688 XFTP modules structure 2022-12-27 21:54:02 +00:00