Commit Graph

585 Commits

Author SHA1 Message Date
Evgeny Poberezkin c970674452 faster lazy secret_box 2023-02-19 10:54:54 +00:00
Evgeny Poberezkin fcef7deec0 Merge branch 'master' into xftp 2023-02-18 18:46:28 +00:00
Evgeny Poberezkin 6a48b735d1 increase timeout for QUOTA errors 2023-02-18 18:09:58 +00:00
Evgeny Poberezkin 15637e545b Merge branch 'master' into xftp 2023-02-17 21:40:35 +00:00
spaced4ndy c0dcf283eb server_key_hash fields (#643)
* server_key_hash fields

* test

* refactor

* fix

* order

* use sync command in test

* refactor

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-02-17 21:24:32 +00:00
Evgeny Poberezkin 2ddfb044fc parameterize protocol by error type (#644) 2023-02-17 20:46:01 +00:00
spaced4ndy 4e8657097c xftp: cli client draft (#635)
* xftp: cli client draft

* more stubs

* compiles

* hash, app

* options parsers, random

* tmp

* xftp CLI client agent, simplify CLI command syntax

* only allow argument as a second parameter

* pivot signature draft

* receive file

* pivot sent chunks to recipients

* encryptFile - temp, chunks, specs

* send (upload) file and save file descriptions

* refactor, remove encrypted file

* save file size in description as string

* include filename inside padded encrypted file

* call chunk uploads concurrently, using queueing in HTTP2 as library client does not support concurrent streaming uploads

* download file (does not work yet)

* add digests to sent chunks

* fix recv - save file using AppendMode

* encrypt/decrypt sent file with secretbox

* remove print

* fix file description parsing in tests

* fix test

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-02-16 14:06:03 +04:00
Evgeny Poberezkin d8f81b5cdb Merge branch 'master' into xftp 2023-02-15 22:02:19 +00:00
Evgeny Poberezkin 2ae3100bed lazy pad/unpad, secretbox encrypt/decrypt (#639) 2023-02-15 22:01:33 +00:00
Evgeny Poberezkin 88c6b0d663 Merge branch 'master' into xftp 2023-02-15 12:51:34 +00:00
Evgeny Poberezkin 8659d4de05 NaCL secretbox based on cryptonite (#636) 2023-02-15 12:51:08 +00:00
Stanislav Dmitrenko bccef0ba47 files: server and client spike - basic upload/download (#591)
* Files: main, env, stats, storeLog

* Better + transport

* Executable

* Env

* Update Client.hs, Server.hs, and 4 more files...

* Answer on request

* Delay

* Temp file

* Bypass cert check

* update package.yml, rename

* update store log

* extend HTTP2 transport

* refactor caStore

* HTTP2 body

* update server stats

* file server/client framework

* verify server commands

* process FNEW command, CLI test works

* simple XFTP server test (fails)

* fix test, refactor

* upload chunk works

* receive file chunk in the client

* remove transport handshake

* typo

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* fix names

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2023-02-13 13:36:02 +00:00
Evgeny Poberezkin d7a008985b Merge branch 'xftp' of github.com:simplex-chat/simplexmq into xftp 2023-02-12 15:53:48 +00:00
spaced4ndy 9775861b6f files: file description parsing & serialization, tests (#633)
* files: file description parsing & serialization, tests

* chunk list

* processFileDescription

* name

* different values

* undo chunk objects to strings

* parser, serializer

* use Either as monad

* refactor

* remove comments

* fix 2 tests

* fix test

* enable all tests

* remove comment

* move

---------

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>
Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-02-12 15:53:37 +00:00
Evgeny Poberezkin 4d333a90fa Merge branch 'master' into xftp 2023-02-10 19:07:12 +00:00
Evgeny Poberezkin eb1723acce refactor transport encoding, make client IO error "temporary" (#634)
* file server/client framework

* refactor

* make IOError temporary

* fix test
2023-02-10 19:06:28 +00:00
Evgeny Poberezkin eff501ddcc Merge branch 'master' into xftp 2023-02-09 23:10:46 +00:00
Evgeny Poberezkin b342b1dc59 extend HTTP2 transport (#632)
* extend HTTP2 transport

* refactor caStore

* HTTP2 body

* enable test

* remove maxBodySize
2023-02-09 23:09:24 +00:00
JRoberts 1523c00bcc files: file description types (#631) 2023-02-09 21:22:58 +04:00
Evgeny Poberezkin 360e55a48f Merge branch 'master' into xftp 2023-02-08 20:57:24 +00:00
Evgeny Poberezkin ee2a764f93 agent: more strict parsing of TransportHost, tests (#629) 2023-02-04 22:59:28 +00:00
Evgeny Poberezkin cfe995325a agent: prevent batch deletions from failing on one connection sql error (#628)
* agent: prevent error reading one connection failing batched subscription

* prevent batch deletions from failing on one connection sql error

* rename
2023-02-04 20:46:45 +00:00
Evgeny Poberezkin 5d8febc353 avoid possible race conditions when cancelled clients/asyncs can be removed after the new ones are added (so that the new are removed as well) (#627) 2023-01-30 17:46:06 +00:00
Evgeny Poberezkin 6858f74897 Merge branch 'master' into xftp 2023-01-27 18:00:11 +00:00
Evgeny Poberezkin 7fb48930f6 increase database transaction timeout, add test for user deletion (#622)
* increase database transaction timeout, add test for user deletion

* more test logs
2023-01-25 20:58:03 +00:00
JRoberts ff038b492c fix DEL_RCVQ queue parameter type (#621) 2023-01-24 15:59:32 +00:00
Evgeny Poberezkin b59669a65e refactor functions operating on single connections using batch functions (#620)
* refactor functions operating on single connections using batch functions

* update tests
2023-01-24 13:14:13 +00:00
Evgeny Poberezkin 2ccef1690b events sent by the agent when queue, connection and user are deleted asynchronously (#619)
* events sent by the agent when queue, connection and user are deleted asynchronously

* update test

* more tests

* add user tests

* queries

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>
2023-01-24 11:12:07 +00:00
Evgeny Poberezkin d4fc638478 async batch connection deletion (#617)
* async batch connection deletion

* delete user record with connections, async connection deletion

* updates

* update query
2023-01-21 13:41:16 +00:00
Evgeny Poberezkin f66e8239f4 Merge branch 'master' into users 2023-01-19 17:45:31 +00:00
Evgeny Poberezkin 56eea29ec3 end SMP client connection after configured number of PING errors, reset PING error count on any success (#615)
* end SMP client connection after configured number of PING errors, reset PING error count on any success

* only terminate client on PING timeout

* refactor

* comment
2023-01-19 17:45:06 +00:00
Evgeny Poberezkin a854b5f2b2 batched queue deletion (#611)
* batched queue deletion

* only re-connect client on subscribe errors, not in all batch commands

* add test, fix

* correction

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>

* refactor

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>
2023-01-19 16:53:28 +00:00
Evgeny Poberezkin 63b7956083 Merge branch 'master' into users 2023-01-18 22:20:29 +00:00
Evgeny Poberezkin 4f4b4f7eee only reconnect client on temporary errors in single subscription too (not currently used) (#616)
* only reconnect client on temporary errors in single subscription too (not currently used)

* refactor
2023-01-18 22:03:43 +00:00
Evgeny Poberezkin f9c613ffb8 Merge branch 'master' into users 2023-01-18 21:18:42 +00:00
Evgeny Poberezkin 14cb88e725 remove completed async action handles from memory (#614)
* remove completed async action handles from memory

* name

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>

Co-authored-by: JRoberts <8711996+jr-simplex@users.noreply.github.com>
2023-01-18 18:22:17 +00:00
Evgeny Poberezkin 6ccbe5e66e retry unsuccessful subscriptions in case of temporary errors (#613)
* retry unsuccessful subscriptions in case of temporary errors

* do not send DOWN if connection has any active queues
2023-01-18 14:30:25 +00:00
Evgeny Poberezkin c4f377a85b fix SMP server stats (#612)
* fix SMP server stats

* add server stats test

* fix test
2023-01-18 11:07:25 +00:00
Evgeny Poberezkin ca5c4b75eb Merge branch 'master' into users 2023-01-17 20:32:23 +00:00
JRoberts 42df6a421d add missing fkey indexes (#610) 2023-01-17 16:07:13 +04:00
Evgeny Poberezkin bc8acf2abf Merge branch 'master' into xftp 2023-01-16 19:29:55 +00:00
Evgeny Poberezkin 324e01300d execute asynchronous commands for correct users, include session in messages from SMP clients to agent (#608)
* execute asynchronous commands for correct users

* pass transport session to message processing to avoid race condition

* account for server changes when determining unused servers

* enable one test

* enable all tests, remove log
2023-01-16 11:06:19 +00:00
Evgeny Poberezkin b549910c17 support switching session mode, transport session now uses connection ID (#607)
* support switching session mode, transport session now uses connection ID

* refactor
2023-01-16 08:51:41 +00:00
Evgeny Poberezkin a1eda12475 increase sending retries on ERR QUOTA 2023-01-15 18:14:20 +00:00
Evgeny Poberezkin cac3432b03 Merge branch 'master' into users 2023-01-14 15:34:18 +00:00
Evgeny Poberezkin 56cc2bc71f additional SMP server stats (#605)
* additional SMP server stats

* refactor
2023-01-14 13:23:37 +00:00
Evgeny Poberezkin 9c9ba8c25c subscribe users in different sessions (#602)
* subscribe users in different sessions

* remove comments

* include userId to rcv queue map key

* use hash of userId[:entityId] as SOCKS proxy username
2023-01-13 09:58:39 +00:00
JRoberts 8e024590bc fix createUserRecord query (#604) 2023-01-13 11:31:35 +04:00
Evgeny Poberezkin 5e31c60c35 do not show warning on server restart when restoring "quota" message and quota is exceeded (#603)
* do not show warning on server restart when restoring "quota" message and quota is exceeded

* complete case

* line break
2023-01-12 23:26:21 +00:00
Evgeny Poberezkin a0a092747a Merge branch 'master' into users 2023-01-12 16:37:59 +00:00