Commit Graph

128 Commits

Author SHA1 Message Date
Alexander Bondarenko ff1d7f2fac ntf-server: break processing transaction (#1067)
* ntf-server: break processing transaction

* make resubscribe sync

* diff

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2024-03-25 15:35:33 +00:00
Evgeny Poberezkin dd2bd11584 parameterize version scopes with phantom types (#1026)
* parameterize version scopes with phantom types

* move Version to another module

* parens
2024-03-04 19:06:51 +00:00
Evgeny Poberezkin 89140e0ff0 use package version for all servers (#1001) 2024-02-18 08:56:35 +00:00
Evgeny Poberezkin 51be2fea20 agent: test NTF server (#1000) 2024-02-17 14:55:40 +00:00
Evgeny Poberezkin 416f1b1721 smp: command authorization (#982)
* smp: command authorization

* fix encoding, most tests

* remove old tests

* authorize via crypto_box

* extract authenticator to Crypto module

* make TransmissionAuth Maybe

* rfc

* support authenticators in NTF protocol, test matrix (no backwards compatibility yet from new clients to old servers)

* fix/add tests, add version config to "small" agent

* separate client and server versions for SMP protocol

* test batching SMP v7

* do not send session ID in each transmission

* refactor auth verification in the server, split tests

* server "warm up" fixes timing test

* uncomment SUB timing test

* comments, disable two timing tests

* rename version

* increase auth timing test failure threshold

* use different algorithms to authorize snd/rcv commands, use random correlation ID

* transport: fetch and store server certificate (#985)

* THandleParams (WIP, does not compile)

* transport: fetch and store server certificate

* smp: add getOnlinePubKey example to smpClientHandshake

* add server certs and sign authPub

* cleanup

* update

* style

* load server certs from test fixtures

* sign ntf authPubKey

* fix onServerCertificate

* increase delay before sending messages

* require certificate with key in SMP server handshake

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>

* remove dhSecret from THandle

* remove v8, merge all changes to one version

* parameterize THandle

* rfc: transmission ecnryption

* Revert "parameterize THandle"

This reverts commit 75adfc94fb.

* use batch syntax for ntf server commands

* separate encodeTransmission when there is no key

* typo

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* rename

* diff

---------

Co-authored-by: Alexander Bondarenko <486682+dpwiz@users.noreply.github.com>
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2024-02-16 11:45:54 +00:00
Evgeny Poberezkin 9254d8dac5 v5.5.3 2024-02-16 11:33:56 +00:00
Alexander Bondarenko 9ab34bca7d cli: add cert command to xftp and ntf servers (#991)
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2024-02-16 10:56:54 +00:00
Evgeny Poberezkin e64b6cba4b 5.5.2.0 2024-02-10 22:07:02 +00:00
Evgeny Poberezkin 1e49f1c92d 5.5.1.0 2024-01-19 19:47:39 +00:00
Evgeny Poberezkin 8ff89c19dc servers: do not keep stats file open during delays (#974) 2024-01-19 17:47:47 +00:00
Evgeny Poberezkin f6ed4640d4 5.5.0.6 2024-01-16 17:07:15 +00:00
Evgeny Poberezkin 7f7a77c4eb SMP protocol: optimize batching transactions, remove Builder (#961)
* remove Builder

* fewer chunks

* remove lazy bytestrings

* optimize

* pad
2024-01-14 20:42:47 +00:00
Evgeny Poberezkin ad8cd1d515 5.5.0.5 2024-01-09 19:44:15 +00:00
Evgeny Poberezkin 55808b0c82 5.5.0.4 2024-01-06 11:40:15 +00:00
Evgeny Poberezkin d0588bd0ac 5.5.0.2 2023-12-27 20:35:56 +00:00
Evgeny Poberezkin a57066a826 5.5.0.1 2023-12-23 13:42:22 +00:00
Evgeny Poberezkin 13a60d1d39 use ChaChaDRG as the source of randomness (#920)
* use ChaChaDRG as the source of randomness

* remove functions using entropy directly

* comment
2023-12-21 00:12:08 +00:00
Evgeny Poberezkin 7627ce6b69 5.5.0-beta.0: SMP server/agent 5.5, XFTP server 1.2, NTF server 1.7 2023-12-19 16:07:11 +00:00
Alexander Bondarenko 7c27357eb2 server: stop inactive clients without subscriptions, additional monitoring via control port (#901)
* fix thread labelling points

* add subscription thread labels

* add explicit sections for sockets/sessions/disconnects

* adjust label threads wrt. next blocking op

* WIP: emit event on server finish

So the event log wouldn't be cut short giving
false impression of unterminated sessions.

* add handshake timeout

* hack server socket stats

* trace closeTLS exceptions

* fix build

* clean up disconnectTransport

* make disconnectTransport terminating

Thus, participating in racyAny_ around it.

* trach snd/rcv time separately, log clients as CSV

* add direct client counting/stats

* add csv dump for socket threads

* guard socket threads dump from older GHC

* cut socket thread listing into socket-threads command

* store client creation time for stats

* wrap tls setup in timeout and labels

* server: expire clients without subscriptions

* fix check for subscriptions

* remove excess tracing

* move hardcoded timeouts to configs

* cleanup

* fix tests

* wrap direct queue waits in timeout

* rewrite old TCP wrappers with their SocketState variants

* add testInactiveWithSubs

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-12-19 15:50:45 +00:00
Alexander Bondarenko 6bffcc8503 compatibility with GHC 8.10.7 (#906)
* build with ghc-8.10

* add 8.10 to matrix

* cleanup

* stats

* remove getField

* agent

* server

* remote

* space

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-11-29 12:53:06 +00:00
Evgeny Poberezkin 8fa369f9d3 5.4.0: xftp 1.1.3, ntf 1.6.4 (#903)
* 5.4.0: xftp 1.2.0, ntf 1.7.0

* upgrade ghc in github job

* change xftp, ntf versions

* shorter predicate for exception in test
2023-11-26 22:20:42 +00:00
Evgeny Poberezkin 9460551a04 5.4.0.4: xftp 1.1.2, ntf 1.6.3 2023-11-11 09:16:40 +00:00
Evgeny Poberezkin 511d793b92 derive JSON instances with TH (#869)
* derive JSON instances with TH

* defaultJSON

* remove ToJSON
2023-10-26 09:20:30 +01:00
Evgeny Poberezkin cf8b9c12ff remove test dependencies (#861)
* remove test dependencies

* .hlint.yaml
2023-10-22 10:50:36 +01:00
Evgeny Poberezkin 9d12d76078 add fourmolu (#868)
* add fourmolu

* linebreak

* simplify

* comment
2023-10-22 09:20:14 +01:00
Alexander Bondarenko 96a38505d6 add FromJSON instances (#856)
* Add FromJSON instances

* add missing FromJSON instances

* more JSON instances

* update comments

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-10-05 15:57:53 +01:00
Evgeny Poberezkin 57d08c0e36 Merge branch 'master' into master-ghc9 2023-09-17 22:30:46 +01:00
Evgeny Poberezkin 00b48babf3 5.4.0.3: ntf 1.6.2 2023-09-17 22:23:00 +01:00
Evgeny Poberezkin 0b488ceb66 ntf server: faster restoring from store log (#846)
* ntf server: faster restoring from store log

* only log sub status when changed
2023-09-17 15:16:24 +01:00
Evgeny Poberezkin 887ccbcf6c Merge branch 'master' into master-ghc9 2023-09-10 18:00:33 +01:00
Evgeny Poberezkin 0cabe0690b 5.4.0.2: xftp 1.1.1, ntf 1.6.1 2023-09-10 17:44:37 +01:00
Moritz Angermann 002f36dde0 5.4.0.0: use ghc 9.6.2 (#791)
* Add 9.6 compat

* compile with GHC9.6.2: dependencies, imports, code

* refactor typeclasses

* refactor record dot

* update cabal version

* update github actions

* update direct-sqlcipher

* 5.4.0.0

* update cabal.project

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-08-24 20:22:02 +01:00
Evgeny Poberezkin 745a144e0c 5.2.1 (ntf 1.5.1, xftp 1.0.1) (#799) 2023-07-13 19:23:06 +01:00
Evgeny Poberezkin 4c6c436e7f resubscribe in batches in Client.Agent (#798)
* resubscribe in batches in Client.Agent

* fix test
2023-07-12 07:35:38 +01:00
Evgeny Poberezkin 54dc8d42e7 5.2.0, ntf server 1.5.0 (#790) 2023-07-03 13:32:58 +01:00
Evgeny Poberezkin 94540a2c71 servers: add TCP timeouts to avoid memory leaks (#776)
* servers: add TCP timeouts to avoid memory leaks

* fix tests

* only use RecvTimeOut

* servers: simple timeout for TCP transport

* revert dependency change

* simplify

* simplify

* simplify 2
2023-06-30 16:22:01 +01:00
Evgeny Poberezkin 4a927d1ae2 ntf server: better batching and logging (#780)
* ntf server: better batching and logging

* reduce batch delay for ntf server

* comments

* 5.1.3, ntf 1.4.2

* more logging

* more logging

* split large batches, more logging

* remove some logs
2023-06-26 20:14:35 +01:00
Evgeny Poberezkin e3fff98b1b 5.1.2, ntf server 1.4.1 2023-06-25 09:31:55 +01:00
Evgeny Poberezkin acea477aab batch subscriptions for ntf server (#778)
* return updated ConnectionStats from switchConnectionAsync (#777)

* batch ntf server subscriptions to SMP servers

* refactor

* fix filtering queues in reconnect

* test, remove repeated grouping

* enable all tests

* reduce subscriptions count

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2023-06-25 08:27:50 +01:00
Evgeny Poberezkin a83f85dc58 5.1.0: smp 1.4.0, xftp 1.0.0 2023-05-12 22:01:32 +01:00
spaced4ndy bdff274f56 use Int64 for time calculations (#706) 2023-04-04 14:51:13 +04:00
Tristan de Cacqueray 5743069021 Remove duplicated serverIdentity field (#491) 2023-03-26 12:37:31 +01:00
Evgeny Poberezkin 02eb1e9822 add stateTVar imports for compatibility with older hackage index in nix builds 2023-03-18 07:57:25 +00:00
spaced4ndy 9255d48b69 xftp: move delay to replica (#688) 2023-03-16 18:12:38 +04:00
Evgeny Poberezkin 781f8e0000 xftp: increase timeouts for file uploads, show progress (#658)
* xftp: increase timeouts for file uploads, show progress

* refactor

* use default port 443

* typo

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* upload/download to/from chosen servers concurrently

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2023-02-27 13:42:15 +00:00
Evgeny Poberezkin bbf9e28696 xftp: add server to client encryption layer with cryptobox (#646)
* xftp: add server to client encryption layer with cryptobox

* workarounds for http2

* update http2 commit

* remove TODOs
2023-02-22 09:03:18 +00:00
Evgeny Poberezkin 15637e545b Merge branch 'master' into xftp 2023-02-17 21:40:35 +00:00
Evgeny Poberezkin 2ddfb044fc parameterize protocol by error type (#644) 2023-02-17 20:46:01 +00:00
Stanislav Dmitrenko bccef0ba47 files: server and client spike - basic upload/download (#591)
* Files: main, env, stats, storeLog

* Better + transport

* Executable

* Env

* Update Client.hs, Server.hs, and 4 more files...

* Answer on request

* Delay

* Temp file

* Bypass cert check

* update package.yml, rename

* update store log

* extend HTTP2 transport

* refactor caStore

* HTTP2 body

* update server stats

* file server/client framework

* verify server commands

* process FNEW command, CLI test works

* simple XFTP server test (fails)

* fix test, refactor

* upload chunk works

* receive file chunk in the client

* remove transport handshake

* typo

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* fix names

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2023-02-13 13:36:02 +00:00
Evgeny Poberezkin eb1723acce refactor transport encoding, make client IO error "temporary" (#634)
* file server/client framework

* refactor

* make IOError temporary

* fix test
2023-02-10 19:06:28 +00:00