Commit Graph

605 Commits

Author SHA1 Message Date
Evgeny dab1980d79 xftp: report receive file error with redirected file ID, when redirect is present (#1304)
* xftp: report receive file error with redirected file ID, when redirect is present

* fix test
2024-09-09 08:08:16 +01:00
Evgeny 344a295845 agent: error when user record is not in database (#1303) 2024-09-08 16:50:22 +01:00
Evgeny d5efe3406a agent: fix race when sending a message to the deleted connection (#1296) 2024-09-02 15:07:16 +01:00
Evgeny ce6777b68d newtype for server entity IDs, fix TRcvQueues (#1290)
* put DRG state to IORef, split STM transaction of sending notification (#1288)

* put DRG state to IORef, split STM transaction of sending notification

* remove comment

* remove comment

* add comment

* revert version

* newtype for server entity IDs, fix TRcvQueues

* Revert "put DRG state to IORef, split STM transaction of sending notification (#1288)"

This reverts commit 517933d189.

* logServer
2024-08-30 12:50:02 +01:00
Evgeny Poberezkin aa60bd6770 use strict Maps, fix stats for sent END batches 2024-08-26 19:35:57 +01:00
Evgeny 53e829a21c agent: store query errors, reduce slow query threshold to 1ms (#1277)
* agent: collect query errors stats

* simplify

* test

* use microseconds

* parens

* revert change to track all queries, reduce threshold to 1 ms
2024-08-24 14:51:26 +01:00
Evgeny ab4acadcc1 agent: fix stuck connection commands when server is not responding (#1276)
* agent: fix stuck connection commands when server is not responding

* fix

* rename

* test SKEY retry
2024-08-24 13:54:20 +01:00
Diogo 1cbf8c0015 agent: support changing user of the new connection (#1267)
* agent: support new connection user id update

* another way for assertion

* add more tests to setConnUserId

* remove fdescribes

* allow rcv connection to change user id

* add functional test to api

* remove fdescribe

* refactor

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2024-08-20 10:57:09 +01:00
Evgeny 571d148bdf agent: do not start delivery workers when there are no messages to deliver (#1263)
* agent: use weak ThreadId and forkIO in workers instead of async (reduce memory)

* agent: do not start and exit delivery workers when there are no messages to deliver (#1264)

* agent: exit delivery workers when no messages to deliver

* only start delivery workers when there are pending messages

* fix

* focus test

* enable all tests

* lift

* do not exit workers when there is no work
2024-08-18 21:30:06 +01:00
Evgeny f229e135e3 agent: reduce subscription transactions (#1259)
* agent: reduce subscription transactions

* nub

* remove commented
2024-08-16 10:06:22 +01:00
Evgeny 47dd0c4c64 agent: do not send worker crash errors when agent is not active (stopped) (#1256) 2024-08-09 21:10:27 +01:00
Evgeny 7762b4d364 reduce the number of STM transactions (#1255)
* reduce the number of STM transactions

* reduce STM more

* refactor

* remove more
2024-08-09 16:38:46 +01:00
Evgeny f4d1a33ed8 agent: retry loop that resumes subscriptions as soon as agent is moved to foreground, suspend retry loops while agent is suspended (#1249)
* agent: retry loop that resumes subscriptions as soon as agent is moved to foreground, suspend retry loops while agent is suspended

* reset retry enterval when moving to foreground

* account for network state too

* simplify

* typo

* simplify
2024-08-05 15:09:46 +01:00
Evgeny Poberezkin 3753379ae4 agent: support dababase access with high priority for more responsive user actions (#1248)
* agent: support dababase access with high priority for more responsive user actions

* rename, export

* fix
2024-07-29 18:57:16 +01:00
Evgeny Poberezkin 9093c7b120 agent, ntf server: only mark subscriptions as pending that were created by the disconnected client (#1242)
* ntf server: only mark subscriptions as pending if the disconnected client is current

* add sessionId to subscribed queue

* add sessionId to subscriptions in ntf server agent

* fix
2024-07-25 13:07:28 +01:00
Evgeny Poberezkin 47ae921b86 use strict Map (#1241) 2024-07-23 08:27:54 +01:00
spaced4ndy c605156302 agent: getAgentSubsTotal api (#1238)
* agent: getAgentSubsTotal api

* move

* export

* count sessions

* simplify

* simplify 2

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2024-07-22 15:43:38 +01:00
Evgeny Poberezkin 051bf38bc7 agent: remove delays in notification processing, add notification stats (#1235)
* agent: remove delays in notification processing, add notification stats

* do not replace token after failed check

* more stats

* refactor

* fix

* backwards compatible JSON parsing

* retry deleting subscription on temporary error

* remove attempts to get multiple notification messages

* fix JSON decoding to be backwards compatible
2024-07-22 15:42:34 +01:00
spaced4ndy 7565ddd91c agent: initialize ratchet on processing confirmation to support decryption of messages received before allowConnection; return SndQueueSecured from joinConnection, acceptContact (#1233) 2024-07-18 19:54:14 +04:00
Evgeny Poberezkin 8dd54ced0e agent: retry proxied command on NO_SESSION error, to prevent failure of proxied interactive commands (like joining connection) (#1227) 2024-07-13 10:06:48 +01:00
Evgeny Poberezkin e56bd0b47b agent: add known servers (#1225)
* agent: add known servers

* test delays

* ServerCfg

* json encoding

* enabledServerCfg

* checkUserServers
2024-07-12 12:41:55 +01:00
Evgeny Poberezkin 9d0774a58e agent: add queue information (#1217)
* agent: add queue information to "debug delivery" response

* fix test

* rename

* encodings
2024-07-03 19:32:27 +01:00
spaced4ndy ae8e1c5e9a agent: servers stats improvements, fixes (#1208)
* agent: reset stats startedAt time in memory

* getAgentSubsSummary

* change sub counting

* ack statistics

* add import

* instance

* Revert "instance"

This reverts commit 1f63740d56.

* Revert "add import"

This reverts commit ef72df8014.

* modify sub counting

* modify conn creation counting

* use int64

* file size stats

* remove import

* ack err counting

* conn del stats

* format

* new data

* add data

* toKB

* restore connCompleted

* use Int for counts

* use rq from scope

* remove getAgentSubsSummary

* fix connCompleted

* fix

* revert disabling stats

* use srv from scope

* combine ack stats

* modify

* comment

* count subs

* refactor

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2024-07-03 12:36:15 +01:00
Evgeny Poberezkin 5c839b40c9 Merge branch 'master' into fast-connection 2024-07-01 21:14:32 +01:00
spaced4ndy a50e2e74a5 agent: disable saving stats (#1214)
* agent: disable stat saving

* disable migration

* schema

* disable restore
2024-07-01 21:06:38 +01:00
Evgeny Poberezkin a99ce6122c secure queue by sender via proxy (proxy SKEY command) (#1210)
* client: secure queue by sender via proxy (proxy SKEY command)

* agent and server: proxy SKEY command
2024-06-30 16:20:54 +01:00
Evgeny Poberezkin a6b542b301 Merge branch 'master' into fast-connection 2024-06-30 15:00:28 +01:00
Evgeny Poberezkin 6a54a58a0d agent: remove legacy statistics (#1211)
* agent: remove legacy statistics

* delays after disposeAgent

* delays

* enable all tests

* more delays
2024-06-30 12:50:42 +01:00
Evgeny Poberezkin 9ee684b0f4 rfc: faster handshake protocol (#1203)
* rfc: faster handshake protocol

* update

* 1 message

* SKEY

* use SKEY for both parties

* test

* update doc

* NEW command parameter

* add k=s param to queue URI

* fix

* add sndSecure field to queues

* make sender key non-optional in SndQueue (WIP, tests fail)

* fast handshake sometimes works (many tests fail)

* correctly handle SKEY retries, avoiding to re-generate the keys

* handle SKEY retries during async connection

* fix most tests (1 test fails)

* remove do

* fix contact requests encoding/tests

* export

* fix: ignore duplicate confirmations, fixes testBatchedPendingMessages

* do not store sndSecure in store log if it is false to allow server downgrade

* add connection invitation encoding tests
2024-06-30 08:36:24 +01:00
spaced4ndy c788692687 agent: servers summary types, api (#1202)
* agent: servers summary types, api [wip]

* encoding

* export

* Revert "export"

This reverts commit cd9f315fe8.

* comment

* rename

* simplify types

* uncomment

* comment

* rework

* comment, exports

* save, restore stats wip

* remove

* rename

* save stats periodically

* sigint, sigterm experiments

* corrections

* remove some proxy stats

* increase stat

* proposed stats

* fields

* Revert "sigint, sigterm experiments"

This reverts commit f876fbd418.

* wip

* retries -> attempts

* errs

* fix

* other errs

* more stat tracking

* sub stats

* remove xftp successes stats

* xftp stats tracking

* revert

* revert

* refactor

* remove imports

* comment

* Revert "refactor"

This reverts commit 26c368d82a.

* Revert "revert"

This reverts commit 4c9e3753b5.

* Revert "revert"

This reverts commit 6f65644053.

* todos

* persistence

* rename, fix

* config

* comment

* add started at to summary

* delete stats on user deletion

* reset api

* move

* getAgentServersSummary collect state logic

* corrections

* corrections

* remove

* rework

* decrease contention

* update

* more stats

* count sentProxied

* count subs

* remove unused

* comment

* remove comment

* comment

* export

* refactor

* cleanup

* intervals

* refactor

* refactor2

* refactor3

* refactor4

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2024-06-25 09:42:59 +04:00
Evgeny Poberezkin d47c099ac9 docs: update protocol specifications (#1204)
* docs: update protocol specifications

* update SMP protocol (WIP)

* add proxy protocol commands and responses, amend envelope sizes in SMP protocol

* docs: update XFTP protocol (#1205)

* docs: update XFTP protocol

* commands

* fix table of contents, move sections

* add about ids

* download encryption

* qualities

* diagram

* crypto

* sending file diagram

* fix svg

* receiving file diagram

* update commands

* update handshake

* Add updated XRCP (#1207)

* add XRCP protocol

* add ToC

* update

* update

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>

* add XFTP handshake description

* update agent protocol

* fast duplex connection in agent protocol

* update agent protocol

* update overview

* typos

* queue rotation, agent API, updates

* push notifications specification

* add XRCP threat model

* XFTP threat model

* update PQDR

* agent protocol end-to-end encryption

* versions

* remove TOC details

* update

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
Co-authored-by: Alexander Bondarenko <486682+dpwiz@users.noreply.github.com>
2024-06-23 22:53:45 +01:00
spaced4ndy 4bbadffa37 agent: api to reconnect single server (#1198)
* agent: api to reconnect single server

* refactor

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2024-06-16 06:27:25 -07:00
spaced4ndy 3c0cd7efcc agent: separate type for agent file errors (#1185) 2024-06-05 18:44:32 +04:00
Evgeny Poberezkin 3d605310ed agent: remove protocol encodings and agent TCP server (#1189)
* rfc: remove agent protocol encodings

* agent: remove protocol encodings and agent TCP server

* update

* remove unused code

* remove

* move tests

* add delay to tests

* stabilize test

* test

* more delays

* reduce delays

* enable all tests

* delays

* style
2024-06-05 14:34:40 +01:00
Evgeny Poberezkin 3dab330480 use throwE instead of throwError (#1187)
* use throwE instead of throwError

* test delay
2024-06-05 11:20:50 +01:00
Evgeny Poberezkin d28b17e787 xftp server: send HTTP2 error as timeout error to the client so it is treated as temporary (#1186)
* xftp server: log file reception error

* report HTTP2 error as timeout error

* reduce timeout to 5 min

* process timeout error in protocol response

* log warning on timeout/HTTP2 error
2024-05-31 22:18:28 +01:00
Evgeny Poberezkin 88f1b727e0 SMP protocol extension to debug subscribed SMP queues (#1181)
* SMP protocol extension to debug subscribed SMP queues

* fix, test

* corrections

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* increase delays

* increase timeout

* delay

* delay

* enable all tests

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2024-05-30 18:49:43 +01:00
spaced4ndy 97a953550f agent: getAgentQueuesInfo (#1180) 2024-05-30 14:21:29 +04:00
Evgeny Poberezkin 5c2c88315a SMP server information (#1072)
* SMP server information

* fix tests

* country codes

* smp-server: serve contact and link pages from static files (#1084)

* smp-server: serve contact and link pages from static files

* generate index

* use params from ini

* render using ServerInformation

* tweak templates

* update

* fix some html

* smp-server: fix layout (#1097)

* smp-server: fix layout

* port fixes to link page

---------

Co-authored-by: Alexander Bondarenko <486682+dpwiz@users.noreply.github.com>

* update server information page

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
Co-authored-by: M. Sarmad Qadeer <MSarmadQadeer@gmail.com>

* update server info

* web: improve server info page design (#1166)

* web: improve server info page design

* web: fix font errors & some tags

* web: improve contact & invitation page layout and header

* update

* remove unused files/css

* cleanup

* fix link page

* remove unused font links

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>

* show contact address as is

---------

Co-authored-by: Alexander Bondarenko <486682+dpwiz@users.noreply.github.com>
Co-authored-by: M. Sarmad Qadeer <MSarmadQadeer@gmail.com>
2024-05-29 11:30:42 +01:00
Evgeny Poberezkin 63f5e76f9c agent: treat absent proxy session as a temporary error to retry sending (#1178)
* agent: treat absent proxy session as a temporary error to retry sending

* enable all tests
2024-05-29 08:06:01 +01:00
Evgeny Poberezkin 2ff5f5a832 agent: add context to CMD error (#1167)
* agent: add context to CMD error

* tests, more warnings

* fix tests

* log TBQueue sizes

* log locks

* more logs

* log sendMessagesB

* fix test

* log length

* refactor

* remove logging

* revert lock scope change

* cleanup

* add string error to A_PROHIBITED

* remove

* remove test limitations

* language
2024-05-24 14:13:01 +01:00
Evgeny Poberezkin e7a73a4c89 Merge branch 'stable' 2024-05-23 22:04:06 +01:00
Evgeny Poberezkin 6309f92c68 agent: fail if non-unique connection IDs are passed to sendMessages (#1170) 2024-05-23 22:01:57 +01:00
Evgeny Poberezkin 875a56b75a Merge branch 'stable' 2024-05-23 16:48:11 +01:00
Alexander Bondarenko 984394d906 core: remove MonadUnliftIO ExceptT orphans (#1169) 2024-05-23 16:44:00 +01:00
Evgeny Poberezkin 2b09ada392 agent: reduce interval for storing server connection errors to 3 seconds 2024-05-22 14:36:54 +01:00
Evgeny Poberezkin 8097df5540 agent: persist server connection error (#1165)
* agent: persist server connection

* comment, refactor

* fix tests, reduce interval for ntf server

* cleanup

* 0
2024-05-22 13:25:49 +01:00
Evgeny Poberezkin 5bff2f5537 Merge branch 'stable' 2024-05-21 23:42:49 +01:00
Evgeny Poberezkin f50589b31a agent: remove external timeout to resubscribe (#1164)
* agent: remove external timeout to resubscribe

* liftIO

* fix tests
2024-05-21 22:52:22 +01:00
spaced4ndy 2e5433676e xftp: check proxy before downloading from unknown server (#1102)
* xftp: check proxy before downloading from unknown server

* corrections

* remove import

* tests

* comment

* tests

* don't wrap into internal error

* fix tests
2024-05-20 14:14:04 +01:00