Commit Graph

447 Commits

Author SHA1 Message Date
Evgeny Poberezkin f097c24e6b functions to backup/restore/remove store 2023-09-30 17:21:40 +01:00
Evgeny Poberezkin ad52fdda7a revert block size change, it is causing corruption in SQLCipher (sqlcipher/sqlcipher#485) 2023-09-30 11:46:54 +01:00
Evgeny Poberezkin ee895a14f5 change block size, checkpoint after vacuum, disable auto-vacuum 2023-09-29 10:08:29 +01:00
Evgeny Poberezkin 5cc8728733 check and change page size when opening connection 2023-09-29 08:25:54 +01:00
Evgeny Poberezkin 459b6d39bf missing import 2023-09-28 17:41:08 +01:00
Evgeny Poberezkin 053c0b513a backup wal files 2023-09-28 16:31:34 +01:00
Evgeny Poberezkin d334a2be79 add toggling WAL mode, doc 2023-09-28 15:54:37 +01:00
Evgeny Poberezkin 3d63e7507c optionally switch database to delete mode when checkpointing 2023-09-28 14:56:08 +01:00
Evgeny Poberezkin 8e520b17d4 add checkpointSQLiteStore 2023-09-28 14:35:24 +01:00
Evgeny Poberezkin 91df416840 Merge branch 'master' into ep/journal-mode-wal 2023-09-28 13:15:26 +01:00
Evgeny Poberezkin fda1284ae4 agent: support closing and re-opening store (#855)
* agent: support closing and re-opening store

* add closed flag, tests
2023-09-27 12:50:24 +01:00
Evgeny Poberezkin 52dc402194 sql: set journal_mode to WAL 2023-09-25 22:49:22 +01:00
Evgeny Poberezkin 887ccbcf6c Merge branch 'master' into master-ghc9 2023-09-10 18:00:33 +01:00
Evgeny Poberezkin 84ce037867 extend SMP protocol to allow creating new queues without subscriptions (#839)
* Trace auto-subs flag

* Replace Bools with SubscriptionMode

* Handle SMOnlyCreate

* Wire remaining todos

* Update tests and fix

* Bump protocol level

* Apply suggestions from code review

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>

* Scrub needs_sub from agent DB

* Scrub a few more needSubs from the agent api

* change API, fix test

* agent: do not subscribe to queue when creating reply queue

* fix encoding

* WIP: SMOnlyCreate test

* Add SM guard for confirmQueue

Allows the test case to pump the allowConnection
reply without getting PROHIBITED.

* Remove tracing

* add noMessages, remove unnecessary getConnectionMessage from test

* add sending messages to the test

---------

Co-authored-by: IC Rainbow <aenor.realm@gmail.com>
Co-authored-by: Alexander Bondarenko <486682+dpwiz@users.noreply.github.com>
2023-09-10 17:07:19 +01:00
Evgeny Poberezkin 980e5c4d1e agent: add debugging info (#840) 2023-09-04 22:02:12 +01:00
Evgeny Poberezkin 5dc3d739b2 agent: support encrypted local files (#837)
* agent: support encrypted local files

* migration, update store, api

* tests, fix

* use CF.plain
2023-08-31 22:43:58 +01:00
Evgeny Poberezkin dc65197cfd Merge branch 'master' into master-ghc9 2023-08-26 16:54:09 +01:00
Evgeny Poberezkin 4c0b8a31d2 agent: create parent folders when creating the database 2023-08-26 16:53:31 +01:00
Evgeny Poberezkin ed05428227 Merge branch 'master' into master-ghc9 2023-08-25 14:11:51 +01:00
Evgeny Poberezkin 066d91b0f5 agent: export SubInfo 2023-08-25 09:20:12 +01:00
Evgeny Poberezkin 414b019ad4 agent: function to debug agent subscriptions (#834)
* agent: function to debug agent subscriptions

* add Show instances
2023-08-24 23:19:48 +01:00
Moritz Angermann 002f36dde0 5.4.0.0: use ghc 9.6.2 (#791)
* Add 9.6 compat

* compile with GHC9.6.2: dependencies, imports, code

* refactor typeclasses

* refactor record dot

* update cabal version

* update github actions

* update direct-sqlcipher

* 5.4.0.0

* update cabal.project

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-08-24 20:22:02 +01:00
Evgeny Poberezkin d09adb5cd6 agent: wait for db (Revert Revert db error busy treatments (#796) (#808)) (#828)
* Revert "Revert "db error busy treatments (#796)" (#808)"

This reverts commit 82aec2cd8f.

* update test

* combine pragmas
2023-08-21 09:06:51 +01:00
Evgeny Poberezkin b329eb9a86 agent: fix async JOIN (#827)
* agent: fix async JOIN

* refactor
2023-08-20 18:45:16 +01:00
spaced4ndy cf2a17b80c agent: track count and average time for slow queries (#824) 2023-08-16 20:14:27 +04:00
spaced4ndy e586bef57a agent: add indexes to improve slow queries performance (#823) 2023-08-16 10:29:09 +04:00
Evgeny Poberezkin e2065ab352 agent: track slow SQL queries (#822)
* agent: track slow SQL queries

* add executeMany

* reduce threshold for slow queries to 50ms
2023-08-12 18:18:10 +01:00
Evgeny Poberezkin 82aec2cd8f Revert "db error busy treatments (#796)" (#808)
This reverts commit 1afcefa5e7.
2023-08-09 12:24:03 +01:00
spaced4ndy e81f3b5529 add indexes (#809) 2023-07-22 13:20:49 +01:00
Evgeny Poberezkin 6314bb1706 agent: partially fix message queries, debug message times (#810)
* agent: partially fix message queries, debug message times

* fix more joins
2023-07-22 13:00:25 +01:00
spaced4ndy 7e2b309450 migration to delete expired messages (#801 is suspected to have caused temporary traffic increase due to flood of non expired messages) (#806) 2023-07-20 16:15:33 +04:00
Evgeny Poberezkin 58cb2855d2 agent: delivery receipts (#752)
* rfc: delivery receipts

* update doc

* update rfc

* implementation plan, types, schema

* migration, update types

* update types

* rename migration

* export MsgReceiptStatus, JSON encoding

* update rfc, schema

* correction

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* skeleton of the implementation

* more implementation (some tests fail)

* more code, 1 test fails

* fix encoding

* refactor

* refactor

* test, fix

* only send receipts in v3+, test

* flip condition

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* flip condition

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>

* agent version 4 required to send receipts

* fix test

---------

Co-authored-by: spaced4ndy <8711996+spaced4ndy@users.noreply.github.com>
2023-07-13 22:33:48 +01:00
spaced4ndy 1afcefa5e7 db error busy treatments (#796) 2023-07-11 20:19:20 +04:00
Evgeny Poberezkin 532cd2f39c fix allFinally 2023-07-09 21:36:23 +01:00
Evgeny Poberezkin f76a5ca5b6 agent: catch IO errors correctly in MonadError (#795)
* agent: catch IO errors correctly in MonadError

* correction

* correction

* utils

* agentFinally to catch IO exceptions in ExceptT

* rename

* remove, inline

* rename utils

* utils unit test

* test to show catch and finally problems

* tryAllErrors

* enable all tests
2023-07-09 18:04:45 +01:00
spaced4ndy f2657f9c0b add crypto error to RSYNC event (#794) 2023-07-05 18:14:51 +04:00
Evgeny Poberezkin 75fe28a8a6 lint 2023-07-03 17:43:56 +01:00
spaced4ndy 0e1562deae increase connection agent version according to received messages; add version and flags to connection stats, bump currentSMPAgentVersion (#789) 2023-06-30 20:19:10 +04:00
Evgeny Poberezkin 94540a2c71 servers: add TCP timeouts to avoid memory leaks (#776)
* servers: add TCP timeouts to avoid memory leaks

* fix tests

* only use RecvTimeOut

* servers: simple timeout for TCP transport

* revert dependency change

* simplify

* simplify

* simplify 2
2023-06-30 16:22:01 +01:00
spaced4ndy 8be2505fa0 ratchet re-synchronization (#774)
* ratchet re-synchronization rfc wip

* additions

* additions, types

* fix tests

* re-sync implementation wip

* re-sync implementation 1st rev.

* test wip

* test passes

* doc

* wording

* improve doc schema

* single agreed state

* refactor (1 state variable)

* allowed -> required

* prohibit enqueue

* enqueue

* send via multiple queues

* test with server offline

* clarify errors

* rename

* more tests

* refactor

* rename AgentRKey

* rename AM_CONN_RATCHET_KEY

* more tests

* rename

* write encoded AgentRatchetInfo to AgentRatchetKey info

* move withConnLock

* refactor qDuplex

* re-create ratchet on receiving second key

* invert condition

* refactor

* simplify

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-06-30 14:17:08 +04:00
spaced4ndy c8a8e2c297 differentiate agent INACTIVE error, treat as temporary (#784) 2023-06-29 10:00:53 +04:00
Evgeny Poberezkin f7d038ef20 agent: more batched commands (#782)
* agent: more batched commands

* layout
2023-06-28 19:50:11 +01:00
Evgeny Poberezkin 4a927d1ae2 ntf server: better batching and logging (#780)
* ntf server: better batching and logging

* reduce batch delay for ntf server

* comments

* 5.1.3, ntf 1.4.2

* more logging

* more logging

* split large batches, more logging

* remove some logs
2023-06-26 20:14:35 +01:00
spaced4ndy 3a74558e84 fix connection switch status db encoding (#781) 2023-06-26 21:03:46 +04:00
spaced4ndy 2efe1496d2 rename "stop switch" -> "abort switch", add canAbortSwitch to rcv queue info (#775) 2023-06-16 13:41:01 +04:00
spaced4ndy 011e1109e9 connection switch - track status, stop switch command (#764)
* connection switch improvements

* stop in transaction

* refactor

* checks in transaction

* withSwitchingRQ in transaction

* withSwitchedSQ in transaction

* refactor

* refactor

* inverse some cases

* reset snd switch progress on repeat qadd

* remove unused functions

* connection info type

* correction

* SPFinalizing phase, correctly update connection in event

* refactor

* tests

* refactor switch conn lock taking, take lock in switch stop, remove ineligible test

* comment

* corrections

* corrections

* corrections

* refactor

* remove checks in transactions, remove some statuses

* remove diff

* refactor

* refactor tests

* update connection in memory

* revert diff

* corrections

* more tests

* remove statuses, change switching queue lookup

* async queue deletion

* refactor

* lock, phases

* test status

* refactor

---------

Co-authored-by: Evgeny Poberezkin <2769109+epoberezkin@users.noreply.github.com>
2023-06-15 11:15:46 +04:00
spaced4ndy 4a3c5abf32 add with withTransaction version with logging (#757) 2023-05-26 20:33:48 +04:00
spaced4ndy 6c6f22051d check previous encrypted message hashes for duplicates on decryption errors (#756) 2023-05-17 15:42:57 +04:00
spaced4ndy d693868bc0 change implementation of time diff calculation (#754) 2023-05-15 20:26:18 +04:00
spaced4ndy ce64c91d5a add indexes for reading pending files (#751) 2023-05-10 15:17:26 +04:00