types and migration

This commit is contained in:
Evgeny @ SimpleX Chat
2026-09-05 09:44:20 +00:00
parent e6ad21801f
commit bf8dc4b3c5
12 changed files with 436 additions and 122 deletions
+159 -100
View File
@@ -41,9 +41,20 @@ message id used here.
item -> instance with feed_item_id = feed item AND the contact_id /
group_id of the connection`.
- Instances are ordinary sent items of their chats: per-chat editing,
deletion, reactions, expiration, disappearing messages.
`chat_items.feed_item_id` (`ON DELETE SET NULL`) links an instance to the
feed item and is the file join; `CIMeta.itemFeed` exposes it.
deletion, reactions, forwarding, expiration, disappearing messages. A
per-chat edit, or a per-chat deletion that keeps the row (a marked or a
moderated item), detaches the instance: `chat_items.item_feed` changes
from 1 (linked) to 2 (detached). Feed edits and feed deletes are applied
to linked instances only; feed file descriptions are sent for every
undeleted instance. The per-chat `XMsgUpdate` of an instance is sent with
`feed = Just True`, so a recipient with `dropFeed` discards it. Bulk
removals (`APIClearChat`, contact or group deletion, chat item expiration,
disappearing messages) remove instances; those recipients, and the
recipients of detached instances, are outside later feed edits and
deletes: the instances are the record of delivery.
`chat_items.feed_item_id` (`ON DELETE SET NULL`) links every instance,
linked or detached, to the feed item and is the file join; after the feed
item is removed a remaining instance shows no file.
- A feed message takes the TTL of the chat on both sides: the sender's
instance from `sndContactCITimed False ct Nothing` (`Internal.hs:174`), the
recipient's item from the chat's own TTL in place of the absent message
@@ -72,8 +83,11 @@ message id used here.
- Wire marker: `feed :: Maybe Bool` in `MsgContainer` and in `XMsgUpdate`,
serialized only as `true`. Older clients ignore the field. The chat version
is unchanged.
- Received feed messages are stored with `chat_items.item_feed = 1`;
`CIMeta.itemFeed = Just CIFeedRcv`.
- `CIMeta.itemFeed :: Maybe CIFeed` marks the items of a broadcast in direct
and group chats on both sides: the sender's instances (`CIFLinked` or
`CIFDetached`) and the recipients' received feed messages (`CIFLinked`).
The feed item and every other item have `Nothing`. Received feed
messages are stored with `chat_items.item_feed = 1`.
- Content: text, link, image, video, voice, file. One XFTP upload per
broadcast; one recipient description for everyone. Quotes, mentions, live
messages and the `ttl` parameter are rejected.
@@ -88,8 +102,9 @@ message id used here.
only; `SendRef` is unchanged.
- `CRBroadcastSent` is removed: the broadcast is a job, so counts are not
known at command time. `/feed` returns `CRNewChatItems` with the feed item.
- Commands on the feed hold a `CLFeed` entity lock and only write the feed
item and a job. The worker holds no entity locks (as relay job workers).
- Commands on the feed hold the `CLFeed feedId` entity lock and only write
the feed item and a job. The worker holds no entity locks (as relay job
workers).
- Feed items are outside global chat item expiration (as notes).
## Types
@@ -151,19 +166,19 @@ returns `(Maybe ContactId, Maybe GroupId, Maybe FeedId)`.
matches the constructors and gains a `FeedId` branch returning
`SEInternalError`: received messages have a connection or a group.
- `CIMeta` (:509): field `itemFeed :: Maybe CIFeed` after `msgVerified`;
`mkCIMeta` (:535) gains the parameter; `dummyMeta` (:551) sets `Nothing`.
`mkCIMeta` (:535) gains the parameter; `deletable` and `editable` (:537-538)
are unchanged; `dummyMeta` (:551) sets `Nothing`.
```haskell
data CIFeed
= CIFeedSnd {feedItemId :: Maybe ChatItemId}
| CIFeedRcv
data CIFeed = CIFLinked | CIFDetached
```
JSON `sumTypeJSON $ dropPrefix "CIFeed"`. Stored as
`chat_items.item_feed` (1 for both) and `chat_items.feed_item_id`
(`CIFeedSnd` only; `Nothing` after the feed item is removed or the feed is
cleared). Row mapping: `item_feed = 1` with `item_sent = 1` ->
`CIFeedSnd feed_item_id`; with `item_sent = 0` -> `CIFeedRcv`.
JSON `enumJSON $ dropPrefix "CIF"`. Stored in `chat_items.item_feed`:
0 -> `Nothing`, 1 -> `Just CIFLinked`, 2 -> `Just CIFDetached`
(`DBCIFeedLinked`, `DBCIFeedDetached` patterns next to `DBCIDeleted`,
`Store/Messages.hs:2933`). `chat_items.feed_item_id` stays in the store:
the link of a sender's instance to the feed item for jobs, delivery events
and the file join.
`Protocol.hs`:
@@ -172,7 +187,9 @@ returns `(Maybe ContactId, Maybe GroupId, Maybe FeedId)`.
- `XMsgUpdate` (:450): field `feed :: Maybe Bool`; parser (:1403) reads
`opt "feed"`; encoder (:1486) adds `("feed" .=? feed)`. Positional
patterns and constructions gain the argument: `Subscriber.hs:554`, `:730`,
`:1039`, `:1286`, `:3882`; `Commands.hs:780`, `:814`, `:1443`.
`:1039`, `:1286`, `:3882`; `Commands.hs:1443` (`Nothing`); `Commands.hs:780`
and `:814` pass `justTrue (isJust itemFeed)` (`justTrue`,
`Protocol.hs:1037`), the edited item's marker.
- `cmFeed :: AChatMsgEvent -> Bool` next to `cmToQuotedMsg` (:622):
`ACME _ (XMsgNew MsgContainer {feed = Just True}) -> True`.
@@ -238,61 +255,50 @@ data FeedJobSpec
## Schema
SQLite `M20260904_feeds`; Postgres with `BIGINT GENERATED ALWAYS AS IDENTITY`,
`TIMESTAMPTZ`, `SMALLINT` and named constraints, as `M20241220_initial.hs:619-635`.
```sql
CREATE TABLE feeds(
feed_id INTEGER PRIMARY KEY AUTOINCREMENT,
user_id INTEGER NOT NULL REFERENCES users ON DELETE CASCADE,
created_at TEXT NOT NULL DEFAULT(datetime('now')),
updated_at TEXT NOT NULL DEFAULT(datetime('now')),
chat_ts TEXT NOT NULL DEFAULT(datetime('now')),
favorite INTEGER NOT NULL DEFAULT 0,
unread_chat INTEGER NOT NULL DEFAULT 0
) STRICT;
PRAGMA writable_schema=1;
UPDATE sqlite_master
SET sql = replace(sql, 'group_id INTEGER NOT NULL REFERENCES groups ON DELETE CASCADE', 'group_id INTEGER REFERENCES groups ON DELETE CASCADE')
WHERE type = 'table' AND name = 'delivery_jobs';
PRAGMA writable_schema=RESET;
ALTER TABLE delivery_jobs ADD COLUMN feed_id INTEGER REFERENCES feeds ON DELETE CASCADE;
ALTER TABLE delivery_jobs ADD COLUMN chat_item_id INTEGER REFERENCES chat_items ON DELETE CASCADE;
ALTER TABLE delivery_jobs ADD COLUMN delete_mode TEXT;
ALTER TABLE delivery_jobs ADD COLUMN message_ids TEXT;
ALTER TABLE delivery_jobs ADD COLUMN cursor_contact_id INTEGER;
ALTER TABLE delivery_jobs ADD COLUMN cursor_group_id INTEGER;
CREATE INDEX idx_delivery_jobs_feed_next ON delivery_jobs(feed_id, worker_scope, failed, job_status);
CREATE INDEX idx_delivery_jobs_chat_item_id ON delivery_jobs(chat_item_id);
ALTER TABLE chat_items ADD COLUMN feed_id INTEGER DEFAULT NULL REFERENCES feeds ON DELETE CASCADE;
ALTER TABLE chat_items ADD COLUMN feed_item_id INTEGER DEFAULT NULL REFERENCES chat_items ON DELETE SET NULL;
ALTER TABLE chat_items ADD COLUMN item_feed INTEGER NOT NULL DEFAULT 0;
ALTER TABLE messages ADD COLUMN feed_id INTEGER DEFAULT NULL REFERENCES feeds ON DELETE CASCADE;
ALTER TABLE files ADD COLUMN feed_id INTEGER DEFAULT NULL REFERENCES feeds ON DELETE CASCADE;
ALTER TABLE contacts ADD COLUMN drop_feed INTEGER NOT NULL DEFAULT 0;
ALTER TABLE groups ADD COLUMN drop_feed INTEGER NOT NULL DEFAULT 0;
CREATE INDEX idx_feeds_user_id ON feeds(user_id);
CREATE INDEX idx_chat_items_feed_id ON chat_items(feed_id);
CREATE INDEX idx_chat_items_feeds_created_at ON chat_items(user_id, feed_id, created_at);
CREATE INDEX idx_chat_items_feed_item_contact ON chat_items(feed_item_id, contact_id);
CREATE INDEX idx_chat_items_feed_item_group ON chat_items(feed_item_id, group_id);
CREATE INDEX idx_messages_feed_id ON messages(feed_id);
CREATE INDEX idx_files_feed_id ON files(feed_id);
INSERT INTO feeds (user_id) SELECT user_id FROM users;
```
SQLite: `M20260905_feeds.hs` (written; registered in `SQLite/Migrations.hs`
and `simplex-chat.cabal`; `chat_schema.sql` regenerated; the up, down and
repeated up dumps and `.lint fkey-indexes` verified with `sqlite3`).
Postgres: the same statements with `BIGINT GENERATED ALWAYS AS IDENTITY`,
`TIMESTAMPTZ`, `SMALLINT`, named constraints and
`ALTER TABLE delivery_jobs ALTER COLUMN group_id DROP NOT NULL`, as
`M20241220_initial.hs:619-635`.
- `feed_item_id` self-reference precedent: `fwd_from_chat_item_id`
(`chat_schema.sql:504`).
(`chat_schema.sql:504`). `item_feed`: 0 none, 1 linked, 2 detached.
- `delivery_jobs.group_id` becomes nullable through the `sqlite_master`
edit of `M20251230_strict_tables.hs:18-26`; Postgres uses
`ALTER COLUMN group_id DROP NOT NULL`. A feed job has `feed_id`,
`chat_item_id` (the feed item), `message_ids` and the feed cursors; a
group job has `group_id` and `cursor_group_member_id`.
edit of `M20251230_strict_tables.hs:18-26`; `PRAGMA writable_schema=RESET`
reloads the schema before the `ADD COLUMN` statements of the same
transaction, so their column offsets are computed from the edited text.
A feed job has `feed_id`, `chat_item_id` (the feed item), `message_ids`
and the feed cursors; a group job has `group_id` and
`cursor_group_member_id`.
- `message_ids`: comma-separated decimal ids, the encoding of
`delivery_jobs.sender_group_member_ids` (`Store/Delivery.hs:266-270, :329`).
- Registration: `SQLite/Migrations.hs`, `Postgres/Migrations.hs`, the module
list in `simplex-chat.cabal` (:85), regenerated `chat_schema.sql` dumps, the
down-migration round-trip.
- Indexes and the queries they serve (plans checked with
`EXPLAIN QUERY PLAN` on the migrated schema):
- `idx_delivery_jobs_feed_next (feed_id, worker_scope, failed, job_status)`:
`getNextDeliveryJob` by feed, a covering search.
- `idx_delivery_jobs_chat_item_id`: the `chat_item_id` FK.
- `idx_chat_items_feed_item_contact (feed_item_id, contact_id)` and
`idx_chat_items_feed_item_group (feed_item_id, group_id)`: the instance
cursor reads, the range guards, `updateFeedInstances` and the
delivery-event join (`feed_item_id = ? AND contact_id > ?`, in cursor
order without a sort).
- `idx_contacts_user_id (user_id)`: `getFeedContactsByCursor`
(`user_id = ? AND rowid > ? ORDER BY rowid`); without it the planner
picks `idx_contacts_chat_ts` and sorts every contact of the user per
bucket.
- `idx_groups_user_id_business_chat (user_id, business_chat)`:
`getFeedCustomerGroupsByCursor`
(`user_id = ? AND business_chat = ? AND rowid > ? ORDER BY rowid`).
- `idx_chat_items_feed_id`, `idx_chat_items_feeds_created_at (user_id, feed_id, created_at)`:
the `feed_id` FK and the feed chat reads.
- `idx_messages_feed_id`, `idx_files_feed_id`, `idx_feeds_user_id`: FKs
and `deleteFeedCIs`.
- Existing: `idx_msg_deliveries_message_id` (delivery guards),
`idx_group_members_group_id (user_id, group_id)` (members by group
range), `idx_chat_tags_chats_*` (tags by range),
`idx_chat_item_reactions_shared_msg_id` (feed reactions).
## Store
@@ -325,16 +331,20 @@ New module `Store/Feeds.hs`:
(the `getGroupMembers` condition, `Store/Groups.hs:1236`;
`idx_group_members_group_id (user_id, group_id)`); the `groups` join
keeps members of other groups in the id range out.
- `getFeedContactInstancesByCursor db cxt user feedItemId cursor_ count :: IO [(Contact, CChatItem 'CTDirect)]` —
- `getFeedContactInstancesByCursor db cxt user feedItemId spec cursor_ count :: IO [(Contact, CChatItem 'CTDirect)]` —
the `getDirectChatItem` SELECT (`Store/Messages.hs:2700-2716`) composed
with `contactQueryFields` (`:.` rows), `FROM chat_items i JOIN contacts ct ON ct.contact_id = i.contact_id`
+ `contactQueryFrom` joins,
`WHERE i.user_id = ? AND i.feed_item_id = ? AND i.contact_id > ? ORDER BY i.contact_id, c.connection_id LIMIT ?`
(`idx_chat_items_feed_item_contact`).
- `getFeedGroupInstancesByCursor db cxt user feedItemId cursor_ count :: IO [(GroupInfo, CChatItem 'CTGroup)]` —
`WHERE i.user_id = ? AND i.feed_item_id = ? AND i.contact_id > ? AND <spec> ORDER BY i.contact_id, c.connection_id LIMIT ?`
(`idx_chat_items_feed_item_contact`); `<spec>` is `i.item_feed = 1` for
`FJUpdate` and `FJDelete`, `i.item_deleted = 0` for `FJFileDescr`.
- `getFeedGroupInstancesByCursor db cxt user feedItemId spec cursor_ count :: IO [(GroupInfo, CChatItem 'CTGroup)]` —
the `getGroupChatItem` SELECT (:3094-3139) composed with
`groupInfoQueryFields`, `WHERE i.user_id = ? AND i.feed_item_id = ? AND i.group_id > ? ORDER BY i.group_id LIMIT ?`
`groupInfoQueryFields`, `WHERE i.user_id = ? AND i.feed_item_id = ? AND i.group_id > ? AND <spec> ORDER BY i.group_id LIMIT ?`
(`idx_chat_items_feed_item_group`); members from `getCustomerGroupsMembersByRange`.
- `detachFeedInstances db itemIds` —
`UPDATE chat_items SET item_feed = 2 WHERE chat_item_id = ? AND item_feed = 1`
by `executeMany`.
- `getFeedInstanceContactIdsByRange db user feedItemId fromId toId :: IO [ContactId]`
(and groups) — the instance guard of a repeated `FJNew` bucket.
- `getDeliveredContactIdsByRange db msgId fromId toId :: IO [ContactId]` —
@@ -366,7 +376,7 @@ New module `Store/Feeds.hs`:
- Bucket writers, one statement or one `executeMany` each, in one
transaction per bucket: `updateFeedInstances` (`item_content`, `item_text`,
`item_edited = 1`, `has_link`, `updated_at` for
`user_id = ? AND feed_item_id = ? AND contact_id > ? AND contact_id <= ?`,
`user_id = ? AND feed_item_id = ? AND item_feed = 1 AND contact_id > ? AND contact_id <= ?`,
and the group range), `deleteFeedInstances` and `markFeedInstancesDeleted`
by `executeMany` over the ids of each subset (the full-delete split is
decided in Haskell from `mergedPreferences`), reaction deletion by
@@ -414,17 +424,18 @@ the `note_folders` shape (`Store/NoteFolders.hs:61`).
`connOrGroupId`. `Just smId` inserts with it; `Nothing` keeps
`createWithRandomId'`. `SharedMsgId` has a `ToField` instance
(`Types.hs:256`). `FeedId feedId` writes `messages.feed_id`.
- `createNewChatItem_` (:590): parameter `Maybe CIFeed`; `idsRow` gains
- `createNewChatItem_` (:590): parameters `Maybe CIFeed` and
`Maybe ChatItemId` (the feed item of a sender's instance); `idsRow` gains
`Maybe FeedId` from `CDFeedSnd Feed {feedId}`; the INSERT adds `feed_id`,
`feed_item_id`, `item_feed`. `createNewSndChatItem` (:548) passes the
parameter through; `createNewRcvChatItem` (:564) passes
`if cmFeed chatMsgEvent then Just CIFeedRcv else Nothing`;
`createNewChatItemNoMsg` (:583) and `createLocalChatItems`
(`Internal.hs:3185`) pass `Nothing`.
- `ChatItemModeRow` (:2281) gains `(BoolInt, Maybe ChatItemId)` from
`i.item_feed, i.feed_item_id`; `toLocalChatItem` (:1096),
`toDirectChatItem` (:2307), `toGroupChatItem` (:2375) build `Maybe CIFeed`
and pass it to `mkCIMeta`; SELECT lists at :2706, :3100, :3213.
`feed_item_id`, `item_feed`. `createNewRcvChatItem` (:564) passes
`(if cmFeed chatMsgEvent then Just CIFLinked else Nothing) Nothing`;
`createNewSndChatItem` (:548), `createNewChatItemNoMsg` (:583) and
`createLocalChatItems` (`Internal.hs:3185`) pass `Nothing Nothing`; the
feed job calls `createNewChatItem_` with `(Just CIFLinked) (Just feedItemId)`.
- `ChatItemModeRow` (:2281) gains `Int` from `i.item_feed`;
`toCIFeed :: Int -> Maybe CIFeed` maps 0, 1, 2; `toLocalChatItem` (:1096),
`toDirectChatItem` (:2307), `toGroupChatItem` (:2375) pass the value to
`mkCIMeta`; SELECT lists at :2706, :3100, :3213.
- The file join in `getDirectChatItem` (:2712) and `getGroupChatItem`
(:3129) becomes
`LEFT JOIN files f ON f.chat_item_id = COALESCE(i.feed_item_id, i.chat_item_id)`:
@@ -501,14 +512,21 @@ the `note_folders` shape (`Store/NoteFolders.hs:61`).
- `createMemberSndStatuses` (`Commands.hs:4866`) moves to the top level of
`Internal.hs` unchanged.
- `mkChatItem_` (:2859) gains the `Maybe CIFeed` parameter for `mkCIMeta`;
`mkChatItem` (:2853), `saveRcvChatItem'` (:2825) and
`saveSndChatItems.createItem` (:2789) pass through (`NewSndChatItemData`
:2761 gains `itemFeed :: Maybe CIFeed`, `Nothing` at existing sites).
`saveRcvChatItem'` (:2825) passes
`if cmFeed chatMsgEvent then Just CIFLinked else Nothing`; `mkChatItem`
(:2853) and `saveSndChatItems.createItem` (:2793) pass `Nothing`; the feed
job passes `Just CIFLinked`.
- The file-info lambda repeated in `deleteDirectCIs` (:521),
`deleteGroupCIs` (:533), `deleteLocalCIs` (:597), `markDirectCIsDeleted`
(:616), `markGroupCIsDeleted` (:628) becomes `itemsFilesInfo`, which skips
items with `itemFeed = Just (CIFeedSnd _)`: an instance's file is the feed
item's file and is not cancelled or deleted with the instance.
`CChatItem SMDSnd` items with `itemFeed` set: a sender's instance renders
the feed item's file, which is not cancelled or deleted with the instance.
A recipient's received feed item owns its file and is unaffected.
- `detachedInstance :: ChatItem c 'MDSnd -> ChatItem c 'MDSnd` sets
`itemFeed = Just CIFDetached`; `detachFeedInstances :: [CChatItem c] -> CM [CChatItem c]`
runs the store `detachFeedInstances` for the `CChatItem SMDSnd` items with
`itemFeed = Just CIFLinked`, returns those through `detachedInstance` and
the other items unchanged, in the input order.
`Commands.hs`, `APISendFeedMessage feedId cm`, under `withFeedLock "sendFeed" feedId`:
@@ -582,8 +600,9 @@ delivery, and a crash between delivery and the cursor sends nothing twice.
Contact bucket, one range read: `FJNew` reads
`getFeedContactsByCursor` and `getContactsTagsByRange`; the other types read
`getFeedContactInstancesByCursor` (contact and instance together) and the
tags. Then, in memory:
`getFeedContactInstancesByCursor` (contact and instance together, linked
instances for `FJUpdate` and `FJDelete`, undeleted instances for
`FJFileDescr`) and the tags. Then, in memory:
1. Eligible for `FJNew`: `directOrUsed`, not `contactConnIncognito`,
`contactSendConn_` returns a connection. For the other types every
@@ -595,7 +614,7 @@ tags. Then, in memory:
`getFeedInstanceContactIdsByRange` are skipped; for the rest
`updateChatTsStats` and `createNewChatItem_` with `CDDirectSnd ct`, no
message id, the shared id, `CISndMsgContent` from the container,
`Just (CIFeedSnd (Just feedItemId))`, `timed_`; the items are built with
`Just CIFLinked`, `Just feedItemId`, `timed_`; the items are built with
`mkChatItem_`.
4. Delivery: one `deliverMessagesB` over
`(conn, MsgFlags {notification = hasNotification tag}, (vor, messageIds))`
@@ -697,8 +716,8 @@ description per relay, forwarded to all subscribers).
and `getDeliveryJobWorker True (DEFeed feedId, DWSFeed)`.
5. Feed item: `addInitialAndNewCIVersions db itemId (chatItemTs' ci, oldMC) (currentTs, mc)`;
`updateFeedChatItem' db user feedId ci (CISndMsgContent mc) True`;
response `CRChatItemUpdated`. Instance versions are not recorded; the
feed item holds the history.
response `CRChatItemUpdated`. The job records no instance versions; the
feed item holds the history of feed edits.
## Deleting
@@ -721,6 +740,33 @@ description per relay, forwarded to all subscribers).
item in `CIDeleting`); the job removes the item at the end.
4. Response `CRChatItemsDeleted user deletions True False`.
## Per-chat editing and deletion of an instance
`APIUpdateChatItem` on `CTDirect` (:768) and `CTGroup` (:793): the existing
paths, with two changes in the `(CISndMsgContent oldMC, Just itemSharedMId, True)`
branch: the event's `feed` is `justTrue (isJust itemFeed)` (:780, :814), and
the item passed to `updateDirectChatItem'` (:787) and `updateGroupChatItem`
(:822) is `detachedInstance ci` after `detachFeedInstances db [itemId]` in
the same transaction, when `itemFeed == Just CIFLinked`. The instance
receives its own versions from `addInitialAndNewCIVersions` (:785, :820),
starting from its content at the time of the edit. The response and
`startUpdatedTimedItemThread` are unchanged.
`APIDeleteChatItem` on `CTDirect` (:844): `CIDMInternalMark` (:848) and the
marking branch of `CIDMBroadcast` (:859) call `markDirectCIsDeleted` with
`detachFeedInstances items`; `CIDMInternal` and the full-delete branch
remove the rows. On `CTGroup`: `CIDMInternalMark` (:870) likewise;
`delGroupChatItems` (:4154) detaches before `markGroupCIsDeleted` and
before `deleteGroupCIs` with a moderating member (`updateGroupChatItemModerated`
keeps the row, `Internal.hs:549`), which covers `CIDMBroadcast` (:877),
`CIDMHistory` (:883) and `APIDeleteMemberChatItem` (:918). Received items
in the same command are not detached (`detachFeedInstances` selects
`CChatItem SMDSnd`).
Reactions on an instance (`APIChatItemReaction`), forwarding from an
instance, and every operation on a recipient's received feed item are
unchanged.
## Receiving
- `newContentMessage` (`Subscriber.hs:1870`): when
@@ -744,8 +790,8 @@ description per relay, forwarded to all subscribers).
- `newGroupContentMessage` (:2121) and `groupMessageUpdate` (:2209): the same
checks against `GroupInfo {chatSettings}`; a discarded message returns
`Nothing` (no delivery task).
- Received feed items get `itemFeed = Just CIFeedRcv` through
`createNewRcvChatItem` (`cmFeed`).
- Received feed items get `itemFeed = Just CIFLinked` through
`createNewRcvChatItem` and `saveRcvChatItem'` (`cmFeed`).
- `APISetChatSettings` (:1898) stores `dropFeed`; `SetDropFeed cName on`
uses `updateChatSettings` (:4347). The command replaces the whole record,
so a client without the field (an older remote controller) resets
@@ -766,7 +812,8 @@ description per relay, forwarded to all subscribers).
- `APIChatRead` (:1231): `CTFeed -> getUserByFeedId; ok user`.
`APIChatItemsRead`: "not supported". `APIChatUnread` (:1296):
`updateFeedUnreadChat`. `APIClearChat` (:1386): `deleteFeedFiles`,
`deleteFeedCIs`; instances stay in their chats, detached by the FK.
`deleteFeedCIs`; instances stay in their chats with `feed_item_id` set
to NULL by the FK and `item_feed` unchanged.
- `APIChatItemReaction` (:943): `CTFeed -> throwCmdError "not supported"`.
- `APIDeleteChat`, `APISetChatTags`, `APISetChatSettings`,
`APISetChatUIThemes`, `APISetChatTTL`: existing "not supported" branches.
@@ -801,8 +848,10 @@ description per relay, forwarded to all subscribers).
`CIDeleted.Deleting`, `CIMeta.itemFeed: CIFeed?`, `ChatSettings.dropFeed`,
the `StoreError` constructors. Chat list row and chat view for the feed;
a `CIDeleting` item rendered as deletion in progress; a feed marker on
instances and on received feed items; "edit" on an instance opens the feed
item when `feedItemId` is present; a "Drop feed messages" toggle in contact
instances and on received feed items; before an edit or a deletion of a
`linked` instance, a notice that the message is detached from the feed and
later feed edits and deletions skip this chat (`detached` instances show
no notice); a "Drop feed messages" toggle in contact
and group settings; a privacy notice in the feed chat before the first
broadcast: every recipient receives the same message id, so recipients can
establish a common sender by comparing messages.
@@ -822,14 +871,16 @@ description per relay, forwarded to all subscribers).
the UI refreshes the feed item meanwhile.
- `getChatRefViaItemId` (`Store/Messages.hs:3323`) for local and feed items.
- Forwarding into the feed.
- Navigation from an instance to its feed item (a lookup by the shared
`itemSharedMsgId` in the feed chat).
## Tests
`ChatTests`, with `feedBucketSize = 2` in the test config:
1. `/feed` to three contacts and a customer group: instances in each chat
with `itemFeed = CIFeedSnd`; recipients' items with `CIFeedRcv`; the feed
item reaches `CISSndSent SSPComplete`.
and recipients' items with `itemFeed = Just CIFLinked`; the feed item
reaches `CISSndSent SSPComplete`.
2. Instance statuses after `SENT` and receipts.
3. A contact with `dropFeed` set receives nothing; a later feed edit and
delete are silent.
@@ -864,3 +915,11 @@ description per relay, forwarded to all subscribers).
the following file description.
15. `Direct.hs:995`, `Groups.hs:6731`, `Bots/BroadcastTests.hs` updated to
`CRNewChatItems`.
16. Per-chat edit of one instance: the recipient receives the edit, the
instance becomes `CIFDetached` with its own versions; a following feed
edit updates the other instances and recipients, and this chat keeps the
per-chat text; a following feed delete keeps this instance and the
recipient's message. A per-chat broadcast delete of an instance in a
chat without full delete marks and detaches it; a recipient with
`dropFeed` set receives nothing from a per-chat edit of a dropped
message.
+1
View File
@@ -329,6 +329,7 @@ library
Simplex.Chat.Store.SQLite.Migrations.M20260813_auto_accept_group_invitations
Simplex.Chat.Store.SQLite.Migrations.M20260822_forward_link
Simplex.Chat.Store.SQLite.Migrations.M20260828_file_expiry
Simplex.Chat.Store.SQLite.Migrations.M20260905_feeds
other-modules:
Paths_simplex_chat
hs-source-dirs:
+1
View File
@@ -120,6 +120,7 @@ defaultChatConfig =
highlyAvailable = False,
deliveryWorkerDelay = 0,
deliveryBucketSize = 10000,
feedBucketSize = 1000,
webPreviewConfig = Nothing,
channelSubscriberRole = GRObserver,
relayChecksInterval = 15 * 60, -- 15 minutes
+3
View File
@@ -165,6 +165,7 @@ data ChatConfig = ChatConfig
ciExpirationInterval :: Int64, -- microseconds
deliveryWorkerDelay :: Int64, -- microseconds
deliveryBucketSize :: Int,
feedBucketSize :: Int,
webPreviewConfig :: Maybe WebPreviewConfig,
channelSubscriberRole :: GroupMemberRole, -- TODO [relays] starting role should be communicated in protocol from owner to relays
relayChecksInterval :: NominalDiffTime,
@@ -392,6 +393,7 @@ data ChatCommand
| APIUpdateChatTag ChatTagId ChatTagData
| APIReorderChatTags (NonEmpty ChatTagId)
| APICreateChatItems {noteFolderId :: NoteFolderId, composedMessages :: NonEmpty ComposedMessage}
| APISendFeedMessage {feedId :: FeedId, composedMessage :: ComposedMessage}
| APIReportMessage {groupId :: GroupId, chatItemId :: ChatItemId, reportReason :: ReportReason, reportText :: Text}
| ReportMessage {groupName :: GroupName, contactName_ :: Maybe ContactName, reportReason :: ReportReason, reportedMessage :: Text}
| APIUpdateChatItem {chatRef :: ChatRef, chatItemId :: ChatItemId, liveMessage :: Bool, updatedMessage :: UpdatedMessage}
@@ -514,6 +516,7 @@ data ChatCommand
| APIEnableGroupMember GroupId GroupMemberId
| SetShowMessages ChatName MsgFilter
| SetSendReceipts ChatName (Maybe Bool)
| SetDropFeed ChatName Bool
| SetShowMemberMessages GroupName ContactName Bool
| ContactInfo ContactName
| ShowGroupInfo GroupName
+37 -3
View File
@@ -10,7 +10,8 @@ import Data.ByteString.Char8 (ByteString)
import Data.Int (Int64)
import Data.Maybe (fromMaybe)
import Data.Time.Clock (UTCTime)
import Simplex.Chat.Messages (GroupChatScopeInfo (..), MessageId, ShowGroupAsSender)
import Simplex.Chat.Messages (ChatItemId, GroupChatScopeInfo (..), MessageId, ShowGroupAsSender)
import Simplex.Chat.Messages.CIContent (CIDeleteMode (..))
import Simplex.Chat.Options.DB (FromField (..), ToField (..))
import Simplex.Chat.Protocol
import Simplex.Chat.Types
@@ -18,11 +19,15 @@ import Simplex.Chat.Types.Shared
import Simplex.Messaging.Agent.Store.DB (fromTextField_)
import Simplex.Messaging.Encoding.String
type DeliveryWorkerKey = (GroupId, DeliveryWorkerScope)
data DeliveryEntity = DEGroup GroupId | DEFeed FeedId
deriving (Eq, Ord, Show)
type DeliveryWorkerKey = (DeliveryEntity, DeliveryWorkerScope)
data DeliveryWorkerScope
= DWSGroup
| DWSMemberSupport
| DWSFeed
-- | DWSMemberProfileUpdate
deriving (Eq, Ord, Show)
@@ -34,11 +39,13 @@ instance TextEncoding DeliveryWorkerScope where
textDecode = \case
"group" -> Just DWSGroup
"member_support" -> Just DWSMemberSupport
"feed" -> Just DWSFeed
-- "member_profile_update" -> Just DWSMemberProfileUpdate
_ -> Nothing
textEncode = \case
DWSGroup -> "group"
DWSMemberSupport -> "member_support"
DWSFeed -> "feed"
-- DWSMemberProfileUpdate -> "member_profile_update"
-- Context for creating a delivery task. Separate from DeliveryJobScope because
@@ -54,17 +61,30 @@ data DeliveryTaskContext = DeliveryTaskContext
data DeliveryJobScope
= DJSGroup {jobSpec :: DeliveryJobSpec}
| DJSMemberSupport {supportGMId :: GroupMemberId}
| DJSFeed {feedItemId :: ChatItemId, feedJobSpec :: FeedJobSpec}
-- | DJSMemberProfileUpdate
deriving (Show)
data DeliveryJobSpec
= DJDeliveryJob {includePending :: Bool}
| DJRelayRemoved
| DJFeed FeedJobSpec
deriving (Show)
data FeedJobSpec
= FJNew
| FJFileDescr
| FJUpdate
| FJDelete CIDeleteMode
deriving (Show)
data DeliveryJobSpecTag
= DJSTDeliveryJob
| DJSTRelayRemoved
| DJSTFeedNew
| DJSTFeedFileDescr
| DJSTFeedUpdate
| DJSTFeedDelete
deriving (Show)
instance FromField DeliveryJobSpecTag where fromField = fromTextField_ textDecode
@@ -75,15 +95,24 @@ instance TextEncoding DeliveryJobSpecTag where
textDecode = \case
"delivery_job" -> Just DJSTDeliveryJob
"relay_removed" -> Just DJSTRelayRemoved
"feed_new" -> Just DJSTFeedNew
"feed_file_descr" -> Just DJSTFeedFileDescr
"feed_update" -> Just DJSTFeedUpdate
"feed_delete" -> Just DJSTFeedDelete
_ -> Nothing
textEncode = \case
DJSTDeliveryJob -> "delivery_job"
DJSTRelayRemoved -> "relay_removed"
DJSTFeedNew -> "feed_new"
DJSTFeedFileDescr -> "feed_file_descr"
DJSTFeedUpdate -> "feed_update"
DJSTFeedDelete -> "feed_delete"
toWorkerScope :: DeliveryJobScope -> DeliveryWorkerScope
toWorkerScope = \case
DJSGroup _ -> DWSGroup
DJSMemberSupport _ -> DWSMemberSupport
DJSFeed {} -> DWSFeed
-- DJSMemberProfileUpdate -> DWSMemberProfileUpdate
isRelayRemoved :: DeliveryJobScope -> Bool
@@ -97,11 +126,13 @@ jobScopeImpliedSpec :: DeliveryJobScope -> DeliveryJobSpec
jobScopeImpliedSpec = \case
DJSGroup {jobSpec} -> jobSpec
DJSMemberSupport {} -> DJDeliveryJob {includePending = False}
DJSFeed {feedJobSpec} -> DJFeed feedJobSpec
jobSpecImpliedPending :: DeliveryJobSpec -> Bool
jobSpecImpliedPending = \case
DJDeliveryJob {includePending} -> includePending
DJRelayRemoved -> True
DJFeed _ -> False
infoToDeliveryContext :: GroupInfo -> Maybe GroupChatScopeInfo -> ShowGroupAsSender -> DeliveryTaskContext
infoToDeliveryContext GroupInfo {membership} scopeInfo sentAsGroup = DeliveryTaskContext {jobScope, sentAsGroup}
@@ -162,8 +193,11 @@ data MessageDeliveryJob = MessageDeliveryJob
{ jobId :: Int64,
jobScope :: DeliveryJobScope,
senderGMIds :: [GroupMemberId],
messageIds :: [MessageId],
body :: ByteString,
cursorGMId_ :: Maybe GroupMemberId
cursorGMId_ :: Maybe GroupMemberId,
cursorContactId_ :: Maybe ContactId,
cursorGroupId_ :: Maybe GroupId
}
deriving (Show)
+37 -5
View File
@@ -58,7 +58,7 @@ import Simplex.Messaging.Parsers (defaultJSON, dropPrefix, enumJSON, parseAll, s
import Simplex.Messaging.Protocol (BlockingInfo, MsgBody, XFTPServer)
import Simplex.Messaging.Util (eitherToMaybe, safeDecodeUtf8, (<$?>))
data ChatType = CTDirect | CTGroup | CTLocal | CTContactRequest | CTContactConnection
data ChatType = CTDirect | CTGroup | CTLocal | CTFeed | CTContactRequest | CTContactConnection
deriving (Eq, Show, Ord)
$(JQ.deriveJSON (enumJSON $ dropPrefix "CT") ''ChatType)
@@ -67,6 +67,7 @@ data SChatType (c :: ChatType) where
SCTDirect :: SChatType 'CTDirect
SCTGroup :: SChatType 'CTGroup
SCTLocal :: SChatType 'CTLocal
SCTFeed :: SChatType 'CTFeed
SCTContactRequest :: SChatType 'CTContactRequest
SCTContactConnection :: SChatType 'CTContactConnection
@@ -76,6 +77,7 @@ instance TestEquality SChatType where
testEquality SCTDirect SCTDirect = Just Refl
testEquality SCTGroup SCTGroup = Just Refl
testEquality SCTLocal SCTLocal = Just Refl
testEquality SCTFeed SCTFeed = Just Refl
testEquality SCTContactRequest SCTContactRequest = Just Refl
testEquality SCTContactConnection SCTContactConnection = Just Refl
testEquality _ _ = Nothing
@@ -91,6 +93,8 @@ instance ChatTypeI 'CTGroup where chatTypeI = SCTGroup
instance ChatTypeI 'CTLocal where chatTypeI = SCTLocal
instance ChatTypeI 'CTFeed where chatTypeI = SCTFeed
instance ChatTypeI 'CTContactRequest where chatTypeI = SCTContactRequest
instance ChatTypeI 'CTContactConnection where chatTypeI = SCTContactConnection
@@ -100,6 +104,7 @@ toChatType = \case
SCTDirect -> CTDirect
SCTGroup -> CTGroup
SCTLocal -> CTLocal
SCTFeed -> CTFeed
SCTContactRequest -> CTContactRequest
SCTContactConnection -> CTContactConnection
@@ -108,6 +113,7 @@ aChatType = \case
CTDirect -> ACT SCTDirect
CTGroup -> ACT SCTGroup
CTLocal -> ACT SCTLocal
CTFeed -> ACT SCTFeed
CTContactRequest -> ACT SCTContactRequest
CTContactConnection -> ACT SCTContactConnection
@@ -157,6 +163,7 @@ chatTypeStr = \case
CTDirect -> "@"
CTGroup -> "#"
CTLocal -> "*"
CTFeed -> "%"
CTContactRequest -> "<@"
CTContactConnection -> ":"
@@ -170,6 +177,7 @@ data ChatInfo (c :: ChatType) where
DirectChat :: Contact -> ChatInfo 'CTDirect
GroupChat :: GroupInfo -> Maybe GroupChatScopeInfo -> ChatInfo 'CTGroup
LocalChat :: NoteFolder -> ChatInfo 'CTLocal
FeedChat :: Feed -> ChatInfo 'CTFeed
ContactRequest :: UserContactRequest -> ChatInfo 'CTContactRequest
ContactConnection :: PendingContactConnection -> ChatInfo 'CTContactConnection
CInfoInvalidJSON :: SChatType c -> J.Object -> ChatInfo c -- this constructor is needed to catch JSON errors for Remote connection parsing
@@ -192,6 +200,7 @@ chatInfoToRef = \case
DirectChat Contact {contactId} -> Just $ ChatRef CTDirect contactId Nothing
GroupChat GroupInfo {groupId} scopeInfo -> Just $ ChatRef CTGroup groupId (toChatScope <$> scopeInfo)
LocalChat NoteFolder {noteFolderId} -> Just $ ChatRef CTLocal noteFolderId Nothing
FeedChat Feed {feedId} -> Just $ ChatRef CTFeed feedId Nothing
ContactRequest UserContactRequest {contactRequestId} -> Just $ ChatRef CTContactRequest contactRequestId Nothing
ContactConnection PendingContactConnection {pccConnId} -> Just $ ChatRef CTContactConnection pccConnId Nothing
CInfoInvalidJSON {} -> Nothing
@@ -205,6 +214,7 @@ data JSONChatInfo
= JCInfoDirect {contact :: Contact}
| JCInfoGroup {groupInfo :: GroupInfo, groupChatScope :: Maybe GroupChatScopeInfo}
| JCInfoLocal {noteFolder :: NoteFolder}
| JCInfoFeed {feed :: Feed}
| JCInfoContactRequest {contactRequest :: UserContactRequest}
| JCInfoContactConnection {contactConnection :: PendingContactConnection}
| JCInfoInvalidJSON {chatType :: ChatType, json :: J.Object}
@@ -231,6 +241,7 @@ jsonChatInfo = \case
DirectChat c -> JCInfoDirect c
GroupChat g s -> JCInfoGroup g s
LocalChat l -> JCInfoLocal l
FeedChat f -> JCInfoFeed f
ContactRequest g -> JCInfoContactRequest g
ContactConnection c -> JCInfoContactConnection c
CInfoInvalidJSON c o -> JCInfoInvalidJSON (toChatType c) o
@@ -244,6 +255,7 @@ jsonAChatInfo = \case
JCInfoDirect c -> AChatInfo SCTDirect $ DirectChat c
JCInfoGroup g s -> AChatInfo SCTGroup $ GroupChat g s
JCInfoLocal l -> AChatInfo SCTLocal $ LocalChat l
JCInfoFeed f -> AChatInfo SCTFeed $ FeedChat f
JCInfoContactRequest g -> AChatInfo SCTContactRequest $ ContactRequest g
JCInfoContactConnection c -> AChatInfo SCTContactConnection $ ContactConnection c
JCInfoInvalidJSON cType o -> case aChatType cType of ACT c -> AChatInfo c $ CInfoInvalidJSON c o
@@ -298,6 +310,7 @@ data CIDirection (c :: ChatType) (d :: MsgDirection) where
CIChannelRcv :: CIDirection 'CTGroup 'MDRcv
CILocalSnd :: CIDirection 'CTLocal 'MDSnd
CILocalRcv :: CIDirection 'CTLocal 'MDRcv
CIFeedSnd :: CIDirection 'CTFeed 'MDSnd
deriving instance Show (CIDirection c d)
@@ -313,6 +326,7 @@ data JSONCIDirection
| JCIChannelRcv
| JCILocalSnd
| JCILocalRcv
| JCIFeedSnd
deriving (Show)
jsonCIDirection :: CIDirection c d -> JSONCIDirection
@@ -324,6 +338,7 @@ jsonCIDirection = \case
CIChannelRcv -> JCIChannelRcv
CILocalSnd -> JCILocalSnd
CILocalRcv -> JCILocalRcv
CIFeedSnd -> JCIFeedSnd
jsonACIDirection :: JSONCIDirection -> ACIDirection
jsonACIDirection = \case
@@ -334,6 +349,7 @@ jsonACIDirection = \case
JCIChannelRcv -> ACID SCTGroup SMDRcv CIChannelRcv
JCILocalSnd -> ACID SCTLocal SMDSnd CILocalSnd
JCILocalRcv -> ACID SCTLocal SMDRcv CILocalRcv
JCIFeedSnd -> ACID SCTFeed SMDSnd CIFeedSnd
data CIReactionCount = CIReactionCount {reaction :: MsgReaction, userReacted :: Bool, totalReacted :: Int}
deriving (Show)
@@ -396,6 +412,7 @@ data ChatDirection (c :: ChatType) (d :: MsgDirection) where
CDChannelRcv :: GroupInfo -> Maybe GroupChatScopeInfo -> ChatDirection 'CTGroup 'MDRcv
CDLocalSnd :: NoteFolder -> ChatDirection 'CTLocal 'MDSnd
CDLocalRcv :: NoteFolder -> ChatDirection 'CTLocal 'MDRcv
CDFeedSnd :: Feed -> ChatDirection 'CTFeed 'MDSnd
toCIDirection :: ChatDirection c d -> CIDirection c d
toCIDirection = \case
@@ -406,6 +423,7 @@ toCIDirection = \case
CDChannelRcv _ _ -> CIChannelRcv
CDLocalSnd _ -> CILocalSnd
CDLocalRcv _ -> CILocalRcv
CDFeedSnd _ -> CIFeedSnd
toChatInfo :: ChatDirection c d -> ChatInfo c
toChatInfo = \case
@@ -416,6 +434,7 @@ toChatInfo = \case
CDChannelRcv g s -> GroupChat g s
CDLocalSnd l -> LocalChat l
CDLocalRcv l -> LocalChat l
CDFeedSnd f -> FeedChat f
signMessagesRequired :: ChatDirection c d -> Bool
signMessagesRequired = \case
@@ -525,6 +544,7 @@ data CIMeta (c :: ChatType) (d :: MsgDirection) = CIMeta
forwardedByMember :: Maybe GroupMemberId,
showGroupAsSender :: ShowGroupAsSender,
msgVerified :: Maybe MsgVerified,
itemFeed :: Maybe CIFeed,
createdAt :: UTCTime,
updatedAt :: UTCTime
}
@@ -532,12 +552,15 @@ data CIMeta (c :: ChatType) (d :: MsgDirection) = CIMeta
type ShowGroupAsSender = Bool
mkCIMeta :: forall c d. ChatTypeI c => ChatItemId -> CIContent d -> Text -> CIStatus d -> Maybe Bool -> Maybe SharedMsgId -> Maybe CIForwardedFrom -> Maybe (CIDeleted c) -> Bool -> Maybe CITimed -> Maybe Bool -> Bool -> Bool -> UTCTime -> ChatItemTs -> Maybe GroupMemberId -> Bool -> Maybe MsgVerified -> UTCTime -> UTCTime -> CIMeta c d
mkCIMeta itemId itemContent itemText itemStatus sentViaProxy itemSharedMsgId itemForwarded itemDeleted itemEdited itemTimed itemLive userMention hasLink_ currentTs itemTs forwardedByMember showGroupAsSender msgVerified createdAt updatedAt =
data CIFeed = CIFLinked | CIFDetached
deriving (Eq, Show)
mkCIMeta :: forall c d. ChatTypeI c => ChatItemId -> CIContent d -> Text -> CIStatus d -> Maybe Bool -> Maybe SharedMsgId -> Maybe CIForwardedFrom -> Maybe (CIDeleted c) -> Bool -> Maybe CITimed -> Maybe Bool -> Bool -> Bool -> UTCTime -> ChatItemTs -> Maybe GroupMemberId -> Bool -> Maybe MsgVerified -> Maybe CIFeed -> UTCTime -> UTCTime -> CIMeta c d
mkCIMeta itemId itemContent itemText itemStatus sentViaProxy itemSharedMsgId itemForwarded itemDeleted itemEdited itemTimed itemLive userMention hasLink_ currentTs itemTs forwardedByMember showGroupAsSender msgVerified itemFeed createdAt updatedAt =
let deletable = deletable' itemContent itemDeleted itemTs nominalDay currentTs
editable = deletable && isNothing itemForwarded
hasLink = BoolDef hasLink_
in CIMeta {itemId, itemTs, itemText, itemStatus, sentViaProxy, itemSharedMsgId, itemForwarded, itemDeleted, itemEdited, itemTimed, itemLive, userMention, hasLink, deletable, editable, forwardedByMember, showGroupAsSender, msgVerified, createdAt, updatedAt}
in CIMeta {itemId, itemTs, itemText, itemStatus, sentViaProxy, itemSharedMsgId, itemForwarded, itemDeleted, itemEdited, itemTimed, itemLive, userMention, hasLink, deletable, editable, forwardedByMember, showGroupAsSender, msgVerified, itemFeed, createdAt, updatedAt}
deletable' :: forall c d. ChatTypeI c => CIContent d -> Maybe (CIDeleted c) -> UTCTime -> NominalDiffTime -> UTCTime -> Bool
deletable' itemContent itemDeleted itemTs allowedInterval currentTs =
@@ -569,6 +592,7 @@ dummyMeta itemId ts itemText =
forwardedByMember = Nothing,
showGroupAsSender = False,
msgVerified = Nothing,
itemFeed = Nothing,
createdAt = ts,
updatedAt = ts
}
@@ -673,6 +697,7 @@ jsonACIQDirection = \case
JCIChannelRcv -> Right $ ACIQDirection SCTGroup $ CIQGroupRcv Nothing
JCILocalSnd -> Left "unquotable"
JCILocalRcv -> Left "unquotable"
JCIFeedSnd -> Left "unquotable"
quoteMsgDirection :: CIQDirection c -> MsgDirection
quoteMsgDirection = \case
@@ -1187,7 +1212,7 @@ data RcvMessage = RcvMessage
type MessageId = Int64
data ConnOrGroupId = ConnectionId Int64 | GroupId Int64
data ConnOrGroupId = ConnectionId Int64 | GroupId Int64 | FeedId Int64
data SndMsgDelivery = SndMsgDelivery
{ connId :: Int64,
@@ -1284,6 +1309,7 @@ data CIDeleted (c :: ChatType) where
CIBlocked :: Maybe UTCTime -> CIDeleted 'CTGroup
CIBlockedByAdmin :: Maybe UTCTime -> CIDeleted 'CTGroup
CIModerated :: Maybe UTCTime -> GroupMember -> CIDeleted 'CTGroup
CIDeleting :: Maybe UTCTime -> CIDeleted 'CTFeed
deriving instance Show (CIDeleted c)
@@ -1294,6 +1320,7 @@ data JSONCIDeleted
| JCIDBlocked {deletedTs :: Maybe UTCTime}
| JCIDBlockedByAdmin {deletedTs :: Maybe UTCTime}
| JCIDModerated {deletedTs :: Maybe UTCTime, byGroupMember :: GroupMember}
| JCIDDeleting {deletedTs :: Maybe UTCTime}
deriving (Show)
jsonCIDeleted :: forall d. ChatTypeI d => CIDeleted d -> JSONCIDeleted
@@ -1302,6 +1329,7 @@ jsonCIDeleted = \case
CIBlocked ts -> JCIDBlocked ts
CIBlockedByAdmin ts -> JCIDBlockedByAdmin ts
CIModerated ts m -> JCIDModerated ts m
CIDeleting ts -> JCIDDeleting ts
jsonACIDeleted :: JSONCIDeleted -> ACIDeleted
jsonACIDeleted = \case
@@ -1309,6 +1337,7 @@ jsonACIDeleted = \case
JCIDBlocked ts -> ACIDeleted SCTGroup $ CIBlocked ts
JCIDBlockedByAdmin ts -> ACIDeleted SCTGroup $ CIBlockedByAdmin ts
JCIDModerated ts m -> ACIDeleted SCTGroup (CIModerated ts m)
JCIDDeleting ts -> ACIDeleted SCTFeed $ CIDeleting ts
itemDeletedTs :: CIDeleted d -> Maybe UTCTime
itemDeletedTs = \case
@@ -1316,6 +1345,7 @@ itemDeletedTs = \case
CIBlocked ts -> ts
CIBlockedByAdmin ts -> ts
CIModerated ts _ -> ts
CIDeleting ts -> ts
data CIForwardedFrom
= CIFFUnknown
@@ -1445,6 +1475,8 @@ $(JQ.deriveJSON defaultJSON ''MemberDeliveryStatus)
$(JQ.deriveJSON defaultJSON ''ChatItemVersion)
$(JQ.deriveJSON (enumJSON $ dropPrefix "CIF") ''CIFeed)
instance (ChatTypeI c, MsgDirectionI d) => FromJSON (CIMeta c d) where
parseJSON = $(JQ.mkParseJSON defaultJSON ''CIMeta)
+8 -5
View File
@@ -447,7 +447,7 @@ signChatMsgBody MsgSigning {bindingTag, bindingData, keyRef, privKey} msgBody =
data ChatMsgEvent (e :: MsgEncoding) where
XMsgNew :: MsgContainer -> ChatMsgEvent 'Json
XMsgFileDescr :: {msgId :: SharedMsgId, fileDescr :: FileDescr, fileExpires :: Maybe UTCTime} -> ChatMsgEvent 'Json
XMsgUpdate :: {msgId :: SharedMsgId, content :: MsgContent, mentions :: Map MemberName MsgMention, ttl :: Maybe Int, live :: Maybe Bool, scope :: Maybe MsgScope, asGroup :: Maybe Bool} -> ChatMsgEvent 'Json
XMsgUpdate :: {msgId :: SharedMsgId, content :: MsgContent, mentions :: Map MemberName MsgMention, ttl :: Maybe Int, live :: Maybe Bool, scope :: Maybe MsgScope, asGroup :: Maybe Bool, feed :: Maybe Bool} -> ChatMsgEvent 'Json
XMsgDel :: {msgId :: SharedMsgId, memberId :: Maybe MemberId, scope :: Maybe MsgScope, onlyHistory :: Bool} -> ChatMsgEvent 'Json
XMsgDeleted :: ChatMsgEvent 'Json
XMsgReact :: {msgId :: SharedMsgId, memberId :: Maybe MemberId, scope :: Maybe MsgScope, reaction :: MsgReaction, add :: Bool} -> ChatMsgEvent 'Json
@@ -689,7 +689,8 @@ data MsgContainer = MsgContainer
quote :: Maybe QuotedMsg,
parent :: Maybe MsgRef,
forward :: Maybe Bool,
forwardLink :: Maybe ForwardLink
forwardLink :: Maybe ForwardLink,
feed :: Maybe Bool
}
deriving (Eq, Show)
@@ -715,7 +716,8 @@ mcSimple content =
quote = Nothing,
parent = Nothing,
forward = Nothing,
forwardLink = Nothing
forwardLink = Nothing,
feed = Nothing
}
mcQuote :: QuotedMsg -> MsgContent -> MsgContainer
@@ -1400,7 +1402,8 @@ appJsonToCM AppMessageJson {v, msgId, event, params} = do
live <- opt "live"
scope <- opt "scope"
asGroup <- opt "asGroup"
pure XMsgUpdate {msgId = msgId', content, mentions, ttl, live, scope, asGroup}
feed <- opt "feed"
pure XMsgUpdate {msgId = msgId', content, mentions, ttl, live, scope, asGroup, feed}
XMsgDel_ -> XMsgDel <$> p "msgId" <*> opt "memberId" <*> opt "scope" <*> (fromMaybe False <$> opt "onlyHistory")
XMsgDeleted_ -> pure XMsgDeleted
XMsgReact_ -> XMsgReact <$> p "msgId" <*> opt "memberId" <*> opt "scope" <*> p "reaction" <*> p "add"
@@ -1483,7 +1486,7 @@ chatToAppMessage chatMsg@ChatMessage {chatVRange, msgId, chatMsgEvent} = case en
J.Object obj -> obj
_ -> JM.empty
XMsgFileDescr msgId' fileDescr fileExpires -> o $ ("fileExpires" .=? fileExpires) ["msgId" .= msgId', "fileDescr" .= fileDescr]
XMsgUpdate {msgId = msgId', content, mentions, ttl, live, scope, asGroup} -> o $ ("asGroup" .=? asGroup) $ ("ttl" .=? ttl) $ ("live" .=? live) $ ("scope" .=? scope) $ ("mentions" .=? nonEmptyMap mentions) ["msgId" .= msgId', "content" .= content]
XMsgUpdate {msgId = msgId', content, mentions, ttl, live, scope, asGroup, feed} -> o $ ("feed" .=? feed) $ ("asGroup" .=? asGroup) $ ("ttl" .=? ttl) $ ("live" .=? live) $ ("scope" .=? scope) $ ("mentions" .=? nonEmptyMap mentions) ["msgId" .= msgId', "content" .= content]
XMsgDel msgId' memberId scope onlyHistory -> o $ ("memberId" .=? memberId) $ ("scope" .=? scope) $ ("onlyHistory" .=? justTrue onlyHistory) ["msgId" .= msgId']
XMsgDeleted -> JM.empty
XMsgReact msgId' memberId scope reaction add -> o $ ("memberId" .=? memberId) $ ("scope" .=? scope) ["msgId" .= msgId', "reaction" .= reaction, "add" .= add]
+3 -1
View File
@@ -172,6 +172,7 @@ import Simplex.Chat.Store.SQLite.Migrations.M20260723_contact_request_rejection
import Simplex.Chat.Store.SQLite.Migrations.M20260813_auto_accept_group_invitations
import Simplex.Chat.Store.SQLite.Migrations.M20260822_forward_link
import Simplex.Chat.Store.SQLite.Migrations.M20260828_file_expiry
import Simplex.Chat.Store.SQLite.Migrations.M20260905_feeds
import Simplex.Messaging.Agent.Store.Shared (Migration (..))
schemaMigrations :: [(String, Query, Maybe Query)]
@@ -343,7 +344,8 @@ schemaMigrations =
("20260723_contact_request_rejection", m20260723_contact_request_rejection, Just down_m20260723_contact_request_rejection),
("20260813_auto_accept_group_invitations", m20260813_auto_accept_group_invitations, Just down_m20260813_auto_accept_group_invitations),
("20260822_forward_link", m20260822_forward_link, Just down_m20260822_forward_link),
("20260828_file_expiry", m20260828_file_expiry, Just down_m20260828_file_expiry)
("20260828_file_expiry", m20260828_file_expiry, Just down_m20260828_file_expiry),
("20260905_feeds", m20260905_feeds, Just down_m20260905_feeds)
]
-- | The list of migrations in ascending order by date
@@ -0,0 +1,102 @@
{-# LANGUAGE QuasiQuotes #-}
module Simplex.Chat.Store.SQLite.Migrations.M20260905_feeds where
import Database.SQLite.Simple (Query)
import Database.SQLite.Simple.QQ (sql)
m20260905_feeds :: Query
m20260905_feeds =
[sql|
CREATE TABLE feeds(
feed_id INTEGER PRIMARY KEY AUTOINCREMENT,
user_id INTEGER NOT NULL REFERENCES users ON DELETE CASCADE,
created_at TEXT NOT NULL DEFAULT(datetime('now')),
updated_at TEXT NOT NULL DEFAULT(datetime('now')),
chat_ts TEXT NOT NULL DEFAULT(datetime('now')),
favorite INTEGER NOT NULL DEFAULT 0,
unread_chat INTEGER NOT NULL DEFAULT 0
) STRICT;
CREATE INDEX idx_feeds_user_id ON feeds(user_id);
INSERT INTO feeds (user_id) SELECT user_id FROM users;
ALTER TABLE chat_items ADD COLUMN feed_id INTEGER DEFAULT NULL REFERENCES feeds ON DELETE CASCADE;
ALTER TABLE chat_items ADD COLUMN feed_item_id INTEGER DEFAULT NULL REFERENCES chat_items ON DELETE SET NULL;
ALTER TABLE chat_items ADD COLUMN item_feed INTEGER NOT NULL DEFAULT 0;
CREATE INDEX idx_chat_items_feed_id ON chat_items(feed_id);
CREATE INDEX idx_chat_items_feeds_created_at ON chat_items(user_id, feed_id, created_at);
CREATE INDEX idx_chat_items_feed_item_contact ON chat_items(feed_item_id, contact_id);
CREATE INDEX idx_chat_items_feed_item_group ON chat_items(feed_item_id, group_id);
ALTER TABLE messages ADD COLUMN feed_id INTEGER DEFAULT NULL REFERENCES feeds ON DELETE CASCADE;
CREATE INDEX idx_messages_feed_id ON messages(feed_id);
ALTER TABLE files ADD COLUMN feed_id INTEGER DEFAULT NULL REFERENCES feeds ON DELETE CASCADE;
CREATE INDEX idx_files_feed_id ON files(feed_id);
ALTER TABLE contacts ADD COLUMN drop_feed INTEGER NOT NULL DEFAULT 0;
ALTER TABLE groups ADD COLUMN drop_feed INTEGER NOT NULL DEFAULT 0;
CREATE INDEX idx_contacts_user_id ON contacts(user_id);
CREATE INDEX idx_groups_user_id_business_chat ON groups(user_id, business_chat);
PRAGMA writable_schema=1;
UPDATE sqlite_master
SET sql = replace(sql, 'group_id INTEGER NOT NULL REFERENCES groups ON DELETE CASCADE', 'group_id INTEGER REFERENCES groups ON DELETE CASCADE')
WHERE type = 'table' AND name = 'delivery_jobs';
PRAGMA writable_schema=RESET;
ALTER TABLE delivery_jobs ADD COLUMN feed_id INTEGER REFERENCES feeds ON DELETE CASCADE;
ALTER TABLE delivery_jobs ADD COLUMN chat_item_id INTEGER REFERENCES chat_items ON DELETE CASCADE;
ALTER TABLE delivery_jobs ADD COLUMN delete_mode TEXT;
ALTER TABLE delivery_jobs ADD COLUMN message_ids TEXT;
ALTER TABLE delivery_jobs ADD COLUMN cursor_contact_id INTEGER;
ALTER TABLE delivery_jobs ADD COLUMN cursor_group_id INTEGER;
CREATE INDEX idx_delivery_jobs_feed_next ON delivery_jobs(feed_id, worker_scope, failed, job_status);
CREATE INDEX idx_delivery_jobs_chat_item_id ON delivery_jobs(chat_item_id);
|]
down_m20260905_feeds :: Query
down_m20260905_feeds =
[sql|
DROP INDEX idx_delivery_jobs_chat_item_id;
DROP INDEX idx_delivery_jobs_feed_next;
DELETE FROM delivery_jobs WHERE group_id IS NULL;
ALTER TABLE delivery_jobs DROP COLUMN cursor_group_id;
ALTER TABLE delivery_jobs DROP COLUMN cursor_contact_id;
ALTER TABLE delivery_jobs DROP COLUMN message_ids;
ALTER TABLE delivery_jobs DROP COLUMN delete_mode;
ALTER TABLE delivery_jobs DROP COLUMN chat_item_id;
ALTER TABLE delivery_jobs DROP COLUMN feed_id;
PRAGMA writable_schema=1;
UPDATE sqlite_master
SET sql = replace(sql, 'group_id INTEGER REFERENCES groups ON DELETE CASCADE', 'group_id INTEGER NOT NULL REFERENCES groups ON DELETE CASCADE')
WHERE type = 'table' AND name = 'delivery_jobs';
PRAGMA writable_schema=RESET;
DROP INDEX idx_groups_user_id_business_chat;
DROP INDEX idx_contacts_user_id;
ALTER TABLE groups DROP COLUMN drop_feed;
ALTER TABLE contacts DROP COLUMN drop_feed;
DROP INDEX idx_files_feed_id;
ALTER TABLE files DROP COLUMN feed_id;
DROP INDEX idx_messages_feed_id;
ALTER TABLE messages DROP COLUMN feed_id;
DROP INDEX idx_chat_items_feed_item_group;
DROP INDEX idx_chat_items_feed_item_contact;
DROP INDEX idx_chat_items_feeds_created_at;
DROP INDEX idx_chat_items_feed_id;
ALTER TABLE chat_items DROP COLUMN item_feed;
ALTER TABLE chat_items DROP COLUMN feed_item_id;
ALTER TABLE chat_items DROP COLUMN feed_id;
DROP INDEX idx_feeds_user_id;
DROP TABLE feeds;
|]
@@ -106,6 +106,7 @@ CREATE TABLE contacts(
grp_direct_inv_from_group_member_id INTEGER REFERENCES group_members(group_member_id) ON DELETE SET NULL,
grp_direct_inv_from_member_conn_id INTEGER REFERENCES connections(connection_id) ON DELETE SET NULL,
grp_direct_inv_started_connection INTEGER NOT NULL DEFAULT 0,
drop_feed INTEGER NOT NULL DEFAULT 0,
FOREIGN KEY(user_id, local_display_name)
REFERENCES display_names(user_id, local_display_name)
ON DELETE CASCADE
@@ -208,7 +209,8 @@ CREATE TABLE groups(
roster_blob BLOB,
group_domain_verified INTEGER,
stored_roster_version INTEGER,
applied_complete_roster_version INTEGER, -- received
applied_complete_roster_version INTEGER,
drop_feed INTEGER NOT NULL DEFAULT 0, -- received
FOREIGN KEY(user_id, local_display_name)
REFERENCES display_names(user_id, local_display_name)
ON DELETE CASCADE
@@ -303,6 +305,8 @@ CREATE TABLE files(
roster_transfer_id INTEGER,
file_digest BLOB,
file_expires_at TEXT
,
feed_id INTEGER DEFAULT NULL REFERENCES feeds ON DELETE CASCADE
) STRICT;
CREATE TABLE snd_files(
file_id INTEGER NOT NULL REFERENCES files ON DELETE CASCADE,
@@ -456,6 +460,8 @@ CREATE TABLE messages(
broker_ts TEXT,
msg_chat_binding TEXT,
msg_signatures BLOB
,
feed_id INTEGER DEFAULT NULL REFERENCES feeds ON DELETE CASCADE
) STRICT;
CREATE TABLE pending_group_messages(
pending_group_message_id INTEGER PRIMARY KEY,
@@ -521,6 +527,10 @@ CREATE TABLE chat_items(
fwd_from_public_group_id BLOB,
fwd_from_member_id BLOB,
fwd_from_shared_msg_id BLOB
,
feed_id INTEGER DEFAULT NULL REFERENCES feeds ON DELETE CASCADE,
feed_item_id INTEGER DEFAULT NULL REFERENCES chat_items ON DELETE SET NULL,
item_feed INTEGER NOT NULL DEFAULT 0
) STRICT;
CREATE TABLE sqlite_sequence(name,seq);
CREATE TABLE chat_item_messages(
@@ -785,7 +795,7 @@ CREATE TABLE delivery_tasks(
) STRICT;
CREATE TABLE delivery_jobs(
delivery_job_id INTEGER PRIMARY KEY AUTOINCREMENT,
group_id INTEGER NOT NULL REFERENCES groups ON DELETE CASCADE,
group_id INTEGER REFERENCES groups ON DELETE CASCADE,
worker_scope TEXT NOT NULL,
job_scope_spec_tag TEXT,
job_scope_include_pending INTEGER,
@@ -799,6 +809,13 @@ CREATE TABLE delivery_jobs(
updated_at TEXT NOT NULL DEFAULT(datetime('now'))
,
sender_group_member_ids TEXT
,
feed_id INTEGER REFERENCES feeds ON DELETE CASCADE,
chat_item_id INTEGER REFERENCES chat_items ON DELETE CASCADE,
delete_mode TEXT,
message_ids TEXT,
cursor_contact_id INTEGER,
cursor_group_id INTEGER
) STRICT;
CREATE TABLE group_member_status_predicates(
member_status TEXT NOT NULL PRIMARY KEY,
@@ -1418,3 +1435,41 @@ BEGIN
)
WHERE group_id = NEW.group_id;
END;
CREATE TABLE feeds(
feed_id INTEGER PRIMARY KEY AUTOINCREMENT,
user_id INTEGER NOT NULL REFERENCES users ON DELETE CASCADE,
created_at TEXT NOT NULL DEFAULT(datetime('now')),
updated_at TEXT NOT NULL DEFAULT(datetime('now')),
chat_ts TEXT NOT NULL DEFAULT(datetime('now')),
favorite INTEGER NOT NULL DEFAULT 0,
unread_chat INTEGER NOT NULL DEFAULT 0
) STRICT;
CREATE INDEX idx_feeds_user_id ON feeds(user_id);
CREATE INDEX idx_chat_items_feed_id ON chat_items(feed_id);
CREATE INDEX idx_chat_items_feeds_created_at ON chat_items(
user_id,
feed_id,
created_at
);
CREATE INDEX idx_chat_items_feed_item_contact ON chat_items(
feed_item_id,
contact_id
);
CREATE INDEX idx_chat_items_feed_item_group ON chat_items(
feed_item_id,
group_id
);
CREATE INDEX idx_messages_feed_id ON messages(feed_id);
CREATE INDEX idx_files_feed_id ON files(feed_id);
CREATE INDEX idx_contacts_user_id ON contacts(user_id);
CREATE INDEX idx_groups_user_id_business_chat ON groups(
user_id,
business_chat
);
CREATE INDEX idx_delivery_jobs_feed_next ON delivery_jobs(
feed_id,
worker_scope,
failed,
job_status
);
CREATE INDEX idx_delivery_jobs_chat_item_id ON delivery_jobs(chat_item_id);
+4
View File
@@ -73,6 +73,7 @@ data ChatLockEntity
| CLUserContact Int64
| CLContactRequest Int64
| CLFile Int64
| CLFeed FeedId
deriving (Eq, Ord)
-- These error type constructors must be added to mobile apps
@@ -113,6 +114,9 @@ data StoreError
| SENoteFolderAlreadyExists {noteFolderId :: NoteFolderId}
| SENoteFolderNotFound {noteFolderId :: NoteFolderId}
| SEUserNoteFolderNotFound
| SEFeedAlreadyExists {feedId :: FeedId}
| SEFeedNotFound {feedId :: FeedId}
| SEUserFeedNotFound
| SESndFileNotFound {fileId :: FileTransferId}
| SESndFileInvalid {fileId :: FileTransferId}
| SERcvFileNotFound {fileId :: FileTransferId}
+24 -6
View File
@@ -588,20 +588,22 @@ data GroupLink = GroupLink
}
deriving (Show)
data ContactOrGroup = CGContact Contact | CGGroup GroupInfo [GroupMember]
data ContactOrGroup = CGContact Contact | CGGroup GroupInfo [GroupMember] | CGFeed Feed
data PreparedChatEntity = PCEContact Contact | PCEGroup {groupInfo :: GroupInfo, hostMember :: GroupMember}
contactAndGroupIds :: ContactOrGroup -> (Maybe ContactId, Maybe GroupId)
contactAndGroupIds :: ContactOrGroup -> (Maybe ContactId, Maybe GroupId, Maybe FeedId)
contactAndGroupIds = \case
CGContact Contact {contactId} -> (Just contactId, Nothing)
CGGroup GroupInfo {groupId} _ -> (Nothing, Just groupId)
CGContact Contact {contactId} -> (Just contactId, Nothing, Nothing)
CGGroup GroupInfo {groupId} _ -> (Nothing, Just groupId, Nothing)
CGFeed Feed {feedId} -> (Nothing, Nothing, Just feedId)
-- TODO when more settings are added we should create another type to allow partial setting updates (with all Maybe properties)
data ChatSettings = ChatSettings
{ enableNtfs :: MsgFilter,
sendRcpts :: Maybe Bool,
favorite :: Bool
favorite :: Bool,
dropFeed :: BoolDef
}
deriving (Eq, Show)
@@ -610,7 +612,8 @@ defaultChatSettings =
ChatSettings
{ enableNtfs = MFAll,
sendRcpts = Nothing,
favorite = False
favorite = False,
dropFeed = BoolDef False
}
chatHasNtfs :: ChatSettings -> Bool
@@ -2218,6 +2221,19 @@ data NoteFolder = NoteFolder
type NoteFolderId = Int64
data Feed = Feed
{ feedId :: FeedId,
userId :: UserId,
createdAt :: UTCTime,
updatedAt :: UTCTime,
chatTs :: UTCTime,
favorite :: Bool,
unread :: Bool
}
deriving (Eq, Show)
type FeedId = Int64
data ChatVersion
instance VersionScope ChatVersion
@@ -2410,4 +2426,6 @@ $(JQ.deriveJSON defaultJSON ''ContactRef)
$(JQ.deriveJSON defaultJSON ''NoteFolder)
$(JQ.deriveJSON defaultJSON ''Feed)
$(JQ.deriveJSON defaultJSON ''ChatTag)