diff --git a/plans/2026-09-04-feed-broadcast-chat.md b/plans/2026-09-04-feed-broadcast-chat.md index 2aae5e82ac..b74e643fd4 100644 --- a/plans/2026-09-04-feed-broadcast-chat.md +++ b/plans/2026-09-04-feed-broadcast-chat.md @@ -41,9 +41,20 @@ message id used here. item -> instance with feed_item_id = feed item AND the contact_id / group_id of the connection`. - Instances are ordinary sent items of their chats: per-chat editing, - deletion, reactions, expiration, disappearing messages. - `chat_items.feed_item_id` (`ON DELETE SET NULL`) links an instance to the - feed item and is the file join; `CIMeta.itemFeed` exposes it. + deletion, reactions, forwarding, expiration, disappearing messages. A + per-chat edit, or a per-chat deletion that keeps the row (a marked or a + moderated item), detaches the instance: `chat_items.item_feed` changes + from 1 (linked) to 2 (detached). Feed edits and feed deletes are applied + to linked instances only; feed file descriptions are sent for every + undeleted instance. The per-chat `XMsgUpdate` of an instance is sent with + `feed = Just True`, so a recipient with `dropFeed` discards it. Bulk + removals (`APIClearChat`, contact or group deletion, chat item expiration, + disappearing messages) remove instances; those recipients, and the + recipients of detached instances, are outside later feed edits and + deletes: the instances are the record of delivery. + `chat_items.feed_item_id` (`ON DELETE SET NULL`) links every instance, + linked or detached, to the feed item and is the file join; after the feed + item is removed a remaining instance shows no file. - A feed message takes the TTL of the chat on both sides: the sender's instance from `sndContactCITimed False ct Nothing` (`Internal.hs:174`), the recipient's item from the chat's own TTL in place of the absent message @@ -72,8 +83,11 @@ message id used here. - Wire marker: `feed :: Maybe Bool` in `MsgContainer` and in `XMsgUpdate`, serialized only as `true`. Older clients ignore the field. The chat version is unchanged. -- Received feed messages are stored with `chat_items.item_feed = 1`; - `CIMeta.itemFeed = Just CIFeedRcv`. +- `CIMeta.itemFeed :: Maybe CIFeed` marks the items of a broadcast in direct + and group chats on both sides: the sender's instances (`CIFLinked` or + `CIFDetached`) and the recipients' received feed messages (`CIFLinked`). + The feed item and every other item have `Nothing`. Received feed + messages are stored with `chat_items.item_feed = 1`. - Content: text, link, image, video, voice, file. One XFTP upload per broadcast; one recipient description for everyone. Quotes, mentions, live messages and the `ttl` parameter are rejected. @@ -88,8 +102,9 @@ message id used here. only; `SendRef` is unchanged. - `CRBroadcastSent` is removed: the broadcast is a job, so counts are not known at command time. `/feed` returns `CRNewChatItems` with the feed item. -- Commands on the feed hold a `CLFeed` entity lock and only write the feed - item and a job. The worker holds no entity locks (as relay job workers). +- Commands on the feed hold the `CLFeed feedId` entity lock and only write + the feed item and a job. The worker holds no entity locks (as relay job + workers). - Feed items are outside global chat item expiration (as notes). ## Types @@ -151,19 +166,19 @@ returns `(Maybe ContactId, Maybe GroupId, Maybe FeedId)`. matches the constructors and gains a `FeedId` branch returning `SEInternalError`: received messages have a connection or a group. - `CIMeta` (:509): field `itemFeed :: Maybe CIFeed` after `msgVerified`; - `mkCIMeta` (:535) gains the parameter; `dummyMeta` (:551) sets `Nothing`. + `mkCIMeta` (:535) gains the parameter; `deletable` and `editable` (:537-538) + are unchanged; `dummyMeta` (:551) sets `Nothing`. ```haskell - data CIFeed - = CIFeedSnd {feedItemId :: Maybe ChatItemId} - | CIFeedRcv + data CIFeed = CIFLinked | CIFDetached ``` - JSON `sumTypeJSON $ dropPrefix "CIFeed"`. Stored as - `chat_items.item_feed` (1 for both) and `chat_items.feed_item_id` - (`CIFeedSnd` only; `Nothing` after the feed item is removed or the feed is - cleared). Row mapping: `item_feed = 1` with `item_sent = 1` -> - `CIFeedSnd feed_item_id`; with `item_sent = 0` -> `CIFeedRcv`. + JSON `enumJSON $ dropPrefix "CIF"`. Stored in `chat_items.item_feed`: + 0 -> `Nothing`, 1 -> `Just CIFLinked`, 2 -> `Just CIFDetached` + (`DBCIFeedLinked`, `DBCIFeedDetached` patterns next to `DBCIDeleted`, + `Store/Messages.hs:2933`). `chat_items.feed_item_id` stays in the store: + the link of a sender's instance to the feed item for jobs, delivery events + and the file join. `Protocol.hs`: @@ -172,7 +187,9 @@ returns `(Maybe ContactId, Maybe GroupId, Maybe FeedId)`. - `XMsgUpdate` (:450): field `feed :: Maybe Bool`; parser (:1403) reads `opt "feed"`; encoder (:1486) adds `("feed" .=? feed)`. Positional patterns and constructions gain the argument: `Subscriber.hs:554`, `:730`, - `:1039`, `:1286`, `:3882`; `Commands.hs:780`, `:814`, `:1443`. + `:1039`, `:1286`, `:3882`; `Commands.hs:1443` (`Nothing`); `Commands.hs:780` + and `:814` pass `justTrue (isJust itemFeed)` (`justTrue`, + `Protocol.hs:1037`), the edited item's marker. - `cmFeed :: AChatMsgEvent -> Bool` next to `cmToQuotedMsg` (:622): `ACME _ (XMsgNew MsgContainer {feed = Just True}) -> True`. @@ -238,61 +255,50 @@ data FeedJobSpec ## Schema -SQLite `M20260904_feeds`; Postgres with `BIGINT GENERATED ALWAYS AS IDENTITY`, -`TIMESTAMPTZ`, `SMALLINT` and named constraints, as `M20241220_initial.hs:619-635`. - -```sql -CREATE TABLE feeds( - feed_id INTEGER PRIMARY KEY AUTOINCREMENT, - user_id INTEGER NOT NULL REFERENCES users ON DELETE CASCADE, - created_at TEXT NOT NULL DEFAULT(datetime('now')), - updated_at TEXT NOT NULL DEFAULT(datetime('now')), - chat_ts TEXT NOT NULL DEFAULT(datetime('now')), - favorite INTEGER NOT NULL DEFAULT 0, - unread_chat INTEGER NOT NULL DEFAULT 0 -) STRICT; -PRAGMA writable_schema=1; -UPDATE sqlite_master -SET sql = replace(sql, 'group_id INTEGER NOT NULL REFERENCES groups ON DELETE CASCADE', 'group_id INTEGER REFERENCES groups ON DELETE CASCADE') -WHERE type = 'table' AND name = 'delivery_jobs'; -PRAGMA writable_schema=RESET; -ALTER TABLE delivery_jobs ADD COLUMN feed_id INTEGER REFERENCES feeds ON DELETE CASCADE; -ALTER TABLE delivery_jobs ADD COLUMN chat_item_id INTEGER REFERENCES chat_items ON DELETE CASCADE; -ALTER TABLE delivery_jobs ADD COLUMN delete_mode TEXT; -ALTER TABLE delivery_jobs ADD COLUMN message_ids TEXT; -ALTER TABLE delivery_jobs ADD COLUMN cursor_contact_id INTEGER; -ALTER TABLE delivery_jobs ADD COLUMN cursor_group_id INTEGER; -CREATE INDEX idx_delivery_jobs_feed_next ON delivery_jobs(feed_id, worker_scope, failed, job_status); -CREATE INDEX idx_delivery_jobs_chat_item_id ON delivery_jobs(chat_item_id); -ALTER TABLE chat_items ADD COLUMN feed_id INTEGER DEFAULT NULL REFERENCES feeds ON DELETE CASCADE; -ALTER TABLE chat_items ADD COLUMN feed_item_id INTEGER DEFAULT NULL REFERENCES chat_items ON DELETE SET NULL; -ALTER TABLE chat_items ADD COLUMN item_feed INTEGER NOT NULL DEFAULT 0; -ALTER TABLE messages ADD COLUMN feed_id INTEGER DEFAULT NULL REFERENCES feeds ON DELETE CASCADE; -ALTER TABLE files ADD COLUMN feed_id INTEGER DEFAULT NULL REFERENCES feeds ON DELETE CASCADE; -ALTER TABLE contacts ADD COLUMN drop_feed INTEGER NOT NULL DEFAULT 0; -ALTER TABLE groups ADD COLUMN drop_feed INTEGER NOT NULL DEFAULT 0; -CREATE INDEX idx_feeds_user_id ON feeds(user_id); -CREATE INDEX idx_chat_items_feed_id ON chat_items(feed_id); -CREATE INDEX idx_chat_items_feeds_created_at ON chat_items(user_id, feed_id, created_at); -CREATE INDEX idx_chat_items_feed_item_contact ON chat_items(feed_item_id, contact_id); -CREATE INDEX idx_chat_items_feed_item_group ON chat_items(feed_item_id, group_id); -CREATE INDEX idx_messages_feed_id ON messages(feed_id); -CREATE INDEX idx_files_feed_id ON files(feed_id); -INSERT INTO feeds (user_id) SELECT user_id FROM users; -``` +SQLite: `M20260905_feeds.hs` (written; registered in `SQLite/Migrations.hs` +and `simplex-chat.cabal`; `chat_schema.sql` regenerated; the up, down and +repeated up dumps and `.lint fkey-indexes` verified with `sqlite3`). +Postgres: the same statements with `BIGINT GENERATED ALWAYS AS IDENTITY`, +`TIMESTAMPTZ`, `SMALLINT`, named constraints and +`ALTER TABLE delivery_jobs ALTER COLUMN group_id DROP NOT NULL`, as +`M20241220_initial.hs:619-635`. - `feed_item_id` self-reference precedent: `fwd_from_chat_item_id` - (`chat_schema.sql:504`). + (`chat_schema.sql:504`). `item_feed`: 0 none, 1 linked, 2 detached. - `delivery_jobs.group_id` becomes nullable through the `sqlite_master` - edit of `M20251230_strict_tables.hs:18-26`; Postgres uses - `ALTER COLUMN group_id DROP NOT NULL`. A feed job has `feed_id`, - `chat_item_id` (the feed item), `message_ids` and the feed cursors; a - group job has `group_id` and `cursor_group_member_id`. + edit of `M20251230_strict_tables.hs:18-26`; `PRAGMA writable_schema=RESET` + reloads the schema before the `ADD COLUMN` statements of the same + transaction, so their column offsets are computed from the edited text. + A feed job has `feed_id`, `chat_item_id` (the feed item), `message_ids` + and the feed cursors; a group job has `group_id` and + `cursor_group_member_id`. - `message_ids`: comma-separated decimal ids, the encoding of `delivery_jobs.sender_group_member_ids` (`Store/Delivery.hs:266-270, :329`). -- Registration: `SQLite/Migrations.hs`, `Postgres/Migrations.hs`, the module - list in `simplex-chat.cabal` (:85), regenerated `chat_schema.sql` dumps, the - down-migration round-trip. +- Indexes and the queries they serve (plans checked with + `EXPLAIN QUERY PLAN` on the migrated schema): + - `idx_delivery_jobs_feed_next (feed_id, worker_scope, failed, job_status)`: + `getNextDeliveryJob` by feed, a covering search. + - `idx_delivery_jobs_chat_item_id`: the `chat_item_id` FK. + - `idx_chat_items_feed_item_contact (feed_item_id, contact_id)` and + `idx_chat_items_feed_item_group (feed_item_id, group_id)`: the instance + cursor reads, the range guards, `updateFeedInstances` and the + delivery-event join (`feed_item_id = ? AND contact_id > ?`, in cursor + order without a sort). + - `idx_contacts_user_id (user_id)`: `getFeedContactsByCursor` + (`user_id = ? AND rowid > ? ORDER BY rowid`); without it the planner + picks `idx_contacts_chat_ts` and sorts every contact of the user per + bucket. + - `idx_groups_user_id_business_chat (user_id, business_chat)`: + `getFeedCustomerGroupsByCursor` + (`user_id = ? AND business_chat = ? AND rowid > ? ORDER BY rowid`). + - `idx_chat_items_feed_id`, `idx_chat_items_feeds_created_at (user_id, feed_id, created_at)`: + the `feed_id` FK and the feed chat reads. + - `idx_messages_feed_id`, `idx_files_feed_id`, `idx_feeds_user_id`: FKs + and `deleteFeedCIs`. + - Existing: `idx_msg_deliveries_message_id` (delivery guards), + `idx_group_members_group_id (user_id, group_id)` (members by group + range), `idx_chat_tags_chats_*` (tags by range), + `idx_chat_item_reactions_shared_msg_id` (feed reactions). ## Store @@ -325,16 +331,20 @@ New module `Store/Feeds.hs`: (the `getGroupMembers` condition, `Store/Groups.hs:1236`; `idx_group_members_group_id (user_id, group_id)`); the `groups` join keeps members of other groups in the id range out. - - `getFeedContactInstancesByCursor db cxt user feedItemId cursor_ count :: IO [(Contact, CChatItem 'CTDirect)]` — + - `getFeedContactInstancesByCursor db cxt user feedItemId spec cursor_ count :: IO [(Contact, CChatItem 'CTDirect)]` — the `getDirectChatItem` SELECT (`Store/Messages.hs:2700-2716`) composed with `contactQueryFields` (`:.` rows), `FROM chat_items i JOIN contacts ct ON ct.contact_id = i.contact_id` + `contactQueryFrom` joins, - `WHERE i.user_id = ? AND i.feed_item_id = ? AND i.contact_id > ? ORDER BY i.contact_id, c.connection_id LIMIT ?` - (`idx_chat_items_feed_item_contact`). - - `getFeedGroupInstancesByCursor db cxt user feedItemId cursor_ count :: IO [(GroupInfo, CChatItem 'CTGroup)]` — + `WHERE i.user_id = ? AND i.feed_item_id = ? AND i.contact_id > ? AND ORDER BY i.contact_id, c.connection_id LIMIT ?` + (`idx_chat_items_feed_item_contact`); `` is `i.item_feed = 1` for + `FJUpdate` and `FJDelete`, `i.item_deleted = 0` for `FJFileDescr`. + - `getFeedGroupInstancesByCursor db cxt user feedItemId spec cursor_ count :: IO [(GroupInfo, CChatItem 'CTGroup)]` — the `getGroupChatItem` SELECT (:3094-3139) composed with - `groupInfoQueryFields`, `WHERE i.user_id = ? AND i.feed_item_id = ? AND i.group_id > ? ORDER BY i.group_id LIMIT ?` + `groupInfoQueryFields`, `WHERE i.user_id = ? AND i.feed_item_id = ? AND i.group_id > ? AND ORDER BY i.group_id LIMIT ?` (`idx_chat_items_feed_item_group`); members from `getCustomerGroupsMembersByRange`. + - `detachFeedInstances db itemIds` — + `UPDATE chat_items SET item_feed = 2 WHERE chat_item_id = ? AND item_feed = 1` + by `executeMany`. - `getFeedInstanceContactIdsByRange db user feedItemId fromId toId :: IO [ContactId]` (and groups) — the instance guard of a repeated `FJNew` bucket. - `getDeliveredContactIdsByRange db msgId fromId toId :: IO [ContactId]` — @@ -366,7 +376,7 @@ New module `Store/Feeds.hs`: - Bucket writers, one statement or one `executeMany` each, in one transaction per bucket: `updateFeedInstances` (`item_content`, `item_text`, `item_edited = 1`, `has_link`, `updated_at` for - `user_id = ? AND feed_item_id = ? AND contact_id > ? AND contact_id <= ?`, + `user_id = ? AND feed_item_id = ? AND item_feed = 1 AND contact_id > ? AND contact_id <= ?`, and the group range), `deleteFeedInstances` and `markFeedInstancesDeleted` by `executeMany` over the ids of each subset (the full-delete split is decided in Haskell from `mergedPreferences`), reaction deletion by @@ -414,17 +424,18 @@ the `note_folders` shape (`Store/NoteFolders.hs:61`). `connOrGroupId`. `Just smId` inserts with it; `Nothing` keeps `createWithRandomId'`. `SharedMsgId` has a `ToField` instance (`Types.hs:256`). `FeedId feedId` writes `messages.feed_id`. -- `createNewChatItem_` (:590): parameter `Maybe CIFeed`; `idsRow` gains +- `createNewChatItem_` (:590): parameters `Maybe CIFeed` and + `Maybe ChatItemId` (the feed item of a sender's instance); `idsRow` gains `Maybe FeedId` from `CDFeedSnd Feed {feedId}`; the INSERT adds `feed_id`, - `feed_item_id`, `item_feed`. `createNewSndChatItem` (:548) passes the - parameter through; `createNewRcvChatItem` (:564) passes - `if cmFeed chatMsgEvent then Just CIFeedRcv else Nothing`; - `createNewChatItemNoMsg` (:583) and `createLocalChatItems` - (`Internal.hs:3185`) pass `Nothing`. -- `ChatItemModeRow` (:2281) gains `(BoolInt, Maybe ChatItemId)` from - `i.item_feed, i.feed_item_id`; `toLocalChatItem` (:1096), - `toDirectChatItem` (:2307), `toGroupChatItem` (:2375) build `Maybe CIFeed` - and pass it to `mkCIMeta`; SELECT lists at :2706, :3100, :3213. + `feed_item_id`, `item_feed`. `createNewRcvChatItem` (:564) passes + `(if cmFeed chatMsgEvent then Just CIFLinked else Nothing) Nothing`; + `createNewSndChatItem` (:548), `createNewChatItemNoMsg` (:583) and + `createLocalChatItems` (`Internal.hs:3185`) pass `Nothing Nothing`; the + feed job calls `createNewChatItem_` with `(Just CIFLinked) (Just feedItemId)`. +- `ChatItemModeRow` (:2281) gains `Int` from `i.item_feed`; + `toCIFeed :: Int -> Maybe CIFeed` maps 0, 1, 2; `toLocalChatItem` (:1096), + `toDirectChatItem` (:2307), `toGroupChatItem` (:2375) pass the value to + `mkCIMeta`; SELECT lists at :2706, :3100, :3213. - The file join in `getDirectChatItem` (:2712) and `getGroupChatItem` (:3129) becomes `LEFT JOIN files f ON f.chat_item_id = COALESCE(i.feed_item_id, i.chat_item_id)`: @@ -501,14 +512,21 @@ the `note_folders` shape (`Store/NoteFolders.hs:61`). - `createMemberSndStatuses` (`Commands.hs:4866`) moves to the top level of `Internal.hs` unchanged. - `mkChatItem_` (:2859) gains the `Maybe CIFeed` parameter for `mkCIMeta`; - `mkChatItem` (:2853), `saveRcvChatItem'` (:2825) and - `saveSndChatItems.createItem` (:2789) pass through (`NewSndChatItemData` - :2761 gains `itemFeed :: Maybe CIFeed`, `Nothing` at existing sites). + `saveRcvChatItem'` (:2825) passes + `if cmFeed chatMsgEvent then Just CIFLinked else Nothing`; `mkChatItem` + (:2853) and `saveSndChatItems.createItem` (:2793) pass `Nothing`; the feed + job passes `Just CIFLinked`. - The file-info lambda repeated in `deleteDirectCIs` (:521), `deleteGroupCIs` (:533), `deleteLocalCIs` (:597), `markDirectCIsDeleted` (:616), `markGroupCIsDeleted` (:628) becomes `itemsFilesInfo`, which skips - items with `itemFeed = Just (CIFeedSnd _)`: an instance's file is the feed - item's file and is not cancelled or deleted with the instance. + `CChatItem SMDSnd` items with `itemFeed` set: a sender's instance renders + the feed item's file, which is not cancelled or deleted with the instance. + A recipient's received feed item owns its file and is unaffected. +- `detachedInstance :: ChatItem c 'MDSnd -> ChatItem c 'MDSnd` sets + `itemFeed = Just CIFDetached`; `detachFeedInstances :: [CChatItem c] -> CM [CChatItem c]` + runs the store `detachFeedInstances` for the `CChatItem SMDSnd` items with + `itemFeed = Just CIFLinked`, returns those through `detachedInstance` and + the other items unchanged, in the input order. `Commands.hs`, `APISendFeedMessage feedId cm`, under `withFeedLock "sendFeed" feedId`: @@ -582,8 +600,9 @@ delivery, and a crash between delivery and the cursor sends nothing twice. Contact bucket, one range read: `FJNew` reads `getFeedContactsByCursor` and `getContactsTagsByRange`; the other types read -`getFeedContactInstancesByCursor` (contact and instance together) and the -tags. Then, in memory: +`getFeedContactInstancesByCursor` (contact and instance together, linked +instances for `FJUpdate` and `FJDelete`, undeleted instances for +`FJFileDescr`) and the tags. Then, in memory: 1. Eligible for `FJNew`: `directOrUsed`, not `contactConnIncognito`, `contactSendConn_` returns a connection. For the other types every @@ -595,7 +614,7 @@ tags. Then, in memory: `getFeedInstanceContactIdsByRange` are skipped; for the rest `updateChatTsStats` and `createNewChatItem_` with `CDDirectSnd ct`, no message id, the shared id, `CISndMsgContent` from the container, - `Just (CIFeedSnd (Just feedItemId))`, `timed_`; the items are built with + `Just CIFLinked`, `Just feedItemId`, `timed_`; the items are built with `mkChatItem_`. 4. Delivery: one `deliverMessagesB` over `(conn, MsgFlags {notification = hasNotification tag}, (vor, messageIds))` @@ -697,8 +716,8 @@ description per relay, forwarded to all subscribers). and `getDeliveryJobWorker True (DEFeed feedId, DWSFeed)`. 5. Feed item: `addInitialAndNewCIVersions db itemId (chatItemTs' ci, oldMC) (currentTs, mc)`; `updateFeedChatItem' db user feedId ci (CISndMsgContent mc) True`; - response `CRChatItemUpdated`. Instance versions are not recorded; the - feed item holds the history. + response `CRChatItemUpdated`. The job records no instance versions; the + feed item holds the history of feed edits. ## Deleting @@ -721,6 +740,33 @@ description per relay, forwarded to all subscribers). item in `CIDeleting`); the job removes the item at the end. 4. Response `CRChatItemsDeleted user deletions True False`. +## Per-chat editing and deletion of an instance + +`APIUpdateChatItem` on `CTDirect` (:768) and `CTGroup` (:793): the existing +paths, with two changes in the `(CISndMsgContent oldMC, Just itemSharedMId, True)` +branch: the event's `feed` is `justTrue (isJust itemFeed)` (:780, :814), and +the item passed to `updateDirectChatItem'` (:787) and `updateGroupChatItem` +(:822) is `detachedInstance ci` after `detachFeedInstances db [itemId]` in +the same transaction, when `itemFeed == Just CIFLinked`. The instance +receives its own versions from `addInitialAndNewCIVersions` (:785, :820), +starting from its content at the time of the edit. The response and +`startUpdatedTimedItemThread` are unchanged. + +`APIDeleteChatItem` on `CTDirect` (:844): `CIDMInternalMark` (:848) and the +marking branch of `CIDMBroadcast` (:859) call `markDirectCIsDeleted` with +`detachFeedInstances items`; `CIDMInternal` and the full-delete branch +remove the rows. On `CTGroup`: `CIDMInternalMark` (:870) likewise; +`delGroupChatItems` (:4154) detaches before `markGroupCIsDeleted` and +before `deleteGroupCIs` with a moderating member (`updateGroupChatItemModerated` +keeps the row, `Internal.hs:549`), which covers `CIDMBroadcast` (:877), +`CIDMHistory` (:883) and `APIDeleteMemberChatItem` (:918). Received items +in the same command are not detached (`detachFeedInstances` selects +`CChatItem SMDSnd`). + +Reactions on an instance (`APIChatItemReaction`), forwarding from an +instance, and every operation on a recipient's received feed item are +unchanged. + ## Receiving - `newContentMessage` (`Subscriber.hs:1870`): when @@ -744,8 +790,8 @@ description per relay, forwarded to all subscribers). - `newGroupContentMessage` (:2121) and `groupMessageUpdate` (:2209): the same checks against `GroupInfo {chatSettings}`; a discarded message returns `Nothing` (no delivery task). -- Received feed items get `itemFeed = Just CIFeedRcv` through - `createNewRcvChatItem` (`cmFeed`). +- Received feed items get `itemFeed = Just CIFLinked` through + `createNewRcvChatItem` and `saveRcvChatItem'` (`cmFeed`). - `APISetChatSettings` (:1898) stores `dropFeed`; `SetDropFeed cName on` uses `updateChatSettings` (:4347). The command replaces the whole record, so a client without the field (an older remote controller) resets @@ -766,7 +812,8 @@ description per relay, forwarded to all subscribers). - `APIChatRead` (:1231): `CTFeed -> getUserByFeedId; ok user`. `APIChatItemsRead`: "not supported". `APIChatUnread` (:1296): `updateFeedUnreadChat`. `APIClearChat` (:1386): `deleteFeedFiles`, - `deleteFeedCIs`; instances stay in their chats, detached by the FK. + `deleteFeedCIs`; instances stay in their chats with `feed_item_id` set + to NULL by the FK and `item_feed` unchanged. - `APIChatItemReaction` (:943): `CTFeed -> throwCmdError "not supported"`. - `APIDeleteChat`, `APISetChatTags`, `APISetChatSettings`, `APISetChatUIThemes`, `APISetChatTTL`: existing "not supported" branches. @@ -801,8 +848,10 @@ description per relay, forwarded to all subscribers). `CIDeleted.Deleting`, `CIMeta.itemFeed: CIFeed?`, `ChatSettings.dropFeed`, the `StoreError` constructors. Chat list row and chat view for the feed; a `CIDeleting` item rendered as deletion in progress; a feed marker on - instances and on received feed items; "edit" on an instance opens the feed - item when `feedItemId` is present; a "Drop feed messages" toggle in contact + instances and on received feed items; before an edit or a deletion of a + `linked` instance, a notice that the message is detached from the feed and + later feed edits and deletions skip this chat (`detached` instances show + no notice); a "Drop feed messages" toggle in contact and group settings; a privacy notice in the feed chat before the first broadcast: every recipient receives the same message id, so recipients can establish a common sender by comparing messages. @@ -822,14 +871,16 @@ description per relay, forwarded to all subscribers). the UI refreshes the feed item meanwhile. - `getChatRefViaItemId` (`Store/Messages.hs:3323`) for local and feed items. - Forwarding into the feed. +- Navigation from an instance to its feed item (a lookup by the shared + `itemSharedMsgId` in the feed chat). ## Tests `ChatTests`, with `feedBucketSize = 2` in the test config: 1. `/feed` to three contacts and a customer group: instances in each chat - with `itemFeed = CIFeedSnd`; recipients' items with `CIFeedRcv`; the feed - item reaches `CISSndSent SSPComplete`. + and recipients' items with `itemFeed = Just CIFLinked`; the feed item + reaches `CISSndSent SSPComplete`. 2. Instance statuses after `SENT` and receipts. 3. A contact with `dropFeed` set receives nothing; a later feed edit and delete are silent. @@ -864,3 +915,11 @@ description per relay, forwarded to all subscribers). the following file description. 15. `Direct.hs:995`, `Groups.hs:6731`, `Bots/BroadcastTests.hs` updated to `CRNewChatItems`. +16. Per-chat edit of one instance: the recipient receives the edit, the + instance becomes `CIFDetached` with its own versions; a following feed + edit updates the other instances and recipients, and this chat keeps the + per-chat text; a following feed delete keeps this instance and the + recipient's message. A per-chat broadcast delete of an instance in a + chat without full delete marks and detaches it; a recipient with + `dropFeed` set receives nothing from a per-chat edit of a dropped + message. diff --git a/simplex-chat.cabal b/simplex-chat.cabal index 3c50d92968..719d98bee2 100644 --- a/simplex-chat.cabal +++ b/simplex-chat.cabal @@ -329,6 +329,7 @@ library Simplex.Chat.Store.SQLite.Migrations.M20260813_auto_accept_group_invitations Simplex.Chat.Store.SQLite.Migrations.M20260822_forward_link Simplex.Chat.Store.SQLite.Migrations.M20260828_file_expiry + Simplex.Chat.Store.SQLite.Migrations.M20260905_feeds other-modules: Paths_simplex_chat hs-source-dirs: diff --git a/src/Simplex/Chat.hs b/src/Simplex/Chat.hs index 3eb2045317..f53522096c 100644 --- a/src/Simplex/Chat.hs +++ b/src/Simplex/Chat.hs @@ -120,6 +120,7 @@ defaultChatConfig = highlyAvailable = False, deliveryWorkerDelay = 0, deliveryBucketSize = 10000, + feedBucketSize = 1000, webPreviewConfig = Nothing, channelSubscriberRole = GRObserver, relayChecksInterval = 15 * 60, -- 15 minutes diff --git a/src/Simplex/Chat/Controller.hs b/src/Simplex/Chat/Controller.hs index 54dfb58fc3..b8f3367459 100644 --- a/src/Simplex/Chat/Controller.hs +++ b/src/Simplex/Chat/Controller.hs @@ -165,6 +165,7 @@ data ChatConfig = ChatConfig ciExpirationInterval :: Int64, -- microseconds deliveryWorkerDelay :: Int64, -- microseconds deliveryBucketSize :: Int, + feedBucketSize :: Int, webPreviewConfig :: Maybe WebPreviewConfig, channelSubscriberRole :: GroupMemberRole, -- TODO [relays] starting role should be communicated in protocol from owner to relays relayChecksInterval :: NominalDiffTime, @@ -392,6 +393,7 @@ data ChatCommand | APIUpdateChatTag ChatTagId ChatTagData | APIReorderChatTags (NonEmpty ChatTagId) | APICreateChatItems {noteFolderId :: NoteFolderId, composedMessages :: NonEmpty ComposedMessage} + | APISendFeedMessage {feedId :: FeedId, composedMessage :: ComposedMessage} | APIReportMessage {groupId :: GroupId, chatItemId :: ChatItemId, reportReason :: ReportReason, reportText :: Text} | ReportMessage {groupName :: GroupName, contactName_ :: Maybe ContactName, reportReason :: ReportReason, reportedMessage :: Text} | APIUpdateChatItem {chatRef :: ChatRef, chatItemId :: ChatItemId, liveMessage :: Bool, updatedMessage :: UpdatedMessage} @@ -514,6 +516,7 @@ data ChatCommand | APIEnableGroupMember GroupId GroupMemberId | SetShowMessages ChatName MsgFilter | SetSendReceipts ChatName (Maybe Bool) + | SetDropFeed ChatName Bool | SetShowMemberMessages GroupName ContactName Bool | ContactInfo ContactName | ShowGroupInfo GroupName diff --git a/src/Simplex/Chat/Delivery.hs b/src/Simplex/Chat/Delivery.hs index a6ccc74247..7a6efb8135 100644 --- a/src/Simplex/Chat/Delivery.hs +++ b/src/Simplex/Chat/Delivery.hs @@ -10,7 +10,8 @@ import Data.ByteString.Char8 (ByteString) import Data.Int (Int64) import Data.Maybe (fromMaybe) import Data.Time.Clock (UTCTime) -import Simplex.Chat.Messages (GroupChatScopeInfo (..), MessageId, ShowGroupAsSender) +import Simplex.Chat.Messages (ChatItemId, GroupChatScopeInfo (..), MessageId, ShowGroupAsSender) +import Simplex.Chat.Messages.CIContent (CIDeleteMode (..)) import Simplex.Chat.Options.DB (FromField (..), ToField (..)) import Simplex.Chat.Protocol import Simplex.Chat.Types @@ -18,11 +19,15 @@ import Simplex.Chat.Types.Shared import Simplex.Messaging.Agent.Store.DB (fromTextField_) import Simplex.Messaging.Encoding.String -type DeliveryWorkerKey = (GroupId, DeliveryWorkerScope) +data DeliveryEntity = DEGroup GroupId | DEFeed FeedId + deriving (Eq, Ord, Show) + +type DeliveryWorkerKey = (DeliveryEntity, DeliveryWorkerScope) data DeliveryWorkerScope = DWSGroup | DWSMemberSupport + | DWSFeed -- | DWSMemberProfileUpdate deriving (Eq, Ord, Show) @@ -34,11 +39,13 @@ instance TextEncoding DeliveryWorkerScope where textDecode = \case "group" -> Just DWSGroup "member_support" -> Just DWSMemberSupport + "feed" -> Just DWSFeed -- "member_profile_update" -> Just DWSMemberProfileUpdate _ -> Nothing textEncode = \case DWSGroup -> "group" DWSMemberSupport -> "member_support" + DWSFeed -> "feed" -- DWSMemberProfileUpdate -> "member_profile_update" -- Context for creating a delivery task. Separate from DeliveryJobScope because @@ -54,17 +61,30 @@ data DeliveryTaskContext = DeliveryTaskContext data DeliveryJobScope = DJSGroup {jobSpec :: DeliveryJobSpec} | DJSMemberSupport {supportGMId :: GroupMemberId} + | DJSFeed {feedItemId :: ChatItemId, feedJobSpec :: FeedJobSpec} -- | DJSMemberProfileUpdate deriving (Show) data DeliveryJobSpec = DJDeliveryJob {includePending :: Bool} | DJRelayRemoved + | DJFeed FeedJobSpec + deriving (Show) + +data FeedJobSpec + = FJNew + | FJFileDescr + | FJUpdate + | FJDelete CIDeleteMode deriving (Show) data DeliveryJobSpecTag = DJSTDeliveryJob | DJSTRelayRemoved + | DJSTFeedNew + | DJSTFeedFileDescr + | DJSTFeedUpdate + | DJSTFeedDelete deriving (Show) instance FromField DeliveryJobSpecTag where fromField = fromTextField_ textDecode @@ -75,15 +95,24 @@ instance TextEncoding DeliveryJobSpecTag where textDecode = \case "delivery_job" -> Just DJSTDeliveryJob "relay_removed" -> Just DJSTRelayRemoved + "feed_new" -> Just DJSTFeedNew + "feed_file_descr" -> Just DJSTFeedFileDescr + "feed_update" -> Just DJSTFeedUpdate + "feed_delete" -> Just DJSTFeedDelete _ -> Nothing textEncode = \case DJSTDeliveryJob -> "delivery_job" DJSTRelayRemoved -> "relay_removed" + DJSTFeedNew -> "feed_new" + DJSTFeedFileDescr -> "feed_file_descr" + DJSTFeedUpdate -> "feed_update" + DJSTFeedDelete -> "feed_delete" toWorkerScope :: DeliveryJobScope -> DeliveryWorkerScope toWorkerScope = \case DJSGroup _ -> DWSGroup DJSMemberSupport _ -> DWSMemberSupport + DJSFeed {} -> DWSFeed -- DJSMemberProfileUpdate -> DWSMemberProfileUpdate isRelayRemoved :: DeliveryJobScope -> Bool @@ -97,11 +126,13 @@ jobScopeImpliedSpec :: DeliveryJobScope -> DeliveryJobSpec jobScopeImpliedSpec = \case DJSGroup {jobSpec} -> jobSpec DJSMemberSupport {} -> DJDeliveryJob {includePending = False} + DJSFeed {feedJobSpec} -> DJFeed feedJobSpec jobSpecImpliedPending :: DeliveryJobSpec -> Bool jobSpecImpliedPending = \case DJDeliveryJob {includePending} -> includePending DJRelayRemoved -> True + DJFeed _ -> False infoToDeliveryContext :: GroupInfo -> Maybe GroupChatScopeInfo -> ShowGroupAsSender -> DeliveryTaskContext infoToDeliveryContext GroupInfo {membership} scopeInfo sentAsGroup = DeliveryTaskContext {jobScope, sentAsGroup} @@ -162,8 +193,11 @@ data MessageDeliveryJob = MessageDeliveryJob { jobId :: Int64, jobScope :: DeliveryJobScope, senderGMIds :: [GroupMemberId], + messageIds :: [MessageId], body :: ByteString, - cursorGMId_ :: Maybe GroupMemberId + cursorGMId_ :: Maybe GroupMemberId, + cursorContactId_ :: Maybe ContactId, + cursorGroupId_ :: Maybe GroupId } deriving (Show) diff --git a/src/Simplex/Chat/Messages.hs b/src/Simplex/Chat/Messages.hs index 5a62ca7a81..62cceead43 100644 --- a/src/Simplex/Chat/Messages.hs +++ b/src/Simplex/Chat/Messages.hs @@ -58,7 +58,7 @@ import Simplex.Messaging.Parsers (defaultJSON, dropPrefix, enumJSON, parseAll, s import Simplex.Messaging.Protocol (BlockingInfo, MsgBody, XFTPServer) import Simplex.Messaging.Util (eitherToMaybe, safeDecodeUtf8, (<$?>)) -data ChatType = CTDirect | CTGroup | CTLocal | CTContactRequest | CTContactConnection +data ChatType = CTDirect | CTGroup | CTLocal | CTFeed | CTContactRequest | CTContactConnection deriving (Eq, Show, Ord) $(JQ.deriveJSON (enumJSON $ dropPrefix "CT") ''ChatType) @@ -67,6 +67,7 @@ data SChatType (c :: ChatType) where SCTDirect :: SChatType 'CTDirect SCTGroup :: SChatType 'CTGroup SCTLocal :: SChatType 'CTLocal + SCTFeed :: SChatType 'CTFeed SCTContactRequest :: SChatType 'CTContactRequest SCTContactConnection :: SChatType 'CTContactConnection @@ -76,6 +77,7 @@ instance TestEquality SChatType where testEquality SCTDirect SCTDirect = Just Refl testEquality SCTGroup SCTGroup = Just Refl testEquality SCTLocal SCTLocal = Just Refl + testEquality SCTFeed SCTFeed = Just Refl testEquality SCTContactRequest SCTContactRequest = Just Refl testEquality SCTContactConnection SCTContactConnection = Just Refl testEquality _ _ = Nothing @@ -91,6 +93,8 @@ instance ChatTypeI 'CTGroup where chatTypeI = SCTGroup instance ChatTypeI 'CTLocal where chatTypeI = SCTLocal +instance ChatTypeI 'CTFeed where chatTypeI = SCTFeed + instance ChatTypeI 'CTContactRequest where chatTypeI = SCTContactRequest instance ChatTypeI 'CTContactConnection where chatTypeI = SCTContactConnection @@ -100,6 +104,7 @@ toChatType = \case SCTDirect -> CTDirect SCTGroup -> CTGroup SCTLocal -> CTLocal + SCTFeed -> CTFeed SCTContactRequest -> CTContactRequest SCTContactConnection -> CTContactConnection @@ -108,6 +113,7 @@ aChatType = \case CTDirect -> ACT SCTDirect CTGroup -> ACT SCTGroup CTLocal -> ACT SCTLocal + CTFeed -> ACT SCTFeed CTContactRequest -> ACT SCTContactRequest CTContactConnection -> ACT SCTContactConnection @@ -157,6 +163,7 @@ chatTypeStr = \case CTDirect -> "@" CTGroup -> "#" CTLocal -> "*" + CTFeed -> "%" CTContactRequest -> "<@" CTContactConnection -> ":" @@ -170,6 +177,7 @@ data ChatInfo (c :: ChatType) where DirectChat :: Contact -> ChatInfo 'CTDirect GroupChat :: GroupInfo -> Maybe GroupChatScopeInfo -> ChatInfo 'CTGroup LocalChat :: NoteFolder -> ChatInfo 'CTLocal + FeedChat :: Feed -> ChatInfo 'CTFeed ContactRequest :: UserContactRequest -> ChatInfo 'CTContactRequest ContactConnection :: PendingContactConnection -> ChatInfo 'CTContactConnection CInfoInvalidJSON :: SChatType c -> J.Object -> ChatInfo c -- this constructor is needed to catch JSON errors for Remote connection parsing @@ -192,6 +200,7 @@ chatInfoToRef = \case DirectChat Contact {contactId} -> Just $ ChatRef CTDirect contactId Nothing GroupChat GroupInfo {groupId} scopeInfo -> Just $ ChatRef CTGroup groupId (toChatScope <$> scopeInfo) LocalChat NoteFolder {noteFolderId} -> Just $ ChatRef CTLocal noteFolderId Nothing + FeedChat Feed {feedId} -> Just $ ChatRef CTFeed feedId Nothing ContactRequest UserContactRequest {contactRequestId} -> Just $ ChatRef CTContactRequest contactRequestId Nothing ContactConnection PendingContactConnection {pccConnId} -> Just $ ChatRef CTContactConnection pccConnId Nothing CInfoInvalidJSON {} -> Nothing @@ -205,6 +214,7 @@ data JSONChatInfo = JCInfoDirect {contact :: Contact} | JCInfoGroup {groupInfo :: GroupInfo, groupChatScope :: Maybe GroupChatScopeInfo} | JCInfoLocal {noteFolder :: NoteFolder} + | JCInfoFeed {feed :: Feed} | JCInfoContactRequest {contactRequest :: UserContactRequest} | JCInfoContactConnection {contactConnection :: PendingContactConnection} | JCInfoInvalidJSON {chatType :: ChatType, json :: J.Object} @@ -231,6 +241,7 @@ jsonChatInfo = \case DirectChat c -> JCInfoDirect c GroupChat g s -> JCInfoGroup g s LocalChat l -> JCInfoLocal l + FeedChat f -> JCInfoFeed f ContactRequest g -> JCInfoContactRequest g ContactConnection c -> JCInfoContactConnection c CInfoInvalidJSON c o -> JCInfoInvalidJSON (toChatType c) o @@ -244,6 +255,7 @@ jsonAChatInfo = \case JCInfoDirect c -> AChatInfo SCTDirect $ DirectChat c JCInfoGroup g s -> AChatInfo SCTGroup $ GroupChat g s JCInfoLocal l -> AChatInfo SCTLocal $ LocalChat l + JCInfoFeed f -> AChatInfo SCTFeed $ FeedChat f JCInfoContactRequest g -> AChatInfo SCTContactRequest $ ContactRequest g JCInfoContactConnection c -> AChatInfo SCTContactConnection $ ContactConnection c JCInfoInvalidJSON cType o -> case aChatType cType of ACT c -> AChatInfo c $ CInfoInvalidJSON c o @@ -298,6 +310,7 @@ data CIDirection (c :: ChatType) (d :: MsgDirection) where CIChannelRcv :: CIDirection 'CTGroup 'MDRcv CILocalSnd :: CIDirection 'CTLocal 'MDSnd CILocalRcv :: CIDirection 'CTLocal 'MDRcv + CIFeedSnd :: CIDirection 'CTFeed 'MDSnd deriving instance Show (CIDirection c d) @@ -313,6 +326,7 @@ data JSONCIDirection | JCIChannelRcv | JCILocalSnd | JCILocalRcv + | JCIFeedSnd deriving (Show) jsonCIDirection :: CIDirection c d -> JSONCIDirection @@ -324,6 +338,7 @@ jsonCIDirection = \case CIChannelRcv -> JCIChannelRcv CILocalSnd -> JCILocalSnd CILocalRcv -> JCILocalRcv + CIFeedSnd -> JCIFeedSnd jsonACIDirection :: JSONCIDirection -> ACIDirection jsonACIDirection = \case @@ -334,6 +349,7 @@ jsonACIDirection = \case JCIChannelRcv -> ACID SCTGroup SMDRcv CIChannelRcv JCILocalSnd -> ACID SCTLocal SMDSnd CILocalSnd JCILocalRcv -> ACID SCTLocal SMDRcv CILocalRcv + JCIFeedSnd -> ACID SCTFeed SMDSnd CIFeedSnd data CIReactionCount = CIReactionCount {reaction :: MsgReaction, userReacted :: Bool, totalReacted :: Int} deriving (Show) @@ -396,6 +412,7 @@ data ChatDirection (c :: ChatType) (d :: MsgDirection) where CDChannelRcv :: GroupInfo -> Maybe GroupChatScopeInfo -> ChatDirection 'CTGroup 'MDRcv CDLocalSnd :: NoteFolder -> ChatDirection 'CTLocal 'MDSnd CDLocalRcv :: NoteFolder -> ChatDirection 'CTLocal 'MDRcv + CDFeedSnd :: Feed -> ChatDirection 'CTFeed 'MDSnd toCIDirection :: ChatDirection c d -> CIDirection c d toCIDirection = \case @@ -406,6 +423,7 @@ toCIDirection = \case CDChannelRcv _ _ -> CIChannelRcv CDLocalSnd _ -> CILocalSnd CDLocalRcv _ -> CILocalRcv + CDFeedSnd _ -> CIFeedSnd toChatInfo :: ChatDirection c d -> ChatInfo c toChatInfo = \case @@ -416,6 +434,7 @@ toChatInfo = \case CDChannelRcv g s -> GroupChat g s CDLocalSnd l -> LocalChat l CDLocalRcv l -> LocalChat l + CDFeedSnd f -> FeedChat f signMessagesRequired :: ChatDirection c d -> Bool signMessagesRequired = \case @@ -525,6 +544,7 @@ data CIMeta (c :: ChatType) (d :: MsgDirection) = CIMeta forwardedByMember :: Maybe GroupMemberId, showGroupAsSender :: ShowGroupAsSender, msgVerified :: Maybe MsgVerified, + itemFeed :: Maybe CIFeed, createdAt :: UTCTime, updatedAt :: UTCTime } @@ -532,12 +552,15 @@ data CIMeta (c :: ChatType) (d :: MsgDirection) = CIMeta type ShowGroupAsSender = Bool -mkCIMeta :: forall c d. ChatTypeI c => ChatItemId -> CIContent d -> Text -> CIStatus d -> Maybe Bool -> Maybe SharedMsgId -> Maybe CIForwardedFrom -> Maybe (CIDeleted c) -> Bool -> Maybe CITimed -> Maybe Bool -> Bool -> Bool -> UTCTime -> ChatItemTs -> Maybe GroupMemberId -> Bool -> Maybe MsgVerified -> UTCTime -> UTCTime -> CIMeta c d -mkCIMeta itemId itemContent itemText itemStatus sentViaProxy itemSharedMsgId itemForwarded itemDeleted itemEdited itemTimed itemLive userMention hasLink_ currentTs itemTs forwardedByMember showGroupAsSender msgVerified createdAt updatedAt = +data CIFeed = CIFLinked | CIFDetached + deriving (Eq, Show) + +mkCIMeta :: forall c d. ChatTypeI c => ChatItemId -> CIContent d -> Text -> CIStatus d -> Maybe Bool -> Maybe SharedMsgId -> Maybe CIForwardedFrom -> Maybe (CIDeleted c) -> Bool -> Maybe CITimed -> Maybe Bool -> Bool -> Bool -> UTCTime -> ChatItemTs -> Maybe GroupMemberId -> Bool -> Maybe MsgVerified -> Maybe CIFeed -> UTCTime -> UTCTime -> CIMeta c d +mkCIMeta itemId itemContent itemText itemStatus sentViaProxy itemSharedMsgId itemForwarded itemDeleted itemEdited itemTimed itemLive userMention hasLink_ currentTs itemTs forwardedByMember showGroupAsSender msgVerified itemFeed createdAt updatedAt = let deletable = deletable' itemContent itemDeleted itemTs nominalDay currentTs editable = deletable && isNothing itemForwarded hasLink = BoolDef hasLink_ - in CIMeta {itemId, itemTs, itemText, itemStatus, sentViaProxy, itemSharedMsgId, itemForwarded, itemDeleted, itemEdited, itemTimed, itemLive, userMention, hasLink, deletable, editable, forwardedByMember, showGroupAsSender, msgVerified, createdAt, updatedAt} + in CIMeta {itemId, itemTs, itemText, itemStatus, sentViaProxy, itemSharedMsgId, itemForwarded, itemDeleted, itemEdited, itemTimed, itemLive, userMention, hasLink, deletable, editable, forwardedByMember, showGroupAsSender, msgVerified, itemFeed, createdAt, updatedAt} deletable' :: forall c d. ChatTypeI c => CIContent d -> Maybe (CIDeleted c) -> UTCTime -> NominalDiffTime -> UTCTime -> Bool deletable' itemContent itemDeleted itemTs allowedInterval currentTs = @@ -569,6 +592,7 @@ dummyMeta itemId ts itemText = forwardedByMember = Nothing, showGroupAsSender = False, msgVerified = Nothing, + itemFeed = Nothing, createdAt = ts, updatedAt = ts } @@ -673,6 +697,7 @@ jsonACIQDirection = \case JCIChannelRcv -> Right $ ACIQDirection SCTGroup $ CIQGroupRcv Nothing JCILocalSnd -> Left "unquotable" JCILocalRcv -> Left "unquotable" + JCIFeedSnd -> Left "unquotable" quoteMsgDirection :: CIQDirection c -> MsgDirection quoteMsgDirection = \case @@ -1187,7 +1212,7 @@ data RcvMessage = RcvMessage type MessageId = Int64 -data ConnOrGroupId = ConnectionId Int64 | GroupId Int64 +data ConnOrGroupId = ConnectionId Int64 | GroupId Int64 | FeedId Int64 data SndMsgDelivery = SndMsgDelivery { connId :: Int64, @@ -1284,6 +1309,7 @@ data CIDeleted (c :: ChatType) where CIBlocked :: Maybe UTCTime -> CIDeleted 'CTGroup CIBlockedByAdmin :: Maybe UTCTime -> CIDeleted 'CTGroup CIModerated :: Maybe UTCTime -> GroupMember -> CIDeleted 'CTGroup + CIDeleting :: Maybe UTCTime -> CIDeleted 'CTFeed deriving instance Show (CIDeleted c) @@ -1294,6 +1320,7 @@ data JSONCIDeleted | JCIDBlocked {deletedTs :: Maybe UTCTime} | JCIDBlockedByAdmin {deletedTs :: Maybe UTCTime} | JCIDModerated {deletedTs :: Maybe UTCTime, byGroupMember :: GroupMember} + | JCIDDeleting {deletedTs :: Maybe UTCTime} deriving (Show) jsonCIDeleted :: forall d. ChatTypeI d => CIDeleted d -> JSONCIDeleted @@ -1302,6 +1329,7 @@ jsonCIDeleted = \case CIBlocked ts -> JCIDBlocked ts CIBlockedByAdmin ts -> JCIDBlockedByAdmin ts CIModerated ts m -> JCIDModerated ts m + CIDeleting ts -> JCIDDeleting ts jsonACIDeleted :: JSONCIDeleted -> ACIDeleted jsonACIDeleted = \case @@ -1309,6 +1337,7 @@ jsonACIDeleted = \case JCIDBlocked ts -> ACIDeleted SCTGroup $ CIBlocked ts JCIDBlockedByAdmin ts -> ACIDeleted SCTGroup $ CIBlockedByAdmin ts JCIDModerated ts m -> ACIDeleted SCTGroup (CIModerated ts m) + JCIDDeleting ts -> ACIDeleted SCTFeed $ CIDeleting ts itemDeletedTs :: CIDeleted d -> Maybe UTCTime itemDeletedTs = \case @@ -1316,6 +1345,7 @@ itemDeletedTs = \case CIBlocked ts -> ts CIBlockedByAdmin ts -> ts CIModerated ts _ -> ts + CIDeleting ts -> ts data CIForwardedFrom = CIFFUnknown @@ -1445,6 +1475,8 @@ $(JQ.deriveJSON defaultJSON ''MemberDeliveryStatus) $(JQ.deriveJSON defaultJSON ''ChatItemVersion) +$(JQ.deriveJSON (enumJSON $ dropPrefix "CIF") ''CIFeed) + instance (ChatTypeI c, MsgDirectionI d) => FromJSON (CIMeta c d) where parseJSON = $(JQ.mkParseJSON defaultJSON ''CIMeta) diff --git a/src/Simplex/Chat/Protocol.hs b/src/Simplex/Chat/Protocol.hs index 7c932d89ad..7ff0d427de 100644 --- a/src/Simplex/Chat/Protocol.hs +++ b/src/Simplex/Chat/Protocol.hs @@ -447,7 +447,7 @@ signChatMsgBody MsgSigning {bindingTag, bindingData, keyRef, privKey} msgBody = data ChatMsgEvent (e :: MsgEncoding) where XMsgNew :: MsgContainer -> ChatMsgEvent 'Json XMsgFileDescr :: {msgId :: SharedMsgId, fileDescr :: FileDescr, fileExpires :: Maybe UTCTime} -> ChatMsgEvent 'Json - XMsgUpdate :: {msgId :: SharedMsgId, content :: MsgContent, mentions :: Map MemberName MsgMention, ttl :: Maybe Int, live :: Maybe Bool, scope :: Maybe MsgScope, asGroup :: Maybe Bool} -> ChatMsgEvent 'Json + XMsgUpdate :: {msgId :: SharedMsgId, content :: MsgContent, mentions :: Map MemberName MsgMention, ttl :: Maybe Int, live :: Maybe Bool, scope :: Maybe MsgScope, asGroup :: Maybe Bool, feed :: Maybe Bool} -> ChatMsgEvent 'Json XMsgDel :: {msgId :: SharedMsgId, memberId :: Maybe MemberId, scope :: Maybe MsgScope, onlyHistory :: Bool} -> ChatMsgEvent 'Json XMsgDeleted :: ChatMsgEvent 'Json XMsgReact :: {msgId :: SharedMsgId, memberId :: Maybe MemberId, scope :: Maybe MsgScope, reaction :: MsgReaction, add :: Bool} -> ChatMsgEvent 'Json @@ -689,7 +689,8 @@ data MsgContainer = MsgContainer quote :: Maybe QuotedMsg, parent :: Maybe MsgRef, forward :: Maybe Bool, - forwardLink :: Maybe ForwardLink + forwardLink :: Maybe ForwardLink, + feed :: Maybe Bool } deriving (Eq, Show) @@ -715,7 +716,8 @@ mcSimple content = quote = Nothing, parent = Nothing, forward = Nothing, - forwardLink = Nothing + forwardLink = Nothing, + feed = Nothing } mcQuote :: QuotedMsg -> MsgContent -> MsgContainer @@ -1400,7 +1402,8 @@ appJsonToCM AppMessageJson {v, msgId, event, params} = do live <- opt "live" scope <- opt "scope" asGroup <- opt "asGroup" - pure XMsgUpdate {msgId = msgId', content, mentions, ttl, live, scope, asGroup} + feed <- opt "feed" + pure XMsgUpdate {msgId = msgId', content, mentions, ttl, live, scope, asGroup, feed} XMsgDel_ -> XMsgDel <$> p "msgId" <*> opt "memberId" <*> opt "scope" <*> (fromMaybe False <$> opt "onlyHistory") XMsgDeleted_ -> pure XMsgDeleted XMsgReact_ -> XMsgReact <$> p "msgId" <*> opt "memberId" <*> opt "scope" <*> p "reaction" <*> p "add" @@ -1483,7 +1486,7 @@ chatToAppMessage chatMsg@ChatMessage {chatVRange, msgId, chatMsgEvent} = case en J.Object obj -> obj _ -> JM.empty XMsgFileDescr msgId' fileDescr fileExpires -> o $ ("fileExpires" .=? fileExpires) ["msgId" .= msgId', "fileDescr" .= fileDescr] - XMsgUpdate {msgId = msgId', content, mentions, ttl, live, scope, asGroup} -> o $ ("asGroup" .=? asGroup) $ ("ttl" .=? ttl) $ ("live" .=? live) $ ("scope" .=? scope) $ ("mentions" .=? nonEmptyMap mentions) ["msgId" .= msgId', "content" .= content] + XMsgUpdate {msgId = msgId', content, mentions, ttl, live, scope, asGroup, feed} -> o $ ("feed" .=? feed) $ ("asGroup" .=? asGroup) $ ("ttl" .=? ttl) $ ("live" .=? live) $ ("scope" .=? scope) $ ("mentions" .=? nonEmptyMap mentions) ["msgId" .= msgId', "content" .= content] XMsgDel msgId' memberId scope onlyHistory -> o $ ("memberId" .=? memberId) $ ("scope" .=? scope) $ ("onlyHistory" .=? justTrue onlyHistory) ["msgId" .= msgId'] XMsgDeleted -> JM.empty XMsgReact msgId' memberId scope reaction add -> o $ ("memberId" .=? memberId) $ ("scope" .=? scope) ["msgId" .= msgId', "reaction" .= reaction, "add" .= add] diff --git a/src/Simplex/Chat/Store/SQLite/Migrations.hs b/src/Simplex/Chat/Store/SQLite/Migrations.hs index e3f5f9385b..96f933b8ad 100644 --- a/src/Simplex/Chat/Store/SQLite/Migrations.hs +++ b/src/Simplex/Chat/Store/SQLite/Migrations.hs @@ -172,6 +172,7 @@ import Simplex.Chat.Store.SQLite.Migrations.M20260723_contact_request_rejection import Simplex.Chat.Store.SQLite.Migrations.M20260813_auto_accept_group_invitations import Simplex.Chat.Store.SQLite.Migrations.M20260822_forward_link import Simplex.Chat.Store.SQLite.Migrations.M20260828_file_expiry +import Simplex.Chat.Store.SQLite.Migrations.M20260905_feeds import Simplex.Messaging.Agent.Store.Shared (Migration (..)) schemaMigrations :: [(String, Query, Maybe Query)] @@ -343,7 +344,8 @@ schemaMigrations = ("20260723_contact_request_rejection", m20260723_contact_request_rejection, Just down_m20260723_contact_request_rejection), ("20260813_auto_accept_group_invitations", m20260813_auto_accept_group_invitations, Just down_m20260813_auto_accept_group_invitations), ("20260822_forward_link", m20260822_forward_link, Just down_m20260822_forward_link), - ("20260828_file_expiry", m20260828_file_expiry, Just down_m20260828_file_expiry) + ("20260828_file_expiry", m20260828_file_expiry, Just down_m20260828_file_expiry), + ("20260905_feeds", m20260905_feeds, Just down_m20260905_feeds) ] -- | The list of migrations in ascending order by date diff --git a/src/Simplex/Chat/Store/SQLite/Migrations/M20260905_feeds.hs b/src/Simplex/Chat/Store/SQLite/Migrations/M20260905_feeds.hs new file mode 100644 index 0000000000..efa954e201 --- /dev/null +++ b/src/Simplex/Chat/Store/SQLite/Migrations/M20260905_feeds.hs @@ -0,0 +1,102 @@ +{-# LANGUAGE QuasiQuotes #-} + +module Simplex.Chat.Store.SQLite.Migrations.M20260905_feeds where + +import Database.SQLite.Simple (Query) +import Database.SQLite.Simple.QQ (sql) + +m20260905_feeds :: Query +m20260905_feeds = + [sql| +CREATE TABLE feeds( + feed_id INTEGER PRIMARY KEY AUTOINCREMENT, + user_id INTEGER NOT NULL REFERENCES users ON DELETE CASCADE, + created_at TEXT NOT NULL DEFAULT(datetime('now')), + updated_at TEXT NOT NULL DEFAULT(datetime('now')), + chat_ts TEXT NOT NULL DEFAULT(datetime('now')), + favorite INTEGER NOT NULL DEFAULT 0, + unread_chat INTEGER NOT NULL DEFAULT 0 +) STRICT; +CREATE INDEX idx_feeds_user_id ON feeds(user_id); +INSERT INTO feeds (user_id) SELECT user_id FROM users; + +ALTER TABLE chat_items ADD COLUMN feed_id INTEGER DEFAULT NULL REFERENCES feeds ON DELETE CASCADE; +ALTER TABLE chat_items ADD COLUMN feed_item_id INTEGER DEFAULT NULL REFERENCES chat_items ON DELETE SET NULL; +ALTER TABLE chat_items ADD COLUMN item_feed INTEGER NOT NULL DEFAULT 0; +CREATE INDEX idx_chat_items_feed_id ON chat_items(feed_id); +CREATE INDEX idx_chat_items_feeds_created_at ON chat_items(user_id, feed_id, created_at); +CREATE INDEX idx_chat_items_feed_item_contact ON chat_items(feed_item_id, contact_id); +CREATE INDEX idx_chat_items_feed_item_group ON chat_items(feed_item_id, group_id); + +ALTER TABLE messages ADD COLUMN feed_id INTEGER DEFAULT NULL REFERENCES feeds ON DELETE CASCADE; +CREATE INDEX idx_messages_feed_id ON messages(feed_id); + +ALTER TABLE files ADD COLUMN feed_id INTEGER DEFAULT NULL REFERENCES feeds ON DELETE CASCADE; +CREATE INDEX idx_files_feed_id ON files(feed_id); + +ALTER TABLE contacts ADD COLUMN drop_feed INTEGER NOT NULL DEFAULT 0; +ALTER TABLE groups ADD COLUMN drop_feed INTEGER NOT NULL DEFAULT 0; +CREATE INDEX idx_contacts_user_id ON contacts(user_id); +CREATE INDEX idx_groups_user_id_business_chat ON groups(user_id, business_chat); + +PRAGMA writable_schema=1; + +UPDATE sqlite_master +SET sql = replace(sql, 'group_id INTEGER NOT NULL REFERENCES groups ON DELETE CASCADE', 'group_id INTEGER REFERENCES groups ON DELETE CASCADE') +WHERE type = 'table' AND name = 'delivery_jobs'; + +PRAGMA writable_schema=RESET; + +ALTER TABLE delivery_jobs ADD COLUMN feed_id INTEGER REFERENCES feeds ON DELETE CASCADE; +ALTER TABLE delivery_jobs ADD COLUMN chat_item_id INTEGER REFERENCES chat_items ON DELETE CASCADE; +ALTER TABLE delivery_jobs ADD COLUMN delete_mode TEXT; +ALTER TABLE delivery_jobs ADD COLUMN message_ids TEXT; +ALTER TABLE delivery_jobs ADD COLUMN cursor_contact_id INTEGER; +ALTER TABLE delivery_jobs ADD COLUMN cursor_group_id INTEGER; +CREATE INDEX idx_delivery_jobs_feed_next ON delivery_jobs(feed_id, worker_scope, failed, job_status); +CREATE INDEX idx_delivery_jobs_chat_item_id ON delivery_jobs(chat_item_id); +|] + +down_m20260905_feeds :: Query +down_m20260905_feeds = + [sql| +DROP INDEX idx_delivery_jobs_chat_item_id; +DROP INDEX idx_delivery_jobs_feed_next; +DELETE FROM delivery_jobs WHERE group_id IS NULL; +ALTER TABLE delivery_jobs DROP COLUMN cursor_group_id; +ALTER TABLE delivery_jobs DROP COLUMN cursor_contact_id; +ALTER TABLE delivery_jobs DROP COLUMN message_ids; +ALTER TABLE delivery_jobs DROP COLUMN delete_mode; +ALTER TABLE delivery_jobs DROP COLUMN chat_item_id; +ALTER TABLE delivery_jobs DROP COLUMN feed_id; + +PRAGMA writable_schema=1; + +UPDATE sqlite_master +SET sql = replace(sql, 'group_id INTEGER REFERENCES groups ON DELETE CASCADE', 'group_id INTEGER NOT NULL REFERENCES groups ON DELETE CASCADE') +WHERE type = 'table' AND name = 'delivery_jobs'; + +PRAGMA writable_schema=RESET; + +DROP INDEX idx_groups_user_id_business_chat; +DROP INDEX idx_contacts_user_id; +ALTER TABLE groups DROP COLUMN drop_feed; +ALTER TABLE contacts DROP COLUMN drop_feed; + +DROP INDEX idx_files_feed_id; +ALTER TABLE files DROP COLUMN feed_id; + +DROP INDEX idx_messages_feed_id; +ALTER TABLE messages DROP COLUMN feed_id; + +DROP INDEX idx_chat_items_feed_item_group; +DROP INDEX idx_chat_items_feed_item_contact; +DROP INDEX idx_chat_items_feeds_created_at; +DROP INDEX idx_chat_items_feed_id; +ALTER TABLE chat_items DROP COLUMN item_feed; +ALTER TABLE chat_items DROP COLUMN feed_item_id; +ALTER TABLE chat_items DROP COLUMN feed_id; + +DROP INDEX idx_feeds_user_id; +DROP TABLE feeds; +|] diff --git a/src/Simplex/Chat/Store/SQLite/Migrations/chat_schema.sql b/src/Simplex/Chat/Store/SQLite/Migrations/chat_schema.sql index 90706e70dd..f6a72169b7 100644 --- a/src/Simplex/Chat/Store/SQLite/Migrations/chat_schema.sql +++ b/src/Simplex/Chat/Store/SQLite/Migrations/chat_schema.sql @@ -106,6 +106,7 @@ CREATE TABLE contacts( grp_direct_inv_from_group_member_id INTEGER REFERENCES group_members(group_member_id) ON DELETE SET NULL, grp_direct_inv_from_member_conn_id INTEGER REFERENCES connections(connection_id) ON DELETE SET NULL, grp_direct_inv_started_connection INTEGER NOT NULL DEFAULT 0, + drop_feed INTEGER NOT NULL DEFAULT 0, FOREIGN KEY(user_id, local_display_name) REFERENCES display_names(user_id, local_display_name) ON DELETE CASCADE @@ -208,7 +209,8 @@ CREATE TABLE groups( roster_blob BLOB, group_domain_verified INTEGER, stored_roster_version INTEGER, - applied_complete_roster_version INTEGER, -- received + applied_complete_roster_version INTEGER, + drop_feed INTEGER NOT NULL DEFAULT 0, -- received FOREIGN KEY(user_id, local_display_name) REFERENCES display_names(user_id, local_display_name) ON DELETE CASCADE @@ -303,6 +305,8 @@ CREATE TABLE files( roster_transfer_id INTEGER, file_digest BLOB, file_expires_at TEXT + , + feed_id INTEGER DEFAULT NULL REFERENCES feeds ON DELETE CASCADE ) STRICT; CREATE TABLE snd_files( file_id INTEGER NOT NULL REFERENCES files ON DELETE CASCADE, @@ -456,6 +460,8 @@ CREATE TABLE messages( broker_ts TEXT, msg_chat_binding TEXT, msg_signatures BLOB + , + feed_id INTEGER DEFAULT NULL REFERENCES feeds ON DELETE CASCADE ) STRICT; CREATE TABLE pending_group_messages( pending_group_message_id INTEGER PRIMARY KEY, @@ -521,6 +527,10 @@ CREATE TABLE chat_items( fwd_from_public_group_id BLOB, fwd_from_member_id BLOB, fwd_from_shared_msg_id BLOB + , + feed_id INTEGER DEFAULT NULL REFERENCES feeds ON DELETE CASCADE, + feed_item_id INTEGER DEFAULT NULL REFERENCES chat_items ON DELETE SET NULL, + item_feed INTEGER NOT NULL DEFAULT 0 ) STRICT; CREATE TABLE sqlite_sequence(name,seq); CREATE TABLE chat_item_messages( @@ -785,7 +795,7 @@ CREATE TABLE delivery_tasks( ) STRICT; CREATE TABLE delivery_jobs( delivery_job_id INTEGER PRIMARY KEY AUTOINCREMENT, - group_id INTEGER NOT NULL REFERENCES groups ON DELETE CASCADE, + group_id INTEGER REFERENCES groups ON DELETE CASCADE, worker_scope TEXT NOT NULL, job_scope_spec_tag TEXT, job_scope_include_pending INTEGER, @@ -799,6 +809,13 @@ CREATE TABLE delivery_jobs( updated_at TEXT NOT NULL DEFAULT(datetime('now')) , sender_group_member_ids TEXT + , + feed_id INTEGER REFERENCES feeds ON DELETE CASCADE, + chat_item_id INTEGER REFERENCES chat_items ON DELETE CASCADE, + delete_mode TEXT, + message_ids TEXT, + cursor_contact_id INTEGER, + cursor_group_id INTEGER ) STRICT; CREATE TABLE group_member_status_predicates( member_status TEXT NOT NULL PRIMARY KEY, @@ -1418,3 +1435,41 @@ BEGIN ) WHERE group_id = NEW.group_id; END; +CREATE TABLE feeds( + feed_id INTEGER PRIMARY KEY AUTOINCREMENT, + user_id INTEGER NOT NULL REFERENCES users ON DELETE CASCADE, + created_at TEXT NOT NULL DEFAULT(datetime('now')), + updated_at TEXT NOT NULL DEFAULT(datetime('now')), + chat_ts TEXT NOT NULL DEFAULT(datetime('now')), + favorite INTEGER NOT NULL DEFAULT 0, + unread_chat INTEGER NOT NULL DEFAULT 0 +) STRICT; +CREATE INDEX idx_feeds_user_id ON feeds(user_id); +CREATE INDEX idx_chat_items_feed_id ON chat_items(feed_id); +CREATE INDEX idx_chat_items_feeds_created_at ON chat_items( + user_id, + feed_id, + created_at +); +CREATE INDEX idx_chat_items_feed_item_contact ON chat_items( + feed_item_id, + contact_id +); +CREATE INDEX idx_chat_items_feed_item_group ON chat_items( + feed_item_id, + group_id +); +CREATE INDEX idx_messages_feed_id ON messages(feed_id); +CREATE INDEX idx_files_feed_id ON files(feed_id); +CREATE INDEX idx_contacts_user_id ON contacts(user_id); +CREATE INDEX idx_groups_user_id_business_chat ON groups( + user_id, + business_chat +); +CREATE INDEX idx_delivery_jobs_feed_next ON delivery_jobs( + feed_id, + worker_scope, + failed, + job_status +); +CREATE INDEX idx_delivery_jobs_chat_item_id ON delivery_jobs(chat_item_id); diff --git a/src/Simplex/Chat/Store/Shared.hs b/src/Simplex/Chat/Store/Shared.hs index ceaf905df0..8ecc3d4e6a 100644 --- a/src/Simplex/Chat/Store/Shared.hs +++ b/src/Simplex/Chat/Store/Shared.hs @@ -73,6 +73,7 @@ data ChatLockEntity | CLUserContact Int64 | CLContactRequest Int64 | CLFile Int64 + | CLFeed FeedId deriving (Eq, Ord) -- These error type constructors must be added to mobile apps @@ -113,6 +114,9 @@ data StoreError | SENoteFolderAlreadyExists {noteFolderId :: NoteFolderId} | SENoteFolderNotFound {noteFolderId :: NoteFolderId} | SEUserNoteFolderNotFound + | SEFeedAlreadyExists {feedId :: FeedId} + | SEFeedNotFound {feedId :: FeedId} + | SEUserFeedNotFound | SESndFileNotFound {fileId :: FileTransferId} | SESndFileInvalid {fileId :: FileTransferId} | SERcvFileNotFound {fileId :: FileTransferId} diff --git a/src/Simplex/Chat/Types.hs b/src/Simplex/Chat/Types.hs index c0e44277a4..936e74089f 100644 --- a/src/Simplex/Chat/Types.hs +++ b/src/Simplex/Chat/Types.hs @@ -588,20 +588,22 @@ data GroupLink = GroupLink } deriving (Show) -data ContactOrGroup = CGContact Contact | CGGroup GroupInfo [GroupMember] +data ContactOrGroup = CGContact Contact | CGGroup GroupInfo [GroupMember] | CGFeed Feed data PreparedChatEntity = PCEContact Contact | PCEGroup {groupInfo :: GroupInfo, hostMember :: GroupMember} -contactAndGroupIds :: ContactOrGroup -> (Maybe ContactId, Maybe GroupId) +contactAndGroupIds :: ContactOrGroup -> (Maybe ContactId, Maybe GroupId, Maybe FeedId) contactAndGroupIds = \case - CGContact Contact {contactId} -> (Just contactId, Nothing) - CGGroup GroupInfo {groupId} _ -> (Nothing, Just groupId) + CGContact Contact {contactId} -> (Just contactId, Nothing, Nothing) + CGGroup GroupInfo {groupId} _ -> (Nothing, Just groupId, Nothing) + CGFeed Feed {feedId} -> (Nothing, Nothing, Just feedId) -- TODO when more settings are added we should create another type to allow partial setting updates (with all Maybe properties) data ChatSettings = ChatSettings { enableNtfs :: MsgFilter, sendRcpts :: Maybe Bool, - favorite :: Bool + favorite :: Bool, + dropFeed :: BoolDef } deriving (Eq, Show) @@ -610,7 +612,8 @@ defaultChatSettings = ChatSettings { enableNtfs = MFAll, sendRcpts = Nothing, - favorite = False + favorite = False, + dropFeed = BoolDef False } chatHasNtfs :: ChatSettings -> Bool @@ -2218,6 +2221,19 @@ data NoteFolder = NoteFolder type NoteFolderId = Int64 +data Feed = Feed + { feedId :: FeedId, + userId :: UserId, + createdAt :: UTCTime, + updatedAt :: UTCTime, + chatTs :: UTCTime, + favorite :: Bool, + unread :: Bool + } + deriving (Eq, Show) + +type FeedId = Int64 + data ChatVersion instance VersionScope ChatVersion @@ -2410,4 +2426,6 @@ $(JQ.deriveJSON defaultJSON ''ContactRef) $(JQ.deriveJSON defaultJSON ''NoteFolder) +$(JQ.deriveJSON defaultJSON ''Feed) + $(JQ.deriveJSON defaultJSON ''ChatTag)