* WIP: Haskell impl of resizeImageToStrSize * WIP: add FFI export wrapper * WIP: switch ios resizeImageToStrSizeSync to backend * WIP: fix resizeImageToStrSizeSync * WIP: add JNI wrappers * Add non-base64 resize and cWriteImage * WIP: add chat_write_image * fix resizeToStr * fixes * fix syntax, raise jpeg quality back * add chatWriteImage * Fix cfArgs in tests * Use jpeg-turbo encoder * add nix pins * bump jpeg-turbo * add CI deps * add encoding funs to android lib * bump jpeg-turbo for 20.04 fix * jpeg-turbo fix for 8.10 * android fixes * Bump jpeg-turbo * set static flag for jpeg-turbo * Fix nix for aarch64/android * Fix android C api * Fix resize in android * Bump hs-libjpeg-turbo * target correct override * override android lib from android pkgs * add macos build tweaks * bump hs-jpeg-turbo for 8.10 fixes * more build fixes * make ios compile * Always use pkgconfig for libjpeg-turbo * WIP: debug libjpeg in ubuntu pkg-config CI * Add pkg-config to dockerfiles * Add libturbojpeg to dockerfiles * Use -dev version of libturbojpeg in dockerfiles * Install pkg-config in windows CI * Disable flag for custom libturbojpeg linking options Workarounds relocation error on x86_64-linux with gcc: /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/13/../../../x86_64-linux-gnu/libturbojpeg.a(turbojpeg.c.o): relocation R_X86_64_TPOFF32 against `errStr' can not be used when making a shared object; recompile with -fPIC /usr/bin/ld: failed to set dynamic section sizes: bad value * WIP: debug libjpeg in windows pkg-config CI * WIP: debug libjpeg package in windows CI * core: image resize module and buffer FFI * ci: link libjpeg-turbo statically on all platforms * ui: resize images in the core * core: rename image resize functions to chat_resize_image and chat_resize_image_uri * ci: restore whitespace in build.yml * ci: build libjpeg-turbo 3.1.4.1 from a submodule * core: hs-jpeg-turbo from github.com/simplex-chat * nodejs: resize images * python: resize images * core: hs-jpeg-turbo from gitlab.com, the GitHub import lacks LFS objects * core: generate /img previews * ci: link libjpeg-turbo on Windows without pkg-config * ci: build libjpeg-turbo on Windows with the GHC toolchain * docs: plan for image resizing in the core * docs: update image resizing plan * ios: prevent crash when wallpaper image cannot be resized * android, desktop: raise an error when an image cannot be resized * docs: update spec references for image resizing * docker: build the CLI image from a clone without submodules * docs: apply review fixes to image resizing plan * docker: build the CLI from a git worktree with the submodule present * docs: apply second review to image resizing plan * docs: point image resizing plan at the rewritten commit * core: choose image format by transparency in chat_resize_image * nodejs: resize images to the app sizes * python: resize images to the app sizes * nodejs, python: describe the core modules as used for image resizing * docs: add license notices of the image libraries * ci: describe the avatar method without attributing it to the apps * ci, ios, nix: remove the libjpeg-turbo build outside cabal * python: name resize_image in the core module text * nodejs, python: describe resizing to the app image sizes * core: refuse images larger than the supported size * core: resize to JPEG images with a side above 65500 pixels * docs: record image resizing decisions for the bundled libjpeg-turbo * core, docs: build libjpeg-turbo and stb from the fork submodules * docs: record the fork pins in the image resizing plan * core: resize images to at most 4320 pixels per side, refuse above 16384 * docs: record the image side limits in the image resizing plan * core: pin the rebased hs-jpeg-turbo fork * core: pin the hs-jpeg-turbo fork without package.yaml * docs: plan scaled JPEG decoding and EXIF orientation of images * docs: copy the JuicyPixels-stbir licence of the pinned fork * core: pin the hs-jpeg-turbo fork with scaled JPEG decoding * core: decode JPEG at the display size, up to 65500 pixels per side * core: rotate JPEG images by their EXIF orientation * docs: refer the image plan to the JPEG decoding code * docs: record the approved image processing decisions * core: limit JPEG decoding memory to 1024 MB outside iOS and Android * docs: name JPEG decoding in the libjpeg-turbo notice * docs: plan image resizing from files and the decisions of 26 Oct * core: scale images in one Catmull-Rom pass with sharpening * docs: plan image sizes at a fixed quality * core: resize images at a fixed quality to fill the size limit * core: resize images from a file path or a file descriptor * ios, android, desktop: resize picked images from their files * core: encode resized images with mozjpeg * tests: check descriptor closing by the file it refers to * core: raise JPEG quality only for a full-size image below 97% of the limit * ci: name the mozjpeg submodule in the Windows cleanup comments * docs: refer the image plan to the current code * docs: refer the image plan to the code merged with master * core: export every image resizing function from each library * core: close the file descriptor when its handle cannot be created * docs: count the C sources of mozjpeg with the arithmetic decoder * core: use the reviewed hs-jpeg-turbo and JuicyPixels-stbir forks * ci: build the armv7a Android core library on request * core: round image heights half up in the size search * ios: describe the file image resizing functions in SimpleX.h * ci: pick, preview and send images in the Claude app tests * core: decode PNG images with libspng and refuse other formats * core: encode PNG with libspng, keeping the smaller filter choice * ci: pick, preview and send images in the scripted app smoke tests * flake: build the PNG decoder package in nix builds * ios: remove the copy of a picked image after sending it * core: refuse image files larger than the decode memory limit * core: scale a JPEG image to its largest size once per resize * core: bisect JPEG quality without a step function * core: reduce the memory used to sharpen and to decode PNG images * android, desktop: pass the image format to the core as an integer * android, desktop: fuse the preview conditions of picked media * android, desktop, ci: check that the core resized the smoke test images * python: remove the docstrings of the explicit image resizing functions * docs: add the second review fixes to the image plan * ci: tap the iOS attachment button by its accessibility label * core: keep the colour profiles of resized images * docs: add the colour profiles to the image plan * ci: wait up to 30 s for the new group screens in the iOS smoke test * ci: pick the Android smoke test images from their own photo folders * ci: tap the iOS photo picker by position, as its tree is not shown * ci: pick the test card and the logo by their iOS picker positions * core: update JuicyPixels-stbir with the stb_image_resize licence terms * core: write a fixed Display P3 profile for Display P3 images only * core: do not keep the input image in memory for its colour profile * core: read the JPEG segments once for the orientation and the profile * core: read an image file into one buffer of its size * docs: add the third review fixes to the image plan * core: resize images with Display P3 or no profile, not profile bytes * ci: check the pixel sizes of the images sent in the iOS smoke test * core: update hs-jpeg-turbo with PNG chunk limits and encoder fixes * docs: list Compact ICC Profiles, libspng and miniz in the dependencies * android, desktop: reword the picked media preview comment * docs: refer the image plan to the code merged with master * ci: render the iOS snapshots before the images case * ci: dismiss the snapshot tests' camera alert before the iOS images case * core: update hs-jpeg-turbo with tests and the JuicyPixels-spng synopsis * core: pin hs-jpeg-turbo and JuicyPixels-stbir in simplex-chat repositories * update ios core library * core, ui: accept only JPEG and PNG image data * core: update JuicyPixels-stbir * docs: plan for the iOS copy of picked images --------- Co-authored-by: IC Rainbow <aenor.realm@gmail.com> Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com> Co-authored-by: Alexandre Esteves <alexfmpe@proton.me>
20 KiB
Message Composition Specification
Source: common/src/commonMain/kotlin/chat/simplex/common/views/chat/ComposeView.kt, SendMsgView.kt
Table of Contents
- Overview
- ComposeState Data Class
- ComposePreview Sealed Class
- ComposeContextItem Sealed Class
- SendMsgView
- Attachment Handling
- Draft Persistence
- Source Files
- Emoji Picker
Executive Summary
Message composition in SimpleX Chat is managed by ComposeView (line ~345 in ComposeView.kt) backed by the serializable ComposeState data class. The compose area supports text input, link previews, media/file/voice attachments, reply/edit/forward contexts, live (streaming) messages, member @mentions, message reports, and timed (disappearing) messages. The SendMsgView composable (in SendMsgView.kt) provides the text field and action buttons. Draft state persists across chat switches when the privacy preference is enabled.
1. Overview
ComposeView
|-- contextItemView()
| |-- ContextItemView (QuotedItem) [reply indicator]
| |-- ContextItemView (EditingItem) [edit indicator]
| |-- ContextItemView (ForwardingItems) [forward indicator]
| +-- ContextItemView (ReportedItem) [report indicator]
|-- ReportReasonView [report reason header]
|-- MsgNotAllowedView [disabled send reason]
|-- previewView()
| |-- ComposeLinkView [link preview card]
| |-- ComposeImageView [media thumbnails]
| |-- ComposeVoiceView [voice recording waveform]
| +-- ComposeFileView [file name display]
|-- ComposeActionButtons
| |-- CommandsButton [bot commands "//"]
| |-- AttachmentButton [paperclip icon]
| +-- ComposeEmojiButton [desktop emoji picker popup]
+-- SendMsgView
|-- PlatformTextField [multiline text input]
|-- DeleteTextButton [clear text, shown on long text]
|-- SendMsgButton [arrow/check icon]
|-- RecordVoiceView [microphone + hold-to-record]
|-- StartLiveMessageButton [bolt icon]
|-- CancelLiveMessageButton [cancel live]
+-- TimedMessageDropdown [disappearing message timer]
2. ComposeState Data Class
Location: ComposeView.kt#L98
@Serializable
data class ComposeState(
val message: ComposeMessage = ComposeMessage(),
val parsedMessage: List<FormattedText> = emptyList(),
val liveMessage: LiveMessage? = null,
val preview: ComposePreview = ComposePreview.NoPreview,
val contextItem: ComposeContextItem = ComposeContextItem.NoContextItem,
val inProgress: Boolean = false,
val progressByTimeout: Boolean = false,
val useLinkPreviews: Boolean,
val mentions: MentionedMembers = emptyMap()
)
Fields
| Field | Type | Description |
|---|---|---|
message |
ComposeMessage |
Current text and cursor selection (TextRange) |
parsedMessage |
List<FormattedText> |
Markdown-parsed representation of message text |
liveMessage |
LiveMessage? |
Active live (streaming) message state |
preview |
ComposePreview |
Attachment preview (link, media, voice, file) |
contextItem |
ComposeContextItem |
Reply/edit/forward/report context |
inProgress |
Boolean |
Send operation in flight |
progressByTimeout |
Boolean |
Show spinner after 1-second send delay |
useLinkPreviews |
Boolean |
Link preview feature flag |
mentions |
MentionedMembers |
Map of mention display name to CIMention |
Computed Properties
| Property | Type | Description |
|---|---|---|
editing |
Boolean |
True when contextItem is EditingItem |
forwarding |
Boolean |
True when contextItem is ForwardingItems |
reporting |
Boolean |
True when contextItem is ReportedItem |
sendEnabled |
() -> Boolean |
True when there is content to send and not in progress |
linkPreviewAllowed |
Boolean |
True when no media/voice/file preview is active |
linkPreview |
LinkPreview? |
Extracts link preview from CLinkPreview |
attachmentDisabled |
Boolean |
True when editing, forwarding, live, in-progress, or reporting |
attachmentPreview |
Boolean |
True when a file or media preview is showing |
empty |
Boolean |
True when no text, no preview, and no context item |
whitespaceOnly |
Boolean |
True when message text contains only whitespace |
placeholder |
String |
Input placeholder text (report reason text or default) |
memberMentions |
Map<String, Long> |
Extracted member ID map for API calls |
ComposeMessage
@Serializable
data class ComposeMessage(
val text: String = "",
val selection: TextRange = TextRange.Zero
)
LiveMessage
@Serializable
data class LiveMessage(
val chatItem: ChatItem,
val typedMsg: String,
val sentMsg: String,
val sent: Boolean
)
Tracks a live (streaming) message: the associated ChatItem, the currently typed text, the last sent text, and whether the initial send has occurred.
Serialization
ComposeState is fully @Serializable with a custom Saver (line ~214) that uses json.encodeToString/decodeFromString for rememberSaveable persistence across configuration changes.
3. ComposePreview Sealed Class
Location: ComposeView.kt#L52
sealed class ComposePreview {
object NoPreview : ComposePreview()
class CLinkPreview(val linkPreview: LinkPreview?) : ComposePreview()
class MediaPreview(val images: List<String>, val content: List<UploadContent>) : ComposePreview()
data class VoicePreview(val voice: String, val durationMs: Int, val finished: Boolean) : ComposePreview()
class FilePreview(val fileName: String, val uri: URI) : ComposePreview()
}
| Variant | Fields | View |
|---|---|---|
NoPreview |
-- | Nothing shown |
CLinkPreview |
linkPreview: LinkPreview? (null = loading) |
ComposeLinkView: title, description, image thumbnail, cancel button |
MediaPreview |
images: List<String> (base64 thumbnails), content: List<UploadContent> |
ComposeImageView: horizontal thumbnail strip, cancel button |
VoicePreview |
voice: String (file path), durationMs: Int, finished: Boolean |
ComposeVoiceView: waveform visualization, duration, play/pause |
FilePreview |
fileName: String, uri: URI |
ComposeFileView: file icon, file name, cancel button |
UploadContent
Used within MediaPreview to track the source type:
SimpleImage(uri: URI)-- still imageAnimatedImage(uri: URI)-- GIF or animated WebPVideo(uri: URI, duration: Int)-- video with duration in seconds
4. ComposeContextItem Sealed Class
Location: ComposeView.kt#L61
sealed class ComposeContextItem {
object NoContextItem : ComposeContextItem()
class QuotedItem(val chatItem: ChatItem) : ComposeContextItem()
class EditingItem(val chatItem: ChatItem) : ComposeContextItem()
class ForwardingItems(val chatItems: List<ChatItem>, val fromChatInfo: ChatInfo) : ComposeContextItem()
class ReportedItem(val chatItem: ChatItem, val reason: ReportReason) : ComposeContextItem()
}
| Variant | Trigger | Compose Behavior |
|---|---|---|
NoContextItem |
Default state | Normal message composition |
QuotedItem |
Swipe-to-reply or reply menu action | Shows quoted message indicator; sends with quoted parameter |
EditingItem |
Edit menu action | Populates text field with existing message; send button becomes checkmark; calls apiUpdateChatItem |
ForwardingItems |
Forward action from another chat | Shows forwarded items indicator; calls apiForwardChatItems; can include optional text message |
ReportedItem |
Report menu action | Shows report indicator with reason; placeholder changes to reason text; calls apiReportMessage |
Context Item View
contextItemView() (line ~1098 in ComposeView.kt) renders the active context as a dismissible bar above the text input:
- Icon: reply (ic_reply), edit (ic_edit_filled), forward (ic_forward), report (ic_flag)
- Content: quoted message preview text with sender name
- Close button: resets
contextItemtoNoContextItem(orclearState()for editing)
5. SendMsgView
Location: SendMsgView.kt#L36
fun SendMsgView(
composeState: MutableState<ComposeState>,
showVoiceRecordIcon: Boolean,
recState: MutableState<RecordingState>,
isDirectChat: Boolean,
liveMessageAlertShown: SharedPreference<Boolean>,
sendMsgEnabled: Boolean,
userCantSendReason: Pair<String, String?>?,
sendButtonEnabled: Boolean,
sendToConnect: (() -> Unit)?,
hideSendButton: Boolean,
nextConnect: Boolean,
needToAllowVoiceToContact: Boolean,
allowedVoiceByPrefs: Boolean,
sendButtonColor: Color,
allowVoiceToContact: () -> Unit,
timedMessageAllowed: Boolean,
customDisappearingMessageTimePref: SharedPreference<Int>?,
placeholder: String,
sendMessage: (Int?) -> Unit,
sendLiveMessage: (suspend () -> Unit)?,
updateLiveMessage: (suspend () -> Unit)?,
cancelLiveMessage: (() -> Unit)?,
editPrevMessage: () -> Unit,
onFilesPasted: (List<URI>) -> Unit,
onMessageChange: (ComposeMessage) -> Unit,
textStyle: MutableState<TextStyle>,
focusRequester: FocusRequester?
)
Layout
The view is a Box containing:
- PlatformTextField: Multiline text input (platform-specific
expect). Handles text changes viaonMessageChange, up-arrow toeditPrevMessage, file paste viaonFilesPasted, and Enter to send. - DeleteTextButton: Shown when text is long; clears the field.
- Action area (bottom-right, stacked):
- Progress indicator: Shown when
progressByTimeoutis true. - Report confirm button: Checkmark icon when context is
ReportedItem. - Voice record button: Shown when message is empty, not editing/forwarding, no preview active.
RecordVoiceView: Hold-to-record with waveform display.DisallowedVoiceButton: Shown when voice is disabled by preferences.VoiceButtonWithoutPermissionByPlatform: Shown when microphone permission is not granted.
- Live message button: Bolt icon, starts streaming message (calls
sendLiveMessage). - Send button: Arrow icon (new message) or checkmark (editing/live). Long-press opens dropdown:
- "Send live message" option
- Timed message options (1min, 5min, 1hr, 8hr, 1day, 1week, 1month, custom)
- Progress indicator: Shown when
RecordingState
sealed class RecordingState {
object NotStarted : RecordingState()
class Started(val filePath: String, val progressMs: Int) : RecordingState()
class Finished(val filePath: String, val durationMs: Int) : RecordingState()
}
Voice recording of 300ms or less is auto-cancelled.
Disabled State
When sendMsgEnabled is false (e.g., contact not ready, group permissions), an overlay covers the text field. If userCantSendReason is provided, tapping the overlay shows an alert explaining why sending is disabled.
6. Attachment Handling
Attachment Selection
The AttachmentSelection composable (line ~263 in ComposeView.kt) is an expect function with platform-specific implementations:
Android:
- Camera launcher (image capture)
- Gallery launcher (image/video picker, multi-select)
- File picker (any file type)
Desktop:
- File chooser dialog (filters for images or all files)
ChooseAttachmentView
Bottom sheet (ModalBottomSheetLayout) presenting attachment type options:
| Option | Result |
|---|---|
| Camera (Android) | Launches camera intent; result processed as SimpleImage |
| Gallery | Launches media picker; results processed via processPickedMedia |
| File | Launches file picker; result processed via processPickedFile |
File Processing
processPickedFile (line ~281):
- Checks file size against
maxFileSize(XFTP limit). - Extracts file name from URI.
- Sets
ComposePreview.FilePreviewon compose state.
processPickedMedia (line ~300):
- For each URI, determines type (image, animated image, video).
- Images: the preview of a non-animated image is made by core from the file (
resizeImageFileToStrSize); otherwise, or when core returns no preview, from the bitmap; createsSimpleImageorAnimatedImageupload content. - Videos: Extracts thumbnail and duration, creates
Videoupload content. - Generates base64 preview thumbnails (max 14KB).
- Sets
ComposePreview.MediaPreviewwith thumbnails and content list.
onFilesAttached (line ~270):
Groups dropped/pasted files into images and non-images; routes to processPickedMedia or processPickedFile.
Send Flow
On send (line ~603, sendMessageAsync):
- Forwarding: Calls
apiForwardChatItems, then optionally sends a text message quoting the last forwarded item. - Editing: Calls
apiUpdateChatItemwith updatedMsgContent. - Reporting: Calls
apiReportMessagewith reason and text. - New message: Iterates over
msgs(one per media item or single for text/file/voice):- Saves file to app storage (or remote host).
- For voice: encrypts if
privacyEncryptLocalFilesis enabled. - Calls
apiSendMessagesorapiCreateChatItems(local notes).
- On failure of the last message, restores compose state for retry.
Link Preview
When privacyLinkPreviews is enabled and the message contains a URL:
showLinkPreviewextracts first non-SimpleX, non-cancelled link from parsed markdown.- Sets
ComposePreview.CLinkPreview(null)(loading state). - After 1.5s debounce, calls
getLinkPreview(url). - On success, updates to
CLinkPreview(linkPreview). - Cancel button adds the URL to
cancelledLinksset.
7. Draft Persistence
Location: ComposeView.kt#L1394 (KeyChangeEffect(chatModel.chatId.value))
Controlled by the privacySaveLastDraft preference.
Save Behavior
When the user navigates away from a chat (chatModel.chatId.value changes):
| Compose State | Action |
|---|---|
| Live message active (text present or already sent) | Sends the live message immediately, clears draft |
| In progress | Clears in-progress flag, clears previous draft |
| Non-empty (text, preview, or context) | If saveLastDraft is true: saves composeState.value to chatModel.draft.value and chatModel.draftChatId.value |
| Empty but draft exists for current chat | Restores draft from chatModel.draft |
| Empty, no draft | Clears previous draft, deletes unused files |
Restore Behavior
When entering a chat (line ~132 in ChatView.kt):
- Checks if
chatModel.draftChatId.valuematches the chat ID. - If match and draft is not null (and not a cross-chat forward), initializes
composeStatefrom the draft. - Otherwise, creates a fresh
ComposeState.
Desktop-specific
On desktop, a DisposableEffect (line ~1256) saves the draft on dispose when forwarding content, since the KeyChangeEffect mechanism is Android-specific.
Draft Display in Chat List
When a draft exists for a chat, ChatPreviewView shows a pencil icon with the draft text instead of the last message preview.
8. Source Files
| File | Description |
|---|---|
ComposeView.kt |
ComposeState, ComposePreview, ComposeContextItem, ComposeView composable, send logic, link preview, draft persistence |
SendMsgView.kt |
Text input field, send/voice/live/timed buttons, recording state |
ComposeFileView.kt |
File attachment preview (name, cancel) |
ComposeImageView.kt |
Media attachment preview (thumbnails, cancel) |
ComposeVoiceView.kt |
Voice recording preview (waveform, duration, play) |
ContextItemView.kt |
Reply/edit/forward/report context bar |
ComposeContextContactRequestActionsView.kt |
Contact request action buttons in compose area |
ComposeContextGroupDirectInvitationActionsView.kt |
Group direct invitation compose actions |
ComposeContextPendingMemberActionsView.kt |
Pending member compose actions |
ComposeContextProfilePickerView.kt |
Profile picker in compose context |
SelectableChatItemToolbars.kt |
Multi-select mode toolbar (delete, forward, moderate) |
ComposeEmojiPicker.kt (+ .android.kt, .desktop.kt) |
Desktop emoji button and picker, emoji insertion |
9. Emoji Picker
Source: ComposeEmojiPicker.kt, ComposeEmojiPicker.desktop.kt
Desktop only; on Android the ComposeEmojiButton() actual is empty, as keyboards have their own emoji panel.
An emoji button in ComposeActionButtons() toggles emojiPickerShown, owned by ComposeView. A picked emoji goes through onEmoji(): insertingEmoji() replaces the selection (clamped to the text) and moves the cursor after the emoji, then onMessageChange applies markdown parsing, link sanitizing and the large-emoji font. Emoji are ignored while a message is in progress. The picker stays open after picking. The text field collapses its selection to the end when the popup takes focus, so the emoji goes after selected text.
emojiEnabled is false when sending is not allowed or a voice message is previewed; the picker closes then. When the picker closes, the text field gets focus through focusRequester.
| Function | Behavior |
|---|---|
ComposeEmojiButton() |
Button 4 dp after the attachment button (EMOJI_BUTTON_START_PADDING, against the row's -8 dp overlap). Focusable Popup above the button; closes on Esc (on key release, so the window does not also close its top modal), outside click or the button; its height is capped to the space above the button |
EmojiPicker() |
Search field, category tabs, LazyVerticalGrid with one section per EmojiCategory; tabs and cells are not focusable, so typing after a click goes to search. Picked emoji are not stored, so nothing about what the user typed is kept in preferences |
Data: desktopMain/resources/emoji/emoji.txt is generated by scripts/desktop/generate-emoji-data.py from Unicode emoji-test.txt 15.0 (the version covered by the bundled NotoColorEmoji): fully-qualified emoji without skin tone variants, @<group> lines followed by <emoji>\t<name> lines. parseEmojiData() maps groups to EmojiCategory and fails on an unknown group; emojiSections loads it lazily. searchEmoji() matches the English name case-insensitively.
[GAP] No skin tone selection, and search uses English emoji names only.