Files
simplex-chat/apps/ios/spec/architecture.md
T
fbca2cae81 core, ui: resize images in the core for sharper previews and images (#7941)
* WIP: Haskell impl of resizeImageToStrSize

* WIP: add FFI export wrapper

* WIP: switch ios resizeImageToStrSizeSync to backend

* WIP: fix resizeImageToStrSizeSync

* WIP: add JNI wrappers

* Add non-base64 resize and cWriteImage

* WIP: add chat_write_image

* fix resizeToStr

* fixes

* fix syntax, raise jpeg quality back

* add chatWriteImage

* Fix cfArgs in tests

* Use jpeg-turbo encoder

* add nix pins

* bump jpeg-turbo

* add CI deps

* add encoding funs to android lib

* bump jpeg-turbo for 20.04 fix

* jpeg-turbo fix for 8.10

* android fixes

* Bump jpeg-turbo

* set static flag for jpeg-turbo

* Fix nix for aarch64/android

* Fix android C api

* Fix resize in android

* Bump hs-libjpeg-turbo

* target correct override

* override android lib from android pkgs

* add macos build tweaks

* bump hs-jpeg-turbo for 8.10 fixes

* more build fixes

* make ios compile

* Always use pkgconfig for libjpeg-turbo

* WIP: debug libjpeg in ubuntu pkg-config CI

* Add pkg-config to dockerfiles

* Add libturbojpeg to dockerfiles

* Use -dev version of libturbojpeg in dockerfiles

* Install pkg-config in windows CI

* Disable flag for custom libturbojpeg linking options

Workarounds relocation error on x86_64-linux with gcc:
  /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/13/../../../x86_64-linux-gnu/libturbojpeg.a(turbojpeg.c.o): relocation R_X86_64_TPOFF32 against `errStr' can not be used when making a shared object; recompile with -fPIC
  /usr/bin/ld: failed to set dynamic section sizes: bad value

* WIP: debug libjpeg in windows pkg-config CI

* WIP: debug libjpeg package in windows CI

* core: image resize module and buffer FFI

* ci: link libjpeg-turbo statically on all platforms

* ui: resize images in the core

* core: rename image resize functions to chat_resize_image and chat_resize_image_uri

* ci: restore whitespace in build.yml

* ci: build libjpeg-turbo 3.1.4.1 from a submodule

* core: hs-jpeg-turbo from github.com/simplex-chat

* nodejs: resize images

* python: resize images

* core: hs-jpeg-turbo from gitlab.com, the GitHub import lacks LFS objects

* core: generate /img previews

* ci: link libjpeg-turbo on Windows without pkg-config

* ci: build libjpeg-turbo on Windows with the GHC toolchain

* docs: plan for image resizing in the core

* docs: update image resizing plan

* ios: prevent crash when wallpaper image cannot be resized

* android, desktop: raise an error when an image cannot be resized

* docs: update spec references for image resizing

* docker: build the CLI image from a clone without submodules

* docs: apply review fixes to image resizing plan

* docker: build the CLI from a git worktree with the submodule present

* docs: apply second review to image resizing plan

* docs: point image resizing plan at the rewritten commit

* core: choose image format by transparency in chat_resize_image

* nodejs: resize images to the app sizes

* python: resize images to the app sizes

* nodejs, python: describe the core modules as used for image resizing

* docs: add license notices of the image libraries

* ci: describe the avatar method without attributing it to the apps

* ci, ios, nix: remove the libjpeg-turbo build outside cabal

* python: name resize_image in the core module text

* nodejs, python: describe resizing to the app image sizes

* core: refuse images larger than the supported size

* core: resize to JPEG images with a side above 65500 pixels

* docs: record image resizing decisions for the bundled libjpeg-turbo

* core, docs: build libjpeg-turbo and stb from the fork submodules

* docs: record the fork pins in the image resizing plan

* core: resize images to at most 4320 pixels per side, refuse above 16384

* docs: record the image side limits in the image resizing plan

* core: pin the rebased hs-jpeg-turbo fork

* core: pin the hs-jpeg-turbo fork without package.yaml

* docs: plan scaled JPEG decoding and EXIF orientation of images

* docs: copy the JuicyPixels-stbir licence of the pinned fork

* core: pin the hs-jpeg-turbo fork with scaled JPEG decoding

* core: decode JPEG at the display size, up to 65500 pixels per side

* core: rotate JPEG images by their EXIF orientation

* docs: refer the image plan to the JPEG decoding code

* docs: record the approved image processing decisions

* core: limit JPEG decoding memory to 1024 MB outside iOS and Android

* docs: name JPEG decoding in the libjpeg-turbo notice

* docs: plan image resizing from files and the decisions of 26 Oct

* core: scale images in one Catmull-Rom pass with sharpening

* docs: plan image sizes at a fixed quality

* core: resize images at a fixed quality to fill the size limit

* core: resize images from a file path or a file descriptor

* ios, android, desktop: resize picked images from their files

* core: encode resized images with mozjpeg

* tests: check descriptor closing by the file it refers to

* core: raise JPEG quality only for a full-size image below 97% of the limit

* ci: name the mozjpeg submodule in the Windows cleanup comments

* docs: refer the image plan to the current code

* docs: refer the image plan to the code merged with master

* core: export every image resizing function from each library

* core: close the file descriptor when its handle cannot be created

* docs: count the C sources of mozjpeg with the arithmetic decoder

* core: use the reviewed hs-jpeg-turbo and JuicyPixels-stbir forks

* ci: build the armv7a Android core library on request

* core: round image heights half up in the size search

* ios: describe the file image resizing functions in SimpleX.h

* ci: pick, preview and send images in the Claude app tests

* core: decode PNG images with libspng and refuse other formats

* core: encode PNG with libspng, keeping the smaller filter choice

* ci: pick, preview and send images in the scripted app smoke tests

* flake: build the PNG decoder package in nix builds

* ios: remove the copy of a picked image after sending it

* core: refuse image files larger than the decode memory limit

* core: scale a JPEG image to its largest size once per resize

* core: bisect JPEG quality without a step function

* core: reduce the memory used to sharpen and to decode PNG images

* android, desktop: pass the image format to the core as an integer

* android, desktop: fuse the preview conditions of picked media

* android, desktop, ci: check that the core resized the smoke test images

* python: remove the docstrings of the explicit image resizing functions

* docs: add the second review fixes to the image plan

* ci: tap the iOS attachment button by its accessibility label

* core: keep the colour profiles of resized images

* docs: add the colour profiles to the image plan

* ci: wait up to 30 s for the new group screens in the iOS smoke test

* ci: pick the Android smoke test images from their own photo folders

* ci: tap the iOS photo picker by position, as its tree is not shown

* ci: pick the test card and the logo by their iOS picker positions

* core: update JuicyPixels-stbir with the stb_image_resize licence terms

* core: write a fixed Display P3 profile for Display P3 images only

* core: do not keep the input image in memory for its colour profile

* core: read the JPEG segments once for the orientation and the profile

* core: read an image file into one buffer of its size

* docs: add the third review fixes to the image plan

* core: resize images with Display P3 or no profile, not profile bytes

* ci: check the pixel sizes of the images sent in the iOS smoke test

* core: update hs-jpeg-turbo with PNG chunk limits and encoder fixes

* docs: list Compact ICC Profiles, libspng and miniz in the dependencies

* android, desktop: reword the picked media preview comment

* docs: refer the image plan to the code merged with master

* ci: render the iOS snapshots before the images case

* ci: dismiss the snapshot tests' camera alert before the iOS images case

* core: update hs-jpeg-turbo with tests and the JuicyPixels-spng synopsis

* core: pin hs-jpeg-turbo and JuicyPixels-stbir in simplex-chat repositories

* update ios core library

* core, ui: accept only JPEG and PNG image data

* core: update JuicyPixels-stbir

* docs: plan for the iOS copy of picked images

---------

Co-authored-by: IC Rainbow <aenor.realm@gmail.com>
Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
Co-authored-by: Alexandre Esteves <alexfmpe@proton.me>
2026-10-30 12:14:51 +00:00

20 KiB

SimpleX Chat iOS -- System Architecture

Technical specification for the iOS app's layered architecture, FFI bridge, event system, and extension model.

Related specs: README | API Reference | State Management | Database Related product: Product Overview

Source: SimpleXApp.swift | AppDelegate.swift | ContentView.swift | ChatModel.swift | SimpleXAPI.swift | AppAPITypes.swift | APITypes.swift | API.swift


Table of Contents

  1. Layered Architecture
  2. FFI Bridge
  3. Event Streaming
  4. Database Architecture
  5. App Lifecycle
  6. Extension Architecture
  7. Remote Desktop Control

1. Layered Architecture

The app follows a strict layered model where each layer communicates only with its immediate neighbor:

┌─────────────────────────────────────────┐
│  SwiftUI Views                          │  Rendering, user interaction
│  (ChatListView, ChatView, ComposeView)  │
├─────────────────────────────────────────┤
│  ChatModel (ObservableObject)           │  App state, @Published properties
│  ItemsModel, Chat, ChatTagsModel        │  Per-chat state, tag filtering
├─────────────────────────────────────────┤
│  SimpleXAPI (FFI Bridge)                │  chatSendCmd/chatApiSendCmd
│  AppAPITypes (ChatCommand/Response)     │  JSON serialization/deserialization
├─────────────────────────────────────────┤
│  C FFI Layer                            │  chat_send_cmd_retry, chat_recv_msg_wait
│  (SimpleX.h, libsimplex.a)             │  Compiled Haskell via GHC cross-compiler
├─────────────────────────────────────────┤
│  Haskell Core (chat_ctrl)               │  Chat logic, chat protocol (x-events),
│  (Simplex.Chat.Controller)              │  database operations, file management
├─────────────────────────────────────────┤
│  simplexmq library (external)           │  SMP/XFTP protocols, SMP Agent,
│  (github.com/simplex-chat/simplexmq)    │  double-ratchet (PQDR), transport (TLS)
└─────────────────────────────────────────┘

Key invariant: No SwiftUI view directly calls FFI functions. All communication flows through ChatModel or dedicated API functions in SimpleXAPI.swift.

Source Files

Layer File Role Line
Views Shared/Views/ChatList/ChatListView.swift Chat list rendering
Views Shared/Views/Chat/ChatView.swift Conversation rendering
State Shared/Model/ChatModel.swift ChatModel, ItemsModel, Chat classes L337, L74, L1271
API Shared/Model/SimpleXAPI.swift FFI bridge functions L93
API Shared/Model/AppAPITypes.swift ChatCommand, ChatResponse, ChatEvent enums L15, L649, L1055
FFI SimpleXChat/SimpleX.h C header declaring Haskell exports
FFI SimpleXChat/APITypes.swift APIResult<R>, ChatError, ChatCmdProtocol L27, L699, L17
Core ../../src/Simplex/Chat/Controller.hs Haskell command processor — see processCommand in Controller.hs

2. FFI Bridge

C Functions (SimpleX.h)

The Haskell core exposes these C functions, declared in SimpleXChat/SimpleX.h:

typedef void* chat_ctrl;

// Initialize database, apply migrations, return controller
char *chat_migrate_init_key(char *path, char *key, int keepKey, char *confirm,
                            int backgroundMode, chat_ctrl *ctrl);

// Send command string, return JSON response string
char *chat_send_cmd_retry(chat_ctrl ctl, char *cmd, int retryNum);

// Block until next async event arrives (or timeout)
char *chat_recv_msg_wait(chat_ctrl ctl, int wait);

// Close/reopen database store
char *chat_close_store(chat_ctrl ctl);
char *chat_reopen_store(chat_ctrl ctl);

// Utility: markdown parsing, server validation, password hashing
char *chat_parse_markdown(char *str);
char *chat_parse_server(char *str);
char *chat_password_hash(char *pwd, char *salt);

// File encryption/decryption
char *chat_write_file(chat_ctrl ctl, char *path, char *data, int len);
char *chat_read_file(char *path, char *key, char *nonce);
char *chat_encrypt_file(chat_ctrl ctl, char *fromPath, char *toPath);
char *chat_decrypt_file(char *fromPath, char *key, char *nonce, char *toPath);

// Image resizing
char *chat_resize_image(char *data, int len, int maxSize, int png);
char *chat_resize_image_uri(char *data, int len, int maxSize);
char *chat_resize_image_file(char *path, int maxSize, int png);
char *chat_resize_image_file_uri(char *path, int maxSize);

Swift Bridge Functions (SimpleXAPI.swift)

// Synchronous send -- blocks calling thread
func chatSendCmdSync<R: ChatAPIResult>(_ cmd: ChatCommand, bgTask: Bool = true,
    bgDelay: Double? = nil, ctrl: chat_ctrl? = nil) throws -> R                      // L91

// Async send -- dispatches to background
func chatApiSendCmd<R: ChatAPIResult>(_ cmd: ChatCommand, bgTask: Bool = true,
    bgDelay: Double? = nil, ctrl: chat_ctrl? = nil) async -> APIResult<R>             // L215

// Low-level FFI call -- serializes command to string, calls chat_send_cmd_retry, decodes JSON
func sendSimpleXCmd<R: ChatAPIResult>(_ cmd: ChatCmdProtocol, _ ctrl: chat_ctrl?,
    retryNum: Int32 = 0) -> APIResult<R>                                              // SimpleXChat/API.swift L114

Data Flow

  1. Swift constructs a ChatCommand enum value (e.g., .apiSendMessages(type:id:scope:live:ttl:composedMessages:))
  2. ChatCommand.cmdString serializes it to a command string (e.g., "/_send @1 json {...}")
  3. sendSimpleXCmd passes the string to chat_send_cmd_retry via C FFI
  4. Haskell core processes the command, returns JSON response string
  5. Swift decodes JSON into APIResult<R> where R: ChatAPIResult
  6. Result is either .result(R), .error(ChatError), or .invalid(type, json)

Background Task Protection

All FFI calls are wrapped in beginBGTask() / endBackgroundTask() to prevent iOS from killing the app mid-operation. The maxTaskDuration is 15 seconds.


3. Event Streaming

The Haskell core emits async events (new messages, connection status changes, file progress, etc.) that are not direct responses to commands. These are received via polling:

Haskell Core --[chat_recv_msg_wait]--> Swift event loop --> ChatModel update --> SwiftUI re-render

The event loop is implemented in ChatReceiver, and events are dispatched by processReceivedMsg.

Event Types (ChatEvent enum)

Key async events delivered from core to UI:

Event Description Line
newChatItems New messages received L1070
chatItemUpdated Message edited by sender L1072
chatItemsDeleted Messages deleted L1074
chatItemReaction Reaction added/removed L1073
contactConnected New contact connected L1062
contactUpdated Contact profile changed L1066
receivedGroupInvitation Group invitation received L1077
groupMemberUpdated Group member info changed L1067
callInvitation Incoming call L1115
chatSuspended Core suspended (background) L1056
rcvFileComplete File download finished L1099
sndFileCompleteXFTP File upload finished L1110

Events are decoded as ChatEvent enum in Shared/Model/AppAPITypes.swift and dispatched to update ChatModel / ItemsModel properties, triggering SwiftUI view re-renders via @Published property observation.


4. Database Architecture

Two SQLite databases in the app group container (shared with NSE):

Database File Contents
Chat DB simplex_v1_chat.db Messages, contacts, groups, profiles, files, tags, preferences
Agent DB simplex_v1_agent.db SMP connections, keys, queues, server info

Both databases use the DB_FILE_PREFIX = "simplex_v1" prefix. The database path is resolved via getAppDatabasePath() in SimpleXChat/FileUtils.swift, which checks dbContainerGroupDefault to determine whether to use the app group container or legacy documents directory.

See Database & Storage specification for full details.


5. App Lifecycle

Initialization Sequence (SimpleXApp.swift)

// SimpleXApp.init()
1. haskell_init()                    // Initialize Haskell RTS (background queue, sync)
2. UserDefaults.register(defaults:)  // Register app preference defaults
3. setGroupDefaults()                // Sync preferences to app group container
4. setDbContainer()                  // Set database path                          L122
5. BGManager.shared.register()       // Register background task handlers
6. NtfManager.shared.registerCategories()  // Register notification action categories

State Transitions

              ┌──────────┐
              │ Launched  │
              └─────┬─────┘
                    │ initChatAndMigrate()
                    v
              ┌──────────┐
              │ DB Setup  │ chat_migrate_init_key()
              └─────┬─────┘
                    │ startChat()                                    SimpleXAPI.swift L2098
                    v
              ┌──────────┐
              │  Active   │ apiActivateChat()                        SimpleXAPI.swift L358
              └─────┬─────┘
                    │ scenePhase == .background
                    v
              ┌──────────┐
              │Background │ apiSuspendChat(timeoutMicroseconds:)     SimpleXAPI.swift L368
              └─────┬─────┘
                    │ scenePhase == .active
                    v
              ┌──────────┐
              │  Active   │ startChatAndActivate()
              └──────────┘

Scene Phase Handling (SimpleXApp.swift)

  • .active: Calls startChatAndActivate(), processes pending notification responses, refreshes chat list and call invitations
  • .background: Records authentication timestamp, calls suspendChat() (unless CallKit call active), schedules BGManager background refresh, updates badge count
  • .inactive: No explicit handling (transitional state)

CallKit Exception

When a CallKit call is active during backgrounding, chat suspension is deferred (CallController.shared.shouldSuspendChat = true) until the call ends, to maintain the WebRTC session.


6. Extension Architecture

Notification Service Extension (NSE)

The NSE (SimpleX NSE/NotificationService.swift) is a separate process that:

  1. Receives encrypted push notification payload from APNs
  2. Initializes its own Haskell core instance (chat_ctrl) with shared database access
  3. Decrypts the push payload using stored keys
  4. Generates a visible UNMutableNotificationContent with the decrypted message preview
  5. Delivers the notification to the user

Database sharing: Both main app and NSE access the same database files in the app group container (APP_GROUP_NAME). Coordination uses file locks to prevent concurrent write conflicts.

Lifecycle: The NSE has a ~30-second execution window per notification. It must initialize Haskell RTS, open the database, decrypt, and deliver within this window.

Share Extension (SE)

The Share Extension (SimpleX SE/) allows sharing content (text, images, files) from other apps into SimpleX conversations.


7. Remote Desktop Control

Optional desktop pairing allows controlling the mobile app from a desktop client:


8. Chat Relay Management

Overview

Chat relays are SMP servers that forward messages to channel subscribers. They are configured in the Network & Servers settings and selected during channel creation.

Data Model

Type Location Description
UserChatRelay ChatTypes.swift Relay server config: chatRelayId, address, name, domains, preset, tested, enabled, deleted
UserOperatorServers.chatRelays AppAPITypes.swift Array of UserChatRelay per operator
UserServersWarning AppAPITypes.swift Enum with .noChatRelays(user:) case
ServerSettings.serverWarnings ChatListView.swift [UserServersWarning] field on ServerSettings struct (exposed via SaveableSettings.servers)

Relay Management Views

View File Description
ChatRelayView ChatRelayView.swift Edit/view relay: name, address, test, enable toggle, delete
ChatRelayViewLink ChatRelayView.swift NavigationLink row showing relay status icon + display name
NewChatRelayView ChatRelayView.swift Form to add new relay (name + address + test + enable toggle)
ServersWarningView NetworkAndServers.swift Orange exclamation triangle + warning text

Key Functions

Function File Description
addChatRelay(...) ChatRelayView.swift Validates name/address, appends to userServers[nil operator].chatRelays, calls validateServers_
deleteChatRelay(...) ProtocolServersView.swift Marks relay as deleted or removes if no chatRelayId
validRelayName(_:) ChatRelayView.swift Non-empty + valid display name check
validRelayAddress(_:) ChatRelayView.swift Parses via parseSimpleXMarkdown, validates .simplexLink(_, .relay, _, _)
showRelayTestStatus(relay:) ChatRelayView.swift ViewBuilder returning checkmark/multiply/clear icons
validateServers_ NetworkAndServers.swift Extended signature: now accepts optional Binding<[UserServersWarning]>?; calls validateServers which returns ([UserServersError], [UserServersWarning]) tuple
globalServersWarning(_:) NetworkAndServers.swift Extracts .noChatRelays warning text for display
bindingForChatRelays(_:_:) NetworkAndServers.swift Creates binding for chatRelays at operator index

Relay Sections in Settings

"Chat relays" sections appear in:

  • OperatorView: lists relays for the operator, with header and footer
  • YourServersView (in ProtocolServersView): lists relays for non-operator servers, with delete support and "Add server" -> "Chat relay" option

serverWarnings Plumbing

Binding<[UserServersWarning]> is threaded through: NetworkAndServers -> OperatorView -> ProtocolServersView -> ProtocolServerView / NewServerView / ScanProtocolServer. All validateServers_ calls pass the warnings binding.


Source Files

File Path Line
App entry point Shared/SimpleXApp.swift L17
App delegate Shared/AppDelegate.swift L15
Root view Shared/ContentView.swift L24
FFI bridge Shared/Model/SimpleXAPI.swift L93
Low-level FFI SimpleXChat/API.swift L115
App state Shared/Model/ChatModel.swift L337
API types Shared/Model/AppAPITypes.swift L15
Shared types SimpleXChat/APITypes.swift L27
C header SimpleXChat/SimpleX.h
NSE SimpleX NSE/NotificationService.swift
Haskell core ../../src/Simplex/Chat/Controller.hs — see processCommand in Controller.hs
Chat protocol (x-events, message envelopes) ../../src/Simplex/Chat/Protocol.hs

External: simplexmq Library

The lower-level protocol and encryption layers are in the separate simplexmq library:

Component Spec Implementation
SMP protocol simplexmq/protocol/simplex-messaging.md simplexmq/src/Simplex/Messaging/Protocol.hs
XFTP protocol simplexmq/protocol/xftp.md simplexmq/src/Simplex/FileTransfer/Protocol.hs
SMP Agent (duplex connections) simplexmq/protocol/agent-protocol.md simplexmq/src/Simplex/Messaging/Agent.hs
Double ratchet (PQDR) simplexmq/protocol/pqdr.md simplexmq/src/Simplex/Messaging/Crypto/Ratchet.hs
Post-quantum KEM (sntrup761) simplexmq/protocol/pqdr.md simplexmq/src/Simplex/Messaging/Crypto/SNTRUP761.hs
TLS transport — simplexmq/src/Simplex/Messaging/Transport.hs
File encryption — simplexmq/src/Simplex/Messaging/Crypto/File.hs