simplex-chat/apps/ios/spec/database.md

SimpleX Chat iOS -- Database & Storage

Source: FileUtils.swift

Technical specification for the database architecture, encryption, file storage, and export/import functionality.

Related specs: Architecture | State Management | README Related product: Product Overview

---

Table of Contents

1. Database Overview
2. Database Files & Paths
3. Haskell Store Modules
4. Migrations
5. Database Encryption
6. File Storage
7. Export & Import
8. App Group Sharing

---

1. Database Overview

SimpleX Chat uses two SQLite databases managed entirely by the Haskell core. The iOS Swift layer never reads or writes directly to the databases -- all data access goes through the FFI command/response API.

Database	Suffix	Contents
Chat DB	_chat.db	Messages, contacts, groups, user profiles, files, tags, preferences, call history
Agent DB	_agent.db	SMP agent connections, cryptographic keys, message queues, server state, XFTP chunks

Both databases are initialized and migrated via the C FFI function chat_migrate_init_key(), which applies pending migrations and returns a chat_ctrl pointer.

---

2. Database Files & Paths

Path Resolution (FileUtils.swift)

let DB_FILE_PREFIX = "simplex_v1"

// Database path depends on container preference
func getAppDatabasePath() -> URL {
    dbContainerGroupDefault.get() == .group
    ? getGroupContainerDirectory().appendingPathComponent(DB_FILE_PREFIX)
    : getLegacyDatabasePath()
}

// Full database file paths:
// Chat:  {container}/simplex_v1_chat.db
// Agent: {container}/simplex_v1_agent.db

File Constants

let CHAT_DB: String = "_chat.db"
let AGENT_DB: String = "_agent.db"
private let CHAT_DB_BAK: String = "_chat.db.bak"
private let AGENT_DB_BAK: String = "_agent.db.bak"

Container Locations

See getDocumentsDirectory() and getGroupContainerDirectory().

Container	Path	Used When
App Group	FileManager.containerURL(forSecurityApplicationGroupIdentifier: APP_GROUP_NAME)	Default (shared with NSE)
Documents	FileManager.urls(for: .documentDirectory)	Legacy installations

The container choice is stored in dbContainerGroupDefault (GroupDefaults).

Backup Exclusion

excludeAppDataFromBackup(_:) sets isExcludedFromBackup on the app group container as requested, clears it on the database files (FileManager.copyItem in restoreBackup() copies it from the .bak files), and always sets it on the Documents directory (legacy database, exported archives, migration files), temp_files (decrypted copies of files) and the .bak copies of the database (made by the core on passphrase changes and migrations, possibly unencrypted or with an old passphrase).

The app group container is included in backup when the "Enable iCloud backup" toggle in "Database passphrase & export" is on (DEFAULT_ICLOUD_BACKUP, on by default) and iCloudBackupBlock returns nil: the database is in the app group container, encrypted, and its passphrase was set by the user, not the initial random one. Otherwise the toggle is disabled and shown off, and its footer states the reason. The keychain passphrase is stored with kSecAttrAccessibleAfterFirstUnlockThisDeviceOnly, so on another device the user has to enter it after a restore.

updateAppDataBackup() applies the rule in initializeChat after the database is opened and the file paths are set (the core re-creates temp_files then), after a passphrase change in DatabaseEncryptionView (the core makes new .bak copies), after restoring the database backup in DatabaseErrorView, and when the toggle changes. prepareForLaunch() applies it with updateAppDataBackupOnLaunch() before the database is opened, treating it as encrypted when its passphrase is not stored in the keychain (it was set by the user) or a key is stored there, as chatMigrateInit opens the database with the stored key (importing an archive removes it, so an unencrypted imported database is excluded); while protected data is unavailable it does nothing, keeping the last applied state.

Accepted gaps: after deleting or importing a database, the previous decision stays until the app restarts; .bak copies made during a passphrase change or migration, and temp_files re-created by the NSE or SE, are excluded only at the next update; files in app_files are backed up as stored (unencrypted when "Encrypt local files" is off).

---

3. Haskell Store Modules

All database operations are implemented in Haskell. Key store modules (paths relative to repo root):

Module	Path	Size	Description
Messages	src/Simplex/Chat/Store/Messages.hs	~178KB	Message CRUD, pagination, search, reactions, delivery receipts
Groups	src/Simplex/Chat/Store/Groups.hs	~126KB	Group CRUD, member management, roles, links, invitations
Direct	src/Simplex/Chat/Store/Direct.hs	~52KB	Direct contact connections, contact requests. See createDirectChat in Store/Direct.hs
Files	src/Simplex/Chat/Store/Files.hs	~43KB	File transfer state, XFTP chunks, inline files
Profiles	src/Simplex/Chat/Store/Profiles.hs	~42KB	User profiles, contact profiles, incognito profiles
Connections	src/Simplex/Chat/Store/Connections.hs	~17KB	Connection lifecycle, queue management

Data Model (key tables)

users              -- User profiles (userId, displayName, fullName, image, ...)
contacts           -- Contact records (contactId, userId, localDisplayName, ...)
groups             -- Group records (groupId, userId, groupProfile, ...)
group_members      -- Group membership (groupMemberId, groupId, memberId, role, ...)
messages           -- Message records (messageId, chatItemId, msgBody, ...)
chat_items         -- Chat items (chatItemId, chatType, chatId, content, ...)
files              -- File transfer records (fileId, chatItemId, fileName, fileSize, ...)
connections        -- SMP connections (connId, agentConnId, ...)
chat_tags          -- User-defined chat tags
chat_tags_chats    -- Tag-to-chat assignments

---

4. Migrations

Database migrations are managed by the Haskell core. Migration files are located in:

src/Simplex/Chat/Store/SQLite/Migrations/

Migrations are numbered sequentially starting from M20220101 through M20260122 (200+ migrations). Each migration is a Haskell module containing SQL statements for schema changes.

The migration process:

1. chat_migrate_init_key() is called with the database path
2. Haskell reads the current schema version from the database
3. Pending migrations are applied in order
4. If migration fails, the function returns an error string (not a chat_ctrl)
5. On success, a chat_ctrl pointer is returned

Migration results are decoded in Swift as DBMigrationResult:

• .ok -- migrations applied successfully
• .invalidConfirmation -- migration requires user confirmation
• .errorNotADatabase(dbFile:) -- file is not a valid SQLite database
• .errorMigration(dbFile:, migrationError:) -- migration failed
• .errorSQL(dbFile:, migrationSQLError:) -- SQL error during migration
• .errorKeychain -- keychain access failed
• .unknown(json:) -- unrecognized response

---

5. Database Encryption

Encryption Configuration

Database encryption uses SQLCipher (AES-256) and is managed through the API:

// Set or change encryption
ChatCommand.apiStorageEncryption(config: DBEncryptionConfig)

// Test if a key is correct
ChatCommand.testStorageEncryption(key: String)

DBEncryptionConfig contains:

• currentKey: String -- current encryption key (empty if unencrypted)
• newKey: String -- new encryption key (empty to decrypt)

Key Storage

The encryption key is stored in the iOS Keychain via kcDatabasePassword:

• On first launch with encryption, the key is generated and stored
• The storeDBPassphraseGroupDefault flag controls whether the key is auto-stored
• If the user opts out of auto-storage, they must enter the key on each launch

UI

• DatabaseEncryptionView.swift -- Encryption settings UI
• DatabaseView.swift -- Database management UI (size, export, import, encryption)

---

6. File Storage

Directory Structure

{App Container}/
├── Documents/
│   ├── app_files/          -- Downloaded and sent files
│   ├── temp_files/         -- Temporary files during transfer
│   └── assets/wallpapers/  -- Custom wallpaper images
├── {App Group Container}/
│   ├── simplex_v1_chat.db  -- Chat database
│   ├── simplex_v1_agent.db -- Agent database
│   └── ...

File Size Constants (FileUtils.swift)

public let MAX_IMAGE_SIZE: Int64 = 261_120        // 255 KB -- inline image compression target
public let MAX_IMAGE_SIZE_AUTO_RCV: Int64 = 522_240  // 510 KB -- auto-receive images
public let MAX_VOICE_SIZE_AUTO_RCV: Int64 = 522_240  // 510 KB -- auto-receive voice
public let MAX_VIDEO_SIZE_AUTO_RCV: Int64 = 1_047_552 // 1023 KB -- auto-receive video
public let MAX_FILE_SIZE_XFTP: Int64 = 1_073_741_824 // 1 GB -- max XFTP transfer
public let MAX_FILE_SIZE_SMP: Int64 = 8_000_000      // ~7.6 MB -- max SMP inline
public let MAX_FILE_SIZE_LOCAL: Int64 = Int64.max     // No limit for local files
public let MAX_VOICE_MESSAGE_LENGTH = TimeInterval(300) // 5 minutes

CryptoFile (Encrypted File Storage)

When apiSetEncryptLocalFiles(enable: true) is set, files stored on device are AES-encrypted:

• Encryption/decryption uses chat_encrypt_file / chat_decrypt_file C FFI functions
• Each file gets a unique key and nonce stored alongside the file reference
• The CryptoFile type wraps (filePath: String, cryptoArgs: CryptoFileArgs?) where CryptoFileArgs contains (fileKey: String, fileNonce: String)

File Path Helpers

public func getDocumentsDirectory() -> URL      // Standard documents dir
public func getGroupContainerDirectory() -> URL // App group container
func getAppFilesDirectory() -> URL              // {appDir}/app_files/
func getTempFilesDirectory() -> URL             // {appDir}/temp_files/
func getWallpaperDirectory() -> URL             // {appDir}/assets/wallpapers/

See also saveFile(), removeFile(), and getMaxFileSize().

Cleanup

• Files are deleted when their associated ChatItem is deleted. See cleanupFile() and cleanupDirectFile().
• Timed message expiry triggers file deletion
• deleteAppDatabaseAndFiles() removes all databases, files, temp files, and wallpapers
• deleteAppFiles() removes only the files directory (preserving databases)

---

7. Export & Import

Export

ChatCommand.apiExportArchive(config: ArchiveConfig)
// Response: ChatResponse2.archiveExported(archiveErrors: [ArchiveError])

ArchiveConfig specifies:

• archivePath: String -- destination path for the archive
• disableCompression: Bool? -- optional flag to skip compression

The archive contains both databases and optionally files. The Haskell core handles the actual export, creating a ZIP archive.

Import

ChatCommand.apiImportArchive(config: ArchiveConfig)
// Response: ChatResponse2.archiveImported(archiveErrors: [ArchiveError])

Import replaces the current databases with the archive contents. The app must be restarted after import.

Archive Errors

ArchiveError is an array returned with both export and import results, listing any non-fatal issues encountered (e.g., missing files, corrupt entries).

---

8. App Group Sharing

Shared Access Model

The main app and NSE share database access through the iOS App Group container:

Main App ──┐
            ├── {App Group}/simplex_v1_chat.db
            ├── {App Group}/simplex_v1_agent.db
NSE ────────┘

Coordination

• Both processes can initialize their own chat_ctrl instance pointing to the same database files
• SQLite WAL mode allows concurrent reads
• Write coordination uses chat_close_store / chat_reopen_store to manage database locks
• The main app suspends its chat controller when entering background, allowing NSE to access the database
• NSE is short-lived (~30 seconds per notification) and releases its lock quickly

App State Communication

The appStateGroupDefault in GroupDefaults communicates app state between main app and NSE:

• .active -- main app is in foreground
• .suspended -- main app is in background
• .stopped -- main app is terminated

The NSE checks this flag to determine whether to process notifications (it avoids processing if the main app is active).

---

Source Files

File	Path
File utilities & constants	SimpleXChat/FileUtils.swift
Database management UI	Shared/Views/Database/DatabaseView.swift
Encryption settings UI	Shared/Views/Database/DatabaseEncryptionView.swift
C FFI (migration, file ops)	SimpleXChat/SimpleX.h
Haskell store root	../../src/Simplex/Chat/Store/
Haskell migrations	../../src/Simplex/Chat/Store/SQLite/Migrations/