Files
Evgeny @ SimpleX ChatandEvgeny Poberezkin c3c733e0d4 ios, ui, ci: terminal chat layout in the iOS app (#7898)
* plan: UI improvements to improve LLM usage via SimpleX

* core, ui: types for chat layout

* core, ui: settings for alternative terminal-like chat layout for managing LLMs or other service bots

* plan: message priority

* fixes

* query plan

* update plans

* remove silent messages plan

* rename migrations

* implement layout

* amend

* fix

* fix

* fix

* improve

* time on the right, sender name in blue

* no avatar in direct chats

* better previews

* update, fixes

* update

* time gap

* changes

* ui: terminal layout line heights, gaps and time placement

* ui: simplify terminal meta layout, correct its docs and tests

* ui: right-to-left emoji edge in terminal rows, time placement tests

* android, desktop: raise Kotlin compiler memory to 4 GB

* ui: terminal captions as follow-up text, classic event separators

* ui: terminal media edge and overlay time, plan decisions

* ui: simplify terminal row separation, correct doc line references

* ui: keep the media time inside terminal rows, restore caption conditions

* ui: terminal heading capitals at large font scales, idle live message time

* ui: rename the classic voice-with-text flag, correct the meta copy docs

* ui: smaller terminal layout diff, sender gap and meta names

* ui: terminal row as tall as the media time

* docs: remove the chat layout section from the bots readme

* docs: shorten the terminal layout section of the chat view

* core: address settings API as on master, /auto_accept reads the stored layout

* ui: smaller terminal headings

* ui: classic sender rows and avatar rule as on master

* ui: classic reply quote text colour as on master

* ios: keep the chat layout setting with other UI settings

* ui: go-to button at the end of terminal rows, no gutter in direct chats

* ui: forwarded arrow on the header line of terminal rows, above the time

* ui: forwarded arrow centred above the time line of terminal rows

* ios: terminal chat layout, chosen per app, profile and chat

* ios: event time reserve of three spaces, app layout picker as Kotlin

* ios: spec and plan of the terminal chat layout

* ios: classic message text at 87% in dark themes, as terminal rows

* ios: terminal gaps scaled by the body size, 17/16 of Kotlin values

* ios: terminal quote sender at 13.5pt, thumbnail at the item end

* ios: product docs of the terminal layout, plans updated for iOS

* ios: chat scenes rendered to PNG by a unit test in the simulator CI

* ios: snapshots rendered by the smoke test script, workflow unchanged

* ios: terminal names at 13.5 x 17/16 pt, as the gaps

* ios: Markdown views at the 87% message colour in dark themes

* ios: terminal quote thumbnail 12pt after the quote text or the bar

* ios: terminal rule-only message widened by a wider quote or header

* ios: member avatars scaled with the text size, as in the chat list

* ios: snapshot rows at their own height, as list cells, so text wraps

* ios: member avatars back at 34pt, for the snapshot before scaling

* ios: member avatars scaled with the text size, as in the chat list

* ui: terminal quote thumbnail 12dp after the quote text or the bar

* ios: classic group bubble width follows the scaled member avatar

* ci: ios-sim README describes the snapshots rendered after the smoke test

* ios: scratch, padding variants for the I7 and I8 renders, to revert

* ios: revert the scratch padding variants after their renders

* ui: contrast of the voice message button in terminal rows

* ios: scratch, spacing variants and list margins to render, to revert

* ios: revert the scratch spacing variants after their renders

* ios: snapshots at a fixed date, in UTC and the en_US locale

* ios: snapshots drawn as 8-bit sRGB without alpha

* ci: compare iOS snapshots with the references in the repository

* ci: record and check iOS snapshot references on GitHub's runner

* ios: scratch, list margins on the remaining screens, to revert

* ios: revert the scratch list margins after their measurement

* ci: snapshot scripts executable, as the other CI scripts

* ios: scratch, voice circle variants in light themes, to revert

* ios: revert the scratch voice circle variants

* ios: one 12pt padding at the edges and after avatars in terminal rows

* ios: classic group sender name on the bubble edge

* ui: one 12dp padding at the edges and after avatars in terminal rows

* ios: one heading scale in both chat layouts

* ios: snapshot references of the chat scenes

* docs: leave the headings and classic lines of the plan to ep/classic-bubbles

* ios: declare terminal layout functions inside types

* ios: keep layout constants at file scope

* ios: keep file-scope functions, move only extension members into types

* ios: snapshot tests record row heights at first layout and the appearance layout switch

* ios: appearance preview follows the chat layout setting at once

* ios: snapshot test measures first-layout rows by SwiftUI and by cells

* ios: chat rows measured at the chat width when they first appear

* ios: snapshot test checks that chat rows keep their first height after a reload

* ios, ci: references of terminal and classic chat scenes, dark message text

* ios: message text at 95% in dark themes

* ios, ci: references with message text at 95% in dark themes

* ios: media in terminal rows end at the time's right inset

* ios, ci: references with terminal media at the time's right inset

---------

Co-authored-by: Evgeny Poberezkin <evgeny@poberezkin.com>
2026-10-29 12:45:34 +00:00

493 lines
20 KiB
Swift

//
// CIVoiceView.swift
// SimpleX (iOS)
//
// Created by JRoberts on 22.11.2022.
// Copyright © 2022 SimpleX Chat. All rights reserved.
//
// Spec: spec/client/chat-view.md
import SwiftUI
import SimpleXChat
// Spec: spec/client/chat-view.md#CIVoiceView
struct CIVoiceView: View {
@ObservedObject var chat: Chat
@EnvironmentObject var theme: AppTheme
@Environment(\.terminalLayout) private var terminal
var chatItem: ChatItem
let recordingFile: CIFile?
let duration: Int
@State var audioPlayer: AudioPlayer? = nil
@State var playbackState: VoiceMessagePlaybackState = .noPlayback
@State var playbackTime: TimeInterval? = nil
@Binding var allowMenu: Bool
var smallViewSize: CGFloat?
@State private var seek: (TimeInterval) -> Void = { _ in }
var body: some View {
Group {
if smallViewSize != nil {
HStack(spacing: 10) {
player()
playerTime()
.allowsHitTesting(false)
if .playing == playbackState || (playbackTime ?? 0) > 0 || !allowMenu {
playbackSlider()
}
}
} else if chatItem.chatDir.sent && !terminal {
VStack (alignment: .trailing, spacing: 6) {
HStack {
if .playing == playbackState || (playbackTime ?? 0) > 0 || !allowMenu {
playbackSlider()
}
playerTime()
player()
}
.frame(alignment: .trailing)
metaView().padding(.trailing, 10)
}
} else {
ItemWithMeta {
metaView()
} content: { inlineMeta in
VStack (alignment: .leading, spacing: 6) {
HStack {
player()
playerTime()
if .playing == playbackState || (playbackTime ?? 0) > 0 || !allowMenu {
playbackSlider()
}
}
.frame(alignment: .leading)
if let inlineMeta {
inlineMeta.padding(.leading, -6)
}
}
}
}
}
.padding(.top, terminal ? terminalGap(4) : 4)
.padding(.bottom, terminal ? terminalGap(6) : 8)
.padding(.leading, terminal ? chatItemTextPadding : 0)
}
private func player() -> some View {
let sizeMultiplier: CGFloat = if let sz = smallViewSize {
voiceMessageSizeBasedOnSquareSize(sz) / 56
} else {
1
}
return VoiceMessagePlayer(
chat: chat,
chatItem: chatItem,
recordingFile: recordingFile,
recordingTime: TimeInterval(duration),
showBackground: true,
seek: $seek,
audioPlayer: $audioPlayer,
playbackState: $playbackState,
playbackTime: $playbackTime,
allowMenu: $allowMenu,
sizeMultiplier: sizeMultiplier
)
}
private func playerTime() -> some View {
VoiceMessagePlayerTime(
recordingTime: TimeInterval(duration),
playbackState: $playbackState,
playbackTime: $playbackTime
)
.foregroundColor(theme.colors.secondary)
}
private func playbackSlider() -> some View {
ComposeVoiceView.SliderBar(
length: TimeInterval(duration),
progress: $playbackTime,
seek: {
let time = max(0.0001, $0)
seek(time)
playbackTime = time
})
.onChange(of: .playing == playbackState || (playbackTime ?? 0) > 0) { show in
if !show {
allowMenu = true
}
}
.tint(theme.colors.primary)
}
private func metaView() -> some View {
CIMetaView(chat: chat, chatItem: chatItem, metaColor: theme.colors.secondary)
}
}
struct VoiceMessagePlayerTime: View {
var recordingTime: TimeInterval
@Binding var playbackState: VoiceMessagePlaybackState
@Binding var playbackTime: TimeInterval?
var body: some View {
ZStack(alignment: .leading) {
Text(String("66:66")).foregroundColor(.clear)
switch playbackState {
case .noPlayback:
Text(voiceMessageTime(recordingTime))
case .playing:
Text(voiceMessageTime_(playbackTime))
case .paused:
Text(voiceMessageTime_(playbackTime))
}
}
}
}
struct VoiceMessagePlayer: View {
@ObservedObject var chat: Chat
@EnvironmentObject var chatModel: ChatModel
@EnvironmentObject var theme: AppTheme
@Environment(\.terminalLayout) private var terminal
var chatItem: ChatItem
var recordingFile: CIFile?
var recordingTime: TimeInterval
var showBackground: Bool
@Binding var seek: (TimeInterval) -> Void
@Binding var audioPlayer: AudioPlayer?
@Binding var playbackState: VoiceMessagePlaybackState
@Binding var playbackTime: TimeInterval?
@Binding var allowMenu: Bool
var sizeMultiplier: CGFloat
var body: some View {
ZStack {
if let recordingFile = recordingFile {
switch recordingFile.fileStatus {
case .sndStored:
if recordingFile.fileProtocol == .local {
playbackButton()
} else {
loadingIcon()
}
case .sndTransfer: loadingIcon()
case .sndComplete: playbackButton()
case .sndCancelled: playbackButton()
case let .sndError(sndFileError):
fileStatusIcon("multiply", 14)
.simultaneousGesture(TapGesture().onEnded {
showFileErrorAlert(sndFileError)
})
case let .sndWarning(sndFileError):
fileStatusIcon("exclamationmark.triangle.fill", 16)
.simultaneousGesture(TapGesture().onEnded {
showFileErrorAlert(sndFileError, temporary: true)
})
case .rcvInvitation: downloadButton(recordingFile, recordingFile.expired ? "multiply" : "play.fill")
case .rcvAccepted: loadingIcon()
case .rcvTransfer: loadingIcon()
case .rcvAborted: downloadButton(recordingFile, "exclamationmark.arrow.circlepath")
case .rcvComplete: playbackButton()
case .rcvCancelled: playPauseIcon("play.fill", Color(uiColor: .tertiaryLabel))
case let .rcvError(rcvFileError):
fileStatusIcon("multiply", 14)
.simultaneousGesture(TapGesture().onEnded {
showFileErrorAlert(rcvFileError, recordingFile)
})
case let .rcvWarning(rcvFileError):
fileStatusIcon("exclamationmark.triangle.fill", 16)
.simultaneousGesture(TapGesture().onEnded {
showFileErrorAlert(rcvFileError, temporary: true)
})
case .invalid: playPauseIcon("play.fill", Color(uiColor: .tertiaryLabel))
}
} else {
playPauseIcon("play.fill", Color(uiColor: .tertiaryLabel))
}
}
.onAppear {
if audioPlayer == nil {
let small = sizeMultiplier != 1
audioPlayer = small ? VoiceItemState.smallView[VoiceItemState.id(chat, chatItem)]?.audioPlayer : VoiceItemState.chatView[VoiceItemState.id(chat, chatItem)]?.audioPlayer
playbackState = (small ? VoiceItemState.smallView[VoiceItemState.id(chat, chatItem)]?.playbackState : VoiceItemState.chatView[VoiceItemState.id(chat, chatItem)]?.playbackState) ?? .noPlayback
playbackTime = small ? VoiceItemState.smallView[VoiceItemState.id(chat, chatItem)]?.playbackTime : VoiceItemState.chatView[VoiceItemState.id(chat, chatItem)]?.playbackTime
}
seek = { to in audioPlayer?.seek(to) }
let audioPath: URL? = if let recordingSource = getLoadedFileSource(recordingFile) {
getAppFilePath(recordingSource.filePath)
} else {
nil
}
let chatId = chatModel.chatId
let userId = chatModel.currentUser?.userId
audioPlayer?.onTimer = {
playbackTime = $0
notifyStateChange()
// Manual check here is needed because when this view is not visible, SwiftUI don't react on stopPreviousRecPlay, chatId and current user changes and audio keeps playing when it should stop
if (audioPath != nil && chatModel.stopPreviousRecPlay != audioPath) || chatModel.chatId != chatId || chatModel.currentUser?.userId != userId {
stopPlayback()
}
}
audioPlayer?.onFinishPlayback = {
playbackState = .noPlayback
playbackTime = TimeInterval(0)
notifyStateChange()
}
// One voice message was paused, then scrolled far from it, started to play another one, drop to stopped state
if let audioPath, chatModel.stopPreviousRecPlay != audioPath {
stopPlayback()
}
}
.onChange(of: chatModel.stopPreviousRecPlay) { it in
if let recordingFileName = getLoadedFileSource(recordingFile)?.filePath,
chatModel.stopPreviousRecPlay != getAppFilePath(recordingFileName) {
stopPlayback()
}
}
.onChange(of: playbackState) { state in
allowMenu = state == .paused || state == .noPlayback
// Notify activeContentPreview in ChatPreviewView that playback is finished
if state == .noPlayback, let recordingFileName = getLoadedFileSource(recordingFile)?.filePath,
chatModel.stopPreviousRecPlay == getAppFilePath(recordingFileName) {
chatModel.stopPreviousRecPlay = nil
}
}
.onChange(of: chatModel.chatId) { _ in
stopPlayback()
}
.onDisappear {
if sizeMultiplier == 1 && chatModel.chatId == nil {
stopPlayback()
}
}
}
private func playbackButton() -> some View {
let icon = switch playbackState {
case .noPlayback: "play.fill"
case .playing: "pause.fill"
case .paused: "play.fill"
}
return playPauseIcon(icon, theme.colors.primary)
.simultaneousGesture(TapGesture().onEnded { _ in
switch playbackState {
case .noPlayback:
if let recordingSource = getLoadedFileSource(recordingFile) {
startPlayback(recordingSource)
}
case .playing:
audioPlayer?.pause()
playbackState = .paused
notifyStateChange()
case .paused:
audioPlayer?.play()
playbackState = .playing
notifyStateChange()
}
})
}
private func playPauseIcon(_ image: String, _ color: Color/* = .accentColor*/) -> some View {
ZStack {
Image(systemName: image)
.resizable()
.aspectRatio(contentMode: .fit)
.frame(width: 20 * sizeMultiplier, height: 20 * sizeMultiplier)
.foregroundColor(color)
.padding(.leading, image == "play.fill" ? 4 : 0)
.frame(width: 56 * sizeMultiplier, height: 56 * sizeMultiplier)
.background(showBackground ? backgroundColor : .clear)
.clipShape(Circle())
if recordingTime > 0 {
ProgressCircle(length: recordingTime, progress: $playbackTime)
.frame(width: 53 * sizeMultiplier, height: 53 * sizeMultiplier) // this + ProgressCircle lineWidth = background circle diameter
}
}
}
private func downloadButton(_ recordingFile: CIFile, _ icon: String) -> some View {
playPauseIcon(icon, theme.colors.primary)
.simultaneousGesture(TapGesture().onEnded {
chatModel.openWhenReceived(recordingFile) { file in
if let recordingSource = getLoadedFileSource(file) {
startPlayback(recordingSource)
}
}
Task {
if let user = chatModel.currentUser {
await receiveFile(user: user, fileId: recordingFile.fileId)
}
}
})
}
func notifyStateChange() {
if sizeMultiplier != 1 {
VoiceItemState.smallView[VoiceItemState.id(chat, chatItem)] = VoiceItemState(audioPlayer: audioPlayer, playbackState: playbackState, playbackTime: playbackTime)
} else {
VoiceItemState.chatView[VoiceItemState.id(chat, chatItem)] = VoiceItemState(audioPlayer: audioPlayer, playbackState: playbackState, playbackTime: playbackTime)
}
}
private struct ProgressCircle: View {
@EnvironmentObject var theme: AppTheme
var length: TimeInterval
@Binding var progress: TimeInterval?
var body: some View {
Circle()
.trim(from: 0, to: ((progress ?? TimeInterval(0)) / length))
.stroke(
theme.colors.primary,
style: StrokeStyle(lineWidth: 3)
)
.rotationEffect(.degrees(-90))
.animation(.linear, value: progress)
}
}
private var backgroundColor: Color {
terminal ? chatItemFrameContextColor(chatItem, theme) : chatItemFrameColor(chatItem, theme)
}
private func fileStatusIcon(_ image: String, _ size: CGFloat) -> some View {
Image(systemName: image)
.resizable()
.aspectRatio(contentMode: .fit)
.frame(width: size * sizeMultiplier, height: size * sizeMultiplier)
.foregroundColor(Color(uiColor: .tertiaryLabel))
.frame(width: 56 * sizeMultiplier, height: 56 * sizeMultiplier)
.background(showBackground ? backgroundColor : .clear)
.clipShape(Circle())
}
private func loadingIcon() -> some View {
ProgressView()
.frame(width: 30 * sizeMultiplier, height: 30 * sizeMultiplier)
.frame(width: 56 * sizeMultiplier, height: 56 * sizeMultiplier)
.background(showBackground ? backgroundColor : .clear)
.clipShape(Circle())
}
private func startPlayback(_ recordingSource: CryptoFile) {
let audioPath = getAppFilePath(recordingSource.filePath)
let chatId = chatModel.chatId
let userId = chatModel.currentUser?.userId
chatModel.stopPreviousRecPlay = audioPath
audioPlayer = AudioPlayer(
onTimer: {
playbackTime = $0
notifyStateChange()
// Manual check here is needed because when this view is not visible, SwiftUI don't react on stopPreviousRecPlay, chatId and current user changes and audio keeps playing when it should stop
if chatModel.stopPreviousRecPlay != audioPath || chatModel.chatId != chatId || chatModel.currentUser?.userId != userId {
stopPlayback()
}
},
onFinishPlayback: {
playbackState = .noPlayback
playbackTime = TimeInterval(0)
notifyStateChange()
}
)
audioPlayer?.start(fileSource: recordingSource, at: playbackTime)
playbackState = .playing
notifyStateChange()
}
private func stopPlayback() {
audioPlayer?.stop()
playbackState = .noPlayback
playbackTime = TimeInterval(0)
notifyStateChange()
}
}
@inline(__always)
func voiceMessageSizeBasedOnSquareSize(_ squareSize: CGFloat) -> CGFloat {
let squareToCircleRatio = 0.935
return squareSize + squareSize * (1 - squareToCircleRatio)
}
class VoiceItemState {
var audioPlayer: AudioPlayer?
var playbackState: VoiceMessagePlaybackState
var playbackTime: TimeInterval?
init(audioPlayer: AudioPlayer? = nil, playbackState: VoiceMessagePlaybackState, playbackTime: TimeInterval? = nil) {
self.audioPlayer = audioPlayer
self.playbackState = playbackState
self.playbackTime = playbackTime
}
@inline(__always)
static func id(_ chat: Chat, _ chatItem: ChatItem) -> String {
"\(chat.id) \(chatItem.id)"
}
@inline(__always)
static func id(_ chatInfo: ChatInfo, _ chatItem: ChatItem) -> String {
"\(chatInfo.id) \(chatItem.id)"
}
static func stopVoiceInSmallView(_ chatInfo: ChatInfo, _ chatItem: ChatItem) {
let id = id(chatInfo, chatItem)
if let item = smallView[id] {
item.audioPlayer?.stop()
ChatModel.shared.stopPreviousRecPlay = nil
}
}
static func stopVoiceInChatView(_ chatInfo: ChatInfo, _ chatItem: ChatItem) {
let id = id(chatInfo, chatItem)
if let item = chatView[id] {
item.audioPlayer?.stop()
ChatModel.shared.stopPreviousRecPlay = nil
}
}
static var smallView: [String: VoiceItemState] = [:]
static var chatView: [String: VoiceItemState] = [:]
}
struct CIVoiceView_Previews: PreviewProvider {
static var previews: some View {
let im = ItemsModel.shared
let sentVoiceMessage: ChatItem = ChatItem(
chatDir: .directSnd,
meta: CIMeta.getSample(1, previewDate, "", .sndSent(sndProgress: .complete), itemEdited: true),
content: .sndMsgContent(msgContent: .voice(text: "", duration: 30)),
quotedItem: nil,
file: CIFile.getSample(fileStatus: .sndComplete)
)
let voiceMessageWtFile = ChatItem(
chatDir: .directRcv,
meta: CIMeta.getSample(1, previewDate, "", .rcvRead),
content: .rcvMsgContent(msgContent: .voice(text: "", duration: 30)),
quotedItem: nil,
file: nil
)
Group {
let ci1 = previewDated(ChatItem.getVoiceMsgContentSample())
CIVoiceView(
chat: Chat.sampleData,
chatItem: ci1,
recordingFile: CIFile.getSample(fileName: "voice.m4a", fileSize: 65536, fileStatus: .rcvComplete),
duration: 30,
allowMenu: Binding.constant(true)
)
.modifier(ChatItemClipped(ci1, tailVisible: true))
ChatItemView(chat: Chat.sampleData, im: im, chatItem: sentVoiceMessage, scrollToItem: { _ in }, scrollToItemId: Binding.constant(nil), allowMenu: .constant(true)).modifier(ChatItemClipped(sentVoiceMessage, tailVisible: true))
let ci2 = previewDated(ChatItem.getVoiceMsgContentSample())
ChatItemView(chat: Chat.sampleData, im: im, chatItem: ci2, scrollToItem: { _ in }, scrollToItemId: Binding.constant(nil), allowMenu: .constant(true)).modifier(ChatItemClipped(ci2, tailVisible: true))
let ci3 = previewDated(ChatItem.getVoiceMsgContentSample(fileStatus: .rcvTransfer(rcvProgress: 7, rcvTotal: 10)))
ChatItemView(chat: Chat.sampleData, im: im, chatItem: ci3, scrollToItem: { _ in }, scrollToItemId: Binding.constant(nil), allowMenu: .constant(true)).modifier(ChatItemClipped(ci3, tailVisible: true))
ChatItemView(chat: Chat.sampleData, im: im, chatItem: voiceMessageWtFile, scrollToItem: { _ in }, scrollToItemId: Binding.constant(nil), allowMenu: .constant(true)).modifier(ChatItemClipped(voiceMessageWtFile, tailVisible: true))
}
.previewLayout(.fixed(width: 360, height: 360))
}
}