From ce30e9bd6cd38e066debea351f646af27ddd9a04 Mon Sep 17 00:00:00 2001 From: spacedandy-agents-simplex <342056592+spacedandy-agents-simplex@users.noreply.github.com> Date: Mon, 19 Oct 2026 09:23:26 +0000 Subject: [PATCH] ci: check in the iOS smoke test and plan that known directory results are chats, and the note for a page of them After joining ci_cards, the smoke test expects the note "All results on this page are already in your chats" for its name, and, in both layouts and in New chat, that a search the local list doesn't match shows ci_cards as its chat row, which opens the channel, not "#ci_cards" with "Join channel". The plan gets U4 and W12 and the new U2, the prompt describes known entries and the note, and the Claude scope runs the changed cases (Haiku 5.5, 300 turns, 3 USD). Run: 008-dirsearch-known-chats, slice 4 --- scripts/ci/ios-sim/README.md | 2 +- scripts/ci/ios-sim/claude-prompt.md | 6 +- scripts/ci/ios-sim/claude-scope | 12 +-- scripts/ci/ios-sim/claude-test.sh | 2 +- scripts/ci/ios-sim/device-test-plan.md | 6 +- scripts/ci/ios-sim/smoke-test.sh | 102 ++++++++++++++++++++++++- 6 files changed, 114 insertions(+), 16 deletions(-) diff --git a/scripts/ci/ios-sim/README.md b/scripts/ci/ios-sim/README.md index bf78ed498f..14a0af51a1 100644 --- a/scripts/ci/ios-sim/README.md +++ b/scripts/ci/ios-sim/README.md @@ -47,7 +47,7 @@ Haiku 5.5 drives the app well: the smoke plan takes a few dozen turns and cents; - a local SMP server with a `[NAMES]` section and `names-resolver.py`, the names `cinews.testing`, `ciboth.testing`, `cialice.testing` and `cibobby.testing` claimed by their owners, and the contacts cialice and Robert, online and accepting connections; its address goes to `/names-server`; - checks before the app starts: a search sent to the directory as a message must list `ci_group` and `ci_channel`, `/_connect plan` must resolve each name from a fresh peer, and a search request as the app sends it (`/_service_request`) must find `ci_group` and `ci_channel`; then the searches "ci seed" and "ci avatar" are followed page by page, recording each page's entries and size against the response limit in `seed.md`. -With the backend, the workflow points the app at the test directory before building it (`DIRECTORY_SERVICE_LINK` in `apps/ios/Shared/Model/DirectorySearch.swift`, the only change it makes to app sources), and the smoke test goes on to search it: the Directory header above the results and their avatars, in both layouts; once the profile has joined `ci_cards`, the chat list's cards shown while typing and hidden under the results, in both layouts; and "Show more" disabled while its page loads. +With the backend, the workflow points the app at the test directory before building it (`DIRECTORY_SERVICE_LINK` in `apps/ios/Shared/Model/DirectorySearch.swift`, the only change it makes to app sources), and the smoke test goes on to search it: the Directory header above the results and their avatars, in both layouts; once the profile has joined `ci_cards`, the note "All results on this page are already in your chats" when that channel, listed as a chat, is the page's only result; the chat list's cards shown while typing and hidden under the results, in both layouts; `ci_cards` found by `ci cards` (which the local filter doesn't match) shown as its chat row, with no `#ci_cards` or "Join channel", and opening the channel, on the chat list in both layouts and in New chat; and "Show more" disabled while its page loads. It writes what it seeded to `/seed.md` (the Claude sessions get it) and logs to `/*.log`; `stop`, `directory-stop`, `directory-start` and `listings` are its other commands. The app reaches it through the simulator, which shares the runner's network; pointing the app at the names server is a few steps in Settings, described in `claude-prompt.md`. diff --git a/scripts/ci/ios-sim/claude-prompt.md b/scripts/ci/ios-sim/claude-prompt.md index b98af8859b..bc48b176da 100644 --- a/scripts/ci/ios-sim/claude-prompt.md +++ b/scripts/ci/ios-sim/claude-prompt.md @@ -13,8 +13,8 @@ You are testing the SimpleX Chat iOS app in an iPhone 17 simulator (iOS 26.5, sc - **Positions.** "Above" means a smaller y. Judge order and position from the tree's coordinates, and look at a screenshot when you judge layout (truncation, overlap, images). - **The search field** has no label; tap it with `ui tap-value "Search or paste SimpleX link"` when empty, or by its text. To clear it, tap the field so it is focused, then "Cancel" next to it ("Cancel" exists only while the field is focused; it also hides the keyboard); tapping the "Close" image inside the field did not clear it in an earlier run. Type text as one quoted argument (`ui type "ci seed"`). The keyboard capitalizes the first letter and autocorrects words (an earlier run saw "ci" become "Ci" and a typed name become another word when a button was tapped), so check the field's value in the tree after typing, and before typing check that the field shows its placeholder. The keyboard's Search key is `ui key return`; the keyboard's keys are below the screen and not listed in the tree, and the screenshots do not show the keyboard. - **Pasting** (C4, W3): `ui clipboard ""` with the link given under "This session", tap the empty field once to focus it and once more to open its menu, then tap its "Paste" item (two elements carry that label: use `ui tap-xy` on the one `ui find Paste` shows in the menu). -- **Directory search.** "Search in Directory" (a row) searches the directory; the first time after the hints are reset the alert "Search in Directory?" asks first, with "Cancel", "Search" and "Search and don't show again" (`ui tap Search` taps the alert's button). Results come as a heading `Heading "Directory"`, then per entry an avatar image, `"#name"`, the description, "N members" or "N subscriber(s)", and "Join group" or "Join channel" as separate texts. A failed search shows "Search failed, tap to retry"; more results show "Show more". A search takes 1–2 s here (it sets up an encrypted connection) and times out after 10 s. A channel's row shows the subscriber count the directory publishes for it (`publicMemberCount`), not the listings' members column, which counts every member record (owner, relay, directory), so the two differ by design. -- **Order with the one-hand UI.** Results read from the bottom like the chats: the first result is nearest the search field, each "Show more" adds its page above the earlier ones, and "Show more" sits right under the "Directory" heading. That is the current design; what must hold is that the heading is above all of the part's rows. +- **Directory search.** "Search in Directory" (a row) searches the directory; the first time after the hints are reset the alert "Search in Directory?" asks first, with "Cancel", "Search" and "Search and don't show again" (`ui tap Search` taps the alert's button). Results come as a heading `Heading "Directory"`, then per entry an avatar image, `"#name"`, the description, "N members" or "N subscriber(s)", and "Join group" or "Join channel" as separate texts. An entry you already have that the screen doesn't already list is shown instead as that chat's row, with no `#` and no "Join …": on the chat list the same row as your other chats (name, last message, time), in New chat a row like a contact's. When every result of the last page loaded is a chat already listed, the part shows "All results on this page are already in your chats". A failed search shows "Search failed, tap to retry"; more results show "Show more". A search takes 1–2 s here (it sets up an encrypted connection) and times out after 10 s. A channel's row shows the subscriber count the directory publishes for it (`publicMemberCount`), not the listings' members column, which counts every member record (owner, relay, directory), so the two differ by design. +- **Order with the one-hand UI.** Results read from the bottom like the chats: the first result is nearest the search field, each "Show more" adds its page above the earlier ones, and "Show more" sits right under the "Directory" heading. This order is settled (the user's decision: the list is mirrored; in New chat, Names stays above Directory); what must hold is that the heading is above all of the part's rows. - **Disabled elements** end with "disabled" in the tree. - **Holding a search open.** A search answers in 1–2 s here, too fast to see its loading state. For the cases about a search in progress (D1, D12, D13, W9, G3, and "Show more" while loading in D7), stop the directory first (`ui directory stop`): the search then waits 10 s before it fails. Start it again afterwards. The 10 s count from the search's start, and each of your turns takes several seconds, so do the case's action (for G3, `ui tap Compose`) as the very next command after starting the search, without a shot in between; an action after the search has failed tests nothing. - **Ambiguous labels.** When `ui tap` says several elements match a label (seen for "Paste", "Channels", "Account"), tap the one you mean with `ui tap-xy` at its centre, taken from `ui find`. @@ -47,7 +47,7 @@ When "This session" gives the test directory's address, the runner also runs a t ## How to run the cases -- Run each case of the session's list, in the plan's order unless a case needs another first (for example D15 needs a profile without conversations, U1 and U2 need a joined channel from D6). +- Run each case of the session's list, in the plan's order unless a case needs another first (for example D15 needs a profile without conversations; U1, U2, U4 and W12 need a joined channel: use the one named under "This session", or one joined in D6). - The plan's tables (ID, Do, Expect) are at the end of `prompt.md` in the current directory: read the row before running a case and do exactly what it says. An earlier session did not find them and filed other actions under some IDs. - For each case, do what the plan says, decide pass or fail against the plan's "Expect" column, and take at least one `ui shot` at the moment that shows the result, named after the case and layout without a number, for example `ui shot D2-reachable` (the helper numbers the files). - Pass only what you saw: quote the evidence (element labels, values and coordinates from the tree, or what a screenshot shows). If an expectation cannot be checked here, say which part and why, and mark the case passed for the parts you checked only if those parts are the substance of the case; otherwise skipped. diff --git a/scripts/ci/ios-sim/claude-scope b/scripts/ci/ios-sim/claude-scope index 3be75bd450..e4aea54d3f 100644 --- a/scripts/ci/ios-sim/claude-scope +++ b/scripts/ci/ios-sim/claude-scope @@ -3,9 +3,9 @@ # entries- (optional) tells that session which directory entries and names its cases use, so none meets an entry already known. model: claude-haiku-5-5 layouts: reachable top -cases-reachable: the SimpleX names setup, then N1-N11, W4 with names, W7, W8, G1 and the full search (chat list and New chat); after N10, and after the full search joins cinews_digest, check with the search text unchanged that the Names or Directory row you tapped is gone once the chat it opened is listed -cases-top: N1 and N4 (this layout places the "Connect to …" row and the Names and Directory parts differently) and the full search on the chat list only (New chat looks the same in both layouts); the names setup is already done -entries-reachable: N7 uses a second profile you add (the first profile has no other); N10 connects to @cialice.testing, which N5, W7 and G1 then use; W8 first connects to @cibobby.testing (Robert) from its Names row; N11 starts its connect from the Directory row #cinews_weekly and cancels its alert; the full search joins the group cinews_digest; leave the channel cinews unopened, as the top-bar session's N1 needs it -entries-top: the earlier session joined cinews_digest and connected to cialice and Robert; cinews is not opened; N1 and N4 join nothing -max-turns: 900 -max-budget-usd: 5 +cases-reachable: U1, U2, U4 and W12 +cases-top: U2 and U4 (the list's structure differs in this layout) +entries-reachable: the smoke test joined the channel ci_cards; U1 and U4 search `ci_cards`, U2 and W12 search `ci cards` (the space keeps the local list from matching it); join nothing else +entries-top: the same entries: U4 searches `ci_cards`, U2 searches `ci cards`; nothing else is joined +max-turns: 300 +max-budget-usd: 3 diff --git a/scripts/ci/ios-sim/claude-test.sh b/scripts/ci/ios-sim/claude-test.sh index 870aef196b..377f9d407d 100755 --- a/scripts/ci/ios-sim/claude-test.sh +++ b/scripts/ci/ios-sim/claude-test.sh @@ -73,7 +73,7 @@ write_prompt() { echo "- No test backend runs in this session." fi if [ -z "$previous" ] && [ -n "$BACKEND_DIR" ]; then - echo "- At the start: the smoke test onboarded a profile (typed as \"citester\"; the keyboard may have autocorrected the name), searched the directory in both layouts, choosing \"Search\" in the alert each time, and joined the channel ci_cards, which no case uses; it is the profile's only conversation." + echo "- At the start: the smoke test onboarded a profile (typed as \"citester\"; the keyboard may have autocorrected the name), searched the directory in both layouts, choosing \"Search\" in the alert each time, and joined the channel ci_cards; it is the profile's only conversation." elif [ -z "$previous" ]; then echo "- At the start: the smoke test onboarded a profile (typed as \"citester\"; the keyboard may have autocorrected the name) and checked the chat list in both layouts; the profile has no conversations." else diff --git a/scripts/ci/ios-sim/device-test-plan.md b/scripts/ci/ios-sim/device-test-plan.md index 699e1424a3..0f51375900 100644 --- a/scripts/ci/ios-sim/device-test-plan.md +++ b/scripts/ci/ios-sim/device-test-plan.md @@ -37,7 +37,7 @@ Run these on an iOS simulator or device on the Mac. Each case says what to do an | ID | Do | Expect | |----|----|--------| | D1 | Search for a term that matches listed groups | Spinner in the search field after about 1 s. While loading: **no "Directory" header**, no "No chats found" overlay | -| D2 | Results arrive | A "Directory" header row, then rows: avatar, `#` name, the directory's SimpleX name (if any), description up to 2 lines, member or subscriber count, and **"Join channel"** for channels or **"Join group"** for groups at the trailing edge | +| D2 | Results arrive | A "Directory" header row, then rows: avatar, `#` name, the directory's SimpleX name (if any), description up to 2 lines, member or subscriber count, and **"Join channel"** for channels or **"Join group"** for groups at the trailing edge. "Join …" rows are for entries you don't have; an entry you already have is a chat row (U2) | | D3 (R1-4) | Long-named entry, small phone, large text | The verb stays on one line; the name and description truncate instead | | D4 | Scroll the list, especially in one-hand mode | The "Directory" header scrolls with the rows and is not pinned to the screen edge | | D5 | Entry with markdown in its name or description | Shown literally: no bold, no tappable link | @@ -57,8 +57,9 @@ Run these on an iOS simulator or device on the Mac. Each case says what to do an | ID | Do | Expect | |----|----|--------| | U1 | Search for the name of a channel you joined (it is also listed) | It appears once, as your local chat; no Directory row for it | -| U2 | Search for a word only in that channel's description, so the local list does not show it | It appears in the Directory part; tapping it opens the "already joined" alert for the known channel | +| U2 | Search for a word only in that channel's description, so the local list does not show it | It appears in the Directory part as your chat's row (as in the chat list: name without `#`, last message, time), with no "Join channel"; tapping it opens the channel, with no alert | | U3 | A listed regular group you joined through a different link (or by invitation) | It shows twice, local and Directory. This is the documented limit, not a regression | +| U4 | Search for the name of a joined channel that is the page's only result, so the list shows it as your chat | Under the "Directory" header: "All results on this page are already in your chats" (grey), and "Show more" when there are more pages (with the one-hand UI it sits right under the header, as in D7); nothing loads more pages by itself | ## Chat list: names (AC5) @@ -91,6 +92,7 @@ Run these on an iOS simulator or device on the Mac. Each case says what to do an | W9 | Close the sheet while a search runs | The spinner stops; the chat list is unaffected | | W10 (R1-1) | Sheet open with results, tap a notification from another profile | The results disappear | | W11 | New chat → Archived contacts, type and press return | The key is "return", not Search; no online rows; empty states as before | +| W12 | In New chat, search the directory for a word in the description of a channel or group you joined | In the Directory section it is a chat row like a contact's (avatar, name without `#`, a star if favourite, a mask if incognito), with no "Join …"; tapping it closes the sheet and opens the chat | ## Regression checks diff --git a/scripts/ci/ios-sim/smoke-test.sh b/scripts/ci/ios-sim/smoke-test.sh index 2900792345..e782c212a3 100755 --- a/scripts/ci/ios-sim/smoke-test.sh +++ b/scripts/ci/ios-sim/smoke-test.sh @@ -4,8 +4,9 @@ # chat list with the one-hand UI (the default), then the chat list again with the top bar. It leaves the app in the one-hand UI. # With BACKEND_DIR, the work dir of the running test backend, it then searches the test directory, whose results # must show their avatars and have the Directory header above them in both layouts; then, with a joined channel, -# the chat list's cards must show while typing and hide under the results; and "Show more" must be disabled while -# its page loads (a screenshot shows its colour). +# the chat list's cards must show while typing and hide under the results; a page whose only result is that channel +# must say so; the channel found by its description must be its chat row, opening it, on the chat list in both layouts +# and in New chat; and "Show more" must be disabled while its page loads (a screenshot shows its colour). # Usage: scripts/ci/ios-sim/smoke-test.sh # Each step saves a screenshot and an accessibility tree to the output dir and prints the tree to the log. # Later steps of the job get SIM_UDID and UI_OUT, with the simulator still running. @@ -195,6 +196,95 @@ expect_cards_hidden_by_results() { echo "in the $1 layout the cards ($(paste -sd ',' - <<< "$typing")) show while typing, are hidden under the results and come back when the search is cleared" } +# the joined channel must not be offered to join again +expect_no_join_row() { + local tree + tree=$("$ui" tree) + if grep -qF '"#ci_cards"' <<< "$tree" || grep -qF 'Join channel' <<< "$tree"; then + echo "error: in $1 the joined ci_cards is a Directory row (#ci_cards, Join channel), not its chat" + exit 1 + fi +} + +# with the space the local filter does not match ci_cards, so the list does not show it, and the directory finds it by its description +open_known_chat_from_results() { + type_search "ci cards" + search_directory "ci_cards" + "$ui" shot "known-chat-$1" + expect_in_tree -F 'StaticText "ci_cards"' + expect_no_join_row "the $1 layout (chat list)" + "$ui" tap ci_cards + "$ui" wait "you are subscriber" 30 + "$ui" shot "known-chat-opened-$1" + "$ui" tap Back + clear_search + echo "in the $1 layout the directory result for the joined ci_cards is its chat row, which opens the channel" +} + +# the elements labelled exactly $1, one per line +labelled() { + "$ui" tree | grep -F "\"$1\" value=" || true +} + +# the elements labelled $1 that are not among the lines $2 +added_labelled() { + local now + now=$(labelled "$1") + if [ -z "$2" ]; then + printf '%s' "$now" + else + grep -vxF -f <(printf '%s\n' "$2") <<< "$now" || true + fi +} + +# the topmost of these tree lines +topmost() { + sed -E 's/.* at [0-9-]+,([0-9-]+) .*/\1 &/' <<< "$1" | sort -n | head -n 1 | cut -d ' ' -f 2- +} + +tap_element() { + local x y w h + read -r x y w h <<< "$(sed -nE 's/.* at ([0-9]+),([0-9]+) size ([0-9]+)x([0-9]+).*/\1 \2 \3 \4/p' <<< "$1")" + if [ -z "$h" ]; then + echo "error: no on-screen element to tap in: $1" + exit 1 + fi + "$ui" tap-xy $((x + w / 2)) $((y + h / 2)) +} + +# New chat lists only contacts, so there the joined channel is a contact-style chat row. The chat list behind the sheet +# may be in the tree too, with its own search field (at the bottom with the one-hand UI) and ci_cards row, so the sheet's +# field is the topmost one and its row is the ci_cards element the search adds. +open_known_chat_from_new_chat() { + local before row deadline + before=$(labelled ci_cards) + "$ui" tap Compose + "$ui" wait "Create 1-time link" 10 + tap_element "$(topmost "$("$ui" tree | grep -F 'TextField null value="Search or paste SimpleX link"')")" + "$ui" type "ci cards" + sleep 2 + "$ui" tap "Search in Directory" + if "$ui" wait "Search in Directory?" 10; then + "$ui" tap "Search" + fi + deadline=$((SECONDS + 60)) + until row=$(added_labelled ci_cards "$before") && [ -n "$row" ]; do + if [ "$SECONDS" -ge "$deadline" ]; then + "$ui" shot known-chat-new-chat-missing + echo "error: New chat shows no ci_cards row after searching the directory for \"ci cards\"" + exit 1 + fi + sleep 2 + done + "$ui" shot known-chat-new-chat + expect_no_join_row "New chat" + tap_element "$row" + "$ui" wait "you are subscriber" 30 + "$ui" shot known-chat-opened-new-chat + "$ui" tap Back + echo "in New chat the directory result for the joined ci_cards is its chat row ($row), which closes the sheet and opens the channel" +} + xcrun simctl install "$SIM_UDID" "$app" xcrun simctl launch "$SIM_UDID" "$bundle_id" @@ -292,12 +382,18 @@ if grep -qF '"#ci_cards"' <<< "$tree"; then exit 1 fi echo "after joining from its Directory row, ci_cards is a chat in the search results and its Directory row is gone" +# the page's only result is that channel, which the list shows, so the Directory part says the page added none +expect_in_tree -F '"All results on this page are already in your chats"' +echo "the Directory part says all results on the page are already in your chats" clear_search expect_cards_hidden_by_results reachable +open_known_chat_from_results reachable "$ui" layout top dismiss_lock_notice expect_cards_hidden_by_results top +open_known_chat_from_results top "$ui" layout reachable +open_known_chat_from_new_chat # with the directory stopped, the next page waits 10 s before it fails: long enough to see "Show more" while it loads type_search "ci avatar" @@ -308,4 +404,4 @@ search_directory "Show more" expect_in_tree -E '^Button "Show more" .* disabled$' "$ui" directory start clear_search -echo "smoke test passed: onboarding reaches the chat list in both layouts; the directory search for \"$search_text\" shows #ci_group (Join group) and #ci_channel (Join channel) with their avatars and the Directory header above them; with a chat, the list's cards show while typing and hide under the results; \"Show more\" is disabled while its page loads" +echo "smoke test passed: onboarding reaches the chat list in both layouts; the directory search for \"$search_text\" shows #ci_group (Join group) and #ci_channel (Join channel) with their avatars and the Directory header above them; with a chat, the list's cards show while typing and hide under the results; a page of chats already listed says so; the joined channel found by its description is its chat row on the chat list in both layouts and in New chat, and opens it; \"Show more\" is disabled while its page loads"