diff --git a/.github/workflows/desktop-linux.yml b/.github/workflows/desktop-linux.yml index 40ad0f164c..a340b1ea50 100644 --- a/.github/workflows/desktop-linux.yml +++ b/.github/workflows/desktop-linux.yml @@ -411,6 +411,14 @@ jobs: timeout-minutes: 30 run: scripts/ci/desktop-linux/backend.sh start ~/backend/${{ env.BACKEND_PKG }} "$RUNNER_TEMP/backend" + - name: Point the app at the test directory + if: needs.plan.outputs.run-backend == 'true' + run: | + link=$(cat "$RUNNER_TEMP/backend/directory-link") + file=apps/multiplatform/common/src/commonMain/kotlin/chat/simplex/common/model/DirectorySearch.kt + sed -i "s|^const val DIRECTORY_SERVICE_LINK = \".*\"$|const val DIRECTORY_SERVICE_LINK = \"$link\"|" "$file" + grep -F "const val DIRECTORY_SERVICE_LINK = \"$link\"" "$file" + - name: Setup Java uses: actions/setup-java@v5 with: @@ -431,7 +439,14 @@ jobs: sudo chmod +x /usr/local/bin/ui - name: Start the app - run: scripts/ci/desktop-linux/smoke-test.sh apps/multiplatform/release/main/app/simplex "$RUNNER_TEMP/claude-test/screens" start + env: + BACKEND: ${{ needs.plan.outputs.run-backend }} + run: | + if [ "$BACKEND" = true ]; then + export NAMES_SERVER=$(cat "$RUNNER_TEMP/backend/names-server") + cases=setup + fi + scripts/ci/desktop-linux/smoke-test.sh apps/multiplatform/release/main/app/simplex "$RUNNER_TEMP/claude-test/screens" ${cases:-start} # Claude's Bash is limited to single ui commands by the guard hook in claude-test.sh. Claude Code's credential # scrub stays off: on Linux it forces its whole Bash sandbox (bubblewrap, socat), which ui isn't set up for. @@ -440,7 +455,8 @@ jobs: env: CLAUDE_CODE_OAUTH_TOKEN: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }} CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC: 1 - run: scripts/ci/desktop-linux/claude-test.sh "$RUNNER_TEMP/claude-test" "$RUNNER_TEMP/claude-transcript.jsonl" + TEST_PROMPT: scripts/ci/desktop-linux/${{ needs.plan.outputs.run-backend == 'true' && 'claude-search.md' || 'claude-test.md' }} + run: scripts/ci/desktop-linux/claude-test.sh "$RUNNER_TEMP/claude-test" "$RUNNER_TEMP/claude-transcript.jsonl" "$TEST_PROMPT" - name: Report Claude usage if: ${{ !cancelled() && steps.claude.outcome != 'skipped' }} diff --git a/scripts/ci/desktop-linux/claude-search.md b/scripts/ci/desktop-linux/claude-search.md new file mode 100644 index 0000000000..892773e5c8 --- /dev/null +++ b/scripts/ci/desktop-linux/claude-search.md @@ -0,0 +1,62 @@ +# Test: online search in the SimpleX Chat desktop app + +You are testing the SimpleX Chat desktop app, which runs on a virtual display on this machine. Drive it only with the `ui` helper, through Bash, one command per call: `ui [args]`. Anything else is refused; don't look for ways around that, and if something blocks the test, report it. The commands are listed at the end. + +Your working directory is where your output goes. `ui shot ` saves `screens/NN-.png` and prints the text read from it, one phrase per line after its centre `x,y` in screen pixels. Read the PNG to see the screen (1440×900, the app window at the top left). OCR can misread or split words; when the text and the picture disagree, trust the picture. "Above" means a smaller y. + +## What is set up + +- The app is already running, past onboarding and set up as below: never run `ui start` or `ui stop`. Begin with `ui shot start` to see it. +- The app has a profile and one chat, a group of its own called "seeded notes". The chat list is in the left pane (x 1 to 389). Its toolbar and the search field "Search or paste SimpleX link" are at the bottom: the default "Reachable app toolbars" layout. +- A test directory lists a group, #ci_group ("CI seeded group", a red avatar with a white G), and a channel, #ci_channel ("CI seeded channel", a green avatar with a white C). Their avatars are as large as the app makes them, so the directory returns one of them per response: a search first shows one result and "Show more", which loads the other. +- A test server resolves the SimpleX name `seeded.testing` to the address of a contact, ci_contact ("CI seeded contact", a blue avatar with a white N), and the app uses only that server to resolve names. + +## The test + +Run all of A and B in the default layout, with the toolbar at the bottom. Then move the toolbar to the top and repeat A and B with only the checks marked **(both layouts)**: the two layouts lay the list out differently, but draw the rows themselves the same way. + +**A. Search in Directory.** + +1. Click the search field and type `ci seed`. A row "Search in Directory" appears. Take a shot: in the first layout, check that the chat list still shows its banner "Invest in SimpleX Chat", the "Bottom bar" / "Top bar" card and the "New 1-time link" / "Paste link" cards. Then click "Search in Directory". +2. The alert "Search in Directory?" asks before searching. In the first layout click "Search"; in the second, "Search and don't show again". +3. Wait for results, up to 60 s. If "Search failed, tap to retry" shows, tap it once and note it. +4. Use "Show more" until both #ci_group and #ci_channel are listed, and note how many pages that took. +5. Check, noting the evidence (screenshot names, positions): + - each result's row shows its name, its description, its count ("2 members" or "1 subscriber") and "Join group" (#ci_group) or "Join channel" (#ci_channel); + - the count's text is smaller than the description's above it: compare `ui ink "1 subscriber"` with `ui ink "CI seeded channel"`, and look at the screenshot; + - each shows its avatar, filling the picture: run `ui avatar "#ci_group" red` and `ui avatar "#ci_channel" green`, and look at the screenshot; + - that banner and those cards are hidden while the results show; + - **(both layouts)** the "Directory" header is above its results; + - **(both layouts)** no "Show more" is left, and nothing is cut off, overlapping or garbled. +6. Press Escape to clear the search field. In the first layout, take a shot and check that the banner and the cards show again. + +**B. The full search.** + +1. Click the search field and type `seeded`. A row "Connect to seeded.testing" appears. +2. Press Enter (`ui key Return`): on desktop it is the Search key. In the first layout the same alert asks first: click "Search". In the second layout it should not ask. +3. Wait up to 60 s for the results, and use "Show more" until both directory entries are listed. +4. Check, noting the evidence: + - **(both layouts)** three parts: the local chat "seeded notes"; a "Names" header with one row for ci_contact, showing `(@seeded.testing)` and "CI seeded contact"; and the "Directory" header with #ci_group and #ci_channel; + - "Connect to seeded.testing" is gone once the name is found; + - ci_contact shows its avatar: `ui avatar "ci_contact" blue`; + - **(both layouts)** each header is above all of its part's rows; + - **(both layouts)** the order of the parts: with the toolbar at the bottom the list reads upwards from the search field (the local chat nearest the field, then Names, then Directory furthest up); with the toolbar at the top it reads downwards from the field in the same order. +5. Press Escape to clear the search field. + +**Between the layouts:** click the profile picture at the left end of the bottom toolbar (about 27,730) to open the menu, open "Settings", then "Appearance", and turn off the switch of "Reachable app toolbars", at the right end of its row (about x 335). Press Escape until the chat list shows again; its search field should now be at the top. + +You have at most 60 turns, and the whole test fits in about 40 if you chain steps whose outcome you can predict with `ui do` and look at the screen after each chain. For example, in the first layout: + +- `ui do click-text "Search or paste SimpleX link" -- type "ci seed" -- wait "Search in Directory" -- shot bottom-typed` +- `ui do click-text "Search in Directory" -- wait "Search and don't show again" -- click-text "Search" -- wait "Join" 60 -- shot bottom-directory-1` +- `ui do click-text "Show more" -- wait "#ci_group" 60 -- shot bottom-directory-2` (wait for whichever result is missing) +- `ui do avatar "#ci_group" red -- avatar "#ci_channel" green` +- `ui do key Escape -- click-text "Search or paste SimpleX link" -- type seeded -- wait "Connect to seeded.testing" -- key Return -- wait "Search and don't show again" -- click-text "Search" -- wait ci_contact 60 -- shot bottom-full-1` + +`ui do` stops at the first command that fails, so a failed chain tells you where it stopped. + +## Report + +Write `report.md` in your working directory: for each layout and each of A and B, each check you ran as PASS or FAIL with its evidence, the number of pages and retries, and anything else unexpected. End your final message with one line: `CLAUDE-TEST: PASS` if every check you ran passed, otherwise `CLAUDE-TEST: FAIL` followed by the checks that failed. + +## The ui helper diff --git a/scripts/ci/desktop-linux/claude-test.sh b/scripts/ci/desktop-linux/claude-test.sh index d254226b75..c3fc500950 100755 --- a/scripts/ci/desktop-linux/claude-test.sh +++ b/scripts/ci/desktop-linux/claude-test.sh @@ -1,9 +1,10 @@ #!/bin/bash -# Runs Claude Code to repeat the smoke test with the ui helper, as claude-test.md describes, and passes when -# its final message ends with "CLAUDE-TEST: PASS". -# Usage: scripts/ci/desktop-linux/claude-test.sh -# Expects the app started (smoke-test.sh ... start) with UI_OUT under , claude and ui on PATH, +# Runs Claude Code to repeat the smoke test with the ui helper, as claude-test.md describes, or to run the test the +# given prompt describes, and passes when its final message ends with "CLAUDE-TEST: PASS". +# Usage: scripts/ci/desktop-linux/claude-test.sh [prompt .md] +# Expects the app started as the prompt says (smoke-test.sh ... start for claude-test.md, ... setup for +# claude-search.md) with UI_OUT under , claude and ui on PATH, # CLAUDE_MODEL and CLAUDE_CODE_OAUTH_TOKEN. Claude works in : the screenshots, its report.md and the # prompt it was given end up there. The transcript, Claude Code's stderr (.stderr) and the guard hook's # log (.guard.log) are kept outside it, where Claude can't write. If any of them holds the token, all are @@ -15,13 +16,14 @@ set -euo pipefail here=$(cd "$(dirname "$0")" && pwd) out=$(cd "$1" && pwd) transcript=$2 +test_prompt=${3:-$here/claude-test.md} stderr=$transcript.stderr settings=$(mktemp) export CLAUDE_GUARD_LOG=$transcript.guard.log : > "$CLAUDE_GUARD_LOG" { - cat "$here/claude-test.md" + cat "$test_prompt" echo echo '```' "$here/ui" || true