diff --git a/scripts/ci/desktop-linux/README.md b/scripts/ci/desktop-linux/README.md index b696e9b528..cabebcbe1f 100644 --- a/scripts/ci/desktop-linux/README.md +++ b/scripts/ci/desktop-linux/README.md @@ -41,7 +41,7 @@ It writes the directory's address to `/directory-link` and the SMP server's ## The Claude job -It needs the GitHub environment `app-sim-claude` with the secret `CLAUDE_CODE_OAUTH_TOKEN`, a Claude subscription token. Which branches may use it is decided only by that environment's branch rule in the repository settings: on a branch the rule doesn't allow, a requested Claude run fails at the environment check. It runs once the app job has passed, on that job's distributable, one per branch at a time. The token goes only to the step that runs `claude`; `claude-test.sh` passes it to the `claude` process alone, and every `ui` call unsets it too. The job starts the test backend when `claude-scope` says `backend: yes`, as it does on this branch; a run that asks for Claude must then ask for the backend too, or the plan job fails. The app job built the distributable pointing at its own backend's directory, which a new backend would not have, so the app job saves that backend's stopped work dir (the artifact `desktop-backend-state`, without the binaries) and the Claude job starts it again with `backend.sh resume`: the same servers' keys and databases, so the same addresses. That artifact holds the test backend's databases and server keys, unencrypted, and like every artifact of this public repository anyone signed in to GitHub can download it while it is kept (a day); with it, someone could act as the test directory or relay while the Claude job runs and show the session crafted results. This is an accepted risk: these are throwaway test identities, the session has no shell and no token in reach of its tools, so what is at stake is the test's integrity, and encrypting the state would need a key only the two jobs hold, a second secret, which this repository's rules for the Claude job don't allow. With the backend it sets the app up at the chat list (`smoke-test.sh … setup`) and gives Claude `claude-search.md`, the online search test, in place of the onboarding path in `claude-test.md`: the directory search with its consent, Open, the load row and the right-click menu (part A) and the Search key, which searches the directory only and keeps "Connect to …" (part B), in both layouts, then part C, which opens `ci_channel` with its row's Open, without an alert, joins it, and checks that later searches show it as a chat (no row and no note for a page of chats already listed, its chat row among the results of the chat list and of New chat, opening without an alert). The job installs Claude Code at the version the iOS workflow pins too, and runs `claude-haiku-5-5` with `--max-turns 120`, `--max-budget-usd 3` and the iOS sessions' other flags (`--restricted`, `--permission-prompts none`, `--effort high`). Its Bash runs only single `ui` commands (`claude-guard.py`, a PreToolUse hook: Claude Code runs read-only commands without asking in every permission mode). Before anything the session wrote is printed or uploaded, `claude-test.sh` checks it for the token; if it's there, the output, transcript, Claude Code's stderr and the guard log are deleted and the job fails. It passes when Claude's final message ends with `CLAUDE-TEST: PASS`; `report.md`, the transcript, the guard's log and `claude-usage.json` are in `desktop-claude-test`. +It needs the GitHub environment `app-sim-claude` with the secret `CLAUDE_CODE_OAUTH_TOKEN`, a Claude subscription token. Which branches may use it is decided only by that environment's branch rule in the repository settings: on a branch the rule doesn't allow, a requested Claude run fails at the environment check. It runs once the app job has passed, on that job's distributable, one per branch at a time. The token goes only to the step that runs `claude`; `claude-test.sh` passes it to the `claude` process alone, and every `ui` call unsets it too. The job starts the test backend when `claude-scope` says `backend: yes`, as it does on this branch; a run that asks for Claude must then ask for the backend too, or the plan job fails. The app job built the distributable pointing at its own backend's directory, which a new backend would not have, so the app job saves that backend's stopped work dir (the artifact `desktop-backend-state`, without the binaries) and the Claude job starts it again with `backend.sh resume`: the same servers' keys and databases, so the same addresses. That artifact holds the test backend's databases and server keys, unencrypted, and like every artifact of this public repository anyone signed in to GitHub can download it while it is kept (a day); with it, someone could act as the test directory, the relay or `ci_admin` (the owner of `ci_group` and `ci_channel`, so it can post in the channel the session joins), all with their queues on the preset servers, while the Claude job runs. This is an accepted risk: these are throwaway test identities, so what is at stake is the test's integrity (crafted results, posts and messages shown to the session), and encrypting the state would need a key only the two jobs hold, a second secret, which this repository's rules for the Claude job don't allow. The session's Bash inherits the token (Claude Code's credential scrub is off), which stays out of its reach only as long as `claude-guard.py` lets through just one `ui` command with plain arguments and no shell expansion outside quotes, and `ui` treats every argument as data (it checks the numbers it calculates with and passes text on quoted) and unsets the token: a change to either is a change to this boundary. With the backend it sets the app up at the chat list (`smoke-test.sh … setup`) and gives Claude `claude-search.md`, the online search test, in place of the onboarding path in `claude-test.md`: the directory search with its consent, Open, the load row and the right-click menu (part A) and the Search key, which searches the directory only and keeps "Connect to …" (part B), in both layouts, then part C, which opens `ci_channel` with its row's Open, without an alert, joins it, and checks that later searches show it as a chat (no row and no note for a page of chats already listed, its chat row among the results of the chat list and of New chat, opening without an alert). The job installs Claude Code at the version the iOS workflow pins too, and runs `claude-haiku-5-5` with `--max-turns 120`, `--max-budget-usd 3` and the iOS sessions' other flags (`--restricted`, `--permission-prompts none`, `--effort high`). Its Bash runs only single `ui` commands (`claude-guard.py`, a PreToolUse hook: Claude Code runs read-only commands without asking in every permission mode). Before anything the session wrote is printed or uploaded, `claude-test.sh` checks it for the token; if it's there, the output, transcript, Claude Code's stderr and the guard log are deleted and the job fails. It passes when Claude's final message ends with `CLAUDE-TEST: PASS`; `report.md`, the transcript, the guard's log and `claude-usage.json` are in `desktop-claude-test`. ## Cost diff --git a/scripts/ci/ios-sim/README.md b/scripts/ci/ios-sim/README.md index 867deb6191..0d41ab324d 100644 --- a/scripts/ci/ios-sim/README.md +++ b/scripts/ci/ios-sim/README.md @@ -22,7 +22,7 @@ The macOS jobs run on `macos-26`, the image with Xcode 26 and the iPhone 17 simu ## The Claude session -It runs in its own job, `claude`, once the app job has passed: the job installs that job's build (the artifact `ios-sim-app`, kept a day) in a new simulator, runs the smoke test to reach the sessions' starting state, then one session per layout listed in `claude-scope`, with Claude Code installed from npm at the version the desktop workflow pins too. Only this job references the environment `app-sim-claude`, which holds the secret `CLAUDE_CODE_OAUTH_TOKEN` (a Claude subscription token) and admits the branches its rule allows; one runs per branch at a time. Only the Claude step gets the token, and `claude-test.sh` passes it to the `claude` processes alone. The job checks out without persisting its GitHub token and installs a pinned AXe (`install-axe.sh`), since it holds the Claude token. It starts the test backend when `claude-scope` says `backend: yes`; a run that asks for Claude must then ask for the backend too, or the plan job fails. On master it says `no`, since the smoke plan doesn't use the backend. The app job built the app pointing at its own backend's directory, which a new backend would not have, so the app job saves that backend's stopped work dir (the artifact `ios-sim-backend-state`, without the binaries) and the Claude job starts it again with `backend.sh resume`: the same servers' keys and databases, so the same addresses. That artifact holds the test backend's databases and server keys, unencrypted, and like every artifact of this public repository anyone signed in to GitHub can download it while it is kept (a day); with it, someone could act as the test directory or relay while the Claude job runs and show the session crafted results. This is an accepted risk: these are throwaway test identities, the session has no shell and no token in reach of its tools, so what is at stake is the test's integrity, and encrypting the state would need a key only the two jobs hold, a second secret, which this repository's rules for the Claude job don't allow. +It runs in its own job, `claude`, once the app job has passed: the job installs that job's build (the artifact `ios-sim-app`, kept a day) in a new simulator, runs the smoke test to reach the sessions' starting state, then one session per layout listed in `claude-scope`, with Claude Code installed from npm at the version the desktop workflow pins too. Only this job references the environment `app-sim-claude`, which holds the secret `CLAUDE_CODE_OAUTH_TOKEN` (a Claude subscription token) and admits the branches its rule allows; one runs per branch at a time. Only the Claude step gets the token, and `claude-test.sh` passes it to the `claude` processes alone. The job checks out without persisting its GitHub token and installs a pinned AXe (`install-axe.sh`), since it holds the Claude token. It starts the test backend when `claude-scope` says `backend: yes`; a run that asks for Claude must then ask for the backend too, or the plan job fails. On master it says `no`, since the smoke plan doesn't use the backend. The app job built the app pointing at its own backend's directory, which a new backend would not have, so the app job saves that backend's stopped work dir (the artifact `ios-sim-backend-state`, without the binaries) and the Claude job starts it again with `backend.sh resume`: the same servers' keys and databases, so the same addresses. That artifact holds the test backend's databases and server keys, unencrypted, and like every artifact of this public repository anyone signed in to GitHub can download it while it is kept (a day); with it, someone could act as the test directory, the relay, `ci_admin` (the owner of the seeded groups and channels, so it can post in those the session joins) or the contacts cialice and Robert, all with their queues on the preset servers, while the Claude job runs. This is an accepted risk: these are throwaway test identities, so what is at stake is the test's integrity (crafted results, posts and messages shown to the session), and encrypting the state would need a key only the two jobs hold, a second secret, which this repository's rules for the Claude job don't allow. The session's Bash inherits the token (Claude Code's credential scrub is off), which stays out of its reach only as long as `claude-guard.py` lets through just one `ui` command with plain arguments and no shell expansion outside quotes, and `ui` treats every argument as data (it checks the numbers it calculates with and passes text on quoted) and unsets the token: a change to either is a change to this boundary. Each session's final message ends with `CLAUDE-TEST: PASS`, or `CLAUDE-TEST: FAIL` and the failed cases; the step fails unless every session passes.