mirror of
https://github.com/simplex-chat/simplex-chat.git
synced 2026-11-04 06:32:59 +00:00
ci: Claude test plans for channel message signing and channel moderators
This branch's iOS device test plan and desktop prompt replace master's smoke tests: the owner creates a channel on the preset relays, turns on Sign messages through its alert, and two more profiles in the same app join by the copied link; the owner makes one a moderator, who then sees the subscriber list, moderates a contributor's message and blocks the contributor. Signatures are checked as text in message Info. One layout, no test backend. Run: 001-channel-signing-moderators, slice 6
This commit is contained in:
1 parent
13287fd9d8
commit
af7e1caae6
4 files changed
+86
-35
No files matched your search
@@ -2,3 +2,5 @@
|
||||
# backend: yes when the prompt uses the test backend; a run that asks for Claude must then start it too.
|
||||
# max-turns and max-budget-usd (optional) bound the session; claude-test.sh uses 60 and 3 when they're absent.
|
||||
backend: no
|
||||
max-turns: 300
|
||||
max-budget-usd: 5
|
||||
@@ -1,52 +1,73 @@
|
||||
# Test: onboarding and the chat list in the SimpleX Chat desktop app
|
||||
# Test: channel message signing and channel moderators in the SimpleX Chat desktop app
|
||||
|
||||
You are testing the SimpleX Chat desktop app, which runs on a virtual display on this machine. Drive it only with the `ui` helper, through Bash, one command per call: `ui <command> [args]`. Anything else is refused; don't look for ways around that, and if something blocks the test, report it. The commands are listed at the end.
|
||||
|
||||
Your working directory is where your output goes. `ui shot <name>` saves `screens/NN-<name>.png` and prints the text read from it, one phrase per line after its centre `x,y` in screen pixels. Read the PNG to see the screen (1440×900, the app window at the top left). OCR can misread or split words; when the text and the picture disagree, trust the picture. "Above" means a smaller y.
|
||||
Your working directory is where your output goes. `ui shot <name>` saves `screens/NN-<name>.png` and prints the text read from it, one phrase per line after its centre `x,y` in screen pixels. Read the PNG to see the screen (1440×900, the app window at the top left). OCR can misread or split words and can't read icons; when the text and the picture disagree, trust the picture. "Above" means a smaller y.
|
||||
|
||||
## What is set up
|
||||
|
||||
- The app is already running, on its first screen, with a new, empty database: never run `ui start` or `ui stop`. Begin with `ui shot start` to see it.
|
||||
- After onboarding, the chat list is in the left pane (x 1 to 389), and the right pane shows the empty chat list's cards. An empty chat list shows only its toolbar, with the profile picture at about 27,730 and a pencil at about 363,730: the search field "Search or paste SimpleX link" appears once there is a chat. In the default "Reachable app toolbars" layout, the toolbar and the search field are at the bottom of the left pane.
|
||||
- After onboarding, the chat list is in the left pane (x 1 to 389) and chats open in the right pane. An empty chat list shows only its toolbar, with the profile picture at about 27,730 and a pencil (New chat) at about 363,730: the search field "Search or paste SimpleX link" appears once there is a chat. The toolbar is at the bottom of the left pane.
|
||||
- The channel uses the app's preset chat relays, on the internet. Creating it and joining it wait for them: wait up to 120 s there (`ui wait "<text>" 120`).
|
||||
|
||||
## How to do things in the app
|
||||
|
||||
- **Send a message:** click the message field at the bottom of the right pane (its placeholder is "Broadcast" for the channel's owner, "Message" for others), `ui type "<text>"`, then `ui key Return`. Check the message then shows in the chat.
|
||||
- **Channel info:** click the channel name at the top of the right pane. Press Escape to close it and other panes (preferences, member info, a message's info) and get back to the chat.
|
||||
- **Message menu:** `ui right-click <x> <y>` on the message's text (centre from `ui find`) opens its menu ("Info", "Moderate", …).
|
||||
- **Signatures are read as text, never from the seal icons:** a message's "Info" shows "Signed" (your own message), "Signed & verified" (someone else's, checked) or "Signature missing".
|
||||
- **Switch profile:** click the profile picture (about 27,730); the menu lists your other profiles by name: click the one you want. The chat list then shows that profile's chats; click "ci signing" in it.
|
||||
- **Add a profile:** profile picture → "Your chat profiles" → "Add profile". On "Create profile", click the field about 40 px below the text "Enter your name:", type the name, and click the "Create profile" button (the lower of the two "Create profile" texts; the upper one is the title). The new profile becomes the active one; press Escape until the chat list shows.
|
||||
- **Join by the copied link** (from a profile without chats): the pencil (about 363,730) → "Paste link" → "Tap to paste link". An alert names the channel with "Open channel": click it. The channel opens with "Tap Join channel" and a "Join channel" bar at the bottom: click the bar, and wait up to 120 s until the bottom of the chat shows "you are subscriber".
|
||||
|
||||
## The test
|
||||
|
||||
**A. Onboarding.** Each screen waits for the one before it; wait for a text before clicking it.
|
||||
The parts build on each other: run them in this order. cisub must join only after cimod has become a moderator (B2), so that cimod learns about cisub.
|
||||
|
||||
1. The first screen has "Create your profile": click it.
|
||||
2. On "Create profile", click the field "Enter profile name", type `citester`, and click "Create profile".
|
||||
3. The next screen offers a database passphrase: click "Use random passphrase", then "OK" in the alert that follows.
|
||||
4. Click "Continue", then "Accept" on the screen after it.
|
||||
5. Wait up to 60 s for the chat list: the right pane shows "Talk to someone". Take a shot.
|
||||
**S. Setup.**
|
||||
|
||||
**B. A group, and the chat list with the toolbar at the bottom.**
|
||||
1. (S1) Onboarding. Each screen waits for the one before it; wait for a text before clicking it. The first screen has "Create your profile": click it. On "Create profile", click the field "Enter profile name", type `citester`, and click "Create profile". The next screen offers a database passphrase: click "Use random passphrase", then "OK" in the alert that follows. Click "Continue", then "Accept" on the screen after it. Wait up to 60 s for the chat list: the right pane shows "Talk to someone".
|
||||
2. (S2) Click the pencil, then "Create public channel". Click the field about 40 px below the text "Channel name", type `ci signing`, and click the "Create public channel" button (the lower of the two texts with that name). Wait up to 120 s for "Share link": the channel opens and its "Channel link" pane shows over it. If "N/M relays active" stays below M for 120 s with N at least 1, click "Continue", and "Continue" in the alert. On "Channel link", click "Continue". Check: the chat "ci signing" is open.
|
||||
3. (S3) Send `before signing`. Check: it shows in the chat.
|
||||
|
||||
1. Click the pencil (about 363,730) to open New chat, then "Create group". Click the name field, about 40 px below the text "Enter group name", type `seeded notes`, and click "Create group". If "Skip inviting members" shows, click it.
|
||||
2. The group opens in the right pane, and stays there. Press Escape until the left pane shows the chat list again, then take a shot.
|
||||
3. Check, noting the evidence (screenshot names, positions):
|
||||
- "seeded notes" is listed in the left pane;
|
||||
- the search field "Search or paste SimpleX link" is at the bottom of the left pane (y over 600), and "Add list" is just above it;
|
||||
- nothing is cut off, overlapping or garbled.
|
||||
**A. "Sign messages" (as citester).**
|
||||
|
||||
**Between the layouts:** click the profile picture at the left end of the bottom toolbar (about 27,730) to open the menu, open "Settings", then "Appearance", and turn off the switch of "Reachable app toolbars", at the right end of its row (about x 335). Press Escape until the chat list shows again; its search field should now be at the top.
|
||||
1. (A1) Channel info → "Channel preferences". Check: the first row is "Sign messages", and its footer says "Do not require signing messages." (the switch is off).
|
||||
2. (A2) Click the switch of "Sign messages", at the right end of its row. Check: the alert "Sign messages?" says "Messages sent before will show "signature missing" to subscribers who join later." and has "Enable" and "Cancel". Click "Cancel". Check: the footer says "Do not require signing messages." again.
|
||||
3. (A3) Click the switch again, then "Enable". Check: the footer says "Require signing messages.". Click "Save and notify channel subscribers", then press Escape until the chat shows. Check: the chat shows "Sign messages: on".
|
||||
4. (A4) Send `after signing`, right-click it → "Info". Check: the info says "Signed".
|
||||
5. (A5) Channel info → "Channel preferences". Check: the footer of "Chat with admins" says "Prohibit chats with admins." (saving in A3 did not turn it on). Press Escape until the chat shows.
|
||||
|
||||
**C. The chat list, toolbar at the top.** Check, noting the evidence:
|
||||
**J. A subscriber joins.**
|
||||
|
||||
- "seeded notes" is still listed;
|
||||
- the search field is at the top of the left pane (y under 200), and "Add list" is just below it;
|
||||
- nothing is cut off, overlapping or garbled. The pencil is now a floating button at the bottom right of the left pane, drawn over whatever the list shows there: note what it covers, but that alone is not a failure.
|
||||
1. (J1) Channel info → "Channel link" → "Share link". Check: the toast "Copied to clipboard". Press Escape until the chat list shows.
|
||||
2. (J2) Add the profile `cimod`, and as cimod join by the copied link. Check: "you are subscriber".
|
||||
3. (A6) As cimod in "ci signing": click the message "before signing". Check: the alert "Signature missing" says "This channel requires signed messages, but this message has no signature. It may have been sent before signing was required, or a relay may have forwarded it without one." Close it, right-click "before signing" → "Info". Check: "Signature missing". Then right-click "after signing" → "Info". Check: "Signed & verified".
|
||||
|
||||
You have at most 60 turns, and the whole test fits in about 25 if you chain steps whose outcome you can predict with `ui do` and look at the screen after each chain. For example:
|
||||
**B. A moderator.**
|
||||
|
||||
1. (B1) Switch to citester → "ci signing" → channel info → "Subscribers" → "cimod". Click the "Change role" row. Check: the list offers exactly "subscriber", "contributor" and "moderator" (no admin, no owner).
|
||||
2. (B2) Click "moderator", then "Change" in the alert "Change role?". Check: cimod's role shows moderator, and the section above it is titled "Moderator". Press Escape until the chat shows. Check: the chat shows "you changed role of cimod to moderator".
|
||||
3. (B3) Switch to cimod → "ci signing". Check: the chat shows "changed your role to moderator". Send `moderator post`. Check: it shows in the chat.
|
||||
4. (J3) Add the profile `cisub`, and as cisub join by the same link (it is still on the clipboard; if not, copy it again as citester as in J1). Check: "you are subscriber".
|
||||
5. (J4) Switch to citester → "ci signing" → channel info → "Subscribers" → "cisub" → "Change role" → "contributor" → "Change". Check: cisub's role shows contributor, in the section titled "Contributor".
|
||||
6. (J5) Switch to cisub → "ci signing". Send `contributor post`. Check: it shows in the chat.
|
||||
7. (B4) Switch to cimod → "ci signing" → channel info. Check: it shows "Subscribers"; click it: the list includes "cisub".
|
||||
8. (B5) Click "cisub". Check: its info offers "Block for all" (don't click it yet). Press Escape until the chat shows.
|
||||
9. (B6) Right-click "contributor post" → "Moderate" → "Delete" in the alert "Delete member message?". Check: the message shows "moderated by cimod" in its place.
|
||||
10. (B7) Channel info → "Subscribers" → "cisub" → "Block for all" → "Block for all" in the alert "Block subscriber for all?". Check: cisub's info now offers "Unblock for all". Press Escape until the chat shows. Check: the chat shows "you blocked cisub".
|
||||
|
||||
You have at most 300 turns, and the whole test fits in about 150 if you chain steps whose outcome you can predict with `ui do` and look at the screen after each chain. For example:
|
||||
|
||||
- `ui do click-text "Create your profile" -- wait "Enter profile name" -- click-text "Enter profile name" -- type citester -- shot profile`
|
||||
- `ui do click-text "Create profile" -- wait "Use random passphrase" 60 -- shot passphrase`
|
||||
- `ui do click 363 730 -- click-text "Create group" 10 -- wait "Enter group name" -- shot new-group`
|
||||
- `ui do click 27 730 -- click-text Settings 10 -- click-text Appearance 10 -- wait "Reachable app toolbars" -- shot appearance`
|
||||
- `ui do click 363 730 -- click-text "Create public channel" 10 -- wait "Channel name" -- shot new-channel`
|
||||
- `ui do click-text "Broadcast" -- type "after signing" -- key Return -- wait "after signing" -- shot after-signing`
|
||||
- `ui do click 27 730 -- click-text "Your chat profiles" 10 -- click-text "Add profile" 10 -- wait "Enter your name" -- shot add-profile`
|
||||
|
||||
`ui do` stops at the first command that fails, so a failed chain tells you where it stopped.
|
||||
`ui do` stops at the first command that fails, so a failed chain tells you where it stopped. Take a shot at each check, named after its step (`ui shot A2`).
|
||||
|
||||
## Report
|
||||
|
||||
Write `report.md` in your working directory: for each of A, B and C, each check you ran as PASS or FAIL with its evidence, and anything else unexpected. End your final message with one line: `CLAUDE-TEST: PASS` if every check you ran passed, otherwise `CLAUDE-TEST: FAIL` followed by the checks that failed.
|
||||
Write `report.md` in your working directory as you go, so a session that is cut short still leaves its results: one row per step (S1 to S3, A1 to A5, J1, J2, A6, B1 to B3, J3 to J5, B4 to B7), each check you ran as PASS or FAIL with its evidence (the text OCR read, screenshot names), and anything else unexpected. End your final message with one line: `CLAUDE-TEST: PASS` if every check you ran passed, otherwise `CLAUDE-TEST: FAIL` followed by the steps that failed.
|
||||
|
||||
## The ui helper
|
||||
@@ -5,6 +5,6 @@
|
||||
backend: no
|
||||
model: claude-haiku-5-5
|
||||
layouts: reachable
|
||||
cases-reachable: S1-S3
|
||||
max-turns: 60
|
||||
max-budget-usd: 3
|
||||
cases-reachable: S2, S3, A1, A2, A3, A4, A5, J1, J2, A6, B1, B2, B3, J3, J4, J5, B4, B5, B6, B7
|
||||
max-turns: 350
|
||||
max-budget-usd: 5
|
||||
@@ -1,9 +1,37 @@
|
||||
# Device test plan: smoke test
|
||||
# Device test plan: channel message signing and channel moderators
|
||||
|
||||
The app as the scripted smoke test leaves it: a profile created through onboarding, without conversations. A branch that tests a feature replaces this plan with its own (same table format) and names its cases in `claude-scope`.
|
||||
The app as the scripted smoke test leaves it: the profile "citester", without conversations. The cases run in the order of the tables and build on each other: one channel, "ci signing", owned by citester, and two more profiles in this app, "cimod" and "cisub", that join it by its link. Do not reorder them: cisub must join only after cimod has become a moderator (B2), so that cimod learns about cisub.
|
||||
|
||||
## Setup
|
||||
|
||||
- The channel uses the app's preset chat relays. Creating it and joining it wait for the relays: use `ui wait "<text>" 120` there, and take a shot if a wait fails.
|
||||
- **Sending a message:** in the chat, tap the message field at the bottom (find it in `ui tree`: the text view next to the send button; for the channel's owner its placeholder is "Broadcast"), `ui type "<text>"`, then tap the send button at the right end of the field. Check the message then shows in the chat.
|
||||
- **Channel info:** tap the channel name at the top of the chat (`ui find "ci signing"`, the element with the smallest y; tap it with `ui tap-xy`). Go back with the navigation back buttons; close a sheet by swiping it down.
|
||||
- **Message menu:** `ui long-press <x> <y>` on the message's text (centre from `ui find`) opens its menu ("Info", "Moderate", …). "Info" opens the message's details; close them by swiping down.
|
||||
- **Signatures are read as text, never from the seal icons:** a message's "Info" shows "Signed" (your own message), "Signed & verified" (someone else's, checked) or "Signature missing".
|
||||
- **Switching profile:** open the picker with your profile image (`Image "Account"`), then tap the other profile's name in the row at the top of the picker (`ui find <name>`, then `ui tap-xy`; the picker's elements are in the tree even while it is closed, so open it first). The chat list then shows that profile's chats; open "ci signing" from it.
|
||||
- **Adding a profile:** picker → "Your chat profiles" → "Add profile" → tap the field "Enter your name…" (`ui tap-value "Enter your name…"`), type the name, check the field's value in the tree (the keyboard may capitalize it: then fix it), tap "Create profile". The new profile becomes the active one; close "Your chat profiles" by swiping it down.
|
||||
- **Joining by the copied link** (from a profile without conversations): "Compose" → "Scan / Paste link" → "Tap to paste link". If iOS asks to allow pasting, tap "Allow Paste". An alert names the channel with "Open channel": tap it; the channel opens with "Tap Join channel" and a "Join channel" bar at the bottom: tap the bar, and wait up to 120 s until the chat shows "you are subscriber".
|
||||
|
||||
| ID | Do | Expect |
|
||||
|----|----|--------|
|
||||
| S1 | Look at the screen: `ui tree`, then `ui shot S1-<layout>` | The chat list of a profile without conversations: the heading "Talk to someone", the cards "Let someone connect to you" and "Connect via link or QR code", and the toolbar with your profile image (`Image "Account"`, 32 x 32) and "Compose": at the bottom of the screen with the one-hand UI, at the top with the top bar |
|
||||
| S2 | Tap "Let someone connect to you", then "Back" | The page "Create your link" with the cards "Invite someone privately" and "Create your public address" and a "Back" button; after "Back", "Talk to someone" again |
|
||||
| S3 | Open the profile picker by tapping the toolbar's profile image, tap "Settings", then close Settings by swiping it down | The picker shows your profile name and "Settings"; Settings opens with "Your settings" and "Network & servers"; after closing, "Talk to someone" again |
|
||||
| S2 | "Compose" → "Create public channel" → `ui tap-value "Enter channel name…"`, type `ci signing` (check the value) → tap the "Create public channel" button (the list row, not the title; use `ui tap-xy` if the label is ambiguous) → wait up to 120 s for "Channel link"; if "%d/%d relays active" stays below the total for 120 s with at least 1 active, tap "Continue" and "Continue" in the alert → on "Channel link" tap "Continue" (top right) | The channel "ci signing" opens as its chat |
|
||||
| S3 | Send `before signing` | "before signing" shows in the chat as sent |
|
||||
| A1 | Channel info → "Channel preferences" | The first row is the switch "Sign messages", off; its footer says "Do not require signing messages." |
|
||||
| A2 | Turn "Sign messages" on | Alert "Sign messages?" with the message "Messages sent before will show "signature missing" to subscribers who join later." and the buttons "Enable" and "Cancel". Tap "Cancel": the switch is off again |
|
||||
| A3 | Turn "Sign messages" on again, tap "Enable", then "Save and notify subscribers"; go back to the chat | After "Enable" the switch stays on (footer "Require signing messages."); after saving and going back, the chat shows the event "Sign messages: on" |
|
||||
| A4 | Send `after signing`, then long-press it → "Info" | The message's info says "Signed" |
|
||||
| A5 | Channel info → "Channel preferences" | "Chat with admins" is still off (its footer says "Prohibit chats with admins."); saving in A3 did not turn it on |
|
||||
| J1 | Channel info → "Channel link" → "Share link" → in the share sheet tap "Copy" (`ui find Copy`; if it is not in the tree, `ui tap-xy` at its position in the screenshot) → close the sheet and go back to the chat list | The link is copied (the sheet closes or shows it was copied) |
|
||||
| J2 | Add the profile `cimod`; as cimod, join by the copied link | The chat "ci signing" shows "you are subscriber" |
|
||||
| A6 | As cimod in "ci signing": tap the message "before signing"; then long-press it → "Info"; then long-press "after signing" → "Info" | Tapping "before signing" shows the alert "Signature missing" with "This channel requires signed messages, but this message has no signature. It may have been sent before signing was required, or a relay may have forwarded it without one."; its info says "Signature missing"; the info of "after signing" says "Signed & verified" |
|
||||
| B1 | Switch to citester → "ci signing" → channel info → "Subscribers" → "cimod" → tap the "Change role" row | The role menu lists exactly "subscriber", "contributor" and "moderator" (no admin, no owner) |
|
||||
| B2 | Choose "moderator" → "Change role?" → "Change"; go back to the chat | cimod's info shows the role moderator under the section header "Moderator" (any case); the chat shows "you changed role of cimod to moderator" |
|
||||
| B3 | Switch to cimod → "ci signing"; send `moderator post` | The chat shows "changed your role to moderator"; "moderator post" shows as sent |
|
||||
| J3 | Add the profile `cisub`; as cisub, join by the same link (it is still on the clipboard; if not, copy it again as citester as in J1) | cisub's chat "ci signing" shows "you are subscriber" |
|
||||
| J4 | Switch to citester → "ci signing" → channel info → "Subscribers" → "cisub" → "Change role" → "contributor" → "Change" | cisub's role shows contributor, under the section header "Contributor" |
|
||||
| J5 | Switch to cisub → "ci signing"; send `contributor post` | "contributor post" shows as sent |
|
||||
| B4 | Switch to cimod → "ci signing" → channel info | The info shows "Subscribers"; its list includes "cisub" |
|
||||
| B5 | Open "cisub" in that list | cisub's info offers "Block for all" (do not tap it yet) |
|
||||
| B6 | Back in the chat, long-press "contributor post" → "Moderate" → "Delete member message?" → "Delete" | The message shows "moderated by cimod" in its place |
|
||||
| B7 | Channel info → "Subscribers" → "cisub" → "Block for all" → "Block subscriber for all?" → "Block for all" | cisub's info now offers "Unblock for all"; the chat shows "you blocked cisub" |
|
||||
Reference in new issue
Block a user