mirror of
https://github.com/Kpa-clawbot/meshcore-analyzer.git
synced 2026-04-25 07:42:19 +00:00
5ff4b75a07
Automates QA plan §10.1 (nodeBlacklist hide) and §10.2 (DB retain), flipping both rows from `human` to `auto`. Stacks on top of #808. **What** - New `qa/scripts/blacklist-test.sh` — env-driven harness: - Args: `BASELINE_URL TARGET_URL TEST_PUBKEY` - Env: `TARGET_SSH_HOST`, `TARGET_SSH_KEY` (default `/root/.ssh/id_ed25519`), `TARGET_CONFIG_PATH`, `TARGET_CONTAINER`, optional `TARGET_DB_PATH` / `ADMIN_API_TOKEN`. - Edits `nodeBlacklist` on target via remote `jq` (python3 fallback), atomic move with preserved perms. - Restarts container, waits up to 120 s for `/api/stats == 200`. - §10.1 asserts `/api/nodes/{pk}` is 404 **or** absent from `/api/nodes` listing, and `/api/topology` does not reference the pubkey. - §10.2 prefers `/api/admin/transmissions` if `ADMIN_API_TOKEN` set, else falls back to `sqlite3` inside the container (and host as last resort). - **Teardown is mandatory** (`trap … EXIT INT TERM`): removes pubkey, restarts, verifies the node is visible again. Teardown failures count toward exit code. - Exit code = number of failures; per-step ✅/❌ with classified failure modes (`ssh-failed`, `restart-stuck`, `hide-failed`, `retain-failed`, `teardown-failed`). - `qa/plans/v3.6.0-rc.md` §10.1 / §10.2 mode → `auto (qa/scripts/blacklist-test.sh)`. **Why** Manual blacklist verification was the slowest item in the §10 block and the easiest to get wrong (forgetting teardown leaks state into the next QA pass). Now it's a single command, public-repo-safe (zero PII / hardcoded hosts), and the trap guarantees the target is restored. `bash -n` passes locally. Live run requires staging credentials. --------- Co-authored-by: meshcore-agent <agent@meshcore> Co-authored-by: meshcore-agent <meshcore@openclaw.local>
CoreScope QA artifacts
Project-specific assets for the qa-suite skill.
Layout
qa/
├── README.md ← this file
├── plans/
│ └── <release>.md ← per-release test plans (one file per RC)
└── scripts/
└── api-contract-diff.sh ← CoreScope-tuned API contract diff
How to run
qa staging # use the latest plans/v*-rc.md against staging
qa pr 806 # use plans/pr-806.md if it exists, else latest plans/v*-rc.md
qa v3.6.0-rc # use plans/v3.6.0-rc.md
The parent agent loads the qa-suite skill, which reads:
- The plan file from
qa/plans/ - Bundled scripts from
qa/scripts/ - The reusable engine + qa-engineer persona from the skill itself
Adding a new plan
For each release candidate, copy the latest plans/v*-rc.md to plans/<new-tag>.md and update:
- The commit-range header (
vN.M..master) - Any new sections for new features in the release
- The "Test data" section if new fixture types are needed
- The GO criteria (which sections are blockers)
Adding a new script
Custom scripts go in qa/scripts/ with mode=auto: <script-name> referenced from the plan. The qa-engineer subagent runs them with two args: BASELINE_URL TARGET_URL.
Authoring rules from the qa-suite skill:
- 4-way error classification:
curl-failed/parse-empty/shape-diff/ field-missing - Distinguish HTTP errors from jq parse failures
- Don't silence stderr — script bugs must surface
- Exit code = number of failures