Commit Graph

2 Commits

Author SHA1 Message Date
Kpa-clawbot
5ff4b75a07 qa: automate §10.1/§10.2 nodeBlacklist test (#822)
Automates QA plan §10.1 (nodeBlacklist hide) and §10.2 (DB retain),
flipping both rows from `human` to `auto`. Stacks on top of #808.

**What**
- New `qa/scripts/blacklist-test.sh` — env-driven harness:
  - Args: `BASELINE_URL TARGET_URL TEST_PUBKEY`
- Env: `TARGET_SSH_HOST`, `TARGET_SSH_KEY` (default
`/root/.ssh/id_ed25519`), `TARGET_CONFIG_PATH`, `TARGET_CONTAINER`,
optional `TARGET_DB_PATH` / `ADMIN_API_TOKEN`.
- Edits `nodeBlacklist` on target via remote `jq` (python3 fallback),
atomic move with preserved perms.
  - Restarts container, waits up to 120 s for `/api/stats == 200`.
- §10.1 asserts `/api/nodes/{pk}` is 404 **or** absent from `/api/nodes`
listing, and `/api/topology` does not reference the pubkey.
- §10.2 prefers `/api/admin/transmissions` if `ADMIN_API_TOKEN` set,
else falls back to `sqlite3` inside the container (and host as last
resort).
- **Teardown is mandatory** (`trap … EXIT INT TERM`): removes pubkey,
restarts, verifies the node is visible again. Teardown failures count
toward exit code.
- Exit code = number of failures; per-step / with classified failure
modes (`ssh-failed`, `restart-stuck`, `hide-failed`, `retain-failed`,
`teardown-failed`).
- `qa/plans/v3.6.0-rc.md` §10.1 / §10.2 mode → `auto
(qa/scripts/blacklist-test.sh)`.

**Why**
Manual blacklist verification was the slowest item in the §10 block and
the easiest to get wrong (forgetting teardown leaks state into the next
QA pass). Now it's a single command, public-repo-safe (zero PII /
hardcoded hosts), and the trap guarantees the target is restored.

`bash -n` passes locally. Live run requires staging credentials.

---------

Co-authored-by: meshcore-agent <agent@meshcore>
Co-authored-by: meshcore-agent <meshcore@openclaw.local>
2026-04-21 04:53:55 +00:00
Kpa-clawbot
f701121672 Add qa/ — project-specific QA artifacts for the qa-suite skill (#808)
Adds the CoreScope-side artifacts that pair with the generic [`qa-suite`
skill](https://github.com/Kpa-clawbot/ai-sdlc/pull/1).

## Layout

```
qa/
├── README.md
├── plans/
│   └── v3.6.0-rc.md       # 34-commit test plan since v3.5.1
└── scripts/
    └── api-contract-diff.sh  # CoreScope-tuned API contract diff
```

The skill ships the reusable engine + qa-engineer persona + an example
plan. This PR adds the CoreScope-tuned plan and the CoreScope-tuned
script (correct seed lookups for our `{packets, total}` response shape,
our endpoint list, our `resolved_path` requirement). Read by the parent
agent at runtime.

## How to use

From chat:

- `qa staging` — runs the latest `qa/plans/v*-rc.md` against staging,
files a fresh GH issue with the report
- `qa pr <N>` — uses `qa/plans/pr-<N>.md` if present, else latest RC
plan; comments on the PR
- `qa v3.6.0-rc` — runs that specific plan

The qa-engineer subagent walks every step, classifying each as `auto`
(script) / `browser` (UI assertion) / `human` (manual) / `browser+auto`.
Quantified pass criteria are mandatory — banned phrases: 'visually
aligned' / 'fast' / 'no regression'.

## Plan v3.6.0-rc contents

Covers the 34 commits since v3.5.1:
- §1 Memory & Load (#806, #790, #807) — heap thresholds, sawtooth
pattern
- §2 API contract (#806) — every endpoint that should carry
`resolved_path`, auto-checked by `api-contract-diff.sh`
- §3 Decoder & hashing (#787, #732, #747, #766, #794, #761)
- §4 Channels (#725 series M1–M5)
- §5 Clock skew (#690 series M1–M3)
- §6 Observers (#764, #774)
- §7 Multi-byte hash adopters (#758, #767)
- §8 Frontend nav & deep linking (#739, #740, #779, #785, #776, #745)
- §9 Geofilter (#735, #734)
- §10 Node blacklist (#742)
- §11 Deploy/ops

Release blockers: §1.2, §2, §3. §4 is the headline-feature gate.

## Adding new plans

Per release: copy `plans/v<last>-rc.md` to `plans/v<new>-rc.md` and
update commit-range header, new sections, GO criteria.

Per PR: create `plans/pr-<N>.md` with the bare minimum for that PR's
surface area.

Co-authored-by: you <you@example.com>
2026-04-20 21:46:57 -07:00