fix(ci): aggregate e2e pass/fail across all suites instead of broken digits-before-slash regex (#1298)

RED 33d789c4f3 (test) → GREEN
b43bd70f43 (fix). CI:
https://github.com/Kpa-clawbot/CoreScope/actions/workflows/deploy.yml?query=branch%3Afix%2Fe2e-badge-aggregate

Fixes #1296

## Problem
`.github/workflows/deploy.yml` was computing the e2e-tests badge with:

```
E2E_PASS=$(grep -oP '[0-9]+(?=/)' e2e-output.txt | tail -1 || echo "0")
```

This regex matched any digit-run immediately followed by `/` anywhere in
the combined output of 45+ Playwright suites, then took the **last**
match. The result was usually a small number scraped out of intermediate
per-suite progress text (often `2` from something like `2/3 …`), so the
badge perpetually showed `{"label":"e2e tests","message":"2
passed","color":"brightgreen"}` regardless of how many tests actually
ran.

## Fix
- New `scripts/aggregate-e2e-pass.sh` parses every per-suite summary
shape emitted by `test-*-e2e.js` (`N passed, M failed` / `passed N
failed M` / `N/T tests passed` / `N/T PASS` / `<file>.js: PASS|FAIL`)
and sums them. Per-test progress lines (`✓`, `PASS:`) are skipped so
they can't double-count.
- `deploy.yml` sources the aggregator, sets the badge to `"X passed"`
(brightgreen) when `FAIL=0` and `"X passed, Y failed"` (red) otherwise.
Badge schema (`schemaVersion / label / message / color`) unchanged.

## TDD
- **RED** 33d789c4f3: adds
`test-e2e-badge-aggregate.sh` + vendored fixture
`test-fixtures/e2e-output-sample.txt` (45 suites of realistic output).
Aggregator stub returns zeros → test fails on assertion (`PASS=108
FAIL=0` expected, `PASS=0 FAIL=0` got).
- **GREEN** b43bd70f43: real aggregator
implementation → all five sub-tests pass (fixture aggregate,
broken-regex sanity, synthetic mixed pass/fail, per-test-progress-line
guard, missing-file fallback).

No force-push. PII preflight clean.

---------

Co-authored-by: openclaw-bot <bot@openclaw.local>
This commit is contained in:
Kpa-clawbot
2026-05-19 22:40:10 -07:00
committed by GitHub
parent 4525c87963
commit e267fb754d
4 changed files with 431 additions and 2 deletions
+15 -2
View File
@@ -279,7 +279,13 @@ jobs:
- name: Generate frontend coverage badges
if: success()
run: |
E2E_PASS=$(grep -oP '[0-9]+(?=/)' e2e-output.txt | tail -1 || echo "0")
# Aggregate per-suite PASS/FAIL across every test-*-e2e.js summary.
# The previous regex (grep -oP '[0-9]+(?=/)' | tail -1) caught a
# stray digits-before-slash like the '2' in '2/3 tests passed' from
# some sub-output and stamped the badge as '2 passed'. See #1296.
eval "$(bash scripts/aggregate-e2e-pass.sh e2e-output.txt)"
E2E_PASS=${PASS:-0}
E2E_FAIL=${FAIL:-0}
mkdir -p .badges
if [ -f .nyc_output/frontend-coverage.json ] || [ -f .nyc_output/e2e-coverage.json ]; then
@@ -292,7 +298,14 @@ jobs:
echo "{\"schemaVersion\":1,\"label\":\"frontend coverage\",\"message\":\"${FE_COVERAGE}%\",\"color\":\"${FE_COLOR}\"}" > .badges/frontend-coverage.json
echo "## Frontend: ${FE_COVERAGE}% coverage" >> $GITHUB_STEP_SUMMARY
fi
echo "{\"schemaVersion\":1,\"label\":\"e2e tests\",\"message\":\"${E2E_PASS:-0} passed\",\"color\":\"brightgreen\"}" > .badges/e2e-tests.json
if [ "${E2E_FAIL:-0}" -gt 0 ]; then
E2E_MSG="${E2E_PASS:-0} passed, ${E2E_FAIL} failed"
E2E_COLOR="red"
else
E2E_MSG="${E2E_PASS:-0} passed"
E2E_COLOR="brightgreen"
fi
echo "{\"schemaVersion\":1,\"label\":\"e2e tests\",\"message\":\"${E2E_MSG}\",\"color\":\"${E2E_COLOR}\"}" > .badges/e2e-tests.json
- name: Stop test server
if: always()