mirror of
https://github.com/Kpa-clawbot/meshcore-analyzer.git
synced 2026-05-24 22:55:20 +00:00
Kpa-clawbot
e267fb754d
RED33d789c4f3(test) → GREENb43bd70f43(fix). CI: https://github.com/Kpa-clawbot/CoreScope/actions/workflows/deploy.yml?query=branch%3Afix%2Fe2e-badge-aggregate Fixes #1296 ## Problem `.github/workflows/deploy.yml` was computing the e2e-tests badge with: ``` E2E_PASS=$(grep -oP '[0-9]+(?=/)' e2e-output.txt | tail -1 || echo "0") ``` This regex matched any digit-run immediately followed by `/` anywhere in the combined output of 45+ Playwright suites, then took the **last** match. The result was usually a small number scraped out of intermediate per-suite progress text (often `2` from something like `2/3 …`), so the badge perpetually showed `{"label":"e2e tests","message":"2 passed","color":"brightgreen"}` regardless of how many tests actually ran. ## Fix - New `scripts/aggregate-e2e-pass.sh` parses every per-suite summary shape emitted by `test-*-e2e.js` (`N passed, M failed` / `passed N failed M` / `N/T tests passed` / `N/T PASS` / `<file>.js: PASS|FAIL`) and sums them. Per-test progress lines (`✓`, `PASS:`) are skipped so they can't double-count. - `deploy.yml` sources the aggregator, sets the badge to `"X passed"` (brightgreen) when `FAIL=0` and `"X passed, Y failed"` (red) otherwise. Badge schema (`schemaVersion / label / message / color`) unchanged. ## TDD - **RED**33d789c4f3: adds `test-e2e-badge-aggregate.sh` + vendored fixture `test-fixtures/e2e-output-sample.txt` (45 suites of realistic output). Aggregator stub returns zeros → test fails on assertion (`PASS=108 FAIL=0` expected, `PASS=0 FAIL=0` got). - **GREEN**b43bd70f43: real aggregator implementation → all five sub-tests pass (fixture aggregate, broken-regex sanity, synthetic mixed pass/fail, per-test-progress-line guard, missing-file fallback). No force-push. PII preflight clean. --------- Co-authored-by: openclaw-bot <bot@openclaw.local>
99 lines
3.1 KiB
Bash
Executable File
99 lines
3.1 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Test for scripts/aggregate-e2e-pass.sh — verifies aggregate across 45+
|
|
# Playwright suites in test-fixtures/e2e-output-sample.txt is correct, not the
|
|
# broken old behavior of "grep digits-before-slash | tail -1" (which returned 2).
|
|
#
|
|
# Regression for #1296.
|
|
set -u
|
|
script_dir=$(cd "$(dirname "$0")" && pwd)
|
|
aggregator="$script_dir/scripts/aggregate-e2e-pass.sh"
|
|
fixture="$script_dir/test-fixtures/e2e-output-sample.txt"
|
|
|
|
if [ ! -x "$aggregator" ]; then
|
|
chmod +x "$aggregator"
|
|
fi
|
|
|
|
# --- Test 1: fixture aggregate ---------------------------------------------
|
|
out=$("$aggregator" "$fixture")
|
|
# Count expected pass: sum N for every per-suite summary in the fixture.
|
|
# Computed by hand from the fixture (45 suites, see file).
|
|
EXPECTED_PASS=108
|
|
EXPECTED_FAIL=0
|
|
EXPECTED="PASS=$EXPECTED_PASS FAIL=$EXPECTED_FAIL"
|
|
|
|
if [ "$out" != "$EXPECTED" ]; then
|
|
echo "FAIL: fixture aggregate"
|
|
echo " expected: $EXPECTED"
|
|
echo " got: $out"
|
|
exit 1
|
|
fi
|
|
echo "PASS: fixture aggregates to $out"
|
|
|
|
# --- Test 2: the broken old regex would have returned something tiny -------
|
|
# (sanity check that we are NOT just reproducing the bug). Requires grep -P
|
|
# (PCRE), which is available in GitHub-hosted Ubuntu runners and most Linux
|
|
# distros but not in BusyBox; skip gracefully if absent.
|
|
if echo "x1/2" | grep -qoP '[0-9]+(?=/)' 2>/dev/null; then
|
|
old=$(grep -oP '[0-9]+(?=/)' "$fixture" | tail -1)
|
|
if [ "$old" = "$EXPECTED_PASS" ]; then
|
|
echo "FAIL: old broken regex coincidentally matches expected — fixture is not discriminating"
|
|
exit 1
|
|
fi
|
|
echo "PASS: old broken regex returned '$old' (NOT $EXPECTED_PASS) — fixture proves the bug"
|
|
else
|
|
echo "SKIP: grep -P unavailable, cannot verify old broken regex sanity"
|
|
fi
|
|
|
|
# --- Test 3: synthetic with failures, ensures FAIL accounting --------------
|
|
tmp=$(mktemp)
|
|
cat > "$tmp" <<'EOF'
|
|
=== Results: 4 passed, 1 failed ===
|
|
=== Results: 2 passed, 0 failed ===
|
|
test-foo.js: 3/5 passed
|
|
test-bar.js: PASS
|
|
test-baz.js: FAIL — boom
|
|
passed 7 failed 2
|
|
EOF
|
|
out2=$("$aggregator" "$tmp")
|
|
rm -f "$tmp"
|
|
EXP2="PASS=17 FAIL=4"
|
|
if [ "$out2" != "$EXP2" ]; then
|
|
echo "FAIL: synthetic mixed pass/fail"
|
|
echo " expected: $EXP2"
|
|
echo " got: $out2"
|
|
exit 1
|
|
fi
|
|
echo "PASS: synthetic mixed pass/fail aggregates to $out2"
|
|
|
|
# --- Test 4: per-test progress lines must NOT be counted -------------------
|
|
tmp=$(mktemp)
|
|
cat > "$tmp" <<'EOF'
|
|
✓ test alpha
|
|
✓ test beta
|
|
✗ test gamma failed
|
|
PASS: detail line
|
|
FAIL: detail line
|
|
=== Results: 2 passed, 1 failed ===
|
|
EOF
|
|
out3=$("$aggregator" "$tmp")
|
|
rm -f "$tmp"
|
|
EXP3="PASS=2 FAIL=1"
|
|
if [ "$out3" != "$EXP3" ]; then
|
|
echo "FAIL: per-test progress double-count"
|
|
echo " expected: $EXP3"
|
|
echo " got: $out3"
|
|
exit 1
|
|
fi
|
|
echo "PASS: per-test progress lines correctly ignored ($out3)"
|
|
|
|
# --- Test 5: empty / missing file ------------------------------------------
|
|
out4=$("$aggregator" /nonexistent/path/nope.txt)
|
|
if [ "$out4" != "PASS=0 FAIL=0" ]; then
|
|
echo "FAIL: missing file should yield PASS=0 FAIL=0, got $out4"
|
|
exit 1
|
|
fi
|
|
echo "PASS: missing file → PASS=0 FAIL=0"
|
|
|
|
echo
|
|
echo "ALL TESTS PASS"
|