mirror of
https://github.com/Kpa-clawbot/meshcore-analyzer.git
synced 2026-10-15 12:29:52 +00:00
Fixes #2203. ## Problem The frontend coverage badge (35.16% on run 38413268001) came from two sources on shard 2: one dump part-way through `test-e2e-playwright.js` and the `collect-frontend-coverage.js` tour. The other E2E suites run against the instrumented build but never save `window.__coverage__`, including the suites written for single pages (`path-inspector.js` 4.72%, `touch-gestures.js` 14% on that run). ## Change 1. `9d3bfad5` `test(e2e): red:` `tests/unit/test-e2e-coverage.js`, with fake pages, contexts and browsers (no Chromium needed). On the placeholder module the first assertion fails. 2. `c00bbb12` `feat(e2e):` the collector. - `scripts/e2e-coverage-hook.js` is a preload (`node --require`) that wraps `chromium.launch` from `playwright`. `playwright` re-exports `playwright-core`, so a suite's own `const { chromium } = require('playwright')` gets the same, wrapped object. - `scripts/e2e-coverage.js` reads `window.__coverage__` before every call that replaces or ends a page's document: `goto`, `reload`, `setContent`, `goBack`, `goForward`, `close`, and context and browser `close`. Pages from `newPage`, `newContext` and the context `page` event are all tracked. - Each read zeroes the page's counters, so a document read twice is not counted twice. Maps travel once per file; later reads carry counts only. - The page returns a JSON string. Returning the object cost about 10 s per suite, because Playwright serialises it node by node. - A read is given up after 5 s, so a busy page cannot hold up a suite's cleanup. - On exit the hook writes one file per suite to `.nyc_output/`, or nothing without coverage. A failed write is logged and never changes the exit code. 3. `64487c4c` `ci:` the workflow. - On master pushes the E2E run step sets `NODE_OPTIONS=--require ./scripts/e2e-coverage-hook.js`. On PRs it is empty, so PR runs are unchanged. - Each shard runs `nyc merge` (one file per suite is about 6 MB, mostly the same maps) and uploads `coverage-shard.json` with its results instead of `.badges/`. - The `e2e-test` job merges the three shards and makes the frontend badge (it used to come from shard 2 alone). - The merge, setup-node and badge steps are `continue-on-error`. When `nyc report` gives no statement percentage the step warns and writes no badge, so a registry hiccup cannot publish 0%. - `test-e2e-playwright.js` no longer writes its own `e2e-coverage.json`: the hook covers that suite completely, including the tests after the old dump. `collect-frontend-coverage.js` stays as an extra source. ## Performance Measured locally against an instrumented server (Windows, Chrome): | Suite | Without hook | With hook | |---|---|---| | `test-path-inspector-coverage-e2e.js` | 1.9 s | 2.2 s | | `test-touch-gestures-coverage-e2e.js` | 17 s | 16 s | | `test-e2e-playwright.js` (stops at the same local-fixture failure in both runs) | 52 s | 52 s | | `browser.close()` on a page in a 30 s busy loop | 153 ms | 5.2 s | The hook only loads on master pushes. ## Testing - `tests/unit/test-e2e-coverage.js` covers: - reads before each document-replacing call and before close; - pages opened by the page itself; - context and browser close; - summing without double counts; - a failed read, and a read that never answers; - the file writer, including a failing write. - Nine deliberate breaks of the library each turn it red: - dropping the counter reset; - dropping the `page` event, the `goto` read, the `goForward` read, the `close` read or the browser `close` read; - removing the timeout or the write guard, or writing an empty store. - The test also fails if it stops before its end. Without that check, a read that never returns would let node exit 0 half-way. - Real suites with the hook, measured on their own (the badge run had all suites): | File | Badge run 38413268001 | Own suite with the hook | |---|---|---| | `path-inspector.js` | 4.72% | 52.75% | | `touch-gestures.js` | 14% | 72.96% | - Exit codes and pass counts were unchanged with the hook for these suites (the first three were run before the 5 s read timeout was added): - `test-slideover-1056-e2e.js` (6 contexts); - `test-a11y-axe-1668.js` (`@axe-core/playwright`); - `test-issue-1150-404-state-e2e.js`; - the three above. - The badge step, run locally against two merged shard files: - It gives the same per-file numbers as `nyc report` on the inputs. - With no shard coverage, or with `nyc report` failing, it warns and writes no badge. - `sh test-all.sh` passes locally, apart from `test-issue-1956-release-routing.js`, which needs the workflow's `GITHUB_REF_NAME` and was skipped. The test inventory classifies 321 suites. `actionlint` 1.7.7 reports only the self-hosted runner label, as on master. **Not tested:** the shard and badge steps in CI. They only run on a master push, so the first master run after merge is the real test. I do not know what the merged number will be. I will compare the badge with the merged report in that run's `e2e-test` log. **Not counted:** a full page load the suite does not start through one of the wrapped calls, such as a link that leaves the SPA or `location.reload()` from page code. ## Not in this PR - The `tests/unit/` suites, which load frontend files through `vm.createContext` without instrumentation (noted in #2203). - Coverage on PR runs. The overhead above is small, so this could follow if wanted. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
155 lines
5.3 KiB
JavaScript
155 lines
5.3 KiB
JavaScript
'use strict';
|
|
// Collects window.__coverage__ from every page a Playwright E2E suite opens
|
|
// (#2203). Loaded through scripts/e2e-coverage-hook.js; kept free of
|
|
// Playwright and fs so tests/unit/test-e2e-coverage.js can drive it with fakes.
|
|
//
|
|
// A page's counters live until its document is replaced, so they are read
|
|
// (and zeroed) before every call that can replace or end the document. A
|
|
// full load the suite does not start through one of these calls, such as a
|
|
// click on a link that leaves the SPA, is not counted.
|
|
|
|
const PAGE_CALLS = ['goto', 'reload', 'setContent', 'goBack', 'goForward', 'close'];
|
|
|
|
// A read must never hold up the call it runs before: a page stuck in a busy
|
|
// loop would otherwise keep browser.close() in a suite's cleanup waiting for
|
|
// as long as the loop runs. Past this, the read is given up.
|
|
const READ_TIMEOUT_MS = 5000;
|
|
|
|
// Runs inside the page. Returns the hit counts since the last read and zeroes
|
|
// them, so a document read twice is not counted twice. A file the caller has
|
|
// not seen yet comes back whole (with its statement, function and branch maps).
|
|
// The result is a JSON string: Playwright serialises a returned object node by
|
|
// node, which took about 10 s for the roughly 6 MB of maps of the first read.
|
|
function readAndReset(known) {
|
|
const cov = window.__coverage__;
|
|
if (!cov) return null;
|
|
const seen = new Set(known);
|
|
const out = {};
|
|
for (const file of Object.keys(cov)) {
|
|
const fc = cov[file];
|
|
out[file] = seen.has(file) ? { s: fc.s, f: fc.f, b: fc.b } : fc;
|
|
}
|
|
const json = JSON.stringify(out);
|
|
for (const file of Object.keys(cov)) {
|
|
const fc = cov[file];
|
|
for (const k in fc.s) fc.s[k] = 0;
|
|
for (const k in fc.f) fc.f[k] = 0;
|
|
for (const k in fc.b) fc.b[k].fill(0);
|
|
}
|
|
return json;
|
|
}
|
|
|
|
// Adds a snapshot from readAndReset to store, summing the hit counts.
|
|
function mergeInto(store, snapshot) {
|
|
for (const file of Object.keys(snapshot)) {
|
|
const add = snapshot[file];
|
|
const into = store[file];
|
|
if (!into) {
|
|
store[file] = add;
|
|
continue;
|
|
}
|
|
for (const k in add.s) into.s[k] = (into.s[k] || 0) + add.s[k];
|
|
for (const k in add.f) into.f[k] = (into.f[k] || 0) + add.f[k];
|
|
for (const k in add.b) {
|
|
const sum = into.b[k] || add.b[k].map(() => 0);
|
|
add.b[k].forEach((n, i) => { sum[i] = (sum[i] || 0) + n; });
|
|
into.b[k] = sum;
|
|
}
|
|
}
|
|
}
|
|
|
|
function createCollector({ readTimeoutMs = READ_TIMEOUT_MS } = {}) {
|
|
const store = {};
|
|
const pages = new WeakSet();
|
|
const contexts = new WeakSet();
|
|
|
|
async function collect(page) {
|
|
if (page.isClosed()) return;
|
|
let timer;
|
|
const timeout = new Promise(resolve => { timer = setTimeout(resolve, readTimeoutMs, null); });
|
|
let snapshot;
|
|
try {
|
|
snapshot = await Promise.race([page.evaluate(readAndReset, Object.keys(store)), timeout]);
|
|
} catch {
|
|
return; // page crashed or navigated away mid-read: nothing to add
|
|
} finally {
|
|
clearTimeout(timer);
|
|
}
|
|
if (snapshot) mergeInto(store, JSON.parse(snapshot));
|
|
}
|
|
|
|
function readFirst(obj, names, read) {
|
|
for (const name of names) {
|
|
const orig = obj[name];
|
|
if (typeof orig !== 'function') continue;
|
|
obj[name] = async function (...args) {
|
|
await read();
|
|
return orig.apply(this, args);
|
|
};
|
|
}
|
|
}
|
|
|
|
function trackPage(page) {
|
|
if (pages.has(page)) return page;
|
|
pages.add(page);
|
|
readFirst(page, PAGE_CALLS, () => collect(page));
|
|
return page;
|
|
}
|
|
|
|
function trackContext(context) {
|
|
if (contexts.has(context)) return context;
|
|
contexts.add(context);
|
|
context.pages().forEach(trackPage);
|
|
context.on('page', trackPage);
|
|
readFirst(context, ['close'], () => Promise.all(context.pages().map(collect)));
|
|
const newPage = context.newPage;
|
|
context.newPage = async function (...args) {
|
|
return trackPage(await newPage.apply(this, args));
|
|
};
|
|
return context;
|
|
}
|
|
|
|
function trackBrowser(browser) {
|
|
const newContext = browser.newContext;
|
|
browser.newContext = async function (...args) {
|
|
return trackContext(await newContext.apply(this, args));
|
|
};
|
|
const newPage = browser.newPage;
|
|
browser.newPage = async function (...args) {
|
|
const page = await newPage.apply(this, args);
|
|
trackContext(page.context());
|
|
return trackPage(page);
|
|
};
|
|
readFirst(browser, ['close'], () =>
|
|
Promise.all(browser.contexts().flatMap(c => c.pages()).map(collect)));
|
|
return browser;
|
|
}
|
|
|
|
function wrapLaunch(browserType) {
|
|
const launch = browserType.launch;
|
|
browserType.launch = async function (...args) {
|
|
return trackBrowser(await launch.apply(this, args));
|
|
};
|
|
}
|
|
|
|
return { store, collect, trackPage, trackContext, trackBrowser, wrapLaunch };
|
|
}
|
|
|
|
// Writes store as one nyc input file, or nothing when it is empty. Never
|
|
// throws: it runs in an exit handler, where a throw would change the suite's
|
|
// exit code. Returns the file written, or null.
|
|
function writeCoverage(fs, path, store, outDir, suite, pid) {
|
|
if (Object.keys(store).length === 0) return null;
|
|
try {
|
|
fs.mkdirSync(outDir, { recursive: true });
|
|
const file = path.join(outDir, `e2e-${suite}-${pid}.json`);
|
|
fs.writeFileSync(file, JSON.stringify(store));
|
|
return file;
|
|
} catch (e) {
|
|
console.error(`e2e-coverage: not written: ${e.message}`);
|
|
return null;
|
|
}
|
|
}
|
|
|
|
module.exports = { readAndReset, mergeInto, createCollector, writeCoverage };
|