Three optimizations to the CI frontend test pipeline:
1. Run E2E tests and coverage collection concurrently
- Previously sequential (E2E ~1.5min, then coverage ~5.75min)
- Now both run in parallel against the same instrumented server
- Expected savings: ~5 min (coverage runs alongside E2E instead of after)
2. Replace networkidle with domcontentloaded in coverage collector
- SPA uses hash routing — networkidle waits 500ms for network silence
on every navigation, adding ~10-15s of dead time across 23 navigations
- domcontentloaded fires immediately once HTML is parsed; JS initializes
the route handler synchronously
- For in-page hash changes, use 200ms setTimeout instead of
waitForLoadState (which would never re-fire for same-document nav)
3. Extract coverage from E2E tests too
- E2E tests already exercise the app against the instrumented server
- Now writes window.__coverage__ to .nyc_output/e2e-coverage.json
- nyc merges both coverage files for higher total coverage
Also:
- Split Playwright install into browser + deps steps (deps skip if present)
- Replace sleep 5 with health-check poll in quick E2E path
Remove all 169 waitForTimeout() calls (totaling 104.1s of blind sleeping)
from scripts/collect-frontend-coverage.js:
- Helper functions (safeClick, safeFill, safeSelect, clickAll, cycleSelect):
removed 300-400ms waits after every interaction — Playwright's built-in
actionability checks handle waiting for elements automatically
- Post-navigation waits: removed redundant sleeps after page.goto() calls
that already use waitUntil: 'networkidle'
- Hash-change navigations: replaced waitForTimeout with
waitForLoadState('networkidle') for proper SPA route settling
- Toggle/button waits: removed — event handlers execute synchronously
before click() resolves
- Post-evaluate waits: removed — evaluate() is synchronous
Local benchmark (Windows, sparse test data):
Before: 744.8s
After: 484.8s (35% faster, 260s saved)
On CI runner (ARM Linux with real mesh data), savings will be
proportionally better since most elements exist and the 104s
of blind sleeping was the dominant bottleneck.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The page.evaluate() calls corrupting localStorage and firing fake events
caused page error-reloads, losing accumulated coverage. Reverting to
the 42% version which was the actual high water mark.
Exercise every major code path across all frontend files:
app.js: all routes, bad routes, hashchange, theme toggle x4,
hamburger menu, favorites dropdown, global search, Ctrl+K,
apiPerf(), timeAgo/truncate/routeTypeName utils
nodes.js: sort every column (both directions), every role tab,
every status filter, cycle all Last Heard options, click rows
for side pane, navigate to detail page, copy URL, show all
paths, node analytics day buttons (1/7/30/365), scroll target
packets.js: 12 filter expressions including bad ones, cycle all
time windows, group by hash toggle, My Nodes toggle, observer
menu, type filter menu, hash input, node filter, observer sort,
column toggle menu, hex hash toggle, pause button, resize handle,
deep-link to packet hash
map.js: all role checkboxes toggle, clusters/heatmap/neighbors/
hash labels toggles, cycle Last Heard, status filter buttons,
jump buttons, markers, zoom controls, dark mode tile swap
analytics.js: all 9 tabs clicked, deep-link to each tab via URL,
observer selector on topology, navigate rows on collisions/
subpaths, sortable headers on nodes tab, region filter
customize.js: all 5 tabs, all preset themes, branding text inputs,
theme color inputs, node color inputs, type color inputs, reset
buttons, home tab fields (hero, journey steps, checklist, links),
export tab, reset preview/user theme
live.js: VCR pause/speed/missed/prompt buttons, all visualization
toggles (heat/ghost/realistic/favorites/matrix/rain), audio
toggle + BPM slider, timeline click, resize event
channels.js: click rows, navigate to specific channel
observers.js: click rows, navigate to detail, cycle days select
traces.js: click rows
perf.js: refresh + reset buttons
home.js: both chooser paths, search + suggest, my-node cards,
health/packets buttons, remove buttons, toggle level, timeline
Also exercises packet-filter parser and region-filter directly.
- Install nyc for Istanbul instrumentation
- Add scripts/instrument-frontend.sh to instrument public/*.js
- Add scripts/collect-frontend-coverage.js to extract window.__coverage__
- Add scripts/combined-coverage.sh for combined server+frontend coverage
- Make server.js serve public-instrumented/ when COVERAGE=1 is set
- Add test:full-coverage npm script
- Add public-instrumented/ and .nyc_output/ to .gitignore
Creates transmissions and observations tables from existing packets table.
- Groups packets by hash → 1 transmission per unique hash
- Creates 1 observation per original packet row with FK to transmission
- Idempotent: drops and recreates new tables on each run
- Does NOT modify the original packets table
- Prints stats and verifies counts match
Tested on test DB: 33813 packets → 11530 transmissions (2.93x dedup ratio)
