Commit Graph

24 Commits

Author SHA1 Message Date
Kpa-clawbot eae1b915ca feat: load Aldrich webfont for #1137 logo SVG (#1138)
Adds Aldrich webfont so the merged #1137 logo renders in the intended
typeface.

## Problem
The inline SVG logo merged in #1137 declares `font-family="Aldrich,
monospace"` in `public/index.html` and `public/home.js`, but the page
never loaded the Aldrich font face. Browsers silently fell back to
monospace.

## Fix
Self-hosted webfont:
- `public/fonts/aldrich-regular.woff2` — Regular 400, ~16KB, downloaded
from Google Fonts (latin subset). Self-hosted to avoid third-party CDN
dependency, privacy concern, and FOUT delay.
- `@font-face` declaration added at the top of `public/style.css` with
`font-display: swap`.

Aldrich only ships in 400; the SVG `font-weight="700"` on the wordmark
synthesizes bold (matches the design intent of #1137).

## TDD
- Red commit: E2E test asserting `document.fonts.check('1em Aldrich')`
is true and the navbar SVG `<text>` `font-family` contains "Aldrich".
Without the font face declaration, both assertions fail on an assertion
(not a build error).
- Green commit: adds the woff2 + `@font-face` rule, both assertions
pass.

## Files
- `public/fonts/aldrich-regular.woff2` (new, 16460 bytes)
- `public/style.css` — `@font-face` rule
- `test-e2e-playwright.js` — new test

---------

Co-authored-by: openclaw-bot <bot@openclaw.local>
2026-05-06 21:24:57 -07:00
Kpa-clawbot 494d3022f9 Partial fix for #1128: close audit gaps (z-scale, css-var lint, multi-viewport E2E, Bug 1+5 polish) (#1133)
## Partial fix for #1128 — closes the gaps PR #1131 left behind

PR #1131 was a partial fix for the packets-page layout chaos
(merged 2026-05-06 ~01:55 UTC, then the issue was reopened by the
maintainer). #1131 shipped Bug 4 (`--surface` definition), the
`.path-popover` flip + lower z-index, the debounced re-measure for
Bug 1, the `.filter-bar` row-gap + `.multi-select-trigger`
truncation for Bug 3, the new z-index TOKENS, and a single-viewport
E2E with five individual-component assertions.

This PR closes everything else the issue body and the
`specs/packets-layout-audit.md` audit asked for.

### What changed (per gap)

**Gap A — apply the z-index scale (audit Section 2)**
#1131 added `--z-dropdown` / `--z-popover` / `--z-modal` /
`--z-tooltip` but explicitly left existing literal values in place.
This PR renumbers the 7 dropdowns/popovers the audit named:

| Selector | Before | After |
|---|---:|---:|
| `.col-toggle-menu` | 50 | `var(--z-dropdown)` (100) |
| `.multi-select-menu` | 90 | `var(--z-dropdown)` |
| `.region-dropdown-menu` | 90 | `var(--z-dropdown)` |
| `.node-filter-dropdown` | 100 | `var(--z-dropdown)` |
| `.fux-saved-menu` | `var(--z-tooltip)` (9200) | `var(--z-dropdown)` |
| `.fux-ac-dropdown` | `var(--z-tooltip)` | `var(--z-dropdown)` |
| `.hop-conflict-popover` | `var(--z-tooltip)` | `var(--z-popover)`
(300) |

`.fux-ctx-menu` deliberately retains the tooltip band — context
menus must float above all toolbar UI. `.region-filter-options-menu`
no longer exists in the source (was renamed
`.region-dropdown-menu`).

The `style.css` doc-block at the top is rewritten to record the
applied scale and to point operators at the new lint.

**Gap B — CSS-var lint (audit Section 5 #1, "single highest-value
addition")**
Adds `scripts/check-css-vars.js` (~70 lines). Walks
`public/*.css`, extracts every `var(--name)` reference WITHOUT a
fallback, asserts the name is defined in some `public/*.css`.
References WITH a fallback are tolerated. Wired into CI in the
`go-test` job before the JS unit tests.

The red commit (`608d81f`) shipped this lint exiting 1 against the
master tree — three undefined vars that bypassed earlier review:

```
public/style.css:2628  var(--text-primary)
public/style.css:2675  var(--bg-hover)
public/style.css:2924  var(--primary)
```

The green commit (`1369d1e`) defines those three as aliases in the
:root block (`--text-primary` → `--text`, `--bg-hover` →
`--hover-bg`, `--primary` → `--accent`). Light + dark themes
inherit through the existing tokens.

**Gap C — multi-viewport E2E (issue acceptance criterion)**
Adds `test-issue-1128-multi-viewport-e2e.js` — sister of the
existing single-viewport test. At each of three viewports
(1280×900, 1080×800, 768×1024):

  - takes a screenshot to `e2e-screenshots/issue-1128-<viewport>.png`
  - asserts no two `.filter-group` siblings vertically overlap
  - on desktop+laptop, opens the Saved menu and the Types
    multi-select and asserts the dropdown does not vertically
    overlap any `.filter-group` below it

Plus three viewport-agnostic assertions:

  - dropdown selectors compute z-index in `[100,199]`
    (`.col-toggle-menu`, `.multi-select-menu`,
    `.region-filter-options-menu`, `.fux-saved-menu`,
    `.fux-ac-dropdown`)
  - `.path-hops .hop / .hop-named / .arrow` compute
    `line-height ≤ 18px`
  - `.col-path` computes `height ≤ 28px`

Wired into the e2e-test job after the existing #1128 test.

**Gap D — Bug 5 polish (toolbar reorder)**
Audit Section 3 Bug 5: swaps `filter-group-dropdowns` and
`filter-group-toggles` in `public/packets.js` so time range +
Group by Hash + ★ My Nodes sit next to the search input. Pure
markup reorder. No CSS / no JS-handler changes.

**Gap E — Bug 1 belt-and-suspenders**
Audit Section 3 Bug 1 sub-bullets:

  - locks `.path-hops .hop / .hop-named / .arrow` to
    `line-height: 18px` so a chip with mixed font metrics cannot
    overflow the 22px host vertically and bleed into the row above
  - converts `.col-path { max-height: 28px }` → `height: 28px`
    because browsers widely ignore `max-height` on `<td>`s; the
    earlier rule was a no-op

### TDD discipline (red → green)

```
$ git log --oneline origin/master..HEAD
68b0426 fix(#1128): Bug 5 — toolbar group reorder (toggles before dropdowns)
6d16e6f fix(#1128): apply z-index scale to dropdowns + Bug 1 chip line-height lock
b9850c9 fix(check-css-vars): strip /* ... */ comments before scanning
1369d1e fix(#1128): define --text-primary, --bg-hover, --primary aliases (lint green)
0d4660f test(#1128): multi-viewport E2E + wire CSS-var lint into CI (red commit)
608d81f test(#1128): add scripts/check-css-vars.js — fails on 3 undefined vars (red commit)
```

Both red commits (`608d81f`, `0d4660f`) were verified to fail
locally before the green commits landed:

  - `608d81f` runs the lint and exits 1 on the three undefined vars
    listed above (proven against master).
  - `0d4660f` introduces the multi-viewport E2E and wires the lint
    into CI — the lint then fails the build on master, and the E2E
    z-scale assertion fails because pre-fix `.col-toggle-menu` is
    50, the multi-selects are 90, etc.

### Acceptance criteria status

From the original issue body:
  -  Bug 4 root cause fixed (#1131 + this PR's lint guard)
  -  Bug 1 chip-spill (debounced re-measure from #1131 +
       line-height lock + col-path height fix from this PR)
  -  Bug 2 +N popover positioning (#1131)
  -  Bug 3 toolbar overlap (#1131 + #1131 row-gap)
  -  Bug 5 group reorder (this PR)
  -  Z-index scale documented + applied (this PR)
  -  E2E screenshots at multiple viewports (this PR)
  -  Bounding-rect collision detection on visible interactive
       elements (this PR — `.filter-group` siblings + dropdown vs.
       toolbar)
  -  CSS-var lint in CI (this PR)

### Why this is "Partial fix for #1128", not "Fixes #1128"

Per `AGENTS.md` rule 34, automated closure is reserved for the
operator after they verify on staging. The acceptance criteria
above appear satisfied in code, but the user should confirm the
visual outcome on staging before closing.

### Files changed

- `scripts/check-css-vars.js` (new — ~70 lines)
- `test-issue-1128-multi-viewport-e2e.js` (new)
- `.github/workflows/deploy.yml` (lint step + e2e step wiring)
- `public/style.css` (z-renumber, doc-block, Bug 1 polish, alias defs)
- `public/packets.js` (Bug 5 reorder)

Refs #1128, follows #1131

---------

Co-authored-by: Kpa-clawbot <bot@kpa-clawbot.local>
Co-authored-by: openclaw-bot <bot@openclaw.local>
2026-05-07 03:41:12 +00:00
Kpa-clawbot 364c5766fc feat(logo): wire new CoreScope SVG logo into navbar + home hero (#1137)
## Adds new logo and home hero

Replaces the navbar mushroom emoji + "CoreScope" text spans with the new
CoreScope SVG mark, and adds a hero SVG (with the MESH ANALYZER tagline)
above the home page H1.

### What changed
- `public/img/corescope-logo.svg` — navbar mark, no tagline (locked
"aggressive low-amp chirp" variant: facing-arcs + low-amp chirp
connector between the two nodes).
- `public/img/corescope-hero.svg` — home hero version, includes the MESH
ANALYZER tagline.
- `public/index.html` — replaces `<span class="brand-icon">🍄</span><span
class="brand-text">CoreScope</span>` with `<img class="brand-logo"
src="img/corescope-logo.svg?__BUST__" …>`. `.nav-brand` link still
routes to `#/`. `.live-dot` retained.
- `public/style.css` — adds `.brand-logo { height: 36px }` (32px on
tablet ≤900px). Existing 52px nav height unchanged.
- `public/home.js` / `public/home.css` — adds `<img
class="home-hero-logo">` above the hero `<h1>`, sized `max-width:
min(720px, 90vw)` and centered.

### TDD
Red→green is visible in the branch:
- `3159b82` — `test(logo): add failing E2E …` (red commit). Adds
`test-logo-rebrand-e2e.js` and wires it into the `e2e-test` job in
`deploy.yml` with `CHROMIUM_REQUIRE=1`. On this commit `index.html`
still has the emoji + text spans, `home.js` has no hero img, and the SVG
asset files do not exist — the test asserts on each so CI fails on
assertion.
- `19434e1` — `feat(logo): wire new CoreScope SVG logo …` (green
commit). Implements the fix.

### E2E asserts
1. `.nav-brand img` exists with `src` ending `corescope-logo.svg`
2. legacy `.brand-icon` / `.brand-text` are gone
3. `.live-dot` is present, visible, and to the right of the logo (no
overlap)
4. `.home-hero img.home-hero-logo` exists with `src` ending
`corescope-hero.svg`, positioned BEFORE the `<h1>`
5. both `/img/corescope-{logo,hero}.svg` return 200 with svg
content-type

### Customizer compatibility
- `customize.js` still does `querySelector('.brand-text')` /
`.brand-icon` for live branding updates. Both now return `null`;
existing `if (el)` guards make those branches silent no-ops. **No JS
errors, but the customizer's `branding.siteName` and `branding.logoUrl`
fields no longer rewrite the navbar brand** — the brand is now a fixed
SVG asset.
- **Theme accent does NOT recolor the SVG.** SVGs loaded via `<img src>`
are isolated documents and cannot inherit document CSS variables; the
SVG falls back to its embedded brand colors. This is appropriate for a
brand mark; if recoloring per theme is desired later, swap to inline SVG
(separate PR).

### Browser validation
Local Chromium not available in this env; the E2E test soft-skips
locally and hard-fails in CI (`CHROMIUM_REQUIRE=1`). Server-side checks
done locally:
- `curl http://localhost:13581/` → confirmed `<img class="brand-logo"
src="img/corescope-logo.svg?<bust>" …>` rendered, no
`.brand-icon`/`.brand-text` spans.
- `curl -I /img/corescope-logo.svg` and `/img/corescope-hero.svg` → both
200.

### Performance
No hot-path changes. Two new static SVG assets (~7.6KB each), served
directly by the Go static handler. Cache-busted via `?__BUST__`
(auto-replaced server-side).

---------

Co-authored-by: OpenClaw Bot <bot@openclaw.local>
Co-authored-by: Kpa-clawbot <bot@kpa-clawbot.local>
2026-05-06 19:17:46 -07:00
Kpa-clawbot ac0cf5ac7d fix(channels): #1087 QR library + share modal + PSK persistence (#1090)
Red commit: 5def4d073c (CI run pending —
see Checks tab)

Fixes #1087

## What's broken (4 bugs)
1. **"QR library not loaded"** — `channel-qr.js` checked `root.QRCode`
(capital), but the vendored library exports lowercase `qrcode` (Kazuhiko
Arase API). Generate & Show QR always fell into the "library not loaded"
branch.
2. **QR encodes `name=psk:hex`** — the Share button (and parts of the
Generate path) passed the internal `psk:<hex8>` lookup key to
`ChannelQR.generate`, ignoring the user's display label stored in
`LABELS_KEY`.
3. **PSK channel doesn't persist on refresh** — the persistence path was
scattered, and the read-back wasn't verified. Added channels disappeared
on refresh and "reappeared" only when a later add ran the persist hook.
4. **Share button reuses the Add Channel modal** — wrong intent reuse
(Add = INPUT, Share = OUTPUT). Replaced with a dedicated `#chShareModal`
(separate DOM id, separate title, share-only affordances, privacy
warning).

## TDD
Red commit (this) lands ONLY the failing tests:
- `test-channel-issue-1087.js` — source-string contract assertions for
all 4 bugs
- `test-channel-issue-1087-e2e.js` — Playwright E2E covering generate →
QR render, QR display name, persistence across refresh, Share opens
dedicated modal

Green commit (follow-up) lands the production fixes.

## E2E assertion added
E2E assertion added: test-channel-issue-1087-e2e.js:55

## CI wiring
- `test-channel-issue-1087.js` added to `.github/workflows/deploy.yml`
(go-test JS unit step) + `test-all.sh`
- `test-channel-issue-1087-e2e.js` added to
`.github/workflows/deploy.yml` (e2e-test step)

---------

Co-authored-by: bot <bot@corescope>
Co-authored-by: meshcore-bot <bot@meshcore.local>
Co-authored-by: clawbot <clawbot@users.noreply.github.com>
2026-05-05 03:24:52 -07:00
Kpa-clawbot a8e1cea683 fix: use payload type bits only in content hash (not full header byte) (#787)
## Problem

The firmware computes packet content hash as:

```
SHA256(payload_type_byte + [path_len for TRACE] + payload)
```

Where `payload_type_byte = (header >> 2) & 0x0F` — just the payload type
bits (2-5).

CoreScope was using the **full header byte** in its hash computation,
which includes route type bits (0-1) and version bits (6-7). This meant
the same logical packet produced different content hashes depending on
route type — breaking dedup and packet lookup.

**Firmware reference:** `Packet.cpp::calculatePacketHash()` uses
`getPayloadType()` which returns `(header >> PH_TYPE_SHIFT) &
PH_TYPE_MASK`.

## Fix

- Extract only payload type bits: `payloadType := (headerByte >> 2) &
0x0F`
- Include `path_len` byte in hash for TRACE packets (matching firmware
behavior)
- Applied to both `cmd/server/decoder.go` and `cmd/ingestor/decoder.go`

## Tests Added

- **Route type independence:** Same payload with FLOOD vs DIRECT route
types produces identical hash
- **TRACE path_len inclusion:** TRACE packets with different `path_len`
produce different hashes
- **Firmware compatibility:** Hash output matches manual computation of
firmware algorithm

## Migration Impact

Existing packets in the DB have content hashes computed with the old
(incorrect) formula. Options:

1. **Recompute hashes** via migration (recommended for clean state)
2. **Dual lookup** — check both old and new hash on queries (backward
compat)
3. **Accept the break** — old hashes become stale, new packets get
correct hashes

Recommend option 1 (migration) as a follow-up. The volume of affected
packets depends on how many distinct route types were seen for the same
logical packet.

Fixes #786

---------

Co-authored-by: you <you@example.com>
2026-04-18 11:52:22 -07:00
efiten fe314be3a8 feat: geo_filter enforcement, DB pruning, geofilter-builder tool, HB column (#215)
## Summary

Several features and fixes from a live deployment of the Go v3.0.0
backend.

### geo_filter — full enforcement

- **Go backend config** (`cmd/server/config.go`,
`cmd/ingestor/config.go`): added `GeoFilterConfig` struct so
`geo_filter.polygon` and `bufferKm` from `config.json` are parsed by
both the server and ingestor
- **Ingestor** (`cmd/ingestor/geo_filter.go`, `cmd/ingestor/main.go`):
ADVERT packets from nodes outside the configured polygon + buffer are
dropped *before* any DB write — no transmission, node, or observation
data is stored
- **Server API** (`cmd/server/geo_filter.go`, `cmd/server/routes.go`):
`GET /api/config/geo-filter` endpoint returns the polygon + bufferKm to
the frontend; `/api/nodes` responses filter out any out-of-area nodes
already in the DB
- **Frontend** (`public/map.js`, `public/live.js`): blue polygon overlay
(solid inner + dashed buffer zone) on Map and Live pages, toggled via
"Mesh live area" checkbox, state shared via localStorage

### Automatic DB pruning

- Add `retention.packetDays` to `config.json` to delete transmissions +
observations older than N days on a daily schedule (1 min after startup,
then every 24h). Nodes and observers are never pruned.
- `POST /api/admin/prune?days=N` for manual runs (requires `X-API-Key`
header if `apiKey` is set)

```json
"retention": {
  "nodeDays": 7,
  "packetDays": 30
}
```

### tools/geofilter-builder.html

Standalone HTML tool (no server needed) — open in browser, click to
place polygon points on a Leaflet map, set `bufferKm`, copy the
generated `geo_filter` JSON block into `config.json`.

### scripts/prune-nodes-outside-geo-filter.py

Utility script to clean existing out-of-area nodes from the database
(dry-run + confirm). Useful after first enabling geo_filter on a
populated DB.

### HB column in packets table

Shows the hop hash size in bytes (1–4) decoded from the path byte of
each packet's raw hex. Displayed as **HB** between Size and Type
columns, hidden on small screens.

## Test plan

- [x] ADVERT from node outside polygon is not stored (no new row in
nodes or transmissions)
- [x] `GET /api/config/geo-filter` returns polygon + bufferKm when
configured, `{polygon: null, bufferKm: 0}` when not
- [x] `/api/nodes` excludes nodes outside polygon even if present in DB
- [x] Map and Live pages show blue polygon overlay when configured;
checkbox toggles it
- [x] `retention.packetDays: 30` deletes old transmissions/observations
on startup and daily
- [x] `POST /api/admin/prune?days=30` returns `{deleted: N, days: 30}`
- [x] `tools/geofilter-builder.html` opens standalone, draws polygon,
copies valid JSON
- [x] HB column shows 1–4 for all packets in grouped and flat view

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-31 01:10:56 -07:00
Kpa-clawbot a555b68915 feat: expand frontend coverage collector for 60%+ target (#275)
## Summary

Expands the parallel frontend coverage collector to boost coverage from
~43% toward 60%+. This covers Phases 1 and 2 of the coverage improvement
plan.

### Phase 1 — Visit unvisited pages

- **Compare page** (`#/compare`): Navigates with query params selecting
two real observers from fixture DB, also exercises UI controls
- **Node analytics** (`#/nodes/{pubkey}/analytics`): Visits analytics
for two real nodes from fixture DB, clicks day buttons
- **Traces search** (`#/traces`): Searches for two valid packet hashes
from fixture DB
- **Personalized home**: Sets `localStorage.myNodes` with real pubkeys
before visiting `#/home`
- **Observer detail pages**: Direct navigation to
`#/observers/test-obs-1` and `#/observers/test-status-obs`
- **Real packet detail**: Navigates to `#/packets/b6b839cb61eead4a`
(real hash)
- **Rapid route transitions**: Exercises destroy/init cycles across all
pages
- **Compare in route list**: Added to the full route transition exercise

### Phase 2 — page.evaluate() for interactive code paths

| File | Functions exercised |
|------|-------------------|
| **live.js** | `vcrPause`, `vcrSpeedCycle`, `vcrReplayFromTs`,
`drawLcdText`, `vcrResumeLive`, `vcrUnpause`, `vcrRewind`,
`updateVCRClock`, `updateVCRLcd`, `updateVCRUI`, `bufferPacket`
(synthetic WS packets), `dbPacketToLive`, `renderPacketTree` |
| **packets.js** | `renderDecodedPacket` (ADVERT + GRP_TXT), `obsName`,
`renderPath`, `renderHop` |
| **packet-filter.js** | 30+ filter expressions now **evaluated against
4 synthetic packets** (previously only compiled, not run). Covers
`resolveField` for all field types including `payload.*` dot notation |
| **nodes.js** | `getStatusInfo`, `renderNodeBadges`,
`renderStatusExplanation`, `renderHashInconsistencyWarning` with varied
node types/roles |
| **roles.js** | `getHealthThresholds` (all roles), `getNodeStatus` (all
roles × active/stale), `getTileUrl`, `syncBadgeColors`, `miniMarkdown`
(bold, italic, code, links, lists), `copyToClipboard` |
| **channels.js** | `hashCode`, `getChannelColor`, `getSenderColor`,
`highlightMentions`, `formatSecondsAgo` |
| **app.js** | `escapeHtml`, `debouncedOnWS`, extended
`timeAgo`/`truncate` edge cases, extended
`routeTypeName`/`payloadTypeName`/`payloadTypeColor` ranges |

### What changed

- `scripts/collect-frontend-coverage.js` — +336 lines across existing
groups (no new groups added)

### Testing

- `npm test` passes (all 13 tests)
- No other files modified

Co-authored-by: you <you@example.com>
2026-03-29 20:41:02 -07:00
Kpa-clawbot 5777780fc8 refactor: parallel coverage collector (~30-60s vs 8min) (#272)
## Summary

Redesigned frontend coverage collector with 7 parallel browser contexts.
Coverage collector runs on master pushes only (skipped on PRs).

### Architecture
7 groups run simultaneously via `Promise.allSettled()`:
- G1: Home + Customizer
- G2: Nodes + Node Detail
- G3: Packets + Packet Detail
- G4: Map
- G5: Analytics + Channels + Observers
- G6: Live + Perf + Traces + Globals
- G7: Utility functions (page.evaluate)

### Speed gains
- `safeClick` 500ms → 100ms
- `navHash` 150ms → 50ms
- Removed redundant page visits and E2E-duplicate interactions
- Wall time = slowest group (~30-60s estimated)

### 821 lines → ~450 lines
Each group writes its own coverage JSON, nyc merges automatically.

### CI behavior
- **PRs:** Coverage collector skipped (fast CI)
- **Master:** Coverage collector runs (full synthetic user validation)

Co-authored-by: you <you@example.com>
2026-03-29 19:46:01 -07:00
Kpa-clawbot f438411a27 chore: remove deprecated Node.js backend (-11,291 lines) (#265)
## Summary

Removes all deprecated Node.js backend server code. The Go server
(`cmd/server/`) has been the production backend — the Node.js server was
kept "just in case" but is no longer needed.

### Removed (19 files, -11,291 lines)

**Backend server (6 files):**
`server.js`, `db.js`, `decoder.js`, `server-helpers.js`,
`packet-store.js`, `iata-coords.js`

**Backend tests (9 files):**
`test-decoder.js`, `test-decoder-spec.js`, `test-server-helpers.js`,
`test-server-routes.js`, `test-packet-store.js`, `test-db.js`,
`test-db-migration.js`, `test-regional-filter.js`,
`test-regional-integration.js`

**Backend tooling (4 files):**
`tools/e2e-test.js`, `tools/frontend-test.js`, `benchmark.js`,
`benchmark-ab.sh`

### Updated
- `AGENTS.md` — Rewritten architecture section for Go, explicit
deprecation warnings
- `test-all.sh` — Only runs frontend tests
- `package.json` — Updated test:unit
- `scripts/validate.sh` — Removed Node.js server syntax check
- `docker/supervisord.conf` — Points to Go binary

### NOT touched
- `public/` (active frontend) 
- `test-e2e-playwright.js` (frontend E2E tests) 
- Frontend test files (`test-packet-filter.js`, `test-aging.js`,
`test-frontend-helpers.js`) 
- `package.json` / Playwright deps 

### Follow-up
- Server-only npm deps (express, better-sqlite3, mqtt, ws, supertest)
can be cleaned from package.json separately
- `Dockerfile.node` can be removed separately

---------

Co-authored-by: you <you@example.com>
2026-03-29 15:53:51 -07:00
Kpa-clawbot ec7ae19bb5 ci: restructure pipeline — sequential fail-fast, Go server E2E, remove deprecated JS tests (#256)
## Summary

Complete CI pipeline restructure. Sequential fail-fast chain, E2E tests
against Go server with real staging data, all deprecated Node.js server
tests removed.

### Pipeline (PR):
1. **Go unit tests** — fail-fast, coverage + badges
2. **Playwright E2E** — against Go server with fixture DB, frontend
coverage, fail-fast on first failure
3. **Docker build** — verify containers build

### Pipeline (master merge):
Same chain + deploy to staging + badge publishing

### Removed:
- All Node.js server-side unit tests (deprecated JS server)
- `npm ci` / `npm run test` steps
- JS server coverage collection (`COVERAGE=1 node server.js`)
- Changed-files detection logic
- Docs-only CI skip logic
- Cancel-workflow API hacks

### Added:
- `test-fixtures/e2e-fixture.db` — real data from staging (200 nodes, 31
observers, 500 packets)
- `scripts/capture-fixture.sh` — refresh fixture from staging API
- Go server launches with `-port 13581 -db test-fixtures/e2e-fixture.db
-public public-instrumented`

---------

Co-authored-by: Kpa-clawbot <kpabap+clawdbot@gmail.com>
Co-authored-by: you <you@example.com>
2026-03-29 11:24:22 -07:00
you 13cab9bede perf: optimize frontend coverage collector (~2x faster)
Three optimizations to reduce wall-clock time:

1. Reduce safeClick timeout from 3000ms to 500ms
   - Elements either exist immediately after navigation or don't exist at all
   - ~75 safeClick calls; if ~30 miss, saves ~75s of dead wait time

2. Replace 18 page.goto() calls with SPA hash navigation
   - After initial page load, the SPA shell is already in the DOM
   - page.goto() reloads the entire page (network round-trip + parse)
   - Hash navigation via location.hash triggers the SPA router instantly
   - Only 3 page.goto() remain: initial load + 2 home page loads after localStorage.clear()

3. Remove redundant final route sweep
   - All 10 routes were already visited during the page-specific sections
   - The sweep just re-navigated to pages that had already been exercised
   - Saves ~2s of redundant navigation

Also:
- Reduce inter-route wait from 200ms to 50ms (SPA router is synchronous)
- Merge utility function + packet filter exercises into single evaluate() call
- Use navHash() helper for consistent hash navigation with 150ms settle time
2026-03-29 10:32:42 -07:00
you 12d1174e39 perf: speed up frontend coverage tests (~3x faster)
Three optimizations to the CI frontend test pipeline:

1. Run E2E tests and coverage collection concurrently
   - Previously sequential (E2E ~1.5min, then coverage ~5.75min)
   - Now both run in parallel against the same instrumented server
   - Expected savings: ~5 min (coverage runs alongside E2E instead of after)

2. Replace networkidle with domcontentloaded in coverage collector
   - SPA uses hash routing — networkidle waits 500ms for network silence
     on every navigation, adding ~10-15s of dead time across 23 navigations
   - domcontentloaded fires immediately once HTML is parsed; JS initializes
     the route handler synchronously
   - For in-page hash changes, use 200ms setTimeout instead of
     waitForLoadState (which would never re-fire for same-document nav)

3. Extract coverage from E2E tests too
   - E2E tests already exercise the app against the instrumented server
   - Now writes window.__coverage__ to .nyc_output/e2e-coverage.json
   - nyc merges both coverage files for higher total coverage

Also:
- Split Playwright install into browser + deps steps (deps skip if present)
- Replace sleep 5 with health-check poll in quick E2E path
2026-03-29 09:12:23 -07:00
Kpa-clawbot 72161ba8fe perf: replace 169 blind sleeps with Playwright waits in coverage script
Remove all 169 waitForTimeout() calls (totaling 104.1s of blind sleeping)
from scripts/collect-frontend-coverage.js:

- Helper functions (safeClick, safeFill, safeSelect, clickAll, cycleSelect):
  removed 300-400ms waits after every interaction — Playwright's built-in
  actionability checks handle waiting for elements automatically
- Post-navigation waits: removed redundant sleeps after page.goto() calls
  that already use waitUntil: 'networkidle'
- Hash-change navigations: replaced waitForTimeout with
  waitForLoadState('networkidle') for proper SPA route settling
- Toggle/button waits: removed — event handlers execute synchronously
  before click() resolves
- Post-evaluate waits: removed — evaluate() is synchronous

Local benchmark (Windows, sparse test data):
  Before: 744.8s
  After:  484.8s (35% faster, 260s saved)

On CI runner (ARM Linux with real mesh data), savings will be
proportionally better since most elements exist and the 104s
of blind sleeping was the dominant bottleneck.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-26 23:02:03 -07:00
you 9b0c740537 revert: aggressive coverage interactions dropped score from 42% to 39%
The page.evaluate() calls corrupting localStorage and firing fake events
caused page error-reloads, losing accumulated coverage. Reverting to
the 42% version which was the actual high water mark.
2026-03-24 05:48:06 +00:00
you 556e3b19db coverage: aggressive branch coverage push — target 80%+
Add ~900 lines of deep branch-coverage interactions:
- Utility functions with all edge cases (timeAgo, truncate, escapeHtml, formatHex, etc.)
- roles.js: getHealthThresholds/getNodeStatus for all roles + edge inputs
- PacketFilter: compile+match with mock packets, all operators, bad expressions
- HopResolver/HopDisplay: init, resolve, renderPath with various inputs
- RegionFilter: onChange, getSelected, isEnabled, setRegions, render
- Customize: deep tab cycling, import/export, bad JSON, theme preview
- WebSocket reconnection trigger
- Keyboard shortcuts (Ctrl+K, Meta+K, Escape)
- Window resize (mobile/tablet/desktop) for responsive branches
- Error routes: nonexistent nodes/packets/observers/channels
- localStorage corruption to trigger catch branches
- Theme toggling (dark/light rapid switching, custom vars)
- Live page: VCR modes, timeline clicks, speed cycling, all toggles
- Audio Lab: play/stop/loop, BPM/volume sliders, voice selection
- All analytics tabs via deep-link + sort headers
- Packets: complex filter expressions, scroll-to-load, double-click
- Nodes: special char search, all sort columns, fav stars
- Channels: resize handle drag, theme observer, node tooltips
- Observers/Observer Detail: sort, tabs, day cycling
- Node Analytics: day buttons, tabs
- Home: both new/experienced flows, empty search results
- debouncedOnWS/onWS/offWS exercise
2026-03-24 05:26:22 +00:00
you 86dce6f350 coverage: massively expand frontend interaction coverage
Exercise every major code path across all frontend files:

app.js: all routes, bad routes, hashchange, theme toggle x4,
  hamburger menu, favorites dropdown, global search, Ctrl+K,
  apiPerf(), timeAgo/truncate/routeTypeName utils

nodes.js: sort every column (both directions), every role tab,
  every status filter, cycle all Last Heard options, click rows
  for side pane, navigate to detail page, copy URL, show all
  paths, node analytics day buttons (1/7/30/365), scroll target

packets.js: 12 filter expressions including bad ones, cycle all
  time windows, group by hash toggle, My Nodes toggle, observer
  menu, type filter menu, hash input, node filter, observer sort,
  column toggle menu, hex hash toggle, pause button, resize handle,
  deep-link to packet hash

map.js: all role checkboxes toggle, clusters/heatmap/neighbors/
  hash labels toggles, cycle Last Heard, status filter buttons,
  jump buttons, markers, zoom controls, dark mode tile swap

analytics.js: all 9 tabs clicked, deep-link to each tab via URL,
  observer selector on topology, navigate rows on collisions/
  subpaths, sortable headers on nodes tab, region filter

customize.js: all 5 tabs, all preset themes, branding text inputs,
  theme color inputs, node color inputs, type color inputs, reset
  buttons, home tab fields (hero, journey steps, checklist, links),
  export tab, reset preview/user theme

live.js: VCR pause/speed/missed/prompt buttons, all visualization
  toggles (heat/ghost/realistic/favorites/matrix/rain), audio
  toggle + BPM slider, timeline click, resize event

channels.js: click rows, navigate to specific channel
observers.js: click rows, navigate to detail, cycle days select
traces.js: click rows
perf.js: refresh + reset buttons
home.js: both chooser paths, search + suggest, my-node cards,
  health/packets buttons, remove buttons, toggle level, timeline

Also exercises packet-filter parser and region-filter directly.
2026-03-24 04:57:09 +00:00
you 26d4bbc39d ci: fix coverage collector — use Playwright bundled chromium on CI 2026-03-24 04:26:19 +00:00
you 5ee976055b feat(coverage): add targeted Playwright interactions for higher frontend coverage
Add redundant selectors (data-sort, data-role, data-status, data-tab,
placeholder-based search, emoji theme toggle, .cust-close, #fTimeWindow,
broader preset selectors) to exercise more frontend code paths.

All interactions wrapped in try/catch for resilience.
2026-03-24 04:19:32 +00:00
you 860d5c574e test: expanded frontend coverage collection with page interactions
367 lines of Playwright interactions covering nodes, packets, map,
analytics, customizer, channels, live, home pages.
Fixed e2e channels assertion (chList vs chResp.channels).
2026-03-24 03:43:27 +00:00
you d7faa4d978 Add frontend code coverage via Istanbul instrumentation + Playwright
- Install nyc for Istanbul instrumentation
- Add scripts/instrument-frontend.sh to instrument public/*.js
- Add scripts/collect-frontend-coverage.js to extract window.__coverage__
- Add scripts/combined-coverage.sh for combined server+frontend coverage
- Make server.js serve public-instrumented/ when COVERAGE=1 is set
- Add test:full-coverage npm script
- Add public-instrumented/ and .nyc_output/ to .gitignore
2026-03-24 03:11:13 +00:00
you 91a6a2c525 fix: migration handles concurrent dual-write with INSERT OR IGNORE 2026-03-20 22:52:12 +00:00
you d7e415daa7 fix: raw_hex NOT NULL in transmissions schema — deleted 4 junk test rows 2026-03-20 20:24:13 +00:00
you 2c6148fd2d Add dedup migration script (Milestone 1)
Creates transmissions and observations tables from existing packets table.
- Groups packets by hash → 1 transmission per unique hash
- Creates 1 observation per original packet row with FK to transmission
- Idempotent: drops and recreates new tables on each run
- Does NOT modify the original packets table
- Prints stats and verifies counts match

Tested on test DB: 33813 packets → 11530 transmissions (2.93x dedup ratio)
2026-03-20 20:22:30 +00:00
you 10b11106f6 ci: add pre-deploy JS validation — syntax check + undefined reference detection
Validation runs BEFORE docker build. If it fails, deployment is blocked.
No more broken code reaching production.
2026-03-20 19:24:11 +00:00