mirror of
https://github.com/Kpa-clawbot/meshcore-analyzer.git
synced 2026-06-27 10:11:40 +00:00
f0addfdabf3dbeb2121dd954e03362bf4ebe7cd1
30 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Kpa-clawbot
|
55e4d957b1 |
M1: emoji → Phosphor Icons — top-nav, mobile nav, Compare (#1648) (#1649)
Red commit:
|
||
|
Kpa-clawbot
|
63bfa3d910 |
feat(security): detect CDN-fronted deployment + document bypass requirement (closes #1561) (#1564)
Closes #1561. Follow-up to #1551. ## Why #1551 added `Cache-Control: no-store` to all `/api/*` responses. That's sufficient for CDNs that honour origin headers (Varnish, nginx). It is **not** sufficient for Cloudflare zones where Cache Rules / Page Rules override origin Cache-Control. Field evidence from the meshat.se diagnosis (2026-06-04): observers behind Cloudflare were returning `cf-cache-status: HIT` with `age` up to ~6 hours despite the origin emitting `no-store`. The CDN was caching per zone policy and ignoring the upstream directive — exactly the failure mode #1551 cannot reach. The application has no way to inject CDN rules; the only durable fix is operator-side. This PR makes that operator step discoverable and verifiable. ## What ### Server-side detection (log-only) `cmd/server/cdn_detection.go` adds a middleware wired into the `/api/*` chain after `noStoreAPIMiddleware`. On the **first** request bearing any CDN-typical header (`CF-Connecting-IP`, `CF-Ray`, `X-Forwarded-For`, `X-Real-IP`, `Fastly-Client-IP`, `True-Client-IP`) it logs: ``` [security] WARNING: detected request via CDN (CF-Ray header present). Ensure /api/* is bypassed in your CDN config — see docs/deployment-behind-cdn.md. Cached API responses cause observer-flap and incorrect dashboards. ``` `sync.Once` guarantees the warning fires at most once per process boot. The middleware never blocks, never modifies the response, never adds headers. Detection is observational only — operators who run behind a CDN without bypass have a real bug; the warning is appropriate. ### Operator documentation `docs/deployment.md` gains a new **"Behind a CDN (Cloudflare, Fastly)"** section covering: 1. Curl verification command + healthy vs unhealthy output examples 2. Cloudflare Cache Rule creation (URI Path starts-with `/api/` → Bypass cache) 3. Legacy Page Rules equivalent 4. Fastly note 5. Re-verification 6. Meaning of the startup log warning 7. Why we can't fix this server-side `docs/deployment-behind-cdn.md` is the canonical path the log message references — it's a short TL;DR that links back to the full section. ### Healthcheck script `scripts/check-cdn-bypass.sh` — POSIX sh, no dependencies beyond curl + grep + awk. Operators run: ```sh scripts/check-cdn-bypass.sh https://your-domain.example.com ``` Exits `0` with `OK: no CDN caching detected ...` or `1` with a precise diagnostic naming the offending header (`cf-cache-status: HIT` or stale `age`). ## TDD - **Red commit `e90ccaba`** (`test(security): RED ...`) — `cmd/server/cdn_detection_test.go` (4 Go tests + 6 subtests for each header) and `scripts/test-check-cdn-bypass.sh` (3 shell harness cases). Middleware stub returns `next` unchanged so tests compile and fail on assertions, not build errors. - **Green commit `5e6a60b5`** (`feat(security): GREEN ...`) — real middleware, wiring in `routes.go`, healthcheck script, doc. ## Deliverables | File | Status | Purpose | |------|--------|---------| | `cmd/server/cdn_detection.go` | new | middleware + sync.Once warning | | `cmd/server/cdn_detection_test.go` | new | 4 Go tests (1 stand-alone + 1 silence + 1 once + 1 table-driven over 6 headers) | | `cmd/server/routes.go` | modified | `r.Use(cdnDetectionMiddleware)` after no-store | | `docs/deployment.md` | modified | TOC entry + "Behind a CDN" section | | `docs/deployment-behind-cdn.md` | new | canonical path referenced by log message + script output | | `scripts/check-cdn-bypass.sh` | new | operator-runnable healthcheck | | `scripts/test-check-cdn-bypass.sh` | new | shell harness with fake curl | ## What this PR explicitly does NOT do - Does not block requests based on CDN detection (log-only). - Does not enforce CDN bypass (impossible — operator-controlled). - Does not spoof, strip or modify CDN headers. - Does not add CSP / HSTS / other security headers (out of scope). - Warning is not configurable — operators behind a CDN without bypass have a real bug, surfacing it is correct. ## Verification - `go test ./...` in `cmd/server/` — full suite green. - `sh scripts/test-check-cdn-bypass.sh` — 3/3 pass. - Preflight checklist — all 11 gates clean (PII, branch scope, red commit, CSS vars, CSS self-fallback, LIKE-on-JSON, sync migration, async-migration annotation, XSS sinks, img/SVG ratio, themed-img/SVG, fixture coverage). --------- Co-authored-by: openclaw-bot <bot@openclaw.local> Co-authored-by: clawbot <bot@clawbot.invalid> |
||
|
Kpa-clawbot
|
e4a21fc9ab |
feat(preflight): hard-fail gate on unescaped node-controlled HTML sinks (#1543)
## Summary Closes the "XSS regression in newly-added sink" class. Follow-up to #1537 (10 stored-XSS sinks in node names) and the post-#1537 audit (TRACE-1, OBS-1, ANL-1 — 3 additional HIGH XSS in files #1537 didn't touch). After those fixes land, the project still has **zero automated catch for the next one**. Every future PR can re-introduce the same class freely. This PR closes that gap with a hard-fail pr-preflight gate that runs at PR-creation time and in CI. ## What the gate does A NEW or MODIFIED line in the PR diff under `public/**/*.{js,html}` is flagged when it matches any of these sink patterns: | Pattern | What it catches | |---|---| | `.innerHTML = \`…\`` / `'…'` | template-literal or string-concat HTML injection | | `insertAdjacentHTML(…, \`…\`)` | DOM-adjacent injection | | `.bindPopup(\`…\`)` / `.bindTooltip(\`…\`)` | Leaflet popup/tooltip injection (the OBS-1 class) | | `.setAttribute('on<event>', …)` | inline event-handler injection | | `.setAttribute('href'\|'src'\|'action'\|'formaction', <interp>)` | `javascript:` URI class | For each flagged line, the gate then walks the dynamic substring (`${…}`, post-`+`, or `setAttribute` value arg) and only fires if it interpolates an identifier from the node-controlled allowlist (`name`, `observer`, `sender`, `pubkey`, `body`, `hash`, …). This keeps the regex off static CSS classes like `text-center`. A flagged line is accepted (no fail) when ANY of: - **(a)** wrapped in `escapeHtml(` / `escapeAttr(` / `safeEsc(` / local `esc(` — the audited helpers - **(b)** a same-PR `test*.js` file DOM-greps the audit payload (`' onfocus=` or `onerror=alert`) AND references the sink file's basename - **(c)** the PR body carries `PREFLIGHT-XSS-OPTOUT: <file>:<line> reason="…"` — explicit author opt-out logged for reviewer attention Otherwise: **HARD FAIL** with `file:line: flagged: <token>` plus a suggested fix. ## Split - **Skill directory** (local, no PR): - `~/.openclaw/skills/pr-preflight/scripts/check-xss-sinks.sh` — canonical gate - `~/.openclaw/skills/pr-preflight/data/xss-node-controlled-fields.txt` — allowlist (27 identifiers, easy to extend without a repo PR) - wired into `~/.openclaw/skills/pr-preflight/scripts/run-all.sh` - **This PR** (in repo): - `testdata/preflight-xss/` — fixtures (`bad-1..bad-3`, `good-1..good-2`, `test-good-2.js`) - `scripts/check-xss-sinks.sh` — local mirror of the canonical gate, so CI can exercise the gate without depending on the skill dir - `test-preflight-xss-gate.js` — Node test wrapper that asserts bad fixtures fail (exit 1) and good fixtures pass (exit 0) - `public/app.js` — `escapeHtml` docstring marked CANONICAL with links to the enforcing gate - `.github/workflows/deploy.yml` — invoke `node test-preflight-xss-gate.js` alongside the existing `test-xss-escape-sinks.js` ## TDD red → green | | Commit | Test result | |---|---|---| | **Red** | `test(preflight-xss): RED — fixtures + assertion wrapper for XSS sink gate` | `test-preflight-xss-gate.js` exits 1 — bad fixtures unexpectedly pass because `scripts/check-xss-sinks.sh` is a no-op stub. Genuine assertion failure (not a build error). | | **Green** | `feat(preflight): GREEN — implement XSS-sink check + escapeHtml docstring` | stub replaced with real check; all 5 fixtures behave as expected. | The red commit ships a working stub script so the test runs to completion and fails on an **assertion**, not on a missing-file error. ## Coverage proof — would the gate have caught the originals? - **PR #1537 (10 sinks):** synthetic file from the deleted lines of #1537 → gate flags `n.name` in `innerHTML \`tpl\`` and two `bindPopup(\`…${n.name}\`)` lines. Yes, the gate would have caught these the moment they hit a PR diff. - **Post-#1537 audit:** - **TRACE-1** (`traces.js` `${e.message}` / `${urlHash}` in innerHTML): yes — the `hash`/`urlHash` tokens are allowlisted and the innerHTML template-literal pattern matches. - **OBS-1** (`observer-detail.js` URL fragment + MQTT fields into innerHTML / bindPopup): yes — the `observer`, `text`, `hash` tokens are allowlisted and both sink patterns match. - **ANL-1** (`analytics.js` attribute-mutation roundtrip): yes for `setAttribute('on*', …)` and `setAttribute('href', \`…${interp}…\`)` patterns. (Note: pure innerHTML lines with only `${e.message}` are not node-controlled and are intentionally not flagged.) ## Allowlist (initial 27 identifiers) ``` adv_name name observer observer_name sender from_node channel channel_name model firmware client_version radio iata hopNames nodeLabel obsName n.name o.name obs.name public_key pubkey area_key region_name text body message preview hash urlHash ``` Extend in `~/.openclaw/skills/pr-preflight/data/xss-node-controlled-fields.txt` whenever a new node-controlled field surfaces in an audit — no repo PR required. ## Hard rules respected - No build step, no ESLint plugin, no AST analysis — grep + heuristics + opt-out escape valves - Hard fail (exit 1), not warning-only (exit 2) - PII preflight grep on every commit + this PR body - Same split as the sibling migration-gate PR ## Three-axis merge-readiness - **Mergeable:** yes — branch is clean off `origin/master`, no conflicts - **CI:** will report on push; red commit expected to fail, green commit expected to pass - **Threads:** none open yet (new PR) --------- Co-authored-by: meshcore-bot <bot@local> Co-authored-by: mc-bot <bot@meshcore.local> Co-authored-by: corescope-bot <bot@corescope> |
||
|
efiten
|
8151185ede |
fix(ci): Dockerfile COPY invariant check — prevent missing internal/<pkg> Docker failures (#1316) (#1432)
## Summary - Adds `scripts/check-dockerfile-internal-pkgs.sh`: reads `replace => ../../internal/<pkg>` directives from `cmd/server/go.mod` and `cmd/ingestor/go.mod`, then verifies each referenced package has the correct number of `COPY internal/<pkg>/` lines in `Dockerfile` (one per builder section that needs it) - Wired into CI as a step in the `go-test` job, before CSS lint — runs on every PR, adds ~0.1s - Prevents the recurring failure pattern (#1316): new `internal/<pkg>` added to go.mod but COPY line forgotten in Dockerfile; non-Docker CI passes, Docker build fails after merge with a cryptic module error Key details: - Counts COPY occurrences per package: if a pkg is referenced in both go.mods (both binaries need it), it must appear in at least 2 builder sections - Anchored regex: only matches actual `replace` directives (not comments) - Anchored grep: skips commented-out `COPY internal/...` lines Closes #1316. ## Test plan - [ ] Run `bash scripts/check-dockerfile-internal-pkgs.sh` locally — exits 0 on current Dockerfile - [ ] Manually remove a `COPY internal/perfio/` line from Dockerfile → script exits 1 with a clear error - [ ] CI step visible in the `go-test` job on this PR 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Kpa-clawbot
|
e267fb754d |
fix(ci): aggregate e2e pass/fail across all suites instead of broken digits-before-slash regex (#1298)
RED |
||
|
Kpa-clawbot
|
43203b09b7 |
fix(#1249): IATA badge missing on fixture + mobile clipping (#1252)
Failing test commit: `bdb4eefb` (added in #1189 R1) — original CI failure: https://github.com/Kpa-clawbot/CoreScope/actions/runs/25995819598 Fixes #1249. ## Root cause Two independent bugs surfaced by the same E2E test: 1. **Fixture join broken.** `scripts/capture-fixture.sh` wrote the text observer hash into `observations.observer_idx`, but the v3 join in `cmd/server` is `observers.rowid = observations.observer_idx`. The join silently nulled out `observer_id` / `observer_iata` for every packet. 2. **Mobile clipping.** `.col-observer` had `data-priority=3` (hides at ≤1024px) and was in the narrow-viewport `defaultHidden` list, so at 375px the cell collapsed to `display:none` and `.badge-iata` had a 0×0 box. ## Changes - `test-fixtures/e2e-fixture.db`: remap `observer_idx` text hash → integer rowid (500/500 rows resolved). - `scripts/capture-fixture.sh`: build an `observer_id → rowid` map before insert; skip rows whose observer isn't in the fixture. Comment explains the trap. - `public/packets.js`: bump `.col-observer` priority `3 → 1` and drop `observer` from narrow-viewport `defaultHidden`. ## Verification All three sub-tests in `test-observer-iata-1188-e2e.js` pass locally against the freshened fixture. `curl /api/packets?limit=5` returns real IATA codes (OAK / MRY / SFO) instead of empty strings. Co-authored-by: OpenClaw Bot <bot@openclaw.local> |
||
|
Kpa-clawbot
|
eae1b915ca |
feat: load Aldrich webfont for #1137 logo SVG (#1138)
Adds Aldrich webfont so the merged #1137 logo renders in the intended typeface. ## Problem The inline SVG logo merged in #1137 declares `font-family="Aldrich, monospace"` in `public/index.html` and `public/home.js`, but the page never loaded the Aldrich font face. Browsers silently fell back to monospace. ## Fix Self-hosted webfont: - `public/fonts/aldrich-regular.woff2` — Regular 400, ~16KB, downloaded from Google Fonts (latin subset). Self-hosted to avoid third-party CDN dependency, privacy concern, and FOUT delay. - `@font-face` declaration added at the top of `public/style.css` with `font-display: swap`. Aldrich only ships in 400; the SVG `font-weight="700"` on the wordmark synthesizes bold (matches the design intent of #1137). ## TDD - Red commit: E2E test asserting `document.fonts.check('1em Aldrich')` is true and the navbar SVG `<text>` `font-family` contains "Aldrich". Without the font face declaration, both assertions fail on an assertion (not a build error). - Green commit: adds the woff2 + `@font-face` rule, both assertions pass. ## Files - `public/fonts/aldrich-regular.woff2` (new, 16460 bytes) - `public/style.css` — `@font-face` rule - `test-e2e-playwright.js` — new test --------- Co-authored-by: openclaw-bot <bot@openclaw.local> |
||
|
Kpa-clawbot
|
494d3022f9 |
Partial fix for #1128: close audit gaps (z-scale, css-var lint, multi-viewport E2E, Bug 1+5 polish) (#1133)
## Partial fix for #1128 — closes the gaps PR #1131 left behind PR #1131 was a partial fix for the packets-page layout chaos (merged 2026-05-06 ~01:55 UTC, then the issue was reopened by the maintainer). #1131 shipped Bug 4 (`--surface` definition), the `.path-popover` flip + lower z-index, the debounced re-measure for Bug 1, the `.filter-bar` row-gap + `.multi-select-trigger` truncation for Bug 3, the new z-index TOKENS, and a single-viewport E2E with five individual-component assertions. This PR closes everything else the issue body and the `specs/packets-layout-audit.md` audit asked for. ### What changed (per gap) **Gap A — apply the z-index scale (audit Section 2)** #1131 added `--z-dropdown` / `--z-popover` / `--z-modal` / `--z-tooltip` but explicitly left existing literal values in place. This PR renumbers the 7 dropdowns/popovers the audit named: | Selector | Before | After | |---|---:|---:| | `.col-toggle-menu` | 50 | `var(--z-dropdown)` (100) | | `.multi-select-menu` | 90 | `var(--z-dropdown)` | | `.region-dropdown-menu` | 90 | `var(--z-dropdown)` | | `.node-filter-dropdown` | 100 | `var(--z-dropdown)` | | `.fux-saved-menu` | `var(--z-tooltip)` (9200) | `var(--z-dropdown)` | | `.fux-ac-dropdown` | `var(--z-tooltip)` | `var(--z-dropdown)` | | `.hop-conflict-popover` | `var(--z-tooltip)` | `var(--z-popover)` (300) | `.fux-ctx-menu` deliberately retains the tooltip band — context menus must float above all toolbar UI. `.region-filter-options-menu` no longer exists in the source (was renamed `.region-dropdown-menu`). The `style.css` doc-block at the top is rewritten to record the applied scale and to point operators at the new lint. **Gap B — CSS-var lint (audit Section 5 #1, "single highest-value addition")** Adds `scripts/check-css-vars.js` (~70 lines). Walks `public/*.css`, extracts every `var(--name)` reference WITHOUT a fallback, asserts the name is defined in some `public/*.css`. References WITH a fallback are tolerated. Wired into CI in the `go-test` job before the JS unit tests. The red commit (`608d81f`) shipped this lint exiting 1 against the master tree — three undefined vars that bypassed earlier review: ``` public/style.css:2628 var(--text-primary) public/style.css:2675 var(--bg-hover) public/style.css:2924 var(--primary) ``` The green commit (`1369d1e`) defines those three as aliases in the :root block (`--text-primary` → `--text`, `--bg-hover` → `--hover-bg`, `--primary` → `--accent`). Light + dark themes inherit through the existing tokens. **Gap C — multi-viewport E2E (issue acceptance criterion)** Adds `test-issue-1128-multi-viewport-e2e.js` — sister of the existing single-viewport test. At each of three viewports (1280×900, 1080×800, 768×1024): - takes a screenshot to `e2e-screenshots/issue-1128-<viewport>.png` - asserts no two `.filter-group` siblings vertically overlap - on desktop+laptop, opens the Saved menu and the Types multi-select and asserts the dropdown does not vertically overlap any `.filter-group` below it Plus three viewport-agnostic assertions: - dropdown selectors compute z-index in `[100,199]` (`.col-toggle-menu`, `.multi-select-menu`, `.region-filter-options-menu`, `.fux-saved-menu`, `.fux-ac-dropdown`) - `.path-hops .hop / .hop-named / .arrow` compute `line-height ≤ 18px` - `.col-path` computes `height ≤ 28px` Wired into the e2e-test job after the existing #1128 test. **Gap D — Bug 5 polish (toolbar reorder)** Audit Section 3 Bug 5: swaps `filter-group-dropdowns` and `filter-group-toggles` in `public/packets.js` so time range + Group by Hash + ★ My Nodes sit next to the search input. Pure markup reorder. No CSS / no JS-handler changes. **Gap E — Bug 1 belt-and-suspenders** Audit Section 3 Bug 1 sub-bullets: - locks `.path-hops .hop / .hop-named / .arrow` to `line-height: 18px` so a chip with mixed font metrics cannot overflow the 22px host vertically and bleed into the row above - converts `.col-path { max-height: 28px }` → `height: 28px` because browsers widely ignore `max-height` on `<td>`s; the earlier rule was a no-op ### TDD discipline (red → green) ``` $ git log --oneline origin/master..HEAD |
||
|
Kpa-clawbot
|
364c5766fc |
feat(logo): wire new CoreScope SVG logo into navbar + home hero (#1137)
## Adds new logo and home hero Replaces the navbar mushroom emoji + "CoreScope" text spans with the new CoreScope SVG mark, and adds a hero SVG (with the MESH ANALYZER tagline) above the home page H1. ### What changed - `public/img/corescope-logo.svg` — navbar mark, no tagline (locked "aggressive low-amp chirp" variant: facing-arcs + low-amp chirp connector between the two nodes). - `public/img/corescope-hero.svg` — home hero version, includes the MESH ANALYZER tagline. - `public/index.html` — replaces `<span class="brand-icon">🍄</span><span class="brand-text">CoreScope</span>` with `<img class="brand-logo" src="img/corescope-logo.svg?__BUST__" …>`. `.nav-brand` link still routes to `#/`. `.live-dot` retained. - `public/style.css` — adds `.brand-logo { height: 36px }` (32px on tablet ≤900px). Existing 52px nav height unchanged. - `public/home.js` / `public/home.css` — adds `<img class="home-hero-logo">` above the hero `<h1>`, sized `max-width: min(720px, 90vw)` and centered. ### TDD Red→green is visible in the branch: - `3159b82` — `test(logo): add failing E2E …` (red commit). Adds `test-logo-rebrand-e2e.js` and wires it into the `e2e-test` job in `deploy.yml` with `CHROMIUM_REQUIRE=1`. On this commit `index.html` still has the emoji + text spans, `home.js` has no hero img, and the SVG asset files do not exist — the test asserts on each so CI fails on assertion. - `19434e1` — `feat(logo): wire new CoreScope SVG logo …` (green commit). Implements the fix. ### E2E asserts 1. `.nav-brand img` exists with `src` ending `corescope-logo.svg` 2. legacy `.brand-icon` / `.brand-text` are gone 3. `.live-dot` is present, visible, and to the right of the logo (no overlap) 4. `.home-hero img.home-hero-logo` exists with `src` ending `corescope-hero.svg`, positioned BEFORE the `<h1>` 5. both `/img/corescope-{logo,hero}.svg` return 200 with svg content-type ### Customizer compatibility - `customize.js` still does `querySelector('.brand-text')` / `.brand-icon` for live branding updates. Both now return `null`; existing `if (el)` guards make those branches silent no-ops. **No JS errors, but the customizer's `branding.siteName` and `branding.logoUrl` fields no longer rewrite the navbar brand** — the brand is now a fixed SVG asset. - **Theme accent does NOT recolor the SVG.** SVGs loaded via `<img src>` are isolated documents and cannot inherit document CSS variables; the SVG falls back to its embedded brand colors. This is appropriate for a brand mark; if recoloring per theme is desired later, swap to inline SVG (separate PR). ### Browser validation Local Chromium not available in this env; the E2E test soft-skips locally and hard-fails in CI (`CHROMIUM_REQUIRE=1`). Server-side checks done locally: - `curl http://localhost:13581/` → confirmed `<img class="brand-logo" src="img/corescope-logo.svg?<bust>" …>` rendered, no `.brand-icon`/`.brand-text` spans. - `curl -I /img/corescope-logo.svg` and `/img/corescope-hero.svg` → both 200. ### Performance No hot-path changes. Two new static SVG assets (~7.6KB each), served directly by the Go static handler. Cache-busted via `?__BUST__` (auto-replaced server-side). --------- Co-authored-by: OpenClaw Bot <bot@openclaw.local> Co-authored-by: Kpa-clawbot <bot@kpa-clawbot.local> |
||
|
Kpa-clawbot
|
ac0cf5ac7d |
fix(channels): #1087 QR library + share modal + PSK persistence (#1090)
Red commit: 5def4d073c61058fc9f327a3c60ece27e21cbc69 (CI run pending — see Checks tab) Fixes #1087 ## What's broken (4 bugs) 1. **"QR library not loaded"** — `channel-qr.js` checked `root.QRCode` (capital), but the vendored library exports lowercase `qrcode` (Kazuhiko Arase API). Generate & Show QR always fell into the "library not loaded" branch. 2. **QR encodes `name=psk:hex`** — the Share button (and parts of the Generate path) passed the internal `psk:<hex8>` lookup key to `ChannelQR.generate`, ignoring the user's display label stored in `LABELS_KEY`. 3. **PSK channel doesn't persist on refresh** — the persistence path was scattered, and the read-back wasn't verified. Added channels disappeared on refresh and "reappeared" only when a later add ran the persist hook. 4. **Share button reuses the Add Channel modal** — wrong intent reuse (Add = INPUT, Share = OUTPUT). Replaced with a dedicated `#chShareModal` (separate DOM id, separate title, share-only affordances, privacy warning). ## TDD Red commit (this) lands ONLY the failing tests: - `test-channel-issue-1087.js` — source-string contract assertions for all 4 bugs - `test-channel-issue-1087-e2e.js` — Playwright E2E covering generate → QR render, QR display name, persistence across refresh, Share opens dedicated modal Green commit (follow-up) lands the production fixes. ## E2E assertion added E2E assertion added: test-channel-issue-1087-e2e.js:55 ## CI wiring - `test-channel-issue-1087.js` added to `.github/workflows/deploy.yml` (go-test JS unit step) + `test-all.sh` - `test-channel-issue-1087-e2e.js` added to `.github/workflows/deploy.yml` (e2e-test step) --------- Co-authored-by: bot <bot@corescope> Co-authored-by: meshcore-bot <bot@meshcore.local> Co-authored-by: clawbot <clawbot@users.noreply.github.com> |
||
|
Kpa-clawbot
|
a8e1cea683 |
fix: use payload type bits only in content hash (not full header byte) (#787)
## Problem The firmware computes packet content hash as: ``` SHA256(payload_type_byte + [path_len for TRACE] + payload) ``` Where `payload_type_byte = (header >> 2) & 0x0F` — just the payload type bits (2-5). CoreScope was using the **full header byte** in its hash computation, which includes route type bits (0-1) and version bits (6-7). This meant the same logical packet produced different content hashes depending on route type — breaking dedup and packet lookup. **Firmware reference:** `Packet.cpp::calculatePacketHash()` uses `getPayloadType()` which returns `(header >> PH_TYPE_SHIFT) & PH_TYPE_MASK`. ## Fix - Extract only payload type bits: `payloadType := (headerByte >> 2) & 0x0F` - Include `path_len` byte in hash for TRACE packets (matching firmware behavior) - Applied to both `cmd/server/decoder.go` and `cmd/ingestor/decoder.go` ## Tests Added - **Route type independence:** Same payload with FLOOD vs DIRECT route types produces identical hash - **TRACE path_len inclusion:** TRACE packets with different `path_len` produce different hashes - **Firmware compatibility:** Hash output matches manual computation of firmware algorithm ## Migration Impact Existing packets in the DB have content hashes computed with the old (incorrect) formula. Options: 1. **Recompute hashes** via migration (recommended for clean state) 2. **Dual lookup** — check both old and new hash on queries (backward compat) 3. **Accept the break** — old hashes become stale, new packets get correct hashes Recommend option 1 (migration) as a follow-up. The volume of affected packets depends on how many distinct route types were seen for the same logical packet. Fixes #786 --------- Co-authored-by: you <you@example.com> |
||
|
efiten
|
fe314be3a8 |
feat: geo_filter enforcement, DB pruning, geofilter-builder tool, HB column (#215)
## Summary
Several features and fixes from a live deployment of the Go v3.0.0
backend.
### geo_filter — full enforcement
- **Go backend config** (`cmd/server/config.go`,
`cmd/ingestor/config.go`): added `GeoFilterConfig` struct so
`geo_filter.polygon` and `bufferKm` from `config.json` are parsed by
both the server and ingestor
- **Ingestor** (`cmd/ingestor/geo_filter.go`, `cmd/ingestor/main.go`):
ADVERT packets from nodes outside the configured polygon + buffer are
dropped *before* any DB write — no transmission, node, or observation
data is stored
- **Server API** (`cmd/server/geo_filter.go`, `cmd/server/routes.go`):
`GET /api/config/geo-filter` endpoint returns the polygon + bufferKm to
the frontend; `/api/nodes` responses filter out any out-of-area nodes
already in the DB
- **Frontend** (`public/map.js`, `public/live.js`): blue polygon overlay
(solid inner + dashed buffer zone) on Map and Live pages, toggled via
"Mesh live area" checkbox, state shared via localStorage
### Automatic DB pruning
- Add `retention.packetDays` to `config.json` to delete transmissions +
observations older than N days on a daily schedule (1 min after startup,
then every 24h). Nodes and observers are never pruned.
- `POST /api/admin/prune?days=N` for manual runs (requires `X-API-Key`
header if `apiKey` is set)
```json
"retention": {
"nodeDays": 7,
"packetDays": 30
}
```
### tools/geofilter-builder.html
Standalone HTML tool (no server needed) — open in browser, click to
place polygon points on a Leaflet map, set `bufferKm`, copy the
generated `geo_filter` JSON block into `config.json`.
### scripts/prune-nodes-outside-geo-filter.py
Utility script to clean existing out-of-area nodes from the database
(dry-run + confirm). Useful after first enabling geo_filter on a
populated DB.
### HB column in packets table
Shows the hop hash size in bytes (1–4) decoded from the path byte of
each packet's raw hex. Displayed as **HB** between Size and Type
columns, hidden on small screens.
## Test plan
- [x] ADVERT from node outside polygon is not stored (no new row in
nodes or transmissions)
- [x] `GET /api/config/geo-filter` returns polygon + bufferKm when
configured, `{polygon: null, bufferKm: 0}` when not
- [x] `/api/nodes` excludes nodes outside polygon even if present in DB
- [x] Map and Live pages show blue polygon overlay when configured;
checkbox toggles it
- [x] `retention.packetDays: 30` deletes old transmissions/observations
on startup and daily
- [x] `POST /api/admin/prune?days=30` returns `{deleted: N, days: 30}`
- [x] `tools/geofilter-builder.html` opens standalone, draws polygon,
copies valid JSON
- [x] HB column shows 1–4 for all packets in grouped and flat view
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
|
||
|
Kpa-clawbot
|
a555b68915 |
feat: expand frontend coverage collector for 60%+ target (#275)
## Summary
Expands the parallel frontend coverage collector to boost coverage from
~43% toward 60%+. This covers Phases 1 and 2 of the coverage improvement
plan.
### Phase 1 — Visit unvisited pages
- **Compare page** (`#/compare`): Navigates with query params selecting
two real observers from fixture DB, also exercises UI controls
- **Node analytics** (`#/nodes/{pubkey}/analytics`): Visits analytics
for two real nodes from fixture DB, clicks day buttons
- **Traces search** (`#/traces`): Searches for two valid packet hashes
from fixture DB
- **Personalized home**: Sets `localStorage.myNodes` with real pubkeys
before visiting `#/home`
- **Observer detail pages**: Direct navigation to
`#/observers/test-obs-1` and `#/observers/test-status-obs`
- **Real packet detail**: Navigates to `#/packets/b6b839cb61eead4a`
(real hash)
- **Rapid route transitions**: Exercises destroy/init cycles across all
pages
- **Compare in route list**: Added to the full route transition exercise
### Phase 2 — page.evaluate() for interactive code paths
| File | Functions exercised |
|------|-------------------|
| **live.js** | `vcrPause`, `vcrSpeedCycle`, `vcrReplayFromTs`,
`drawLcdText`, `vcrResumeLive`, `vcrUnpause`, `vcrRewind`,
`updateVCRClock`, `updateVCRLcd`, `updateVCRUI`, `bufferPacket`
(synthetic WS packets), `dbPacketToLive`, `renderPacketTree` |
| **packets.js** | `renderDecodedPacket` (ADVERT + GRP_TXT), `obsName`,
`renderPath`, `renderHop` |
| **packet-filter.js** | 30+ filter expressions now **evaluated against
4 synthetic packets** (previously only compiled, not run). Covers
`resolveField` for all field types including `payload.*` dot notation |
| **nodes.js** | `getStatusInfo`, `renderNodeBadges`,
`renderStatusExplanation`, `renderHashInconsistencyWarning` with varied
node types/roles |
| **roles.js** | `getHealthThresholds` (all roles), `getNodeStatus` (all
roles × active/stale), `getTileUrl`, `syncBadgeColors`, `miniMarkdown`
(bold, italic, code, links, lists), `copyToClipboard` |
| **channels.js** | `hashCode`, `getChannelColor`, `getSenderColor`,
`highlightMentions`, `formatSecondsAgo` |
| **app.js** | `escapeHtml`, `debouncedOnWS`, extended
`timeAgo`/`truncate` edge cases, extended
`routeTypeName`/`payloadTypeName`/`payloadTypeColor` ranges |
### What changed
- `scripts/collect-frontend-coverage.js` — +336 lines across existing
groups (no new groups added)
### Testing
- `npm test` passes (all 13 tests)
- No other files modified
Co-authored-by: you <you@example.com>
|
||
|
Kpa-clawbot
|
5777780fc8 |
refactor: parallel coverage collector (~30-60s vs 8min) (#272)
## Summary Redesigned frontend coverage collector with 7 parallel browser contexts. Coverage collector runs on master pushes only (skipped on PRs). ### Architecture 7 groups run simultaneously via `Promise.allSettled()`: - G1: Home + Customizer - G2: Nodes + Node Detail - G3: Packets + Packet Detail - G4: Map - G5: Analytics + Channels + Observers - G6: Live + Perf + Traces + Globals - G7: Utility functions (page.evaluate) ### Speed gains - `safeClick` 500ms → 100ms - `navHash` 150ms → 50ms - Removed redundant page visits and E2E-duplicate interactions - Wall time = slowest group (~30-60s estimated) ### 821 lines → ~450 lines Each group writes its own coverage JSON, nyc merges automatically. ### CI behavior - **PRs:** Coverage collector skipped (fast CI) - **Master:** Coverage collector runs (full synthetic user validation) Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbot
|
f438411a27 |
chore: remove deprecated Node.js backend (-11,291 lines) (#265)
## Summary Removes all deprecated Node.js backend server code. The Go server (`cmd/server/`) has been the production backend — the Node.js server was kept "just in case" but is no longer needed. ### Removed (19 files, -11,291 lines) **Backend server (6 files):** `server.js`, `db.js`, `decoder.js`, `server-helpers.js`, `packet-store.js`, `iata-coords.js` **Backend tests (9 files):** `test-decoder.js`, `test-decoder-spec.js`, `test-server-helpers.js`, `test-server-routes.js`, `test-packet-store.js`, `test-db.js`, `test-db-migration.js`, `test-regional-filter.js`, `test-regional-integration.js` **Backend tooling (4 files):** `tools/e2e-test.js`, `tools/frontend-test.js`, `benchmark.js`, `benchmark-ab.sh` ### Updated - `AGENTS.md` — Rewritten architecture section for Go, explicit deprecation warnings - `test-all.sh` — Only runs frontend tests - `package.json` — Updated test:unit - `scripts/validate.sh` — Removed Node.js server syntax check - `docker/supervisord.conf` — Points to Go binary ### NOT touched - `public/` (active frontend) ✅ - `test-e2e-playwright.js` (frontend E2E tests) ✅ - Frontend test files (`test-packet-filter.js`, `test-aging.js`, `test-frontend-helpers.js`) ✅ - `package.json` / Playwright deps ✅ ### Follow-up - Server-only npm deps (express, better-sqlite3, mqtt, ws, supertest) can be cleaned from package.json separately - `Dockerfile.node` can be removed separately --------- Co-authored-by: you <you@example.com> |
||
|
Kpa-clawbot
|
ec7ae19bb5 |
ci: restructure pipeline — sequential fail-fast, Go server E2E, remove deprecated JS tests (#256)
## Summary Complete CI pipeline restructure. Sequential fail-fast chain, E2E tests against Go server with real staging data, all deprecated Node.js server tests removed. ### Pipeline (PR): 1. **Go unit tests** — fail-fast, coverage + badges 2. **Playwright E2E** — against Go server with fixture DB, frontend coverage, fail-fast on first failure 3. **Docker build** — verify containers build ### Pipeline (master merge): Same chain + deploy to staging + badge publishing ### Removed: - All Node.js server-side unit tests (deprecated JS server) - `npm ci` / `npm run test` steps - JS server coverage collection (`COVERAGE=1 node server.js`) - Changed-files detection logic - Docs-only CI skip logic - Cancel-workflow API hacks ### Added: - `test-fixtures/e2e-fixture.db` — real data from staging (200 nodes, 31 observers, 500 packets) - `scripts/capture-fixture.sh` — refresh fixture from staging API - Go server launches with `-port 13581 -db test-fixtures/e2e-fixture.db -public public-instrumented` --------- Co-authored-by: Kpa-clawbot <kpabap+clawdbot@gmail.com> Co-authored-by: you <you@example.com> |
||
|
you
|
13cab9bede |
perf: optimize frontend coverage collector (~2x faster)
Three optimizations to reduce wall-clock time: 1. Reduce safeClick timeout from 3000ms to 500ms - Elements either exist immediately after navigation or don't exist at all - ~75 safeClick calls; if ~30 miss, saves ~75s of dead wait time 2. Replace 18 page.goto() calls with SPA hash navigation - After initial page load, the SPA shell is already in the DOM - page.goto() reloads the entire page (network round-trip + parse) - Hash navigation via location.hash triggers the SPA router instantly - Only 3 page.goto() remain: initial load + 2 home page loads after localStorage.clear() 3. Remove redundant final route sweep - All 10 routes were already visited during the page-specific sections - The sweep just re-navigated to pages that had already been exercised - Saves ~2s of redundant navigation Also: - Reduce inter-route wait from 200ms to 50ms (SPA router is synchronous) - Merge utility function + packet filter exercises into single evaluate() call - Use navHash() helper for consistent hash navigation with 150ms settle time |
||
|
you
|
12d1174e39 |
perf: speed up frontend coverage tests (~3x faster)
Three optimizations to the CI frontend test pipeline:
1. Run E2E tests and coverage collection concurrently
- Previously sequential (E2E ~1.5min, then coverage ~5.75min)
- Now both run in parallel against the same instrumented server
- Expected savings: ~5 min (coverage runs alongside E2E instead of after)
2. Replace networkidle with domcontentloaded in coverage collector
- SPA uses hash routing — networkidle waits 500ms for network silence
on every navigation, adding ~10-15s of dead time across 23 navigations
- domcontentloaded fires immediately once HTML is parsed; JS initializes
the route handler synchronously
- For in-page hash changes, use 200ms setTimeout instead of
waitForLoadState (which would never re-fire for same-document nav)
3. Extract coverage from E2E tests too
- E2E tests already exercise the app against the instrumented server
- Now writes window.__coverage__ to .nyc_output/e2e-coverage.json
- nyc merges both coverage files for higher total coverage
Also:
- Split Playwright install into browser + deps steps (deps skip if present)
- Replace sleep 5 with health-check poll in quick E2E path
|
||
|
Kpa-clawbot
|
72161ba8fe |
perf: replace 169 blind sleeps with Playwright waits in coverage script
Remove all 169 waitForTimeout() calls (totaling 104.1s of blind sleeping)
from scripts/collect-frontend-coverage.js:
- Helper functions (safeClick, safeFill, safeSelect, clickAll, cycleSelect):
removed 300-400ms waits after every interaction — Playwright's built-in
actionability checks handle waiting for elements automatically
- Post-navigation waits: removed redundant sleeps after page.goto() calls
that already use waitUntil: 'networkidle'
- Hash-change navigations: replaced waitForTimeout with
waitForLoadState('networkidle') for proper SPA route settling
- Toggle/button waits: removed — event handlers execute synchronously
before click() resolves
- Post-evaluate waits: removed — evaluate() is synchronous
Local benchmark (Windows, sparse test data):
Before: 744.8s
After: 484.8s (35% faster, 260s saved)
On CI runner (ARM Linux with real mesh data), savings will be
proportionally better since most elements exist and the 104s
of blind sleeping was the dominant bottleneck.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
||
|
you
|
9b0c740537 |
revert: aggressive coverage interactions dropped score from 42% to 39%
The page.evaluate() calls corrupting localStorage and firing fake events caused page error-reloads, losing accumulated coverage. Reverting to the 42% version which was the actual high water mark. |
||
|
you
|
556e3b19db |
coverage: aggressive branch coverage push — target 80%+
Add ~900 lines of deep branch-coverage interactions: - Utility functions with all edge cases (timeAgo, truncate, escapeHtml, formatHex, etc.) - roles.js: getHealthThresholds/getNodeStatus for all roles + edge inputs - PacketFilter: compile+match with mock packets, all operators, bad expressions - HopResolver/HopDisplay: init, resolve, renderPath with various inputs - RegionFilter: onChange, getSelected, isEnabled, setRegions, render - Customize: deep tab cycling, import/export, bad JSON, theme preview - WebSocket reconnection trigger - Keyboard shortcuts (Ctrl+K, Meta+K, Escape) - Window resize (mobile/tablet/desktop) for responsive branches - Error routes: nonexistent nodes/packets/observers/channels - localStorage corruption to trigger catch branches - Theme toggling (dark/light rapid switching, custom vars) - Live page: VCR modes, timeline clicks, speed cycling, all toggles - Audio Lab: play/stop/loop, BPM/volume sliders, voice selection - All analytics tabs via deep-link + sort headers - Packets: complex filter expressions, scroll-to-load, double-click - Nodes: special char search, all sort columns, fav stars - Channels: resize handle drag, theme observer, node tooltips - Observers/Observer Detail: sort, tabs, day cycling - Node Analytics: day buttons, tabs - Home: both new/experienced flows, empty search results - debouncedOnWS/onWS/offWS exercise |
||
|
you
|
86dce6f350 |
coverage: massively expand frontend interaction coverage
Exercise every major code path across all frontend files: app.js: all routes, bad routes, hashchange, theme toggle x4, hamburger menu, favorites dropdown, global search, Ctrl+K, apiPerf(), timeAgo/truncate/routeTypeName utils nodes.js: sort every column (both directions), every role tab, every status filter, cycle all Last Heard options, click rows for side pane, navigate to detail page, copy URL, show all paths, node analytics day buttons (1/7/30/365), scroll target packets.js: 12 filter expressions including bad ones, cycle all time windows, group by hash toggle, My Nodes toggle, observer menu, type filter menu, hash input, node filter, observer sort, column toggle menu, hex hash toggle, pause button, resize handle, deep-link to packet hash map.js: all role checkboxes toggle, clusters/heatmap/neighbors/ hash labels toggles, cycle Last Heard, status filter buttons, jump buttons, markers, zoom controls, dark mode tile swap analytics.js: all 9 tabs clicked, deep-link to each tab via URL, observer selector on topology, navigate rows on collisions/ subpaths, sortable headers on nodes tab, region filter customize.js: all 5 tabs, all preset themes, branding text inputs, theme color inputs, node color inputs, type color inputs, reset buttons, home tab fields (hero, journey steps, checklist, links), export tab, reset preview/user theme live.js: VCR pause/speed/missed/prompt buttons, all visualization toggles (heat/ghost/realistic/favorites/matrix/rain), audio toggle + BPM slider, timeline click, resize event channels.js: click rows, navigate to specific channel observers.js: click rows, navigate to detail, cycle days select traces.js: click rows perf.js: refresh + reset buttons home.js: both chooser paths, search + suggest, my-node cards, health/packets buttons, remove buttons, toggle level, timeline Also exercises packet-filter parser and region-filter directly. |
||
|
you
|
26d4bbc39d | ci: fix coverage collector — use Playwright bundled chromium on CI | ||
|
you
|
5ee976055b |
feat(coverage): add targeted Playwright interactions for higher frontend coverage
Add redundant selectors (data-sort, data-role, data-status, data-tab, placeholder-based search, emoji theme toggle, .cust-close, #fTimeWindow, broader preset selectors) to exercise more frontend code paths. All interactions wrapped in try/catch for resilience. |
||
|
you
|
860d5c574e |
test: expanded frontend coverage collection with page interactions
367 lines of Playwright interactions covering nodes, packets, map, analytics, customizer, channels, live, home pages. Fixed e2e channels assertion (chList vs chResp.channels). |
||
|
you
|
d7faa4d978 |
Add frontend code coverage via Istanbul instrumentation + Playwright
- Install nyc for Istanbul instrumentation - Add scripts/instrument-frontend.sh to instrument public/*.js - Add scripts/collect-frontend-coverage.js to extract window.__coverage__ - Add scripts/combined-coverage.sh for combined server+frontend coverage - Make server.js serve public-instrumented/ when COVERAGE=1 is set - Add test:full-coverage npm script - Add public-instrumented/ and .nyc_output/ to .gitignore |
||
|
you
|
91a6a2c525 | fix: migration handles concurrent dual-write with INSERT OR IGNORE | ||
|
you
|
d7e415daa7 | fix: raw_hex NOT NULL in transmissions schema — deleted 4 junk test rows | ||
|
you
|
2c6148fd2d |
Add dedup migration script (Milestone 1)
Creates transmissions and observations tables from existing packets table. - Groups packets by hash → 1 transmission per unique hash - Creates 1 observation per original packet row with FK to transmission - Idempotent: drops and recreates new tables on each run - Does NOT modify the original packets table - Prints stats and verifies counts match Tested on test DB: 33813 packets → 11530 transmissions (2.93x dedup ratio) |
||
|
you
|
10b11106f6 |
ci: add pre-deploy JS validation — syntax check + undefined reference detection
Validation runs BEFORE docker build. If it fails, deployment is blocked. No more broken code reaching production. |