Commit Graph

2358 Commits

Author SHA1 Message Date
Kpa-clawbot 90b25a91ab ci: update frontend-coverage.json [skip ci] 2026-05-29 21:22:24 +00:00
Kpa-clawbot 44bab97d6c ci: update e2e-tests.json [skip ci] 2026-05-29 21:22:23 +00:00
Eric Muehlstein 788a509e73 refactor: move version/commit badge from navbar to Perf dashboard (#1503)
## Summary

The version/commit badge currently rendered in the nav stats bar
(alongside packet counts, node counts, and observer counts) is
operator-facing diagnostic information — not something end users need
visible on every page load. For most visitors, it adds visual noise
without adding value.

## Changes

- **perf.js**: Add a **Version** card to the Perf dashboard overview
row. Shows `version` + short `commit` hash, both already available from
`/api/health` (no new API surface needed). Card renders conditionally —
if neither field is set it stays hidden.
- **app.js**: Remove `formatVersionBadge()` and `formatEngineBadge()`
helper functions (now unused); strip the badge call from
`updateNavStats()` so the navbar shows only packet/node/observer counts.
- **style.css**: Remove now-dead `.nav-stats .version-badge`,
`.nav-stats .engine-badge`, and their link sub-rules.

## Rationale

The Perf page is explicitly the right place for this information — it's
already scoped to operators and developers who want to know what version
is running. The navbar is a high-visibility surface shared by all users;
version strings belong in a diagnostic context, not a navigation bar.

Net result: navbar is cleaner for end users; operators can still find
version info immediately on the Perf tab.
2026-05-29 14:03:03 -07:00
Kpa-clawbot e0fea3fe1d ci: update go-server-coverage.json [skip ci] 2026-05-29 17:35:51 +00:00
Kpa-clawbot 1d4840221d ci: update go-ingestor-coverage.json [skip ci] 2026-05-29 17:35:50 +00:00
Kpa-clawbot 642f947d6b ci: update frontend-tests.json [skip ci] 2026-05-29 17:35:49 +00:00
Kpa-clawbot 1a91982e01 ci: update frontend-coverage.json [skip ci] 2026-05-29 17:35:48 +00:00
Kpa-clawbot 47aa5dfe5c ci: update e2e-tests.json [skip ci] 2026-05-29 17:35:47 +00:00
Kpa-clawbot 9bed0e85ed fix(#1496): Reset All clears every customizer-touched state (#1497)
## Summary

`🗑️ Reset All Customizations` only stripped `cs-theme-overrides`,
leaving CB-preset, encrypted-channel toggle, dark-tile pick,
marker-stroke vars and the per-role `--mc-role-*` body.style writes from
PRs #1361/#1430/#1448/#1454/#1488 stuck. Operators had to clear
localStorage by hand to actually reset.

Single source of truth lands as `_resetAll()` in
`public/customize-v2.js` (exposed on `_customizerV2.resetAll` for
tests). The Reset button delegates to it. Future customizer features
extend ONE function — not 12 scattered call-sites.

## What is cleared

| surface | keys / props |
|---|---|
| localStorage | `cs-theme-overrides`, `meshcore-cb-preset`,
`channels-show-encrypted`, `mc-dark-tile-provider` |
| body attr | `data-cb-preset` |
| body.style | `--mc-role-{role}`, `--mc-role-{role}-text` for
repeater/companion/room/sensor/observer |
| :root style | `--mc-role-*`, `--mc-role-*-text`, `--node-*`,
`--mc-marker-stroke-{color,width,opacity}`,
`--mc-mb-{confirmed,suspected,unknown}`, `--mc-rt-ramp-{0..4}`,
`--logo-accent`, `--logo-accent-hi`, every value in `THEME_CSS_MAP` |

CB-preset teardown delegates to `MeshCorePresets.clearPreset()` so
`cb-preset-changed` fires and downstream consumers re-sync to server
config without a reload. Tile-provider teardown re-applies the active id
(which now falls through to server default / `carto-dark`) so
`mc-tile-provider-changed` fires and the live map swaps tiles, then
re-clears the just-rewritten localStorage entry.

## What is explicitly preserved (per issue body)

- `meshcore-theme` — separate user preference, not a customization
- `meshcore-gesture-hints-*` — has its own dedicated Reset button
- `meshcore-favorites` — operator's favorites list, not a customizer
pick
- `mc-channels-*` — channel selection state, not a customization

## TDD

- Red commit (`7a986fce`): adds `test-issue-1496-reset-all-complete.js`
+ a stub `resetAll: function () {}` so the test fails on assertions (9
of 14), not on a missing symbol. The 5 "must NOT clear" assertions pass
trivially against the stub.
- Green commit (`45c88154`): wires `_resetAll()`; all 14 pass.

```
14 passed, 0 failed
```

Existing customizer tests (`test-customize-display-e2e.js` shape only;
`test-issue-1361-cb-presets.js` 82/82;
`test-issue-1412-customizer-no-override.js` 13/13) unaffected. Two
pre-existing failures in `test-customizer-v2.js` and
`test-issue-1438-customizer-mcrole.js` reproduce on `origin/master`
without this change.

Closes #1496

---------

Co-authored-by: mc-bot <bot@meshcore.local>
2026-05-29 10:01:24 -07:00
Kpa-clawbot 2196102eae test(channels): skip processWSBatch-explicit-sender step pending #1498 root-cause (#1502)
Master CI failing across all recent PRs due to this single test. The
#1499 find-by-hash fix didn't resolve it — root cause is deeper than the
index-vs-hash race (possibly closure staleness on
`_channelsProcessWSBatchForTest` vs `_channelsGetStateForTest`).

Skipping to unblock master per operator directive. Filed #1498 for
proper diagnosis with CDP repro.

Co-authored-by: mc-bot <bot@meshcore.local>
2026-05-29 17:01:00 +00:00
Kpa-clawbot ebdf51ba54 fix(test): channels-ws-batch race — find injected message by hash not index (#1499)
## Why master CI keeps failing

Real WS messages from the staging ingestor race with the test's
synthetic injection. messages.length jumps prev+2 instead of prev+1, and
messages[length-1] is some XMD packet instead of the synthetic WsAlice —
assertion fails.

Failure log:
```
✗ processWSBatch with explicit sender appends to messages: expected sender WsAlice, got XMD Tag 1
```

Started flaking ~v3.8.2-track when test timing shifted. Test was
authored in #1300.

## Fix

Find injected message by its synthetic hash:
```js
s.messages.find((m) => m.hash === 'wsbatch-explicit-1' || m.id === 'pkt-wsbatch-1')
```

Race-immune regardless of real WS noise. Unblocks master CI.

Co-authored-by: mc-bot <bot@meshcore.local>
2026-05-29 16:04:54 +00:00
Kpa-clawbot 6af82cf3a8 fix(test): #1487 E2E desktop-only (BYOP button hidden on mobile per #1471) (#1495)
## Why CI was failing on master

PR #1493 (BYOP modal fix for #1487) shipped an E2E test that runs at
BOTH 390×844 mobile + 1280×800 desktop. The test calls
`waitForSelector('[data-action=pkt-byop]')` which defaults to `state:
visible`.

But #1471 mobile UX rules explicitly hide BYOP on mobile:
`#pktLeft .page-header [data-action="pkt-byop"] { display: none
!important }`

So the test times out on the mobile pass, breaking master CI on every
commit since c841dbcc.

## Fix
Drop the mobile viewport from the test loop. Reporter (@EldoonNemar)'s
bug was on desktop — that's where we test.

If BYOP ever gets surfaced on mobile, re-enable the mobile pass.

Co-authored-by: mc-bot <bot@meshcore.local>
2026-05-29 15:39:19 +00:00
Kpa-clawbot 7fcb226cd8 fix(#1486): collapse chevron no longer reopens closed detail panel (#1492)
## Summary

Fixes #1486 — clicking the collapse chevron on a grouped packet row in
the packets table no longer reopens the detail panel that the operator
just closed.

## Root cause

In the `#pktBody` row click handler the `toggle-select` action ran
**both** `pktToggleGroup(value)` and `pktSelectHash(value)` on every
chevron click. `pktToggleGroup()` already opens the detail panel itself
(via `selectPacket()`) when it expands a row, so the trailing
`pktSelectHash()` was:

  - redundant on **expand** (the panel was already opening), and
  - harmful on **collapse** — after the operator closed the detail panel
    via the ✕ in `#pktRight`, clicking the same chevron a second time
    to collapse the tree re-fetched `/packets/<hash>` and re-populated
    the panel with the same packet, exactly the behavior the issue
    describes.

## Fix

Drop the unconditional `pktSelectHash(value)` call inside the
`toggle-select` branch. `pktToggleGroup()` already handles the
expand-side panel open; the collapse branch does no panel work, so a
closed panel stays closed.

```js
else if (action === 'toggle-select') {
  // #1486: pktToggleGroup() already opens the detail panel on EXPAND
  // (via selectPacket()), and must NOT open it on COLLAPSE.
  pktToggleGroup(value);
}
```

## Tests

- New Playwright E2E `test-issue-1486-collapse-reopens-detail-e2e.js`
  walks the operator-visible repro: expand → assert panel open →
  click ✕ → assert panel empty → click chevron again → assert row
  collapsed AND panel STILL empty.
- Committed red-first: the test was added in its own commit and FAILS
  on the unpatched code (3 passed / 1 failed), then GREEN on the fix
  commit (4 passed / 0 failed).
- CI workflow seeds two extra observations onto the newest fixture
  transmission so a grouped (`toggle-select`) row exists; without this
  the fixture renders only flat rows and the chevron can't be
  exercised.

## Reproduction (manual, against staging or local)

1. Open `/#/packets` on desktop.
2. Click a grouped row's `▶` chevron — the tree expands and the detail
   panel opens on the right.
3. Click the `✕` in the top-right of the detail panel — panel goes back
   to "Select a packet to view details".
4. Click the same chevron (now `▼`) again — **before:** detail panel
   reopens with the same packet. **After:** the row collapses and the
   panel stays empty.

---------

Co-authored-by: mc-bot <bot@meshcore.local>
2026-05-29 15:17:16 +00:00
Kpa-clawbot 268751ff56 fix(#1485): put live map animations on custom pane z=650 (above markerPane) (#1491)
## Summary

Animations on the live map (packet pulses, hop-to-hop trails,
drawAnimatedLine, pulseNode rings, matrix chars) render BEHIND the node
base layer — community-confirmed by @EldoonNemar in #1485 after pulling
latest and rebuilding. The live map looks completely static because
every node marker paints on top of moving packets.

Closes #1485

## Root cause

PR #1334 ("role-aware marker shapes + outline-ring highlight") swapped
node markers:

- **Before:** `L.circleMarker([n.lat, n.lon], {...})` — rendered into
the default Leaflet `overlayPane` (z=400) alongside other vector shapes.
- **After:** `L.marker([n.lat, n.lon], { icon: L.divIcon({...}) })` —
rendered into the default Leaflet `markerPane` (z=600).

`animLayer` and `pathsLayer` (built from `L.polyline` / `L.circleMarker`
shapes) still default to `overlayPane` @ 400. With nodes now in pane
600, every node marker occluded every animation. CDP confirmed pre-fix:

```
overlayPane z=400  (animations live here)  ← 2 children
markerPane  z=600  (nodes live here)        ← 516 children  ← occludes
```

## Fix

Create a custom Leaflet pane `liveAnimPane` at `z-index: 650` (strictly
above markerPane) and pin both `animLayer` and `pathsLayer` to it via
the `{ pane: 'liveAnimPane' }` option on `L.layerGroup`. Polylines +
circleMarkers added to those groups inherit the pane from their parent,
so all `drawAnimatedLine` / `pulseNode` / `animatePath` / matrix-char
shapes now paint above markers.

`pointerEvents: 'none'` on the pane so it does not steal hover/click
events from the markerPane beneath (`clickablePathsLayer` keeps the
default overlayPane and continues to handle path clicks).

Diff is +14 / -2 in `public/live.js`. No CSS changes, no API changes, no
protocol changes.

## TDD

Red commit (`b7ca794f`): test asserts on `public/live.js` source —
1. `map.createPane('liveAnimPane')` is called in init
2. that pane is assigned `style.zIndex` ≥ 650 (strictly above markerPane
@ 600)
3. `animLayer` AND `pathsLayer` are constructed with `{ pane:
'liveAnimPane' }`
4. (sanity) animLayer still hosts ≥3 animation shapes, pathsLayer ≥3
trail shapes — regression detector if someone moves circles to the
default pane.

CI must fail on `b7ca794f` (RED). Fix lands in `627ce341` (GREEN). Test
reruns 5× clean — non-flaky (source invariants).

## Browser verified

Local headless chromium (CDP) against
`http://analyzer-stg.00id.net/#/live`:

- **Before fix:** overlayPane z=400 (2 anim children), markerPane z=600
(516 marker children) — animations buried.
- **After hot-deploy:** liveAnimPane z=650 above markerPane z=600 —
animations visible on top. Will attach screenshot post-merge once
staging redeploys.

E2E assertion added: `test-issue-1485-live-anim-z.js:54` (`liveAnimPane
z-index >= 650`).

## Test wiring

`test-all.sh` line 51 added; CI runs the new test alongside the existing
1418/1420/1438/1470 suite.

## Credit

Reported by @EldoonNemar in #1485 — pulled via git, built the docker
image, noticed the regression same day. Bug-report quality was excellent
(concise repro: "live map now shows the animated packets behind the node
base layer so you can't actually see the nodes moving").

---------

Co-authored-by: mc-bot <bot@meshcore.local>
2026-05-29 07:46:00 -07:00
Kpa-clawbot ca2c3d6c79 feat(1488): customize marker stroke (color, width, opacity) (#1494)
## Summary

Reporter (@EldoonNemar in #1488) found the new white marker stroke
overwhelming with hundreds of nodes on screen. This PR exposes the
stroke through CSS vars + a customizer panel so operators can dial
color/width/opacity (or remove it) without code edits.

**Scope:** ship stroke customization only. The reporter also asked for
the old glow-style highlight ring as an alternative — that's a separate
visual feature that needs design discussion, so it's deferred to a
follow-up issue.

## Changes

- **`public/style.css`** `:root` declares `--mc-marker-stroke-color` /
`--mc-marker-stroke-width` / `--mc-marker-stroke-opacity` with sensible
defaults (white, 1, 1) that match current behavior.
- **`public/roles.js`** `makeRoleMarkerSVG` — replaced the 6 baked
`stroke="#fff" stroke-width="1"` literals with a single shared
`strokeAttr` referencing the CSS vars. One source of truth for all role
shapes.
- **`public/map.js`** `makeMarkerIcon` — same migration. The observer
star overlay keeps its narrow 0.8 width but routes color + opacity
through the same vars.
- **`public/live.js`** `addNodeMarker` fallback SVG — same migration.
- **`public/customize-v2.js`** — new `markerStroke` object section
(color/width/opacity) with validation, `applyCSS` writes, three controls
on the Colors tab → "Marker Stroke" panel (color picker + width slider
0–4 + opacity slider 0–100%). Optimistic CSS-var writes on the `input`
event so markers repaint live as the operator drags.
- **`cmd/server/{config,types,routes}.go`** — `ThemeFile` / `Config` /
`ThemeResponse` pick up `MarkerStroke` so `theme.json` and `config.json`
can ship server-side defaults. Defaults mirror the `:root` CSS values so
no breaking change for current operators.
- **`config.example.json`** — documented `markerStroke` section with
usage hint.

## TDD

- **Red commit** `92183f95` — `test-issue-1488-marker-stroke-vars.js` (5
sections, 18 assertions); failed 14/18 before implementation.
- **Green commit** `ce39637e` — implementation; same test now passes
18/18.
- Existing `#1438` (marker CSS-var migration) and `#1293` (marker
shapes) regression tests still pass.
- Go tests (`cmd/server/...`) all green.

## CDP validation

Synthetic page with 600 markers, three blocks proving CSS-var control
works end-to-end:

| Block | Stroke setting | Computed `getComputedStyle().stroke` / width
/ opacity |
| --- | --- | --- |
| Default | `var(--mc-marker-stroke-color)` (no override) |
`rgba(255,255,255,0.85)` / `1px` / `1` |
| Tuned | inline `--mc-marker-stroke-*` (operator override) |
`rgb(255,255,255)` / `0.5px` / `0.3` |
| Cyan | inline `--mc-marker-stroke-*` (branding/CB) | `rgb(0,229,255)`
/ `2px` / `1` |

Same SVG source, three different rendered strokes — that's the whole
point. Runtime `documentElement.style.setProperty(...)` (which is
exactly what the customizer slider's `input` handler does) repaints
mounted markers without reload. CDP screenshot attached to the
implementation note.

## Hot-deploy

Frontend + Go binary changes. Safe to hot-deploy frontend files
(`public/*.js`, `public/style.css`) via the standard staging path; Go
binary update needs a container restart.

## Defer

Glow highlight ring (the second half of #1488) — separate follow-up
issue. This PR delivers the immediately-useful, smaller deliverable.

Partial fix for #1488 (stroke customization shipped; glow ring deferred
to a follow-up issue).

---------

Co-authored-by: meshcore-bot <bot@meshcore.local>
2026-05-29 14:31:36 +00:00
Kpa-clawbot c841dbccdd fix(#1487): BYOP modal — bounded header, no body occlusion (#1493)
## Fixes #1487

Reporter (@EldoonNemar): "The dialog text can't be seen due to the title
bar being massive."

### Root cause
`.byop-header` swelled to ~73px on mobile because:
1. `position: sticky` + `margin: -24px -24px 12px` assumed `.modal`
desktop padding (24px) — but `.modal` switches to 16px padding at
mobile, so the sibling-margin pushed the description paragraph UP into
the sticky-pinned header band, occluding it.
2. `.btn-icon` close button floors at 48×48 (touch target) → forced
header height ≥48px+padding.
3. H3 inherited a default emoji line-height that added more height on
platforms with tall emoji ascent metrics.

### Fix (`public/style.css`)
- Drop full-bleed negative-margin gymnastics — header uses normal
in-flow padding (`4px 0`); `.modal` padding handles inset.
- `max-height: 48px` on header so emoji ascent / btn-icon floor can't
blow it past safe range.
- Bound H3 explicitly (`font-size: 1rem; line-height: 1.3`).
- Override `.byop-x` to compact 32px visual size; preserve ≥44px
effective tap target via invisible `::before` pad (a11y safe).

### Verification
Hot-swapped onto staging, CDP-measured both viewports:

| viewport | hdrH | descTop ≥ hdrBottom | result |
|---|---|---|---|
| 390×844 mobile | 41px (was 73) | 341 ≥ 329  | clean |
| 1280×800 desktop | 41px | 318 ≥ 306  | clean |

### TDD
- **Red commit**: bb1a9f48 — `test-issue-1487-byop-modal-layout-e2e.js`
asserts header ≤56px AND description top ≥ header bottom on both
viewports. Pre-fix: header=73px ⇒ FAIL.
- **Green commit**: 72a69b3e — CSS fix; assertions all pass against
hot-swapped staging.
- E2E added: `test-issue-1487-byop-modal-layout-e2e.js`; wired into
`.github/workflows/deploy.yml` e2e job.

### Screenshots
Before (mobile): description "Paste raw hex bytes..." clipped by
oversized header. After: header 41px, description fully visible above
textarea.

---------

Co-authored-by: corescope-bot <bot@corescope.local>
2026-05-29 14:29:57 +00:00
Kpa-clawbot 022f3d8f0d ci: update go-server-coverage.json [skip ci] 2026-05-29 12:04:33 +00:00
Kpa-clawbot c0ca47e9be ci: update go-ingestor-coverage.json [skip ci] 2026-05-29 12:04:32 +00:00
Kpa-clawbot 3f97d2eca2 ci: update frontend-tests.json [skip ci] 2026-05-29 12:04:31 +00:00
Kpa-clawbot 13c35c0aa8 ci: update frontend-coverage.json [skip ci] 2026-05-29 12:04:30 +00:00
Kpa-clawbot ec1525e195 ci: update e2e-tests.json [skip ci] 2026-05-29 12:04:29 +00:00
Kpa-clawbot d837166158 test(coverage): add Playwright E2E for channels page (#1297 B3) (#1300)
## #1297 B3 — Playwright E2E coverage for `public/channels.js`

Pure-coverage PR. Adds five Playwright suites targeting the largest
under-tested branches of `public/channels.js` (1950 LOC, was **19.9%
statements** per the live coverage refinement in #1297 — the single
biggest delta opportunity in the umbrella). No production code changes.

### Coverage exemption

Per repo `AGENTS.md` TDD rule: this is the **net-new test coverage**
case — there is no production change to gate, so a failing-then-passing
red commit isn't applicable. All five suites exercise existing channels
init() code paths that ship today.

### New test files

| File | Scenarios exercised |
| --- | --- |
| `test-channels-list-render-e2e.js` | Sectioned sidebar (My Channels /
Network / Encrypted) headers, encrypted collapse toggle + localStorage
persistence, row badges + previews, color dot + color clear control,
sidebar resize handle width persist |
| `test-channels-selection-flow-e2e.js` | `selectChannel()` header
update + URL replaceState, message row rendering (avatars, sender
colors, packet links), node detail panel open via mouse + keyboard +
close-with-focus-restore, deep-link route restoration, scroll button
initial state |
| `test-channels-add-modal-e2e.js` | Generate PSK Channel (key + QR +
status banner + localStorage persist), Add PSK invalid hex error path,
Add PSK valid hex success + close + My Channels row, Monitor Hashtag
with and without leading `#`, empty-hashtag no-op, Scan QR unavailable
fallback, Escape close, Remove ✕ flow |
| `test-channels-share-color-e2e.js` | Share modal normal mode
(dedicated `#chShareModal` with QR + Hex Key + Copy success label),
Share modal error mode (`openShareModalError` when no stored key — field
groups hidden), Escape close, `ChannelColorPicker.show` invocation on
color-dot click, keyboard Enter on a `[data-share-channel]` span |
| `test-channels-ws-batch-e2e.js` | `processWSBatch` via
`_channelsProcessWSBatchForTest`: explicit-sender append, `"Sender:
text"` parsing branch, packetHash dedup + observer accumulation,
new-channel append (channel previously unseen), scroll-button branch
when user not at bottom, region-filter exclusion code path |

All five tests wired into `.github/workflows/deploy.yml` after the
existing `test-channel-fluid-e2e.js` step.

### Preflight

`bash ~/.openclaw/skills/pr-preflight/scripts/run-all.sh origin/master`
→
exit 0, all gates pass (PII, CSS vars, branch scope, etc.).

Refs #1297

---------

Co-authored-by: openclaw-bot <openclaw-bot@users.noreply.github.com>
Co-authored-by: openclaw-bot <bot@openclaw.local>
Co-authored-by: mc-bot <bot@meshcore.local>
2026-05-29 11:46:51 +00:00
Kpa-clawbot c4576d547f ci: update go-server-coverage.json [skip ci] 2026-05-29 10:03:17 +00:00
Kpa-clawbot ed18702061 ci: update go-ingestor-coverage.json [skip ci] 2026-05-29 10:03:15 +00:00
Kpa-clawbot a0dccd943c ci: update frontend-tests.json [skip ci] 2026-05-29 10:03:14 +00:00
Kpa-clawbot 5e939a3647 ci: update frontend-coverage.json [skip ci] 2026-05-29 10:03:13 +00:00
Kpa-clawbot 7e9a3c5e85 ci: update e2e-tests.json [skip ci] 2026-05-29 10:03:12 +00:00
Kpa-clawbot 13bdee57d4 perf: P0 hot-path fixes (observers, neighbor-graph, observer-analytics) (#1481) (#1483)
## What

Three of the four P0s from #1481's scale-test findings. Each cuts a
distinct
hot path; together they target /api/observers,
/api/analytics/neighbor-graph,
and /api/observers/{id}/analytics — the top three live offenders.

### P0-1: 5-min atomic-pointer cache for default neighbor-graph response
- Live p95 10.8s on the most-trafficked organic endpoint.
- Background recomputer (5-min cadence per operator directive) builds
the
  default-filter (`minCount=5 minScore=0.1`, no region, no role)
  `NeighborGraphResponse` and stores it via `atomic.Pointer`.
- `handleNeighborGraph` short-circuits on the default shape; non-default
filters take the extracted `computeNeighborGraphResponse` path
(identical
  semantics to the previous inline build).

### P0-2: cache parsed `StoreObs.Timestamp` + drop RLock window
- `handleObserverAnalytics` re-parsed the RFC3339 timestamp three times
  per observation, for 60k+ observations per active observer, under
  `s.store.mu.RLock` — blocking writers for the full scan.
- `StoreObs.ParsedTime()` parses once via `sync.Once` (mirrors
  `StoreTx.ParsedDecoded`).
- Handler snapshots the `byObserver[id]` pointer slice, releases the
  RLock immediately, then iterates locally.

### P0-3: 30s cache for `/api/observers` + sargable `IN` + covering
index
- Three SQL queries on every request → ~1.7s p50 at 50-concurrent.
- Atomic-pointer 30s cache for the default (no-filter) query.
- `GetNodeLocationsByKeys` drops `LOWER(public_key) IN (...)`
(non-sargable);
  callers pre-lowercase in Go and the plain `IN` matches the existing
  `public_key` index.
- New ingestor migration `obs_observer_ts_idx_v1` adds composite index
  `idx_observations_observer_idx_timestamp(observer_idx, timestamp)` so
  `GetObserverPacketCounts` can resolve its GROUP-BY + range filter from
  the index without scanning the 1.9M-row observations table.

### P0-4: deferred
`perfMiddleware`'s global mutex was claimed to serialize every API
request.
A direct test (`50 concurrent requests through the middleware, handler
sleeps 20ms each`) shows total elapsed ≈ 25ms, not 1s — the lock is held
only for the post-handler bookkeeping (a few µs). Real impact is below
measurement noise. Skipping to avoid invasive churn on PerfStats
consumers
without a demonstrable win.

## Test plan

Red → green per P0:
- `observers_cache_test.go` — handler reads `s.observersCache` before
SQL,
  TTL boundary, atomic.Pointer (no mutex contention).
- `storeobs_parsedtime_test.go` — parses three timestamp shapes, caches
  result, no race under concurrent readers.
- `neighbor_graph_cache_test.go` — handler serves from atomic pointer
  when set, bypasses cache when `?region=` (or any non-default filter)
  is passed.

Full server + ingestor suites pass: `go test -count=1 ./...`.

## Perf proof

Before/after p50/p95/p99 (50 requests × 50 concurrent) against prod
(before)
and staging once CI deploys (after) will be posted as a PR comment per
the
operator's "no merge without proof of improvement" gate.

Closes #1481


## TDD exemption — P0-1 and P0-2 (net-new surfaces, AGENTS.md)

Per CoreScope `AGENTS.md` § "Exemptions": **net-new code surfaces with
no
prior tests to break** may land tests in the same PR without a strict
test-first → impl commit split.

- **P0-1 (neighbor-graph atomic-pointer cache)** — `neighborGraphCache`,
  `recomputeNeighborGraphCache`, `loadNeighborGraphCacheBytes`,
  `startNeighborGraphRecomputer` and the default-shape short-circuit in
  `handleNeighborGraph` were brand-new code with no pre-existing
  assertions covering them. There was no green test to first turn red.
- **P0-2 (cached `StoreObs.Timestamp` + RLock window drop)** —
  `StoreObs.ParsedTime()` and the snapshot+release pattern in
  `handleObserverAnalytics` were new surfaces; the prior code did the
  parse inline per call with no behavioural test to break.

P0-3 was authored properly red-then-green (commit `6e63ec6a` red, then
`83ae129b` green) and does NOT use this exemption.

## Default-filter detection vs frontend reality (#1483 follow-up)

The Neighbor Graph analytics tab in `public/analytics.js` fetches
`/analytics/neighbor-graph?min_count=1&min_score=0` because the
client-side sliders need the full edge set to filter from. That shape
did NOT match the `(5, 0.1)` cached default, so the UI tab still paid
the cold compute cost despite #1481 P0-1.

The #1483 follow-up commit caches BOTH shapes in the same recomputer
pass:
- `(minCount=5, minScore=0.1, no region, no role)` — `live.js`
  affinity-scoring consumer.
- `(minCount=1, minScore=0, no region, no role)` — analytics tab.

Both are served from `atomic.Pointer` with an `X-Cache-Age-Seconds`
header. The per-shape cost in the background goroutine is roughly
linear in edge count; total recompute time stays well under the
5-minute cadence on prod-scale graphs.

---------

Co-authored-by: openclaw-bot <bot@openclaw.dev>
Co-authored-by: mc-bot <mc-bot@users.noreply.github.com>
2026-05-29 02:42:21 -07:00
Kpa-clawbot 544c36d60e ci: update go-server-coverage.json [skip ci] 2026-05-29 08:39:58 +00:00
Kpa-clawbot d2d59566f6 ci: update go-ingestor-coverage.json [skip ci] 2026-05-29 08:39:57 +00:00
Kpa-clawbot 2c32813f45 ci: update frontend-tests.json [skip ci] 2026-05-29 08:39:56 +00:00
Kpa-clawbot e2edd6e284 ci: update frontend-coverage.json [skip ci] 2026-05-29 08:39:55 +00:00
Kpa-clawbot df08cb537a ci: update e2e-tests.json [skip ci] 2026-05-29 08:39:54 +00:00
Kpa-clawbot 139fc5e6a3 ci: update go-server-coverage.json [skip ci] 2026-05-29 08:19:36 +00:00
Kpa-clawbot 08893bc566 ci: update go-ingestor-coverage.json [skip ci] 2026-05-29 08:19:35 +00:00
Kpa-clawbot f68fb2208e ci: update frontend-tests.json [skip ci] 2026-05-29 08:19:34 +00:00
Kpa-clawbot 5ccb9201e4 ci: update frontend-coverage.json [skip ci] 2026-05-29 08:19:33 +00:00
Kpa-clawbot c80637ddb9 ci: update e2e-tests.json [skip ci] 2026-05-29 08:19:31 +00:00
Kpa-clawbot 43b93c6bb9 feat(observers): surface naive-clock observers as ⚠️ chip + detail banner (#1478) (#1480)
## Summary

Issue #1478 — surface observers whose envelope timestamps are being
clamped because they're emitting zone-less local-time strings (UTC-N
observers showed up perpetually as "Stale" before #1466, and per-packet
rxTime is still clamped to ingest time for them, muddying
propagation-delay analytics).

Now the UI tells operators which observers are misconfigured + how to
fix it.

## What changed

### Ingestor (cmd/ingestor)
- New `observers_clock_naive_v1` migration adds three columns to
`observers`:
- `clock_skew_seconds INTEGER` (signed: negative = behind UTC, positive
= ahead)
  - `clock_skew_count_24h INTEGER` (rolling 24h event count)
  - `clock_last_naive_at TEXT` (RFC3339 timestamp of last clamp)
- `resolveRxTime` now returns `(rxTime, naiveSkewSec)`. The
packet-handler call site invokes `store.RecordNaiveSkew(observerID,
deltaSec)` whenever a naive envelope is clamped (the existing >15 min
naive-tolerance path). The counter resets to 1 if no event in the prior
24h, else increments. Single INSERT-or-UPDATE round trip per clamp.

### Server (cmd/server)
- `Observer` struct + `GetObservers` / `GetObserverByID` extended to
scan the three new columns.
- `ObserverResp` gains four JSON fields exposed by `/api/observers` and
`/api/observers/{id}`:
- `clock_naive` (bool, derived from `clock_last_naive_at` being within
24h)
  - `clock_skew_seconds`, `clock_skew_count_24h`, `clock_last_naive_at`
- Decay is **read-side**: a stale event yields `clock_naive=false` with
zero counts. No background sweep, no writes from the read-only server,
no race with the ingestor.

### Frontend (public)
- `window.ObserversNaiveChip.render(o)` — total render helper, returns
⚠️ chip HTML when `o.clock_naive===true`, `""` otherwise. Used inline in
the observers-list `name` cell and in the row-detail slide-over. Tooltip
explains magnitude + direction + count + fix.
- `window.ObserverDetailNaiveBanner.render(obs)` — yellow alert banner
at the top of the observer-detail page with the skew magnitude,
last-event timestamp, and the actionable fix ("Set host clock to UTC, OR
emit Z-suffixed/offset-aware timestamps from the observer script").

## TDD trail
- `5ddd5b42` red: backend `cmd/server/observer_naive_clock_1478_test.go`
(3 tests asserting JSON fields + 24h decay) + frontend
`test-observer-naive-clock-1478.js` (8 jsdom-style tests asserting
helpers exist and render correctly). Both failed on master with
field-missing / export-missing assertions.
- `4ecc79c8` green backend: schema + Observer / GetObservers /
ObserverResp / handler decay.
- `2137ab81` green frontend: chip + banner helpers and call sites.

## Tests
- `cd cmd/server && go test ./...` → all green (full suite, 46s)
- `cd cmd/ingestor && go test ./...` → all green (full suite, 98s)
- `node test-observer-naive-clock-1478.js` → 8/8 pass
- `node test-frontend-helpers.js` → unchanged from master (pre-existing
failures only)

## Acceptance (issue #1478)
-  Observer running with `python datetime.now().isoformat()` (naive,
off by N hours) → `clock_naive=true` after the next clamp → UI shows ⚠️
chip + banner.
-  Observer with `datetime.now(timezone.utc).isoformat()` (Z-suffixed)
→ never clamped → never flagged.
-  Observer that fixed its clock → `clock_naive` returns to `false` 24h
after the last clamp event (read-side decay).

Closes #1478.

---------

Co-authored-by: openclaw <bot@openclaw.local>
2026-05-29 01:08:12 -07:00
Kpa-clawbot 1fd95f6771 ci: update go-server-coverage.json [skip ci] 2026-05-29 07:57:46 +00:00
Kpa-clawbot f135e114f5 ci: update go-ingestor-coverage.json [skip ci] 2026-05-29 07:57:45 +00:00
Kpa-clawbot d9593df243 ci: update frontend-tests.json [skip ci] 2026-05-29 07:57:44 +00:00
Kpa-clawbot f1be30dc1f ci: update frontend-coverage.json [skip ci] 2026-05-29 07:57:43 +00:00
Kpa-clawbot 50bc073813 ci: update e2e-tests.json [skip ci] 2026-05-29 07:57:42 +00:00
efiten d4280befd4 fix(packets): use route-aware path byte offset for HB column (#1469)
## Summary

- The **HB** (hash bytes) column in the packet list always read byte 1
of `raw_hex` to compute the hash size
- For TRANSPORT routes (`route_type` 0 or 3), the path_len byte sits at
offset 5 — bytes 1–4 are transport codes
- Reading byte 1 for these packets produced the wrong hash size (e.g.
`0xBB` → bits 7-6 = `10` → **3** instead of the correct **2**)
- Fix: use `getPathLenOffset(route_type)` at all three render sites
(grouped header, grouped children, flat row)
- For grouped children that have no `raw_hex`, fall back to deriving
hash size from the path_json hop string lengths

## Test plan

- [ ] Open a TRANSPORT FLOOD packet (`route_type=0`) in the packet list
— HB column now shows the correct value (e.g. 2 instead of 3)
- [ ] Verify FLOOD packets (`route_type=1`) still show the correct hash
size (byte 1 unchanged for non-transport routes)
- [ ] Expand a grouped packet row and confirm child rows show correct
hash size from path_json hop lengths

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-29 00:52:53 -07:00
efiten b71b26a438 fix(live): decouple live animation from VCR.speed — always 1× in LIVE mode (#1427)
## Summary

- `drawAnimatedLine` and `drawMatrixLine` both used `33 / VCR.speed` and
`1100 / VCR.speed` as timing constants
- `VCR.speed` persists in localStorage, so a 4× or 8× replay setting
carried into live mode made packet animations run near-instantaneously
(8.25ms steps vs 33ms)
- Guard both constants behind `VCR.mode === 'REPLAY'` so live mode
always animates at the baseline rate regardless of saved speed

## Test plan

- [ ] Set replay speed to 4×, end replay, reload page → live animation
runs at normal speed (~660ms for a full hop animation)
- [ ] Verify replay still respects slow-mo: 0.25× is visibly slower, 4×
is faster
- [ ] Verify live animations are unaffected by the stored
`live-vcr-speed` localStorage value

Closes #1346

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-29 00:52:31 -07:00
efiten 8151185ede fix(ci): Dockerfile COPY invariant check — prevent missing internal/<pkg> Docker failures (#1316) (#1432)
## Summary

- Adds `scripts/check-dockerfile-internal-pkgs.sh`: reads `replace =>
../../internal/<pkg>` directives from `cmd/server/go.mod` and
`cmd/ingestor/go.mod`, then verifies each referenced package has the
correct number of `COPY internal/<pkg>/` lines in `Dockerfile` (one per
builder section that needs it)
- Wired into CI as a step in the `go-test` job, before CSS lint — runs
on every PR, adds ~0.1s
- Prevents the recurring failure pattern (#1316): new `internal/<pkg>`
added to go.mod but COPY line forgotten in Dockerfile; non-Docker CI
passes, Docker build fails after merge with a cryptic module error

Key details:
- Counts COPY occurrences per package: if a pkg is referenced in both
go.mods (both binaries need it), it must appear in at least 2 builder
sections
- Anchored regex: only matches actual `replace` directives (not
comments)
- Anchored grep: skips commented-out `COPY internal/...` lines

Closes #1316.

## Test plan

- [ ] Run `bash scripts/check-dockerfile-internal-pkgs.sh` locally —
exits 0 on current Dockerfile
- [ ] Manually remove a `COPY internal/perfio/` line from Dockerfile →
script exits 1 with a clear error
- [ ] CI step visible in the `go-test` job on this PR

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-29 00:52:08 -07:00
Kpa-clawbot e11ce54059 fix(#1480): update E2E #534 to click navbar mirror; simplify CSS (#1484)
Sequence of errors:
- #1475: hid in-page button with visibility:hidden \u2192 Playwright
won't click visibility:hidden \u2192 broke E2E #534
- #1482: tried opacity:0 instead \u2192 Playwright won't click opacity:0
either \u2192 still broken
- This PR: UPDATE THE TEST instead of fighting Playwright. The mobile UX
since #1471 is: operator-visible Filters control = navbar mirror
(.filter-toggle-btn-mirror). The test should click THAT, not the
now-hidden in-page button.

Test now tries the mirror first, falls back to in-page button for any
test rig without the mirror script. CSS simplified to display:none.

Unblocks #1480 (#1478 naive-TS observer UI surface) CI. Also any other
PR inheriting this same regression.

Hot-deploy candidate (CSS + test only).

Co-authored-by: openclaw-bot <bot@openclaw.local>
2026-05-29 07:38:20 +00:00
Kpa-clawbot b6e005009c fix(#1475 followup): opacity:0 not visibility:hidden so E2E #534 click works (#1482)
Regression I introduced in #1475. Playwright's elementHandle.click()
refuses to act on elements with visibility:hidden — the in-page Filters
button became unclickable, breaking E2E test #534 'Mobile filter toggle
expands filter bar on packets page'.

Caught by CI on #1480.

Switch to opacity:0 + 0×0 + position:absolute. Element renders zero
pixels for the user but stays 'visible' per Playwright's actionability
check — E2E #534 click works, no duplicate Filters button visible.

Hot-deploy candidate (CSS-only).

Co-authored-by: openclaw-bot <bot@openclaw.local>
2026-05-29 07:15:43 +00:00
hrtndev 462cb2cb5a chore: update MeshCore URLs to use new site (#1445)
# Summary
The main MeshCore website is https://meshcore.io. Reasons for the new
website are listed here: https://blog.meshcore.io/2026/04/23/the-split

# Changes
Any occurrence of `meshcore.co.uk` was replaced with `meshcore.io`. No
logic was changed, only updated strings.

Co-authored-by: hrtndev <hrtndev@users.noreply.github.com>
2026-05-29 00:06:29 -07:00