Commit Graph
9 Commits
Author SHA1 Message Date
b74a64ccfa fix(ui): canonical payload label map across packets/live/packet-filter (#1799) (#1804)
## Summary

Replaces the three drifted per-surface payload-type label vocabularies
with a single canonical map keyed by firmware enum name.

Per the locked triage comment on #1799
([comment-4823975431](https://github.com/Kpa-clawbot/CoreScope/issues/1799#issuecomment-4823975431)):

> Create `public/payload-labels.js` exporting `{GRP_DATA: {short:'Group
Data', long:'Group data packet', enumId:6}, ...}`. Migrate `packets.js
typeMap`, `packet-filter.js FW_PAYLOAD_TYPES`, `live.js TYPE_COLORS
legend` to consume it. E2E that scrapes each surface and asserts label
equality.

## Changes

- **`public/payload-labels.js`** (new) — canonical map exposed as
`window.PayloadLabels` and `window.PayloadLabelsApi`. Keys are firmware
enum names; values carry `{short, long, enumId}` plus derived
`SHORT_BY_ID` / `FW_PAYLOAD_TYPES` / `TYPE_ALIASES` for legacy callers.
- **`public/packets.js`** — `TYPE_NAMES` + `typeMap` now read from
`PayloadLabelsApi.SHORT_BY_ID`. Literal kept only as a defensive
fallback for the case where the script tag fails to load.
- **`public/packet-filter.js`** — `FW_PAYLOAD_TYPES` + `TYPE_ALIASES`
now sourced from `PayloadLabelsApi`. Literal fallback retained so `node
test-packet-filter.js` still works headlessly.
- **`public/live.js`** — legend `<li>` rows are now generated from
`window.PayloadLabels` in stable order, killing the third-vocabulary
`Message — Group text` / `Direct — Direct message` drift the #1797
review surfaced.
- **`public/index.html`** — `<script src="payload-labels.js">` loaded
before `roles.js` / `packet-filter.js` / `packets.js`.
- **`test-issue-1799-label-vocab-e2e.js`** (new) — Playwright E2E.
Scrapes `#liveLegend` rows and the `/packets` type-filter checklist,
asserts each label matches `window.PayloadLabels[ENUM].short` for
`TXT_MSG`, `GRP_TXT`, `GRP_DATA`. Also verifies `window.PacketFilter`
still recognises the enum names.
- **`.github/workflows/deploy.yml`** — wired the new E2E into the
existing Playwright block.

## TDD trail

- Red commit `eb392d4` — adds the failing E2E only (asserts
`window.PayloadLabels` exists and labels match; both fail).
- Green commit `44e902a` — introduces the canonical map and migrates the
three surfaces.

## Verification

- `node test-packet-filter.js` — 92/92 pass with the new fallback
wiring.
- Preflight: `bash ~/.openclaw/skills/pr-preflight/scripts/run-all.sh
origin/master` — clean.

Browser verified: E2E `test-issue-1799-label-vocab-e2e.js` exercises
`/live` legend + `/packets` type filter against a Playwright headless
Chromium; CI's Playwright block runs it on every push.

E2E assertion added: `test-issue-1799-label-vocab-e2e.js:139` —
`assert(fromLegend === canon, ...)` and `assert(fromPackets === canon,
...)` per enum.

Fixes #1799

---------

Co-authored-by: mc-bot <bot@corescope>
Co-authored-by: openclaw-bot <bot@openclaw.local>
Co-authored-by: clawbot <clawbot@kpa.com>
Co-authored-by: clawbot <bot@clawbot.local>
2026-06-30 05:48:47 -07:00
Kpa-clawbotandclawbot b3189c613a fix(#1802): decode CONTROL DISCOVER_REQ/RESP subtype + body fields (#1806)
## Summary
Extend CONTROL packet decoding to surface DISCOVER_REQ / DISCOVER_RESP
subtype plus body fields in the packet detail view. Previously only the
byte0 zero-hop flag was decoded; the body was rendered as opaque hex.

## What changed

**Backend** — `cmd/ingestor/decoder.go` `decodeControl()`
- New `Payload` fields (all omitempty): `CtrlSubtype`, `CtrlFilter`,
`CtrlTag`, `CtrlSince`, `CtrlNodeType`, `CtrlSNR`, `CtrlPubKey`.
- Subtype derived from `byte0 & 0xF0`: `0x80` → `DISCOVER_REQ`, `0x90` →
`DISCOVER_RESP`, otherwise `UNKNOWN`.
- REQ body parsed when `len(buf) >= 6`: `filter:u8 | tag:u32 LE`, plus
optional `since:u32 LE` when 4 more bytes remain.
- RESP body parsed when `len(buf) >= 6`: `node_type` (low nibble of
byte0), `snr:i8`, `tag:u32 LE`, and `pubkey` hex — 32 bytes when full, 8
bytes when prefix-only.
- Every field gated on length; short/truncated bodies emit subtype only
and never panic.
- `CtrlZeroHop` retained for backwards compatibility (rename flagged for
follow-up per triage).

**Frontend** — `public/packets.js` `getDetailPreview()`
- New `decoded.type === 'CONTROL'` branch renders subtype + present body
fields (filter / tag / since / node_type / snr / pubkey). Each field
shown only when populated, so truncated CONTROL still gets a subtype
label.

## Wire format reference
- `firmware/src/Mesh.cpp:69` — `CTL_TYPE_NODE_DISCOVER_REQ=0x80`,
`CTL_TYPE_NODE_DISCOVER_RESP=0x90`.
- `firmware/examples/simple_repeater/MyMesh.cpp:773-820` — body parse /
build.

## Tests (red → green, per AGENTS.md STRICT TDD)
- `cmd/ingestor/issue1802_test.go` — 6 cases: REQ full body (with
since), REQ no-since, RESP 32B pubkey, RESP 8B prefix pubkey, RESP
truncated pubkey (no panic, no pubkey emitted), short body (subtype
only), unknown subtype. Red commit `43713d3a` → green commit `d4b28180`.
Pre-existing CONTROL tests (`TestDecodeControlZeroHop`,
`TestDecodeControlMultiHop`) still pass.
- `test-packets.js` — 3 cases on `getDetailPreview`: DISCOVER_REQ
(filter+tag rendered), DISCOVER_RESP (snr+pubkey rendered), UNKNOWN
subtype label. Red commit `be23e349` → green commit `845d6c48`.

## Preflight overrides
- `check-branch-clean` (cross-stack): justified — issue #1802 explicitly
spans backend decoder (`cmd/ingestor/decoder.go`) and frontend renderer
(`public/packets.js`) per triage comment. Tests in both layers.
Single-purpose PR.

## Scope discipline
Files touched: `cmd/ingestor/decoder.go`,
`cmd/ingestor/issue1802_test.go`, `public/packets.js`,
`test-packets.js`. No other files. No firmware changes. No
`cmd/server/decoder.go` changes. No `CtrlZeroHop` rename (deferred per
triage).

Fixes #1802

---------

Co-authored-by: clawbot <bot@meshcore.local>
2026-06-28 06:32:07 -07:00
d5ceb27334 fix(#1792): decode GRP_DATA channel hash + inner data_type/len/blob in details cell (#1796)
Red commit: 11d8c51e8a (CI:
https://github.com/Kpa-clawbot/CoreScope/actions?query=branch%3Afix%2Fissue-1792)

## What

Render GRP_DATA (PAYLOAD_TYPE 0x06) channel hash + (when decrypted)
inner `data_type` / `data_len` / blob hex in the packets table "details"
cell, mirroring the existing GRP_TXT branch in `public/packets.js
getDetailPreview()`.

Previously these packets showed only the opaque payload bytes —
operators had no way to see channel distribution or recognize specific
data_type values at a glance.

## How

`public/packets.js` — one new branch right after the GRP_TXT branch:

- Always renders `Ch 0xNN` (from `channelHashHex` or computed from
`channelHash`).
- `decryptionStatus = no_key | decryption_failed` → status label, same
shape as GRP_TXT.
- `decryptionStatus = decrypted` → adds `type=0xNNNN len=N` plus a
`<code>` block with the blob hex, truncated to 32 hex chars (16 bytes)
with `…` when longer.

Inner layout per `firmware/src/helpers/BaseChatMesh.cpp:382-385` (uint16
LE data_type, u8 data_len, blob).

## TDD

- **Red commit** `11d8c51e`: 4 new assertion-shaped failures in
`test-packets.js` getDetailPreview suite (`packets.js tests: 70 passed,
17 failed` → +4 vs baseline 13).
- **Green commit** `3466ed09`: 17 → 13 failed (baseline
emoji-vs-Phosphor drift, unrelated). All 4 GRP_DATA assertions pass.

## Optional check #2 — backend JSON parity

Confirmed `cmd/server/decoder.go` and `cmd/ingestor/decoder.go` use
identical JSON tags (`channelHashHex`, `decryptionStatus`, `dataType`,
`dataLen`, `decryptedBlob`). No backend change needed — server emits
envelope-only fields, ingestor adds the inner fields when a channel key
matches; the frontend handles both shapes.

## Scope

- 2 files: `public/packets.js` (+17 lines), `test-packets.js` (+48 lines
test).
- No public API change. No CSS. No migration.

Preflight clean (PII, branch scope, red commit, CSS, LIKE-on-JSON,
sync/async migration, XSS sinks — all pass).

Fixes #1792

---------

Co-authored-by: Kpa-clawbot <bot@example.com>
Co-authored-by: meshcore-bot <bot@meshcore.local>
Co-authored-by: Kpa-clawbot <bot@openclaw.local>
2026-06-28 02:17:27 +00:00
Kpa-clawbotandyou aea0a9caee fix(packets): preserve scroll position on filter change + group expand/collapse (closes #431) (#996)
## Summary

Closes #431. Preserves scroll position on the packets page when filters
change or groups are expanded/collapsed.

## Problem

When an operator scrolls down through packet history then changes a
filter (type, observer, packet-filter expression) or expands/collapses a
group, `renderTableRows()` rebuilds the DOM which resets `scrollTop` to
0. This forces the user back to the top — frustrating when digging
through hundreds of packets.

## Fix

Save `scrollContainer.scrollTop` at the start of `renderTableRows()`,
restore it after DOM rebuild completes. Two restore points:
1. **Empty-results path** (line ~1821): after `tbody.innerHTML = ...` 
2. **Normal virtual-scroll path** (line ~1840): after
`renderVisibleRows()`

### Key lines changed
- `public/packets.js` lines 1748–1749: save scrollTop
- `public/packets.js` line 1821: restore after empty-state DOM write  
- `public/packets.js` line 1840: restore after renderVisibleRows

## TDD evidence

- **Red commit:** a99ba21 — test asserts scrollTop preserved; fails
without fix
- **Green commit:** 35cc4bf — adds save/restore; test passes

## Anti-tautology

Removing the `scrollContainer.scrollTop = savedScrollTop` lines causes
the test to fail (scrollTop becomes 0 instead of 500). Verified locally.

## Verification

- `node test-packets.js` — 83 passed, 0 failed
- `node test-packet-filter.js` — 62 passed, 0 failed

---------

Co-authored-by: you <you@example.com>
2026-05-02 21:03:01 -07:00
Kpa-clawbotandKpa-clawbot f84142b1d2 fix(packets): hash filter must bypass saved region filter (#939)
## Summary

Direct packet links like `/#/packets?hash=<HASH>` silently returned zero
rows when the user's saved region filter excluded the packet's observer
region. The packet existed and rendered in the side panel (which fetches
without region filter), but the main packet table was empty — leaving
the user with no rows to click and no obvious diagnostic.

## Root cause

`loadPackets()` in `public/packets.js` always added the `region` query
param to `/api/packets`, even when `filters.hash` was set. The
time-window filter is already correctly suppressed when `filters.hash`
is present (see line 619: `if (windowMin > 0 && !filters.hash)`); the
region filter should follow the same rule. A specific hash is an exact
identifier — the user wants THAT packet regardless of where their saved
region selection points.

## Change

Extracted the param-building logic into a pure helper
`buildPacketsParams(...)` so it's testable, then suppressed the `region`
param when `filters.hash` is set.

## Tests

Added 7 unit tests in `test-packets.js` covering:

- hash filter suppresses region (the bug)
- hash filter suppresses region with default windowMin=0
- region applies normally when no hash filter
- empty regionParam doesn't produce spurious `region=` param
- node/observer/channel filters still pass through alongside a hash
- groupByHash=true / false flag handling

Anti-tautology gate verified: reverting the one-line fix (`!filters.hash
&&` → removed) causes 3 of the 7 new tests to fail. The fix is the
smallest change that makes them pass.

`node test-packets.js`: 80 passed, 0 failed.

## Reproduction

1. Set region filter to e.g. `SJC`
2. Open `/#/packets?hash=<HASH_FROM_ANOTHER_REGION>`
3. Before fix: empty table, no diagnostic
4. After fix: packet renders

---------

Co-authored-by: Kpa-clawbot <bot@example.invalid>
2026-04-30 19:51:53 -07:00
you 16a72b66a9 test: fix hash_size test for zero-hop behavior change (#653)
The buildFieldTable test expected hash_size=4 for path byte 0xC0 with
hash_count=0. After #653, zero hash_count shows 'hash_count=0 (direct
advert)' instead. Updated test and added new test verifying hash_size
IS shown when hash_count > 0.
2026-04-08 04:53:10 +00:00
77b7c33d0f perf: incremental DOM diff in renderVisibleRows (#414) (#596)
## Summary

- Replace full \`tbody\` teardown+rebuild on every scroll frame with a
range-diff that only adds/removes the delta rows at the edges of the
visible window
- \`buildFlatRowHtml\` / \`buildGroupRowHtml\` now accept an
\`entryIdx\` parameter and emit \`data-entry-idx\` on every \`<tr>\` so
the diff can target rows precisely (including expanded group children)
- Full rebuild is retained for initial render and large scroll jumps
past the buffer (no range overlap)
- Also loads \`packet-helpers.js\` in the test sandbox, fixing 7
pre-existing test failures for the builder functions; adds 4 new tests
covering \`data-entry-idx\` output

Fixes #414

## Test plan

- [x] Open packets page with 500+ packets, scroll rapidly — DOM
inspector should show incremental \`<tr>\` adds/removes rather than full
\`tbody\` teardown
- [x] Expand a grouped packet, scroll away and back — expanded children
re-render correctly
- [x] Large scroll jump (jump to bottom via scrollbar) — full rebuild
fires, no visual glitch
- [x] \`node test-packets.js\` — 72 passed, 0 failed

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: you <you@example.com>
2026-04-04 19:41:33 -07:00
Kpa-clawbotandyou 8e42febc9c fix: virtual scroll height accounts for expanded group rows (#410) (#547)
## Summary

Fixes #410 — virtual scroll height miscalculation for expanded group
rows.

## Root Cause

When WebSocket messages add children to an already-expanded packet
group, `_rowCounts` becomes stale during the 200ms render debounce
window. Scroll events during this window call `renderVisibleRows()` with
stale row counts, causing wrong total height, spacer heights, and
visible range calculations.

## Changes

**public/packets.js:**
- Added `_rowCountsDirty` flag to track when row counts need
recomputation
- Added `_invalidateRowCounts()` — marks row counts as stale and clears
cumulative cache
- Added `_refreshRowCountsIfDirty()` — lazily recomputes `_rowCounts`
from `_displayPackets`
- Called `_invalidateRowCounts()` when WS handler adds children to
expanded groups (line ~402)
- Called `_refreshRowCountsIfDirty()` at top of `renderVisibleRows()`
before using row counts
- Reset `_rowCountsDirty` in all cleanup paths (destroy, empty display)

**test-packets.js:**
- Added 4 regression tests for `_invalidateRowCounts` /
`_refreshRowCountsIfDirty`

## Complexity

O(n) recomputation of `_rowCounts` when dirty (same as existing
`renderTableRows` path). Only triggers when WS modifies expanded group
children, which is infrequent relative to scroll events.

Co-authored-by: you <you@example.com>
2026-04-03 13:55:23 -07:00
Kpa-clawbotandyou 016b87b33c test: add 64 unit tests for packets.js (Part of #344) (#488)
## Summary

Adds 64 unit tests for `packets.js` — the largest untested frontend file
(2000+ lines) covering filter engine integration, time window logic,
groupByHash rendering, and packet detail display.

Part of #344 — packets.js coverage.

## Approach

Follows the existing `test-frontend-helpers.js` pattern: loads real
source files into a `vm.createContext` sandbox and tests actual code (no
copies).

Added a `window._packetsTestAPI` export at the end of the packets.js
IIFE to expose pure functions for testing without changing any runtime
behavior.

## What's Tested

| Function | Tests | What it covers |
|----------|-------|----------------|
| `typeName` | 2 | Type code → name mapping, unknown fallback |
| `obsName` | 2 | Observer name lookup, falsy/missing handling |
| `kv` | 1 | Key-value HTML helper |
| `sectionRow` / `fieldRow` | 3 | Table section/field HTML builders |
| `getDetailPreview` | 17 | All packet types: CHAN, ADVERT
(repeater/room/sensor/companion), GRP_TXT
(no_key/decryption_failed/channelHashHex), TXT_MSG, PATH, REQ, RESPONSE,
ANON_REQ, text fallback, public_key fallback, empty |
| `getPathHopCount` | 4 | Valid path, empty, null, invalid JSON |
| `sortGroupChildren` | 3 | Default observer sort, header update, null
safety |
| `renderTimestampCell` | 2 | Timestamp HTML output, null handling |
| `renderPath` | 3 | Empty/null, multi-hop with arrows, single hop |
| `renderDecodedPacket` | 6 | Header/path/payload/nested objects/null
skip/raw hex |
| `buildFieldTable` | 11 | All payload types (ADVERT with
flags/location/name, GRP_TXT, CHAN, ACK, destHash, raw fallback),
transport codes, path hops, hash_size calculation, empty hex |
| `_getRowCount` | 1 | Virtual scroll row counting |
| `buildFlatRowHtml` | 3 | Row rendering, size calculation, missing hex
|
| `buildGroupRowHtml` | 3 | Single/multi group, observation badge |
| Test API exposure | 1 | Verifies window._packetsTestAPI |

## Constraints Met

- No new test dependencies
- Tests real code via `vm.createContext`, not copies
- No build step — vanilla JS
- All existing tests still pass (254 frontend-helpers, 62 packet-filter,
29 aging)

Co-authored-by: you <you@example.com>
2026-04-02 16:42:25 -07:00