Commit Graph

1669 Commits

Author SHA1 Message Date
Kpa-clawbot e2bd9a8fa2 ci: update go-server-coverage.json [skip ci] 2026-05-05 02:07:56 +00:00
Kpa-clawbot 1f3c8130ef ci: update go-ingestor-coverage.json [skip ci] 2026-05-05 02:07:55 +00:00
Kpa-clawbot e5606058c1 ci: update frontend-tests.json [skip ci] 2026-05-05 02:07:54 +00:00
Kpa-clawbot 47b4021346 ci: update frontend-coverage.json [skip ci] 2026-05-05 02:07:53 +00:00
Kpa-clawbot c93c008867 ci: update e2e-tests.json [skip ci] 2026-05-05 02:07:52 +00:00
Kpa-clawbot cea2c70d12 feat(#1034): channel UX redesign PR1 — Add Channel modal + sectioned sidebar (#1037)
## Summary

PR 1 of 3 for #1034 — channel UX redesign. Replaces the cramped inline
"type a name or 32-hex blob" form with a clear modal dialog, and
reorganizes the sidebar into three labeled sections.

**Scope of this PR:** Modal UI + sectioned sidebar. QR generation/scan
is deferred to PR #2 (placeholders are wired and ready).
`channel-decrypt.js` crypto is untouched.

## What changed

### New modal: `[+ Add Channel]`

Triggered by the new sidebar button. Three sections:

1. **Generate PSK Channel** — name + `[Generate & Show QR]` →
`crypto.getRandomValues(16)` → hex → `ChannelDecrypt.storeKey`. QR
rendering ships in PR #2; for now `#qr-output` surfaces the hex key as
text.
2. **Add Private Channel (PSK)** — 32-hex input (regex-validated),
optional display name, `[Add]`. `[📷 Scan QR]` placeholder is present but
`disabled` (PR #2 wires it).
3. **Monitor Hashtag Channel** — non-editable `#` prefix + free text +
case-sensitivity warning + `[Monitor]`. Reuses
`ChannelDecrypt.deriveKey`.

Privacy footer: _"🔒 Keys stay in your browser. CoreScope is a passive
observer..."_

Close ✕, backdrop click, and Escape all dismiss.

### Sectioned sidebar

`renderChannelList()` rewritten to render three sections:

- **My Channels** — `userAdded` channels. ✕ always visible. Last sender
+ relative time.
- **Network** — server-known cleartext channels.
- **Encrypted (N)** — collapsed by default (toggle persists in
`localStorage`). Shows hash byte + packet count.

The legacy "🔒 No key" checkbox and `#chShowEncrypted` toggle are removed
entirely. Encrypted channels are always fetched; the renderer groups
them.

## Tests

- **Unit** — `test-channel-modal-ux.js` (33 assertions): added to
`test-all.sh`. Covers sidebar button, modal markup, three sections, QR
placeholders, privacy footer, sectioned sidebar, modal handlers (incl.
`crypto.getRandomValues(16)`).
- **E2E** — `test-channel-modal-e2e.js` (Playwright, 14 steps). Covers
modal open/close, section rendering, invalid-hex error, valid-hex
storage, encrypted-section toggle. Run with:
  ```
CHROMIUM_PATH=/usr/bin/chromium-browser BASE_URL=http://localhost:38201
node test-channel-modal-e2e.js
  ```
- `test-channel-psk-ux.js` — updated to reference `#chPskName` (was
`#chKeyLabelInput`).

### Red→green proof

- Red commit (`7ee421b`): test added with 31 expected assertion
failures, no source change.
- Green commit (`897be8f`): implementation lands, test passes 33/33.

## Browser-validated

Built `cmd/server/`, ran against `test-fixtures/e2e-fixture.db`,
exercised modal open → invalid hex → valid hex → key persisted → modal
closes → sectioned sidebar renders + Encrypted toggle expands. All 14
E2E steps pass.

## What's NOT in this PR

- QR code rendering (PR #2)
- Camera/QR scanning (PR #2)
- Migration of legacy localStorage format (PR #3, if needed — current
key format is unchanged)
- `channel-decrypt.js` changes (none — UI-only PR)

## Acceptance criteria from #1034

- [x] Modal opens on `[+ Add Channel]` click
- [x] Three sections clearly separated with labels
- [x] Add PSK: accepts 32-hex (QR scan = PR #2)
- [x] Monitor Hashtag: derives key, case-sensitivity warning shown
- [x] Privacy footer present
- [x] Sidebar: three sections (My Channels / Network / Encrypted)
- [x] ✕ button visible and functional on My Channels entries
- [x] "No key" checkbox removed
- [ ] Generate PSK QR display — text fallback only; QR is PR #2
- [ ] Old stored keys migrate seamlessly — no migration needed (storage
format unchanged)

Refs #1034

---------

Co-authored-by: meshcore-bot <bot@meshcore.local>
2026-05-04 18:40:46 -07:00
Kpa-clawbot 71f82d5d25 ci: update go-server-coverage.json [skip ci] 2026-05-05 01:39:54 +00:00
Kpa-clawbot 81430cf4c4 ci: update go-ingestor-coverage.json [skip ci] 2026-05-05 01:39:53 +00:00
Kpa-clawbot 1178bae18f ci: update frontend-tests.json [skip ci] 2026-05-05 01:39:52 +00:00
Kpa-clawbot 27c8514d70 ci: update frontend-coverage.json [skip ci] 2026-05-05 01:39:51 +00:00
Kpa-clawbot a24ec6e767 ci: update e2e-tests.json [skip ci] 2026-05-05 01:39:50 +00:00
Kpa-clawbot c1d0daf200 feat(#1034): channel QR generate + scan module (PR 2/3) (#1035)
## PR #2 of channel UX redesign (#1034) — QR generation + scanning

Self-contained QR module for MeshCore channel sharing. Wirable but **not
wired** — PR #3 wires this into the modal placeholders shipped by PR #1.

### What's in
- **`public/channel-qr.js`** — new module exporting `window.ChannelQR`:
- `buildUrl(name, secretHex)` →
`meshcore://channel/add?name=<urlencoded>&secret=<32hex>`
- `parseChannelUrl(url)` → `{name, secret}` or `null` (strict: scheme,
path, hex32 secret)
- `generate(name, secretHex, target)` — renders QR (via vendored
qrcode.js) + the URL string + a "Copy Key" button into `target`
- `scan()` → `Promise<{name, secret} | null>` — opens a camera overlay,
decodes with jsQR, parses, auto-closes on first valid match. Graceful
no-camera/permission-denied fallback ("Camera not available — paste key
manually").
- **`public/vendor/jsqr.min.js`** — vendored jsQR 1.4.0
- **`public/index.html`** — loads `vendor/jsqr.min.js` + `channel-qr.js`
after `channel-decrypt.js`
- **`test-channel-qr.js`** + wired into `test-all.sh` — 16 assertions on
`buildUrl` / `parseChannelUrl` (DOM/camera paths covered by Playwright
in #3)

### TDD
- Red commit `d6ba89e` — stub module + failing assertions on `buildUrl`
/ `parseChannelUrl` (compiles, runs, fails on assertion)
- Green commit `25328ac` — real impl, 16/16 pass

### License note
Brief specified jsQR as MIT — it's actually **Apache-2.0**
(https://github.com/cozmo/jsQR/blob/master/package.json). Apache-2.0 is
permissive and compatible with the repo's ISC license; flagging here so
reviewers can confirm. Cited in the file header.

### Independence guarantees
- Does **not** touch `channels.js` or `channel-decrypt.js`
- Does not call any UI from `channels.js`; PR #3 will call
`ChannelQR.generate(...)` into `#qr-output` and wire `#scan-qr-btn` to
`ChannelQR.scan()`

Refs #1034

---------

Co-authored-by: openclaw-bot <bot@openclaw.local>
2026-05-04 18:29:48 -07:00
Kpa-clawbot d967170dd3 fix(channels): sidebar layout for user-added (PSK) rows — nested <button> bug (#1033)
## Problem

Channel sidebar layout broke for user-added (PSK) channels. Visible
symptoms in the screenshot:

- No ✕ (delete) button on user-added rows
- 🔑 emoji floating in the wrong position
- Message preview text (e.g. `KpaPocket: Тест`) orphaned **between**
channel entries instead of inside the row
- Spinner/loading dots misaligned

## Root cause

**HTML5 forbids nested `<button>` elements.** The `.ch-item` row is a
`<button>`, and #1024 added a `<button class="ch-remove-btn">` inside
it. The HTML parser implicitly closes the outer `.ch-item` the moment it
sees the inner `<button>`, then re-parents everything after it (✕ and
the `.ch-item-preview` line) outside the row.

Resulting DOM tree (parser-corrected, simplified):

```
<button class="ch-item">[icon] Levski 🔑</button>   <-- closes early
<button class="ch-remove-btn">✕</button>            <-- orphaned, "floating"
<div class="ch-item-preview">KpaPocket: Тест</div>  <-- orphaned
<button class="ch-item">[icon] #bookclub …</button>
```

Compounded by `.ch-remove-btn { opacity: 0 }` (only visible on row
hover), which made the ✕ undiscoverable on touch devices even before the
parser bug.

## Fix

`public/channels.js`
- Replace the inner `<button class="ch-remove-btn">` with `<span
class="ch-remove-btn" role="button" tabindex="0">`. Click delegation
already keys off `[data-remove-channel]` so behavior is unchanged.
- Add `keydown` (Enter / Space) handler on `#chList` so the role=button
span stays keyboard-accessible.
- Relabel the ambiguous `🔒 No key` toggle to `🔒 Show encrypted (no
key)`, with an explanatory `title` ("Show encrypted channels you don't
have a key for (locked, can't decrypt)") so users understand it controls
visibility of channels they haven't added a PSK for.

`public/style.css`
- `.ch-remove-btn`: drop `opacity: 0` default. Now `0.55` idle, `0.9` on
row hover, `1` on direct hover/focus. Added `:focus` outline removal +
`display: inline-flex` so the ✕ centers cleanly.
- Add `.ch-user-badge` rule (was unstyled — contributed to the
misalignment of the 🔑).

## TDD

- Red commit `eeb94ad` — `test-channel-sidebar-layout.js` (7 assertions,
3 failing on master).
- Green commit `2959c3d` — fix; all 7 pass.
- Wire commit `4d6100d` — added to `test-all.sh`.

Existing channel test files still pass (`test-channel-psk-ux.js`,
`test-channel-live-decrypt.js`,
`test-channel-live-decrypt-userprefix.js`,
`test-channel-decrypt-m345.js`,
`test-channel-decrypt-insecure-context.js`).

## Files changed
- `public/channels.js`
- `public/style.css`
- `test-channel-sidebar-layout.js` (new)
- `test-all.sh`
2026-05-04 18:29:45 -07:00
Kpa-clawbot 2f0c97604b feat(map): cluster markers with Leaflet.markercluster (#1036) (#1038)
## Summary
Implements map marker clustering for large meshes (500+ nodes) using
vendored `Leaflet.markercluster@1.5.3`. Closes the long-standing no-op
`Show clusters` checkbox.

## What changed
**Vendored library** — `public/vendor/leaflet.markercluster.js` +
`MarkerCluster.css` + `MarkerCluster.Default.css`. No CDN: this runs
offline on mesh-operator deployments.

**`map.js`**
- `createClusterGroup()` instantiates `L.markerClusterGroup` with:
  - `chunkedLoading: true` (no frame drops on initial render)
- `removeOutsideVisibleBounds: true` (viewport culling — key win at 2k+
nodes)
  - `disableClusteringAtZoom: 16` (fully expanded at high zoom)
  - `spiderfyOnMaxZoom: true` (fan out at max zoom)
  - `showCoverageOnHover: false`
  - `animate` disabled on mobile UA for perf
- `makeClusterIcon(cluster)` produces a CoreScope-themed `L.divIcon`:
  - Bold total count, centered
- Up to 4 role-color mini-pills (repeater / companion / room / sensor /
observer) using `ROLE_COLORS`
- Bucketed `mc-sm` / `mc-md` / `mc-lg` background (info / warning /
accent CSS vars)
- `#mcClusters` checkbox repurposed from no-op `Show clusters` →
`Cluster markers`, default **ON**, persisted to
`localStorage['meshcore-map-clustering']`
- Render branches at the marker-add step: clustering ON → `addLayers()`
to `clusterGroup`, skip `deconflictLabels` + `_updateOffsetIndicator`
polylines + `_repositionMarkers` on zoom/resize. Clustering OFF →
original flow unchanged.
- Route polylines (`drawPacketRoute`) already removed both layers — no
change needed beyond actually instantiating `clusterGroup`.
- `?node=PUBKEY` deep-link lookup now searches both `markerLayer` and
`clusterGroup` so it works in either mode.

**`style.css`** — cluster bubble + role-pill styles using `--info` /
`--warning` / `--accent` CSS variables; hover scale.

**`index.html`** — vendor CSS + JS tags after the Leaflet bundle
(cache-busted via `__BUST__`).

## TDD
- **Red commit** `e10af23` — `test-map-clustering.js` + stub
`createClusterGroup`/`makeClusterIcon` returning null/empty divIcon.
Compiles, runs, fails 4/5 on assertions.
- **Green commit** `482ea2e` — real implementation. 5/5 pass.

```
=== map.js: clustering ===
   exposes test hooks (__meshcoreMapInternals)
   createClusterGroup returns an L.MarkerClusterGroup with required options
   cluster group accepts markers via addLayer
   makeClusterIcon: includes total count and role-pill counts
   makeClusterIcon: bucket sm/md/lg by total
```

## Behavior preserved
- Clustering OFF (existing checkbox unchecked) → all original behavior
intact: deconfliction spiral, offset-indicator polylines, per-zoom
reposition.
- Default ON. Operators with small meshes can disable via the checkbox;
choice persists.
- Spiderfying enabled at max zoom (built-in markercluster behavior).

## Performance target
Smooth pan/zoom at 2000 nodes — `chunkedLoading` keeps the main thread
responsive during initial add, `removeOutsideVisibleBounds` keeps DOM
bounded to the viewport. Per AGENTS.md rule 0: complexity is O(n) for
the initial add (chunked across frames), per-zoom re-cluster is internal
to markercluster (well-tested at 10k+ scale).

## Out of scope (filed as follow-ups in spec)
- Canvas marker renderer — only if 5k+ nodes per viewport materializes
- Server-side viewport culling (`/api/nodes?bbox=`)
- Cluster-by-role split groups
- 2k-node fixture + Playwright DOM assertions — repo doesn't currently
ship a `fixture=` query param; the unit test exercises the integration
deterministically.

Fixes #1036

---------

Co-authored-by: corescope-bot <bot@corescope>
2026-05-04 18:29:42 -07:00
Kpa-clawbot 0b0fda5bb2 ci: update go-server-coverage.json [skip ci] 2026-05-04 23:54:17 +00:00
Kpa-clawbot e966ecc71a ci: update go-ingestor-coverage.json [skip ci] 2026-05-04 23:54:16 +00:00
Kpa-clawbot e7aa8eded8 ci: update frontend-tests.json [skip ci] 2026-05-04 23:54:16 +00:00
Kpa-clawbot d652b7c39d ci: update frontend-coverage.json [skip ci] 2026-05-04 23:54:15 +00:00
Kpa-clawbot 6a8ed98d8f ci: update e2e-tests.json [skip ci] 2026-05-04 23:54:14 +00:00
Kpa-clawbot 26daa760cd fix(channels): live PSK decrypt for user-added channels (#1029 follow-up) (#1031)
## Problem

PR #1030 added live PSK decrypt for GRP_TXT WS packets, but in
production it still didn't work for **user-added** PSK channels. New
messages never appeared in real time on a channel added via the sidebar
key form — users had to refresh the page to see them via the REST fetch
path (regression #1029).

## Root cause

`decryptLivePSKBatch` rewrites the payload with the raw channel name:

```js
payload.channel = dec.channelName;   // e.g. "medusa"
```

But user-added channels live in `channels[]` under the key produced by
`addUserChannel`:

```js
hash: 'user:' + name,                // e.g. "user:medusa"
```

`selectedHash` also uses the `user:`-prefixed key while a user-added
channel is open. Downstream in `processWSBatch`:

| Line | Check | Result |
|---|---|---|
| 962 | `c.hash === channelName` | `"medusa" !== "user:medusa"` → user
channel never matched |
| 982 | `channelName === selectedHash` | `"medusa" !== "user:medusa"` →
message never appended to open chat |
| 974 | `channels.push({ hash: channelName, ... })` | duplicate plain
`"medusa"` entry pushed into sidebar |

The unread bumper (`channels.js:1086`) compared `chName === prior` with
the same mismatch, so it bumped an unread badge on the channel currently
being viewed.

Verified end to end against staging WS traffic (live `decryption_status:
"decrypted"` packets observed; user-added channel never updated,
duplicate entry created).

## Fix

`decryptLivePSKBatch` now also stamps a canonical sidebar key on the
payload:

```js
payload.channelKey = hasUserCh ? ('user:' + dec.channelName) : dec.channelName;
```

`processWSBatch` and the unread bumper route on `payload.channelKey`
(falling back to `payload.channel` for server-known CHAN packets — no
behavior change there).

After the fix:
-  live message appends to the open user-added chat
-  sidebar row's `lastMessage` / `messageCount` / `lastActivityMs`
update
-  no duplicate non-prefixed sidebar entry
-  unread bumped only on channels NOT being viewed

## TDD

Red commit `f1719a8` — `test-channel-live-decrypt-userprefix.js`, fails
6/9 on assertions (NOT build error) on pristine `channels.js`.
Green commit `da87018` — minimal fix in `channels.js`, all 9/9 pass.

Verified red gates the change: stashed `public/channels.js`, re-ran test
on red commit alone → 6 assertion failures (open channel got 0 messages,
duplicate sidebar entry, unread bumped on viewed channel).

## Files changed

- `public/channels.js` — stamp/route on `channelKey`
- `test-channel-live-decrypt-userprefix.js` (new) — red-then-green
regression test

---------

Co-authored-by: corescope-bot <bot@corescope>
v3.7.1
2026-05-04 16:44:35 -07:00
Kpa-clawbot c196030ec0 ci: update go-server-coverage.json [skip ci] v3.7.0 2026-05-04 05:06:19 +00:00
Kpa-clawbot 7b07761fb9 ci: update go-ingestor-coverage.json [skip ci] 2026-05-04 05:06:19 +00:00
Kpa-clawbot e47257222e ci: update frontend-tests.json [skip ci] 2026-05-04 05:06:18 +00:00
Kpa-clawbot 6f2d70599a ci: update frontend-coverage.json [skip ci] 2026-05-04 05:06:17 +00:00
Kpa-clawbot c120b5eef2 ci: update e2e-tests.json [skip ci] 2026-05-04 05:06:16 +00:00
Kpa-clawbot 3290ff1ed5 fix(channels): auto-decrypt PSK channels on WebSocket live feed (#1029) (#1030)
Closes #1029.

## Problem

PSK-decrypted channels show new messages only after a full page refresh.
The WebSocket live feed delivers `GRP_TXT` packets as encrypted blobs
and the channel UI has no hook to auto-decrypt them with stored keys.
The REST fetch path (used on initial load + on `selectChannel`) already
decrypts; the WS path silently dropped on the floor.

## Fix

Two new helpers in `public/channel-decrypt.js`:

- `buildKeyMap()` → `Map<channelHashByte, { channelName, keyBytes,
keyHex }>`
  built from `getStoredKeys()`. Cached and invalidated on `saveKey` /
  `removeKey`, so the WS hot path is O(1) per packet after the first
  build.
- `tryDecryptLive(payload, keyMap)` → returns
`{ sender, text, channelName, channelHashByte }` when the payload is an
  encrypted `GRP_TXT` whose channel hash matches a stored key and whose
  MAC verifies; `null` otherwise.

`public/channels.js` wraps `debouncedOnWS` with an async pre-pass
(`decryptLivePSKBatch`) that:

1. Skips the work entirely when no encrypted `GRP_TXT` is in the batch
   or no PSK keys are stored.
2. For each match, rewrites `payload.channel`, `payload.sender`, and
   `payload.text` so the existing `processWSBatch` consumes the packet
   exactly the same way it consumes a server-decrypted `CHAN`.
3. Bumps a per-channel `unread` counter for any decrypted message
   whose channel is not currently selected. The badge renders in the
   sidebar (`.ch-unread-badge`) and resets on `selectChannel`.

`processWSBatch` itself is untouched, so the existing channel-view
behavior, dedup-by-packet-hash, region filtering, and timestamp ticker
all continue to work as before.

## TDD

- **Red** (`2e1ff05`): `test-channel-live-decrypt.js` asserts the new
  helpers + the channels.js integration contract. With stub
  `buildKeyMap`/`tryDecryptLive` returning empty/null, the test compiles
  and runs to completion with **8/14 assertion failures** (no crashes,
  no missing-symbol errors).
- **Green** (`1783658`): real implementation lands; **14/14 pass**.

## Verification (Rule 18)

- `node test-channel-live-decrypt.js` → 14/14 pass
- All other channel tests still pass:
  - `test-channel-decrypt-ecb.js` 7/7
  - `test-channel-decrypt-insecure-context.js` 8/8
  - `test-channel-decrypt-m345.js` 24/24
  - `test-channel-psk-ux.js` 19/19
- `cd cmd/server && go build ./...` clean
- Booted the server against the fixture DB and curled
  `/channel-decrypt.js`, `/channels.js`, `/style.css` — all three serve
  the new code with the auto-injected `__BUST__` cache buster.

## Performance

The WS pre-pass is gated by a quick scan: zero-cost when no encrypted
`GRP_TXT` is present in the batch. When PSK keys exist, the key map is
cached (sig-keyed on the stored-keys snapshot) so `crypto.subtle.digest`
runs once per stored key per change, not per packet. Each match costs
one MAC verify + one ECB decrypt — the same work
`fetchAndDecryptChannel`
already does, just amortized over time instead of in a single batch.

## Out of scope

- Decoupling the badge from the live feed (server should ideally tag
  packets with `decryptionStatus` before broadcast). Tracked separately.
- Persisting the `unread` counter across reloads (currently in-memory).

---------

Co-authored-by: clawbot <bot@corescope.local>
2026-05-04 04:56:43 +00:00
Kpa-clawbot 505206feb4 ci: update go-server-coverage.json [skip ci] 2026-05-04 04:52:13 +00:00
Kpa-clawbot 41762a873a ci: update go-ingestor-coverage.json [skip ci] 2026-05-04 04:52:12 +00:00
Kpa-clawbot 7ab05c5a19 ci: update frontend-tests.json [skip ci] 2026-05-04 04:52:11 +00:00
Kpa-clawbot c3138a96f7 ci: update frontend-coverage.json [skip ci] 2026-05-04 04:52:10 +00:00
Kpa-clawbot 03c895addc ci: update e2e-tests.json [skip ci] 2026-05-04 04:52:09 +00:00
Kpa-clawbot c9301fee9c fix(ingestor): extract per-hop SNR for TRACE packets at ingest time (#1028)
## Problem

PR #1007 added per-hop SNR extraction (`snrValues`) for TRACE packets to
`cmd/server/decoder.go`. That code path is only hit by the on-demand
re-decode endpoint (packet detail). The actual ingest pipeline runs
`cmd/ingestor/decoder.go`, decodes the packet once, and persists
`decoded_json` into SQLite. The server then serves `decoded_json` as-is
for list/feed queries.

Net effect: `snrValues` never appears in any production response,
because the ingestor's decoder was never updated.

Confirmed empirically: `strings /app/corescope-ingestor | grep snrVal`
returns nothing.

## Fix

Port the SNR extraction logic from `cmd/server/decoder.go` (lines
410–422) into `cmd/ingestor/decoder.go`. For TRACE packets, the header
path bytes are int8 SNR values in quarter-dB encoding; extract them into
`payload.SNRValues` **before** `path.Hops` is overwritten with
payload-derived hop IDs.

Also adds the matching `SNRValues []float64` field to the ingestor's
`Payload` struct so it serializes into `decoded_json`.

## TDD

- **Red commit** (`6ae4c07`): adds `TestDecodeTraceExtractsSNRValues` +
`SNRValues` field stub. Compiles, fails on assertion (`len(SNRValues)=0,
want 2`).
- **Green commit** (`4a4f3f3`): adds extraction loop. Test passes.

Test packet: `26022FF8116A23A80000000001C0DE1000DEDE`
- header `0x26` = TRACE + DIRECT
- pathByte `0x02` = hash_size 1, hash_count 2
- header path `2F F8` → SNR `[int8(0x2F)/4, int8(0xF8)/4]` = `[11.75,
-2.0]`

## Files

- `cmd/ingestor/decoder.go` — `+16` (field + extraction)
- `cmd/ingestor/decoder_test.go` — `+29` (red test)

## Out of scope

- `cmd/server/decoder.go` is already correct (PR #1007). Untouched.
- Backfill of historical `decoded_json` rows. New TRACE packets get SNR;
old rows do not until re-decoded.

---------

Co-authored-by: corescope-bot <bot@corescope.local>
2026-05-03 21:42:14 -07:00
Kpa-clawbot dd66f678be ci: update go-server-coverage.json [skip ci] 2026-05-04 04:17:59 +00:00
Kpa-clawbot 8ec355c6d6 ci: update go-ingestor-coverage.json [skip ci] 2026-05-04 04:17:58 +00:00
Kpa-clawbot 98e5fe6adf ci: update frontend-tests.json [skip ci] 2026-05-04 04:17:57 +00:00
Kpa-clawbot b40719a21e ci: update frontend-coverage.json [skip ci] 2026-05-04 04:17:56 +00:00
Kpa-clawbot a695110ea4 ci: update e2e-tests.json [skip ci] 2026-05-04 04:17:54 +00:00
Kpa-clawbot 3aaa21bbc0 fix(channel-decrypt): pure-JS SHA-256/HMAC fallback for HTTP context (P0 follow-up to #1021) (#1027)
## P0: PSK channel decryption silently failed on HTTP origins

User reported PSK key `372a9c93260507adcbf36a84bec0f33d` "still doesn't
work" after PRs #1021 (AES-ECB pure-JS) and #1024 (PSK UX) merged.
Reproduced end-to-end and found the actual remaining bug.

### Root cause

PR #1021 fixed the AES-ECB path by vendoring a pure-JS core, but
**SHA-256 and HMAC-SHA256 in `public/channel-decrypt.js` are still
pinned to `crypto.subtle`**. `SubtleCrypto` is exposed **only in secure
contexts** (HTTPS / localhost); when CoreScope is served over plain HTTP
— common for self-hosted instances — `crypto.subtle` is `undefined`,
and:

- `computeChannelHash(key)` → `Cannot read properties of undefined
(reading 'digest')`
- `verifyMAC(...)` → `Cannot read properties of undefined (reading
'importKey')`

Both throws are swallowed by `addUserChannel`'s `try/catch`, so the only
user-visible signal is the toast `"Failed to decrypt"` with no
console-friendly explanation. Verdict: PR #1021 only fixed half of the
crypto-in-insecure-context problem.

### Reproduction (no browser required)

`test-channel-decrypt-insecure-context.js` loads the production
`public/channel-decrypt.js` in a `vm` sandbox where `crypto.subtle` is
undefined (mirrors HTTP browser). Pre-fix it failed 8/8 with the exact
error above; post-fix it passes 8/8.

### Fix

- New `public/vendor/sha256-hmac.js`: minimal pure-JS SHA-256 +
HMAC-SHA256 (FIPS-180-4 + RFC 2104, ~120 LOC, MIT). Verified against
Node `crypto` for SHA-256 (empty / "abc" / 1000 bytes) and RFC 4231
HMAC-SHA256 TC1.
- `public/channel-decrypt.js`: `hasSubtle()` guard. `deriveKey`,
`computeChannelHash`, and `verifyMAC` use `crypto.subtle` when available
and fall back to `window.PureCrypto` otherwise. Same API, same return
types, same async signatures.
- `public/index.html`: load `vendor/sha256-hmac.js` immediately before
`channel-decrypt.js` (mirrors the `vendor/aes-ecb.js` wiring from
#1021).

### TDD

- **Red** (`8075b55`): `test-channel-decrypt-insecure-context.js` — runs
the **unmodified** prod module in a no-`subtle` sandbox, asserts on the
known PSK key (hash byte `0xb7`) and synthetic encrypted packet
round-trip. Compiles, runs, **fails 8/8 on assertions** (not on import
errors).
- **Green** (`232add6`): vendor + delegate. Test passes 8/8.
- Wired into `test-all.sh` and `.github/workflows/deploy.yml` so CI
gates the regression.

### Validation (all green post-fix)

| Test | Result |
|---|---|
| `test-channel-decrypt-insecure-context.js` | 8/8 |
| `test-channel-decrypt-ecb.js` (#1021 KAT) | 7/7 |
| `test-channel-decrypt-m345.js` (existing) | 24/24 |
| `test-channel-psk-ux.js` (#1024) | 19/19 |
| `test-packet-filter.js` | 69/69 |

### Files changed

- `public/vendor/sha256-hmac.js` — **new** (~150 LOC, MIT, decrypt-side
only)
- `public/channel-decrypt.js` — `hasSubtle()` guard + fallback in
`deriveKey`/`computeChannelHash`/`verifyMAC`
- `public/index.html` — script tag for `vendor/sha256-hmac.js`
- `test-channel-decrypt-insecure-context.js` — **new** (8 assertions,
pure Node, no browser)
- `test-all.sh` + `.github/workflows/deploy.yml` — wire the test

### Risk / scope

- Frontend-only, decrypt-side only. No server, schema, or config changes
(Config Documentation Rule N/A).
- Secure-context behaviour unchanged (still uses Web Crypto when
present).
- HMAC `secret` building, MAC truncation (2 bytes), and AES-ECB
delegation untouched.
- Hash vector for the user's PSK key matches:
`SHA-256(372a9c93260507adcbf36a84bec0f33d) = b7ce04…`, channel hash byte
`0xb7` (183) — confirmed against Node `crypto` and against the new
pure-JS path.

### Note on the FIPS test data in the new test

The PSK `372a9c93260507adcbf36a84bec0f33d` is shared test data from the
bug report, not a real channel secret.

---------

Co-authored-by: openclaw-bot <bot@openclaw.local>
2026-05-03 21:06:59 -07:00
Kpa-clawbot 4def3ed7c4 ci: update go-server-coverage.json [skip ci] 2026-05-04 03:19:34 +00:00
Kpa-clawbot cfb4d652a7 ci: update go-ingestor-coverage.json [skip ci] 2026-05-04 03:19:33 +00:00
Kpa-clawbot 9bf4c103d8 ci: update frontend-tests.json [skip ci] 2026-05-04 03:19:32 +00:00
Kpa-clawbot 49857dd748 ci: update frontend-coverage.json [skip ci] 2026-05-04 03:19:31 +00:00
Kpa-clawbot 8815b194d8 ci: update e2e-tests.json [skip ci] 2026-05-04 03:19:30 +00:00
Kpa-clawbot 9f55ef802b fix(#804): attribute analytics by repeater home region, not observer (#1025)
Fixes #804.

## Problem
Analytics filtered region purely by **observer** region: a multi-byte
repeater whose home is PDX would leak into SJC results whenever its
flood
adverts were relayed past an SJC observer. Per-node groupings
(`multiByteNodes`, `distributionByRepeaters`) inherited the same bug.

## Fix

Two new helpers in `cmd/server/store.go`:

- `iataMatchesRegion(iata, regionParam)` — case-insensitive IATA→region
  match using the existing `normalizeRegionCodes` parser.
- `computeNodeHomeRegions()` — derives each node's HOME IATA from its
  zero-hop DIRECT adverts. Path byte for those packets is set locally on
  the originating radio and the packet has not been relayed, so the
  observer that hears it must be in direct RF range. Plurality vote when
  zero-hop adverts span multiple regions.

`computeAnalyticsHashSizes` now applies these in two ways:

1. **Observer-region filter is relaxed for ADVERT packets** when the
   originator's home region matches the requested region. A flood advert
   from a PDX repeater that's only heard by an SJC observer still
   attributes to PDX.
2. **Per-node grouping** (`multiByteNodes`, `distributionByRepeaters`)
   excludes nodes whose HOME region disagrees with the requested region.
   Falls back to the observer-region filter when home is unknown.

Adds `attributionMethod` to the response (`"observer"` or `"repeater"`)
so operators can tell which method was applied.

## Backwards compatibility

- No region filter requested → behavior unchanged (`attributionMethod`
  is `"observer"`).
- Region filter requested but no zero-hop direct adverts seen for a node
  → falls back to the prior observer-region check for that node.
- Operators without IATA-tagged observers see no change.

## TDD

- **Red commit** (`c35d349`): adds
`TestIssue804_AnalyticsAttributesByRepeaterRegion`
with three subtests (PDX leak into SJC, attributionMethod field present,
  SJC leak into PDX). Compiles, runs, fails on assertions.
- **Green commit** (`11b157f`): the implementation. All subtests pass,
  full `cmd/server` package green.

## Files changed
- `cmd/server/store.go` — helpers + analytics filter logic (+236/-51)
- `cmd/server/issue804_repeater_region_test.go` — new test (+147)

---------

Co-authored-by: CoreScope Bot <bot@corescope.local>
Co-authored-by: openclaw-bot <bot@openclaw.local>
2026-05-03 20:10:02 -07:00
Kpa-clawbot 019ace3645 ci: update go-server-coverage.json [skip ci] 2026-05-04 02:59:47 +00:00
Kpa-clawbot c5139f5de5 ci: update go-ingestor-coverage.json [skip ci] 2026-05-04 02:59:46 +00:00
Kpa-clawbot 0add429d24 ci: update frontend-tests.json [skip ci] 2026-05-04 02:59:45 +00:00
Kpa-clawbot c8b29d0482 ci: update frontend-coverage.json [skip ci] 2026-05-04 02:59:44 +00:00
Kpa-clawbot 9c5e13d133 ci: update e2e-tests.json [skip ci] 2026-05-04 02:59:42 +00:00
Kpa-clawbot 1f4969c1a6 fix(#770): treat region 'All' as no-filter + document region behavior (#1026)
## Summary

Fixes #770 — selecting "All" in the region filter dropdown produced an
empty channel list.

## Root cause

`normalizeRegionCodes` (cmd/server/db.go) treated any non-empty input as
a literal IATA code. The frontend region filter labels its catch-all
option **"All"**; while `region-filter.js` normally sends an empty
string when "All" is selected, any code path that ends up sending
`?region=All` (deep-link URLs, manual queries, future callers) caused
the function to return `["ALL"]`. Downstream queries then filtered
observers for `iata = 'ALL'`, which never matches anything → empty
response.

## Fix

`normalizeRegionCodes` now treats `All` / `ALL` / `all`
(case-insensitive, with optional whitespace, mixed in CSV) as equivalent
to an empty value, returning `nil` to signal "no filter". Real IATA
codes (`SJC`, `PDX`, `sjc,PDX` → `[SJC PDX]`) still pass through
unchanged.

This is a defensive server-side fix: a single chokepoint that all
region-aware endpoints already flow through (channels, packets,
analytics, encrypted channels, observer ID resolution).

## Documentation

Expanded `_comment_regions` in `config.example.json` to explain:
- How IATA codes are resolved (payload > topic > source config — set in
#1012)
- What the `regions` map controls (display labels) vs runtime-discovered
codes
- That observers without an IATA tag only appear under "All Regions"
- That the `All` sentinel is server-side safe

## TDD

- **Red commit** (`4f65bf4`): `cmd/server/region_filter_test.go` —
`TestNormalizeRegionCodes_AllIsNoFilter` asserts `All` / `ALL` / `all` /
`""` / `"All,"` all collapse to `nil`. Compiles, runs, fails on
assertion (`got [ALL], want nil`). Companion test
`TestNormalizeRegionCodes_RealCodesPreserved` locks in that `sjc,PDX`
still returns `[SJC PDX]`.
- **Green commit** (`c9fb965`): two-line change in
`normalizeRegionCodes` + docs update.

## Verification

```
$ go test -run TestNormalizeRegionCodes -count=1 ./cmd/server
ok      github.com/corescope/server     0.023s

$ go test -count=1 ./cmd/server
ok      github.com/corescope/server    21.454s
```

Full suite green; no existing region tests regressed.

Fixes #770

---------

Co-authored-by: Kpa-clawbot <bot@corescope>
2026-05-03 19:50:01 -07:00