Commit Graph

2125 Commits

Author SHA1 Message Date
Kpa-clawbot 101c11b4b3 fix(#1361): theme customizer — colorblind presets [WIP] (#1378)
WIP — draft PR for CI to exercise the RED test commit. Will be promoted
out of draft once the GREEN commit lands.

Red commit: 8b37c918 (test-only, expected CI failure on assertions)

Tracks #1361.

---------

Co-authored-by: openclaw-bot <bot@openclaw.local>
2026-05-25 22:35:42 -07:00
efiten 0b35c7eef3 feat(server): persist multi-byte capability across restart + O(1) per-key lookup (#903) (#1324)
## Summary

Follows the reconciliation recommendation in #916 — extracts only the
NET-NEW persistence layer from that PR (which is now superseded by #1002
for the overlay UI) into a focused 6-file change against current master.

**What this adds:**
- `multibyte_sup_v1` migration: `multibyte_sup INTEGER NOT NULL DEFAULT
0` + `multibyte_evidence TEXT` on `nodes`/`inactive_nodes` so capability
survives restart
- `hasMultibyteSupCols` schema detection gates the persist/load paths
- `loadMultibyteCapFromDB()`: pre-populates `mbCapSnapshot`/`mbCapIndex`
at startup — cold starts serve last-known capability without waiting for
the first ~15s analytics cycle
- `maybePersistMultibyteCapability()` + `persistMultibyteCapability()`:
after each analytics cycle; TryLock-gated (concurrent cycles coalesce);
skips `sup==0` entries (data-destruction guard)
- `GetMultibyteCapFor(pk)`: O(1) map lookup; both `handleNodes` and
node-detail call sites updated from the O(N)-alloc
`GetMultiByteCapMap()`

**What this explicitly does NOT change:**
- API field names (`multi_byte_status`, `multi_byte_evidence`,
`multi_byte_max_hash_size`)
- `EnrichNodeWithMultiByte` — unchanged
- `GetMultiByteCapMap` — still present for any external callers
- `public/map.js`, `public/live.css`, `Dockerfile`, `docs/` — zero
frontend churn

## Test plan

- [x] `TestMultibyteCapPersistRoundTrip` — confirmed values survive
persist → fresh-store load
- [x] `TestMultibyteCapPersistSkipsUnknown` — data-destruction guard:
`sup==0` entry does not overwrite DB-confirmed value
- [x] `TestMultibyteCapMaybePersistCoalesces` — TryLock coalesces 10
concurrent callers without deadlock
- [x] `TestMultibyteCapGetMultibyteCapForO1` — O(1) index returns
correct entry / false for unknown pubkey
- [x] `TestMultibyteCapLoadFromDB` — only `sup>0` rows loaded; `sup==0`
row excluded
- [x] `TestSchemaMultibyteSupColumns` — migration adds columns to both
tables; idempotent on second `OpenStore`
- [x] All existing `TestMultiByteCapability_*` tests pass unchanged
- [x] Full ingestor test suite: `ok` in 27s
- [x] `go build ./cmd/server/ && go build ./cmd/ingestor/` clean

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: openclaw-bot <bot@openclaw>
2026-05-25 22:35:35 -07:00
Kpa-clawbot 9d3dd8df0a fix(packets): order by ingest id, not rxTime — fresh activity visible on packets page (#1345) (#1349)
## Summary
Fixes #1345 — the packets page shows "no recent activity" while MQTT
ingest is healthy because the default `/api/packets` query was `ORDER BY
first_seen DESC`, and PR #1233 redefined `first_seen` as the observer's
radio receive time (rxTime). When an observer buffers offline and
uploads hours later, its packets land with hours-old `first_seen`
values; older-ingested packets with fresher rxTime then crowd the top of
the list and the visually freshest activity disappears.

## Fix
Switch the default ordering to `t.id DESC` (ingest order) on
`/api/packets` and the closely-related endpoints. `id` is monotonic with
ingest time and immune to buffered uploads.

Endpoints changed (all use the same fix for the same reason):

| Path | Function | File |
|------|----------|------|
| `GET /api/packets` (default) | `DB.QueryPackets`, `Store.QueryPackets`
| `cmd/server/db.go`, `cmd/server/store.go` |
| `GET /api/packets?nodes=…` | `DB.QueryMultiNodePackets`,
`Store.QueryMultiNodePackets` | same |
| Node detail "recent transmissions" |
`DB.GetRecentTransmissionsForNode` | `cmd/server/db.go` |

## `since=` semantic — preserved
`since=` still filters by `first_seen` (RFC3339 path uses the
observations.timestamp subquery), i.e. "packets the network received
since X." Buffered uploads of older packets are still excluded from a
`since=15m` view even if they were ingested in the last 15 minutes. Only
the **display order** changes; filtering by receive time is unchanged.

## Audit — NOT changed
- `Store.QueryGroupedPackets` already sorts by `LatestSeen` (max
observation timestamp), which is correct for the grouped view and immune
to the buffered-upload regression.
- `GetChannelMessages` and channel `sample_json` subqueries keep
`first_seen DESC` — channel message chronology is meaningful for message
UX; if buffered uploads become a problem here too it's a separate UX
call (out of scope for #1345).
- `s.packets` insertion ordering (Load + ingest) — untouched. The fix
sorts at query time so we don't perturb `oldestLoaded` invariants.

## Tests — TDD red → green
- Red: `508f4371` adds `cmd/server/packets_order_test.go` with two cases
— order assertion (failed on master with `[fresh, buffered]`) and
since-filter semantic (RFC3339 path uses observation timestamps).
- Green: `0fd685e7` switches the SQL + in-memory ordering. Tests pass;
full `cmd/server` suite green locally (44s).

## Out of scope
- Re-thinking #1233's first_seen semantics
- Adding a UI sort toggle (issue's option 2)
- Channel-message page ordering

## Preflight
Clean (`bash ~/.openclaw/skills/pr-preflight/scripts/run-all.sh
origin/master`).

---------

Co-authored-by: openclaw-bot <bot@openclaw.local>
2026-05-25 22:32:00 -07:00
Kpa-clawbot dc6c79cff8 fix(mqtt): watchdog forces paho reconnect on stall — recovers from half-open TCP (closes #1335) (#1336)
RED `f06887` — GREEN `8f53c1`. CI: (will populate on PR open)

`Fixes #1335`

## Problem
PR #1216 added per-source stall **detection** (`LivenessStalled`) but
only **logged**. Staging's `lincomatic` source has been silently losing
~14k pkts/hr behind a half-open TCP socket the Azure NAT abandons: paho
reports `IsConnected==true`, no messages arrive for 1h+, container
restart is the only known recovery. Prod (MikroTik networking) doesn't
see it.

## Fix
Make the watchdog actually recover.

- **`SourceLivenessState.ForceReconnectFn`** — per-source closure wired
in `main.go` next to `IsConnectedFn`, wraps `client.Disconnect(250) +
client.Connect()`.
- **`processLivenessTransition`** — on the `LivenessStalled` edge AND on
every heartbeat re-emit while still Stalled, invoke
`maybeForceReconnect`. `LivenessNeverReceived` (cold-start ACL deny /
wrong hash) is **deliberately not** force-reconnected — a new TCP socket
won't fix an ACL deny and would just churn the broker.
- **`maybeForceReconnect`** — throttled at `forceReconnectThrottle =
60s` per source so a stall→reconnect→re-stall loop self-recovers without
hammering the broker. The Disconnect+Connect runs in a goroutine so a
single slow source can't stall the watchdog tick.
- **`buildMQTTOpts`** — explicit `SetKeepAlive(30 * time.Second)`.
paho's default happens to be 30s, but the #1335 RCA called this out —
making it explicit so it can't drift and so operators reading the code
know it's intentional.
- **Telemetry** — `WATCHDOG forcing reconnect` (intent), `WATCHDOG
reconnect attempt issued` (post-goroutine), `WATCHDOG suppressing forced
reconnect` (throttle window).

## TDD
- **RED** `f06887` — `mqtt_watchdog_force_reconnect_test.go`. Stub field
+ constant added so the file compiles; assertions fail because
`processLivenessTransition` never invokes `ForceReconnectFn`. Reverting
just the `s.ForceReconnectFn()` call line from GREEN re-fails the same
assertion (mutation verified).
- **GREEN** `8f53c1` — wiring + throttle + keepalive.

## Scope discipline
Additive only. No regression to currently-flowing sources: `LivenessOK`,
`LivenessRecovered`, `LivenessDisconnected`, `LivenessHeartbeat`, and
`LivenessNeverReceived` transitions are unchanged. Throttle bound = ≤1
reconnect/min/source = ≤60/hr worst-case across all sources, well within
any broker rate limit.

Preflight: clean (all gates pass).

---------

Co-authored-by: openclaw-bot <bot@openclaw.local>
2026-05-25 22:31:56 -07:00
Kpa-clawbot 2ea84e2237 chore(agents): codify 'no new map[string]interface{}' rule from #1383 (#1384)
Adds a "What NOT to Do" entry to `AGENTS.md` codifying the
no-new-`map[string]interface{}` rule from #1383.

Every subagent brief in this project requires `AGENTS.md` as step 1;
this puts the rule in front of every future contributor automatically.

Rule text:
> Don't introduce new `map[string]interface{}` in API response builders,
handler returns, or internal data structures that cross domain
boundaries. Use a named Go struct with explicit JSON tags. CoreScope
already carries 694 occurrences (see #1383); the count must
monotonically decrease. If your change adds even one new occurrence in a
touched file, the PR is wrong-shaped — fix the design, don't paper over
with `interface{}`. Exempt: third-party library boundaries that
genuinely return `interface{}`, and ad-hoc test fixture assertions.

Refs #1383.

Co-authored-by: CoreScope Bot <bot@corescope>
2026-05-26 05:31:53 +00:00
Kpa-clawbot ec98a43d68 feat(ci): frontend eslint no-undef gate — catches renamed-function-caller class of bugs (fixes #1342) (#1344)
**TDD:** red commit `03ea965` (canary undef var → CI fails) → green
commit `b514aeb` (canary removed → CI passes). CI URL appears in the
Checks tab once GitHub Actions queues this branch.

`Fixes #1342`

## What ships

- **`.eslintrc.json`** at repo root — eslint 8 legacy-config format.
`no-undef: error`, `no-unused-vars: warn` (with `^_` allowlist).
- **CI step** in `.github/workflows/deploy.yml` (job `go-test`, after JS
unit tests, before proto + Playwright): `npm install --no-save eslint@8
&& npx eslint public/*.js`. `--no-save` keeps `node_modules` and
`package-lock.json` out of the tree (already gitignored).
- **One pre-existing fix** in `public/map.js`: `typeof esc ===
'function'` → `typeof globalThis.esc === 'function'`. `esc` is a *local*
IIFE var in 5 other files, never exported as a true global; the optional
lookup was structurally invalid under `no-undef`. Behavior unchanged.

## How this would have caught #1318 / PR #923

PR #923 renamed `drawAnimatedLine`, updated one caller in
`public/live.js`, missed the other — leaving a reference to the
undefined `hash` var. Playwright didn't hit that path. Reverting #1325
locally (re-introducing the bug) → eslint flags `hash` as `no-undef` →
red. With the gate in place, #923 never lands.

## The "quiet pile of globals" reality

The config declares **257 globals**. They were discovered by walking
`public/*.js` for two patterns:
1. `window.X = ...` assignments (the explicit exports — 168 of them)
2. Top-level `function`/`const`/`let`/`var` declarations in non-IIFE
files (the implicit exports — Go-style cross-file linking via shared
HTML `<script>` order)

Plus 9 vendor/runtime names (`L`, `Chart`, `QRCode`, `qrcode`, `module`,
`global`, `process`, `require`, `exports`, `__filename`, `__dirname`)
for dual-runtime files like `url-state.js`, `packet-filter.js`,
`hash-color.js`, `filter-ux.js` that are also `require()`-d by Node
tests.

This is honest documentation of an architectural reality, not a
workaround. Future refactor → modules will collapse this list.

## Latent bugs discovered

**Zero `no-undef` errors against the current `public/*.js` tree** after
globals were enumerated honestly. The would-be-#1318-class bug count
today: 0. The gate's job is forward-looking — block the next one.

## Out of scope (acknowledged from acceptance criteria)

- Inline `<script>` blocks in `public/*.html` — separate ticket.
- Per-PR delta-coverage gate — separate ticket.
- pr-preflight grep for arg-count mismatch — separate ticket.

## Preflight

`bash ~/.openclaw/skills/pr-preflight/scripts/run-all.sh origin/master`
→ exit 0, clean.

---------

Co-authored-by: openclaw-bot <bot@openclaw.local>
2026-05-25 22:31:40 -07:00
Kpa-clawbot 791c8ae1bc fix(#1367): channels page chat-app redesign — restore prod row layout, drop analytics chip, add detail view (#1376)
Red commit: ae8838ef (CI: pending — see Checks tab once attached)

## What
Channels page mobile UX overhaul (#1367). Restores prod's chat-app row
layout, drops the analytics chip, and adds a per-channel detail view.

## Status
Draft — RED commit on the wire. Greens will follow in subsequent commits
before this is moved to Ready.

Fixes #1367

---------

Co-authored-by: bot <bot@example.com>
2026-05-25 22:30:19 -07:00
Kpa-clawbot bfebf200b7 fix(#1375): scope-stats fetch path — drop duplicate /api prefix (Scopes tab JSON.parse fix) (#1379)
## What

Drop the leading `/api` from the Scopes-tab `scope-stats` fetch in
`public/analytics.js`. The `api()` helper already prefixes `/api`;
passing `/api/scope-stats` produced a runtime URL of
`/api/api/scope-stats`, which 404s, falls through to the SPA HTML, and
crashes the Scopes tab with `JSON.parse: unexpected character`.

Single-line behavior change.

## Why

`api()` (defined earlier in the same file) prepends `/api`. Every other
caller in `public/analytics.js` correctly passes a helper-relative path
(`/observers`, `/nodes`, …). The Scopes loader was the lone offender.
The same fix originally landed on the PR #915 branch (commit `2fd22cee`)
but that branch never merged, so the bug resurfaced on subsequent
rebases.

The Scopes tab is therefore broken on production today — open
`/analytics` → Scopes and the panel never renders.

## TDD

- Red commit `b1fbc5601a985f20eb0ffee9181b7df5333248ca` adds
`test-issue-1375-scope-stats-fetch.js`, which reads
`public/analytics.js` and asserts:
  - ZERO matches of literal `api('/api/scope-stats'` (regression guard).
  - Exactly one match of `api('/scope-stats'` (positive — fix present).
- Green commit edits the loader to drop the duplicate `/api`.
- Test wired into `.github/workflows/deploy.yml` next to the existing
`test-issue-*` entries.

## Manual verification

After deploy, open `https://analyzer.00id.net/analytics`, click
**Scopes**: panel renders cards instead of throwing a JSON parse error
in DevTools console.

Fixes #1375

---------

Co-authored-by: openclaw-bot <bot@openclaw.local>
2026-05-25 22:16:17 -07:00
Kpa-clawbot 88bc5d9d3b fix(#1373): drop ghost "unknown" channel bucket from /api/channels for encrypted-no-key packets (#1377)
## What

Drops the ghost `unknown` channel bucket from `/api/channels` for
encrypted GRP_TXT packets whose decoded JSON sets `channel=""` (server
has no PSK to decrypt). Fix A from issue #1373 — cosmetic / immediate.
Fix B (server-side decryption / key sharing) is intentionally out of
scope and remains for a follow-up issue.

## Why

When an operator adds a PSK channel key client-side (via the channel
customizer), the channel list shows the newly-decrypted channel
correctly — but it ALSO shows a stale `unknown` bucket holding the SAME
packets the new channel just decrypted. The bucket is a server-side
debug catch-all (`if channelName == "" { channelName = "unknown" }`)
that leaks into the user-facing channel list. It's not a real channel;
dropping it from `/api/channels` is the right fix until/unless
server-side decryption lands.

Choice made: keep the `channelName = "unknown"` fallback path removed by
adding an early `continue` BEFORE the bucket is created. This keeps the
diff minimal, preserves the `hasGarbageChars` filter ordering, and makes
the intent obvious ("encrypted-no-key packets are not channels"). The DB
path (`cmd/server/db.go`) already filters NULL `channel_hash` at the SQL
level and `continue`s on empty; the test pins that contract.

## TDD

- Red commit: `35b8ba51c74dcc6200d5cf4a87dc7a0b63b2b2c2` — seeds 5
encrypted GRP_TXT (Channel="") + 3 decrypted (#real) into both
PacketStore and DB paths; asserts `GetChannels` returns exactly 1
channel (#real). Fails on assertions, not compile.
- Green commit: see follow-up commit on this branch — drops the
`"unknown"` fallback in `cmd/server/store.go` `GetChannels`; DB path
unchanged (already correct, test pins it).

## Manual verification (staging)

After deploy, on a staging instance with encrypted GRP_TXT traffic and
no PSKs configured:
1. `curl -s https://staging/api/channels | jq '[.[] | select(.name ==
"unknown")] | length'` → `0`
2. Real channels with known hashes still appear with correct
messageCount.

## Files changed

- `cmd/server/store.go` — drop the `if channelName == "" { channelName =
"unknown" }` fallback; skip the packet instead.
- `cmd/server/channels_no_unknown_bucket_1373_test.go` — new test
covering both code paths.

Fixes #1373

---------

Co-authored-by: openclaw-bot <bot@openclaw.local>
2026-05-25 22:16:14 -07:00
Kpa-clawbot 7742fbe7b1 ci: update go-server-coverage.json [skip ci] 2026-05-26 03:17:48 +00:00
Kpa-clawbot a6224e2325 ci: update go-ingestor-coverage.json [skip ci] 2026-05-26 03:17:47 +00:00
Kpa-clawbot 9f92b1331c ci: update frontend-tests.json [skip ci] 2026-05-26 03:17:46 +00:00
Kpa-clawbot d7dd2dca1e ci: update frontend-coverage.json [skip ci] 2026-05-26 03:17:45 +00:00
Kpa-clawbot 7f9bad452f ci: update e2e-tests.json [skip ci] 2026-05-26 03:17:44 +00:00
Kpa-clawbot 0f7cce3a5f fix(#1370): revert ingestor envelope-timestamp path — server ingest time for packet/observation storage (counters #1233) (#1372)
## Summary

Reverts the part of PR #1233 (commit `498fbc03`) that routed the MQTT
envelope's `timestamp` field into `PacketData.Timestamp` for
`transmissions.first_seen` and `observations.timestamp`. Packet
ordering is restored to server ingest time — the client clock is
untrusted.

`UpsertObserverAt` + `MAX(MIN(existing, ingestNow), rxTime)` for
observer/node `last_seen` (PR #1233's other half) is preserved
unchanged. `parseEnvelopeTime` / `resolveRxTime` helpers are
preserved — they still feed the observer.last_seen path.

## Diagnosis — Voodoo3 tx 304114 on staging

Staging `tx_id = 304114` in channel `#test` has 5 observations:

| # | observer  | reported timestamp | comment |
|---|-----------|--------------------|---------|
| 1 | Voodoo3 | 18:42 | broken client RTC — ingested first, locks
`first_seen` |
| 2 | Voodoo3   | 18:42  | broken client RTC |
| 3 | Voodoo3   | 18:42  | broken client RTC |
| 4 | Voodoo3   | 18:42  | broken client RTC |
| 5 | other obs | 01:42  | genuine receive time |

4 of 5 observations carry stale 18:42 timestamps from Voodoo3's own
broken clock. Because Voodoo3 ingested first, PR #1233's code wrote
`transmissions.first_seen = 18:42` (envelope value). Downstream
aggregators that compute `MAX(first_seen)` per channel saw 18:42 as
the latest activity, and `/api/channels` for `#test` displayed
`lastActivity` ~7h+ in the past plus a stale heartbeat in the row
preview — hiding the genuinely-newest message (Voodoo3's `tst hmdpt`
at 01:42).

## Why PR #1233's premise fails

PR #1233 assumed:
> Uploaders stamp `timestamp` when the radio receives the frame and
> freeze it; the MQTT message is published late, but the timestamp
> field is not re-stamped at publish. A buffered packet uploaded
> hours late still carries its true receive time.

That holds ONLY when the uploader's wall clock is correct. Observers
in the field (Voodoo3 here, surely others) have broken local clocks.
Their envelope timestamps are not a true receive time — they're a
broken-clock receive time, which is just garbage with extra steps.
The server clock is the only one we control, so packet ordering must
use it.

## Fix

### `cmd/ingestor/db.go`
- `BuildPacketData`: `PacketData.Timestamp =
time.Now().UTC().Format(time.RFC3339)`,
  NOT `msg.Timestamp`. Docstring updated to cite #1370 and explain
  why `msg.Timestamp` is no longer read here.

### `cmd/ingestor/main.go`
- Channel-companion path: `Timestamp: ingestNow` (was `rxTime`).
- DM-companion path: `Timestamp: ingestNow` (was `rxTime`).
- Local `rxTime := resolveRxTime(msg, tag)` removed from both paths
  (no remaining consumers in those scopes).

### Preserved (NOT touched)
- `resolveRxTime`, `parseEnvelopeTime` — still used by `handleMessage`
  to populate `mqttMsg.Timestamp` and to call `UpsertObserverAt`,
  which feeds `observer.last_seen` and `observer.last_packet_at`.
- All three `MAX(MIN(existing, ingestNow), rxTime)` guards (#1233
  observer.last_seen, observer.last_packet_at, node.last_seen).
- `MQTTPacketMessage.Timestamp` struct field.

## Tests

| File | Asserts |
|------|---------|
| `cmd/ingestor/ingest_time_regression_1370_test.go` (3 cases) |
Raw-packet, channel-companion, and DM-companion `handleMessage` paths.
Feed envelope `timestamp = T_now - 7h`; assert stored
`transmissions.first_seen` (RFC3339) and `observations.timestamp`
(epoch) are server wall clock (±5s). Each case fails on master under PR
#1233's premise. |

### Adjusted test
- `cmd/ingestor/db_test.go::TestBuildPacketData` — PR #1233 had asserted
  `pkt.Timestamp == "2026-05-16T10:00:00Z"` (the envelope value
  propagating). Now asserts the opposite: `pkt.Timestamp` is non-empty
  AND is NOT the envelope value. Comment cites #1370 and why the
  expectation flipped.

### Verified still-green
- `cmd/ingestor/rxtime_test.go` (`TestParseEnvelopeTime`,
  `TestResolveRxTime`) — helpers untouched, still cover envelope
  parsing for the observer.last_seen path.
- `cmd/server/channels_message_order_1366_test.go` (#1366).
- `cmd/server/db_channel_messages_perf_test.go` (#1368 perf budget).

## Commits

- `a9b7efc3` — RED: 3 `handleMessage` assertion-fail tests + test name
  collision check.
- `5a0891f0` — GREEN: revert envelope→PacketData.Timestamp plumbing in
  `cmd/ingestor/{db,main}.go` + flip `TestBuildPacketData`.

Fixes #1370

---------

Co-authored-by: corescope-bot <bot@corescope.dev>
2026-05-25 19:56:49 -07:00
Kpa-clawbot c0c5b66ca9 ci: update go-server-coverage.json [skip ci] 2026-05-26 01:05:12 +00:00
Kpa-clawbot 954148ae8e ci: update go-ingestor-coverage.json [skip ci] 2026-05-26 01:05:11 +00:00
Kpa-clawbot 988f64a27d ci: update frontend-tests.json [skip ci] 2026-05-26 01:05:10 +00:00
Kpa-clawbot b81256976c ci: update frontend-coverage.json [skip ci] 2026-05-26 01:05:09 +00:00
Kpa-clawbot ddc353aab7 ci: update e2e-tests.json [skip ci] 2026-05-26 01:05:08 +00:00
Kpa-clawbot c7ab5f3eb9 fix(#1366): channels view shows latest message time — backend emits LatestSeen, not FirstSeen (#1368)
Red commit: 702d82eb5e (CI: see Actions
tab for fix/issue-1366)

## What
Channel view emits the max observation timestamp (`tx.LatestSeen`)
instead of the analyzer's first-observation time (`tx.FirstSeen`) as the
rendered `timestamp` field. A new `first_seen` field is exposed
alongside for debug surfaces. `sender_timestamp` continues to be
returned in the JSON response but is intentionally NOT used as the
rendered time (client clocks are unreliable).

## Root cause

Two parallel call sites both emitted the wrong field:

- `cmd/server/store.go` — `GetChannelMessages` (~line 4807): set
`entry.Data["timestamp"] = strOrNil(tx.FirstSeen)` for every new dedup
entry. `tx.FirstSeen` is the analyzer's first-ever observation time of a
`transmissions.hash` row; for heartbeat-style packets (e.g. `BlorkoBot
🤖` posting the same status line periodically), the hash is stable, so
FirstSeen stays pinned at the very first observation while the message
keeps retransmitting hours later. Operator sees "old" message timestamps
for live messages.
- `cmd/server/db.go` — `GetChannelMessages` (~line 1757): same problem
against the SQLite-backed query path. Used `nullStr(fs)` (where `fs` is
`t.first_seen`) for the `timestamp` field.

### Repro from staging
Same packet, same hash `aba4f0493249de57`, sender `BlorkoBot 🤖`:
- `/api/channels/%23test/messages` → `timestamp: "2026-05-25T15:53:20Z"`
(FirstSeen, 7h+ in the past)
- `/api/packets?hash=aba4f0493249de57` → `first_seen:
"2026-05-25T22:53:19Z"` (latest obs), `observation_count: 84`

The packets view used max-obs correctly; the channels view did not. 7h
gap matches operator screenshot.

## TDD red → green

Red: `cmd/server/channels_message_order_1366_test.go` — three tests:
- `TestChannelMessages_TimestampUsesLatestSeen`: seeds a CHAN tx with
observations 7h apart, asserts returned `timestamp` ≈ latest observation
epoch (±1s). Fails under FirstSeen with Δ=−25200s.
- `TestChannelMessages_TimestampNotSenderTimestamp`: seeds a CHAN tx
whose decoded `sender_timestamp` is year-2000 (bad RTC). Asserts the
rendered `timestamp` parses to current year — guards against the
tempting "just use sender_timestamp" alt-fix that would let bad client
clocks corrupt the view.
- `TestChannelMessages_TimestampIsUTCZ`: asserts the emitted string is
unambiguously UTC (suffix `Z` or `+00:00`) so browsers don't apply a
local-zone shift.

Green commit changes:
- `store.go`: emit `tx.LatestSeen` (with FirstSeen fallback if no obs);
add `first_seen` field.
- `db.go`: join `o.timestamp` per-observation, track max epoch per tx,
emit RFC3339 UTC at the end; add `first_seen` field.

`sender_timestamp` remains in the response — unchanged shape, frontend
never read it for the rendered time (verified: only `msg.timestamp` is
consumed in `public/channels.js:1902`).

## Manual verification (post-merge)

1. Deploy to staging.
2. Curl `/api/channels/%23test/messages?limit=5` and
`/api/packets?hash=<recent>`. The channel `timestamp` field MUST equal
the packets `first_seen` (max obs) for the same hash, NOT lag it.
3. Send a fresh GRP_TXT via a MeshCore client into a watched channel.
Within 15s, refresh the Channels view at `/channels`. The new message
MUST render at the bottom with the correct (current) time.

## Why not `sender_timestamp`?

It's a per-client field, decoded from the payload. Many MeshCore
firmware builds run without RTC/NTP/GPS and report bogus values.
Trusting it for display would propagate bad client clocks into the
analyzer UI — the analyzer is the source of truth for UTC, not the
client.

Fixes #1366

---------

Co-authored-by: CoreScope Bot <bot@corescope>
Co-authored-by: bot <bot@kpa-clawbot.dev>
Co-authored-by: openclaw-bot <bot@openclaw.local>
2026-05-25 17:45:32 -07:00
Kpa-clawbot fa52c0887e ci: update go-server-coverage.json [skip ci] 2026-05-25 22:22:21 +00:00
Kpa-clawbot 73d9f06f9a ci: update go-ingestor-coverage.json [skip ci] 2026-05-25 22:22:21 +00:00
Kpa-clawbot ea849d226a ci: update frontend-tests.json [skip ci] 2026-05-25 22:22:19 +00:00
Kpa-clawbot cf74d6cfa4 ci: update frontend-coverage.json [skip ci] 2026-05-25 22:22:18 +00:00
Kpa-clawbot 7906524340 ci: update e2e-tests.json [skip ci] 2026-05-25 22:22:17 +00:00
Kpa-clawbot 91d90d48fb fix(#1364): drop over-aggressive .mc-pill max-width — restore multi-digit count visibility (#1365)
Red commit: 482ffe69e6 (CI: pending)

## What

Drops `max-width: 4ch` from `.mc-cluster .mc-pill` in
`public/style.css`. Keeps `overflow: hidden` + `text-overflow: ellipsis`
as belt-only graceful degradation.

## Why

#1362 added `max-width: 4ch` as defense-in-depth for the `999+` JS cap.
But `4ch` is applied to the BOX including the `1px 3px` padding, so
effective text width is ~2.5ch — enough for `R6` but not `R60`. Result:
post-merge regression on staging where multi-digit cluster pills render
`R…` instead of `R60`/`C30`.

The JS cap in `public/map.js` already clamps counts to `999+` (max 5
chars: `R999+`). That's the load-bearing safety. The CSS `max-width` was
overcaution and went too aggressive. Option A from the issue: drop the
cap entirely, keep ellipsis as graceful-degrade if JS ever fails.

## TDD red→green

- RED: `test-issue-1364-pill-no-clamp.js` asserts `.mc-pill` CSS does
NOT contain `max-width: 4ch` (regression guard) and DOES contain
`overflow: hidden` + `text-overflow: ellipsis` (graceful degradation).
Fails on the unchanged CSS.
- GREEN: deletes the `max-width: 4ch;` line from `.mc-pill`. Test
passes.

Wired into `.github/workflows/deploy.yml` alongside the #1360 test.

## Visual verification

Open `/map` zoomed-out on staging. Cluster pills must render full counts
(`R60`, `C30`, `R250`, capped `R999+`) — no `R…` ellipsis. No horizontal
scrollbar even on synthetic 4-digit injection.

Fixes #1364

---------

Co-authored-by: openclaw-bot <bot@openclaw.local>
2026-05-25 14:56:43 -07:00
Kpa-clawbot 78da393737 ci: update go-server-coverage.json [skip ci] 2026-05-25 20:51:26 +00:00
Kpa-clawbot 83feae228a ci: update go-ingestor-coverage.json [skip ci] 2026-05-25 20:51:25 +00:00
Kpa-clawbot a279ab736c ci: update frontend-tests.json [skip ci] 2026-05-25 20:51:24 +00:00
Kpa-clawbot 3bb9dc16ef ci: update frontend-coverage.json [skip ci] 2026-05-25 20:51:23 +00:00
Kpa-clawbot 2e08305b1d ci: update e2e-tests.json [skip ci] 2026-05-25 20:51:22 +00:00
Kpa-clawbot 40aa02b438 fix(#1360): cluster pill shows letter+count — restore count visibility regressed by #1357 (#1362)
Red commit: c0de33a952 (CI:
https://github.com/Kpa-clawbot/CoreScope/actions/runs/26416117686)
Green commit: c268248d — CI:
https://github.com/Kpa-clawbot/CoreScope/actions/runs/26416069319

## What

Fix #1360 regression: cluster role pills on `/map` show ONLY the role
letter (R/C/M/S/O); the per-role count number that was visible pre-#1357
is gone. This PR restores the count by concatenating it after the letter
inside the pill body, so each pill renders as `R60`, `C30`, `M5`, etc.

- `public/map.js` `makeClusterIcon`: pill body becomes `letter + n` (was
`letter`).
- `aria-label` / `title` (`"60 repeaters"`) untouched — already correct.
- DOM, classes, CSS, `--mc-*` constants, border-style ramp, multi-byte
labels — untouched.

### Adversarial follow-up (commit on top of green)

- **JS cap**: `makeClusterIcon` clamps `n > 999` → `"999+"`, so
pathological clusters render as e.g. `R999+` instead of `R10000`. Pill
width stays bounded.
- **CSS guard** on `.mc-pill`: `max-width: 4ch; overflow: hidden;
text-overflow: ellipsis;` as defense-in-depth if a render slips past the
JS cap.
- **+3 test assertions**: one for the JS cap, two for the CSS guard.
Mutation-verified (removing the cap fails ONLY the new cap assertion).

## Why

#1357 fixed WCAG 1.4.1 for cluster role pills by promoting the role
letter to the pill body, but in doing so dropped the count number that
sighted operators relied on for at-a-glance per-role counts. The letter
is the WCAG carrier; the count is the data. Both belong in the pill body
— they always did before #1357. The audit's intent was to PAIR them, not
REPLACE one with the other.

## TDD red→green

- **Red** (`c0de33a9`): added `test-issue-1360-pill-letter-count.js`
with assertions that pill body concatenates `letter + n` and is no
longer the bare `letter`. Fails by assertion against current `master`.
Red CI:
https://github.com/Kpa-clawbot/CoreScope/actions/runs/26416117686
- **Green** (`c268248d`): one-line change in `public/map.js` (`letter +
'</span>'` → `letter + n + '</span>'`). All assertions pass. Green CI:
https://github.com/Kpa-clawbot/CoreScope/actions/runs/26416069319
- **Follow-up** (this push): JS `"999+"` cap + CSS width guard + 3 new
assertions. #1356 (40), #1293, and `marker-outline-weight` tests remain
green.
- New test wired into `.github/workflows/deploy.yml` right after
`test-issue-1356-map-a11y.js`.

## Visual verification

Open https://analyzer.00id.net/#/map after deploy and confirm cluster
pills display `R<count>`, `C<count>`, `M<count>`, etc. (e.g. `R60 C30
M5`) instead of bare letters. `aria-label="60 repeaters"` remains for
screen readers. For very large clusters, pills cap at `R999+` / `C999+`
etc.

Fixes #1360

---------

Co-authored-by: openclaw-bot <bot@openclaw.local>
Co-authored-by: CoreScope Bot <bot@corescope>
2026-05-25 12:59:55 -07:00
Kpa-clawbot e545f315ca ci: update go-server-coverage.json [skip ci] 2026-05-25 18:58:40 +00:00
Kpa-clawbot f798b59c4d ci: update go-ingestor-coverage.json [skip ci] 2026-05-25 18:58:39 +00:00
Kpa-clawbot 0e305d880d ci: update frontend-tests.json [skip ci] 2026-05-25 18:58:38 +00:00
Kpa-clawbot e7debe7b13 ci: update frontend-coverage.json [skip ci] 2026-05-25 18:58:37 +00:00
Kpa-clawbot 1b7dc34e74 ci: update e2e-tests.json [skip ci] 2026-05-25 18:58:36 +00:00
Kpa-clawbot 933ef4e6ef fix(#1356): WCAG 2.2 AA map a11y — cluster bubbles, role pills, multi-byte labels (#1357)
Red commit: d48c1add88 (CI:
https://github.com/Kpa-clawbot/CoreScope/actions/runs/26411462973)

Green commit CI:
https://github.com/Kpa-clawbot/CoreScope/actions/runs/26411699037

## What

Brings the map's three visual surfaces — cluster bubbles, role pills
inside cluster bubbles, and multi-byte hash labels on repeater markers —
up to WCAG 2.2 AA. Replaces the prior color-only signaling with
structural carriers (size, border-style, glyph, letter prefix) so color
is no longer the only channel.

## How

Locked design = Tufte's structural framing ([issue
comment](https://github.com/Kpa-clawbot/CoreScope/issues/1356#issuecomment-4535244400))
WITH the WCAG audit's "Minimal patch to reach AA" applied as overrides
([issue
comment](https://github.com/Kpa-clawbot/CoreScope/issues/1356#issuecomment-4535849354)).
Where the audit and the original proposal disagreed (border color, pill
text color, V3 accent palette, font sizes), the audit's values won.

## V1 cluster bubbles

- Neutral fill `rgba(33,41,54,0.92)` via new `--mc-cluster-fill` (was
per-bucket `--info / --warning / --accent`).
- Border-style ramp as the redundant non-color carrier of the count
bucket: `mc-sm` `1.5px solid`, `mc-md` `2.5px solid`, `mc-lg` `2px
double`.
- Border color `#666` + dark halo `box-shadow: 0 0 0 1px
rgba(0,0,0,0.5), 0 1px 2px rgba(0,0,0,0.35)` so the border edge is
visible against both Carto Positron (`#f8f9fa`) and Carto Dark Matter
(`#262626`).
- `<div role="img" aria-label="<n> nodes — <breakdown>">` with the count
+ pills wrapped `aria-hidden="true"` so the AT announcement is the
summary, not the literal glyphs.

## V2 role pills

- `ROLE_LETTERS` map (`R` / `C` / `M` / `S` / `O`) is the primary
carrier — visible inside every pill, so protanopes/deuteranopes can read
the role without depending on hue.
- Wong (2011) palette as the secondary carrier, declared as
`--mc-role-repeater/companion/room/sensor/observer` — does NOT touch the
reserved `--info / --warning / --accent` system vars.
- `color: #1a1a1a` on **all five** pills (CSS rule + inline
defense-in-depth). Passes SC 1.4.3 small-text (≥4.5:1) against every
Wong hue.
- Font now `0.625rem/1.1 ui-monospace` (was `9px`, audit bumped to
`10px`, this PR converts to `rem` so user font-size preferences scale
the pill).
- Per-pill `aria-label="<n> <role>s"`, `overflow: visible` so a user
`letter-spacing` override doesn't clip (SC 1.4.12).

## V3 multi-byte hash labels

- `MB_GLYPHS` prefix (`✓` / `?` / `✗`) is the primary non-color status
carrier; the hash text is the data.
- Neutral dark fill `--mc-mb-fill` + colored 3px left border via
per-status `--mc-mb-confirmed/suspected/unknown` (high-luminance set
`#56F0A0` / `#FFD966` / `#FF8888` — audit override of original Tol
"vibrant" set, which failed border-stripe SC 1.4.11).
- Font now `0.75rem/1.2 ui-monospace` (was `11px`, audit bumped to
`12px`, this PR converts to `rem` for SC 1.4.4 robustness).
- `<div role="img" aria-label="multi-byte <status>, hash <ID>"><span
aria-hidden="true">` so AT reads the meaningful label (not the literal
`✓ 3E`). Observer-overlay `★` carries `aria-hidden="true"` for the same
reason. Null `mbStatus` falls through to `"repeater hash <ID>"` cleanly
— no `"multi-byte undefined"`.
- Forced-colors graceful degradation via `@media (forced-colors:
active)` block mapping all three surfaces to `Canvas` / `CanvasText`
with `forced-color-adjust: auto` (NOT `none`).

## TDD red→green

| Commit | Files | CI |
|---|---|---|
| `d48c1add` (red) | `test-issue-1356-map-a11y.js`,
`.github/workflows/deploy.yml` (test + wiring only) | [**failure** — 27
assertion ✗, exit
1](https://github.com/Kpa-clawbot/CoreScope/actions/runs/26411462973) |
| `b94755e6` (green) | `public/map.js`, `public/style.css`,
`test-issue-1356-map-a11y.js` (impl) |
[**success**](https://github.com/Kpa-clawbot/CoreScope/actions/runs/26411699037)
|
| `ac63e6ab` | refactor: drop `MB_COLORS` alias, hoist `MB_MARKER_TINT`
(round-1 #3 + #4) | (round-2) |
| `8aad60cb` | style: font sizes to `rem` for SC 1.4.4 (round-1 #2) |
(round-2) |
| `50a1aab1` | test: round-1 coverage adds + de-tautologise V2.c / V3.h
(round-1 #5) | (round-2) |

Red commit failed on **assertions** (not compile error) — the harness
loaded `public/map.js` + `public/style.css` end-to-end and exhausted all
27 string-presence checks. Green commit lands the audit-overridden
design and clears 32/32. Round-2 commits extend coverage to 40/40
without altering the original red→green gate.

## WCAG SC addressed

- **SC 1.4.1 Use of Color (A)**: cluster size + border-style ramp; pill
capital-letter prefix; MB label glyph prefix. Every visual is now
carried by at least one non-color channel.
- **SC 1.4.3 Contrast Minimum (AA)**: cluster `#fff` count on composited
fill = 10.12:1 vs Positron / 14.64:1 vs Dark Matter. MB label text =
11.48:1 / 14.65:1. Pill `#1a1a1a` on Wong hues: R 5.43, C 9.10, M 6.14,
S 13.16, O 6.86 — all ≥4.5:1.
- **SC 1.4.11 Non-text Contrast (AA)**: cluster border `#666` = 4.83:1
vs Positron, 3.30:1 vs Dark Matter; MB stripes vs `--mc-mb-fill`:
`#56F0A0` 5.13, `#FFD966` 8.66, `#FF8888` 4.62. Stripe-vs-basemap edge
is mitigated by the 1px dark halo box-shadow on `.mc-mb-label`.
- **SC 1.3.1 Info & Relationships (A)**: every divIcon now has
`role="img"` + a descriptive `aria-label`; visible glyph spans are
`aria-hidden="true"` so AT reads the meaning, not the typography.
- **SC 1.4.5 Images of Text (AA)**: implemented surfaces use live text
(`<span>` + `<div>` with CSS font), not rasterised glyphs — user
font-size / zoom scale them. Where SVG markers are used (non-label
path), the textual information is also exposed via `marker.alt` + popup,
satisfying the "essential" exception.

## Manual verification

1. **Both Carto themes on staging.** Open https://analyzer.00id.net and
switch the basemap (Positron and Dark Matter) — cluster bubbles, pills,
and MB labels must remain legible on both. Border edge of cluster bubble
visible on Positron (was the original bug).
2. **Screen-reader (NVDA / VoiceOver) test.**
- Focus a cluster bubble → expect `"<n> nodes — <role breakdown>"` and
NO literal letter/number announce per pill.
- Focus a MB label on a repeater marker → expect `"multi-byte confirmed,
hash 3E"` (or whatever status/hash applies) and NO `"check mark thin
space 3 E"`.
- Observer-also-repeater label → still announces the meaningful label
only; ★ is silent.
3. **Coblis simulation** (or equivalent). Run cluster + pills + MB
labels through deuteranopia / protanopia / tritanopia simulation.
Cluster bucket must be distinguishable by size + border-style (without
hue). Pill role must be distinguishable by the letter (without hue). MB
status must be distinguishable by glyph (without hue).
4. **Windows High Contrast / forced-colors.** Toggle on; all three
surfaces should fall back to `Canvas` / `CanvasText` (no invisible
elements, no `forced-color-adjust: none` regression).

## Out of scope

Filed for separate follow-up issues (audit explicitly tagged these as
either pre-existing or modern-interpretation non-blockers):

1. **SC 2.1.1 Keyboard (A)** — cluster click-to-zoom is mouse-only today
(Leaflet markercluster limitation). Needs `role="button"` + `tabindex=0`
+ `keydown` handler. Pre-existing, not introduced by this PR.
2. **SC 2.4.7 Focus Visible (AA)** — moot until #1 is addressed (no
focusable target). When the cluster becomes focusable, a
`:focus-visible` outline must be added.
3. **`prefers-reduced-motion` gate** — `.mc-cluster:hover { transform:
scale(1.06) }` and the 120ms transition are untouched from pre-PR.
Should be gated on `@media (prefers-reduced-motion: reduce)` in a
follow-up hygiene pass.
4. **px → rem for non-font sizes** — this PR converts font sizes (the SC
1.4.4 sensitive surface). Border widths and small paddings are kept in
px because physical-pixel snapping matters more for borders than user
font-zoom.

Fixes #1356

---------

Co-authored-by: Kpa-clawbot <bot@kpa-clawbot.local>
2026-05-25 11:38:50 -07:00
Kpa-clawbot bbd185a826 ci: update go-server-coverage.json [skip ci] 2026-05-25 15:13:30 +00:00
Kpa-clawbot e4c6246257 ci: update go-ingestor-coverage.json [skip ci] 2026-05-25 15:13:29 +00:00
Kpa-clawbot 30a20c388e ci: update frontend-tests.json [skip ci] 2026-05-25 15:13:28 +00:00
Kpa-clawbot 3170cbdea5 ci: update frontend-coverage.json [skip ci] 2026-05-25 15:13:26 +00:00
Kpa-clawbot de3424533c ci: update e2e-tests.json [skip ci] 2026-05-25 15:13:25 +00:00
Kpa-clawbot 0d131808d4 fix(map): thinner always-on marker outline — was dominating at zoomed-out levels (#1347)
## Operator feedback on #1334

PR #1334 (the #1293 marker a11y change) added a baked-in white outline
at `stroke-width=2` to every node marker via `makeRoleMarkerSVG`.
Operator reports it's too heavy and dominates the map at zoomed-out
levels — every node reads as a "big white blob with a colour core",
which actually drowns out the per-role shape silhouette at the exact
zoom levels where the shape distinction matters most.

## Fix

Drop the always-on stroke from **2 → 1** across all marker producers:

| Producer | Before | After |
|----------|--------|-------|
| `public/roles.js` `makeRoleMarkerSVG` (circle / square / triangle /
diamond / hexagon) | `stroke-width="2"` | `stroke-width="1"` |
| `public/roles.js` `makeRoleMarkerSVG` (star branch) |
`stroke-width="1.5"` | `stroke-width="1"` |
| `public/live.js` `addNodeMarker` inline fallback SVG |
`stroke-width="2"` | `stroke-width="1"` |
| `public/map.js` `makeMarkerIcon` switch (all shapes) |
`stroke-width="2"` / `"1.5"` | `stroke-width="1"` |
| `_highlightRing` (pulse on selected/active) | `weight: 3 → 2` |
**unchanged** |

The highlight ring used by `pulseNodeMarker` is the one place where a
heavy outline carries real signal (selected state), so it stays at
weight 3 → 2. The always-on shape stroke is now just enough to keep
silhouettes distinct on both Carto dark and light basemaps without
dominating the surrounding terrain.

## Constraints preserved

- Shape variation (#1293) — per-role shapes still rendered, helper
untouched except for stroke width.
- Colorblind palette — fills/colors unchanged, all via CSS variables /
`ROLE_COLORS`.
- Highlight ring still visible — pulse weight ≥ 2 retained and asserted.

## Tests

New: `test-marker-outline-weight.js` (added to `test-all.sh` unit suite)

- Asserts every `stroke-width` literal in `makeRoleMarkerSVG` is `<= 1`.
- Asserts `live.js` inline fallback SVG `stroke-width <= 1`.
- Asserts the `_highlightRing` (`ringHl.setStyle({ weight: N })`) keeps
at least one `weight >= 2` so highlight stays visible.

Red commit (`d17cfcc`) fails on assertion; green commit (`6cfe99b`)
flips it.

Existing `test-issue-1293-marker-shapes.js` still passes — the
shape-variation and outline-ring highlight contracts are intact.

---------

Co-authored-by: openclaw-bot <bot@openclaw>
2026-05-25 07:53:33 -07:00
Kpa-clawbot bfb652c1e8 ci: update go-server-coverage.json [skip ci] 2026-05-25 06:31:44 +00:00
Kpa-clawbot c1423ee5dd ci: update go-ingestor-coverage.json [skip ci] 2026-05-25 06:31:44 +00:00
Kpa-clawbot f4a1db023d ci: update frontend-tests.json [skip ci] 2026-05-25 06:31:43 +00:00
Kpa-clawbot c5c2b8c483 ci: update frontend-coverage.json [skip ci] 2026-05-25 06:31:42 +00:00
Kpa-clawbot 01f6a4707a ci: update e2e-tests.json [skip ci] 2026-05-25 06:31:41 +00:00