mirror of
https://github.com/Kpa-clawbot/meshcore-analyzer.git
synced 2026-11-13 00:38:08 +00:00
7c00a42947ff7670dbce3849285fd65d37191a31
72
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 efitenandClaude Opus 5.5
|
2d09335cfb |
fix(packets): keep the node filter pick on a slow click or late search (#2243) (#2244)
Fixes #2243. ## Problem The packets node filter selected a suggestion on `click`, while the input's `blur` handler hides the dropdown 200 ms after focus leaves. A press held longer than that, or a search response that rebuilt the list between mousedown and mouseup, lost the pick: the typed text stayed and no `node=` filter was applied. ## Change - `public/packets.js`: suggestions select on `mousedown` with `preventDefault`, so the input keeps focus and the pick happens before any rebuild. `live.js` already does this for its node filter. - `public/packets.js`: a sequence counter drops search responses for an older query, bumped on every search and on a pick, so a late response can't replace or reopen the list. - `tests/e2e/test-packets-node-filter-e2e.js` (new), registered in `scripts/non-unit-tests.json` and on shard 1 in `deploy.yml`. ## Tests The new E2E has three cases: a normal click, a 400 ms press, and a search response released mid-click. | Code | Normal click | 400 ms press | Response mid-click | |---|---|---|---| | master | pass | fail | fail | | this PR without the sequence guard | pass | pass | fail (list reopens) | | this PR | pass | pass | pass | Run locally against the freshened e2e fixture. `sh test-all.sh` passes (locally `test-preflight-xss-gate.js` needs a real `python3` on PATH and `PYTHONUTF8=1`, unrelated to this change). ## Not done - Full `test-e2e-playwright.js` not run locally. - Could not reproduce on the reporter's instance with the browser tool, whose clicks are shorter than 200 ms. The 400 ms Playwright press reproduces it on the fixture. - `live.js` and `packets.js` still carry two separate node-autocomplete implementations. Merging them is out of scope here. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
efitenandClaude Opus 5.5
|
a910a3e480 |
fix(packets): re-fetch when the observer filter changes (#2241) (#2242)
Fixes #2241. ## Problem In grouped mode, ticking an observer on `#/packets` updated the URL and the trigger label but left the table unchanged until a reload. Since #1821 `applyObserverFilter` leaves grouped rows to the server's EXISTS filter, while the observer menu's change handler still only called `renderTableRows()`, which re-renders the rows already loaded without a new `/api/packets` request. ## Change - `public/packets.js`: the observer menu change handler calls `loadPackets()` instead of `renderTableRows()`, so the request carries the new `observer=` set. - `tests/unit/test-observer-menu-interactions.js`: two tests asserting the handler re-fetches, for ticking one observer and for "All Observers". Both failed before the change. ## Verification - `sh test-all.sh`: all suites pass (locally `test-preflight-xss-gate.js` needs a real `python3` on PATH and `PYTHONUTF8=1`, unrelated to this change). - Browser, local server on the freshened e2e fixture: 55 rows; one observer ticked gives 2 rows, all from that observer; back to "All Observers" gives 53 rows. No reload in between. - Reproduced the original bug on an operator's v3.14.0 instance before the change. ## Not done - Playwright E2E suite not run locally. - Perf: one `/api/packets` request per checkbox toggle, the same request a reload or the node filter already makes. No per-item calls. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Alex BandClaude Opus 5.5
|
8b5ca5735c |
fix(analytics): Packets / Hour readable on long histories (#2193) (#2194)
Fixes #2193. ## Problem On an instance that keeps months of packets, the Overview tab's Packets / Hour chart becomes unreadable: 2,932 hourly bars in 720 units of plot, a date label every day, and the hours with no packets (which the server leaves out) squeezed off the axis. ## Before and after Both use the real `/api/analytics/rf` answer from the instance in the issue (2,933 hourly buckets over 124 days), served to the page in place of the fixture's own. **Before** (master, page on "All data"):  **After**, same data:  Phone: <img src="https://raw.githubusercontent.com/A13xB0/CoreScope/7862abd8/2026-10/pph-2193/after-phone.png" width="300"> **A short store** (an instance holding 7 days in memory, plus a few old packets a day for weeks before that because they were heard again). The chart starts where the real data does (this shot predates the "since MM-DD" note now in the title when that crop applies):  ## Change All in `public/analytics.js`; no API change. - **`packetsPerHourCard()`** replaces the inline block: - rolls the hours up to hourly, 6-hourly, daily or weekly bars, at most 180, and titles the card to match (Packets / Day); - counts missing hours as zero; - puts about 10 labels on round UTC times; - **under "All data" only**, starts on the first UTC day with at least 2% of a typical recent day (the median of the seven days before the last one), at its first hour with a tenth of a typical hour, and says "since MM-DD" in the title when that cuts anything. Scanning forward, an outage after the start is never taken for it. In a picked window (1h to 30d) the server has already cut the data, so nothing is cropped and a quieter early stretch stays visible. - **Theme refresh before the data** (separate commit; this also happens on master without the new chart): every load fires `theme-refresh` about 300 ms after the theme resolves, and the page re-rendered the current tab on it. When the analytics answers came later (a busy or just-restarted server), the Overview was drawn from an empty object and threw `Cannot read properties of undefined (reading 'totalTransmissions')`. It now skips that refresh until the data has landed; `loadAnalytics()` renders the tab with the current theme when it does. The span is still picked by the page's time window. ## Tests - `tests/unit/test-issue-2193-packets-per-hour.js` (in `test-all.sh`, red then green): 122 days gives 123 daily bars with 4 to 12 labels; 7 days gives 168 hourly bars; missing hours are drawn as zero; the chart starts at the data, not at the stragglers, and not after an outage; in a picked window nothing is cropped (90 days with 30 quiet ones gives 91 bars); a cropped chart says "since MM-DD" and an uncropped one does not; no data says so. - `tests/unit/test-analytics-theme-refresh-before-data.js` (in `test-all.sh`): drives `init()` with answers that never land, fires the `theme-refresh` handler, and checks that nothing throws and the tab is not re-rendered. Without the guard it fails with the production error. - `sh test-all.sh` passes; the #1925 neighbour-graph theme-refresh test and the analytics chart test still pass; the screenshots show no page errors and no sideways scroll at 1280 px or 390 px. ## Later - The 2% floor, the tenth-of-an-hour start and the 180-bar cap are hardcoded. Per AGENTS.md rule 8 they can move to the customizer in a later step. - Edge bars are partial periods (data starting at 17h gives a 7-hour first day; the last bar is still filling), and weekly bars align on epoch weeks (Thursday). Both are left as they are here. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01KaR4xu8gVykQBkH8yG8m6H --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
hermes-gadget
|
fdcec91578 |
fix(route-view): "N hops" counts only true hops, not payload endpoints (#2240)
Fixes #2206 ## What Follow-up to #2207, as sketched in its review: the route meta (desktop header + mobile summary) printed `positions.length` as "N hops". For a DM, `positions` includes the payload-decoded sender and recipient rows — endpoints, not hops — so the count was inflated. Live on corescope.ukmesh.com right now, the reported 6-relay chain reads **"8 hops · 913 km"** with "from payload" chips on rows 1 and 8. ## Change - `public/route-view.js` — new `countHops(positions)` helper: counts rows **without** `_fromPayload` (both builders set the flag since #2207). The desktop header and the mobile summary both render it, so they can't disagree. Only the displayed figure changes; the row-count `total` still drives distances, stripe colours and row slicing. - `tests/unit/test-issue-2206-hop-count.js` — regression test (listed in `test-all.sh`): extracts `countHops()` from the shipping file and exercises it (6-relay DM → 6; payload-only → 0; plain chains and non-payload origin/dest unchanged), and pins both meta strings to the helper so a regression back to `total` fails. The count intentionally counts relay rows (per the review note), not edges: a 6-relay DM reads "6 hops". Frontend-only, static assets, no build step, no new dependencies. Cache buster untouched (AGENTS.md rule 3). ## Perf One `.filter()` over a ≤ ~10-entry array, once per sidebar build; no hot-path impact. ## Testing - `node tests/unit/test-issue-2206-hop-count.js` — 11 passed, 0 failed - `sh test-all.sh` — full unit suite green, exit 0 - Live cross-check against the deployed instances (pre-fix logic): `https://corescope.ukmesh.com/#/map?packet=44259e35b7c826a8&obs=97234` shows 8 rows, meta "8 hops · 913 km", payload chips on rows 1 and 8 → this change makes that same chain read "6 hops · 913 km". The new count ships with the next build that carries it; happy to re-check on a deployment once one does. |
||
|
|
127dd511a7 |
fix(health): judge favorites and home status cards by the node's role (#2215)
## The problem A starred repeater can show two different health statuses. The My Repeaters tab (#1761) and `getNodeStatus` judge it by the infra thresholds (degraded after 24 h, silent after 72 h). The favorites dropdown and the home page cards judge every node by the node thresholds (degraded after 1 h, silent after 24 h), because they never look at the node's role. | repeater last heard | favorites / home cards (before) | My Repeaters tab | after this PR | |---|---|---|---| | 5 h ago | degraded | healthy | healthy | | 30 h ago | silent | degraded | degraded | | 80 h ago | silent | silent | silent | If you set `healthThresholds` in `config.json`, every view follows your values. ## The change Three call sites now use `getHealthThresholds(role)`, the helper the My Repeaters tab already uses: - `renderFavDropdown` in `app.js` - the "My Nodes" grid card in `home.js` - the hero search-result card in `home.js` The search card's fallback text, "Not heard in 24+ hours", was only true for non-infra nodes, so it now reads "Not heard recently". ## Scope - I kept the existing three-state healthy / degraded / silent logic and changed only which thresholds apply. - Not included: counting a repeater that is relaying traffic as alive (#1598, which `getNodeStatus` does). That is a bigger behavior change and would be its own PR. ## Risk Low. It is 9 lines in two files, and the helper and thresholds already exist and are used elsewhere. ## Tests New `tests/unit/test-health-card-role-thresholds.js`, added to `test-all.sh`. It checks that infra roles get looser thresholds than nodes, and that each of the three call sites goes through `getHealthThresholds(role)` with no node-only comparison left. On current `master` 5 of its 7 assertions fail; with this change all pass. `sh test-all.sh` passes. It prints a `STALE ALLOWLIST` warning from an a11y self-test, which it also does on `master`. ## Update after review (commit `0f2376df`) The text above is the description efiten reviewed on 2026-10-15. This section describes the code as it is now. ### The change - New `getHealthStatus(role, heard, now)` in `roles.js`. It returns `healthy`, `degraded` or `silent` by the role's thresholds, or `null` when there is no usable timestamp. It only applies the cut-offs; each caller picks the timestamp. - All six places that compared an age against `degradedMs` / `silentMs` now call it: - the favorites dropdown (`app.js`); - the "My Nodes" card and the home health card (`home.js`); - the analytics network status and My Repeaters (`analytics.js`); - the live node panel (`live.js`). Each keeps its own labels and timestamp, so the last three show the same as before. `analytics.js` and `live.js` no longer guard with `typeof getHealthThresholds`. `roles.js` loads before both, and both already used it without a guard elsewhere: the old network status called `getHealthThresholds` directly, and `live.js` reads `ROLE_COLORS`. - The favorites dropdown and both home cards now fall back to the node's `last_seen` when the packet store has no packet for it. My Repeaters (`last_heard || last_seen`) and the node page (`nodes.js:729`) already do this. Without it, with a packet store window under 72 h, a repeater could be silent on the cards and degraded on My Repeaters. The "ago" text uses the same timestamp. With the default thresholds, the fallback changes a status only when the packet store holds less than 72 h of packets. `retentionHours` unset (unlimited) or at the example's 168 h holds more than that. Otherwise a node with no packet in the window is silent either way, and the views now show how long ago it was last seen instead of "never" or nothing. - With no timestamp at all, the home health card said "Not heard in 24+ hours". It now says "No packets in the current window". ### Why a new helper and not `getNodeStatus` `getNodeStatus` has two states (active / stale) and counts a repeater that relays traffic as alive (#1598). Using it here would change more than the thresholds. `getHealthStatus` keeps these views' three states. ### Scope - Not included: counting a repeater that relays traffic as alive (#1598) in these views. That is a bigger behavior change and would be its own PR. - The live node panel still judges by `last_seen` alone, and the node page stays relay-aware. Both now use the shared thresholds; aligning their timestamps would also be its own change. ### Tests - `tests/unit/test-health-status.js` runs `roles.js` in `vm`. It covers: - repeater, room, companion, sensor, an unknown role and a missing role, at ages either side of 1 h, 24 h and 72 h; - ISO and epoch input; - null, empty and invalid timestamps; - a `healthThresholds` override through `/api/config/client`. One source guard remains: no age-vs-threshold comparison outside `roles.js`. - `tests/unit/test-my-repeaters-dashboard.js` now uses the real helper instead of a stub, which gave repeaters the node thresholds. It also checks that a repeater heard 30 h ago is Degraded. - New `tests/e2e/test-health-status-e2e.js` mocks `/api/nodes/<pk>/health` for six nodes. It checks five views: the favorites dropdown, the "My Nodes" card, the home health card, the analytics network status and the live node panel. It fails on `master` and on this PR's first commit. I put it on shard 2, which was the shortest shard in the last CI run. - Mutation check: I made 33 deliberate breaks. In the helper they cover each comparison and boundary. At each of the six call sites they cover the timestamp field, the role argument and at least one label mapping. Each one makes the suite fail. - I checked a `config.json` with `infraDegradedHours: 2` and `infraSilentHours: 6` in Chromium. Repeaters heard 3 h and 7 h ago show Degraded and Silent. - `sh test-all.sh` passes. It prints a `STALE ALLOWLIST` warning, which comes from a deliberate fixture in the a11y self-test. ### Cost The analytics network status loops over every node. In headless Chromium, 30,000 synthetic nodes take 9.0 to 9.2 ms before this change and 9.1 ms after (median of 21 runs, on one machine). ### Risk Low. The change is frontend only, with no API, config or data change. Rolling back means reverting the commits. --- 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: nullrouten <201262769+nullrouten0@users.noreply.github.com> Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com> |
||
|
hermes-gadgetandhermes-gadget
|
550829af6e |
fix(route-view): badge payload-decoded sender/recipient rows in the sequence renderer (#2207)
Refs #2206 ## What DM packets on the route view showed **sender** and **recipient** rows — decoded from the packet payload (`destHash`/`srcHash`, one byte each) rather than from the outer path bytes — as ordinary hops, without the "from payload" badge, whenever the chain renders through the multi-path/sequence builder. Only the single-path builder set `_fromPayload`, and the isolate/sequence list renderer never drew the chip. Effects: payload endpoints are indistinguishable from observed hops, and the displayed hop count is inflated (a 6-relay DM renders 8 rows, with the last "hop" being a single-byte recipient prefix resolved to whichever node shares it). ## Change - `public/map.js` — set `_fromPayload: true` on the sequence builder's origin (sender) and destination (recipient) entries (3 + 2 sites). - `public/route-view.js` — render the `mc-rt-status-chip mc-rt-status-payload` "from payload" chip in the isolate/sequence list, matching the canonical renderer. The chip markup now lives in one `payloadChip(p)` helper called by both row builders (it was duplicated between them, so they could drift apart). - `tests/unit/test-issue-2206-payload-badge.js` — regression test (listed in `test-all.sh`): extracts `payloadChip()` and `rowFor()` from the shipping `route-view.js` and exercises them in a vm sandbox, and pins every sender/recipient entry in `drawPacketRouteMulti` to `_fromPayload: true`. Frontend-only, static assets, no build step, no new dependencies. Per AGENTS.md rule 3 the cache buster in `public/index.html` is intentionally untouched — `__BUST__` is replaced automatically at server startup. ## Perf Two ternaries + one string concat per rendered row in a display-only list; no impact on hot paths. ## Testing - `node tests/unit/test-issue-2206-payload-badge.js` — 22 passed, 0 failed (new regression test) - `sh test-all.sh` — full unit suite green, exit 0 (201 suites) - Browser validation, re-run against the live deployments (which carry the equivalent patch): on https://corescope.nemesh.uk/#/map?packet=44259e35b7c826a8&obs=97234 the canonical chain marks rows 1 and 5 "from payload"; isolating the 6-hop path (`B4CE24→…→8C197F`) renders the 8-row chain with rows 1 and 8 marked. The refactor keeps the renderer output identical (pinned by the new test). The identical patched assets have been running on two independent deployments since 2026-10-13. ## Notes The underlying ambiguity (a 1-byte prefix matched against many nodes, resolved to the most-recently-seen candidate) is what the badge surfaces to the user; changing the resolution strategy itself is out of scope for this PR. The `N hops` header still counts every row — counting only non-payload rows would change every DM header, so it stays a follow-up; hence `Refs #2206`. --------- Co-authored-by: hermes-gadget <hermes-gadget@users.noreply.github.com> |
||
|
efitenandClaude Opus 5.5
|
927a8eec8a |
fix(nodes): node analytics rates cover the data in memory, and say so (#748) (#2211)
Refs #748 (M3a: correct numbers within the in-memory window; the SQL history in M3b is a separate plan, see the end). ## Problem The node analytics range picker offers 24h, 7d, 30d and All (365d), but the store only holds its in-memory window (`retentionHours`, then eviction). `GetNodeAnalytics` still divided availability and packets/day by the requested range. Measured on our live instance (14-day window), repeater BE-BRE-ON8AR: | Range | Availability | Packets/day | |---|---|---| | 7d | 88.7% | 91 | | 30d | 42.6% | 265 | | All | 3.5% | 22 | Nothing on the page said that the data stopped at the edge of the window. ## What changes - `coverageStart` returns the edge of the in-memory window: the latest of `oldestLoaded` (background-load progress), `now - retentionHours`, and a new `evictedThrough` that eviction sets to the newest `FirstSeen` it removed, so memory-pressure eviction inside the retention window moves the edge too. - `s.packets[0]` is not that edge, and the first version of this PR used it. Chunks are selected by `last_seen`, so a transmission first heard weeks ago and heard again recently sits at the head with its old `FirstSeen`. On staging (7-day window) it returned 21/07. - When the edge lies more than a minute after the requested start, node analytics counts only transmissions from the edge, computes availability and packets/day over that span, and returns `timeRange.dataFrom` and `timeRange.partial`. - The minute of slack keeps a range equal to the retention window from reading as partial. Without it every 7d request on staging (168h) was partial, because the two edges came from separate clock reads. - The analytics page shows "Data from 2026-10-07: the server keeps about 7 days of packets in memory. Availability and packets/day cover that period, not the full 365 days." when partial. - **UTC fix.** The window bounds of node analytics and node hop analytics are formatted in UTC. They are compared as strings with UTC `FirstSeen` values, so on a non-UTC host the 7d window lost its oldest hours (2 at +02:00). Docker images run in UTC, which is why this did not show in production. - `nodeTxsSince` (one caller) is inclusive, so a packet exactly on the edge is kept. ## Tests - Go, `node_analytics_coverage_test.go`: - full coverage within memory; - 30d over 3 days in memory; - empty store; - a re-heard 85-day-old transmission at the head of `s.packets` (edge stays at retention, packet not counted, no bucket before `dataFrom`); - memory-eviction edge; - eviction records `evictedThrough`; - retention 30 s shorter than the range is not partial, while 30d is; - window bounds under a +02:00 `time.Local`. - Two tests were checked to fail on the old code: the local-zone test (166 of 168 packets, a `+02:00` bound) and the retention test without the slack (`dataFrom 09:57:08` vs `from 09:56:38`). - Full `cmd/server` suite passes locally (skipping `TestSaveGeoFilterPreservesFileMode`, a Windows-only failure on clean master). One run of five reported `FAIL` and four reruns were green; I did not capture which test failed. - Frontend unit: `tests/unit/test-issue-748-node-analytics-coverage.js` (5 assertions on the real `node-analytics.js` in `vm`), in `test-all.sh`; the full suite passes. - E2E: `tests/e2e/test-issue-748-node-analytics-coverage-e2e.js` (registered in `deploy.yml` and `scripts/non-unit-tests.json`), against the real fixture response with no interception: 7d is not partial and shows no notice; All is partial and the notice names the real start date. 2/2 locally. ## Validation Staging (retentionHours 168), same repeater, after the background load completed: | Range | Before | After | |---|---|---| | 7d | 82.7% | 83.3%, not partial | | 30d | 19.3% | 83.3%, partial from 2026-10-07 | | All | 1.6% | 83.3%, partial from 2026-10-07 | In the browser the notice appears under All and not under 7d. ## Not done - M3b, older history from SQLite. In the database only the sender is indexed (`from_pubkey`). Recipients sit in `decoded_json`, and relay hops only in `observations.resolved_path` (filled for about 27% of observations on our live instance, not indexed). A fallback today would show only a node's own packets for older days, a different meaning than the in-memory part. Doing it properly needs a per-node index written by the ingestor (pubkey, transmission, role); that is a separate plan. - The other analytics endpoints with a range (observer analytics, RF) were not checked for the same dilution. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
efitenandClaude Opus 5.5
|
cab4f71c64 |
feat(nodes): LoRa radio settings per node from the MeshCore map (#1268) (#2209)
Fixes #1268. Frequency, spreading factor, bandwidth and coding rate are per-device settings that never travel in a packet. The public MeshCore map is the only source. This PR adds them to the node detail views, opt-in, and answers the open policy questions from the triage comments. ## Policy decisions | Question (triage) | Answer in this PR | |---|---| | Opt-out for self-hosted instances | Opt-**in**: `rfParams.nodesUrl` empty (default) = nothing fetched, `/api/rf-params` returns `enabled: false`. Same model as `knownChannelsUrl`. | | 33 MB every 6 h, thundering herd | Random 0-60 s delay before the first fetch, then every `refreshHours` (default 6). The map sends gzip (17 MB on the wire). It sends no ETag or Last-Modified, so conditional requests are not possible. | | Size cap | 128 MB on the decoded body (it is 51.5 MB today). Exceeding it fails the fetch instead of parsing a truncated body. | | Field shapes at the parse boundary | 64-hex key; 100-1100 MHz; SF 5-12; bandwidth from the LoRa set (7.8 to 500 kHz); CR 5-8. Invalid rows are dropped. A body that is not a well-formed array fails the fetch. | | Keep stale cache on anomaly | Any failure keeps the previous snapshot; a failed presets fetch keeps the previous presets. `fetched_at` is returned so an outage is visible rather than hidden. | | Freshness | Only adverts within `maxAgeDays` (default 7) are kept at fetch time and re-checked at serve time. | | TLS pinning | Not done: Go's certificate validation plus the field validation above. | ## What changes - `cmd/server/rf_params_cache.go`: stream decoder, validation, cap reader, preset parser (presets with identical values are merged, titles joined), background refresh. In memory only, no DB writes, so the read-only server invariant holds. - `GET /api/rf-params`: `{enabled, fetched_at, source, presets, nodes}`; `nodes` holds only nodes this instance knows, keyed by lower-case pubkey. `/api/nodes` is unchanged, so its payload does not grow for every consumer and the 500-row paging is unaffected. - Node detail (side panel and full page): `LoRa` row with the four values, the preset name when all four match one, and the age of the map advert. Shown only when the feature is on and the server has fetched the map at least once. - No column in the Nodes table, on purpose: on our network such a column showed one preset title in 814 of 1,366 rows and nothing in 443. Showing only the nodes that differ from their region is the follow-up PR. - `config.example.json` documents the block with the two URLs to use. ## Cost - Measured on the real map payload (51.5 MB, 66,336 nodes): parse 0.52 s, 15 MB allocated in total, 26,815 nodes kept (advert within 7 days). Runs in a background goroutine every 6 h. - The endpoint walks the cached node list once with map lookups: O(nodes). - Frontend: one bulk fetch of `/api/rf-params` when a node detail opens (client TTL 5 min), not one per node. ## Tests - Go (`rf_params_cache_test.go`): parse with every rejection reason, malformed bodies (not an array, truncated, wrong type, empty), preset merge and invalid presets, cap reader (one byte over refused, exactly at the cap accepted), fail-soft against an `httptest` upstream (503, HTML body, presets-only failure), config defaults, route (disabled shape, only known nodes, entry aged past max age dropped at serve time). Full `cmd/server` suite passes locally (skipping `TestSaveGeoFilterPreservesFileMode`, a Windows-only failure on clean master). - Frontend unit: `tests/unit/test-issue-1268-rf-params.js` (7 assertions on the real file in `vm`), in `test-all.sh`; the full suite passes. - E2E: `tests/e2e/test-issue-1268-rf-params-e2e.js` (registered in `deploy.yml` and `scripts/non-unit-tests.json`). It checks the real disabled state on the fixture server (no detail row), then serves `/api/rf-params` through `page.route`: no table column, full detail page with preset, side panel with raw values, and the hint for a node that is not on the map. 4/4 locally, plus the existing node table and node detail E2E files. ## Validation On a staging instance with `rfParams` set to the map and preset URLs: - Log: `[rf-params] 26821 nodes within max age, 20 presets`, about 6 s after start. - `/api/rf-params` returned 1,115 of our nodes: 988 on 869.618/SF8/BW62.5/CR8, 111 on the same with CR5, 15 on 433.65 MHz, 1 on SF7. - In the browser the detail row shows in the side panel and on the full page, and the Nodes table keeps its original columns. ## Not done - Highlighting nodes whose settings differ from their region's majority (a badge in the Nodes list plus a filter) is the follow-up (separate PR). In that follow-up a CR-only difference stays informational until it is confirmed whether MeshCore radios use the explicit LoRa header (then CR travels in the header and a mismatch still decodes). - `maxAgeDays` is config-only; the customizer is a later step. - No data is written to the DB, so the RF parameters are gone after a restart until the next fetch (at most 60 s plus the fetch time). --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
efitenandClaude Opus 5.5
|
1ae4c74913 |
feat(observers): rolling 1 h TX airtime and a duty-cycle warning (#661) (#2208)
Fixes #661 (the narrowed scope from the 2026-10-14 comment). ## What changes - `GET /api/observers/metrics/summary` adds `tx_airtime_pct_1h` and `rx_airtime_pct_1h` per observer: the airtime share over the hour before now, whatever `window` is, from the cumulative `tx_air_secs` / `rx_air_secs` counters in `observer_metrics`. - A counter that goes down marks a reboot; only the stretch after the last reset counts. - Under 15 minutes of samples (one sample, or a reboot late in the hour) returns `null`. - Clamped to 0..100, rounded to 2 decimals. - The response also carries `duty_cycle_warn_pct`, from the new config key `dutyCycleWarnPct` (default 10, the EU 868 MHz g3 sub-band limit). Added to `config.example.json`. - RF Health tiles show `TX 1h: x%`; the observer list gets a sortable numeric `TX 1h` column. Both use the shared `public/observer-airtime.js` helper and flag values at or over the threshold with `.airtime-over` (`var(--status-red-text)`). - The handler now returns a named `MetricsSummaryResponse` struct instead of `map[string]interface{}` (one occurrence fewer). ## Why the warning matters Repeater and room-server firmware default to `airtime_factor = 1.0` (`examples/simple_repeater/MyMesh.cpp:890`), which `Dispatcher.cpp:26` turns into a 50% TX budget per 1 h window. The device therefore does not keep itself under the EU 10% limit; this view is where an operator can see it. ## Cost - Server: one extra query per summary request, `SELECT observer_id, timestamp, tx_air_secs, rx_air_secs FROM observer_metrics WHERE timestamp >= now-1h`, on the existing `idx_observer_metrics_timestamp`, then one O(rows) pass. On our instance that is 35 observers x about 12 samples (5 min interval). - Observer list: one extra bulk call `summary?window=1h` per load (client TTL 30 s, failure ignored), not a call per observer. ## Tests - Go: `TestRollingAirtimePct` (steady, reboot mid-hour, single sample, span under 15 min, reboot leaving under 15 min, nil counters, clamp, rounding), `TestGetMetricsSummaryAirtime1h` (samples older than 1 h ignored inside a 24 h window, single-sample observer gets null), `TestMetricsSummaryDutyCycleWarnPct` (default 10, config override, `observers` is `[]` not `null`). Full `cmd/server` suite passes locally (skipping `TestSaveGeoFilterPreservesFileMode`, which fails on Windows on clean master too). - Frontend unit: `tests/unit/test-issue-661-observer-airtime.js` (8 assertions, real file in `vm`), added to `test-all.sh`. `test-observers-headings.js` updated for the new column. Full `test-all.sh` passes locally. - E2E: `tests/e2e/test-issue-661-airtime-e2e.js`, registered in `deploy.yml` and `scripts/non-unit-tests.json`. The fixture DB has no `observer_metrics` table, so the summary is served through `page.route` with known values for the fixture's own observer ids. It checks the column header, over/under/unknown cells, sort order and the RF Health tile. 4/4 locally. It caught a real bug during development: `loadRFHealthData` declares a local `const window`, so the tile calls the helper as the global `ObserverAirtime`. ## Validation Deployed to a staging instance with live MQTT data (35 observers with metrics) and checked in the browser: - The observer list shows the column for 42 rows (33 with a value, 9 without enough samples). Sorting puts the highest share (5.3%) first. - RF Health shows `TX 1h` on all 35 tiles. With a fourth stat the line wrapped inside items on narrow tiles; the stats row now wraps between items only. - No observer was over 10% at the time, so the red state was checked by rendering one cell client-side at 12.5% (computed colour `rgb(185, 28, 28)` in light mode), and by the E2E test. ## Not done - The threshold is config-only; exposing it in the customizer is a follow-up. - No airtime comparison across observers (that is #671) and no per-packet airtime (Relay Airtime Share covers it). - Dark mode was not checked in the browser; `--status-red-text` is defined for both themes. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
efitenandClaude Opus 5.5
|
fbeb9e4d71 |
ci: frontend coverage from every E2E suite, merged across the shards (#2204)
Fixes #2203. ## Problem The frontend coverage badge (35.16% on run 38413268001) came from two sources on shard 2: one dump part-way through `test-e2e-playwright.js` and the `collect-frontend-coverage.js` tour. The other E2E suites run against the instrumented build but never save `window.__coverage__`, including the suites written for single pages (`path-inspector.js` 4.72%, `touch-gestures.js` 14% on that run). ## Change 1. `9d3bfad5` `test(e2e): red:` `tests/unit/test-e2e-coverage.js`, with fake pages, contexts and browsers (no Chromium needed). On the placeholder module the first assertion fails. 2. `c00bbb12` `feat(e2e):` the collector. - `scripts/e2e-coverage-hook.js` is a preload (`node --require`) that wraps `chromium.launch` from `playwright`. `playwright` re-exports `playwright-core`, so a suite's own `const { chromium } = require('playwright')` gets the same, wrapped object. - `scripts/e2e-coverage.js` reads `window.__coverage__` before every call that replaces or ends a page's document: `goto`, `reload`, `setContent`, `goBack`, `goForward`, `close`, and context and browser `close`. Pages from `newPage`, `newContext` and the context `page` event are all tracked. - Each read zeroes the page's counters, so a document read twice is not counted twice. Maps travel once per file; later reads carry counts only. - The page returns a JSON string. Returning the object cost about 10 s per suite, because Playwright serialises it node by node. - A read is given up after 5 s, so a busy page cannot hold up a suite's cleanup. - On exit the hook writes one file per suite to `.nyc_output/`, or nothing without coverage. A failed write is logged and never changes the exit code. 3. `64487c4c` `ci:` the workflow. - On master pushes the E2E run step sets `NODE_OPTIONS=--require ./scripts/e2e-coverage-hook.js`. On PRs it is empty, so PR runs are unchanged. - Each shard runs `nyc merge` (one file per suite is about 6 MB, mostly the same maps) and uploads `coverage-shard.json` with its results instead of `.badges/`. - The `e2e-test` job merges the three shards and makes the frontend badge (it used to come from shard 2 alone). - The merge, setup-node and badge steps are `continue-on-error`. When `nyc report` gives no statement percentage the step warns and writes no badge, so a registry hiccup cannot publish 0%. - `test-e2e-playwright.js` no longer writes its own `e2e-coverage.json`: the hook covers that suite completely, including the tests after the old dump. `collect-frontend-coverage.js` stays as an extra source. ## Performance Measured locally against an instrumented server (Windows, Chrome): | Suite | Without hook | With hook | |---|---|---| | `test-path-inspector-coverage-e2e.js` | 1.9 s | 2.2 s | | `test-touch-gestures-coverage-e2e.js` | 17 s | 16 s | | `test-e2e-playwright.js` (stops at the same local-fixture failure in both runs) | 52 s | 52 s | | `browser.close()` on a page in a 30 s busy loop | 153 ms | 5.2 s | The hook only loads on master pushes. ## Testing - `tests/unit/test-e2e-coverage.js` covers: - reads before each document-replacing call and before close; - pages opened by the page itself; - context and browser close; - summing without double counts; - a failed read, and a read that never answers; - the file writer, including a failing write. - Nine deliberate breaks of the library each turn it red: - dropping the counter reset; - dropping the `page` event, the `goto` read, the `goForward` read, the `close` read or the browser `close` read; - removing the timeout or the write guard, or writing an empty store. - The test also fails if it stops before its end. Without that check, a read that never returns would let node exit 0 half-way. - Real suites with the hook, measured on their own (the badge run had all suites): | File | Badge run 38413268001 | Own suite with the hook | |---|---|---| | `path-inspector.js` | 4.72% | 52.75% | | `touch-gestures.js` | 14% | 72.96% | - Exit codes and pass counts were unchanged with the hook for these suites (the first three were run before the 5 s read timeout was added): - `test-slideover-1056-e2e.js` (6 contexts); - `test-a11y-axe-1668.js` (`@axe-core/playwright`); - `test-issue-1150-404-state-e2e.js`; - the three above. - The badge step, run locally against two merged shard files: - It gives the same per-file numbers as `nyc report` on the inputs. - With no shard coverage, or with `nyc report` failing, it warns and writes no badge. - `sh test-all.sh` passes locally, apart from `test-issue-1956-release-routing.js`, which needs the workflow's `GITHUB_REF_NAME` and was skipped. The test inventory classifies 321 suites. `actionlint` 1.7.7 reports only the self-hosted runner label, as on master. **Not tested:** the shard and badge steps in CI. They only run on a master push, so the first master run after merge is the real test. I do not know what the merged number will be. I will compare the badge with the merged report in that run's `e2e-test` log. **Not counted:** a full page load the suite does not start through one of the wrapped calls, such as a link that leaves the SPA or `location.reload()` from page code. ## Not in this PR - The `tests/unit/` suites, which load frontend files through `vm.createContext` without instrumentation (noted in #2203). - Coverage on PR runs. The overhead above is small, so this could follow if wanted. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
45680a22cf |
fix(#2191): apply the channel filter to live WebSocket packets (#2192)
Fixes #2191. ## Problem On the Packets page the channel filter only applies to the initial load (`/api/packets?channel=` → server-side `transmissions.channel_hash` match). The WebSocket handler re-checks time window, type, observer, region and node, but not `filters.channel`, and the channel filter is not re-applied at render time. With a channel selected, every new live packet is appended regardless of its channel. ## Fix - New pure helper `packetMatchesChannel(p, channel)` in `public/packets.js`, exported as `window._packetsMatchesChannelForTest` (same pattern as `_packetsBuildChannelOptionsForTest`). - It derives the value the server filters on from `decoded_json`, because WS packets don't carry `channel_hash`: - `channel` when the server decrypted the `GRP_TXT`, - otherwise `'enc_' + channelHashHex.toUpperCase()`. - Only `payload_type === 5` can match, mirroring the server (the column is only set for `GRP_TXT`). If a packet ever does carry `channel_hash`, that wins. - The WS filter calls it: `if (filters.channel && !packetMatchesChannel(p, filters.channel)) return false;` Checked the derivation against a production DB (live.saarmesh.de): it equals `channel_hash` for all 3,193 `GRP_TXT` rows of the last two days, 0 mismatches. ## Performance WS hot path, but the new check runs only while a channel filter is active, and for non-`GRP_TXT` packets it returns before any parsing. For `GRP_TXT` it is one `JSON.parse` of the already-received `decoded_json` string, O(1) per packet. WS messages are already debounced/coalesced by `debouncedOnWS`, so at typical rates (a few packets per second) this is negligible. No new state, no allocations beyond the parse, no API calls. ## Tests New `tests/unit/test-packets-live-channel-filter.js` (registered in `test-all.sh`), 10 cases: - no filter accepts everything - decrypted name match / mismatch - non-`GRP_TXT` rejected while a channel filter is active - `enc_<HH>` match incl. hex case normalisation and `enc_00` - `channel_hash` passthrough takes precedence - already-parsed `decoded_json` object - malformed / missing `decoded_json` doesn't throw and doesn't match - source check that the WS filter block actually applies the helper Without the `packets.js` change the new test fails; with it all pass. Existing `test-packets.js` (117), `test-packets-local-channels.js` (14), `test-packet-filter.js` (99) and `test-frontend-helpers.js` still pass. ## Browser validation The same change is deployed on live.saarmesh.de (hot-patched `packets.js`, server restarted so `__BUST__` refreshed; the served file contains the fix). I have not taken a screenshot of the filtered live table myself, so treat the in-browser check as pending until a reviewer or operator confirms. No config or customizer implications. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01HJcbiRjQC5cSmydVFXbPbR Co-authored-by: SaarMesh-Bot <300107934+SaarMesh-Bot@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com> |
||
|
efitenandClaude Opus 5.5
|
60af9bd469 |
fix(hops): narrow only the directly heard hop to the observer region (#2098) (#2188)
Fixes #2098. ## Problem `HopResolver.resolve()` narrowed every hop of a path to the observer's IATA region (300 km around the code's coordinates). Only the last hop was heard by the observer directly; every earlier hop can be anywhere the mesh reaches, so the region says nothing about it. This stayed invisible on deployments whose observer codes were missing from `/api/iata-coords`: there `nodeInRegion()` returns `null` and nothing is filtered. A Belgian deployment's 42 observers use 10 codes and none was in the table. Filling the table alone would have switched on a filter that is wrong for every hop but the last. ## Why only the last hop #2098 lists three options and argues against simply repopulating the table, because the filter it would switch on is not sound. This PR does fill the table, but only after limiting the filter to the one hop where it is sound: - In a MeshCore path the last hop is the repeater the observer received the packet from, so it is within radio range of the observer. Measured below: p99 30 km for directly heard multi-byte hops. - An earlier hop says nothing about the observer's position. On the Belgian deployment nearly every observer hears nearly every flood, so a region filter on earlier hops would drop the right candidate whenever the packet started outside the observer's region. This was decided while preparing the PR, not on the issue thread, so the reasoning is recorded here. ## Change - **`public/hop-resolver.js`:** `resolve()` takes an optional `opts.directHops`, the hops the observer heard directly. Without it, the input is one path and its last hop is the direct one. The region check runs only on direct hops. An earlier hop gets `regional: false` and, with an observer, `regionSkipped: true`. `globalFallback` now means checked against the region with no candidate in it, so it is only set on a direct hop. - **`public/hop-display.js`:** the badge counts every candidate for `globalFallback` or `regionSkipped`. The dashed underline and the popover's "(global fallback)" stay for `globalFallback` only, so an earlier hop is not labelled as matched outside the region when no region check ran on it. - **`public/packets.js`:** `resolveHopsForPackets()` resolves the last hop of each path apart from the earlier hops. `hopCacheKey(h, observerId, direct)` keeps the two in separate cache entries, because the same prefix can be the last hop in one packet and a relayed hop in another. `renderPath()` reads the direct entry for the path's last hop, and the detail-pane resolve writes it. The region-narrowed entry never goes to the bare (observer-less) key, in the list or in the detail pane. The detail pane resolves and caches the shown observation's path with that observation's observer (`effectivePkt.observer_id`), the observer `renderPath` already used; before, it resolved with the packet row's observer, so when the first observation came from another observer the last hop missed its direct entry and showed unfiltered. The route button uses the same observer. - **`cmd/server/routes.go`:** `iataCoords` gains 104 entries from OurAirports `airports.csv` (public domain): every IATA code in Belgium, the Netherlands and Luxembourg, and the airports with scheduled service in Germany and France. AMS, CDG, FRA and MUC were already present and keep their values. ## Measured before building On 5000 packets from the Belgian deployment (3661 with a path): - 1-byte last hops: 1212, all ambiguous, 1204 of them without a server `resolved_path`, so they go through the client resolver. - Distance from the observer to a directly heard node, on 1892 unique multi-byte last hops: p95 25 km, p99 30 km, max 82 km. - With this change, on 906 1-byte last hops from observers whose code is now in the table: the badge counts 6.1 candidates on average instead of 6.6. The displayed name changes for 52, all from observers without their own lat/lon, where distance to the region now orders the candidates instead of list order. For observers with a position, no displayed name changes, because the #2097 observer anchor already picked the nearest candidate. ## Tests - `tests/unit/test-issue-2098-last-hop-region.js` (new, in `test-all.sh`, 25 assertions): - HopResolver: the last hop of a path is narrowed and an earlier hop is not; an earlier hop keeps every candidate and reports `regionSkipped`, not `globalFallback`; a direct hop with no candidate in region reports `globalFallback`; `directHops` overrides the position; a one-hop path is narrowed. - HopDisplay (real file): an earlier hop keeps a badge with every candidate but no fallback class or popover text; a direct hop outside the region keeps both. - packets.js (real file, real resolver and display): with one observer, a prefix that is the last hop of one packet and an earlier hop of another renders narrowed in the first and with a badge in the second; after the detail pane resolves a path, the observer-less render does not show the narrowed last hop; when the shown observation comes from another observer than the packet row, its last hop is narrowed to the region of the observer that heard it. - `TestIataCoordsIncludesBelgianObserverCodes`: ANR, BRU, GNE, KJK, LGG, OBL, OST and MST with their OurAirports coordinates. - Red commits first (`26000a22`, `bb91ff9d`, `189f55d7`, failing on assertions), then green. Mutation check on the packets.js tests: inverting `isDirect` fails 3 assertions, swapping the `directHops` argument fails 4. - `test-issue-2097-hop-ambiguity-badge.js`: two structural regexes now allow the new `direct` argument; every assertion still holds. - All 198 `node tests/unit/*` suites in `test-all.sh` pass. `cd cmd/server && go test ./...` passes (with `TestSaveGeoFilterPreservesFileMode` skipped, a Windows-only failure on master too); `go vet` clean. ## Browser validation Staging (this branch on top of the deployment's current build), headless Chrome on `#/packets/<hash>` for packets whose path ends in a 1-byte hop: the last hop's popover lists only candidates within the region with their distances (for example `7E`: 3 candidates at 46, 207 and 296 km), earlier hops list every candidate (`1D`: 13). The same prefix `7E` showed 3 in region as the last hop of one path and 4 as a relayed hop of another. No page errors. `/api/iata-coords` returns 158 codes. ## Browser validation of the follow-up Staging again (this branch at `4e129b49` on top of the deployment's build), headless Chrome: - Packets list, 41 paths of 2+ hops: the global-fallback class appears on 2 last hops and on 0 earlier hops. - Detail pane, 5 packets whose path ends in a 1-byte hop (1 to 13 hops shown): no earlier hop carries the class or "(global fallback)" in its popover, and every ambiguous earlier hop keeps its badge. Every last hop was checked against the region: 4 show "N candidates in region", 1 shows "(global fallback)" with the class. - Before the observer fix, 2 of 5 detail panes showed a last hop with neither label, because the shown observation came from another observer than the packet row (`b66a8f4c`: row from one observer with 13 hops, shown observation from another with 9). - No page errors. ## Not done - The 300 km radius is unchanged. The measured direct-hearing distances suggest it is wide, but a smaller radius would also have to hold for less dense networks, and I have no data for those. - `live.js` passes no observer, so the region filter does not run there, before or after this change. `map.js` passes the observer of the chosen path in path order, so once the resolver knows the observer regions it narrows that path's last hop only. - The "Cross-observer aggregate" line in the detail pane renders the longest path with the packet's own observer, so its last hop is narrowed to that observer's region even if another observer reported that path. The per-observer keying there predates this PR (#2097). - Two of the Belgian deployment's codes, HEP and NRW, are not airport codes and are not in OurAirports; those observers stay without a region filter. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Alex BandClaude Opus 5.5
|
955386b63f |
test(map): settle the router's focus frame before the teardown snapshot (#2179)
## Summary Fixes an intermittent failure in `tests/e2e/test-map-nodes-pagination-e2e.js`, the "late response after map teardown" cases added for #2030 in #2048: ``` ✗ late /observers response after map teardown: old response must not change the destination page ``` It failed on #2164 ([run 38080812509](https://github.com/Kpa-clawbot/CoreScope/actions/runs/38080812509), shard 2/3), a Go-only change, and passed on the rerun. ## Cause `checkMapTeardown` compares `#app`'s HTML before and after releasing the old map's pending response. When it failed, the only difference was on the destination page's heading: ``` before: <div class="tools-landing"><h2>Tools</h2>… after: <div class="tools-landing"><h2 tabindex="-1">Tools</h2>… ``` That attribute comes from the router, not the old response. Since #630 (`app.js`, "SPA focus management") it focuses the new page's first heading in a `requestAnimationFrame` after the page renders. The before-snapshot was taken as soon as `#leaflet-map` detached, sometimes ahead of that frame. ## Fix The test now waits two frames before the before-snapshot, the same settle it already does before the after-snapshot. The router's frame is queued by then, so it always lands first. The change is test-only: no app code changes, and every assertion stays as it was. ## Tests Run locally with the e2e fixture (`tools/freshen-fixture.sh`, migrated) served by this branch's server, headless Chromium: | | runs | failed | |---|---|---| | test on `master` (`352a0ccd`) | 20 | 2 | | this branch | 50 | 0 | - **Before:** the two failures on `master` were "old response must not change the destination page" (teardown, `/config/regions` and `nodes`), plus one 30 s timeout in an "after map replacement" case in the same runs. - **After:** none of these failures or the timeout came back in 50 runs. ## Checks - **Lint:** ESLint passes with the repo config and with Node globals. SonarQube rules on the file report nothing on the added lines; its four existing minor findings are on lines this PR doesn't touch. - **Repo rules:** one logical change, explicit `git add`, no private data, no new dependencies. This is a test-only fix, so there's no red/green pair: the "red" is the intermittent failure above, measured as 2 in 20. Related: #2030, #2048 (where the test was added). 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01KaR4xu8gVykQBkH8yG8m6H Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Alex BandClaude Opus 5.5
|
a4ee3a7328 |
feat(scope-audit): observer region filter for both views (#2171)
## Summary Builds on the transport view from #2167 and #2169 (both merged); rebased onto `master` (`352a0ccd`), so the diff is this PR's six commits only. The Scope Audit was the one analysis page without the region selector. This adds the shared region filter to it, for both the declared and the transport view. `?region=SFO,SJC` means what it means on every other endpoint: count only forwarding **heard by observers in those regions**, not repeaters located there. - **API** (`GET /api/scope-audit`, both modes): - The region codes are resolved to their observers first (a handful of rows), then the audit's hop scan gets `AND o.observer_idx IN (…)` on v3, or `o.observer_id IN (…)` on v2. The 3.5M-row hop scan never joins `observers`. - `All` means no filter (shared `normalizeRegionCodes`). Codes with no observer match nothing, instead of silently widening to every observer. - Responses echo the normalised codes as `region`. The declared and transport caches and their singleflight keys now include the region, so each region set is cached separately with the existing TTLs. - Declared-region verification (`regionEvidence`) is deliberately left unfiltered: it corroborates what a repeater declares from its own traffic, which doesn't depend on where a hop was heard. - **Page:** the shared `RegionFilter` sits with the view and window controls; a change reloads the view, and the selection persists like every other page. The transport view's text box is relabelled **Scope** (it already sends `?scope=` since the rename in #2167), so the page no longer has two controls called Region. - **Docs:** `docs/api-spec.md` (parameter, both response shapes) and the OpenAPI parameters for the route (`mode`, `scope` and `region` were all missing). Partial fix for #2142 (its region filter for rollout tracking), together with #2167 and #2169. ## Performance The unfiltered path must not get slower: it's the same query with an empty clause. `BenchmarkComputeScopeTransport` (from #2167, 1,200 repeaters), the PR it stacks on vs this branch, `-count 8`: | | before | after | | |---|---|---|---| | sec/op | 220.3 ms | 218.0 ms | ~ (p=0.065) | | B/op | 43.77 MiB | 43.70 MiB | ~ | | allocs/op | 833.5k | 833.5k | ~ | **Filtered requests, measured on staging** (a copy of a production database): the first version drove `o.observer_idx IN (…)` from `idx_observations_observer_idx`, reading every observation the region's observers ever made. Fixed in `80414242` with a unary `+` (the same treatment as #2166), so the scan starts from the `first_seen` window and filters per row. Red test `67e063b1` asserts the query plan against a fixture with the real indexes. | scan, one region over 24h (13 observers) | plan | time | hop rows | |---|---|---|---| | `o.observer_idx IN (…)` | observer index | 13.44 s | 102,114 | | `+o.observer_idx IN (…)` | `first_seen` window | **0.25 s** | 102,114 | | unfiltered | `first_seen` window | 0.35 s | 225,983 | Cold end-to-end requests on staging after the fix: 24h declared, unfiltered 0.84 s · one region 0.48 s (was 16.7 s) · two regions 0.22 s · transport, one region 0.42 s · transport, six regions 0.75 s · 7d, one region 3.46 s. 24h transport, live data: 383 repeaters unfiltered, 204 heard by one country's observers. ## Tests - Red `629e4f36` (Go): extends the audit fixture with an `observers` table and `observations.observer_idx`. Covers the declared view ignoring a scope heard only by another region's observer, the transport view listing only repeaters heard in the region, `region=All`, per-region caching, a region with no observers, and the echoed codes. Five fail on assertions; `region=All` passes because the stub ignores the filter, which is the behaviour it pins. - Green `6f1b1e1c`: the implementation, plus a v2-schema case (`observer_id`), so both branches of the clause are covered. - Red `b4933d68` / green `08ecc172` (frontend): `apiPath` carries the selection in both views and adds nothing when none is selected. The region control on the page is a net-new UI surface. - `go test` (cmd/server) passes. Frontend suites: 193 of 194. The one failure, `test-issue-1956-release-routing.js`, needs CI's `GITHUB_REF_NAME` and fails the same way on `master` locally. ## Screenshot The same transport view with the shared region filter set to **SJC**: only forwarding heard by San Jose observers counts, so 7 repeaters are listed instead of 44, with their carried scopes counted from those observers alone. Taken against `test-fixtures/e2e-fixture.db` with fictional scopes.  ## Browser validation The branch's server was run locally with 14 configured regions (dropdown layout), checked in headless Chromium at 1280, 900, 700 and 390 px (touch): | check | 1280 | 900 | 700 | 390 touch | |---|---|---|---|---| | ticking SJC re-requests with `®ion=SJC` | yes | yes | yes | yes | | menu fully on screen | yes | yes | yes | yes | | page wider than viewport | no | no | no | no | | page errors | none | none | none | none | Found and fixed this way: the filter is the right-most control on desktop, and the menu (which opens rightwards) ran 66 px off the page. It now opens leftwards above 640 px. On a phone the controls wrap, the filter starts a line, and the default applies. ## SonarLint SonarQube Community rules (the SonarLint engine) were run on the files this PR touches, and only issues on lines it adds are listed. - **S3776 on `ScopeAuditForwardingHeardIn`:** complexity 55. The function it generalises, `ScopeAuditForwarding`, is 54 on `master`; the region clause adds one point. - **S3776 on `computeScopeAudit`:** complexity 65, unchanged from `master`. - Both pre-date this PR. Splitting them is better as a refactor PR of its own than mixed into a feature. - The page code's findings were fixed in the transport-view PR it stacks on. ## Checked against the repo rules - Rebased onto `master` (`673442cc`). Each red commit was re-run and still fails on its assertions; each green commit passes (TDD.md). - `make test`: all 16 modules pass. Frontend suites: all pass except `test-issue-1956-release-routing.js`, which needs CI's `GITHUB_REF_NAME` and fails the same way on `master` locally (#2160 addresses it). - No private data: test fixtures and examples use the codes the existing tests use (SJC/SFO/LAX, `#be`-style scopes). - No new `map[string]interface{}`, no hard-coded colours. `git add` was explicit, and there is one logical change per commit. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01KaR4xu8gVykQBkH8yG8m6H --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
efitenandClaude Opus 5.5
|
55dd28164d |
test(e2e): assert the configured-scope row after the theme-refresh re-render (#2180)
## What `test-issue-1648-m2-icons-e2e.js` now asserts the configured-scope row only after the page's `theme-refresh` re-render has fired and the detail has rendered again. ## Why On 2026-10-10 this assertion failed CI on four PRs (#2169, #2171, #2154, #2164), each time at the first live-map case: `confirmed scope row must be visible, including an empty scope`. A re-run of the failed jobs passed every time. The code path: - `customize-v2.js:814` dispatches `theme-changed` once `/api/config/theme` resolves, on every page load. - `app.js:1309-1315` turns that into `theme-refresh` 300 ms later. - `live.js:4713` re-runs `showNodeDetail(activeNodeDetailKey)` on `theme-refresh`, and `showNodeDetail` first sets `#nodeDetailContent` to "Loading…" (`live.js:2598`). This is the only caller besides the marker click (`live.js:3090`). - The test waited for `#nodeDetailContent table` and then called `row.isVisible()`, which does not wait. A refresh landing between the two fails the check. The same race could let the null-scope case pass on the "Loading…" panel, because `row.count() === 0` holds there too. The fix uses the same `addInitScript` flag pattern as `test-e2e-playwright.js:83`. ## Testing - Locally against a server on the E2E fixture: 5 of 5 runs pass, 27 of 27 checks each. - **Not reproduced locally.** The unchanged test also passed every local run: 5 plain, 3 with `/api/config/theme` delayed by 1 s, 5 with 8x CPU throttling. So the reasoning above comes from the code path and the four CI failures, not from a local red-then-green. - Test-only change, no product code touched. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Alex BandClaude Opus 5.5
|
352a0ccd19 |
feat(region-filter): configurable region quick picks (#2170)
## Summary **Independent of every other open PR.** Choosing a pick saves the selection, redraws the control and notifies the page's listeners itself, exactly as a click in the control does, so it re-queries straight away whether or not #2168 (`setSelected` notifies) is merged; with both in, nothing fires twice. Region quick picks are named groups of region (IATA) codes, offered as one-tap choices inside the region filter, e.g. "Northern California" for a dozen airport codes instead of ticking each box. One deployment has been injecting these into the page through an nginx `sub_filter` and a script that wraps `window.RegionFilter`; this makes them a configured feature. ```jsonc "regionQuickPicks": [ { "name": "Northern California", "description": "Bay Area and Sacramento observers.", "regions": ["SJC", "SFO", "OAK", "SMF"] } ] ``` - **Server:** `GET /api/config/region-quick-picks` returns the groups cleaned up: names and codes trimmed, codes upper-cased and de-duplicated in their configured order, groups without a name or codes dropped, at most 20 groups of 200 codes, always a list. `/api/config/regions` is unchanged. - **Region filter:** loads the groups with the regions and offers every group with at least one observer today, inside the region control: - dropdown layout: rows of the same kind as **All** (checkbox and bold name), directly under All with a divider before the single regions; the row is ticked while that pick is the selection, ticking it selects the pick and closes the menu, unticking goes back to all; - pill layout: the picks lead the same bar, set off from the single regions by a thin divider. - A tap selects the group's codes that have an observer and re-queries straight away. - The selector then reads the group's name ("South Bay ▾") instead of "2 Regions", and the button shows as pressed. Selecting the same codes by hand does the same. - Tapping the active group again goes back to all regions. - A failed fetch, or no configured groups, leaves the filter exactly as before. - **Mobile:** the rows take the menu's 44 px row height under `(pointer: coarse)`; the pill-bar picks reuse `.region-pill` and its touch target. Inside `.filter-bar` (Packets) the existing touch rule for inputs made every region checkbox a 44 px square; the checkbox is now 20 px and the 44 px row is the tap target. - **Semantics:** like every region filter, a group means "heard by observers in these regions", not "located there"; `config.example.json` says so. No existing issue covers this. ## Screenshot Nodes page with two picks configured (example config below), after tapping **South Bay**: the pick and its codes (SJC, MRY) are selected, and the list re-queried for them. Taken against `test-fixtures/e2e-fixture.db`.  ## Config example ```json "regionQuickPicks": [ { "name": "South Bay", "description": "San Jose and Monterey observers", "regions": ["SJC", "MRY"] }, { "name": "East and North Bay", "description": "Oakland and San Francisco observers", "regions": ["OAK", "SFO"] } ] ``` Each pick needs a `name` and at least one code; `description` is optional (shown as the tooltip). Codes are trimmed, upper-cased and de-duplicated; a pick with no code that has an observer today is not shown. At most 20 picks of 200 codes. ## Browser validation The branch's server was run locally against a migrated copy of `test-fixtures/e2e-fixture.db` (observers in MRY, OAK, SFO, SJC) with three groups configured: "South Bay" (SJC, MRY, SCZ), "East and North Bay" (OAK, SFO, sts) and "Nowhere yet" (XXX). Checked in headless Chromium: | check | desktop 1280 px | phone 390 px (touch) | |---|---|---| | groups shown | South Bay, East and North Bay ("Nowhere yet" hidden) | same | | tap South Bay selects | SJC, MRY (SCZ has no observer) | same | | next `/api/packets` request | `region=SJC,MRY` | `region=SJC,MRY` | | selector label | "South Bay ▾" | "South Bay ▾" | | second tap | back to all regions | same | | button height | 24 px | 44 px | | page wider than viewport | no | no | | page errors | none | none | Checked on Packets (dropdown layout) and Nodes (pill layout). Three layout problems found this way on the phone were fixed before opening: a sideways-scrolling row clipped at the screen edge, the selector squeezed into a narrow column (the filter group is a non-wrapping flex row), and the two wrapped rows spread apart on Nodes (`align-content`). **Revision 1:** the first version put the picks in a separate row beside the selector; they moved inside the dropdown. Re-checked with a 14-region list and three picks on Packets: desktop 1280 px dark and phone 390 px light both keep the menu on screen (menu 340 px wide, page not wider than the viewport), the pick closes the menu and the trigger names the pick. **Revision 2:** picks are checkbox rows like **All** instead of chips. Re-checked on Packets at 1280 px dark and 390 px light: rows render bold under All, the divider separates the single regions, ticking a pick ticks its codes and names the trigger, no sideways scroll. Unrelated and not changed: on Nodes at 390 px the region row (existing pills included) starts flush with the screen edge. ## Tests - Red commit `81ea67d6`: the config type, a stub normaliser and endpoint returning nothing, and tests. - Go: cleaning (trim, upper-case, de-duplicate, drop incomplete groups), caps, nil config, and the endpoint's JSON. - Frontend, `tests/e2e/test-region-quick-picks.js` (moved from `tests/unit/` in `fced69a7`; it needs jsdom, so it runs in the E2E job like `test-table-sort.js`): the real `region-filter.js` in **jsdom** against faked config endpoints. Nine cases: rendering and hiding groups with no observer, selecting and re-querying, the selector label, toggling back, a hand-made selection marked active, the pill layout, no groups, a failed fetch, HTML escaping. Seven fail on the stub, all on assertions. - Green commit `0e33210e`: the implementation, styles, `config.example.json` (`regionQuickPicks` + `_comment_regionQuickPicks`), `docs/api-spec.md`, and the OpenAPI route list (`TestOpenAPICompleteness` requires it). The jsdom test also had a cross-realm comparison bug fixed in this commit (`Array.from` on arrays coming out of the jsdom window). - `make test`: 16 of 16 modules pass. Frontend suites: 195 of 196; the one failure, `test-issue-1956-release-routing.js`, needs CI's `GITHUB_REF_NAME` and fails the same way on `master` locally. - Red commit `f991060d` (revision): picks live inside the dropdown menu before the checkboxes, choosing one closes the menu, and in the pill layout they share the region bar; two fail on assertions against the old layout. Green commit `1f720e35`. Frontend 195 of 196 (same environment-only failure). - Red `134f5256` / green `da0adb50` (second revision): picks are checkbox rows between All and the single regions with a divider, no chips in the menu; ticking and unticking a pick row. The green commit also repairs `style.css`: `1f720e35` had prepended its block to the top of the file (its edit matched an earlier `.region-pill {`) and left the previous rules in place. The file now differs from `master` by 13 lines. ## Review notes - Groups are offered only when at least one of their codes appears in `/api/config/regions`, and only those codes are selected. The stored selection is pruned to known codes on load anyway, so selecting absent codes would not survive a reload. - Customizer (AGENTS.md rule 8): groups are operator configuration; a customizer editor could come later. ## SonarLint SonarQube Community rules (the SonarLint engine) were run on the files this PR touches, and only issues on lines it adds are listed. - **Fixed** in `b84ace4e`: - `normalizeQuickPickCodes` split out of the config normaliser (S3776, 17 > 15); - a comment on why a failed quick-pick fetch is ignored (S2486); - `for…of` in `activePick` (S4138); - in the tests: `node:` specifiers, `includes()`, and an explicit string comparator for sorted selections (S7772, S7765, S2871). - **Kept on purpose:** - S2486 is still reported on that `catch`. Swallowing the error is the intended behaviour: the filter works as before, just without picks. - S1523 on the `vm` test loader, as above. ## Checked against the repo rules - Rebased onto `master` (`673442cc`). Each red commit was re-run and still fails on its assertions; each green commit passes (TDD.md). - `make test`: all 16 modules pass. Frontend suites: all pass except `test-issue-1956-release-routing.js`, which needs CI's `GITHUB_REF_NAME` and fails the same way on `master` locally (#2160 addresses it). - No private data: test fixtures and examples use the codes the existing tests use (SJC/SFO/LAX, `#be`-style scopes). - No new `map[string]interface{}`, no hard-coded colours. `git add` was explicit, and there is one logical change per commit. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01KaR4xu8gVykQBkH8yG8m6H --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Alex BandClaude Opus 5.5
|
99efaed33d |
fix(region-filter): setSelected notifies change listeners on a real change (#2168)
## Summary `RegionFilter.setSelected()` saved the selection and redrew the control but never ran the `onChange` listeners; only a click (`toggleRegion`) did. A programmatic selection therefore updated the dropdown while the page kept showing data for the old selection until the next click. Affected today: the Packets page applies a `?region=` URL parameter through `setSelected`. It registers its listener afterwards, so its first load is fine, but any other page's listener that is still registered sees nothing. And anything that offers one-click region groups has to wrap `setSelected` to make it re-query (an nginx-injected overlay on one deployment does exactly that). This PR stands alone; #2170 (configurable quick picks) does not need it. `setSelected` now compares the new selection with the current one (order-insensitive; empty and null both mean "all") and, only when they differ, runs the listeners after saving and re-rendering, exactly as a click does. Re-applying the current selection stays silent, so a caller restoring state does not trigger a reload. No existing issue covers this. ## Tests - Red commit `75569b78`: `tests/unit/test-region-filter-setselected.js` loads the real `public/region-filter.js` in a `vm` sandbox. Two cases fail on `master` (a programmatic selection notifies once with the new codes; a clear notifies once with `null`). Two pass on both commits on purpose: re-applying the current selection in another order, and clearing an empty selection, must not notify. - Green commit `8a7ddfe1`: the fix. - Listed in `test-all.sh` in sorted order; `test-test-inventory.js` passes. `test-1108-region-hide-nodes.js` (the other region-filter suite) passes. Frontend suites: 194 of 195; the one failure, `test-issue-1956-release-routing.js`, needs CI's `GITHUB_REF_NAME` and fails the same way on `master` locally. ## Review notes - `packets.js` "clear all filters" calls `setSelected([])` and then reloads itself. When a region was selected, that path now also triggers the page's region listener, so the packets list is requested twice. It is a cosmetic double fetch on an explicit user action; left as is to keep this PR to the component. - No performance claim; this is a correctness fix. ## SonarLint SonarQube Community rules (the SonarLint engine) were run on the files this PR touches, and only issues on lines it adds are listed. - **Fixed** in `9e732706`: `node:` specifiers (S7772). - **Kept on purpose:** S1523 on `vm.runInContext`, which loads the repo's own `public/region-filter.js` as the other frontend unit tests do. ## Checked against the repo rules - Rebased onto `master` (`673442cc`). Each red commit was re-run and still fails on its assertions; each green commit passes (TDD.md). - `make test`: all 16 modules pass. Frontend suites: all pass except `test-issue-1956-release-routing.js`, which needs CI's `GITHUB_REF_NAME` and fails the same way on `master` locally (#2160 addresses it). - No private data: test fixtures and examples use the codes the existing tests use (SJC/SFO/LAX, `#be`-style scopes). - No new `map[string]interface{}`, no hard-coded colours. `git add` was explicit, and there is one logical change per commit. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01KaR4xu8gVykQBkH8yG8m6H --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Alex BandClaude Opus 5.5
|
deb55dc82e |
feat(scope-audit): transport view on the Scope Audit page (#2142, page) (#2169)
> **Requires #2167 to be merged first.** This page calls the transport view API that #2167 adds (`/api/scope-audit?mode=transport`), so it cannot be split from it; until #2167 merges this PR also shows its commits. ## Summary Page half of #2142. **Stacked on #2167** (the API); review that first. #2171 adds the observer region filter on top. Partial fix for #2142 until all three are in. Adds a **Declared / Transport** toggle to `#/scope-audit`. The transport view calls `?mode=transport` and lists every repeater seen forwarding in the window: - **Seen carrying:** one chip per region scope with its packet count (first/last seen in the tooltip); - **Declared:** the repeater's declared regions coloured by whether they were seen carried (the declared view's own chips), or *not asked* when no declared-regions answer exists. Never asked is worded differently from declaring nothing; - **Other traffic:** unscoped, unmatched and ambiguous-hop counts. A **Region** box (suggestions from the scopes seen) adds a *Carries X* column, lists the repeaters not yet carrying it first, and summarises carriers against active non-carriers: the rollout report the issue asks for. Window, view and region are deep-linked (`#/scope-audit?window=7d&mode=transport&scope=be`) and survive a reload. The page says plainly that *transported* means seen carrying, not configured, and that over a short window a quiet region looks the same as a missing one. The intro text changes with the view. ## Screenshot The transport view with scope `be` typed in: every repeater seen forwarding in the last 24 h, whether it carries `be` (non-carriers first), the scopes it was seen carrying with counts, and its declared regions or *not asked*. Taken against `test-fixtures/e2e-fixture.db` with fictional `be` / `be-van` / `nl` scopes.  ## Browser validation (staging, real data) Headless Chromium against a staging copy of a production database with its live feed: - `#/scope-audit?window=24h`: the declared view, 16 rows, as before. - Clicking **Transport**: 385 rows; the hash gains `mode=transport`. - Typing a scope as `#XX-YY` in capitals: normalised to `xx-yy`; the hash gains it (now `scope=xx-yy`); summary "385 repeaters seen forwarding in the last 24h · 16 with a declared-regions answer · 51 carried xx-yy · 334 active but not seen carrying it"; the first rows are the non-carriers. - Reloading that hash restores view and region. Switching back to Declared hides the Region box and restores the declared intro. No page errors. - Checked at 1280 px and 390 px wide. Two layout bugs found this way were fixed before opening: the region box inherited `width: 100%` from `.nodes-search`, and `display: flex` overrode the `hidden` attribute on the region bar. ## Tests `tests/unit/test-frontend-helpers.js` gains 10 cases on rendered markup and state, using the page's exported internals like the existing scope-audit tests: hash and API paths (and that a region never leaks into the declared view's link), region normalisation (`#`, capitals, empty), *not asked* vs declared chips (with the `*` wildcard), carried-scope chips with counts and the filtered region marked, the carries column only with a region, HTML escaping of names, the summary counts, region suggestions, and the per-view intro. 768 passing in that file. Net-new UI surface, so per TDD.md the tests land in this PR rather than as a preceding red commit. Frontend suites on this branch: 193 of 194; the one failure, `test-issue-1956-release-routing.js`, needs CI's `GITHUB_REF_NAME` and fails identically on `master` locally (#2149 and #2160 address that). ## Review notes - Colours only through existing CSS variables (no hex literals); the filtered region is marked with the link colour, not a new status colour. - The window note's wording is in the same spirit as the declared view's `windowHonestyNote`. - No API calls per row: one request per view/window/region, cached client-side for 30 s like the declared view. ## SonarLint SonarQube Community rules (the SonarLint engine) were run on the files this PR touches, and only issues on lines it adds are listed. - **Fixed** in `df53e770`: - one `modeQuery` helper for the page hash and the API path (S3358; the two copies were identical); - the carries-scope cell built before the row (S3358); - `localeCompare` for the region suggestions (S2871), `for…of` and `.dataset` (S4138, S7761), optional chaining (S6582); - object spread in the helper tests (S6661). - Optional chaining and spread are already used throughout `public/`. No behaviour change. ## Checked against the repo rules - Rebased onto `master` (`673442cc`). Each red commit was re-run and still fails on its assertions; each green commit passes (TDD.md). - `make test`: all 16 modules pass. Frontend suites: all pass except `test-issue-1956-release-routing.js`, which needs CI's `GITHUB_REF_NAME` and fails the same way on `master` locally (#2160 addresses it). - No private data: test fixtures and examples use the codes the existing tests use (SJC/SFO/LAX, `#be`-style scopes). - No new `map[string]interface{}`, no hard-coded colours. `git add` was explicit, and there is one logical change per commit. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01KaR4xu8gVykQBkH8yG8m6H --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Alex BandClaude Opus 5.5
|
323b8040d4 |
perf(frontend): throttle nav stats to one refresh per 15 s, none in hidden tabs (#2162)
## Summary The nav bar's packet / node / observer counts were refreshed three ways at once: a 15 s timer, every batch of live WebSocket packets (`debouncedOnWS`, 250 ms), and the `/stats` client cache being cleared every 5 s while packets arrive. None of them checked whether the tab was visible. So every open tab, including hidden ones, asked `/api/stats` every few seconds for as long as it stayed open. On one production instance's nginx log for a single day, `/api/stats` was **135,654 of 187,411** CoreScope API requests (72%), about 20,000 a day for each browser that kept a tab open. Each response is cheap thanks to the 10 s server-side cache (#1910), so this is about request volume, bandwidth and log noise, not server CPU. This PR adds `createVisibleThrottle()` to `app.js` and routes all three triggers through one throttled refresh: - at most one `/api/stats` request per 15 s from a visible tab (the old timer's own period); - none from a hidden tab; - one immediate refresh when a hidden tab becomes visible again. No existing issue covers this. ## Performance Measured with headless Chromium, counting `/api/stats` requests from a real tab: 60 s visible, then 60 s with the page hidden (`document.hidden` and a `visibilitychange` event), then back to visible. | `/api/stats` requests | production (current release) | staging, `master` | staging, this PR | |---|---|---|---| | visible tab, per minute | 9 | 4 | 4 (at +10, +25, +40, +55 s) | | hidden tab, per minute | 6 | 5 | **0** | | on return to visible | n/a | n/a | 1, immediately | The same check confirmed the nav bar still renders its counts (`… pkts · … nodes · … obs`). ## Tests - Red commit `2c81a3e4`: `tests/unit/test-nav-stats-throttle.js` loads the real `public/app.js` in a `vm` context. `createVisibleThrottle` lands as a pass-through stub, so the test fails on its first assertion (40 refreshes where 1 is expected). - Green commit `f0ba92ab`: the throttle, and the nav stats timer, WS hook and a new `visibilitychange` listener calling one throttled trigger. All 6 cases pass: one refresh per burst, again after the interval, at most 241 an hour, none while hidden, one on return, and a source check that `updateNavStats` is wrapped and not called by the timer directly. - Added to `test-all.sh` in sorted order; `test-test-inventory.js` passes. - Frontend suites: 194 of 195 pass. The one failure, `test-issue-1956-release-routing.js`, needs CI's `GITHUB_REF_NAME` and fails the same way on `master` locally. ## Review notes - `home.js` keeps its own one-shot `/stats` load on the home page; only the nav bar's polling changed. - A tab opened in the background loads its counts when it is first shown. - The 15 s interval is a constant at the call site. Could be exposed in the customizer later (AGENTS.md rule 8); not added here. - No Playwright E2E added: the change is timing behaviour, covered by the unit test against the real file and by the browser measurement above. Happy to add an E2E if wanted. ## SonarLint SonarQube Community rules (the SonarLint engine) were run on the files this PR touches, and only issues on lines it adds are listed. - **Fixed** in `9eb94ea1`: `node:` specifiers (S7772). - **Kept on purpose:** - S7773 wants `Number.isFinite`/`Number.parseInt` for the `isFinite` and `parseInt` handed to the `vm` sandbox, but `app.js` calls the global functions, so those are the ones it needs. - S1523 flags `vm.runInContext`, which is how the frontend unit tests load the repo's own `public/app.js`. ## Checked against the repo rules - Rebased onto `master` (`673442cc`). Each red commit was re-run and still fails on its assertions; each green commit passes (TDD.md). - `make test`: all 16 modules pass. Frontend suites: all pass except `test-issue-1956-release-routing.js`, which needs CI's `GITHUB_REF_NAME` and fails the same way on `master` locally (#2160 addresses it). - No private data: test fixtures and examples use the codes the existing tests use (SJC/SFO/LAX, `#be`-style scopes). - No new `map[string]interface{}`, no hard-coded colours. `git add` was explicit, and there is one logical change per commit. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01KaR4xu8gVykQBkH8yG8m6H --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
n30nex
|
a5b60ca594 |
test: make local validation independent of runner environment (#2160)
Fixes #2159. Related to #1995. Standalone release tests failed when `GITHUB_REF_NAME` was absent and could use an unrelated runner tag when it was present. Supply the simulated tag explicitly; exercise the real release-notes step with both missing and unrelated host values, including present and absent notes files. Two filesystem fixtures now report unsupported Windows capabilities explicitly: POSIX permission bits, and symlink creation denied with error 1314. Unix mode checks and all symlink safety assertions remain intact; other symlink setup errors still fail. Production code and workflows are unchanged. This is 29 added lines across three test files. Validation: reproduced all three failures before fixing them; focused regressions, the full standalone frontend suite, and native Go server/ingestor suites pass. The POSIX permission and unprivileged Windows symlink fixtures are explicitly skipped on this host and remain enabled in Linux CI. The browser smoke suite passed 148 checks, with three fixture-dependent skips and two pre-existing disabled cases. No configuration or customizer changes. |
||
|
efitenandClaude Opus 5.5
|
0737f10a5d |
feat(notify): optional external node alerts feed and the node.external event (#2177) (#2178)
Closes #2177. ## What Node notifications can mail alerts computed outside CoreScope. With `userManagement.notifications.externalAlerts` configured, users can choose a new event, `node.external`, which is off by default and only offered when configured: ```json "notifications": { "enabled": true, "externalAlerts": { "url": "http://coverage-job/alerts.json", "maxAgeHours": 48, "label": "Coverage" } } ``` The feed: ```json { "generatedAt": "2026-10-11T01:35:12Z", "alerts": [ { "pubkey": "<64 hex>", "key": "SE", "text": "never heard to the south-east", "url": "https://example.org/coverage" } ] } ``` Mail lines: `<name>: Coverage: never heard to the south-east, <time>` when an alert appears, `<name>: Coverage resolved (SE), <time>` when it is gone from a fresh feed. ## How - `cmd/server/notify_external.go`: feed types, `parseExternalFeed` (RFC 3339 `generatedAt`, age check, per-alert validation), `fetchExternalFeed` (200 only, 1 MB cap, 10 s timeout). - `notify_eval.go`: `evalExternal`. Subjects are `<pubkey>/<key>`; a `<pubkey>/*` row is the per-node baseline, so the first fresh feed after watching stores the current alerts without a mail (like `foreign.new`), and later alerts mail at once. `External == nil` (not configured, nobody chose it, or stale) leaves every `node.external` state as it is. - `notify_service.go`: the feed is fetched only when configured and at least one user chose the event. Each new reason the feed is unusable is logged once, and so is the recovery. Mail text and link: the alert's `url`, otherwise the node page. - `notify_handlers.go`: `availableEvents` and the new `externalLabel` include the event only when configured; PUT answers 400 for `node.external` otherwise. - `internal/users/notify.go`: event constant, `OptionalNodeNotifyEvents` (canonical order: node, optional, admin), `RemoveWatch` also deletes the node's `<pubkey>/...` rows. - Account page label, OpenAPI descriptions, `config.example.json` comment, `docs/user-guide/accounts.md` section. Decisions on the three open questions in #2177: the generic name `node.external`; the resolved mail names the key instead of repeating the text, so there is **no `users.db` schema change** (a bump would also make older binaries refuse the file); one feed. ## Invariants - Read/write separation (#1283): the server only reads the feed over HTTP. State rows go to `users.db` through `internal/users`, like the other events. - A dead, garbled or old feed never mails "resolved": `TestEvaluateExternalStaleFeedChangesNothing` and `TestNotifierMailsExternalAlertsAndSkipsAStaleFeed` (HTTP 502 and a 72 h old feed in a row, then a fresh empty feed resolves). ## Performance The evaluator adds one pass over the stored states (already read every tick) plus, per user with the event on and per watched node, the node's current alerts. Bounded by `maxWatchesPerUser` (50) times the alerts per node. The feed is fetched once per tick (default 5 minutes) only when someone chose the event, capped at 1 MB. No packet-store lock is taken for it. Nothing changes for instances without `externalAlerts`. ## Tests - `cmd/server`: `go test` green locally (Windows; `TestSaveGeoFilterPreservesFileMode` skipped as it needs Unix mode bits). New: `notify_external_test.go` (feed parsing and limits, HTTP fetch 200/404/oversized, config resolution, evaluator baseline/new/resolved/returning/stale/not chosen/notifications off, notifier end to end with a fake feed, account routes). - `internal/users`: green, 2 new tests (optional event and canonical order, `RemoveWatch` deletes external subjects only for that node). - Frontend: `test-all.sh` green, 1 new test in `test-notifications-ui.js` (label from `externalLabel`, escaped). - `scripts/check-xss-sinks.sh --file public/notifications.js`: clean. ## Not done - No E2E test: the account page change is one label, covered by the unit test. - Not run against a live instance yet. The first feed in use is the coverage job of on8ar.eu (`alerts.json`, 29 alerts at the moment). Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
efitenandClaude Opus 5.5
|
c2a590b741 |
feat(companions): linked-only ingest, account Companions and My coverage (#2176)
Part 3 of 3 for #2173 (companion linking): the optional linked-only ingest filter, the account page and the "My coverage" toggle, plus the docs. Builds on #2174 and #2175 (both merged); `upstream/master` is merged in, so the diff shows only this part. ## Changes in this PR **Ingestor: linked-only ingest** (`cmd/ingestor/linked_companions.go`) - With `clientRxCoverage.requireLinkedCompanion` on (default off) and user management enabled, every `meshcore/client/<pubkey>/...` message whose pubkey is not in `companion_links` is dropped before any client handler runs, and counted (`client_unlinked_dropped` in the stats file). - The ingestor opens `users.db` read-only (`mode=ro`, raw SQL, no `internal/users` import), like the approved-channels reader, so the #1283 write invariant holds. `TestLinkedCompanionSetIsReadOnly` pins it. - The set refreshes every 60 s. A miss re-reads at most once per 5 s, so a companion linked a moment ago is accepted within seconds while a flood of unknown pubkeys costs one read per 5 s. - It passes everything until its first successful read, and when `users.db` has no `companion_links` table yet (a server older than this). After the first read it keeps the last set on a read error. It is a filter, not a security boundary: anyone with the shared broker credential can still publish under a linked pubkey. **Frontend** - Account page: device tokens show under Devices as "CoreDrive RX" with their label; a new Companions section lists linked companions (name, short key, linked since, last seen) with an Unlink button. - Coverage page (`#/rx-coverage`): a "My coverage" toggle for logged-in users (`mine=1`). Only the newest coverage reply is drawn, and a refused `mine=1` turns the toggle off. Under "My coverage" the page does not ask for the "nothing received" cells from #2172, and their legend entry is hidden. **Docs**: user guide (accounts: devices, companions), `docs/client-rx-coverage.md` (linking, linked-only ingest, the CORS exception), `config.example.json`, and the spec status. ## Performance The filter runs on every client message. Benchmarks with 1000 linked companions (12-core dev box, `go test -bench`, 3 runs each): | Path | Time | Allocations | |---|---|---| | linked pubkey (lock-free map lookup) | 54 ns/op | 0 | | linked pubkey, parallel | 11-14 ns/op | 0 | | unlinked pubkey inside the 5 s miss gap | 85-95 ns/op | 0 | | one `users.db` re-read | 0.38 ms | 3030 | The unlinked benchmark asserts that no extra `users.db` read happens. A re-read happens at most once per 5 s on a miss and once a minute from the refresh loop. ## Verification - `cmd/ingestor`: 15 new tests (filter off changes nothing, linked passes, unlinked dropped before every handler, miss refresh accepts a new link, the 5 s cap holds under a flood, unlink honoured after the periodic refresh, missing table and missing file, fail-closed only after the first read, read-only open, startup, stats counter) and 4 benchmarks. Full ingestor suite green. - Frontend unit tests for the account page (device tokens, companions list escaping, empty state, Unlink confirm and cancel, load errors) and `tests/unit/test-rx-coverage-mine.js`. - E2E `tests/e2e/test-user-management-e2e.js` gains a companion step (device token under Devices, a companion linked with a real Ed25519 signature under Companions, Unlink). Run locally against the e2etest build: 28/28. The coverage suites (`test-rx-coverage-gaps-e2e.js`, `-noise-`, `-viewport-`) still pass. - Frontend unit suite (`test-all.sh`) green. ## Not in this PR - Narrowing the "nothing received" cells to the caller's companions under "My coverage" (the track query takes one companion today). - Per-user broker credentials or ACLs, which would turn the filter into an access control (non-goal in the spec). --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
efitenandClaude Opus 5.5
|
f3004d977e |
feat(rx-coverage): show where a companion drove and received nothing (#2172)
The Mobile RX coverage page (`#/rx-coverage`) shows only cells where a companion received something. Where a companion drove and received nothing is just as useful for operators, and the data is already there: the RF sample track (`client_rf_samples`, #1905/#1906) and the receptions (`client_receptions`) use the same hex grid (`hexCellAt`, `zoomToHexRes`). This PR adds those cells as a hatched layer under the reception cells. ## Changes in this PR **Server** - `GET /api/rx-coverage?...&gaps=1` adds a `gaps` member to the FeatureCollection: cells with moving (non-stationary) track samples and no reception in the same `days` window and `bbox`, each with `samples`. `gaps_truncated` is set when more than 5000 cells qualify (the most-sampled are kept, same pattern as `coverageFeatureCap` and `rfNoiseFeatureCap`). - Gaps are only returned when `clientRfSamples` is on and there is no `node` filter: with `?node=` the question would be "this node was not heard", which these cells do not answer. `?rx=` limits both the track and the receptions to that companion. - Without `gaps=1` the payload is unchanged: the `gaps` member is omitted. - New code lives in `cmd/server/rx_coverage_gaps.go` (pure `aggregateGaps` plus `queryTrackPoints`); the handler change in `rx_dashboard.go` is 14 lines. - `/api/rx-coverage` is now documented in `openapi.go` and removed from `openapi_known_gaps.json` (#1670). **Frontend** (`public/rx-coverage.js`, `public/node-reach-coverage.css`) - A "Nothing received" toggle on the signal layer, on by default, only rendered with `MC_CLIENT_RF_SAMPLES`. Hidden on the noise layer. - Gap cells use a 45-degree hatch (an SVG pattern in a dedicated renderer, so they sit under the reception cells). The hatch is denser for cells driven through more often (1 / 2-3 / 4+ samples). Tooltip: sample count. Legend entry: "driven, nothing received". - Off state is deep-linkable as `gaps=0`. - New token `--nq-cov-gap` for light and dark. Plain grey (`--nq-cov-grey`) already means "received, no SNR", so the hatch keeps the two apart. ## Threshold: one sample Measured on a live network (analyzer.on8ar.eu, Belgium, September to October 2026), with cells of about 250 m: a cell with track samples and no reception in one half of a month, driven again in the other half. How often did it receive something the second time? | Minimum samples | Cells "nothing received" | Driven again | Received something later | |---|---|---|---| | 1 | 3971 | 1263 | 29% | | 2 | 1565 | 592 | 32% | | 3 | 619 | 243 | 35% | | 5 | 142 | 63 | 40% | A higher threshold hides cells without making the rest more reliable, so the threshold is 1 and the wording is "nothing received in this period", never "no coverage". Using every decoded packet as reception evidence (including 1-byte path hashes) gave 25% to 32%, so the capture rule for 1-byte hashes is not what drives this. The median cell has 2 samples. ## Verification - Go: 7 new tests in `rx_coverage_gaps_test.go` (gap/no gap, stationary excluded, single sample, cap keeps the most-sampled, endpoint, `?rx=` filter, no `gaps` member when not asked / with `?node=` / with RF samples off). The endpoint tests fail without the handler change. Full `cmd/server` suite green locally, `go vet` and `gofmt` clean. - E2E: `tests/e2e/test-rx-coverage-gaps-e2e.js` (4 steps: no toggle with RF samples off, on by default with hatched cells under the reception cells, toggle off with `gaps=0` and deep link, noise layer hides it). 2 steps fail without the frontend change. Existing `test-rx-coverage-noise-e2e.js` and `test-rx-coverage-viewport-e2e.js` still pass. Listed in `scripts/non-unit-tests.json`. - Frontend unit suite (`test-all.sh`) green. - Real data on a staging instance (7 days, bbox 50.8,4.9,51.3,5.8, z=12): 478 reception cells and 379 gap cells, no cell in both, reception features identical with and without `gaps=1`, 0.48 s to 0.52 s per request both ways. - Browser: checked on staging with real data (light theme), and light, dark and 390 px wide with mocked data. ## Not in this PR - A per-node variant on the Reach page ("this repeater transmitted while a companion was nearby, and was not heard"). That needs the transmit moments from `observations` and is a heavier query. - `deploy.yml` does not get a line for the new E2E file, like the other entries that are only in `non-unit-tests.json` (#2037). Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
nullrouten0andClaude Mythos 5.1
|
57395e2b28 |
fix(ui): escape observer iata, name and id, and node names, in eight render sinks (#2118)
Anyone who can publish to an MQTT broker that CoreScope reads controls three strings: the observer `iata` (a topic segment), the observer `name` (the JSON `origin` field) and the observer `id` (a topic segment). Anyone with a radio controls the node `name` in an ADVERT. Eight places wrote these into the page with `innerHTML` without escaping, so a crafted value runs as script in every visitor's browser: | file | what | |---|---| | `packets.js` | Region badge in the group, child and single packet rows | | `observers.js` | Region badge in the observers table and in the slide-over | | `live.js` | "Heard By — Regions" header in the node detail | | `nodes.js` | observer name in the clock-skew evidence panel | | `analytics.js` | `observer_id` inside `data-observer=` and `id=` attributes | | `app.js` | node name in the favorites dropdown | | `customize-v2.js` | node name in the geofilter prune preview (admin page) | **Fix:** each value now goes through the existing `escapeHtml()` / `esc()` helper. No behaviour change for normal values. **Tests:** one test per sink added to `tests/unit/test-xss-escape-sinks.js`, following the file's existing pattern (pull the template out of the source, render it with a hostile value, check the output is escaped). Reverting any single fix makes its test fail. 41/41 pass. **Notes for reviewers:** - The ingestor uppercases `iata` before storing it. That is not a defence: tag and attribute names are case-insensitive, and script can be written with numeric character references. - An operator who has set an `observerIATAWhitelist` is protected from the `iata` sinks but not from the name/id sinks. - Follow-ups worth a separate PR: validate `iata` and observer `id` to a safe character set at ingest, cap the `origin` length, and add a `Content-Security-Policy` header. This PR is escaping only so it is easy to review and safe to ship. Running in production on our instance since 2026-10-07. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Mythos 5.1 <noreply@anthropic.com> |
||
|
n30nex
|
29095658e9 |
fix(nodes): explain packet and observation counts (#2132)
Red commit: `d8a6d42` ([CI assertion](https://github.com/Kpa-clawbot/CoreScope/actions/runs/37695889461)). Review-regression red: `f73643b` ([CI assertion](https://github.com/Kpa-clawbot/CoreScope/actions/runs/37703995494)). Fixes #2131. Partial fix for #1154: U2 only; other audit items remain open. Both node views explain distinct transmissions versus recorded observations. The shared help is reachable by keyboard and touch, stays readable at either scroll edge, and remains open when the pointer enters its text. Escape dismisses help while preserving node context and focus; a second Escape keeps the existing navigation behavior. Count expressions and API requests are preserved. One controller per nodes page handles one active help element and one queued animation frame. Scroll/resize work is coalesced; listeners are removed on page teardown. Work is constant relative to packet/node counts, with no dataset scans or new settings/dependencies. E2E assertion added: `tests/e2e/test-e2e-playwright.js:114`. Thirteen cases cover exact counts, accessible description, keyboard/touch, hover transfer, Escape, and viewport/scroll bounds. Existing metric assertions are unchanged. Browser verified: http://localhost:63508 using Chromium, controlled fixtures and actual API data. Desktop/mobile screenshots retained locally; additional checks exercised resizing and repeated page cleanup. Local validation: 189 standalone suites; core browser 148 passed, 3 fixture-dependent skips (existing manual skips unchanged); location/metric suite 6 passed; syntax, theme variables and lint passed (0 errors, 92 existing warnings). ## Preflight overrides - OpenClaw preflight/profile unavailable; repository checks and local Chromium used. - Windows ingestor symlink fixture lacks permission. Backend files are unchanged; full Linux Go CI is required. |
||
|
efitenandClaude Opus 5.5
|
2456762179 |
feat: account data export and users.db backup (#2141)
A small follow-up to parts A to E of #2128: users can download all data the instance holds about them, and the server backs up `users.db`. Stacked on #2140. Review the commits after that branch. ## The situation - An instance with accounts holds personal data, but a user has no way to get a copy (GDPR articles 15 and 20). Self-delete already exists. - `users.db` is the only file the server writes, and nothing backs it up: `GET /api/backup` snapshots the analyzer database only. ## What this PR adds **Data export.** `GET /api/account/export` and a "Download my data" button on the account page give one JSON file with: - the profile, including activation and a pending email change; - sessions (times and user agent); - synced settings, own proposals, notification prefs and watches; - audit rows where the user is actor or target, with other accounts as an id only; - the mail log. Password hash, session and link tokens and the unsubscribe token are left out: they are credentials. Each export is audited as `user.export`. **users.db backup** - Daily `VACUUM INTO` snapshots through `internal/users`, on by default when user management is on. They go to `backups/` next to `users.db`, the 7 newest are kept, the directory is 0700 and the files 0600. Rotation only touches `users-<timestamp>.db` files and never the snapshot it just wrote. Config: `userManagement.backup {enabled, dir, keep}`. - `GET /api/admin/users-backup`: an admin downloads a fresh snapshot to keep off the server; audited as `user.backup`. - `docs/user-guide/accounts.md` gains a restore procedure. It covers what a restore brings back (accounts deleted after the snapshot, old passwords, revoked sessions) and how to clean that up afterwards. Spec: [`docs/specs/2026-10-08-account-export-and-users-backup-design.md`](https://github.com/efiten/CoreScope/blob/feat/account-export-backup/docs/specs/2026-10-08-account-export-and-users-backup-design.md). ## Verification - `internal/users` with `-race` and the `cmd/server` suite pass locally, 30 new Go tests. The file-mode tests skip on Windows. - `sh test-all.sh` exits 0; the XSS gate in diff mode passes. - User-management E2E: 27 of 27 steps locally. - On our production instance since 8 October 2026; the first snapshot there was 143 kB. ## Not in this PR - Importing an export into another instance. - Off-site upload of the snapshots. The admin download covers that by hand. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
efitenandClaude Opus 5.5
|
de364a8bb6 |
feat: mail notifications for watched nodes (user management part E) (#2140)
Part E of #2128: a logged-in user watches nodes and gets one mail when a watched node goes offline, comes back, or reports a low battery. Admins can add instance events: a new foreign node, and an observer going offline or back. This covers the per-user mail part of #775. Stacked on #2139 (part D, which builds on #2138). Review the commits after that branch. ## The situation A sysop learns that a repeater went silent, or that its battery is running down, only by opening CoreScope. #775 asks for notifications. Accounts (part A) now give a verified address to mail and a mailer with delivery status. ## What this PR adds **Events.** They use the instance thresholds, so a mail does not disagree with the node page: - `node.offline`: silent for the role's `healthThresholds` window. Last heard comes from the packet store, and repeaters and rooms count relayed traffic (#1598). - `node.battery`: advert telemetry below `batteryThresholds.lowMv`, recovered at `lowMv + 100`. - Admins only: `foreign.new` (once per node) and `observer.offline`. **Store.** Schema v5 with `notification_prefs`, `notification_watches` and `notification_state`. **Server** (only with `userManagement.notifications.enabled`) - A notifier next to the janitor, every `intervalMinutes` (5). The evaluator is a pure function of watches, prefs, stored states and node snapshots. - No burst after a restart or a new watch: a subject's first evaluation stores its state without mailing. The loop waits for the startup load. While ingest is stale (newest packet older than 30 minutes) the offline checks pause, and after recovery they wait one silent window. - One mail per user per check. Limits: 20 per user and 100 per instance per rolling 24 hours, 50 watches per user. A change over a limit is recorded and never mailed later. - Every mail has a one-click unsubscribe (`List-Unsubscribe` and `List-Unsubscribe-Post`). The GET only redirects to a confirm page, so link scanners cannot unsubscribe anyone. Node names are cleaned of control and bidi characters before they go into a mail. - Routes: `GET`/`PUT /api/account/notifications`, `PUT`/`DELETE /api/account/notifications/watches/{pubkey}`, `POST /api/account/notifications/watch-my-nodes` (copies the synced "my nodes" list), `GET`/`POST /api/notifications/unsubscribe`. `mailer.Message` gains `Headers`. **Frontend.** A "Notify me" toggle on the node pages, a Notifications section on the account page (`#/account?section=notifications`), an unsubscribe page, and the notification mail count on the admin overview. Spec: [`docs/specs/2026-10-07-node-notifications-design.md`](https://github.com/efiten/CoreScope/blob/feat/notifications/docs/specs/2026-10-07-node-notifications-design.md). ## Performance Each check reads watches, prefs and states from `users.db` (one query each), the watched nodes from the analyzer DB in chunks of 500, and last-heard times from the packet store. | Measurement | Result | |---|---| | One check on our staging instance, 130,000 packets in memory, 1 watcher | 6.9 ms, of which 0.93 ms holding the store read lock | | Evaluator benchmark, 100 users with 50 watches each, 2,000 nodes | 1.75 ms | No work is added to ingest, broadcast or any request path. ## Verification - `internal/users` and `internal/mailer` with `-race`; the `cmd/server` suite plus `-race` on the notifier tests; 74 new Go tests. The same Windows-only failure as noted in #2138 applies. - `sh test-all.sh` exits 0; the XSS gate in diff mode passes. - User-management E2E: 26 of 26 steps locally. With notifications off the new steps fail, so they do test the flag. - On our staging and production instance since 7 October 2026. ## Not in this PR - Other channels (Discord, Telegram, webhooks). Detection is separate from delivery, so one can be added. - The topology, RF and anomaly alerts of #775. - If the whole server starts after a feed outage that already ended, the grace window is unknown and a watcher can get one wrong offline mail. The user guide says so. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
efitenandClaude Opus 5.5
|
abbe6846ff |
feat: propose and approve hashtag channels (user management part D) (#2139)
Part D of #2128: logged-in users propose a hashtag channel, an admin approves, rejects or later revokes it, and the ingestor decrypts approved channels without a restart. It is the first consumer of a generic proposal table, and answers the request in #2092. Stacked on #2138 (part C). Review the commits after that branch; GitHub shows C's commits here until it is merged. ## The situation - An instance decrypts only the hashtag channels listed in `hashChannels`. Users who want a city or club channel shown have to ask the operator, who edits `config.json` and restarts the ingestor. - #2092 asked for a suggest-and-approve flow; a downstream fork (dborup/CoreScope#99) runs one with anonymous suggestions and a file queue. ## What this PR adds **Name rules (`internal/channel`).** `ValidateHashtagName`, mirrored in the browser: - at most 31 UTF-8 bytes including `#` (firmware `ChannelDetails.name[32]`), case preserved, `Public` refused in any case; - control, bidi, separator and format characters refused except ZWJ, plus invisible fillers (`Other_Default_Ignorable_Code_Point`, U+2800, non-ASCII spaces). Go and JS give the same verdict for every code point. **Store.** Schema v4, one generic `proposals` table (kind, subject, status pending/approved/rejected/revoked, proposer, reviewer, note). One row per (kind, subject); limits are checked in the same transaction as the change. **Server** (only with `userManagement.channelProposals.enabled`) - `POST /api/proposals`, `GET /api/account/proposals`, `GET /api/admin/proposals`, `POST /api/admin/proposals/{id}/approve|reject|revoke`. - 400 for an invalid name. 409 for a duplicate, a rejected name, or a name already in `hashChannels`/`channelKeys` ("this channel is already decrypted on this instance"). 429 over a limit (`maxPending` 100, `maxApproved` 128, `perUserPerDay` 5). - `GET /api/channels` gains `approvedChannels`, so approved channels are listed before they have traffic. With the feature off the response is byte-identical. **Ingestor.** Opens `users.db` with `mode=ro` every 60 seconds, re-validates the names, and swaps its key map through an `atomic.Pointer`. Configured channels always win, and a failed read keeps the last good set. The ingestor never writes `users.db`; AGENTS.md now says so. **Frontend.** "Propose for everyone" in the add-channel dialog, an "approved" marker in the channel list, "My proposals" on the account page, and an admin "Proposals" tab with a warning on approve. The packet detail pane now escapes the channel name: until now only the operator chose those names. Spec: [`docs/specs/2026-10-07-channel-proposals-design.md`](https://github.com/efiten/CoreScope/blob/feat/channel-proposals/docs/specs/2026-10-07-channel-proposals-design.md). ## Performance Every key is tried on each GRP_TXT that no key opens (`decodeGrpTxt`). Benchmark of that worst case: | Keys | Time per packet | |---|---| | 320 configured | 204 to 216 µs | | 320 configured + 128 approved | 272 to 320 µs | That is about +33%, linear in the key count and capped by `maxApproved`. With the feature off the decoder gets the identical map and no ticker runs. ## Verification - `internal/channel`, `internal/users` (with `-race`), `cmd/server` and `cmd/ingestor` pass locally, 53 new Go tests. The ingestor's `TestWriteStatsAtomic_SymlinkAtDestIsReplaced` needs symlink rights on Windows and fails on `master` too. - `sh test-all.sh` exits 0; the XSS gate in diff mode passes. - User-management E2E: 21 of 21 steps locally (propose, approve, the channel appears, revoke removes it). - On our staging and production instance since 7 October 2026. ## Not in this PR - Private (PSK) channels: their keys stay in the browser (#725). - Anonymous proposals. - Names that need ZWNJ (some Persian and Urdu spellings) and subdivision flags are refused, because ZWNJ and tag characters are format characters. Allowing them later only widens the rule, so no approved name gets stranded. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
efitenandClaude Opus 5.5
|
18a634c115 |
feat: admin dashboard with overview and audit log (user management part C) (#2138)
Part C of #2128: an admin area with an overview, the user table and a global audit log. Off by default like the rest of user management: with `userManagement` off nothing changes, and non-admins get no new UI. This is the first of 4 stacked PRs (C, D, E, then a small export/backup follow-up). Each later one contains this branch; review them in order. ## The situation - An admin could manage users one by one in `#/admin/users`, but nothing showed whether the instance needs attention: accounts stuck in activation, bouncing mail, someone guessing a password, an MQTT source that dropped. - The audit log existed (`internal/users/audit.go`) but could only be read per user, and logins were not recorded. ## What this PR adds **Store (`internal/users`)** - Schema v3: an index on `audit_log(at)`. - `AuditList` with filters (action or group prefix like `user.login.*`, user as actor or target, period) and keyset pagination; `PruneAudit`; `Stats` for the user figures. **Server** - `GET /api/admin/stats` (typed struct): accounts by status, admins, registrations and active users over 7 and 30 days, logins and failed logins in 24 hours, mail by final status, and the attention items computed server-side. - `GET /api/admin/audit`: filtered, newest first, `next` cursor. - `GET /api/admin/users` gains `bouncing=1`. - Logins are audited as `user.login` and `user.login.failed` (reason `wrong_password`, `pending` or `disabled`). An unknown address writes no row. The writes are asynchronous, so the login response does not wait on `users.db`. Login rows are pruned after 90 days. **Frontend** - `#/admin?tab=overview|users|audit`: `admin.js` (tab shell), `admin-overview.js` ("Needs attention", Users card, System card from the existing health, MQTT and observer endpoints), `admin-audit.js` (filters in the URL, "Load more"). `admin-users.js` becomes the Users tab; the old `#/admin/users` link rewrites to it. - `/api/healthz` is read on open and on Refresh only, not on the 60-second timer, because it walks every packet under a read lock. Spec: [`docs/specs/2026-10-07-admin-dashboard-design.md`](https://github.com/efiten/CoreScope/blob/feat/admin-dashboard/docs/specs/2026-10-07-admin-dashboard-design.md). ## Verification - `internal/users` and `cmd/server`: `go vet` and `go test` pass locally, 26 new Go tests. One upstream test, `TestSaveGeoFilterPreservesFileMode`, also fails on Windows on plain `master` (file modes) and is unrelated. - `sh test-all.sh` exits 0; the XSS gate in diff mode passes. - User-management E2E with the `e2etest` build: 13 of 13 steps locally. - Running on our staging and production instance since 7 October 2026. ## Not in this PR - Restricting existing pages (perf, MQTT status) to logged-in users or admins. The admin area adds no access mechanism of its own, so that stays a later router rule plus endpoint check. - Server-enforced customizer tab restrictions (#1508). - The 24-hour, 5-attempt and 10-minute attention thresholds are constants for now (AGENTS.md rule 8: customizer later). --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
efitenandClaude Opus 5.5
|
a4445b4985 |
ci: run E2E, Go tests and the image build side by side (#2135)
Closes #2134. ## What changes The test jobs and the image build now run side by side. Only the GHCR push waits for all of them. ``` changes ─┬─ go-test ─────────────────────────┐ ├─ race-test (ingestor changes only) │ ├─ e2e-shard ×3 ── e2e-test (gate) ───┼─ build-and-publish → deploy, publish └─ image-check (two-arch build) ──────┘ ``` - **`e2e-shard` (new, matrix of 3).** It needs only `changes`. The suite list stays in the workflow: each line now reads `suite <shard> <file> [VAR=value ...]`, with the same environment variables as before. - Shards are balanced on the measured suite times, about 6 minutes each. - A shard number outside 1..3 fails the step, so a typo cannot drop a suite silently. - `test-issue-1648-m4-icons-e2e.js` stays in the same shard as, and after, the a11y suite. Its distance-tab check only counts once the lazy distance index (#1011) has been built, and the a11y suite is what first requests it. - **`e2e-test`** is now a gate under the old check name. It runs only if all three shards passed, checks that three results arrived, and builds the same `e2e-badges` artifact as before. - **`image-check` (new).** It runs the two-arch build (no push) and the arm64 QEMU smoke on every event, beside the tests. It also computes the build metadata once, which the push reuses. - **`build-and-publish`** now needs `go-test`, `e2e-test` and `image-check`. On a PR every step is skipped. On push and tag refs it only pushes, with the same build args as `image-check`, so every layer should be a cache hit. - **The unused `docker compose build` of the staging image is gone.** `docker compose config --quiet` still validates the file. - `AXE_SCREENSHOT_DIR` now actually reaches the a11y suite. - `tests/unit/test-issue-1956-release-routing.js` follows the new jobs, and checks more than before: - a failure in `go-test`, `e2e-shard`, `e2e-test` or `image-check` skips `build-and-publish`; - the only non-publishing build lives in `image-check` and runs on every event; - the push takes its build args from `image-check`. ## Measurements The old numbers are from upstream; the new ones are from PR runs on my fork (a branch on top of this commit, plus one temporary commit that also triggers the workflow for PRs into a test base branch). | | Before (PR run 37766553643) | After (fork runs 37812660655 / 37814554422) | |---|---|---| | Go Build & Test | 4.9 min | 7.2 / 7.5 min, in parallel | | E2E | 18.4 min | 3 shards of 7.1–9.0 min, in parallel | | Image build | 6.9 min, after E2E | 5.9 / 5.4 min, in parallel | | **Total** | **30.5 min** | **9.6 / 10.6 min** | **Same tests:** I compared the second fork run with the E2E log of the upstream run, suite by suite. All 118 suites ran, and each one passed the same number of checks (1161 in total). **Cost:** more runner minutes, because each shard repeats about 2.5 minutes of setup. Standard runners are free for a public repository. ## Not verified here - **A master push.** On a fork the GHCR push cannot run. The expectation is that `Build and push to GHCR` gets cache hits from `image-check` and finishes in a minute or two instead of 4–4.5. The first master run after merge will show whether it does. - **Branch protection.** If it requires other job names than `🎭 Playwright E2E Tests` and `🏗️ Build & Publish Docker Image`, those may need adding. I cannot read the protection settings, so I could not check. ## Tests run locally - `sh test-all.sh` passes (on Windows with `PYTHONUTF8=1`). - `test-issue-1956-release-routing.js` passes, and fails as expected when `go-test` is removed from the `needs` of `build-and-publish`. - actionlint 1.7.7 reports only the existing self-hosted runner label. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
nullrouten0andClaude Mythos 5.1
|
523a5edfa1 |
fix(ui): escape the route string on the unknown-route page (#2124)
`app.js` wrote the URL fragment into `innerHTML` as the heading of the "Page not yet implemented" view for routes it does not know. Browsers percent-encode `<`, `>` and `"` in fragments, so this is unlikely to be exploitable today, but it is a plain `innerHTML` sink on a URL-controlled string and costs one `escapeHtml()` to close. **Tests:** one test added to `tests/unit/test-xss-escape-sinks.js` in the file's existing style. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Mythos 5.1 <noreply@anthropic.com> |
||
|
efitenandClaude Opus 5.5
|
f3ae8ac25a |
feat: sync a logged-in user's settings across devices (part B) (#2130)
Part B of #2128: a logged-in user's settings follow them across devices. Log in on a phone and your own nodes, favorites, customizer and filters are there; a change on one device reaches the others within a minute or when you return to the tab. **This PR builds on #2129.** Until that one is merged, the diff here includes it. The commits for this part start at `docs(specs): settings sync for optional user management (sub-project B)`. ## The situation Everything a visitor sets up lives in one browser's `localStorage` (about 100 keys in `public/`). A second device or a cleared cache starts from zero (#895). ## What this PR adds **Storage.** `users.db` schema v2: one JSON document per user in `user_settings`, with a revision number and a generation id. A write succeeds only when the client's revision and generation match the stored ones, so two devices cannot overwrite each other silently. **Server.** `GET`, `PUT` and `DELETE /api/account/settings`, behind the same session and CSRF checks as the account routes. - The server owns the list of synced keys (61 keys, [`settings_allowlist.go`](https://github.com/efiten/CoreScope/blob/feat/settings-sync/cmd/server/settings_allowlist.go)) and sends it to the client, so the two cannot drift. - A hard denylist, checked first, refuses `meshcore-api-key`, every `corescope_channel_*` key and `live-channel-colors` (#725). The colour map is keyed by channel hash, and for a user-added channel that hash is `user:<name>`, which would expose hashtag channel names. - Documents are capped at 256 KiB, measured like `JSON.stringify`. PUT is limited to 60 requests per hour per user. A stale revision gets 409 with the current document. **Client** ([`settings-sync.js`](https://github.com/efiten/CoreScope/blob/feat/settings-sync/public/settings-sync.js)). Inert unless the feature is on and someone is logged in. - It wraps `localStorage.setItem` and `removeItem` for allowlisted keys only and pushes 2 seconds after the last change. - It pulls on login, page load, tab focus and every 60 seconds while the tab is visible. - **Merge:** three-way, against a per-device baseline that belongs to one user and one document generation. Lists (own nodes, favorites, saved filters) merge per item, so an item added anywhere is kept and an item removed on one device does not come back from another. Single values: the profile wins unless only this device changed it. - Remote changes are written without a push, theme and colour-blind preset are re-applied, and the current page re-renders (skipped on account pages and while the geofilter editor is open). **UI.** - Logout asks: keep my settings on this device (default), remove them from this device, or cancel. Channel keys are never removed: no copy exists anywhere else. - The account page gets a "Settings sync" section: last synced time, "Sync now", what is and is not synced, and "Delete synced settings from my account". ## Not synced Layout and device state (panel and column widths, collapsed panels, map positions, geofilter drafts), channel data (#725), the API key, and all `sessionStorage`. The full list is in the [spec](https://github.com/efiten/CoreScope/blob/feat/settings-sync/docs/specs/2026-10-06-user-settings-sync-design.md). ## Performance - One GET per page load, tab focus and minute while visible; one debounced PUT per burst of changes. - The `setItem` wrapper costs one Set lookup per write for non-synced keys. A synced write reads one small revision key, not the stored document. - The server reads or writes one row per request. ## Verification - `internal/users` and `cmd/server`: `go vet` and `go test` pass locally (22 new Go tests), including a test that every allowlisted key still occurs in `public/`, and denylist tests. - `tests/unit/test-settings-sync.js`: 79 passing (vm, real module). The cases cover the merge table, two tabs sharing one storage, stale answers after a push, delete while a push is in flight, and logout while the final push fails. - `sh test-all.sh` exits 0. - `tests/e2e/test-user-management-e2e.js` (10 steps, 4 of them new) passed locally with two browser contexts as two devices: a favorite and the packet time window travel from device 1 to device 2, a removal does not come back, and "remove from this device" clears the synced keys while a channel key stays. - Checked by hand on a staging instance with a desktop and a phone on one account. ## Not in this PR - On a shared browser where the previous user chose "keep", the next user's first login merges those settings into their own account. The user guide says to choose "remove" on shared computers. - Saved filter expressions are synced as typed, including any channel names written in them. The guide says so. - Realtime push between devices; the minute pull is the sync interval. --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
efitenandClaude Opus 5.5
|
d232072f85 |
feat: optional user accounts (part A: foundation) (#2129)
Part A of #2128: optional, off-by-default user accounts. With the feature off nothing changes; with it on, visitors can register and log in, and admins manage users and use the operator actions without the API key. PR #2130 (settings sync) builds on this one. The two are meant to be merged together. ## The situation - Operator actions (geofilter save and prune, backup, perf reset) need the shared `apiKey`. There is no per-person right. - Nothing in CoreScope knows who a visitor is, so the requests in #2128 that need that (#1835, #2092, #1508, #730) have nothing to build on. ## What this PR adds **Two new Go modules** - `internal/users`: a separate `users.db` (SQLite through `modernc.org/sqlite`) with users, sessions, single-use tokens, an audit log and a mail log. Passwords use argon2id. - `internal/mailer`: a `Mailer` interface with a Brevo client (send, delivery events, webhook parsing) and an in-memory fake for tests. **Server (`cmd/server`)**, active only with `userManagement.enabled` - 24 routes, all documented in OpenAPI under the `users` tag ([`auth_routes.go`](https://github.com/efiten/CoreScope/blob/feat/user-management/cmd/server/auth_routes.go)): - auth: register, activate, login, logout, me, forgot, reset; - account: profile, password, email change with confirmation, sessions, self-delete; - admin: list, detail, disable, enable, delete, role, resend activation, manual activation, mail status refresh; - a Brevo webhook, registered only when `mail.webhookSecret` is set. - `requireAdmin` replaces `requireAPIKey` at the 7 operator call sites: the API key **or** an admin session. With the feature off it is the old API-key gate (`TestRequireAdminWithoutUserManagementIsAPIKeyGate`). - `/api/config/client` gets `userManagement: {enabled: true}` only when the service started; with the feature off the response is byte-identical. **Frontend** - `auth.js` (header account control, request helper that adds the CSRF header), `account.js` (login, register, activate, forgot, reset, confirm email, my account), `admin-users.js` (`#/admin/users`, deep-linked filters), `account.css` (theme tokens only). - On phones the top-bar control is hidden, so a conditional entry goes into the bottom-nav "More" sheet and the nav drawer. - The customizer geofilter tab and the Perf "Reset stats" button use the admin session when there is one. **Config.** A `userManagement` block (`config.example.json`, [`docs/user-guide/accounts.md`](https://github.com/efiten/CoreScope/blob/feat/user-management/docs/user-guide/accounts.md)). The Brevo key can come from `CORESCOPE_BREVO_API_KEY`. The server refuses to start when the block is enabled but incomplete. ## Security choices - Session cookie `cs_session`: HttpOnly, SameSite=Lax, Secure when `publicBaseUrl` is https. Every cookie-authenticated state change needs the `X-CS-CSRF` header and a matching Origin. - Activation needs the token **and** the account password. Without the password, an attacker who keeps re-registering a known address could get the owner to activate an account that carries the attacker's password. - Register, forgot and email change answer identically for known and unknown addresses. A password reset ends all sessions, a password change ends all other sessions, and both end outstanding email-change links. - Rate limits: login 10 per 15 minutes, register and forgot 5 per hour, per IP and per address. The bucket count is capped. `trustedProxies` makes the per-IP limits see real client IPs behind a proxy. - Server logs carry `#<user id>`, never addresses, tokens or passwords; mail-provider error texts are redacted before logging. ## Performance No change to an existing hot path with the feature off. With it on: - One `users.db` lookup per authenticated request (session by token hash). - The admin user table rebuilds its `tbody` on each filter change. `users.List` caps the result at 1000 rows (`internal/users/users.go`), which bounds the rebuild. - `map[string]interface{}` in `openapi.go`: 79 before, 78 after. ## Verification - `internal/users`, `internal/mailer` and `cmd/server`: `go vet` and `go test -race` pass locally. 121 new Go tests. - `cmd/server` with `-tags e2etest`: vet and the e2e hook tests pass. - `sh test-all.sh` exits 0. `tests/unit/test-user-management-ui.js`: 67 passing (vm, real modules). - `tests/e2e/test-user-management-e2e.js` (6 steps) passed locally against an `e2etest` build with the fake mailer and against a feature-off build. CI builds the `e2etest` binary and runs the suite on a second server (`deploy.yml`). - On a staging instance with a real Brevo key: register, activation mail delivered, activate, admin table, "Refresh status" showing sent, deferred, delivered, opened and clicked. ## Not in this PR - Settings sync (#2130), the admin dashboard, approval flows and notifications (parts B to E of #2128). - A `requireReadAuth` mode (#1835). Sessions from this PR are what such a mode would accept. - Binary size and build time with `modernc.org/sqlite` linked next to `mattn/go-sqlite3` were not measured. Their driver names do not collide. #1992 discusses the driver choice. - No Brevo webhook was configured on staging; delivery status there came from "Refresh status". --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Sylvain Rabot
|
b5b230e884 |
feat(channels): show the sender's path hash size on each message (#2089)
Each channel message now shows the hash size its sender's path uses, read from bits 7-6 of the path byte that the originator writes and repeaters keep. The server sends it as path_hash_size (packetpath.HashSize); the frontend helper pathHashSize() applies the same rule and returns 0 for unknown. One rule on both pages: the packet detail Hash Size row and the hex breakdown call pathHashSize() too, so a 0-hop flood message reports the same size on Channels and on its packet page. A direct packet with no hops left reports no size, matching cmd/server/decoder.go. The cases live in test-fixtures/path-hash-size-cases.json, read by the Go and JS tests. |
||
|
efitenandClaude Opus 5.5
|
000d9ab030 |
feat(coverage): RF noise-floor layer on the Mobile RX coverage page (#2113)
The ingestor already stores the noise floor that CoreDrive RX companions
report with each GPS fix (`client_rf_samples`, opt-in through
`clientRfSamples`, `cmd/ingestor/client_rf_sample.go:62`), but nothing
reads it back. An operator who enables it collects the data and cannot
see it. This adds the read side: `GET /api/rf-noise` and a Signal/Noise
toggle on the Mobile RX coverage page.
## What it does
- **`GET /api/rf-noise?bbox=&z=&days=`** returns a GeoJSON hex grid with
the median, quietest and noisiest noise floor per cell, in the same
shape as `/api/rx-coverage`. It is registered always and 404s unless
`clientRfSamples.enabled` is true, the same pattern as the coverage
routes.
- **Stationary samples are excluded.** A parked companion logs hundreds
of readings at one point, which would otherwise define its cell.
- **Coverage page:** a Signal/Noise toggle, rendered only when
`/api/config/client` reports `clientRfSamples: true`. The noise layer
reuses the coverage colour tokens with the axis inverted, because a
lower dBm is quieter. Tiers are at -115 and -108 dBm.
- **Deep link:** `#/rx-coverage?layer=noise` opens on the noise layer.
- **Empty and failed answers** are labelled on the map ("No RF samples
in this view yet", or a retry hint), so a blank map never reads as
"feature off".
No new configuration key: it reads the existing `clientRfSamples`
section that the ingestor already uses. Default off, so nothing changes
for an instance that has not opted in.
## Where it comes from
Ported from the efiten/CoreScope fork, where it has run on
analyzer.on8ar.eu since September (fork commits `42d09f0e`, `80581c43`,
`cde95078`). The cherry-picks conflicted with upstream's newer
`routes.go`, `types.go` and `rx-coverage.js`, so the final state was
ported by hand. The fork-only `/scopes` route that sat next to it in the
same hunk is deliberately left out.
## Performance
The query is bounded by `sampled_at` (indexed, `idx_crf_prune`) and the
bbox, aggregation is one pass plus a per-cell sort, and the response is
capped at 5000 cells. Measured on analyzer.on8ar.eu, all of Belgium
(`bbox=49.4,2.4,51.6,6.5&z=9`), from a client in Belgium, so network
time included:
| window | samples | cells | response time |
|---|---|---|---|
| 7 days | 10,835 | 241 | 0.37 s |
| 30 days | 43,303 | 429 | 0.39 s |
That table holds 45,270 rows in total. It is only read when someone
opens the noise layer, never on ingest or WebSocket paths.
## Tests
- `cmd/server/rf_noise_test.go`: 8 tests for the aggregation (median and
extremes, stationary exclusion, the cell cap, empty input) and the gate
(404 when off, even with data present). All pass, and the full
`cmd/server` suite passes locally.
- `tests/unit/test-rx-coverage-noise.js` (new, in `test-all.sh`): the
colour axis runs the right way, including both tier bounds and a string
median from the API. Slices the real function out of `rx-coverage.js`
and fails on master's copy, which has no thresholds.
- `tests/e2e/test-rx-coverage-noise-e2e.js` (new, wired in `deploy.yml`
and `scripts/non-unit-tests.json`): no toggle when the flag is off;
Noise fetches `/api/rf-noise`, draws the cells, swaps legend and
subtitle, puts `layer=noise` in the hash; a deep link opens on the noise
layer and an empty answer shows the message. 3/3 locally; against
master's `rx-coverage.js` and `roles.js`, 1/3 (only the "flag off" case
passes, as it should).
- `test-rx-coverage-viewport-e2e.js` still passes. ESLint 8 reports 0
errors on the changed files.
## Not done
- The thresholds (-115 / -108 dBm) are fitted to the fork's own data
(1,241 moving samples at the time) and are constants in
`rx-coverage.js`. Per AGENTS.md rule 8 they belong in the customizer
eventually; not in this PR.
- The E2E suite mocks the API. The real endpoint is covered by the Go
tests and by the measurements above, not by a fixture with seeded
samples.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
efitenandClaude Opus 5.5
|
ef12535364 |
test(map): wait for the new route sidebar before measuring it (#2109)
Step (9) of `tests/e2e/test-path-inspector-e2e.js` fails intermittently on master with `(3-9) fixture route coverage: 0 !== 700`: 3 of the 18 runs that executed it since 2026-09-30, including master runs [37120155356](https://github.com/Kpa-clawbot/CoreScope/actions/runs/37120155356) and [37156362836](https://github.com/Kpa-clawbot/CoreScope/actions/runs/37156362836). A red master run skips the image build, so `:edge` is not rebuilt. ## Cause The step clicks a candidate and then waits for any `.mc-rt-sidebar`. The previous route's sidebar survives the hash-only `goto` to `#/map`, and `drawPacketRoute` (`public/map.js:1080`) replaces it only after `/api/resolve-hops` answers. So the wait matched the old sidebar, and the checks that follow measured either the old sidebar or, if the replacement landed between the locator resolving and the evaluation, a detached element, whose `getBoundingClientRect()` width is 0. This is a test defect. The app keeps the previous route on screen while the next one loads, which is intended. ## Fix Remember the sidebar before the click and wait until a different one is in the DOM. Test-only, one file. ## Evidence Locally against the fixture, with `/api/resolve-hops` delayed through `page.route` in a throwaway copy of the test, measuring which sidebar the step-9 assertion sees: | Delay | Before | After | |---|---|---| | 0 ms | new sidebar, pass | new sidebar, pass | | 80 to 100 ms | old sidebar, `isConnected: false`, width 0 (3 of 5 runs) | new sidebar, pass (5 of 5) | | 1500 ms | old sidebar, pass: the step checked the wrong page | new sidebar, pass | The unmodified fixed test then passed 3 of 3 runs. ## Not done - I did not look for the same wait pattern in other suites. - The delay probe is not committed. It needs a timing window to be useful, and that window would be a fixed sleep in CI. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
n30nex
|
9e49db9725 |
feat(nodes): group adverts by observed route evidence (#2085)
Red commit: `ac60248` ([two intended label failures](https://github.com/Kpa-clawbot/CoreScope/actions/runs/37137429333)). Original grouping red: `767da9b` ([CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/36353796690)). Fixes #2073. #2088 is merged. Rebased onto master `415362c`; all eight patches are unchanged, including the test-first history. Recent Adverts groups the bounded sample by authoritative `advert_kind`: Flood, Mixed flood / direct (empty path), Direct (empty path), and Other / unknown. Missing evidence stays unknown. Each advert appears once, preserving reception metadata, order, packet links, counts and origin/history explanations. Wording describes observed remaining paths without inferring original send mode or RF distance. Validation at `6317baf`: - All 185 standalone suites passed; syntax, XSS and lint passed (zero errors, 92 existing warnings). - Browser verified: `http://127.0.0.1:51827` (temporary Go fixture server, now stopped), desktop pane/full view and mobile. Mixed evidence, exact links and caveats checked again after pushing. - The expanded fixture exposes the existing #2104 failure in the unmodified core harness. Supplemental integration with #2105's exact test correction passes 135 checks, with three existing skips. Both PRs remain separate. - [Fresh green CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/37138366726) passed; three independent automated reviews found no must-fix issues. E2E assertion added: `tests/e2e/test-e2e-playwright.js:142`. Grouping is O(n) over the existing 20-advert sample. No new requests, settings, colors or dependencies. ## Preflight overrides External OpenClaw runner/profile unavailable; repository checks and actual local Chromium were used. |
||
|
n30nex
|
99a326a6b3 |
test(packets): verify observation hex after rerenders (#2105)
Red commit: `ca553e4` ([expected assertion failure](https://github.com/Kpa-clawbot/CoreScope/actions/runs/37137426691)). Fixes #2104. The observation hex test now opens the existing grouped fixture directly, selects A → B → A with stable-ID locators, and asserts the selected ID and exact raw bytes after each render. Missing fixture data fails explicitly. Distinct observation frames are seeded after schema migration; this test's fixed sleeps and silent skips are removed. Rebased onto master `415362c` after #2090 merged. Both patches are unchanged. The previous #1122 layout blocker now passes against the actual current server and assets. Validation at `5db8017`: - All 185 standalone frontend suites passed. - Core browser: 133 passed, three existing skips; #1122 layout 6/6, filter 11/11, grouped-collapse 4/4. - Syntax, YAML, fixture SQL, XSS and lint passed (zero errors, 92 existing warnings). - Fresh red CI reached the intended detached-observation assertion. [Green CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/37138187496) passed; three independent automated reviews found no must-fix issues. Only test code and fixture setup change; no configuration/customizer implications. ## Preflight overrides External OpenClaw runner/profile unavailable; repository checks and actual local Chromium were used. |
||
|
efitenandClaude Opus 5.5
|
415362c5af |
fix(packets): resolve path hops with the observer that heard them (#2097) (#2099)
Fixes #2097. Every 1-byte path prefix on this network is shared. Measured on the live deployment: 254 prefixes cover all 2043 nodes, nine of them on a single prefix, and not one node has a prefix to itself. The packets page printed one of those nine as a certainty. It could already have said otherwise. `hop-resolver.js` marks such a hop `ambiguous` and returns its full candidate list; `hop-display.js` renders a warning badge with a count and a candidate popover; `renderHop(h, observerId)` reads a per-observer cache key. None of it fired, because `HopResolver.resolve()` takes six parameters and `packets.js` passed one. Without `observerId`, `packetIata` is null, `nodeInRegion()` never runs, no candidate is flagged `regional`, `globalFallback` stays false, and `hop-display.js` computes a badge count of 0. The whole chain stayed silent while the data said it was a guess. ## What this changes **Resolution carries the observer.** `resolveHops()` takes it and writes the per-observer cache key that `renderHop()` has always read and nothing ever wrote. `resolveHopsForPackets()` groups a multi-packet call by observer so each group is filtered on its own. One site was the bug in miniature: it wrote its result under `hopNameCache[k + ':' + pkt.observer_id]` after resolving without that observer, so the key promised something the value was not. **The observer's position is the anchor.** `resolve()` has always accepted `observerLat`/`observerLon` as the anchor at the receiving end: when nothing later in the path is resolved, the observer is the next known position, which is what `pickByAffinity` needs. `observerPosition()` is the single lookup, used by both resolve paths. This matters more than the IATA route, which turns out to be dead on this network. `nodeInRegion()` looks the observer's code up in `/api/iata-coords`; **0 of 42 observers have their code in that table** (`ANR BRU GNE HEP KJK LGG MST NRW OBL OST` are all missing, while its 54 entries are `AMS`, `APC`, `ATL` and the like). So the 300 km filter has never excluded anything here. 27 of 42 observers report lat/lon directly, and that works. Filed separately as #2098, since it is a network-wide lookup that resolves nothing and hop resolution may not be its only consumer. **Each badge belongs to its pill.** The badge is a sibling after the pill, so a path read `A [8] → B [6] → C` with nothing to say which name the 8 qualified. Pill and badge now render inside one `white-space: nowrap` `.hop-group`, leaving the separator outside the pair. Only a hop that carries a badge is wrapped; wrapping every hop would change the layout of every path for nothing. That markup predates this work, but these commits are what made it visible — before them the badge essentially never appeared on this page. **The list summarises, the detail pane does not.** Every 1-byte hop getting its own badge turned a table row into a line of warning triangles. `renderPath()` takes `{ summary: true }` for the three list call sites: names with no per-hop badge, and one indicator reading "N of M hops have more than one candidate". The two detail-pane sites are unchanged and keep a badge per hop, because that is where the candidates actually get read. `HopDisplay` gains `opts.badge === false` for it, and the hop keeps its `hop-ambiguous` class either way so CSS can still mark it. ## Scope Deliberately not included: chaining the next resolved hop to narrow a candidate set to one. `pickByAffinity` already scores on graph edges and would do it; it needs neighbouring context this call does not yet supply, and that belongs in its own change to `hop-resolver.js`. What is here makes the uncertainty visible and picks a defensible candidate rather than the first in index order. ## Verification 30 unit tests in `tests/unit/test-issue-2097-hop-ambiguity-badge.js`, registered in `test-all.sh`. Full frontend suite exits 0. Four of them are structural guards over `packets.js`, and each was mutation-checked by reverting the change it protects: - no `HopResolver.resolve()` call passes the hops alone - no call passes an observer id and drops its position - one helper does the observer lookup - the detail pane calls `renderPath` without `summary` The behavioural test for the anchor runs with `iataCoords` deliberately empty, which is the live state, and lists the distant candidate first: without an anchor the resolver keeps candidate order and picks it, with one it does not, and the hop stays reported as ambiguous with all candidates listed. **Browser validated** on a staging deployment of this branch, against live data: | | list | detail pane | |---|---|---| | per-hop badges | 0 | 24 | | path indicators | 1 | 0 | | badges outside a `.hop-group` | — | 0 | On the packet that prompted the issue, the first hop now resolves to a repeater 10.6 km from the transmitter instead of one 126 km away, and the second hop resolves to the only candidate that has a recorded neighbour edge matching the next hop in the path. ## Note for reviewers The last commit exists because staging disagreed with the tests. After the anchor landed, the detail pane still picked the distant candidate: it resolves through its own `HopResolver.resolve` call, which had the observer id but a null position. The guard that now fails on exactly that shape is what would have caught it before the deploy. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
efitenandClaude Opus 5.5
|
8933e2c703 |
fix(channels): keep client-only state across a channel-list refresh (#2095) (#2096)
Fixes #2095. `loadChannels()` replaced the `channels` array with the server snapshot and carried nothing across, and `mergeUserChannels()` only ever ran from `init()`. So any refresh destroyed every field that exists only in the tab. **No race needed for the worst of it.** Change the region filter or toggle show-encrypted with a PSK channel open, and: - the **My Channels** section disappears — `renderChannelList()` derives it as `channels.filter(c => c.userAdded === true)`, and the server returns neither `userAdded` nor a `user:`-prefixed hash - every unread badge resets to 0 - the user's own labels vanish - `reconcileSelectionAfterChannelRefresh()` cannot find the `user:*` hash in the snapshot, so it nulls `selectedHash`, sets `messages = []`, rewrites the URL to `#/channels` and replaces the open conversation with "Choose a channel from the sidebar" ## What this does `mergeClientChannelState(fresh, prev)` mirrors `mergeWsAppendedIntoRest()`, which already does exactly this job for `messages` (#1498). Same shape: pure, takes both arrays as parameters, returns a fresh array, never aliases or mutates an input. It carries `unread`, `userAdded` and `userLabel` by hash, and keeps `lastActivityMs` / `lastSender` / `lastMessage` / `messageCount` when a WebSocket batch landed while the request was in flight and is therefore newer than the snapshot. Those four move together: a sender without its message reads as a different message. `mergeUserChannels()` now runs inside `loadChannels()`, before the render and before the reconcile, so all three call sites get it instead of `init()` alone. ## What it deliberately does not do **It does not resurrect a channel the snapshot left out.** A region-filter change legitimately narrows the list, so carrying survivors over would defeat the filter, which is a worse bug than the one being fixed. The helper only enriches rows already present in the fresh snapshot. That leaves half of finding 2 in the issue unfixed: a channel pushed by the WebSocket handler during the initial in-flight window is still dropped. That one self-heals on the channel's next packet, and the reverted preview line is overwritten by the next WS batch. Fixing it properly needs a way to tell "dropped because the snapshot is stale" from "dropped because the filter excludes it", which is a larger change than this. ## Tests `tests/unit/test-issue-2095-channels-client-state.js`, 11 cases, registered in `test-all.sh`. Part 1 exercises the helper directly. Part 2 drives the real `loadChannels()` through the existing `_channelsLoadChannelsForTest` hook with a stubbed `api()`, which is what proves the helper is wired in rather than merely defined. **Verified by mutation.** With the wiring removed from `loadChannels()` but the helper left in place, the three reproduction cases fail with exactly the reported symptoms: ``` FAIL a refresh keeps the My Channels rows My Channels lost on refresh (got ["public1"]) FAIL a refresh keeps unread badges the unread badge reset to 0 on refresh undefined !== 7 FAIL a refresh does not close an open PSK conversation the open PSK channel was deselected + null - 'user:MyPSK' ``` The fourth case, "a refresh still drops a channel the server filtered out", stays green throughout, so the fix cannot be defeating the region filter. Full frontend suite green: `sh test-all.sh` exits 0. ## Not done **No browser validation.** This is frontend JS covered by unit tests that call the production function through its own hook, but I did not run Playwright against a server, so I am not claiming a browser check I did not do. `init()` still calls `mergeUserChannels()` and `renderChannelList()` after `loadChannels()` resolves, which is now redundant and costs one extra full sidebar render per page load. Left alone deliberately: it is idempotent, and the comment there records the regression it was added to fix (`test-channel-issue-1111-e2e.js`, case 2). Worth removing separately with that e2e test watched. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
n30nex
|
3b415fb007 |
fix(live): enable node filtering after its handlers are ready (#2103)
Red commit: `8974616` ([CI assertion failures](https://github.com/Kpa-clawbot/CoreScope/actions/runs/37064461392)). Fixes #2094. The Live node-filter input now starts natively disabled and becomes editable after its input, keyboard and blur handlers attach. Previously it accepted text while initial node loading was pending, then never processed that text without another keystroke. The change adds one HTML attribute and one initialization assignment. Failed initial loading still reaches the existing error handler and installs the controls. Saved filters, URL precedence, autocomplete and keyboard selection keep their existing behavior. No new settings, styling, dependencies, requests or packet-processing work. ## Validation - The existing CI-wired Live browser suite delays or aborts the real initial node request. It verifies rejected premature typing, later editability, actual suggestions, keyboard selection and URL state. Both new cases failed on the red commit; all six cases pass locally on `11fab46`. - All 183 standalone frontend suites passed, plus lint and XSS gates. Separate browser checks passed saved-filter restoration and URL override; desktop/mobile loading/ready screenshots were inspected. - A broader local core run exposed the unchanged stale observation-handle test tracked in #2104. It is outside this Live fix. Full [CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/37067337557) passed at `11fab46`, including Go, browsers and both container architectures; three independent reviews are clear. E2E assertion added: `tests/e2e/test-1110-live-filter.js:69`. Preflight overrides: external script/OpenClaw profile unavailable; repository checks and local Chromium used. |
||
|
n30nex
|
b9fb3d244c |
feat(analytics): split adverts using recorded route evidence (#2088)
Red commits: `dc7dcbe`, `76e3217` ([assertion failures](https://github.com/Kpa-clawbot/CoreScope/actions/runs/37059977385)). Fixes #2041; supplies #2085's backend contract. Records at most two route bits per retained advert. Node flood counts now use the same evidence, including mixed and transport-flood adverts. Analytics failures log/count errors while core observation, path and liveness writes continue. The unrelated recent-advert limit clamp is removed. Direct adverts (empty path) describes the observed remaining path across valid hash widths. Firmware can forward an advert down to an empty path; this cannot establish original send mode or RF distance. The `zero_hop` API spelling remains compatible. Successful post-upgrade evidence writes survive observation replacement and restart. Older overwritten frames and failed writes can leave gaps. The UI explains automatic background backfill. ## Measured validation Synthetic Windows workloads; timings are not production guarantees: - 300K retained adverts, one-hour window: median 64.35→39.84 ms; 13.98 MB→145 KB allocated. Seven-day allocations unchanged; five paired samples. - 1M transmissions/11M observations: backfill 5m43s. Simple 10 Hz writer: all 3,423 observations persisted; p99 28 ms, maximum 323 ms. - Simulated pre-backfill cursor: retained evidence ready 4m21s; full catch-up 6m41s, 122 cache invalidations. Normal completed-migration startup bulk-loads current evidence. Portable opt-in scale tests and window benchmarks are included. All 183 frontend suites, full server tests, targeted races, lint and real-API desktop/mobile checks passed. Full [Linux CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/37063210229) passed at `36def9c`; three independent reviews are clear. Local Windows testing hit the unchanged symlink-privilege test. No configuration changes. Preflight overrides: external script/OpenClaw profile unavailable; repository checks and Chromium used. |
||
|
Sylvain Rabot
|
a4a447acea |
fix(packets): pack short columns, one expand arrow, Full Names toggle (#2090)
The packets table wastes most of its width on short columns, draws two arrows on grouped rows, and cuts every observer and hop name short. This fixes all three and adds a way to read the full path when it does not fit. ## Why the columns were wide `makeColumnsResizable` measured each column once on load and stored the widths as **percentages** of the table. Percentages grow with the screen: on a 2000px display "17s ago" sat in a 180px column. The measurement also read the virtual-scroll spacer row — a single `colspan` cell as wide as the table — as column 0, so the expand column came out 160px wide locally and about 950px on a live instance with grouped rows. Measured on the fixture at 2000px: | Column | Before | After | |---|---|---| | Expand | 162–230px | 32px | | Time | 111px | 67px | | Size | 64px | 46px | | HB | 44px | 29px | | Scope | 80px | 48px | | Path / Details | ~180px each | ~650px each | ## The change **Column packing.** A new `fitColumnsToContent` (`app.js`) sizes every column except Path and Details to its widest cell, in px; those two split what is left. Every visible column gets an explicit width and the table gets their sum: a spacer row keeps a slot for each hidden column, and leaving any column `auto` hands those slots a share of the slack as a blank strip at the right edge. When space is short (detail panel open, narrow window) the widest fixed columns give way first, then Path and Details shrink to 60px, and only then does the wrapper scroll. The split itself is a pure function, `distributeColumnWidths`, with unit tests. It re-lays out when the container resizes (detail panel), when TableResponsive hides or reveals columns (a new `table-columns-changed` event), and after sorting or a column toggle. After each render `grow()` widens columns for the rows just inserted and never narrows them, so nothing jitters while scrolling. Drag handles store px; double-clicking one resets it. Path and Details have no handle. `makeColumnsResizable`, still used by the nodes, observers and analytics tables, now skips `colspan` rows when measuring; other rows are measured as before. **One expand arrow.** A CSS `::before` "▶" sat next to the SVG caret, which pointed up when collapsed. The glyph is gone and the caret points right, then down. **Full Names.** A toolbar toggle next to Hex Paths shows observer and hop names untruncated. It is saved per browser; a link can carry `fullNames=1`, which applies to that page without overwriting the visitor's saved preference. Paths stay on one line: the virtual scroller positions rows at one fixed height, so wrapping would make scrolling jump. **The "+N" path pill, and a full-path popover.** Hops past the column edge were meant to go behind a "+N" pill (#1124), but the pill was appended after the overflowing chips and so sat past the clipped edge: it never showed. It is now sticky at the right edge, opaque on hover, and counts hop chips only (it used to count warning icons too). Hovering or focusing it shows the full path, one hop per line and numbered; click, Enter or Space pins it until an outside click or Escape, and it closes when the page changes or its row scrolls away. Clicking the pill no longer selects the row underneath. **Two related fixes.** - Rows drawn before `/api/observers` resolves show raw 64-char pubkeys. `loadObservers` now re-renders and re-measures, instead of leaving Observer at the pubkey's width (~600px under Full Names). - Group child rows indented every cell by 20px, which misaligned them with the headers and widened every column on expand. Only the Time cell is indented now. Anyone who had dragged packet columns gets the new defaults once: the old percentage key is dropped. ## Performance `grow()` runs after every `renderVisibleRows`, so it is on the hot path. Measured in Chromium on the fixture, 300 iterations, three runs: | Case | Cost | |---|---| | Full render, 63 rows measured | +0.88–0.93ms over the forced reflow alone | | Scroll step (incremental render) | ~73 `Range` rect reads, only the inserted rows | | Large jump / re-render | ~569 rect reads | Before narrowing `grow()` to inserted rows, one scroll step with 92 rendered rows made 5,472 rect reads. The DOM row count is bounded by the virtual-scroll window, so the cost does not grow with the packet count. `distributeColumnWidths` is a binary search over at most ~10 columns. ## Tests - `tests/e2e/test-packets-compact-columns-e2e.js`, 16 cases: short columns stay under fixed px bounds at 1920px; Path and Details take the freed width; hiding both still fills the wrapper; no blank strip and no horizontal scroll, with and without the detail panel, at 1920, 1100, 1000 and 375px; responsive reveal and re-hide refit; stale percentage widths dropped; drag follows the pointer and double-click resets; expanding a group leaves widths alone; Full Names untruncates, keeps paths on one line, persists and round-trips through the URL; the Observer column shrinks back when observer names arrive late; the pill popover lists every hop vertically, closes when the pointer leaves, pins on click without selecting the row, closes on Escape and on navigation, and opens on keyboard focus. Registered in `scripts/non-unit-tests.json` and `deploy.yml`. - On `master` the suite fails where it should, and restoring each fix's old behaviour (child-row indent, the late-observer refit) turns its case red. - `tests/unit/test-packets.js`: `distributeColumnWidths` (room to spare, rounding, widest-first capping, floors, dragged columns, the 60px minimum, overflow), the single chevron and no `::before` glyph, Full Names on flat and grouped observer cells, and the Full Names CSS. - `tests/unit/test-xss-escape-sinks.js` now builds the real `obsCellName` from source: escaping moved into it. Removing its `escapeHtml` turns both observer-cell tests red. - `test-issue-1189-composed-cell.js` and `test-issue-2012-clear-filters-selection.js` evaluate packets.js fragments with injected helpers and needed `obsCellName` / `showFullNames` added. - Existing packets E2E suites pass on this branch and on master against a CI-prepared fixture: scope column, #1128 (layout and multi-viewport), #1122, #1188, #1648, table sort, #1692, #1486. - Unit: 182 of 183 suites pass locally; `test-issue-1956-release-routing.js` could not create a temp directory in my sandbox and is unrelated. ## Browser validation Checked in Chromium against the fixture at 2000, 1920, 1280 (panel open and closed), 1100 and 375px, and with the seeded grouped row expanded and collapsed. Screenshots taken at each step; the table fills its container exactly, grouped rows show one arrow, and the pill popover lists full names. ## Not done, deliberately - **No width cap on Observer under Full Names.** A cap brings back the truncation the toggle removes, and Observer is already the first column to give way when space is short. - **Customizer.** The floors (60px flex minimum, 64px shrink floor, 32px expand, 58px Rpt) are hardcoded, per AGENTS rule 8; exposing them in the customizer can follow. - **Wrapping paths.** Rejected because of the fixed row height above; the pill popover is the way to read a long path. |
||
|
n30nex
|
093e320c2b |
fix(map): keep Path Inspector candidates accessible and cover route replacement (#2082)
## Fix Fixes #2060. Fixes #2081. Path Inspector route drawing and candidate replacement now have deterministic browser coverage. An idempotent SQL seed adds four synthetic repeaters and four edges after fixture migration, without changing packet ordering. The test uses the real API, normal clicks, visible Leaflet geometry, and removal of every previous route object. Restoring coverage exposed a desktop layout bug: drawing the first route moved the map over the next candidate button. The route sidebar now participates in the existing flex layout, including resizing and collapse, while preserving the mobile bottom sheet. The historical zero-candidate result did not reproduce on the current baseline. No search thresholds or API behavior changed. No new dependencies or customizer settings. ## Validation - Local assertion-red history: `042db2b` requires seeded candidates; `ca1d22a` exposes the blocked second click. Subsequent commits repair layout and narrow-window behavior. - All 183 standalone suites passed. Unchanged-base Go failures: #2083 readiness race (fixed in #2084) and Windows symlink privilege. - Real Chromium: 11 Path Inspector/layout checks, 33 related map checks, and core E2E (131 passed, 3 existing fixture skips). - CSS-variable checks, XSS diff check, inventory, whitespace and PII checks passed. Seed remains valid after periodic graph refresh. Local browser validation used Chromium with a 60-second navigation budget; the OpenClaw profile and external preflight script were unavailable. Repository checks ran directly. [CI run 36343343569](https://github.com/Kpa-clawbot/CoreScope/actions/runs/36343343569) records the initial assertion-red commit. Final CI must pass before merge. |
||
|
n30nex
|
ded27a8312 |
fix(packets): stabilize fixture tests and preserve mobile empty states (#2080)
Fixes #2054 and #2079. Packet-row tests could outlive the fixture's default 15-minute window. An audit of 115 wired suites identified four needing explicit fixture windows: 180 minutes on mobile, 1440 on desktop. Mobile rejects larger saved windows, so a desktop-only preference was insufficient. Packet-icon checks now require a real path-bearing packet and both detail controls. Default-window coverage exposed #2079: responsive column hiding also hid the spanning empty-message cell. Skip structural single-cell spanning rows when hiding individual columns, preserving status messages and virtual-scroll spacer height. Ordinary and partial-span data cells retain their behavior; application defaults stay unchanged. - Red evidence: `bae5e78` fails the aged-fixture row assertion; `9b75cf6` fails spanning-row visibility. Guards/setup make them pass. - Validation: 183 standalone suites; 82 targeted browser cases on a fixture aged over 30 minutes (one pre-existing node-gesture skip). Broader browser run: 131 passed, 3 fixture-dependent skips. - Browser verified: fixture-backed Go server and Chromium; `coverage/2079-mobile-empty.png`. Real mobile scrolling retains spacer height and has no horizontal overflow. - E2E assertions: `test-observer-iata-1188-e2e.js` default-window cases and `test-table-fluid-e2e.js:58` shared-helper regression, both under `tests/e2e/`. - Three independent reviews completed; packet-icon coverage feedback addressed in `760057d`. - Populated mobile axe coverage remains part of #1995. - One constant-time guard per already-visited row; no new requests or collections. ## Preflight overrides - External `run-all.sh` unavailable; repository checks run directly. Local runs use UTF-8, the release harness's `GITHUB_REF_NAME`, and a 60s navigation budget; repository timeouts are unchanged. |
||
|
n30nex
|
410c82c02c |
fix(ui): use consistent 48px navigation and channel buttons (#2078)
Fixes #2052. Navigation (`.nav-btn`) and channel icon (`.ch-icon-btn`) buttons now use the shared 48px house minimum. Remove competing 44px component/coarse-pointer rules and the legacy 32px mobile rule, and correct comments that confused the house preference with accessibility requirements. At 768px, user-added channel rows already clipped Remove; the larger Share target exposed the same constraint. Allow only those rows' controls to wrap so both actions remain within the sidebar. Navigation height and normal network-channel rows are unchanged. - Red `7787d9b`: rendered target assertions fail at 375/390/768/1280px. Green `e6ed9c8` changes CSS only. - Validation: all 183 standalone suites; 33 computed touch-target checks; 15 actual-page layout/click checks. Broader browser suite: 131 passed, 3 fixture-dependent skips. Three independent reviews found no required changes. - Browser verified: local Chromium and fixture-backed Go server; screenshots `coverage/2052-targets-375.png` and `coverage/2052-targets-768.png`. - E2E assertion added: `tests/e2e/test-channel-fluid-e2e.js:112`, covering dimensions, containment, overflow and normal Filter/Share clicks. Existing CI selection is retained; Chromium-required runs cannot silently skip the touch suite. - Test-only follow-up `5144503` waits for packet readiness before measuring navbar controls. Five fresh mobile contexts passed; size and click assertions remain intact. - No new settings, requests, dependencies or runtime JavaScript. ## Preflight overrides - External `run-all.sh` unavailable; repository checks run directly. - Local unit runner uses UTF-8 and `GITHUB_REF_NAME=local-validation` for the existing release harness. Browser navigation uses a 60s local budget for slow assets; repository assertions/timeouts are unchanged. |
||
|
n30nex
|
f1edbbef3f |
fix(packets): preserve observation selection in detail URLs (#2093)
Red commit: `c165087` ([CI assertion failure](https://github.com/Kpa-clawbot/CoreScope/actions/runs/36644030012)). Fixes #2091. Packet detail links now retain the selected observation when page initialization or filter changes rebuild the URL. Initialization also restores `obs` from the complete hash after the router strips its query. Explicit ID links render the requested observation and retain it in Copy Link state. The shared updater reads selection from the current route, so returning to the list or selecting another packet cannot resurrect an old observation. Existing filter serialization and Clear Filters behavior remain intact. No new requests, configuration, dependencies or layout changes. ## Validation - Real fixture browser checks use nondefault observation 502: hash/ID load, type/observer/time-window changes, refresh, Clear, and another refresh. Both URL and selected row are asserted. - All 183 standalone frontend suites passed; focused filter browser suite 11/11. - Broader local core run reached the unrelated Live input readiness bug #2094, reproduced on unchanged master. The complete CI browser suite passed. - ESLint 8: zero errors; 91 existing warnings. XSS diff, syntax and whitespace checks passed. - Three independent reviews found no production defects; assertions additionally prove filter controls changed before checking selection. E2E assertion added: `tests/e2e/test-filter-ux-e2e.js:179`. OpenClaw profile/external preflight were unavailable; Chromium and repository checks ran directly. Local navigation used a 60-second budget. Final CI passed at `280ced5`, including Go, browsers, coverage and both container architectures ([run](https://github.com/Kpa-clawbot/CoreScope/actions/runs/36646175352)). |
||
|
n30nex
|
dc4db17c48 |
fix(nodes): remove unsupported region fields from Heard By (#2077)
Fixes #2062. The node health API does not emit observer region data, but Heard By rendered a Region column containing only dashes and a Regions summary that could never appear. Remove that column, its sort control, and the unsupported region displays from the full node page and side panel. Keep Observer, Packets, Avg SNR and Avg RSSI, along with existing escaping, signal placeholders, relay counts and badges. No API, dependency or configuration changes. - Red commit `4396492` fails on the unwanted Region header and summary; `a360711` removes the unsupported fields. - Validation: 183 standalone suites and 7 focused browser checks passed. Broader browser run: 131 passed, 3 fixture-dependent skips. Independent reviews completed; the sorting-test finding was addressed in `1452981`. - Unit coverage evaluates the real templates. The existing CI-selected browser suite checks four-column alignment and sorting on desktop/mobile using the actual API field names. - Browser verified: local Chromium against a fixture-backed Go server; screenshots recorded in `coverage/issue-2062-heard-by-1400.png` and `coverage/issue-2062-heard-by-390.png`. - E2E assertion added: `tests/e2e/test-issue-1151-orphan-separators-e2e.js:125`, the two #2062 desktop/mobile cases. - No added requests, loops or data structures. ## Preflight overrides - External `run-all.sh` unavailable; repository syntax, whitespace, CSS-variable, XSS and PII checks run directly. - Local unit execution supplies UTF-8 settings and `GITHUB_REF_NAME=local-validation`, required by the existing release-routing test harness. - Local browser navigation budget raised to 60s for slow local asset responses; repository assertions and timeout settings unchanged. |
||
|
n30nex
|
31744c6da2 |
test(channels): isolate WS assertions from initial loading (#2087)
Fixes #2086. The #1468 WebSocket browser checks could fail when initial channel loading completed between their separate before/after evaluations. An orphan message added no channel, yet unrelated loading changed the count from 0 to 5; the positive control could also lose its sentinel when loading replaced the list. Each check now captures before state, processes its packet, and captures after state in one synchronous browser evaluation. All four original assertions and both message payloads are unchanged. This updates one test file only, with no production, dependency or configuration changes. ## Validation - Delayed real API loading reproduces both failures on unchanged master. - Both corrected checks pass with loading held and with loading completed; each uses one evaluation. - Parent independently ran the delayed-loading harness and full browser suite: 131 passed, three existing fixture skips. - All 183 standalone frontend suites passed. Syntax, inventory, whitespace and privacy checks passed. TDD justification: test synchronization repair only. Existing assertions demonstrate the baseline failure; no production behavior or manufactured failing test was added. Browser checks used Chromium against unchanged production source. OpenClaw profile/external preflight were unavailable; repository checks ran directly, with a 60-second local navigation budget. Final CI must pass before merge. |
||
|
efitenandClaude Opus 5
|
3caa847323 |
fix(nodes): the advert section is called "Recent Adverts", and says why (#2071)
Closes #2042, reported by @damn-simple-scripts. ## Which of the two fixes The report offered widening the section to all packets the node originated, or renaming it, and proposed the rename. Widening sounds like the better fix, so I measured before agreeing. On a production database: | | | |---|---| | transmissions total | 1,251,967 | | with `from_pubkey` populated | 191,237 | | of those, `payload_type = 4` (ADVERT) | **191,237** | | with `from_pubkey` and **not** an advert | **0** | The ingestor fills that column for adverts only. Attributing a relayed CHAN or TXT packet back to its sender is the path-resolution problem, not a filter this section could apply — so "show all packets from this node" is not a small change, it is a different feature resting on attribution the data does not carry. So the rename is correct, and the numbers say so rather than my preference. ## The change Both copies renamed — the full node page and the side pane. Both already read `nodeData.recentAdverts`, so the field feeding them said "adverts" while the heading said "packets". The heading also gained a `title` naming `from_pubkey` as the reason it is adverts only. Renaming without explaining invites the same report from the next reader; the tooltip is where that explanation costs nothing. ## Tests `tests/unit/test-issue-2042-recent-adverts-label.js`, four cases: both copies present and headed "Recent Adverts", no copy back to "Recent Packets", the advert field still feeding them, and the explanation still in place. The third matters most — it ties the label to its data source, so pointing this section at a different field in future fails here rather than silently making the label wrong again. ## Not done, deliberately The reporter's follow-up idea, splitting flood adverts from zero-hop adverts into two lists. They wrote that it can be dropped if an issue should tackle one thing, so it is not here. It is feasible now: a zero-hop advert is `ROUTE_TYPE_DIRECT` with an empty path, established while fixing #2064. That deserves its own issue rather than a paragraph in this one — say the word and I will open it. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
