mirror of
https://github.com/Kpa-clawbot/meshcore-analyzer.git
synced 2026-11-22 21:58:03 +00:00
af732d18aa273a97f1822c7a5c19bfe7f314fc7b
3164
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
af732d18aa |
fix(repeater-info): log bulk read errors, cap long notes (#2416)
Follow-up to #2415, see #2414. |
||
|
|
0cf22cdd79 |
feat: repeater info, claims and community-maintained details (#2415)
Closes #2414 |
||
|
|
4ebb5ccc11 |
feat: radio line-of-sight on packet routes + repeater planner (#2391)
Both features are OPT-IN ("los" and "planner" config blocks, default
off); with them off nothing is fetched and payloads are unchanged.
Route view
- Multi-path picker discoverability: on-map "N hops shown · tap to see
all" badge, picker pulse (reduced-motion safe), one-time tip, row hover
arrow; "no GPS" hops counted apart from "unresolved".
- Line of sight for every leg of a packet route: clear / marginal /
blocked chips (first-Fresnel clearance, k=4/3), obstruction markers on
the map, docked terrain-profile panel with antenna/frequency inputs and
a &los= deep link. Backed by GET /api/elevation/profile, an Open-Meteo
proxy that only accepts known node positions, with weighted budgets,
back-off and a per-IP limit.
Repeater planner (#/tools/planner)
- Chain: LOS on every leg between any number of points or nodes.
- Relay: best single relay for A→B among known repeaters and open
terrain.
- Area: suggested new repeater sites that best join the existing
repeater backbone in a box up to 100×100 km, picked greedily to merge
clusters, then remove single points of failure, then reach unlinked
repeaters.
- Terrain: SRTM 1° tiles from AWS Open Data, in memory only (~90 m,
bounded LRU); the server stays read-only. Exact-length gunzip, fetch
budget with back-off, missing terrain reported rather than read as sea
level, fuzzed pixel maths.
- GET /api/planner/{profile,relay,coverage}: quantised cache keys, 202 +
progress then cached 200, separate compute slots, per-IP and global
limits, failed jobs remembered, unpolled jobs cancelled. Nodes pass the
same blacklist / hidden-name / geo_filter rules as /api/nodes. The Go
LOS maths matches public/los.js on shared test vectors.
Also: the per-IP client limiter is extracted from elevation.go into
client_limit.go (behaviour-preserving) so both features share it.
Perf (real SRTM): coverage cold 1.5-5.6 s (tile download), warm 10 ms;
worst real compute 545 ms; peak RSS 91 MiB; live /api/nodes and
/api/packets p95 at most 1.6x while a job computes (GOMAXPROCS=2).
Tests: Go suites for elevation, terrain (fuzz), LOS vectors, graph
(brute-force property test), search and planner API; frontend unit
suites in test-all.sh; Playwright e2e for route LOS and the planner.
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: efiten <erwin.fiten@gmail.com>
|
||
|
|
138473d8c2 |
test(tile-resolver): load CSApi into the sandbox before roles.js (#2402)
## Problem `tests/unit/test-tile-resolver.js` fails 0/26 on master with `CSApi is not defined`, and master's CI on `e7fd48a0` is red because of it ([run](https://github.com/Kpa-clawbot/CoreScope/actions/runs/43880608436)). Two PRs that each passed CI on their own met on master: - #2386 added this suite, which runs `public/roles.js` in a `vm` sandbox. - #2383 made `roles.js` call `CSApi.fetch('/api/config/client')` as it loads. The sandbox never had `CSApi`. ## Fix The suite now loads `deployment.js` and `api-base.js` into the sandbox first, through `tests/unit/api-base-sandbox.js`, as the other suites that load `roles.js` already do. That's two lines in a test file, with no app code changes. ## Verification - `node tests/unit/test-tile-resolver.js`: 26 passed, 0 failed (it was 0/26). - `sh test-all.sh` on this branch: exits 0. ## Related #2391 needs this. It now includes #2383 from master, so its CI hits the same failure. Once this lands, I'll merge master into #2391 again. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
e7fd48a054 |
feat(frontend): serve the UI from a web container or a static host, three deployment modes (#2383)
## What this does The frontend half of serving the UI apart from the server. The server half was #2362. It gives CoreScope three deployment modes. The server container is the same image in all three, and it still serves everything, the classic UI included. | Mode | The UI is served by | API | Server CORS setting | Logins | |---|---|---|---|---| | All-in-one container (today) | the server container | same origin | none | yes | | Split containers (new) | a web container that forwards the API | same origin | none | yes | | Static host or CDN (new) | Cloudflare Pages, a CDN or any web server | another origin | `CORS_ALLOWED_ORIGINS` | no, links to the classic UI | ### Commit 1: the UI as static files, for any host - **One place for API URLs:** `public/deployment.js` says where the API is, and `public/api-base.js` (`CSApi`) builds every API and WebSocket URL. A lint rule keeps raw `/api` fetches out of the other files. The embedded UI keeps `apiBase: ''`, so it behaves as before, except that its WebSocket now uses `/ws` instead of `/`. - **Build:** `make web` (`scripts/static/build.sh`) writes `dist/web`, which is `public/` byte for byte except for the cache buster in `index.html` and `deployment.js`. - It is `dist/web` rather than `dist/` because the Go binaries are built into `dist/` and `make clean` removes that folder. - `API_BASE=same-origin` is for a host that forwards `/api` and `/ws` itself, as the web container does. - **Anonymous build (`AUTH=false`):** for another origin. It has no account features: login, account, admin and other writes link to the same page on the classic UI. It sends no cookies, and its three JSON POSTs go as `text/plain`, so there is no preflight. - **Error notices:** a blocked API answer or an HTML answer each show a notice. - **Channel badge:** `beta`, `next` and `dev` builds show a badge that opens a pre-filled issue, plus an "Open in classic" link. - **Runbook:** `deploy/static/README.md` covers Cloudflare Pages, previews, server CORS and rollback. ### Commit 2: the web container and the three modes - **`Dockerfile.web`:** Caddy on `alpine:3.24`, like the main image. It packages the UI with `build.sh` when the image is built, serves it, and forwards `/api/*` and `/ws` to `BACKEND` (default `corescope:3000`). `SITE_ADDRESS` set to a domain name turns on automatic HTTPS. - **Cache rules:** `docker/web/Caddyfile` sends the same no-cache rules as `deploy/static/_headers`, and compresses responses with zstd or gzip. - **`docker-compose.split.yml`:** the stock server image (its own Caddy off) plus the web container: `CORESCOPE_IMAGE=ghcr.io/kpa-clawbot/corescope:edge docker compose -f docker-compose.split.yml up -d --build`. - `CORESCOPE_IMAGE` is required and must match the checkout's version, so a release server is never paired with a newer UI by default. - The server's built-in broker, which accepts anonymous clients, is published on `127.0.0.1` unless `MQTT_BIND` says otherwise. - **Docs:** `docs/deployment.md` has a new "Deployment modes" section, and `DEPLOY.md` points to it. ### Keeping new code on `CSApi` - **Lint rule:** `.eslintrc.json` now rejects every direct network call in `public/*.js`. That covers `fetch`, `XMLHttpRequest`, `EventSource`, `sendBeacon` and `WebSocket` without `CSApi.wsUrl`, plus `/api` paths in `href`, `src` and `action` (assigned, set with `setAttribute` or written into markup), `window.open`, `location.assign`, `location.replace` and `location =`. `api-base.js` is the one file exempt. - **HTML pages:** `eslint public/*.js` never sees the inline scripts and markup of `public/*.html`, so `test-lint-network-calls.js` applies the same rule to them, at their page lines. `area-map.html`'s two fetches carry an `eslint-disable` with the reason: its Server field can name another server. - **Why it changed:** the earlier rule only caught a literal `/api…` passed straight to `fetch`. - **Check:** CI already fails on lint errors, and `tests/e2e/test-lint-network-calls.js` pins exactly what the rule reports: 25 bad statements reported, 15 good ones not, `api-base.js` exempt. - **AGENTS.md:** a new "API calls" section covers: - which helper to use for each need; - the lint rule; - how anonymous builds gate login-only features; - loading `api-base.js` in vm and jsdom tests (the region quick-picks suite missed that, and failed shard 1 on the first CI run). ### Review follow-ups - **Account tokens:** the channel badge's pre-filled issue drops the query on `#/account/…` routes, so mail-link tokens never reach a GitHub URL. "Open in classic" keeps the full route, since the classic UI needs the token. - **HTML answers:** on a packaged UI (static host, web container), `api()` rejects an HTML answer before status and retry handling. An HTML 404 or 503 (a wrong `API_BASE`, a proxy error page) shows "Server returned an unexpected response" after one request. It used to give a generic error, or six "warming up" retries for a 503. JSON errors and the server's JSON 503 warm-up behave as before. The embedded UI keeps its old handling (`CSApi.embedded`). - **Logins:** `docs/deployment.md`, `Dockerfile.web` and `docker-compose.split.yml` say `userManagement.publicBaseUrl` must be the web container's public origin. ### Audit fixes - **No broken logins build:** `build.sh` refuses `AUTH=true` with a cross-origin `API_BASE`. The server allows no credentialed CORS, so every API request of such a build was blocked; in Chromium all 12 were, and the pages stayed empty. - **No accidental deletes:** `build.sh` refuses an `OUT` that is neither empty nor a previous build. It used to wipe whatever folder or file `OUT` named. - **Lint:** the rule now also reports `/api/` paths in `href`, `src` and `action` attributes in single-quoted strings and template literals. `tests/e2e/test-lint-network-calls.js` pins exactly what it reports. - **Compose:** `CORESCOPE_IMAGE` has no default, and MQTT is published on localhost only. - **Runbook:** the deploy and rollback examples use `vX.Y.Z`, since v3.16.0 and v3.15.0 predate the static build. It also names the server version the static build needs: before #2362 the warm-up headers stay hidden and wildcard previews are refused. ### Audit notes, fixed - **Cache busting on every page:** the tool pages (`area-map.html`, `geofilter-builder.html`) load their scripts with `?v=__BUST__`. The server and `build.sh` replace it in every top-level page, and `_headers` and the Caddyfile make every page revalidate. Behind Caddy, a returning visitor could otherwise get a new tool page with an old `api-base.js`. - **HTTP/3:** the split compose publishes 443 over UDP too, for the HTTP/3 Caddy advertises once `SITE_ADDRESS` is a domain. Checked with Chromium forced onto QUIC: the page and `/api/healthz` loaded over `h3`. - **WebSocket on `/`:** a Go test pins that an upgrade on `/` still works, for UIs cached from before `/ws`. ### Second review - **Trusted proxies:** the docs said to trust the compose subnet, and also to publish port 3000 if wanted. Together they let a client spoof its address: the subnet holds the gateway, where a client reaching a published 3000 through Docker's proxy arrives from. The compose file now fixes the subnet (`SPLIT_NET`, default `172.30.83.0/24`) and the web container's address (`172.30.83.200`), and the docs say to trust that address alone. - **Classic UI unchanged:** the HTML-answer notice above applies to packaged UIs only. On the embedded UI a proxy's HTML 503 is retried as before, and an unknown `/api` path fails to parse as before, with no notice. - **Lint coverage:** the HTML pages and the extra sinks above. AGENTS.md says what the rule cannot see: a URL assembled elsewhere and passed in a variable. - **AGENTS.md:** `build.sh` is described as packaging (a copy, `deployment.js` and `__BUST__`, never a code transform), and rule 3 says `__BUST__` is filled in every top-level page. - **CI builds the web image:** the image-check job (no publish) builds `Dockerfile.web`, runs it, and checks the busted, no-cache `index.html`, the same-origin `deployment.js` and that `/api` reaches `BACKEND`. It also validates the split compose file. - **Cross-origin logins:** a hand-edited `deployment.js` with `auth: true` and another origin now logs a warning naming the setting, instead of failing with CORS errors alone. ## Upgrade notes - **WebSocket path:** the UI's live feed connects to `/ws` instead of `/`. The server has answered `/ws` since #1793 and still upgrades `/`, so browsers holding the previous UI keep working. - A reverse proxy that forwards WebSocket upgrades only for an exact `/` must forward `/ws` too. - The nginx example in `docs/deployment.md` (`location /`) already covers it. - **Static UI from another origin:** - The server needs `CORS_ALLOWED_ORIGINS`, and a version with #2362 for the warm-up banner and wildcard preview origins. - `build.sh` refuses `AUTH=true` there, since the server allows no credentialed CORS. - **Split containers:** - `CORESCOPE_IMAGE` is required. - With user accounts on, `userManagement.publicBaseUrl` must be the web container's public origin. - The built-in MQTT broker is published on `127.0.0.1` unless `MQTT_BIND` says otherwise. - The compose network is `172.30.83.0/24`, with the web container at `.200`. If Docker reports that the pool overlaps another network, set `SPLIT_NET`. ## Testing - `go test ./...`, `go vet` and `gofmt` are clean in `cmd/server`, with new tests for busting every page and the WebSocket upgrade on `/`. - `sh test-all.sh` passes. New unit suites: - `test-static-build.js`: the build and its input checks, `same-origin` included, cross-origin logins refused, and an unrelated `OUT` left intact. - `test-web-container.js`: the Dockerfile, Caddyfile and compose file agree on packaging, forwarded paths, cache rules and the server port; the server image has no default and MQTT stays on localhost. - The `api-base`, anonymous mode, badge and `dev-static` suites. - `eslint public/*.js` has 0 errors. `test-lint-network-calls.js` passes 47/47, the four pages included; it needs eslint, so CI runs it on E2E shard 1. - **Static host or CDN mode:** `test-split-anonymous-e2e.js` passes 11/11. That is the static build on one origin against an API-only server on another, which CI runs on E2E shard 1. - **Split containers mode:** - The image builds, and `caddy validate` passes (Caddy 2.11.4). The new CI step passes locally, and fails when the Caddyfile stops forwarding `/api`. - Trusted proxies, with the stack up: the web container gets `172.30.83.200`. With only that address trusted, a spoofed `X-Forwarded-For` sent straight to a published 3000 is ignored, and one sent through Caddy is replaced with the client's address. With the subnet trusted, as the docs used to say, the spoofed address was believed: a deny entry for it rejected the upgrade. - With `docker-compose.split.yml` on the E2E fixture, seeded the way CI seeds it, `test-e2e-playwright.js` passes 149 of 152 (3 skipped) both through the web container and against the server directly. - Headers, checked with curl: - `no-cache` on `/`, `/index.html` and `/deployment.js`; - zstd on static assets; - `/api/*` proxied (`Via: 1.1 Caddy`); - a `101` WebSocket upgrade on `/ws`. - In Chromium: - every API request and the WebSocket stay on the web container's origin; - there are no page errors and no horizontal overflow at 390 px; - no badge shows on the `static` channel. - Logins through the web container: with user management on and `userManagement.publicBaseUrl` set to the web container's origin, a Chromium run passed every step, all on the web container's origin with no page errors: - registration, and activation from the mail link (which uses `publicBaseUrl`); - the session surviving a reload; - a CSRF-protected `PATCH /api/account`; - logout, then login with the form. - The same login with the backend's origin as `Origin` is refused with 403. The server checks `Origin` against `publicBaseUrl`, not `Host`, which is why the docs ask for it. ## Performance There are no hot-path changes. API calls go through `CSApi.url()`, a string concatenation. In the web container, Caddy serves files from disk and compresses them; API responses pass through with one proxy hop, the same as the all-in-one image's built-in Caddy. ## Notes - **Per-IP limits:** behind the web container, as behind the all-in-one image's Caddy, requests reach the server from the proxy's address. The per-IP limits stay off until `trustedProxies` lists it: `172.30.83.200`, the web container's fixed address. The docs say so. - **`depends_on`:** the web container starts with the server, not after its healthcheck, like the all-in-one image's Caddy. Gating on health would make every `up` wait for the first check (30 s) and fail outright when a large database loads for longer than the healthcheck allows; until the server listens, Caddy answers `/api` with 502. - **Not in this PR:** - publishing a `corescope-web` image (CI builds and smokes it, without publishing); - Cross-origin login—including a static frontend and API on sibling custom domains—is outside this PR. In the deployments supported by this PR, account support requires the frontend and API to share one browser origin, either served together or through a reverse proxy. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
d80ce90976 |
fix(store): release evicted packet references from secondary indexes (#2327)
Addresses #2324 for upstream SQLite; PostgreSQL qualification remains separate. Eviction releases actual node memberships, pointer tails and cache roots without SQL reconstruction. Borrowed distance snapshots and bounded hash-migration ownership remain protected. Review correction `0b1049b5` skips each distance-array copy when no record is evicted from that array. When affected, it clones once and filters the suffix beginning at the first match, preserving borrowed snapshots and the surviving prefix. Work remains O(distance records); no new locks, reader bookkeeping, settings or general speedup claim. Tests-only `16509577` reaches [three real assertion failures in CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/43200112374/job/131121073845). GREEN leaves those tests unchanged. Cases cover neither/hops-only/paths-only impact, repeated removals, live prefixes, tail removal and snapshot contents. Parent independently reproduced RED and reran GREEN. Full native server/ingestor suites pass. Targeted checks: 167 passed; race: 165 passed, with two existing timing skips that pass in the ordinary run. Current master `a0b4e6cf` is normally merged; original RED/GREEN history is preserved. Historical November 10 measurements: synthetic retained heap 98.00→27.14 MiB with equal live data; path-update cost at 30k hops/10k paths 0.591→3.675ms and 2,824→5,368,584 B/op. The older ~30ms extra eviction lock time at 240k transmissions is unremeasured. Large-window qualification/optimization: #2393. Invalidated migration progress: #2394; no SQL migration redesign here. Browser verified: populated matching revision (500 transmissions/502 observations), 3/3 interactions. [Final CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/43202356164) and three independent reviews pass. No settings changes. --------- Co-authored-by: n30nex <6686206+n30nex@users.noreply.github.com> |
||
|
|
0e23c24847 |
fix(scope-stats): report total and split unscoped by route (#2364) (#2368)
Fixes #2364. ## Problem Since #1838, `GetScopeStats` folds every non-transport transmission (FLOOD 1, DIRECT 2) into `summary.unscoped`, while `transportTotal` and `scoped` still cover transport routes (0, 3) only. The documented `transportTotal = scoped + unscoped` no longer holds, and a client computing an unscoped share against `transportTotal` gets >100% (the reporter shows `transportTotal 7620`, `unscoped 14320`). ## Fix — additive, no existing field changes value `summary` gains: | field | meaning | |---|---| | `total` | all observed transmissions, routes 0–3 (= `scoped + unscoped`) | | `unscopedTransport` | route 0/3 with Code1 = 0000 | | `unscopedFlood` | route 1 — unscoped by protocol | | `unscopedDirect` | route 2 — unscoped by protocol | Invariants (now asserted in `TestGetScopeStats`): ``` transportTotal = scoped + unscopedTransport unscoped = unscopedTransport + unscopedFlood + unscopedDirect total = scoped + unscoped ``` The non-transport count query keeps its single scan; it just splits route 1 vs 2 with `CASE`. On the design question in the issue: rather than changing what `unscoped` means (the Scopes tab and #1842 rely on it), the split lets a rollout view use `unscopedFlood` directly: repeaters apply region rejection to flood routes only (`isRouteFlood()` in `examples/simple_repeater/MyMesh.cpp`), so DIRECT traffic is not subject to flood-region filtering. `docs/api-spec.md` now says so, spells out the route mapping, and notes that `timeSeries` is transport-only (its `unscoped` matches `unscopedTransport`). ## Frontend Scopes tab uses `summary.total` when present and shows `N transport, N flood, N direct` on the Unscoped card; falls back unchanged for older servers. The cards are built by `scopeSummaryCardsHtml(summary)` (exposed as `window._analyticsScopeSummaryCardsHtml`). ## Tested - `go vet ./...` (cmd/server) clean, `gofmt -l` clean - `go test -run ScopeStats ./...` — pass (incl. extended `TestGetScopeStats` and `/api/scope-stats` route test) - `node --check public/analytics.js`; `tests/unit/test-issue-1375-scope-stats-fetch.js`, `test-issue-1979-scope-adverts-by-role.js` pass - new `tests/unit/test-issue-2364-scope-summary-cards.js` (vm sandbox, 11 assertions: shares of `total`, route split, fallback for servers without `total`/split, partial split, empty window); full `test-all.sh` green Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: SaarMesh-Bot <300107934+SaarMesh-Bot@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
ea87784257 |
refactor(live): reuse shared timestamp renderer (#2395)
Live timestamps duplicated the shared timestamp markup in `app.js`. Keep `formatLiveTimestampHtml` as the input adapter and delegate its final markup to `renderTimestampHtml`, retaining the existing missing-helper fallback and Date/number/string normalization. Partial follow-up to #1154 U10 and the [maintainer note on #2283](https://github.com/Kpa-clawbot/CoreScope/pull/2283#pullrequestreview-5850899652). This addresses only the Live wrapper; the umbrella remains open. Validation: - All 229 existing standalone frontend suites pass. Two shell-dependent suites passed after selecting the installed shell/interpreter for the local test process. - 5,040 exact-base/candidate comparisons match byte-for-byte across timestamp inputs, preferences, escaping, future warnings, and missing-helper fallbacks. - Actual Go fixture and Chromium: node-panel and replay-feed timestamps checked on desktop and phone, with no page exceptions. The existing phone portrait layout hides the feed; its replay DOM remains populated. - ESLint: zero errors. No new settings, requests, dependencies, or caches; constant work per timestamp, with no speedup claimed. Pure-refactor TDD exemption: **tests/ files diff: no changes**. No test files added or modified; existing assertions remain green. [Final CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/43202375909) and three independent reviews pass. Parent independently repeated the Live unit, parity and post-push browser checks. Co-authored-by: n30nex <6686206+n30nex@users.noreply.github.com> |
||
|
|
c2e770ef65 |
docs: archive superseded repeater liveness plans (#2392)
Partial fix for #1385 (PR-E). Move the two April repeater-liveness documents from `docs/superpowers/` into existing `docs/archive/` with `git mv`. Preserve their dated filenames, content and headings; add historical notices clarifying that the proposed `relaying` status/signature is outdated and linking current guidance. Validation: exact commit blobs preserve 41,483 original bytes and 24 headings after removing only notices; four local links resolve; no tracked old-path references; whitespace and PII checks pass. TDD exemption: pure documentation, no behavior change. No runtime, configuration, customizer, test, hierarchy or ignore-rule changes. Application suites were not rerun. Both published Markdown pages and their guidance links render correctly. [Docs-only CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/43199869942) and three independent reviews pass. The umbrella issue stays open pending maintainer reconciliation. Co-authored-by: n30nex <6686206+n30nex@users.noreply.github.com> |
||
|
|
a0b4e6cf5f |
fix(nodes): light-mode map follows tile picks, key and theme changes (#1470/#1420) (#2386)
fix(nodes): light-mode map follows tile picks, key and theme changes
(#1470/#1420)
## Problem
The node-detail maps (`#/nodes/<pubkey>` full page + side pane — and
their
reusers `node-reach-map.js`, `rx-coverage.js`) pick a different basemap
in
**light mode** than the rest of the app:
- hardcode Carto Positron: the user's *Light Mode Provider* customizer
pick
and the operator's `map.tiles.lightDefault` are ignored
- render Carto's `API KEY REQUIRED` watermark when `/api/config/client`
lands after the map is created — the tile requests stay keyless until a
remount (they come back HTTP 200, so nothing surfaces as an error)
- never react to a theme flip or provider change while the page is open
Dark mode is unaffected, which is why it only shows up in light mode.
Reproduce on `master`: open `#/nodes/<id>` in light mode with
`mc-light-tile-provider = carto-voyager` — the inset requests
`light_all`
while the main map requests `rastertiles/voyager`.
## Root cause
`_applyTilesToNodeMap` (`public/nodes.js`) asked `window.getTileUrl()`
for a
single URL string, and roles.js's light branch returned the frozen
`TILE_LIGHT` scalar — the dark branch was the only one that consulted
the
runtime registry (`public/map-tile-providers.js`), which is what owns
localStorage picks, server defaults, the Carto key, and per-provider
attribution/maxZoom/refUrl/invertFilter. map.js and live.js keep the
only
other registry-aware resolvers of the same logic.
## What this PR does
- adds one resolver, `MC_resolveTile(isDark)`, in map-tile-providers.js,
returning the full descriptor
`{ type, id, url, attribution, maxZoom, refUrl, invertFilter }`:
- `url` is always a resolved string — function-typed providers are
invoked at call time (contract of #1614)
- one precedence rule everywhere: explicit operator overrides
(`cfg.tiles.dark|light`, `map.tiles.darkUrl|lightUrl`, tracked via
`MC_LEGACY_TILE_OVERRIDE`) > user/server registry pick >
`TILE_DARK`/`TILE_LIGHT` constants — **this is a deliberate behavior
change on map.js/live.js, see below**
- fully lazy: registry rebuilds / pick changes are picked up by the next
resolve, so config landing after page load fixes itself
- roles.js `getTileUrl()` becomes a thin wrapper over it; the public API
and
the #1614 sandbox contract are unchanged; `getActiveTileProvider()`
keeps
its documented null-in-light contract
- `nodes.js _applyTilesToNodeMap` resolves through the descriptor and
stays
reactive: one shared `mc-tile-provider-changed` listener + one
`data-theme`
MutationObserver re-sync every registered map in place — old layers
removed
before the new ones are added, never stacked; maps whose container left
the
DOM are pruned at sync time, so SPA navigation cannot leak listeners
- map.js `_resolveTileUrl` / live.js `_liveResolveTile` delegate to the
shared resolver (their legacy bodies remain only as sandbox fallbacks)
- provider `maxZoom` replaces the hardcoded 18/19 caps (OpenTopoMap 17
and
USGS 16 used to 404 above their max zoom)
- analytics.js subpath minimap gets provider `maxZoom` as well
## Behavior change (deliberate): operator URL overrides now win on every
surface
On `master`, the two tile-selection mechanisms conflicted and each
surface
resolved the conflict differently:
- `map.js` / `live.js`: user/server **registry pick won** — an
operator's
`tiles.dark|light` / `map.tiles.darkUrl|lightUrl` was effectively dead
config on those pages
- the legacy roles.js path (nodes page, reach map, rx-coverage,
analytics
minimap): **operator override won**
This PR unifies on **operator-override-wins**, because that is the only
reading that gives live operator config a consistent meaning:
- these keys are still applied today. The deprecation warning says they
would "be ignored in v3.5.0", and v3.5.0 shipped 2026-04-08 — the
instance line is now at v3.16.0 and the keys still take effect, so they
are live configuration, not dead letters. An explicit operator pin
should not be silently defeated by a per-user pick on some pages and
honored on others; whichever way the mechanism eventually goes, it must
behave identically everywhere until it is actually removed.
- user choice is not reduced in the general case: picks remain
authoritative for every provider the registry offers, and overrides
only exist where an operator explicitly wrote a literal URL.
If maintainers would rather have pick-wins (operator URL acts only as a
fallback default), flipping the precedence in `MC_resolveTile` is a
small,
well-tested change — say so in review and it will be reworked.
Related cleanup, out of scope here: the stale "ignored in v3.5.0"
deprecation warning in `cmd/server/config.go` should either be honored
or
reworded now that the project is past v3.16.0.
## No change without operator overrides or picks
## Verification
Unit (all deterministic, `vm`-sandbox based):
- new `tests/unit/test-tile-resolver.js` — 22 cases: light/dark pick
honored,
operator `lightDefault`, disabled-provider fallback, override
precedence,
lazy function-URLs, key-after-async-config, in-place swap without
stacking,
detached-map prune, multi-map single-listener sync, provider maxZoom;
listed in `test-all.sh`
- existing suites green: `test-issue-1420-tile-providers` (47),
`test-issue-1470-node-tile-helper` (6),
`test-issue-1614-tile-url-function`
(3), `test-frontend-helpers` (79), `test-settings-sync`,
`test-packet-filter` (105), `test-aging`
- every one of the 227 suites listed in `test-all.sh`, run individually:
all green except `test-manage-workflow` (2 failures that are local-env
only — CRLF bootstrap artifact — and pass in CI; `sh test-all.sh` aborts
there locally under `set -e`, which is why the suites were run one by
one this time; the scripted pipeline runs fully in CI)
- `eslint` on all six touched files: 0 errors
Browser (headless Chromium against this branch, `/api/config/client`
delayed
~800 ms to force the config race):
- light + `/#/nodes/<key>`: 12 keyless tile requests before config, 12
keyed
re-requests after — no reload, no stacked layer
- light pick `carto-voyager`: inset requests `rastertiles/voyager` from
the
start (parity with the main map)
- theme flip on an open node page: `dark_all` swaps in in place
- dark defaults unchanged (`dark_all`, keyed after config)
The Playwright E2E suite was not run locally — no Go toolchain on the
workstation to serve the backend — CI will cover it.
## Scope / follow-ups (intentionally not here)
- `home.js` mini-map (OSM only) and the geo-filter mini-maps in the
customizers are untouched on purpose: they are outside this bug's path,
and migrating them to `MC_resolveTile` later is mechanical
## Performance
Not a hot path: one registry lookup + string build per map creation and
per
provider/theme-change event (each is rare); no per-packet or per-render
work;
the listener footprint is O(1) app-wide (single shared listener pair + a
pruned registry), so there is no growth across SPA navigations.
Co-authored-by: OpenCode <git-llm@trix.moe>
Co-authored-by: OpenCode:GLM-5.3-Flash <git-llm@trix.moe>
Assisted-by: OpenCode:GLM-5.3-Flash eslint
---------
Co-authored-by: OpenCode <git-llm@trix.moe>
|
||
|
|
6dd62711bc |
docs(release): v3.17.0 release notes and changelog entry (#2390)
Release notes for v3.17.0: `docs/release-notes/v3.17.0.md` (used as the GitHub release body by `deploy.yml`) and the `CHANGELOG.md` entry. 24 commits since v3.16.0, counted with #2389, which is merged before this PR. Issues closed come from each PR's `closingIssuesReferences`. Docs only. After merge: tag `v3.17.0` on the merge commit once master CI is green and the `:edge` image carries that commit.v3.17.0 |
||
|
|
873e8b0e16 |
feat(rx-coverage): QR code and link to the operator's CoreDrive RX deployment (#2389)
## What An operator who runs a CoreDrive RX deployment (the web and Android companion app that feeds `clientRxCoverage`) can now point visitors of the Coverage page at it. - New optional config key `clientRxCoverage.appUrl`. `/api/config/client` sends it as `clientRxAppUrl`, only while `clientRxCoverage.enabled` is true and the value is an absolute `https` URL with a host name. Any other value is left out and logged once at startup with a `[config]` line. - `#/rx-coverage` shows a panel under the subtitle when the field is present: a QR code of exactly that URL (vendored `qrcode.js`, same call as the node detail QR), the URL as a link, and one line on scanning it in the app or opening it on a phone. Without the field there is no panel. The page still needs no login. - The subtitle named the app "CoreScope-RX" and linked to `github.com/efiten/corescope-rx`, the app's old repository name. It now says "CoreDrive RX" and links to `github.com/efiten/coredrive-rx`, or carries no link when the panel already shows the deployment. ## Why The CoreDrive RX Android app is not tied to one server: on first start it asks for the address of a deployment. Scanning a QR code on the Coverage page is the shortest way for a visitor to get there, including people without an account. Disclosure: CoreDrive RX (`efiten/coredrive-rx`) is my own project. ## Changes - `cmd/server`: `config.go` (`validClientRxAppURL`, startup warning), `types.go`, `routes.go`, `main.go`, `openapi.go`. - `public/rx-coverage.js` (`appPanelHtml`, `appLinkHtml`), `public/roles.js` (`window.MC_CLIENT_RX_APP_URL`), `public/node-reach-coverage.css` (existing CSS variables only). - Docs: `config.example.json`, `docs/client-rx-coverage.md`, `docs/api-spec.md`. ## Tests - 3 new Go tests in `cmd/server/client_rx_app_url_test.go`: URL validation over 15 inputs, the endpoint field in 6 config states, the startup warning. Full `go test ./cmd/server` passes. - `tests/unit/test-rx-coverage-app-qr.js` (real `rx-coverage.js` and `qrcode.js` in a vm): panel present and absent, QR content, HTML escaping, subtitle link in both cases, old names gone. Added to `test-all.sh`. All 6 `tests/unit/test-rx-coverage-*.js` pass. ## Not done - No E2E test. - Browser-checked on a local server with and without `appUrl` in the light theme only. The QR keeps black modules on white in both themes, unlike the node QR, which is inverted in dark mode; dark mode itself was not checked in a browser. - `cmd/ingestor` is unchanged; it does not need the URL. - The same change runs on analyzer.on8ar.eu (a fork build) since 2026-11-10. |
||
|
|
4140028165 |
fix(ui): keep slide-over close controls above navigation (#2304)
Fixes #2302. Navigation covered tablet slide-over close controls and intercepted pointer input. Two CSS substitutions put the panel/backdrop on the existing modal layers. The regression uses actual close/backdrop clicks and taps, retaining the existing keyboard, focus, scroll and resize assertions. Refreshed at `cd834802` by normally merging current master `d1bf4ca0`, with zero conflicts. The exact three-file feature patch and original RED/GREEN commits are preserved. Both feature tests are unchanged by this refresh, including the supported 180-minute fixture window. [Original RED job](https://github.com/Kpa-clawbot/CoreScope/actions/runs/40212233915/job/121225789154): 40 passes, 17 pointer assertion failures, zero skips. The job completed with failure before its aggregate run was cancelled. Merged #2289 resolves the old unrelated RF test failure. Validation: four 57-case slide-over runs; three focus/scroll cases; 20 supplemental cases; 224 frontend suites; related navigation/gesture checks; browser smoke with 149 passes/three existing fixture skips. Parent rerun passes 57+3 after an initial-response timeout; tests were unchanged. [Fresh CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/42324795657) passes, including the repeat gate. Independent reviews found no introduced must-fixes; Kent Beck Gate: PASS. Existing observer-focus and packet-navigation problems are separately tracked in #2379 and #2380; runtime JavaScript is unchanged. Browser verified: disposable Go fixture at `http://127.0.0.1:14964`. E2E assertion added: `tests/e2e/test-slideover-1056-e2e.js`. No new settings, dependencies, requests or hot-loop work. ## Preflight overrides - check-red-commit: use the completed RED job’s 17 assertion failures; its aggregate run was subsequently cancelled. --------- Co-authored-by: n30nex <6686206+n30nex@users.noreply.github.com> |
||
|
|
9e78e5c98d |
fix(live): show the newest certain node activity (#2283)
Fixes #2233. Live node details use the newest certain activity from the node and health responses for both status and Last Activity. Repeaters and rooms with only recent shared-prefix relay evidence remain Uncertain. Stale/missing values and a failed health request retain conservative behavior. Refreshed at `4d645dbc` by normally merging current master `d1bf4ca0`, with zero conflicts. The three-file feature patch and original RED/GREEN history are unchanged. [Original RED CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/40190088413) remains; the preserved unit regression reaches 15 passes/eight assertion failures locally. Validation: all 224 frontend suites; 23 health-unit cases; eight health browser groups including 12 Live cases; four RF airtime groups; and 48 Live cases across desktop/phone and light/dark. Parent reran the unit and health browser suites. Backend trees are byte-identical to the already-qualified master; no new native Go result is claimed. The old unrelated RF test failure was reproduced during an actual theme refresh. Upstream #2289’s locator waits correctly while retaining its value/warning assertions; it is included in this refresh. Browser verified: disposable Go fixture at `http://127.0.0.1:14963`. Readable screenshots use the existing hidden-legend setting; the default desktop legend overlap is unchanged. E2E assertion added: `tests/e2e/test-health-status-e2e.js:259`. Fixed-size work for one opened panel; no extra requests, dependencies or settings. [Fresh CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/42323634522) passes. Independent correctness, UI and test-quality reviews found no introduced must-fixes; Kent Beck Gate: PASS. --------- Co-authored-by: n30nex <6686206+n30nex@users.noreply.github.com> |
||
|
|
fad303a4f4 |
docs: split the Docker minimum into running and building the image (#2361)
`docs/deployment.md` listed Docker 20.10+ as the minimum, without separating running the published image from building it. The Dockerfile needs BuildKit (`# syntax=docker/dockerfile:1` on line 1, `RUN --mount` from line 71), and BuildKit is the default builder only from Docker 23.0. On 20.10 to 22.x the legacy builder rejects `RUN --mount` unless `DOCKER_BUILDKIT=1` is set. Found while reviewing #2359. Changes: - `docs/deployment.md`: the requirements table gets two Docker rows, 20.10+ to run the published image, and 23.0+ (or 20.10+ with `DOCKER_BUILDKIT=1`) to build it, plus one sentence on why. - `docs/go-migration.md`: that guide builds the image, so its prerequisite becomes 23.0+ (or 20.10+ with `DOCKER_BUILDKIT=1`). Not verified: I did not run a build on Docker 20.10 or 22.x; the legacy-builder behaviour comes from Docker's documentation, not from a test here. Podman and buildah are not covered. Docs only. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
9ddb29c946 |
fix/channels-mobile-header-overflow (#2382)
Fixes #2381. Drops the 56px header cap on mobile; title + Add on row 1, region pills on a sideways-scrolling row 2. See the issue for details --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
afbae53b71 |
ui(nodes): add direct map and packet navigation (#2295)
Fixes #2290; addresses U6 of #1154. Full node details, the side preview and tablet slide-over offer Open in Map and View packets through existing deep links. A shared renderer validates coordinates, including zero, and encodes identifiers without extra fetches. Closing a preview preserves newly selected destinations and titles; manual close returns to Nodes. Tablet targets stay separate, and modified clicks retain native behavior. Refreshed at `538cb169` by normally merging current master `d1bf4ca0`, with zero conflicts. The original five commits and exact seven-file feature patch are preserved. The [original RED CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/40203064817) remains; the preserved RED also reproduces 39 assertions locally. Upstream #2333 now supplies the correction for the previous unrelated navigation-focus CI failure. Current validation: all 225 frontend suites; 52 focused unit checks; 31 shortcut browser cases in each theme; 24 additional title/native-click cases; navigation 60/60, branding 17/17 and slide-over 27/27. Parent independently reran 52 unit and 31 browser cases. Native server and ingestor suites pass. General browser smoke reports 149 passes and three existing fixture skips. Unchanged serial retries resolved two documented local test-server/navigation setup failures. Browser verified: disposable local Go fixture at `http://127.0.0.1:14954`. E2E assertion added: `tests/e2e/test-node-quick-actions-e2e.js`. Constant work for one detail view; no new requests, dependencies or settings. [Final refreshed CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/42318272198) passes. Independent adversarial, UI and test-quality reviews found no must-fixes; Kent Beck Gate: PASS. --------- Co-authored-by: n30nex <6686206+n30nex@users.noreply.github.com> |
||
|
|
a59b42d8c2 |
fix(observers): keep charts and controls within narrow pages (#2344)
Fixes #2341. Observer chart grids fit narrow screens, the doughnut stays square while resizing, and empty canvases/long comparison choices no longer widen the page. Radio formatting, requests and chart calculations are unchanged. Refreshed at `f2ad231e` with a conflict-free normal merge of current master `1577b57d`. Merged #2343 delayed-focus coverage and #2374 shared payload labels are preserved. In response to review, the layout test now checks one column at 760px and two at 900px, safely away from the exact breakpoint. All original geometry, reload, disposal and request-count assertions remain. Original RED proof is retained: [four chart-width failures](https://github.com/Kpa-clawbot/CoreScope/actions/runs/42011115036/job/127190202787) and [four empty-data/control failures](https://github.com/Kpa-clawbot/CoreScope/actions/runs/42016791855/job/127209299782). Current validation: all 226 frontend suites and 68 ownership browser cases pass. Parent independently reran four chart cases covering 32 layouts. Restoring current-base production through browser interception makes all eight layout cases fail their overflow assertions; tests remain sensitive to the bug. Browser verified: disposable current-base Go fixture at `http://127.0.0.1:14973`, desktop/mobile and both themes. E2E assertion added: `tests/e2e/test-observer-detail-ownership-e2e.js`. No new settings, dependencies, API calls or resize code. [Current CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/42803853923) and independent adversarial, visualization and Kent/TDD review pass at `f2ad231e`. --------- Co-authored-by: n30nex <6686206+n30nex@users.noreply.github.com> |
||
|
|
c03147cf0f |
fix(ui): share timestamp cells and retain future-clock warnings (#2387)
Partial fix for #1154 U10; follow-up to the maintainer's markup/warning note on #2377. Peer Interactions Last Contact now retains the same future-clock warning as Nodes and Packets. One shared HTML renderer in app.js owns the escaped timestamp, tooltip and existing warning markup; the three surfaces delegate to it. Existing wrapper names/fallbacks, timestamp preferences, placeholders and date algorithms are preserved. No new settings, dependencies, requests or render triggers; work stays constant per cell. Tests-only RED `28e09e7c` reached [three unit assertion failures](https://github.com/Kpa-clawbot/CoreScope/actions/runs/42800749635/job/129777366878) and [six browser missing-warning assertions](https://github.com/Kpa-clawbot/CoreScope/actions/runs/42800749635/job/129777366837). GREEN `27f9fa49` leaves those tests unchanged and passes 115 unit assertions, all 226 frontend suites and 10 browser cases. Parent independently reran the unit and browser checks. Twenty-two Nodes/Packets outputs match base exactly across 11 input/preference cases. Browser verified: disposable Go fixture at `http://127.0.0.1:14975`, desktop/phone with UTC/local and relative/absolute preferences. Visible warning/tooltip checks pass. The fixture's missing battery table and existing final-row bottom-navigation overlap are unchanged limitations. E2E assertion added: `tests/e2e/test-issue-2218-node-analytics-stale-e2e.js`. No production use of a test-only export, and no broader timestamp audit claimed. [Current CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/42804968476) and independent adversarial, visualization and Kent/TDD review pass at `27f9fa49`. --------- Co-authored-by: n30nex <6686206+n30nex@users.noreply.github.com> |
||
|
|
6c37fc59e1 |
chore: ignore root local worktree directories (#2388)
Partial fix for #1385 (PR-G). Root `_wt-*` worktrees and `.worktrees/` folders currently appear as untracked content. Add anchored, directory-only ignore rules for those two local worktree conventions. Nested folders and similarly named files remain visible. Validation: - Isolated Git repositories with empty global and local excludes: 21 before/after cases, 42 assertions passed. - No currently tracked path matches either new rule; all other tracked content and `.dockerignore` are unchanged. - Existing planning-file and attachment visibility is preserved; `git diff --check` passed. TDD config-change exemption: no test files modified. [Final CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/42809414204) passes without test edits. No runtime, browser UI, or customizer behavior changes. Parent reran the Git checks and post-push browser smoke (3/3); served assets match the candidate. Three independent reviews pass. This covers only PR-G's transient worktree ignore audit. Other #1385 work, including PR-E documentation, remains outside this change; the issue stays open. Co-authored-by: n30nex <6686206+n30nex@users.noreply.github.com> |
||
|
|
1577b57d67 |
docs(accounts): approved channels are backfilled since #2133 (#2385)
`docs/user-guide/accounts.md` said that messages received before a channel proposal is approved stay encrypted. Since #2133 that is no longer the case: when the key set grows, the ingestor re-decodes the stored rows it could not decrypt (`cmd/ingestor/key_backfill.go`), and the server tails `tx_rewrite_feed`, so no restart is needed. Checked on an instance running #2133: approving four hashtag channels triggered one pass (`[key-backfill] channel: 4 of 410 key(s) not yet applied to history`, then `pass complete, 801 row(s) rewritten`, 13 s), and the 801 rows are exactly the stored messages of those four channels, the oldest from three months before the approval. Docs only, one sentence. Not done: I did not check the other docs pages for the same claim beyond a grep for "stay encrypted" and "received before" in `docs/` and `README.md`, which found only this one. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
d66a219bcb |
feat(cors): let a UI on another origin connect, static host previews included (#2362)
## What this does Server changes so the UI can be served from another origin, such as a Cloudflare Pages project and its previews, against an existing backend. It ships before the frontend, which is parked on [`backup/static-frontend-1a`](https://github.com/ppicazo/CoreScope/tree/backup/static-frontend-1a) and will come as its own PR. - **Exposed headers:** allowed origins get `Access-Control-Expose-Headers: Retry-After, X-CoreScope-Load-Status`. The UI reads both, and cross-origin they are `null` without this. - **Wildcard origins:** an entry may be `scheme://*.host[:port]`, such as `https://*.myproject.pages.dev`. It matches exactly one leftmost label (each deploy and branch preview) for both CORS and `/ws`, but not `myproject.pages.dev` itself. The part after `*.` needs at least three labels, so `*.pages.dev` is refused. Refused or malformed wildcards are dropped at startup with a log line. That floor is a sanity check, not a Public Suffix List lookup: `https://*.s3.amazonaws.com` passes and would admit every S3 website endpoint, so a wildcard belongs only over a host the operator controls. - **Case:** exact entries now match case-insensitively for CORS (was `==`), as they already did for `/ws`. - **Test only:** pins that the three anonymous POSTs (`/api/decode`, `/api/packets/observations`, `/api/paths/inspect`) accept a `text/plain` JSON body, which keeps them free of a preflight. Credentials stay off, so nothing a listed origin sends rides on cookies. Beyond GET, a listed origin, and so every deployment under a wildcard entry, can: - call those three anonymous POSTs as simple requests. That predates this PR; they compute an answer and write nothing. `Access-Control-Allow-Methods: GET, HEAD, OPTIONS` only governs preflighted requests. - with `userManagement.enabled`, use the existing bearer tier for companion linking (`/api/auth/device-token`, `/api/auth/me`, `/api/auth/logout`, `/api/account/settings`, `/api/account/companions*`) with POST, PUT and DELETE. Those routes need a device token in `Authorization`, not a cookie, but a preview deployment gets them too. On such a server, list a wildcard only if previews may use them. ## Configuring ```yaml - CORS_ALLOWED_ORIGINS=https://existing-embed.example,https://myproject.pages.dev,https://*.myproject.pages.dev ``` The env var replaces `corsAllowedOrigins`, so keep the origins already listed there. ## Testing - `go test ./...`, `go vet` and `gofmt` are clean in `cmd/server`. The new tests were written first and failed first. - In Chromium, the static build from the backup branch, served on `pr-1.proj.preview.localhost`, passed its split E2E (9/9, WebSocket included) against this server. - From the same setup, the project origin connected, a two-level subdomain was blocked, and `X-CoreScope-Load-Status` read `ready` where master gives `null`. ## Performance These run per request in the CORS middleware and the `/ws` handshake, and they allocate nothing. Same-origin requests skip them. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
899a3ad0b1 |
fix(map): normalize saved viewport longitudes (#2378)
Partial fix for #2375. Restoring a saved longitude such as -352.7 previously put markers one world-width off-screen while tiles looked correct. Saved positions now use Leaflet’s native wrapping and the existing viewport validator; movement saves canonical coordinates. Invalid saved values fall back to configured defaults. Valid URL viewports and node highlighting keep precedence. This repairs saved state, not live dragging across world copies. No new settings, dependencies, requests, marker loops or render triggers; added work is constant per initialization/existing moveend. The reported-node URL diagnosis is separate: no-slash returns JSON when enabled and intentionally 404 when disabled; trailing-slash returns HTML. See #2372. The endpoint/gate remain unchanged; #2375 stays open for maintainer reconciliation. RED `cb453c0d` produced [14 passes and 10 intended CI assertions](https://github.com/Kpa-clawbot/CoreScope/actions/runs/42317069830/job/128232241526). Production fix `e6b70dfd` passed CI, 224 frontend suites, 24 browser cases, six estimated-marker and three mobile checks. Marker placement changes from x=262754/offscreen to x=610/visible in the same fixture. Review follow-up `99a4746e` changes only test data: configured and saved centers now differ, while invalid storage independently expects the default. Parent rerun passes 24/24; a browser-only mutation discarding saved coordinates now fails seven cases. All previous assertions and production files are retained. The single review finding is resolved. Browser verified: disposable Go fixture at `http://127.0.0.1:14956`, real Leaflet rendering. E2E assertion added: `tests/e2e/test-map-nodes-pagination-e2e.js:129`. Final follow-up CI is blocked by the unchanged observer disposal assertion (#2342; fix #2343). A maintainer must rerun the workflow; this remains draft. --------- Co-authored-by: n30nex <6686206+n30nex@users.noreply.github.com> |
||
|
|
8420a7ac0f |
feat(map): identify historical foreign-advert flags (#2371)
Partial implementation of #730 M2; the issue remains open. The map marks nodes whose stored `foreign` Boolean is true in dot and hash-label modes. Titles, accessible names and popups explain that an advert was flagged outside the configured area at ingestion. This historical flag does not establish current location, bridging, an observer or a recent count. Analytics/history remain separate. The warning uses existing theme tokens and stays clear of nearby markers and phone controls. Footprint preparation adds O(n) work and numeric metadata to the existing marker list (around 2K nodes), with one typography read per applicable render and no per-marker DOM measurement or requests. Existing bounded-offset placement retains its worst-case quadratic search; this is not a speedup claim. No new settings or render triggers. RED: `4ad11148` produced [44 missing-warning assertions](https://github.com/Kpa-clawbot/CoreScope/actions/runs/42012004213/job/127193151694). Follow-up `c4a157de` produced [11 warning-occlusion assertions](https://github.com/Kpa-clawbot/CoreScope/actions/runs/42023035345/job/127230364384). GREEN `ad465e32`: parent browser check 66/66, 224 frontend suites, 38 existing map/route checks and CSS audit. Incorrect-copy controls also prove the tests reject a current-location inference. Browser verified: local Go fixture, desktop/phone, both themes. E2E assertion added: `tests/e2e/test-map-foreign-advert-e2e.js` covers keyboard access, missing/false flags, no-GPS, refresh, clustering, overlays, zoom and resize. ## Preflight overrides - check-red-commit: initial RED includes the new browser regression plus necessary CI/inventory registration, with no production changes; CI reached the intended assertions. [Final CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/42026358900) passes. Three independent reviews were consolidated; both findings are fixed and parent-verified. Kent Beck Gate: PASS. --------- Co-authored-by: n30nex <6686206+n30nex@users.noreply.github.com> |
||
|
|
65990873ad |
feat(map): deep-link the route path picker by observer or All view (#2376)
## Summary The map route deep link (`#/map?packet=<hash>[&obs=<id>]`) always opened in the canonical-path view. Two optional query params now choose the path picker's starting view, so a shared link shows exactly what the sender meant. | Param | Effect | |---|---| | `paths=all` | Opens the picker's **All** view (every unique path), the same as clicking **All** | | `observer=<pubkey>` | Full public key or a hex prefix of 8+ characters, case-insensitive. Picks that observer's **earliest** observation (by timestamp, then id), makes it the canonical route and isolates its path row, the same as clicking the row. "Back to packet" then links to that observation. | Examples: ``` #/map?packet=974ad4ecefbe3761&paths=all #/map?packet=974ad4ecefbe3761&observer=a1b2c3d4 ``` When params are combined, `obs=` wins over `observer=`, and `observer=` wins over `paths=all`. An unknown or malformed observer falls back to the current default view and logs a console warning. ## Changes - `public/route-view-utils.js`: two new pure helpers. - `resolveDeepLinkObservation(observations, params)` picks the observation and the initial view. - `pathGroupKey(rawHops, filterHops)` is the path picker's grouping key. It was written out twice in `route-view.js` and now lives in one place, which keeps the deep link and the picker rows in sync. - `public/map.js`: `loadRouteFromDeepLink` uses the helper and passes `initialPathView` through `drawPacketRouteMulti` to the renderer. - `public/route-view.js`: the picker count and rows use `pathGroupKey`. `sidebar._applyPathView` applies the view through the existing All and row click handlers, in the frame after the route's initial fit. ## Performance This doesn't touch a hot path. The work is one pass over a single packet's observations (tens at most) plus a scan of the picker rows, and it runs once per deep-link load. It adds no API calls. ## Config / customizer No new configurable values. ## Tests - New unit suite `tests/unit/test-map-deeplink-path-params.js` (29 checks). It covers prefix vs full key, case, short/non-hex keys, earliest-observation choice with tie-break, the precedence rules, `pathGroupKey`, and the source wiring. - `tests/unit/test-issue-1633-hide-1byte-hops.js` replayed a copy of the key logic. It now tests the real `pathGroupKey`. - New Playwright suite `tests/e2e/test-map-deeplink-path-params-e2e.js` (5 checks) on the seeded #1486 packet. It's registered in `scripts/non-unit-tests.json` and CI shard 3. With the `route-view.js` change removed, 3 of its 5 checks fail. - Local runs against the CI-prepared fixture: `test-all.sh` passes. Route-map a11y E2E 20/20, map mobile E2E 3/3, `test-e2e-playwright.js` 149/152 (3 skipped by the fixture, 0 failed). ## Browser validation Checked in a browser against a local server, using a scratch copy of the fixture that gives the seeded packet multi-hop paths through real GPS nodes. One observer had two observations, and the API listed the later one first. - `observer=<prefix>` picked that observer's earliest observation and drew only its path, with numbered markers and the map fitted to it. - `paths=all` drew the same picture as clicking **All** by hand. |
||
|
|
3100d7a4c9 |
fix(ui): route remaining payload label surfaces through payload-labels.js (#2373) (#2374)
Fixes #2373 ## Summary Finishes the #1799 / #1804 migration: every frontend surface that names a payload type now reads it from `public/payload-labels.js`. Node Analytics, Observer detail and Observer comparison showed GRP_DATA, MULTIPART and RAW_CUSTOM as `Type 6` / `Type 10` / `Type 15`. The map route sidebar had RAW_CUSTOM on id 12; the firmware uses `0x0F`. ## Changes - **`public/payload-labels.js`**: new `PayloadLabels.api.displayName(id)` returns the canonical short label, or `Type N` for ids the firmware does not define (e.g. a corrupted header byte stays visible as `Type 13`). It lives on `.api` only; the legacy root aliases are not extended. - **`public/node-analytics.js`, `public/observer-detail.js`, `public/compare.js`**: drop the local `PAYLOAD_LABELS` maps and use `displayName()`. - **`public/nodes.js`**: drop the local `PAYLOAD_TYPES` map; Recent Adverts uses `shortById()` and keeps its `Packet` fallback. - **`public/app.js`**: drop `PAYLOAD_TYPES`; `payloadTypeName()` uses `shortById()` and keeps its `UNKNOWN` fallback. - **`public/map.js`**: drop `PAYLOAD_TYPE_MAP`; `inferredType` uses `enumNameById()`, which fixes RAW_CUSTOM (15 → `RAW_CUSTOM` instead of `OTHER`). - **`public/packets.js`**: `typeName()` is now `displayName` (same output, one implementation). Consumers resolve labels at render time rather than at file load, so the many unit sandboxes that load these files without `payload-labels.js` keep working. `index.html` already loads `payload-labels.js` before all of them. ## Tests - **New `tests/unit/test-payload-labels-surfaces.js`** (listed in `test-all.sh`): - drift guard: fails if any `public/*.js` other than `payload-labels.js` pairs a numeric id with a short label or firmware enum name (it caught the `map.js` copy, which was not on the original list); - `displayName()` for every enum, string ids, and the `Type N` fallback; - loads the real `node-analytics.js` in a vm sandbox and asserts the doughnut labels for ids 6, 10, 15 and 13. - Before the fix: 11 failures. After: 82 passed. - **`tests/e2e/test-issue-1799-label-vocab-e2e.js`**: two new steps intercept `/api/nodes/:pk/analytics` and `/api/observers/:id/analytics` (the fixture has no GRP_DATA in the store), inject ids 6, 10, 15 and 13, and read the labels from `Chart.getChart()`. Against upstream `master`: 11 passed, 2 failed (`["Type 6","Type 10","Type 15","Type 13"]`). With this branch: 13 passed. - **Sandbox updates**: `test-compare-overlap.js` and `test-observer-detail-ownership.js` now load `payload-labels.js` before the page script, as `index.html` does. `test-issue-1418-raw-hex-extraction.js` asserted the hand-rolled `PAYLOAD_TYPE_MAP` literal; it now asserts that `inferredType` uses the canonical map and that no local map exists. - Unit suites, each `test-all.sh` entry run on its own: 223 pass on this branch vs 222 on `master` (+1 new suite). `test-manage-workflow.js` and `test-preflight-xss-gate.js` fail identically on `master` in my Windows environment (CRLF in the shell fixture, no `python`); they are unrelated to this change. - E2E, run against a local server on the CI fixture (freshened, seeded and migrated as in `deploy.yml`): `test-issue-1799-label-vocab-e2e.js`, `test-charts-fluid-1058-e2e.js`, `test-issue-1147-section-order-e2e.js`, `test-issue-1640-compare-discovery-e2e.js`, `test-issue-1648-m2/m3-icons-e2e.js` and `test-observer-detail-ownership-e2e.js` all pass. `test-e2e-playwright.js` fails one step (`Version info lives on Perf dashboard`, `#navStats` wait timeout) with the same result on `master` locally. ## Performance No hot-path change. Every lookup is still one object-key read per packet: the packets-table sort accessor calls `displayName(t)` instead of `TYPE_NAMES[t] ?? ...`. No new loops, requests or allocations. ## Browser validation Checked against the local fixture server, with the analytics responses patched in the page to contain GRP_DATA: - Node Analytics → Packet Types legend: `Group Data, Advert, Channel Msg, Direct Msg, Multipart, Raw Custom, Type 13` - Observer detail → doughnut and the first Recent Packets row read `Group Data` - Observer comparison → packet table rows read `Group Data` - Map route deep link (`#/map?packet=…`) renders the route panel; no new console errors 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
6618820836 |
fix(nodes): show timestamp context for Last Contact (#2377)
Partial fix for #1154 U10. Peer Interactions Last Contact now shows timestamp context through the existing shared formatter: relative text with an absolute tooltip, or the reverse when absolute display is selected. Existing timezone, precision and custom-format preferences apply; missing/invalid values remain a dash. Other U10 surfaces and tracking items stay open. The production change only formats the existing cell. Work stays O(P) over the existing maximum 20 peers; no requests, settings, dependencies, caches or render triggers are added. Tooltip and text are HTML-escaped. RED `c47cce20` reached [11 unit assertion failures](https://github.com/Kpa-clawbot/CoreScope/actions/runs/42316165311/job/128229216777) and [six browser missing-tooltip assertions](https://github.com/Kpa-clawbot/CoreScope/actions/runs/42316165311/job/128229216764). GREEN `2c29eccb` keeps those tests unchanged and passes 82 unit assertions, 10 focused browser cases (parent rerun), and all 224 frontend suites. Existing browser smoke reports 149 passes and three fixture skips. One separate navigation timeout passed on an unchanged serial retry. Browser verified: disposable Go fixture at `http://127.0.0.1:14953`, desktop/phone; tests retain the real API response and supply deterministic peer contacts. Its unrelated missing battery-metrics table is a fixture limitation. E2E assertion added: `tests/e2e/test-issue-2218-node-analytics-stale-e2e.js:123`. [Final CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/42317025356) passes. Independent adversarial, UI and test-quality reviews found no must-fixes; Kent Beck Gate: PASS. --------- Co-authored-by: n30nex <6686206+n30nex@users.noreply.github.com> |
||
|
|
0e2ed89829 |
fix(packets): explain observer-report coverage (#2370)
Red commit: `e86294b6` ([four intended CI assertion failures](https://github.com/Kpa-clawbot/CoreScope/actions/runs/42009863423/job/127185896565)). Partial fix for #667, Part B. The Packets page now explains that its counts come from observer reports received by this instance—a partial view, rather than network-wide totals or device firmware counters. The note remains visible on phones and describes the table for assistive technology. This does not change TX/RX storage, counts, filtering or grouping, and leaves the rest of #667 open. It invents no contributing-observer number and makes no RX-only claim. No new settings, dependencies, requests or processing work. Validation on `08036329`: parent-run browser regression 10/10; required frontend unit suites 932 assertions. The RED run completes six existing cases and fails the four new missing-context assertions. Browser verified: disposable local fixture at `http://127.0.0.1:14943` (desktop, phone, populated/empty filters, grouping and refresh). E2E assertion added: `tests/e2e/test-issue-1122-packets-filter-ux-e2e.js`. Updated CI passes; independent adversarial, UI and test-quality reviews found no must-fix issues. --------- Co-authored-by: n30nex <6686206+n30nex@users.noreply.github.com> |
||
|
|
199df07c21 |
docs(proto): correct route_type values to match the decoder (#2363) (#2367)
Fixes #2363. The published contract described `route_type` as `0=DIRECT, 1=FLOOD, 2=reserved, 3=TRANSPORT`. Both decoders (`cmd/ingestor/decoder.go`, `cmd/server/decoder.go` — `routeTypeNames`) and MeshCore's `docs/packet_format.md` use: ``` 0 = TRANSPORT_FLOOD, 1 = FLOOD, 2 = DIRECT, 3 = TRANSPORT_DIRECT ``` Updated: - `proto/decoded.proto` — `DecodedHeader.route_type` and the `route_type_name` value list - `proto/packet.proto` — `Transmission.route_type` (same stale comment) - `docs/api-spec.md` — Packet Object `route_type` line (same stale comment) Comment/doc-only; no generated code in-tree, no behaviour change. Co-Authored-By: Claude <noreply@anthropic.com> Co-authored-by: SaarMesh-Bot <300107934+SaarMesh-Bot@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
c4dc46f349 |
test(e2e): wait for comparison focus before disposal assertions (#2343)
Red commit: `eedadc43` ([four intended CI assertion failures](https://github.com/Kpa-clawbot/CoreScope/actions/runs/40586787653/job/122480320591)). Fixes #2342. The disposal test could capture comparison results before the router's accessibility-focus frame added the heading tabindex, then mistake that legitimate setup change for a late observer response. Wait for the actual tabindex in attached state before capturing the baseline, including when the mobile heading is hidden. A controlled late-focus variant proves the regression. Full HTML equality, chart cleanup, route and control assertions remain intact; no production behavior or settings change. Current head `cb537ebe` includes current master while preserving the original RED/GREEN commits and newer progressive-render coverage. Local validation: ownership browser suite 60/60 (independently rerun by parent), navigation 60/60, branding 17/17 and all 224 standalone frontend suites. Browser verified: real local Go fixture server at `http://127.0.0.1:14944`. E2E assertion added: `tests/e2e/test-observer-detail-ownership-e2e.js`. Updated CI passes; independent adversarial, UI and test-quality reviews found no must-fix issues. --------- Co-authored-by: n30nex <6686206+n30nex@users.noreply.github.com> |
||
|
|
d1bf4ca042 |
fix(docker): build natively on arm64 hosts (#2359)
Building the image natively on an arm64 machine (a Raspberry Pi 4 or 5 on 64-bit OS, Apple silicon) fails at the first `RUN` of the build stage: ``` => ERROR [builder 2/40] RUN apk add --no-cache curl xz && ... 0.841 exec /bin/sh: exec format error ``` **Cause:** the Dockerfile declares `ARG BUILDPLATFORM=linux/amd64`. A default on that ARG overrides the value BuildKit sets for the machine doing the build, so an arm64 host pulls the amd64 `golang` image and can't run it. CI didn't catch it because its arm64 image is cross-built on an amd64 runner, where `linux/amd64` was correct anyway. **Fix:** declare `ARG BUILDPLATFORM` without a default. BuildKit, the default builder since Docker 23 and what buildx uses, fills it in on every host. amd64 builds and the CI cross-build are unchanged. **Workaround on current master:** `docker build --build-arg BUILDPLATFORM=linux/arm64 .` **Test:** `tests/unit/test-dockerfile-buildplatform.js` fails on the old Dockerfile and passes on this one. It checks three things: - `BUILDPLATFORM` has no default. - The Go builder stays pinned to `$BUILDPLATFORM`. - No stage hard-codes a platform. It's wired into `test-all.sh`. Found while building the image on a Raspberry Pi running Raspberry Pi OS Trixie (arm64) with Docker 29. --- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
|
9341ec744b |
fix(observers): honour node visibility in neighbour reports (#2357)
Fixes #2356. Observer neighbour reports now apply the existing node blacklist and hidden-name-prefix rules before returning displayed identities. Names are already batch-loaded, so the fix filters the request-owned slice without another SQL query. It preserves reported totals, timestamps, stored data, public/unknown neighbours and the existing reporter-only observer-blacklist semantics. An all-hidden list remains `[]`. The work is bounded by the existing 500-entry ingestion cap plus the existing visibility checks; no new configuration, frontend change or database write is introduced. The filter reuses the existing report slice. Validation: - [RED CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/40971452075/job/123789013428) at `7341b334` compiled and reached the four expected visibility assertions. - GREEN `b38e95e6`: focused native checks passed in 6.607 s; parent rerun passed in 1.865 s. - Full native server passed in 189.743 s; parent ingestor suite passed in 399.859 s. Existing neighbour frontend checks passed 7/7. - Tests use the actual registered router, direct-node 404 controls, casing/shared-name/unknown-node cases, metadata preservation and an unchanged stored report. Local standalone browser replay is unavailable because the fixture-server launch was blocked. Fresh GREEN CI passed, including server race checks and browser shards. Independent adversarial, security-boundary and TDD reviews found no must-fix items. No live-instance incident or deployment is claimed. --------- Co-authored-by: n30nex <6686206+n30nex@users.noreply.github.com> |
||
|
|
d083f9ade9 |
test: cover partial observer chart construction failures (#2354)
Partial follow-up for #1778; addresses the chart-cleanup test gap identified in the review of #2339. The observer detail unit harness now throws during the second or fourth chart constructor. Both cases require existing chart instances to be destroyed, metadata/calibration to remain visible, the error to be rendered as text, and a later time-window retry to recover without extra requests. Production cleanup already behaves correctly; this PR adds coverage only. The parent-run ownership suite passes 30/30. Removing the actual cleanup produces exactly the two intended assertion failures. The remaining unit sources and runner match the current-master baseline whose 222 frontend suites passed; only this updated unit file differs within that test set. TDD scope: no production change or new behavior. The negative control removes existing cleanup to prove the new assertions detect its loss. No configuration, dependencies or runtime performance changes. Exact-head CI passed, including the browser shards. Independent adversarial, correctness and TDD reviews found no must-fix items. Co-authored-by: n30nex <6686206+n30nex@users.noreply.github.com> |
||
|
|
b3a9f2a9e9 |
docs: make scheduled backups portable and bounded (#2353)
Fixes #2352. Follow-up to #2345.
The older deployment guide recommended copying a live telemetry
database. It now points to the consistent-snapshot procedure. The
scheduled-download example uses a trailing-X temporary filename, bounds
connection and total download time, and explains the weak-key denylist.
Validation: the parent reran the extracted command against eight
disposable HTTP cases: valid SQLite, unauthorized, server error, empty
content, HTML, corrupt SQLite, truncated transfer and timeout. Failed
downloads removed partial files and preserved existing completed
backups; the timeout exited 28. Shell syntax and diff checks passed. GNU
tools were exercised; BSD/macOS was not run natively.
TDD exemption: pure documentation, no application or repository test
files changed. No runtime/browser behaviour, dependencies or
configuration changed. Independent reviews are complete; the one
timeout-wording finding was addressed in
|
||
|
|
f8c3b32633 |
docs(api): describe readiness responses and ingestor liveness (#2358)
Partial follow-up for #1670. Documents public `GET /api/healthz`: 503/loading, current in-memory counts, the fixed legacy backfill compatibility object, and optional ingestor liveness timestamps. Receipt and handler-entry clocks have different reset semantics; handler entry does not prove a completed database write. The legacy object does not track real migration progress. The 200 response has a typed schema, including its dynamic source-name map. Only `/api/healthz` leaves the known-gap list (16 → 15). Handlers, configuration and frontend behavior are unchanged; formal non-200 response-schema work remains separate. Validation: - The new documentation assertion failed before metadata was added; actual handler-response checks passed. - Full native server/ingestor suites passed in 257.438 s / 452.613 s. - After review corrections, focused checks passed in 2.757 s and the parent rerun in 2.097 s, including exact compatibility values and retained-receipt/zero-handler timestamps. - Independent adversarial, correctness and TDD reviews completed; the parent verified both semantic corrections against actual call sites. - [Exact-head CI passed](https://github.com/Kpa-clawbot/CoreScope/actions/runs/40979351261), including browser shards. TDD exemption: documentation/API metadata only. Local standalone browser smoke was blocked; no local-browser or deployment claim is made. Production untyped-map count is unchanged. --------- Co-authored-by: n30nex <6686206+n30nex@users.noreply.github.com> |
||
|
|
faf6f792ae |
docs(release): v3.16.0 release notes and changelog entry (#2355)
Release notes for v3.16.0 (`docs/release-notes/v3.16.0.md`) and the matching `CHANGELOG.md` entry. The tag pipeline uses this file as the GitHub release body. Range `v3.15.0..af2e96d9`, 47 commits: 23 fix, 9 feat, 6 docs, 2 test, 2 perf, 2 ci, 1 ui, 1 deps, 1 chore. "Read this before upgrading" covers three changes: - relay hops refresh Last Heard only with a unique prefix and only on flood packets (#2334, #2308, #2298, #2336); - `users.db` schema v6 to v8 with user accounts on (#2306, #2348); - Go 1.26 for source builds and Alpine 3.24 in the image (#2297). Not verified: the mosquitto, Caddy and supervisor versions are taken from #2297's local build, not from a pull of the published image. Performance numbers are copied from the PRs, not re-measured. Docs only, no code change. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>v3.16.0 |
||
|
|
af2e96d92a |
fix(reach): queue a cold request up to 3 s for a build slot before answering 429 (#2249)
Follow-up to #2127, from @dborup's review there. The limiter, most of its tests and the docs text are dborup's (dborup/CoreScope@195ec4c3), adapted to master; the commit carries `Co-authored-by: dborup`. ## What changes `GET /api/nodes/{pubkey}/reach` still runs at most 2 uncached reports at once. What changes is the next uncached request: - **Before:** 429 at once, `Retry-After: 5`. - **After:** it waits up to 3 s for a slot. It gets 429 with `Retry-After: 2` and a JSON body only if no slot frees in that time. Unchanged: cached answers never wait, and concurrent requests for the same node and `days` share one computation and one slot. One addition beyond dborup's version: the shared computation runs on the context of the request that started it. If that request disconnects, while queued or mid-scan, the other requests sharing it that are still connected now get 429 instead of 500. The 3 s wait made that case more likely. ## Why dborup's three points on #2127: 1. An instant 429 fails two tabs, or a quick switch between 7/14/30 days, when the database is not under real pressure. 2. The page cannot recover from it, because `api()` retries only 503. The panel retry is a separate PR so that this one stays server-only. 3. No test covered the promised property: a cached request still answers while every slot is held. Waiting parks a goroutine, not a database connection, so the pool is protected exactly as before. ## What I checked - **Tests.** `make test` (all 16 modules), `make vet`, `gofmt`, `sh test-all.sh` and eslint (0 errors) all pass. The reach tests also ran 10 times under `-race`. - **New tests.** 14 tests in `node_reach_cold_limit_test.go`: dborup's 10 plus 4 new. `TestNodeReachConcurrentBuildCap` now drives the new limiter through the production router. Point 3 is `TestNodeReach_WarmHitSkipsColdLimit`. The 4 new tests are: - a queued request is served when a slot frees; - a cache filled just before the flight skips the limiter; - coalesced waiters get 429 when the starter disconnects, queued or mid-scan (two tests). - **Mutations.** I wrote 21 from the code before running the tests, and the tests caught all 21: instant reject, cap +1, release that leaks, slot taken before the in-flight recheck, busy → 500, Retry-After header or value, status 503, dropped context mapping, and others. - **Real binary.** Master `2d09335c` and this branch, run one after the other on a copy of a 10.5M-observation, 11 GB database. Each burst fires its reach requests at once, while `/api/nodes?limit=50` and `/api/packets?limit=50` are polled every 200 ms. | Burst (distinct uncached keys) | master | this branch | |---|---|---| | 3 keys, `days=7`, warm page cache (2 runs) | 2 × 200, 1 × 429 after 0.01 s | 3 × 200; the third answered at 4.1–4.3 s | | 6 keys, three of them `days=14` (2 runs) | 2 × 200, 4 × 429 after ≤ 0.01 s | 2 × 200, 4 × 429 after 3.0 s | | 24 keys, cold page cache | 2 × 200, 22 × 429 | 2 × 200, 22 × 429 after 3.00 s | | `/api/nodes` during every burst | median 1 ms, max 53 ms | median 1 ms, max 66 ms | ## Not checked - **Slow scans.** On a database this size, uncached scans took from about 9 s to over 60 s, so the 3 s wait only helps when page cache is warm or the database is smaller. Otherwise the result is the same 429, 3 s later. A 30-day uncached scan hit the 60 s `WriteTimeout` on both builds; that is the scan #2214's opt-in hop index addresses. - **Pool contention under a live store.** On this copy the in-memory store loaded 0 packets, so `/api/nodes` latency does not show contention under a live load. - **Platforms.** One machine (linux/amd64). No browser check: the server change has no UI, and the panel retry comes with its own PR. **Base.** Rebased on master `7c00a429`, which includes #2214; every test and mutation above was re-run on the rebased commit. The real-binary runs used `2d09335c`, just before #2214, on a database without the hop index. That is the default after #2214 too (`hopIndexDays` unset means off). With the index on, scans are shorter, so more bursts fit in the 3 s wait; I did not measure that. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- ## Update after review (commits `c65f4d3b`, `1d2cd99d`, `04e49854`) efiten asked how many requests may wait at once. With only the 3 s timeout, the number of waiting distinct keys grew with the request rate. n30nex wrote a cap (nullrouten0/Baymc-CoreScope#1, tracked in #2264). Its two commits are now on this branch unchanged, as a fast-forward. - At most 8 distinct cold requests wait for a slot. Any further one gets 429 with `Retry-After: 2` at once, before a timer is set. - A waiter gives its place back on every exit: it gets a slot, disconnects or times out. - Cached answers and same-key sharing are unchanged. Requests for a key that is already queued or building wait on that one place and are not counted. `04e49854` corrects the code comment that said the ceiling was independent of request rate. **What I checked on `04e49854`:** - The two new tests fail on `28da016f` and pass now. - 12 mutations of the cap, all caught: cap 7 and cap 9; the place never returned, returned before waiting, returned only on promotion, leaked on cancel, leaked on timeout; a full queue that blocks, returns the wrong error or skips the counter; a timer armed before the check; a queue sized by the slot count. - `make test`, `make vet`, `make fmt-check`, `sh test-all.sh` and eslint 8.57.1 (0 errors) pass, and the reach tests pass 5 times under `-race`. On a local merge with current master `660cbbf5`, which includes #2277, the reach tests pass 3 times under `-race` and the `cmd/server` package passes. - Real binary: both heads, alternately, on a copy of a 2.5 GB soak database, with 24 distinct uncached `days=7` requests fired at once, two runs each: | | `28da016f` | `04e49854` | |---|---|---| | 200 | 2, 2 | 2, 3 | | 429 at once | 0, 0 | 14, 14 | | 429 after 3 s | 22, 22 | 8, 7 | **Not checked:** a sustained flood, a burst of same-key requests against the cap, `/api/nodes` latency during these bursts, and platforms other than linux/amd64. The real-binary table further up was measured before the cap. --------- Co-authored-by: nullrouten0 <201262769+nullrouten0@users.noreply.github.com> Co-authored-by: dborup <kontakt@meshview.dk> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: n30nex <blushellneon@outlook.com> Co-authored-by: efiten <erwin.fiten@gmail.com> |
||
|
|
bb3c8cc001 |
fix(rx-coverage): My coverage and the observer filter on the noise layer (#2326)
"My coverage" and a selected observer only narrowed the signal layer on the coverage page. On the noise layer the toggle stayed pressed, but `/api/rf-noise` was called without `mine` or `rx` and returned every companion's samples. ## Changes - **Backend:** `/api/rf-noise` now accepts `rx` and `mine=1`, with the same session rules as `/api/rx-coverage`: 401 without a session, 403 for a device token, 404 with user management off. The `rx_pubkey` filter moved into 1 helper, `appendRxFilter` (`cmd/server/rx_dashboard.go`), which both queries use. Both parameters are in the OpenAPI entry. - **Frontend:** `public/rx-coverage.js` fetches both layers through `fetchLayer`. It builds the URL with the time window, observer and toggle, and judges the reply with the existing `coverageReply`. The request sequence (`signalSeq`, now `drawSeq`) covers both layers, so the noise layer also drops a superseded reply. Before, a slow unfiltered noise reply could overwrite a newer filtered one. ## Performance The noise query gets 1 extra `WHERE` clause on `rx_pubkey`. There are no new requests: the page still makes 1 request per redraw. ## Verification - New Go tests in `cmd/server/rf_noise_mine_test.go`: `rx` narrows the result (case-insensitive), `mine=1` returns 401 without a session, returns empty with nothing linked, returns the linked companion's cells, returns 403 for a device token, and returns 404 with user management off. Both failed before the fix. - `tests/unit/test-rx-coverage-mine.js` is updated for the new `coverageUrl(path, ...)` signature and checks that both layers fetch through `fetchLayer`. - New step in `tests/e2e/test-rx-coverage-noise-e2e.js`: a `?layer=noise&rx=...` link sends `rx`, and clicking "My coverage" refetches with `mine=1`. It fails against the current master frontend and passes with this change. - Local runs (Windows): `go test` and `go vet` in `cmd/server`, `test-all.sh`, and the noise, gaps and viewport E2E in Chrome. All green. I did not run the full Playwright suite locally. ## Not in this PR - The "nothing received" gap cells still skip `mine=1`, unchanged: the track query cannot narrow to a set of companions yet. - The time window (`days`) already worked on the noise layer, so it is not touched. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
417c8dd00a |
feat(observers): latest /neighbors report on the observers list and page (#2270, #2271) (#2300)
Closes #2270, closes #2271. The ingestor took only the confirmed region scopes out of an observer's `/neighbors` report (#1865) and dropped the rest. Nothing could therefore say when an observer last reported, or whom it heard. ## Ingestor `observer_neighbor_reports` (created in `dbschema.Apply`) keeps the latest report of each observer, one row per observer, replaced by every live report. The report is unauthenticated, so `neighbors[]` is stored sanitised: - only 64-hex keys - `snr` and `heard_secs_ago` finite and in range - `status` limited to `[a-z_]` - `scopes` clamped - at most 500 rows `total_neighbors` is believed only when it is not below the number of rows that arrived. A **retained replay** only fills a missing row, and it is dated by the report's own `timestamp`. After an ingestor restart, a long-dead observer therefore does not look as if it just reported. ## API The table is optional for the server (no `AssertReady` change); without it the fields are simply absent. - `GET /api/observers`: `neighbors_report_at` and `neighbors_count`, from one batch query in the existing cached builder. - `GET /api/observers/{id}`: also `neighbor_report`, which is `{received_at, reported_at, neighbor_count, queried_neighbors, self_scopes, self_default_scope, neighbors: [{pubkey, name, role, snr, heard_secs_ago, status, scopes}]}`. Names and roles come from the node table in one query. ## Frontend `public/observer-neighbors.js` is shared by both pages and escapes everything. - **#2270, observers list:** a sortable **Neighbors** column, e.g. `3h ago · 12`, or `—` when the observer never reported. The same value appears in the slide-over. - **#2270, observer page:** **Neighbors Report** and **Neighbors** stat cards. - **#2271, observer page:** a **Neighbors** table listing each neighbor by name (a link to the node) or key prefix, with SNR, heard, scope query status and scopes. The heading says when the list is shorter than the count, e.g. `(2 of 3 listed)`. ## Tests - **ingestor:** sanitising, the cap, live replace, the retained rules (fill only, dated by the report, undated ignored) and the MQTT topic end to end. The existing #1865 scope tests are unchanged and still pass. - **server:** the list summary, the detail with names, and both endpoints without the table. - **frontend:** - unit tests of the helpers - an XSS case for every rendered field; I checked that it fails when escaping is removed - `test-observers-headings` now expects the new column after `Last Packet` - **E2E:** `test-observer-neighbors-2270-e2e.js` intercepts the observer endpoints, since the fixture has no reports. It is listed in `scripts/non-unit-tests.json` and on shard 2. Gates that pass: - `go test -race` in `cmd/server`, `cmd/ingestor` and `internal/dbschema` - `sh test-all.sh` - `eslint@8`, with 0 errors - the XSS diff gate - these E2E suites: the new one, `test-issue-1639-observers-sort-e2e.js`, `test-observer-iata-1188-e2e.js` and `test-e2e-playwright.js` Performance: the list adds one primary-key-sized query per cache fill, and the detail adds two small queries per request. Also fixes #2330 (folded in from #2337, second commit). 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
202bafea28 |
fix(observers): show metadata while analytics loads (#2339)
Partial follow-up to #1778, limited to observer details; other pages remain separate. Observer identity, status and radio information now render when the basic request finishes. Analytics and clock calibration load into their own regions, so either can finish first. An analytics failure preserves usable metadata. Identity failures and obsolete responses cannot overwrite the current view, and failed chart construction is cleaned up. The same three requests start together. Generation ownership, request deduplication, date-window behavior and disposal remain covered. No new fetches, settings, dependencies or customizer options; secondary replies preserve the actual metadata DOM node. [RED CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/40426343054/job/121932999182) proves 14 new assertion failures while all 11 existing unit cases pass. Local RED browsers reproduce 20 new failures while all 32 old cases pass. GREEN `52eabad7` passes 221 standalone suites, 28 ownership unit checks and 56 browser cases across desktop/mobile and both themes; the parent independently repeated the 56 browser cases. All three independent review roles passed. Held-response assertions prove the rendering behavior without claiming a server-latency speedup. [Final CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/40427087948) passes on this exact head, including all three browser shards, the repeated slide-over stability gate and the architecture build. --------- Co-authored-by: n30nex <6686206+n30nex@users.noreply.github.com> |
||
|
|
1ed5e4ab3f |
fix(compare): exclude observer TX-only packets from overlap (#2346)
Red commit: `39295da8` ([intended CI assertion failure](https://github.com/Kpa-clawbot/CoreScope/actions/runs/40605770951/job/122544822813)). Green commit: `d1bda496`. Observer comparison counted an observer's TX-only packets as receptions, producing false overlap. It now requests `excludeTx=true`: the selected observer and non-TX direction must match the same observation in memory and SQLite v2/v3. Normal packet lists and stored TX records remain unchanged; missing directions retain legacy compatibility. Refs #667. This completes a comparison slice, not the issue's remaining observer counters or #671's fleet/SNR feature. Representative display fields and existing time-filter semantics are unchanged. API/OpenAPI docs describe the filter and multi-node bypass. Validation: - RED: Go tests compiled with 32 intended failures/16 control passes; unit and browser regressions reproduced false overlap. CI reached the new unit assertion. - GREEN: all 48 packet cases pass, plus grouped-cache isolation and OpenAPI tests. The added cache regression caught a malformed-radio cache-key collision before correction. - Full local server suite: 174s; ingestor: 185s; `npm run test:unit`: pass. Final focused packet/grouped/OpenAPI checks: pass. - E2E assertion added to `tests/e2e/test-issue-1640-compare-discovery-e2e.js`; all 10 checks pass, including correct 1/2 reception totals and 50% overlap. Performance: still two capped bulk requests, indexed candidate lookups retained, one direction check within the existing observation scan/SQL predicate, no new persistent structures. No configuration/customizer changes. Final-head CI passed; independent reviews found no must-fix items. --------- Co-authored-by: n30nex <6686206+n30nex@users.noreply.github.com> |
||
|
|
c211ef7643 |
feat(nodes): show relay activity from certain forwarding evidence (#2336)
Fixes #2331 with the additive API option. The Last Relayed row shows green **actively relaying** only when `last_certain_relay` falls within the existing `healthThresholds.relayActiveHours` window. Missing, invalid, future or expired evidence adds no idle/alive claim. Existing `relay_active`, attributed timestamps and counts retain their meanings. The optional node list/detail field uses the existing certain-relay path: an eligible observed frame, matching hop, unique prefix, no retired namesake and existing signature checks. Originating an advert does not count. Forwarding another node's advert can count, even with zero non-advert attributions. Adverts with an unknown origin cannot establish forwarding. Evidence uses transmission `first_seen`, not late observation receipt time, and follows node-membership eviction. Snapshots take one read lock over requested keys, O(page size), with no packet scan or new database query. Work per qualifying hop stays bounded. No new dependency or setting; configuration documentation and OpenAPI explain the distinction. The test-first history includes assertion-failing RED commits and GREEN fixes. The initial RED's nil-return compile stub is explicitly permitted by `docs/agents/TDD.md`. Local native server/ingestor suites, focused race checks, 809 frontend helper assertions and 70 browser cases pass. Browser coverage includes both themes and full/side/tablet detail views. [Review regression RED](https://github.com/Kpa-clawbot/CoreScope/actions/runs/40421464618/job/121916723339) proves both malformed-origin failures. All three independent review roles pass at `d1fe79cb`. [Final CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/40422655728) passes on that exact commit, including all three browser shards and the two-architecture image check. --------- Co-authored-by: n30nex <6686206+n30nex@users.noreply.github.com> |
||
|
|
1e1bfa0255 |
fix(ingestor): require unique hop prefixes for relay liveness (#2334)
Fixes #2329 (option 1: unique-prefix liveness). The ingestor could select a likely relay through neighbour adjacency and refresh its `last_seen` even when its prefix matched two nodes. Contextual resolution remains available in stored `resolved_path`; liveness now uses the existing unique-prefix resolver, sharing the parsed path and prefix-index snapshot. The current flood-frame eligibility gate remains unchanged. Retired-prefix catalogue parity with the server is separate. The native SQLite regression exercises real `InsertTransmission`: the displayed path still selects the adjacent candidate, neither colliding relay is touched, and the unique preceding hop is refreshed. Existing direct/TRACE/flood checks remain intact. [RED CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/40407478987/job/121868990704) fails the timestamp/touch-count assertions; [GREEN race CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/40409860054/job/121877671472) passes at `77be50e8`. Full local server and ingestor suites, repeated focused race checks and all 221 frontend suites pass. Existing test assertions are unchanged; one misleading resolver comment is corrected. Frame/path assumptions were checked against MeshCore firmware revision `81dbae59`. Performance: one additional indexed lookup per hop, O(h), with no node scan or extra database query. No schema, dependency, configuration or customizer change. [Final CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/40409860054) passes on `77be50e8`, including the race detector, all browser shards and architecture builds. Three independent reviews found no blockers; final repository checks pass. Ready for maintainer review. |
||
|
|
f9b7c66a8b |
fix(ingestor): open buffered writes before startup packet retention (#2328)
Partial fix for #2325; leave the issue open for the remaining startup work. MQTT subscribed early but its buffer could not drain until the entire packet-retention backlog finished. Required startup work and the migration barrier still finish first; ingestion now opens before the initial packet sweep, so buffered writes can use the gaps between the existing 250-transmission transactions. The sweep remains synchronous with its count/error logging. No new worker, shutdown owner, setting, retention window or batch size. The readiness log reflects the actual gate opening. Earlier non-packet maintenance, initial vacuum ordering and diagnostics publication during maintenance remain unchanged. Native SQLite tests exercise blocked migrations, inter-batch writes, FIFO/drop accounting, stopped buffers, callback counts, rollback/retry and main-function wiring. Removing the migration wait fails the regression. Existing tests are unchanged; GREEN strengthens the new RED test with a blocked-gate assertion and early-failure cleanup. [RED CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/40308382930) fails the intended assertions. [Final-head CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/40311222867) passes on `f8a4d175`, including Go/JS tests, race checks, all browser shards and architecture builds. Full local ingestor tests, repeated focused race checks and local browser/API smoke checks pass. Three independent reviews found no blockers in this partial scope. This removes the whole-sweep startup gate; it does not bound an individual deletion transaction or claim a measured live latency improvement. |
||
|
|
ef17d78909 |
docs(api): document node resolution and battery history (#2338)
Partial follow-up to #1670; the broader documentation issue stays open. Documents the existing node-prefix resolution and node battery-history GET APIs: parameters, normalization and lookup rules, defaults, ambiguity/visibility, ordered samples, nullable latest values and error media types. The generator now has concrete success schemas for both. Error statuses remain documented in prose under its existing convention. Removes exactly these two entries from the known-gap list: 18 → 16, with 115 of 131 routes described. No handler, query, configuration or dependency changes. [RED CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/40426035729/job/121931981058) reaches the missing-documentation assertions. GREEN `521fb3ef` passes both full native CGO suites and focused OpenAPI/real-handler contract tests. Parent browser acceptance passed 28 checks through the rendered Swagger UI and actual endpoints, including defaults, errors and empty/null history. All three independent review roles passed. [Final CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/40427081025) passes on this exact head, including all browser shards and the two-architecture image check. --------- Co-authored-by: n30nex <6686206+n30nex@users.noreply.github.com> |
||
|
|
8437d22356 |
docs: use consistent snapshots for backup and restore (#2345)
The deployment guide copied a running `meshcore.db`, which can omit committed WAL data or produce an inconsistent backup. It now uses the existing authenticated `/api/backup` snapshot, checks the download before publishing it, and explains how to restore with all database users stopped while preserving the old database and sidecars for rollback. Fixes #703, following the [maintainer's remaining documentation scope](https://github.com/Kpa-clawbot/CoreScope/issues/703#issuecomment-6163256620). - Protects API credentials in a header file and keeps incomplete downloads out of the completed backup set. - Links the separate account-backup guide; configuration and secrets remain separate backup responsibilities. - Replaces unsafe live-copy scheduling instructions with the same checked download procedure. Validation: - Exact shell example with real curl/SQLite: valid download, HTTP 401/500, empty, HTML, corrupt database, and truncated transfer. Failures preserved the previous backup and published no new `.db`. - Existing `TestBackup*` passed. Chromium downloaded an authenticated snapshot from a disposable instance; a row committed only in its nonempty WAL survived and `quick_check` returned `ok`. - Disposable stopped-file restore and rollback recovered the original WAL-only row and preserved the original files byte-for-byte. - `git diff --check` and privacy preflight passed. TDD exemption: pure docs; no application behavior changed and no test files modified. No new config/customizer settings. Live operator services were not used for validation. Co-authored-by: n30nex <6686206+n30nex@users.noreply.github.com> |
||
|
|
15e2d2a4a7 |
fix(packets): read HB from the row's own raw_hex header (#2351)
## Problem The HB column on the packets page can show `4`. Path mode 3 (hash size 4) is reserved: `Packet::isValidPathLen` in `src/Packet.cpp` returns false for it, and `Dispatcher::checkRecv` drops such a packet. So HB 4 is never a real value. The three HB cells in `public/packets.js` found `path_len` with `getPathLenOffset(p.route_type)`, the route type of the **transmission**. The raw_hex in a child row (and in a live-updated row) is an **observation's** raw_hex. The packet hash does not cover the route bits, so the same packet is heard as FLOOD by one observer and as TRANSPORT_FLOOD by another, and the transmission keeps the route type of its first observation. Example from a production DB (transmission 3645334): | source | raw_hex start | path_len at | |---|---|---| | transmission | `01 12 3A646A…` (FLOOD) | byte 1 | | observation | `00 E2E0 0000 1D 3A646A…` (TRANSPORT_FLOOD) | byte 5 | Read at the FLOOD offset, the observation gives `0xE2`, `0xE2 >> 6 = 3`, `+ 1 = 4`. In the last 300,000 observations of that DB, about 4,300 have a header route type that differs from their transmission's. ## Fix `pathHashSize(rawHex)` in `public/app.js` already takes the route type from the header byte of the same raw_hex, and the detail pane uses it. The three HB cells now call it through a small `hashBytesText` helper instead of carrying their own copy of the offset logic. Behaviour change: a raw_hex with no hash size (reserved mode 3, a DIRECT packet with no hops, missing raw_hex) shows the same placeholder as TRACE rows instead of a number. Before, missing raw_hex showed `1`. ## Tests - New `tests/unit/test-hb-column-header-route.js` (7 cases, registered in `test-all.sh`): transport raw_hex on a FLOOD transmission gives 1, flood raw_hex with 2-byte hashes on a TRANSPORT_FLOOD transmission gives 2, reserved mode 3 never gives 4, for the helper, the flat row and the group header. It fails on `upstream/master` with `'4' !== '1'`. - `sh test-all.sh`: all standalone frontend suites pass. - Browser: local server on a migrated copy of the E2E fixture, with one observation rewritten to a TRANSPORT_FLOOD raw_hex. The expanded group shows HB 2 on all three rows. The old code reads `0xE2` on that child and shows 4. ## Performance Same per-cell work as before: one header parse and one byte parse per rendered row, O(1), no new allocations beyond the existing regex test in `pathHashSize`. Only visible rows are rendered. ## Not done - The child-row call site is not hit by a unit test: child rows only render when the group is in `expandedHashes`, which the test API does not expose. It is covered by the helper test and the browser check above. - The HB sort accessor (`hash_byte_count` / `hash_size`, sort key `hb` in `packets.js`) is unchanged; it does not read raw_hex. - `transportBadge(c.route_type)` on child rows still follows the stored route type, not the observation header. Left out of scope. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
b65412ab83 |
feat(observer-logins): turn on neighbour reports in the firmware block of the credentials mail (#2349)
The firmware block of the observer credentials mail now ends with `set
mqtt.neighbors on`, so firmware observers send the same `/neighbors`
report that the OpenHop block already turns on ("Publish Neighbours",
#2347). The ingestor reads that report for region scopes (#1865).
## Changes
- `cmd/server/observer_mail.go`: one extra line in the firmware block,
and one sentence in its note. The sentence says the command sends the
neighbour table and region scopes once a day, and that older firmware
does not know it and should skip it.
- `cmd/server/observer_mail_test.go`: the full firmware block assertion
includes the new line, and the slot test checks the line and the note.
- `docs/observer-mqtt-logins.md`: the mail section describes the new
line.
## Tests
- `go test ./...` in `cmd/server`: ok. Skipped
`TestSaveGeoFilterPreservesFileMode`, which needs Unix file modes; this
run was on Windows.
- `TestObserverCredentialsMail*` failed before the change, on the
missing line, and passes after it.
## Not checked
- The command name and the 24-hour default come from #1865. I did not
read the observer firmware source.
- The wording of the error older firmware gives for an unknown command
is not quoted in the mail. The note only says to skip the line.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
ddce76a685 |
ci(e2e): run the E2E shards on ubuntu-24.04 and fail on a broken browser install (#2350)
`ubuntu-latest` now hands out `ubuntu-26.04` runners. Every E2E shard that lands on one fails before a single test runs: ``` browserType.launch: Executable doesn't exist at /home/runner/.cache/ms-playwright/chromium_headless_shell-1208/... ``` The step "Install Playwright browser" runs `npx playwright install chromium 2>/dev/null || true`, so its own failure is invisible and the job only breaks at the first browser launch. ## Evidence - Run 40779894898 (PR #2349, a mail-text change): three attempts, each failing on `Executable doesn't exist`. The job log shows `Image: ubuntu-26.04`, OS `26.04.1 LTS`. - The master run at 2026-10-28 21:31Z passed. Its shard log shows `Image: ubuntu-24.04`. - `playwright` in package-lock.json is 1.58.2. ## Change - `e2e-shard` gets `runs-on: ubuntu-24.04`, with a comment saying when to unpin. The other jobs do not install Playwright and stay on `ubuntu-latest`. - "Install Playwright browser" no longer has `2>/dev/null || true`, so a failed install fails at that step. ## Not done - I did not find out why the install fails on 26.04: its stderr was discarded. Once this is merged, a run with the pin removed would show it. - No Playwright upgrade. That is a separate change, and it may make the pin unnecessary. - Before this change, `install-deps` errors were also swallowed. If it fails on 24.04, this PR's own CI will show it. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
cadc666bd9 |
feat(observer-logins): OpenHop block in the credentials mail (#2347)
The observer credentials mail (#2306) had blocks for meshcoretomqtt, the Gessaman/Dutch-MeshCore firmware, Home Assistant and plain values. An operator running OpenHop (repeater software with its own MQTT uploader) could not tell which of these matched its broker form, and picked the format by trial. ## Changes - `cmd/server/observer_mail.go`: a fifth block, "4. OpenHop (repeater software)", before the plain values. It uses the field and option labels of the OpenHop broker form: - `Format: Letsmesh`, `Transport: TCP` or `Websockets` (from `observerLogins.transport`), host, port, username, password, `TLS: on/off`, `Use JWT Auth: off`, `Retain Status: on`, `Publish Neighbours: on`. - The note says Meshcoretomqtt works too, but the format labelled "openHop" (value `mqtt`) does not: it publishes to `meshcore/repeater/<name>`, which a personal login may not use. - Publish Neighbours is on because the ingestor reads the `neighbors` topic for region scopes (#1865). - `openapi.go` and `docs/observer-mqtt-logins.md`: "four formats" becomes five. ## Sources Read from OpenHop, not tested on a device by me: - format family and topics: `openhop_repeater/repeater/data_acquisition/mqtt_handler.py` (`MC2MQTT_FORMATS = ("meshcoretomqtt", "letsmesh", "waev")`, the `mqtt` format's `meshcore/repeater/<node_name>` base topic, transport values `tcp`/`websockets`); - labels: the built web UI (`Configuration-*.js`: "Add Broker", option labels `Letsmesh`, `Meshcoretomqtt`, `openHop`, `TCP`, `Websockets`, "Retain Status", "Publish Neighbours", "Use JWT Auth"). An operator on our instance set it up from a screenshot of that form with format Letsmesh, TCP, Retain Status and Publish Neighbours on; the observer publishes, and its `neighbors` report set `configured_scope` on its node. ## Tests - `TestObserverCredentialsMailOpenHopBlock`: the block text for TCP without TLS and for Websockets with TLS, the "openHop" warning, HTML escaping, and the block order. - `TestObserverCredentialsMailBlockTitles`: five titles in order. - `cmd/server` suite green locally (the two Windows-only failures skipped, as on master). ## Not in this PR - The websocket path: OpenHop uses paho's default, the doc already tells operators to match the listener path. - Anything with the other fields of the `neighbors` report (SNR, heard_secs_ago, default_scope): we are measuring how many observers send it first. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |