mirror of
https://github.com/Kpa-clawbot/meshcore-analyzer.git
synced 2026-09-25 22:23:50 +00:00
2cfe9cbbc5fe956f681b1be7cf1764fd7d7addde
266
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 efitenandClaude Opus 5
|
25f8426d32 |
test: make every suite in tests/e2e run, and tell the truth when it does (#2053)
Closes #2037. Step 1 (#2045) wired in the nine suites that already passed. This is steps 2 and 3: the four that ran and failed, and the five that could not run at all. After this, the count of suites in `tests/e2e` invoked by nothing goes from 18 to 0. ## Step 2 — the four that ran and failed Triaged against a fixture server in CI with full output kept, not the four-line tail the first probe saved. | suite | verdict | |---|---| | `test-packets-scope-column.js` | passes on master today. It failed on 2026-09-18, so something between the two fixed it. Wired in unchanged rather than investigated. | | `test-node-reach-e2e.js` | test wrong. It waited for the reach map whenever any *link* had GPS; `public/node-reach.js` builds the map only when the *node* has coordinates. Guaranteed 10s timeout on a node with positioned neighbours and no position of its own. | | `test-channel-modal-e2e.js` | both failures test-side. The Add button's visible label was shortened to "+ Add" with the accessible name moved to `aria-label` (`public/channels.js:746`); and `.ch-section-mychannels` is conditional on the visitor having added a channel, which has not happened at that point in the suite. | | `test-touch-targets.js` | three test-side, two a product finding. | The three test-side touch-target failures were the harness measuring controls that are not shown: `.compare-btn` (the CTA was removed in #1646, and `style.css` says so), `.ch-back-btn` (`display:none` outside the mobile channels layout), and `.filter-toggle-btn` (`display:none` on mobile since #1461; the control shown is the navbar mirror, which `mobile-page-actions.js:70` builds as a `.nav-btn`, so it was already measured). All three are dropped from the table with the reason recorded in the file. The remaining two are **not** a test problem: `.nav-btn` and `.ch-icon-btn` are each declared twice in `public/style.css`, 48px in the touch-target block and 44px in their own component rule, and the later one wins. Rather than lower the blanket or hide the failures, the suite now has `DEFAULT_MIN = 48` plus a `MIN_OVERRIDES` table holding those two at their effective 44, so a third selector dropping to 44 still fails the build. The contradiction is **#2052**, with both ways out costed; the override entries should go when it is settled. ## Step 3 — the five that could not run None is deleted. I checked each selector and seam against the product before deciding, and every one still targets a surface that exists and that nothing else covers. Four were written against `@playwright/test`, a runner the project neither installs nor uses anywhere else. Adopting a second runner for twelve tests costs more than porting them, and the precedent is already set: `test-path-inspector-coverage-e2e.js` exists, as its own header says, because `test-path-inspector-e2e.js` could not run. So they are ported to the plain-node Chromium pattern the other 109 suites use. - **`test-issue-1522-trace-url-sync-e2e.js`** — the trace hash in the URL, both directions. `test-e2e-playwright.js` covers that the page loads and searches; it never looks at the URL, which is the whole of #1522. - **`test-marker-outline-weight.js`** — the canvas pulse ring never thins below 2px. There is no CSS rule to read and axe cannot see inside a canvas, so sampling the seam is the only way. Added a guard that the ring was actually visible, so the weight check cannot pass vacuously on a pulse that never rendered. - **`test-pr-1490-live-map-gpu-animations-e2e.js`** — the queue drains, the engine sleeps again, the fading trails stay under the cap of 5, and the canvas sits on `animationsPane` rather than under the markers. - **`test-path-inspector-e2e.js`** — reduced to what nothing else covers: the map side pane, the `/#/traces/<hash>` redirect, the tools landing. Its standalone-page test duplicated the wired coverage suite and is dropped. Its "switching candidate clears prior polyline" case ended after the click with a comment and no assertion, which is the same green-but-empty problem this issue is about; it now compares path counts, and skips loudly when the fixture yields too few candidates. The fifth, **`test-table-sort.js`**, needed `jsdom`, which was declared nowhere. It is a unit test of `public/table-sort.js` filed under `tests/e2e`, so: `jsdom` is a devDependency (lockfile updated, `npm ci` stays consistent), the file moved to `tests/unit/`, and the `domIntegration` group in `scripts/non-unit-tests.json` is gone with its only member. It runs 22 tests. 20 passed immediately; 2 had rotted, because #1648 M2 replaced the up/down glyphs with Phosphor sprites and the direction moved out of `textContent` into the `<use href>`. Those two now read the sprite ref and the `aria-sort` value, so they also guard the accessible announcement. ## Verification `tests/unit/test-table-sort.js` 22/22 and `test-test-inventory.js` pass locally; the E2E suites need a fixture server, which I cannot build here (no cgo toolchain since #1992), so CI is their first run as committed. The triage above was measured in CI, not assumed. ## Not done The per-assertion skips named in the second comment on #2037 are untouched: the two flaky packet-detail cases, the fixture-data ones, and the two `clientRxCoverage` suites that skip wholesale while reporting success. Those need a fixture deployment with coverage enabled, which is its own change. I have not opened it. Option 2 from the issue, making `test-test-inventory.js` require a `deploy.yml` line for every `tests/e2e` file, is also not here. It is the right guard and it is now enforceable, since the list is finally at zero, but it belongs in its own change where a red build means what it says. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
efitenandClaude Opus 5
|
5c016a210a |
fix(analytics): show a building state on the distance index's 202, and stop caching it (#2051)
Closes #1997. ## What was wrong `/api/analytics/distance` answers `202 {status:"building", retry_after_seconds:5}` with no `summary` until the lazy index (#1011) has been built. 1. `renderDistanceTab` read `data.summary.totalHops` straight away. The TypeError is caught by the tab's own try/catch, so it never reaches `window.onerror`: it is painted into the tab as `Failed to load distance analytics: Cannot read properties of undefined (reading 'totalHops')`. 2. `api()` caches any `res.ok` body, and `res.ok` is true for 202, so the placeholder was stored for the `analyticsRF` TTL. Even a correct retry read the cached "building" body back. That is the half that made the broken state outlast the index build. ## What changed - `public/app.js:173` skips the cache write when `res.status === 202`. A 200 still caches, unchanged. - `public/analytics.js` renders a building notice and retries itself, honouring `retry_after_seconds` clamped to [1s, 30s]. The timer is cleared on tab switch and in `destroy()`, and a new render supersedes a pending retry, so two renders cannot write into the same tab. The notice uses `.text-center`/`.text-muted` rather than the `.spinner` class used at `analytics.js:1683`, because `.spinner` has no CSS anywhere in the repo and renders nothing. ## Tests `tests/unit/test-issue-1997-distance-building.js` (7 assertions, wired into `test-all.sh`) pins the two pure decisions the renderer makes and `api()`'s refusal to cache a 202 while still caching a 200. Red-run on the unfixed sources: 6 of 7 fail, and the "a 200 is still cached" control stays green. `tests/e2e/test-issue-1997-distance-building-e2e.js` (classified in `scripts/non-unit-tests.json`, invoked from `deploy.yml` with `CHROMIUM_REQUIRE=1`) serves both responses by route interception, so it does not depend on whether the server under test has an index built. It asserts: the building state appears, the tab does not paint the error text, a retry arrives with no interaction, the retry replaces the placeholder once the server answers 200, and no retry fires after leaving the tab. Check (2) deliberately asserts on the rendered text and not on `pageerror`: the TypeError is caught, so a `pageerror` assertion would pass on the broken build too. ## Verification No local cgo toolchain here since #1992, so I could not build a server to run the E2E against. Instead I ran its five steps in Playwright against a live instance with this branch's `public/app.js` and `public/analytics.js` injected in place of the deployed ones (both files are byte-identical between that instance and upstream master, so the injection is faithful): | check | deployed build | this branch | |---|---|---| | (1) building state shown | fail | pass | | (2) not rendered as data | fail | pass | | (3) retried on its own | fail (1 request) | pass (2 requests) | | (4) real payload after retry | fail | pass | | (5) no retry after leaving the tab | pass | pass | (5) passes on the broken build too: it schedules no retry at all, so it is a control and only means anything together with (3). The committed E2E suite has not been run as committed. CI is its first real run. ## Not done - The server still recomputes on every 202 poll rather than signalling readiness. - No other analytics tab was audited for the same assume-a-summary pattern. - One pre-existing unit suite (`test-preflight-xss-gate.js`) fails on this Windows machine with a cp1252 `UnicodeEncodeError` from its Python helper, on a clean tree as well as with this change. Unrelated, and green on Linux CI. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
liquidraver
|
84eba41592 |
fix(live): anchor the legend toggle to .live-page so the VCR bar stops eating its clicks (#2049)
The PACKET TYPES legend could not be dismissed on Live: clicking the palette toggle did nothing, while document.querySelector('#legendToggleBtn').click() from the console worked. That asymmetry was the diagnosis, since a synthesised click skips hit testing.
Residual half of #1833. That fix corrected the button's offset to calc(var(--vcr-bar-height) + 10px) but left the anchor: .legend-toggle-btn was position: fixed, so the offset resolved against the viewport, while the .vcr-bar it clears is absolute inside .live-page, whose height subtracts --bottom-nav-reserve (56px + safe-area at <=768). The two anchors disagreed by exactly the reserve, dropping the button into the bar's band, and .vcr-bar at z-index 1000 against the button's 500 took every real click. The window is 641-768px: below 641 both buttons are display:none, above 768 the reserve is 0, which is why neither the desktop nor the phone layout test saw it.
Both buttons are now position: absolute, resolving against the same containing block as the bar. .feed-show-btn carried the identical defect but missed the bar by 16px because --legend-toggle-stack parks it a row higher, so that change prevents a break rather than repairs one.
Three review rounds, each verified rather than argued:
- The new E2E was classified in non-unit-tests.json but had no deploy.yml line, so it would never have run (#2037). Now wired in, and it covers both buttons.
- The '.feed-show-btn is display:none at <=768' claim was wrong (it is <=640), which the author measured and corrected.
- Its first real execution failed on its own setup: getComputedStyle().getPropertyValue() on an unregistered custom property returns the unevaluated declaration ('calc(56px + 0px)'), so the non-zero check could not work. It now asserts the measured gap instead, and deliberately does not pin 56 because the gap measures 58 on CI.
Final run: 14 passed, 0 failed. The discriminating assertion is (f), each button's offset net of the bar being identical at 720 and 1440 (11px and 123px slack); under the old anchor those differ by the reserve. (g) pins the desktop layout unmoved.
Merged by the interim maintainer without a second human reviewer: CI and the review above are the independent checks.
Fixes #1833
|
||
|
efiten
|
29c0a3ba2f |
ci: name each E2E suite in the log before it runs (#2046)
The E2E steps invoke 105 suites in a row. GitHub echoes a step's whole script once at the top, so the per-suite output then runs together with nothing between it, and a line like 'node-reach-coverage E2E SKIP (clientRxCoverage disabled on this deployment)' cannot be attributed to its suite without reading the sources and guessing. That is how test-node-reach-coverage-e2e.js came to pass every build while asserting nothing, unnoticed until #2037. Each invocation now prints '=== E2E SUITE: <file> ===' first, so the log is greppable per suite. Safe by construction: the banners go to stdout only, never through tee, so e2e-output.txt is byte-identical and scripts/aggregate-e2e-pass.sh sees what it saw before (it keys on digits followed by 'passed', which a banner never matches). The change is mechanical and was checked as such: the diff removes no line and every added line is a banner, 105 for 105 invocations. Verified on CI run 35347268407: all jobs green, and the banners make attribution work. Using them, exactly one wired suite skips wholesale (test-node-reach-coverage-e2e.js) and three skip individual assertions (test-e2e-playwright.js on a flaky case, test-touch-gestures-coverage-e2e.js and test-issue-1306-collisions-terminology-e2e.js on fixture gaps). That list was guesswork before. Refs #2037. Merged by the interim maintainer without a second human reviewer: CI is the independent check. |
||
|
efiten
|
20b0ee9742 |
ci: run the eight tests/e2e suites that had no runner at all (#2045)
Part of #2037. Of the 113 suites in tests/e2e, 18 were invoked by no deploy.yml line, no script and no workflow: written, classified in scripts/non-unit-tests.json, and never executed. All 18 were run once on upstream master to find out which still work (CI run 35340835178, each tolerated and timed so one run produced the whole table). Nine passed; eight are wired in here, 45 seconds together: test-analytics-fluid-charts (2s), test-nodes-export-e2e (2s), test-1110-live-filter (3s), test-e2e-1267-mobile-vcr (6s), test-live-dedup (6s), test-issue-1274-legend-coverage (7s), test-issue-1648-m5-icons (9s), test-show-neighbors (10s). The ninth, test-rx-coverage-mobile-nav-e2e.js, is deliberately left out: it exits 0 with 'SKIP (clientRxCoverage disabled on this deployment)' and coverage is off by default, so it would add the appearance of a guard rather than a guard. Skip paths were checked rather than assumed: test-issue-1648-m5-icons honours CHROMIUM_REQUIRE and gets the flag, test-nodes-export-e2e's skip branch needs an empty dataset and the fixture holds 182 named positioned nodes of 200, and the other six have no skip path. Verified on run 35343304933: all jobs green, each of the eight invoked exactly once in the Playwright job, and none of them printed a SKIP. Left for #2037: the four that run and fail (test-channel-modal-e2e 12/2, test-packets-scope-column 4/3, test-touch-targets 5 assertions, test-node-reach-e2e TimeoutError) and the five that cannot run as node scripts (four import @playwright/test, one jsdom, neither is a dependency). Merged by the interim maintainer without a second human reviewer: CI is the independent check. |
||
|
liquidraver
|
bbf54cfe1b |
fix(rx-coverage): open at the configured map default, with its own saved viewport (#2033)
The coverage page opened at a hardcoded [51.0, 4.8] zoom 8 regardless of deployment, ignoring /api/config/map (#2032). It now follows the same precedence as the main map (URL hash, then saved position, then /api/config/map, then [37.6, -122.1] zoom 9) and persists its own position across visits, syncing lat/lon/zoom into the hash so a view is shareable. The saved position lives under its own key, rx-coverage-view, and the page never reads or writes the main map's map-view: sharing the configured default was the bug, sharing the session position was not. Both suites assert map-view stays untouched after a pan, so reintroducing a shared write fails instead of passing quietly. Also fixed here: selectedRx is now percent-encoded into the hash, and a generation counter stops a late /api/config/map response or a stale 150ms layout timer from building a map for a page that was already left. Reviewed twice. Verified by mutation rather than by reading: writing map-view too, ignoring the saved coverage position, and dropping the /api/config/map fetch each make the unit suite exit 1, so it covers the feature, the fix and the rejected alternative. The deploy.yml invocation was confirmed to land inside Run Playwright E2E tests (fail-fast) by parsing the workflow, and CI run 35261689694 is the E2E suite's first real execution: Go prints 'RX coverage viewport regressions OK' and Playwright prints 'RX coverage viewport browser regressions OK'. Worth recording for the next reviewer: that E2E asserted localStorage.getItem('map-view'), so wiring it into deploy.yml without updating the assertion would have turned the job red on its first ever run. It was registered in scripts/non-unit-tests.json but invoked by nothing, the gap tracked as #2037. Merged by the interim maintainer without a second human reviewer: CI and the mutation checks above are the independent checks. Fixes #2032 |
||
|
Alex B
|
893773338e |
chore(tests): move root test-*.js into tests/unit and tests/e2e (#2036)
Moves 290 root test-*.js into tests/unit (177, listed in test-all.sh) and tests/e2e (113, classified in scripts/non-unit-tests.json), per #1981 and PR-D of #1385. Root goes from 348 entries to 48. test-all.sh and test-fixtures/ stay put. The inventory guard now fails if a test reappears in the root or sits in the wrong folder.
Verified independently of the diff: the invoked sets are unchanged (test-all.sh 177 before and after, deploy.yml 96 before and after, both identical as sets), and a full local run of test-all.sh on master and on the branch produced 4702 output lines each whose only differences are absolute paths, stack-trace line numbers shifted by the REPO_ROOT line, the inventory wording and two perf ratios. The guard was mutation-checked: a test back in the root, a unit suite in tests/e2e, and a suite dropped from test-all.sh each make it exit 1. CI run 35246304316 ran 97 suites from tests/e2e and is green.
Follow-up
|
||
|
efitenandClaude Opus 5
|
b8c8d98e61 |
fix(release): keep every platform when re-tagging :edge as a release (#2031)
## Problem `crane mutate` works on one image, not on an index. Pointed at the multi-arch `:edge` tag it silently resolves the default platform, so the fast path published v3.11.0 as a single amd64 OCI manifest, and `crane tag` then pointed `v3.11`, `v3` and `latest` at that same manifest. `docker pull` on arm64 against any of those four tags fails. Verified in the registry: | tag | shape | arch | |---|---|---| | `v3.9.2`, `v3.10`, `v3.10.1`, `edge` | index, 4 children | multi-arch | | `v3.11.0`, `v3.11`, `v3`, `latest` | `oci.image.manifest.v1`, 17 layers | amd64 only, revision `a2ea18f7` | Earlier releases are indexes, so this only hit v3.11.0. The GitHub release and both `corescope-decrypt` binaries are unaffected. ## Change - The fast path now reads the `:edge` manifest, mutates each runnable platform child by digest (`/app/.image-version` plus the version label, as #1807 intended) and reassembles an index with `crane index append`. A single-platform `:edge` still takes the old single mutate path. - Attestation manifests (`platform.architecture == "unknown"`) are not carried over: they reference the pre-mutation digests, so copying them would attest the wrong images. - A new verification step compares the platform set of `vX.Y.Z`, `vX.Y`, `vX` and `latest` against `:edge` and fails the run if any of them differs. A release tag that resolves to one platform is worse than a slow release, so this should break the build rather than ship. - Scratch tags (`tmp-vX.Y.Z-linux-amd64`, ...) are deleted best-effort afterwards; the index references the manifests by digest, so leaving them behind is only untidy. - Added `workflow_dispatch` with a `tag` input to republish the images for an existing release. A dispatched run resolves the tagged commit itself, because `github.sha` is then the ref the workflow file came from, and it skips the `deploy.yml` dispatch: that release already exists and releases here are immutable (the trap from #1955/#1956). ## Tests None: this repository has no harness that executes workflow files, and the CI jobs cannot reach a step that pushes to GHCR. What the change is verified against instead: - `crane index append` accepts `-m/--manifest` repeated plus `-t/--tag`, with the base index optional, so building an index from scratch is supported (crane docs for `index append`). - The YAML parses and every `run:` block passes `bash -n`. - The platform comparison was run by hand against the live registry: `:edge` reports `linux/amd64,linux/arm64` and `v3.11.0` reports `single`, which is exactly the case the new step must fail on. - The real test is the dispatch on `v3.11.0` right after merge, which is also the repair. If the verification step fails there, nothing is published and the tags stay as they are. ## Not verified - The scratch-tag delete needs `delete:packages`; `GITHUB_TOKEN` may not have it. It cannot fail the run. - Whether GHCR keeps the attestation manifests attached to `:edge` reachable after the index is rebuilt for a release tag (they stay on `:edge` itself, which is untouched). --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Sylvain Rabot
|
a2ea18f778 |
perf(sqlite): swap modernc.org/sqlite for mattn/go-sqlite3, cross-built with zig (#1992)
Swaps the SQLite driver from `modernc.org/sqlite` (pure Go, SQLite
3.46.0) to `github.com/mattn/go-sqlite3` (cgo, bundled SQLite 3.53.4),
and pays the resulting cross-compilation cost with `zig cc`.
Draft because the riskiest part of this deletes rows — see [Please
review this part first](#please-review-this-part-first) — and because
three things remain unverified at the bottom.
`modernc.org/sqlite` is a transpilation of the C amalgamation. This repo
is read-heavy: `cmd/server` chunk-loads a graph at startup and fans out
neighbour/topology/analytics queries per request, and it pays for that
transpilation on exactly those paths. Head-to-head on the same
120k-transmission / 240k-observation database, running our own hot-path
SQL under both drivers (Apple M4, `-count=5`, medians):
| workload | modernc | mattn | |
|---|---:|---:|---|
| chunk load (`chunked_load.go` v3 join, 20k tx) | 449ms | 196ms |
**2.3×** |
| aggregate scan (240k-row join + `GROUP BY`) | 276ms | 137ms | **2.0×**
|
| 1500 prepared-statement lookups | 512ms | 403ms | **1.3×** |
Allocations fall with it: 1.12M vs 1.64M allocs and 21MB vs 30MB on the
chunk load.
**Superseded by a production run.** @efiten measured both drivers on a
real instance — 11,077,038 observations, 9.7GB database, 4-core arm64 —
as server-only containers against the same live volume, one at a time,
with round 2 reversing the order so the page cache favours the old
driver:
| | audit 7d | audit 24h | background fill (13 chunks) | start →
/api/health |
|---|---:|---:|---:|---:|
| modernc, round 1 | 16.67s | 2.27s | 130.2s | 16.6s |
| mattn, round 1 | 7.87s | 1.34s | 93.8s | 13.5s |
| mattn, round 2 | 8.15s | 1.35s | 96.4s | 13.0s |
| modernc, round 2 | 13.46s | 2.29s | 137.8s | 15.5s |
Warm, the old driver improves to 13.46s on the 7d audit and still loses
by ~1.8×. Chunk load is ~1.4×. `/api/nodes?limit=500` is 0.039s against
0.037s — nothing.
**So the real gain is ~1.4–1.8× on the paths that matter, not 2–2.3×.**
The shape the harness predicted holds — scans and joins gain, small
lookups do not — which is more reassuring than the magnitude would have
been. Quote these numbers.
**The counterweight**, cold and native on that machine: a build goes
from **52s to 163s**. An instance that builds its own image pays that
per deploy.
## The build is cgo now, and one thing about that is a trap
**`CGO_ENABLED=0` still builds.** mattn links a stub, and the binary
dies on its first query with `go-sqlite3 requires cgo to work. This is a
stub`. A green build is not evidence of anything here, which is why
`AGENTS.md` now says so explicitly. `GOOS=linux go build` genuinely
cannot cross-compile any more.
A new root `Makefile` is the entry point. `make crossbuild` uses `zig cc
-target {x86_64,aarch64}-linux-musl` and links static, so each artifact
stays a single self-contained file and the `alpine:3.20` runtime no
longer depends on the base image's libc at all.
`-Wl,-s` is load-bearing: Go's own `-s -w` does not reach the musl
objects zig links in, and without it the server binary is 19.8MB instead
of 12.1MB.
The Dockerfile keeps its single `$BUILDPLATFORM` builder — still no QEMU
for compilation — and gains a checksum-pinned zig plus BuildKit cache
mounts. The mounts are not a nicety: without them an image build
recompiles the amalgamation from cold and takes over half an hour.
## Please review this part first
`internal/dbschema/dedup_index.go` **deletes observation rows**. It is
the one part of this change that can lose data, and it exists because
the migration exposed a real bug rather than causing one.
`stmtInsertObservation` resolves its `ON CONFLICT` against
`idx_observations_dedup`, which `cmd/ingestor/db.go` only ever created
inside the branch that creates the `observations` table for the first
time. Any database whose table predates that branch never got one, so
the UPSERT had no conflict target. modernc failed on the first insert;
mattn fails at `OpenStore`. Same bug, found earlier.
Creating the index unconditionally repairs it — but the index is what
was supposed to prevent duplicates, so a database that never had it can
already hold rows violating it. **`test-fixtures/e2e-fixture.db` in this
repo holds one.** So duplicates are collapsed first. Refusing is not the
safer option: without the index the ingestor cannot prepare its UPSERT,
so it cannot start at all.
Replaying that UPSERT faithfully is subtler than it looks, and a first
cut of this got it wrong twice:
- `COALESCE(excluded.x, x)` means the **incoming** value wins, so down a
group in id order the survivor keeps the **last** non-NULL value. Taking
the first silently discarded newer readings.
- The UPSERT names exactly five columns (`snr`, `rssi`, `score`,
`raw_hex`, `resolved_path`). Every other column must keep the surviving
row's own value; merging those too invents history the ingestor would
never have written.
Merge, delete and `CREATE UNIQUE INDEX` now share one transaction. Split
apart, a writer inserting a duplicate in the gap fails the index
creation while leaving the deletions committed — rows destroyed and no
index to show for it.
Cost, measured on 2.4M synthetic rows holding 5 duplicates: **4.1s**,
holding the write lock throughout, once, at ingestor startup before MQTT
subscribe. Materialising the duplicate-group scan once rather than per
column took that from 9.7s; the pathological case (400k of 600k rows
duplicated) is 5.7s, slightly worse than the 4.2s it was before that
change.
## Four more behavioural differences
Full detail in `docs/sqlite-driver-migration.md`. Briefly:
**Statement preparation is eager.** modernc's `newStmt` stored the SQL
and compiled lazily; mattn calls `sqlite3_prepare_v2` inside `Prepare`,
so SQL naming a missing table fails at *open*. 59 server tests failed on
this alone, all fixtures with partial schemas. `OpenDB` keeps failing
loudly (#1901; `main.go` gates on `dbschema.AssertReady` anyway) and the
fixtures now declare what they are prepared against via
`ensurePreparable`. This also exposed nine `nodes(pubkey …)`
declarations across seven files, where production has only ever had
`public_key` — lazy compilation had hidden the mismatch for as long as
it existed.
**`synchronous` silently dropped FULL → NORMAL.** mattn defaults it to
NORMAL and executes the pragma unconditionally, where SQLite's own
default (what modernc left alone) is FULL. In WAL mode that weakens
durability under power loss. Pinned in `dbschema.WriterDSN`, which both
writers now share — `cmd/migrate` kept a bare path at first and so
quietly wrote at NORMAL, which is what a second copy of a DSN buys you.
**The DSN dialects are mutually invisible.** modernc understood only
`_pragma=name(value)`, mattn only `_`-prefixed parameters, and neither
errors on the other's form — a driver-only rename would have dropped
every pragma in silence. `_journal_mode=WAL` is also gone from the
server's read handle: modernc ignored it, mattn honours it, and setting
`journal_mode` on a read-only connection is a write. Dropping
`_busy_timeout` with it costs nothing, since mattn already defaults to
5000ms — which means the read handle finally *gets* the busy timeout it
had silently lacked.
**`mode=ro` survives for a non-obvious reason.** mattn always passes
`READWRITE|CREATE` and its amalgamation has `SQLITE_USE_URI=0`; what
makes the URI work is its C wrapper ORing `SQLITE_OPEN_URI` in. So the
#1283/#1289 invariant holds with no build flags — but it depends on the
`file:` prefix. `cmd/decrypt` had been building its DSN without one, so
its `mode=ro` had never applied and a missing path was created
read-write. Fixed in passing; never a migration regression.
## What did not change
No modernc-specific API was in use: no `RegisterFunction`, no
`*sqlite.Conn`, no `sqlite/lib` error constants, no `sql.Register`. No
`time.Time` is ever bound as a query argument, so driver time handling
is not in play. Both drivers convert declared
`DATE`/`DATETIME`/`TIMESTAMP` columns to `time.Time`, so
`/api/dropped-packets` keeps emitting `dropped_at` as RFC3339 — an
earlier draft "fixed" that with a `CAST` and would have been the
regression.
## Tests and CI
New regression tests, each written because something got through without
it:
- `TestEnsureObservationsDedupIndexKeepsLatestValues` — the merge
ordering. The original test used complementary NULLs, which passes
whichever direction you pick, which is why the bug survived it.
- `TestCollapseDuplicatesAndIndexIsAtomic` — a failed index creation
must roll the deletions back.
- `TestOpenStorePragmas` / `TestWriterDSNPragmas` — every writer pragma,
read back through the store's own connection. A separate `sqlite3`
session or the startup log line would prove nothing.
- `TestOpenDBRefusesMissingDatabase` — the read-only invariant, which
now rests on a detail of the driver's C wrapper.
- `TestEnsurePreparableMatchesPrepareStatements` — fails when a new
prepared statement outgrows the fixture helper.
CI gains test execution for `cmd/migrate` and `internal/dbschema`, which
had none and both open the database. A PR-time two-arch build plus an
arm64 QEMU smoke gate is new: the GHCR push is push/tag-only, so without
it nothing on a PR would exercise zig, static musl linking or arm64, and
the first signal would arrive on master. `cache-dependency-path` widens
from 2 of the 5 tracked `go.sum` files to all of them.
`make test` passes across all 14 modules, `cmd/server` also under `-race
-count=2` with no failures and no races. `gofmt` and `go vet` clean.
Release-routing and Dockerfile COPY-invariant gates pass.
## Verified by running
- All 8 cross-builds static and correct-architecture; both arches of the
container image built, exported and run under QEMU, serving
`/api/health` and `/api/nodes` against a 2.9M-observation production
snapshot.
- The `migrate` binary repairing that snapshot's duplicate on bare
Alpine.
- `CGO_ENABLED=0` producing a binary that builds and then fails on first
query.
## Not verified
- ~~The 2–2.3× figures come from a standalone harness, not this load
under the old driver.~~ **Closed** by @efiten's production run above,
which also corrected the multiplier.
- SQLite 3.46.0 → 3.53.4 query-planner differences on queries with no
total `ORDER BY`.
- Sustained live ingest through the new writer DSN, and the duplicate
collapse against a database an ingestor is actively writing to. Verified
against a static snapshot only, and the collapse is measured at 4.1s on
2.4M synthetic rows with 5 duplicates — well short of an 11M-row
instance. @efiten has offered a staging instance taking real MQTT
traffic; **this is the item to close before the PR leaves draft.**
An earlier revision of this branch shipped the dedup merge in the wrong
direction with a green test suite, and review then found three more
things in the same file: the repair gated on an error string, a
non-atomic TEMP table drop aimed at the wrong connection, and a deletion
whose only record was a row count. All fixed in
|
||
|
n30nex
|
cd9b4c04d0 |
test: unify frontend test runs and prevent inventory drift (#1965)
Make `test-all.sh` the authoritative standalone frontend runner for npm and CI. Restore stale assertions and reject missing, duplicate, removed or undocumented inventory entries. Fixes #1858. Rebased onto `a2f039d4`. Retains release-routing, map scope-state and Scope Audit stylesheet tests, adds `test-packets-local-channels.js` to the sorted runner, and classifies `test-neighbor-map-btn-clip-e2e.js` under browser. Its separate CI browser step is preserved. Inventory: 280 root suites, 167 standalone, 113 requiring separate setup. The icon repair fixes two suites red on master: `test-issue-1648-m2-emoji-scan.js` and `test-issue-1648-m6-final-sweep.js`. Node/live configured-scope confirmations now use the existing accessible Phosphor check sprite. Values, visibility conditions and scanner assertions are preserved. - Red evidence: `89e45a9` inventory assertions; `4e255df` accessible confirmation assertion. The latest rebase also reproduced both unclassified-file failures before adding their entries. This follow-up only changes runner/classification configuration and counts; no test files modified. - Local validation: all 167 standalone suites; 27 M2 browser checks and 16 neighbor geometry checks in Chromium. Syntax, whitespace, PII, CSS-variable and XSS checks passed. - Browser coverage includes populated/empty/null configured scopes in node and live views. - No new dependencies, requests, application settings or Go changes. Workflow outside the unit step matches master. - Windows validation uses process-local UTF-8 settings. Encoding and node-reach confirmation follow-ups remain separate, as requested. ## Preflight override External `run-all.sh` is unavailable; applicable repository checks were run directly. |
||
|
nullrouten0
|
a2f039d4c7 |
fix(packets): show browser-added channels in the channel filter (#2009)
Verified against production: ingestor writes enc_<HH> (db.go:2282), store.go:3536 filters on it, live values are uppercase two-digit hex with thousands of packets (enc_28: 2708), /api/channels omits them, and /api/packets?channel=enc_28 returns rows. New test passes at 14 and fails when the dedupe guard is disabled. |
||
|
nullrouten0
|
6d7490e7da |
fix(tables): stop the trailing action column cropping its own button (#2010)
Reproduced on a live instance: 41px column for a 53px button at 1440px, 29px at 1024px, max-width:0 and overflow:hidden on the cell. Injecting the .col-action rule took it to 65px with nothing cropped. The new test fails 13 of 16 checks when only the CSS block is removed. |
||
|
efiten
|
51a2a7dd2f |
fix(scope-audit): ship the stylesheet the page paints its badges with (#2005)
Fixes #2004. Two review rounds; findings and evidence on the PR. Verified by injecting the stylesheet into a running deployment and reading computed styles before and after: the badges gain background, size, uppercase and padding; at 430px the Config column stays visible; the sorted column keeps its accent. The new test fails on the eight unstyled classes against the pre-fix tree, on .ns-truncated against the first fix, and on a re-added column-hiding rule. |
||
|
efiten
|
296456f9c1 |
feat(map): colour and filter repeaters by scope-configuration state (#2006)
Closes #2001. Two review rounds plus a re-review; findings and evidence on the PR. Verified on a deployment against live data: the field over 1653 nodes, marker tints per filter state, the marker title and popup Scope row reaching the DOM, and the colorblind-preset cascade. The audit and the map are held to the same classification by an end-to-end test that fails when either side's wildcard handling drifts. |
||
|
efiten
|
6c92a8b612 |
test(ingestor): make the suite race-clean, and run the detector when it matters (#1994)
`go test -race ./...` on `cmd/ingestor` reports **six data races** on master. None is in production logic. All six come from test helpers that outlive the test that started them, which is why the detector blames whichever test happens to be running: two different tests failed on two consecutive runs of the same code. ## What was racing **`StartStatsFileWriter` had no way to stop.** Two tests start it at a 50ms interval, and its goroutine then runs for the rest of the process. It reads the package-level `readProcSelfIOFn` hook, which a later test replaces to inject a fake, so the write and the read race. The same leak explains the stray log lines about writing stats into temp directories that were already cleaned up. It now returns a stop function that closes the goroutine and waits for it to exit. Production ignores the return value and runs for the process lifetime exactly as before; the two tests call it through `t.Cleanup`. **The migration test read a log buffer while a goroutine wrote to it.** `log.Logger` serialises its own writes, but `logContains` read `buf.String()` outside that lock while `RunAsyncMigration` kept logging after the call that started it had returned. The capture helper now uses a mutex-protected buffer. That one is worth calling a real race rather than a test artefact: a concurrent read during a buffer grow can panic outright with "concurrent map read and map write"-class behaviour, not merely trip `-race`. ## Measured `go test -race ./...` on linux/arm64 under go1.27.1: **exit 0, zero races, 704s**. ## The CI job, and why it is shaped this way The server has had `-race` since #1208. This closes the same gap for the ingestor, which carries an `atomic.Pointer` snapshot (the region key set from #1989) whose safety has been an argument rather than a measurement. Two deliberate choices, because a check that costs too much gets switched off: - **Its own job, not a step inside "Go Build & Test".** Appending `-race` there puts its ten-odd minutes on the critical path, taking the pipeline from roughly 20 minutes to roughly 32. As a separate job it runs beside the E2E job (15-17 minutes) and hides inside that window. - **Only when `cmd/ingestor/**.go` changed**, decided by the existing change-scope job, which already gates the heavy jobs on documentation-only PRs. A frontend or docs PR cannot introduce a data race in the ingestor. Pushes to master always run it, as they already do for `code`. Nothing `needs:` the new job. Adding it to `build-and-publish` would mean a skipped race job skips everything downstream, which is the opposite of what a conditional check should do. It reports as its own check; whether that blocks a merge is a repository setting rather than workflow logic. ## Scope Test helpers, one production signature (`StartStatsFileWriter` now returns a stop function), and the workflow. No change to what the ingestor does at runtime. |
||
|
Anupam MedirattaandClaude Opus 5
|
de237fc29c |
fix(qa): bind TEST_PUBKEY as a SQLite parameter instead of interpolating it (#1982)
Closes #1977. Supersedes #1952. Follow-up filed as #1983. ## What §10.2 did ```bash q="SELECT COUNT(*) FROM transmissions WHERE from_node = '$TEST_PUBKEY';" qq=$(printf %q "$q") if ! count=$(ssh_t "docker exec … sqlite3 … $qq" 2>/dev/null); then count=$(ssh_t "sqlite3 … $qq" 2>/dev/null || echo "") fi ``` The injection is not reachable today — `TEST_PUBKEY` is hex-gated and the script `exit 2`s before the SQL is built. The problem is that the SQL layer's safety rests entirely on that outer gate rather than on the SQL layer itself. #1952 proposed doubling embedded quotes; that is string escaping, not parameterisation, which is why it was withdrawn in favour of this. ## What this does Per the four points in the sign-off on #1977: **1. Bind the value.** A constant `SELECT` and a bound `:pubkey`, fed to sqlite3 on stdin. The SQL no longer crosses the remote shell as a command word, so there is no `printf %q` on the query at all any more. **Why hex rather than `.parameter set :pk '<value>'`.** Dot-command arguments are split on whitespace, so a payload containing a space produces too many arguments — and sqlite3 responds by printing the `.parameter` help to **stdout**, exiting **0**, and leaving `:pk` **unbound**. `COUNT(*)` then returns 0, which reads exactly like a passing security fix. `-bail` does not catch it. Verified on 3.51.0: ``` $ printf ".parameter set :pk '' OR 1=1 --'\nSELECT COUNT(*) FROM transmissions WHERE from_node = :pk;\n" \ | sqlite3 -bail ptest.db .parameter CMD ... Manage SQL parameter bindings # <- help, on stdout … 0 # <- :pk never bound $ echo $? 0 ``` `.parameter set :pk 1+1` also binds the integer `2` — the value is evaluated as an SQL expression and only falls back to a text literal when evaluation fails. So interpolating into the `.parameter set` line trades one hazard for another. Hex-encoding removes the quoting layer instead of adding one: the value is bound as `cast(x'<hex>' as text)`, so its contribution to the SQL text is drawn from the alphabet `[0-9a-f]` only. Nothing to quote, no tokenizer arity hazard, and it holds for **arbitrary** input rather than only for hex-gated input — which is the point. Verified against a fixture table holding two rows, one of them `deadbeef`: | value | result | exit | |---|---|---| | `deadbeef`, bound as `cast(x'6465616462656566' as text)` | `1` | 0 | | `' OR 1=1 --`, bound the same way | `0` | 0 | | `' OR 1=1 --`, interpolated the current way | `2` (whole table) | 0 | | query against a DB with no `transmissions` table, `-bail` | `Parse error … no such table` on **stderr** | **1** | **2. Probe the capability, not a version.** `resolve_sqlite_runner` binds `corescope-probe-ok` and asserts it comes back — a round trip, not a bare `.parameter init`, so the positive control runs against the operator's actual binary rather than one we pin. If neither the container nor the host qualifies, it fails loudly and names what is needed: ``` ❌ retain-failed: no sqlite3 able to bind a parameter on the target tried: docker exec -i corescope-prod sqlite3, then sqlite3 on runner@example need: the sqlite3 CLI reachable over ssh, supporting '.parameter set' OCI runtime exec failed: exec: "sqlite3": executable file not found in $PATH bash: line 1: sqlite3: command not found ``` There is deliberately **no** interpolating fallback. That would leave the vulnerable path in place under a nicer name. **3. The hex gate is kept**, with its comment updated to say why: for the SQL layer it is now defence in depth rather than the only guard. Redundant is not the same as wrong. **4. The exit status and stderr survive.** `-batch -bail -init /dev/null -noheader -list` (stop at the first SQL error; ignore the operator's `~/.sqliterc`, where a stray `.mode` would make the count unparseable; stdout is exactly the number). Query stderr is captured and printed on failure rather than sent to `/dev/null`, so a broken query is distinguishable from a legitimately empty result. Probe stderr is collected too, and printed only if *both* probes fail — the container miss is the known-normal case, so surfacing it on every run would be noise. ## Also fixed An existing double-count in §10.2: the `TARGET_DB_PATH unset` branch incremented `$fails` and then left `count=""`, so the generic branch incremented it a **second** time for the same failure. `read_retain_count` now gives §10.2 exactly one increment point. Opportunistic cleanup in a file already being touched (AGENTS.md line 318). ## Tests New `qa/scripts/test-blacklist-sql.sh`, wired into the `go-test` job. 24 assertions, modelled on `scripts/staging/test-disk-monitor.sh`. Both directions are asserted, because a zero from a command that failed proves nothing: - **Positive control** — `deadbeef` still returns its row (`1`, exit 0), and so does `cafebabe`; an absent pubkey returns `0`. - **Negative** — `' OR 1=1 --` returns `0` while the table demonstrably holds 2 rows, and the old interpolated form is asserted to leak all `2`. That last assertion is what makes the `0` above worth something. - **Error surfacing** — the same SQL against a DB with no `transmissions` table exits non-zero with a message on stderr and nothing on stdout. - **Alphabet** — `sql_hex_literal` output matches `^x'[0-9a-f]*'$` for the SQL payloads, a backslash, `$(id)` / backticks, an embedded newline, `héllo`, and a 4096-byte repetitive string. That last one is a regression guard for `od -v`: without the flag `od` collapses repeated identical lines to `*`. - `run_sqlite` with no resolved runner refuses rather than guessing. Group 2 skips loudly (rather than silently) if `sqlite3` is not on PATH; group 1 needs no sqlite3 and always runs. **Mutation-tested** — each of these breaks the suite, so the assertions have teeth: | mutation | caught by | |---|---| | restore full interpolation | `injection payload → 0 rows — expected '0' got '2'` | | naive `.parameter set '%s'` | `expected '0' got '.parameter CMD ...'` | | drop `od -v` | alphabet failure on `*`, plus `expected '8192' got '33'` | Commit 1 is a behaviour-neutral refactor that moves the imperative body into `main()` behind a `BASH_SOURCE` guard, so the test can source the script and exercise individual helpers. Same idiom as `scripts/staging/disk-monitor.sh:99`. ## Verification - `bash qa/scripts/test-blacklist-sql.sh` → 24 passed, 0 failed - `bash -n` on both scripts - All three runtime paths exercised end to end with PATH shims for `ssh`/`docker`/`sqlite3` against a real fixture DB: success (`sqlite3 runner: host`, count 2), query failure (classified message + `Parse error … no such table`, `fails=1`), and no-capability (the loud block above, both probe stderrs, `fails=1` — not 2) - The new step lands inside `go-test`, which runs when `changes.outputs.code == 'true'`; `qa/scripts/*.sh` does not match that job's `^docs/|[.]md$|^LICENSE$` documentation filter, so it is not skipped ## Deliberately out of scope - **The `docker exec` branch is dead on current images** → filed as #1983. The app container has no `sqlite3` at all: `Dockerfile:15` is pure-Go SQLite with no CGO, and the `apk add` installs only `mosquitto mosquitto-clients supervisor caddy wget`. So the host "fallback" is the only path that has ever executed, silently, because both branches discarded stderr. This change keeps both branches and merely makes the outcome visible (`sqlite3 runner: …` on every run). - **`-readonly` on the target DB.** Tempting, and verified compatible with `.parameter` (the binding table lives in the TEMP database), but a WAL database needing journal recovery can refuse a read-only open. Adding it here risks exactly the "trades an unreachable injection for a script that does not run" outcome flagged in the #1952 thread. Worth its own issue. - **The other `2>/dev/null` sites** in this file, which also sit awkwardly with `qa/README.md`'s "Don't silence stderr". Only the §10.2 lines named in the sign-off are touched. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Sylvain Rabot
|
3fbff01f64 |
build: upgrade Go toolchain to 1.27 (#1946)
## Summary - Bumps the Go toolchain used to build/test to 1.27: `golang:1.27-alpine` in `Dockerfile` and `Dockerfile.go`, and `go-version: '1.27'` in the three `actions/setup-go` steps in `.github/workflows/deploy.yml`. - Each module's `go.mod` `go` directive is intentionally left at `1.22` — no 1.27-only language features are being adopted, and a 1.27 toolchain builds a `go 1.22`-declared module without issue. ## Test plan - [x] `go build ./...` + `go vet ./...` for all 13 modules (`cmd/server`, `cmd/ingestor`, `cmd/migrate`, `cmd/decrypt`, 10 `internal/*` packages) under Go 1.27.0 - [x] `go test ./...` passes for `cmd/server`, `cmd/ingestor`, `cmd/migrate`, `cmd/decrypt` - [ ] `docker build` against the new `golang:1.27-alpine` base (Docker wasn't available in the sandbox this change was prepared in — needs a check in CI or locally) |
||
|
n30nex
|
2288e28d4e |
fix: publish release artifacts after successful image retagging (#1964)
Successful release fast paths publish image tags but never dispatch the job that creates the GitHub release and decrypt binaries. Dispatch `deploy.yml` from both image routes. A default-off `images_published` input skips E2E and image rebuilding only for an already-published tag; missing or mismatched images keep the complete fallback without requiring new inputs on older workflow definitions. Go validation still gates the release binaries, checkout and version flags retain the tagged source, and the existing release action uploads both architectures before publication. Missing binary files now fail publication. Fixes #1956. Validation: - `node test-issue-1956-release-routing.js` executes the actual workflow shell steps with registry and dispatch commands stubbed. Covers matching, missing and mismatched images; failed retag and Go validation; branch/PR boundaries; and both tagged binary commands. - The original test commit fails because a matching image dispatches zero artifact workflows; the fix passes the same assertion. - Existing release workflow Go checks, decrypt/channel tests, YAML parsing and actionlint pass. - Both static Linux amd64 and arm64 binaries cross-build with verified architecture and version metadata. Actual registry publication and GitHub release creation were not exercised. Existing immutable releases and old tags that contain older workflow definitions are outside this fix. Following #1922, this is a focused release-routing PR. A separate repair for #1858 rewrites the shared frontend test runner; merging this first lets that repair retain this regression in its authoritative list. Please assess current Go and E2E job results separately from workflow-approval or staging-runner state. |
||
|
efitenandClaude Opus 5
|
a3ee37011a |
ci: scope docs-only skipping to jobs, so required checks still report (#1955)
Same change as #1954, opened from a branch on this repository instead of from a fork. #1954 never received a workflow run: zero runs and zero check suites for its head commit, and closing and reopening it changed nothing. A manual `workflow_dispatch` on master ran immediately, so Actions itself is working; the `pull_request` event from the fork is what produces nothing. See the note at the end. Replaces the trigger-level `paths-ignore` from #1949 and #1950. That approach was wrong, and it is currently blocking #1953 from merging. ## What was wrong GitHub documents the distinction I had backwards: > a workflow skipped by path filtering keeps its checks **pending** and blocks the merge, while a **job** skipped by an `if:` conditional reports **Success** and does not. So the filtering has to live on the jobs, not on the trigger. I compounded it by claiming, in both the commit and the description of #1949, that master had no required checks: *"verified: the branch protection endpoint returns 404"*. **That verification was invalid.** A 404 there means the token cannot read protection details, not that none exist. The branch reports `protected=true`, and #1953 was refused with `the base branch policy prohibits the merge`. ## What this does A `🔎 Change scope` job computes whether anything outside `docs/`, `*.md` and `LICENSE` changed. `go-test`, `e2e-test`, `build-and-publish` and `release-artifacts` are gated on its output. A documentation-only pull request skips those jobs, they report Success, and the PR can merge. **Only pull requests are scoped.** A push or a dispatch always runs the full pipeline. That second part is deliberate and it fixes a separate failure. `release-fast-path.yml` re-tags `:edge` to `:vX.Y.Z` only when the `:edge` revision label matches the tagged commit. A master commit with no image breaks tagging, which is what happened to the v3.10.0 tag: the tagged commit was documentation-only, the fast path could not re-tag, it fell back to a dispatch, and the dispatch published nothing (#1951). Master pushes now always produce an image. The cost is running the pipeline on documentation commits to master; pull requests are where the queue pressure was. Two conservative defaults in the scope check: a non-`pull_request` event and an empty diff both count as code, so an unexpected shape runs everything rather than silently skipping. ## Note for whoever has repository settings access Fork pull requests stopped getting workflow runs between 22:36 and 05:40. #1949, #1950 and #1951 all came from the same fork and each got a run; #1954 got none, with no check suite created at all, which is different from a skipped run. `repos/.../actions/permissions` returns 403 for a non-admin token, so this could not be confirmed from the API. If the "Fork pull request workflows from outside collaborators" setting was tightened, that would explain it, and it would affect every outside contributor, not just this branch. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
efitenandClaude Opus 5
|
bb8634312d |
ci: publish images on a tag ref, not only on a push event (#1951)
**v3.10.0 produced no container image.** The build job reported `success` and pushed nothing. ## What happened `release-fast-path.yml` re-tags `:edge` to `:vX.Y.Z` when the `:edge` revision label matches the tagged commit, and dispatches `deploy.yml` when it does not. For v3.10.0 the tagged commit was documentation-only. The `paths-ignore` added in #1949 means documentation-only commits skip `deploy.yml`, so no `:edge` image was ever built for that commit, the labels did not match, and the fallback ran. **That part worked exactly as designed** and correctly refused to re-tag an image built from a different commit. Then `deploy.yml` skipped all five GHCR steps, because each was gated on `github.event_name == 'push'` and a `workflow_dispatch` is not a push: ``` 4. Build Go Docker image (local staging): success 5. Set up Docker Buildx: skipped 7. Log in to GHCR: skipped 9. Build and push to GHCR: skipped ``` **So the fallback has never been able to publish an image.** It dispatches a pipeline that cannot push. That stayed invisible for as long as the fast path kept succeeding, which it did until a release note happened to be the last commit before the tag. ## Fix Gate those five steps on a push **or** a tag ref: ```yaml if: ${{ github.event_name == 'push' || startsWith(github.ref, 'refs/tags/v') }} ``` A dispatch aimed at a tag now publishes. A dispatch aimed at a branch still does not, so this does not turn every manual run into a release. ## The version stamp needed no change Verified rather than assumed. `Compute build metadata` keys on `GITHUB_REF`, not on the event: ```bash if [[ "$GITHUB_REF" == refs/tags/v* ]]; then APP_VERSION="${GITHUB_REF#refs/tags/}"; else APP_VERSION="edge"; fi ``` The failed v3.10.0 run already logged `Build: version=v3.10.0 commit=5bad23b`. Only the publishing was missing. ## Not covered here `Release Artifacts` failed on the same run for an unrelated reason: the GitHub release had been created by hand before the workflow reached it, and `action-gh-release` cannot update an immutable release. That one is process, not code. Push the tag and let the workflow create the release. Once this merges, re-dispatching `deploy.yml` against `v3.10.0` publishes the images for the existing tag. No re-tagging needed. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
efitenandClaude Opus 5
|
6ccef5053b |
ci: match root-level markdown in the docs paths-ignore (#1950)
Follow-up to #1949, correcting a pattern I did not check before proposing it. #1949 used `'**/*.md'`. That reads as requiring a directory component, so it covers `docs/release-notes/v3.10.0.md` but not `CHANGELOG.md` or `README.md` at the repository root. Since `paths-ignore` skips only when **every** changed file matches, one uncovered root file is enough to run the whole pipeline anyway. GitHub's own example for "any file with this extension" is `'**.js'`, with no slash. `'**/*.md'` does not appear in their documentation at all. `'**.md'` covers root and subdirectories both. ## What this does not establish #1948 (`CHANGELOG.md` plus a release note) did start a full run after #1949 landed, and that is what prompted this. But there is a second candidate explanation I did not rule out: that PR's branch predates #1949, so its workflow file may simply not have carried the filter yet. I am not claiming to have proven which one it was. The fix is correct either way, and shipping the documented pattern is better than defending the first cause I noticed. The real test is the next docs-only PR opened from a branch that already contains the filter. The reasoning is in a comment above the `on:` block, not only in this description. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
efitenandClaude Opus 5
|
55aabaa325 |
ci: skip the pipeline for documentation-only changes (#1949)
Two documentation-only PRs were running the full pipeline simultaneously this afternoon: #1948 (`CHANGELOG.md` plus a release note) and #1947 (deleting a stale `docs/DEPLOYMENT.md`). Each spends about 12 minutes on Go Build & Test and about 16 minutes on Playwright to establish that a text file does not break a browser. **The cost is the queue, not the minutes.** On the same afternoon a `pull_request` run was created at 12:13 and its first job did not start until 16:19. Four hours in the queue. Every unnecessary run pushes the ones that matter further back, and this repository has been merging heavily today. ## Checked before adding the filter Rather than assumed: - **Nothing reads markdown at build or test time.** Grepping every Go and JS source for a runtime read (`ReadFile`, `readFileSync`, `os.Open`) of a `.md` path returns nothing. The `docs/` matches in `cmd/` and `test-*.js` are all comments pointing at documentation. - `/api/docs` serves Swagger UI generated from `cmd/server/openapi.go`, not from `docs/`. - `docs/` holds markdown plus screenshots (`png`, `gif`) and no build input. - **This workflow has no tag trigger**, so release tagging is unaffected; that runs from `release-fast-path.yml`. Worth stating explicitly given a `v3.10.0` tag is imminent. `paths-ignore` skips only when **every** changed file matches, so a PR touching both code and documentation still runs the full pipeline. ## The trap, stated in the file If required status checks are ever enabled on master, a skipped workflow never reports, and a docs-only PR would wait forever on a check that cannot arrive. At that point this needs to become a change-detection job with conditional heavy jobs rather than a trigger filter. Master has no required checks today. Verified: the branch protection endpoint returns 404. That caveat is in a comment above the `on:` block, not just in this description, because the person who enables required checks in six months will be reading the workflow and not this PR. ## Note This PR itself changes only `.github/workflows/deploy.yml`, so it is not documentation-only and will run the full pipeline, as it should. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
|
40f664c587 |
chore(#1859): gofmt sweep + gofmt/go vet CI gate (rebase of #1881) (#1941)
Rebase of #1881 by @SaarMesh-Bot onto current master. Their three commits are preserved, two of them cherry-picked with authorship intact; the sweep itself had to be regenerated. Opened as a new PR rather than force-pushing their branch. Closes #1881 once merged. Addresses parts 1 and 3 of #1859; part 2 landed as #1937. ## Why regenerated rather than merged The sweep in #1881 was cut on 2026-09-02 07:13 and roughly forty PRs landed after it, so it went `CONFLICTING/DIRTY`. Re-running `gofmt` on current master is cheaper and less error-prone than resolving 72 conflicts that are all whitespace. The drift it fixes also grew in the meantime: 66 files now, against 72 then, but spread differently. ## The three commits 1. **`style(#1859)`** — `gofmt -w` across the 14 modules. 66 files. 2. **`test(#1859)`** — @SaarMesh-Bot's fix for the one `go vet` copylocks finding, `cmd/ingestor/coverage_boost_test.go`: the range variable copied a `Config` embedding `sync.Once`. Cherry-picked unchanged. 3. **`ci(#1859)`** — @SaarMesh-Bot's CI step that fails on gofmt drift or vet findings, plus `.git-blame-ignore-revs`. Cherry-picked with one change, noted in the commit message: the ignore file pointed at `04bc80ee`, the sweep commit on their branch, which does not exist on this base and would make `git blame --ignore-revs-file` error. Repointed at `d3a02599`, the sweep here. ## Verification The claim "formatting only" is checked twice rather than asserted: - Every changed file is byte-identical to `gofmt(previous content)`. 0 of 66 deviate. - With line comments and all whitespace stripped, 0 of 66 files differ, so no code outside comments changed. 14 of the 66 also show doc-comment reflow. Since Go 1.19 `gofmt` re-indents indented comment blocks to tabs and inserts a blank comment line before them; the behavior matrix above `resolveHopWithContext` in `cmd/ingestor/path_resolver.go` is a clear example. That is gofmt's own output, not an edit, but it is worth naming because it makes the diff look larger than "whitespace" suggests. The gate was run locally exactly as the workflow runs it: `gofmt` clean, and `go vet` clean in all 14 modules, including `cmd/ingestor` which is what commit 2 fixes. Suites: `cmd/server` ok (80.7s), `internal/packetpath` ok (2.3s), `cmd/ingestor` passes except `TestWriteStatsAtomic_SymlinkAtDestIsReplaced`, which fails identically on bare master with "A required privilege is not held by the client" (Windows symlink privilege on my host, not code). ## Sequencing This should go last in the queue. The sweep touches 66 files, so merging it before the remaining open Go PRs gives each of them a conflict about nothing but formatting. After it lands the gate is active, and any PR with drift fails CI until it runs `gofmt -w`. Excluded from the sweep: the misnamed `Dockerfile.go`, which is a Dockerfile that gofmt cannot parse (the workflow excludes it too), and `docs/DEPLOYMENT.md`, which a case-insensitive filesystem surfaces as a spurious modification against `docs/deployment.md` and is unrelated. --------- Co-authored-by: SaarMesh-Bot <300107934+SaarMesh-Bot@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
efitenandClaude Opus 5
|
859173f145 |
ci: unblock the master pipeline — gate the staging deploy, detach the badges job (#1938)
Fixes the problem @sylr reported on #1922. ## What is broken **master has produced no completed pipeline result since 2026-08-31.** Thirty-plus runs today alone were cancelled or left queued, while `go-test`, `e2e-test` and `build-and-publish` were passing inside them. The `deploy` job runs on `[self-hosted, meshcore-runner-2]`, and no such runner has picked up a job since at least 2026-08-31. It has no `timeout-minutes`, so it sits queued indefinitely. That job holds its run open, the run holds the concurrency group `ci-refs/heads/master`, and GitHub then cancels every subsequent master push while one waits. Two runs showing it, one from yesterday and one from right now: ``` |
||
|
|
9ef4179ef1 |
fix(release): report correct version on fast-path retagged images (#1807) (#1814)
Fixes #1807. Implements the fix path from the triage (env → image-version file → baked version, zero rebuild cost): ## Changes **`cmd/server/main.go` — `resolveVersion()` fallback chain** 1. `CORESCOPE_VERSION` env (operator override) 2. `.image-version` file in the working dir (`/app` in the container) — mirrors the existing `.git-commit` pattern in `resolveCommit()` 3. ldflags-baked `Version` 4. `"unknown"` Edge builds are unaffected: no env, no file → baked `"edge"` as before. **`.github/workflows/release-fast-path.yml` — retag step** Instead of a plain `crane tag :edge → :vX.Y.Z`, the fast path now runs `crane mutate` on `:edge` with: - `--append` of a deterministic one-file layer containing `/app/.image-version` = `vX.Y.Z` - `--label org.opencontainers.image.version=vX.Y.Z` - `--tag :vX.Y.Z` `vX.Y`, `vX` and `latest` are then pointed at the mutated image. Still no rebuild — the mutation is a manifest + single ~100-byte layer operation. ## Notes - The release tags no longer share the exact digest with `:edge` (they carry one extra layer); the fallback SHA check is unaffected since it compares the `org.opencontainers.image.revision` label against `github.sha`. - The layer tar uses `--owner=0 --group=0 --mtime='UTC 2020-01-01'` for reproducibility. - Operators can also fix existing deployments immediately with `-e CORESCOPE_VERSION=v3.9.2`, no image change needed. ## Testing - `go build ./cmd/server` + `go vet` clean (golang:1.24) - Workflow YAML validated - Reporter context: running the affected v3.9.2 fast-path image in production (live.saarmesh.de), happy to verify the next tagged release end-to-end. --------- Co-authored-by: Mathias Kasper <fallisaar@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: SaarMesh-Bot <bot@saarmesh.de> |
||
|
Kpa-clawbotandcorescope-bot
|
8c3e397d39 |
fix(1846): drop 1200px cap on .observers-page (#1847)
Red commit:
|
||
|
|
b74a64ccfa |
fix(ui): canonical payload label map across packets/live/packet-filter (#1799) (#1804)
## Summary Replaces the three drifted per-surface payload-type label vocabularies with a single canonical map keyed by firmware enum name. Per the locked triage comment on #1799 ([comment-4823975431](https://github.com/Kpa-clawbot/CoreScope/issues/1799#issuecomment-4823975431)): > Create `public/payload-labels.js` exporting `{GRP_DATA: {short:'Group Data', long:'Group data packet', enumId:6}, ...}`. Migrate `packets.js typeMap`, `packet-filter.js FW_PAYLOAD_TYPES`, `live.js TYPE_COLORS legend` to consume it. E2E that scrapes each surface and asserts label equality. ## Changes - **`public/payload-labels.js`** (new) — canonical map exposed as `window.PayloadLabels` and `window.PayloadLabelsApi`. Keys are firmware enum names; values carry `{short, long, enumId}` plus derived `SHORT_BY_ID` / `FW_PAYLOAD_TYPES` / `TYPE_ALIASES` for legacy callers. - **`public/packets.js`** — `TYPE_NAMES` + `typeMap` now read from `PayloadLabelsApi.SHORT_BY_ID`. Literal kept only as a defensive fallback for the case where the script tag fails to load. - **`public/packet-filter.js`** — `FW_PAYLOAD_TYPES` + `TYPE_ALIASES` now sourced from `PayloadLabelsApi`. Literal fallback retained so `node test-packet-filter.js` still works headlessly. - **`public/live.js`** — legend `<li>` rows are now generated from `window.PayloadLabels` in stable order, killing the third-vocabulary `Message — Group text` / `Direct — Direct message` drift the #1797 review surfaced. - **`public/index.html`** — `<script src="payload-labels.js">` loaded before `roles.js` / `packet-filter.js` / `packets.js`. - **`test-issue-1799-label-vocab-e2e.js`** (new) — Playwright E2E. Scrapes `#liveLegend` rows and the `/packets` type-filter checklist, asserts each label matches `window.PayloadLabels[ENUM].short` for `TXT_MSG`, `GRP_TXT`, `GRP_DATA`. Also verifies `window.PacketFilter` still recognises the enum names. - **`.github/workflows/deploy.yml`** — wired the new E2E into the existing Playwright block. ## TDD trail - Red commit `eb392d4` — adds the failing E2E only (asserts `window.PayloadLabels` exists and labels match; both fail). - Green commit `44e902a` — introduces the canonical map and migrates the three surfaces. ## Verification - `node test-packet-filter.js` — 92/92 pass with the new fallback wiring. - Preflight: `bash ~/.openclaw/skills/pr-preflight/scripts/run-all.sh origin/master` — clean. Browser verified: E2E `test-issue-1799-label-vocab-e2e.js` exercises `/live` legend + `/packets` type filter against a Playwright headless Chromium; CI's Playwright block runs it on every push. E2E assertion added: `test-issue-1799-label-vocab-e2e.js:139` — `assert(fromLegend === canon, ...)` and `assert(fromPackets === canon, ...)` per enum. Fixes #1799 --------- Co-authored-by: mc-bot <bot@corescope> Co-authored-by: openclaw-bot <bot@openclaw.local> Co-authored-by: clawbot <clawbot@kpa.com> Co-authored-by: clawbot <bot@clawbot.local> |
||
|
|
30e4151f7a |
fix: neighbor-graph tab never renders after filtering down (#1758)
The Analytics → Neighbor Graph tab fetches the full (uncapped) graph and, when it exceeds NODE_LIMIT (1000), skips the force simulation with a "use filters to reduce the node count" notice. But filtering never actually re-enabled rendering: - the node-count guard tested _ngState.allNodes (the immutable full fetched set, assigned once in createGraphState and never reassigned) instead of the displayed/filtered _ngState.nodes, so its verdict was fixed at load time; - the entire draw loop lives in startGraphRenderer(), which ran exactly once at load and was never called from applyNGFilters(), so a filter change updated the node/edge arrays and stat cards but never un-hid the canvas or scheduled an animation frame -> the graph stayed blank no matter how few nodes remained. This explains both reported symptoms (selects too many nodes initially AND stays broken once restricted to fewer). Fix: make the render lifecycle filter-aware. - startGraphRenderer() now guards on the displayed set (_ngState.nodes), cancels any running rAF loop before re-deciding, toggles the canvas plus a stable-id "skipped" notice, and restarts cleanly (no double loops). - applyNGFilters() calls startGraphRenderer() so every filter change re-evaluates the guard and (re)starts or stops the loop. - the initial render now goes through applyNGFilters() so the first paint already respects the default filters (observers unchecked, saved min-score) instead of dumping the full fetched graph. Test: `node --check public/analytics.js` passes. Manually: open Analytics → Neighbor Graph on a mesh with >1000 nodes → the "skipped" notice shows; tighten filters (min-score up / roles off) below 1000 → the graph now renders (was blank before); loosen again → notice returns. Frontend-only change (`public/analytics.js`); no backend/API change. --- **TDD note (review round 1):** Single-commit community bug-fix on an existing UI surface (no "net-new UI" exemption). The e2e `test-issue-1758-ng-filter-rerenders-e2e.js` is the red→green gate — it fails on `origin/master` (the renderer kept the node-count guard on the full fetched graph and never un-hid the canvas) and passes with the fix. Per AGENTS.md the separate red/green-commit *form* is a bot rule, not a contributor gate. --------- Co-authored-by: Waydroid Builder <build@waydroid.local> Co-authored-by: Waydroid Builder <claude@michael.arcan.de> |
||
|
|
707d70c738 |
fix(packets): clamp .col-details to one line on mobile (#1770 S path) (#1805)
## Summary Partial fix for #1770 (S quick-fix path only; L refactor remains as follow-up). The packets-view virtual-scroller assumes a constant `VSCROLL_ROW_HEIGHT`, but the base rule at `public/style.css` L1097 lets `td.col-details` wrap on narrow viewports (`white-space: normal; word-break: break-word`). Wrapped rows produce variable row heights → visible jitter when scrolling past ~900px on iOS. **Quick-fix (S path):** under the existing `@media (max-width: 640px)` block in `public/style.css`, clamp `.col-details` to a single line: ```css .data-table td.col-details { white-space: nowrap; overflow: hidden; text-overflow: ellipsis; } ``` Trade-off accepted in triage: Details column truncates on mobile in exchange for smooth scrolling. The base rule keeps wrapping on desktop (≥641px) so nothing changes there. **Out of scope:** the full L-path fix (per-row measurement, `_rowHeightsPx[]`, cumulative offsets, re-measure on hop-resolver finalize) — tracked separately on #1770. ## TDD - **Red commit** `7f58bedc` — adds `test-issue-1770-mobile-row-clamp.js`, a CSS-grep test (same pattern as `test-issue-1364-pill-no-clamp.js`) that walks every `@media (max-width: 640px)` block in `public/style.css` and asserts a `.col-details` rule declares `white-space: nowrap`, `overflow: hidden`, and `text-overflow: ellipsis`. Verified to FAIL on master (assertion failure, not a parse error) and PASS after the CSS change. - **Green commit** `d46271b8` — applies the 5-line CSS clamp inside the existing mobile breakpoint at L2362. ## Files touched - `public/style.css` (+13) - `test-issue-1770-mobile-row-clamp.js` (+101, new) ## Preflight `bash ~/.openclaw/skills/pr-preflight/scripts/run-all.sh origin/master` → all gates pass (PII, branch scope, red commit, css-vars, css self-fallback, LIKE-on-JSON, sync migration, async-migration, XSS). No warnings. --------- Co-authored-by: clawbot <bot@clawbot.local> Co-authored-by: openclaw-bot <bot@openclaw.local> |
||
|
|
770749a8ce |
fix(#1791): add 'Group Data' (payload_type=6) to packets type filter (#1797)
Fixes #1791. ## What Adds `6:'Group Data'` to the `typeMap` in `public/packets.js` so the Packets-view "message type" multi-select shows a Group Data checkbox. The filter pipeline already keys by integer payload_type, so this just registers the missing option. Also aligns the Live-view legend label in `public/live.js` to "Group Data" for cross-view consistency. ## Why Triage (in #1791) confirmed payload_type=6 (GRP_DATA) was the only ordinary type omitted from the static `typeMap`. `packet-filter.js`, `live.js`, `app.js`, and `map.js` all already know about it — only the Packets-page checklist was missing it. ## Test (TDD red → green) Branch history (4 production commits before round-1 review): - `19ed5beb` — **test-only red commit**: adds Playwright E2E that opens the type-filter menu, asserts a `data-type-id="6"` checkbox labeled "Group Data" exists, selects it, and asserts every visible row's type badge reads "Group Data". Also seeds one GRP_DATA packet into the CI fixture (`.github/workflows/deploy.yml`) so the filter has a row to match. - `823a7d8d` — adds the one-line `typeMap` entry. First CI run on this commit failed on an unrelated test (not the #1791 assertion); the #1791 test ran and passed. - `eec2428` — fixture cleanup: `path_json=[]`/`resolved_path=[]` so the seeded GRP_DATA hop-row count matches the raw_hex `path_len=0`. CI green. - `8f85f5f` — labels the type-6 entry "Group Data" (was briefly "Grp Data"). CI green. E2E assertion: `test-e2e-playwright.js` block `Packets type filter includes Group Data (#1791)`. ## Round-1 review follow-ups - `e3651c99` — `public/live.js` legend: `'Grp Data'` → `'Group Data'`. - `4475c2f7` — test cleanup hardening: error string aligned to assertion, duplicated selector extracted, regex tightened to strict equality, `#typeMenu` explicitly closed, `meshcore-time-window` localStorage key cleared, page reloaded so the in-memory `selectedTypes` Set is reset. - `b90bc33f` — `.github/workflows/deploy.yml`: drop self-referential `#1797` citation from fixture comment, switch synthetic fixture id from `-1` to `-1000000` sentinel with explanatory comment. ## Scope Single-line typeMap registration plus its E2E test scaffolding, fixture seed, and the live.js label alignment. --------- Co-authored-by: clawbot <bot@openclaw.dev> Co-authored-by: meshcore-bot <bot@meshcore.local> Co-authored-by: openclaw-bot <bot@openclaw.local> |
||
|
efitenandClaude Opus 4.8
|
5c0de8fb41 |
feat(live): optional "Multibyte only" view filter (#1780) (#1781)
Closes #1780. ## What Adds an opt-in **"Multibyte only"** toggle to the live map controls. When ON, packets whose path hash size is `< 2` bytes (single-byte, or unresolvable) are excluded from the entire live view — feed, map polylines/rain, and the packet counter — in both LIVE and REPLAY modes. - **Default OFF** — no behavior change for existing users. - Persisted in `localStorage` under `live-multibyte-only`. - Distinct from the existing global "hide 1-byte path hops" toggle: that filters individual hops within a path at every render site; this filters whole packets, on the live view only. They share no state. ## How - **`public/hop-filter.js`** — new pure, dependency-free classifier `MC_packetHashSize(rawHex, routeType)` returning `1|2|3`, or `0` when unresolvable. Reads the path-length byte from `raw_hex` (`(pathByte >> 6) + 1`), offset `5` for transport routes (route_type 0/3) else `1` — mirroring the existing `getPathLenOffset`/`computeBreakdownRanges` logic in `app.js`. Lives next to the existing `hopByteLen`/`MC_*` family; `app.js` is untouched (no duplication of the byte math). - **`public/live.js`** — `groupIsMultibyte(packets)` consumes that helper; applied at two render-time sites: the top of `renderPacketTree` (above the counter increment, so the counter reflects multibyte-only) and inside the `rebuildFeedList` group loop (so toggling re-filters the buffered feed). Toggle markup + change handler mirror the existing `liveFavoritesToggle` pattern. ## Why read from `raw_hex` and not the path hops The hash size is a property of the whole packet and is present even for zero-hop packets (where there are no hops to inspect), so reading the path-length byte is correct in all cases. Unresolvable size is treated as single-byte (excluded when ON) — we only show packets we can positively confirm are multibyte. ## Performance (hot path) The filter runs in the packet-render hot path, so: classification is **O(1) per packet group** — it reads the first resolvable observation's `raw_hex` (a short hex string, single `parseInt` of one byte) and short-circuits. No per-packet API calls, no allocation in the loop, no added O(n²). When the toggle is OFF (default) the check is a single boolean guard and does nothing else. The buffered-feed re-filter reuses the existing `rebuildFeedList` pass — no extra traversal. ## Tests - **Unit** (`test-live-multibyte-filter.js`, 9 cases): single/2-byte/3-byte classification, transport-route offset, missing/short/garbage `raw_hex` → 0, whitespace tolerance. - **E2E** (`test-live-multibyte-only-e2e.js`, Playwright): toggle present and defaults OFF; ON hides a single-byte packet while a multibyte one renders; OFF restores it; setting persists across reload. Registered in the CI live-E2E block in `deploy.yml`. ## Docs User-guide entry added in `docs/user-guide/live.md`. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Kpa-clawbotandopenclaw-bot
|
db5520f70f |
fix(nodes): copy URL buttons produce malformed origin#frag URLs (#1753) (#1755)
## Problem The **Copy URL** and **Copy short URL** buttons on the node detail page produced URLs like: ``` https://analyzer.00id.net#/nodes/abcdef… ``` The `/` between the authority and the fragment is missing. RFC 3986 allows that form, but several mobile browsers (and some link-detection heuristics) reject or mis-parse it. ## Fix Three sites in `public/nodes.js` concatenated `location.origin` with a literal that started with `'#/'`. Prepend `/`: - `public/nodes.js:772` — full Copy URL (full pubkey) - `public/nodes.js:783` — Copy short URL (8-char prefix) - `public/nodes.js:1580` — side-pane Copy URL All three now build `https://analyzer.00id.net/#/nodes/…`, which every browser accepts. ## Tests `test-issue-1753-copy-url-slash.js` — extracts every `location.origin + '<literal>'` site from `public/nodes.js` and asserts the literal starts with `/`. Wired into `.github/workflows/deploy.yml`. - **Red commit** `b4df2786` — test added; CI fails on assertion (3 of 4 cases) because the literals still start with `'#/'`. - **Green commit** `2c59f7c0` — three literals fixed to `'/#/nodes/'`; test passes (4/4). ## Verification ``` $ node test-issue-1753-copy-url-slash.js issue-1753 copy-URL slash regression ✅ found at least 3 location.origin + literal sites in public/nodes.js ✅ public/nodes.js:772 literal starts with "/#/" (got "/#/nodes/") ✅ public/nodes.js:783 literal starts with "/#/" (got "/#/nodes/") ✅ public/nodes.js:1580 literal starts with "/#/" (got "/#/nodes/") 4 passed, 0 failed ``` `bash ~/.openclaw/skills/pr-preflight/scripts/run-all.sh origin/master` → clean (all gates + warnings green). Fixes #1753 --------- Co-authored-by: openclaw-bot <bot@openclaw.dev> |
||
|
|
f780fe7d0b |
ci: bump Go test timeout 15m -> 20m (server + ingestor) (#1750)
## Problem The server Go suite runs ~13–15m against a **15m** `go test -timeout`, so on slower CI runners it intermittently hits `panic: test timed out after 15m0s` (`cmd/server`, e.g. `db_test.go`) — false-red CI that a plain rerun clears. Observed on PR #1728 (15m25s on the passing attempt — right at the ceiling). ## Fix Bump both `go test` invocations (`cmd/server` and `cmd/ingestor`) from `-timeout 15m` to `-timeout 20m` for headroom. No test or application code changes — CI workflow only. ## Verification Workflow-only change; this PR's own CI is the confirmation. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Erwin Fiten <e.fiten@opteco.be> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
22fe929da2 |
feat: opt-in mobile client-RX coverage (crowdsourced RF reach) + /api/nodes/resolve (#1728)
Implements #1727. ## What this adds **Mobile client-RX coverage** — an opt-in, crowdsourced RF-coverage feature. A roaming MeshCore **companion** radio (driven by the open-source [corescope-rx](https://github.com/efiten/corescope-rx) PWA, GPLv3) reports which nodes it heard directly, tagged with the phone's GPS and the packet's SNR/RSSI. CoreScope ingests these into a new `client_receptions` table and renders per-node **hex coverage** on the Reach page, plus a standalone **Coverage dashboard** (`#/rx-coverage`) with a top-mobile-observers leaderboard. Also includes **`GET /api/nodes/resolve?prefix=<hex>`** — a read-only node-name lookup by pubkey prefix (`{name, pubkey, ambiguous}`), used by the companion app for friendly names. ## Opt-in — default OFF (zero impact on existing deployments) The whole feature is gated behind one config flag, **disabled by default**: ```jsonc "clientRxCoverage": { "enabled": false } ``` When disabled (the default): the ingestor writes **no** `client_receptions`; the three coverage endpoints return a clean **404**; the UI hides the Coverage nav link, the `#/rx-coverage` route, and the Reach-page toggle. `/api/nodes/resolve` is always available (not coverage-specific). ## How it works ``` companion ──BLE 0x88 (snr+rssi+raw)──▶ corescope-rx PWA ──▶ MQTT meshcore/client/{pubkey}/packets │ ingestor (gated) ──▶ client_receptions (GPS + SNR + heard-key) │ server: pure-Go hex grid ──▶ GeoJSON ──▶ Reach hex overlay + Coverage dashboard ``` - **Direct-only capture:** records only what the companion heard itself and directly — a 0-hop advert's pubkey, or `path[last]` (last forwarder) for FLOOD routes; ≥2-byte path-hash required. Upstream hops discarded. - **No new deps:** hexbins are a pure-Go pointy-top grid over Web Mercator (`cmd/server/hexgrid.go`) computed at query time (`CGO_ENABLED=0` / `modernc.org/sqlite` friendly); frontend uses the existing Leaflet. - **Trust:** companion pubkey = identity; an EMQX ACL binds each client to publish only to its own `meshcore/client/{pubkey}/packets` topic. Payload contract in `docs/client-rx-coverage.md`. ## How to enable / try it 1. In `config.json`, set `"clientRxCoverage": { "enabled": true }` and restart server + ingestor. 2. Point an EMQX (or any broker) listener so a client can publish to `meshcore/client/<pubkey>/packets`; the ingestor already subscribes under `meshcore/#`. 3. Run the [corescope-rx](https://github.com/efiten/corescope-rx) PWA on an Android phone paired (BLE) to a MeshCore companion — it captures heard nodes + GPS and publishes. 4. View results: per-node Reach page → toggle **coverage**, or the **Coverage** dashboard at `#/rx-coverage`. ## What's where - **Ingestor:** `cmd/ingestor/client_reception.go` (ingest), `db.go` (`client_receptions` + `client_observers` schema), `main.go` (gated dispatch), `config.go` (flag). - **Server:** `cmd/server/rx_coverage.go` + `rx_dashboard.go` (endpoints, self-guard 404 when off), `hexgrid.go` (pure-Go grid), `node_resolve.go` (resolve), `routes.go` / `types.go` / `config.go` (wiring + flag + `/api/config/client` field). - **Frontend:** `public/rx-coverage.js` (dashboard), `node-reach-coverage.js` + `.css` (overlay), `node-reach.js` (Reach toggle, flag-gated), `roles.js` (reads the flag, hides nav when off). - **Docs:** `docs/client-rx-coverage.md`. ## Testing - Go: `cd cmd/server && go test ./...` and `cd cmd/ingestor && go test ./...` — green, including new gate tests (`coverage_gate_test.go` in both: off → no rows / 404, on → works) and the rx-coverage / resolve / hexgrid suites. - JS: `node test-coverage-gate.js`, `node test-node-reach-coverage.js` (wired into CI). The Playwright `test-node-reach-coverage-e2e.js` is wired into the e2e job and **skips when `clientRxCoverage` is disabled**, so it's safe under the default-off config. ## Notes for reviewers - The four new routes are registered in `cmd/server/openapi_known_gaps.json` (the existing OpenAPI-completeness ratchet), matching how other not-yet-spec'd routes are tracked. Happy to write full OpenAPI spec entries instead if you prefer. - Commits are split per layer (ingestor / server endpoints / resolve / frontend / CI) for review. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Erwin Fiten <e.fiten@opteco.be> |
||
|
Kpa-clawbotandopenclaw-bot
|
1476b857d9 |
fix(#1716): drop rf-health a11y allowlist entry — subsumed by #1720 (#1731)
Fixes #1716. PR #1720 (merged 2026-06-13) consolidated `.rf-range-btn.active` — along with `.clock-filter-btn.active`, `.subpath-jump-nav a`, `.node-filter-option.node-filter-active`, `.subpath-selected`, and `.analytics-time-range button.active` — into the shared `.btn-active-accent` rule that paints `background: var(--accent-strong)` (`#2563eb`) + `color: var(--text-on-accent)` (`#f9fafb`) = **4.95:1**, WCAG AA pass in both themes. That makes the `#1716` axe allowlist entry obsolete: the underlying violation no longer reproduces. This PR drops the entry and adds a dedicated per-issue regression gate so a future refactor that only breaks `.rf-range-btn.active` (without touching the other consolidated selectors covered by `#1719`) trips with a clear `#1716` citation. ## Strict TDD red→green ### RED — `bce51a60` (test-only) Adds `test-a11y-1716-rf-range-btn-active.js`, a pure-CSS probe with three assertions: - **A1** — `.rf-range-btn.active` is routed through a rule whose body sets `background: var(--accent-strong)` + `color: var(--text-on-accent)`. - **A2** — the legacy `var(--accent)` + `#fff` pair (2.75:1) does NOT reappear on any block listing `.rf-range-btn.active`. - **A3** — numeric contrast on the resolved tokens is ≥ 4.5:1 in both light and dark themes. Locally verified the test FAILS when the consolidated active-button block is reverted to `background: var(--accent); color: #fff`: ``` PASS A3[light]: 4.95:1 (fg=#f9fafb bg=#2563eb) PASS A3[dark]: 4.95:1 (fg=#f9fafb bg=#2563eb) FAIL A1: .rf-range-btn.active is NOT routed through the consolidated (--accent-strong / --text-on-accent) pair — PR #1720 regression FAIL A2: legacy 2.75:1 pair re-emerged on .rf-range-btn.active (bg=var(--accent) fg=#fff) FAIL: 2 assertion(s) tripped on .rf-range-btn.active (issue #1716) ``` Then restored CSS — test passes green on the consolidated state from master. ### GREEN — `eea79791` Removes from `tests/a11y-allowlist.yaml`: ```yaml - route: '/analytics?tab=rf-health' selector: 'button[data-range="24h"]' rule: color-contrast issue: 1716 expires_at: 2026-09-11 ``` ### CI wiring — `b300ce6d` Hooks the new probe into the same `.github/workflows/deploy.yml` step that already runs `test-a11y-axe-1668-selftest.js` and `test-issue-1705-subpath-contrast.js`, so the gate runs on every PR. ## Gate output (after green) ``` $ node test-a11y-1716-rf-range-btn-active.js PASS A1: .rf-range-btn.active routes to var(--accent-strong) + var(--text-on-accent) PASS A2: no legacy var(--accent) + #fff pair on .rf-range-btn.active PASS A3[light]: 4.95:1 (fg=#f9fafb bg=#2563eb) PASS A3[dark]: 4.95:1 (fg=#f9fafb bg=#2563eb) PASS: .rf-range-btn.active gated by consolidated --accent-strong / --text-on-accent pair (issue #1716) ``` The umbrella `#1719` probe also still passes (`PASS: all 4 root-cause patterns ≥ 4.5:1 in both themes`). ## Scope Only the `rf-health` / `.rf-range-btn.active` line. The sibling allowlist entries for `#1714` (nodes), `#1715` (neighbor-graph), and `#1718` (prefix-tool) are out of scope — separate issues, separate PRs. No production CSS touched (PR #1720 did the substantive fix). ## Files changed - `tests/a11y-allowlist.yaml` (−5 lines: drop `#1716` entry) - `test-a11y-1716-rf-range-btn-active.js` (+177 lines: new regression gate) - `.github/workflows/deploy.yml` (+1 line: wire the new gate) ## Preflight All hard gates clean (PII, branch scope, red commit, CSS-var defined, CSS self-fallback, LIKE-on-JSON, sync migration, async migration, XSS sinks). All warnings clean. --------- Co-authored-by: openclaw-bot <bot@openclaw.local> |
||
|
Kpa-clawbotandKpa-clawbot
|
cbe6e94b1a |
fix(#1706): expand axe route coverage to remaining analytics tabs (#1707)
Adds an anti-drift coverage selftest that fails CI if a future analytics tab is added to `public/analytics.js` but not registered in `test-a11y-axe-1668.js` `ROUTES`. Wires it into the `.github/workflows/deploy.yml` axe job alongside the existing reciprocity selftest. ## Relationship to #1706 / #1711 Follow-up to #1706 / #1711 — #1711 already added the 8 missing analytics tabs to `ROUTES` (`subpaths`, `nodes`, `distance`, `neighbor-graph`, `rf-health`, `clock-health`, `scopes`, `prefix-tool`). This PR locks that in: a future tab added to `analytics.js` without a corresponding `ROUTES` entry now breaks CI on this assertion. NOT closing #1706 — that issue is already closed by #1711. ## What the gate does (headline) `test-a11y-axe-routes-coverage.js` scrapes `<button class="tab-btn" data-tab="...">` declarations from `public/analytics.js` and asserts every declared tab is exercised by `test-a11y-axe-1668.js` `ROUTES` as `/analytics?tab=<tab>`. Asymmetry vs the existing selftest, intentional: - `test-a11y-axe-1668-selftest.js` — checks `REGISTERED_ANALYTICS_TABS ⊆ analytics.js` dispatch arms (no dead registrations). - `test-a11y-axe-routes-coverage.js` (new) — checks `analytics.js data-tab buttons ⊆ ROUTES` (no axe-blind tabs). Together they keep the axe matrix honest in both directions. ## Diff scope Only two files vs merge base: - `.github/workflows/deploy.yml` (+1 line — wires the new test into the deploy-job batch) - `test-a11y-axe-routes-coverage.js` (+74 lines, new file) No production code changes, no `ROUTES` changes (those landed in #1711). ## TDD framing Net-new drift gate — no prior assertions to break, no behavior change in shipped UI. Per workspace `AGENTS.md` net-new-test exemption ("net-new UI surfaces … test must land in the SAME PR but doesn't need to be the FIRST commit"), this analogous net-new gate ships green from commit 1. A red→green pair would require a synthetic regression in `analytics.js` or `ROUTES`, which isn't appropriate for an anti-drift guard. ## Local run Local Alpine chromium 136 crashes under Playwright's CDP probe (`posix_fallocate64: symbol not found`) — affects the axe runner generally, not this selftest. The coverage assertion itself is pure Node (file read + regex + set diff) and runs locally clean. Real verdict comes from CI's Playwright-bundled chromium. --------- Co-authored-by: Kpa-clawbot <bot@openclaw.local> |
||
|
|
9b8b613832 |
fix(#1705): WCAG AA contrast on .subpath-selected .hop-prefix (#1712)
## Summary **Regression guard for an already-fixed bug.** The WCAG AA contrast BLOCKER on `.subpath-selected .hop-prefix` called out in #1705 was already resolved by PR #1708 (commit `293efdb6`), which is an ancestor of this branch's base. This PR adds the missing automated test that locks the fix in — it does **not** ship a CSS change. ### Why this is not a TDD red→green sequence Test commit `0d58d1d5` is **green-on-arrival**: the fix it asserts against had already landed days earlier on master. There is no red commit on this branch — running the test at any point on this branch passes. This PR therefore claims the **net-new-test exemption** per `~/.openclaw/workspace-meshcore/AGENTS.md`: bug fixes on EXISTING UI normally require red→green, but where the fix shipped first and the test is a *post-hoc regression guard*, the test lands in the same PR series but does not need to be the first commit. No production behavior is changed by anything in this branch. (The earlier revisions of this body presented a misleading red→green table; that framing was wrong and has been removed.) ## What this PR actually ships Two test files plus CI wiring — all test-only / config-only: 1. `test-issue-1705-subpath-contrast.js` — parses `public/style.css`, extracts `--accent-strong` / `--text-on-accent` per theme, sRGB-composites any `rgba()` foreground against the resolved background, asserts WCAG AA ≥4.5:1 on `.subpath-selected .hop-prefix` in both themes. 2. `test-issue-1705-subpath-contrast-e2e.js` — Playwright/headless-Chromium variant that loads the real `public/style.css` into a DOM mirroring `analytics.js` `renderSubpathsTable`, then asserts `getComputedStyle` contrast on a live `tr.subpath-selected > .hop-prefix`. Catches specificity/cascade regressions the static parser cannot. 3. `.github/workflows/deploy.yml` PR test stage wiring — runs both tests on every PR. ## The bug (historical, already fixed) For context: `.subpath-selected .hop-prefix` measured ~1.87:1 in dark theme (`rgba(255,255,255,0.6)` composited against `var(--accent)` = `#4a9eff`). #1708 (`293efdb6`) swapped `.subpath-selected` background to `var(--accent-strong)` (`#2563eb`) and color to `var(--text-on-accent)` (`#f9fafb`), yielding **4.95:1** in both themes. The child `.hop-prefix` color was set to `inherit` so the prefix cannot be decoratively muted below AA again. ## Parser test — hardening (review r1) Round-1 review pass surfaced 7 must-fix items on the parser test; all addressed: | # | Concern | Fix | |---|---------|-----| | MF3 | Parser asserted declared cascade, not computed style | Added the Playwright E2E variant; `getComputedStyle` resolves specificity natively | | MF4 | CSS comment cited "4.83:1+" while measured value is 4.95:1 | Comment updated to `#f9fafb on #2563eb = 4.95:1` | | MF5 | `extractBlockTokens` silently returned `{}` when the regex didn't match | Throws with selector label; defensive assertion verifies `:root` declares the three tokens | | MF6 | `'inherit'` on the child color fell back to `#ffffff` silently | Now throws; callers either declare the parent color or use the Playwright variant where `inherit` resolves natively | | MF7 | `extractDecl` picked the last regex match across rules with no specificity model | Now throws on N>1 distinct values; warns on N>1 identical values | ## E2E test — hardening (polish v2) - M2: removed dead `<link rel="stylesheet" href="file://...">` element from the test HTML template — `setContent` runs against `about:blank` origin and cannot fetch `file://`, so the link was dead. CSS is injected inline via `page.evaluate` (unchanged). Removed the now-unused `cssHref` declaration. - M3: fixed misleading class comment — the production table uses `.analytics-table` (see `analytics.js` `renderSubpathsTable`), so the fixture's `class="analytics-table subpaths-table"` was misleading. Stripped `.subpaths-table` from the fixture and corrected the comment to cite the real production class. ## Acceptance criteria - [x] `.subpath-selected .hop-prefix` ≥4.5:1 in both themes — verified by both tests (parser + E2E) - [x] Regression test added covering the selected state, wired into PR CI Partial fix for #1705 — leaves the issue open for the audit-probe alpha-compositing work (private tooling, out of scope here). ## Local test results - `node test-issue-1705-subpath-contrast.js` (parser): **PASS** — `light 4.95:1 / dark 4.95:1` - `node test-issue-1705-subpath-contrast-e2e.js` (Playwright): **SKIP** in dev sandbox (chromium relocation error — musl/arm sandbox-in-sandbox); CI installs the Playwright-bundled binary via `npx playwright install chromium` and runs with `CHROMIUM_REQUIRE=1` --------- Co-authored-by: CoreScope Bot <bot@corescope.local> Co-authored-by: Kpa-clawbot <bot@openclaw.local> Co-authored-by: clawbot <clawbot@users.noreply.github.com> |
||
|
Kpa-clawbotandKpa-clawbot
|
76e130b313 |
fix(#1702): grant actions: write to release-fast-path workflow (#1703)
## Summary Fixes the missing `actions: write` permission on `.github/workflows/release-fast-path.yml` so the fallback `gh workflow run deploy.yml` dispatch no longer returns HTTP 403. ## Triage verdict From issue #1702 root-cause section: > Fast-path workflow YAML likely lacks: > ```yaml > permissions: > contents: read > packages: write > actions: write # MISSING — required to dispatch other workflows > ``` > ## Fix > One-line addition to `.github/workflows/release-fast-path.yml` permissions block. ## Root cause `.github/workflows/release-fast-path.yml` lines 16-18 (before this change) only granted `contents: read` and `packages: write`. The fallback step (`gh workflow run deploy.yml` when `:edge`'s `org.opencontainers.image.revision` label doesn't match the tag SHA) calls the GitHub Actions REST API, which requires `actions: write` on `GITHUB_TOKEN`. Without it, the dispatch fails with `Resource not accessible by integration` and the release stalls until an operator manually re-runs the fast-path job after `:edge` rebuilds. ## Change - `.github/workflows/release-fast-path.yml`: add `actions: write` to the workflow-level `permissions:` block. - `cmd/server/release_fast_path_workflow_test.go`: extend the existing config-gate test (issue #1677) to require `actions: write` alongside the previously asserted `contents: read` and `packages: write`. Two commits, red→green: 1. `test(#1702): assert release-fast-path.yml requires actions: write` — extends the assertion. Verified to fail on this commit (`release-fast-path.yml: missing required permission "actions: write"`). 2. `fix(#1702): grant actions: write to release-fast-path workflow` — adds the permission. Test green. ## TDD posture The repo already had a YAML-config gate at `cmd/server/release_fast_path_workflow_test.go` (parses the workflow as text and asserts required permission strings). Strict TDD applied: red commit extends the test, green commit fixes the workflow. No exemption needed. ## Acceptance criteria (from #1702) - [x] `permissions.actions: write` added to the fast-path workflow - [ ] Manual test: tag a scratch SHA where `:edge` is stale; confirm fallback dispatches deploy.yml without 403 — by-design out of CI scope (would require a throwaway tag + race condition); covered by next real release. - [ ] Operator-felt: next release where notes-commit lands AFTER `:edge` build completes works in one pass without manual rerun — verifiable only on next release; in-scope of `Closes #1702` because bullet 1 (the structural defect) is the cause of bullets 2 and 3. ## Preflight `bash ~/.openclaw/skills/pr-preflight/scripts/run-all.sh origin/master` → **clean** (all hard gates pass, no warnings). Closes #1702 --------- Co-authored-by: Kpa-clawbot <kpa-clawbot@users.noreply.github.com> |
||
|
Kpa-clawbotandKpa-clawbot
|
eaac816280 |
feat(#1668): M6 — expanded axe ruleset (mobile + image-alt + label) (#1700)
# M6 — expanded axe ruleset (#1668) **Closes #1668.** M1-M5 already merged: M2 palette/contrast, M3 typography, M4 per-route polish, M5 axe gate + 443→0 fixes. ## What this PR adds ### Expanded axe ruleset - New rules: image-alt, label, aria-required-attr, aria-valid-attr (12 total, verified 0 violations on master after one fix) - Mobile viewport (375×812) added alongside existing 1200×900 desktop - TDD: RED commit `d3e4309e` expands the rule/viewport set deliberately to fail; GREEN commit `5599068f` adds the one needed aria-label fix on audio-lab BPM + Volume sliders ## What this PR does NOT include The letsmesh A/B verification artifact (initially scoped for M6) is split out to a follow-up issue. The capture script needs more work to reliably navigate post-onboarding state on both sites. Tracked separately so the gate-expansion work isn't held up by tooling. ## Test plan - `test-a11y-axe-1668.js` runs new ruleset across both viewports — 0 violations baseline (on master pre-merge AND post-merge) - `test-a11y-axe-1668-selftest.js` unchanged (allowlist semantics still apply) - Anti-tautology: reverting `5599068f` produces 8 net violations on `#alabBPM`/`#alabVol` × 2 themes × 2 viewports ## Notes - Allowlist still empty (per M5 policy — issue# + expires_at required) - M5 token work covered all color-contrast surfaces; M6's image/aria additions only required one fix (audio-lab sliders) --------- Co-authored-by: Kpa-clawbot <bot@openclaw.local> |
||
|
|
d954ea7444 |
feat(#1668): axe-core CI gate for WCAG AA color-contrast (M5) (#1696)
Partial fix for #1668 (M5 of 6). After M1 (audit), M2 (color tokens, #1676), M3 (typography floor, #1679), and M4 (per-route polish, #1681) cleared ~95% of contrast/typography violations, M5 **locks in the wins** by adding an axe-core CI gate that fails the build on any new WCAG AA color-contrast regression. ## What's in the box - `test-a11y-axe-1668.js` — Playwright + `@axe-core/playwright`. Runs every major CoreScope route × `{dark, light}` at 1200×900 desktop, injects axe, runs only the `color-contrast` rule, asserts net violations === 0. - `test-a11y-axe-1668-selftest.js` — fast, deterministic, browser-free unit test that exercises the YAML allowlist parser, the `violationAllowed` matcher, and the route/theme metadata. Runs in the JS unit block (no browser needed). - `tests/a11y-allowlist.yaml` — operator-flagged false-positive allowlist. **0 entries at M5 baseline.** ## Allowlist format Each entry MUST cite a GH issue # and an `expires_at` date. Missing fields = refused. Expired `expires_at` = refused (warning logged). This **forces a periodic revisit** — no permanent suppressions. ```yaml - route: /analytics?tab=channels selector: ".some-known-stale-element" rule: color-contrast issue: 1234 expires_at: 2026-09-01 ``` ## Routes covered (19 × 2 themes = 38 cells) `/`, `/packets`, `/nodes`, `/channels`, `/live`, `/map`, `/observers`, `/compare`, `/analytics?tab={overview,rf,topology,channels,hashsizes,collisions,roles,airtime}`, `/audio-lab`, `/customize`, `/replay`. ## TDD red→green - **RED** (`08adafdb`) — adds the gate + deliberately regresses `--text-muted` from `palette-gray-700` (~10:1) to `#9ca3af` (~2.4:1). axe-core fails on every light-theme cell. - **GREEN** (`f62fb1e0`) — restores the M2 token. Net violations = 0 across all 38 cells. ## Scope discipline - Only `color-contrast` (matches M2/M3/M4 scope). M6 owns `image-alt`, `aria-required-attr`, `label`, mobile viewports, and letsmesh A/B. - No new design tokens. - M2-M4 tokens untouched. ## CI wiring - `.github/workflows/deploy.yml:155` — selftest in JS unit block. - `.github/workflows/deploy.yml:367` — real axe browser run in the Playwright E2E block after the fixture server is up. ## Deps `@axe-core/playwright@4.11.3` + `axe-core@4.12.1` added to `devDependencies`. Pinned versions. --------- Co-authored-by: openclaw-bot <bot@openclaw.local> Co-authored-by: clawbot <clawbot@users.noreply.github.com> |
||
|
Kpa-clawbotandopenclaw-bot
|
547b141530 |
fix(#1697): MQTT sources panel — mobile card layout at ≤640px (#1698)
## Fix
At ≤640px viewports, `public/mqtt-status-panel.js::renderPanel` now
emits a stacked
card per source instead of the 7-column desktop table that overflowed
375px screens
and ran `connected`/`never` together. Desktop (≥641px) keeps the
original table verbatim.
Each mobile card surfaces all 7 data points:
```
[●] gomesh connected 27s ago
wss://mqtt.gomesh.dev
5m: 27 Total: 1247 Disc: 0
```
## Implementation
- `renderTable(sources, now)` — extracted desktop layout (no behavior
change)
- `renderCards(sources, now)` — new mobile card layout, M2 tokens + M3
typography
- `renderPanel` reads `window.innerWidth` and picks one
- Debounced (150ms) `resize` listener flips layout when crossing the
640px bucket
- All colors via `var(--status-green/-red/-yellow)`,
`var(--text-muted)`,
`var(--border)`, `var(--card-bg)` — no inline hex
- All type via `var(--fs-sm)` + `var(--fw-medium)` — no hardcoded px
font sizes in cards
- Broker URL wraps with `word-break: break-all`
- No width ≥400px declared anywhere — eliminates 375px horizontal
overflow
## TDD — red→green visible
- Red commit: `d127d08f` (test only — fails on master with assertion
errors)
- Green commit: `816afc9b` (implementation — all 5 tests pass)
- Wired into `.github/workflows/deploy.yml` JS unit-test block.
## Browser verification (staging 375×812, dark + light)
Overflow probe results (staging, real fixture):
| | scrollWidth | clientWidth | overflow? |
|---|---|---|---|
| BEFORE (master) | 517 | 335 | YES (+182px) |
| AFTER (this PR) | 335 | 335 | no |
Staging URL: http://analyzer-stg.00id.net/#/observers (hot-patched with
the new file).
E2E assertion added: `test-issue-1697-mqtt-mobile-e2e.js:60` ("mobile
375px: renders cards (no desktop table)").
Browser verified: screenshots at
`workspace-meshcore/a11y-audit/operator-reports/1697-{before,after}-{dark,light}-375.png`.
## Preflight gates
All hard gates pass — PII / branch scope / red-commit / CSS-var / CSS
self-fallback /
LIKE-on-JSON / sync-migration / async-migration / XSS sinks (false
positive on
`innerHTML='str'` literal — string is hard-coded constant in empty-state
branch,
no payload data).
Fixes #1697.
---------
Co-authored-by: openclaw-bot <bot@openclaw.local>
|
||
|
|
a4af0285fd |
fix(#1692): parallelize loadObservers + loadPackets in /packets init() (#1693)
## Summary Fixes #1692 — `public/packets.js::init()` serialized `loadObservers()` and `loadPackets()`, blocking `/api/packets` behind `/api/observers`. On loaded CI runners the cumulative wait pushed first-row render to 25–40s, which is the root cause of the persistent #1662 slideover flake and a real operator-felt latency on slow links. ## Fix (Option B — `Promise.all`) ```js // before await loadObservers(); loadPackets(); // after await Promise.all([loadObservers(), loadPackets()]); ``` Option B chosen over fire-and-forget (Option A) because `renderLeft()` synchronously iterates `observers` to build the observer-filter dropdown (`for (const o of observers)` at packets.js:1636). With Option A the menu would render empty on first paint and not refresh until the next user-triggered render. Promise.all preserves the existing render contract while halving worst-case latency — the two fetches now run in parallel and the slower one gates `renderLeft()`. ## TDD - **RED `c7184188`** — `test-issue-1692-packets-init-parallel-e2e.js` stubs `/api/observers` with a 4s delay via `page.route()`, asserts first `tr[data-hash]` < 3000ms. Fails on serial init (blocked at 4s). - **GREEN `903020c5`** — init refactor + wire test into `.github/workflows/deploy.yml` deploy job. ## Out of scope (separate PR per #1692 acceptance #2/#3) The 30s row-wait timeout and 3-iter flake-gate in `test-slideover-1056-e2e.js` + `deploy.yml` were stop-gaps for the underlying serialization. They stay in this PR — they should be reverted in a follow-up after operators confirm the latency fix holds in production. ## Preflight `bash ~/.openclaw/skills/pr-preflight/scripts/run-all.sh origin/master` → all gates pass (PII, branch scope, red commit, CSS vars, LIKE-on-JSON, sync/async migration, XSS). ## Browser verification Local headless chromium on this sandbox crashes on the heavy `/packets` page (small `/dev/shm`, ARM constraints documented in AGENTS.md). Test is gated on CI runner where the harness runs. --------- Co-authored-by: CoreScope Bot <bot@corescope.local> Co-authored-by: clawbot <clawbot@users.noreply.github.com> Co-authored-by: Kpa-clawbot <bot@kpa-clawbot> |
||
|
|
6dfe589b57 |
fix(#1668): per-route polish — hash cells, badges, /live, modals (M4) (#1681)
Partial fix for #1668 (M4 of 6). After M2 (color tokens, PR #1676, ~85% BLOCKER) and M3 (typography floor, PR #1679, ~87% MAJOR), what's left are route-specific structural issues that token/floor passes can't reach. M4 closes those with surgical carve-outs — no new top-level tokens, no semantic encoding flattened. ## Route × selector × fix | Route | Selector | Before | After | |---|---|---|---| | `/analytics?tab=hashsizes` `/analytics?tab=collisions` | `td.hash-cell` + `-collision/-taken/-possible` (302+ M1 violations) | 11px/400; collision-fg 3.61, taken-fg 2.5, possible-fg 1.9 on respective bg | 12px base, 12px/700 on semantic cells. Bg palette preserved (green/yellow/orange still distinct). Inline style in analytics.js bumped 11→12. | | `/packets` `/live` `/nodes` (everywhere `<span class="badge badge-*">`) | All 14 TYPE_COLORS badges (ADVERT, REQUEST, RESPONSE, …) | `${color}20` translucent wash with `color: ${color}` — ratio **1.0–4.25, all BLOCKER** | `syncBadgeColors` rewritten: pick readable fg by luminance, darken bg in 8% steps until AA (≥4.5:1). All 14 PASS (4.57–7.94). TYPE_COLORS itself unchanged — map dots / live-feed dots keep full hue. | | `/live` | `.vcr-live-btn` ("LIVE") | `rgba(239,68,68,0.2)` + status-red fg = **1.0:1** | Solid `--status-red` + #fff = 5.25:1; 12px/700 | | `/live` | `.vcr-scope-btn.active` (1h/6h/12h/24h selected) | `--accent-bg` wash + `--text` = 2.98:1 BLOCKER | `--accent-strong` + `--text-on-accent` (M2 tokens, AA) | | `/live` | `.vcr-btn` `.vcr-scope-btn` | 0.9rem/400, 0.75rem/400 (thin-small) | 14px/500, 12px/500 desktop; 12px/600 ≤640px | | `/live` | `.live-feed-empty` | 12px/400 (thin-small) | 12px/500 | | `/packets` (path hops) | `.path-hops .hop-named` | font-size inherited (variable) | explicit 12px/600 | ## TDD & gating - **RED** `341f47f1` — 23 assertion failures (9 typography + 14 badge-contrast). New gate `test-issue-1668-m4-per-route.js` executes `syncBadgeColors` in a VM sandbox and asserts each emitted `.badge-*` rule clears WCAG AA; also checks rule-level font-size/font-weight floors. - **GREEN** `6ef17491` — both axes 0/0. - Test wired into `.github/workflows/deploy.yml:144` alongside M3. - Anti-tautology proven locally: `git stash public/roles.js` returns the test to FAIL with the badge assertions; pop restores GREEN. ## Re-scan findings `a11y-audit/m4-rescan.jsonl` — `/live` (timed out in M1) now probes cleanly: 29 dark / 39 light residuals all caught by this PR. Channel-add and customize modals probed clean (M2 tokens already cover; nothing chip-level needed). ## Out of scope M5 (axe CI gate) and M6 (letsmesh side-by-side A/B) are next milestones. --------- Co-authored-by: agent <agent@openclaw.local> Co-authored-by: meshcore-bot <bot@meshcore> Co-authored-by: Kpa-clawbot <bot@kpa-clawbot> Co-authored-by: openclaw-bot <bot@openclaw> |
||
|
|
79cf453660 |
feat(#1633): customizer toggle to hide 1-byte path hops everywhere (#1689)
## What Customize-v2 toggle **Hide 1-byte path hops** (Display tab). Default OFF — operators opt in. When ON, 1-byte path-hash prefixes are filtered at every render site without touching what's stored or what the firmware does. Render sites wired: - **Packets list / detail** (`packets.js renderPath`) — group header, child observations, detail dt/dd, BYOP overlay. Empty result renders `(1-byte filtered)`. - **Map polylines** (`map.js drawPacketRoute`) — intermediate hops tagged `_hopHex`; origin/destination (from payload, no `_hopHex`) always survive. - **Route view** (`route-view.js`) — unique-paths picker + group counts key on the filtered hop list, so routes that only differ by 1-byte hops collapse. - **Analytics route patterns** (`analytics.js`) — filters INPUT rows whose `rawHops` contain any 1-byte token; header reports filtered/total. ## Why 1-byte hashes collide ~8-way at ~2k relay nodes (Cascadia scale). The collisions inflate polyline noise, route-pattern row counts, and chip clutter without adding signal. See #1633 for the full hypothesis. ## How (pure render-time) New `public/hop-filter.js`: - `MC_getHide1ByteHops()` / `MC_setHide1ByteHops(on)` — localStorage `meshcore-hide-1byte-hops`, default OFF. - `MC_isVisibleHop(hop, opts)` — predicate. - `MC_filterPathHops(hops, opts)` — non-mutating array filter. Nothing in the ingest / store / decode path changes. The hop hex stays in `path_json`; only the render iterators drop it. ## Tests `test-issue-1633-hide-1byte-hops.js` — 8 assertions: - Default OFF (back-compat). - `hopByteLen` semantics. - `isVisibleHop` ON drops 1-byte, keeps 2/3-byte. - `filterPathHops` non-mutating. - `HopDisplay.renderPath` chip set after filter. - Map polyline positions[] filter preserves origin/destination. - Analytics route-pattern aggregation key collapses on filtered hops. Wired into `.github/workflows/deploy.yml`. Red commit: `6baa3f13` (5/8 ON-branch assertions failed on stubs). Green commit: `5c0bbdba` (8/8 pass). ## Browser verify Staging deploy of changed files. Packet `99ef781f42eb7249` (all 1-byte path): - BEFORE (toggle OFF): `3 HOPS — Station Rat → KO6IFX-R5 → little russia`. - AFTER (toggle ON): `3 HOPS — (1-byte filtered)`. Customizer toggle visible + working in Display tab. Fixes #1633. --------- Co-authored-by: openclaw-bot <bot@openclaw.dev> Co-authored-by: clawbot <bot@openclaw.local> |
||
|
Kpa-clawbot
|
dd2b3d2e21 | ci(#1662): cut slideover flake-gate from 20× to 3× — 5% per-iter flake = 64% per-run fail at N=20 | ||
|
|
a8c99c61fd |
fix(#1659): block analytics endpoint until first pass complete (503 Retry-After) (#1688)
## Summary Fixes #1659 — analytics cards no longer show the post-restart slice when "All data" is selected. ## Root cause After server restart, `s.recompRF` / `s.recompTopology` / `s.recompChannels` cache the FIRST computation, which is the small in-RAM observations slice (background chunk-loader has not yet backfilled history). The recomputer serves that slice through `GetAnalyticsRFWithWindow`'s default shortcut for an entire recompute interval, while the client pins it via `CLIENT_TTL.analyticsRF`. UX: cards show a tiny window even when the user selects "All data". ## Fix shape (option B from the issue body) Server-side per-recomputer warm-up gate: - `cmd/server/analytics_warmup_1659.go` adds a per-recomputer `firstPassDoneNs` atomic timestamp, set ONLY by the first successful `runOnce()` (CAS-guarded for idempotency). `IsWarmingUp_1659()` / `FirstPassDoneAt_1659()` are lock-free reads. - `cmd/server/analytics_recomputer.go` `runOnce()` calls `markFirstPassDone_1659()` after every successful compute. - `cmd/server/routes.go` handlers for RF / Topology / Channels: when the request is the default shape (`region=="" && area=="" && window.IsZero()`) AND the matching recomputer is still warming up, return `503` + `Retry-After: 5` + `{"error":"analytics warming up","retry_after_s":5}`. Windowed / region-filtered requests bypass the gate (they already bypass the recomputer cache, so they are unaffected by the warm-up bug). Client-side: - `public/app.js` `api()` helper retries any 503 response, honoring `Retry-After`, with exponential backoff capped at 30s, max 6 attempts (~63s total). - Small "Computing analytics…" banner appears while any warm-up retry is in flight, dismissed once the request resolves. Pages can override via `window.onWarmup_1659`. ## Tests RED commit `8b2b2d7` ships failing-on-assertion tests + a stub. GREEN commit `2716c23` lands the fix and flips them green. - `cmd/server/analytics_warmup_1659_test.go` — 3 cases: 503 during warmup, 200 after first pass, windowed request bypasses gate. - `test-1659-analytics-warmup.js` — 3 cases: Retry-After honored, retry cap bounded, non-503 errors not retried. Wired into `.github/workflows/deploy.yml`. ## Preflight overrides - cross-stack: justified — server-side 503 contract MUST be paired with client-side retry-and-banner handling; splitting across two PRs would land a half-working fix. Fixes #1659. --------- Co-authored-by: corescope-bot <bot@corescope.local> Co-authored-by: openclaw <openclaw@local> |
||
|
|
d910ea0208 |
feat(#1638): confidence rating weighted by hash mode (#1687)
Fixes #1638. ## Problem `getConfidenceIndicator` in `public/nodes.js` treats every observation as equal evidence, so a node seen 5 times via 1-byte hash prefixes (which collide ~8-way across a typical mesh) scores the same as a node seen 5 times via 6-byte prefixes (effectively unambiguous). The user asked for confidence to respect ambiguity. ## Change - `cmd/server/neighbor_graph.go` — new `CountsByMode map[int]int` on `NeighborEdge`, bumped in `upsertEdge` / `upsertEdgeWithCandidates` based on the observation's hash-prefix byte length (1/2/4/6). Merged in `resolveEdge` when ambiguous→resolved edges collapse. - `cmd/server/neighbor_api.go` — `NeighborEntry.counts_by_mode` exposed (omitempty), and `dedupPrefixEntries` merges per-mode counts when an unresolved prefix entry collapses into a resolved one. Flat `Count` field preserved for back-compat. - `public/nodes.js::getConfidenceIndicator` — weights observations by mode: 1-byte=0.125, 2-byte=0.5, 4/6-byte=1.0. A single 6-byte sighting counts ~8× a raw 1-byte one. HIGH triggers when EITHER the legacy heuristic clears OR weighted count ≥3. Legacy entries without `counts_by_mode` keep working (default weight 0.5). - Tooltip now shows the per-mode breakdown (e.g. "Observations: 5 (1-byte: 3, 6-byte: 2)"). ## TDD - RED: `cmd/server/neighbor_graph_test.go::TestBuildNeighborGraph_CountsByMode` — fixture with 1/2/4-byte sightings asserts per-mode tally (commit `838965f3`). - RED: `test-confidence-indicator.js` — 6-byte mostly-sighted neighbor must outrank 1-byte mostly-sighted neighbor at equal flat count (commit `4bd5e18e`). - GREEN: implementation in commit `7511606d`. All 4 JS tests pass; new Go test passes; full Go suite passes (two pre-existing flakes unrelated, both pass when isolated). ## Browser verification Synthetic side-by-side of OLD vs NEW classifier against representative inputs — see screenshot. 1-byte-only and 6-byte-only at the same flat count diverge from MEDIUM/MEDIUM to MEDIUM/HIGH, and 3 6-byte sightings now upgrade where 20 1-byte sightings stay MEDIUM. ## Preflight overrides - check-branch-scope: cross-stack: justified — backend exposes the new `counts_by_mode` field and the frontend consumes it; the whole point of the change. ## Compat - `Count` field unchanged in shape and value. - `counts_by_mode` is `omitempty`; legacy persisted edges (loaded from `neighbor_edges` via `neighbor_persist.go`) get no per-mode breakdown and fall back to the default weight (0.5) — no UI regression. --------- Co-authored-by: bot <bot@local> Co-authored-by: corescope-bot <bot@corescope.local> |
||
|
|
a2004351d3 |
fix(#1684): staging disk monitor + cleanup cron (#1686)
## Summary Adds a staging VM disk-usage monitor + daily cleanup cron, fixing the gap surfaced by #1684 (staging hit 100% disk during a hot-patch, no alert, no cleanup). ## What landed - **`scripts/staging/disk-monitor.sh`** — parses `df -P <mount>`, classifies usage `<80 ok / >=80 warn / >=90 error / >=95 alert`, emits to stderr + journald via `logger -p`, exits non-zero on `error|alert` so the systemd unit surfaces as failed. - **`scripts/staging/disk-cleanup.sh`** — daily prune of `/tmp` snapshot patterns (`*.db`, `staging-snap.*`, `cs-*`, `node-compile-cache`) older than 7d + `docker builder/image prune --filter until=72h --filter label!=keep`. Honors `CORESCOPE_CLEANUP_DRY_RUN=1`. - **`scripts/staging/test-disk-monitor.sh`** — pure-bash unit tests for the testable helpers (22 cases covering threshold boundaries, df parsing, invalid input, severity→priority mapping). - **`DEPLOY.md`** — install one-liner with full inline systemd unit + timer content (15-min monitor, daily 03:30 cleanup). Uses `<STAGING_HOST>` placeholder. - **`.github/workflows/deploy.yml`** — wires `test-disk-monitor.sh` into the Go build & test job. ## TDD - Commit `26185967` (RED): tests against stub helpers — `PASS=5 FAIL=17` on assertions. - Commit `d31a1082` (GREEN): real helpers — `PASS=22 FAIL=0`. ## Phase 3 — `staging-snap.db` root cause `grep -rn staging-snap.db cmd/ public/ scripts/` → **zero hits**. The 4.4 GB orphan was a manual debug artifact, not committed code. The cleanup retention rule prevents recurrence. Partial fix for #1684 — leaves issue open for operator to verify install on staging and confirm alert fires at 85%. --------- Co-authored-by: corescope-bot <bot@corescope.local> Co-authored-by: clawbot <bot@openclaw.dev> |
||
|
|
6aa5146b93 |
fix(#1660): FE warm-up banner reads X-Corescope-Load-Status + polls /api/healthz (#1683)
## Summary Partial fix for #1660 — adds an FE-only global warm-up banner that surfaces server-side load state to users instead of letting "data may be incomplete" look like silent breakage. Implements sub-deliverables **(1)** and **(3)** from the triage. Sub-deliverable (2) (per-card "recomputing" pill) is deferred — it depends on a new server-side `recomputer.first_pass_done` flag that pairs with #1659. ## What it does - New `public/warmup-banner.js` mounts a sticky `role="status"` live region at the top of `<body>`. Pure helper `getWarmupMessages()` is fully unit-tested in isolation. - Consumes both signals the server already exposes: - `X-Corescope-Load-Status` response header (set by `cmd/server/chunked_load.go:446` on every API response) — captured via a thin `window.fetch` wrapper. - `GET /api/healthz` — polled every 30s while not in steady-state, torn down once `ready=true` AND `from_pubkey_backfill.done=true`. - Messages per acceptance criteria: - `loading` → "⏳ Loading historical data — counts may be incomplete." - `from_pubkey_backfill.done=false` → "Backfilling pubkey index: 12,400 / 87,500 (14%)" - `ingest_liveness.<src>.lastReceiptUnix` older than 5 min → "No packets from `<src>` in N min." - Banner fades out (opacity + max-height transition) once steady-state is reached. ## Files - `public/warmup-banner.js` — new module (pure helpers + DOM mount + poll + fetch interceptor). - `public/style.css` — `.warmup-banner` rules; all colors via existing `--warn-bg` / `--warn-text` / `--warning` CSS variables (customizer-safe, no inline hexes). - `public/index.html` — loads `warmup-banner.js` immediately before `app.js` so the fetch wrapper is installed before other modules issue requests. - `test-warmup-banner.js` — 8 tests: 6 pure-helper + 2 vm-DOM E2E that stub `/api/healthz` returning `ready:false` → asserts banner visible, then flips to `ready:true` → asserts the `warmup-banner--hidden` class is applied (sub-deliverable 3). ## TDD red → green - **Red:** `ca5f9837` — `test(#1660): RED — failing tests for warmup banner message derivation` — stub `getWarmupMessages` returns `[]`; CI fails on 3 assertion failures (compiles cleanly, fails on `assert.ok(msgs.length >= 1)` etc — not on import/build). - **Green:** `0d07efdf` — `feat(#1660): GREEN — warmup banner reads X-Corescope-Load-Status + polls /api/healthz` — implementation lands; all 8 tests pass. ## Test output ``` warmup-banner.js (#1660): ✅ exports getWarmupMessages and shouldShowBanner ✅ loading header alone produces a "historical data" message ✅ from_pubkey_backfill.done=false produces a progress message with pct ✅ stale ingest source >5min produces a "No packets from" message ✅ steady-state ready=true + backfill done + fresh ingest → no banner ✅ isSteadyState reflects ready+backfill predicate ✅ E2E: stub /api/healthz ready=false → banner visible ✅ E2E: flip /api/healthz to ready=true → banner fades (hidden class) passed=8 failed=0 ``` ## Preflight `bash ~/.openclaw/skills/pr-preflight/scripts/run-all.sh origin/master` — **clean** (PII / branch scope / red commit / CSS-var defined / CSS self-fallback / LIKE-on-JSON / sync migration / async-migration gate / XSS sinks all PASS, no warnings). ## Performance - Poll runs every 30s and only while `ready=false || from_pubkey_backfill.done=false`. Stops immediately on steady state. No hot-path impact. - Fetch wrapper adds one `.then()` per response to read a single header — O(1). - Banner DOM is one `<div>` with a `<ul>` of ≤3 `<li>`s. Re-render is a single innerHTML set. ## Out of scope (explicit) - Sub-deliverable (2) — per-card "↻ Recomputing…" pill. Requires a new `recomputer.first_pass_done` field on `/api/healthz` (small `cmd/server/analytics_recomputer.go` addition) and is grouped with the #1659 recomputer redesign. Not in this PR. - No backend code changed. Partial fix for #1660. --------- Co-authored-by: Kpa-clawbot <bot@kpa-clawbot> Co-authored-by: corescope-bot <bot@corescope.local> |
||
|
Kpa-clawbotandopenclaw-bot
|
efd66ea3f5 |
feat(mqtt): per-source status endpoint + Observers panel (#1682)
## Summary Adds MQTT source status visibility per #1043 acceptance criteria: - **Ingestor:** per-source counter registry (`cmd/ingestor/source_status.go`) tracking `connected`, `lastConnectUnix`, `lastDisconnectUnix`, `lastPacketUnix`, `connectCount`, `disconnectCount`, `packetsTotal`, `packetsLast5m` (sliding 5-min window via per-second buckets keyed by unix second — no stale-leak), `lastError`. Wired at the existing OnConnect / ConnectionLost / DefaultPublish callsites alongside the liveness watchdog. Idempotent registration so counters survive reconnects. Snapshot emitted in the existing stats file under `source_statuses` (additive, `omitempty`). - **Backend:** new `GET /api/mqtt/status` handler reads the ingestor stats file and returns the per-source list. **Broker passwords are masked** via a regex over the `scheme://user:pass@host` form (covers mqtt/mqtts/tcp/ssl/ws/wss). Mask is also applied to `lastError` as defense-in-depth (broker libs occasionally quote the failing URL). OpenAPI completeness gate satisfied with a `routeDescriptions` entry. - **Frontend:** small self-contained panel (`public/mqtt-status-panel.js`) mounted above the Observers table. Auto-refreshes every 10s, color-codes each row (green = connected + recent packet, yellow = connected idle, red = disconnected), and tears down its timer on SPA route change. ## TDD - Red commit `f19a93b5` — stub `/api/mqtt/status` handler + assertion test that the broker password is `****`-redacted. Test fails on the assertion (handler passes the URL through verbatim). Compile-clean — assertion-fail, not build-fail. - Green commit `77042e41` — `maskBrokerURL` helper + table-driven unit tests across all schemes + handler rewires to mask both `Broker` and `LastError`. - Subsequent commits land the ingestor wiring and the frontend panel. ## Tests ``` $ cd cmd/server && go test -run 'TestMqttStatus|TestMaskBrokerURL' -v ./... PASS: TestMqttStatus_MasksBrokerPassword PASS: TestMqttStatus_EmptyWhenNoStatsFile PASS: TestMaskBrokerURL_Patterns (10 subtests) $ cd cmd/ingestor && go test -run 'TestSourceStatus|TestSnapshotSourceStatuses' -v ./... PASS: TestSourceStatus_BasicLifecycle PASS: TestSourceStatus_Disconnect PASS: TestSnapshotSourceStatuses_ReturnsAll $ node test-mqtt-status-panel.js 7 passed, 0 failed ``` Full `go test ./...` clean in both `cmd/server` and `cmd/ingestor`. ## Preflight overrides - `cross-stack`: justified — issue #1043 is intrinsically full-stack (ingestor stats → server endpoint → observers panel). Per-stack split would land an unreachable endpoint or a fetch with no backend. - `check-xss-sinks` (public/mqtt-status-panel.js:55): justified — the flagged `innerHTML=` is a fully-static literal (empty-state placeholder, no payload data interpolated). All payload-bearing `innerHTML=` sites in this file run through `escapeHTML` (defined in the same file); the test `renderPanel never echoes a plaintext password (defense-in-depth)` exercises the rendered HTML against payload strings. ## Acceptance criteria - [x] `/api/mqtt/status` returns per-source connection state — `cmd/server/mqtt_status.go` - [x] UI panel shows all configured sources with live status — `public/mqtt-status-panel.js` - [x] Connection state updates on reconnect/disconnect events — `MarkConnect` / `MarkDisconnect` wired in `cmd/ingestor/main.go` - [x] Broker URLs don't expose passwords in the API response — `maskBrokerURL` + 13 test cases - [x] Works with 1-N sources — registry is keyed per-source, snapshot iterates the map **Partial fix for #1043** — per-packet `mqtt_source` attribution (the issue's "Follow-up" section) is **deferred** per the `mc-bot-triaged:v1` triage and the autofix comment ("Per-packet attribution deferred to follow-up issue"). That work requires a new observation-row column and DB schema migration, both explicitly out of scope for this PR. Refs #1043 --------- Co-authored-by: openclaw-bot <bot@openclaw.local> |