Commit Graph
10 Commits
Author SHA1 Message Date
n30nex d07186af43 fix(nodes): preserve names and metadata in preview panels (#2315)
Fixes #2309 and #2312. Partial follow-up to #1154 (U1 and tablet
timestamp parity).

Node previews omitted Last Relayed, Traffic share and Hash Prefix, while
tablet previews read `advert_name` instead of the API's `name`. Shared
rendering now presents those fields consistently across full detail,
side preview and tablet, with the existing timestamp tooltips.

Relay attribution is labelled neutrally: ambiguous relay evidence no
longer claims a node is alive. Existing certainty/status logic is
unchanged. Measured zero, legacy score fallback, role applicability,
missing values and escaped names/hash tooltips are covered. A resize
regression prevents duplicate score-row IDs across retained desktop and
tablet previews.

Only the selected node's metadata is formatted. Existing request counts
remain five for full/side and two for tablet; no new fetch, cache, hot
loop, setting, customizer value or dependency.

Validation: 806 helper checks, all 218 standalone suites, 46 browser
checks in both themes, eight health checks, and the main browser suite
(149 passes, three existing fixture skips). The unchanged backend tree
passed full server/ingestor suites. Independent reviews completed; the
parent verified the two-line follow-up from its resize check.

Test-only RED `e0f681ae` produced [eight expected assertion
failures](https://github.com/Kpa-clawbot/CoreScope/actions/runs/40232409958/job/121286783705).
Follow-up `2bb1ccd` repairs the help fixture's scroll range without
relaxing its checks; all 13 help checks pass across three font/CPU
configurations. [Final-head CI
passes](https://github.com/Kpa-clawbot/CoreScope/actions/runs/40234768485),
including all browser shards and architecture builds.
2026-10-26 15:44:10 +01:00
n30nex cb38ac356c fix(packets): preserve All time across reloads and links (#2279)
Desktop All time now survives reloads and shared links, including a
selected older grouped packet and observation. An explicit zero is
retained in storage and the URL; missing/invalid values still use the
default. Mobile requests retain their existing time and row limits.

Fixes #2231.

Validation: RED reproduces the lost zero; all standalone frontend suites
and the unchanged server suite pass. The local main browser suite
(`test-e2e-playwright.js`) passes 149 checks with three explicit
fixture-dependent skips. The gesture suite also preserves the existing
time window when opening a packet trace. The new regression covers
selection, reload, fresh links, older grouped observations and mobile
bounds.

Normalization is constant work; URL updates reuse the existing
serializer and add no requests. No new settings or dependencies. Common
ingestor validation is covered by the separate #2272 test-stability
change.


[Exact-head
CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/40160982457)
passed on commit 5278ade6, including all three browser shards.
2026-10-25 12:11:20 +01:00
efitenandClaude Opus 5.5 3960fadd96 fix(pages): drop late responses that belong to an earlier page visit (#2218, #2230, #2200) (#2248)
Fixes #2218, fixes #2230, fixes #2200.

Node analytics, the account page and analytics await their data, then
write into the shared `#app` container. A response that lands after the
user has navigated away replaces the new page. In #2218's reproduction,
the Nodes list, with 200 rows, became an Analytics heading.

## The shared part

`public/app.js` keeps one view generation, `viewGeneration()`:
- `navigate()` bumps it before the old page's `destroy()`, so every page
init counts as a new visit, including a switch to another node of the
same page type.
- A page captures the value at init, and after each `await` drops the
result when the value has moved on.

`.eslintrc.json` declares it as a global.

## Per page

**#2218, `node-analytics.js`**
- `loadAnalytics` and `loadBatteryChart` check the generation on both
success and error. A page-local `loadSeq` makes a time-range click a new
load on the same visit, so an older range can't land over a newer one.
- `NodeHopAnalytics.load` already checked `container.isConnected`, so it
is unchanged.

**#2230, `account.js`**
- The render after `CSAuth.ready()` runs only while the generation is
unchanged.
- `destroy()` was empty. It now clears `shown.app` and `shown.userId`.

**#2200, `analytics.js`**
- A page-local `_loadSeq` sits next to the generation. Results and
errors of an older window/region selection or an earlier visit are
dropped, so a late failure of an old load no longer shows an error over
newer results.
- The bulk load renders only the six tabs drawn from its data. A tab
that loads its own data (Prefix Tool, Nodes, Roles, ...) is no longer
rebuilt or replaced by the bulk result: a deep-linked one renders at
init, and a window, region or area change re-renders it directly.
- `renderTab` guards the six tabs drawn from the bulk data (overview,
rf, topology, channels, hashsizes, collisions). Until the data arrives
they show "Loading analytics…", or the current load's error. Overview no
longer dereferences an undefined `rf`.
- Three related fixes in the same lines:
  - The error message is now escaped with `esc()`; it was inserted raw.
- `destroy()` removes the RegionFilter and AreaFilter listeners. They
piled up on every visit and called `loadAnalytics` from other pages.
  - `destroy()` resets the stored error.

## Tests

| File | Assertions | Covers |
|---|---|---|
| `tests/unit/test-issue-2218-node-analytics-stale-render.js` | 12 |
Loads the real router: the generation rises on every init, including
another node. Late success and error after leaving, a stale visit of the
same node, an older range landing last, late battery success and error.
The current response, battery and error still render |
| `tests/unit/test-issue-2230-account-stale-render.js` | 6 | Leaving
before ready (login view and "not enabled"), re-entering before ready,
the current render, an auth change after `destroy()` |
| `tests/unit/test-issue-2200-analytics-stale-load.js` | 19 | An older
selection landing last, a late rejection of an older selection, leave
and re-enter (success and error), the current error rendered escaped and
kept on a tab click, Overview and RF clicked before the data |
| `tests/e2e/test-issue-2218-node-analytics-stale-e2e.js` | 4 steps |
Holds `/api/nodes/*/analytics` with `page.route`, returns to `#/nodes`,
then releases the response as a 200 and as a 500. The 200 rows stay and
no Analytics heading appears; a normal visit still renders |

How I checked them:
- With the production files reverted and the tests kept, 7 of 12, 4 of 6
and 7 of 12 unit assertions fail. With the fix, all pass.
- The E2E fails on the unfixed frontend with "Nodes rows 200 before
release, 0 after".
- Single mutations, each reverted:
- removing the success check in `loadAnalytics` (analytics) fails 2
assertions;
  - removing the check after `CSAuth.ready()` fails 3.

Other test changes:
- `test-analytics-theme-refresh-before-data.js` and
`test-user-management-ui.js` get a `viewGeneration: () => 0` stub.
- The new E2E is registered in `scripts/non-unit-tests.json` and
`deploy.yml` (shard 2, next to the #748 suite). The unit tests are in
`test-all.sh`.
- The broad suite's "Node side panel Details link navigates" test
selected an anchor that now opens Analytics, as #2218 notes. It now
clicks `#nodesRight .node-detail-btn`.

Results:
- `sh test-all.sh` passes, and `npx eslint public/*.js` reports 0
errors.
- E2E on a fixture seeded as in CI:

  | Suite | Result |
  |---|---|
  | `test-e2e-playwright.js` | 148 passed, 3 skipped |
  | `test-user-management-e2e.js` | 28/28 |
  | the new #2218 suite | 4/4 |
  | `test-issue-748-node-analytics-coverage-e2e.js` | 2/2 |
  | `test-analytics-fluid-charts.js` | 8/8 |
  | `test-issue-1657-analytics-channels-group-sprites-e2e.js` | 5/5 |

## Not done

- Two tabs that each fetch their own data can still overwrite each other
after a fast tab switch between them. #2200 does not ask for that.
- `nodes.js` `navigateToNode` calls its own `init` without the router,
so it does not bump the generation. None of the three pages fixed here
depends on that.
- I placed the E2E in shard 2 without measuring suite times, which the
`deploy.yml` comment asks for.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 10:39:15 +02:00
efitenandClaude Opus 5.5 fbeb9e4d71 ci: frontend coverage from every E2E suite, merged across the shards (#2204)
Fixes #2203.

## Problem

The frontend coverage badge (35.16% on run 38413268001) came from two
sources on shard 2: one dump part-way through `test-e2e-playwright.js`
and the `collect-frontend-coverage.js` tour. The other E2E suites run
against the instrumented build but never save `window.__coverage__`,
including the suites written for single pages (`path-inspector.js`
4.72%, `touch-gestures.js` 14% on that run).

## Change

1. `9d3bfad5` `test(e2e): red:` `tests/unit/test-e2e-coverage.js`, with
fake pages, contexts and browsers (no Chromium needed). On the
placeholder module the first assertion fails.
2. `c00bbb12` `feat(e2e):` the collector.
- `scripts/e2e-coverage-hook.js` is a preload (`node --require`) that
wraps `chromium.launch` from `playwright`. `playwright` re-exports
`playwright-core`, so a suite's own `const { chromium } =
require('playwright')` gets the same, wrapped object.
- `scripts/e2e-coverage.js` reads `window.__coverage__` before every
call that replaces or ends a page's document: `goto`, `reload`,
`setContent`, `goBack`, `goForward`, `close`, and context and browser
`close`. Pages from `newPage`, `newContext` and the context `page` event
are all tracked.
- Each read zeroes the page's counters, so a document read twice is not
counted twice. Maps travel once per file; later reads carry counts only.
- The page returns a JSON string. Returning the object cost about 10 s
per suite, because Playwright serialises it node by node.
- A read is given up after 5 s, so a busy page cannot hold up a suite's
cleanup.
- On exit the hook writes one file per suite to `.nyc_output/`, or
nothing without coverage. A failed write is logged and never changes the
exit code.
3. `64487c4c` `ci:` the workflow.
- On master pushes the E2E run step sets `NODE_OPTIONS=--require
./scripts/e2e-coverage-hook.js`. On PRs it is empty, so PR runs are
unchanged.
- Each shard runs `nyc merge` (one file per suite is about 6 MB, mostly
the same maps) and uploads `coverage-shard.json` with its results
instead of `.badges/`.
- The `e2e-test` job merges the three shards and makes the frontend
badge (it used to come from shard 2 alone).
- The merge, setup-node and badge steps are `continue-on-error`. When
`nyc report` gives no statement percentage the step warns and writes no
badge, so a registry hiccup cannot publish 0%.
- `test-e2e-playwright.js` no longer writes its own `e2e-coverage.json`:
the hook covers that suite completely, including the tests after the old
dump. `collect-frontend-coverage.js` stays as an extra source.

## Performance

Measured locally against an instrumented server (Windows, Chrome):

| Suite | Without hook | With hook |
|---|---|---|
| `test-path-inspector-coverage-e2e.js` | 1.9 s | 2.2 s |
| `test-touch-gestures-coverage-e2e.js` | 17 s | 16 s |
| `test-e2e-playwright.js` (stops at the same local-fixture failure in
both runs) | 52 s | 52 s |
| `browser.close()` on a page in a 30 s busy loop | 153 ms | 5.2 s |

The hook only loads on master pushes.

## Testing

- `tests/unit/test-e2e-coverage.js` covers:
  - reads before each document-replacing call and before close;
  - pages opened by the page itself;
  - context and browser close;
  - summing without double counts;
  - a failed read, and a read that never answers;
  - the file writer, including a failing write.
- Nine deliberate breaks of the library each turn it red:
  - dropping the counter reset;
- dropping the `page` event, the `goto` read, the `goForward` read, the
`close` read or the browser `close` read;
  - removing the timeout or the write guard, or writing an empty store.
- The test also fails if it stops before its end. Without that check, a
read that never returns would let node exit 0 half-way.
- Real suites with the hook, measured on their own (the badge run had
all suites):

| File | Badge run 38413268001 | Own suite with the hook |
|---|---|---|
| `path-inspector.js` | 4.72% | 52.75% |
| `touch-gestures.js` | 14% | 72.96% |

- Exit codes and pass counts were unchanged with the hook for these
suites (the first three were run before the 5 s read timeout was added):
  - `test-slideover-1056-e2e.js` (6 contexts);
  - `test-a11y-axe-1668.js` (`@axe-core/playwright`);
  - `test-issue-1150-404-state-e2e.js`;
  - the three above.
- The badge step, run locally against two merged shard files:
  - It gives the same per-file numbers as `nyc report` on the inputs.
- With no shard coverage, or with `nyc report` failing, it warns and
writes no badge.
- `sh test-all.sh` passes locally, apart from
`test-issue-1956-release-routing.js`, which needs the workflow's
`GITHUB_REF_NAME` and was skipped. The test inventory classifies 321
suites. `actionlint` 1.7.7 reports only the self-hosted runner label, as
on master.

**Not tested:** the shard and badge steps in CI. They only run on a
master push, so the first master run after merge is the real test. I do
not know what the merged number will be. I will compare the badge with
the merged report in that run's `e2e-test` log.

**Not counted:** a full page load the suite does not start through one
of the wrapped calls, such as a link that leaves the SPA or
`location.reload()` from page code.

## Not in this PR

- The `tests/unit/` suites, which load frontend files through
`vm.createContext` without instrumentation (noted in #2203).
- Coverage on PR runs. The overhead above is small, so this could follow
if wanted.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 22:04:03 +02:00
n30nex 29095658e9 fix(nodes): explain packet and observation counts (#2132)
Red commit: `d8a6d42` ([CI
assertion](https://github.com/Kpa-clawbot/CoreScope/actions/runs/37695889461)).
Review-regression red: `f73643b` ([CI
assertion](https://github.com/Kpa-clawbot/CoreScope/actions/runs/37703995494)).

Fixes #2131. Partial fix for #1154: U2 only; other audit items remain
open.

Both node views explain distinct transmissions versus recorded
observations. The shared help is reachable by keyboard and touch, stays
readable at either scroll edge, and remains open when the pointer enters
its text. Escape dismisses help while preserving node context and focus;
a second Escape keeps the existing navigation behavior.

Count expressions and API requests are preserved. One controller per
nodes page handles one active help element and one queued animation
frame. Scroll/resize work is coalesced; listeners are removed on page
teardown. Work is constant relative to packet/node counts, with no
dataset scans or new settings/dependencies.

E2E assertion added: `tests/e2e/test-e2e-playwright.js:114`. Thirteen
cases cover exact counts, accessible description, keyboard/touch, hover
transfer, Escape, and viewport/scroll bounds. Existing metric assertions
are unchanged.

Browser verified: http://localhost:63508 using Chromium, controlled
fixtures and actual API data. Desktop/mobile screenshots retained
locally; additional checks exercised resizing and repeated page cleanup.

Local validation: 189 standalone suites; core browser 148 passed, 3
fixture-dependent skips (existing manual skips unchanged);
location/metric suite 6 passed; syntax, theme variables and lint passed
(0 errors, 92 existing warnings).

## Preflight overrides
- OpenClaw preflight/profile unavailable; repository checks and local
Chromium used.
- Windows ingestor symlink fixture lacks permission. Backend files are
unchanged; full Linux Go CI is required.
2026-10-08 22:37:15 +02:00
n30nex 9e49db9725 feat(nodes): group adverts by observed route evidence (#2085)
Red commit: `ac60248` ([two intended label
failures](https://github.com/Kpa-clawbot/CoreScope/actions/runs/37137429333)).
Original grouping red: `767da9b`
([CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/36353796690)).

Fixes #2073.

#2088 is merged. Rebased onto master `415362c`; all eight patches are
unchanged, including the test-first history.

Recent Adverts groups the bounded sample by authoritative `advert_kind`:
Flood, Mixed flood / direct (empty path), Direct (empty path), and Other
/ unknown. Missing evidence stays unknown. Each advert appears once,
preserving reception metadata, order, packet links, counts and
origin/history explanations. Wording describes observed remaining paths
without inferring original send mode or RF distance.

Validation at `6317baf`:
- All 185 standalone suites passed; syntax, XSS and lint passed (zero
errors, 92 existing warnings).
- Browser verified: `http://127.0.0.1:51827` (temporary Go fixture
server, now stopped), desktop pane/full view and mobile. Mixed evidence,
exact links and caveats checked again after pushing.
- The expanded fixture exposes the existing #2104 failure in the
unmodified core harness. Supplemental integration with #2105's exact
test correction passes 135 checks, with three existing skips. Both PRs
remain separate.
- [Fresh green
CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/37138366726)
passed; three independent automated reviews found no must-fix issues.

E2E assertion added: `tests/e2e/test-e2e-playwright.js:142`.

Grouping is O(n) over the existing 20-advert sample. No new requests,
settings, colors or dependencies.

## Preflight overrides
External OpenClaw runner/profile unavailable; repository checks and
actual local Chromium were used.
2026-10-03 23:49:08 +02:00
n30nex 99a326a6b3 test(packets): verify observation hex after rerenders (#2105)
Red commit: `ca553e4` ([expected assertion
failure](https://github.com/Kpa-clawbot/CoreScope/actions/runs/37137426691)).

Fixes #2104.

The observation hex test now opens the existing grouped fixture
directly, selects A → B → A with stable-ID locators, and asserts the
selected ID and exact raw bytes after each render. Missing fixture data
fails explicitly. Distinct observation frames are seeded after schema
migration; this test's fixed sleeps and silent skips are removed.

Rebased onto master `415362c` after #2090 merged. Both patches are
unchanged. The previous #1122 layout blocker now passes against the
actual current server and assets.

Validation at `5db8017`:
- All 185 standalone frontend suites passed.
- Core browser: 133 passed, three existing skips; #1122 layout 6/6,
filter 11/11, grouped-collapse 4/4.
- Syntax, YAML, fixture SQL, XSS and lint passed (zero errors, 92
existing warnings).
- Fresh red CI reached the intended detached-observation assertion.
[Green
CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/37138187496)
passed; three independent automated reviews found no must-fix issues.

Only test code and fixture setup change; no configuration/customizer
implications.

## Preflight overrides
External OpenClaw runner/profile unavailable; repository checks and
actual local Chromium were used.
2026-10-03 23:48:32 +02:00
n30nex b9fb3d244c feat(analytics): split adverts using recorded route evidence (#2088)
Red commits: `dc7dcbe`, `76e3217` ([assertion
failures](https://github.com/Kpa-clawbot/CoreScope/actions/runs/37059977385)).

Fixes #2041; supplies #2085's backend contract.

Records at most two route bits per retained advert. Node flood counts
now use the same evidence, including mixed and transport-flood adverts.
Analytics failures log/count errors while core observation, path and
liveness writes continue. The unrelated recent-advert limit clamp is
removed.

Direct adverts (empty path) describes the observed remaining path across
valid hash widths. Firmware can forward an advert down to an empty path;
this cannot establish original send mode or RF distance. The `zero_hop`
API spelling remains compatible.

Successful post-upgrade evidence writes survive observation replacement
and restart. Older overwritten frames and failed writes can leave gaps.
The UI explains automatic background backfill.

## Measured validation

Synthetic Windows workloads; timings are not production guarantees:

- 300K retained adverts, one-hour window: median 64.35→39.84 ms; 13.98
MB→145 KB allocated. Seven-day allocations unchanged; five paired
samples.
- 1M transmissions/11M observations: backfill 5m43s. Simple 10 Hz
writer: all 3,423 observations persisted; p99 28 ms, maximum 323 ms.
- Simulated pre-backfill cursor: retained evidence ready 4m21s; full
catch-up 6m41s, 122 cache invalidations. Normal completed-migration
startup bulk-loads current evidence.

Portable opt-in scale tests and window benchmarks are included. All 183
frontend suites, full server tests, targeted races, lint and real-API
desktop/mobile checks passed. Full [Linux
CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/37063210229)
passed at `36def9c`; three independent reviews are clear. Local Windows
testing hit the unchanged symlink-privilege test. No configuration
changes.

Preflight overrides: external script/OpenClaw profile unavailable;
repository checks and Chromium used.
2026-10-03 12:16:55 +02:00
n30nex 31744c6da2 test(channels): isolate WS assertions from initial loading (#2087)
Fixes #2086.

The #1468 WebSocket browser checks could fail when initial channel
loading completed between their separate before/after evaluations. An
orphan message added no channel, yet unrelated loading changed the count
from 0 to 5; the positive control could also lose its sentinel when
loading replaced the list.

Each check now captures before state, processes its packet, and captures
after state in one synchronous browser evaluation. All four original
assertions and both message payloads are unchanged. This updates one
test file only, with no production, dependency or configuration changes.

## Validation

- Delayed real API loading reproduces both failures on unchanged master.
- Both corrected checks pass with loading held and with loading
completed; each uses one evaluation.
- Parent independently ran the delayed-loading harness and full browser
suite: 131 passed, three existing fixture skips.
- All 183 standalone frontend suites passed. Syntax, inventory,
whitespace and privacy checks passed.

TDD justification: test synchronization repair only. Existing assertions
demonstrate the baseline failure; no production behavior or manufactured
failing test was added.

Browser checks used Chromium against unchanged production source.
OpenClaw profile/external preflight were unavailable; repository checks
ran directly, with a 60-second local navigation budget. Final CI must
pass before merge.
2026-09-30 11:39:34 +02:00
Alex B 893773338e chore(tests): move root test-*.js into tests/unit and tests/e2e (#2036)
Moves 290 root test-*.js into tests/unit (177, listed in test-all.sh) and tests/e2e (113, classified in scripts/non-unit-tests.json), per #1981 and PR-D of #1385. Root goes from 348 entries to 48. test-all.sh and test-fixtures/ stay put. The inventory guard now fails if a test reappears in the root or sits in the wrong folder.

Verified independently of the diff: the invoked sets are unchanged (test-all.sh 177 before and after, deploy.yml 96 before and after, both identical as sets), and a full local run of test-all.sh on master and on the branch produced 4702 output lines each whose only differences are absolute paths, stack-trace line numbers shifted by the REPO_ROOT line, the inventory wording and two perf ratios. The guard was mutation-checked: a test back in the root, a unit suite in tests/e2e, and a suite dropped from test-all.sh each make it exit 1. CI run 35246304316 ran 97 suites from tests/e2e and is green.

Follow-up 9335c51d finished the instruction files: no bare root test command is left in AGENTS.md, the squad charters, .github or docs, and every tests/ path they name resolves.

Merged by the interim maintainer without a second human reviewer: CI and the local runs above are the independent checks.

Known and deliberately out of scope: 18 of the 113 files in tests/e2e are invoked by no runner at all, and one of them cannot run anywhere because it requires jsdom, which is not a declared dependency. Tracked separately.
2026-09-17 19:06:07 +02:00