Files
meshcore-analyzer/tests/e2e/test-issue-748-node-analytics-coverage-e2e.js
efitenandClaude Opus 5.5 927a8eec8a fix(nodes): node analytics rates cover the data in memory, and say so (#748) (#2211)
Refs #748 (M3a: correct numbers within the in-memory window; the SQL
history in M3b is a separate plan, see the end).

## Problem

The node analytics range picker offers 24h, 7d, 30d and All (365d), but
the store only holds its in-memory window (`retentionHours`, then
eviction). `GetNodeAnalytics` still divided availability and packets/day
by the requested range. Measured on our live instance (14-day window),
repeater BE-BRE-ON8AR:

| Range | Availability | Packets/day |
|---|---|---|
| 7d | 88.7% | 91 |
| 30d | 42.6% | 265 |
| All | 3.5% | 22 |

Nothing on the page said that the data stopped at the edge of the
window.

## What changes

- `coverageStart` returns the edge of the in-memory window: the latest
of `oldestLoaded` (background-load progress), `now - retentionHours`,
and a new `evictedThrough` that eviction sets to the newest `FirstSeen`
it removed, so memory-pressure eviction inside the retention window
moves the edge too.
- `s.packets[0]` is not that edge, and the first version of this PR used
it. Chunks are selected by `last_seen`, so a transmission first heard
weeks ago and heard again recently sits at the head with its old
`FirstSeen`. On staging (7-day window) it returned 21/07.
- When the edge lies more than a minute after the requested start, node
analytics counts only transmissions from the edge, computes availability
and packets/day over that span, and returns `timeRange.dataFrom` and
`timeRange.partial`.
- The minute of slack keeps a range equal to the retention window from
reading as partial. Without it every 7d request on staging (168h) was
partial, because the two edges came from separate clock reads.
- The analytics page shows "Data from 2026-10-07: the server keeps about
7 days of packets in memory. Availability and packets/day cover that
period, not the full 365 days." when partial.
- **UTC fix.** The window bounds of node analytics and node hop
analytics are formatted in UTC. They are compared as strings with UTC
`FirstSeen` values, so on a non-UTC host the 7d window lost its oldest
hours (2 at +02:00). Docker images run in UTC, which is why this did not
show in production.
- `nodeTxsSince` (one caller) is inclusive, so a packet exactly on the
edge is kept.

## Tests

- Go, `node_analytics_coverage_test.go`:
  - full coverage within memory;
  - 30d over 3 days in memory;
  - empty store;
- a re-heard 85-day-old transmission at the head of `s.packets` (edge
stays at retention, packet not counted, no bucket before `dataFrom`);
  - memory-eviction edge;
  - eviction records `evictedThrough`;
  - retention 30 s shorter than the range is not partial, while 30d is;
  - window bounds under a +02:00 `time.Local`.
- Two tests were checked to fail on the old code: the local-zone test
(166 of 168 packets, a `+02:00` bound) and the retention test without
the slack (`dataFrom 09:57:08` vs `from 09:56:38`).
- Full `cmd/server` suite passes locally (skipping
`TestSaveGeoFilterPreservesFileMode`, a Windows-only failure on clean
master). One run of five reported `FAIL` and four reruns were green; I
did not capture which test failed.
- Frontend unit: `tests/unit/test-issue-748-node-analytics-coverage.js`
(5 assertions on the real `node-analytics.js` in `vm`), in
`test-all.sh`; the full suite passes.
- E2E: `tests/e2e/test-issue-748-node-analytics-coverage-e2e.js`
(registered in `deploy.yml` and `scripts/non-unit-tests.json`), against
the real fixture response with no interception: 7d is not partial and
shows no notice; All is partial and the notice names the real start
date. 2/2 locally.

## Validation

Staging (retentionHours 168), same repeater, after the background load
completed:

| Range | Before | After |
|---|---|---|
| 7d | 82.7% | 83.3%, not partial |
| 30d | 19.3% | 83.3%, partial from 2026-10-07 |
| All | 1.6% | 83.3%, partial from 2026-10-07 |

In the browser the notice appears under All and not under 7d.

## Not done

- M3b, older history from SQLite. In the database only the sender is
indexed (`from_pubkey`). Recipients sit in `decoded_json`, and relay
hops only in `observations.resolved_path` (filled for about 27% of
observations on our live instance, not indexed). A fallback today would
show only a node's own packets for older days, a different meaning than
the in-memory part. Doing it properly needs a per-node index written by
the ingestor (pubkey, transmission, role); that is a separate plan.
- The other analytics endpoints with a range (observer analytics, RF)
were not checked for the same dilution.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 09:27:21 +02:00

72 lines
3.2 KiB
JavaScript

/**
* E2E test (#748): the node analytics page says from when its data runs when
* the requested range reaches past the server's in-memory window.
*
* The CI server runs without a retention window or memory cap and the
* fixture's packets span a few months: 7d is fully covered, and "All"
* (365d) reaches past the oldest packet in the database. That is the
* "database" case, so the notice must say the database holds no older
* packets, not that the server keeps them in memory. The memory and
* loading wordings are covered by the unit tests.
*
* Usage: BASE_URL=http://localhost:13581 node test-issue-748-node-analytics-coverage-e2e.js
*/
const { chromium } = require('playwright');
const BASE = process.env.BASE_URL || 'http://localhost:3000';
async function test(name, fn) {
try {
await fn();
console.log(` ✅ ${name}`);
} catch (err) {
console.log(` ❌ ${name}: ${err.message}`);
process.exit(1);
}
}
function assert(cond, msg) {
if (!cond) throw new Error(msg || 'Assertion failed');
}
async function run() {
const browser = await chromium.launch({
headless: true,
executablePath: process.env.CHROMIUM_PATH || undefined,
args: ['--no-sandbox', '--disable-gpu', '--disable-dev-shm-usage']
});
const page = await (await browser.newContext({ viewport: { width: 1400, height: 900 } })).newPage();
page.setDefaultTimeout(15000);
console.log(`\nRunning #748 node analytics coverage E2E tests against ${BASE}\n`);
const nodes = (await (await page.request.get(`${BASE}/api/nodes?limit=50`)).json()).nodes || [];
assert(nodes.length > 0, 'fixture has no nodes');
const pk = nodes[0].public_key;
await test('7d inside the in-memory window: not partial, no notice', async () => {
const r = await (await page.request.get(`${BASE}/api/nodes/${pk}/analytics?days=7`)).json();
assert(!r.timeRange.partial && !r.timeRange.dataFrom, `7d timeRange: ${JSON.stringify(r.timeRange)}`);
await page.goto(`${BASE}/#/nodes/${encodeURIComponent(pk)}/analytics`, { waitUntil: 'domcontentloaded' });
await page.waitForSelector('#timeRangeBtns');
assert(!(await page.$('[data-testid="analytics-coverage-notice"]')), 'notice shown for a fully covered range');
});
await test('All reaches past the oldest packet: partial, notice names the start date and the database', async () => {
const r = await (await page.request.get(`${BASE}/api/nodes/${pk}/analytics?days=365`)).json();
assert(r.timeRange.partial && r.timeRange.dataFrom, `365d timeRange: ${JSON.stringify(r.timeRange)}`);
assert(r.timeRange.dataFromReason === 'database', `365d reason: ${r.timeRange.dataFromReason}`);
await page.click('#timeRangeBtns button[data-days="365"]');
await page.waitForSelector('[data-testid="analytics-coverage-notice"]');
const text = await page.$eval('[data-testid="analytics-coverage-notice"]', (el) => el.textContent);
const day = r.timeRange.dataFrom.slice(0, 10);
assert(text.includes('Data from ' + day + ': the database holds no older packets'), `notice: ${text}`);
assert(!text.includes('in memory'), `notice claims a memory limit: ${text}`);
});
await browser.close();
console.log('\nAll #748 node analytics coverage E2E tests passed.');
}
run().catch((e) => { console.error(e); process.exit(1); });