Files
meshcore-analyzer/tests/unit/test-issue-748-node-analytics-coverage.js
efitenandClaude Opus 5.5 927a8eec8a fix(nodes): node analytics rates cover the data in memory, and say so (#748) (#2211)
Refs #748 (M3a: correct numbers within the in-memory window; the SQL
history in M3b is a separate plan, see the end).

## Problem

The node analytics range picker offers 24h, 7d, 30d and All (365d), but
the store only holds its in-memory window (`retentionHours`, then
eviction). `GetNodeAnalytics` still divided availability and packets/day
by the requested range. Measured on our live instance (14-day window),
repeater BE-BRE-ON8AR:

| Range | Availability | Packets/day |
|---|---|---|
| 7d | 88.7% | 91 |
| 30d | 42.6% | 265 |
| All | 3.5% | 22 |

Nothing on the page said that the data stopped at the edge of the
window.

## What changes

- `coverageStart` returns the edge of the in-memory window: the latest
of `oldestLoaded` (background-load progress), `now - retentionHours`,
and a new `evictedThrough` that eviction sets to the newest `FirstSeen`
it removed, so memory-pressure eviction inside the retention window
moves the edge too.
- `s.packets[0]` is not that edge, and the first version of this PR used
it. Chunks are selected by `last_seen`, so a transmission first heard
weeks ago and heard again recently sits at the head with its old
`FirstSeen`. On staging (7-day window) it returned 21/07.
- When the edge lies more than a minute after the requested start, node
analytics counts only transmissions from the edge, computes availability
and packets/day over that span, and returns `timeRange.dataFrom` and
`timeRange.partial`.
- The minute of slack keeps a range equal to the retention window from
reading as partial. Without it every 7d request on staging (168h) was
partial, because the two edges came from separate clock reads.
- The analytics page shows "Data from 2026-10-07: the server keeps about
7 days of packets in memory. Availability and packets/day cover that
period, not the full 365 days." when partial.
- **UTC fix.** The window bounds of node analytics and node hop
analytics are formatted in UTC. They are compared as strings with UTC
`FirstSeen` values, so on a non-UTC host the 7d window lost its oldest
hours (2 at +02:00). Docker images run in UTC, which is why this did not
show in production.
- `nodeTxsSince` (one caller) is inclusive, so a packet exactly on the
edge is kept.

## Tests

- Go, `node_analytics_coverage_test.go`:
  - full coverage within memory;
  - 30d over 3 days in memory;
  - empty store;
- a re-heard 85-day-old transmission at the head of `s.packets` (edge
stays at retention, packet not counted, no bucket before `dataFrom`);
  - memory-eviction edge;
  - eviction records `evictedThrough`;
  - retention 30 s shorter than the range is not partial, while 30d is;
  - window bounds under a +02:00 `time.Local`.
- Two tests were checked to fail on the old code: the local-zone test
(166 of 168 packets, a `+02:00` bound) and the retention test without
the slack (`dataFrom 09:57:08` vs `from 09:56:38`).
- Full `cmd/server` suite passes locally (skipping
`TestSaveGeoFilterPreservesFileMode`, a Windows-only failure on clean
master). One run of five reported `FAIL` and four reruns were green; I
did not capture which test failed.
- Frontend unit: `tests/unit/test-issue-748-node-analytics-coverage.js`
(5 assertions on the real `node-analytics.js` in `vm`), in
`test-all.sh`; the full suite passes.
- E2E: `tests/e2e/test-issue-748-node-analytics-coverage-e2e.js`
(registered in `deploy.yml` and `scripts/non-unit-tests.json`), against
the real fixture response with no interception: 7d is not partial and
shows no notice; All is partial and the notice names the real start
date. 2/2 locally.

## Validation

Staging (retentionHours 168), same repeater, after the background load
completed:

| Range | Before | After |
|---|---|---|
| 7d | 82.7% | 83.3%, not partial |
| 30d | 19.3% | 83.3%, partial from 2026-10-07 |
| All | 1.6% | 83.3%, partial from 2026-10-07 |

In the browser the notice appears under All and not under 7d.

## Not done

- M3b, older history from SQLite. In the database only the sender is
indexed (`from_pubkey`). Recipients sit in `decoded_json`, and relay
hops only in `observations.resolved_path` (filled for about 27% of
observations on our live instance, not indexed). A fallback today would
show only a node's own packets for older days, a different meaning than
the in-memory part. Doing it properly needs a per-node index written by
the ingestor (pubkey, transmission, role); that is a separate plan.
- The other analytics endpoints with a range (observer analytics, RF)
were not checked for the same dilution.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 09:27:21 +02:00

81 lines
3.5 KiB
JavaScript

/**
* #748 — node analytics coverage notice: shown when the server's in-memory
* window starts after the requested range. Loads the real
* public/node-analytics.js in a vm sandbox.
*/
'use strict';
const fs = require('fs');
const path = require('path');
const vm = require('vm');
const assert = require('assert');
const REPO_ROOT = path.resolve(__dirname, '..', '..');
let passed = 0, failed = 0;
function t(name, fn) {
try { fn(); passed++; console.log(' ✓ ' + name); }
catch (e) { failed++; console.error(' ✗ ' + name + ': ' + e.message); }
}
const ctx = { window: {}, registerPage: () => {}, Date, Math, Number, String, isNaN, console };
vm.createContext(ctx);
vm.runInContext(fs.readFileSync(path.join(REPO_ROOT, 'public', 'node-analytics.js'), 'utf8'), ctx);
const notice = ctx.window.NodeAnalyticsCoverage && ctx.window.NodeAnalyticsCoverage.notice;
console.log('node-analytics.js coverage notice');
t('exposes NodeAnalyticsCoverage.notice', () => {
assert.strictEqual(typeof notice, 'function');
});
t('memory limit: names the start date, the window and the requested days', () => {
const html = notice({ from: '2025-10-14T12:00:00Z', to: '2026-10-14T12:00:00Z', days: 365, dataFrom: '2026-09-30T09:12:00Z', dataFromReason: 'memory', partial: true });
assert.ok(html.includes('data-testid="analytics-coverage-notice"'), html);
assert.ok(html.includes('Data from 2026-09-30'), html);
assert.ok(html.includes('about 14 days'), html);
assert.ok(html.includes('not the full 365 days'), html);
});
t('database younger than the range: says so, no memory claim', () => {
const html = notice({ to: '2026-10-14T12:00:00Z', days: 365, dataFrom: '2026-07-01T00:00:00Z', dataFromReason: 'database', partial: true });
assert.ok(html.includes('Data from 2026-07-01: the database holds no older packets'), html);
assert.ok(!html.includes('in memory'), html);
assert.ok(html.includes('not the full 365 days'), html);
});
t('background load still running: says so', () => {
const html = notice({ to: '2026-10-14T12:00:00Z', days: 30, dataFrom: '2026-10-13T12:00:00Z', dataFromReason: 'loading', partial: true });
assert.ok(html.includes('the server is still loading older packets'), html);
});
t('window too short for rates: says so instead of the rates claim', () => {
const html = notice({ to: '2026-10-14T12:00:00Z', days: 7, dataFrom: '2026-10-14T06:00:00Z', dataFromReason: 'memory', partial: true },
{ availabilityPct: null, avgPacketsPerDay: null });
assert.ok(html.includes('too short for availability and packets/day'), html);
assert.ok(!html.includes('not the full'), html);
});
t('rateText: null reads as a dash, numbers keep their suffix', () => {
const rateText = ctx.window.NodeAnalyticsCoverage.rateText;
assert.strictEqual(rateText(null, '%'), '—');
assert.strictEqual(rateText(undefined), '—');
assert.strictEqual(rateText(0, '%'), '0%');
assert.strictEqual(rateText(12.5), '12.5');
});
t('full coverage: no notice', () => {
assert.strictEqual(notice({ from: '2026-10-07T12:00:00Z', to: '2026-10-14T12:00:00Z', days: 7 }), '');
});
t('missing or malformed time range: no notice', () => {
assert.strictEqual(notice(null), '');
assert.strictEqual(notice({ partial: true, dataFrom: 'garbage', to: '2026-10-14T12:00:00Z', days: 30 }), '');
});
t('one day of data reads singular', () => {
const html = notice({ to: '2026-10-14T12:00:00Z', days: 7, dataFrom: '2026-10-13T13:00:00Z', partial: true });
assert.ok(html.includes('about 1 day of packets'), html);
});
console.log(`\n${passed} passed, ${failed} failed`);
process.exit(failed ? 1 : 0);