2 Commits

Include renames

Author	SHA1	Message	Date
efitenandClaude Opus 5.5	a08f2f4e7c	fix(liveness): count only evidence that can be the node alone (#2260) (#2261) ... Fixes #2260 (part 1 of 2: liveness; the adaptive offline threshold follows in a separate PR). ## Problem A repeater silent since 2026-10-18 showed **Active — Last relayed 4m ago** and **Last Heard 36m**, and its offline mail never fired. Every hit after that date was the 1-byte hop `67`, shared by nine nodes: - the live resolver's affinity tiebreak (`resolvePathForObs`) put those transmissions in `byNode`, which fed `stats.lastHeard` and `LastHeardMap`; - `last_relayed` folds in the 1-byte prefix bucket on purpose (#662), and drove the status in `roles.js` and the notify offline check. ## Fix The store keeps `certainHeard` per node: the newest transmission that can only be that node. | Evidence | Counts? | |---|---| | Own packet (`pubKey`, `srcPubKey`) | ✅ | | Flood relay hop whose prefix, at the packet's path hash size, matches exactly one path-capable node | ✅ | | 1-byte hop shared with other repeaters | ❌ | | Direct-route path (hops still to come, `Mesh::removeSelfFromPath`) | ❌ | | TRACE (path bytes are SNR) | ❌ | | Destination (`destPubKey`) | ❌ | A multibyte repeater legitimately appears as a 1-byte hop (the originator picks the hash size, `Mesh::routeRecvPacket` appends at that size), so only uniqueness decides. The rule is applied in every load path through `indexResolvedPathHops` and the new `indexColdLoadRelayHops`: Load, loadChunk, scanAndMergeChunk, live ingest, late observations. Liveness reads only that: - `/api/nodes` and the node detail: `last_heard` is the newer of `last_seen` and the certain last heard. Before, it was a plain copy of `last_seen`. - health / bulk-health `stats.lastHeard`. - `LastHeardMap` → notify offline check. The separate relay input (`lastRelayed`, `relayTimes`, `notifyInput.Relayed`) is removed. - `roles.js getEffectiveHeardMs`: newest known timestamp, without `last_relayed`. The side pane no longer says "Last relayed"; the Last Relayed row stays (analytics) with an updated tooltip. - New status **`uncertain`** (repeaters and rooms): nothing certain within the silent window, but the shared prefix (`last_relayed`) still in paths. Grey badge with explanation, an *Uncertain* filter on the nodes list and the map, dimmed like stale. The offline mail for such a node says only its shared prefix was still seen. - A prefix also held by a path-capable node in `inactive_nodes` is not unique; adverts with `signatureValid: false` are no evidence; `/api/nodes/search` reports the same `last_heard`. Attribution for analytics (`byNode`, packet counts, `last_relayed`, relay counters, usefulness) is unchanged. ## Performance Hot path: `noteCertainRelays` runs per observation on ingest under the write lock: two hex bytes of header parsing plus one prefix-map lookup per resolved hop. ``` BenchmarkNoteCertainRelays-16 10307640 116.2 ns/op 0 B/op 0 allocs/op (8-hop 2-byte path, 2,000 repeaters) BenchmarkLastHeardMap-16 1902 0.77 ms/op (5,000 nodes; was 3.7 ms scanning byNode) ``` Memory: one string per node ever certainly heard, dropped together with the node's `byNode` list on eviction. ## Tests - `certain_heard_loadpaths_test.go`: every load path (Load, loadChunk, LoadChunked — each with and without resolved_path — live ingest, late observations) feeds `certainHeard`; each of the 8 call sites verified red with a nil prefix map. - `certain_heard_test.go`: hop width per route/payload type; retired namesake prefix; failed signature; uncertain offline mail; shared vs unique 1-byte vs 2-byte hops; direct/TRACE/nil prefix map; cold-load path; sender vs destination; eviction; `lastHeardValue`; `/api/nodes`, detail and health agree. - Regression for #2260: last certain packet on day X, 20 shared-prefix hops after it → `LastHeardMap` stays at X and `node.offline` goes bad after 72 h. Verified red with the uniqueness check mutated out. - Notify: `TestEvaluateCertainHeard` replaces the relay-input test. - Frontend: `last_relayed` alone no longer keeps a repeater or room active; newest timestamp wins (an old `_liveSeen` can't hide a fresh certain relay). - `cmd/server` `go test ./...` and `test-all.sh` pass locally, also on a trial merge into a downstream fork with 457 extra commits. ## Validation Ran the server on the freshened e2e fixture: list, detail and health report the same last heard; e.g. a repeater whose `last_relayed` is 3 min newer than anything certain keeps its certain time. **No browser screenshot**: Playwright isn't installed on this machine; CI's E2E covers the pages. ## Not changed - The ingestor's relay touch of `nodes.last_seen` (#1854) resolves a shared hop only when exactly one candidate is adjacent in the neighbor graph; it did not fire in this case. Left as is. - The `lastHeard=` list filter still filters `last_seen` in SQL. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-10-22 19:11:30 +02:00
efitenandClaude Opus 5.5	de364a8bb6	feat: mail notifications for watched nodes (user management part E) (#2140) ... Part E of #2128: a logged-in user watches nodes and gets one mail when a watched node goes offline, comes back, or reports a low battery. Admins can add instance events: a new foreign node, and an observer going offline or back. This covers the per-user mail part of #775. Stacked on #2139 (part D, which builds on #2138). Review the commits after that branch. ## The situation A sysop learns that a repeater went silent, or that its battery is running down, only by opening CoreScope. #775 asks for notifications. Accounts (part A) now give a verified address to mail and a mailer with delivery status. ## What this PR adds **Events.** They use the instance thresholds, so a mail does not disagree with the node page: - `node.offline`: silent for the role's `healthThresholds` window. Last heard comes from the packet store, and repeaters and rooms count relayed traffic (#1598). - `node.battery`: advert telemetry below `batteryThresholds.lowMv`, recovered at `lowMv + 100`. - Admins only: `foreign.new` (once per node) and `observer.offline`. **Store.** Schema v5 with `notification_prefs`, `notification_watches` and `notification_state`. **Server** (only with `userManagement.notifications.enabled`) - A notifier next to the janitor, every `intervalMinutes` (5). The evaluator is a pure function of watches, prefs, stored states and node snapshots. - No burst after a restart or a new watch: a subject's first evaluation stores its state without mailing. The loop waits for the startup load. While ingest is stale (newest packet older than 30 minutes) the offline checks pause, and after recovery they wait one silent window. - One mail per user per check. Limits: 20 per user and 100 per instance per rolling 24 hours, 50 watches per user. A change over a limit is recorded and never mailed later. - Every mail has a one-click unsubscribe (`List-Unsubscribe` and `List-Unsubscribe-Post`). The GET only redirects to a confirm page, so link scanners cannot unsubscribe anyone. Node names are cleaned of control and bidi characters before they go into a mail. - Routes: `GET`/`PUT /api/account/notifications`, `PUT`/`DELETE /api/account/notifications/watches/{pubkey}`, `POST /api/account/notifications/watch-my-nodes` (copies the synced "my nodes" list), `GET`/`POST /api/notifications/unsubscribe`. `mailer.Message` gains `Headers`. **Frontend.** A "Notify me" toggle on the node pages, a Notifications section on the account page (`#/account?section=notifications`), an unsubscribe page, and the notification mail count on the admin overview. Spec: [`docs/specs/2026-10-07-node-notifications-design.md`](https://github.com/efiten/CoreScope/blob/feat/notifications/docs/specs/2026-10-07-node-notifications-design.md). ## Performance Each check reads watches, prefs and states from `users.db` (one query each), the watched nodes from the analyzer DB in chunks of 500, and last-heard times from the packet store. | Measurement | Result | |---|---| | One check on our staging instance, 130,000 packets in memory, 1 watcher | 6.9 ms, of which 0.93 ms holding the store read lock | | Evaluator benchmark, 100 users with 50 watches each, 2,000 nodes | 1.75 ms | No work is added to ingest, broadcast or any request path. ## Verification - `internal/users` and `internal/mailer` with `-race`; the `cmd/server` suite plus `-race` on the notifier tests; 74 new Go tests. The same Windows-only failure as noted in #2138 applies. - `sh test-all.sh` exits 0; the XSS gate in diff mode passes. - User-management E2E: 26 of 26 steps locally. With notifications off the new steps fail, so they do test the flag. - On our staging and production instance since 7 October 2026. ## Not in this PR - Other channels (Discord, Telegram, webhooks). Detection is separate from delivery, so one can be added. - The topology, RF and anomaly alerts of #775. - If the whole server starts after a feed outage that already ended, the grace window is unknown and a watcher can get one wrong offline mail. The user guide says so. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-08 22:02:39 +02:00