dandri/meshcore-analyzer
1
0
Fork 0
mirror of https://github.com/Kpa-clawbot/meshcore-analyzer.git synced 2026-10-13 17:37:24 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,011 Commits 84 Branches 39 Tags
9897d52cf6c51fce4e200c13f2ea3579b43fe01f
Commit Graph
21 Commits
Author SHA1 Message Date
efitenandClaude Opus 5.5 9897d52cf6 feat(users): companion linking store (schema v6, device sessions, link challenges) (#2174) 
Part 1 of 3 for #2173 (companion linking). This PR adds only the
`users.db` side: the schema and store methods the server (part 2) and
ingestor (part 3) build on. No route, no UI and no behaviour change for
a running server.

## Changes in this PR

- **Design spec** `docs/specs/2026-10-08-companion-linking-design.md`,
including the optional linked-only ingest.
- **Schema v6** (`internal/users/schema.go`):
- `sessions` gains `kind` (`'web'` or `'device'`), `label` and `scopes`
(`''` = full, for web sessions).
- `companion_links (pubkey PRIMARY KEY, user_id, name, linked_at)`,
cascading on user delete.
- `link_challenges`: single-use challenges bound to a user and a pubkey,
with an expiry.
- **Device sessions** (`sessions.go`): create a device session with a
label and scopes, list sessions with kind and label.
- **Challenges** (`challenges.go`): issue and consume a challenge
exactly once; consumed, expired, or bound to another user or pubkey all
fail.
- **Companion links** (`companions.go`): link, list, unlink. Linking a
pubkey that belongs to another user moves it and reports the previous
owner (transfer detection), so the server can audit and mail.
- **Janitor**: the existing janitor also prunes expired challenges
(`cmd/server/auth_service.go`, 3 lines).

## Upgrade note

The store migrates `users.db` to v6 on start. A binary that only knows
v5 refuses to open a v6 file (`users: database schema version 6 is newer
than this binary supports (5)`), so rolling back after this lands needs
the pre-upgrade `users.db` backup.

## Verification

- `internal/users`: new tests for the v6 migration from v5, device
sessions, challenges (single use, expiry, binding) and companion links
(transfer, cascade on user delete). `go vet` and `go test ./...` green.
- `cmd/server` full suite green against the v6 store.

## Not in this PR

- Routes, bearer auth and the link handshake: part 2.
- Ingestor filter, account page and "My coverage": part 3.

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-10 18:08:51 +02:00
efitenandClaude Opus 5.5 2456762179 feat: account data export and users.db backup (#2141) 
A small follow-up to parts A to E of #2128: users can download all data
the instance holds about them, and the server backs up `users.db`.

Stacked on #2140. Review the commits after that branch.

## The situation

- An instance with accounts holds personal data, but a user has no way
to get a copy (GDPR articles 15 and 20). Self-delete already exists.
- `users.db` is the only file the server writes, and nothing backs it
up: `GET /api/backup` snapshots the analyzer database only.

## What this PR adds

**Data export.** `GET /api/account/export` and a "Download my data"
button on the account page give one JSON file with:
- the profile, including activation and a pending email change;
- sessions (times and user agent);
- synced settings, own proposals, notification prefs and watches;
- audit rows where the user is actor or target, with other accounts as
an id only;
- the mail log.

Password hash, session and link tokens and the unsubscribe token are
left out: they are credentials. Each export is audited as `user.export`.

**users.db backup**
- Daily `VACUUM INTO` snapshots through `internal/users`, on by default
when user management is on. They go to `backups/` next to `users.db`,
the 7 newest are kept, the directory is 0700 and the files 0600.
Rotation only touches `users-<timestamp>.db` files and never the
snapshot it just wrote. Config: `userManagement.backup {enabled, dir,
keep}`.
- `GET /api/admin/users-backup`: an admin downloads a fresh snapshot to
keep off the server; audited as `user.backup`.
- `docs/user-guide/accounts.md` gains a restore procedure. It covers
what a restore brings back (accounts deleted after the snapshot, old
passwords, revoked sessions) and how to clean that up afterwards.

Spec:
[`docs/specs/2026-10-08-account-export-and-users-backup-design.md`](https://github.com/efiten/CoreScope/blob/feat/account-export-backup/docs/specs/2026-10-08-account-export-and-users-backup-design.md).

## Verification

- `internal/users` with `-race` and the `cmd/server` suite pass locally,
30 new Go tests. The file-mode tests skip on Windows.
- `sh test-all.sh` exits 0; the XSS gate in diff mode passes.
- User-management E2E: 27 of 27 steps locally.
- On our production instance since 8 October 2026; the first snapshot
there was 143 kB.

## Not in this PR

- Importing an export into another instance.
- Off-site upload of the snapshots. The admin download covers that by
hand.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 22:16:49 +02:00
efitenandClaude Opus 5.5 de364a8bb6 feat: mail notifications for watched nodes (user management part E) (#2140) 
Part E of #2128: a logged-in user watches nodes and gets one mail when a
watched node goes offline, comes back, or reports a low battery. Admins
can add instance events: a new foreign node, and an observer going
offline or back. This covers the per-user mail part of #775.

Stacked on #2139 (part D, which builds on #2138). Review the commits
after that branch.

## The situation

A sysop learns that a repeater went silent, or that its battery is
running down, only by opening CoreScope. #775 asks for notifications.
Accounts (part A) now give a verified address to mail and a mailer with
delivery status.

## What this PR adds

**Events.** They use the instance thresholds, so a mail does not
disagree with the node page:
- `node.offline`: silent for the role's `healthThresholds` window. Last
heard comes from the packet store, and repeaters and rooms count relayed
traffic (#1598).
- `node.battery`: advert telemetry below `batteryThresholds.lowMv`,
recovered at `lowMv + 100`.
- Admins only: `foreign.new` (once per node) and `observer.offline`.

**Store.** Schema v5 with `notification_prefs`, `notification_watches`
and `notification_state`.

**Server** (only with `userManagement.notifications.enabled`)
- A notifier next to the janitor, every `intervalMinutes` (5). The
evaluator is a pure function of watches, prefs, stored states and node
snapshots.
- No burst after a restart or a new watch: a subject's first evaluation
stores its state without mailing. The loop waits for the startup load.
While ingest is stale (newest packet older than 30 minutes) the offline
checks pause, and after recovery they wait one silent window.
- One mail per user per check. Limits: 20 per user and 100 per instance
per rolling 24 hours, 50 watches per user. A change over a limit is
recorded and never mailed later.
- Every mail has a one-click unsubscribe (`List-Unsubscribe` and
`List-Unsubscribe-Post`). The GET only redirects to a confirm page, so
link scanners cannot unsubscribe anyone. Node names are cleaned of
control and bidi characters before they go into a mail.
- Routes: `GET`/`PUT /api/account/notifications`, `PUT`/`DELETE
/api/account/notifications/watches/{pubkey}`, `POST
/api/account/notifications/watch-my-nodes` (copies the synced "my nodes"
list), `GET`/`POST /api/notifications/unsubscribe`. `mailer.Message`
gains `Headers`.

**Frontend.** A "Notify me" toggle on the node pages, a Notifications
section on the account page (`#/account?section=notifications`), an
unsubscribe page, and the notification mail count on the admin overview.

Spec:
[`docs/specs/2026-10-07-node-notifications-design.md`](https://github.com/efiten/CoreScope/blob/feat/notifications/docs/specs/2026-10-07-node-notifications-design.md).

## Performance

Each check reads watches, prefs and states from `users.db` (one query
each), the watched nodes from the analyzer DB in chunks of 500, and
last-heard times from the packet store.

| Measurement | Result |
|---|---|
| One check on our staging instance, 130,000 packets in memory, 1
watcher | 6.9 ms, of which 0.93 ms holding the store read lock |
| Evaluator benchmark, 100 users with 50 watches each, 2,000 nodes |
1.75 ms |

No work is added to ingest, broadcast or any request path.

## Verification

- `internal/users` and `internal/mailer` with `-race`; the `cmd/server`
suite plus `-race` on the notifier tests; 74 new Go tests. The same
Windows-only failure as noted in #2138 applies.
- `sh test-all.sh` exits 0; the XSS gate in diff mode passes.
- User-management E2E: 26 of 26 steps locally. With notifications off
the new steps fail, so they do test the flag.
- On our staging and production instance since 7 October 2026.

## Not in this PR

- Other channels (Discord, Telegram, webhooks). Detection is separate
from delivery, so one can be added.
- The topology, RF and anomaly alerts of #775.
- If the whole server starts after a feed outage that already ended, the
grace window is unknown and a watcher can get one wrong offline mail.
The user guide says so.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 22:02:39 +02:00
efitenandClaude Opus 5.5 abbe6846ff feat: propose and approve hashtag channels (user management part D) (#2139) 
Part D of #2128: logged-in users propose a hashtag channel, an admin
approves, rejects or later revokes it, and the ingestor decrypts
approved channels without a restart. It is the first consumer of a
generic proposal table, and answers the request in #2092.

Stacked on #2138 (part C). Review the commits after that branch; GitHub
shows C's commits here until it is merged.

## The situation

- An instance decrypts only the hashtag channels listed in
`hashChannels`. Users who want a city or club channel shown have to ask
the operator, who edits `config.json` and restarts the ingestor.
- #2092 asked for a suggest-and-approve flow; a downstream fork
(dborup/CoreScope#99) runs one with anonymous suggestions and a file
queue.

## What this PR adds

**Name rules (`internal/channel`).** `ValidateHashtagName`, mirrored in
the browser:
- at most 31 UTF-8 bytes including `#` (firmware
`ChannelDetails.name[32]`), case preserved, `Public` refused in any
case;
- control, bidi, separator and format characters refused except ZWJ,
plus invisible fillers (`Other_Default_Ignorable_Code_Point`, U+2800,
non-ASCII spaces).

Go and JS give the same verdict for every code point.

**Store.** Schema v4, one generic `proposals` table (kind, subject,
status pending/approved/rejected/revoked, proposer, reviewer, note). One
row per (kind, subject); limits are checked in the same transaction as
the change.

**Server** (only with `userManagement.channelProposals.enabled`)
- `POST /api/proposals`, `GET /api/account/proposals`, `GET
/api/admin/proposals`, `POST
/api/admin/proposals/{id}/approve|reject|revoke`.
- 400 for an invalid name. 409 for a duplicate, a rejected name, or a
name already in `hashChannels`/`channelKeys` ("this channel is already
decrypted on this instance"). 429 over a limit (`maxPending` 100,
`maxApproved` 128, `perUserPerDay` 5).
- `GET /api/channels` gains `approvedChannels`, so approved channels are
listed before they have traffic. With the feature off the response is
byte-identical.

**Ingestor.** Opens `users.db` with `mode=ro` every 60 seconds,
re-validates the names, and swaps its key map through an
`atomic.Pointer`. Configured channels always win, and a failed read
keeps the last good set. The ingestor never writes `users.db`; AGENTS.md
now says so.

**Frontend.** "Propose for everyone" in the add-channel dialog, an
"approved" marker in the channel list, "My proposals" on the account
page, and an admin "Proposals" tab with a warning on approve. The packet
detail pane now escapes the channel name: until now only the operator
chose those names.

Spec:
[`docs/specs/2026-10-07-channel-proposals-design.md`](https://github.com/efiten/CoreScope/blob/feat/channel-proposals/docs/specs/2026-10-07-channel-proposals-design.md).

## Performance

Every key is tried on each GRP_TXT that no key opens (`decodeGrpTxt`).
Benchmark of that worst case:

| Keys | Time per packet |
|---|---|
| 320 configured | 204 to 216 µs |
| 320 configured + 128 approved | 272 to 320 µs |

That is about +33%, linear in the key count and capped by `maxApproved`.
With the feature off the decoder gets the identical map and no ticker
runs.

## Verification

- `internal/channel`, `internal/users` (with `-race`), `cmd/server` and
`cmd/ingestor` pass locally, 53 new Go tests. The ingestor's
`TestWriteStatsAtomic_SymlinkAtDestIsReplaced` needs symlink rights on
Windows and fails on `master` too.
- `sh test-all.sh` exits 0; the XSS gate in diff mode passes.
- User-management E2E: 21 of 21 steps locally (propose, approve, the
channel appears, revoke removes it).
- On our staging and production instance since 7 October 2026.

## Not in this PR

- Private (PSK) channels: their keys stay in the browser (#725).
- Anonymous proposals.
- Names that need ZWNJ (some Persian and Urdu spellings) and subdivision
flags are refused, because ZWNJ and tag characters are format
characters. Allowing them later only widens the rule, so no approved
name gets stranded.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 21:51:42 +02:00
efitenandClaude Opus 5.5 18a634c115 feat: admin dashboard with overview and audit log (user management part C) (#2138) 
Part C of #2128: an admin area with an overview, the user table and a
global audit log. Off by default like the rest of user management: with
`userManagement` off nothing changes, and non-admins get no new UI.

This is the first of 4 stacked PRs (C, D, E, then a small export/backup
follow-up). Each later one contains this branch; review them in order.

## The situation

- An admin could manage users one by one in `#/admin/users`, but nothing
showed whether the instance needs attention: accounts stuck in
activation, bouncing mail, someone guessing a password, an MQTT source
that dropped.
- The audit log existed (`internal/users/audit.go`) but could only be
read per user, and logins were not recorded.

## What this PR adds

**Store (`internal/users`)**
- Schema v3: an index on `audit_log(at)`.
- `AuditList` with filters (action or group prefix like `user.login.*`,
user as actor or target, period) and keyset pagination; `PruneAudit`;
`Stats` for the user figures.

**Server**
- `GET /api/admin/stats` (typed struct): accounts by status, admins,
registrations and active users over 7 and 30 days, logins and failed
logins in 24 hours, mail by final status, and the attention items
computed server-side.
- `GET /api/admin/audit`: filtered, newest first, `next` cursor.
- `GET /api/admin/users` gains `bouncing=1`.
- Logins are audited as `user.login` and `user.login.failed` (reason
`wrong_password`, `pending` or `disabled`). An unknown address writes no
row. The writes are asynchronous, so the login response does not wait on
`users.db`. Login rows are pruned after 90 days.

**Frontend**
- `#/admin?tab=overview|users|audit`: `admin.js` (tab shell),
`admin-overview.js` ("Needs attention", Users card, System card from the
existing health, MQTT and observer endpoints), `admin-audit.js` (filters
in the URL, "Load more"). `admin-users.js` becomes the Users tab; the
old `#/admin/users` link rewrites to it.
- `/api/healthz` is read on open and on Refresh only, not on the
60-second timer, because it walks every packet under a read lock.

Spec:
[`docs/specs/2026-10-07-admin-dashboard-design.md`](https://github.com/efiten/CoreScope/blob/feat/admin-dashboard/docs/specs/2026-10-07-admin-dashboard-design.md).

## Verification

- `internal/users` and `cmd/server`: `go vet` and `go test` pass
locally, 26 new Go tests. One upstream test,
`TestSaveGeoFilterPreservesFileMode`, also fails on Windows on plain
`master` (file modes) and is unrelated.
- `sh test-all.sh` exits 0; the XSS gate in diff mode passes.
- User-management E2E with the `e2etest` build: 13 of 13 steps locally.
- Running on our staging and production instance since 7 October 2026.

## Not in this PR

- Restricting existing pages (perf, MQTT status) to logged-in users or
admins. The admin area adds no access mechanism of its own, so that
stays a later router rule plus endpoint check.
- Server-enforced customizer tab restrictions (#1508).
- The 24-hour, 5-attempt and 10-minute attention thresholds are
constants for now (AGENTS.md rule 8: customizer later).

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 21:38:30 +02:00
efitenandClaude Opus 5.5 f3ae8ac25a feat: sync a logged-in user's settings across devices (part B) (#2130) 
Part B of #2128: a logged-in user's settings follow them across devices.
Log in on a phone and your own nodes, favorites, customizer and filters
are there; a change on one device reaches the others within a minute or
when you return to the tab.

**This PR builds on #2129.** Until that one is merged, the diff here
includes it. The commits for this part start at `docs(specs): settings
sync for optional user management (sub-project B)`.

## The situation

Everything a visitor sets up lives in one browser's `localStorage`
(about 100 keys in `public/`). A second device or a cleared cache starts
from zero (#895).

## What this PR adds

**Storage.** `users.db` schema v2: one JSON document per user in
`user_settings`, with a revision number and a generation id. A write
succeeds only when the client's revision and generation match the stored
ones, so two devices cannot overwrite each other silently.

**Server.** `GET`, `PUT` and `DELETE /api/account/settings`, behind the
same session and CSRF checks as the account routes.
- The server owns the list of synced keys (61 keys,
[`settings_allowlist.go`](https://github.com/efiten/CoreScope/blob/feat/settings-sync/cmd/server/settings_allowlist.go))
and sends it to the client, so the two cannot drift.
- A hard denylist, checked first, refuses `meshcore-api-key`, every
`corescope_channel_*` key and `live-channel-colors` (#725). The colour
map is keyed by channel hash, and for a user-added channel that hash is
`user:<name>`, which would expose hashtag channel names.
- Documents are capped at 256 KiB, measured like `JSON.stringify`. PUT
is limited to 60 requests per hour per user. A stale revision gets 409
with the current document.

**Client**
([`settings-sync.js`](https://github.com/efiten/CoreScope/blob/feat/settings-sync/public/settings-sync.js)).
Inert unless the feature is on and someone is logged in.
- It wraps `localStorage.setItem` and `removeItem` for allowlisted keys
only and pushes 2 seconds after the last change.
- It pulls on login, page load, tab focus and every 60 seconds while the
tab is visible.
- **Merge:** three-way, against a per-device baseline that belongs to
one user and one document generation. Lists (own nodes, favorites, saved
filters) merge per item, so an item added anywhere is kept and an item
removed on one device does not come back from another. Single values:
the profile wins unless only this device changed it.
- Remote changes are written without a push, theme and colour-blind
preset are re-applied, and the current page re-renders (skipped on
account pages and while the geofilter editor is open).

**UI.**
- Logout asks: keep my settings on this device (default), remove them
from this device, or cancel. Channel keys are never removed: no copy
exists anywhere else.
- The account page gets a "Settings sync" section: last synced time,
"Sync now", what is and is not synced, and "Delete synced settings from
my account".

## Not synced

Layout and device state (panel and column widths, collapsed panels, map
positions, geofilter drafts), channel data (#725), the API key, and all
`sessionStorage`. The full list is in the
[spec](https://github.com/efiten/CoreScope/blob/feat/settings-sync/docs/specs/2026-10-06-user-settings-sync-design.md).

## Performance

- One GET per page load, tab focus and minute while visible; one
debounced PUT per burst of changes.
- The `setItem` wrapper costs one Set lookup per write for non-synced
keys. A synced write reads one small revision key, not the stored
document.
- The server reads or writes one row per request.

## Verification

- `internal/users` and `cmd/server`: `go vet` and `go test` pass locally
(22 new Go tests), including a test that every allowlisted key still
occurs in `public/`, and denylist tests.
- `tests/unit/test-settings-sync.js`: 79 passing (vm, real module). The
cases cover the merge table, two tabs sharing one storage, stale answers
after a push, delete while a push is in flight, and logout while the
final push fails.
- `sh test-all.sh` exits 0.
- `tests/e2e/test-user-management-e2e.js` (10 steps, 4 of them new)
passed locally with two browser contexts as two devices: a favorite and
the packet time window travel from device 1 to device 2, a removal does
not come back, and "remove from this device" clears the synced keys
while a channel key stays.
- Checked by hand on a staging instance with a desktop and a phone on
one account.

## Not in this PR

- On a shared browser where the previous user chose "keep", the next
user's first login merges those settings into their own account. The
user guide says to choose "remove" on shared computers.
- Saved filter expressions are synced as typed, including any channel
names written in them. The guide says so.
- Realtime push between devices; the minute pull is the sync interval.

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-07 11:20:44 +02:00
efitenandClaude Opus 5.5 d232072f85 feat: optional user accounts (part A: foundation) (#2129) 
Part A of #2128: optional, off-by-default user accounts. With the
feature off nothing changes; with it on, visitors can register and log
in, and admins manage users and use the operator actions without the API
key.

PR #2130 (settings sync) builds on this one. The two are meant to be
merged together.

## The situation

- Operator actions (geofilter save and prune, backup, perf reset) need
the shared `apiKey`. There is no per-person right.
- Nothing in CoreScope knows who a visitor is, so the requests in #2128
that need that (#1835, #2092, #1508, #730) have nothing to build on.

## What this PR adds

**Two new Go modules**
- `internal/users`: a separate `users.db` (SQLite through
`modernc.org/sqlite`) with users, sessions, single-use tokens, an audit
log and a mail log. Passwords use argon2id.
- `internal/mailer`: a `Mailer` interface with a Brevo client (send,
delivery events, webhook parsing) and an in-memory fake for tests.

**Server (`cmd/server`)**, active only with `userManagement.enabled`
- 24 routes, all documented in OpenAPI under the `users` tag
([`auth_routes.go`](https://github.com/efiten/CoreScope/blob/feat/user-management/cmd/server/auth_routes.go)):
  - auth: register, activate, login, logout, me, forgot, reset;
- account: profile, password, email change with confirmation, sessions,
self-delete;
- admin: list, detail, disable, enable, delete, role, resend activation,
manual activation, mail status refresh;
  - a Brevo webhook, registered only when `mail.webhookSecret` is set.
- `requireAdmin` replaces `requireAPIKey` at the 7 operator call sites:
the API key **or** an admin session. With the feature off it is the old
API-key gate (`TestRequireAdminWithoutUserManagementIsAPIKeyGate`).
- `/api/config/client` gets `userManagement: {enabled: true}` only when
the service started; with the feature off the response is
byte-identical.

**Frontend**
- `auth.js` (header account control, request helper that adds the CSRF
header), `account.js` (login, register, activate, forgot, reset, confirm
email, my account), `admin-users.js` (`#/admin/users`, deep-linked
filters), `account.css` (theme tokens only).
- On phones the top-bar control is hidden, so a conditional entry goes
into the bottom-nav "More" sheet and the nav drawer.
- The customizer geofilter tab and the Perf "Reset stats" button use the
admin session when there is one.

**Config.** A `userManagement` block (`config.example.json`,
[`docs/user-guide/accounts.md`](https://github.com/efiten/CoreScope/blob/feat/user-management/docs/user-guide/accounts.md)).
The Brevo key can come from `CORESCOPE_BREVO_API_KEY`. The server
refuses to start when the block is enabled but incomplete.

## Security choices

- Session cookie `cs_session`: HttpOnly, SameSite=Lax, Secure when
`publicBaseUrl` is https. Every cookie-authenticated state change needs
the `X-CS-CSRF` header and a matching Origin.
- Activation needs the token **and** the account password. Without the
password, an attacker who keeps re-registering a known address could get
the owner to activate an account that carries the attacker's password.
- Register, forgot and email change answer identically for known and
unknown addresses. A password reset ends all sessions, a password change
ends all other sessions, and both end outstanding email-change links.
- Rate limits: login 10 per 15 minutes, register and forgot 5 per hour,
per IP and per address. The bucket count is capped. `trustedProxies`
makes the per-IP limits see real client IPs behind a proxy.
- Server logs carry `#<user id>`, never addresses, tokens or passwords;
mail-provider error texts are redacted before logging.

## Performance

No change to an existing hot path with the feature off. With it on:
- One `users.db` lookup per authenticated request (session by token
hash).
- The admin user table rebuilds its `tbody` on each filter change.
`users.List` caps the result at 1000 rows (`internal/users/users.go`),
which bounds the rebuild.
- `map[string]interface{}` in `openapi.go`: 79 before, 78 after.

## Verification

- `internal/users`, `internal/mailer` and `cmd/server`: `go vet` and `go
test -race` pass locally. 121 new Go tests.
- `cmd/server` with `-tags e2etest`: vet and the e2e hook tests pass.
- `sh test-all.sh` exits 0. `tests/unit/test-user-management-ui.js`: 67
passing (vm, real modules).
- `tests/e2e/test-user-management-e2e.js` (6 steps) passed locally
against an `e2etest` build with the fake mailer and against a
feature-off build. CI builds the `e2etest` binary and runs the suite on
a second server (`deploy.yml`).
- On a staging instance with a real Brevo key: register, activation mail
delivered, activate, admin table, "Refresh status" showing sent,
deferred, delivered, opened and clicked.

## Not in this PR

- Settings sync (#2130), the admin dashboard, approval flows and
notifications (parts B to E of #2128).
- A `requireReadAuth` mode (#1835). Sessions from this PR are what such
a mode would accept.
- Binary size and build time with `modernc.org/sqlite` linked next to
`mattn/go-sqlite3` were not measured. Their driver names do not collide.
#1992 discusses the driver choice.
- No Brevo webhook was configured on staging; delivery status there came
from "Refresh status".

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-07 09:25:29 +02:00
Kpa-clawbotmc-botopenclaw-botclawbotclawbot
 		b74a64ccfa fix(ui): canonical payload label map across packets/live/packet-filter (#1799) (#1804) 
## Summary

Replaces the three drifted per-surface payload-type label vocabularies
with a single canonical map keyed by firmware enum name.

Per the locked triage comment on #1799
([comment-4823975431](https://github.com/Kpa-clawbot/CoreScope/issues/1799#issuecomment-4823975431)):

> Create `public/payload-labels.js` exporting `{GRP_DATA: {short:'Group
Data', long:'Group data packet', enumId:6}, ...}`. Migrate `packets.js
typeMap`, `packet-filter.js FW_PAYLOAD_TYPES`, `live.js TYPE_COLORS
legend` to consume it. E2E that scrapes each surface and asserts label
equality.

## Changes

- **`public/payload-labels.js`** (new) — canonical map exposed as
`window.PayloadLabels` and `window.PayloadLabelsApi`. Keys are firmware
enum names; values carry `{short, long, enumId}` plus derived
`SHORT_BY_ID` / `FW_PAYLOAD_TYPES` / `TYPE_ALIASES` for legacy callers.
- **`public/packets.js`** — `TYPE_NAMES` + `typeMap` now read from
`PayloadLabelsApi.SHORT_BY_ID`. Literal kept only as a defensive
fallback for the case where the script tag fails to load.
- **`public/packet-filter.js`** — `FW_PAYLOAD_TYPES` + `TYPE_ALIASES`
now sourced from `PayloadLabelsApi`. Literal fallback retained so `node
test-packet-filter.js` still works headlessly.
- **`public/live.js`** — legend `<li>` rows are now generated from
`window.PayloadLabels` in stable order, killing the third-vocabulary
`Message — Group text` / `Direct — Direct message` drift the #1797
review surfaced.
- **`public/index.html`** — `<script src="payload-labels.js">` loaded
before `roles.js` / `packet-filter.js` / `packets.js`.
- **`test-issue-1799-label-vocab-e2e.js`** (new) — Playwright E2E.
Scrapes `#liveLegend` rows and the `/packets` type-filter checklist,
asserts each label matches `window.PayloadLabels[ENUM].short` for
`TXT_MSG`, `GRP_TXT`, `GRP_DATA`. Also verifies `window.PacketFilter`
still recognises the enum names.
- **`.github/workflows/deploy.yml`** — wired the new E2E into the
existing Playwright block.

## TDD trail

- Red commit `eb392d4` — adds the failing E2E only (asserts
`window.PayloadLabels` exists and labels match; both fail).
- Green commit `44e902a` — introduces the canonical map and migrates the
three surfaces.

## Verification

- `node test-packet-filter.js` — 92/92 pass with the new fallback
wiring.
- Preflight: `bash ~/.openclaw/skills/pr-preflight/scripts/run-all.sh
origin/master` — clean.

Browser verified: E2E `test-issue-1799-label-vocab-e2e.js` exercises
`/live` legend + `/packets` type filter against a Playwright headless
Chromium; CI's Playwright block runs it on every push.

E2E assertion added: `test-issue-1799-label-vocab-e2e.js:139` —
`assert(fromLegend === canon, ...)` and `assert(fromPackets === canon,
...)` per enum.

Fixes #1799

---------

Co-authored-by: mc-bot <bot@corescope>
Co-authored-by: openclaw-bot <bot@openclaw.local>
Co-authored-by: clawbot <clawbot@kpa.com>
Co-authored-by: clawbot <bot@clawbot.local>
 2026-06-30 05:48:47 -07:00
efiten 9c5faab1e4 Revert "feat(nodes): per-node Reach page (#1625)" (#1626) 
Reverts #1625.

#1625 was merged before the round-1 reviews (Independent / Kent Beck /
Tufte) were addressed. Reverting to land it cleanly: a fresh PR will
re-add the feature with the perf pass, the backend correctness/safety +
test-coverage fixes, and the UI/a11y (Tufte) batch folded in, so it goes
through review in a single hardened state rather than as a string of
post-merge follow-ups.

No functional loss — the feature returns in the replacement PR.
 2026-06-08 12:35:12 +00:00
efitenandClaude Opus 4.8 47f85f6c4c feat(nodes): per-node Reach page + GET /api/nodes/{pubkey}/reach (directional link quality) (#1625) 
## What

Adds a per-node **Reach** view that answers "how well does this specific
node hear, and get heard by, its neighbours?" — both as a standalone
page (`#/nodes/{pubkey}/reach`) and as a section on the node detail
page.

New endpoint: **`GET /api/nodes/{pubkey}/reach`**.

## What it measures

For the target node it derives, from raw `path_json` adjacency (a path
travels origin→observer, so in `[A,B]` B received A directly):

- **Directional link counts** per neighbour: `we_hear` (how often we
received them) vs `they_hear` (how often they received us).
- **Bidirectional / bottleneck**: a link is two-way stable when both
directions > 0; the weaker direction is the bottleneck and rates real
two-way reliability.
- **Importance**: neighbour degree + rank, relay-observation volume,
bidirectional-link count, direct-observer count.
- **Direct observers**: who received the node at 0 hops, with SNR.

Reliability rule: a neighbour is only attributed when its pubkey
**prefix is unique** at the path's byte length (collisions are skipped,
never misattributed).

## UI

- Standalone Reach page + node-detail section.
- Reusable bidirectional link map (OSM) with links coloured by
bottleneck.
- Incoming/outgoing toggles to isolate each direction.

## Naming note (deliberate, no collision)

This is distinct from the existing **per-observer reachability** in
topology analytics (`ReachNode` / `ObserverReach` / `perObserverReach`).
This PR adds its own `NodeReach*` response structs in a new
`node_reach.go` and a new `/api/nodes/{pubkey}/reach` route — there are
no symbol or route collisions (verified: `go build ./...` clean). Happy
to rename to disambiguate further (e.g. "Link Quality") if you'd prefer
to reserve "Reach" for the per-observer feature.

## Testing

- `cmd/server`: endpoint shape/404/limit-clamp + unit tests for token
derivation and directional attribution, plus a scan benchmark — all
pass.
- Frontend: helper tests + Reach-page E2E (`test-node-reach-e2e.js`),
standalone route + incoming/outgoing toggles.
- `go build ./...` and `eslint public/*.js` (no-undef) clean.

## Docs

Design spec, implementation plan, and the `GET
/api/nodes/{pubkey}/reach` API contract are included under `docs/`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-08 13:11:06 +02:00
you 6b9946d9c6 docs: timestamp-based packet filter spec (#289) 2026-04-06 01:22:15 +00:00
you 24335164d6 docs: table sorting consistency spec (#620) 2026-04-05 21:56:09 +00:00
Kpa-clawbotandyou f7000992ca fix(rf-health): auto-scale airtime Y-axis + hover tooltips (#600) (#623) 
## Summary

Addresses user feedback on #600 — two improvements to RF Health detail
panel charts:

### 1. Auto-scale airtime Y-axis
Previously fixed 0-100% which made low-activity nodes unreadable (e.g.
0.1% TX barely visible). Now auto-scales to the actual data range with
20% headroom (minimum 1%), matching how the noise floor chart already
works.

### 2. Hover tooltips on all chart data points
Invisible SVG `<circle>` elements with native `<title>` tooltips on
every data point across all 4 charts:
- **Noise floor**: `NF: -112.3 dBm` + UTC timestamp
- **Airtime**: `TX: 2.1%` or `RX: 8.3%` + UTC timestamp  
- **Error rate**: `Err: 0.05%` + UTC timestamp
- **Battery**: `Batt: 3.85V` + UTC timestamp

Uses native browser SVG tooltips — zero dependencies, accessible, no JS
event handlers.

### Design rationale (Tufte)
- Auto-scaling increases data-ink ratio by eliminating wasted vertical
space
- Tooltips provide detail-on-demand without cluttering the chart with
labels on every point

### Spec update
Added M2 feedback improvements section to
`docs/specs/rf-health-dashboard.md`.

---------

Co-authored-by: you <you@example.com>
 2026-04-05 13:08:05 -07:00
you 8a94c43334 docs: startup performance spec — serve HTTP within 2 minutes on any DB size 2026-04-05 07:09:55 +00:00
you 788005bff7 docs: clarify Docker tag strategy — pin to vX.Y.Z for production, edge for testing 2026-04-05 07:09:44 +00:00
you af03f9aa57 docs: deployment simplification spec — pre-built Docker images + one-line deploy 2026-04-05 07:06:35 +00:00
you cbc3e3ce13 docs: movable UI panels spec — draggable panel positioning (#279) 2026-04-05 06:54:45 +00:00
you 1796493ec0 docs: channel color highlighting spec (#271) 
Custom color assignment for hash channels in Live tab.
Reviewed by Tufte, Torvalds, and Doshi personas.
 2026-04-05 06:45:53 +00:00
you aaf00d0616 docs: add M5 Prometheus/Grafana metrics export to RF Health spec 2026-04-05 05:02:36 +00:00
you 41c046c974 docs: RF Health Dashboard spec — observer radio metrics 
Per-observer time-series charts for noise floor, TX/RX airtime, CRC errors,
and battery. Small multiples grid design. MVP-first milestones.

Reviewed by Carmack (perf), Munger (failure modes), radio expert (hardware),
Tufte (visualization), and Doshi (product strategy).
 2026-04-05 04:42:32 +00:00
Kpa-clawbotandyou 64745f89b1 feat: customizer v2 — event-driven state management (#502) (#503) 
## Summary

Implements the customizer v2 per the [approved
spec](docs/specs/customizer-rework.md), replacing the v1 customizer's
scattered state management with a clean event-driven architecture.
Resolves #502.

## What Changed

### New: `public/customize-v2.js`
Complete rewrite of the customizer as a self-contained IIFE with:

- **Single localStorage key** (`cs-theme-overrides`) replacing 7
scattered keys
- **Three state layers:** server defaults (immutable) → user overrides
(delta) → effective config (computed)
- **Full data flow pipeline:** `write → read-back → merge → atomic
SITE_CONFIG assign → apply CSS → dispatch theme-changed`
- **Color picker optimistic CSS** (Decision #12): `input` events update
CSS directly for responsiveness; `change` events trigger the full
pipeline
- **Override indicator dots** (●) on each field — click to reset
individual values
- **Section-level override count badges** on tabs
- **Browser-local banner** in panel header: "These settings are saved in
your browser only"
- **Auto-save status indicator** in footer: "All changes saved" /
"Saving..." / "⚠️ Storage full"
- **Export/Import** with full shape validation (`validateShape()`)
- **Presets** flow through the standard pipeline
(`writeOverrides(presetData) → pipeline`)
- **One-time migration** from 7 legacy localStorage keys (exact field
mapping per spec)
- **Validation** on all writes: color format, opacity range, timestamp
enum values
- **QuotaExceededError handling** with visible user warning

### Modified: `public/app.js`
Replaced ~80 lines of inline theme application code with a 15-line
`_customizerV2.init(cfg)` call. The customizer v2 handles all merging,
CSS application, and global state updates.

### Modified: `public/index.html`
Swapped `customize.js` → `customize-v2.js` script tag.

### Added: `docs/specs/customizer-rework.md`
The full approved spec, included in the repo for reference.

## Migration

On first page load:
1. Checks if `cs-theme-overrides` already exists → skip if yes
2. Reads all 7 legacy keys (`meshcore-user-theme`,
`meshcore-timestamp-*`, `meshcore-heatmap-opacity`,
`meshcore-live-heatmap-opacity`)
3. Maps them to the new delta format per the spec's field-by-field
mapping
4. Writes to `cs-theme-overrides`, removes all legacy keys
5. Continues with normal init

Users with existing customizations will see them preserved
automatically.

## Dark/Light Mode

- `theme` section stores light mode overrides, `themeDark` stores dark
mode overrides
- `meshcore-theme` localStorage key remains **separate** (view
preference, not customization)
- Switching modes re-runs the full pipeline with the correct section

## Testing

- All existing tests pass (`test-packet-filter.js`, `test-aging.js`,
`test-frontend-helpers.js`)
- Old `customize.js` is NOT modified — left in place for reference but
no longer loaded

## Not in Scope (per spec)

- Undo/redo stack
- Cross-tab synchronization
- Server-side admin import endpoint
- Map config / geo-filter overrides

---------

Co-authored-by: you <you@example.com>
 2026-04-02 21:14:38 -07:00