meshcore-analyzer

dandri/meshcore-analyzer

0 Fork 0

mirror of https://github.com/Kpa-clawbot/meshcore-analyzer.git synced 2026-11-11 12:37:33 +00:00

Commit Graph

Author SHA1 Message Date

Author	SHA1	Message	Date
efitenandClaude Opus 5.5	e149568c6c	feat(observer-logins): personal MQTT logins for observers (#2306 ) Observers publish to the broker with one shared MQTT login. Anyone who has that login can publish under any observer's topic, and the ingestor takes the observer id from the topic (`cmd/ingestor/main.go:1009`), so the data in the analyzer DB is only as trustworthy as the shared password. This PR gives each user a personal MQTT login, generated and mailed by CoreScope, and lets the broker ask CoreScope whether a login may connect and publish. Closes #2319. The feature is optional and off by default (`userManagement.observerLogins.enabled`). It needs `userManagement.enabled`. With it off, nothing changes. ## Changes in this PR Broker hooks. `POST /api/mqtt-hook/emqx/authn` and `/authz`, for the EMQX HTTP authenticator and authorizer. - They check a shared secret header, `X-CoreScope-Hook-Secret`, in constant time. - A username CoreScope does not know gets `ignore`, so EMQX falls through to its built-in database and existing users keep working. - A personal login may only publish `meshcore/<iata>/<pubkey>/<type>`, with a 64-hex pubkey that is unbound or bound to the same user. - The first publish for an unbound pubkey binds that observer to the user (first come wins), until the user owns 20 observers (admin assignments count too). - An admin can rebind any observer. Credentials. - Username `obs_` + 12 random `[a-z0-9]`, password 32 random `[A-Za-z0-9]`, both from `crypto/rand`. - The password is stored twice: SHA-256 for the auth check, and sealed with AES-256-GCM (key from config or `CORESCOPE_OBSERVER_KEY`) so the user can ask for the same credentials again. - The UI and the API never show the password. It leaves the server only in the mail. Mail. One mail with four formats: - the meshcoretomqtt TOML; - the observer firmware CLI commands (Gessaman or Dutch-MeshCore, same `mqttN` CLI), on slot 4: slots 1-2 (Gessaman, Let's Mesh US/EU) and 1-3 (Dutch-MeshCore, `MQTTDefaults.h:19-34`) hold default broker presets on a fresh install, slot 4 is empty on both. `set mqtt4.server` gets the bare host: the firmware picks the scheme from the port (`MQTTBridge.cpp:2094-2145`), and builds before merge 32dd8ad (2026-10-19) do not accept a scheme there (read from the source, not tested on a device). A scheme is written only when the port would pick the wrong one, for example TLS on 8884; - the Home Assistant meshcore-ha broker form fields; - the plain values. Each block starts with a numbered title and one line on where it goes. The account page can resend the mail, at most 3 times per 24 h. Account and admin UI. - The account page gets an "Observer MQTT login" section (`#/account?section=observers`) with the username, the bound observers and the resend button. - The admin page gets an "Observer logins" tab (`#/admin?tab=observer-logins`) that lists logins with revoke and reissue, and every observer with its owner, "not migrated" or "orphaned", with a reassign control. Notifications. Creating a login adds `observer.offline.own` ("My observers go offline") to the owner's notification events; mails go out once the user has notifications switched on, and the user can untick it. Storage. users.db schema v7 adds `mqtt_logins` and `observer_bindings`. The server still writes only users.db through `internal/users`, and the analyzer DB stays read-only. Docs. `docs/observer-mqtt-logins.md` covers the EMQX setup (authn and authz chains, the hook body templates, the closing `^obs_` deny rule and why it is needed), how to enable it on a running EMQX through the REST API step by step, with a check and a rollback per step, and how to migrate observers off a shared login. `config.example.json` has the block with placeholder values. ## Enabling it on an instance Full steps in `docs/observer-mqtt-logins.md`. In short: 1. CoreScope: add `userManagement.observerLogins` to `config.json` with `enabled: true`, a `hookSecret` (`openssl rand -hex 32`), an `encryptionKey` (`openssl rand -base64 32`) and the `broker` host, port, transport and TLS that observers should use. Restart. Keep the key safe: without it, mails cannot be resent and logins must be reissued. 2. EMQX, authorization: add a file source with `{deny, {username, {re, "^obs_"}}, all, ["#"]}.` and move it to the end, then add the HTTP authorizer pointing at `/api/mqtt-hook/emqx/authz` (it lands at the front). 3. EMQX, authentication: add the HTTP authenticator pointing at `/api/mqtt-hook/emqx/authn` and move it to the front, before the built-in database. 4. Check: the shared login still connects, an unknown `obs_` login is refused, and `metrics.deny` on the authorizer stays 0 for existing clients. 5. Owners: they request a login on their account page and paste the block for their uploader. Each observer binds to them on its first publish. An uploader that was refused before step 3 may need a restart. 6. Later: remove the shared login from the broker once enough observers moved. ## Who can claim an observer Any active account can request a login, and its first publish under an unbound pubkey binds that observer, also when CoreScope already knew it from the shared login (first come wins, a deliberate choice: owners migrate without waiting for an admin). Observer pubkeys are public, and registration cannot be closed, so with this feature on anyone with a mail address can claim up to 20 unbound observers per account. Before, publishing needed the shared password. What limits it: each claim of a known observer mails the admins, an admin can unbind or reassign it and revoke the login, and a bound observer cannot be claimed by another login. The doc spells this out and suggests assigning known observers to their owners before announcing the feature. A stricter mode (admin approval of logins, or no auto-bind for known observers) is not in this PR. ## Performance `authorize()` runs for every publish that misses the EMQX authz cache. It takes two short read locks, does map lookups and one topic split. `BenchmarkAuthorizeBound` (2000 observers, 500 logins): 533.5 ns/op, 96 B/op, 2 allocs/op. The admin observers table renders once per load and updates single rows after a change. ## Verification - All 16 Go modules: `bash scripts/allmod.sh test ./...` green. - Skipped `TestSaveGeoFilterPreservesFileMode` and `TestWriteStatsAtomic_SymlinkAtDestIsReplaced`: both fail on Windows on a clean master too. - `sh test-all.sh` green. - `tests/e2e/test-user-management-e2e.js` 36/36 locally. It covers: request, mail, resend, revoke, reissue, reassign, no password in the page or API body, and axe. The CI job for this suite gets the `observerLogins` block with fake values in `deploy.yml`. - Local EMQX 5.8.6 in Docker against a local CoreScope: - a personal login connects and publishes, which binds the observer; a second personal login is denied on that observer's topic, and subscribe is denied; - with CoreScope stopped, the built-in users still connect and publish, and new personal-login connects are refused; - the `^obs_` deny rule blocks an already-connected personal login while CoreScope is down; without the rule, the same publish was delivered. - Production: running on an EMQX 5.8.9 broker with about 50 clients since 2026-10-25, set up through the REST API as in the doc. After the hooks went in, the authorizer answered `ignore` for every existing login (0 denies, 0 dropped messages), and 4 observers were bound to their owners through personal logins on the same evening, with the shared login still in use by the rest. ## Known behaviour EMQX's HTTP client can mark the hook resource down for one health-check interval (15 s measured) when CoreScope closes an idle keep-alive connection just as a health check runs. During that window new personal-login connects are refused and the uploader reconnects. I saw it 3 times in about 25 minutes of runs with a forced 1-5 s idle timeout, and could not trigger it on demand. EMQX 5.8.6 has no setting for it, and 5.10.5 ships the same client code. The doc describes it and the alarm to watch for. ## Not in this PR - mosquitto (mosquitto-go-auth). EMQX only. - A websocket path field: the mail does not carry one, the doc tells operators to match the listener path. - Disabling the shared login. That is an operator step after migration, described in the doc. - Tested on EMQX 5.8.6 (Docker) and 5.8.9 (production) only. Note on `cb9f4ff3`: its message says the scheme problem was reported by an operator. That report turned out to come from before the broker used the hook, so it does not show this cause; the change rests on the firmware source only. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-26 09:50:20 +01:00
efitenandClaude Opus 5.5	abbe6846ff	feat: propose and approve hashtag channels (user management part D) (#2139 ) Part D of #2128: logged-in users propose a hashtag channel, an admin approves, rejects or later revokes it, and the ingestor decrypts approved channels without a restart. It is the first consumer of a generic proposal table, and answers the request in #2092. Stacked on #2138 (part C). Review the commits after that branch; GitHub shows C's commits here until it is merged. ## The situation - An instance decrypts only the hashtag channels listed in `hashChannels`. Users who want a city or club channel shown have to ask the operator, who edits `config.json` and restarts the ingestor. - #2092 asked for a suggest-and-approve flow; a downstream fork (dborup/CoreScope#99) runs one with anonymous suggestions and a file queue. ## What this PR adds Name rules (`internal/channel`). `ValidateHashtagName`, mirrored in the browser: - at most 31 UTF-8 bytes including `#` (firmware `ChannelDetails.name[32]`), case preserved, `Public` refused in any case; - control, bidi, separator and format characters refused except ZWJ, plus invisible fillers (`Other_Default_Ignorable_Code_Point`, U+2800, non-ASCII spaces). Go and JS give the same verdict for every code point. Store. Schema v4, one generic `proposals` table (kind, subject, status pending/approved/rejected/revoked, proposer, reviewer, note). One row per (kind, subject); limits are checked in the same transaction as the change. Server (only with `userManagement.channelProposals.enabled`) - `POST /api/proposals`, `GET /api/account/proposals`, `GET /api/admin/proposals`, `POST /api/admin/proposals/{id}/approve\|reject\|revoke`. - 400 for an invalid name. 409 for a duplicate, a rejected name, or a name already in `hashChannels`/`channelKeys` ("this channel is already decrypted on this instance"). 429 over a limit (`maxPending` 100, `maxApproved` 128, `perUserPerDay` 5). - `GET /api/channels` gains `approvedChannels`, so approved channels are listed before they have traffic. With the feature off the response is byte-identical. Ingestor. Opens `users.db` with `mode=ro` every 60 seconds, re-validates the names, and swaps its key map through an `atomic.Pointer`. Configured channels always win, and a failed read keeps the last good set. The ingestor never writes `users.db`; AGENTS.md now says so. Frontend. "Propose for everyone" in the add-channel dialog, an "approved" marker in the channel list, "My proposals" on the account page, and an admin "Proposals" tab with a warning on approve. The packet detail pane now escapes the channel name: until now only the operator chose those names. Spec: [`docs/specs/2026-10-07-channel-proposals-design.md`](https://github.com/efiten/CoreScope/blob/feat/channel-proposals/docs/specs/2026-10-07-channel-proposals-design.md). ## Performance Every key is tried on each GRP_TXT that no key opens (`decodeGrpTxt`). Benchmark of that worst case: \| Keys \| Time per packet \| \|---\|---\| \| 320 configured \| 204 to 216 µs \| \| 320 configured + 128 approved \| 272 to 320 µs \| That is about +33%, linear in the key count and capped by `maxApproved`. With the feature off the decoder gets the identical map and no ticker runs. ## Verification - `internal/channel`, `internal/users` (with `-race`), `cmd/server` and `cmd/ingestor` pass locally, 53 new Go tests. The ingestor's `TestWriteStatsAtomic_SymlinkAtDestIsReplaced` needs symlink rights on Windows and fails on `master` too. - `sh test-all.sh` exits 0; the XSS gate in diff mode passes. - User-management E2E: 21 of 21 steps locally (propose, approve, the channel appears, revoke removes it). - On our staging and production instance since 7 October 2026. ## Not in this PR - Private (PSK) channels: their keys stay in the browser (#725). - Anonymous proposals. - Names that need ZWNJ (some Persian and Urdu spellings) and subdivision flags are refused, because ZWNJ and tag characters are format characters. Allowing them later only widens the rule, so no approved name gets stranded. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-08 21:51:42 +02:00
efitenandClaude Opus 5.5	18a634c115	feat: admin dashboard with overview and audit log (user management part C) (#2138 ) Part C of #2128: an admin area with an overview, the user table and a global audit log. Off by default like the rest of user management: with `userManagement` off nothing changes, and non-admins get no new UI. This is the first of 4 stacked PRs (C, D, E, then a small export/backup follow-up). Each later one contains this branch; review them in order. ## The situation - An admin could manage users one by one in `#/admin/users`, but nothing showed whether the instance needs attention: accounts stuck in activation, bouncing mail, someone guessing a password, an MQTT source that dropped. - The audit log existed (`internal/users/audit.go`) but could only be read per user, and logins were not recorded. ## What this PR adds Store (`internal/users`) - Schema v3: an index on `audit_log(at)`. - `AuditList` with filters (action or group prefix like `user.login.`, user as actor or target, period) and keyset pagination; `PruneAudit`; `Stats` for the user figures. Server* - `GET /api/admin/stats` (typed struct): accounts by status, admins, registrations and active users over 7 and 30 days, logins and failed logins in 24 hours, mail by final status, and the attention items computed server-side. - `GET /api/admin/audit`: filtered, newest first, `next` cursor. - `GET /api/admin/users` gains `bouncing=1`. - Logins are audited as `user.login` and `user.login.failed` (reason `wrong_password`, `pending` or `disabled`). An unknown address writes no row. The writes are asynchronous, so the login response does not wait on `users.db`. Login rows are pruned after 90 days. Frontend - `#/admin?tab=overview\|users\|audit`: `admin.js` (tab shell), `admin-overview.js` ("Needs attention", Users card, System card from the existing health, MQTT and observer endpoints), `admin-audit.js` (filters in the URL, "Load more"). `admin-users.js` becomes the Users tab; the old `#/admin/users` link rewrites to it. - `/api/healthz` is read on open and on Refresh only, not on the 60-second timer, because it walks every packet under a read lock. Spec: [`docs/specs/2026-10-07-admin-dashboard-design.md`](https://github.com/efiten/CoreScope/blob/feat/admin-dashboard/docs/specs/2026-10-07-admin-dashboard-design.md). ## Verification - `internal/users` and `cmd/server`: `go vet` and `go test` pass locally, 26 new Go tests. One upstream test, `TestSaveGeoFilterPreservesFileMode`, also fails on Windows on plain `master` (file modes) and is unrelated. - `sh test-all.sh` exits 0; the XSS gate in diff mode passes. - User-management E2E with the `e2etest` build: 13 of 13 steps locally. - Running on our staging and production instance since 7 October 2026. ## Not in this PR - Restricting existing pages (perf, MQTT status) to logged-in users or admins. The admin area adds no access mechanism of its own, so that stays a later router rule plus endpoint check. - Server-enforced customizer tab restrictions (#1508). - The 24-hour, 5-attempt and 10-minute attention thresholds are constants for now (AGENTS.md rule 8: customizer later). --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-08 21:38:30 +02:00

efitenandClaude Opus 5.5

e149568c6c

feat(observer-logins): personal MQTT logins for observers (#2306 )

Observers publish to the broker with one shared MQTT login. Anyone who
has that login can publish under any observer's topic, and the ingestor
takes the observer id from the topic (`cmd/ingestor/main.go:1009`), so
the data in the analyzer DB is only as trustworthy as the shared
password. This PR gives each user a personal MQTT login, generated and
mailed by CoreScope, and lets the broker ask CoreScope whether a login
may connect and publish.

Closes #2319.

The feature is optional and off by default
(`userManagement.observerLogins.enabled`). It needs
`userManagement.enabled`. With it off, nothing changes.

## Changes in this PR

**Broker hooks.** `POST /api/mqtt-hook/emqx/authn` and `/authz`, for the
EMQX HTTP authenticator and authorizer.
- They check a shared secret header, `X-CoreScope-Hook-Secret`, in
constant time.
- A username CoreScope does not know gets `ignore`, so EMQX falls
through to its built-in database and existing users keep working.
- A personal login may only publish `meshcore/<iata>/<pubkey>/<type>`,
with a 64-hex pubkey that is unbound or bound to the same user.
- The first publish for an unbound pubkey binds that observer to the
user (first come wins), until the user owns 20 observers (admin
assignments count too).
- An admin can rebind any observer.

**Credentials.**
- Username `obs_` + 12 random `[a-z0-9]`, password 32 random
`[A-Za-z0-9]`, both from `crypto/rand`.
- The password is stored twice: SHA-256 for the auth check, and sealed
with AES-256-GCM (key from config or `CORESCOPE_OBSERVER_KEY`) so the
user can ask for the same credentials again.
- The UI and the API never show the password. It leaves the server only
in the mail.

**Mail.** One mail with four formats:
- the meshcoretomqtt TOML;
- the observer firmware CLI commands (Gessaman or Dutch-MeshCore, same
`mqttN` CLI), on slot 4: slots 1-2 (Gessaman, Let's Mesh US/EU) and 1-3
(Dutch-MeshCore, `MQTTDefaults.h:19-34`) hold default broker presets on
a fresh install, slot 4 is empty on both. `set mqtt4.server` gets the
bare host: the firmware picks the scheme from the port
(`MQTTBridge.cpp:2094-2145`), and builds before merge 32dd8ad
(2026-10-19) do not accept a scheme there (read from the source, not
tested on a device). A scheme is written only when the port would pick
the wrong one, for example TLS on 8884;
- the Home Assistant meshcore-ha broker form fields;
- the plain values.

Each block starts with a numbered title and one line on where it goes.
The account page can resend the mail, at most 3 times per 24 h.

**Account and admin UI.**
- The account page gets an "Observer MQTT login" section
(`#/account?section=observers`) with the username, the bound observers
and the resend button.
- The admin page gets an "Observer logins" tab
(`#/admin?tab=observer-logins`) that lists logins with revoke and
reissue, and every observer with its owner, "not migrated" or
"orphaned", with a reassign control.

**Notifications.** Creating a login adds `observer.offline.own` ("My
observers go offline") to the owner's notification events; mails go out
once the user has notifications switched on, and the user can untick it.

**Storage.** users.db schema v7 adds `mqtt_logins` and
`observer_bindings`. The server still writes only users.db through
`internal/users`, and the analyzer DB stays read-only.

**Docs.** `docs/observer-mqtt-logins.md` covers the EMQX setup (authn
and authz chains, the hook body templates, the closing `^obs_` deny rule
and why it is needed), how to enable it on a running EMQX through the
REST API step by step, with a check and a rollback per step, and how to
migrate observers off a shared login. `config.example.json` has the
block with placeholder values.

## Enabling it on an instance

Full steps in `docs/observer-mqtt-logins.md`. In short:

1. **CoreScope:** add `userManagement.observerLogins` to `config.json`
with `enabled: true`, a `hookSecret` (`openssl rand -hex 32`), an
`encryptionKey` (`openssl rand -base64 32`) and the `broker` host, port,
transport and TLS that observers should use. Restart. Keep the key safe:
without it, mails cannot be resent and logins must be reissued.
2. **EMQX, authorization:** add a file source with `{deny, {username,
{re, "^obs_"}}, all, ["#"]}.` and move it to the end, then add the HTTP
authorizer pointing at `/api/mqtt-hook/emqx/authz` (it lands at the
front).
3. **EMQX, authentication:** add the HTTP authenticator pointing at
`/api/mqtt-hook/emqx/authn` and move it to the front, before the
built-in database.
4. **Check:** the shared login still connects, an unknown `obs_` login
is refused, and `metrics.deny` on the authorizer stays 0 for existing
clients.
5. **Owners:** they request a login on their account page and paste the
block for their uploader. Each observer binds to them on its first
publish. An uploader that was refused before step 3 may need a restart.
6. **Later:** remove the shared login from the broker once enough
observers moved.

## Who can claim an observer

Any active account can request a login, and its first publish under an
unbound pubkey binds that observer, also when CoreScope already knew it
from the shared login (first come wins, a deliberate choice: owners
migrate without waiting for an admin). Observer pubkeys are public, and
registration cannot be closed, so with this feature on anyone with a
mail address can claim up to 20 unbound observers per account. Before,
publishing needed the shared password.

What limits it: each claim of a known observer mails the admins, an
admin can unbind or reassign it and revoke the login, and a bound
observer cannot be claimed by another login. The doc spells this out and
suggests assigning known observers to their owners before announcing the
feature. A stricter mode (admin approval of logins, or no auto-bind for
known observers) is not in this PR.

## Performance

`authorize()` runs for every publish that misses the EMQX authz cache.
It takes two short read locks, does map lookups and one topic split.
`BenchmarkAuthorizeBound` (2000 observers, 500 logins): 533.5 ns/op, 96
B/op, 2 allocs/op. The admin observers table renders once per load and
updates single rows after a change.

## Verification

- All 16 Go modules: `bash scripts/allmod.sh test ./...` green.
- Skipped `TestSaveGeoFilterPreservesFileMode` and
`TestWriteStatsAtomic_SymlinkAtDestIsReplaced`: both fail on Windows on
a clean master too.
- `sh test-all.sh` green.
- `tests/e2e/test-user-management-e2e.js` 36/36 locally. It covers:
request, mail, resend, revoke, reissue, reassign, no password in the
page or API body, and axe. The CI job for this suite gets the
`observerLogins` block with fake values in `deploy.yml`.
- Local EMQX 5.8.6 in Docker against a local CoreScope:
- a personal login connects and publishes, which binds the observer; a
second personal login is denied on that observer's topic, and subscribe
is denied;
- with CoreScope stopped, the built-in users still connect and publish,
and new personal-login connects are refused;
- the `^obs_` deny rule blocks an already-connected personal login while
CoreScope is down; without the rule, the same publish was delivered.

- Production: running on an EMQX 5.8.9 broker with about 50 clients
since 2026-10-25, set up through the REST API as in the doc. After the
hooks went in, the authorizer answered `ignore` for every existing login
(0 denies, 0 dropped messages), and 4 observers were bound to their
owners through personal logins on the same evening, with the shared
login still in use by the rest.

## Known behaviour

EMQX's HTTP client can mark the hook resource down for one health-check
interval (15 s measured) when CoreScope closes an idle keep-alive
connection just as a health check runs. During that window new
personal-login connects are refused and the uploader reconnects. I saw
it 3 times in about 25 minutes of runs with a forced 1-5 s idle timeout,
and could not trigger it on demand. EMQX 5.8.6 has no setting for it,
and 5.10.5 ships the same client code. The doc describes it and the
alarm to watch for.

## Not in this PR

- mosquitto (mosquitto-go-auth). EMQX only.
- A websocket path field: the mail does not carry one, the doc tells
operators to match the listener path.
- Disabling the shared login. That is an operator step after migration,
described in the doc.
- Tested on EMQX 5.8.6 (Docker) and 5.8.9 (production) only.

Note on cb9f4ff3: its message says the scheme problem was reported by an
operator. That report turned out to come from before the broker used the
hook, so it does not show this cause; the change rests on the firmware
source only.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

2026-10-26 09:50:20 +01:00

efitenandClaude Opus 5.5

abbe6846ff

feat: propose and approve hashtag channels (user management part D) (#2139 )

Part D of #2128: logged-in users propose a hashtag channel, an admin
approves, rejects or later revokes it, and the ingestor decrypts
approved channels without a restart. It is the first consumer of a
generic proposal table, and answers the request in #2092.

Stacked on #2138 (part C). Review the commits after that branch; GitHub
shows C's commits here until it is merged.

## The situation

- An instance decrypts only the hashtag channels listed in
`hashChannels`. Users who want a city or club channel shown have to ask
the operator, who edits `config.json` and restarts the ingestor.
- #2092 asked for a suggest-and-approve flow; a downstream fork
(dborup/CoreScope#99) runs one with anonymous suggestions and a file
queue.

## What this PR adds

**Name rules (`internal/channel`).** `ValidateHashtagName`, mirrored in
the browser:
- at most 31 UTF-8 bytes including `#` (firmware
`ChannelDetails.name[32]`), case preserved, `Public` refused in any
case;
- control, bidi, separator and format characters refused except ZWJ,
plus invisible fillers (`Other_Default_Ignorable_Code_Point`, U+2800,
non-ASCII spaces).

Go and JS give the same verdict for every code point.

**Store.** Schema v4, one generic `proposals` table (kind, subject,
status pending/approved/rejected/revoked, proposer, reviewer, note). One
row per (kind, subject); limits are checked in the same transaction as
the change.

**Server** (only with `userManagement.channelProposals.enabled`)
- `POST /api/proposals`, `GET /api/account/proposals`, `GET
/api/admin/proposals`, `POST
/api/admin/proposals/{id}/approve|reject|revoke`.
- 400 for an invalid name. 409 for a duplicate, a rejected name, or a
name already in `hashChannels`/`channelKeys` ("this channel is already
decrypted on this instance"). 429 over a limit (`maxPending` 100,
`maxApproved` 128, `perUserPerDay` 5).
- `GET /api/channels` gains `approvedChannels`, so approved channels are
listed before they have traffic. With the feature off the response is
byte-identical.

**Ingestor.** Opens `users.db` with `mode=ro` every 60 seconds,
re-validates the names, and swaps its key map through an
`atomic.Pointer`. Configured channels always win, and a failed read
keeps the last good set. The ingestor never writes `users.db`; AGENTS.md
now says so.

**Frontend.** "Propose for everyone" in the add-channel dialog, an
"approved" marker in the channel list, "My proposals" on the account
page, and an admin "Proposals" tab with a warning on approve. The packet
detail pane now escapes the channel name: until now only the operator
chose those names.

Spec:
[`docs/specs/2026-10-07-channel-proposals-design.md`](https://github.com/efiten/CoreScope/blob/feat/channel-proposals/docs/specs/2026-10-07-channel-proposals-design.md).

## Performance

Every key is tried on each GRP_TXT that no key opens (`decodeGrpTxt`).
Benchmark of that worst case:

| Keys | Time per packet |
|---|---|
| 320 configured | 204 to 216 µs |
| 320 configured + 128 approved | 272 to 320 µs |

That is about +33%, linear in the key count and capped by `maxApproved`.
With the feature off the decoder gets the identical map and no ticker
runs.

## Verification

- `internal/channel`, `internal/users` (with `-race`), `cmd/server` and
`cmd/ingestor` pass locally, 53 new Go tests. The ingestor's
`TestWriteStatsAtomic_SymlinkAtDestIsReplaced` needs symlink rights on
Windows and fails on `master` too.
- `sh test-all.sh` exits 0; the XSS gate in diff mode passes.
- User-management E2E: 21 of 21 steps locally (propose, approve, the
channel appears, revoke removes it).
- On our staging and production instance since 7 October 2026.

## Not in this PR

- Private (PSK) channels: their keys stay in the browser (#725).
- Anonymous proposals.
- Names that need ZWNJ (some Persian and Urdu spellings) and subdivision
flags are refused, because ZWNJ and tag characters are format
characters. Allowing them later only widens the rule, so no approved
name gets stranded.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

2026-10-08 21:51:42 +02:00

efitenandClaude Opus 5.5

18a634c115

feat: admin dashboard with overview and audit log (user management part C) (#2138 )

Part C of #2128: an admin area with an overview, the user table and a
global audit log. Off by default like the rest of user management: with
`userManagement` off nothing changes, and non-admins get no new UI.

This is the first of 4 stacked PRs (C, D, E, then a small export/backup
follow-up). Each later one contains this branch; review them in order.

## The situation

- An admin could manage users one by one in `#/admin/users`, but nothing
showed whether the instance needs attention: accounts stuck in
activation, bouncing mail, someone guessing a password, an MQTT source
that dropped.
- The audit log existed (`internal/users/audit.go`) but could only be
read per user, and logins were not recorded.

## What this PR adds

**Store (`internal/users`)**
- Schema v3: an index on `audit_log(at)`.
- `AuditList` with filters (action or group prefix like `user.login.*`,
user as actor or target, period) and keyset pagination; `PruneAudit`;
`Stats` for the user figures.

**Server**
- `GET /api/admin/stats` (typed struct): accounts by status, admins,
registrations and active users over 7 and 30 days, logins and failed
logins in 24 hours, mail by final status, and the attention items
computed server-side.
- `GET /api/admin/audit`: filtered, newest first, `next` cursor.
- `GET /api/admin/users` gains `bouncing=1`.
- Logins are audited as `user.login` and `user.login.failed` (reason
`wrong_password`, `pending` or `disabled`). An unknown address writes no
row. The writes are asynchronous, so the login response does not wait on
`users.db`. Login rows are pruned after 90 days.

**Frontend**
- `#/admin?tab=overview|users|audit`: `admin.js` (tab shell),
`admin-overview.js` ("Needs attention", Users card, System card from the
existing health, MQTT and observer endpoints), `admin-audit.js` (filters
in the URL, "Load more"). `admin-users.js` becomes the Users tab; the
old `#/admin/users` link rewrites to it.
- `/api/healthz` is read on open and on Refresh only, not on the
60-second timer, because it walks every packet under a read lock.

Spec:
[`docs/specs/2026-10-07-admin-dashboard-design.md`](https://github.com/efiten/CoreScope/blob/feat/admin-dashboard/docs/specs/2026-10-07-admin-dashboard-design.md).

## Verification

- `internal/users` and `cmd/server`: `go vet` and `go test` pass
locally, 26 new Go tests. One upstream test,
`TestSaveGeoFilterPreservesFileMode`, also fails on Windows on plain
`master` (file modes) and is unrelated.
- `sh test-all.sh` exits 0; the XSS gate in diff mode passes.
- User-management E2E with the `e2etest` build: 13 of 13 steps locally.
- Running on our staging and production instance since 7 October 2026.

## Not in this PR

- Restricting existing pages (perf, MQTT status) to logged-in users or
admins. The admin area adds no access mechanism of its own, so that
stays a later router rule plus endpoint check.
- Server-enforced customizer tab restrictions (#1508).
- The 24-hour, 5-attempt and 10-minute attention thresholds are
constants for now (AGENTS.md rule 8: customizer later).

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

2026-10-08 21:38:30 +02:00