Commit Graph

57 Commits

Author SHA1 Message Date
efiten
b7c2cb070c docs: geofilter manual + config.example.json entry (#734)
## Summary

- Add missing `geo_filter` block to `config.example.json` with polygon
example, `bufferKm`, and inline `_comment`
- Add `docs/user-guide/geofilter.md`: full operator guide covering
config schema, GeoFilter Builder workflow, and prune script as one-time
migration tool
- Add Geographic filtering section to `docs/user-guide/configuration.md`
with link to the full guide

Closes #669 (M1: documentation)

## Test plan

- [x] `config.example.json` parses cleanly (no JSON errors)
- [x] `docs/user-guide/geofilter.md` renders correctly in GitHub preview
- [x] Link from `configuration.md` to `geofilter.md` resolves

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 22:43:19 -07:00
efiten
1de80a9eaf feat: serve geofilter builder from app, link from customizer (#735)
## Summary

Part of #669 — M2: Link the builder from the app.

- **`public/geofilter-builder.html`** — the existing
`tools/geofilter-builder.html` is now served by the static file server
at `/geofilter-builder.html`. Additions vs the original: a `← CoreScope`
back-link in the header, inline code comments explaining the output
format, and a help bar below the output panel with paste instructions
and a link to the documentation.
- **`public/customize-v2.js`** — adds a "Tools" section at the bottom of
the Export tab with a `🗺️ GeoFilter Builder →` link and a one-line
description.
- **`docs/user-guide/customization.md`** — documents the new GeoFilter
Builder entry in the Export tab.

> **Note:** `tools/geofilter-builder.html` is kept as-is for
local/offline use. The `public/` copy is what the server serves.

> **Depends on:** #734 (M1 docs) for `docs/user-guide/geofilter.md` —
the link in the help bar references that file. Can be merged
independently; the link still works once M1 lands.

## Test plan

- [x] Open the app, go to Customizer → Export tab — "Tools" section
appears with GeoFilter Builder link
- [x] Click the link — opens `/geofilter-builder.html` in a new tab
- [x] Builder loads the Leaflet map, draw 3+ points — JSON output
appears
- [x] Copy button works, output is valid `{ "geo_filter": { ... } }`
JSON
- [x] `← CoreScope` back-link navigates to `/`
- [x] Help bar shows paste instructions

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 22:42:27 -07:00
Kpa-clawbot
26c47df814 fix: entrypoint .env support + deployment docs for bare docker run (#704)
## Summary

Fixes #702 — `.env` file `DISABLE_MOSQUITTO`/`DISABLE_CADDY` ignored
when using `docker run`.

## Changes

### Entrypoint sources `/app/data/.env`
The entrypoint now sources `/app/data/.env` (if present) before the
`DISABLE_*` checks. This works regardless of how the container is
started — `docker run`, compose, or `manage.sh`.

```bash
if [ -f /app/data/.env ]; then
  set -a
  . /app/data/.env
  set +a
fi
```

### `DISABLE_CADDY` added to compose files
Both `docker-compose.yml` and `docker-compose.staging.yml` now forward
`DISABLE_CADDY` to the container environment (was missing — only
`DISABLE_MOSQUITTO` was wired).

### Deployment docs updated
- `docs/deployment.md`: bare `docker run` is now the primary/recommended
approach with a full parameter reference table
- Documents the `/app/data/.env` convenience feature
- Compose and `manage.sh` marked as legacy alternatives
- `DISABLE_CADDY` added to the environment variable reference

### README quick start updated
Shows the full `docker run` command with `--restart`, ports, and
volumes. Includes HTTPS variant. Documents `-e` flags and `.env` file.

### v3.5.0 release notes
Updated the env var documentation to mention the `.env` file support.

## Testing
- All Go server tests pass
- All Go ingestor tests pass
- No logic changes to Go code — entrypoint shell script + docs only

---------

Co-authored-by: you <you@example.com>
2026-04-11 20:43:16 -07:00
you
111b03cea1 docs: lead with pre-built Docker image as the headline 2026-04-08 07:22:07 +00:00
you
34c56d203e docs: promote API docs to own section with live analyzer.00id.net links, fix transition section 2026-04-08 07:21:11 +00:00
you
cc9f25e5c8 docs: fix release notes — bind mount for caddy-data, no personal paths, add Caddyfile example 2026-04-08 07:20:02 +00:00
you
2e33eb7050 docs: add HTTPS/Caddyfile mount to release notes and upgrade steps 2026-04-08 07:14:15 +00:00
you
6dd0957507 docs: use v3.5.0 tag in release notes, :latest requires git tag 2026-04-08 07:05:58 +00:00
you
e22ee3f0ad docs: docker run based upgrade, no compose 2026-04-08 07:03:05 +00:00
you
f7f1bb08d0 docs: add cd to compose dir in upgrade steps 2026-04-08 07:01:43 +00:00
you
84da4d962d docs: release notes with juice 2026-04-08 07:00:59 +00:00
you
ad0a10c009 docs: fix transition steps — compose-based, not docker run 2026-04-08 06:59:54 +00:00
you
c1f268d3b9 docs: add concrete transition steps to release notes 2026-04-08 06:58:39 +00:00
you
f5d25f75c6 docs: trim release notes — less book, more changelog 2026-04-08 06:56:50 +00:00
you
cde62166cb docs: v3.5.0 release notes + API documentation across README, deployment guide, FAQ
- Release notes for 95 commits since v3.4.1
- OpenAPI/Swagger docs: /api/spec and /api/docs called out everywhere
- Deployment guide: new API Documentation section
- README: API docs link added
- FAQ: 'Where is the API documentation?' entry
- Test plans for v3.4.2 validation
2026-04-08 06:55:25 +00:00
efiten
2bff89a546 feat: deep link P1 UI states — nodes tab, packets filters, channels node panel (#536) (#618)
## Summary

- **nodes.js**: `#/nodes?tab=repeater` and `#/nodes?search=foo` — role
tab and search query are now URL-addressable; state resets to defaults
on re-navigation
- **packets.js**: `#/packets?timeWindow=60` and
`#/packets?region=US-SFO` — time window and region filter survive
refresh and are shareable
- **channels.js**: `#/channels/{hash}?node=Name` — node detail panel is
URL-addressable; auto-opens on load, URL updates on open/close
- **region-filter.js**: adds `RegionFilter.setSelected(codesArray)` to
public API (needed for URL-driven init)

All changes use `history.replaceState` (not `pushState`) to avoid
polluting browser history. URL params override localStorage on load;
localStorage remains fallback.

## Implementation notes

- Router strips query string before computing `routeParam`, so all pages
read URL params directly from `location.hash`
- `buildNodesQuery(tab, searchStr)` and `buildPacketsUrl(timeWindowMin,
regionParam)` are pure functions exposed on `window` for testability
- Region URL param is applied after `RegionFilter.init()` via a
`_pendingUrlRegion` module-level var to keep ordering explicit
- `showNodeDetail` captures `selectedHash` before the async `lookupNode`
call to avoid stale URL construction

## Test plan

- [x] `node test-frontend-helpers.js` — 459 passed, 0 failed (includes 6
`buildNodesQuery` + 5 `buildPacketsUrl` unit tests)
- [x] Navigate to `#/nodes?tab=repeater` — Repeaters tab active on load
- [x] Click a tab, verify URL updates to `#/nodes?tab=room`
- [x] Navigate to `#/packets?timeWindow=60` — time window dropdown shows
60 min
- [x] Change time window, verify URL updates
- [x] Navigate to `#/channels/{hash}` and click a sender name — URL
updates to `?node=Name`
- [x] Reload that URL — node panel re-opens

Closes #536

🤖 Generated with [Claude Code](https://claude.ai/claude-code)

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 15:43:25 -07:00
you
6b9946d9c6 docs: timestamp-based packet filter spec (#289) 2026-04-06 01:22:15 +00:00
Kpa-clawbot
a068e3e086 feat: zero-config defaults + deployment docs (M3-M4, #610) (#631)
## Zero-Config Defaults + Deployment Docs

Make CoreScope start with zero configuration — no `config.json`
required. The ingestor falls back to sensible defaults (local MQTT
broker, standard topics, default DB path) when no config file exists.

### What changed

**`cmd/ingestor/config.go`** — `LoadConfig` no longer errors on missing
config file. Instead it logs a message and uses defaults. If no MQTT
sources are configured (from file or env), defaults to
`mqtt://localhost:1883` with `meshcore/#` topic.

**`cmd/ingestor/main.go`** — Removed redundant "no MQTT sources" fatal
(now handled in config layer). Improved the "no connections established"
fatal with actionable hints.

**`README.md`** — Replaced "Docker (Recommended)" section with a
one-command quickstart using the pre-built image. No build step, no
config file, just `docker run`.

**`docs/deployment.md`** — New comprehensive deployment guide covering
Docker, Compose, config reference, MQTT setup, TLS/HTTPS, monitoring,
backup, and troubleshooting.

### Zero-config flow

```
docker run -d -p 80:80 -v corescope-data:/app/data ghcr.io/kpa-clawbot/corescope:latest
```

1. No config.json found → defaults used, log message printed
2. No MQTT sources → defaults to `mqtt://localhost:1883`
3. Internal Mosquitto broker already running in container → connection
succeeds
4. Dashboard shows empty, ready for packets

### Review fixes (commit 13b89bb)

- Removed `DISABLE_CADDY` references from all docs — this env var was
never implemented in the entrypoint
- Fixed `/api/stats` example in deployment guide — showed nonexistent
fields (`mqttConnected`, `uptimeSeconds`, `activeNodes`)
- Improved MQTT connection failure message with actionable
troubleshooting hints

Closes #610

---------

Co-authored-by: you <you@example.com>
2026-04-05 15:04:49 -07:00
you
24335164d6 docs: table sorting consistency spec (#620) 2026-04-05 21:56:09 +00:00
Kpa-clawbot
f7000992ca fix(rf-health): auto-scale airtime Y-axis + hover tooltips (#600) (#623)
## Summary

Addresses user feedback on #600 — two improvements to RF Health detail
panel charts:

### 1. Auto-scale airtime Y-axis
Previously fixed 0-100% which made low-activity nodes unreadable (e.g.
0.1% TX barely visible). Now auto-scales to the actual data range with
20% headroom (minimum 1%), matching how the noise floor chart already
works.

### 2. Hover tooltips on all chart data points
Invisible SVG `<circle>` elements with native `<title>` tooltips on
every data point across all 4 charts:
- **Noise floor**: `NF: -112.3 dBm` + UTC timestamp
- **Airtime**: `TX: 2.1%` or `RX: 8.3%` + UTC timestamp  
- **Error rate**: `Err: 0.05%` + UTC timestamp
- **Battery**: `Batt: 3.85V` + UTC timestamp

Uses native browser SVG tooltips — zero dependencies, accessible, no JS
event handlers.

### Design rationale (Tufte)
- Auto-scaling increases data-ink ratio by eliminating wasted vertical
space
- Tooltips provide detail-on-demand without cluttering the chart with
labels on every point

### Spec update
Added M2 feedback improvements section to
`docs/specs/rf-health-dashboard.md`.

---------

Co-authored-by: you <you@example.com>
2026-04-05 13:08:05 -07:00
you
dc635775b5 docs: TUI spec updated with expert feedback + MVP definition 2026-04-05 07:12:11 +00:00
you
8a94c43334 docs: startup performance spec — serve HTTP within 2 minutes on any DB size 2026-04-05 07:09:55 +00:00
you
6aaa5cdc20 docs: add user guide — getting started, pages, config, FAQ 2026-04-05 07:09:54 +00:00
you
788005bff7 docs: clarify Docker tag strategy — pin to vX.Y.Z for production, edge for testing 2026-04-05 07:09:44 +00:00
you
af03f9aa57 docs: deployment simplification spec — pre-built Docker images + one-line deploy 2026-04-05 07:06:35 +00:00
you
14732135b7 docs: proposal for terminal/TUI interface into CoreScope 2026-04-05 06:56:33 +00:00
you
cbc3e3ce13 docs: movable UI panels spec — draggable panel positioning (#279) 2026-04-05 06:54:45 +00:00
you
1796493ec0 docs: channel color highlighting spec (#271)
Custom color assignment for hash channels in Live tab.
Reviewed by Tufte, Torvalds, and Doshi personas.
2026-04-05 06:45:53 +00:00
you
aaf00d0616 docs: add M5 Prometheus/Grafana metrics export to RF Health spec 2026-04-05 05:02:36 +00:00
you
41c046c974 docs: RF Health Dashboard spec — observer radio metrics
Per-observer time-series charts for noise floor, TX/RX airtime, CRC errors,
and battery. Small multiples grid design. MVP-first milestones.

Reviewed by Carmack (perf), Munger (failure modes), radio expert (hardware),
Tufte (visualization), and Doshi (product strategy).
2026-04-05 04:42:32 +00:00
you
0a55717283 docs: add PSK brute-force attack with timestamp oracle to security analysis
Weak passphrases with no KDF stretching are the #1 practical threat.
Timestamp in plaintext block 0 serves as known-plaintext oracle for
instant key verification from a single captured packet.

Key findings:
- decode_base64() output used directly as AES key, no KDF
- Short passphrases produce <16 byte keys (reduced key space)
- No salt means global precomputed attacks work
- 3-word passphrase crackable in ~2 min on commodity GPU

Reviewed by djb and Dijkstra personas. Corrections applied:
- GPU throughput upgraded from 10^9 to 10^10 AES/sec baseline
- Oracle strengthened: bytes 4+ (type byte, sender name) also predictable
- Dictionary size assumptions made explicit
- Zipf's law caveat added (humans don't choose uniformly)
- base64 short-passphrase key truncation issue documented
2026-04-05 00:58:57 +00:00
you
bcab31bf72 docs: AES-128-ECB security analysis — block-level vulnerability assessment
Formal analysis of MeshCore's ECB encryption for channel and direct messages.
Reviewed by djb and Dijkstra expert personas through 3 revisions.

Key findings:
- Block 0 has accidental nonce (4-byte timestamp) preventing repetition
- Blocks 1+ are pure deterministic ECB with no nonce — vulnerable to
  frequency analysis for repeated message content
- Partial final block attack: zero-padding reduces search space
- HMAC key reuse: AES key is first 16 bytes of HMAC key (same material)
- Recommended fix: switch to AES-128-CTR mode
2026-04-05 00:44:21 +00:00
you
9099154514 docs: add v3.4 release notes 2026-04-03 08:26:05 +00:00
Kpa-clawbot
64745f89b1 feat: customizer v2 — event-driven state management (#502) (#503)
## Summary

Implements the customizer v2 per the [approved
spec](docs/specs/customizer-rework.md), replacing the v1 customizer's
scattered state management with a clean event-driven architecture.
Resolves #502.

## What Changed

### New: `public/customize-v2.js`
Complete rewrite of the customizer as a self-contained IIFE with:

- **Single localStorage key** (`cs-theme-overrides`) replacing 7
scattered keys
- **Three state layers:** server defaults (immutable) → user overrides
(delta) → effective config (computed)
- **Full data flow pipeline:** `write → read-back → merge → atomic
SITE_CONFIG assign → apply CSS → dispatch theme-changed`
- **Color picker optimistic CSS** (Decision #12): `input` events update
CSS directly for responsiveness; `change` events trigger the full
pipeline
- **Override indicator dots** (●) on each field — click to reset
individual values
- **Section-level override count badges** on tabs
- **Browser-local banner** in panel header: "These settings are saved in
your browser only"
- **Auto-save status indicator** in footer: "All changes saved" /
"Saving..." / "⚠️ Storage full"
- **Export/Import** with full shape validation (`validateShape()`)
- **Presets** flow through the standard pipeline
(`writeOverrides(presetData) → pipeline`)
- **One-time migration** from 7 legacy localStorage keys (exact field
mapping per spec)
- **Validation** on all writes: color format, opacity range, timestamp
enum values
- **QuotaExceededError handling** with visible user warning

### Modified: `public/app.js`
Replaced ~80 lines of inline theme application code with a 15-line
`_customizerV2.init(cfg)` call. The customizer v2 handles all merging,
CSS application, and global state updates.

### Modified: `public/index.html`
Swapped `customize.js` → `customize-v2.js` script tag.

### Added: `docs/specs/customizer-rework.md`
The full approved spec, included in the repo for reference.

## Migration

On first page load:
1. Checks if `cs-theme-overrides` already exists → skip if yes
2. Reads all 7 legacy keys (`meshcore-user-theme`,
`meshcore-timestamp-*`, `meshcore-heatmap-opacity`,
`meshcore-live-heatmap-opacity`)
3. Maps them to the new delta format per the spec's field-by-field
mapping
4. Writes to `cs-theme-overrides`, removes all legacy keys
5. Continues with normal init

Users with existing customizations will see them preserved
automatically.

## Dark/Light Mode

- `theme` section stores light mode overrides, `themeDark` stores dark
mode overrides
- `meshcore-theme` localStorage key remains **separate** (view
preference, not customization)
- Switching modes re-runs the full pipeline with the correct section

## Testing

- All existing tests pass (`test-packet-filter.js`, `test-aging.js`,
`test-frontend-helpers.js`)
- Old `customize.js` is NOT modified — left in place for reference but
no longer loaded

## Not in Scope (per spec)

- Undo/redo stack
- Cross-tab synchronization
- Server-side admin import endpoint
- Map config / geo-filter overrides

---------

Co-authored-by: you <you@example.com>
2026-04-02 21:14:38 -07:00
Kpa-clawbot
5aa4fbb600 chore: normalize all files to LF line endings 2026-03-30 22:52:46 -07:00
KpaBap
d538d2f3e7 Merge branch 'master' into rename/corescope-migration 2026-03-28 16:21:57 -07:00
Kpa-clawbot
a6413fb665 fix: address review — stale URLs, manage.sh branding, proto comment
- docs/go-migration.md: update clone URL meshcore-dev/meshcore-analyzer → Kpa-clawbot/meshcore-analyzer
- manage.sh: rename header comment and help footer from 'MeshCore Analyzer' to 'CoreScope'
- proto/config.proto: update default branding comment from 'MeshCore Analyzer' to 'CoreScope'

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-28 14:44:53 -07:00
Kpa-clawbot
cdcaa476f2 rename: MeshCore Analyzer → CoreScope (Phase 1 — backend + infra)
Rename product branding, binary names, Docker images, container names,
Go modules, proto go_package, CI, manage.sh, and documentation.

Preserved (backward compat):
- meshcore.db database filename
- meshcore-data / meshcore-staging-data directory paths
- MQTT topics (meshcore/#, meshcore/+/+/packets, etc.)
- proto package namespace (meshcore.v1)
- localStorage keys

Changes by category:
- Go modules: github.com/corescope/{server,ingestor}
- Binaries: corescope-server, corescope-ingestor
- Docker images: corescope:latest, corescope-go:latest
- Containers: corescope-prod, corescope-staging, corescope-staging-go
- Supervisord programs: corescope, corescope-server, corescope-ingestor
- Branding: siteName, heroTitle, startup logs, fallback HTML
- Proto go_package: github.com/corescope/proto/v1
- CI: container refs, deploy path
- Docs: 8 markdown files updated

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-28 14:08:15 -07:00
Kpa-clawbot
1453fb6492 docs: add CoreScope rename migration guide
Documents what existing users need to update when the rename
from MeshCore Analyzer to CoreScope lands:
- Git remote URL update
- Docker image/container name changes
- Config branding.siteName (if customized)
- CI/CD references (if applicable)
- Confirms data dirs, MQTT, browser state unchanged

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-28 13:51:41 -07:00
Kpa-clawbot
69eae6415a docs: add Go migration guide for existing Node.js users
Comprehensive guide covering:
- Prerequisites and backup steps
- Config compatibility (MQTT URL normalization, shared fields)
- Switch procedure via Docker Compose or manage.sh
- DB compatibility (v3 schema shared, bidirectional)
- Verification steps (engine field, packet counts, MQTT, WebSocket)
- Rollback instructions
- Known differences (companion bridge gaps, process model)
- Migration gaps to track (manage.sh --engine, Docker Hub, etc.)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-28 00:29:50 -07:00
Kpa-clawbot
47ee63ed55 fix: #191 #192 #193 #194 — repeater-only collision matrix, expand=observations, store-based node health, goRuntime in perf
#191: Hash collision matrix now filters to role=repeater only (routing-relevant)
#192: expand=observations in /api/packets now returns full observation details (txToMap includes observations, stripped by default)
#193: /api/nodes/:pubkey/health uses in-memory PacketStore when available instead of slow SQL queries
#194: goRuntime (heapMB, sysMB, numGoroutine, numGC, gcPauseMs) restored in /api/perf response

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-27 21:25:19 -07:00
Kpa-clawbot
3ddd9662e6 docs: add formal API contract spec for all REST endpoints and WebSocket messages
Document the exact response shape, query parameters, and type information
for every endpoint in server.js. This is the authoritative contract that
both Node.js and Go backends must conform to.

Covers:
- All 30+ REST endpoints with full JSON response schemas
- WebSocket message envelope and data shapes
- Shared object shapes (Packet, Observation, DecodedHeader, DecodedPath)
- Query parameter documentation with types and defaults
- Null rules, pagination conventions, error response format
- Frontend consumer matrix (which page reads which WS fields)
- Payload type and route type reference tables

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-27 13:55:42 -07:00
you
77adc311cb docs: simplify backup/update sections — use manage.sh, drop volume inspect
Backup section now shows manage.sh commands instead of raw docker
volume inspect paths. Update section is one command.
2026-03-25 00:46:04 +00:00
you
eaa42ed097 feat: add manage.sh helper script for setup and management
Interactive setup: checks Docker, creates config, prompts for domain,
validates DNS, builds and runs the container.

Commands: setup, start, stop, restart, status, logs, update, backup,
restore, mqtt-test. Colored output, error handling, confirmations
for destructive operations.

Updated DEPLOYMENT.md to show manage.sh as the primary quick start.
2026-03-25 00:40:08 +00:00
you
3f7fa89acb docs: full rewrite of deployment guide
- Added HTTPS Options section: auto (Caddy), bring your own cert,
  Cloudflare Tunnel, behind existing proxy, HTTP-only
- Expanded MQTT Security into its own section with 3 options + recommendation
- Fixed DB backup to use volume path not docker cp
- Added restore instructions
- Expanded troubleshooting table (rate limits → use own cert or different subdomain)
- Clarified that MQTT 1883 is NOT exposed by default in quick start
- Added tip to save docker run as a script
- Restructured for cleaner TOC
- Removed condescension, kept clarity
2026-03-25 00:38:06 +00:00
you
3cb6c43c52 docs: fix backup instructions — DB is on volume, not inside container 2026-03-25 00:35:32 +00:00
you
3ada1f1c3d docs: replace ASCII art with Mermaid diagrams in deployment guide
- Traffic flow: browser/observers → Caddy/Mosquitto → Node.js → SQLite
- Container internals: supervisord → services
- Data flow sequence: LoRa → observer → MQTT → server → browser
- Quick start flow: 5-step overview
2026-03-25 00:32:05 +00:00
you
69182828b3 docs: add table of contents to deployment guide 2026-03-25 00:31:00 +00:00
you
9b80cc5923 docs: remove condescending tone from deployment guide 2026-03-25 00:30:22 +00:00
you
ef1e84e3d9 docs: rewrite deployment guide for complete beginners
Added: what is Docker, how to install it, what is a server,
where to get a domain, how to open ports. Every command explained.
Assumes zero DevOps knowledge.
2026-03-25 00:28:46 +00:00