Files
meshcore-analyzer/docs/user-guide/configuration.md
T
efiten e460932668 fix(store): apply retentionHours cutoff in Load() to prevent OOM on cold start (#917)
## Problem

`Load()` loaded all transmissions from the DB regardless of
`retentionHours`, so `buildSubpathIndex()` processed the full DB history
on every startup. On a DB with ~280K paths this produces ~13.5M subpath
index entries, OOM-killing the process before it ever starts listening —
causing a supervisord crash loop with no useful error message.

## Fix

Apply the same `retentionHours` cutoff to `Load()`'s SQL that
`EvictStale()` already uses at runtime. Both conditions
(`retentionHours` window and `maxPackets` cap) are combined with AND so
neither safety limit is bypassed.

Startup now builds indexes only over the retention window, making
startup time and memory proportional to recent activity rather than
total DB history.

## Docs

- `config.example.json`: adds `retentionHours` to the `packetStore`
block with recommended value `168` (7 days) and a warning about `0` on
large DBs
- `docs/user-guide/configuration.md`: documents the field and adds an
explicit OOM warning

## Test plan

- [x] `cd cmd/server && go test ./... -run TestRetentionLoad` — covers
the retention-filtered load: verifies packets outside the window are
excluded, and that `retentionHours: 0` still loads everything
- [x] Deploy on an instance with a large DB (>100K paths) and
`retentionHours: 168` — server reaches "listening" in seconds instead of
OOM-crashing
- [x] Verify `config.example.json` has `retentionHours: 168` in the
`packetStore` block
- [x] Verify `docs/user-guide/configuration.md` documents the field and
warning

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Kpa-clawbot <kpaclawbot@outlook.com>
2026-05-01 06:47:55 +00:00

6.5 KiB
Raw Blame History

Configuration

CoreScope is configured via config.json in the server's working directory. Copy config.example.json to get started.

Core settings

Field Default Description
port 3000 HTTP server port
apiKey Secret key for admin API endpoints (POST/PUT routes)
dbPath Path to SQLite database file (optional, defaults to meshcore.db)

MQTT

"mqtt": {
  "broker": "mqtt://localhost:1883",
  "topic": "meshcore/+/+/packets"
}

The ingestor connects to this MQTT broker and subscribes to the topic pattern.

Multiple MQTT sources

Use mqttSources for multiple brokers:

"mqttSources": [
  {
    "name": "local",
    "broker": "mqtt://localhost:1883",
    "topics": ["meshcore/#"]
  },
  {
    "name": "remote",
    "broker": "mqtts://mqtt.example.com:8883",
    "username": "user",
    "password": "pass",
    "topics": ["meshcore/SJC/#"]
  }
]

Branding

Field Description
branding.siteName Site title shown in the nav bar
branding.tagline Subtitle on the home page
branding.logoUrl URL to a custom logo image
branding.faviconUrl URL to a custom favicon

Theme

Colors used throughout the UI. All values are hex color codes.

Field Description
theme.accent Primary accent color (links, buttons)
theme.navBg Navigation bar background
theme.navBg2 Secondary nav background
theme.statusGreen Healthy status color
theme.statusYellow Degraded status color
theme.statusRed Silent/error status color

See Customization for the full list — the theme customizer exposes every color.

Node colors

Default marker colors by role:

"nodeColors": {
  "repeater": "#dc2626",
  "companion": "#2563eb",
  "room": "#16a34a",
  "sensor": "#d97706",
  "observer": "#8b5cf6"
}

Health thresholds

How long (in hours) before a node is marked degraded or silent:

Field Default Description
healthThresholds.infraDegradedHours 24 Repeaters/rooms → degraded after this many hours
healthThresholds.infraSilentHours 72 Repeaters/rooms → silent after this many hours
healthThresholds.nodeDegradedHours 1 Companions/others → degraded
healthThresholds.nodeSilentHours 24 Companions/others → silent

Retention

Field Default Description
retention.nodeDays 7 Nodes not seen in N days move to inactive
retention.packetDays 30 Packets older than N days are deleted daily

Note: Lowering retention does not immediately shrink the database file. SQLite marks deleted pages as free but does not return them to the filesystem unless incremental auto-vacuum is enabled. New databases created after v0.x.x have auto-vacuum enabled automatically. Existing databases require a one-time migration — see the Database guide.

Database

Field Default Description
db.vacuumOnStartup false Run a one-time full VACUUM on startup to enable incremental auto-vacuum (blocks for minutes on large DBs)
db.incrementalVacuumPages 1024 Free pages returned to the OS after each retention reaper cycle

See Database for details on SQLite auto-vacuum, WAL, and manual maintenance. See #919 for background.

Channel decryption

Field Description
channelKeys Object of "label": "hex-key" pairs for decrypting channel messages
hashChannels Array of channel names (e.g., "#LongFast") to match by hash

See Channels for details.

Map defaults

"mapDefaults": {
  "center": [37.45, -122.0],
  "zoom": 9
}

Initial map center and zoom level.

Regions

"regions": {
  "SJC": "San Jose, US",
  "SFO": "San Francisco, US"
}

Named regions for the region filter dropdown. The defaultRegion field sets which region is selected by default.

Cache TTL

All values in seconds. Controls how long the server caches API responses:

"cacheTTL": {
  "stats": 10,
  "nodeList": 90,
  "nodeDetail": 300,
  "analyticsRF": 1800
}

Lower values = fresher data but more server load.

Packet store

Field Default Description
packetStore.maxMemoryMB 1024 Maximum RAM for in-memory packet store
packetStore.estimatedPacketBytes 450 Estimated bytes per packet (for memory budgeting)
packetStore.retentionHours 0 Only load packets younger than N hours on startup and keep them in memory. Set this on any instance with a large DB. 0 = unlimited (loads full DB history — causes OOM on cold start when the DB has hundreds of thousands of paths). Recommended: same as retention.packetDays × 24 (e.g. 168 for 7 days).

Warning: Leaving retentionHours at 0 on a large database will cause the server to OOM-kill itself on every cold start. The full packet history is loaded into the subpath index at startup; a DB with ~280K paths produces ~13M index entries before the process is killed.

Timestamps

Field Default Description
timestamps.defaultMode "ago" Display mode: "ago" (relative) or "absolute"
timestamps.timezone "local" "local" or "utc"
timestamps.formatPreset "iso" Date format preset

Live map

Field Default Description
liveMap.propagationBufferMs 5000 How long to buffer observations before animating

HTTPS

"https": {
  "cert": "/path/to/cert.pem",
  "key": "/path/to/key.pem"
}

Provide cert and key paths to enable HTTPS.

Geographic filtering

"geo_filter": {
  "polygon": [[51.55, 3.80], [51.55, 5.90], [50.65, 5.90], [50.65, 3.80]],
  "bufferKm": 20
}

Restricts ingestion and API responses to nodes within the polygon plus a buffer margin. Remove the block to disable filtering. Nodes with no GPS fix always pass through.

See Geographic Filtering for the full guide including the visual polygon builder and the prune script for cleaning up historical data.

Home page

The home section customizes the onboarding experience. See config.example.json for the full structure including steps, checklist, and footerLinks.