mirror of
https://github.com/Kpa-clawbot/meshcore-analyzer.git
synced 2026-05-15 09:55:15 +00:00
Compare commits
13 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Kpa-clawbot
|
4e4de3fe0d | ||
|
you
|
86ca793b60 | ||
|
you
|
4291b387f5 | ||
|
you
|
3cd7186563 | ||
|
you
|
86a4403136 | ||
|
you
|
c46a60f78a | ||
|
you
|
d4b1aa40d0 | ||
|
you
|
d617a55155 | ||
|
you
|
2106cc0b8b | ||
|
you
|
0acbac6fde | ||
|
you
|
2c675f5ab2 | ||
|
you
|
545df2788d | ||
|
you
|
f872fd90bf |
@@ -1 +1 @@
|
||||
{"schemaVersion":1,"label":"e2e tests","message":"89 passed","color":"brightgreen"}
|
||||
{"schemaVersion":1,"label":"e2e tests","message":"83 passed","color":"brightgreen"}
|
||||
|
||||
@@ -1 +1 @@
|
||||
{"schemaVersion":1,"label":"frontend coverage","message":"36.12%","color":"red"}
|
||||
{"schemaVersion":1,"label":"frontend coverage","message":"37.74%","color":"red"}
|
||||
|
||||
@@ -1,207 +0,0 @@
|
||||
# v3.6.0 - The Forensics
|
||||
|
||||
CoreScope just got eyes everywhere. This release drops **path inspection**, **color-by-hash markers**, **clock skew detection**, **full channel encryption**, an **observer graph**, and a pile of robustness fixes that make your mesh network feel like it's being watched by someone who actually cares.
|
||||
|
||||
134 commits, 105 PRs merged, 18K+ lines added. Here's what shipped.
|
||||
|
||||
---
|
||||
|
||||
## 🚀 New Features
|
||||
|
||||
### Path-Prefix Candidate Inspector (#944, #945)
|
||||
The marquee feature. Click any path segment and CoreScope opens an interactive inspector showing every candidate node that could match that hop prefix - plotted on a map with scoring by neighbor-graph affinity and geographic centroid. Ambiguous hops? Now you can see *why* they're ambiguous and pick the right one.
|
||||
|
||||
**Why you'll love it:** No more guessing which `0xA3` is the real repeater. The inspector lays out every candidate, scores them, and lets you drill in visually.
|
||||
|
||||
### Color-by-Hash Packet Markers (#948, #951)
|
||||
Every packet type gets a vivid, hash-derived color - on the live feed, map polylines, and flying-packet animations. Bright fill with dark outline for contrast. No more monochrome blobs - you can visually track packet flows by color at a glance.
|
||||
|
||||
### Node Filter on Live Page (#924, #771)
|
||||
Filter the live packet stream to show only traffic flowing through a specific node. Pick a repeater, see exactly what it's carrying. That simple.
|
||||
|
||||
### Clock Skew Detection (#746, #752, #828, #850)
|
||||
Full pipeline: backend computes drift using Theil-Sen regression with outlier rejection (#828), the UI shows per-node badges, detail sparklines, and fleet-wide analytics (#752). Bimodal clock severity (#850) surfaces flaky-RTC nodes that toggle between accurate and drifted - instead of hiding them as "No Clock."
|
||||
|
||||
**Why you'll love it:** Nodes with bad clocks silently corrupt your timeline. Now they glow red before they ruin your analysis.
|
||||
|
||||
### Observer Graph (M1+M2) (#774)
|
||||
Observers are now first-class graph citizens. CoreScope builds a neighbor graph from observation overlaps, scores hop-resolver candidates by graph edges (#876), and uses geographic centroid for tiebreaking. The observer topology is visible and queryable.
|
||||
|
||||
### Channel Encryption - Full Stack (#726, #733, #750, #760)
|
||||
Three milestones landed as one: DB-backed channel message history (#726), client-side PSK decryption in the browser (#733), and PSK channel management with add/remove UX and message caching (#750). Add a channel key in the UI, and CoreScope decrypts messages client-side - no server-side key storage. The add-channel button (#760) makes it dead simple.
|
||||
|
||||
**Why you'll love it:** Encrypted channels are no longer black boxes. Add your PSK, see the messages, search history - all without exposing keys to the server.
|
||||
|
||||
### Hash Collision Inspector (#758)
|
||||
The Hash Usage Matrix now shows collision details for all hash sizes. When two nodes share a prefix, you see exactly who collides and at what size.
|
||||
|
||||
### Geofilter Builder - In-App (#735, #900)
|
||||
The geofilter polygon builder is now served directly from CoreScope with a full docs page (#900). No more hunting for external tools. Link from the customizer, draw your polygon, done.
|
||||
|
||||
### Node Blacklist (#742)
|
||||
`nodeBlacklist` in config hides abusive or troll nodes from all views. They're gone.
|
||||
|
||||
### Observer Retention (#764)
|
||||
Stale observers are automatically pruned after a configurable number of days. Your observer list stays clean without manual intervention.
|
||||
|
||||
### Advert Signature Validation (#794)
|
||||
Corrupt packets with invalid advert signatures are now rejected at ingest. Bad data never hits your store.
|
||||
|
||||
### Bounded Cold Load (#790)
|
||||
`Load()` now respects a memory budget - no more OOM on cold start with a fat database. Combined with retention-hours cutoff (#917), cold start is safe on constrained hardware.
|
||||
|
||||
### Multi-Arch Docker Images (#869)
|
||||
Official images now publish `amd64` + `arm64` in a single multi-arch manifest. Raspberry Pi operators: pull and run. No special tags needed.
|
||||
|
||||
### /nodes Detail Panel + Search (#868)
|
||||
The nodes detail panel ships with search improvements (#862) - find nodes fast, see their full detail in a slide-out panel.
|
||||
|
||||
### Deduplicated Top Longest Hops (#848)
|
||||
Longest hops are now deduplicated by pair with observation count and SNR cues. No more seeing the same link 47 times.
|
||||
|
||||
---
|
||||
|
||||
## 🔥 Performance Wins
|
||||
|
||||
### StoreTx ResolvedPath Elimination (#806)
|
||||
The per-transaction `ResolvedPath` computation is gone - replaced by a membership index with on-demand decode. This was one of the hottest paths in the ingestor.
|
||||
|
||||
### Node Packet Queries (#803)
|
||||
Raw JSON text search for node packets replaced with a proper `byNode` index (#673). Night and day.
|
||||
|
||||
### Channel Query Performance (#762, #763)
|
||||
New `channel_hash` column enables SQL-level channel filtering. No more full-table scan to find messages in a channel.
|
||||
|
||||
### SQLite Auto-Vacuum (#919, #920)
|
||||
Incremental auto-vacuum enabled - the database file actually shrinks after retention pruning. No more 2GB database holding 200MB of live data.
|
||||
|
||||
### Retention-Hours Cutoff on Load (#917)
|
||||
`Load()` now applies `retentionHours` at read time, preventing OOM when the DB has more history than memory allows.
|
||||
|
||||
---
|
||||
|
||||
## 🛡️ Security & Robustness
|
||||
|
||||
### MQTT Reconnect with Bounded Backoff (#947, #949)
|
||||
The ingestor now reconnects to MQTT brokers with exponential backoff, observability logging, and bounded retry. No more silent disconnects that kill your data stream.
|
||||
|
||||
---
|
||||
|
||||
## 🐛 Bugs Squashed
|
||||
|
||||
This release exterminates **40+ bugs** — from protocol-level hash mismatches to pixel-level CSS breakage. Operators told us what hurt; we listened.
|
||||
|
||||
- **Path inspector "Show on Map" missed origin and first hop** (#950) - map view now includes all hops
|
||||
- **Content hash used full header byte** (#787) - content hashing now uses payload type bits only, fixing hash collisions between packets that differ only in header flags
|
||||
- **Encrypted channel deep links showed broken UI** (#825, #826, #815) - deep links to encrypted channels now show a lock message instead of broken UI when you don't have the key
|
||||
- **Geofilter longitude wrapping** (#925) - geofilter builder wraps longitude to [-180, 180]; southern hemisphere polygons no longer invert
|
||||
- **Hash filter bypasses saved region filter** (#939) - hash lookups now skip the geo filter as intended
|
||||
- **Companion-as-repeater excluded from path hops** (#935, #936) - non-repeater nodes no longer pollute hop resolution
|
||||
- **Customize panel re-renders while typing** (#927) - text fields keep focus during config changes
|
||||
- **Per-observation raw_hex** (#881, #882) - each observer's hex dump now shows what *that observer* actually received
|
||||
- **Per-observation children in packet groups** (#866, #880) - expanded groups show per-obs data, not cross-observer aggregates
|
||||
- **Full-page obs-switch** (#866, #870) - switching observers updates hex, path, and direction correctly
|
||||
- **Packet detail shows wrong observation** (#849, #851) - clicking a specific observation opens *that* observation
|
||||
- **Byte breakdown hop count** (#844, #846) - derived from `path_len`, not aggregated `_parsedPath`
|
||||
- **Transport-route path_len offset** (#852, #853) - correct offset calculation + CSS variable fix
|
||||
- **Packets/hour chart bars + x-axis** (#858, #865) - bars render correctly, x-axis labels properly decimated
|
||||
- **Channel timeline capped to top 8** (#860, #864) - no more 47-channel chart spaghetti
|
||||
- **Reachability row opacity removed** (#859, #863) - clean rows without misleading gradient
|
||||
- **Sticky table headers on mobile** (#861, #867) - restored after regression
|
||||
- **Map popup 'Show Neighbors' on iOS Safari** (#840, #841) - link actually works now
|
||||
- **Node detail Recent Packets invisible text** (#829, #830) - CSS fix
|
||||
- **/api/packets/{hash} falls back to DB** (#827, #831) - when in-memory store misses, DB catches it
|
||||
- **IATA filter bypass for status messages** (#694, #802) - status packets no longer filtered out by airport codes
|
||||
- **Desktop node click URL hash** (#676, #739) - clicking a node updates the URL for deep linking
|
||||
- **Filter params in URL hash** (#682, #740) - all filter state serialized for shareable links
|
||||
- **Hide undecryptable channel messages** (#727, #728) - clean default view
|
||||
- **TRACE path_json uses path_sz** (#732) - correct field from flags byte, not header hash_size
|
||||
- **Multi-byte adopters** (#754, #767) - all node types, role column, advert precedence
|
||||
- **Channel key case sensitivity** (#761) - Public decode works correctly
|
||||
- **Transport route field offsets** (#766) - correct offsets in field table
|
||||
- **Clock skew sanity checks** (#769) - filter epoch-0, cap drift, require minimum samples
|
||||
- **Neighbor graph slider persistence** (#776) - default 0.7, persisted to localStorage
|
||||
- **Node detail panel navigation** (#779, #785) - Details/Analytics links actually navigate
|
||||
- **Channel key removal** (#898) - user-added keys for server-known channels can be removed
|
||||
- **Side-panel Details on desktop** (#892) - opens full-screen correctly
|
||||
- **Hex-dump byte ranges client-side** (#891) - computed from per-obs raw_hex
|
||||
- **path_json derived from raw_hex at ingest** (#886, #887) - single source of truth
|
||||
- **Path pill and byte breakdown hop agreement** (#885) - they match now
|
||||
- **Mobile close button + toolbar scroll** (#797, #805) - accessible and scrollable
|
||||
- **/health.recentPackets resolved_path fallback** (#810, #821) - falls back to longest sibling observation
|
||||
- **Channel filter on Packets page** (#812, #816) - UI and API both fixed
|
||||
- **Clock-skew section in side panel** (#813, #814) - renders correctly
|
||||
- **Real RSS in /api/stats** (#832, #835) - surface actual RSS alongside tracked store bytes
|
||||
- **Hash size detection for transport routes + zero-hop adverts** (#747) - correct detection
|
||||
- **Repeater+observer merged map marker** (#745) - single marker, not two overlapping
|
||||
|
||||
---
|
||||
|
||||
## 🎨 UI Polish
|
||||
|
||||
- QA findings applied across the board (#832, #833, #836, #837, #838) - dozens of small UX fixes from systematic QA pass
|
||||
|
||||
---
|
||||
|
||||
## 📦 Upgrading
|
||||
|
||||
```bash
|
||||
git pull
|
||||
docker compose down
|
||||
docker compose build prod
|
||||
docker compose up -d prod
|
||||
```
|
||||
|
||||
Your existing `config.json` works as-is. New optional config keys:
|
||||
- `nodeBlacklist` - array of node hashes to hide
|
||||
- `observerRetentionDays` - days before stale observers are pruned
|
||||
- `memoryBudgetMB` - cap on in-memory packet store
|
||||
|
||||
### Verify
|
||||
|
||||
```bash
|
||||
curl -s http://localhost/api/health | jq .version
|
||||
# "3.6.0"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🙏 External Contributors
|
||||
|
||||
- **#735** ([@efiten](https://github.com/efiten)) - Serve geofilter builder from app, link from customizer
|
||||
- **#739** ([@efiten](https://github.com/efiten)) - Desktop node click updates URL hash for deep linking
|
||||
- **#740** ([@efiten](https://github.com/efiten)) - Serialize filter params in URL hash for shareable links
|
||||
- **#742** ([@Joel-Claw](https://github.com/Joel-Claw)) - Add nodeBlacklist config to hide abusive/troll nodes
|
||||
- **#761** ([@copelaje](https://github.com/copelaje)) - Fix channel key case sensitivity for Public decode
|
||||
- **#764** ([@Joel-Claw](https://github.com/Joel-Claw)) - Add observer retention - prune stale observers after configurable days
|
||||
- **#802** ([@efiten](https://github.com/efiten)) - Bypass IATA filter for status messages, fill SNR on duplicate observations
|
||||
- **#803** ([@efiten](https://github.com/efiten)) - Replace raw JSON text search with byNode index for node packet queries
|
||||
- **#805** ([@efiten](https://github.com/efiten)) - Mobile close button accessible + toolbar scrollable
|
||||
- **#900** ([@efiten](https://github.com/efiten)) - App-served geofilter docs page
|
||||
- **#917** ([@efiten](https://github.com/efiten)) - Apply retentionHours cutoff in Load() to prevent OOM on cold start
|
||||
- **#924** ([@efiten](https://github.com/efiten)) - Node filter on live page - show only traffic through a specific node
|
||||
- **#925** ([@efiten](https://github.com/efiten)) - Fix geobuilder longitude wrapping for southern hemisphere polygons
|
||||
- **#927** ([@efiten](https://github.com/efiten)) - Skip customize panel re-render while text field has focus
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ Breaking Changes
|
||||
|
||||
**None.** All API endpoints remain backwards-compatible. New fields are additive only.
|
||||
|
||||
---
|
||||
|
||||
## 📊 By the Numbers
|
||||
|
||||
| Stat | Count |
|
||||
|------|-------|
|
||||
| Commits | 134 |
|
||||
| PRs merged | 105 |
|
||||
| Lines added | 18,480 |
|
||||
| Lines removed | 1,632 |
|
||||
| Files changed | 110 |
|
||||
| Contributors | 4 |
|
||||
|
||||
---
|
||||
|
||||
*Previous release: [v3.5.2](https://github.com/Kpa-clawbot/CoreScope/releases/tag/v3.5.2)*
|
||||
+18
-32
@@ -129,7 +129,23 @@ func main() {
|
||||
tag = source.Broker
|
||||
}
|
||||
|
||||
opts := buildMQTTOpts(source)
|
||||
opts := mqtt.NewClientOptions().
|
||||
AddBroker(source.Broker).
|
||||
SetAutoReconnect(true).
|
||||
SetConnectRetry(true).
|
||||
SetOrderMatters(true)
|
||||
|
||||
if source.Username != "" {
|
||||
opts.SetUsername(source.Username)
|
||||
}
|
||||
if source.Password != "" {
|
||||
opts.SetPassword(source.Password)
|
||||
}
|
||||
if source.RejectUnauthorized != nil && !*source.RejectUnauthorized {
|
||||
opts.SetTLSConfig(&tls.Config{InsecureSkipVerify: true})
|
||||
} else if strings.HasPrefix(source.Broker, "ssl://") {
|
||||
opts.SetTLSConfig(&tls.Config{})
|
||||
}
|
||||
|
||||
opts.SetOnConnectHandler(func(c mqtt.Client) {
|
||||
log.Printf("MQTT [%s] connected to %s", tag, source.Broker)
|
||||
@@ -149,11 +165,7 @@ func main() {
|
||||
})
|
||||
|
||||
opts.SetConnectionLostHandler(func(c mqtt.Client, err error) {
|
||||
log.Printf("MQTT [%s] disconnected from %s: %v", tag, source.Broker, err)
|
||||
})
|
||||
|
||||
opts.SetReconnectingHandler(func(c mqtt.Client, options *mqtt.ClientOptions) {
|
||||
log.Printf("MQTT [%s] reconnecting to %s", tag, source.Broker)
|
||||
log.Printf("MQTT [%s] disconnected: %v", tag, err)
|
||||
})
|
||||
|
||||
// Capture source for closure
|
||||
@@ -194,32 +206,6 @@ func main() {
|
||||
log.Println("Done.")
|
||||
}
|
||||
|
||||
// buildMQTTOpts creates MQTT client options for a source with bounded reconnect
|
||||
// backoff, connect timeout, and TLS/auth configuration.
|
||||
func buildMQTTOpts(source MQTTSource) *mqtt.ClientOptions {
|
||||
opts := mqtt.NewClientOptions().
|
||||
AddBroker(source.Broker).
|
||||
SetAutoReconnect(true).
|
||||
SetConnectRetry(true).
|
||||
SetOrderMatters(true).
|
||||
SetMaxReconnectInterval(30 * time.Second).
|
||||
SetConnectTimeout(10 * time.Second).
|
||||
SetWriteTimeout(10 * time.Second)
|
||||
|
||||
if source.Username != "" {
|
||||
opts.SetUsername(source.Username)
|
||||
}
|
||||
if source.Password != "" {
|
||||
opts.SetPassword(source.Password)
|
||||
}
|
||||
if source.RejectUnauthorized != nil && !*source.RejectUnauthorized {
|
||||
opts.SetTLSConfig(&tls.Config{InsecureSkipVerify: true})
|
||||
} else if strings.HasPrefix(source.Broker, "ssl://") {
|
||||
opts.SetTLSConfig(&tls.Config{})
|
||||
}
|
||||
return opts
|
||||
}
|
||||
|
||||
func handleMessage(store *Store, tag string, source MQTTSource, m mqtt.Message, channelKeys map[string]string, cfg *Config) {
|
||||
defer func() {
|
||||
if r := recover(); r != nil {
|
||||
|
||||
@@ -1,76 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestBuildMQTTOpts_ReconnectSettings(t *testing.T) {
|
||||
source := MQTTSource{
|
||||
Broker: "tcp://localhost:1883",
|
||||
Name: "test",
|
||||
}
|
||||
opts := buildMQTTOpts(source)
|
||||
|
||||
if opts.MaxReconnectInterval != 30*time.Second {
|
||||
t.Errorf("MaxReconnectInterval = %v, want 30s", opts.MaxReconnectInterval)
|
||||
}
|
||||
if opts.ConnectTimeout != 10*time.Second {
|
||||
t.Errorf("ConnectTimeout = %v, want 10s", opts.ConnectTimeout)
|
||||
}
|
||||
if opts.WriteTimeout != 10*time.Second {
|
||||
t.Errorf("WriteTimeout = %v, want 10s", opts.WriteTimeout)
|
||||
}
|
||||
if !opts.AutoReconnect {
|
||||
t.Error("AutoReconnect should be true")
|
||||
}
|
||||
if !opts.ConnectRetry {
|
||||
t.Error("ConnectRetry should be true")
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildMQTTOpts_Credentials(t *testing.T) {
|
||||
source := MQTTSource{
|
||||
Broker: "tcp://broker:1883",
|
||||
Username: "user1",
|
||||
Password: "pass1",
|
||||
}
|
||||
opts := buildMQTTOpts(source)
|
||||
|
||||
if opts.Username != "user1" {
|
||||
t.Errorf("Username = %q, want %q", opts.Username, "user1")
|
||||
}
|
||||
if opts.Password != "pass1" {
|
||||
t.Errorf("Password = %q, want %q", opts.Password, "pass1")
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildMQTTOpts_TLS_InsecureSkipVerify(t *testing.T) {
|
||||
f := false
|
||||
source := MQTTSource{
|
||||
Broker: "ssl://broker:8883",
|
||||
RejectUnauthorized: &f,
|
||||
}
|
||||
opts := buildMQTTOpts(source)
|
||||
|
||||
if opts.TLSConfig == nil {
|
||||
t.Fatal("TLSConfig should be set")
|
||||
}
|
||||
if !opts.TLSConfig.InsecureSkipVerify {
|
||||
t.Error("InsecureSkipVerify should be true when RejectUnauthorized=false")
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildMQTTOpts_TLS_SSL_Prefix(t *testing.T) {
|
||||
source := MQTTSource{
|
||||
Broker: "ssl://broker:8883",
|
||||
}
|
||||
opts := buildMQTTOpts(source)
|
||||
|
||||
if opts.TLSConfig == nil {
|
||||
t.Fatal("TLSConfig should be set for ssl:// brokers")
|
||||
}
|
||||
if opts.TLSConfig.InsecureSkipVerify {
|
||||
t.Error("InsecureSkipVerify should be false by default")
|
||||
}
|
||||
}
|
||||
+165
-255
@@ -12,20 +12,28 @@ import (
|
||||
type SkewSeverity string
|
||||
|
||||
const (
|
||||
SkewOK SkewSeverity = "ok" // < 5 min
|
||||
SkewWarning SkewSeverity = "warning" // 5 min – 1 hour
|
||||
SkewCritical SkewSeverity = "critical" // 1 hour – 30 days
|
||||
SkewAbsurd SkewSeverity = "absurd" // > 30 days
|
||||
SkewNoClock SkewSeverity = "no_clock" // > 365 days — uninitialized RTC
|
||||
SkewBimodalClock SkewSeverity = "bimodal_clock" // mixed good+bad recent samples (flaky RTC)
|
||||
SkewDefault SkewSeverity = "default" // firmware-default epoch + uptime
|
||||
SkewOK SkewSeverity = "ok" // |skew| <= 15s
|
||||
SkewDegrading SkewSeverity = "degrading" // 15s < |skew| <= 60s
|
||||
SkewDegraded SkewSeverity = "degraded" // 60s < |skew| <= 600s
|
||||
SkewWrong SkewSeverity = "wrong" // |skew| > 600s and not default
|
||||
)
|
||||
|
||||
// Known firmware default epochs. Nodes with advert_ts in
|
||||
// [epoch, epoch + maxPlausibleUptimeSec] are classified as "default".
|
||||
// See docs/clock-skew-redesign.md for provenance of each value.
|
||||
var defaultEpochs = []int64{0, 1609459200, 1672531200, 1715770351}
|
||||
|
||||
// Default thresholds in seconds.
|
||||
const (
|
||||
skewThresholdWarnSec = 5 * 60 // 5 minutes
|
||||
skewThresholdCriticalSec = 60 * 60 // 1 hour
|
||||
skewThresholdAbsurdSec = 30 * 24 * 3600 // 30 days
|
||||
skewThresholdNoClockSec = 365 * 24 * 3600 // 365 days — uninitialized RTC
|
||||
// maxPlausibleUptimeSec caps how far past a default epoch we still
|
||||
// consider "default + uptime ticking". 730 days ≈ 2 years.
|
||||
maxPlausibleUptimeSec = 1095 * 86400 // 3 years — covers solar repeater deployment lifetimes at firmware default
|
||||
|
||||
// Severity band boundaries (absolute skew in seconds).
|
||||
skewThresholdOKSec = 15
|
||||
skewThresholdDegradingSec = 60
|
||||
skewThresholdDegradedSec = 600
|
||||
|
||||
// minDriftSamples is the minimum number of advert transmissions needed
|
||||
// to compute a meaningful linear drift rate.
|
||||
@@ -35,54 +43,52 @@ const (
|
||||
// drift rates (> 1 day/day) indicate insufficient or outlier samples.
|
||||
maxReasonableDriftPerDay = 86400.0
|
||||
|
||||
// recentSkewWindowCount is the number of most-recent advert samples
|
||||
// used to derive the "current" skew for severity classification (see
|
||||
// issue #789). The all-time median is poisoned by historical bad
|
||||
// samples (e.g. a node that was off and then GPS-corrected); severity
|
||||
// must reflect current health, not lifetime statistics.
|
||||
recentSkewWindowCount = 5
|
||||
|
||||
// recentSkewWindowSec bounds the recent-window in time as well: only
|
||||
// samples from the last N seconds count as "recent" for severity.
|
||||
// The effective window is min(recentSkewWindowCount, samples in 1h).
|
||||
recentSkewWindowSec = 3600
|
||||
|
||||
// bimodalSkewThresholdSec is the absolute skew threshold (1 hour)
|
||||
// above which a sample is considered "bad" — likely firmware emitting
|
||||
// a nonsense timestamp from an uninitialized RTC, not real drift.
|
||||
// Chosen to match the warning/critical severity boundary: real clock
|
||||
// drift rarely exceeds 1 hour, while epoch-0 RTCs produce ~1.7B sec.
|
||||
bimodalSkewThresholdSec = 3600.0
|
||||
|
||||
// maxPlausibleSkewJumpSec is the largest skew change between
|
||||
// consecutive samples that we treat as physical drift. Anything larger
|
||||
// (e.g. a GPS sync that jumps the clock by minutes/days) is rejected
|
||||
// as an outlier when computing drift. Real microcontroller drift is
|
||||
// fractions of a second per advert; 60s is a generous safety factor.
|
||||
// consecutive samples that we treat as physical drift.
|
||||
maxPlausibleSkewJumpSec = 60.0
|
||||
|
||||
// theilSenMaxPoints caps the number of points fed to Theil-Sen
|
||||
// regression (O(n²) in pairs). For nodes with thousands of samples we
|
||||
// keep the most-recent points, which are also the most relevant for
|
||||
// current drift.
|
||||
// regression (O(n²) in pairs).
|
||||
theilSenMaxPoints = 200
|
||||
)
|
||||
|
||||
// classifySkew maps absolute skew (seconds) to a severity level.
|
||||
// Float64 comparison is safe: inputs are rounded to 1 decimal via round(),
|
||||
// and thresholds are integer multiples of 60 — no rounding artifacts.
|
||||
func classifySkew(absSkewSec float64) SkewSeverity {
|
||||
// isDefaultEpoch returns true if the raw advert timestamp falls within
|
||||
// [epoch, epoch + maxPlausibleUptimeSec] for any known firmware default.
|
||||
// If matched, returns the matched epoch; otherwise returns 0.
|
||||
func isDefaultEpoch(advertTS int64) (bool, int64) {
|
||||
// Find the largest epoch <= advertTS (closest match). Since ranges
|
||||
// overlap, picking the closest avoids attributing a 2023-firmware
|
||||
// node's timestamp to the 2024 epoch.
|
||||
bestEpoch := int64(-1)
|
||||
for _, epoch := range defaultEpochs {
|
||||
if epoch <= advertTS && epoch > bestEpoch {
|
||||
bestEpoch = epoch
|
||||
}
|
||||
}
|
||||
if bestEpoch >= 0 && advertTS <= bestEpoch+maxPlausibleUptimeSec {
|
||||
return true, bestEpoch
|
||||
}
|
||||
return false, 0
|
||||
}
|
||||
|
||||
// classifySkew maps a raw advert timestamp and corrected skew (signed)
|
||||
// to a severity level. Takes math.Abs internally so callers may pass
|
||||
// signed values. Default detection runs on the raw advert_ts
|
||||
// (independent of observer calibration).
|
||||
func classifySkew(advertTS int64, skewSec float64) (SkewSeverity, int64) {
|
||||
if ok, epoch := isDefaultEpoch(advertTS); ok {
|
||||
return SkewDefault, epoch
|
||||
}
|
||||
abs := math.Abs(skewSec)
|
||||
switch {
|
||||
case absSkewSec >= skewThresholdNoClockSec:
|
||||
return SkewNoClock
|
||||
case absSkewSec >= skewThresholdAbsurdSec:
|
||||
return SkewAbsurd
|
||||
case absSkewSec >= skewThresholdCriticalSec:
|
||||
return SkewCritical
|
||||
case absSkewSec >= skewThresholdWarnSec:
|
||||
return SkewWarning
|
||||
case abs <= skewThresholdOKSec:
|
||||
return SkewOK, 0
|
||||
case abs <= skewThresholdDegradingSec:
|
||||
return SkewDegrading, 0
|
||||
case abs <= skewThresholdDegradedSec:
|
||||
return SkewDegraded, 0
|
||||
default:
|
||||
return SkewOK
|
||||
return SkewWrong, 0
|
||||
}
|
||||
}
|
||||
|
||||
@@ -90,38 +96,35 @@ func classifySkew(absSkewSec float64) SkewSeverity {
|
||||
|
||||
// skewSample is a single raw skew measurement from one advert observation.
|
||||
type skewSample struct {
|
||||
advertTS int64 // node's advert Unix timestamp
|
||||
observedTS int64 // observation Unix timestamp
|
||||
observerID string // which observer saw this
|
||||
hash string // transmission hash (for multi-observer grouping)
|
||||
advertTS int64 // node's advert Unix timestamp
|
||||
observedTS int64 // observation Unix timestamp
|
||||
observerID string // which observer saw this
|
||||
hash string // transmission hash (for multi-observer grouping)
|
||||
}
|
||||
|
||||
// ObserverCalibration holds the computed clock offset for an observer.
|
||||
type ObserverCalibration struct {
|
||||
ObserverID string `json:"observerID"`
|
||||
OffsetSec float64 `json:"offsetSec"` // positive = observer clock ahead
|
||||
Samples int `json:"samples"` // number of multi-observer packets used
|
||||
OffsetSec float64 `json:"offsetSec"` // positive = observer clock ahead
|
||||
Samples int `json:"samples"` // number of multi-observer packets used
|
||||
}
|
||||
|
||||
// NodeClockSkew is the API response for a single node's clock skew data.
|
||||
type NodeClockSkew struct {
|
||||
Pubkey string `json:"pubkey"`
|
||||
MeanSkewSec float64 `json:"meanSkewSec"` // corrected mean skew (positive = node ahead)
|
||||
MedianSkewSec float64 `json:"medianSkewSec"` // corrected median skew
|
||||
LastSkewSec float64 `json:"lastSkewSec"` // most recent corrected skew
|
||||
RecentMedianSkewSec float64 `json:"recentMedianSkewSec"` // median across most-recent samples (drives severity, see #789)
|
||||
DriftPerDaySec float64 `json:"driftPerDaySec"` // linear drift rate (sec/day)
|
||||
Severity SkewSeverity `json:"severity"`
|
||||
SampleCount int `json:"sampleCount"`
|
||||
Calibrated bool `json:"calibrated"` // true if observer calibration was applied
|
||||
LastAdvertTS int64 `json:"lastAdvertTS"` // most recent advert timestamp
|
||||
LastObservedTS int64 `json:"lastObservedTS"` // most recent observation timestamp
|
||||
Samples []SkewSample `json:"samples,omitempty"` // time-series for sparklines
|
||||
GoodFraction float64 `json:"goodFraction"` // fraction of recent samples with |skew| <= 1h
|
||||
RecentBadSampleCount int `json:"recentBadSampleCount"` // count of recent samples with |skew| > 1h
|
||||
RecentSampleCount int `json:"recentSampleCount"` // total recent samples in window
|
||||
NodeName string `json:"nodeName,omitempty"` // populated in fleet responses
|
||||
NodeRole string `json:"nodeRole,omitempty"` // populated in fleet responses
|
||||
Pubkey string `json:"pubkey"`
|
||||
MeanSkewSec float64 `json:"meanSkewSec"` // corrected mean skew (positive = node ahead)
|
||||
MedianSkewSec float64 `json:"medianSkewSec"` // corrected median skew
|
||||
LastSkewSec float64 `json:"lastSkewSec"` // most recent corrected skew
|
||||
DriftPerDaySec float64 `json:"driftPerDaySec"` // linear drift rate (sec/day)
|
||||
Severity SkewSeverity `json:"severity"`
|
||||
SampleCount int `json:"sampleCount"`
|
||||
Calibrated bool `json:"calibrated"` // true if observer calibration was applied
|
||||
LastAdvertTS int64 `json:"lastAdvertTS"` // most recent advert timestamp
|
||||
LastObservedTS int64 `json:"lastObservedTS"` // most recent observation timestamp
|
||||
DefaultEpoch *int64 `json:"defaultEpoch,omitempty"` // matched epoch when severity=default
|
||||
Samples []SkewSample `json:"samples,omitempty"` // time-series for sparklines
|
||||
NodeName string `json:"nodeName,omitempty"` // populated in fleet responses
|
||||
NodeRole string `json:"nodeRole,omitempty"` // populated in fleet responses
|
||||
}
|
||||
|
||||
// SkewSample is a single (timestamp, skew) point for sparkline rendering.
|
||||
@@ -130,28 +133,26 @@ type SkewSample struct {
|
||||
SkewSec float64 `json:"skew"` // corrected skew in seconds
|
||||
}
|
||||
|
||||
// txSkewResult maps tx hash → per-transmission skew stats. This is an
|
||||
// intermediate result keyed by hash (not pubkey); the store maps hash → pubkey
|
||||
// when building the final per-node view.
|
||||
// txSkewResult maps tx hash → per-transmission skew stats.
|
||||
type txSkewResult = map[string]*NodeClockSkew
|
||||
|
||||
// ── Clock Skew Engine ──────────────────────────────────────────────────────────
|
||||
|
||||
// ClockSkewEngine computes and caches clock skew data for nodes and observers.
|
||||
type ClockSkewEngine struct {
|
||||
mu sync.RWMutex
|
||||
observerOffsets map[string]float64 // observerID → calibrated offset (seconds)
|
||||
observerSamples map[string]int // observerID → number of multi-observer packets used
|
||||
nodeSkew txSkewResult
|
||||
lastComputed time.Time
|
||||
computeInterval time.Duration
|
||||
mu sync.RWMutex
|
||||
observerOffsets map[string]float64 // observerID → calibrated offset (seconds)
|
||||
observerSamples map[string]int // observerID → number of multi-observer packets used
|
||||
nodeSkew txSkewResult
|
||||
lastComputed time.Time
|
||||
computeInterval time.Duration
|
||||
}
|
||||
|
||||
func NewClockSkewEngine() *ClockSkewEngine {
|
||||
return &ClockSkewEngine{
|
||||
observerOffsets: make(map[string]float64),
|
||||
observerOffsets: make(map[string]float64),
|
||||
observerSamples: make(map[string]int),
|
||||
nodeSkew: make(txSkewResult),
|
||||
nodeSkew: make(txSkewResult),
|
||||
computeInterval: 30 * time.Second,
|
||||
}
|
||||
}
|
||||
@@ -188,7 +189,6 @@ func (e *ClockSkewEngine) Recompute(store *PacketStore) {
|
||||
|
||||
// Swap results under brief write lock.
|
||||
e.mu.Lock()
|
||||
// Re-check: another goroutine may have computed while we were working.
|
||||
if time.Since(e.lastComputed) < e.computeInterval {
|
||||
e.mu.Unlock()
|
||||
return
|
||||
@@ -214,13 +214,13 @@ func collectSamples(store *PacketStore) []skewSample {
|
||||
if decoded == nil {
|
||||
continue
|
||||
}
|
||||
// Extract advert timestamp from decoded JSON.
|
||||
advertTS := extractTimestamp(decoded)
|
||||
if advertTS <= 0 {
|
||||
if advertTS < 0 {
|
||||
continue
|
||||
}
|
||||
// Sanity: skip timestamps before year 2020 or after year 2100.
|
||||
if advertTS < 1577836800 || advertTS > 4102444800 {
|
||||
// Allow epoch 0 and above (needed for default-epoch detection).
|
||||
// Upper bound: year 2100.
|
||||
if advertTS > 4102444800 {
|
||||
continue
|
||||
}
|
||||
|
||||
@@ -240,21 +240,43 @@ func collectSamples(store *PacketStore) []skewSample {
|
||||
return samples
|
||||
}
|
||||
|
||||
// timestampMissing is the sentinel returned by extractTimestamp when no
|
||||
// timestamp field is present in the decoded advert. Using -1 lets us
|
||||
// distinguish "field absent" from a real epoch-0 timestamp (ts == 0).
|
||||
const timestampMissing int64 = -1
|
||||
|
||||
// extractTimestamp gets the Unix timestamp from a decoded ADVERT payload.
|
||||
// Returns timestampMissing (-1) if no timestamp field is found.
|
||||
func extractTimestamp(decoded map[string]interface{}) int64 {
|
||||
// Try payload.timestamp first (nested in "payload" key).
|
||||
if payload, ok := decoded["payload"]; ok {
|
||||
if pm, ok := payload.(map[string]interface{}); ok {
|
||||
if ts := jsonNumber(pm, "timestamp"); ts > 0 {
|
||||
if ts, ok := jsonNumberOk(pm, "timestamp"); ok {
|
||||
return ts
|
||||
}
|
||||
}
|
||||
}
|
||||
// Fallback: top-level timestamp.
|
||||
if ts := jsonNumber(decoded, "timestamp"); ts > 0 {
|
||||
if ts, ok := jsonNumberOk(decoded, "timestamp"); ok {
|
||||
return ts
|
||||
}
|
||||
return 0
|
||||
return timestampMissing
|
||||
}
|
||||
|
||||
// jsonNumberOk extracts an int64 from a JSON-parsed map, returning (value, true)
|
||||
// if the key exists and is numeric, or (0, false) otherwise.
|
||||
func jsonNumberOk(m map[string]interface{}, key string) (int64, bool) {
|
||||
v, ok := m[key]
|
||||
if !ok || v == nil {
|
||||
return 0, false
|
||||
}
|
||||
switch n := v.(type) {
|
||||
case float64:
|
||||
return int64(n), true
|
||||
case int64:
|
||||
return n, true
|
||||
case int:
|
||||
return int64(n), true
|
||||
}
|
||||
return 0, false
|
||||
}
|
||||
|
||||
// jsonNumber extracts an int64 from a JSON-parsed map (handles float64 and json.Number).
|
||||
@@ -281,7 +303,6 @@ func parseISO(s string) int64 {
|
||||
}
|
||||
t, err := time.Parse(time.RFC3339, s)
|
||||
if err != nil {
|
||||
// Try with fractional seconds.
|
||||
t, err = time.Parse("2006-01-02T15:04:05.999999999Z07:00", s)
|
||||
if err != nil {
|
||||
return 0
|
||||
@@ -295,19 +316,16 @@ func parseISO(s string) int64 {
|
||||
// calibrateObservers computes each observer's clock offset using multi-observer
|
||||
// packets. Returns offset map and sample count map.
|
||||
func calibrateObservers(samples []skewSample) (map[string]float64, map[string]int) {
|
||||
// Group observations by packet hash.
|
||||
byHash := make(map[string][]skewSample)
|
||||
for _, s := range samples {
|
||||
byHash[s.hash] = append(byHash[s.hash], s)
|
||||
}
|
||||
|
||||
// For each multi-observer packet, compute per-observer deviation from median.
|
||||
deviations := make(map[string][]float64) // observerID → list of deviations
|
||||
deviations := make(map[string][]float64)
|
||||
for _, group := range byHash {
|
||||
if len(group) < 2 {
|
||||
continue // single-observer packet, can't calibrate
|
||||
continue
|
||||
}
|
||||
// Compute median observation timestamp for this packet.
|
||||
obsTimes := make([]float64, len(group))
|
||||
for i, s := range group {
|
||||
obsTimes[i] = float64(s.observedTS)
|
||||
@@ -319,7 +337,6 @@ func calibrateObservers(samples []skewSample) (map[string]float64, map[string]in
|
||||
}
|
||||
}
|
||||
|
||||
// Each observer's offset = median of its deviations.
|
||||
offsets := make(map[string]float64, len(deviations))
|
||||
counts := make(map[string]int, len(deviations))
|
||||
for obsID, devs := range deviations {
|
||||
@@ -333,8 +350,6 @@ func calibrateObservers(samples []skewSample) (map[string]float64, map[string]in
|
||||
|
||||
// computeNodeSkew calculates corrected skew statistics for each node.
|
||||
func computeNodeSkew(samples []skewSample, obsOffsets map[string]float64) txSkewResult {
|
||||
// Compute corrected skew per sample, grouped by hash (each hash = one
|
||||
// node's advert transmission). The caller maps hash → pubkey via byNode.
|
||||
type correctedSample struct {
|
||||
skew float64
|
||||
observedTS int64
|
||||
@@ -349,8 +364,6 @@ func computeNodeSkew(samples []skewSample, obsOffsets map[string]float64) txSkew
|
||||
rawSkew := float64(s.advertTS - s.observedTS)
|
||||
corrected := rawSkew
|
||||
if hasCal {
|
||||
// Observer offset = obs_ts - median(all_obs_ts). If observer is ahead,
|
||||
// its obs_ts is inflated, making raw_skew too low. Add offset to correct.
|
||||
corrected = rawSkew + obsOffset
|
||||
}
|
||||
byHash[s.hash] = append(byHash[s.hash], correctedSample{
|
||||
@@ -361,10 +374,7 @@ func computeNodeSkew(samples []skewSample, obsOffsets map[string]float64) txSkew
|
||||
hashAdvertTS[s.hash] = s.advertTS
|
||||
}
|
||||
|
||||
// Each hash represents one advert from one node. Compute median corrected
|
||||
// skew per hash (across multiple observers).
|
||||
|
||||
result := make(map[string]*NodeClockSkew) // keyed by hash for now
|
||||
result := make(map[string]*NodeClockSkew)
|
||||
for hash, cs := range byHash {
|
||||
skews := make([]float64, len(cs))
|
||||
for i, c := range cs {
|
||||
@@ -373,29 +383,37 @@ func computeNodeSkew(samples []skewSample, obsOffsets map[string]float64) txSkew
|
||||
medSkew := median(skews)
|
||||
meanSkew := mean(skews)
|
||||
|
||||
// Find latest observation.
|
||||
var latestObsTS int64
|
||||
// Pick the skew from the most recent observation (max observedTS),
|
||||
// not the last-appended sample which may be non-chronological.
|
||||
var latest correctedSample
|
||||
var anyCal bool
|
||||
for _, c := range cs {
|
||||
if c.observedTS > latestObsTS {
|
||||
latestObsTS = c.observedTS
|
||||
if c.observedTS > latest.observedTS {
|
||||
latest = c
|
||||
}
|
||||
if c.calibrated {
|
||||
anyCal = true
|
||||
}
|
||||
}
|
||||
lastCorrectedSkew := latest.skew
|
||||
advTS := hashAdvertTS[hash]
|
||||
severity, matchedEpoch := classifySkew(advTS, lastCorrectedSkew)
|
||||
|
||||
absMedian := math.Abs(medSkew)
|
||||
result[hash] = &NodeClockSkew{
|
||||
ncs := &NodeClockSkew{
|
||||
MeanSkewSec: round(meanSkew, 1),
|
||||
MedianSkewSec: round(medSkew, 1),
|
||||
LastSkewSec: round(cs[len(cs)-1].skew, 1),
|
||||
Severity: classifySkew(absMedian),
|
||||
LastSkewSec: round(lastCorrectedSkew, 1),
|
||||
Severity: severity,
|
||||
SampleCount: len(cs),
|
||||
Calibrated: anyCal,
|
||||
LastAdvertTS: hashAdvertTS[hash],
|
||||
LastObservedTS: latestObsTS,
|
||||
LastAdvertTS: advTS,
|
||||
LastObservedTS: latest.observedTS,
|
||||
}
|
||||
if severity == SkewDefault {
|
||||
ep := matchedEpoch
|
||||
ncs.DefaultEpoch = &ep
|
||||
}
|
||||
result[hash] = ncs
|
||||
}
|
||||
return result
|
||||
}
|
||||
@@ -457,124 +475,45 @@ func (s *PacketStore) getNodeClockSkewLocked(pubkey string) *NodeClockSkew {
|
||||
medSkew := median(allSkews)
|
||||
meanSkew := mean(allSkews)
|
||||
|
||||
// Severity is derived from RECENT samples only (issue #789). The
|
||||
// all-time median is poisoned by historical bad data — a node that
|
||||
// was off for hours and then GPS-corrected can have median = -59M sec
|
||||
// while its current skew is -0.8s. Operators need severity to reflect
|
||||
// current health, so they trust the dashboard.
|
||||
//
|
||||
// Sort tsSkews by time and take the last recentSkewWindowCount samples
|
||||
// (or all samples within recentSkewWindowSec of the latest, whichever
|
||||
// gives FEWER samples — we want the more-current view; a chatty node
|
||||
// can fit dozens of samples in 1h, in which case the count cap wins).
|
||||
sort.Slice(tsSkews, func(i, j int) bool { return tsSkews[i].ts < tsSkews[j].ts })
|
||||
// Classify using the most recent advert's raw timestamp and
|
||||
// the most recent corrected skew. No windowing or median-driven
|
||||
// severity — per-advert classification per the spec.
|
||||
severity, matchedEpoch := classifySkew(lastAdvTS, lastSkew)
|
||||
|
||||
recentSkew := lastSkew
|
||||
var recentVals []float64
|
||||
if n := len(tsSkews); n > 0 {
|
||||
latestTS := tsSkews[n-1].ts
|
||||
// Index-based window: last K samples.
|
||||
startByCount := n - recentSkewWindowCount
|
||||
if startByCount < 0 {
|
||||
startByCount = 0
|
||||
}
|
||||
// Time-based window: samples newer than latestTS - windowSec.
|
||||
startByTime := n - 1
|
||||
for i := n - 1; i >= 0; i-- {
|
||||
if latestTS-tsSkews[i].ts <= recentSkewWindowSec {
|
||||
startByTime = i
|
||||
} else {
|
||||
break
|
||||
}
|
||||
}
|
||||
// Pick the narrower (larger-index) of the two windows — the most
|
||||
// current view of the node's clock health.
|
||||
start := startByCount
|
||||
if startByTime > start {
|
||||
start = startByTime
|
||||
}
|
||||
recentVals = make([]float64, 0, n-start)
|
||||
for i := start; i < n; i++ {
|
||||
recentVals = append(recentVals, tsSkews[i].skew)
|
||||
}
|
||||
if len(recentVals) > 0 {
|
||||
recentSkew = median(recentVals)
|
||||
}
|
||||
}
|
||||
|
||||
// ── Bimodal detection (#845) ─────────────────────────────────────────
|
||||
// Split recent samples into "good" (|skew| <= 1h, real clock) and
|
||||
// "bad" (|skew| > 1h, firmware nonsense from uninitialized RTC).
|
||||
// Classification order (first match wins):
|
||||
// no_clock — goodFraction < 0.10 (essentially no real clock)
|
||||
// bimodal_clock — 0.10 <= goodFraction < 0.80 AND badCount > 0
|
||||
// ok/warn/etc. — goodFraction >= 0.80 (normal, outliers filtered)
|
||||
var goodSamples []float64
|
||||
for _, v := range recentVals {
|
||||
if math.Abs(v) <= bimodalSkewThresholdSec {
|
||||
goodSamples = append(goodSamples, v)
|
||||
}
|
||||
}
|
||||
recentSampleCount := len(recentVals)
|
||||
recentBadCount := recentSampleCount - len(goodSamples)
|
||||
var goodFraction float64
|
||||
if recentSampleCount > 0 {
|
||||
goodFraction = float64(len(goodSamples)) / float64(recentSampleCount)
|
||||
}
|
||||
|
||||
var severity SkewSeverity
|
||||
if goodFraction < 0.10 {
|
||||
// Essentially no real clock — classify as no_clock regardless
|
||||
// of the raw skew magnitude.
|
||||
severity = SkewNoClock
|
||||
} else if goodFraction < 0.80 && recentBadCount > 0 {
|
||||
// Bimodal: use median of GOOD samples as the "real" skew.
|
||||
severity = SkewBimodalClock
|
||||
if len(goodSamples) > 0 {
|
||||
recentSkew = median(goodSamples)
|
||||
}
|
||||
} else {
|
||||
// Normal path: if there are good samples, use their median
|
||||
// (filters out rare outliers in ≥80% good case).
|
||||
if len(goodSamples) > 0 && recentBadCount > 0 {
|
||||
recentSkew = median(goodSamples)
|
||||
}
|
||||
severity = classifySkew(math.Abs(recentSkew))
|
||||
}
|
||||
|
||||
// For no_clock / bimodal_clock nodes, skip drift when data is unreliable.
|
||||
// Drift: display only, not a classifier input.
|
||||
var drift float64
|
||||
if severity != SkewNoClock && severity != SkewBimodalClock && len(tsSkews) >= minDriftSamples {
|
||||
if severity != SkewDefault && len(tsSkews) >= minDriftSamples {
|
||||
drift = computeDrift(tsSkews)
|
||||
// Cap physically impossible drift rates.
|
||||
if math.Abs(drift) > maxReasonableDriftPerDay {
|
||||
drift = 0
|
||||
}
|
||||
}
|
||||
|
||||
// Build sparkline samples from tsSkews (already sorted by time above).
|
||||
// Build sparkline samples.
|
||||
sort.Slice(tsSkews, func(i, j int) bool { return tsSkews[i].ts < tsSkews[j].ts })
|
||||
samples := make([]SkewSample, len(tsSkews))
|
||||
for i, p := range tsSkews {
|
||||
samples[i] = SkewSample{Timestamp: p.ts, SkewSec: round(p.skew, 1)}
|
||||
}
|
||||
|
||||
return &NodeClockSkew{
|
||||
Pubkey: pubkey,
|
||||
MeanSkewSec: round(meanSkew, 1),
|
||||
MedianSkewSec: round(medSkew, 1),
|
||||
LastSkewSec: round(lastSkew, 1),
|
||||
RecentMedianSkewSec: round(recentSkew, 1),
|
||||
DriftPerDaySec: round(drift, 2),
|
||||
Severity: severity,
|
||||
SampleCount: totalSamples,
|
||||
Calibrated: anyCal,
|
||||
LastAdvertTS: lastAdvTS,
|
||||
LastObservedTS: lastObsTS,
|
||||
Samples: samples,
|
||||
GoodFraction: round(goodFraction, 2),
|
||||
RecentBadSampleCount: recentBadCount,
|
||||
RecentSampleCount: recentSampleCount,
|
||||
result := &NodeClockSkew{
|
||||
Pubkey: pubkey,
|
||||
MeanSkewSec: round(meanSkew, 1),
|
||||
MedianSkewSec: round(medSkew, 1),
|
||||
LastSkewSec: round(lastSkew, 1),
|
||||
DriftPerDaySec: round(drift, 2),
|
||||
Severity: severity,
|
||||
SampleCount: totalSamples,
|
||||
Calibrated: anyCal,
|
||||
LastAdvertTS: lastAdvTS,
|
||||
LastObservedTS: lastObsTS,
|
||||
Samples: samples,
|
||||
}
|
||||
if severity == SkewDefault {
|
||||
ep := matchedEpoch
|
||||
result.DefaultEpoch = &ep
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
// GetFleetClockSkew returns clock skew data for all nodes that have skew data.
|
||||
@@ -583,7 +522,6 @@ func (s *PacketStore) GetFleetClockSkew() []*NodeClockSkew {
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
|
||||
// Build name/role lookup from DB cache (requires s.mu held).
|
||||
allNodes, _ := s.getCachedNodesAndPM()
|
||||
nameMap := make(map[string]nodeInfo, len(allNodes))
|
||||
for _, ni := range allNodes {
|
||||
@@ -596,12 +534,10 @@ func (s *PacketStore) GetFleetClockSkew() []*NodeClockSkew {
|
||||
if cs == nil {
|
||||
continue
|
||||
}
|
||||
// Enrich with node name/role.
|
||||
if ni, ok := nameMap[pubkey]; ok {
|
||||
cs.NodeName = ni.Name
|
||||
cs.NodeRole = ni.Role
|
||||
}
|
||||
// Omit samples in fleet response (too much data).
|
||||
cs.Samples = nil
|
||||
results = append(results, cs)
|
||||
}
|
||||
@@ -626,7 +562,6 @@ func (s *PacketStore) GetObserverCalibrations() []ObserverCalibration {
|
||||
Samples: s.clockSkew.observerSamples[obsID],
|
||||
})
|
||||
}
|
||||
// Sort by absolute offset descending.
|
||||
sort.Slice(result, func(i, j int) bool {
|
||||
return math.Abs(result[i].OffsetSec) > math.Abs(result[j].OffsetSec)
|
||||
})
|
||||
@@ -667,38 +602,20 @@ type tsSkewPair struct {
|
||||
}
|
||||
|
||||
// computeDrift estimates linear drift in seconds per day from time-ordered
|
||||
// (timestamp, skew) pairs. Issue #789: a single GPS-correction event (huge
|
||||
// skew jump in seconds) used to dominate ordinary least squares and produce
|
||||
// absurd drift like 1.7M sec/day. We now:
|
||||
//
|
||||
// 1. Drop pairs whose consecutive skew jump exceeds maxPlausibleSkewJumpSec
|
||||
// (clock corrections, not physical drift). This protects both OLS-style
|
||||
// consumers and Theil-Sen.
|
||||
// 2. Use Theil-Sen regression — the slope is the median of all pairwise
|
||||
// slopes, naturally robust to remaining outliers (breakdown point ~29%).
|
||||
//
|
||||
// For very small samples after filtering we fall back to a simple slope
|
||||
// between first and last calibrated samples.
|
||||
// (timestamp, skew) pairs using Theil-Sen regression with outlier filtering.
|
||||
func computeDrift(pairs []tsSkewPair) float64 {
|
||||
if len(pairs) < 2 {
|
||||
return 0
|
||||
}
|
||||
// Sort by timestamp.
|
||||
sort.Slice(pairs, func(i, j int) bool {
|
||||
return pairs[i].ts < pairs[j].ts
|
||||
})
|
||||
|
||||
// Time span too short? Skip.
|
||||
spanSec := float64(pairs[len(pairs)-1].ts - pairs[0].ts)
|
||||
if spanSec < 3600 { // need at least 1 hour of data
|
||||
if spanSec < 3600 {
|
||||
return 0
|
||||
}
|
||||
|
||||
// Outlier filter: drop samples where the skew jumps more than
|
||||
// maxPlausibleSkewJumpSec from the running "stable" baseline.
|
||||
// We anchor on the first sample, then accept each subsequent point
|
||||
// that's within the threshold of the most recent accepted point —
|
||||
// this preserves a slow drift while rejecting correction events.
|
||||
filtered := make([]tsSkewPair, 0, len(pairs))
|
||||
filtered = append(filtered, pairs[0])
|
||||
for i := 1; i < len(pairs); i++ {
|
||||
@@ -707,30 +624,23 @@ func computeDrift(pairs []tsSkewPair) float64 {
|
||||
filtered = append(filtered, pairs[i])
|
||||
}
|
||||
}
|
||||
// If the filter killed too much (e.g. unstable node), fall back to the
|
||||
// raw series so we at least produce *something* — it'll be capped by
|
||||
// maxReasonableDriftPerDay downstream.
|
||||
if len(filtered) < 2 || float64(filtered[len(filtered)-1].ts-filtered[0].ts) < 3600 {
|
||||
filtered = pairs
|
||||
}
|
||||
|
||||
// Cap point count for Theil-Sen (O(n²) on pairs). Keep most-recent.
|
||||
if len(filtered) > theilSenMaxPoints {
|
||||
filtered = filtered[len(filtered)-theilSenMaxPoints:]
|
||||
}
|
||||
|
||||
return theilSenSlope(filtered) * 86400 // sec/sec → sec/day
|
||||
return theilSenSlope(filtered) * 86400
|
||||
}
|
||||
|
||||
// theilSenSlope returns the Theil-Sen estimator: median of all pairwise
|
||||
// slopes (yj - yi) / (tj - ti) for i < j. Naturally robust to outliers.
|
||||
// Pairs must be sorted by timestamp ascending.
|
||||
// theilSenSlope returns the Theil-Sen estimator: median of all pairwise slopes.
|
||||
func theilSenSlope(pairs []tsSkewPair) float64 {
|
||||
n := len(pairs)
|
||||
if n < 2 {
|
||||
return 0
|
||||
}
|
||||
// Pre-allocate: n*(n-1)/2 pairs.
|
||||
slopes := make([]float64, 0, n*(n-1)/2)
|
||||
for i := 0; i < n; i++ {
|
||||
for j := i + 1; j < n; j++ {
|
||||
|
||||
+318
-600
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,241 @@
|
||||
# Clock Skew Classifier — Redesign
|
||||
|
||||
**Status:** spec, pre-implementation
|
||||
**Supersedes:** parts of #690 / #789 / #845 / PR #894
|
||||
**Date drafted:** 2026-04-24
|
||||
|
||||
## Problem
|
||||
|
||||
The current classifier (`cmd/server/clock_skew.go`) uses windowed medians, hysteresis, "good fraction" floors, and a 365-day `no_clock` threshold. It produces:
|
||||
|
||||
- False `no_clock` flags on nodes whose clocks are working today but had garbage timestamps in recent samples.
|
||||
- Symmetric severity bands that conflate "clock at firmware default" with "operator set the clock wrong by a year" — completely different operator actions required.
|
||||
- Compounding over-engineering as each operator complaint added a new tier or window.
|
||||
|
||||
The actual physical reality of these devices is much simpler than the classifier assumes.
|
||||
|
||||
## Hardware reality
|
||||
|
||||
Most MeshCore nodes have **no auto-updating RTC**. There are two hardware paths:
|
||||
|
||||
1. **Volatile RTC nodes** (`firmware/src/helpers/ArduinoHelpers.h:11` — `VolatileRTCClock`):
|
||||
- On boot, `base_time` is hardcoded to a firmware-build constant (currently `1715770351` = 2024-05-15 20:52:31 UTC).
|
||||
- `getCurrentTime()` returns `base_time + millis()/1000`.
|
||||
- On reboot the value snaps back to the constant.
|
||||
- User must manually sync via companion app (`set time` CLI invokes `setCurrentTime(...)`) to set a real wall-clock time, which then ticks until the next reboot.
|
||||
|
||||
2. **Hardware-RTC nodes** (`firmware/src/helpers/AutoDiscoverRTCClock.cpp` — DS3231 / RV3028 / PCF8563):
|
||||
- Real-time chip with battery backup. Holds the time across reboots.
|
||||
- Behaves correctly once set; no default-snap behavior.
|
||||
|
||||
The `set time RESET` CLI command (`firmware/src/helpers/CommonCLI.cpp:215`) explicitly calls `setCurrentTime(1715770351)` regardless of hardware — so even hardware-RTC nodes can be deliberately reset to the default epoch.
|
||||
|
||||
**Therefore every node is in exactly one of these states:**
|
||||
|
||||
| State | Description |
|
||||
|---|---|
|
||||
| **Default / never set** | RTC is at a firmware-default epoch + ticking up since the last boot. |
|
||||
| **Set, drifting normally** | RTC was synced; small skew accumulating at ~0.8s/day per #789 reports. |
|
||||
| **Set, drifted past tolerance** | Like above but skew has grown beyond what's useful. |
|
||||
| **Wrong** | Operator-set incorrect time, or genuine RTC malfunction not matching any known default. |
|
||||
|
||||
There is no "bimodal RTC bug" — what looked bimodal in #845 is just a sequence of `defaulted → user sync → reboot → defaulted again`. The "bad" timestamps are not noise; they're a constant (the default epoch + a small uptime).
|
||||
|
||||
## Production data analysis (2026-04-24)
|
||||
|
||||
### 00id.net (this deployment, 416 nodes, commit `abd9c46`)
|
||||
|
||||
`lastSkewSec` (advert_ts − observed_ts) distribution:
|
||||
|
||||
| Bucket | Count | Pct |
|
||||
|---|---:|---:|
|
||||
| OK ≤15s | 90 | 22% |
|
||||
| Degrading ≤60s | 93 | 22% |
|
||||
| Degraded ≤10m | 13 | 3% |
|
||||
| off ≤1d | 5 | 1% |
|
||||
| off ≤1y | 110 | 26% |
|
||||
| absurd >1y | 105 | 25% |
|
||||
|
||||
Per-node `lastAdvertTS` raw timestamp distribution shows a sharp default cluster:
|
||||
|
||||
```
|
||||
+0 days count=19 samples=114969 ← exactly at 1715770351 (just rebooted)
|
||||
+1d count=9 samples=24766
|
||||
+2d count=7 samples=58101
|
||||
+3d count=2 samples=360
|
||||
... ← decay through ~110 days
|
||||
+113d count=2 samples=53776
|
||||
```
|
||||
|
||||
103 of 416 nodes (25%) have `lastAdvertTS` between `1715770351` and `1715770351 + 1095 days`, consistent with the volatile-RTC-default-ticking-up pattern.
|
||||
|
||||
A second cluster of 5 nodes has `lastAdvertTS = 1672531542 ≈ 1672531200 + 5min` = **2023-01-01 00:00:00 UTC** + small uptime. This is a *different* firmware-default epoch from an older firmware version.
|
||||
|
||||
### Cascadia (analyzer.cascadiamesh.org, 433 nodes in 5000-packet sample, commit `111b03c` v3.5.0)
|
||||
|
||||
ADVERT timestamp by year-month:
|
||||
|
||||
```
|
||||
1970-01 1 ← epoch zero (ESP32 native fallback OR ancient firmware)
|
||||
2021-01 1 ← possible third default epoch
|
||||
2023-01 2 ← old firmware default (matches 00id)
|
||||
2024-05 60 ← current VolatileRTCClock + days uptime
|
||||
2024-06 39 ← same default + weeks uptime
|
||||
2024-07 21
|
||||
2024-08 10
|
||||
2024-09 2
|
||||
2024-10 1
|
||||
2024-11 2 ← decays out as fewer nodes have multi-month uptime since reboot
|
||||
2025-10 1 ← pre-current-now miscellany
|
||||
2025-11 2
|
||||
2026-03 4
|
||||
2026-04 285 ← currently set clocks (this is "now-ish")
|
||||
2027-04 1 ← operator set wrong by ~1 year (typo?)
|
||||
2067-12 1 ← operator set wildly wrong / corrupted RTC
|
||||
```
|
||||
|
||||
Confirms the model: ~67% of nodes have a current clock, ~32% are at known firmware defaults at varying uptime offsets, ~3 outliers represent genuine misconfigurations.
|
||||
|
||||
## Known firmware default epochs
|
||||
|
||||
These are the values discovered in production data so far:
|
||||
|
||||
| Epoch (unix) | UTC | Source |
|
||||
|---:|---|---|
|
||||
| `0` | 1970-01-01 | Likely ESP32 boot when no RTC initialization runs (`time(NULL)` returns 0). |
|
||||
| `1609459200` | 2021-01-01 | Speculation — single-sample evidence, validate as more data arrives. |
|
||||
| `1672531200` | 2023-01-01 | Older firmware `VolatileRTCClock::base_time` value. |
|
||||
| `1715770351` | 2024-05-15 20:52:31 | **Current** `VolatileRTCClock` constructor + `set time RESET` CLI. |
|
||||
|
||||
Treat the table as data, not fixed code. New firmware versions will introduce new defaults; expect to add to the list over time.
|
||||
|
||||
## Reconciliation with #690 — the four timestamps
|
||||
|
||||
#690 lists three timestamps; in practice there are four signals worth distinguishing:
|
||||
|
||||
| Signal | Source | Used for |
|
||||
|---|---|---|
|
||||
| `advert_ts` | Inside MeshCore packet, set by sending node | Per-node classification (THE signal). |
|
||||
| `mqtt_envelope_ts` | Set by observer when it forwards via MQTT | Observer-side calibration only — *not* a direct node-skew signal because observer clock can itself be wrong. |
|
||||
| `corescope_received_ts` | Wall clock when CoreScope ingested the message | Reference "now"; calibration cross-check. |
|
||||
| `same_packet_across_observers` | Multiple observers seeing the same hash | Phase 2 calibration (triangulation). |
|
||||
|
||||
**Inputs flow:**
|
||||
|
||||
1. **Phase 2 (existing, kept):** for each packet hash seen by ≥2 observers, compute each observer's deviation from the per-packet median observed_ts → `observerOffset`. This is the triangulation #690 calls for ("Same packet observed by more than one (ideally 3+) observers gives good indication if one observer is off"). Observer offsets are the calibration table.
|
||||
2. **Per-advert correction (existing, kept):** `correctedSkew = (advert_ts - observed_ts) + observerOffset[observer_id]`. If no calibration exists for an observer, fall back to raw skew with `calibrated: false`.
|
||||
3. **Default detection (new):** runs on RAW `advert_ts`, not corrected. The firmware default is a fixed wall-clock value; observer offsets are seconds-to-minutes scale and cannot move `advert_ts` from 2024 to 2026. Default check is independent of calibration.
|
||||
4. **Severity classification (new):** if `is_default(advert_ts)` → `default`; else classify by `|correctedSkew|` band.
|
||||
|
||||
This keeps everything #690 asks for (observer detection, bias subtraction, triangulation), and adds the firmware-default cluster as a new pre-empting tier.
|
||||
|
||||
## UI: explain WHY (#690 requirement)
|
||||
|
||||
The classifier alone doesn't satisfy #690's "present on the UI why clock skew is obvious or suspected." The evidence panel from PR #906 (per-hash observer breakdown showing raw vs corrected skew per observer) is the WHY.
|
||||
|
||||
For each per-node clock card the UI must show:
|
||||
|
||||
- **Tier badge** (default / ok / degrading / degraded / wrong) + magnitude.
|
||||
- **Plain-English reason line**: e.g. "Last advert at 2024-05-15 + 3.2 days uptime — matches firmware default (volatile RTC, not yet user-set)" or "Last advert −12s vs wall clock — within OK tolerance."
|
||||
- **Calibration footnote**: "Skew corrected using observer X offset +1.7s (computed from 412 multi-observer packets)" or "Single-observer measurement, no calibration available."
|
||||
- **Evidence accordion** (PR #906 shape, retained): for the most recent N hashes, each observer's raw vs corrected skew + the observer's offset.
|
||||
|
||||
For the per-observer page (also from PR #906): show the observer's offset, the multi-observer sample count, and a tier badge using the same scale (treating `|observerOffset|` as the skew).
|
||||
|
||||
## Proposed classifier
|
||||
|
||||
Per-advert classification, no windowing:
|
||||
|
||||
```python
|
||||
DEFAULT_EPOCHS = [0, 1609459200, 1672531200, 1715770351]
|
||||
MAX_PLAUSIBLE_UPTIME_SEC = 1095 * 86400 # 3 years
|
||||
|
||||
def is_default(ts):
|
||||
return any(d <= ts <= d + MAX_PLAUSIBLE_UPTIME_SEC for d in DEFAULT_EPOCHS)
|
||||
|
||||
def classify(advert_ts, corrected_skew_sec):
|
||||
if is_default(advert_ts):
|
||||
return "default" # gray
|
||||
abs_skew = abs(corrected_skew_sec)
|
||||
if abs_skew <= 15: return "ok" # green
|
||||
if abs_skew <= 60: return "degrading" # yellow
|
||||
if abs_skew <= 600: return "degraded" # orange
|
||||
return "wrong" # red
|
||||
```
|
||||
|
||||
`corrected_skew_sec` is the observer-bias-subtracted skew per Phase 2 calibration. Default detection is independent of calibration (runs on raw `advert_ts`).
|
||||
|
||||
Per-node state = classification of the node's most-recent advert (per hash, picking the most recent observation across all observers). No medians, no good-fraction, no hysteresis.
|
||||
|
||||
## Severity tier definitions
|
||||
|
||||
| Tier | Condition | Color | UI label | Meaning |
|
||||
|---|---|---|---|---|
|
||||
| `default` | Advert ts within `[default, default + 3y]` of any known epoch | Gray | "Default" | Volatile RTC at firmware boot constant; never set or rebooted and not re-synced. |
|
||||
| `ok` | abs(skew) ≤ 15s | Green | "OK" | Working clock. |
|
||||
| `degrading` | 15s < abs(skew) ≤ 60s | Yellow | "Degrading" | Real but accumulating drift. |
|
||||
| `degraded` | 60s < abs(skew) ≤ 600s | Orange | "Degraded" | Off by minutes — needs re-sync. |
|
||||
| `wrong` | abs(skew) > 600s and not `default` | Red | "Wrong" | Operator-set error or RTC malfunction. |
|
||||
|
||||
## What this kills
|
||||
|
||||
- The 365-day `no_clock` threshold and the entire `recentSkewWindow{Count,Sec}` machinery.
|
||||
- The hysteresis / `goodFraction` / `longTermGoodFraction` logic from PR #894.
|
||||
- The proposed `bimodal_clock` tier from #845 — the pattern is not bimodal, it's defaulted vs set.
|
||||
- All Theil-Sen drift calculations as classifier inputs (drift remains a derived display value).
|
||||
|
||||
## What this preserves
|
||||
|
||||
- **Phase 2 observer calibration** (`calibrateObservers()`) — kept verbatim. It's what powers the "subtract observer bias" requirement from #690 and provides the triangulation evidence the UI needs.
|
||||
- **Drift display** (computed but not classifying).
|
||||
- **PR #906 evidence UI** — orthogonal to the classifier; it is in fact the implementation of #690's "explain WHY" requirement. Only label strings change to match the new tier names.
|
||||
- **`/api/observers/clock-skew`** — unchanged shape.
|
||||
|
||||
## API impact
|
||||
|
||||
`/api/nodes/{pubkey}/clock-skew` response changes:
|
||||
|
||||
- `severity` enum: `default | ok | degrading | degraded | wrong` (no more `no_clock | severe | warn | absurd`).
|
||||
- New field `defaultEpoch` (int, optional): if `severity == "default"`, the matched epoch.
|
||||
- Drop fields: `recentMedianSkewSec`, `goodFraction`, `recentBadSampleCount`, `longTermGoodFraction`.
|
||||
- Keep: `lastSkewSec`, `medianSkewSec`, `meanSkewSec`, `driftPerDaySec`, `sampleCount`, `calibrated`, `lastAdvertTS`, `lastObservedTS`, `nodeName`, `nodeRole`.
|
||||
|
||||
`/api/nodes/clock-skew` (fleet) shape unchanged except severity enum values.
|
||||
|
||||
## UI impact
|
||||
|
||||
- New CSS classes `skew-badge--default`, `skew-badge--degrading`, `skew-badge--degraded`, `skew-badge--wrong`. Drop `--no_clock`, `--severe`, `--warn`, `--absurd`, `--bimodal_clock`.
|
||||
- Tooltip text updated per tier.
|
||||
- "Default" badge tooltip should explain the clock is at firmware default plus uptime since boot, and the operator hasn't set it yet (or hasn't re-set it since the last reboot).
|
||||
|
||||
## Migration
|
||||
|
||||
Single PR replaces the classifier in `clock_skew.go` and updates the frontend badges/labels. No database schema change, no data migration — all per-call computation.
|
||||
|
||||
## Open issues to close
|
||||
|
||||
- **#789** (median hides corrected clocks) — resolved by per-advert classification.
|
||||
- **#845** (bimodal_clock tier) — replaced by `default` tier; the pattern that motivated it is correctly captured.
|
||||
- **PR #894** — close without merging; this design supersedes Option C entirely.
|
||||
- **#690** UI completion (PR #906) — keeps moving in parallel; only label updates needed.
|
||||
|
||||
## Validation plan
|
||||
|
||||
1. Hand-run the classifier against a snapshot of `/api/nodes/clock-skew` from 00id and cascadia. Confirm:
|
||||
- All 103 00id "absurd" nodes reclassify as `default`.
|
||||
- All 5 cascadia 2023-01 nodes reclassify as `default`.
|
||||
- The 2027 / 2067 cascadia outliers reclassify as `wrong`.
|
||||
- The 285 cascadia 2026-04 nodes reclassify as `ok` (or `degrading` if drift exceeds 15s).
|
||||
2. Add per-tier unit tests in `cmd/server/clock_skew_test.go`.
|
||||
3. Add a regression test for each known default epoch (synthesize advert at `default + 0s`, `default + 1d`, `default + 3y - 1s` → all classify as `default`).
|
||||
4. Edge cases:
|
||||
- `advert_ts == 0` → matches default epoch 0.
|
||||
- `advert_ts == 1715770351 + 731 days` → no longer matches (uptime cap exceeded) — should fall through to time-based classification, likely `wrong`.
|
||||
- Future timestamps beyond `now + 600s` → `wrong`.
|
||||
|
||||
## Out of scope (follow-ups)
|
||||
|
||||
- Per-firmware-version known-default lookup (when `firmware_version` field becomes reliable on adverts).
|
||||
- Reboot-count / flakiness indicator ("this node has hit default N times in last 30d").
|
||||
- Auto-discovery of new default epochs from clustering analysis (could detect a 4th default emerging in the wild).
|
||||
- Filtering defaulted-clock adverts out of time-windowed analytics queries (separate spec — affects path attribution).
|
||||
@@ -1,204 +0,0 @@
|
||||
# Scope Stats Page — Design Spec
|
||||
|
||||
**Issue**: Kpa-clawbot/CoreScope#899
|
||||
**Date**: 2026-04-23
|
||||
**Branch target**: `master`
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Add a dedicated **Scopes** page showing scope/region statistics for MeshCore transport-route packets. Scope filtering in MeshCore uses `TRANSPORT_FLOOD` (route_type 0) and `TRANSPORT_DIRECT` (route_type 3) packets that carry two 16-bit transport codes. Code1 ≠ `0000` means the packet is region-scoped.
|
||||
|
||||
Feature 3 from the issue (default scope per client via advert) is **not implemented** — the advert format has no scope field in the current firmware.
|
||||
|
||||
---
|
||||
|
||||
## How Scopes Work (Firmware)
|
||||
|
||||
Transport code derivation (authoritative source: `meshcore-dev/MeshCore`):
|
||||
|
||||
```
|
||||
key = SHA256("#regionname")[:16] // TransportKeyStore::getAutoKeyFor
|
||||
Code1 = HMAC-SHA256(key, type || payload) // TransportKey::calcTransportCode, 2-byte output
|
||||
```
|
||||
|
||||
Code1 is a **per-message** HMAC — the same region produces a different Code1 for every message. Identifying a region from Code1 requires knowing the region name in advance and recomputing the HMAC.
|
||||
|
||||
`Code1 = 0000` is the "no scope" sentinel (also `FFFF` is reserved). Packets with route_type 1 or 2 (plain FLOOD/DIRECT) carry no transport codes.
|
||||
|
||||
---
|
||||
|
||||
## Config
|
||||
|
||||
Add `hashRegions` to the ingestor `Config` struct in `cmd/ingestor/config.go`, mirroring `hashChannels`:
|
||||
|
||||
```json
|
||||
"hashRegions": ["#belgium", "#eu", "#brussels"]
|
||||
```
|
||||
|
||||
Normalization (same rules as `hashChannels`):
|
||||
- Trim whitespace
|
||||
- Prepend `#` if missing
|
||||
- Skip empty entries
|
||||
|
||||
---
|
||||
|
||||
## Ingestor Changes
|
||||
|
||||
### Key derivation (`loadRegionKeys`)
|
||||
|
||||
```go
|
||||
func loadRegionKeys(cfg *Config) map[string][]byte {
|
||||
// key = first 16 bytes of SHA256("#regionname")
|
||||
}
|
||||
```
|
||||
|
||||
Returns `map[string][]byte` (region name → 16-byte HMAC key). Called once at startup, stored on the `Store`.
|
||||
|
||||
### Decoder: expose raw payload bytes
|
||||
|
||||
Add `PayloadRaw []byte` to `DecodedPacket` in `cmd/ingestor/decoder.go`. Populated from the raw `buf` slice at the payload offset — zero-copy slice, no allocation. This is the **encrypted** payload bytes, matching what the firmware feeds into `calcTransportCode`.
|
||||
|
||||
### At-ingest region matching
|
||||
|
||||
In `BuildPacketData`:
|
||||
- Skip if `route_type` not in `{0, 3}` → `scope_name` stays `nil`
|
||||
- If `Code1 == "0000"` → `scope_name = nil` (unscoped transport, no scope involvement)
|
||||
- If `Code1 != "0000"` → try each region key:
|
||||
```
|
||||
HMAC-SHA256(key, payloadType_byte || PayloadRaw) → first 2 bytes as uint16
|
||||
```
|
||||
First match → `scope_name = "#regionname"`. No match → `scope_name = ""` (unknown scope).
|
||||
|
||||
Add `ScopeName *string` to `PacketData`.
|
||||
|
||||
### MQTT-sourced packets (DM / CHAN paths in main.go)
|
||||
|
||||
These are injected directly without going through `BuildPacketData`. They use `route_type = 1` (FLOOD), so they are never transport-route packets. No scope matching needed for these paths.
|
||||
|
||||
---
|
||||
|
||||
## Database
|
||||
|
||||
### Migration
|
||||
|
||||
```sql
|
||||
ALTER TABLE transmissions ADD COLUMN scope_name TEXT DEFAULT NULL;
|
||||
CREATE INDEX idx_tx_scope_name ON transmissions(scope_name) WHERE scope_name IS NOT NULL;
|
||||
```
|
||||
|
||||
### Column semantics
|
||||
|
||||
| Value | Meaning |
|
||||
|-------|---------|
|
||||
| `NULL` | Either: non-transport-route packet (route_type 1/2), or transport-route with Code1=0000 |
|
||||
| `""` (empty string) | Transport-route, Code1 ≠ 0000, but no configured region matched |
|
||||
| `"#belgium"` | Matched named region |
|
||||
|
||||
The API stats queries resolve the NULL ambiguity by always filtering `route_type IN (0, 3)` first:
|
||||
- `unscoped` count = `route_type IN (0,3) AND scope_name IS NULL`
|
||||
- `scoped` count = `route_type IN (0,3) AND scope_name IS NOT NULL`
|
||||
|
||||
### Backfill
|
||||
|
||||
On migration, re-decode `raw_hex` for all rows where `route_type IN (0, 3)` and `scope_name IS NULL`. Run the same HMAC matching logic. Rows with `Code1 = 0000` remain `NULL`.
|
||||
|
||||
The backfill runs in the existing migration framework in `cmd/ingestor/db.go`. If no regions are configured, backfill is skipped.
|
||||
|
||||
---
|
||||
|
||||
## API
|
||||
|
||||
### `GET /api/scope-stats`
|
||||
|
||||
**Query param**: `window` — one of `1h`, `24h` (default), `7d`
|
||||
|
||||
**Time-series bucket sizes**:
|
||||
| Window | Bucket |
|
||||
|--------|--------|
|
||||
| `1h` | 5 min |
|
||||
| `24h` | 1 hour |
|
||||
| `7d` | 6 hours|
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
{
|
||||
"window": "24h",
|
||||
"summary": {
|
||||
"transportTotal": 1240,
|
||||
"scoped": 890,
|
||||
"unscoped": 350,
|
||||
"unknownScope": 42
|
||||
},
|
||||
"byRegion": [
|
||||
{ "name": "#belgium", "count": 612 },
|
||||
{ "name": "#eu", "count": 236 }
|
||||
],
|
||||
"timeSeries": [
|
||||
{ "t": "2026-04-23T10:00:00Z", "scoped": 45, "unscoped": 18 },
|
||||
{ "t": "2026-04-23T11:00:00Z", "scoped": 51, "unscoped": 22 }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
- `transportTotal` = `scoped + unscoped` (transport-route packets only)
|
||||
- `scoped` = Code1 ≠ 0000 (named + unknown)
|
||||
- `unscoped` = transport-route with Code1 = 0000
|
||||
- `unknownScope` = scoped but no region name matched (subset of `scoped`)
|
||||
- `byRegion` sorted by count descending, excludes unknown
|
||||
- `timeSeries` covers the full window at the bucket granularity
|
||||
|
||||
Route: `GET /api/scope-stats` registered in `cmd/server/routes.go`.
|
||||
No auth required (same as other read endpoints).
|
||||
TTL cache: 30 seconds (heavier query than `/api/stats`).
|
||||
|
||||
---
|
||||
|
||||
## Frontend
|
||||
|
||||
### Navigation
|
||||
|
||||
Add nav link between Channels and Nodes in `public/index.html`:
|
||||
```html
|
||||
<a href="#/scopes" class="nav-link" data-route="scopes">Scopes</a>
|
||||
```
|
||||
|
||||
### `public/scopes.js`
|
||||
|
||||
Three sections on the page:
|
||||
|
||||
**1. Summary cards** (reuse existing card CSS pattern from home/analytics pages)
|
||||
- Transport total, Scoped, Unscoped, Unknown scope
|
||||
- Each card shows count + percentage of transport total
|
||||
|
||||
**2. Per-region table**
|
||||
Columns: Region, Messages, % of Scoped
|
||||
Sorted by count descending. Last row: "Unknown scope" (italic) if unknownScope > 0.
|
||||
Shows "No regions configured" message if `byRegion` is empty and `unknownScope = 0`.
|
||||
|
||||
**3. Time-series chart**
|
||||
- Window selector: `1h / 24h / 7d` (default 24h)
|
||||
- Two lines: **Scoped** (blue) and **Unscoped** (grey)
|
||||
- Uses the same lightweight canvas chart pattern as other pages (no external chart lib)
|
||||
|
||||
### Cache buster
|
||||
|
||||
`scopes.js` added to the `__BUST__` entries in `index.html` in the same commit.
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
- Unit tests for `loadRegionKeys`: normalization, key bytes match firmware SHA256 derivation
|
||||
- Unit tests for HMAC matching: known Code1 value computed from firmware logic, verified against Go implementation
|
||||
- Integration test: ingest a synthetic transport-route packet with a known region, assert `scope_name` column is set correctly
|
||||
- API test: `GET /api/scope-stats` returns correct summary counts against fixture DB
|
||||
|
||||
---
|
||||
|
||||
## Out of Scope
|
||||
|
||||
- Feature 3 (default scope per client via advert) — firmware has no advert scope field
|
||||
- Drill-down from region row to filtered packet list (deferred)
|
||||
- Private regions (`$`-prefixed) — use secret keys not publicly derivable
|
||||
+5
-5
@@ -3495,12 +3495,12 @@ function destroy() { _analyticsData = {}; _channelData = null; if (_ngState && _
|
||||
});
|
||||
|
||||
// Summary
|
||||
var counts = { ok: 0, warning: 0, critical: 0, absurd: 0 };
|
||||
var counts = { ok: 0, degrading: 0, degraded: 0, wrong: 0, default: 0 };
|
||||
data.forEach(function(n) { if (counts[n.severity] !== undefined) counts[n.severity]++; });
|
||||
|
||||
// Filter buttons (also serve as summary — no separate stats pills needed)
|
||||
var filterColors = { ok: 'var(--status-green)', warning: 'var(--status-yellow)', critical: 'var(--status-orange)', absurd: 'var(--status-purple)', no_clock: 'var(--text-muted)' };
|
||||
var filters = ['all', 'ok', 'warning', 'critical', 'absurd', 'no_clock'];
|
||||
var filterColors = { ok: 'var(--status-green)', degrading: 'var(--status-yellow)', degraded: 'var(--status-orange)', wrong: 'var(--status-red)', default: 'var(--text-muted)' };
|
||||
var filters = ['all', 'ok', 'degrading', 'degraded', 'wrong', 'default'];
|
||||
var filterHtml = '<div style="margin-bottom:10px">' + filters.map(function(f) {
|
||||
var dot = f !== 'all' ? '<span style="display:inline-block;width:8px;height:8px;border-radius:50%;background:' + filterColors[f] + ';margin-right:4px;vertical-align:middle"></span>' : '';
|
||||
return '<button class="clock-filter-btn' + (activeFilter === f ? ' active' : '') + '" data-filter="' + f + '">' +
|
||||
@@ -3513,8 +3513,8 @@ function destroy() { _analyticsData = {}; _channelData = null; if (_ngState && _
|
||||
var rowClass = 'clock-fleet-row--' + (n.severity || 'ok');
|
||||
var lastAdv = n.lastObservedTS ? new Date(n.lastObservedTS * 1000).toISOString().replace('T', ' ').replace(/\.\d+Z/, ' UTC') : '—';
|
||||
var skewVal = window.currentSkewValue(n);
|
||||
var skewText = n.severity === 'no_clock' ? 'No Clock' : formatSkew(skewVal);
|
||||
var driftText = n.severity === 'no_clock' || !n.driftPerDaySec ? '–' : formatDrift(n.driftPerDaySec);
|
||||
var skewText = n.severity === 'default' ? 'Default' : formatSkew(skewVal);
|
||||
var driftText = n.severity === 'default' || !n.driftPerDaySec ? '–' : formatDrift(n.driftPerDaySec);
|
||||
return '<tr class="' + rowClass + '" data-pubkey="' + esc(n.pubkey) + '" style="cursor:pointer">' +
|
||||
'<td><strong>' + esc(n.nodeName || n.pubkey.slice(0, 12)) + '</strong></td>' +
|
||||
'<td style="font-family:var(--mono,monospace)">' + skewText + '</td>' +
|
||||
|
||||
@@ -70,7 +70,7 @@
|
||||
<div id="help-bar">
|
||||
Copy the JSON above → paste as a top-level key in <code>config.json</code> → restart the server.
|
||||
Nodes with no GPS fix always pass through. Remove the <code>geo_filter</code> block to disable filtering.
|
||||
· <a href="/geofilter-docs.html">Documentation</a>
|
||||
· <a href="https://github.com/Kpa-clawbot/CoreScope/blob/master/docs/user-guide/geofilter.md" target="_blank">Documentation ↗</a>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
|
||||
@@ -1,132 +0,0 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>GeoFilter Docs — CoreScope</title>
|
||||
<style>
|
||||
* { box-sizing: border-box; margin: 0; padding: 0; }
|
||||
body { font-family: system-ui, sans-serif; background: #1a1a2e; color: #e0e0e0; min-height: 100vh; display: flex; flex-direction: column; }
|
||||
header { padding: 12px 16px; background: #0f0f23; border-bottom: 1px solid #333; display: flex; align-items: center; gap: 16px; }
|
||||
header h1 { font-size: 1rem; font-weight: 600; color: #4a9eff; }
|
||||
#back-link { font-size: 0.8rem; color: #4a9eff; text-decoration: none; white-space: nowrap; }
|
||||
#back-link:hover { text-decoration: underline; }
|
||||
main { flex: 1; max-width: 800px; margin: 0 auto; padding: 32px 24px; width: 100%; }
|
||||
h2 { font-size: 1.1rem; font-weight: 600; color: #4a9eff; margin: 32px 0 12px; border-bottom: 1px solid #222; padding-bottom: 6px; }
|
||||
h2:first-of-type { margin-top: 0; }
|
||||
h3 { font-size: 0.95rem; font-weight: 600; color: #c0c0c0; margin: 20px 0 8px; }
|
||||
p { font-size: 0.9rem; line-height: 1.6; color: #ccc; margin-bottom: 10px; }
|
||||
ul { padding-left: 20px; margin-bottom: 10px; }
|
||||
li { font-size: 0.9rem; line-height: 1.7; color: #ccc; }
|
||||
code { font-family: monospace; font-size: 0.85rem; color: #7ec8e3; background: #111; border: 1px solid #333; border-radius: 3px; padding: 1px 5px; }
|
||||
pre { background: #111; border: 1px solid #333; border-radius: 6px; padding: 14px 16px; overflow-x: auto; margin: 10px 0 16px; }
|
||||
pre code { background: none; border: none; padding: 0; font-size: 0.82rem; color: #7ec8e3; }
|
||||
.note { background: #1a2a1a; border: 1px solid #2a4a2a; border-radius: 6px; padding: 10px 14px; margin: 12px 0; }
|
||||
.note p { color: #aaddaa; margin: 0; }
|
||||
.warn { background: #2a1a0a; border: 1px solid #5a3a0a; border-radius: 6px; padding: 10px 14px; margin: 12px 0; }
|
||||
.warn p { color: #ddbb88; margin: 0; }
|
||||
table { width: 100%; border-collapse: collapse; margin: 10px 0 16px; font-size: 0.88rem; }
|
||||
th { background: #0f0f23; color: #888; font-weight: 500; text-align: left; padding: 8px 12px; border: 1px solid #333; }
|
||||
td { padding: 8px 12px; border: 1px solid #222; color: #ccc; vertical-align: top; }
|
||||
td code { font-size: 0.82rem; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
|
||||
<header>
|
||||
<a href="/geofilter-builder.html" id="back-link">← GeoFilter Builder</a>
|
||||
<h1>GeoFilter Docs</h1>
|
||||
</header>
|
||||
|
||||
<main>
|
||||
|
||||
<h2>How it works</h2>
|
||||
<p>Geographic filtering restricts which nodes are ingested and returned in API responses. It operates at two levels:</p>
|
||||
<ul>
|
||||
<li><strong>Ingest time</strong> — ADVERT packets carrying GPS coordinates are rejected by the ingestor if the node falls outside the configured area. The node never reaches the database.</li>
|
||||
<li><strong>API responses</strong> — Nodes already in the database are filtered from the <code>/api/nodes</code> response if they fall outside the area. This covers nodes ingested before the filter was configured.</li>
|
||||
</ul>
|
||||
<div class="note"><p>Nodes with no GPS fix (<code>lat=0, lon=0</code> or missing coordinates) always pass the filter regardless of configuration.</p></div>
|
||||
|
||||
<h2>Configuration</h2>
|
||||
<p>Add a <code>geo_filter</code> block to <code>config.json</code>:</p>
|
||||
<pre><code>"geo_filter": {
|
||||
"polygon": [
|
||||
[51.55, 3.80],
|
||||
[51.55, 5.90],
|
||||
[50.65, 5.90],
|
||||
[50.65, 3.80]
|
||||
],
|
||||
"bufferKm": 20
|
||||
}</code></pre>
|
||||
<table>
|
||||
<thead><tr><th>Field</th><th>Type</th><th>Description</th></tr></thead>
|
||||
<tbody>
|
||||
<tr><td><code>polygon</code></td><td><code>[[lat, lon], ...]</code></td><td>Array of at least 3 coordinate pairs defining the boundary</td></tr>
|
||||
<tr><td><code>bufferKm</code></td><td>number</td><td>Extra distance (km) around the polygon edge that is also accepted. <code>0</code> = exact boundary</td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<p>Both the server and the ingestor read <code>geo_filter</code> from <code>config.json</code>. Restart both after changing this section.</p>
|
||||
<p>To disable filtering entirely, remove the <code>geo_filter</code> block.</p>
|
||||
|
||||
<h2>Coordinate ordering</h2>
|
||||
<div class="warn"><p><strong>Important:</strong> Coordinates are <code>[lat, lon]</code> — latitude first, longitude second. This is the opposite of GeoJSON, which uses <code>[lon, lat]</code>. Swapping them will place your polygon in the wrong location.</p></div>
|
||||
|
||||
<h2>Multi-polygon</h2>
|
||||
<p>Only a single polygon is supported. If your deployment area consists of multiple disconnected regions, draw a single convex hull that covers all of them, or use the largest region with a generous <code>bufferKm</code> value.</p>
|
||||
|
||||
<h2>Examples</h2>
|
||||
<h3>Belgium (bounding rectangle)</h3>
|
||||
<pre><code>"geo_filter": {
|
||||
"polygon": [
|
||||
[51.55, 3.80],
|
||||
[51.55, 5.90],
|
||||
[50.65, 5.90],
|
||||
[50.65, 3.80]
|
||||
],
|
||||
"bufferKm": 20
|
||||
}</code></pre>
|
||||
<h3>Irregular shape</h3>
|
||||
<pre><code>"geo_filter": {
|
||||
"polygon": [
|
||||
[51.10, 3.70],
|
||||
[51.55, 4.20],
|
||||
[51.30, 5.10],
|
||||
[50.80, 5.50],
|
||||
[50.50, 4.80],
|
||||
[50.70, 3.90]
|
||||
],
|
||||
"bufferKm": 10
|
||||
}</code></pre>
|
||||
|
||||
<h2>Legacy bounding box</h2>
|
||||
<p>An older bounding box format is also supported as a fallback when no <code>polygon</code> is present:</p>
|
||||
<pre><code>"geo_filter": {
|
||||
"latMin": 50.65,
|
||||
"latMax": 51.55,
|
||||
"lonMin": 3.80,
|
||||
"lonMax": 5.90
|
||||
}</code></pre>
|
||||
<p>Prefer the polygon format — it supports irregular shapes and the <code>bufferKm</code> margin.</p>
|
||||
|
||||
<h2>Cleaning up historical nodes</h2>
|
||||
<p>The ingestor prevents new out-of-bounds nodes from being ingested, but does not retroactively remove nodes stored before the filter was configured. Use the prune script for that:</p>
|
||||
<pre><code># Dry run — shows what would be deleted without making any changes
|
||||
python3 scripts/prune-nodes-outside-geo-filter.py --dry-run
|
||||
|
||||
# Default paths: /app/data/meshcore.db and /app/config.json
|
||||
python3 scripts/prune-nodes-outside-geo-filter.py
|
||||
|
||||
# Custom paths
|
||||
python3 scripts/prune-nodes-outside-geo-filter.py /path/to/meshcore.db \
|
||||
--config /path/to/config.json
|
||||
|
||||
# In Docker — run inside the container
|
||||
docker exec -it meshcore-analyzer \
|
||||
python3 /app/scripts/prune-nodes-outside-geo-filter.py --dry-run</code></pre>
|
||||
<p>The script reads <code>geo_filter.polygon</code> and <code>geo_filter.bufferKm</code> from config, lists nodes that fall outside, then asks for <code>yes</code> confirmation before deleting. Nodes without coordinates are always kept.</p>
|
||||
<p>This is a one-time migration tool — run it once after first configuring <code>geo_filter</code> to clean up pre-filter data.</p>
|
||||
|
||||
</main>
|
||||
</body>
|
||||
</html>
|
||||
@@ -1,70 +0,0 @@
|
||||
/* hash-color.js — Deterministic HSL color from packet hash
|
||||
* IIFE attaching window.HashColor = { hashToHsl, hashToOutline }
|
||||
* Pure function: no DOM access, no state, works in Node vm.createContext sandbox.
|
||||
*/
|
||||
(function() {
|
||||
'use strict';
|
||||
|
||||
/**
|
||||
* Derive a deterministic HSL color string from a hex hash.
|
||||
* Uses bytes 0-1 for hue, byte 2 for saturation, byte 3 for lightness.
|
||||
* Produces bright vivid fills; contrast is provided by a dark outline (hashToOutline).
|
||||
* @param {string|null|undefined} hashHex - Hex string (e.g. "a1b2c3d4...")
|
||||
* @param {string} theme - "light" or "dark"
|
||||
* @returns {string} CSS hsl() string
|
||||
*/
|
||||
function hashToHsl(hashHex, theme) {
|
||||
if (!hashHex || hashHex.length < 8) {
|
||||
return 'hsl(0, 0%, 50%)';
|
||||
}
|
||||
|
||||
var b0 = parseInt(hashHex.slice(0, 2), 16) || 0;
|
||||
var b1 = parseInt(hashHex.slice(2, 4), 16) || 0;
|
||||
var b2 = parseInt(hashHex.slice(4, 6), 16) || 0;
|
||||
var b3 = parseInt(hashHex.slice(6, 8), 16) || 0;
|
||||
|
||||
// Hue: 0-360 from bytes 0-1 (16-bit)
|
||||
var hue = Math.round(((b0 << 8) | b1) / 65535 * 360);
|
||||
// Saturation: 55-95% from byte 2
|
||||
var S = 55 + Math.round(b2 / 255 * 40);
|
||||
// Lightness: vivid range per theme from byte 3
|
||||
// Light: 50-65%, Dark: 55-72%
|
||||
var L;
|
||||
if (theme === 'dark') {
|
||||
L = 55 + Math.round(b3 / 255 * 17);
|
||||
} else {
|
||||
L = 50 + Math.round(b3 / 255 * 15);
|
||||
}
|
||||
|
||||
return 'hsl(' + hue + ', ' + S + '%, ' + L + '%)';
|
||||
}
|
||||
|
||||
/**
|
||||
* Derive a dark outline color (same hue) for contrast against backgrounds.
|
||||
* @param {string|null|undefined} hashHex - Hex string
|
||||
* @param {string} theme - "light" or "dark"
|
||||
* @returns {string} CSS hsl() string
|
||||
*/
|
||||
function hashToOutline(hashHex, theme) {
|
||||
if (!hashHex || hashHex.length < 8) {
|
||||
return 'hsl(0, 0%, 30%)';
|
||||
}
|
||||
|
||||
var b0 = parseInt(hashHex.slice(0, 2), 16) || 0;
|
||||
var b1 = parseInt(hashHex.slice(2, 4), 16) || 0;
|
||||
var hue = Math.round(((b0 << 8) | b1) / 65535 * 360);
|
||||
|
||||
// Dark outline: same hue, low lightness for contrast
|
||||
if (theme === 'dark') {
|
||||
return 'hsl(' + hue + ', 30%, 15%)';
|
||||
}
|
||||
return 'hsl(' + hue + ', 70%, 25%)';
|
||||
}
|
||||
|
||||
// Export
|
||||
if (typeof window !== 'undefined') {
|
||||
window.HashColor = { hashToHsl: hashToHsl, hashToOutline: hashToOutline };
|
||||
} else if (typeof module !== 'undefined') {
|
||||
module.exports = { hashToHsl: hashToHsl, hashToOutline: hashToOutline };
|
||||
}
|
||||
})();
|
||||
@@ -94,7 +94,6 @@
|
||||
<script src="home.js?v=__BUST__"></script>
|
||||
<script src="table-sort.js?v=__BUST__"></script>
|
||||
<script src="packet-filter.js?v=__BUST__"></script>
|
||||
<script src="hash-color.js?v=__BUST__"></script>
|
||||
<script src="packet-helpers.js?v=__BUST__"></script>
|
||||
<script src="channel-decrypt.js?v=__BUST__"></script>
|
||||
<script src="channel-colors.js?v=__BUST__"></script>
|
||||
|
||||
+9
-133
@@ -22,12 +22,6 @@
|
||||
let showOnlyFavorites = localStorage.getItem('live-favorites-only') === 'true';
|
||||
let matrixMode = localStorage.getItem('live-matrix-mode') === 'true';
|
||||
let matrixRain = localStorage.getItem('live-matrix-rain') === 'true';
|
||||
let colorByHash = localStorage.getItem('meshcore-color-packets-by-hash') !== 'false';
|
||||
/** Current theme string for hash-color functions. */
|
||||
function _liveTheme() { return document.documentElement.dataset.theme || (window.matchMedia('(prefers-color-scheme: dark)').matches ? 'dark' : 'light'); }
|
||||
let nodeFilterKeys = (localStorage.getItem('live-node-filter') || '').split(',').map(s => s.trim()).filter(Boolean);
|
||||
let nodeFilterTotal = 0;
|
||||
let nodeFilterShown = 0;
|
||||
let rainCanvas = null, rainCtx = null, rainDrops = [], rainRAF = null;
|
||||
const propagationBuffer = new Map(); // hash -> {timer, packets[]}
|
||||
let _onResize = null;
|
||||
@@ -831,8 +825,6 @@
|
||||
<span id="ghostDesc" class="sr-only">Show interpolated ghost markers for unknown hops</span>
|
||||
<label><input type="checkbox" id="liveRealisticToggle" aria-describedby="realisticDesc"> Realistic</label>
|
||||
<span id="realisticDesc" class="sr-only">Buffer packets by hash and animate all paths simultaneously</span>
|
||||
<label><input type="checkbox" id="liveColorHashToggle" aria-describedby="colorHashDesc"> Color by hash</label>
|
||||
<span id="colorHashDesc" class="sr-only">Color flying-packet dots and contrails by packet hash for propagation tracing</span>
|
||||
<label><input type="checkbox" id="liveMatrixToggle" aria-describedby="matrixDesc"> Matrix</label>
|
||||
<span id="matrixDesc" class="sr-only">Animate packet hex bytes flowing along paths like the Matrix</span>
|
||||
<label><input type="checkbox" id="liveMatrixRainToggle" aria-describedby="rainDesc"> Rain</label>
|
||||
@@ -841,12 +833,6 @@
|
||||
<span id="audioDesc" class="sr-only">Sonify packets — turn raw bytes into generative music</span>
|
||||
<label><input type="checkbox" id="liveFavoritesToggle" aria-describedby="favDesc"> ⭐ Favorites</label>
|
||||
<span id="favDesc" class="sr-only">Show only favorited and claimed nodes</span>
|
||||
<div class="live-node-filter-wrap">
|
||||
<input type="text" id="liveNodeFilterInput" list="liveNodeFilterList" placeholder="Filter by node…" autocomplete="off" class="live-node-filter-input">
|
||||
<datalist id="liveNodeFilterList"></datalist>
|
||||
<button id="liveNodeFilterClear" class="vcr-btn" title="Clear node filter" style="display:none">×</button>
|
||||
</div>
|
||||
<div id="liveNodeFilterCount" class="live-filter-count hidden"></div>
|
||||
<label id="liveGeoFilterLabel" style="display:none"><input type="checkbox" id="liveGeoFilterToggle"> Mesh live area</label>
|
||||
</div>
|
||||
<div class="audio-controls hidden" id="audioControls">
|
||||
@@ -997,14 +983,6 @@
|
||||
localStorage.setItem('live-realistic-propagation', realisticPropagation);
|
||||
});
|
||||
|
||||
const colorHashToggle = document.getElementById('liveColorHashToggle');
|
||||
colorHashToggle.checked = colorByHash;
|
||||
colorHashToggle.addEventListener('change', (e) => {
|
||||
colorByHash = e.target.checked;
|
||||
localStorage.setItem('meshcore-color-packets-by-hash', colorByHash);
|
||||
window.dispatchEvent(new Event('storage'));
|
||||
});
|
||||
|
||||
const favoritesToggle = document.getElementById('liveFavoritesToggle');
|
||||
favoritesToggle.checked = showOnlyFavorites;
|
||||
favoritesToggle.addEventListener('change', (e) => {
|
||||
@@ -1013,35 +991,6 @@
|
||||
applyFavoritesFilter();
|
||||
});
|
||||
|
||||
// Node filter input
|
||||
const nodeFilterInput = document.getElementById('liveNodeFilterInput');
|
||||
const nodeFilterClear = document.getElementById('liveNodeFilterClear');
|
||||
if (nodeFilterInput) {
|
||||
// Restore from URL param or localStorage
|
||||
const urlNode = getHashParams && getHashParams().get('node');
|
||||
if (urlNode) setNodeFilter(urlNode.split(',').map(s => s.trim()).filter(Boolean));
|
||||
else if (nodeFilterKeys.length) updateNodeFilterUI();
|
||||
|
||||
nodeFilterInput.addEventListener('change', (e) => {
|
||||
const val = e.target.value.trim();
|
||||
setNodeFilter(val ? val.split(',').map(s => s.trim()).filter(Boolean) : []);
|
||||
const params = getHashParams ? getHashParams() : new URLSearchParams();
|
||||
if (nodeFilterKeys.length) params.set('node', nodeFilterKeys.join(','));
|
||||
else params.delete('node');
|
||||
const base = location.hash.split('?')[0];
|
||||
const qs = params.toString();
|
||||
location.hash = base + (qs ? '?' + qs : '');
|
||||
});
|
||||
}
|
||||
if (nodeFilterClear) {
|
||||
nodeFilterClear.addEventListener('click', () => {
|
||||
if (nodeFilterInput) nodeFilterInput.value = '';
|
||||
setNodeFilter([]);
|
||||
const base = location.hash.split('?')[0];
|
||||
location.hash = base;
|
||||
});
|
||||
}
|
||||
|
||||
// Geo filter overlay
|
||||
(async function () {
|
||||
try {
|
||||
@@ -1707,47 +1656,6 @@
|
||||
return getFavoritePubkeys().some(f => f === pubkey);
|
||||
}
|
||||
|
||||
function packetInvolvesFilterNode(pkt, filterKeys) {
|
||||
if (!filterKeys.length) return true;
|
||||
const hops = (pkt.decoded?.path?.hops) || [];
|
||||
for (const hop of hops) {
|
||||
const h = (hop.id || hop.public_key || hop).toString().toLowerCase();
|
||||
if (filterKeys.some(f => f.toLowerCase().startsWith(h) || h.startsWith(f.toLowerCase()))) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
function setNodeFilter(keys) {
|
||||
nodeFilterKeys = keys;
|
||||
nodeFilterTotal = 0;
|
||||
nodeFilterShown = 0;
|
||||
localStorage.setItem('live-node-filter', keys.join(','));
|
||||
updateNodeFilterUI();
|
||||
}
|
||||
|
||||
function updateNodeFilterUI() {
|
||||
const countEl = document.getElementById('liveNodeFilterCount');
|
||||
const clearBtn = document.getElementById('liveNodeFilterClear');
|
||||
const input = document.getElementById('liveNodeFilterInput');
|
||||
if (nodeFilterKeys.length > 0) {
|
||||
if (clearBtn) clearBtn.style.display = '';
|
||||
if (countEl) { countEl.textContent = `Showing ${nodeFilterShown} of ${nodeFilterTotal}`; countEl.classList.remove('hidden'); }
|
||||
if (input && input.value !== nodeFilterKeys.join(', ')) input.value = nodeFilterKeys.join(', ');
|
||||
} else {
|
||||
if (clearBtn) clearBtn.style.display = 'none';
|
||||
if (countEl) countEl.classList.add('hidden');
|
||||
}
|
||||
updateNodeFilterDatalist();
|
||||
}
|
||||
|
||||
function updateNodeFilterDatalist() {
|
||||
const dl = document.getElementById('liveNodeFilterList');
|
||||
if (!dl) return;
|
||||
dl.innerHTML = Object.values(nodeData).map(n =>
|
||||
`<option value="${n.public_key}">${n.name || n.public_key.slice(0, 8)}</option>`
|
||||
).join('');
|
||||
}
|
||||
|
||||
function rebuildFeedList() {
|
||||
const feed = document.getElementById('liveFeed');
|
||||
if (!feed) return;
|
||||
@@ -1954,9 +1862,6 @@
|
||||
window._liveGetFavoritePubkeys = getFavoritePubkeys;
|
||||
window._livePacketInvolvesFavorite = packetInvolvesFavorite;
|
||||
window._liveIsNodeFavorited = isNodeFavorited;
|
||||
window._livePacketInvolvesFilterNode = packetInvolvesFilterNode;
|
||||
window._liveGetNodeFilterKeys = function() { return nodeFilterKeys; };
|
||||
window._liveSetNodeFilter = setNodeFilter;
|
||||
window._liveFormatLiveTimestampHtml = formatLiveTimestampHtml;
|
||||
window._liveResolveHopPositions = resolveHopPositions;
|
||||
window._liveVcrSpeedCycle = vcrSpeedCycle;
|
||||
@@ -2047,14 +1952,6 @@
|
||||
// --- Favorites filter ---
|
||||
if (showOnlyFavorites && !packets.some(function(p) { return packetInvolvesFavorite(p); })) return;
|
||||
|
||||
// --- Node filter ---
|
||||
if (nodeFilterKeys.length) {
|
||||
nodeFilterTotal++;
|
||||
if (!packets.some(function(p) { return packetInvolvesFilterNode(p, nodeFilterKeys); })) return;
|
||||
nodeFilterShown++;
|
||||
updateNodeFilterUI();
|
||||
}
|
||||
|
||||
// --- Ensure ADVERT nodes appear on map ---
|
||||
for (var pi = 0; pi < packets.length; pi++) {
|
||||
var pkt = packets[pi];
|
||||
@@ -2171,7 +2068,7 @@
|
||||
var completedPositions = allPaths[ai].hopPositions.slice(0, hopsCompleted + 1);
|
||||
var remainingPositions = allPaths[ai].hopPositions.slice(hopsCompleted);
|
||||
if (completedPositions.length >= 2) {
|
||||
animatePath(completedPositions, typeName, color, allPaths[ai].raw, onHop, first.hash);
|
||||
animatePath(completedPositions, typeName, color, allPaths[ai].raw, onHop);
|
||||
} else if (completedPositions.length === 1) {
|
||||
pulseNode(completedPositions[0].key, completedPositions[0].pos, typeName);
|
||||
}
|
||||
@@ -2179,7 +2076,7 @@
|
||||
drawDashedPath(remainingPositions, color);
|
||||
}
|
||||
} else {
|
||||
animatePath(allPaths[ai].hopPositions, typeName, color, allPaths[ai].raw, onHop, first.hash);
|
||||
animatePath(allPaths[ai].hopPositions, typeName, color, allPaths[ai].raw, onHop);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -2288,7 +2185,7 @@
|
||||
return raw.filter(h => h.pos != null);
|
||||
}
|
||||
|
||||
function animatePath(hopPositions, typeName, color, rawHex, onHop, hash) {
|
||||
function animatePath(hopPositions, typeName, color, rawHex, onHop) {
|
||||
if (!animLayer || !pathsLayer) return;
|
||||
if (activeAnims >= MAX_CONCURRENT_ANIMS) return;
|
||||
activeAnims++;
|
||||
@@ -2340,7 +2237,7 @@
|
||||
const nextGhost = hopPositions[hopIndex + 1].ghost;
|
||||
const lineColor = (isGhost || nextGhost) ? '#94a3b8' : color;
|
||||
const lineOpacity = (isGhost || nextGhost) ? 0.3 : undefined;
|
||||
drawAnimatedLine(hp.pos, nextPos, lineColor, () => { hopIndex++; nextHop(); }, lineOpacity, rawHex, hash);
|
||||
drawAnimatedLine(hp.pos, nextPos, lineColor, () => { hopIndex++; nextHop(); }, lineOpacity, rawHex);
|
||||
} else {
|
||||
if (!isGhost) pulseNode(hp.key, hp.pos, typeName);
|
||||
hopIndex++; nextHop();
|
||||
@@ -2695,7 +2592,7 @@
|
||||
requestAnimationFrame(tick);
|
||||
}
|
||||
|
||||
function drawAnimatedLine(from, to, color, onComplete, overrideOpacity, rawHex, hash) {
|
||||
function drawAnimatedLine(from, to, color, onComplete, overrideOpacity, rawHex) {
|
||||
if (!animLayer || !pathsLayer) { if (onComplete) onComplete(); return; }
|
||||
if (matrixMode) return drawMatrixLine(from, to, color, onComplete, rawHex);
|
||||
const steps = 20;
|
||||
@@ -2706,30 +2603,17 @@
|
||||
const mainOpacity = overrideOpacity ?? 0.8;
|
||||
const isDashed = overrideOpacity != null;
|
||||
|
||||
// Hash-derived color for fill + contrail + outline (when toggle ON and not ghost/dashed line)
|
||||
var hashFill = '#fff';
|
||||
var hashOutline = color;
|
||||
var contrailColor = color;
|
||||
if (colorByHash && hash && !isDashed && window.HashColor) {
|
||||
var hsl = HashColor.hashToHsl(hash, _liveTheme());
|
||||
hashFill = hsl;
|
||||
hashOutline = HashColor.hashToOutline(hash, _liveTheme());
|
||||
contrailColor = hsl;
|
||||
}
|
||||
|
||||
const contrail = L.polyline([from], {
|
||||
color: contrailColor, weight: 6, opacity: mainOpacity * 0.2, lineCap: 'round'
|
||||
color: color, weight: 6, opacity: mainOpacity * 0.2, lineCap: 'round'
|
||||
}).addTo(pathsLayer);
|
||||
|
||||
const line = L.polyline([from], {
|
||||
color: (colorByHash && hash && !isDashed && window.HashColor) ? hashFill : color,
|
||||
weight: isDashed ? 1.5 : 2, opacity: mainOpacity, lineCap: 'round',
|
||||
dashArray: isDashed ? '4 6' : null,
|
||||
className: 'live-packet-trace'
|
||||
color: color, weight: isDashed ? 1.5 : 2, opacity: mainOpacity, lineCap: 'round',
|
||||
dashArray: isDashed ? '4 6' : null
|
||||
}).addTo(pathsLayer);
|
||||
|
||||
const dot = L.circleMarker(from, {
|
||||
radius: 3.5, fillColor: hashFill, fillOpacity: 1, color: hashOutline, weight: 1.5
|
||||
radius: 3.5, fillColor: '#fff', fillOpacity: 1, color: color, weight: 1.5
|
||||
}).addTo(animLayer);
|
||||
|
||||
let lastStep = performance.now();
|
||||
@@ -2861,10 +2745,6 @@
|
||||
item.setAttribute('tabindex', '0');
|
||||
item.setAttribute('role', 'button');
|
||||
item.style.cursor = 'pointer';
|
||||
// Hash-color stripe for feed items (mirrors packets table border-left)
|
||||
if (colorByHash && pkt.hash && window.HashColor) {
|
||||
item.style.borderLeft = '4px solid ' + HashColor.hashToHsl(pkt.hash, _liveTheme());
|
||||
}
|
||||
// Channel color highlighting for GRP_TXT packets (#271)
|
||||
var _cs = _getChannelStyle(pkt);
|
||||
if (_cs) item.style.cssText += _cs;
|
||||
@@ -2948,10 +2828,6 @@
|
||||
item.setAttribute('role', 'button');
|
||||
if (hash) item.setAttribute('data-hash', hash);
|
||||
item.style.cursor = 'pointer';
|
||||
// Hash-color stripe for feed items (mirrors packets table border-left)
|
||||
if (colorByHash && hash && window.HashColor) {
|
||||
item.style.borderLeft = '4px solid ' + HashColor.hashToHsl(hash, _liveTheme());
|
||||
}
|
||||
// Channel color highlighting for GRP_TXT packets (#271)
|
||||
var _chanStyle = _getChannelStyle(pkt);
|
||||
if (_chanStyle) item.style.cssText += _chanStyle;
|
||||
|
||||
+2
-14
@@ -388,14 +388,6 @@
|
||||
}
|
||||
|
||||
function drawPacketRoute(hopKeys, origin) {
|
||||
// Defensive: origin must be an object with pubkey/lat/lon/name. A bare
|
||||
// string slips through both branches at lines below and silently no-ops
|
||||
// the originator marker (caused PR #950's bug). Coerce string → object
|
||||
// and warn so callers get a clear signal.
|
||||
if (typeof origin === 'string') {
|
||||
console.warn('drawPacketRoute: origin should be an object {pubkey,lat,lon,name}, got string. Coercing.');
|
||||
origin = { pubkey: origin };
|
||||
}
|
||||
// Hide default markers so only the route is visible
|
||||
if (markerLayer) map.removeLayer(markerLayer);
|
||||
if (clusterGroup) map.removeLayer(clusterGroup);
|
||||
@@ -580,11 +572,7 @@
|
||||
delete window._pendingPathInspectorRoute;
|
||||
if (pending.path && pending.path.length > 0) {
|
||||
if (window.routeLayer) window.routeLayer.clearLayers();
|
||||
// Pass full path as hopKeys; null origin (origin is already the first
|
||||
// hop). slice(1) + path[0] string was wrong — drawPacketRoute expects
|
||||
// origin to be an OBJECT with pubkey/lat/lon, and stripping the head
|
||||
// hid the originating node from the route polyline.
|
||||
drawPacketRoute(pending.path, null);
|
||||
drawPacketRoute(pending.path.slice(1), pending.path[0]);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1119,7 +1107,7 @@
|
||||
var idx = parseInt(btn.dataset.idx);
|
||||
var cand = data.candidates[idx];
|
||||
if (routeLayer) routeLayer.clearLayers();
|
||||
drawPacketRoute(cand.path, null);
|
||||
drawPacketRoute(cand.path.slice(1), cand.path[0]);
|
||||
});
|
||||
});
|
||||
// Expand evidence on row click.
|
||||
|
||||
+23
-9
@@ -808,7 +808,7 @@
|
||||
let _themeRefreshHandler = null;
|
||||
|
||||
let _allNodes = null; // cached full node list
|
||||
let _fleetSkew = null; // cached clock skew map: pubkey → {severity, recentMedianSkewSec, medianSkewSec, ...}
|
||||
let _fleetSkew = null; // cached clock skew map: pubkey → {severity, medianSkewSec, ...}
|
||||
|
||||
/**
|
||||
* Fetch per-node clock skew and render into the given container element.
|
||||
@@ -824,14 +824,28 @@
|
||||
var driftHtml = cs.driftPerDaySec ? '<div style="font-size:12px;color:var(--text-muted);margin-top:2px">Drift: ' + formatDrift(cs.driftPerDaySec) + '</div>' : '';
|
||||
var sparkHtml = renderSkewSparkline(cs.samples, 200, 32);
|
||||
var skewVal = window.currentSkewValue(cs);
|
||||
var skewDisplay = cs.severity === 'no_clock'
|
||||
? '<span style="font-size:18px;font-weight:700;color:var(--text-muted)">No Clock</span>'
|
||||
var skewDisplay = cs.severity === 'default'
|
||||
? '<span style="font-size:18px;font-weight:700;color:var(--text-muted)">Default</span>'
|
||||
: '<span style="font-size:18px;font-weight:700;font-family:var(--mono)">' + formatSkew(skewVal) + '</span>';
|
||||
var bimodalWarning = '';
|
||||
if (cs.severity === 'bimodal_clock') {
|
||||
var totalRecent = cs.recentSampleCount || 0;
|
||||
bimodalWarning = '<div style="font-size:12px;color:var(--status-amber-text);margin-top:4px">⚠️ ' + (cs.recentBadSampleCount || '?') + ' of last ' + (totalRecent || '?') + ' adverts had nonsense timestamps (likely RTC reset)</div>';
|
||||
|
||||
// Per-tier explainer line (plain English reason).
|
||||
var explainer = '';
|
||||
var absSkew = Math.abs(cs.lastSkewSec || 0);
|
||||
var skewStr = Math.round(absSkew) + 's';
|
||||
if (cs.severity === 'default') {
|
||||
var isoAdv = cs.lastAdvertTS ? new Date(cs.lastAdvertTS * 1000).toISOString() : '?';
|
||||
explainer = 'Last advert at ' + isoAdv + ' — matches firmware default (volatile RTC, not user-set since boot)';
|
||||
} else if (cs.severity === 'ok') {
|
||||
explainer = 'Last advert ' + skewStr + ' vs wall clock — within OK tolerance (≤15s)';
|
||||
} else if (cs.severity === 'degrading') {
|
||||
explainer = 'Last advert ' + skewStr + ' vs wall clock — drift accumulating (≤60s)';
|
||||
} else if (cs.severity === 'degraded') {
|
||||
explainer = 'Last advert ' + skewStr + ' vs wall clock — significantly off (≤10m)';
|
||||
} else if (cs.severity === 'wrong') {
|
||||
explainer = 'Last advert ' + skewStr + ' vs wall clock — clock incorrect (operator-set or RTC failure)';
|
||||
}
|
||||
var explainerHtml = explainer ? '<div style="font-size:12px;color:var(--text-muted);margin-top:4px">' + explainer + '</div>' : '';
|
||||
|
||||
container.innerHTML =
|
||||
'<h4 style="margin:0 0 6px">⏰ Clock Skew</h4>' +
|
||||
'<div style="display:flex;align-items:center;gap:12px;flex-wrap:wrap">' +
|
||||
@@ -839,9 +853,9 @@
|
||||
renderSkewBadge(cs.severity, skewVal, cs) +
|
||||
(cs.calibrated ? ' <span style="font-size:10px;color:var(--text-muted)" title="Observer-calibrated">✓ calibrated</span>' : '') +
|
||||
'</div>' +
|
||||
explainerHtml +
|
||||
driftHtml +
|
||||
(sparkHtml ? '<div class="skew-sparkline-wrap" style="margin-top:8px">' + sparkHtml + '<div style="font-size:10px;color:var(--text-muted)">Skew over time (' + (cs.samples || []).length + ' samples)</div></div>' : '') +
|
||||
bimodalWarning;
|
||||
(sparkHtml ? '<div class="skew-sparkline-wrap" style="margin-top:8px">' + sparkHtml + '<div style="font-size:10px;color:var(--text-muted)">Skew over time (' + (cs.samples || []).length + ' samples)</div></div>' : '');
|
||||
} catch (e) {
|
||||
// Non-fatal — section stays hidden
|
||||
}
|
||||
|
||||
+3
-22
@@ -13,9 +13,6 @@
|
||||
return o.iata ? `${o.name} (${o.iata})` : o.name;
|
||||
}
|
||||
let selectedId = null;
|
||||
function _isColorByHash() { return localStorage.getItem('meshcore-color-packets-by-hash') !== 'false'; }
|
||||
function _currentTheme() { return document.documentElement.dataset.theme || (window.matchMedia('(prefers-color-scheme: dark)').matches ? 'dark' : 'light'); }
|
||||
function _hashStripeStyle(hash) { return _isColorByHash() && hash && window.HashColor ? 'border-left:4px solid ' + HashColor.hashToHsl(hash, _currentTheme()) + ';' : ''; }
|
||||
let groupByHash = true;
|
||||
let filters = {};
|
||||
{ const o = localStorage.getItem('meshcore-observer-filter'); if (o) filters.observer = o;
|
||||
@@ -1362,9 +1359,7 @@
|
||||
// Channel color highlighting (#271)
|
||||
const _grpDecoded = getParsedDecoded(p) || {};
|
||||
const _grpChanStyle = window.ChannelColors ? window.ChannelColors.getRowStyle(_grpDecoded.type || groupTypeName, _grpDecoded.channel) : '';
|
||||
const _grpHashStripe = _hashStripeStyle(p.hash);
|
||||
const _grpStyle = _grpHashStripe + _grpChanStyle;
|
||||
let html = `<tr class="${isSingle ? '' : 'group-header'} ${isExpanded ? 'expanded' : ''}" data-hash="${p.hash}" data-action="${isSingle ? 'select-hash' : 'toggle-select'}" data-value="${p.hash}" data-entry-idx="${entryIdx}" tabindex="0" role="row"${_grpStyle ? ' style="' + _grpStyle + '"' : ''}>
|
||||
let html = `<tr class="${isSingle ? '' : 'group-header'} ${isExpanded ? 'expanded' : ''}" data-hash="${p.hash}" data-action="${isSingle ? 'select-hash' : 'toggle-select'}" data-value="${p.hash}" data-entry-idx="${entryIdx}" tabindex="0" role="row"${_grpChanStyle ? ' style="' + _grpChanStyle + '"' : ''}>
|
||||
<td style="width:28px;text-align:center;cursor:pointer">${isSingle ? '' : (isExpanded ? '▼' : '▶')}</td>
|
||||
<td class="col-region">${groupRegion ? `<span class="badge-region">${groupRegion}</span>` : '—'}</td>
|
||||
<td class="col-time">${renderTimestampCell(p.latest)}</td>
|
||||
@@ -1390,8 +1385,7 @@
|
||||
const childRegion = c.observer_id ? (observerMap.get(c.observer_id)?.iata || '') : '';
|
||||
const childPath = getParsedPath(c);
|
||||
const childPathStr = renderPath(childPath, c.observer_id);
|
||||
const _childHashStripe = _hashStripeStyle(c.hash || p.hash);
|
||||
html += `<tr class="group-child" data-id="${c.id}" data-hash="${c.hash || ''}" data-action="select-observation" data-value="${c.id}" data-parent-hash="${p.hash}" data-entry-idx="${entryIdx}" tabindex="0" role="row"${_childHashStripe ? ' style="' + _childHashStripe + '"' : ''}>
|
||||
html += `<tr class="group-child" data-id="${c.id}" data-hash="${c.hash || ''}" data-action="select-observation" data-value="${c.id}" data-parent-hash="${p.hash}" data-entry-idx="${entryIdx}" tabindex="0" role="row">
|
||||
<td></td><td class="col-region">${childRegion ? `<span class="badge-region">${childRegion}</span>` : '—'}</td>
|
||||
<td class="col-time">${renderTimestampCell(c.timestamp)}</td>
|
||||
<td class="mono col-hash">${truncate(c.hash || '', 8)}</td>
|
||||
@@ -1421,9 +1415,7 @@
|
||||
const hashBytes = ((parseInt(p.raw_hex?.slice(2, 4), 16) || 0) >> 6) + 1;
|
||||
const pathStr = renderPath(pathHops, p.observer_id);
|
||||
const detail = getDetailPreview(decoded);
|
||||
const _flatHashStripe = _hashStripeStyle(p.hash);
|
||||
const _flatStyle = _flatHashStripe + _chanStyle;
|
||||
return `<tr data-id="${p.id}" data-hash="${p.hash || ''}" data-action="select-hash" data-value="${p.hash || p.id}" data-entry-idx="${entryIdx}" tabindex="0" role="row" class="${selectedId === p.id ? 'selected' : ''}"${_flatStyle ? ' style="' + _flatStyle + '"' : ''}>
|
||||
return `<tr data-id="${p.id}" data-hash="${p.hash || ''}" data-action="select-hash" data-value="${p.hash || p.id}" data-entry-idx="${entryIdx}" tabindex="0" role="row" class="${selectedId === p.id ? 'selected' : ''}"${_chanStyle ? ' style="' + _chanStyle + '"' : ''}>
|
||||
<td></td><td class="col-region">${region ? `<span class="badge-region">${region}</span>` : '—'}</td>
|
||||
<td class="col-time">${renderTimestampCell(p.timestamp)}</td>
|
||||
<td class="mono col-hash">${truncate(p.hash || String(p.id), 8)}</td>
|
||||
@@ -2564,22 +2556,12 @@
|
||||
} catch {}
|
||||
}
|
||||
|
||||
let _lastColorByHash = _isColorByHash();
|
||||
function _onStorageChange() {
|
||||
var current = _isColorByHash();
|
||||
if (_lastColorByHash !== current) {
|
||||
_lastColorByHash = current;
|
||||
renderVisibleRows();
|
||||
}
|
||||
}
|
||||
|
||||
let _themeRefreshHandler = null;
|
||||
|
||||
registerPage('packets', {
|
||||
init: function(app, routeParam) {
|
||||
_themeRefreshHandler = () => { if (typeof renderTableRows === 'function') renderTableRows(); };
|
||||
window.addEventListener('theme-refresh', _themeRefreshHandler);
|
||||
window.addEventListener('storage', _onStorageChange);
|
||||
var result = init(app, routeParam);
|
||||
// Install channel color picker on packets table (M2, #271)
|
||||
if (window.ChannelColorPicker) window.ChannelColorPicker.installPacketsTable();
|
||||
@@ -2587,7 +2569,6 @@
|
||||
},
|
||||
destroy: function() {
|
||||
if (_themeRefreshHandler) { window.removeEventListener('theme-refresh', _themeRefreshHandler); _themeRefreshHandler = null; }
|
||||
window.removeEventListener('storage', _onStorageChange);
|
||||
return destroy();
|
||||
}
|
||||
});
|
||||
|
||||
@@ -183,12 +183,9 @@
|
||||
// Already on map — draw directly.
|
||||
delete window._pendingPathInspectorRoute;
|
||||
if (window.routeLayer) window.routeLayer.clearLayers();
|
||||
// Pass FULL path as hopKeys (not slice(1)) — drawPacketRoute resolves
|
||||
// each entry against nodes[] for plotting. The 2nd arg is the origin
|
||||
// OBJECT (with pubkey/lat/lon/name); pass null since the origin is
|
||||
// already the first hop in the path itself, and drawPacketRoute draws
|
||||
// a marker for every resolved hop.
|
||||
if (window.drawPacketRoute) window.drawPacketRoute(candidate.path, null);
|
||||
var hops = candidate.path.slice(1);
|
||||
var origin = candidate.path[0] || null;
|
||||
if (window.drawPacketRoute) window.drawPacketRoute(hops, origin);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
+11
-19
@@ -397,17 +397,16 @@
|
||||
|
||||
// #690 — Clock Skew shared helpers
|
||||
var SKEW_SEVERITY_COLORS = {
|
||||
default: 'var(--text-muted)',
|
||||
ok: 'var(--status-green)',
|
||||
warning: 'var(--status-yellow)',
|
||||
critical: 'var(--status-orange)',
|
||||
absurd: 'var(--status-purple)',
|
||||
bimodal_clock: 'var(--status-amber)',
|
||||
no_clock: 'var(--text-muted)'
|
||||
degrading: 'var(--status-yellow)',
|
||||
degraded: 'var(--status-orange)',
|
||||
wrong: 'var(--status-red)'
|
||||
};
|
||||
var SKEW_SEVERITY_LABELS = {
|
||||
ok: 'OK', warning: 'Warning', critical: 'Critical', absurd: 'Absurd', bimodal_clock: 'Bimodal', no_clock: 'No Clock'
|
||||
default: 'Default', ok: 'OK', degrading: 'Degrading', degraded: 'Degraded', wrong: 'Wrong'
|
||||
};
|
||||
var SKEW_SEVERITY_ORDER = { no_clock: 0, bimodal_clock: 1, absurd: 2, critical: 3, warning: 4, ok: 5 };
|
||||
var SKEW_SEVERITY_ORDER = { default: 0, wrong: 1, degraded: 2, degrading: 3, ok: 4 };
|
||||
|
||||
window.SKEW_SEVERITY_COLORS = SKEW_SEVERITY_COLORS;
|
||||
window.SKEW_SEVERITY_LABELS = SKEW_SEVERITY_LABELS;
|
||||
@@ -430,26 +429,19 @@
|
||||
return (secPerDay >= 0 ? '+' : '') + secPerDay.toFixed(1) + ' s/day';
|
||||
};
|
||||
|
||||
/** Pick the skew value that drives current-health UI: prefer the
|
||||
* recent-window median (#789, current health) over the all-time median
|
||||
* (poisoned by historical bad samples). Falls back gracefully if the
|
||||
* field isn't present (older API responses). */
|
||||
/** Pick the skew value that drives current-health UI. Uses lastSkewSec
|
||||
* (most recent corrected skew) when available, falls back to medianSkewSec. */
|
||||
window.currentSkewValue = function(cs) {
|
||||
if (!cs) return null;
|
||||
return cs.recentMedianSkewSec != null ? cs.recentMedianSkewSec : cs.medianSkewSec;
|
||||
return cs.lastSkewSec != null ? cs.lastSkewSec : cs.medianSkewSec;
|
||||
};
|
||||
|
||||
/** Render a clock skew badge HTML */
|
||||
window.renderSkewBadge = function(severity, skewSec, cs) {
|
||||
if (!severity) return '';
|
||||
var cls = 'skew-badge skew-badge--' + severity;
|
||||
if (severity === 'no_clock') {
|
||||
return '<span class="' + cls + '" title="Uninitialized RTC — no valid clock">🚫 No Clock</span>';
|
||||
}
|
||||
if (severity === 'bimodal_clock' && cs) {
|
||||
var badPct = cs.goodFraction != null ? Math.round((1 - cs.goodFraction) * 100) : '?';
|
||||
var label = '⏰ ' + window.formatSkew(skewSec);
|
||||
return '<span class="' + cls + '" title="Clock skew: ' + window.formatSkew(skewSec) + ' (bimodal: ' + badPct + '% of recent adverts have nonsense timestamps)">' + label + '</span>';
|
||||
if (severity === 'default') {
|
||||
return '<span class="' + cls + '" title="Firmware default clock — volatile RTC not yet user-set since boot">⏰ Default</span>';
|
||||
}
|
||||
var label = severity === 'ok' ? '⏰' : '⏰ ' + window.formatSkew(skewSec);
|
||||
return '<span class="' + cls + '" title="Clock skew: ' + window.formatSkew(skewSec) + ' (' + (SKEW_SEVERITY_LABELS[severity] || severity) + ')">' + label + '</span>';
|
||||
|
||||
+8
-9
@@ -2293,22 +2293,21 @@ th.sort-active { color: var(--accent, #60a5fa); }
|
||||
|
||||
/* #690 — Clock Skew badges & fleet table */
|
||||
.skew-badge { display: inline-block; font-size: 10px; padding: 1px 5px; border-radius: 3px; margin-left: 4px; font-weight: 600; white-space: nowrap; }
|
||||
.skew-badge--default { background: var(--text-muted); color: #fff; }
|
||||
.skew-badge--ok { background: var(--status-green); color: #fff; }
|
||||
.skew-badge--warning { background: var(--status-yellow); color: #000; }
|
||||
.skew-badge--critical { background: var(--status-orange); color: #fff; }
|
||||
.skew-badge--absurd { background: var(--status-purple); color: #fff; }
|
||||
.skew-badge--no_clock { background: var(--text-muted); color: #fff; }
|
||||
.skew-badge--bimodal_clock { background: var(--status-amber-light); color: var(--status-amber-text); border: 1px solid var(--status-amber); }
|
||||
.skew-badge--degrading { background: var(--status-yellow); color: #000; }
|
||||
.skew-badge--degraded { background: var(--status-orange); color: #fff; }
|
||||
.skew-badge--wrong { background: var(--status-red); color: #fff; }
|
||||
|
||||
.skew-detail-section { padding: 10px 16px; margin-bottom: 8px; }
|
||||
.skew-sparkline-wrap { margin-top: 6px; }
|
||||
.skew-sparkline-wrap svg { display: block; }
|
||||
|
||||
|
||||
.clock-fleet-row--warning { background: color-mix(in srgb, var(--status-yellow) 10%, transparent); }
|
||||
.clock-fleet-row--critical { background: color-mix(in srgb, var(--status-orange) 10%, transparent); }
|
||||
.clock-fleet-row--absurd { background: color-mix(in srgb, var(--status-purple) 10%, transparent); }
|
||||
.clock-fleet-row--no_clock { background: color-mix(in srgb, var(--text-muted) 10%, transparent); }
|
||||
.clock-fleet-row--degrading { background: color-mix(in srgb, var(--status-yellow) 10%, transparent); }
|
||||
.clock-fleet-row--degraded { background: color-mix(in srgb, var(--status-orange) 10%, transparent); }
|
||||
.clock-fleet-row--wrong { background: color-mix(in srgb, var(--status-red) 10%, transparent); }
|
||||
.clock-fleet-row--default { background: color-mix(in srgb, var(--text-muted) 10%, transparent); }
|
||||
|
||||
.clock-filter-btn { font-size: 12px; padding: 3px 8px; border: 1px solid var(--border); border-radius: 4px; background: var(--card-bg, #fff); color: var(--text); cursor: pointer; margin-right: 4px; }
|
||||
.clock-filter-btn.active { background: var(--accent); color: #fff; border-color: var(--accent); }
|
||||
|
||||
@@ -2141,120 +2141,6 @@ async function run() {
|
||||
assert(isFullScreen, 'Details button should open full-screen node view');
|
||||
});
|
||||
|
||||
// === Hash color toggle E2E tests (#946) ===
|
||||
|
||||
await test('Color-by-hash toggle present on Live page, defaults ON', async () => {
|
||||
await page.goto(BASE + '#/live', { waitUntil: 'domcontentloaded' });
|
||||
// Wait until live.js has initialized the toggle (checked = true by default)
|
||||
await page.waitForFunction(() => {
|
||||
const el = document.getElementById('liveColorHashToggle');
|
||||
return el && el.checked === true;
|
||||
}, { timeout: 10000 });
|
||||
const checked = await page.$eval('#liveColorHashToggle', el => el.checked);
|
||||
assert(checked, 'Color by hash toggle should default to ON');
|
||||
});
|
||||
|
||||
await test('Color-by-hash toggle persists across reload', async () => {
|
||||
await page.goto(BASE + '#/live', { waitUntil: 'domcontentloaded' });
|
||||
await page.waitForSelector('#liveColorHashToggle', { timeout: 10000 });
|
||||
// Uncheck toggle
|
||||
await page.click('#liveColorHashToggle');
|
||||
const unchecked = await page.$eval('#liveColorHashToggle', el => !el.checked);
|
||||
assert(unchecked, 'Toggle should be OFF after click');
|
||||
// Reload
|
||||
await page.goto(BASE + '#/live', { waitUntil: 'domcontentloaded' });
|
||||
await page.waitForSelector('#liveColorHashToggle', { timeout: 10000 });
|
||||
const afterReload = await page.$eval('#liveColorHashToggle', el => !el.checked);
|
||||
assert(afterReload, 'Toggle OFF state should persist after reload');
|
||||
// Reset to ON for other tests
|
||||
await page.click('#liveColorHashToggle');
|
||||
});
|
||||
|
||||
await test('Packets table rows have border-left stripe when toggle ON', async () => {
|
||||
await page.evaluate(() => localStorage.setItem('meshcore-color-packets-by-hash', 'true'));
|
||||
// Hard reload to re-init page handler with the new toggle state.
|
||||
// page.goto with same hash URL is a no-op for re-rendering.
|
||||
await page.goto(BASE + '#/packets', { waitUntil: 'domcontentloaded' });
|
||||
await page.reload({ waitUntil: 'domcontentloaded' });
|
||||
await page.waitForSelector('table tbody tr[data-hash]', { timeout: 15000 });
|
||||
// Wait for hash stripe to be applied (inline style set during render).
|
||||
// Assert specifically 4px (per spec §2.10) so we don't false-pass on the
|
||||
// 3px channel-color highlight which is independent of this toggle.
|
||||
const hasStripe = await page.waitForFunction(() => {
|
||||
const row = document.querySelector('table tbody tr[data-hash]');
|
||||
return row && (row.getAttribute('style') || '').includes('border-left:4px');
|
||||
}, { timeout: 5000 }).then(() => true).catch(() => false);
|
||||
assert(hasStripe, 'At least one <tr> should have hash-color border-left:4px stripe when toggle ON');
|
||||
});
|
||||
|
||||
await test('Packets table rows have NO border-left stripe when toggle OFF', async () => {
|
||||
await page.evaluate(() => {
|
||||
localStorage.setItem('meshcore-color-packets-by-hash', 'false');
|
||||
});
|
||||
// Hard reload (page.goto with same hash URL no-ops — must reload to re-init
|
||||
// the page handler and re-render rows with the new toggle state).
|
||||
await page.reload({ waitUntil: 'domcontentloaded' });
|
||||
await page.waitForSelector('table tbody tr[data-hash]', { timeout: 15000 });
|
||||
await page.waitForTimeout(500);
|
||||
const noStripe = await page.evaluate(() => {
|
||||
const rows = document.querySelectorAll('table tbody tr[data-hash]');
|
||||
for (const r of rows) {
|
||||
// Hash stripe is 4px (per spec §2.10). Channel-color highlight uses
|
||||
// 3px and is independent of the hash-color toggle. Only assert no
|
||||
// 4px hash stripe is present.
|
||||
if ((r.getAttribute('style') || '').includes('border-left:4px')) return false;
|
||||
}
|
||||
return true;
|
||||
});
|
||||
assert(noStripe, 'No <tr> should have hash-color border-left:4px stripe when toggle OFF');
|
||||
// Reset
|
||||
await page.evaluate(() => localStorage.setItem('meshcore-color-packets-by-hash', 'true'));
|
||||
});
|
||||
|
||||
// --- Live feed hash-color stripe ---
|
||||
await test('Live feed items have border-left stripe when toggle ON', async () => {
|
||||
await page.evaluate(() => localStorage.setItem('meshcore-color-packets-by-hash', 'true'));
|
||||
await page.goto(BASE + '/#/live');
|
||||
await page.waitForTimeout(3000); // allow feed to populate
|
||||
const hasStripe = await page.evaluate(() => {
|
||||
const items = document.querySelectorAll('.live-feed-item');
|
||||
for (const item of items) {
|
||||
if ((item.getAttribute('style') || item.style.cssText || '').includes('border-left')) return true;
|
||||
}
|
||||
return false;
|
||||
});
|
||||
// May not have live packets in fixture — skip if no feed items
|
||||
const itemCount = await page.evaluate(() => document.querySelectorAll('.live-feed-item').length);
|
||||
if (itemCount === 0) {
|
||||
console.log(' (skipped — no live feed items in fixture)');
|
||||
return;
|
||||
}
|
||||
assert(hasStripe, 'At least one .live-feed-item should have hash-color border-left stripe when toggle ON');
|
||||
});
|
||||
|
||||
// --- Map polyline uses hash color ---
|
||||
await test('Map trace polyline uses hash-derived color when toggle ON', async () => {
|
||||
await page.evaluate(() => localStorage.setItem('meshcore-color-packets-by-hash', 'true'));
|
||||
await page.goto(BASE + '/#/live');
|
||||
await page.waitForTimeout(3000);
|
||||
// Use the dedicated .live-packet-trace class so we don't pick up
|
||||
// unrelated leaflet paths (geofilter polygons, region overlays, etc).
|
||||
const pathCount = await page.evaluate(() => document.querySelectorAll('path.live-packet-trace').length);
|
||||
if (pathCount === 0) {
|
||||
console.log(' (skipped — no live-packet-trace polylines drawn in 3s window)');
|
||||
return;
|
||||
}
|
||||
const hasHslPolyline = await page.evaluate(() => {
|
||||
const paths = document.querySelectorAll('path.live-packet-trace');
|
||||
for (const p of paths) {
|
||||
const stroke = p.getAttribute('stroke') || '';
|
||||
if (stroke.startsWith('hsl(')) return true;
|
||||
}
|
||||
return false;
|
||||
});
|
||||
assert(hasHslPolyline, 'At least one live-packet-trace polyline should have hsl() stroke color from hash');
|
||||
});
|
||||
|
||||
await browser.close();
|
||||
|
||||
// Summary
|
||||
|
||||
@@ -5904,12 +5904,11 @@ console.log('\n=== channel-decrypt.js: key derivation, MAC, parsing, storage ===
|
||||
assert.strictEqual(ctx.window.renderSkewBadge(null, 0), '');
|
||||
});
|
||||
|
||||
test('renderSkewBadge renders bimodal_clock badge with tooltip (#845)', () => {
|
||||
var cs = { goodFraction: 0.6, recentBadSampleCount: 4, recentSampleCount: 10 };
|
||||
var html = ctx.window.renderSkewBadge('bimodal_clock', -5, cs);
|
||||
assert.ok(html.includes('skew-badge--bimodal_clock'), 'should contain bimodal_clock class');
|
||||
assert.ok(html.includes('bimodal'), 'tooltip should mention bimodal');
|
||||
assert.ok(html.includes('40%'), 'tooltip should show bad percentage');
|
||||
test('renderSkewBadge renders default badge with tooltip', () => {
|
||||
var cs = {};
|
||||
var html = ctx.window.renderSkewBadge('default', 0, cs);
|
||||
assert.ok(html.includes('skew-badge--default'), 'should contain default class');
|
||||
assert.ok(html.toLowerCase().includes('firmware default'), 'tooltip should mention firmware default');
|
||||
assert.ok(html.includes('⏰'), 'should contain clock emoji');
|
||||
});
|
||||
|
||||
@@ -5933,9 +5932,9 @@ console.log('\n=== channel-decrypt.js: key derivation, MAC, parsing, storage ===
|
||||
|
||||
test('SKEW_SEVERITY_ORDER sorts worst first', () => {
|
||||
var order = ctx.window.SKEW_SEVERITY_ORDER;
|
||||
assert.ok(order.absurd < order.critical, 'absurd should sort before critical');
|
||||
assert.ok(order.critical < order.warning, 'critical should sort before warning');
|
||||
assert.ok(order.warning < order.ok, 'warning should sort before ok');
|
||||
assert.ok(order.wrong < order.degraded, 'wrong should sort before degraded');
|
||||
assert.ok(order.degraded < order.degrading, 'degraded should sort before degrading');
|
||||
assert.ok(order.degrading < order.ok, 'degrading should sort before ok');
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
@@ -1,150 +0,0 @@
|
||||
/* test-hash-color.js — Unit tests for hash-color.js (vm.createContext sandbox)
|
||||
* Tests: purity, theme split, saturation variability, lightness variability,
|
||||
* outline darker than fill, sentinel, perceptual distance
|
||||
*/
|
||||
'use strict';
|
||||
const vm = require('vm');
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
|
||||
const src = fs.readFileSync(path.join(__dirname, 'public', 'hash-color.js'), 'utf8');
|
||||
|
||||
function createSandbox() {
|
||||
const sandbox = { window: {}, module: {} };
|
||||
vm.createContext(sandbox);
|
||||
vm.runInContext(src, sandbox);
|
||||
return sandbox.window.HashColor || sandbox.module.exports;
|
||||
}
|
||||
|
||||
const HashColor = createSandbox();
|
||||
let passed = 0;
|
||||
let failed = 0;
|
||||
|
||||
function assert(cond, msg) {
|
||||
if (cond) { passed++; console.log(' ✓ ' + msg); }
|
||||
else { failed++; console.error(' ✗ ' + msg); }
|
||||
}
|
||||
|
||||
function parseHsl(str) {
|
||||
const m = str.match(/hsl\((\d+),\s*(\d+)%,\s*(\d+)%\)/);
|
||||
if (!m) return null;
|
||||
return { h: parseInt(m[1]), s: parseInt(m[2]), l: parseInt(m[3]) };
|
||||
}
|
||||
|
||||
// --- Purity: same input → same output ---
|
||||
console.log('Purity:');
|
||||
const r1 = HashColor.hashToHsl('a1b2c3d4', 'light');
|
||||
const r2 = HashColor.hashToHsl('a1b2c3d4', 'light');
|
||||
assert(r1 === r2, 'Same hash+theme → identical output');
|
||||
const r3 = HashColor.hashToHsl('a1b2c3d4', 'light');
|
||||
assert(r1 === r3, 'Third call still identical (no internal state)');
|
||||
|
||||
// --- Theme split: light vs dark produce different L ---
|
||||
console.log('Theme split:');
|
||||
const light = HashColor.hashToHsl('ff00aa80', 'light');
|
||||
const dark = HashColor.hashToHsl('ff00aa80', 'dark');
|
||||
assert(light !== dark, 'Light and dark produce different colors for same hash');
|
||||
const lightP = parseHsl(light);
|
||||
const darkP = parseHsl(dark);
|
||||
assert(lightP.l >= 50 && lightP.l <= 65, 'Light theme L in [50,65] (got ' + lightP.l + ')');
|
||||
assert(darkP.l >= 55 && darkP.l <= 72, 'Dark theme L in [55,72] (got ' + darkP.l + ')');
|
||||
|
||||
// --- Saturation varies with byte 2 ---
|
||||
console.log('Saturation variability (byte 2):');
|
||||
const lowSat = HashColor.hashToHsl('000000ff', 'light'); // byte2=0x00
|
||||
const highSat = HashColor.hashToHsl('0000ffff', 'light'); // byte2=0xff
|
||||
const lowSatP = parseHsl(lowSat);
|
||||
const highSatP = parseHsl(highSat);
|
||||
assert(lowSatP.s === 55, 'byte2=0x00 → S=55% (got ' + lowSatP.s + ')');
|
||||
assert(highSatP.s === 95, 'byte2=0xff → S=95% (got ' + highSatP.s + ')');
|
||||
// Mid value
|
||||
const midSat = HashColor.hashToHsl('00008000', 'light'); // byte2=0x80
|
||||
const midSatP = parseHsl(midSat);
|
||||
assert(midSatP.s > 55 && midSatP.s < 95, 'byte2=0x80 → S between 55 and 95 (got ' + midSatP.s + ')');
|
||||
|
||||
// --- Lightness varies with byte 3 ---
|
||||
console.log('Lightness variability (byte 3):');
|
||||
const lowL = HashColor.hashToHsl('00000000', 'light'); // byte3=0x00
|
||||
const highL = HashColor.hashToHsl('000000ff', 'light'); // byte3=0xff
|
||||
const lowLP = parseHsl(lowL);
|
||||
const highLP = parseHsl(highL);
|
||||
assert(lowLP.l === 50, 'byte3=0x00 light → L=50 (got ' + lowLP.l + ')');
|
||||
assert(highLP.l === 65, 'byte3=0xff light → L=65 (got ' + highLP.l + ')');
|
||||
const lowLD = HashColor.hashToHsl('00000000', 'dark');
|
||||
const highLD = HashColor.hashToHsl('000000ff', 'dark');
|
||||
assert(parseHsl(lowLD).l === 55, 'byte3=0x00 dark → L=55 (got ' + parseHsl(lowLD).l + ')');
|
||||
assert(parseHsl(highLD).l === 72, 'byte3=0xff dark → L=72 (got ' + parseHsl(highLD).l + ')');
|
||||
|
||||
// --- Outline is darker than fill ---
|
||||
console.log('Outline darker than fill:');
|
||||
['a1b2c3d4', 'ff00aa80', '12345678', 'deadbeef'].forEach(h => {
|
||||
['light', 'dark'].forEach(theme => {
|
||||
const fill = parseHsl(HashColor.hashToHsl(h, theme));
|
||||
const outline = parseHsl(HashColor.hashToOutline(h, theme));
|
||||
assert(outline.l < fill.l, 'Outline L(' + outline.l + ') < Fill L(' + fill.l + ') for ' + h + '/' + theme);
|
||||
});
|
||||
});
|
||||
|
||||
// --- Outline same hue as fill ---
|
||||
console.log('Outline same hue as fill:');
|
||||
['a1b2c3d4', 'deadbeef'].forEach(h => {
|
||||
const fill = parseHsl(HashColor.hashToHsl(h, 'light'));
|
||||
const outline = parseHsl(HashColor.hashToOutline(h, 'light'));
|
||||
assert(fill.h === outline.h, 'Hue matches: fill=' + fill.h + ' outline=' + outline.h + ' for ' + h);
|
||||
});
|
||||
|
||||
// --- Sentinel: null/empty/short hash ---
|
||||
console.log('Sentinel:');
|
||||
assert(HashColor.hashToHsl(null, 'light') === 'hsl(0, 0%, 50%)', 'null → sentinel');
|
||||
assert(HashColor.hashToHsl('', 'light') === 'hsl(0, 0%, 50%)', 'empty string → sentinel');
|
||||
assert(HashColor.hashToHsl('ab', 'dark') === 'hsl(0, 0%, 50%)', 'too short (2 chars) → sentinel');
|
||||
assert(HashColor.hashToHsl('abcdef', 'dark') === 'hsl(0, 0%, 50%)', '6 chars (need 8) → sentinel');
|
||||
assert(HashColor.hashToHsl(undefined, 'dark') === 'hsl(0, 0%, 50%)', 'undefined → sentinel');
|
||||
assert(HashColor.hashToOutline(null, 'light') === 'hsl(0, 0%, 30%)', 'null outline → sentinel');
|
||||
|
||||
// --- Variability: different hashes → different colors (anti-tautology) ---
|
||||
console.log('Variability (anti-tautology):');
|
||||
const colors = new Set();
|
||||
['00008080', '80008080', 'ff008080', '00ff8080', 'ffff8080'].forEach(h => {
|
||||
colors.add(HashColor.hashToHsl(h, 'light'));
|
||||
});
|
||||
assert(colors.size >= 4, 'At least 4 distinct colors from 5 different hashes (got ' + colors.size + ')');
|
||||
|
||||
// Adjacent hashes differ
|
||||
const c1 = HashColor.hashToHsl('01008080', 'light');
|
||||
const c2 = HashColor.hashToHsl('02008080', 'light');
|
||||
assert(c1 !== c2, 'Adjacent hashes produce different colors');
|
||||
|
||||
// --- Perceptual distance: sample 50 hashes, compute pairwise HSL distance ---
|
||||
console.log('Perceptual distance (50 sample hashes):');
|
||||
function hslDistance(a, b) {
|
||||
// Simple cylindrical distance: weight hue wrap, sat, lightness
|
||||
var dh = Math.min(Math.abs(a.h - b.h), 360 - Math.abs(a.h - b.h)) / 180; // 0-1
|
||||
var ds = Math.abs(a.s - b.s) / 100; // 0-1
|
||||
var dl = Math.abs(a.l - b.l) / 100; // 0-1
|
||||
return Math.sqrt(dh*dh + ds*ds + dl*dl);
|
||||
}
|
||||
|
||||
const deterministicHashes = [];
|
||||
for (var i = 0; i < 50; i++) {
|
||||
var hex = ('0000000' + (i * 5347 + 12345).toString(16)).slice(-8);
|
||||
deterministicHashes.push(hex);
|
||||
}
|
||||
|
||||
const parsedColors = deterministicHashes.map(h => parseHsl(HashColor.hashToHsl(h, 'light')));
|
||||
var distances = [];
|
||||
for (var i = 0; i < parsedColors.length; i++) {
|
||||
for (var j = i + 1; j < parsedColors.length; j++) {
|
||||
distances.push(hslDistance(parsedColors[i], parsedColors[j]));
|
||||
}
|
||||
}
|
||||
var avgDist = distances.reduce((a, b) => a + b, 0) / distances.length;
|
||||
var minDist = Math.min(...distances);
|
||||
console.log(' Avg pairwise HSL distance: ' + avgDist.toFixed(4));
|
||||
console.log(' Min pairwise HSL distance: ' + minDist.toFixed(4));
|
||||
assert(avgDist > 0.15, 'Average pairwise distance > 0.15 (got ' + avgDist.toFixed(4) + ')');
|
||||
assert(minDist > 0.01, 'Min pairwise distance > 0.01 (got ' + minDist.toFixed(4) + ')');
|
||||
|
||||
// --- Summary ---
|
||||
console.log('\n' + passed + ' passed, ' + failed + ' failed');
|
||||
if (failed > 0) process.exit(1);
|
||||
@@ -928,56 +928,6 @@ console.log('\n=== live.js: source-level safety checks ===');
|
||||
});
|
||||
}
|
||||
|
||||
// ===== Node filter (M3 — #771) =====
|
||||
console.log('\n=== live.js: node filter ===');
|
||||
{
|
||||
const ctx = makeLiveSandbox();
|
||||
const pktInvolvesFilter = ctx.window._livePacketInvolvesFilterNode;
|
||||
assert.ok(pktInvolvesFilter, '_livePacketInvolvesFilterNode must be exposed');
|
||||
|
||||
const makePkt = (hops) => ({ decoded: { path: { hops }, payload: {} } });
|
||||
|
||||
test('packetInvolvesFilterNode returns true when filter is empty', () => {
|
||||
assert.strictEqual(pktInvolvesFilter(makePkt(['abcd1234']), []), true);
|
||||
});
|
||||
|
||||
test('packetInvolvesFilterNode matches hop by prefix', () => {
|
||||
assert.strictEqual(pktInvolvesFilter(makePkt(['abcd1234', 'ef012345']), ['abcd1234567890ab']), true);
|
||||
});
|
||||
|
||||
test('packetInvolvesFilterNode matches full key against short hop', () => {
|
||||
assert.strictEqual(pktInvolvesFilter(makePkt(['abcd']), ['abcd1234567890ab']), true);
|
||||
});
|
||||
|
||||
test('packetInvolvesFilterNode returns false when no hop matches', () => {
|
||||
assert.strictEqual(pktInvolvesFilter(makePkt(['ffff1234', '00001111']), ['abcd1234567890ab']), false);
|
||||
});
|
||||
|
||||
test('packetInvolvesFilterNode matches any of multiple filter keys (OR logic)', () => {
|
||||
assert.strictEqual(pktInvolvesFilter(makePkt(['ffff0000']), ['abcd1234', 'ffff0000']), true);
|
||||
});
|
||||
|
||||
test('packetInvolvesFilterNode returns false for packet with no hops', () => {
|
||||
assert.strictEqual(pktInvolvesFilter(makePkt([]), ['abcd1234']), false);
|
||||
});
|
||||
|
||||
const getNodeFilterKeys = ctx.window._liveGetNodeFilterKeys;
|
||||
assert.ok(getNodeFilterKeys, '_liveGetNodeFilterKeys must be exposed');
|
||||
|
||||
test('node filter defaults to empty array when localStorage is unset', () => {
|
||||
assert.strictEqual(getNodeFilterKeys().length, 0);
|
||||
});
|
||||
|
||||
test('node filter saves to localStorage when set', () => {
|
||||
const setFilter = ctx.window._liveSetNodeFilter;
|
||||
assert.ok(setFilter, '_liveSetNodeFilter must be exposed');
|
||||
setFilter(['abcd1234', 'ef012345']);
|
||||
assert.strictEqual(ctx.localStorage.getItem('live-node-filter'), 'abcd1234,ef012345');
|
||||
setFilter([]);
|
||||
assert.strictEqual(ctx.localStorage.getItem('live-node-filter'), '');
|
||||
});
|
||||
}
|
||||
|
||||
// ===== SUMMARY =====
|
||||
Promise.allSettled(pendingTests).then(() => {
|
||||
console.log(`\n${'═'.repeat(40)}`);
|
||||
|
||||
Reference in New Issue
Block a user