mirror of
https://github.com/Kpa-clawbot/meshcore-analyzer.git
synced 2026-03-30 14:45:52 +00:00
Compare commits
10 Commits
rename/cor
...
fix/go-pac
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
db1122ef4b | ||
|
|
3f54632b07 | ||
|
|
609b12541e | ||
|
|
4369e58a3c | ||
|
|
8ef321bf70 | ||
|
|
bee705d5d8 | ||
|
|
9b2ad91512 | ||
|
|
d538d2f3e7 | ||
|
|
1453fb6492 | ||
|
|
5cc6064e11 |
144
RELEASE-v3.1.0.md
Normal file
144
RELEASE-v3.1.0.md
Normal file
@@ -0,0 +1,144 @@
|
||||
# v3.1.0 — Now It's CoreScope
|
||||
|
||||
MeshCore Analyzer has a new name: **CoreScope**. Same mesh analysis you rely on, sharper identity, and a boatload of fixes and performance wins since v3.0.0.
|
||||
|
||||
48 commits, 30+ issues closed. Here's what changed.
|
||||
|
||||
---
|
||||
|
||||
## 🏷️ Renamed to CoreScope
|
||||
|
||||
The project is now **CoreScope** — frontend, backend, Docker images, manage.sh, docs, CI — everything has been updated. The URL, the API, the database, and your config all stay the same. Just a better name for the tool the community built.
|
||||
|
||||
---
|
||||
|
||||
## ⚡ Performance
|
||||
|
||||
| What | Before | After |
|
||||
|------|--------|-------|
|
||||
| Subpath analytics | 900 ms | **5 ms** (precomputed at ingest) |
|
||||
| Distance analytics | 1.2 s | **15 ms** (precomputed at ingest) |
|
||||
| Packet ingest (prepend) | O(n) slice copy | **O(1) append** |
|
||||
| Go runtime stats | GC stop-the-world on every call | **cached ReadMemStats** |
|
||||
| All analytics endpoints | computed per-request | **TTL-cached** |
|
||||
|
||||
The in-memory store now precomputes subpaths and distance data as packets arrive, eliminating expensive full-table scans on the analytics endpoints. The O(n) slice prepend on every ingest — the single hottest line in the server — is gone. `ReadMemStats` calls are cached to prevent GC pause spikes under load.
|
||||
|
||||
---
|
||||
|
||||
## 🆕 New Features
|
||||
|
||||
### Telemetry Decode
|
||||
Sensor nodes now report **battery voltage** and **temperature** parsed from advert payloads. Telemetry is gated on the sensor flag — only real sensors emit data, and 0°C is no longer falsely reported. Safe migration with `PRAGMA` column checks.
|
||||
|
||||
### Channel Decryption for Custom Channels
|
||||
The `hashChannels` config now works in the Go ingestor. Key derivation has been ported from Node.js with full AES-128-ECB support and garbage text detection — wrong keys silently fail instead of producing garbled output.
|
||||
|
||||
### Node Pruning
|
||||
Stale nodes are automatically moved to an `inactive_nodes` table after the configurable retention window. Pruning runs hourly. Your active node list stays clean. (#202)
|
||||
|
||||
### Duplicate Node Name Badges
|
||||
Nodes with the same display name but different public keys are flagged with a badge so you can spot collisions instantly.
|
||||
|
||||
### Sortable Channels Table
|
||||
Channel columns are now sortable with click-to-sort headers. Sort preferences persist in `localStorage` across sessions. (#167)
|
||||
|
||||
### Go Runtime Metrics
|
||||
The performance page exposes goroutine count, heap allocation, GC pause percentiles, and memory breakdown when connected to a Go backend.
|
||||
|
||||
---
|
||||
|
||||
## 🐛 Bug Fixes
|
||||
|
||||
- **Channel decryption regression** (#176) — full AES-128-ECB in Go, garbage text detection, hashChannels key derivation ported correctly (#218)
|
||||
- **Packets page not live-updating** (#172) — WebSocket broadcast now includes the nested packet object and timestamp fields the frontend expects; multiple fixes across broadcast and render paths
|
||||
- **Node detail page crashes** (#190) — `Number()` casts and `Array.isArray` guards prevent rendering errors on unexpected data shapes
|
||||
- **Observation count staleness** (#174) — trace page and packet detail now show correct observation counts
|
||||
- **Phantom node cleanup** (#133) — `autoLearnHopNodes` no longer creates fake nodes from 1-byte repeater IDs
|
||||
- **Advert count inflation** (#200) — counts unique transmissions, not total observations (8 observers × 1 advert = 1, not 8)
|
||||
- **SQLite BUSY contention** (#214) — `MaxOpenConns(1)` + `MaxIdleConns(1)` serializes writes; load-tested under concurrent ingest
|
||||
- **Decoder bounds check** (#183) — corrupt/malformed packets no longer crash the decoder with buffer overruns
|
||||
- **noise_floor / battery_mv type mismatches** — consistent `float64` scanning handles SQLite REAL values correctly
|
||||
- **packetsLastHour always zero** (#182) — early `break` in observer loop prevented counting
|
||||
- **Channels stale messages** (#171) — latest message sorted by observation timestamp, not first-seen
|
||||
- **pprof port conflict** — non-fatal bind with separate ports prevents Go server crash on startup
|
||||
|
||||
---
|
||||
|
||||
## ♿ Accessibility & 📱 Mobile
|
||||
|
||||
### WCAG AA Compliance (10 fixes)
|
||||
- Search results keyboard-accessible with `tabindex`, `role`, and arrow-key navigation (#208)
|
||||
- 40+ table headers given `scope` attributes (#211)
|
||||
- 9 Chart.js canvases given accessible names (#210)
|
||||
- Form inputs in customizer/filters paired with labels (#212)
|
||||
|
||||
### Mobile Responsive
|
||||
- **Live page**: bottom-sheet panel instead of full-screen overlay (#203)
|
||||
- **Perf page**: responsive layout with stacked cards (#204)
|
||||
- **Nodes table**: column hiding at narrow viewports (#205)
|
||||
- **Analytics/Compare**: horizontal scroll wrappers (#206)
|
||||
- **VCR bar**: 44px minimum touch targets (#207)
|
||||
|
||||
---
|
||||
|
||||
## 🏗️ Infrastructure
|
||||
|
||||
### manage.sh Refactored (#230)
|
||||
`manage.sh` is now a thin wrapper around `docker compose` — no custom container management, no divergent logic. It reads `.env` for data paths, matching how `docker-compose.yml` works. One source of truth.
|
||||
|
||||
### .env Support
|
||||
Data directory, ports, and image tags are configured via `.env`. Both `docker compose` and `manage.sh` read the same file.
|
||||
|
||||
### Branch Protection & CI on PRs
|
||||
- Branch protection enabled on `master` — CI must pass, PRs required
|
||||
- CI now triggers on `pull_request`, not just `push` — catch failures before merge (#199)
|
||||
|
||||
### Protobuf API Contract
|
||||
10 `.proto` files, 33 golden fixtures, CI validation on every push. API shape drift is caught automatically.
|
||||
|
||||
### pprof Profiling
|
||||
Controlled by `ENABLE_PPROF` env var. When enabled, exposes Go profiling endpoints on separate ports — zero overhead when off.
|
||||
|
||||
### Test Coverage
|
||||
- Go backend: **92%+** coverage
|
||||
- **49 Playwright E2E tests**
|
||||
- Both tracks gate deploy in CI
|
||||
|
||||
---
|
||||
|
||||
## 📦 Upgrading
|
||||
|
||||
```bash
|
||||
git pull
|
||||
./manage.sh stop
|
||||
./manage.sh setup
|
||||
```
|
||||
|
||||
That's it. Your existing `config.json` and database work as-is. The rename is cosmetic — no schema changes, no API changes, no config changes.
|
||||
|
||||
### Verify
|
||||
|
||||
```bash
|
||||
curl -s http://localhost/api/health | grep engine
|
||||
# "engine": "go"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ Breaking Changes
|
||||
|
||||
**None.** All API endpoints, WebSocket messages, and config options are backwards-compatible. The rename affects branding only — Docker image names, page titles, and documentation.
|
||||
|
||||
---
|
||||
|
||||
## 🙏 Thank You
|
||||
|
||||
- **efiten** — PR #222 performance fix (O(n) slice prepend elimination)
|
||||
- **jade-on-mesh**, **lincomatic**, **LitBomb**, **mibzzer15** — ongoing testing, feedback, and issue reports
|
||||
|
||||
And to everyone running CoreScope on their mesh networks — your real-world data drives every fix and feature in this release. 48 commits since v3.0.0, and every one of them came from something the community found, reported, or requested.
|
||||
|
||||
---
|
||||
|
||||
*Previous release: [v3.0.0](RELEASE-v3.0.0.md)*
|
||||
@@ -72,8 +72,8 @@ type Header struct {
|
||||
|
||||
// TransportCodes are present on TRANSPORT_FLOOD and TRANSPORT_DIRECT routes.
|
||||
type TransportCodes struct {
|
||||
NextHop string `json:"nextHop"`
|
||||
LastHop string `json:"lastHop"`
|
||||
Code1 string `json:"code1"`
|
||||
Code2 string `json:"code2"`
|
||||
}
|
||||
|
||||
// Path holds decoded path/hop information.
|
||||
@@ -92,6 +92,8 @@ type AdvertFlags struct {
|
||||
Room bool `json:"room"`
|
||||
Sensor bool `json:"sensor"`
|
||||
HasLocation bool `json:"hasLocation"`
|
||||
HasFeat1 bool `json:"hasFeat1"`
|
||||
HasFeat2 bool `json:"hasFeat2"`
|
||||
HasName bool `json:"hasName"`
|
||||
}
|
||||
|
||||
@@ -111,6 +113,8 @@ type Payload struct {
|
||||
Lat *float64 `json:"lat,omitempty"`
|
||||
Lon *float64 `json:"lon,omitempty"`
|
||||
Name string `json:"name,omitempty"`
|
||||
Feat1 *int `json:"feat1,omitempty"`
|
||||
Feat2 *int `json:"feat2,omitempty"`
|
||||
BatteryMv *int `json:"battery_mv,omitempty"`
|
||||
TemperatureC *float64 `json:"temperature_c,omitempty"`
|
||||
ChannelHash int `json:"channelHash,omitempty"`
|
||||
@@ -123,6 +127,8 @@ type Payload struct {
|
||||
EphemeralPubKey string `json:"ephemeralPubKey,omitempty"`
|
||||
PathData string `json:"pathData,omitempty"`
|
||||
Tag uint32 `json:"tag,omitempty"`
|
||||
AuthCode uint32 `json:"authCode,omitempty"`
|
||||
TraceFlags *int `json:"traceFlags,omitempty"`
|
||||
RawHex string `json:"raw,omitempty"`
|
||||
Error string `json:"error,omitempty"`
|
||||
}
|
||||
@@ -199,14 +205,13 @@ func decodeEncryptedPayload(typeName string, buf []byte) Payload {
|
||||
}
|
||||
|
||||
func decodeAck(buf []byte) Payload {
|
||||
if len(buf) < 6 {
|
||||
if len(buf) < 4 {
|
||||
return Payload{Type: "ACK", Error: "too short", RawHex: hex.EncodeToString(buf)}
|
||||
}
|
||||
checksum := binary.LittleEndian.Uint32(buf[0:4])
|
||||
return Payload{
|
||||
Type: "ACK",
|
||||
DestHash: hex.EncodeToString(buf[0:1]),
|
||||
SrcHash: hex.EncodeToString(buf[1:2]),
|
||||
ExtraHash: hex.EncodeToString(buf[2:6]),
|
||||
ExtraHash: fmt.Sprintf("%08x", checksum),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -231,6 +236,8 @@ func decodeAdvert(buf []byte) Payload {
|
||||
if len(appdata) > 0 {
|
||||
flags := appdata[0]
|
||||
advType := int(flags & 0x0F)
|
||||
hasFeat1 := flags&0x20 != 0
|
||||
hasFeat2 := flags&0x40 != 0
|
||||
p.Flags = &AdvertFlags{
|
||||
Raw: int(flags),
|
||||
Type: advType,
|
||||
@@ -239,6 +246,8 @@ func decodeAdvert(buf []byte) Payload {
|
||||
Room: advType == 3,
|
||||
Sensor: advType == 4,
|
||||
HasLocation: flags&0x10 != 0,
|
||||
HasFeat1: hasFeat1,
|
||||
HasFeat2: hasFeat2,
|
||||
HasName: flags&0x80 != 0,
|
||||
}
|
||||
|
||||
@@ -252,6 +261,16 @@ func decodeAdvert(buf []byte) Payload {
|
||||
p.Lon = &lon
|
||||
off += 8
|
||||
}
|
||||
if hasFeat1 && len(appdata) >= off+2 {
|
||||
feat1 := int(binary.LittleEndian.Uint16(appdata[off : off+2]))
|
||||
p.Feat1 = &feat1
|
||||
off += 2
|
||||
}
|
||||
if hasFeat2 && len(appdata) >= off+2 {
|
||||
feat2 := int(binary.LittleEndian.Uint16(appdata[off : off+2]))
|
||||
p.Feat2 = &feat2
|
||||
off += 2
|
||||
}
|
||||
if p.Flags.HasName {
|
||||
// Find null terminator to separate name from trailing telemetry bytes
|
||||
nameEnd := len(appdata)
|
||||
@@ -469,15 +488,22 @@ func decodePathPayload(buf []byte) Payload {
|
||||
}
|
||||
|
||||
func decodeTrace(buf []byte) Payload {
|
||||
if len(buf) < 12 {
|
||||
if len(buf) < 9 {
|
||||
return Payload{Type: "TRACE", Error: "too short", RawHex: hex.EncodeToString(buf)}
|
||||
}
|
||||
return Payload{
|
||||
Type: "TRACE",
|
||||
DestHash: hex.EncodeToString(buf[5:11]),
|
||||
SrcHash: hex.EncodeToString(buf[11:12]),
|
||||
Tag: binary.LittleEndian.Uint32(buf[1:5]),
|
||||
tag := binary.LittleEndian.Uint32(buf[0:4])
|
||||
authCode := binary.LittleEndian.Uint32(buf[4:8])
|
||||
flags := int(buf[8])
|
||||
p := Payload{
|
||||
Type: "TRACE",
|
||||
Tag: tag,
|
||||
AuthCode: authCode,
|
||||
TraceFlags: &flags,
|
||||
}
|
||||
if len(buf) > 9 {
|
||||
p.PathData = hex.EncodeToString(buf[9:])
|
||||
}
|
||||
return p
|
||||
}
|
||||
|
||||
func decodePayload(payloadType int, buf []byte, channelKeys map[string]string) Payload {
|
||||
@@ -520,8 +546,7 @@ func DecodePacket(hexString string, channelKeys map[string]string) (*DecodedPack
|
||||
}
|
||||
|
||||
header := decodeHeader(buf[0])
|
||||
pathByte := buf[1]
|
||||
offset := 2
|
||||
offset := 1
|
||||
|
||||
var tc *TransportCodes
|
||||
if isTransportRoute(header.RouteType) {
|
||||
@@ -529,12 +554,18 @@ func DecodePacket(hexString string, channelKeys map[string]string) (*DecodedPack
|
||||
return nil, fmt.Errorf("packet too short for transport codes")
|
||||
}
|
||||
tc = &TransportCodes{
|
||||
NextHop: strings.ToUpper(hex.EncodeToString(buf[offset : offset+2])),
|
||||
LastHop: strings.ToUpper(hex.EncodeToString(buf[offset+2 : offset+4])),
|
||||
Code1: strings.ToUpper(hex.EncodeToString(buf[offset : offset+2])),
|
||||
Code2: strings.ToUpper(hex.EncodeToString(buf[offset+2 : offset+4])),
|
||||
}
|
||||
offset += 4
|
||||
}
|
||||
|
||||
if offset >= len(buf) {
|
||||
return nil, fmt.Errorf("packet too short (no path byte)")
|
||||
}
|
||||
pathByte := buf[offset]
|
||||
offset++
|
||||
|
||||
path, bytesConsumed := decodePath(pathByte, buf, offset)
|
||||
offset += bytesConsumed
|
||||
|
||||
@@ -562,16 +593,24 @@ func ComputeContentHash(rawHex string) string {
|
||||
return rawHex
|
||||
}
|
||||
|
||||
pathByte := buf[1]
|
||||
headerByte := buf[0]
|
||||
offset := 1
|
||||
if isTransportRoute(int(headerByte & 0x03)) {
|
||||
offset += 4
|
||||
}
|
||||
if offset >= len(buf) {
|
||||
if len(rawHex) >= 16 {
|
||||
return rawHex[:16]
|
||||
}
|
||||
return rawHex
|
||||
}
|
||||
pathByte := buf[offset]
|
||||
offset++
|
||||
hashSize := int((pathByte>>6)&0x3) + 1
|
||||
hashCount := int(pathByte & 0x3F)
|
||||
pathBytes := hashSize * hashCount
|
||||
|
||||
headerByte := buf[0]
|
||||
payloadStart := 2 + pathBytes
|
||||
if isTransportRoute(int(headerByte & 0x03)) {
|
||||
payloadStart += 4
|
||||
}
|
||||
payloadStart := offset + pathBytes
|
||||
if payloadStart > len(buf) {
|
||||
if len(rawHex) >= 16 {
|
||||
return rawHex[:16]
|
||||
|
||||
@@ -129,7 +129,8 @@ func TestDecodePath3ByteHashes(t *testing.T) {
|
||||
|
||||
func TestTransportCodes(t *testing.T) {
|
||||
// Route type 0 (TRANSPORT_FLOOD) should have transport codes
|
||||
hex := "1400" + "AABB" + "CCDD" + "1A" + strings.Repeat("00", 10)
|
||||
// Firmware order: header + transport_codes(4) + path_len + path + payload
|
||||
hex := "14" + "AABB" + "CCDD" + "00" + strings.Repeat("00", 10)
|
||||
pkt, err := DecodePacket(hex, nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
@@ -140,11 +141,11 @@ func TestTransportCodes(t *testing.T) {
|
||||
if pkt.TransportCodes == nil {
|
||||
t.Fatal("transportCodes should not be nil for TRANSPORT_FLOOD")
|
||||
}
|
||||
if pkt.TransportCodes.NextHop != "AABB" {
|
||||
t.Errorf("nextHop=%s, want AABB", pkt.TransportCodes.NextHop)
|
||||
if pkt.TransportCodes.Code1 != "AABB" {
|
||||
t.Errorf("code1=%s, want AABB", pkt.TransportCodes.Code1)
|
||||
}
|
||||
if pkt.TransportCodes.LastHop != "CCDD" {
|
||||
t.Errorf("lastHop=%s, want CCDD", pkt.TransportCodes.LastHop)
|
||||
if pkt.TransportCodes.Code2 != "CCDD" {
|
||||
t.Errorf("code2=%s, want CCDD", pkt.TransportCodes.Code2)
|
||||
}
|
||||
|
||||
// Route type 1 (FLOOD) should NOT have transport codes
|
||||
@@ -537,10 +538,11 @@ func TestDecodeTraceShort(t *testing.T) {
|
||||
|
||||
func TestDecodeTraceValid(t *testing.T) {
|
||||
buf := make([]byte, 16)
|
||||
buf[0] = 0x00
|
||||
buf[1] = 0x01 // tag LE uint32 = 1
|
||||
buf[5] = 0xAA // destHash start
|
||||
buf[11] = 0xBB
|
||||
// tag(4) + authCode(4) + flags(1) + pathData
|
||||
binary.LittleEndian.PutUint32(buf[0:4], 1) // tag = 1
|
||||
binary.LittleEndian.PutUint32(buf[4:8], 0xDEADBEEF) // authCode
|
||||
buf[8] = 0x02 // flags
|
||||
buf[9] = 0xAA // path data
|
||||
p := decodeTrace(buf)
|
||||
if p.Error != "" {
|
||||
t.Errorf("unexpected error: %s", p.Error)
|
||||
@@ -548,9 +550,18 @@ func TestDecodeTraceValid(t *testing.T) {
|
||||
if p.Tag != 1 {
|
||||
t.Errorf("tag=%d, want 1", p.Tag)
|
||||
}
|
||||
if p.AuthCode != 0xDEADBEEF {
|
||||
t.Errorf("authCode=%d, want 0xDEADBEEF", p.AuthCode)
|
||||
}
|
||||
if p.TraceFlags == nil || *p.TraceFlags != 2 {
|
||||
t.Errorf("traceFlags=%v, want 2", p.TraceFlags)
|
||||
}
|
||||
if p.Type != "TRACE" {
|
||||
t.Errorf("type=%s, want TRACE", p.Type)
|
||||
}
|
||||
if p.PathData == "" {
|
||||
t.Error("pathData should not be empty")
|
||||
}
|
||||
}
|
||||
|
||||
func TestDecodeAdvertShort(t *testing.T) {
|
||||
@@ -833,10 +844,9 @@ func TestComputeContentHashShortHex(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestComputeContentHashTransportRoute(t *testing.T) {
|
||||
// Route type 0 (TRANSPORT_FLOOD) with no path hops + 4 transport code bytes
|
||||
// header=0x14 (TRANSPORT_FLOOD, ADVERT), path=0x00 (0 hops)
|
||||
// transport codes = 4 bytes, then payload
|
||||
hex := "1400" + "AABBCCDD" + strings.Repeat("EE", 10)
|
||||
// Route type 0 (TRANSPORT_FLOOD) with transport codes then path=0x00 (0 hops)
|
||||
// header=0x14 (TRANSPORT_FLOOD, ADVERT), transport(4), path=0x00
|
||||
hex := "14" + "AABBCCDD" + "00" + strings.Repeat("EE", 10)
|
||||
hash := ComputeContentHash(hex)
|
||||
if len(hash) != 16 {
|
||||
t.Errorf("hash length=%d, want 16", len(hash))
|
||||
@@ -870,12 +880,10 @@ func TestComputeContentHashPayloadBeyondBufferLongHex(t *testing.T) {
|
||||
|
||||
func TestComputeContentHashTransportBeyondBuffer(t *testing.T) {
|
||||
// Transport route (0x00 = TRANSPORT_FLOOD) with path claiming some bytes
|
||||
// total buffer too short for transport codes + path
|
||||
// header=0x00, pathByte=0x02 (2 hops, 1-byte hash), then only 2 more bytes
|
||||
// payloadStart = 2 + 2 + 4(transport) = 8, but buffer only 6 bytes
|
||||
hex := "0002" + "AABB" + strings.Repeat("CC", 6) // 20 chars = 10 bytes
|
||||
// header=0x00, transport(4), pathByte=0x02 (2 hops, 1-byte hash)
|
||||
// offset=1+4+1+2=8, buffer needs to be >= 8
|
||||
hex := "00" + "AABB" + "CCDD" + "02" + strings.Repeat("CC", 6) // 20 chars = 10 bytes
|
||||
hash := ComputeContentHash(hex)
|
||||
// payloadStart = 2 + 2 + 4 = 8, buffer is 10 bytes → should work
|
||||
if len(hash) != 16 {
|
||||
t.Errorf("hash length=%d, want 16", len(hash))
|
||||
}
|
||||
@@ -913,8 +921,8 @@ func TestDecodePacketWithNewlines(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestDecodePacketTransportRouteTooShort(t *testing.T) {
|
||||
// TRANSPORT_FLOOD (route=0) but only 3 bytes total → too short for transport codes
|
||||
_, err := DecodePacket("140011", nil)
|
||||
// TRANSPORT_FLOOD (route=0) but only 2 bytes total → too short for transport codes
|
||||
_, err := DecodePacket("1400", nil)
|
||||
if err == nil {
|
||||
t.Error("expected error for transport route with too-short buffer")
|
||||
}
|
||||
@@ -931,16 +939,19 @@ func TestDecodeAckShort(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestDecodeAckValid(t *testing.T) {
|
||||
buf := []byte{0xAA, 0xBB, 0xCC, 0xDD, 0xEE, 0xFF}
|
||||
buf := []byte{0xAA, 0xBB, 0xCC, 0xDD}
|
||||
p := decodeAck(buf)
|
||||
if p.Error != "" {
|
||||
t.Errorf("unexpected error: %s", p.Error)
|
||||
}
|
||||
if p.DestHash != "aa" {
|
||||
t.Errorf("destHash=%s, want aa", p.DestHash)
|
||||
if p.ExtraHash != "ddccbbaa" {
|
||||
t.Errorf("extraHash=%s, want ddccbbaa", p.ExtraHash)
|
||||
}
|
||||
if p.ExtraHash != "ccddeeff" {
|
||||
t.Errorf("extraHash=%s, want ccddeeff", p.ExtraHash)
|
||||
if p.DestHash != "" {
|
||||
t.Errorf("destHash should be empty, got %s", p.DestHash)
|
||||
}
|
||||
if p.SrcHash != "" {
|
||||
t.Errorf("srcHash should be empty, got %s", p.SrcHash)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -54,8 +54,8 @@ type Header struct {
|
||||
|
||||
// TransportCodes are present on TRANSPORT_FLOOD and TRANSPORT_DIRECT routes.
|
||||
type TransportCodes struct {
|
||||
NextHop string `json:"nextHop"`
|
||||
LastHop string `json:"lastHop"`
|
||||
Code1 string `json:"code1"`
|
||||
Code2 string `json:"code2"`
|
||||
}
|
||||
|
||||
// Path holds decoded path/hop information.
|
||||
@@ -74,6 +74,8 @@ type AdvertFlags struct {
|
||||
Room bool `json:"room"`
|
||||
Sensor bool `json:"sensor"`
|
||||
HasLocation bool `json:"hasLocation"`
|
||||
HasFeat1 bool `json:"hasFeat1"`
|
||||
HasFeat2 bool `json:"hasFeat2"`
|
||||
HasName bool `json:"hasName"`
|
||||
}
|
||||
|
||||
@@ -97,6 +99,8 @@ type Payload struct {
|
||||
EphemeralPubKey string `json:"ephemeralPubKey,omitempty"`
|
||||
PathData string `json:"pathData,omitempty"`
|
||||
Tag uint32 `json:"tag,omitempty"`
|
||||
AuthCode uint32 `json:"authCode,omitempty"`
|
||||
TraceFlags *int `json:"traceFlags,omitempty"`
|
||||
RawHex string `json:"raw,omitempty"`
|
||||
Error string `json:"error,omitempty"`
|
||||
}
|
||||
@@ -173,14 +177,13 @@ func decodeEncryptedPayload(typeName string, buf []byte) Payload {
|
||||
}
|
||||
|
||||
func decodeAck(buf []byte) Payload {
|
||||
if len(buf) < 6 {
|
||||
if len(buf) < 4 {
|
||||
return Payload{Type: "ACK", Error: "too short", RawHex: hex.EncodeToString(buf)}
|
||||
}
|
||||
checksum := binary.LittleEndian.Uint32(buf[0:4])
|
||||
return Payload{
|
||||
Type: "ACK",
|
||||
DestHash: hex.EncodeToString(buf[0:1]),
|
||||
SrcHash: hex.EncodeToString(buf[1:2]),
|
||||
ExtraHash: hex.EncodeToString(buf[2:6]),
|
||||
ExtraHash: fmt.Sprintf("%08x", checksum),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -205,6 +208,8 @@ func decodeAdvert(buf []byte) Payload {
|
||||
if len(appdata) > 0 {
|
||||
flags := appdata[0]
|
||||
advType := int(flags & 0x0F)
|
||||
hasFeat1 := flags&0x20 != 0
|
||||
hasFeat2 := flags&0x40 != 0
|
||||
p.Flags = &AdvertFlags{
|
||||
Raw: int(flags),
|
||||
Type: advType,
|
||||
@@ -213,6 +218,8 @@ func decodeAdvert(buf []byte) Payload {
|
||||
Room: advType == 3,
|
||||
Sensor: advType == 4,
|
||||
HasLocation: flags&0x10 != 0,
|
||||
HasFeat1: hasFeat1,
|
||||
HasFeat2: hasFeat2,
|
||||
HasName: flags&0x80 != 0,
|
||||
}
|
||||
|
||||
@@ -226,6 +233,12 @@ func decodeAdvert(buf []byte) Payload {
|
||||
p.Lon = &lon
|
||||
off += 8
|
||||
}
|
||||
if hasFeat1 && len(appdata) >= off+2 {
|
||||
off += 2 // skip feat1 bytes (reserved for future use)
|
||||
}
|
||||
if hasFeat2 && len(appdata) >= off+2 {
|
||||
off += 2 // skip feat2 bytes (reserved for future use)
|
||||
}
|
||||
if p.Flags.HasName {
|
||||
name := string(appdata[off:])
|
||||
name = strings.TrimRight(name, "\x00")
|
||||
@@ -276,15 +289,22 @@ func decodePathPayload(buf []byte) Payload {
|
||||
}
|
||||
|
||||
func decodeTrace(buf []byte) Payload {
|
||||
if len(buf) < 12 {
|
||||
if len(buf) < 9 {
|
||||
return Payload{Type: "TRACE", Error: "too short", RawHex: hex.EncodeToString(buf)}
|
||||
}
|
||||
return Payload{
|
||||
Type: "TRACE",
|
||||
DestHash: hex.EncodeToString(buf[5:11]),
|
||||
SrcHash: hex.EncodeToString(buf[11:12]),
|
||||
Tag: binary.LittleEndian.Uint32(buf[1:5]),
|
||||
tag := binary.LittleEndian.Uint32(buf[0:4])
|
||||
authCode := binary.LittleEndian.Uint32(buf[4:8])
|
||||
flags := int(buf[8])
|
||||
p := Payload{
|
||||
Type: "TRACE",
|
||||
Tag: tag,
|
||||
AuthCode: authCode,
|
||||
TraceFlags: &flags,
|
||||
}
|
||||
if len(buf) > 9 {
|
||||
p.PathData = hex.EncodeToString(buf[9:])
|
||||
}
|
||||
return p
|
||||
}
|
||||
|
||||
func decodePayload(payloadType int, buf []byte) Payload {
|
||||
@@ -327,8 +347,7 @@ func DecodePacket(hexString string) (*DecodedPacket, error) {
|
||||
}
|
||||
|
||||
header := decodeHeader(buf[0])
|
||||
pathByte := buf[1]
|
||||
offset := 2
|
||||
offset := 1
|
||||
|
||||
var tc *TransportCodes
|
||||
if isTransportRoute(header.RouteType) {
|
||||
@@ -336,12 +355,18 @@ func DecodePacket(hexString string) (*DecodedPacket, error) {
|
||||
return nil, fmt.Errorf("packet too short for transport codes")
|
||||
}
|
||||
tc = &TransportCodes{
|
||||
NextHop: strings.ToUpper(hex.EncodeToString(buf[offset : offset+2])),
|
||||
LastHop: strings.ToUpper(hex.EncodeToString(buf[offset+2 : offset+4])),
|
||||
Code1: strings.ToUpper(hex.EncodeToString(buf[offset : offset+2])),
|
||||
Code2: strings.ToUpper(hex.EncodeToString(buf[offset+2 : offset+4])),
|
||||
}
|
||||
offset += 4
|
||||
}
|
||||
|
||||
if offset >= len(buf) {
|
||||
return nil, fmt.Errorf("packet too short (no path byte)")
|
||||
}
|
||||
pathByte := buf[offset]
|
||||
offset++
|
||||
|
||||
path, bytesConsumed := decodePath(pathByte, buf, offset)
|
||||
offset += bytesConsumed
|
||||
|
||||
@@ -367,16 +392,24 @@ func ComputeContentHash(rawHex string) string {
|
||||
return rawHex
|
||||
}
|
||||
|
||||
pathByte := buf[1]
|
||||
headerByte := buf[0]
|
||||
offset := 1
|
||||
if isTransportRoute(int(headerByte & 0x03)) {
|
||||
offset += 4
|
||||
}
|
||||
if offset >= len(buf) {
|
||||
if len(rawHex) >= 16 {
|
||||
return rawHex[:16]
|
||||
}
|
||||
return rawHex
|
||||
}
|
||||
pathByte := buf[offset]
|
||||
offset++
|
||||
hashSize := int((pathByte>>6)&0x3) + 1
|
||||
hashCount := int(pathByte & 0x3F)
|
||||
pathBytes := hashSize * hashCount
|
||||
|
||||
headerByte := buf[0]
|
||||
payloadStart := 2 + pathBytes
|
||||
if isTransportRoute(int(headerByte & 0x03)) {
|
||||
payloadStart += 4
|
||||
}
|
||||
payloadStart := offset + pathBytes
|
||||
if payloadStart > len(buf) {
|
||||
if len(rawHex) >= 16 {
|
||||
return rawHex[:16]
|
||||
|
||||
@@ -33,6 +33,11 @@ type Server struct {
|
||||
memStatsMu sync.Mutex
|
||||
memStatsCache runtime.MemStats
|
||||
memStatsCachedAt time.Time
|
||||
|
||||
// Cached /api/stats response — recomputed at most once every 10s
|
||||
statsMu sync.Mutex
|
||||
statsCache *StatsResponse
|
||||
statsCachedAt time.Time
|
||||
}
|
||||
|
||||
// PerfStats tracks request performance.
|
||||
@@ -380,6 +385,17 @@ func (s *Server) handleHealth(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
|
||||
func (s *Server) handleStats(w http.ResponseWriter, r *http.Request) {
|
||||
const statsTTL = 10 * time.Second
|
||||
|
||||
s.statsMu.Lock()
|
||||
if s.statsCache != nil && time.Since(s.statsCachedAt) < statsTTL {
|
||||
cached := s.statsCache
|
||||
s.statsMu.Unlock()
|
||||
writeJSON(w, cached)
|
||||
return
|
||||
}
|
||||
s.statsMu.Unlock()
|
||||
|
||||
var stats *Stats
|
||||
var err error
|
||||
if s.store != nil {
|
||||
@@ -392,7 +408,7 @@ func (s *Server) handleStats(w http.ResponseWriter, r *http.Request) {
|
||||
return
|
||||
}
|
||||
counts := s.db.GetRoleCounts()
|
||||
writeJSON(w, StatsResponse{
|
||||
resp := &StatsResponse{
|
||||
TotalPackets: stats.TotalPackets,
|
||||
TotalTransmissions: &stats.TotalTransmissions,
|
||||
TotalObservations: stats.TotalObservations,
|
||||
@@ -411,7 +427,14 @@ func (s *Server) handleStats(w http.ResponseWriter, r *http.Request) {
|
||||
Companions: counts["companions"],
|
||||
Sensors: counts["sensors"],
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
s.statsMu.Lock()
|
||||
s.statsCache = resp
|
||||
s.statsCachedAt = time.Now()
|
||||
s.statsMu.Unlock()
|
||||
|
||||
writeJSON(w, resp)
|
||||
}
|
||||
|
||||
func (s *Server) handlePerf(w http.ResponseWriter, r *http.Request) {
|
||||
|
||||
@@ -98,6 +98,11 @@ type PacketStore struct {
|
||||
// computed during Load() and incrementally updated on ingest.
|
||||
distHops []distHopRecord
|
||||
distPaths []distPathRecord
|
||||
|
||||
// Cached GetNodeHashSizeInfo result — recomputed at most once every 15s
|
||||
hashSizeInfoMu sync.Mutex
|
||||
hashSizeInfoCache map[string]*hashSizeNodeInfo
|
||||
hashSizeInfoAt time.Time
|
||||
}
|
||||
|
||||
// Precomputed distance records for fast analytics aggregation.
|
||||
@@ -3722,8 +3727,26 @@ type hashSizeNodeInfo struct {
|
||||
Inconsistent bool
|
||||
}
|
||||
|
||||
// GetNodeHashSizeInfo scans advert packets to compute per-node hash size data.
|
||||
// GetNodeHashSizeInfo returns cached per-node hash size data, recomputing at most every 15s.
|
||||
func (s *PacketStore) GetNodeHashSizeInfo() map[string]*hashSizeNodeInfo {
|
||||
const ttl = 15 * time.Second
|
||||
s.hashSizeInfoMu.Lock()
|
||||
if s.hashSizeInfoCache != nil && time.Since(s.hashSizeInfoAt) < ttl {
|
||||
cached := s.hashSizeInfoCache
|
||||
s.hashSizeInfoMu.Unlock()
|
||||
return cached
|
||||
}
|
||||
s.hashSizeInfoMu.Unlock()
|
||||
result := s.computeNodeHashSizeInfo()
|
||||
s.hashSizeInfoMu.Lock()
|
||||
s.hashSizeInfoCache = result
|
||||
s.hashSizeInfoAt = time.Now()
|
||||
s.hashSizeInfoMu.Unlock()
|
||||
return result
|
||||
}
|
||||
|
||||
// computeNodeHashSizeInfo scans advert packets to compute per-node hash size data.
|
||||
func (s *PacketStore) computeNodeHashSizeInfo() map[string]*hashSizeNodeInfo {
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
|
||||
|
||||
@@ -5,9 +5,12 @@
|
||||
|
||||
services:
|
||||
prod:
|
||||
build: .
|
||||
image: corescope:latest
|
||||
container_name: corescope-prod
|
||||
restart: unless-stopped
|
||||
extra_hosts:
|
||||
- "host.docker.internal:host-gateway"
|
||||
ports:
|
||||
- "${PROD_HTTP_PORT:-80}:${PROD_HTTP_PORT:-80}"
|
||||
- "${PROD_HTTPS_PORT:-443}:${PROD_HTTPS_PORT:-443}"
|
||||
@@ -26,9 +29,12 @@ services:
|
||||
retries: 3
|
||||
|
||||
staging:
|
||||
build: .
|
||||
image: corescope:latest
|
||||
container_name: corescope-staging
|
||||
restart: unless-stopped
|
||||
extra_hosts:
|
||||
- "host.docker.internal:host-gateway"
|
||||
ports:
|
||||
- "${STAGING_HTTP_PORT:-81}:${STAGING_HTTP_PORT:-81}"
|
||||
- "${STAGING_MQTT_PORT:-1884}:1883"
|
||||
@@ -57,6 +63,8 @@ services:
|
||||
image: corescope-go:latest
|
||||
container_name: corescope-staging-go
|
||||
restart: unless-stopped
|
||||
extra_hosts:
|
||||
- "host.docker.internal:host-gateway"
|
||||
ports:
|
||||
- "${STAGING_GO_HTTP_PORT:-82}:80"
|
||||
- "${STAGING_GO_MQTT_PORT:-1885}:1883"
|
||||
|
||||
101
docs/rename-migration.md
Normal file
101
docs/rename-migration.md
Normal file
@@ -0,0 +1,101 @@
|
||||
# CoreScope Migration Guide
|
||||
|
||||
MeshCore Analyzer has been renamed to **CoreScope**. This document covers what you need to update.
|
||||
|
||||
## What Changed
|
||||
|
||||
- **Repository name**: `meshcore-analyzer` → `corescope`
|
||||
- **Docker image name**: `meshcore-analyzer:latest` → `corescope:latest`
|
||||
- **Docker container prefixes**: `meshcore-*` → `corescope-*`
|
||||
- **Default site name**: "MeshCore Analyzer" → "CoreScope"
|
||||
|
||||
## What Did NOT Change
|
||||
|
||||
- **Data directories** — `~/meshcore-data/` stays as-is
|
||||
- **Database filename** — `meshcore.db` is unchanged
|
||||
- **MQTT topics** — `meshcore/#` topics are protocol-level and unchanged
|
||||
- **Browser state** — Favorites, localStorage keys, and settings are preserved
|
||||
- **Config file format** — `config.json` structure is the same
|
||||
|
||||
---
|
||||
|
||||
## 1. Git Remote Update
|
||||
|
||||
Update your local clone to point to the new repository URL:
|
||||
|
||||
```bash
|
||||
git remote set-url origin https://github.com/Kpa-clawbot/corescope.git
|
||||
git pull
|
||||
```
|
||||
|
||||
## 2. Docker (manage.sh) Users
|
||||
|
||||
Rebuild with the new image name:
|
||||
|
||||
```bash
|
||||
./manage.sh stop
|
||||
git pull
|
||||
./manage.sh setup
|
||||
```
|
||||
|
||||
The new image is `corescope:latest`. You can clean up the old image:
|
||||
|
||||
```bash
|
||||
docker rmi meshcore-analyzer:latest
|
||||
```
|
||||
|
||||
## 3. Docker Compose Users
|
||||
|
||||
Rebuild containers with the new names:
|
||||
|
||||
```bash
|
||||
docker compose down
|
||||
git pull
|
||||
docker compose build
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
Container names change from `meshcore-*` to `corescope-*`. Old containers are removed by `docker compose down`.
|
||||
|
||||
## 4. Data Directories
|
||||
|
||||
**No action required.** The data directory `~/meshcore-data/` and database file `meshcore.db` are unchanged. Your existing data carries over automatically.
|
||||
|
||||
## 5. Config
|
||||
|
||||
If you customized `branding.siteName` in your `config.json`, update it to your preferred name. Otherwise the new default "CoreScope" applies automatically.
|
||||
|
||||
No other config keys changed.
|
||||
|
||||
## 6. MQTT
|
||||
|
||||
**No action required.** MQTT topics (`meshcore/#`) are protocol-level and are not affected by the rename.
|
||||
|
||||
## 7. Browser
|
||||
|
||||
**No action required.** Bookmarks/favorites will continue to work at the same host and port. localStorage keys are unchanged, so your settings and preferences are preserved.
|
||||
|
||||
## 8. CI/CD
|
||||
|
||||
If you have custom CI/CD pipelines that reference:
|
||||
|
||||
- The old repository URL (`meshcore-analyzer`)
|
||||
- The old Docker image name (`meshcore-analyzer:latest`)
|
||||
- Old container names (`meshcore-*`)
|
||||
|
||||
Update those references to use the new names.
|
||||
|
||||
---
|
||||
|
||||
## Summary Checklist
|
||||
|
||||
| Item | Action Required? | What to Do |
|
||||
|------|-----------------|------------|
|
||||
| Git remote | ✅ Yes | `git remote set-url origin …corescope.git` |
|
||||
| Docker image | ✅ Yes | Rebuild; optionally `docker rmi` old image |
|
||||
| Docker Compose | ✅ Yes | `docker compose down && build && up` |
|
||||
| Data directories | ❌ No | Unchanged |
|
||||
| Config | ⚠️ Maybe | Only if you customized `branding.siteName` |
|
||||
| MQTT | ❌ No | Topics unchanged |
|
||||
| Browser | ❌ No | Settings preserved |
|
||||
| CI/CD | ⚠️ Maybe | Update if referencing old repo/image names |
|
||||
Reference in New Issue
Block a user