Merge branch 'master' into fix/remove-packets-v-fallbacks

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

.env.example

@@ -1,44 +1,17 @@
 # MeshCore Analyzer — Environment Configuration
-# Copy to .env and customize. All values have sensible defaults.
+# Copy to .env and customize. All values have sensible defaults in docker-compose.yml.
 #
-# This file is read by BOTH docker compose AND manage.sh — one source of truth.
 # Each environment keeps config + data together in one directory:
 #   ~/meshcore-data/config.json, meshcore.db, Caddyfile, theme.json
 #   ~/meshcore-staging-data/config.json, meshcore.db, Caddyfile
 
 # --- Production ---
-# Data directory (database, theme, etc.)
-# Default: ~/meshcore-data
-# Used by: docker compose, manage.sh
+PROD_HTTP_PORT=80
+PROD_HTTPS_PORT=443
+PROD_MQTT_PORT=1883
 PROD_DATA_DIR=~/meshcore-data
 
-# HTTP port for web UI
-# Default: 80
-# Used by: docker compose
-PROD_HTTP_PORT=80
-
-# HTTPS port for web UI (TLS via Caddy)
-# Default: 443
-# Used by: docker compose
-PROD_HTTPS_PORT=443
-
-# MQTT port for observer connections
-# Default: 1883
-# Used by: docker compose
-PROD_MQTT_PORT=1883
-
 # --- Staging (HTTP only, no HTTPS) ---
-# Data directory
-# Default: ~/meshcore-staging-data
-# Used by: docker compose
-STAGING_DATA_DIR=~/meshcore-staging-data
-
-# HTTP port
-# Default: 81
-# Used by: docker compose
 STAGING_HTTP_PORT=81
-
-# MQTT port
-# Default: 1884
-# Used by: docker compose
 STAGING_MQTT_PORT=1884
+STAGING_DATA_DIR=~/meshcore-staging-data

RELEASE-v3.1.0.md

@@ -1,144 +0,0 @@
-# v3.1.0 — Now It's CoreScope
-
-MeshCore Analyzer has a new name: **CoreScope**. Same mesh analysis you rely on, sharper identity, and a boatload of fixes and performance wins since v3.0.0.
-
-48 commits, 30+ issues closed. Here's what changed.
-
----
-
-## 🏷️ Renamed to CoreScope
-
-The project is now **CoreScope** — frontend, backend, Docker images, manage.sh, docs, CI — everything has been updated. The URL, the API, the database, and your config all stay the same. Just a better name for the tool the community built.
-
----
-
-## ⚡ Performance
-
-| What | Before | After |
-|------|--------|-------|
-| Subpath analytics | 900 ms | **5 ms** (precomputed at ingest) |
-| Distance analytics | 1.2 s | **15 ms** (precomputed at ingest) |
-| Packet ingest (prepend) | O(n) slice copy | **O(1) append** |
-| Go runtime stats | GC stop-the-world on every call | **cached ReadMemStats** |
-| All analytics endpoints | computed per-request | **TTL-cached** |
-
-The in-memory store now precomputes subpaths and distance data as packets arrive, eliminating expensive full-table scans on the analytics endpoints. The O(n) slice prepend on every ingest — the single hottest line in the server — is gone. `ReadMemStats` calls are cached to prevent GC pause spikes under load.
-
----
-
-## 🆕 New Features
-
-### Telemetry Decode
-Sensor nodes now report **battery voltage** and **temperature** parsed from advert payloads. Telemetry is gated on the sensor flag — only real sensors emit data, and 0°C is no longer falsely reported. Safe migration with `PRAGMA` column checks.
-
-### Channel Decryption for Custom Channels
-The `hashChannels` config now works in the Go ingestor. Key derivation has been ported from Node.js with full AES-128-ECB support and garbage text detection — wrong keys silently fail instead of producing garbled output.
-
-### Node Pruning
-Stale nodes are automatically moved to an `inactive_nodes` table after the configurable retention window. Pruning runs hourly. Your active node list stays clean. (#202)
-
-### Duplicate Node Name Badges
-Nodes with the same display name but different public keys are flagged with a badge so you can spot collisions instantly.
-
-### Sortable Channels Table
-Channel columns are now sortable with click-to-sort headers. Sort preferences persist in `localStorage` across sessions. (#167)
-
-### Go Runtime Metrics
-The performance page exposes goroutine count, heap allocation, GC pause percentiles, and memory breakdown when connected to a Go backend.
-
----
-
-## 🐛 Bug Fixes
-
-- **Channel decryption regression** (#176) — full AES-128-ECB in Go, garbage text detection, hashChannels key derivation ported correctly (#218)
-- **Packets page not live-updating** (#172) — WebSocket broadcast now includes the nested packet object and timestamp fields the frontend expects; multiple fixes across broadcast and render paths
-- **Node detail page crashes** (#190) — `Number()` casts and `Array.isArray` guards prevent rendering errors on unexpected data shapes
-- **Observation count staleness** (#174) — trace page and packet detail now show correct observation counts
-- **Phantom node cleanup** (#133) — `autoLearnHopNodes` no longer creates fake nodes from 1-byte repeater IDs
-- **Advert count inflation** (#200) — counts unique transmissions, not total observations (8 observers × 1 advert = 1, not 8)
-- **SQLite BUSY contention** (#214) — `MaxOpenConns(1)` + `MaxIdleConns(1)` serializes writes; load-tested under concurrent ingest
-- **Decoder bounds check** (#183) — corrupt/malformed packets no longer crash the decoder with buffer overruns
-- **noise_floor / battery_mv type mismatches** — consistent `float64` scanning handles SQLite REAL values correctly
-- **packetsLastHour always zero** (#182) — early `break` in observer loop prevented counting
-- **Channels stale messages** (#171) — latest message sorted by observation timestamp, not first-seen
-- **pprof port conflict** — non-fatal bind with separate ports prevents Go server crash on startup
-
----
-
-## ♿ Accessibility & 📱 Mobile
-
-### WCAG AA Compliance (10 fixes)
-- Search results keyboard-accessible with `tabindex`, `role`, and arrow-key navigation (#208)
-- 40+ table headers given `scope` attributes (#211)
-- 9 Chart.js canvases given accessible names (#210)
-- Form inputs in customizer/filters paired with labels (#212)
-
-### Mobile Responsive
-- **Live page**: bottom-sheet panel instead of full-screen overlay (#203)
-- **Perf page**: responsive layout with stacked cards (#204)
-- **Nodes table**: column hiding at narrow viewports (#205)
-- **Analytics/Compare**: horizontal scroll wrappers (#206)
-- **VCR bar**: 44px minimum touch targets (#207)
-
----
-
-## 🏗️ Infrastructure
-
-### manage.sh Refactored (#230)
-`manage.sh` is now a thin wrapper around `docker compose` — no custom container management, no divergent logic. It reads `.env` for data paths, matching how `docker-compose.yml` works. One source of truth.
-
-### .env Support
-Data directory, ports, and image tags are configured via `.env`. Both `docker compose` and `manage.sh` read the same file.
-
-### Branch Protection & CI on PRs
-- Branch protection enabled on `master` — CI must pass, PRs required
-- CI now triggers on `pull_request`, not just `push` — catch failures before merge (#199)
-
-### Protobuf API Contract
-10 `.proto` files, 33 golden fixtures, CI validation on every push. API shape drift is caught automatically.
-
-### pprof Profiling
-Controlled by `ENABLE_PPROF` env var. When enabled, exposes Go profiling endpoints on separate ports — zero overhead when off.
-
-### Test Coverage
-- Go backend: **92%+** coverage
-- **49 Playwright E2E tests**
-- Both tracks gate deploy in CI
-
----
-
-## 📦 Upgrading
-
-```bash
-git pull
-./manage.sh stop
-./manage.sh setup
-```
-
-That's it. Your existing `config.json` and database work as-is. The rename is cosmetic — no schema changes, no API changes, no config changes.
-
-### Verify
-
-```bash
-curl -s http://localhost/api/health | grep engine
-# "engine": "go"
-```
-
----
-
-## ⚠️ Breaking Changes
-
-**None.** All API endpoints, WebSocket messages, and config options are backwards-compatible. The rename affects branding only — Docker image names, page titles, and documentation.
-
----
-
-## 🙏 Thank You
-
-- **efiten** — PR #222 performance fix (O(n) slice prepend elimination)
-- **jade-on-mesh**, **lincomatic**, **LitBomb**, **mibzzer15** — ongoing testing, feedback, and issue reports
-
-And to everyone running CoreScope on their mesh networks — your real-world data drives every fix and feature in this release. 48 commits since v3.0.0, and every one of them came from something the community found, reported, or requested.
-
----
-
-*Previous release: [v3.0.0](RELEASE-v3.0.0.md)*

cmd/ingestor/config.go

@@ -26,14 +26,13 @@ type MQTTLegacy struct {
 
 // Config holds the ingestor configuration, compatible with the Node.js config.json format.
 type Config struct {
-	DBPath          string            `json:"dbPath"`
-	MQTT            *MQTTLegacy       `json:"mqtt,omitempty"`
-	MQTTSources     []MQTTSource      `json:"mqttSources,omitempty"`
-	LogLevel        string            `json:"logLevel,omitempty"`
-	ChannelKeysPath string            `json:"channelKeysPath,omitempty"`
-	ChannelKeys     map[string]string `json:"channelKeys,omitempty"`
-	HashChannels    []string          `json:"hashChannels,omitempty"`
-	Retention       *RetentionConfig  `json:"retention,omitempty"`
+	DBPath         string            `json:"dbPath"`
+	MQTT           *MQTTLegacy       `json:"mqtt,omitempty"`
+	MQTTSources    []MQTTSource      `json:"mqttSources,omitempty"`
+	LogLevel       string            `json:"logLevel,omitempty"`
+	ChannelKeysPath string           `json:"channelKeysPath,omitempty"`
+	ChannelKeys    map[string]string `json:"channelKeys,omitempty"`
+	Retention      *RetentionConfig  `json:"retention,omitempty"`
 }
 
 // RetentionConfig controls how long stale nodes are kept before being moved to inactive_nodes.

cmd/ingestor/main.go

@@ -512,64 +512,34 @@ func firstNonEmpty(vals ...string) string {
 	return ""
 }
 
-// deriveHashtagChannelKey derives an AES-128 key from a channel name.
-// Same algorithm as Node.js: SHA-256(channelName) → first 32 hex chars (16 bytes).
-func deriveHashtagChannelKey(channelName string) string {
-	h := sha256.Sum256([]byte(channelName))
-	return hex.EncodeToString(h[:16])
-}
-
 // loadChannelKeys loads channel decryption keys from config and/or a JSON file.
-// Merge priority: rainbow (lowest) → derived from hashChannels → explicit config (highest).
+// Priority: CHANNEL_KEYS_PATH env var > cfg.ChannelKeysPath > channel-rainbow.json next to config.
 func loadChannelKeys(cfg *Config, configPath string) map[string]string {
 	keys := make(map[string]string)
 
-	// 1. Rainbow table keys (lowest priority)
+	// Determine file path for rainbow keys
 	keysPath := os.Getenv("CHANNEL_KEYS_PATH")
 	if keysPath == "" {
 		keysPath = cfg.ChannelKeysPath
 	}
 	if keysPath == "" {
+		// Default: look for channel-rainbow.json next to config file
 		keysPath = filepath.Join(filepath.Dir(configPath), "channel-rainbow.json")
 	}
 
-	rainbowCount := 0
 	if data, err := os.ReadFile(keysPath); err == nil {
 		var fileKeys map[string]string
 		if err := json.Unmarshal(data, &fileKeys); err == nil {
 			for k, v := range fileKeys {
 				keys[k] = v
 			}
-			rainbowCount = len(fileKeys)
-			log.Printf("Loaded %d channel keys from %s", rainbowCount, keysPath)
+			log.Printf("Loaded %d channel keys from %s", len(fileKeys), keysPath)
 		} else {
 			log.Printf("Warning: failed to parse channel keys file %s: %v", keysPath, err)
 		}
 	}
 
-	// 2. Derived keys from hashChannels (middle priority)
-	derivedCount := 0
-	for _, raw := range cfg.HashChannels {
-		trimmed := strings.TrimSpace(raw)
-		if trimmed == "" {
-			continue
-		}
-		channelName := trimmed
-		if !strings.HasPrefix(channelName, "#") {
-			channelName = "#" + channelName
-		}
-		// Skip if explicit config already has this key
-		if _, exists := cfg.ChannelKeys[channelName]; exists {
-			continue
-		}
-		keys[channelName] = deriveHashtagChannelKey(channelName)
-		derivedCount++
-	}
-	if derivedCount > 0 {
-		log.Printf("[channels] %d derived from hashChannels", derivedCount)
-	}
-
-	// 3. Explicit config keys (highest priority — overrides rainbow + derived)
+	// Merge inline config keys (override file keys)
 	for k, v := range cfg.ChannelKeys {
 		keys[k] = v
 	}

cmd/ingestor/main_test.go

@@ -3,8 +3,6 @@ package main
 import (
 	"encoding/json"
 	"math"
-	"os"
-	"path/filepath"
 	"testing"
 	"time"
 )
@@ -494,132 +492,3 @@ func TestAdvertRole(t *testing.T) {
 		})
 	}
 }
-
-func TestDeriveHashtagChannelKey(t *testing.T) {
-	// Test vectors validated against Node.js server-helpers.js
-	tests := []struct {
-		name string
-		want string
-	}{
-		{"#General", "649af2cab73ed5a890890a5485a0c004"},
-		{"#test", "9cd8fcf22a47333b591d96a2b848b73f"},
-		{"#MeshCore", "dcf73f393fa217f6b28fcec6ffc411ad"},
-	}
-	for _, tt := range tests {
-		t.Run(tt.name, func(t *testing.T) {
-			got := deriveHashtagChannelKey(tt.name)
-			if got != tt.want {
-				t.Errorf("deriveHashtagChannelKey(%q) = %q, want %q", tt.name, got, tt.want)
-			}
-		})
-	}
-
-	// Deterministic
-	k1 := deriveHashtagChannelKey("#foo")
-	k2 := deriveHashtagChannelKey("#foo")
-	if k1 != k2 {
-		t.Error("deriveHashtagChannelKey should be deterministic")
-	}
-
-	// Returns 32-char hex string (16 bytes)
-	if len(k1) != 32 {
-		t.Errorf("key length = %d, want 32", len(k1))
-	}
-
-	// Different inputs → different keys
-	k3 := deriveHashtagChannelKey("#bar")
-	if k1 == k3 {
-		t.Error("different inputs should produce different keys")
-	}
-}
-
-func TestLoadChannelKeysMergePriority(t *testing.T) {
-	dir := t.TempDir()
-	cfgPath := filepath.Join(dir, "config.json")
-
-	// Create a rainbow file with two keys: #rainbow (unique) and #override (to be overridden)
-	rainbowPath := filepath.Join(dir, "channel-rainbow.json")
-	t.Setenv("CHANNEL_KEYS_PATH", rainbowPath)
-	rainbow := map[string]string{
-		"#rainbow":  "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
-		"#override": "rainbow_value_should_be_overridden",
-	}
-	rainbowJSON, err := json.Marshal(rainbow)
-	if err != nil {
-		t.Fatal(err)
-	}
-	if err := os.WriteFile(rainbowPath, rainbowJSON, 0o644); err != nil {
-		t.Fatal(err)
-	}
-
-	cfg := &Config{
-		HashChannels: []string{"General", "#override"},
-		ChannelKeys:  map[string]string{"#override": "explicit_wins"},
-	}
-
-	keys := loadChannelKeys(cfg, cfgPath)
-
-	// Rainbow key loaded
-	if keys["#rainbow"] != "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" {
-		t.Errorf("rainbow key missing or wrong: %q", keys["#rainbow"])
-	}
-
-	// HashChannels derived #General
-	expected := deriveHashtagChannelKey("#General")
-	if keys["#General"] != expected {
-		t.Errorf("#General = %q, want %q (derived)", keys["#General"], expected)
-	}
-
-	// Explicit config wins over both rainbow and derived
-	if keys["#override"] != "explicit_wins" {
-		t.Errorf("#override = %q, want explicit_wins", keys["#override"])
-	}
-}
-
-func TestLoadChannelKeysHashChannelsNormalization(t *testing.T) {
-	t.Setenv("CHANNEL_KEYS_PATH", "")
-	dir := t.TempDir()
-	cfgPath := filepath.Join(dir, "config.json")
-
-	cfg := &Config{
-		HashChannels: []string{
-			"NoPound",       // should become #NoPound
-			"#HasPound",     // stays #HasPound
-			"  Spaced  ",   // trimmed → #Spaced
-			"",              // skipped
-		},
-	}
-
-	keys := loadChannelKeys(cfg, cfgPath)
-
-	if _, ok := keys["#NoPound"]; !ok {
-		t.Error("should derive key for #NoPound (auto-prefixed)")
-	}
-	if _, ok := keys["#HasPound"]; !ok {
-		t.Error("should derive key for #HasPound")
-	}
-	if _, ok := keys["#Spaced"]; !ok {
-		t.Error("should derive key for #Spaced (trimmed)")
-	}
-	if len(keys) != 3 {
-		t.Errorf("expected 3 keys, got %d", len(keys))
-	}
-}
-
-func TestLoadChannelKeysSkipExplicit(t *testing.T) {
-	t.Setenv("CHANNEL_KEYS_PATH", "")
-	dir := t.TempDir()
-	cfgPath := filepath.Join(dir, "config.json")
-
-	cfg := &Config{
-		HashChannels: []string{"General"},
-		ChannelKeys:  map[string]string{"#General": "my_explicit_key"},
-	}
-
-	keys := loadChannelKeys(cfg, cfgPath)
-
-	// Explicit key should win — hashChannels derivation should be skipped
-	if keys["#General"] != "my_explicit_key" {
-		t.Errorf("#General = %q, want my_explicit_key", keys["#General"])
-	}
-}

cmd/server/store.go

@@ -62,7 +62,7 @@ type StoreObs struct {
 type PacketStore struct {
 	mu            sync.RWMutex
 	db            *DB
-	packets       []*StoreTx                 // sorted by first_seen ASC (oldest first; newest at tail)
+	packets       []*StoreTx                 // sorted by first_seen DESC
 	byHash        map[string]*StoreTx        // hash → *StoreTx
 	byTxID        map[int]*StoreTx           // transmission_id → *StoreTx
 	byObsID       map[int]*StoreObs          // observation_id → *StoreObs
@@ -176,7 +176,7 @@ func (s *PacketStore) Load() error {
 			FROM transmissions t
 			LEFT JOIN observations o ON o.transmission_id = t.id
 			LEFT JOIN observers obs ON obs.rowid = o.observer_idx
-			ORDER BY t.first_seen ASC, o.timestamp DESC`
+			ORDER BY t.first_seen DESC, o.timestamp DESC`
 	} else {
 		loadSQL = `SELECT t.id, t.raw_hex, t.hash, t.first_seen, t.route_type,
 				t.payload_type, t.payload_version, t.decoded_json,
@@ -184,7 +184,7 @@ func (s *PacketStore) Load() error {
 				o.snr, o.rssi, o.score, o.path_json, o.timestamp
 			FROM transmissions t
 			LEFT JOIN observations o ON o.transmission_id = t.id
-			ORDER BY t.first_seen ASC, o.timestamp DESC`
+			ORDER BY t.first_seen DESC, o.timestamp DESC`
 	}
 
 	rows, err := s.db.conn.Query(loadSQL)
@@ -368,32 +368,28 @@ func (s *PacketStore) QueryPackets(q PacketQuery) *PacketResult {
 	results := s.filterPackets(q)
 	total := len(results)
 
-	// results is oldest-first (ASC). For DESC (default) read backwards from the tail;
-	// for ASC read forwards. Both are O(page_size) — no sort copy needed.
-	start := q.Offset
-	if start >= total {
-		return &PacketResult{Packets: []map[string]interface{}{}, Total: total}
-	}
-	pageSize := q.Limit
-	if start+pageSize > total {
-		pageSize = total - start
+	if q.Order == "ASC" {
+		sorted := make([]*StoreTx, len(results))
+		copy(sorted, results)
+		sort.Slice(sorted, func(i, j int) bool {
+			return sorted[i].FirstSeen < sorted[j].FirstSeen
+		})
+		results = sorted
 	}
 
-	packets := make([]map[string]interface{}, 0, pageSize)
-	if q.Order == "ASC" {
-		for _, tx := range results[start : start+pageSize] {
-			packets = append(packets, txToMap(tx))
-		}
-	} else {
-		// DESC: newest items are at the tail; page 0 = last pageSize items reversed
-		endIdx := total - start
-		startIdx := endIdx - pageSize
-		if startIdx < 0 {
-			startIdx = 0
-		}
-		for i := endIdx - 1; i >= startIdx; i-- {
-			packets = append(packets, txToMap(results[i]))
-		}
+	// Paginate
+	start := q.Offset
+	if start >= len(results) {
+		return &PacketResult{Packets: []map[string]interface{}{}, Total: total}
+	}
+	end := start + q.Limit
+	if end > len(results) {
+		end = len(results)
+	}
+
+	packets := make([]map[string]interface{}, 0, end-start)
+	for _, tx := range results[start:end] {
+		packets = append(packets, txToMap(tx))
 	}
 	return &PacketResult{Packets: packets, Total: total}
 }
@@ -723,16 +719,15 @@ func (s *PacketStore) GetTimestamps(since string) []string {
 	s.mu.RLock()
 	defer s.mu.RUnlock()
 
-	// packets sorted oldest-first — scan from tail until we reach items older than since
+	// packets sorted newest first — scan from start until older than since
 	var result []string
-	for i := len(s.packets) - 1; i >= 0; i-- {
-		tx := s.packets[i]
+	for _, tx := range s.packets {
 		if tx.FirstSeen <= since {
 			break
 		}
 		result = append(result, tx.FirstSeen)
 	}
-	// result is currently newest-first; reverse to return ASC order
+	// Reverse to get ASC order
 	for i, j := 0, len(result)-1; i < j; i, j = i+1, j-1 {
 		result[i], result[j] = result[j], result[i]
 	}
@@ -782,30 +777,23 @@ func (s *PacketStore) QueryMultiNodePackets(pubkeys []string, limit, offset int,
 
 	total := len(filtered)
 
-	// filtered is oldest-first (built by iterating s.packets forward).
-	// Apply same DESC/ASC pagination logic as QueryPackets.
+	if order == "ASC" {
+		sort.Slice(filtered, func(i, j int) bool {
+			return filtered[i].FirstSeen < filtered[j].FirstSeen
+		})
+	}
+
 	if offset >= total {
 		return &PacketResult{Packets: []map[string]interface{}{}, Total: total}
 	}
-	pageSize := limit
-	if offset+pageSize > total {
-		pageSize = total - offset
+	end := offset + limit
+	if end > total {
+		end = total
 	}
 
-	packets := make([]map[string]interface{}, 0, pageSize)
-	if order == "ASC" {
-		for _, tx := range filtered[offset : offset+pageSize] {
-			packets = append(packets, txToMap(tx))
-		}
-	} else {
-		endIdx := total - offset
-		startIdx := endIdx - pageSize
-		if startIdx < 0 {
-			startIdx = 0
-		}
-		for i := endIdx - 1; i >= startIdx; i-- {
-			packets = append(packets, txToMap(filtered[i]))
-		}
+	packets := make([]map[string]interface{}, 0, end-offset)
+	for _, tx := range filtered[offset:end] {
+		packets = append(packets, txToMap(tx))
 	}
 	return &PacketResult{Packets: packets, Total: total}
 }
@@ -938,14 +926,15 @@ func (s *PacketStore) IngestNewFromDB(sinceID, limit int) ([]map[string]interfac
 				DecodedJSON: r.decodedJSON,
 			}
 			s.byHash[r.hash] = tx
-			s.packets = append(s.packets, tx) // oldest-first; new items go to tail
+			// Prepend (newest first)
+			s.packets = append([]*StoreTx{tx}, s.packets...)
 			s.byTxID[r.txID] = tx
 			s.indexByNode(tx)
 			if tx.PayloadType != nil {
 				pt := *tx.PayloadType
-				// Append to maintain oldest-first order (matches Load ordering)
+				// Prepend to maintain newest-first order (matches Load ordering)
 				// so GetChannelMessages reverse iteration stays correct
-				s.byPayloadType[pt] = append(s.byPayloadType[pt], tx)
+				s.byPayloadType[pt] = append([]*StoreTx{tx}, s.byPayloadType[pt]...)
 			}
 
 			if _, exists := broadcastTxs[r.txID]; !exists {
@@ -1090,6 +1079,8 @@ func (s *PacketStore) IngestNewFromDB(sinceID, limit int) ([]map[string]interfac
 		s.cacheMu.Unlock()
 	}
 
+	log.Printf("[poller] IngestNewFromDB: found %d new txs, maxID %d->%d", len(result), sinceID, newMaxID)
+
 	return result, newMaxID
 }
 
@@ -1272,7 +1263,8 @@ func (s *PacketStore) IngestNewObservations(sinceObsID, limit int) int {
 		s.subpathCache = make(map[string]*cachedResult)
 		s.cacheMu.Unlock()
 
-		// analytics caches cleared; no per-cycle log to avoid stdout overhead
+		log.Printf("[poller] IngestNewObservations: updated %d existing txs, maxObsID %d->%d",
+			len(updatedTxs), sinceObsID, newMaxObsID)
 	}
 
 	return newMaxObsID
@@ -1896,7 +1888,7 @@ func (s *PacketStore) GetChannelMessages(channelHash string, limit, offset int)
 	msgMap := map[string]*msgEntry{}
 	var msgOrder []string
 
-	// Iterate type-5 packets oldest-first (byPayloadType is ASC = oldest first)
+	// Iterate type-5 packets oldest-first (byPayloadType is in load order = newest first)
 	type decodedMsg struct {
 		Type            string      `json:"type"`
 		Channel         string      `json:"channel"`
@@ -1907,7 +1899,8 @@ func (s *PacketStore) GetChannelMessages(channelHash string, limit, offset int)
 	}
 
 	grpTxts := s.byPayloadType[5]
-	for _, tx := range grpTxts {
+	for i := len(grpTxts) - 1; i >= 0; i-- {
+		tx := grpTxts[i]
 		if tx.DecodedJSON == "" {
 			continue
 		}
@@ -4076,13 +4069,13 @@ func (s *PacketStore) GetNodeHealth(pubkey string) (map[string]interface{}, erro
 		lhVal = lastHeard
 	}
 
-	// Recent packets (up to 20, newest first — read from tail of oldest-first slice)
+	// Recent packets (up to 20, newest first — packets are already sorted DESC)
 	recentLimit := 20
 	if len(packets) < recentLimit {
 		recentLimit = len(packets)
 	}
 	recentPackets := make([]map[string]interface{}, 0, recentLimit)
-	for i := len(packets) - 1; i >= len(packets)-recentLimit; i-- {
+	for i := 0; i < recentLimit; i++ {
 		p := txToMap(packets[i])
 		delete(p, "observations")
 		recentPackets = append(recentPackets, p)

docker-compose.yml

@@ -1,7 +1,5 @@
-# All container config lives here. manage.sh is just a wrapper around docker compose.
+# Volume paths unified with manage.sh — see manage.sh lines 9-12, 56-68, 98-113
 # Override defaults via .env or environment variables.
-# CRITICAL: All data mounts use bind mounts (~/path), NOT named volumes.
-# This ensures the DB and theme are visible on the host filesystem for backup.
 
 services:
   prod:
@@ -78,7 +76,6 @@ services:
       - staging-go
 
 volumes:
-  # Named volumes for Caddy TLS certificates (not user data — managed by Caddy internally)
   caddy-data:
   caddy-data-staging:
   caddy-data-staging-go:

docs/rename-migration.md

@@ -1,101 +0,0 @@
-# CoreScope Migration Guide
-
-MeshCore Analyzer has been renamed to **CoreScope**. This document covers what you need to update.
-
-## What Changed
-
-- **Repository name**: `meshcore-analyzer` → `corescope`
-- **Docker image name**: `meshcore-analyzer:latest` → `corescope:latest`
-- **Docker container prefixes**: `meshcore-*` → `corescope-*`
-- **Default site name**: "MeshCore Analyzer" → "CoreScope"
-
-## What Did NOT Change
-
-- **Data directories** — `~/meshcore-data/` stays as-is
-- **Database filename** — `meshcore.db` is unchanged
-- **MQTT topics** — `meshcore/#` topics are protocol-level and unchanged
-- **Browser state** — Favorites, localStorage keys, and settings are preserved
-- **Config file format** — `config.json` structure is the same
-
----
-
-## 1. Git Remote Update
-
-Update your local clone to point to the new repository URL:
-
-```bash
-git remote set-url origin https://github.com/Kpa-clawbot/corescope.git
-git pull
-```
-
-## 2. Docker (manage.sh) Users
-
-Rebuild with the new image name:
-
-```bash
-./manage.sh stop
-git pull
-./manage.sh setup
-```
-
-The new image is `corescope:latest`. You can clean up the old image:
-
-```bash
-docker rmi meshcore-analyzer:latest
-```
-
-## 3. Docker Compose Users
-
-Rebuild containers with the new names:
-
-```bash
-docker compose down
-git pull
-docker compose build
-docker compose up -d
-```
-
-Container names change from `meshcore-*` to `corescope-*`. Old containers are removed by `docker compose down`.
-
-## 4. Data Directories
-
-**No action required.** The data directory `~/meshcore-data/` and database file `meshcore.db` are unchanged. Your existing data carries over automatically.
-
-## 5. Config
-
-If you customized `branding.siteName` in your `config.json`, update it to your preferred name. Otherwise the new default "CoreScope" applies automatically.
-
-No other config keys changed.
-
-## 6. MQTT
-
-**No action required.** MQTT topics (`meshcore/#`) are protocol-level and are not affected by the rename.
-
-## 7. Browser
-
-**No action required.** Bookmarks/favorites will continue to work at the same host and port. localStorage keys are unchanged, so your settings and preferences are preserved.
-
-## 8. CI/CD
-
-If you have custom CI/CD pipelines that reference:
-
-- The old repository URL (`meshcore-analyzer`)
-- The old Docker image name (`meshcore-analyzer:latest`)
-- Old container names (`meshcore-*`)
-
-Update those references to use the new names.
-
----
-
-## Summary Checklist
-
-| Item | Action Required? | What to Do |
-|------|-----------------|------------|
-| Git remote | ✅ Yes | `git remote set-url origin …corescope.git` |
-| Docker image | ✅ Yes | Rebuild; optionally `docker rmi` old image |
-| Docker Compose | ✅ Yes | `docker compose down && build && up` |
-| Data directories | ❌ No | Unchanged |
-| Config | ⚠️ Maybe | Only if you customized `branding.siteName` |
-| MQTT | ❌ No | Topics unchanged |
-| Browser | ❌ No | Settings preserved |
-| CI/CD | ⚠️ Maybe | Update if referencing old repo/image names |

manage.sh

@@ -2,20 +2,26 @@
 # CoreScope — Setup & Management Helper
 # Usage: ./manage.sh [command]
 #
-# All container management goes through docker compose.
-# Container config lives in docker-compose.yml — this script is just a wrapper.
-#
 # Idempotent: safe to cancel and re-run at any point.
 # Each step checks what's already done and skips it.
 set -e
 
+CONTAINER_NAME="corescope"
 IMAGE_NAME="corescope"
+DATA_VOLUME="meshcore-data"
+CADDY_VOLUME="caddy-data"
 STATE_FILE=".setup-state"
 
-# Source .env for port/path overrides (same file docker compose reads)
+# Source .env for port/path overrides (if present)
 [ -f .env ] && set -a && . ./.env && set +a
 
-# Resolved paths for prod/staging data (must match docker-compose.yml)
+# Docker Compose mode detection
+COMPOSE_MODE=false
+if [ -f docker-compose.yml ]; then
+  COMPOSE_MODE=true
+fi
+
+# Resolved paths for prod/staging data
 PROD_DATA="${PROD_DATA_DIR:-$HOME/meshcore-data}"
 STAGING_DATA="${STAGING_DATA_DIR:-$HOME/meshcore-staging-data}"
 
@@ -45,6 +51,83 @@ is_done()    { [ -f "$STATE_FILE" ] && grep -qx "$1" "$STATE_FILE" 2>/dev/null;
 
 # ─── Helpers ──────────────────────────────────────────────────────────────
 
+# Determine the correct data volume/mount args for docker run.
+# Detects existing host data directories and uses bind mounts if found.
+get_data_mount_args() {
+  # Check for existing host data directories with a DB file
+  if [ -d "$HOME/meshcore-data" ] && [ -f "$HOME/meshcore-data/meshcore.db" ]; then
+    echo "-v $HOME/meshcore-data:/app/data"
+    return
+  fi
+  if [ -d "$(pwd)/data" ] && [ -f "$(pwd)/data/meshcore.db" ]; then
+    echo "-v $(pwd)/data:/app/data"
+    return
+  fi
+  # Default: Docker named volume
+  echo "-v ${DATA_VOLUME}:/app/data"
+}
+
+# Determine the required port mappings from Caddyfile
+get_required_ports() {
+  local caddyfile_domain
+  caddyfile_domain=$(grep -v '^#' caddy-config/Caddyfile 2>/dev/null | head -1 | tr -d ' {')
+  if echo "$caddyfile_domain" | grep -qE '^:[0-9]+$'; then
+    # HTTP-only on a specific port (e.g., :80, :8080)
+    echo "${caddyfile_domain#:}"
+  else
+    # Domain name — needs 80 + 443 for Caddy auto-TLS
+    echo "80 443"
+  fi
+}
+
+# Get current container port mappings (just the host ports)
+get_current_ports() {
+  docker inspect "$CONTAINER_NAME" 2>/dev/null | \
+    grep -oP '"HostPort":\s*"\K[0-9]+' | sort -u | tr '\n' ' ' | sed 's/ $//'
+}
+
+# Check if container port mappings match what's needed.
+# Returns 0 if they match, 1 if mismatch.
+check_port_match() {
+  local required current
+  required=$(get_required_ports | tr ' ' '\n' | sort | tr '\n' ' ' | sed 's/ $//')
+  current=$(get_current_ports | tr ' ' '\n' | sort | tr '\n' ' ' | sed 's/ $//')
+  [ "$required" = "$current" ]
+}
+
+# Build the docker run command args (ports + volumes)
+get_docker_run_args() {
+  local ports_arg=""
+  for port in $(get_required_ports); do
+    ports_arg="$ports_arg -p ${port}:${port}"
+  done
+
+  local data_mount
+  data_mount=$(get_data_mount_args)
+
+  echo "$ports_arg \
+    -v $(pwd)/config.json:/app/config.json:ro \
+    -v $(pwd)/caddy-config/Caddyfile:/etc/caddy/Caddyfile:ro \
+    $data_mount \
+    -v ${CADDY_VOLUME}:/data/caddy"
+}
+
+# Recreate the container with current settings
+recreate_container() {
+  info "Stopping and removing old container..."
+  docker stop "$CONTAINER_NAME" 2>/dev/null || true
+  docker rm "$CONTAINER_NAME" 2>/dev/null || true
+
+  local run_args
+  run_args=$(get_docker_run_args)
+
+  eval docker run -d \
+    --name "$CONTAINER_NAME" \
+    --restart unless-stopped \
+    $run_args \
+    "$IMAGE_NAME"
+}
+
 # Check config.json for placeholder values
 check_config_placeholders() {
   if [ -f config.json ]; then
@@ -57,7 +140,7 @@ check_config_placeholders() {
 
 # Verify the running container is actually healthy
 verify_health() {
-  local container="corescope-prod"
+  local base_url="http://localhost:3000"
   local use_https=false
 
   # Check if Caddyfile has a real domain (not :80)
@@ -73,7 +156,7 @@ verify_health() {
   info "Waiting for server to respond..."
   local healthy=false
   for i in $(seq 1 45); do
-    if docker exec "$container" wget -qO- http://localhost:3000/api/stats &>/dev/null; then
+    if docker exec "$CONTAINER_NAME" wget -qO- http://localhost:3000/api/stats &>/dev/null; then
       healthy=true
       break
     fi
@@ -89,7 +172,7 @@ verify_health() {
 
   # Check for MQTT errors in recent logs
   local mqtt_errors
-  mqtt_errors=$(docker logs "$container" --tail 50 2>&1 | grep -i 'mqtt.*error\|mqtt.*fail\|ECONNREFUSED.*1883' || true)
+  mqtt_errors=$(docker logs "$CONTAINER_NAME" --tail 50 2>&1 | grep -i 'mqtt.*error\|mqtt.*fail\|ECONNREFUSED.*1883' || true)
   if [ -n "$mqtt_errors" ]; then
     warn "MQTT errors detected in logs:"
     echo "$mqtt_errors" | head -5 | sed 's/^/   /'
@@ -151,13 +234,6 @@ cmd_setup() {
   fi
 
   log "Docker $(docker --version | grep -oP 'version \K[^ ,]+')"
-  
-  # Check docker compose (separate check since it's a plugin/separate binary)
-  if ! docker compose version &>/dev/null; then
-    err "docker compose is required. Install Docker Desktop or docker-compose-plugin."
-    exit 1
-  fi
-  
   mark_done "docker"
 
   # ── Step 2: Config ──
@@ -295,12 +371,12 @@ cmd_setup() {
   if [ -n "$IMAGE_EXISTS" ] && is_done "build"; then
     log "Image already built."
     if confirm "Rebuild? (only needed if you updated the code)"; then
-      docker compose build prod
+      docker build --build-arg APP_VERSION=$(node -p "require('./package.json').version" 2>/dev/null || echo "unknown") --build-arg GIT_COMMIT=$(git rev-parse --short HEAD 2>/dev/null || echo "unknown") --build-arg BUILD_TIME=$(date -u +%Y-%m-%dT%H:%M:%SZ) -t "$IMAGE_NAME" .
       log "Image rebuilt."
     fi
   else
     info "This takes 1-2 minutes the first time..."
-    docker compose build prod
+    docker build --build-arg APP_VERSION=$(node -p "require('./package.json').version" 2>/dev/null || echo "unknown") --build-arg GIT_COMMIT=$(git rev-parse --short HEAD 2>/dev/null || echo "unknown") --build-arg BUILD_TIME=$(date -u +%Y-%m-%dT%H:%M:%SZ) -t "$IMAGE_NAME" .
     log "Image built."
   fi
   mark_done "build"
@@ -309,15 +385,45 @@ cmd_setup() {
   step 5 "Starting container"
 
   # Detect existing data directories
-  if [ -d "$PROD_DATA" ] && [ -f "$PROD_DATA/meshcore.db" ]; then
-    info "Found existing data at $PROD_DATA/ — will use bind mount."
+  if [ -d "$HOME/meshcore-data" ] && [ -f "$HOME/meshcore-data/meshcore.db" ]; then
+    info "Found existing data at \$HOME/meshcore-data/ — will use bind mount."
+  elif [ -d "$(pwd)/data" ] && [ -f "$(pwd)/data/meshcore.db" ]; then
+    info "Found existing data at ./data/ — will use bind mount."
   fi
 
-  if docker ps --format '{{.Names}}' | grep -q "^corescope-prod$"; then
+  if docker ps --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
     log "Container already running."
+    # Check port mappings match
+    if ! check_port_match; then
+      warn "Container port mappings don't match Caddyfile configuration."
+      warn "Current ports: $(get_current_ports)"
+      warn "Required ports: $(get_required_ports)"
+      if confirm "Recreate container with correct ports?"; then
+        recreate_container
+        log "Container recreated with correct ports."
+      fi
+    fi
+  elif docker ps -a --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
+    # Exists but stopped — check ports before starting
+    if ! check_port_match; then
+      warn "Stopped container has wrong port mappings."
+      warn "Current ports: $(get_current_ports)"
+      warn "Required ports: $(get_required_ports)"
+      if confirm "Recreate container with correct ports?"; then
+        recreate_container
+        log "Container recreated with correct ports."
+      else
+        info "Starting existing container (ports unchanged)..."
+        docker start "$CONTAINER_NAME"
+        log "Started (with old port mappings)."
+      fi
+    else
+      info "Container exists but is stopped. Starting..."
+      docker start "$CONTAINER_NAME"
+      log "Started."
+    fi
   else
-    mkdir -p "$PROD_DATA"
-    docker compose up -d prod
+    recreate_container
     log "Container started."
   fi
   mark_done "container"
@@ -325,7 +431,7 @@ cmd_setup() {
   # ── Step 6: Verify ──
   step 6 "Verifying"
 
-  if docker ps --format '{{.Names}}' | grep -q "^corescope-prod$"; then
+  if docker ps --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
     verify_health
 
     CADDYFILE_DOMAIN=$(grep -v '^#' caddy-config/Caddyfile 2>/dev/null | head -1 | tr -d ' {')
@@ -357,7 +463,7 @@ cmd_setup() {
     err "Container failed to start."
     echo ""
     echo "   Check what went wrong:"
-    echo "     docker compose logs prod"
+    echo "     docker logs ${CONTAINER_NAME}"
     echo ""
     echo "   Common fixes:"
     echo "     • Invalid config.json — check JSON syntax"
@@ -429,72 +535,132 @@ cmd_start() {
     WITH_STAGING=true
   fi
 
-  if $WITH_STAGING; then
-    # Prepare staging data and config
-    prepare_staging_db
-    prepare_staging_config
+  if $COMPOSE_MODE; then
+    if $WITH_STAGING; then
+      # Prepare staging data and config
+      prepare_staging_db
+      prepare_staging_config
 
-    info "Starting production container (corescope-prod) on ports ${PROD_HTTP_PORT:-80}/${PROD_HTTPS_PORT:-443}..."
-    info "Starting staging container (corescope-staging) on port ${STAGING_HTTP_PORT:-81}..."
-    docker compose --profile staging up -d
-    log "Production started on ports ${PROD_HTTP_PORT:-80}/${PROD_HTTPS_PORT:-443}/${PROD_MQTT_PORT:-1883}"
-    log "Staging started on port ${STAGING_HTTP_PORT:-81} (MQTT: ${STAGING_MQTT_PORT:-1884})"
+      info "Starting production container (corescope-prod) on ports ${PROD_HTTP_PORT:-80}/${PROD_HTTPS_PORT:-443}..."
+      info "Starting staging container (corescope-staging) on port ${STAGING_HTTP_PORT:-81}..."
+      docker compose --profile staging up -d
+      log "Production started on ports ${PROD_HTTP_PORT:-80}/${PROD_HTTPS_PORT:-443}/${PROD_MQTT_PORT:-1883}"
+      log "Staging started on port ${STAGING_HTTP_PORT:-81} (MQTT: ${STAGING_MQTT_PORT:-1884})"
+    else
+      info "Starting production container (corescope-prod) on ports ${PROD_HTTP_PORT:-80}/${PROD_HTTPS_PORT:-443}..."
+      docker compose up -d prod
+      log "Production started. Staging NOT running (use --with-staging to start both)."
+    fi
   else
-    info "Starting production container (corescope-prod) on ports ${PROD_HTTP_PORT:-80}/${PROD_HTTPS_PORT:-443}..."
-    docker compose up -d prod
-    log "Production started. Staging NOT running (use --with-staging to start both)."
+    # Legacy single-container mode
+    if $WITH_STAGING; then
+      err "--with-staging requires docker-compose.yml. Run setup or add docker-compose.yml first."
+      exit 1
+    fi
+    warn "No docker-compose.yml found — using legacy single-container mode."
+    if docker ps --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
+      warn "Already running."
+    elif docker ps -a --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
+      if ! check_port_match; then
+        warn "Container port mappings don't match Caddyfile configuration."
+        warn "Current ports: $(get_current_ports)"
+        warn "Required ports: $(get_required_ports)"
+        if confirm "Recreate container with correct ports?"; then
+          recreate_container
+          log "Container recreated and started with correct ports."
+          return
+        fi
+      fi
+      docker start "$CONTAINER_NAME"
+      log "Started."
+    else
+      err "Container doesn't exist. Run './manage.sh setup' first."
+      exit 1
+    fi
   fi
 }
 
 cmd_stop() {
   local TARGET="${1:-all}"
 
-  case "$TARGET" in
-    prod)
-      info "Stopping production container (corescope-prod)..."
-      docker compose stop prod
-      log "Production stopped."
-      ;;
-    staging)
-      info "Stopping staging container (corescope-staging)..."
-      docker compose --profile staging stop staging
-      log "Staging stopped."
-      ;;
-    all)
-      info "Stopping all containers..."
-      docker compose --profile staging --profile staging-go down
-      log "All containers stopped."
-      ;;
-    *)
-      err "Usage: ./manage.sh stop [prod|staging|all]"
-      exit 1
-      ;;
-  esac
+  if $COMPOSE_MODE; then
+    case "$TARGET" in
+      prod)
+        info "Stopping production container (corescope-prod)..."
+        docker compose stop prod
+        log "Production stopped."
+        ;;
+      staging)
+        info "Stopping staging container (corescope-staging)..."
+        docker compose stop staging
+        log "Staging stopped."
+        ;;
+      all)
+        info "Stopping all containers..."
+        docker compose --profile staging --profile staging-go down 2>/dev/null
+        docker rm -f "$CONTAINER_NAME" 2>/dev/null
+        log "All containers stopped."
+        ;;
+      *)
+        err "Usage: ./manage.sh stop [prod|staging|all]"
+        exit 1
+        ;;
+    esac
+  else
+    # Legacy mode
+    docker stop "$CONTAINER_NAME" 2>/dev/null && log "Stopped." || warn "Not running."
+  fi
 }
 
 cmd_restart() {
-  local TARGET="${1:-prod}"
-  case "$TARGET" in
-    prod)
-      info "Restarting production container (corescope-prod)..."
-      docker compose up -d --force-recreate prod
-      log "Production restarted."
-      ;;
-    staging)
-      info "Restarting staging container (corescope-staging)..."
-      docker compose --profile staging up -d --force-recreate staging
-      log "Staging restarted."
-      ;;
-    all)
-      info "Restarting all containers..."
-      docker compose --profile staging up -d --force-recreate
-      log "All containers restarted."
-      ;;
-    *)
-      err "Usage: ./manage.sh restart [prod|staging|all]"
+  if $COMPOSE_MODE; then
+    local TARGET="${1:-prod}"
+    case "$TARGET" in
+      prod)
+        info "Restarting production container (corescope-prod)..."
+        docker compose up -d --force-recreate prod
+        log "Production restarted."
+        ;;
+      staging)
+        info "Restarting staging container (corescope-staging)..."
+        docker compose --profile staging up -d --force-recreate staging
+        log "Staging restarted."
+        ;;
+      all)
+        info "Restarting all containers..."
+        docker compose --profile staging up -d --force-recreate
+        log "All containers restarted."
+        ;;
+      *)
+        err "Usage: ./manage.sh restart [prod|staging|all]"
+        exit 1
+        ;;
+    esac
+  else
+    # Legacy mode
+    if docker ps --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
+      if ! check_port_match; then
+        warn "Port mappings have changed. Recreating container..."
+        recreate_container
+        log "Container recreated with correct ports."
+      else
+        docker restart "$CONTAINER_NAME"
+        log "Restarted."
+      fi
+    elif docker ps -a --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
+      if ! check_port_match; then
+        warn "Port mappings have changed. Recreating container..."
+        recreate_container
+        log "Container recreated with correct ports."
+      else
+        docker start "$CONTAINER_NAME"
+        log "Started."
+      fi
+    else
+      err "Not running. Use './manage.sh setup'."
       exit 1
-      ;;
-  esac
+    fi
+  fi
 }
 
 # ─── Status ───────────────────────────────────────────────────────────────
@@ -529,68 +695,143 @@ show_container_status() {
 
 cmd_status() {
   echo ""
-  echo "═══════════════════════════════════════"
-  echo "  CoreScope Status"
-  echo "═══════════════════════════════════════"
-  echo ""
 
-  # Production
-  show_container_status "corescope-prod" "Production"
-  echo ""
+  if $COMPOSE_MODE; then
+    echo "═══════════════════════════════════════"
+    echo "  CoreScope Status (Compose)"
+    echo "═══════════════════════════════════════"
+    echo ""
+
+    # Production
+    show_container_status "corescope-prod" "Production"
+    echo ""
+
+    # Staging
+    if container_running "corescope-staging"; then
+      show_container_status "corescope-staging" "Staging"
+    else
+      info "Staging (corescope-staging): Not running (use --with-staging to start both)"
+    fi
+    echo ""
+
+    # Disk usage
+    if [ -d "$PROD_DATA" ] && [ -f "$PROD_DATA/meshcore.db" ]; then
+      local db_size
+      db_size=$(du -h "$PROD_DATA/meshcore.db" 2>/dev/null | cut -f1)
+      info "Production DB: ${db_size}"
+    fi
+    if [ -d "$STAGING_DATA" ] && [ -f "$STAGING_DATA/meshcore.db" ]; then
+      local staging_db_size
+      staging_db_size=$(du -h "$STAGING_DATA/meshcore.db" 2>/dev/null | cut -f1)
+      info "Staging DB: ${staging_db_size}"
+    fi
 
-  # Staging
-  if container_running "corescope-staging"; then
-    show_container_status "corescope-staging" "Staging"
   else
-    info "Staging (corescope-staging): Not running (use --with-staging to start both)"
-  fi
-  echo ""
+    # Legacy single-container status
+    if docker ps --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
+      log "Container is running."
+      echo ""
+      docker ps --filter "name=${CONTAINER_NAME}" --format "   Status: {{.Status}}"
+      docker ps --filter "name=${CONTAINER_NAME}" --format "   Ports:  {{.Ports}}"
+      echo ""
 
-  # Disk usage
-  if [ -d "$PROD_DATA" ] && [ -f "$PROD_DATA/meshcore.db" ]; then
-    local db_size
-    db_size=$(du -h "$PROD_DATA/meshcore.db" 2>/dev/null | cut -f1)
-    info "Production DB: ${db_size}"
-  fi
-  if [ -d "$STAGING_DATA" ] && [ -f "$STAGING_DATA/meshcore.db" ]; then
-    local staging_db_size
-    staging_db_size=$(du -h "$STAGING_DATA/meshcore.db" 2>/dev/null | cut -f1)
-    info "Staging DB: ${staging_db_size}"
-  fi
+      info "Service health:"
+      # Server
+      if docker exec "$CONTAINER_NAME" wget -qO /dev/null http://localhost:3000/api/stats 2>/dev/null; then
+        STATS=$(docker exec "$CONTAINER_NAME" wget -qO- http://localhost:3000/api/stats 2>/dev/null)
+        PACKETS=$(echo "$STATS" | grep -oP '"totalPackets":\K[0-9]+' 2>/dev/null || echo "?")
+        NODES=$(echo "$STATS" | grep -oP '"totalNodes":\K[0-9]+' 2>/dev/null || echo "?")
+        log "  Server — ${PACKETS} packets, ${NODES} nodes"
+      else
+        err "  Server — not responding"
+      fi
 
+      # Mosquitto
+      if docker exec "$CONTAINER_NAME" pgrep mosquitto &>/dev/null; then
+        log "  Mosquitto — running"
+      else
+        err "  Mosquitto — not running"
+      fi
+
+      # Caddy
+      if docker exec "$CONTAINER_NAME" pgrep caddy &>/dev/null; then
+        log "  Caddy — running"
+      else
+        err "  Caddy — not running"
+      fi
+
+      # Check for MQTT errors in recent logs
+      MQTT_ERRORS=$(docker logs "$CONTAINER_NAME" --tail 50 2>&1 | grep -i 'mqtt.*error\|mqtt.*fail\|ECONNREFUSED.*1883' || true)
+      if [ -n "$MQTT_ERRORS" ]; then
+        echo ""
+        warn "MQTT errors in recent logs:"
+        echo "$MQTT_ERRORS" | head -3 | sed 's/^/   /'
+      fi
+
+      # Port mapping check
+      if ! check_port_match; then
+        echo ""
+        warn "Port mappings don't match Caddyfile. Run './manage.sh restart' to fix."
+      fi
+
+      # Disk usage
+      DB_SIZE=$(docker exec "$CONTAINER_NAME" du -h /app/data/meshcore.db 2>/dev/null | cut -f1)
+      if [ -n "$DB_SIZE" ]; then
+        echo ""
+        info "Database size: ${DB_SIZE}"
+      fi
+    else
+      err "Container is not running."
+      if docker ps -a --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
+        echo "   Start with: ./manage.sh start"
+      else
+        echo "   Set up with: ./manage.sh setup"
+      fi
+    fi
+  fi
   echo ""
 }
 
 # ─── Logs ─────────────────────────────────────────────────────────────────
 
 cmd_logs() {
-  local TARGET="${1:-prod}"
-  local LINES="${2:-100}"
-  case "$TARGET" in
-    prod)
-      info "Tailing production logs..."
-      docker compose logs -f --tail="$LINES" prod
-      ;;
-    staging)
-      if container_running "corescope-staging"; then
-        info "Tailing staging logs..."
-        docker compose logs -f --tail="$LINES" staging
-      else
-        err "Staging container is not running."
-        info "Start with: ./manage.sh start --with-staging"
+  if $COMPOSE_MODE; then
+    local TARGET="${1:-prod}"
+    local LINES="${2:-100}"
+    case "$TARGET" in
+      prod)
+        info "Tailing production logs..."
+        docker compose logs -f --tail="$LINES" prod
+        ;;
+      staging)
+        if container_running "corescope-staging"; then
+          info "Tailing staging logs..."
+          docker compose logs -f --tail="$LINES" staging
+        else
+          err "Staging container is not running."
+          info "Start with: ./manage.sh start --with-staging"
+          exit 1
+        fi
+        ;;
+      *)
+        err "Usage: ./manage.sh logs [prod|staging] [lines]"
         exit 1
-      fi
-      ;;
-    *)
-      err "Usage: ./manage.sh logs [prod|staging] [lines]"
-      exit 1
-      ;;
-  esac
+        ;;
+    esac
+  else
+    # Legacy mode
+    docker logs -f "$CONTAINER_NAME" --tail "${1:-100}"
+  fi
 }
 
 # ─── Promote ──────────────────────────────────────────────────────────────
 
 cmd_promote() {
+  if ! $COMPOSE_MODE; then
+    err "Promotion requires Docker Compose setup (docker-compose.yml)."
+    exit 1
+  fi
+
   echo ""
   info "Promotion Flow: Staging → Production"
   echo ""
@@ -665,10 +906,10 @@ cmd_update() {
   git pull
 
   info "Rebuilding image..."
-  docker compose build prod
+  docker build --build-arg APP_VERSION=$(node -p "require('./package.json').version" 2>/dev/null || echo "unknown") --build-arg GIT_COMMIT=$(git rev-parse --short HEAD 2>/dev/null || echo "unknown") --build-arg BUILD_TIME=$(date -u +%Y-%m-%dT%H:%M:%SZ) -t "$IMAGE_NAME" .
 
   info "Restarting with new image..."
-  docker compose up -d --force-recreate prod
+  recreate_container
 
   log "Updated and restarted. Data preserved."
 }
@@ -683,13 +924,12 @@ cmd_backup() {
   info "Backing up to ${BACKUP_DIR}/"
 
   # Database
-  # Always use bind mount path (from .env or default)
-  DB_PATH="$PROD_DATA/meshcore.db"
+  DB_PATH=$(docker volume inspect "$DATA_VOLUME" --format '{{ .Mountpoint }}' 2>/dev/null)/meshcore.db
   if [ -f "$DB_PATH" ]; then
     cp "$DB_PATH" "$BACKUP_DIR/meshcore.db"
     log "Database ($(du -h "$BACKUP_DIR/meshcore.db" | cut -f1))"
-  elif container_running "corescope-prod"; then
-    docker cp corescope-prod:/app/data/meshcore.db "$BACKUP_DIR/meshcore.db" 2>/dev/null && \
+  elif docker ps --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
+    docker cp "${CONTAINER_NAME}:/app/data/meshcore.db" "$BACKUP_DIR/meshcore.db" 2>/dev/null && \
       log "Database (via docker cp)" || warn "Could not backup database"
   else
     warn "Database not found (container not running?)"
@@ -708,8 +948,7 @@ cmd_backup() {
   fi
 
   # Theme
-  # Always use bind mount path (from .env or default)
-  THEME_PATH="$PROD_DATA/theme.json"
+  THEME_PATH=$(docker volume inspect "$DATA_VOLUME" --format '{{ .Mountpoint }}' 2>/dev/null)/theme.json
   if [ -f "$THEME_PATH" ]; then
     cp "$THEME_PATH" "$BACKUP_DIR/theme.json"
     log "theme.json"
@@ -782,12 +1021,15 @@ cmd_restore() {
   info "Backing up current state..."
   cmd_backup "./backups/corescope-pre-restore-$(date +%Y%m%d-%H%M%S)"
 
-  docker compose stop prod 2>/dev/null || true
+  docker stop "$CONTAINER_NAME" 2>/dev/null || true
 
   # Restore database
-  mkdir -p "$PROD_DATA"
-  DEST_DB="$PROD_DATA/meshcore.db"
-  cp "$DB_FILE" "$DEST_DB"
+  DEST_DB=$(docker volume inspect "$DATA_VOLUME" --format '{{ .Mountpoint }}' 2>/dev/null)/meshcore.db
+  if [ -d "$(dirname "$DEST_DB")" ]; then
+    cp "$DB_FILE" "$DEST_DB"
+  else
+    docker cp "$DB_FILE" "${CONTAINER_NAME}:/app/data/meshcore.db"
+  fi
   log "Database restored"
 
   # Restore config if present
@@ -805,25 +1047,27 @@ cmd_restore() {
 
   # Restore theme if present
   if [ -n "$THEME_FILE" ] && [ -f "$THEME_FILE" ]; then
-    DEST_THEME="$PROD_DATA/theme.json"
-    cp "$THEME_FILE" "$DEST_THEME"
+    DEST_THEME=$(docker volume inspect "$DATA_VOLUME" --format '{{ .Mountpoint }}' 2>/dev/null)/theme.json
+    if [ -d "$(dirname "$DEST_THEME")" ]; then
+      cp "$THEME_FILE" "$DEST_THEME"
+    fi
     log "theme.json restored"
   fi
 
-  docker compose up -d prod
+  docker start "$CONTAINER_NAME"
   log "Restored and restarted."
 }
 
 # ─── MQTT Test ────────────────────────────────────────────────────────────
 
 cmd_mqtt_test() {
-  if ! container_running "corescope-prod"; then
+  if ! docker ps --format '{{.Names}}' | grep -q "^${CONTAINER_NAME}$"; then
     err "Container not running. Start with: ./manage.sh start"
     exit 1
   fi
 
   info "Listening for MQTT messages (10 second timeout)..."
-  MSG=$(docker exec corescope-prod mosquitto_sub -h localhost -t 'meshcore/#' -C 1 -W 10 2>/dev/null)
+  MSG=$(docker exec "$CONTAINER_NAME" mosquitto_sub -h localhost -t 'meshcore/#' -C 1 -W 10 2>/dev/null)
   if [ -n "$MSG" ]; then
     log "Received MQTT message:"
     echo "   $MSG" | head -c 200
@@ -840,19 +1084,21 @@ cmd_mqtt_test() {
 
 cmd_reset() {
   echo ""
-  warn "This will remove all containers, images, and setup state."
-  warn "Your config.json, Caddyfile, and data directory are NOT deleted."
+  warn "This will remove the container, image, and setup state."
+  warn "Your config.json, Caddyfile, and data volume are NOT deleted."
   echo ""
   if ! confirm "Continue?"; then
     echo "   Aborted."
     exit 0
   fi
 
-  docker compose --profile staging --profile staging-go down --rmi local 2>/dev/null || true
+  docker stop "$CONTAINER_NAME" 2>/dev/null || true
+  docker rm "$CONTAINER_NAME" 2>/dev/null || true
+  docker rmi "$IMAGE_NAME" 2>/dev/null || true
   rm -f "$STATE_FILE"
 
   log "Reset complete. Run './manage.sh setup' to start over."
-  echo "   Data directory: $PROD_DATA (not removed)"
+  echo "   Data volume preserved. To delete it: docker volume rm ${DATA_VOLUME}"
 }
 
 # ─── Help ─────────────────────────────────────────────────────────────────
@@ -882,7 +1128,11 @@ cmd_help() {
   echo "    restore <d>        Restore from backup dir or .db file"
   echo "    mqtt-test          Check if MQTT data is flowing"
   echo ""
-  echo "All commands use docker compose with docker-compose.yml."
+  if $COMPOSE_MODE; then
+    info "Docker Compose mode detected (docker-compose.yml present)."
+  else
+    warn "Legacy mode (no docker-compose.yml). Some commands unavailable."
+  fi
   echo ""
 }