Commit Graph
309 Commits
Author SHA1 Message Date
you b114cd6eb0 Add hash size labels for repeater markers on map
- Compute hash_size from ADVERT packets in /api/nodes response
- Show colored rectangle markers with hash size (e.g. '2B') for repeaters
- Add 'Hash size labels' toggle in map controls (default ON, saved to localStorage)
- Non-repeater markers unchanged
2026-03-21 00:19:15 +00:00
you 85c356448e fix: switch all user-facing URLs to hash-based for stability across restarts
After dedup migration, packet IDs from the legacy 'packets' table differ
from transmission IDs in the 'transmissions' table. URLs using numeric IDs
became invalid after restart when _loadNormalized() assigned different IDs.

Changes:
- All packet URLs now use 16-char hex hashes instead of numeric IDs
  (#/packets/HASH instead of #/packet/ID)
- selectPacket() accepts hash parameter, uses hash-based URLs
- Copy Link generates hash-based URLs
- Search results link to hash-based URLs
- /api/packets/:id endpoint accepts both numeric IDs and 16-char hashes
- insert() now calls insertTransmission() to get stable transmission IDs
- Added db.getTransmission() for direct transmission table lookup
- Removed redundant byTransmission map (identical to byHash)
- All byTransmission references replaced with byHash
2026-03-21 00:18:11 +00:00
you d621a9f34a Make ADVERT node names clickable links to node detail page 2026-03-21 00:09:30 +00:00
you 8c2d5d770f fix: Advertised Hash Size field shows byte offset 1 (path byte), not payload offset 2026-03-21 00:03:55 +00:00
you 9d65f041d4 fix: check transmission ID before observation ID in packet lookup 2026-03-20 23:58:23 +00:00
you 8ebce8087b fix: packet detail lookup uses transmission ID index
After dedup, table rows have transmission IDs but getById() maps
observation IDs. Added byTxId index so /api/packets/:id resolves
both observation and transmission IDs correctly.
2026-03-20 23:55:38 +00:00
you e7f2613d1e packets: show Advertised Hash Size prominently in ADVERT field table 2026-03-20 23:43:40 +00:00
you 15e80e56f1 fix: packet links use hash instead of observation ID
After dedup migration, transmission IDs != old packet IDs.
Hash-based links (#/packets/HASH) are stable across the migration.
Affected: node detail, channel messages, live page packet cards.
2026-03-20 23:40:30 +00:00
you 84bf7fb545 packets: show hash_size in detail meta + fix blank hash_size in field table 2026-03-20 23:32:43 +00:00
you 9713e972f9 traces: clickable observer names, path graph, remove redundant observer table
- API returns observer_name + path_json per observation
- Timeline shows resolved names with links to #/observers/<id>
- Path Visualization replaced with SVG graph showing all observed paths
- Removed redundant Observer Details table (same data as timeline)
2026-03-20 23:19:37 +00:00
you 1a87e7410e v2.3.0: Packet Deduplication — normalized storage with transmissions + observations
CI: add setup-node to deploy workflow
v2.3.0
2026-03-20 23:02:58 +00:00
you 91a6a2c525 fix: migration handles concurrent dual-write with INSERT OR IGNORE 2026-03-20 22:52:12 +00:00
you e5a609bbfc Merge dedup-normalize: packet deduplication (M1-M5)
Normalized packet storage: transmissions + observations tables.
- M1: Schema migration script
- M2: Dual-write ingest
- M3: In-memory store restructured around transmissions
- M4: API responses include observation_count, totalTransmissions
- M5: Frontend badges, deeplinks, dedup-aware UI
2026-03-20 22:49:13 +00:00
you 95b59d1792 fix: recent packets deeplink uses route path not query param 2026-03-20 22:34:00 +00:00
you e7aa4246ac fix: live node panel deeplinks — full detail, observer names, recent packets
- Full Detail links to #/nodes/<pubkey> (was #/nodes?selected= which just showed list)
- Heard By observer names link to #/observers/<id>
- Recent Packets link to #/packets?hash=<hash>
2026-03-20 22:17:13 +00:00
you f1aa6caf93 fix: packet expand shows observations, heard-by uses correct field name
- pktToggleGroup fetches ?expand=observations and maps them as children
- Live page heard-by uses o.packetCount (was o.count → undefined)
2026-03-20 22:02:54 +00:00
you a882aae681 M5: Frontend updates for dedup — observation_count badges, totalTransmissions
- packets.js: Show observation_count badge (👁 N) on grouped rows
- nodes.js: Use totalTransmissions (fallback totalPackets), show observation badges on recent packets
- home.js: Use totalTransmissions for network stats
- node-analytics.js: Use totalTransmissions for throughput display
- analytics.js: Use totalTransmissions for overview stats and node rankings
- live.js: Use totalTransmissions in node detail, show observation badges in feed and recent packets
- style.css: Add .badge-obs style for observation count badges
- index.html: Bump cache busters on all changed JS/CSS files

All changes have backward compat fallbacks to totalPackets.
2026-03-20 21:31:10 +00:00
you aa35164252 M4: API response changes for dedup-normalize
- GET /api/packets: returns transmissions with observation_count, strip
  observations[] by default (use ?expand=observations to include)
- GET /api/packets/🆔 includes observation_count and observations[]
- GET /api/nodes/:pubkey/health: stats.totalTransmissions + totalObservations
  (totalPackets kept for backward compat)
- GET /api/nodes/bulk-health: same transmission/observation split
- WebSocket broadcast: includes observation_count
- db.js getStats(): adds totalTransmissions count
- All backward-compatible: old field names preserved alongside new ones
2026-03-20 20:49:34 +00:00
you 84f33aef7b M3: Restructure in-memory store around transmissions
- load() reads from transmissions JOIN observations (with legacy fallback)
- byHash now maps hash → single transmission object (1:1)
- byNode maps pubkey → [transmissions] (deduped, no inflated observations)
- byTransmission is the primary data structure
- byId maps observation IDs for backward-compat packet detail links
- byObserver still maps observer_id → [observations]
- getSiblings() returns observations from transmission
- findPacketsForNode() returns unique transmissions
- query()/queryGrouped() work with transmission-centric model
- All returned objects maintain backward-compatible fields
- SQLite-only fallback path (NO_MEMORY_STORE=1) unchanged
- Tested: 11.6K transmissions from 37.5K observations (3.2x dedup)
2026-03-20 20:44:32 +00:00
you baa60cac0f M2: Dual-write ingest to transmissions/observations tables
- Add transmissions and observations schema to db.js init
- Add insertTransmission() function: upsert transmission by hash,
  always insert observation row
- All 6 pktStore.insert() call sites in server.js now also call
  db.insertTransmission() with try/catch (non-fatal on error)
- packet-store.js: add byTransmission Map index (hash → transmission
  with observations array) for future M3 query migration
- Existing insertPacket() and all read paths unchanged
2026-03-20 20:29:03 +00:00
you d7e415daa7 fix: raw_hex NOT NULL in transmissions schema — deleted 4 junk test rows 2026-03-20 20:24:13 +00:00
you 2c6148fd2d Add dedup migration script (Milestone 1)
Creates transmissions and observations tables from existing packets table.
- Groups packets by hash → 1 transmission per unique hash
- Creates 1 observation per original packet row with FK to transmission
- Idempotent: drops and recreates new tables on each run
- Does NOT modify the original packets table
- Prints stats and verifies counts match

Tested on test DB: 33813 packets → 11530 transmissions (2.93x dedup ratio)
2026-03-20 20:22:30 +00:00
you 2feb2c5b94 fix: escapeHtml crashes on numbers — String(s) before .replace() v2.2.0 2026-03-20 19:25:48 +00:00
you 10b11106f6 ci: add pre-deploy JS validation — syntax check + undefined reference detection
Validation runs BEFORE docker build. If it fails, deployment is blocked.
No more broken code reaching production.
2026-03-20 19:24:11 +00:00
you 326d411c4a fix: esc is not defined — use escapeHtml in live.js node detail 2026-03-20 19:23:20 +00:00
you 15a93d5ea4 feat: clickable nodes on live map with slide-in detail panel
- Click any node marker to see name, role, status, location, stats
- Heard By observers and recent packets shown
- Links to full node detail and analytics pages
- Slide-in panel from right with blur background, matches live page style
- Uses shared ROLE_COLORS and HEALTH_THRESHOLDS
2026-03-20 19:21:30 +00:00
you 055467ca43 fix: live map legend uses shared ROLE_COLORS instead of hardcoded wrong colors 2026-03-20 19:17:57 +00:00
you 4f7b02a91c fix: centralize hardcoded values — roles, thresholds, colors, tiles, limits — closes #104
- New public/roles.js shared module: ROLE_COLORS, ROLE_LABELS, ROLE_STYLE,
  ROLE_EMOJI, ROLE_SORT, HEALTH_THRESHOLDS, TILE_DARK/LIGHT, SNR_THRESHOLDS,
  DIST_THRESHOLDS, MAX_HOP_DIST, LIMITS — all configurable via /api/config/roles
- Removed duplicate ROLE_COLORS from map.js, nodes.js, live.js, analytics.js
- Removed duplicate health thresholds from nodes.js, home.js, observer-detail.js
- Deduplicated CartoDB tile URLs (3 copies → 1 in roles.js)
- Removed hardcoded region names from map.js and packets.js
- channels.js uses ROLE_EMOJI/ROLE_LABELS instead of hardcoded emoji chains
- server.js reads healthThresholds from config.json with defaults
- Unknown roles get gray circle fallback instead of crashing
2026-03-20 17:36:41 +00:00
you f0db317051 fix: observer locations case-insensitive match, regions from API not hardcoded
- Observer ID is uppercase, node pubkey is lowercase — added COLLATE NOCASE
- New /api/config/regions endpoint merges config regions + observed IATAs
- map.js and packets.js fetch regions from API instead of hardcoded maps
2026-03-20 16:10:16 +00:00
you 9bf78bd28d feat: add MRY (Monterey) to lincomatic MQTT topics 2026-03-20 15:29:37 +00:00
you 5fe275b3f8 fix: use region-specific MQTT topics instead of wildcards — saves bandwidth 2026-03-20 15:28:54 +00:00
you 74a08d99b0 fix: observer location from nodes table direct join — observer ID = node pubkey 2026-03-20 15:11:57 +00:00
you 76d63ffe75 feat: observers as map markers (purple stars) with computed locations
- Removed dead 'MQTT Connected Only' checkbox (never worked)
- Added 'observer' role type with purple star marker
- Observer locations computed from average of nodes they've seen
- Observer popup with name, IATA, packets, link to detail page
- Role filter checkbox includes observers with count
2026-03-20 14:56:08 +00:00
you 157dc9a979 fix: spark bars use inline spans instead of div — immune to max-width:0 table crush
The div-based spark bar was always getting crushed to 0px by
table-layout:auto + max-width:0 on td. Inline spans with fixed
width survive because they participate in text flow, not block layout.
2026-03-20 14:48:15 +00:00
you 2f07ae2e5c fix: spark bar z-index so it renders above adjacent packet count cell 2026-03-20 14:39:48 +00:00
you 1f9cd3ead1 fix: add config.json mount to CI deploy workflow 2026-03-20 14:29:32 +00:00
you 4b5b801def docs: v2.1.1 — multi-broker MQTT, observer detail, changelog
- Multi-broker MQTT with per-source IATA filtering
- Observer detail pages with charts and status
- Channel key auto-derivation (SHA256)
- Map dark/light tile swap
- Server-side My Nodes filter
- 12 bug fixes including spark bars, packet ordering, PII cleanup
- Docker deploy.sh fixed with config mount
v2.1.1
2026-03-20 09:37:22 +00:00
you 8ef65def7d fix: make renderTableRows async (await broke packets page entirely) 2026-03-20 09:33:08 +00:00
you 9ef5c1a809 fix: My Nodes filter uses server-side findPacketsForNode for all packet types
Client was matching field names that only exist on ADVERTs. Now sends
pubkeys to server, which uses findPacketsForNode() (byNode index +
text search) to find ALL packet types referencing those nodes.
2026-03-20 09:30:41 +00:00
you e31e4aa356 chore: remove accidentally committed test DB files, update gitignore 2026-03-20 09:23:25 +00:00
you db884f12eb fix: 4 bugs - spark bars inline style, My Nodes filter field names, duplicate pin button, map dark mode
1. Spark bars: inline style override on td (max-width:none, min-width:80px)
2. My Nodes filter: pubkey→pubKey, to/from→srcPubKey/destPubKey/srcHash/destHash
3. Pin button: guard against duplicates in init, remove in destroy
4. Map page: CartoDB dark/light tiles with MutationObserver theme swap
2026-03-20 09:21:17 +00:00
you 116f0c8dfb fix: add col-spark class to spark bar cells, min-width 80px 2026-03-20 09:14:19 +00:00
you b3e8dcaa93 fix: spark bars crushed by max-width:0 on data-table td
Override max-width for td cells containing spark bars so they render.
2026-03-20 09:12:27 +00:00
you 920eab04c1 fix: sort observer analytics packets newest-first, fix recentPackets slice 2026-03-20 09:10:16 +00:00
you d67b531bf2 fix: pass observer name (msg.origin) to packet insert and observer upsert
Lincomatic MQTT packets include origin field with friendly name but
we were only using it in status handler. Now both packet and observer
records get the name.
2026-03-20 09:07:39 +00:00
you 5a6847bbf4 fix: remove duplicate crypto require that crashed server 2026-03-20 08:54:35 +00:00
you 034c68c154 feat: auto-derive hashtag channel keys from SHA256(name)
Scans DB for known #channel names, derives 16-byte AES keys
algorithmically. No need to manually add hashtag channels to config.
Private channels still need manual config.
2026-03-20 08:53:11 +00:00
you 477dcde82f fix: analytics shows total packets + signal data count separately
RF analytics filtered on snr!=null, showing only 3 packets when most
lincomatic data has no SNR. Now shows total packets prominently and
signal-data count as a separate stat.
2026-03-20 08:42:38 +00:00
you c997318cd2 fix: recentPackets in health endpoint — show 20 newest DESC, not 10 oldest 2026-03-20 08:39:01 +00:00
you fa40ede9e7 fix: findPacketsForNode always resolves name, even when called with pubkey 2026-03-20 08:31:07 +00:00