Commit Graph

438 Commits

Author SHA1 Message Date
you f8faafe643 Polish filter bar: consistent sizing, logical grouping with separators, tooltips
- All filter-bar controls now exactly 34px tall with line-height:1 and border-radius:6px
- col-toggle-btn matched to same height/font-size as other controls
- Controls grouped into 4 logical sections (Filters, Display, Sort, Columns) with vertical separators
- Added title attributes with helpful descriptions to all controls
- Added sort help icon (ⓘ) with detailed tooltip explaining each sort mode
- Mobile responsive: separators hidden on small screens
2026-03-22 00:00:02 +00:00
you b03bfc780f fix: shrink region dropdown checkboxes to 14px 2026-03-21 23:55:18 +00:00
you cb622462c2 fix: resolve hops after sort changes header paths
When sort updates header row path_json, new hop hashes may not be in
hopNameCache. Now resolves unknown hops before re-rendering.
2026-03-21 23:54:49 +00:00
you 384b463a28 fix: normalize filter bar heights + widen region dropdown
All filter bar controls now share: height 34px, font-size 13px,
border-radius 6px, same padding. Region dropdown trigger matches
other controls, menu widened to 220px with white-space:nowrap to
prevent text wrapping.
2026-03-21 23:53:36 +00:00
you 352e75ff3a fix: sort change fetches observations for all visible groups
When switching to a non-Observer sort, batch-fetches observations for
all visible multi-observation groups that haven't been expanded yet.
Header rows update immediately without needing manual expand.
2026-03-21 23:52:20 +00:00
you 39894e9555 feat: sort header reflects first sorted child + asc/desc for path & time
- Header row (observer, path) updates to match the first child after sort
- Path sort: ascending (shortest first) and descending (longest first)
- Chronological: ascending (earliest first) and descending (latest first)
- Observer mode unchanged
2026-03-21 23:49:01 +00:00
you 7ead538c7c fix: region dropdown labels show 'IATA - Friendly Name' format
Dropdown items now display 'SJC - San Jose, US' instead of just
'San Jose, US'. Summary shows just IATA codes for brevity.
2026-03-21 23:47:45 +00:00
you 94d63181c3 fix: apply observation sort on all code paths that set _children
Sort was only applied in pktToggleGroup and dropdown change handler.
Missing from: loadPackets restore (re-fetches children for expanded
groups) and WS update path (unshifts new observations). Now all
three paths call sortGroupChildren after modifying _children.
2026-03-21 23:46:30 +00:00
you f40a0796f7 fix: move observation sort to filter bar dropdown, save to localStorage
- Removed per-group sort bar links (broken navigation)
- Added global 'Sort:' dropdown in filter toolbar
- Persists to localStorage across sessions
- Re-sorts all expanded groups on change
2026-03-21 23:41:24 +00:00
you a259891007 feat: add Chronological sort mode for expanded packet groups
Pure timestamp order, no grouping — shows propagation sequence.
2026-03-21 23:32:05 +00:00
you 45df19b51b feat: observation sort toggle — Observer (default) or Path length
Two sort modes for expanded packet groups:
- Observer: group by observer, earliest first, ascending time within
- Path length: shortest paths first, alphabetical observer within

Sort bar appears above expanded children with bold active mode.
2026-03-21 23:31:21 +00:00
you 9e718bd090 fix: trace page accepts route param (#/traces/HASH) 2026-03-21 23:17:32 +00:00
you f0c440151d fix: deeplink uses observation id (not observer_id) + add trace link
- Deeplinks now use ?obs=<observation_id> which is unique per row,
  fixing cases where same observer has multiple paths
- Added '🔍 Trace' link in detail pane actions
2026-03-21 23:15:42 +00:00
you 558781f834 fix: set header observer/path to earliest observation on DB load
During _loadNormalized(), observations load in DESC order so the first
observation processed is the LATEST. tx.observer_id was set from this
latest observation. Added post-load pass that finds the earliest
observation by timestamp and sets tx.observer_id/path_json to match.
2026-03-21 23:10:10 +00:00
you 1b8e8141a5 fix: stop client WS handler from replacing header path with longest path
The WS handler was overwriting the group's path_json with the longest
path from any new observation. Header should always show the first
observer's path — individual observation paths are in the expanded rows.
2026-03-21 23:06:36 +00:00
you daac079cb8 docs: update v2.4.0 release notes with latest fixes 2026-03-21 23:03:13 +00:00
you 849e53ead9 revert: undo unauthorized version bump 2026-03-21 23:01:57 +00:00
you 5c25c6adf6 Disable auto-seeding on empty DB - require --seed flag or SEED_DB=true
Previously db.seed() ran unconditionally on startup and would populate
a fresh database with fake test data. Now seeding only triggers when
explicitly requested via --seed CLI flag or SEED_DB=true env var.

The seed functionality remains available for developers:
  node server.js --seed
  SEED_DB=true node server.js
  node db.js  (direct run still seeds)
2026-03-21 23:00:01 +00:00
you f25f958a9e Add observation-level deeplinks to packet detail page
When viewing a specific observation, the URL now includes ?obs=OBSERVER_ID.
Opening such a link auto-expands the group and selects the observation.
Copy Link button includes the obs parameter when an observation is selected.
2026-03-21 22:59:07 +00:00
you 734e5570c1 fix: header row shows first observer's path, not longest path
Removed server-side longest-path override in /api/packets/:id that
replaced the transmission's path_json with the longest observation
path. The header should always reflect the first observer's path.
Individual observation paths are available in the observations array.
2026-03-21 22:58:37 +00:00
you 6a8933d896 fix: detail pane shows clicked observation data, not parent packet
When expanding a grouped packet and clicking a child observation row,
the detail pane now shows that observation's observer, SNR/RSSI, path,
and timestamp instead of the parent packet's data.

Child rows use a new 'select-observation' action that builds a synthetic
packet object by overlaying observation-specific fields onto the parent
packet data (no extra API fetch needed).
2026-03-21 22:47:01 +00:00
you 40add85cc4 docs: v2.4.0 release notes 2026-03-21 22:44:31 +00:00
you 55406e5ac6 fix: update observer_id, observer_name, path_json when first_seen moves earlier
When a later observation has an earlier timestamp, the transmission's
first_seen was updated but observer_id and path_json were not. This
caused the header row to show the wrong observer and path — whichever
MQTT message arrived first, not whichever observation was actually
earliest.
2026-03-21 22:43:18 +00:00
you 3106257c06 tweak: bump chan-tag size from 0.8em to 0.9em with more padding 2026-03-21 22:33:14 +00:00
you c72b2f712c feat: show channel name tag in packet detail column for CHAN messages 2026-03-21 22:27:51 +00:00
you 845b613d1d fix: use correct field names for observation sort (observer_name, timestamp)
Subagent used observer/rx_at/created_at but API returns
observer_name/timestamp. Sort was comparing empty strings.
2026-03-21 22:26:39 +00:00
you 805e9497a7 cleanup: remove cracker package files and temp docs accidentally committed by subagent 2026-03-21 22:24:12 +00:00
you 7258a05382 Fix expanded packet group observation sorting: group by observer, earliest first 2026-03-21 22:22:44 +00:00
you 2fb0b8ae94 Fix channel list timeAgo counters: use monotonic lastActivityMs instead of ISO strings
- Track lastActivityMs (Date.now()) on each channel object instead of ISO lastActivity
- 1s interval iterates channels[] array and updates DOM text only (no re-render)
- Uses data-channel-hash attribute to find time elements after DOM rebuilds
- Simple formatSecondsAgo: <60s→Xs, <3600s→Xm, <86400s→Xh, else Xd
- Seed lastActivityMs from API ISO string on initial load
- WS handler sets lastActivityMs = Date.now() on receipt
- Bump channels.js cache buster
2026-03-21 22:21:28 +00:00
you d51e9ff7d0 Fix duplicate observations in expanded packet group view
The insert() method had a second code path (for building rows from
packetData) that pushed observations without checking for duplicates.
Added the same dedup check (observer_id + path_json) that exists in
the other insert path and in the load-from-DB path.
2026-03-21 22:18:04 +00:00
you 6d81e13d65 fix: tick channel timeAgo labels every 1s instead of re-rendering every 30s 2026-03-21 22:15:25 +00:00
you b19b57cce8 fix: use current time for channel lastActivity on WS updates
packet.timestamp is first_seen — when the transmission was originally
observed. When multiple observers re-see the same old packet, the
broadcast carries the original (stale) first_seen. For channel list
display, what matters is 'activity happened now', not 'packet was
first seen 10h ago'.
2026-03-21 22:12:56 +00:00
you cb8e20ae7e fix: deduplicate observations with NULL path_json
The UNIQUE index on (hash, observer_id, path_json) didn't prevent
duplicates when path_json was NULL because SQLite treats NULL != NULL
for uniqueness. Fixed by:

1. Using COALESCE(path_json, '') in the UNIQUE index expression
2. Adding migration to clean up existing duplicate rows
3. Adding NULL-safe dedup checks in PacketStore load and insert paths
2026-03-21 22:06:43 +00:00
you 0dcf973e43 debug: log channel WS timestamp values 2026-03-21 22:05:52 +00:00
you 8a18e091e0 fix: use packet hash instead of sender_timestamp for channel message dedup
Device clocks on MeshCore nodes are wildly inaccurate (off by hours or
epoch-near values like 4). The channel messages endpoint was using
sender_timestamp as part of the deduplication key, which could cause
messages to fail deduplication or incorrectly collide.

Changed dedupe key from sender:timestamp to sender:hash, which is the
correct unique identifier for a transmission.

Also added TIMESTAMP-AUDIT.md documenting all device timestamp usage.
2026-03-21 21:53:38 +00:00
you 36c069b3f6 fix: use server timestamp for channel lastActivity, not device clock
WS messages from lincomatic bridge lack packet.timestamp, so the
code fell through to payload.sender_timestamp which reflects the
MeshCore device's clock (often wrong). Use current time as fallback.
2026-03-21 21:50:18 +00:00
you 4a04fbb750 fix: tick channel list relative timestamps every 30s
timeAgo labels were computed once on render and never updated,
showing stale '11h ago' until next WS message triggered re-render.
Added 30s interval to re-render channel list, cleaned up on destroy.
2026-03-21 21:41:30 +00:00
you 9b4a68051f fix: dedup live channel messages by packet hash, not sender+timestamp
Multiple observers seeing the same packet triggered separate message
entries. Now deduplicates by packet hash — additional observations
increment repeats count and add to observers list. Channel messageCount
also only bumps once per unique packet.
2026-03-21 21:38:32 +00:00
you cfcb441aa1 feat: zero-API live updates on channels page via WS
Instead of re-fetching /api/channels and /api/channels/:hash/messages
on every WebSocket event, the channels page now processes WS messages
client-side:

- Extract sender, text, channel, timestamp from WS payload
- Append new messages directly to local messages[] array
- Update channel list entries (lastActivity, lastSender, messageCount)
- Create new channel entries for previously unseen channels
- Deduplicate repeated observations of the same message

API calls now only happen on:
- Initial page load (loadChannels)
- Channel selection (selectChannel)
- Region filter change

This eliminates all polling and WS-triggered re-fetches.
2026-03-21 21:35:30 +00:00
you e58879830f fix: stop channels page from spamming API requests
The channels WS handler was calling invalidateApiCache() before
loadChannels()/refreshMessages(), which nuked the cache and forced
network fetches. Combined with the global WS onmessage handler also
invalidating /channels every 5s, this created excessive API traffic
when sitting idle on the channels page.

Changes:
- channels.js: Remove invalidateApiCache calls from WS handler, use
  bust:true parameter instead to bypass cache only when WS triggers
- channels.js: Add bust parameter to loadChannels() and refreshMessages()
- app.js: Remove /channels from global WS cache invalidation (channels
  page manages its own cache busting via its dedicated WS handler)
2026-03-21 21:34:05 +00:00
you 06f1d286d6 fix: remove ADVERT timestamp validation — field isn't stored or used
Timestamp is decoded from the ADVERT but never persisted to the nodes
table. The validation was rejecting valid nodes with slightly-off clocks
(28h future) and nodes broadcasting timestamp=4. No reason to gate on it.
2026-03-21 21:31:25 +00:00
you 54d0259708 perf: reduce 3 API calls to 1 when expanding grouped packet row
Expanding a grouped row fired: packets?hash=X&expand=observations,
packets?hash=X&limit=1, and packets/HASH — all returning the same
data. Now uses single /packets/HASH call and passes prefetched data
to selectPacket() to skip redundant fetches.
2026-03-21 21:17:24 +00:00
you 7315ce08d5 fix: invalidate channel cache before re-fetching on WS update
The WS handler's 250ms debounce fired loadChannels() before the
global 5s cache invalidation timer cleared the stale entry, so
the fetch returned cached data. Now channels.js invalidates its
own cache entries immediately before re-fetching.
2026-03-21 21:09:48 +00:00
you b81e2b7e56 fix: build insert row from packetData instead of DB round-trip
packets_v view uses observation IDs, not transmission IDs, so
getPacket(transmissionId) returned null. Skip the DB lookup entirely
and construct the row directly from the incoming packetData object
which already has all needed fields.
2026-03-21 21:02:12 +00:00
you 24fa68895a fix: use transmissionId for packet lookup after insert
packets_v view uses transmission IDs, not packets table IDs.
insertPacket returns a packets table ID which doesn't exist in
packets_v, so getPacket returned null and new packets never got
indexed in memory. Use transmissionId from insertTransmission instead.
2026-03-21 20:54:59 +00:00
you ec4ebf2ede fix: re-export insertPacket from db.js (packet-store.js needs it)
Migration subagent removed it from exports but packet-store.js
calls dbModule.insertPacket() on every MQTT ingestion.
2026-03-21 20:46:15 +00:00
you cbec6b3108 Move hop resolution to client side
Create public/hop-resolver.js that mirrors the server's disambiguateHops()
algorithm (prefix index, forward/backward pass, distance sanity check).

Replace all /api/resolve-hops fetch calls in packets.js with local
HopResolver.resolve() calls. The resolver lazily fetches and caches the
full nodes list via /api/nodes on first use.

The server endpoint is kept as fallback but no longer called by the UI,
eliminating 40+ HTTP requests per session.
2026-03-21 20:39:28 +00:00
you 3a1b0b544b refactor: migrate all SQL queries from packets table to packets_v view (transmissions+observations)
- Create packets_v SQL view joining transmissions+observations to match old packets schema
- Replace all SELECT FROM packets with packets_v in db.js, packet-store.js, server.js
- Update countPackets/countRecentPackets to query observations directly
- Update seed() to use insertTransmission instead of insertPacket
- Remove insertPacket from exports (no longer called)
- Keep packets table schema intact (not dropped yet, pending testing)
2026-03-21 20:38:58 +00:00
you 016c4091db perf: disable SQLite auto-checkpoint, use manual PASSIVE checkpoint
SQLite WAL auto-checkpoint (every 1000 pages/4MB) was causing 200ms+
event loop spikes on a 381MB database. This is synchronous I/O that
blocks the Node.js event loop unpredictably.

Fix: disable auto-checkpoint, run PASSIVE (non-blocking) checkpoint
every 5 minutes. PASSIVE won't stall readers or writers — it only
checkpoints pages that aren't currently in use.
2026-03-21 20:19:22 +00:00
you 0ee8992e09 perf: eliminate synchronous blocking during startup pre-warm
Previous pre-warm called computeAllSubpaths() synchronously (500ms+)
directly in the setTimeout callback, then sequentially warmed 8 more
endpoints. Any browser request arriving during this 1.5s+ window
waited the full duration (user saw 4816ms for a 3-hop resolve-hops).

Fix: ALL pre-warm now goes through self-HTTP-requests which yield the
event loop between each computation. Delayed to 5s so initial page
load requests complete first (they populate cache on-demand).

Removed the sync computeAllSubpaths() call and inline subpath cache
population — the /api/analytics/subpaths endpoint handles this itself.
2026-03-21 20:07:27 +00:00