fix(ingestor): extract per-hop SNR for TRACE packets at ingest time (#1028)

## Problem

PR #1007 added per-hop SNR extraction (`snrValues`) for TRACE packets to
`cmd/server/decoder.go`. That code path is only hit by the on-demand
re-decode endpoint (packet detail). The actual ingest pipeline runs
`cmd/ingestor/decoder.go`, decodes the packet once, and persists
`decoded_json` into SQLite. The server then serves `decoded_json` as-is
for list/feed queries.

Net effect: `snrValues` never appears in any production response,
because the ingestor's decoder was never updated.

Confirmed empirically: `strings /app/corescope-ingestor | grep snrVal`
returns nothing.

## Fix

Port the SNR extraction logic from `cmd/server/decoder.go` (lines
410–422) into `cmd/ingestor/decoder.go`. For TRACE packets, the header
path bytes are int8 SNR values in quarter-dB encoding; extract them into
`payload.SNRValues` **before** `path.Hops` is overwritten with
payload-derived hop IDs.

Also adds the matching `SNRValues []float64` field to the ingestor's
`Payload` struct so it serializes into `decoded_json`.

## TDD

- **Red commit** (`6ae4c07`): adds `TestDecodeTraceExtractsSNRValues` +
`SNRValues` field stub. Compiles, fails on assertion (`len(SNRValues)=0,
want 2`).
- **Green commit** (`4a4f3f3`): adds extraction loop. Test passes.

Test packet: `26022FF8116A23A80000000001C0DE1000DEDE`
- header `0x26` = TRACE + DIRECT
- pathByte `0x02` = hash_size 1, hash_count 2
- header path `2F F8` → SNR `[int8(0x2F)/4, int8(0xF8)/4]` = `[11.75,
-2.0]`

## Files

- `cmd/ingestor/decoder.go` — `+16` (field + extraction)
- `cmd/ingestor/decoder_test.go` — `+29` (red test)

## Out of scope

- `cmd/server/decoder.go` is already correct (PR #1007). Untouched.
- Backfill of historical `decoded_json` rows. New TRACE packets get SNR;
old rows do not until re-decoded.

---------

Co-authored-by: corescope-bot <bot@corescope.local>
This commit is contained in:
Kpa-clawbot
2026-05-03 21:42:14 -07:00
committed by GitHub
co-authored by corescope-bot
parent dd66f678be
commit c9301fee9c
2 changed files with 45 additions and 0 deletions
+16
View File
@@ -131,6 +131,7 @@ type Payload struct {
SenderTimestamp uint32 `json:"sender_timestamp,omitempty"`
EphemeralPubKey string `json:"ephemeralPubKey,omitempty"`
PathData string `json:"pathData,omitempty"`
SNRValues []float64 `json:"snrValues,omitempty"`
Tag uint32 `json:"tag,omitempty"`
AuthCode uint32 `json:"authCode,omitempty"`
TraceFlags *int `json:"traceFlags,omitempty"`
@@ -609,6 +610,21 @@ func DecodePacket(hexString string, channelKeys map[string]string, validateSigna
}
// The header path hops count represents SNR entries = completed hops
hopsCompleted := path.HashCount
// Extract per-hop SNR from header path bytes (int8, quarter-dB encoding).
// Mirrors cmd/server/decoder.go — must be done at ingest time so SNR
// values are persisted in decoded_json (server endpoint serves DB as-is).
if hopsCompleted > 0 && len(path.Hops) >= hopsCompleted {
snrVals := make([]float64, 0, hopsCompleted)
for i := 0; i < hopsCompleted; i++ {
b, err := hex.DecodeString(path.Hops[i])
if err == nil && len(b) == 1 {
snrVals = append(snrVals, float64(int8(b[0]))/4.0)
}
}
if len(snrVals) > 0 {
payload.SNRValues = snrVals
}
}
pathBytes, err := hex.DecodeString(payload.PathData)
if err == nil && payload.TraceFlags != nil {
// path_sz from flags byte is a power-of-two exponent per firmware:
+29
View File
@@ -1947,3 +1947,32 @@ func TestDecodeTracePayloadFailSetsAnomaly(t *testing.T) {
t.Error("expected Anomaly to be set when TRACE payload decode fails but observation is stored")
}
}
// TestDecodeTraceExtractsSNRValues verifies that for TRACE packets, the header
// path bytes are interpreted as int8 SNR values (quarter-dB) and exposed via
// payload.SNRValues. Mirrors logic in cmd/server/decoder.go (issue: SNR values
// extracted by server but never written into decoded_json by ingestor).
//
// Packet 26022FF8116A23A80000000001C0DE1000DEDE:
// header 0x26 → TRACE (pt=9), DIRECT (rt=2)
// pathByte 0x02 → hash_size=1, hash_count=2
// header path: 2F F8 → SNR = [int8(0x2F)/4, int8(0xF8)/4] = [11.75, -2.0]
// payload (15B): tag=116A23A8 auth=00000000 flags=0x01 pathData=C0DE1000DEDE
func TestDecodeTraceExtractsSNRValues(t *testing.T) {
pkt, err := DecodePacket("26022FF8116A23A80000000001C0DE1000DEDE", nil, false)
if err != nil {
t.Fatalf("DecodePacket error: %v", err)
}
if pkt.Payload.Type != "TRACE" {
t.Fatalf("payload type=%s, want TRACE", pkt.Payload.Type)
}
if len(pkt.Payload.SNRValues) != 2 {
t.Fatalf("len(SNRValues)=%d, want 2 (got %v)", len(pkt.Payload.SNRValues), pkt.Payload.SNRValues)
}
if pkt.Payload.SNRValues[0] != 11.75 {
t.Errorf("SNRValues[0]=%v, want 11.75", pkt.Payload.SNRValues[0])
}
if pkt.Payload.SNRValues[1] != -2.0 {
t.Errorf("SNRValues[1]=%v, want -2.0", pkt.Payload.SNRValues[1])
}
}