perf(packets): replace N+1 API calls with single expand=observations query (#580)

## Summary

Eliminates the N+1 API call storm when toggling off "Group by Hash" in
the packets table.

## Problem

When ungrouped mode was active, `loadPackets()` fired individual
`/api/packets/{hash}` requests for every multi-observation packet. With
200+ multi-obs packets, this created 200+ parallel HTTP requests —
overwhelming both browser connection limits and the server.

## Fix

The server already supports `expand=observations` on the `/api/packets`
endpoint, which returns observations inline. Instead of:

1. Always fetching grouped (`groupByHash=true`)
2. Then N+1 fetching each packet's children individually

We now:

1. Fetch grouped when grouped mode is active (`groupByHash=true`)
2. Fetch with `expand=observations` when ungrouped — **single API call**
3. Flatten observations client-side

**Result: 200+ API calls → 1 API call.**

## Changes

- `public/packets.js`: Replaced N+1 observation fetching loop with
single `expand=observations` query parameter, flatten inline
observations client-side.

## Testing

- All frontend tests pass (packet-filter: 62/62, frontend-helpers:
445/445)
- All Go backend tests pass

Fixes #382

Co-authored-by: you <you@example.com>
This commit is contained in:
Kpa-clawbot
2026-04-04 10:15:14 -07:00
committed by GitHub
parent 45d8116880
commit b37e8e2da2
+10 -12
View File
@@ -524,7 +524,11 @@
if (filters.hash) params.set('hash', filters.hash);
if (filters.node) params.set('node', filters.node);
if (filters.observer) params.set('observer', filters.observer);
params.set('groupByHash', 'true'); // always fetch grouped
if (groupByHash) {
params.set('groupByHash', 'true');
} else {
params.set('expand', 'observations');
}
const data = await api('/packets?' + params.toString());
packets = data.packets || [];
@@ -532,20 +536,14 @@
for (const p of packets) { if (p.hash) hashIndex.set(p.hash, p); }
totalCount = data.total || packets.length;
// When ungrouped, fetch observations for all multi-obs packets and flatten
// When ungrouped, flatten observations inline (single API call, no N+1)
if (!groupByHash) {
const multiObs = packets.filter(p => (p.observation_count || p.count || 1) > 1);
await Promise.all(multiObs.map(async (p) => {
try {
const d = await api(`/packets/${p.hash}`);
if (d?.observations) p._children = d.observations.map(o => clearParsedCache({...d.packet, ...o, _isObservation: true}));
} catch {}
}));
// Flatten: replace grouped packets with individual observations
const flat = [];
for (const p of packets) {
if (p._children && p._children.length > 1) {
for (const c of p._children) flat.push(c);
if (p.observations && p.observations.length > 1) {
for (const o of p.observations) {
flat.push(clearParsedCache({...p, ...o, _isObservation: true, observations: undefined}));
}
} else {
flat.push(p);
}