Local FlexFEC-03 test harness

Scripts to validate end-to-end FlexFEC-03 between the rust-sdks local_video examples and a locally-built SFU, under simulated packet loss.

FlexFEC is terminated at the SFU (it can't be passed through because the SFU rewrites SSRC/sequence numbers per subscriber), so the two directions are independent and tested separately:

Direction	What's exercised	Loss to inject
Downlink	SFU generates a repair stream per subscriber	`down`
Uplink	SFU receives + decodes a publisher repair flow	`up`

Prerequisites

Go toolchain (to build this SFU).
A rust-sdks5 checkout with the FlexFEC changes, next to this repo (../rust-sdks5) or pointed at via RUST_SDKS_DIR. Its webrtc-sys must be built (the C++ field trials that enable FlexFEC-03 live there).
A camera (the publisher captures real video).
sudo for traffic shaping. macOS uses dnctl/pfctl (dummynet); Linux uses tc netem.

All processes run on one machine, so media flows over loopback. The SFU is pinned to UDP 7882 (see flexfec-local.yaml) so shaping can target it by port.

Files

File	Purpose
`flexfec-local.yaml`	SFU config: flexfec on (pub+sub), fixed loopback port
`run-sfu.sh`	Build + run the SFU (`--dev`, devkey/secret)
`netem.sh`	Start/stop/inspect packet-loss shaping on udp/7882
`run-publisher.sh`	Run the publisher with `--flex-fec`
`run-subscriber.sh`	Run the subscriber, surface `Video FEC ...` logs

Workflow

Use four terminals.

1 — SFU

scripts/flexfec/run-sfu.sh

Connects on ws://127.0.0.1:7880 (API key devkey / secret secret).

2 — Subscriber

scripts/flexfec/run-subscriber.sh
# or only show FEC lines on the console:
FEC_ONLY=1 scripts/flexfec/run-subscriber.sh

3 — Publisher (grant camera permission on first run)

scripts/flexfec/run-publisher.sh

4 — Packet loss (needs sudo)

# Uplink test: drop 8% of publisher -> SFU packets
sudo scripts/flexfec/netem.sh start 8 up

# Downlink test: drop 8% of SFU -> subscriber packets (+ optional delay_ms, see A/B below)
sudo scripts/flexfec/netem.sh start 8 down
sudo scripts/flexfec/netem.sh start 8 down 150   # 8% loss + 150ms one-way delay

sudo scripts/flexfec/netem.sh status
sudo scripts/flexfec/netem.sh stop   # always clean up when done

What success looks like

Downlink (down) — the subscriber recovers SFU-generated repair packets. In the subscriber log (logs/subscriber.log):

Video FEC active: fec_ssrc=..., received_total=..., discarded_total=..., accepted_recovery_payload_total=...
Video FEC recovery payload accepted: +N packets (accepted_total=...)

accepted_recovery_payload_total should climb while loss is applied.

Uplink (up) — the SFU receives + decodes the publisher's repair stream. In the SFU log:

flexfec decoder enabled   {"fecSSRC": ..., "mediaSSRC": ...}
flexfec recovery stats    {"fecPacketsReceived": ..., "packetsRecovered": ..., "packetsRecoveredDelta": ...}

packetsRecovered increasing confirms the SFU rebuilt lost uplink packets.

To also see FlexFEC SDP negotiation (flexfec ... from sdp, codec registration), set logging.level: debug in flexfec-local.yaml.

Subscriber-side A/B (isolating FEC from RTX)

On loopback RTT~=0, so NACK/RTX recovers nearly all loss before FlexFEC matters and its benefit is invisible. To make FEC the primary recovery path, add one-way delay so retransmissions arrive too late, and compare FEC on vs off under the same conditions:

# A) FEC ON (flexfec.subscriber: true). With SFU + pub + sub running:
sudo scripts/flexfec/netem.sh start 8 down 150     # 8% loss + 150ms delay, downlink
# ...observe subscriber for ~20s, then...
sudo scripts/flexfec/netem.sh stop

# B) FEC OFF: set flexfec.subscriber: false in flexfec-local.yaml, restart the SFU,
#    reconnect the subscriber, then apply the SAME shaping and observe.

The subscriber logs a Video quality: line every ~3s:

Video quality: frames_decoded=..., frames_dropped=..., freezes=N (Xs), nack=..., pli=...,
               packets_lost=..., rtx_recovered=..., fec_recv=..., fec_discarded=...

With FEC ON you expect fewer freezes/frames_dropped, lower net packets_lost, and a meaningful fec_recv; with FEC OFF the same loss leans entirely on rtx_recovered and (because of the delay) typically shows more freezes and dropped frames.

Notes & caveats

Loopback shaping (macOS): netem.sh loads its dummynet rules into a dedicated flexfec-loss pf anchor and re-applies /etc/pf.conf non-destructively. stop flushes the pipes and restores pf to its prior enabled/disabled state. If pf is set to skip on lo0 the script warns and shaping won't apply.
Direction isolation is approximate. Both clients share the SFU UDP port, so up/down are distinguished by source vs. destination port. The publisher is the dominant sender to the SFU and the subscriber the dominant receiver, so this is good enough to isolate each path in practice (some RTCP is also affected).
dnctl -q flush clears all dummynet pipes, not just ours — fine on a dev box.
Send-side BWE: if the SFU uses pacer-based send-side BWE, subscriber FEC overhead isn't accounted for by congestion control (logged as a warning by the SFU). The interceptor-based send-side BWE path TWCC-tags FEC packets correctly.
Increase loss gradually (e.g. 3% → 10%). Beyond what a single FlexFEC repair packet can cover, bursts become unrecoverable (expected).

5.8 KiB Raw Blame History