Fix review: H-L8-07 (unread SRTCP streams of a member's media connection)

From mnema-talk v3.3.13, rendered on this site.

Reviewer: phase 3, read-only. I wrote neither the finding nor the fix. Branch msch128/fix-h-l8-07, commit c3ee1762b (on 39286b819). Temporary worktrees of c3ee1762b and of 39286b819 (the base, for comparisons) were removed afterwards. Probe test files lived only inside them. Nothing was committed or pushed. Everything ran on loopback, with no audio device, camera or microphone. Load was 4 to 9 for all runs, nice -n 10, go test -p 1, one job at a time.

Verdict: MERGE AFTER FIXES. The fix bounds the memory for every attack vector I tried. One honest-client risk must be settled before merge (BLOCKER-pending 1), and SECURITY.md overstates one bound (SHOULD-FIX 2).


What the fix does (as read)

  • Every member connection gets its own Pion API (connAPI, sfu_setup.go:135-168). The settings struct is copied, and the media engine and interceptor registry are shared as before. The API carries:
    • a logger factory, rtcpStreamCap, which wraps only the "pc" scope;
    • a buffer factory that limits SRTCP buffers to 128 packets and 16 KB (RTP buffers stay at 1 MB).
  • rtcpCountingLogger.Warnf matches Pion's format string "Incoming unhandled RTCP ssrc(%d), OnTrack will not be fired" exactly and counts each stream Pion keeps.
  • SSRCs that addSender recorded (sender plus RTX) are "forgiven": they count only against the total cap.
  • Past 64 unknown streams, or 2048 in all, the connection is closed on its own goroutine. The close logs once at warn, counts in mnema_sfu_ingress_dropped_total{reason="rtcp_ssrcs"}, and from then on every SRTCP stream gets a closed zero-size buffer.

I checked the Pion sources the fix relies on (webrtc v4.2.26, srtp v3.1.3, logging v0.2.4):

  • peerconnection.go:2327-2351: the processor goroutine logs the line once per accepted stream with ssrc as a uint32, which matches the type assertion in ingress.go.
  • api.go:40-41 and peerconnection.go:143: the "pc" logger comes from the API's LoggerFactory.
  • srtp stream_srtcp.go:116-136: the BufferFactory is called on every stream init. The default buffer was a packetBuffer limited to 100 KB, which attrBuffer reproduces (attributes pass through ReadWithAttributes/WriteWithAttributes).
  • session_srtcp.go:241-251: a write error from the closed buffer ends that compound packet. This happens only after the trip.
  • srtp session.go:163: the read loop logs decrypt errors and goes on.

1. Does it bound memory?

My own flood vectors, unthrottled, each on a fresh member connection, 900 to 3000 distinct SSRCs per vector:

VectorSSRCs named before the connection stopped writingStreams Pion kept (all unknown)Closed
SR, 31 blocks133383yes
BYE, 31 sources (after an RR)300786yes
SDES, 31 chunks111680yes
REMB, 100 SSRCs300090yes
FIR, 30 entries147084yes
NACK, 40 per compound148082yes
XR DLRR, 3090078yes

The author's test adds PLI and 31-block RRs; both are covered too.

Heap per connection. Heap figures are after two forced GCs. While the connection is still open:

  • At the cap with filled streams: my slow drip (one new SSRC every 50 ms, 30 reports each) stood at +1.20 MB with 63 streams. The author's filled test measured +1.19 MB for 62 streams (19.2 KB per stream). Without the fix such a stream costs about 113 KB with 200 packets and about 320 KB when full.
  • After the close, the memory is freed within about a second: the heap one second later was already at or below the baseline in every vector.
  • A slow drip below any rate does not escape the cap, because the cap counts streams, not rate. It closed at 65.

Race between the trip and the close. Four goroutines flooding PLIs unthrottled, three rounds: Pion made 0, 5 and 7 streams after the trip, each with no buffer. The closing flag is atomic and checked by the buffer factory, the pointer to the connection is atomic, the counters are under a leaf mutex, and -race over the package is clean.

The forgiven set. Can an attacker ride it to 2048 × 20 KB? Yes, and faster than the commit message and SECURITY.md say. My probe: a member toggled its subscription to one other member's screen share through Room.Subscribe.

  • Each "on" adds a new sender with two SSRCs (media and RTX). The hub's webrtc_subscribe has no rate limit (the voice gate covers joins and leaves only).
  • 1100 toggles took 12.9 s and recorded 1101 SSRCs. Filling their streams with 31-block RRs cost +20.9 MB (18,975 B per stream), and the connection stayed open.
  • With 2200 toggles (27 s), the total cap tripped at 2049 streams and closed the connection, as designed.

So one member can hold about 2048 × 19 KB ≈ 39 MB indefinitely by stopping at 2047. Reaching it needs only one publisher in the room and about half a minute; it does not take "2048 tracks nobody reaches". How many members does it take to exhaust the server? With the shipped 1 GB container limit (GOMEMLIMIT about 870 MB) and a baseline of a few hundred MB, it takes roughly 15 to 22 cooperating members, each holding its own connection; one member alone cannot. That is a bounded, multi-member cost, so I do not call it a blocker. But the documented bound is wrong (SHOULD-FIX 2), and it is cheap to shrink (see there).

2. The logger hook

  • Log level. Pion filters by level inside DefaultLeveledLogger.logf (logging logger.go:149-151). The wrapper sees each Warnf call before that filter, so the default Error level and any PION_LOG_* setting make no difference. The app's LOG_LEVEL does not touch Pion: the only other Pion logger factory in the repo is the TURN relay's (internal/turnrelay/server.go:85). Every member connection gets its logger factory from connAPI. The shared s.api (connection test: data channel only, no SRTP) has no cap and needs none.
  • Pion changes the text. I changed the matched text in pionUnhandledRTCP (ssrc -> SSRC): TestUnhandledSRTCPStreamsAreCapped FAILS ("Pion kept 0 streams (0 unknown), heap +4038656 bytes ... not capped"). A Pion bump that changes the line therefore turns CI red. Two caveats:
    • TestRTCPStreamCapForgivesWhatTheSFUSent still passes then, because it calls unhandled directly.
    • In production the cap would silently stop working while each stream keeps the smaller buffer (about 20 KB filled, against 2.7 KB unfilled per stream).
  • A sturdier signal. Now that the API is per connection, the buffer factory itself is a per-connection hook that Pion calls for every SRTCP stream, with its SSRC. It cannot tell a read stream from an unread one on its own, but it can carry a backstop that does not depend on Pion's wording: a cap on the total SRTCP buffers it has handed out. The string remains the precise signal, so it is still needed for the 64. NOTE 3: add the backstop.

3. One API per connection

  • What the shared API shared, and what is shared now.

    • The SettingEngine is copied by value, as WithSettingEngine always did. Its pointers are copied with it, so the one ICE UDP mux, the port range, the announced addresses and the network types stay single.
    • The MediaEngine pointer is shared; Pion copies it per PeerConnection anyway (the fix does not set disableMediaEngineCopy).
    • The interceptor registry is shared. It holds the bandwidth estimator factory, NACK, TWCC, telemetry and the viewerGuard, and Pion builds its interceptors per connection, as before.
    • The logger factory is one DefaultLoggerFactory from the builder, as NewAPI would have picked.
    • NewAPI with these options allocates only its own struct and an "api" logger.
    • Nothing that must be single is duplicated.
  • Join cost. 100 s.Join calls into one room, with probe test files on both commits:

    CommitTime per joinMallocs per joinBytes allocated per joinHeap kept per peer
    base 39286b819247 µs10,7361,059,224399 KB
    fix c3ee1762b278 µs10,7391,056,461412 KB

    The differences are within noise.

  • Forwarding path. make bench-allocs shows 0 allocs/op on all four forward benchmarks.

  • Cycle tests. With SFU_CYCLE_ROUNDS=400, each run three times per commit:

    • TestSFUJoinLeaveCycleReturnsToBaseline: base +12.8 / 15.5 / 15.8 MB, fix +15.9 / 12.8 / 12.8 MB.
    • TestSFURoomOpenCloseCycleReturnsToBaseline: base +11.9 / 8.7 / 8.9 MB, fix +9.1 / 9.0 / 9.5 MB.
    • Goroutines are at baseline, except one transient extra goroutine in one fix run.

    H-L8-05's retained memory is not worse.

  • Remaining checks. The full package passes with -race (85.4 s), including the fuzz seeds. gofmt, vet (with and without -tags=integration), internal/repo, internal/metrics, complexity-check and the English check pass, and go mod tidy -diff is empty. pion/transport/v5 is direct at the same version (v5.1.4), with no new module.

4. Honest clients

What creates an unknown stream: RTCP from the client whose destination SSRC (a report block, a feedback media SSRC, an SDES chunk, a BYE source) is something the SFU neither reads nor has sent on that connection.

  • The client's own published SSRCs. The web client starts and stops its camera, screen and screen audio with replaceTrack on stable transceivers (camera.ts:42/90, screen.ts:69/232, state.ts:54 "never renegotiates"). The SSRCs do not change, and the SFU keeps reading a stopped track (stoppedAt), so stopping and starting a share many times creates no unknown stream. Simulcast layers could get an SR before Pion's RID probe opens the stream: at most one per layer SSRC per connection (about 12 for camera and screen with RTX), once each.

  • Chrome, Safari, Electron. libwebrtc gives its receive streams one shared RTCP SSRC. My probe with a Chrome-like viewer of 70 remote audio lines (RR plus SDES with one shared SSRC, three rounds) gave 1 unknown stream, and the connection stayed open.

  • Firefox: BLOCKER-pending 1. Firefox signals an a=ssrc on each recvonly m-line and uses it as the sender SSRC of that line's receiver reports (Mozilla bug 1397756; bug 1335469 is related). If it also sends the SDES CNAME chunk of the compound packet with that SSRC, as compound RTCP normally does, every line the SFU sends to a Firefox viewer adds one unknown stream. Pion does not read the SSRC of a line the remote only receives on.

    My probe with a Firefox-like viewer of 70 remote audio lines (RR plus SDES with one SSRC per line) gave 72 unknown streams, and the cap closed the honest viewer's connection.

    A Firefox member in a Talk with more than about 60 receive lines (49 others' microphones plus visible cameras and a share) would lose all media silently: the client shows one "no media connection" toast and does not rejoin (see 5). The author's instrumented suite cannot see this, because Pion clients send no per-line SDES.

    I could not confirm Firefox's SDES behaviour without a browser (none may run here). Settle it before merge, with either of:

    • a Firefox call spec in CI (they exist: ci.yml:950-973) that checks the server's count of unknown streams after a two-member call. 0 clears this point; 1 or more per received line confirms it;
    • or, independent of the outcome, forgive SSRCs that the remote description declares (the a=ssrc lines of the client's answer), read lazily on the unknown path only. That is cheap and also covers any browser that signals per-line SSRCs.
  • Long sessions and the 2048. An honest viewer adds forgiven streams for senders the SFU removed while late reports were still on the way:

    • The author measured 18 for 41 departures (about 0.44 each).
    • The web client also unsubscribes cameras whose tiles leave the screen (cameraVisibility.ts, 800 ms delay). Each scroll removes senders, and each removal is worth up to 2 forgiven SSRCs.

    At about 0.5 per removal, an honest connection reaches 2048 after roughly 4000 removals (departures plus off-screen cameras), which is rare but possible in a long, large Talk. NOTE 4.

5. The close

The hub is not told: the member keeps a silent seat, as with the existing ingress-abuse close. The web client reacts to ICE "failed" only with a one-time toast (peer.ts:88-92, mediaRoom.ts:99-103). It rejoins only after a WebSocket reconnect (session.ts:177-181), so the member stays without media until they leave and join again by hand.

For an attacker this is fine, and consistent with the existing ingress-abuse close. For a false positive (point 4) it is the reason the honest bound matters.

Reconnect and flood. Each round costs one join (the voice gate allows 1 per second, burst 3) and holds at most about 1.2 MB of streams plus about 0.4 MB of connection until the close, which frees within about a second. One member looping keeps a few MB live at most. There is no back-off beyond the voice gate, and none is needed for memory.

6. Locks, goroutines, documentation

  • Locks.
    • unhandled runs on Pion's processor goroutine, holds no SFU lock, takes only c.mu (a leaf) and starts the close on a new goroutine. The processor goroutine is the one the SRTCP session waits for while closing, so closing inline could hang.
    • sent reads GetParameters before taking c.mu. It is called from addSender, and nothing takes c.mu and then another lock.
    • The peer leaves the room through the existing closed-state handler.
    • I read the changed parts of sfu.go, sfu_join.go (Join, newReceiveConn), room_attach.go (attachPeer), room_signal.go (addSender, removeStaleSenders) and sfu_setup.go fully.
    • SetAnnouncedIPs replaces the builder under apiMu, and connAPI reads it under the read lock. Before bind, the connection pointer is nil and no close can trip; DTLS is not up yet, so no SRTCP can arrive then.
  • Revert check. With the caps and the buffer factory disabled, all three new tests FAIL. With the fix, they pass.
  • SECURITY.md. It is right that the cap counts SSRCs "the SFU never sent". The last sentence ("about 41 MB in the case nobody reaches (2048 tracks sent to it and stopped while it stays connected)") is wrong: subscription toggles reach it in about 27 s with one publisher (SHOULD-FIX 2).
  • doc/operations.md. The rtcp_ssrcs row matches the code.
  • sim-sweep.yml. Only the test name and the step name change; triggers, timeout and permissions are unchanged.
  • Commit. The type fix: is right.

Findings

  • BLOCKER-pending 1 (ingress.go:256-267, unhandled): if Firefox's per-recvonly-line RTCP SSRC (signalled as a=ssrc on that line) appears as an SDES chunk, an honest Firefox viewer of more than about 64 lines is disconnected, silently and without a rejoin. The mechanism is proven with a simulated client (72 unknown streams for 70 lines, connection closed); the Firefox behaviour is supported by Mozilla bug 1397756 but not measured. Settle it with a CI Firefox check, or forgive the SSRCs declared in the remote description.
  • SHOULD-FIX 2 (SECURITY.md new paragraph; commit message "Bound per connection"): the 2048 × 20 KB ≈ 39 to 41 MB per member is reachable by any member in about half a minute through webrtc_subscribe toggles; it is not "the case nobody reaches". Fix the text. Also consider handing a forgiven re-created stream (an SSRC whose sender's own stream was already given a buffer) a discarding buffer: Write returns len, nil; Read blocks until Close. Do not use the closed one, whose write error would cut the rest of an honest compound packet. That brings the forgiven streams to about 3 KB each (about 6 MB per member at 2048).
  • NOTE 3 (ingress.go:212-227): the cap depends on Pion's exact log text. A change is caught by TestUnhandledSRTCPStreamsAreCapped (verified), but nothing caps in production if it slips through. A count of SRTCP buffers handed out in the per-connection buffer factory would be a backstop independent of the text.
  • NOTE 4: honest forgiven streams accumulate with departures and off-screen camera churn for the life of a connection (about 0.5 per removed sender); 2048 is far but finite for a long, large Talk. A false trip costs that member its media until a manual rejoin.

What I ran

  • My own flood probes: seven vectors, a filled slow drip, the trip race (3 rounds), a Chrome-like and a Firefox-like viewer of 70 lines, and the forgiven-set ride (1100 and 2200 toggles).
  • The author's tests, including the mutated log text, and the revert check.
  • 100 joins on base and fix; the cycle tests at 400 rounds, three times each on base and fix.
  • go test -p 1 -race ./internal/sfu/, make bench-allocs, gofmt, vet, internal/repo, internal/metrics, complexity-check, the English check and go mod tidy -diff.

What I did not check

  • Any real browser (Firefox's SDES chunks in particular); a production build; the NAS.
  • The deep 2 × 50,000 run of the author's test (it runs nightly).
  • The upstream Pion change.

Round 2: commit fd4645559 (on c3ee1762b)

Reviewed after git fetch origin, in a temporary worktree (removed). Probe test files lived only inside it. Load was 3 to 5, nice -n 10, -p 1, one job at a time.

Round 2 verdict: MERGE AFTER FIXES. Points 1, 3 and 4 of round 1 are closed. Point 2 is closed for the toggle ride. But the new "forgive an SSRC once a later answer declares it" step lets a member move buffered streams out of the unknown count. That raises the bound for arbitrary SSRCs from about 1.3 MB to about 20 MB per member, and SECURITY.md states the lower figure (SHOULD-FIX R2-1). The bound still holds, so this is not a blocker.

What changed (as read)

  • rtcpStreamCap is now both the logger and the buffer factory of the connection (ingress.go:110-350).
  • Per SRTCP stream, the factory gives a capped rtcpBuffer for an SSRC the SFU reads (the media SSRC of a track it sends now) or one nobody accounts for. It gives a quietBuffer for an SSRC the client declared, an RTX SSRC of a sent track, or one of the last 1024 retired SSRCs. A quietBuffer's Write takes the packet and keeps nothing; its Read waits until Close (transport.go).
  • The connection is closed past 64 unknown SSRCs, 32,768 quiet streams, or 1024 buffered streams open at once. The last cap is counted by the factory, so it does not depend on Pion's log line.
  • SetAnswer calls declare with the applied answer (at most 4 SSRCs per m-line; the set is replaced each time; any SSRC in it leaves the unknown set). removeStaleSenders calls stoppedTrack.

Pion facts checked (webrtc v4.2.26, srtp v3.1.3)

  • RTPSender opens an SRTCP read stream only for the media SSRC (srtp_writer_future.go:63). Its Close closes it (:85-98), which reaches rtcpBuffer.Close, so the buffered count goes down and the SSRC is retired.
  • The SFU reads RTCP only from senders (room_tracks.go:24). No Pion goroutine reads a receiver's or an unhandled stream on its own.
  • The srtp session's close does not close the read streams; their owners do: the processor's deferred loop for unhandled streams (peerconnection.go:2328-2333), and senders and receivers on Stop.

So the quiet buffer's Read blocks nobody in practice, and where a reader exists, Close releases it. Nothing leaks at close. Goroutines were at baseline in the cycle tests, and -race is clean.

Round 1 vectors again (unthrottled, fresh member connection each)

  • PLI, RR(31), SR(31), BYE(31), SDES(31), REMB(100), FIR(30), NACK(40), XR DLRR(30), 3000 SSRCs each: every connection closed at 68 to 76 unknown streams. Buffered open fell back to 0 after the close.
  • 62 filled unknown streams on an open connection: +1.10 MB.
  • Trip race (4 goroutines flooding PLIs, 3 rounds): 66 unknown each, closed.
  • Toggle ride: see below.

Declaring around the cap (your first question)

  • How many m-lines. Every m-line of the answer counts. That is the SFU's fixed lines (8 in a fresh room) plus one per extra track it sends (transceivers are reused), so the count is bounded by the room's tracks, not by the client.

  • More than one SSRC per line. Adding more than one a=ssrc to a line the client sends on crashes the process through a Pion bug that is present on main, independent of this fix. That is a separate defect, reported to the coordinator privately, not detailed here. So in practice an attacker gets about 1 per line without crashing (worse: it can crash the server instead).

  • Rotation, measured. Every answer declared fresh SSRCs on each of the 7 lines without a=ssrc, and the client named them all. Over 20 s: 788 answers, 5530 declared SSRCs named (276 per second), 5578 quiet streams, 14 unknown (reports that landed after the next answer had replaced the set), 15 buffered; the connection stayed open.

    At that rate the 32,768 quiet cap is reached in about 2 minutes, not "a quarter of an hour". In a room with many lines it is reached faster: up to 4 × lines per answer if Pion tolerated it. The bound is still 32,768 × about 0.5 KB, about 16 MB, then the close.

  • Laundering buffered streams through declare (SHOULD-FIX R2-1). declare drops from the unknown set every SSRC the new answer names, but the buffered stream Pion already made for it keeps its 16 KB buffer.

    Unit probe on the cap (no connection, so nothing closes): 40 rounds of 32 unknown streams (each given a buffer, then logged), each followed by an answer declaring them (8 lines × 4). Result: unknown 0, buffered 1280. With a connection, the backstop closes at 1025 buffered.

    So a member can hold about 1024 × 19 KB ≈ 20 MB of filled buffers for arbitrary SSRCs, not 1.3 MB. My end-to-end attempt did not reproduce the climb (buffered stayed at 63; I did not find out why), so this rests on the unit probe and on reading declare.

    The forgiveness is needed: Firefox's first report from a new line can arrive before its answer. It can be bounded, for example by forgiving a formerly unknown SSRC only once per m-line, or by counting forgiven-unknown streams against their own small cap. Then the 1.3 MB in SECURITY.md holds again.

The 1024 retired window (your second question)

An SSRC falls out only after 1024 later retirements. The first late report after a stop creates the stream, and later reports reuse it, so only that first report matters. It comes within one or two RTCP intervals, about 1 to 5 s. Losing one would take more than 1024 senders removed from one viewer within those seconds. No honest Talk does that: a whole 50-member room leaving at once is about 150 to 200 SSRCs. Count: 0 in any plausible case.

Toggle cost (H-L8-05 path, the author's note)

  • 400 subscription events against one publisher's share (one on or off each, 25 ms apart, 10.2 s), no reports: +6.58 MB, 16.4 KB per event (about 33 KB per sender created and removed). 28 quiet streams, connection open.
  • The same with reports on every retired SSRC: +9.88 MB, 401 quiet streams. The difference of 3.3 MB is within the run-to-run noise of these heap numbers; the author's controlled test gives 409 to 546 B per quiet stream.
  • Correction to the author's figure (10 KB per toggle, 20 toggles a second, 200 KB a second): each webrtc_subscribe event is one toggle. The ordinary budget lets 40 a second through (internal/ws/budget.go:13), so I measure about 16 KB × 40 ≈ 650 KB a second of retained heap per member, for the life of its connection.
  • A camera batch event names up to 16 publishers (and all: true names every camera), so in a room with many cameras one event can stop up to 16 senders. By extrapolation, not measured: about 10 MB a second.

This is H-L8-05's retention driven on purpose by one member, unbounded until the connection ends. It is not this fix's path, but it is now the larger member-driven growth, and I recommend recording it as a finding of its own (S3, the same reasoning as H-L8-07).

Honest clients

  • TestFirefoxLikeViewerStaysConnected (70 and 200 lines, 0 unknown) passes. With declare made a no-op it fails; so does TestRTCPStreamCapSortsStreams.
  • Firefox's per-line SSRC stays the same across renegotiations, so replacing the set on each answer does not hurt it.
  • The cycle test at 400 rounds keeps the resident connected (heap +16.6 MB, base runs 12.8 to 15.8 MB, goroutines at baseline).
  • Chrome-like clients name no declared or unknown SSRC (round 1).

Is leaving out a CI Firefox assertion acceptable? Yes for this merge: declare covers bug 1397756's behaviour by construction, whatever Firefox does with SDES. The cheapest real-browser check, as a follow-up: log every unknown SSRC once at debug, and have e2e/run.sh fail a Firefox or WebKit run whose server log contains that line. That needs no metrics token and no new spec, and the existing two-member Firefox call specs then show 0 or not.

Seams, revert checks, gates

  • The seams change no behaviour. pionUnhandledRTCPChanged is an atomic.Bool that only tests set, and production code reads it once per unhandled stream. answerTweak exists only in sfu_test.go.

  • Revert checks (each mutation run against the six new and changed tests):

    MutationResult
    declare keeps no setTestFirefoxLikeViewerStaysConnected/70 and /200, TestRTCPStreamCapSortsStreams FAIL
    no quiet buffersthe same plus TestSubscriptionTogglesMakeQuietStreams FAIL
    no factory backstopTestRTCPStreamCapHoldsWithoutPionsLogLine FAILS
    no forgiveness for retired SSRCsTestRTCPStreamCapSortsStreams, TestSubscriptionTogglesMakeQuietStreams FAIL
  • go test -p 1 -race ./internal/sfu/: ok (93.9 s), including the fuzz seeds. make bench-allocs: 0 allocs/op on all four forward benchmarks. gofmt, vet (with and without -tags=integration), internal/repo, complexity-check and the English check pass. ingress.go is 350 lines, transport.go 245, ssrc_honest_test.go 204.

  • SECURITY.md. True except two figures:

    • "at most about 1.3 MB of buffered streams for arbitrary SSRCs": about 20 MB through declare laundering (R2-1).
    • "a quarter of an hour" to reach the quiet cap: about 2 minutes through declare rotation.

Findings, round 2

  • SHOULD-FIX R2-1 (ingress.go declare, the loop that deletes from c.unknown): a later answer moves buffered unknown streams out of the unknown count, so arbitrary-SSRC buffered streams reach the 1024 backstop (about 20 MB) instead of 64 (about 1.3 MB). Bound the forgiveness (once per m-line, or a separate small cap) and correct the two SECURITY.md figures.
  • NOTE R2-2: a quiet-stream rotation reaches the 32,768 cap in about 2 minutes (16 MB, then the close). This is acceptable, but the doc's time estimate is wrong.
  • NOTE R2-3: follow-up, a debug line per unknown SSRC plus a check in e2e/run.sh, as the cheap real-browser check.
  • NOTE R2-4, outside this fix: subscription toggles retain about 16 KB per event (H-L8-05) at up to 40 events a second, about 650 KB/s per member for the connection's life, more with camera batches. Recommend a separate finding.
  • Separate defect found during this round, present on main 3.0.9: a crafted answer crashes the process through a Pion nil dereference. Reported to the coordinator with a repro file under findings/; not detailed in this public report.

Round 3: commit e365c714d (on fd4645559)

A small delta check after git fetch origin, in a temporary worktree (removed); probe files lived only inside it. Load was 4 to 5.

Round 3 verdict: MERGE AFTER FIXES. The laundering through repeated answers is closed, and the honest race is still forgiven. But the m-line count the budget uses comes from the client's answer, and Pion accepts an answer with more m-lines than the offer. The bound is therefore "65 + whatever m-lines fit in one 64 KB answer", about 9 MB rather than "under 3 MB" (SHOULD-FIX R3-1). It is bounded below the 1024 backstop, so this is not a blocker.

Checked

  • The honest race. Unit probe on the cap: lines were added 1 to 3 per renegotiation, up to 120 lines, and each new line's first report beat its answer. Result: 0 unknown, so every one was forgiven. With the forgiveness set to 0, TestRTCPStreamCapSortsStreams fails, so the case is under test.

  • The round 2 laundering probe (40 rounds of 32 unknown streams, each followed by an answer of 8 lines declaring them): unknown 1272, buffered 1280. Nothing is forgiven after the first answer, so with a connection bound it closes at 65. TestDeclaringReportedSSRCsDoesNotLaunderThem passes ("closed after 3 rounds: 88 unknown, 73 buffered"). With unlimited forgiveness it fails.

  • Can a client inflate the m-line count? Yes (SHOULD-FIX R3-1). End-to-end, through the test client's answer tweak:

    • Three copies of the answer's last m-section with new mids: SetAnswer succeeded, the cap counted 11 lines (8 + 3), and the connection stayed up.
    • Minimal sections without a=rtpmap are refused by Pion ("payload type not found").
    • With a=rtpmap (about 100 bytes per section): +100 lines in a 37 KB answer, accepted, cap lines 108, connected. +450 lines in a 73 KB answer (above the hub's 64 KB message cap, so not reachable through the hub) was also accepted, cap lines 458.

    Nothing refuses an answer whose m-line count differs from the offer before declare runs. A member can therefore raise lines to about 350 to 600, depending on the size of the rest of its answer, and so forgive that many buffered unknown streams: about (65 + about 400) × 19 KB ≈ 9 MB.

    The fix is small: take the line count from the SFU's own offer (its local description, which only the SFU controls), or the smaller of the offer's and the answer's. Then the doc's "65 + its m-lines" holds with m-lines meaning the SFU's. That Pion applies answer m-lines the offer never had is worth a look together with the separate answer defect, which the coordinator is fixing.

  • Other tests. TestFirefoxLikeViewerStaysConnected (70 and 200 lines) and TestSubscriptionTogglesMakeQuietStreams pass.

  • Exit codes. go test -p 1 -race ./internal/sfu/: exit 0. make bench-allocs: exit 0, with 0 allocs/op on all four forward benchmarks. gofmt, vet (with and without -tags=integration), internal/repo, complexity-check and the English check: exit 0.

  • SECURITY.md. "About two minutes" for the quiet streams is right. "At most (65 + its m-lines) ... under 3 MB for 70 lines" is true only if the m-lines are the SFU's; today a client can claim about 400 (R3-1).

Findings, round 3

  • SHOULD-FIX R3-1 (ingress.go declare and forgiveNewLines): the m-line count is taken from the client's answer, which Pion accepts with m-lines the offer never had (measured: +3 and +100 lines accepted, cap lines 11 and 108). Count the SFU's own offer lines instead, and keep SECURITY.md's bound as written.

Round 4: commit d77aa2893 (on e365c714d)

A small delta check after git fetch origin, in a temporary worktree (removed); the probe file lived only inside it. Load was 6 to 9.

Round 4 verdict: MERGE. R3-1 is closed, the moved stoppedTrack call starves no live sender, and nothing new was found.

  • R3-1, end to end. I reran my probe: the answer gets 3, then 100 extra sections, each with a=rtpmap and its own a=ssrc. In both cases SetAnswer succeeds (Pion still accepts such answers), but the cap counts 8 lines (the offer's), and the only declared SSRC is the client's own microphone. Nothing from the extra sections is declared or forgiven. The new TestAnswerWithMoreLinesThanTheOfferForgivesNoMore passes, and fails when the offer's line count is replaced by "unlimited".
  • The offer is taken from PendingLocalDescription() before SetRemoteDescription. Without a pending offer, Pion refuses the answer, so declare never runs with an empty offer. An answer with fewer lines than the offer only shrinks the count.
  • stoppedTrack before RemoveTrack (removeStaleSenders).
    • Pion's RemoveTrack (webrtc v4.2.26 peerconnection.go:2519-2543) fails only when the connection is closed, when the sender is not this connection's (impossible: it comes from the peer's own senders), or after sender.Stop() has already run (a Stop or setSendingTrack error).
    • So whenever stoppedTrack ran and RemoveTrack then failed, the sender is stopped or the connection is closing. No live sender is left with a retired SSRC.
    • Even in that case, a live sender's SRTCP stream is opened once at Send, and its buffer is chosen then. Retiring the SSRC later changes only streams Pion creates afterwards, so forwardRTCP (NACK, PLI) keeps reading the sender's buffered stream.
    • Lock order is unchanged: GetParameters (the sender's lock) before c.mu, a leaf; the call only moved within the same function.
  • Runs:
    • TestSubscriptionTogglesMakeQuietStreams 10 times with -race: exit 0, 0 failures.
    • go test -p 1 -race ./internal/sfu/: exit 0.
    • make bench-allocs: exit 0, 0 allocs/op on all four forward benchmarks.
    • gofmt, vet (with and without -tags=integration), internal/repo, complexity-check and the English check: exit 0.
  • SECURITY.md. "65 + the offer's m-lines" now matches the code.

Final state of the review: round 1 points 1 to 4, R2-1 and R3-1 are all closed. The remaining NOTEs are follow-ups and do not block the merge:

  • the debug-log check for real browsers in e2e (R2-3);
  • the separate finding recommended for subscription toggles retaining per-sender state (R2-4, the H-L8-05 path);
  • the answer crash, which the coordinator is handling.

Something wrong on this page? Tell us.