Review: each lower screen layer gets its own copy of the capture, and the camera's clone gaps are closed (PR #836, heads 577df35b0 and dd4c92fad)
From mnema-talk v3.3.13, rendered on this site.
Reviewer: an agent that did not write the change. Read-only, with a focus on media and privacy. Two rounds, each in its own detached worktree, both now removed.
Final verdict
MERGE once the CI run on dd4c92fad is green. At review time no check on that head had finished.
- The round 1 required finding is fixed and tested. A quality change now gives the lower lines new clones, which carry the new hint and constraints. The new test fails when the fix is removed (round 2, section R2.1).
- Notes 1, 3, 6 and 7 are addressed. One cosmetic point from #824 remains (round 2, section R2.2).
- Nothing new broke. The peer.ts change keeps the reconnect behaviour and drops the per-renegotiation syncs. E2EE is untouched (round 2, section R2.3).
- Nothing blocks. Round 2 has four notes.
Round 1 (head 577df35b0)
Reviewer: an agent that did not write the change. Read-only, with a focus on media and privacy. I used my own detached worktree at 577df35b0, now removed.
Commit reviewed: 577df35b0 (one commit).
Files: web/src/talk/composables/webrtc/ layers.ts, screen.ts, peer.ts, state.ts, cameraLayers.ts, cameraLayerSync.ts, the new layerClones.ts, and three test files. No file under web/src/lib/media/ (E2EE) changed.
I made mutations and one throwaway probe test in the scratch worktree only, and reverted each one after its run. Nothing was committed or pushed.
Verdict
MERGE AFTER FIXES. One fix is required, and it is small:
- The lower screen lines keep a stale
contentHintafter an "Optimise for" change.applyScreenQualitysets the new hint on the capture (senders.ts:82) and thensyncLayersruns (watchers.ts:72). ButlayerClonereuses the held clone because the source is the same and still live (layerClones.ts:16). It copies the hint only when it makes a clone (:19). A probe test confirmed it. After a share starts asdetailand switches to{ resolution: 1080, fps: 30, optimize: 'motion' }, the capture reportsmotion, and the low line still carries the same clone withdetail. MeanwhiledegradationPreferenceon that sender is alreadybalanced. On main, the lower lines carried the capture itself and followed the hint, so this is a regression this PR introduces, and it contradicts the PR's own "contentHint kept" claim. The fix: inlayerClone, setheld.clone.contentHint = source.contentHinton reuse, or stop the screen clones inside theapplyScreenQualitychain step. Either needs a test that pins the hint after a quality change.
Everything else holds:
- No path leaves a screen clone live after the share stops, and none can make a clone after the stop (section 1).
- E2EE is unchanged (section 3).
- Lint and the talk tests pass. The core mutations are caught (section 5).
The notes below do not block.
1. Clone lifecycle (screen)
The clones live in st.screenLayerClones (state.ts:67, :167). layerClone (layerClones.ts:14-22) makes them and dropLayerClone (:25-28) stops them. layerClone reuses a clone only while it copies the same capture and is live. Otherwise it stops the old clone first.
| Path | What happens | Evidence |
|---|---|---|
stopScreenShare (button, menu, logout, gaming action) | stopScreenLayerClones runs synchronously, before the capture tracks are stopped. syncLayers later takes the lines off | screen.ts:239, :245, :254; layers.ts:153-156 |
Capture onended (the browser's own "Stop sharing") | calls stopScreenShare while the stream is current | screen.ts:35 |
| Leaving the call or being taken over | endCall runs cleanupPeerConnection (which keeps the clones) and then stopScreenShare, which stops them | session.ts:217, :219 |
| Call becomes unprotected | onMediaUnprotected runs stopScreenShare | useWebRTC.ts:48-49 |
Source switched (replaceScreenShare) | old clones are stopped synchronously, right before the new stream is set, with no await in between. The new clones come from the next sync | screen.ts:220-223 |
retuneForCapture (the capture changed size or rate) | all clones are stopped synchronously. The queued sync makes new ones | layers.ts:166-170 |
| Line no longer fed (no plan for it) | dropTrack stops its clone and takes the line off | layers.ts:56-59, :122-124 |
| No track or no best sender | releaseLayers drops every line's clone, then all clones again | layers.ts:62-71 |
replaceTrack fails | the clone stays held but is on no sender. The next sync reuses it, and any stop path stops it. The capture is live then anyway | layers.ts:91 |
| Reconnect to an offer with fewer or no layer lines | runSyncLayers first drops the clones of every slot without a sender (dropMissingSenderClones), before its early return. finishOfferNegotiation now runs the sync when a share was bound or clones exist | layers.ts:135-136, peer.ts:241-243 |
Race with the async qualityChain. feedLayer checks the current capture (layers.ts:89) and makes the clone (:90) with no await in between. Every path that ends or swaps the capture does its clone stop and its localScreenStream change in one synchronous block:
stopScreenShare: screen.ts:239 and :246.replaceScreenShare: :220-221.endCall: throughstopScreenShare.
So an in-flight sync either made its clone before the stop, and the stop catches it, or it sees the new or empty stream and returns false. A clone made before the stop can still reach a sender through its pending replaceTrack, but by then it has already ended. A stopped track keeps no capture open. No clone can be created after the share stopped. The camera has the same structure (cameraLayerSync.ts:19-20).
Camera gaps from the #824 review.
- Note 2 (unused clones after a reconnect without layer lines) is closed.
dropMissingSenderClonesruns before the early return (cameraLayerSync.ts:51-52). The sync is now triggered after such a reconnect (peer.ts:244). Mutations M3b and M8 catch the loss of either part. - Note 1 (a line no longer fed keeps its clone) is closed as well. A test now pins it (useWebRTC.cameraLayers.test.ts, "stops a lower layer's clone when the capture becomes too small"). Mutation M6 is caught.
- The #824 property "a reconnect reuses the same clones" still holds. The drop only runs for slots without a sender, and the sync runs after the rebind. The existing test still passes.
2. Side effects
retuneForCapturestops the clones before the re-sync. The lower senders then sit on ended tracks until the queued chain reachesfeedLayer. That wait covers anything already queued, plus onesetParameterson the best line and one per lower line.- In the normal case this is milliseconds. The capture's size just changed, so the encoders reconfigure anyway.
- The SFU moves an automatic viewer only after
adaptStall(2 s, internal/sfu/adapt.go:61, viewer_track.go:217-229). A gap that long would need a slowapplyConstraintsqueued ahead. - Stopping each clone inside the chain, right before its line is fed again, would remove the gap (note 2).
- The changed condition in
finishOfferNegotiation(peer.ts:241-246).bound.screenandbound.cameraare true only when the best line was bound fresh on this connection and a share or camera runs (senders.ts:137-145). That is the reconnect case, and it is cheap: with no layer lines,runSyncLayersreturns after the drop.- The
*LayerClones.some(Boolean)terms are true during every share or camera with layer lines. So every server renegotiation of the same connection (joins, leaves) now runs a full screen sync and a full camera sync. That is up to sixsetParameterscalls, becausetuneSenderdoes not skip unchanged values (senderTuning.ts:72-81). - Nothing behaves wrongly: the values are the same, and
declareLayersdeduplicates (layers.ts:100). But it is needless work on every renegotiation. - The only case those terms add beyond
bound.*andfresh.*is a reconnect whose best line could not be bound. Keying the sync on "this offer bound any sender fresh" would cover that without the per-renegotiation cost (note 3).
- The best line is unchanged. It still carries the capture itself (layers.ts has no clone at index 0, and the tests pin
screen.track === track).
3. E2EE
- The lower screen senders come only from
bindTransceiver(conn, mid, null, SOURCE_SCREEN, i + 1)(senders.ts:148-153). That call runsmediaCall().protectSenderbefore it switches the line tosendrecv, and returnsnullwhen protection fails (:94-104). st.layerSendersis set only from that return value and reset to null with the connection (peer.ts:30).- Clones are placed only in
feedLayer(layers.ts:91), onto entries of a copy ofst.layerSenders(:139). So every clone goes onto a sender whose transform was armed when it was bound. replaceTrackswaps the track and leavessender.transformas it is. The source (screen) and the layer index are the same as before.- An in-flight sync that still holds the senders of a connection closed by a reconnect puts a clone, at most, on a protected sender of a closed connection. Nothing goes out there.
- Conclusion: the lower screen senders are protected exactly as before, and no path puts a clone on an unprotected sender.
4. contentHint and later constraints
- At creation:
layerClonecopiessource.contentHint(layerClones.ts:19), and a new test pins it. - Hint changed later: not followed. This is the required fix.
- Constraints applied later (
applyScreenQuality:track.applyConstraints(trackConstraints(quality)), senders.ts:76): a clone keeps the constraints it was cloned with.- The clones are recreated only through
retuneForCapture. That runs when the capture watch sees newwidth/height/frameRatesettings, after up toCAPTURE_POLL_MS + CAPTURE_SETTLE_MS(1.5 s). - A quality change that changes the delivered size or rate therefore converges after about 1.5 s. Until then, the lower plans are computed from the capture's settings while a clone may still deliver its old size.
- The comment at layers.ts:164-165 gives constraints as the reason for the stop in
retuneForCapture. YetapplyScreenQuality, the only place that applies constraints, does not recreate the clones itself (note 1).
- The clones are recreated only through
- Pre-existing, now wider:
cleanupPeerConnectionstops the capture watch (peer.ts:17), and the reconnect path (startWaitingScreen, peer.ts:183-192) does not start it again. OnlysendScreenVideodoes (screen.ts:76).- After a reconnect, neither the best line nor, now, the clones follow a resize or a quality change until the share is restarted.
- This is not introduced here. With clones, though, it also leaves the lower lines on stale constraints (note 4).
5. Tests
nice -n 10 npx vitest run src/talk: 92 files, 1046 tests passed. nice -n 10 npm run lint: 0 errors, 23 warnings, none in the changed files (ESLint on the changed files alone: clean). i18n strict: 0 findings.
I ran each mutation against src/talk/composables (467 tests) and reverted it:
| # | Mutation | Result |
|---|---|---|
| M1 | Lower screen lines share the capture again (const clone = track, layers.ts:90) | 9 fail (own clone, reuse, stop, retune, 4K layers, announced lines, stop-and-redeclare, quality change) |
| M2 | stopScreenShare no longer stops clones (screen.ts:239 removed) | 1 fails (synchronous stop) |
| M3a | Screen reconnect drop removed (layers.ts:135) | 1 fails (layers.test.ts, "stops clones of lines that have no sender") |
| M3b | Camera reconnect drop removed (cameraLayerSync.ts:51) | 1 fails (camera reconnect test) |
| M4 | retuneForCapture no longer stops clones (layers.ts:166) | 1 fails |
| M5 | replaceScreenShare no longer stops clones (screen.ts:220) | all pass (note 5) |
| M6 | Camera line no longer fed keeps its clone (cameraLayerSync.ts:39) | 1 fails |
| M7 | Screen line no longer fed keeps its clone (layers.ts:57) | all pass (note 5) |
| M8 | peer.ts camera condition back to fresh.cameraLayers only | 1 fails |
| M9 | peer.ts screen condition back to fresh.layers only | all pass (note 5) |
| M10 | The second clone stop in releaseLayers removed (layers.ts:68) | all pass (note 6) |
Probe (throwaway test, deleted): share as detail, switch to optimize: 'motion'. The capture reports motion, and the low line holds the same clone, which reports detail. That is the required fix.
6. Code quality (AGENTS.md)
- Sizes: layers.ts 171, screen.ts 285, peer.ts 312, state.ts 218, cameraLayers.ts 55, cameraLayerSync.ts 75 and layerClones.ts 35 lines (limit 400). The tests are 781 and 259 lines (limit 800; useWebRTC.layers.test.ts is close).
- Functions and complexity: ESLint (the function size and sonarjs cognitive complexity rules) is clean on every changed file. Extracting
syncCameraSenderLayerskeepsrunSyncCameraLayerssmall. - Sharing: moving the clone helpers into layerClones.ts for both camera and screen is the right split.
releaseCameraLayers(cameraLayers.ts:54) andstopScreenLayerClones(layers.ts:153-156) are the same loop. An exporteddropAllLayerClones(slots)in layerClones.ts would remove the duplicate, andstopScreenLayerClonestakescxonly to reachst. - Stale comments:
- state.ts:12 still says "A lower camera layer's own copy", although the type now serves the screen too.
- state.ts:71 still points to
cameraLayers.layerClone, which moved to layerClones.ts. LayerClonestill sits between two import blocks (#824 note 3).
- No dead code beyond the redundant second stop in
releaseLayers(note 6).
Runs
| What | Command / source | Result |
|---|---|---|
| Setup | npm ci in web/ | ok |
| Unit, talk feature | nice -n 10 npx vitest run src/talk | 92 files, 1046 tests passed |
| Web lint | nice -n 10 npm run lint | 0 errors, 23 warnings (none in changed files); i18n strict 0 findings |
| ESLint, changed files | npx eslint src/talk/composables/webrtc/ <changed tests> | clean |
| Mutations | section 5 | 8 of 11 caught; M5, M7, M9 and M10 survive (note 5, note 6) |
| Probe | throwaway test on the contentHint after an optimise change | stale hint confirmed |
| CI on the head | PR checks at review time | secret-scan, govulncheck, analyze (actions) green; frontend, e2e, backend and CodeQL queued |
All local commands ran with nice -n 10. The load stayed under 8.
Notes (none blocking)
- Constraints reach the clones only through the capture watch. After
applyScreenQuality, the clones keep their old constraints untilretuneForCapturefires (up to 1.5 s, and only if the settings change). Stopping the screen clones in theapplyScreenQualitychain step also fixes the required item. The comment at layers.ts:164-165 should then name the right trigger. - Freeze window in
retuneForCapture. The clones are stopped before the queued re-sync, so the lower lines carry ended tracks until the chain reaches them. This is short in practice, and well below the SFU's 2 s stall switch. Stopping each clone inside the chain, right before its line is fed, would close the gap. - Needless sync on every renegotiation.
screenLayerClones.some(Boolean)andcameraLayerClones.some(Boolean)(peer.ts:241, :244) run both full syncs on every server renegotiation during a share or camera, withsetParameterson unchanged values. A condition on "this offer bound a sender fresh" covers the reconnect cases. - Pre-existing: the capture watch is not restarted after a reconnect (peer.ts:17, screen.ts:76). It predates this PR, but now the clones also keep stale settings after a reconnect. Worth a follow-up.
- Test gaps.
- M5: the replace test passes without the explicit stop, because
layerCloneswaps clones lazily. The test does not check that the old clones are stopped before the new capture is fed, nor the case where the sync after a replace aborts. - M7: no screen test checks that a line that loses its plan stops its clone. The capture runs anyway, so this is not an indicator leak.
- M9: the screen side of the peer.ts condition is only unit-tested in
runSyncLayers(layers.test.ts), not through a reconnect with fewer layer lines.
- M5: the replace test passes without the explicit stop, because
- Redundant stop.
releaseLayersstops each line's clone indropTrackand then all clones again (layers.ts:64-68). Together withdropMissingSenderClones(:135), the second loop is never needed. - Housekeeping: the stale comments at state.ts:12 and :71, and the duplicate stop-all loop (section 6).
Not checked
- I did not run e2e or call specs locally (rule: CI only, no camera, microphone or audio on this machine). Most CI checks on 577df35b0 were still queued at review time.
- The browser's adaptation behaviour, and whether
applyConstraintson a display capture also reconfigures its clones, rest on knowledge of Chromium's track adapters, not on a source read or a measurement. Firefox and WebKit may differ. - I did not measure CPU, power or memory with two extra screen clones.
- The SFU and the E2EE code were read only where they bear on this change (stall detection,
protectSender).
Outcome (round 1)
Merge after the contentHint fix and a test for it, once CI is green. Notes 1 to 3 and 5 to 7 are a small follow-up and can go into the same PR. Note 4 predates this change.
Round 2 (head dd4c92fad)
Commit reviewed: dd4c92fad, one fix commit on top of 577df35b0. It touches layerClones.ts, layers.ts, peer.ts, watchers.ts, cameraLayers.ts, state.ts and useWebRTC.layers.test.ts. No file under web/src/lib/media/ changed, and sender binding (senders.ts) is unchanged.
I reviewed it in a fresh detached worktree, and made mutations there only, each reverted after its run.
Verdict (round 2)
MERGE once CI is green on dd4c92fad. Nothing blocks.
R2.1 The required finding
The fix has two parts.
- A new clone after every quality change. The
voice.screenQualitywatcher now runsstopScreenLayerClonesonceapplyScreenQuality's chain step has settled, and thensyncLayers(watchers.ts:73).- The new clones are made after
applyConstraintsand after the capture got its new hint. So they copy both, which also settles round 1 note 1. - The
.thencallback is registered on the chain promise before anything queued later, so it runs before later chain steps start. A layer sync queued in between finds the clones stopped and makes new ones. The watcher's own sync then reuses them. - When the share was stopped or replaced in the meantime, the stop hits no clones, or the replacement's fresh ones, which the queued sync makes again right away. Nothing stays live.
- The new clones are made after
- The hint is refreshed on reuse. A reused clone now takes the capture's current hint (layerClones.ts:16-20).
Test. "gives the lower lines new copies with the new hint when the share switches to motion" (useWebRTC.layers.test.ts:699-716). It checks that the old clones are stopped, that both lower lines carry motion, and that no line still holds an old clone.
| Mutation | Result |
|---|---|
| R1: watcher stop removed (hint refresh kept) | 1 fails (the new test) |
| R2: hint refresh on reuse removed (watcher stop kept) | all pass (note R2-1) |
| R3: both removed, i.e. round 1's code | 1 fails (the new test) |
Conclusion: fixed, and the test fails without the fix.
R2.2 Round 1 notes 1, 3, 6 and 7
| Note | Status | Evidence |
|---|---|---|
| 1: constraints reach the clones only through the capture watch | Fixed: a quality change makes new clones after applyConstraints. The comment in retuneForCapture now gives the right reason | watchers.ts:72-73, layers.ts:159-163 |
| 3: both syncs on every renegotiation | Fixed: finishOfferNegotiation stops the clones of missing lines synchronously and syncs only when layer lines were bound fresh, as on main | peer.ts:242-251 |
6: redundant second stop in releaseLayers | Fixed: removed. dropTrack covers lines with a sender, and dropMissingSenderClones at the start of runSyncLayers covers the rest | layers.ts:61-70, :134 |
| 7: stale comments, duplicate stop-all loop | Fixed: dropAllClones in layerClones.ts serves camera and screen, and the state.ts comments are corrected. LayerClone still sits between two import blocks (cosmetic, #824 note 3) | layerClones.ts:41-44, cameraLayers.ts:54, layers.ts:152-154, state.ts:12-13, :67, :71 |
Not in scope for this round and unchanged: round 1 notes 2 (freeze window), 4 (capture watch not restarted after a reconnect, which predates this PR) and 5 (test gaps M5 and M7).
R2.3 Does anything new break?
peer.ts (finishOfferNegotiation).
- Reconnect with the same lines.
bindPublishSendersbinds the new senders beforefinishOfferNegotiationruns. SodropMissingSenderClonessees bound senders and keeps their clones, and the sync on the freshly bound lines reuses them. The camera reuse test from #824 still passes. - Reconnect with fewer or no layer lines. The clones of the missing lines are stopped right after the offer, with no sync needed. For the camera this is pinned (mutation R5 fails). For the screen it is not (note R2-2).
- Renegotiation on the same connection. No sync and no
setParameterscalls any more; only two cheap synchronous loops. This is the same as main. - Dropped
bound.screen/bound.cameraterms. A share or camera that is rebound after a reconnect gets fresh layer lines whenever the server offers them, sofresh.layers/fresh.cameraLayerstrigger the sync as before. Without layer lines, there is nothing to sync. - Early returns. If the offer turns stale (
isCurrent) or the screen start fails, the drop is skipped. The next offer, or the next sync (layers.ts:134, cameraLayerSync.ts:51), catches it. The capture is live in that window, so the sharing indicator stays truthful.
Clone lifecycle still holds. The synchronous stops in stopScreenShare, replaceScreenShare and retuneForCapture are unchanged (screen.ts:220, :239; layers.ts:163). The check before layerClone still runs with no await in between (layers.ts:88-89).
| Mutation | Result |
|---|---|
| R4: peer.ts screen drop removed | all pass (note R2-2) |
| R5: peer.ts camera drop removed | 1 fails (camera reconnect) |
| R6: every missing-line drop removed (peer.ts and both syncs) | 2 fail |
| R7: lower screen lines share the capture again | 10 fail |
R8: stopScreenShare no longer stops clones | 1 fails |
E2EE: unchanged. The lower senders are still bound only through bindTransceiver with protectSender, and no new path calls replaceTrack.
Size and lint: layers.ts 168, peer.ts 316, watchers.ts 102 and layerClones.ts 44 lines. useWebRTC.layers.test.ts is at 800 lines, the exact limit, and lint accepts it (note R2-3). ESLint is clean on webrtc/.
Runs (round 2)
| What | Command / source | Result |
|---|---|---|
| Setup | npm ci in web/ | ok |
| Unit, talk feature | nice -n 10 npx vitest run src/talk | 92 files, 1047 tests passed |
| Web lint | nice -n 10 npm run lint | 0 errors, 23 warnings (none in changed files) |
ESLint, webrtc/ | npx eslint src/talk/composables/webrtc | clean |
| Mutations | R1-R8 above | 6 of 8 caught; R2 and R4 survive |
| CI on the head | PR checks at review time | none finished yet |
All local commands ran with nice -n 10. The load stayed under 5.
Notes (round 2, none blocking)
- R2-1. The hint refresh on reuse has no observable effect. The hint changes only in
applyScreenQuality(senders.ts:82) and at the start of a share. Every quality change now makes new clones, so the reuse branch never meets a changed hint. Mutation R2 confirms it: all tests pass without it. Keep it as a cheap guard, with its comment saying so, or remove it as dead code (AGENTS.md: no dead code). - R2-2. The screen side of the missing-line drop in peer.ts is not tested. Removing peer.ts:243 leaves every test green; it is the same gap as round 1's M9. Without it, after a reconnect to an offer without screen layer lines, the old screen clones stay live until the share stops. The capture is live then, so this is not an indicator leak. A reconnect test like the camera one would pin it.
- R2-3. useWebRTC.layers.test.ts is at exactly 800 lines. The next screen-layer test has to go into a split file.
- R2-4. A short freeze now also happens on every quality change. It is the same class as round 1 note 2: the lower lines carry ended tracks from the watcher's stop until the queued sync feeds them. The quality change reconfigures the encoders anyway, and the gap is far below the SFU's 2 s stall switch.
Not checked (round 2)
- I did not run e2e or call specs locally (CI only, no camera, microphone or audio on this machine). CI on dd4c92fad had not finished at review time.
- I did not verify in a real browser that a clone made after
applyConstraintscarries the capture's new constraints (Chromium behaviour, from knowledge, not measured).
Outcome (round 2)
Merge once CI on dd4c92fad is green. Notes R2-1 and R2-2 are a small follow-up.