Fix review: H-L6-02 (branch msch128/fix-h-l6-02, commit 1d8e07010)
From mnema-talk v3.3.13, rendered on this site.
Reviewer: an independent agent that wrote neither the finding nor the fix (it did the phase 2 reproduction and refutation of H-L6-02). Read-only: nothing in the repository changed, nothing pushed. Temporary probe tests were written into a throwaway worktree only and deleted afterwards. Base: msch128/hunt-lane6 at d62d54ce2. Diff reviewed: d62d54ce2..1d8e07010 (12 files).
Verdict: MERGE AFTER FIXES
The cause is removed for messages: reply_to_id and parent_id on posting, scheduling and
uploading now answer the same bytes for a hidden message and for an unknown id, and the timing
does not differ. The category variant is fixed for the probe's request only. When the request also
has an invalid field, the hidden and the missing category still answer different bytes, and their
timing differs clearly (B1, S1). Both have the same small fix. The api-stability bullet sits in the
wrong section (S2).
Findings
B1 BLOCKER: a hidden category still answers unlike a missing one when another field is invalid
internal/chat/channels/handler_channels.go:110-116 checks the category's visibility before
CreateChannelWithStage checks the name, the type and the sort order
(internal/chat/channels/channels.go:96-105, cleanChannelInputs and CheckSortOrder). A
missing category is only noticed later, in lockCategory (channels.go:80-92), after those
checks. So a request with an invalid name or type gets "category not found" for a hidden category
and the name or type error for a missing one. Measured as a holder of Manage channels, in the
canary world:
invalid name: hidden 400 "category not found" | missing 400 "name is required"
invalid type: hidden 400 "category not found" | missing 400 the type error
invalid sort order, a Stage on a text channel: the same answer for both
The fix's own claim (commit message, doc/api-stability.md bullet) is that a hidden category
answers like a missing one. This breaks it for a whole class of requests. Reach: Manage channels,
as in the original finding. Fix: check visibility where existence is checked. Pass the visibility
test into CreateChannelWithStage and apply it in lockCategory after the SELECT ... FOR SHARE
finds the row (answer errNoCategory there), or run cleanChannelInputs and CheckSortOrder
before the guard. The first option also fixes S1. Add the two invalid-input cases to the probe
test.
S1 SHOULD-FIX: the hidden category answers faster than the missing one
The hidden path returns from an in-memory snapshot check (internal/chat/core/guard.go:83-95).
The missing path opens a transaction and runs the SELECT ... FOR SHARE first. Measured over
loopback, 300 alternating requests each, all answered 400 (none rate limited):
channel in a category: hidden p10/p50/p90 143/181/250 µs missing 315/438/620 µs
The gap is about 250 µs at every percentile, larger than the spread, so a caller who repeats the
request can tell the two apart. Same reach as B1 (Manage channels). The B1 fix (checking inside
lockCategory after the row is found) gives both paths the same database work.
S2 SHOULD-FIX: the api-stability bullet is under "3.0.0 changes"
doc/api-stability.md:86-90 adds the bullet to the section that lists what the 3.0.0 major
release changed. This change ships in a 3.0.x patch, so the page now says that 3.0.0 made it.
The status change itself is acceptable. api/openapi.json documents 201, 400, 401, 403, 429 and
500 for POST /api/admin/channels, with no 404, so the old 404 broke the documented contract and
the new 400 conforms to it. Say that in the right place (the changelog, or a short "corrections
within 3.x" note) rather than in the 3.0.0 list.
N1 NOTE: the same difference on a route outside the fix (layout)
PUT /api/admin/layout: moving a channel into a hidden category answers 404 "category not found"
(internal/chat/channels/channel_permissions.go:116-117), into a missing one 404 "channel or
category not found" (channel_layout.go:118-119). Ordering a hidden or a missing category answers
the same. Manage channels again. This is not part of H-L6-02's probes. It should be recorded as its
own finding, or folded into this fix; it is not a reason to hold this commit.
N2 NOTE: messages.ValidateReplyTarget (no viewer) has no production caller
internal/chat/messages/messages.go:245-247 is kept "for webhooks", but webhooks call only
core.ValidateParent (internal/webhook/handler_store.go:39). The function is used only by
internal/chat/message_replies_integration_test.go. Harmless; it could go, or its comment could
say it is for tests.
What I checked
Cause, every variant.
core.LoadReferencedMessage(internal/chat/core/access.go:108-135) asks the Viewer only when the target's channel differs from the posting channel, and answers the not-found text when it is not visible. All three member paths pass a Viewer:Service.prepare(service_gate.go:96; app, bot API, slash commands, scheduled delivery),later.createScheduled(scheduled_handler.go:107) and uploads (media/stream.go:162-166, both the member route and the bot upload route, whose context carries the bot's user). The Viewer (post.go:41-46) isperm.Channel→Snapshot.Channel: for a conversation it is membership only, even for the Owner (perm/snapshot_channels.go:76-82), so a DM message stays hidden from the Owner too.Edit paths:
EditMessageRequest(handler_messages.go:283) andUpdateScheduledRequest(scheduled_handler.go:27) carry no reply target, so nothing revalidates one.My own probes (not in the test), outsider or former Staff member against the canary world, the hidden id against a fresh one:
- uploads with
reply_to_id/parent_id(multipart, file part after the fields): same bytes, 400 "message to reply to not found" / "parent message not found"; - poll vote (message in the open channel's path, and in the hidden channel's path), poll end, edit, delete, pin, reaction, report, save, follow thread, forum post read, interaction with a channel option: all the same status and bytes;
- interactions (read): a channel option is checked with
snap.Channel(...).Has(ViewChannel)(bot/commands_validate.go:111-115). Hidden and unknown channels both answer "a channel you can see". - wiki: pages have server-wide visibility (
wiki/handler.go:57-67). A member either reads the whole wiki or none of it, so there is no per-object hidden case to probe.
- uploads with
Timing of the message checks:
messages.ValidateTargetin-process, 3,000 alternating calls each, former Staff member, hidden id against an unknown id:reply_to_id: hidden p10/p50/p90 60.2/83.5/130.8 µs missing 59.0/82.5/129.2 µs parent_id: hidden 67.7/102.3/156.0 µs missing 65.9/100.1/154.3 µsThe extra work on the hidden path is decoding one row and an in-memory snapshot lookup: about 1 to 2 µs, far below network jitter. Both paths make the same single database round trip. The snapshot reloads only for a channel it does not know (
perm/service.go:226-235), which an existing message's channel is not. Cost on the hot path: none worth naming. A post with no target, or one in the same channel, never calls the Viewer.Nothing changed for a caller who can see the object. A reply to a visible message of another channel keeps "can only reply to a message in the same conversation" (the new test asserts it). The Owner still creates a channel in the Staff category (201, my probe). The web client already maps both reply texts (
web/src/stores/chat/messagePost.ts:18,shared/lib/errorSentencesA.ts:114,errorSentencesB.ts:84,161), and its reply fallback treats them alike, so a hidden target now gives the "reply gone" text and the same retry without the quote. Category creation shows the mapped "category not found" text whatever the status.Revert check: I restored the eight production files to d62d54ce2 and kept the new tests.
TestWritesThatNameAHiddenObjectAnswerLikeAMissingOnethen fails on 13 probes: the five old ones, plus the former member and DM cases for reply and parent, posted and scheduled. I then restored the fix.Tests:
go test -tags=integration -count=1 -p 1on internal/chat/..., internal/later, internal/media, internal/webhook, internal/slashcmd and internal/repo: all ok. The full internal/server integration suite, run once: ok (253 s).Gates: gofmt clean; go vet with and without
-tags=integrationon the touched packages clean;make complexity-checkpassed;node scripts/check-english-only.mjspassed;make docs-checkpassed. The diff touches no allowlist, limit, script or workflow, and skips no test. The changed files are well under 400 lines (largest: messages.go, 395). The commit type isfix, with no!. The test changes only tighten:echoKnownDifferencesis now empty, and 8 probes plus one positive control are added. The three changed tests in later and media only adapt to the new signatures.
Not checked
- I did not drive the timing over a real network. The numbers are loopback.
- Bot interactions with a real registered bot command. I read the option check rather than probing it end to end.
- Event comments, wiki images and the other uploads beyond the message upload route.
Round 2 (commit f6baeb86e, on top of 1d8e07010)
Verdict: MERGE AFTER FIXES
Creating a channel is now fixed. Hidden and missing categories give the same bytes for every input I tried and take the same time. The message part stays as it was. One blocker is left, in the layout route that this commit added to the fix (B2). It needs either the change below or to be taken out of this commit and recorded as its own finding.
Round 1 findings
B1 (bytes): fixed. Visibility is now decided in
lockCategory, after theSELECT ... FOR SHARE(internal/chat/channels/channels.go:80-95), so the field checks run first in both cases. The new probes (no name, bad type, bad sort order) answer the same bytes, and they fail when the production files are reverted to 1d8e07010 (below).S1 (timing): fixed. I ran the same measurement as round 1 (300 alternating requests; 290 pairs kept, 10 dropped for a 429 on either side):
round 1: hidden p10/p50/p90 143/181/250 µs missing 315/438/620 µs round 2: hidden p10/p50/p90 507/659/901 µs missing 489/657/917 µsBoth paths now open the transaction and run the same query. The difference is within noise at every percentile.
S2 (api-stability): removing the bullet is right. The 404 for a hidden category was never part of the contract:
api/openapi.jsonlists 201, 400, 401, 403, 429 and 500 forPOST /api/admin/channels, so 400 is the documented answer. The layout route keeps 404 and only the text of the hidden case changes. The commit message, which feeds the changelog, describes both. A bullet under "3.0.0 changes" was wrong, and no other section of the page lists corrections of this kind.N1 (layout): partly fixed. See B2.
N2:
messages.ValidateReplyTargetis removed. Its tests callValidateReplyTargetForwith no viewer, which is the same behaviour.
B2 BLOCKER: the layout route still tells a hidden category from a missing one when a request has two faults
CheckLayout (internal/chat/channels/channel_permissions.go:97-130) now answers a hidden
category in each list on its own like a missing one. A missing category, though, is only noticed
later, in ApplyLayoutTx (orderCategories, then placeChannels). A hidden one is decided in
CheckLayout at a different point. So when a request has a second fault, the order of the checks
gives the hidden one away. Measured as a holder of Manage channels; the hidden and the missing
request differ only in the category id:
| Request | Hidden category | Missing category |
|---|---|---|
| D1: unknown category in the order list, target category hidden or missing | 404 "channel or category not found" | 404 "category not found" |
| D2: category in the order list hidden or missing, plus an unknown channel | 404 "category not found" | 404 "channel not found" |
| D3: as D2, with a channel hidden from the caller instead of an unknown one | 404 "category not found" | 404 "channel not found" |
| D4: target hidden or missing, then an unknown channel | same | same |
| D5: target hidden or missing, then a bad sort order | same | same |
| D6: category in the order list hidden or missing, plus a missing target | same | same |
| Single faults (unknown category in the list, unknown channel, hidden against unknown channel) | same | same |
Does a double fault tell more than a single request does? Yes. Single requests are now equal, so D1 and D2 are exactly the bit the fix sets out to hide: whether a category id the caller cannot see exists. Reach and gain are those of the original category variant (Manage channels, a known category id, one bit), so this is S4. But the commit says "answers like a missing one at every step", and this is a concrete case where it does not. The author already named double faults as not done. My view: the gap is closed by the design, not by more special cases.
Fix (preferred): do not decide the hidden case in CheckLayout. Before ApplyLayoutTx, replace
every category id the caller cannot see, in both lists, with a fresh random id. Keep the
Administrator rule of checkChannelMove, treating the hidden target as unknown, as it does now.
Then the hidden request takes exactly the missing request's path: the same statements, the same
order of answers, the same database work. That also makes errLayoutMissing and the second
sort-order loop unnecessary. Add D1 and D2 as probes.
Alternative: take the layout part out of this commit, so the fix covers exactly H-L6-02's routes, and record the layout difference as its own finding.
Other checks
Body swap. I added temporary instrumentation to
mustMatchMissing. It counts the hidden ids in each probe's path and body, and the ids left in the twin under the new and under the oldswapBody. Result: 43 probes, every one carries at least one hidden id, and the new swap leaves none in any twin. The old swap would have left one id in exactly the five new layout probes (lists of objects): it would have made them fail with "names no hidden id", not pass vacuously. All 38 older probes produce the same twin under both. No older probe became vacuous.- "Layout: a hidden category after a bad channel order" and "layout: order a hidden category" also pass on 1d8e07010 (they were equal there already). They guard the order of checks and are not regression detectors for this commit.
The callers of
CreateChannelWithStage. There are 18 files and 36 call sites, all but the definition in tests (internal/chat, chat/channels, chat/talks, later, poll). The signature is unchanged. It passes a nilcanSee, andlockCategorythen behaves exactly as before (existence only). All those packages pass.Guard.CategoryViewer(internal/chat/core/guard.go:81-91). It treats a category the snapshot does not know as visible, which defers it to the existence check. That is the same rule as the oldRequireCategory, so it is not a new gap. It now reads the snapshot even when no category is named. That is an in-memory read, and its error was already fatal for this route through the Manage channels check (NOTE).The Owner and visible paths behave as before. The Owner moves a channel into the Staff category, orders that category, and moves the channel back: 204 each. The Owner creates a channel in that category: 201. The moderator moves a visible channel within no category: 204. The moderator creates a channel without a category: 201. The Owner sees every category, so the Owner has no hidden case on this route.
Revert check. I restored the five production files to 1d8e07010 and kept the round 2 tests.
TestWritesThatNameAHiddenObjectAnswerLikeAMissingOnethen fails on five probes: the three create-channel ones (no name, bad type, bad sort order), "move a channel into a hidden category" and "move into a hidden category, bad sort order". I then restored the fix.Gates, by exit code:
- gofmt: no output;
- go vet: 0, and 0 with
-tags=integration; make complexity-check: 0;node scripts/check-english-only.mjs: 0;make docs-check: 0.
The largest changed file is 328 lines. No allowlist entry or limit changed, and no test is skipped.
Tests:
go test -tags=integration -count=1 -p 1on internal/chat/..., later, poll, media, webhook and repo: all ok;- the full internal/server integration suite, run once: ok (257 s);
- my temporary probe test and instrumentation were deleted afterwards.
Not checked (round 2)
- Timing of the layout route.
- Layout requests with more than two faults, or with temporary Talks in the list (
dropTempTalksruns only on the missing path, inside the transaction that is rolled back). - Timing over a real network (loopback only).
Round 3 (commit f866222ee, on top of f6baeb86e)
Verdict: MERGE
B2 is fixed by design. SeenLayout (internal/chat/channels/channel_permissions.go:95-143)
replaces every category id and channel id the caller cannot see with a fresh random id before
anything answers. A hidden id therefore runs the missing id's path: the same checks, the same
statements, the same order of answers and the same timing. I found no remaining difference and
no change to the success path. One test-robustness point is left for a follow-up (S3 below); it
does not block.
Checks
D1 to D6 again (moderator, hidden against missing): all six answer the same bytes. D1 "category not found"; D2, D3 and D4 "channel not found"; D5 the sort-order 400; D6 "category not found".
Temporary Talks. I added a temporary Talk everyone sees and one of a conversation the moderator is not in (rows inserted directly, permissions reloaded through a no-op layout by the Owner).
- T1, the hidden conversation Talk against an unknown id: the same ("channel not found").
- T2, the visible temporary Talk plus a move into a hidden or missing category: the same
("channel or category not found").
dropTempTalksdrops the visible Talk in both.
My own generated run. A different seed (777001), 250 requests, one to four faults with at least one hidden, over a larger fault set: a hidden temporary Talk, a hidden channel moved into the hidden category, a visible temporary Talk with and without a bad target or order, a plain visible move. Each request was sent with the hidden ids and again with fresh twins.
- 0 differences, 0 accepted, and no 429 (the run fails on any 429; see S3).
- The number of channels in the Staff category was 3 before and after.
The success path cannot be changed by the replacement:
When every id is visible, nothing is replaced and
ApplyLayoutTxgets the request unchanged (the code:seenCategoryand the channel check return the id itself).A stand-in can never be written.
orderCategoriesandplaceChannelsrefuse the whole request when a row count falls short: a stand-in category matches no row, a stand-in channel matches no row, and a stand-in target fails theEXISTS (categories)guard (channel_layout.go:110-119). The transaction is rolled back. A collision with a real id is as likely as for any v4 UUID.Measured, mixed requests from the moderator:
- a visible move plus ordering the hidden category: 404;
- a visible move plus moving the hidden channel: 404;
- moving a visible channel into the hidden category: 404.
In all three nothing changed (the open channel's order and category, the Staff category's order and the Staff channel's place are the same before and after) and no audit entry was written.
A repeated channel id (last entry wins). In an earlier run, a placement into the hidden category followed by a second placement of the same channel answered 204, and the fresh-id twin did too. Before this commit,
CheckLayoutrefused the hidden variant (404), and a missing id in the same position went through (204, by reading the code). So the replacement also closes this earlier difference. What is applied is the last entry, which names no hidden id. The Staff category's contents did not change.The Owner's path: one request orders the Staff category, moves the Staff channel into it and moves the open channel: 204. The database shows exactly those values. The moderator's plain visible move: 204.
The audit entry of a successful layout change.
layout.apply(classVisPrivileged,internal/audit/classes.go:49) records the raw request body that the audit middleware peeked before the handler ran (internal/audit/middleware.go:122-142), so it holds the request's own ids. It can never hold a stand-in: stand-ins exist only inside the handler, and a request that needs one is always refused. A refused request writes no entry (successful statuses only, :136). The entries I read:- the moderator's:
{"request": {"channels": [<open> 45, <visible temporary Talk> 9], ...}}; - the Owner's: the Staff category and channel ids it named.
The one entry that names a hidden id is the repeated-id case above: an entry that succeeded with an overridden placement records the body as sent, including that id. Only holders of View audit log read it, and they see private placements anyway (the class says so). It was the same before.
- the moderator's:
Timing on the layout route. Same method as round 1: 280 alternating pairs, moving the open channel into the hidden category against a missing one, all answered 404, none rate limited:
hidden p10/p50/p90 1.78/3.43/5.69 ms missing 1.76/3.52/5.50 msEqual within noise.
Revert check. I restored
channel_permissions.goandhandler_channels.goto f6baeb86e and kept the round 3 tests.TestWritesThatNameAHiddenObjectAnswerLikeAMissingOnefails on the three new double-fault probes.TestALayoutThatNamesAHiddenObjectAnswersLikeAMissingOneWhateverElseIsWrongfails on 38 of its 300 generated requests. I then restored the fix.The author's generated test is not vacuous today. Its 600 layout requests answered 66 × 400 and 534 × 404, none 429.
Gates, by exit code:
- gofmt: 0, no output;
- go vet: 0, and 0 with
-tags=integration; make complexity-check: 0;node scripts/check-english-only.mjs: 0;make docs-check: 0.
channel_permissions.gohas 190 lines, the new test 62. No allowlist entry or limit changed, and no test is skipped. The unit testTestLayoutAndCategoryCheckschanges two expectations from refused to accepted ("into a hidden category", "hidden category order"). That is correct: the refusal now comes from applying the layout, and the test adds the assertion that no hidden id is passed on.Tests:
- the full internal/server integration suite, run once: ok (278 s);
- my temporary probe tests were deleted afterwards.
S3 SHOULD-FIX (follow-up, not blocking): a rate limit can make an echo probe pass vacuously
mustMatchMissing counts any status of 400 or more as a refusal and compares the two answers. Once
a route's rate limit is reached, both sides answer the same 429 and the probe passes without
testing anything. My first generated run showed it: after about 600 layout requests from one
moderator, every further request answered 429, and my "0 differences" would have meant nothing
had I not counted the statuses. The author's generated test sends 600 layout requests in one world
and passes today with none rate limited, but it sits close to that point. One more probe for the
same user, or a lower limit, would silently empty it. Fix: in mustMatchMissing (and in the
generated test), fail on a 429 on either side.
Not checked (round 3)
- Layout requests from a member who holds Administrator but is not the Owner (they see every category, so they have no hidden case).
- Timing over a real network (loopback only).