Fix review H-L6-05 (an event located in a conversation's call Talk reaches the audit log)
From mnema-talk v3.3.13, rendered on this site.
Reviewer: independent reviewer. I raised the candidate in the H-L6-04 review, but I wrote neither the test nor the fix. I worked read-only: nothing in the repository changed and nothing was pushed. The scratch tests ran in my worktree and were deleted afterwards.
Branch: msch128/fix-h-l6-05, based on 93dae338e (the H-L6-04 fix).
- 625d71d98: test, known failure.
- 8c95e9466: fix.
Verdict: MERGE AFTER FIXES
The fix removes the cause for every way an event can be created or moved: create, series, edit, and edit with "this and following". It also covers the automod sibling. No BLOCKER. Three SHOULD-FIX items:
- SF-1: events that already name a call Talk still leak when cancelled.
- SF-2: an add/add file name collision with the H-L6-03 branch.
- SF-3: one extra query on the call-Talk path. It is a timing difference, small and hard to use.
The two paths the author reported but did not fix are a separate finding (section 6). They are not a condition for this merge.
1. Reproduction on the base
- At 625d71d98,
TestKnownFailureAnEventLocatedInACallOfAConversationReachesTheAuditLogpasses while the defect reproduces: "H-L6-05 still reproduces: the Owner reads okapievent, the Owner reads, an Administrator reads ...". - At 8c95e9466 with only handler_validate.go reverted, the guard test fails: "create in the call: got 201".
2. A hidden call answers like a missing channel
I posted POST /api/events 120 times per case, all as a holder of Manage events:
| Case | Status | Same bytes as a random id | Median | p10 | p90 |
|---|---|---|---|---|---|
| random id | 400 | (reference) | 0.48 ms | 0.38 ms | 0.69 ms |
| call Talk, caller is a member | 400 | yes | 0.60 ms | 0.46 ms | 0.84 ms |
| call Talk, caller is not a member | 400 | yes | 0.62 ms | 0.49 ms | 0.80 ms |
| text channel | 400 | yes | 0.46 ms | 0.38 ms | 0.62 ms |
- Bytes. Every case returns the same bytes. All of them return the shared
errNotATalkvalue. A text channel and a missing id take the same path (one query). Before the fix, a non-member got 403 "you cannot view this Talk channel", which told them the id was a Talk. Now that answer is gone. - Timing. A call Talk costs one more query:
talks.IsDMTalkruns afterSELECT type. In this run that is about 0.1 ms on every percentile, so it is not within the noise. It is an oracle only for someone who already holds the call's UUID (a random v4 id) and Manage events. - SF-3. Fold the check into the first query (
SELECT type, dm_id IS NOT NULL FROM channels WHERE id = $1). Then all refusals take one path, and the code is simpler. - Lookup errors. A failed lookup returns the error (500), which fails closed.
3. Events that already name a call Talk
events.channel_id is declared ON DELETE CASCADE (migration 0070). Such events die with the call Talk, which goes when it is empty or with its conversation. So they exist only for calls that were running when the server was upgraded.
To test this I inserted one directly, since the API now refuses to create one:
PATCH(title only): 400. The merged payload is validated, so the event can no longer be edited or moved out of the call. That is acceptable.- A comment: 201, not audited.
POST /api/events/{id}/cancel: 200, and it writesevent.talk_cancelwith the call Talk's id and the title. The Owner and Administrators outside the conversation read it. A non-member auditor does not.- The event list does not show the event to outsiders.
SF-1. In auditRecord, skip the entry when ev.ChannelID is a conversation's call Talk, and fail closed on a lookup error. That covers cancel, cancel-following and any other path in one place. The alternative is to refuse cancel for such an event. The window is small, so this is a SHOULD-FIX, not a BLOCKER.
Older audit rows of this kind (event.talk_* naming a call Talk) stay readable by the Owner and Administrators, because the log is immutable. doc/upgrade.md could say so, as H-L6-03 did for its old rows.
4. Automod
parseRule refuses snap.KnowsChannel(id) == false || snap.IsDMTalk(id) as "unknown channel
I checked from an Administrator who is in the conversation:
| Exemption | Result |
|---|---|
| a text channel | 201 |
| a community voice Talk | 201 |
| the conversation's own id | 400, as before (a conversation is not in channelCat) |
| the call Talk | 400 |
So no legitimate channel is refused. Revert check: with automod_rules.go reverted, TestAnAutomodRuleCannotExemptACallOfAConversation fails ("exempt the call: got 201").
5. Tests, revert checks, gates
- Guard tests at 8c95e9466:
TestAnEventCannotBeLocatedInACallOfAConversationandTestAnAutomodRuleCannotExemptACallOfAConversationpass. Each fails when its own production file is reverted: handler_validate.go fails the event test, automod_rules.go fails the automod test. - Integration tag:
go test -tags=integration ./internal/communityevents/... ./internal/moderation/... ./internal/perm/is ok. - Gates by exit code, all 0:
- gofmt (clean);
- go vet with and without
-tags=integration(communityevents, moderation, server); make complexity-check;go test ./internal/repo/ ./internal/perm/ ./internal/communityevents/... ./internal/moderation/;node scripts/check-english-only.mjs;make openapi-check.
- Commits: test and fix types, no attribution line.
- SF-2. Both this branch and msch128/fix-h-l6-03 add
internal/server/visibility_audit_talk_integration_test.go. Whichever merges second hits an add/add conflict. Rename this file, for example tovisibility_event_call_integration_test.go, or merge the two files when integrating. - Docs. doc/events.md and doc/talks.md match the fix. talks.md still lists "renamed it" among things that never reach the log, which section 6 shows is false.
6. The two paths the author did not fix: rename and duplicate a call Talk through the admin channel routes
Probed at 8c95e9466. The actors were an Administrator who is in a group conversation and, separately, a member who holds only Manage channels server-wide, inside their own conversation.
| Action | Result | What the Owner and Administrators outside the conversation read |
|---|---|---|
PATCH /api/admin/channels/{call} {"name": ...} | 200 (Administrator); 200 (Manage channels only) | channel.update: actor, target "Talk 1", the call's id in params, and before/after/changes (name "Talk 1" to the new name). Owner and Administrators skip the class filter, so they read it. |
POST /api/admin/channels/{call}/duplicate | 201 (both actors) | channel.duplicate with the call's id. A permanent community Talk named after the call appears in everyone's channel list. |
PATCH /api/talks/{call} by a non-leader Administrator | 403 "only the Talk leader can change this Talk" | (nothing) |
The details hold the channel snapshot only: name, type, limits, category NULL. They hold no dm_id, no leader and no other member. What reaches the Owner is that the actor had a call Talk at time T, plus its id and name. The other members are not named.
Judgement: a finding, S4. It breaks two documented statements:
- doc/talks.md: "Nothing about it reaches the audit log (who joined it, renamed it or when it went)".
- doc/architecture.md: the administrative channel routes "answer 404 for a conversation id". The call Talk is meant to be "as private as the conversation".
S4 because it needs Manage channels or Administrator, and the actor mostly discloses their own participation. The duplicate also creates a community object from a private one.
H-L6-04 did not cause this: Manage channels is not a channel-scoped bit and was held in the call before. But since H-L6-04, a member without View channels who holds Manage channels can do it too.
Suggested fix: the admin channel routes (update, delete, duplicate, and any other route keyed by a channel id) should answer 404 for a conversation's call Talk, as they already do for the conversation itself, and as incoming and outgoing webhooks already do. I did not probe DELETE /api/admin/channels/{call} (channel.delete); it presumably behaves the same.
What I did not check
- The web client.
DELETE /api/admin/channels/{call}.- Event comments and roles audit for an old call-Talk event, apart from cancel: the comment create was not audited.
Round 2: 7a67c7601 (SF-1, SF-2, SF-3), c7dd7ac8a (H-L6-06 test), 9dbb9a667 (H-L6-06 fix)
Verdict: MERGE AFTER FIXES. H-L6-06 phase 2: STANDS at S4. No BLOCKER. One SHOULD-FIX (R2-1), NOTEs, and one new candidate finding (R2-C).
H-L6-06 phase 2
The known failure at c7dd7ac8a reproduces. I reran the eight routes without the middleware (router_routes.go reverted on 9dbb9a667):
| Route on a call Talk | Status |
|---|---|
| rename | 200 |
| auto-delete | 200 |
| GET overwrites | 200 |
| duplicate | 201 |
| DELETE | 204 |
| PUT layout | 204 |
| lobby mark | 400, different from a missing id, does nothing |
| PUT overwrites | 409, different from a missing id, does nothing |
The audit log carries the call's id, the new name and channel.duplicate. This breaks two documents:
- doc/talks.md: "renamed it ... never reaches the log".
- doc/architecture.md: the administrative channel routes answer 404 for a conversation id.
It needs Manage channels or Administrator, and the actor mostly reveals their own call, hence S4. It is not a duplicate: H-L6-05 is about event locations.
SF-1, SF-2, SF-3 (7a67c7601)
- SF-1: resolved.
auditable(r, ev)skips the entry when the event's channel is a call Talk or the lookup fails. It is used byauditRecord,auditSeries,auditRosterandauditComment. Revert check: with the communityevents changes reverted,TestCancellingAnEventOfACallOfAConversationWritesNoAuditEntryfails ("the cancellation of an event of a call reached the audit log"). Only cancel is tested; comment deletion and roster changes use the same function but have no test of their own (NOTE R2-a).- Comment deletion and roster changes on such an event are now not audited. That is consistent with doc/talks.md ("Nothing about it reaches the audit log"). The event exists only inside a running call that was created before the upgrade, and only the call's members can see it, so nobody outside the conversation has anything to review. doc/moderation.md lists
event.comment_deletewithout exceptions. NOTE R2-b: one clause there ("not for an event of a conversation's call") would make it literally true.
- Comment deletion and roster changes on such an event are now not audited. That is consistent with doc/talks.md ("Nothing about it reaches the audit log"). The event exists only inside a running call that was created before the upgrade, and only the call's members can see it, so nobody outside the conversation has anything to review. doc/moderation.md lists
- SF-2: resolved. The file is now
visibility_call_talk_objects_integration_test.go, so there is no add/add conflict with fix-h-l6-03. - SF-3: resolved. There is one query (
SELECT type, dm_id IS NOT NULL), so every refusal takes the same path. I did not re-time it. With one query and one code path for all refusals, nothing is left to measure, so I accept the author's medians.
The hideCallTalks middleware (9dbb9a667)
What it does. It sits on the /admin router, after RequireAnyPerm and before the audit middleware. For a RoutePath of /channels/{uuid}[/rest] that names a call Talk, it replaces the id in rctx.RoutePath with a fresh uuid before chi chooses the route. Handlers read chi.URLParam from that routing, so they see the fresh id.
Path spellings. Each spelling was sent for the call and for a missing id and compared by status and body. All 13 answer alike:
| Spelling | Answer |
|---|---|
| canonical | 404 "channel not found" |
| upper case | 404 "channel not found" |
| 32 hex digits without dashes | 404 "channel not found" |
urn:uuid: prefix | 404 "channel not found" |
| a percent-encoded dash | 400 "invalid id" |
| a percent-encoded digit | 400 "invalid id" |
| trailing text after the id | 400 "invalid id" |
;x after the id | 400 "invalid id" |
// before or after channels/ | 404 "route not found" |
| trailing slash | 404 "route not found" |
/../ | 404 "route not found" |
/./ | 404 "route not found" |
There is no bypass, for two reasons:
- The middleware parses the same string chi routes on. chi routes on
RawPathwhen one is set, so an encoded id is never decoded on either side. - The middleware and the handlers both use
uuid.Parsefrom google/uuid. Every form one accepts, the other accepts too.
What still holds the real id.
- The audit middleware matches
r.URL.Path, not the route context (internal/audit/middleware.go:59). Its route parameters andcaptureBeforetherefore use the real call id and read the call's real snapshot before the handler runs.- No entry is written today: entries are written only for 2xx answers, and every route answers a missing id with 4xx. The author's ghost comparison and my probe confirm that; the Owner's audit log does not carry the call id.
- A route that answers 2xx for a missing channel, for example an idempotent DELETE or a future GET-like POST, would write an entry with the real call id and the call's name from the before-snapshot.
- The request log records
LogPath(r.URL.Path), so the real id goes into the operator log. Operator logs are not a public surface and already hold call ids from ordinary member requests. - Metrics use
RoutePattern(), so their labels hold no id.
SHOULD-FIX R2-1. When the middleware swaps the id, also set r.URL.Path (and clear or rewrite r.URL.RawPath) on a shallow copy of the request. Then the audit middleware, captureBefore and the request log all see the fresh id too. It is two lines and removes the only route by which the real id could still reach the audit log.
Lookup cost and failure mode. Each admin request whose path starts with /channels/{uuid} costs one primary-key lookup, on reads too: about 0.1 ms. A failed lookup answers 500 before the handler and before the audit middleware, so it fails closed and records nothing.
Middleware or guard? A check inside the admin channel guard is not simpler. There is no single guard that answers 404 for a conversation. The conversation case is handled by type <> 'dm' in each store query, plus Guard.Require(ViewChannel) in four packages:
- chat/channels (rename, delete, duplicate, auto-delete);
- roles (overwrites);
- webhook (
authorize, which already excludesdm_id); - the layout check.
Guard.Require is shared with the member routes, where call Talks must stay visible, so it cannot simply learn about call Talks. A guard-based fix would need an admin variant in four packages and a promise that every future route uses it. The middleware also covers routes added later, and swapping the id gives byte-identical answers per route, where a fixed 404 would not (some routes answer 400 or 409 for a missing id). Keep the middleware, with R2-1.
Admin routes that take a channel id outside the path:
| Route | Behaviour for a call Talk |
|---|---|
| layout (body) | checkChannelMove refuses a call Talk as hidden. Revert check: without it, PUT layout answers 204 and the audit log carries the call id. |
onboarding start_channel_id | requires a text channel, so a call Talk gets the same answer as a missing channel |
| outgoing and incoming webhooks | already excluded through dm_id |
| events and automod | round 1 |
audit log target_id or channel_id filters | GET /api/admin/audit-log?target_id=<call> returns {"entries":[]} now. The Owner reads everything anyway, so the filter tells them nothing they could not read; old rows stay visible (NOTE R2-c). |
Calls deleted on older versions. DELETE on the base deleted a live call (204). The conversation stays, so nothing is left to clean in the data. Duplicates made from calls are ordinary community Talks that their makers created. The audit rows of earlier renames, duplicates and deletes (channel.update, channel.duplicate, channel.delete naming a call Talk) stay readable by the Owner and Administrators, because the log is immutable. NOTE R2-c: doc/upgrade.md should say so, as for H-L6-03. The author did not add an upgrade note.
Leader's own routes still work. The leader's PATCH /api/talks/{id} works (the author's test). The per-Talk disconnect lives under /api/channels, not /admin, so the middleware does not touch it.
Cosmetic. NOTE R2-d: the comment in checkChannelMove (internal/chat/channels/channel_permissions.go) is written twice.
Revert checks:
- Without the middleware: rename, auto-delete, overwrites, duplicate and DELETE act on the call, the audit log carries the call id and "renamed-okapi", and the leader's later rename fails (the call was deleted).
- Without the layout check: PUT layout answers 204 and the call id reaches the log.
- Without
auditable: the cancel test fails.
R2-C: new candidate finding (predates all of this): uploads in a call's chat appear in the admin media list
media.ListMedia (internal/media/admin.go:50-62) excludes only type = 'dm'. A call Talk is type = 'voice' with a dm_id, so its attachments are listed.
Probe: a conversation member uploaded okapicall.png into the call's chat (201). The Owner's GET /api/admin/media then listed the file with the call's id, the channel name and the uploader's name.
This breaks doc/architecture.md: conversations are "not shown in the admin media list". The call is "as private as the conversation" (doc/talks.md).
- Reader: the Owner or an Administrator who holds the media rights, not a conversation member.
- Content: a file name and an uploader inside a call.
- Suggested severity: S4.
- Fix: exclude
c.dm_id IS NOT NULLin the media list, and check the other admin media routes (orphans, stats, delete with its audit entry) the same way.
Tests and gates (round 2)
- Full suite, run once at 9dbb9a667: internal/server, communityevents, moderation and chat/channels with the integration tag, all ok (internal/server 262 s).
- Gates by exit code, all 0: gofmt (clean); go vet with and without the integration tag; complexity-check; the repo, perm, communityevents and chat/channels tests; the English-only check; openapi-check. Commit types are fix and test, with no attribution line.
Round 3: e4a6e3050 (R2-1 and the notes), 714130f51 and 7a1340e0c (H-L6-07 known failures), 6f4519d2b (H-L6-07 fix)
Verdict for the branch: MERGE. H-L6-07 phase 2: STANDS at S4. No BLOCKER and no SHOULD-FIX. NOTEs only.
R2-1 (e4a6e3050)
withFreshChannelID swaps the call's id in a copy of the request: URL.RawPath, URL.Path and the route path. The audit middleware and the handlers then see only the fresh id.
My probe as an Administrator in the conversation: GET /api/admin/channels/{call}/overwrites answered 404. With the earlier round-2 probes included, the Owner's audit log carries no call id.
NOTE R3-a: the request log still records the real id. httpx.Logger sits outside the admin router and logs its own, unchanged request. I found path=/api/admin/channels/<call id>/overwrites status=404 in the integration log. The middleware's comment ("the request log writes it") claims more than the code does. The operator log is not a public surface, and members' own requests already log call ids (/api/channels/<call>/messages). Fix the comment or accept it; it is not a defect.
NOTE R3-b: the swap uses strings.CutSuffix(EscapedPath, RoutePath). When the two differ in escaping, for example a rest part with a character Go escapes, the URL keeps the real id. Such paths match no route, so they answer 4xx and are never audited. Harmless today.
The notes from round 2 are closed:
- doc/upgrade.md now says older entries stay readable.
- doc/events.md is updated.
- Comment deletion and roster changes have tests.
- The duplicated comment in
checkChannelMoveis removed.
H-L6-07 phase 2
Reproduction. With the three fixed files reverted to 9dbb9a667:
TestAReportOfACallChatIsFiledLikeOneOfTheConversationfails: kind "channel", and the report names the call.TestTheAutoDeleteSweepLeavesACallTalkAlonefails: the sweep removed the call's message, and the run reached the audit log.- The canary sweep (seed 2, smoke) reports the Owner reading the call file's id, the call's id and the call's name in
/api/admin/media.
Defeat attempts. The behaviour is not documented and not intended. doc/architecture.md says conversations are "not shown in the admin media list", and doc/talks.md calls the call "as private as the conversation". It is not a duplicate: H-L6-04 to 06 covered other places.
Severity S4.
- The media list is Owner-only: an Administrator got 403 on
/api/admin/media,/orphansand/stats. - Reports reach holders of Manage reports.
- The auto-delete case needs data left over from before H-L6-06.
Verdict: STANDS (S4).
The fix (6f4519d2b), probed as Owner, Administrator and moderator outside the conversation
core.PrivateChannelSQL(alias) is (alias.type = 'dm' OR alias.dm_id IS NOT NULL). It is used in the media list (with COALESCE, so media of a deleted channel stay listed), the report kind and name, SetAutoDelete, the sweep and selectExpired.
Media. The Owner's /api/admin/media lists the community file but not the file uploaded in the call: no call id, no file name. /orphans and /stats show neither. An Administrator gets 403 on all three.
Report of a call-chat message.
- The reporter (a conversation member) files it: 201.
- A moderator outside the conversation with Manage reports lists it as kind "dm", with no call id and no "Talk 1". It reads the detail (200) without the call id, and resolves it (200).
- The Owner's audit log carries no call id.
- The reporter's own flow is unchanged:
Guard.Require(ViewChannel)passes for a member. - Who can read and resolve: exactly the holders of Manage reports, as for every other dm report. They read the copied snapshot and no channel. This matches SECURITY.md: "a moderator sees only what a member chose to report".
Auto-delete. The author's test covers a call Talk that kept auto_delete_days: the sweep leaves it alone and writes no audit entry. The revert check shows the opposite.
The 35-row table
The 11 "different" rows plus row 28, judged against the documents:
| Row | What | My judgement |
|---|---|---|
| 11 | mark-all-read | Agree. The caller's own markers, limited to channels the caller can see. |
| 12 | message numbers | Agree. A number names no call. Migration 0098 keeps call messages out of the public counts. |
| 16 | Talk invite card | Agree. It goes into the caller's own conversation, and joining runs the normal checks. |
| 18 | push | Agree on privacy (probed below). NOTE R3-c: functional difference. |
| 19 | bot access view | Agree. ChannelPermissions(bot) is computed for the bot, which is in no conversation, so a call never appears. Each entry is also filtered by the caller's own view. |
| 20 | scheduled messages and their export | Agree. The member's own data. |
| 28 | DeleteMedia by id | Correction: the storage dashboard is Owner-only (auth.RequireOwner, internal/media/handler.go:122), not "an administrator". The Owner would need a file id, and no id is listed to them any more. Listing it is right. |
| 29 | totals | Agree. Aggregates name no call. |
| 31 | event reminders, poll notices | Agree. Recipients are viewers, and call events can no longer be made. |
| 35 | backups and import | Agree. Backups hold everything by design. |
On row 18 I probed push recipients with the fake push service. A message in a call's chat by alice, mentioning @bobby @carol @cool_guy @everyone, pushed to bobby only: the call's other member. The Owner and carol, who are outside, got nothing. I did not read push titles: the payload is encrypted in the fake.
NOTE R3-c (functional, not privacy): from the code (internal/push/recipients.go:74 and :100), a call's chat is pushed as a channel, not as a conversation.
- The title is the call's name.
- The member's "Mute all direct messages" (
dm_mute_all) does not silence it. - The server-wide notify level applies.
doc/talks.md says the chat follows the conversation's writing rule. It says nothing about notifications. Worth one sentence in the docs, or treating it as the conversation (PrivateChannelSQL there).
Ten more rows at random (6, 7, 9, 10, 14, 22, 25, 27, 30, 33), checked:
| Row | What | Result |
|---|---|---|
| 6 | webhook | Excludes dm_id; probed in H-L6-06. |
| 7 | message audit | channelAuditName uses type = 'dm' OR dm_id IS NOT NULL. |
| 9 | account erase | Uses its own call-media statement. |
| 10 | readstate | Excludes dm rows. The call is covered by the snapshot and the talk pair. |
| 14 | channel list | Snapshot-filtered. The Owner's GET /api/channels lacks the call (probed in H-L6-04 and here). |
| 22 | automod | Uses snap.IsPrivate. |
| 25 | search | No hit for outsiders (probed in H-L6-04). |
| 27 | media serve | Goes through the snapshot. |
| 30 | totals | Migration 0098; not re-run. |
| 33 | WebSocket | No fan-out to outsiders (probed in H-L6-04). |
I agree with every row.
My own grep. I searched for type <> / != / = 'dm', IS DISTINCT FROM 'dm', ChannelKindDM, .Type == "dm", in_dm, IsDM(, KnowsDM(, type = 'voice', IN ( and slug lookups. It found three places the table does not name:
- chat/messages/forward.go:268 (
loadQuote). The quote of a call-chat message carries the call's name, where a conversation's carries ''. I probed it: alice forwarded a call message into an open channel (201). carol, outside the conversation, read the quote as{"available":false}: no name, no id, no text. The quote is checked again for every reader, so the name never reaches an outsider. Correct as it is. - chat/account/account_export_messages.go and later/visibility.go. The member's own data. Correct.
- ws/handle.go:249. Uses
KnowsChannel || KnowsDM. The snapshot decides. Correct.
Nothing new leaks.
Deep sweep, revert checks, suites, gates
- Revert checks: done above, under the reproduction.
- Deep canary sweep,
VIS_SWEEP=1 VIS_SEED=2, run once below load 10: passed. It sent 48,607 requests, including 13,708 pairs comparing a hidden object against a missing one, and found no leak. It took 25.7 min. The control left 2 Staff words unreached ("staff edited before", "staff post first title"); that line is the harness's own info output and did not fail the run. - Full internal/server integration suite, run once at 6f4519d2b, with media, chat/reports and chat/channels: ok (internal/server 259 s).
- go vet over the whole module (
./...), with and without-tags=integration: exit 0 both. - Gates by exit code, all 0: gofmt (clean); complexity-check; the repo, perm, media and chat/... tests; the English-only check; openapi-check. Commit types are fix and test, with no attribution line.