Fix review H-L6-01 (forum audit entries of a private forum reach every auditor)
From mnema-talk v3.3.13, rendered on this site.
Reviewer: fix reviewer, independent of the finding and the fix. Read-only: nothing in the repository changed, nothing pushed.
Fix: branch msch128/fix-h-l6-01, commit 6e4164178 (fix: class the forum audit entries by channel and hide actions without a class), on the lane 6 test commit 6f0cd798e, base origin/main f58bc3fc6 (3.0.9). Only 6e4164178 was reviewed.
Inputs: findings/H-L6-01.md, reviews/repro-H-L6-01.md, reviews/refute-H-L6-01.md.
Verdict: MERGE AFTER FIXES
The fix removes the cause for every variant the finding and the refuter list (forum.post_update, forum.tag_create/update/delete, forum.settings_update, forum.tag_reorder) and also the second cause the author names (a catch-all that failed open). The class change is exactly the intended one: of the 139 actions the server can write today, only the six forum.* actions changed class (Auditors to Channel), nothing widened, and the SQL and Go agree on every action for every viewer kind on a real database. The blocking item is in the new structural test, not in the production change: it claims to find every writer, but it is blind to an aliased import of the audit package, a type alias of audit.Entry, and any raw INSERT INTO audit_log ... VALUES ($1 anywhere in the tree, which together defeat the very guarantee doc/moderation.md now promises ("A test reads the server's source for every action an entry can be written with"). That is a SHOULD-FIX (the production behaviour already fails closed through the new catch-all, so a missed writer is hidden, not leaked); I list it as the condition for merge because the documentation states the test as a guarantee.
No BLOCKER.
1. Class before and after, action by action
Method: a scratch test (in a temporary worktree, deleted) ran the fix's own source scan (newActionScan, 139 actions, 0 scan errors) and evaluated every action with three functions: the old ClassOf copied from origin/main, the old SQL semantics (privileged patterns hide first, then channel patterns, with SQL LIKE where _ is a one-character wildcard), and the new ClassOf. A dynamic action (admin.*, event.talk_*) was probed with a concrete suffix.
Result: old ClassOf, old SQL and new ClassOf agree for 133 actions. Six changed, all Auditors to Channel:
| action | before | after | judgement |
|---|---|---|---|
| forum.post_update | Auditors | Channel | right: carries title, previous_title, author name and the forum id (the finding) |
| forum.tag_create | Auditors | Channel | right: carries the tag name |
| forum.tag_update | Auditors | Channel | right: carries the tag names before and after |
| forum.tag_delete | Auditors | Channel | right: carries the tag name |
| forum.settings_update | Auditors | Channel | right: ids and rules of a possibly private forum; consistent with channel.update |
| forum.tag_reorder | Auditors | Channel | right: names the forum; no text, but the forum id alone is what channel.* already hides |
Every forum.* writer sets Entry.ChannelID to the forum (internal/chat/forum/handler_forum.go:300, handler_forum_extras.go:121 and :178, handler_forum_tags.go:208-209 for create/update/delete), so an auditor who can view the forum keeps reading them; the fix's integration test shows that for a Staff auditor and the Owner.
Unknown actions narrow from Auditors to Owner and Administrators only. I checked that this hides nothing rightly wide that exists:
- All 44 literal Auditors rows are actions the scan finds (no dead row), and every action the scan finds has a row of its own, so the catch-all is reached by no written action today.
- Stored rows of actions that no longer exist: I collected every action-like string literal ever added or removed in non-test Go files on the history of origin/main (
git log -p), plus every dynamic"prefix." + xconstruction. Every historical action literal is still written today (the only one outside the current literal set, system.mail_test, is still found by the scan). The historical dynamic forms areadmin.<method>(Privileged before and after) andevent.talk_<kind>(Channel before and after). So no stored row of a real deployment changes class except the forum ones.
2. Nothing hidden is now shown
No action widened (table above). Structurally: the Auditors rows are literals placed after every prefix row, so any action that an earlier Privileged or Channel row matched keeps that class; and none of the 44 literals was hidden by the old SQL's _ wildcard (checked with the LIKE emulation). The cross-product run below confirms it on a database: for the old code and the new code the only per-row differences are the forum rows and the synthetic _ probes.
3. SQL against Go, every action and viewer kind, on a real database
Scratch integration test (deleted after the run) on the fix: 157 actions (the 139 written ones, plus stored-only and wildcard probes: something.new, user.update, wiki.something_new, forum, forumx.tag, forum., moderation.noteXadd, event.talkXedit, user.passwordXset, user.two_factorXreset, moderation.bulkXdelete, MESSAGE.delete, message, %, _, webhook, channel., voice.) times 6 channel situations (NULL; an open channel; a channel private to Staff; a forum that was open when the rows were written and made private to Staff afterwards; a channel deleted after the rows were written; a random id that never existed) = 942 stored rows, read by: the Owner, a member with Administrator, a View-audit-log holder with Staff, a View-audit-log holder without Staff (who lost the forum after the rows were written), and a plain member. Expected = new ClassOf plus the viewer's channels after the changes.
ZZ viewer=owner shown=942 of 942 mismatches=0
ZZ viewer=admin shown=942 of 942 mismatches=0
ZZ viewer=modSee shown=348 of 942 mismatches=0
ZZ viewer=modNo shown=292 of 942 mismatches=0
plain member: 403
Paged through before at limit 200 (several pages per viewer), and the console's group filter (action=forum|user|message|wiki|moderation|event|channel) returned exactly the expected count for each viewer.
The same test against the old classes.go and filter.go (restored from 6f0cd798e for one run) shows the old SQL disagreeing with the old Go function in 52 row and viewer pairs, all synthetic _ probes (for example moderation.noteXadd and user.passwordXset hidden by the old SQL though old ClassOf says Auditors; event.talkXedit and moderation.bulkXdelete treated as Channel by the old SQL). No real action, current or historical, matches a pattern only through _ as a wildcard (section 1: old SQL and old ClassOf agree on all of them), so no stored row changes class because the new SQL is exact.
4. The structural test (internal/audit/actions_scan_test.go)
What it catches (each tried with a planted writer in a scratch package, deleted afterwards): a new audit.Entry{Action: "x.y"} with no row (reported); Action: fmt.Sprintf(...) (fails closed: "cannot tell which action"); Action: m["k"] (fails closed); var action = "x.y" inside the function (fails closed; DeclStmt is not followed, the error is the safe outcome); an exported helper in another package with the action as a parameter (fails closed, see NOTE 4).
What it misses (planted, the test stayed green):
- SHOULD-FIX 1, import alias.
import aud ".../internal/audit"withaud.Entry{Action: "secret.alias"}:isEntryTypecompares the selector's package identifier with the literal stringaudit(actions_scan_test.go:132), so an aliased file is invisible in full, including the writers that would otherwise fail closed (Sprintf, map lookup). Fix: resolve the local name of the audit import per file fromf.file.Imports(the alias oraudit) and compare with that; also treat a dot import. - SHOULD-FIX 2, raw INSERT with a parameter action.
q.Exec(ctx, "INSERT INTO audit_log (action, ...) VALUES ($1, ...)", "secret.rawparam")anywhere: rawInsert exempts every string containingVALUES ($1(actions_scan_test.go:173), meant for the writer's own INSERT at internal/audit/audit.go:96 but applied tree-wide. Fix: exempt only that file (or that one literal), report every other parameterised INSERT as "cannot tell"; also match case-insensitively (insert into audit_log ... values ('secret.lower'is missed: thestrings.Containsat :168 and the regexp at :163 are case-sensitive). - NOTE 1, type alias.
type E = audit.Entry; E{Action: "secret.typealias"}is missed. Rare; resolving aliases needs go/types, or at least flag= audit.Entryalias declarations as an error. - NOTE 2, entries not built from a literal:
json.Unmarshal(b, &e)into anaudit.Entry, thenRecord(e), ande.Action = "x"in a file of the same package that does not import audit (the receiver's type is declared elsewhere) are missed; the scan looks at literals and imports-gated assignments only, never at the calls ofRecord/RecordTx/RecordOrLog. A sturdier closure: for every call of those three methods, require that the argument be followed back to a literal the scan has classified, and fail otherwise. - NOTE 3, a call through a function value (
r := rec; r(ctx, l, "secret.indirect")) is missed when the function also has one direct call:argumentscounts calls by name only (actions_scan_test.go:316). With no direct call at all, it fails closed. - NOTE 4, the cross-package helper fails closed for the wrong reason: a planted exported helper
Write(ctx, l, action)produced "cannot tell" becauseargumentsmatches every call namedWritein the tree (io writers with three arguments) and one of them is not a literal. Safe, but a reader will be told to "build it from string literals" when they did. Matching the package qualifier of the call would make it precise. - Web-facing code: the web client has no database access; the only writers are in internal/ and cmd/, which the scan covers. Writers in *_test.go files are excluded on purpose.
Does it fail closed when it cannot follow a value: yes, for every value it reaches. It fails open for writers it does not recognise as writers (SHOULD-FIX 1 and 2, NOTEs 1 to 3). Because the production catch-all is now the narrowest class, a missed writer is hidden from auditors, not leaked, so these gaps are about the promise in the documentation (doc/moderation.md, "Access": "A test reads the server's source for every action an entry can be written with and fails for one without a row of its own"), not about a leak today. No aliased import, type alias, or second raw INSERT exists in the tree today (grep).
5. user.talk_kick (the author's open question)
Decision: it should change; it is not part of this fix's scope and need not block it.
The per-Talk disconnect (POST /api/channels/{id}/voice/{userID}/disconnect, internal/auth/memberadmin/users_admin_talk.go:105) writes user.talk_kick with the target member's name, the moderator as actor, the time, and details.channel_id = the Talk, without Entry.ChannelID. Under the Auditors class every holder of View audit log reads that member X was in Talk C at time T and was removed by moderator M, for a private Talk they cannot see. That is exactly the fact the voice.* rows are hidden for ("who was in which Talk", Channel class). The Talk is named by id only, and the id of a private channel is not otherwise readable to such an auditor (channel.* is Channel class), so the leak is presence of a named member in an unnamed private Talk: S4 territory, but inconsistent with voice.join/leave.
What should change: record the per-Talk disconnect under a Channel-class action with ChannelID set to the Talk (for example voice.disconnect, which voice.% already classes; add it to auditKinds in internal/moderation/history.go as talk_removal, whose output carries no channel), and keep user.talk_kick for the admin-wide kick (POST /api/admin/users/{id}/kick, internal/audit/routes.go:67), which names no Talk and is rightly Auditors class. Rows already stored keep their action (audit_log is immutable), so old per-Talk rows stay wide; say so in the changelog. Not verified at run time (it needs a live Talk); this is from the code.
6. Performance of the listing (200,000 generated rows, EXPLAIN ANALYZE)
The SQL of the old and the new filter was rendered with its arguments inlined (as a custom plan sees them) and run in the integration database on 200,000 rows after ANALYZE, best of five, viewer without privileges seeing one channel:
| scenario | query | old | new |
|---|---|---|---|
| actions uniform over the 139 | first page (51) | 0.24 ms | 0.50 ms |
| page of 201 | 0.48 ms | 0.92 ms | |
action=forum 201 | 0.88 ms | 4.7 ms | |
| 90 % auth.login | 51 | 1.4 ms | 1.4 ms |
| 201 | 3.1 ms | 4.4 ms | |
action=forum 201 | 15.6 ms | 19.2 ms | |
| nothing visible to the reader (full scan) | 51 | 66 ms | 169 ms |
| 201 | 66 ms | 184 ms | |
action=forum | 15-21 ms | 14-20 ms |
Index use is unchanged: the unfiltered listing walks audit_log_pkey backwards with the CASE as a filter and stops at the limit (old and new); the group filter uses the same parallel scan in both. The CASE costs about 2.5 to 3 times the old filter per row (it evaluates up to 90 WHEN branches, Auditors literals last), which shows only when the reader sees few rows and the scan has to go deep. NOTE 5: acceptable for an admin-only, paged route (worst case 0.18 s at 200k rows, linear beyond); if it ever matters, put the Auditors literals in one action = ANY($n) branch placed where they stand in the table, which keeps the first-match order and cuts the branch count by about half.
7. Console filters and paging (API level)
The console (web/src/admin/components/AdminAuditTab.vue:59) calls only GET /api/admin/audit-log with group filters and the before cursor; there is no count endpoint. Covered in section 3 for all four reading viewer kinds: each group filter returned the expected number of rows, paging was exact across pages, and the plain member got 403.
Other checks
- Recorded test:
TestKnownFailureForumAuditEntriesOfAPrivateForumReachEveryAuditoris now the guardTestForumAuditEntriesOfAPrivateForumReachOnlyAuditorsWhoSeeIt(internal/server/visibility_audit_forum_integration_test.go), a normal test; the canaryKnownLeaks rows are removed. - Revert check: the new tests fail when the fix is reverted (see "What I ran").
- Defects introduced: none found. The NULL-channel case is unchanged (the old SQL also excluded it through
FALSE OR NULL; the new COALESCE makes it explicit). The CASE has one argument per row (about 90 parameters per query), below any limit. No race (read-time classification, no state). No error path changed. - Public API (doc/api-stability.md): no shape change; the visible set of entries changes as documented.
- Gates: no allowlist entry, no limit raised, no test skipped (the old
ClassOf("something.new") != VisAuditorsassertion was replaced by the opposite, which is the intended policy change, and the old pin list was extended, not shortened). gofmt clean; go vet clean with and without-tags=integration;make complexity-checkexit 0; internal/repo tests pass (file and function sizes; actions_scan_test.go is 470 lines, under the 800 limit for tests);node scripts/check-english-only.mjspasses. Commit typefix, no!, no attribution line. - Documentation: doc/moderation.md and doc/forums.md match the new behaviour, except for the overclaim about the scan (section 4). NOTE 6: the OpenAPI description of
listAuditLog(internal/audit/handler.go,@Description) still says "message and Talk entries only for channels the caller can view" and does not mention forum entries or that actions without a row are shown to the Owner and Administrators only; update it with the next OpenAPI regeneration.
Findings
- SHOULD-FIX 1: internal/audit/actions_scan_test.go:132, the scan ignores files that import the audit package under another name; planted
aud.Entry{Action: "secret.alias"}passed. - SHOULD-FIX 2: internal/audit/actions_scan_test.go:163-175, every
INSERT INTO audit_log ... VALUES ($1in the tree is exempt, and the match is case-sensitive; planted parameterised and lower-case INSERTs passed. - SHOULD-FIX 3 (separate change, not this merge): user.talk_kick from a Talk reveals who was in a private Talk; move the per-Talk disconnect to a Channel-class action with ChannelID (section 5).
- NOTE 1-4: type alias, non-literal entries and same-package assignment without import, calls through function values, imprecise cross-package parameter matching (section 4).
- NOTE 5: the CASE costs 2.5-3x on deep scans (section 6).
- NOTE 6: OpenAPI description of the listing not updated (internal/audit/handler.go).
What I ran
All in a temporary detached worktree of 6e4164178 (removed afterwards); scratch tests and planted packages were deleted before the full suites ran.
- Class comparison of the 139 scanned actions (old ClassOf, old SQL semantics, new ClassOf): unit test, as above.
- History sweep of action literals:
git log -pover non-test Go files on f58bc3fc6. - Cross product on a real database, on the fix and once on the old classes.go/filter.go: section 3.
- EXPLAIN ANALYZE and timing on 200,000 rows, three distributions: section 6.
- Planted writers against
TestEveryWrittenActionHasAClassOfItsOwn: section 4. - Gates: gofmt, go vet (both tag sets),
make complexity-check,go test ./internal/repo/,node scripts/check-english-only.mjs. - Full suites:
DOCKER_HOST=... nice -n 10 go test -p 1 -tags=integration -count=1 ./internal/audit/ok (0.7 s; the package has no integration-tagged tests, so this is its unit suite);./internal/server/ok (422 s). - Revert check: with classes.go and filter.go restored from 6f0cd798e,
TestClassesPinTheVisibilityOfEveryActionfails (the six forum actions and the unknownforum), andTestForumAuditEntriesOfAPrivateForumReachOnlyAuditorsWhoSeeIt,TestAuditListingAppliesTheClassOfEveryStoredActionandTestNoCanaryReachesAnObserverOutsideItsClassfail. With only theforum.%row removed,TestEveryWrittenActionHasAClassOfItsOwnreports all six forum actions as having no row. Fix restored afterwards.
What I did not check
- The admin console in a browser (API level only, as asked).
- The user.talk_kick leak at run time (needs a live Talk); section 5 is from the code.
- The seeded Moderator role itself (the refuter replayed it; I used a role with View audit log only, which the class logic treats the same).
- Other packages' integration suites (only internal/audit and internal/server).