Independent review of pull request 817 (nightly sweeps)

From mnema-talk v3.3.13, rendered on this site.

Reviewer: an agent that did not write the change. Read-only: nothing in the change was edited. The probes ran as a scratch test file inside internal/automod that was deleted afterwards; the check without the fix put the origin/main version of internal/automod/linkhost.go in place for one test run and then restored it with git checkout HEAD --. The worktree was clean afterwards. No real instance was contacted.

Reviewed: one commit, head dc6f83774 ("link rules apply to hosts with a bare ACE label as well"), on merge base 502b1056b. origin/main is one commit further (0945ee104, #816); that commit touches none of the files below.

  • internal/automod/linkhost.go (read in full)
  • internal/automod/linkscan_test.go (read in full)
  • internal/automod/testdata/link_parity.json (the added vectors)
  • web/src/shared/lib/markdown/linkParity.cases.fixture.ts (read in full)
  • .github/workflows/visibility-sweep.yml (read in full)

Read around it, in full: internal/automod/linkscan.go, linkends.go, linkParity.fixture.ts, linkParity.test.ts, linkParity.sweep.test.ts, linkParity.current.test.ts. In part: compile.go (forbidsLinks, coveredBy), rule.go (cleanDomain, hostOf), linkcover.go (the work budget), the two integration tests the workflow runs, and golang.org/x/net@v0.62.0/idna (idna.go process, punycode.go decode).

Verdict: MERGEABLE

No BLOCKER, no SHOULD-FIX. Two informational notes below.

Root cause

golang.org/x/net/idna (v0.62.0, idna.go:393-411) strips the ACE prefix and Punycode-decodes the rest of the label. decode("") returns "", nil (punycode.go:35-37), and the "decoded label is ASCII" error is raised only for len(u) > 0, so a label that is exactly xn-- (after mapping) is replaced by an empty label with no error. With VerifyDNSLength(false), as the URL Standard requires, nothing reports the empty label either. readDomain then read the host without that label and marked it exact. The URL parser in Node 26 keeps an ASCII xn-- label as written. So the server judged a host that differs from the one the client opens.

The fix is in the right place. readDomain is the single host reader both for messages (linkends.go:41) and for configured domains (rule.go:295). keepEmptyACE puts the bare ACE label back and marks the reading as not exact. With that, link rules apply to hosts with a bare ACE label as well: an allow list refuses them, as it refuses every reading it cannot take as exact, and a block list judges the host the client opens. A configured domain with such a label is no longer accepted, because cleanDomain requires an exact reading.

What I checked

  1. Detection condition (linkhost.go:114). An emptied label always shows in Go's output as an empty host, a leading or trailing dot, or ... Hosts without these return early, so ordinary hosts take no extra work. A trailing dot or .. without an ACE label goes through the per-rune mapping but comes back unchanged and exact (allowed.example., a..allowed.example, sub.allowed.example.).
  2. Per-rune mapping (linkhost.go:137-148). The comment says composition never yields an ASCII x, n or -. That holds: NFC never composes into ASCII. Probed inputs: ASCII upper and mixed case (XN--, Xn--, xN--), percent-encoded (%78n--), full-width letters and hyphens, full-width, ideographic and half-width ideographic full stops as separators, and ignorable code points mapped to nothing (U+00AD in three positions, U+034F, U+FE00, U+200B). Each was read with the bare label kept and as not exact. U+200D (kept by non-transitional processing), U+2024, U+FE52 and U+2488 turn the label into another label. Go then encodes it normally and reports an error, so the reading is not exact either.
  3. Several labels, positions, trailing dots: xn--.xn--, xn--.xn--.allowed.example, .xn--, ..xn--, .xn--.allowed.example, xn--..allowed.example, xn--., xn--.., allowed.example.xn--., allowed.example.xn--.., mixed with non-ASCII labels (café.xn--, xn--.é.allowed.example, xn--caf-dma.xn--). All of them kept every label and were not exact. For the ASCII forms the host matched what Node 26's new URL() reports, apart from the trailing dot, which the code trims for every host.
  4. Hosts that come out empty: a lone xn-- (any case, with user info and a port) now reads as host xn--, neither exact nor rejected. Node 26 opens it. Before the fix it was rejected and could be left out when another reading existed.
  5. Label-count mismatch (linkhost.go:122-124). When the per-rune mapping and Go's output disagree on the number of labels, Go's host is returned and marked as not exact. This is the conservative direction. None of the inputs above reached this branch.
  6. IPv4 interplay: xn--.1 and xn--.0x7f are rejected (they end in a number and are no address), and Node 26 rejects them too. 1.2.3.xn-- is a domain and not exact.
  7. No false alarms on allowed links. With an allow list of allowed.example, these still pass: https://allowed.example/x, https://sub.allowed.example./x, see https://allowed.example., **https://allowed.example**, https://xn--caf-dma.allowed.example/x, https://café.allowed.example/x and [a](https://allowed.example/x). The PASSING set of the parity fixture stays green as well.
  8. Block lists. Before and after the fix, a host whose ACE label decodes to ASCII (for example xn--a-) reads as the decoded label and not as exact. Configured domains must be exact, so they contain no such label, and the labels a listed domain covers are left unchanged by Go's processing. A host the client opens under a listed domain therefore stays covered. Nothing to fix here.
  9. Cost. A 3.9 kB message full of tokens with emptied labels takes about 0.20 ms against 0.13 ms for the same message without them (Apple M5, go test -bench). The work budget is unchanged and charges the reading as before.
  10. The new tests fail without the fix. With the origin/main version of linkhost.go, TestAnEmptyACELabelStaysInTheHost fails on all 9 host cases and all 4 messages. TestLinkParityVectors fails on all 5 new vectors. The exact-host half of the new test passes on both versions, as it should. With the fix, everything passes.
  11. Fixture and vectors agree. The five messages added to AGREED are in link_parity.json in the order linkParity.test.ts expects (AGREED, then PASSING, then the sample). The server readings recorded there match what the fixed code produces.
  12. Workflow (visibility-sweep.yml:43-89). The matrix is now parity × sweep. Each include entry matches one existing sweep value and only adds test, so this gives four jobs, each running exactly one test. I confirmed with go test -tags=integration -list that both names exist. The -run pattern is anchored, and the test name comes in through env, not interpolated into the script. VIS_SWEEP=1, the seed formula GITHUB_RUN_NUMBER * 2 + PARITY, the sizes (deep canary sweep, 400 communities) and both tests' assertions are unchanged. Each test gets the same seed as before, so no seed is dropped. The go test timeout goes from 40 to 50 minutes for one test instead of two, inside the unchanged 60-minute job timeout. fail-fast: false still holds. Nothing in the gate is weakened. The four jobs each start their own PostgreSQL service. The two tests do not depend on each other's data.

What I ran

CommandResult
go test -count=1 ./internal/automodok
same, with linkhost.go from origin/mainFAIL: TestLinkParityVectors (5 vectors), TestAnEmptyACELabelStaysInTheHost (13 cases); restored afterwards
scratch zz_review_test.go (46 authorities, 11 messages, two benchmarks)results as in checks 1 to 9; file deleted
Node 26.13.1 new URL() on 29 of the same authoritiesASCII forms kept as written; non-ASCII mapped forms fail to parse
PATH=…/v26.13.1/…:$PATH npx vitest run src/shared/lib/markdown/ (in web/)13 files passed, 1 skipped (the deep sweep, needs LINK_SWEEP); 433 tests passed, 4 skipped
actionlint .github/workflows/visibility-sweep.ymlno findings
go vet -tags=integration ./internal/serverok
go test -tags=integration -list '^(TestNoCanary…|TestRandomCommunities…)$' ./internal/serverboth names listed

What I did not check

  • The deep link sweep itself (LINK_SWEEP=1, 100,000 messages per shard) and the nightly visibility jobs were not run. Their run time on the CI runners under the new matrix is taken from the workflow comment, not measured.
  • Browsers other than Node 26's URL parser (Chromium, WebKit, Firefox) were not compared on these hosts. Where they reject a non-ASCII mapped form, the server's reading as not exact is stricter, not looser.
  • No full UTS #46 conformance run of the per-rune mapping. The probes cover the classes listed above.

Findings

I1. The generator has no host with a bare ACE label (informational)

web/src/shared/lib/markdown/linkParity.fixture.ts:87 (HOSTS). The nightly run found this class by chance, by splicing characters into xn--caf-dma.example. Adding a host such as xn--.allowed.example (and one with the label last) to HOSTS would make the sweep cover the class on purpose. Changing HOSTS changes the 500-message sample, so the vector file would have to be written again. This is optional and can go in a later change.

I2. Non-ASCII mapped forms read as not exact where Node 26 rejects them (informational)

internal/automod/linkhost.go:92-94. A full-width or ignorable-character spelling of the bare ACE label (xn--.allowed.example, x­n--.allowed.example) fails to parse in Node 26, so the client links nothing from it. The server now reads it as not exact, and an allow list refuses the message. This is stricter than the client, which is the intended direction ("never as exact"). Ordinary allowed links are not affected (check 7). No change needed.

Something wrong on this page? Tell us.