Development
From mnema-talk v3.3.13, rendered on this site.
How to build, test and release Mnema Talk. Read AGENTS.md first: the repository is public and the rules there (no secrets, conventional commits, migrations, dependencies) apply to everyone.
Setup
Requirements: Go (see go.mod), Node 26 + npm, Python 3 (make test-scripts
and make check), Docker, GNU make (on Windows use Git Bash or WSL). Run make help for all targets.
make install-hooks # pre-commit: gitleaks + gofmt
make dev # postgres + seaweedfs in Docker (docker-compose.dev.yml publishes
# them on 127.0.0.1), server on :8080 via go run (reads .env)
cd web && npm run dev # Vite dev server on http://localhost:3000, proxies /api to :8080
For the Vite dev server, allow its origin in .env:
CORS_ALLOWED_ORIGINS=http://localhost:8080,http://localhost:3000
DATABASE_URL=postgres://mnema:<password>@localhost:5432/mnema_talk?sslmode=disable
make build produces bin/mnema-talk with the web app embedded (make web
builds only web/dist).
main is protected: changes land through a pull request (no approvals
required) once the secret-scan, backend, frontend, govulncheck,
docker, upgrade-drill (<version>), e2e and CodeQL analyze checks are
green; squash merge only. Work for the next minor goes to its release/X.Y.0
branch the same way.
Frontend source uses strict TypeScript, including Vue scripts, media workers, browser tests and build tools. See the TypeScript contract for compiler scopes, runtime decoding and the generated-vendor exception.
Testing
make test # Go unit tests (-race)
make test-integration # Go integration tests: starts postgres:18-alpine via Docker,
# or uses TEST_DATABASE_URL if set
make test-web # vitest
make coverage # Go (unit + integration) and web coverage: coverage.out,
# web/coverage/lcov.info; CI shows the totals in the job summary
# gates for Go and web: at least 90 % in total and 80 % in every file
# (Go: COVERAGE_MIN / COVERAGE_FILE_MIN in the Makefile;
# web: web/scripts/check-coverage.ts, all four metrics)
make lint # gofmt, go vet, staticcheck, cognitive complexity, eslint
make bench # hub fan-out, permission snapshot load and SFU forward benchmarks against
# scripts/bench-baseline.txt (Docker or TEST_DATABASE_URL); prints the comparison,
# only more allocations per packet on the SFU forward path fail
# (make bench-allocs is that part alone, CI runs it; make bench-baseline
# records a new baseline on an idle machine)
make docs-check # relative links and anchors in the docs, the paths AGENTS.md names, no merge-conflict markers
make english-check # no German outside the locale resources and scripts/english-only-allowlist.txt
make typecheck-web # complete frontend source and strict compiler checks
make contracts-check # generated frontend REST contract drift check
make check # everything CI runs: docs and English checks, lint, tests, govulncheck, builds,
# npm audit, docker build
make e2e # Playwright (Chromium) against the real binary, see below
make load-test # voice/screen-share load test, see loadtest/README.md
make load-test-host # static chat/voice load-test tools for the server host (linux amd64 + arm64)
make test-loadtest # format, vet and tests of the load-test tools (own module)
make upgrade-drill # 0.8.16, 0.9.0, 1.0.0, 1.1.0, 2.0.0, 2.9.12, 2.9.19, 3.0.13, 3.2.1 and 3.3.5 images -> this build with seeded data (FROM_VERSIONS=1.0.0 for one)
make backup-drill # backup -> change -> restore on a throwaway stack (FROM_VERSION=1.0.0: back up on that release, restore into this build, check the downgrade refusal)
make postgres-upgrade-drill # the one-time PostgreSQL 17 -> 18 move
Browser tests (e2e)
make e2e (= e2e/run.sh) builds the web app and the binary, starts a
throwaway PostgreSQL and SeaweedFS with Docker on loopback ports, runs the
Playwright specs in e2e/tests/ with fake camera/microphone devices and tears
everything down again. SKIP_BUILD=1 reuses the last build,
E2E_INSTALL_BROWSER=1 installs the browsers named in E2E_BROWSERS first
(default chromium; also webkit and firefox) and E2E_ONLY is handed to
playwright test, so E2E_ONLY="smoke.spec.ts" runs one file and
E2E_ONLY="--project=webkit" another engine. Without it only the chromium
project runs.
The suite has five Playwright projects (chromium, webkit, firefox and the two phone projects below). Chromium is the required check and
runs everything. WebKit (what Safari, iOS and installed iOS web apps use)
runs every spec that does not join a call in CI as e2e (webkit 1/2) and e2e (webkit 2/2) (Playwright's
Linux WebKit has no RTCRtpScriptTransform, so a sealed call cannot happen
there, see Browsers; the call specs run in four short
e2e (webkit-macos-calls, <group>) jobs on fresh macOS runners (its own workflow,
ci-macos-calls.yml, so a timeout there never holds back a release), which start PostgreSQL and
SeaweedFS as local processes with E2E_NO_DOCKER=1 and are killed from inside by a watchdog after 13 minutes if wedged; neither is a required
check; the macOS jobs leave out the two specs that encode a 1080p or three-layer simulcast screen share (screen-layers.spec.ts and the stream-quality spec of talk-extras.spec.ts): the two-core hosted runner wedges on them, and Chromium and Linux WebKit cover those paths; the Playwright version is pinned exactly in e2e/package.json; the WebKit 27.2 it ships since 1.64 reports a style-src-attr violation for every <select>, which the e2e fixtures ignore in WebKit only, see product status); Firefox runs a smoke subset
(sign-in, chat, search, mentions, sealed Talk, key rotation, screen share) as
e2e (firefox). Neither is a required check yet. Engine differences the specs
account for: Firefox has no context-level camera/microphone grants (its fake
devices are set by preferences in playwright.config.ts); every WebKit context
gets its own browser process, because WebKit mutes the microphone of all but one
page of a process; Chrome DevTools Protocol specs (crashing a tab, synthetic
touch) skip on the other engines.
Two more projects emulate a phone: mobile-chromium (Pixel 7) and
mobile-webkit (iPhone 15) run only tests/mobile.spec.ts, and the other
projects skip that file. The required e2e check runs chromium and both phone
projects.
Runs do not collide, so several can go at once (other worktrees, agents):
every run uses its own Docker Compose project (mnema-e2e-<pid>) and free
ports for PostgreSQL, S3, the app and WebRTC media, picked at start. Pin them
for debugging with E2E_PROJECT, E2E_PG_PORT, E2E_S3_PORT, E2E_APP_PORT,
E2E_UDP_PORT (first of 51 UDP ports) and E2E_LOG.
Without Docker (e.g. in a sandbox): start PostgreSQL 18 and a SeaweedFS
S3 gateway any other way, run the binary with the environment e2e/run.sh
uses (APP_ENV=development, a fresh JWT_SECRET, ADMIN_INITIAL_PASSWORD,
DATABASE_URL, S3_*, UPDATE_CHECK_ENABLED=false,
WEBRTC_NAT_1TO1_IP=127.0.0.1, ...), then point Playwright at it:
cd e2e && npm ci
E2E_BASE_URL=http://127.0.0.1:58080 E2E_ADMIN_USER=cool_guy \
E2E_ADMIN_PASSWORD=<the admin password you set> npx playwright test
The screen share specs need a Chromium with an H.264 encoder (CI's Chrome has one; a locally installed open-source Chromium may not).
Screenshots
The pictures in the README (doc/images/) show one invented community, Lantern
Fox Studio, a small game studio with eight members, four roles, custom emoji, a
poll, pins, a thread and a build-bot webhook. scripts/screenshots/seed.mjs
creates all of it (avatars, emoji and the game picture are drawn in the
browser; nothing is real) and spreads the messages over one working day;
scripts/screenshots/run.sh builds the app, starts a throwaway stack, seeds it,
starts a Talk with a fake screen share and captures the PNGs with Playwright
(1440x900, English, the phone shot at 390 px). SHOTS_ONLY="chat roles" limits
it; SKIP_BUILD=1 reuses the last build. Never capture a real instance.
CI pipeline
On every pull request and every push to main, .github/workflows/ci.yml runs
these jobs in parallel (rough wall times with warm caches):
| Job | What | Time |
|---|---|---|
secret-scan | gitleaks (pinned, checksum verified) over the tree and the history | < 1 min |
backend | Verdict of backend-checks (gofmt, go vet, OpenAPI check, unit tests, build), five parallel backend-integration shards (server-a, server-b, realtime, rest-a, rest-b: unit + integration tests with coverage, a Postgres service each; internal/server is split in two by a hash of the test name, scripts/test-shard.mjs, and CI fails if a test is in no shard or in both; rest is split into two halves by package, balanced by the times in scripts/test-shard-weights.json, with the same check) and backend-coverage (merges the profiles, applies the coverage gates) | 6–8 min |
govulncheck | Go vulnerability scan | 1–2 min |
frontend | eslint, strict TypeScript, contract drift, vitest with coverage, build, npm audit | 2–3 min |
sdk (node 24), sdk (node 26) | Checks out the bot SDKs at pinned commits (bot-sdk-js, bot-sdk-go), runs their tests and the contract test TestSDKContract against a real server | in parallel |
docker | Image build for linux/amd64 + linux/arm64 (docker-image, not pushed), then smoke, install, backup, PostgreSQL upgrade drills and image scan | 6–9 min |
upgrade-drill (<version>) | One leg per supported source release (0.8.16, 0.9.0, 1.0.0, 1.1.0, 2.0.0, 2.9.12, 2.9.19, 3.0.13, 3.2.1, 3.3.5): install it, seed data through its API, upgrade to this build, back up and restore, check the old release refuses the migrated data | per leg, in parallel |
e2e-build | Builds the web bundle and server binary once for all e2e shards | 1–2 min |
e2e | Verdict of four e2e-shard (i/4) jobs: Playwright against the real binary, each shard with its own stack and Owner; all specs on Chromium plus the phone projects. e2e-report merges the shards' blob reports into one HTML report artifact | 8–11 min |
e2e (webkit 1/2), e2e (webkit 2/2), e2e (firefox), e2e (webkit-macos-calls, <group>) | Other engines, see above; not required checks | in parallel |
The server binary (with the embedded web bundle) is built once in e2e-build
and shared as an artifact (e2e-binary) across all end-to-end test shards
(e2e-chromium, clip-sweep, e2e-browsers), which run with SKIP_BUILD=1
to avoid redundant builds.
Pull requests that change only README/SECURITY/doc/ Markdown skip the heavy
steps (the changes job); every required check still reports success, and the
links, conflict-marker and English-only checks run regardless.
codeql.yml (actions, Go incl. tests and the integration tag, JS) runs on the
same events plus weekly. Caches: Go modules and build cache per job (keyed on
go.sum and the Go version), npm (setup-node), Playwright browsers (keyed on
the Playwright version) and Docker layers (GitHub Actions cache, scope image,
shared with the release build). A new push to a pull request cancels its
running CI; runs for pushes to main always finish, because the release
workflow starts from their result. Every job has a timeout-minutes; tests are
never retried.
Web UI building blocks
The look of the web client comes from tokens and a few shared classes, not from per-view tweaks. The client uses Tailwind CSS 4: there is no tailwind.config; the tokens are in @theme.
web/src/style.css@theme: colours, radius scale (rounded-smtorounded-2xl), elevation scale (shadow-xstoshadow-2xl), motion (--ease-mnema, 150 ms default,animate-*entry animations).web/src/styles/buttons.css:.btnwith-primary,-secondary,-ghost,-danger,-danger-solid, sizes-sm/-lg,.btn-icon,.btn-link,.chip(a choice in a row),.choice-card, and the keyboard focus ring. A loading button setsaria-busy="true".web/src/styles/controls.css: checkbox, radio, slider, select and text fields are styled on the elements themselves (no class needed);.switchis the toggle (role="switch"andaria-checked). Slider fill is painted byshared/lib/rangeFill.ts.web/src/styles/surfaces.css: dialog scrim and panel (BaseDialog),.pop-infor menus and popovers, toast motion, theshadow-popoverutility, and the singleprefers-reduced-motionrule.web/src/styles/settings.css:.settings-groupand its label, card, help and row classes for the account and audio settings pages.
All of it respects the CSP (style-src 'self'; style-src-attr 'none'): no inline styles, only classes, tokens and CSSOM. e2e/tests/a11y.spec.ts runs axe (contrast and accessible names) over the main surfaces.
Releases
Releases are automated with release-please
(.github/workflows/release.yml) and driven by conventional commits
(feat: = minor, fix: = patch, feat!: / BREAKING CHANGE: = major). From
1.0.0 on, the public API listed in api-stability.md only
breaks in a major release: make openapi-diff fails a change that breaks
api/openapi.json unless a commit carries a BREAKING CHANGE: footer.
A breaking change already present at the start of the checked range (the base
branch) was waived when it landed there and is not reported again.
After CI passes on a push to
main, release-please opens or updates a release PR that bumpsversion.txtandCHANGELOG.md. Only the CI run for the commit that is stillmain's tip releases; ifmainmoved on, the run for the newer commit decides.The release workflow merges that PR right away and publishes the GitHub Release and tag
vX.Y.Z, so every green push tomainthat contains afeat:orfix:becomes a release (perf:,revert:,refactor:anddocs:also count;chore:,test:,build:,ci:andstyle:do not). A commit of a non-releasing type that lands after a release leavesmain's tip without a release, so the NAS poller deploys nothing until the next release; merge such commits before a releasing one, or close the line with an empty commit carrying aRelease-As: X.Y.Zfooter. A PR opened withGITHUB_TOKENstarts no workflows, so the workflow reports the tested CI run's result as commit statuses on the release commit (only after checking that it touches nothing butversion.txt,CHANGELOG.mdand the manifest and sits directly on the tested commit); that satisfies the required checks onmain.A red night blocks the release. Before release-please can cut a release or the release PR is merged, the workflow reads the most recent completed scheduled run on
mainof each deep nightly sweep (GATEDinscripts/nightly-gate.py:sim-sweep.yml,visibility-sweep.yml,store-trace.yml,plain-sweep.yml,clip-sweep.yml) and stops with an error when one did not succeed (failure, timeout, cancelled, ...), is older than 48 hours (a nightly that stopped running), is disabled, is a re-run whose first attempt did not succeed (or cannot be read), or when a run number is missing after it (a deleted run; seen once a later run of that workflow exists). A run still in progress is no verdict, so the completed one before it decides; a sweep added less than 48 hours ago that has not had its first night yet does not block. Manual runs of a sweep do not count. Every other scheduled workflow is inNOT_GATEDwith its reason, andscripts/tests/test_nightly_gate.pyfails on a scheduled workflow in neither list. The changes wait onmain: the next push after a green night releases them (or re-run the blocked Release run once a night is green, ifmainhas not moved since).Override, for an urgent security fix only. When a security fix cannot wait for the next green night, run the Release workflow by hand on
main(Run workflow) withnightly_overrideset to the reason, for example the advisory it fixes, andmirror_versionempty. It releasesmain's tip only if CI passed on a push of exactly that commit, still waits for and mirrors every required check, and writes the nightly table, the reason and the actor into the job summary (and a warning annotation). The reason and the actor's login also go into the notes of the release it cuts, as one plain line at the end, so the record outlives the run's logs. The override is the owner's decision: agents never start it on their own. Nothing technical stops them, since they work with the owner's credentials; it is a rule.Forbidden ways past the gate. The override is the only permitted one. Never delete a red scheduled run, never re-run a red night until it is green, never edit
scripts/nightly-gate.py, its lists or this gate inrelease.ymlto let a release through, and never add an entry to a sweep's known-failure list (for examplegroupSim.known.fixture.ts,groupSweep.known.fixture.ts,web/src/stores/storeTrace.known.ts,canaryKnownLeaks) to turn a night green; those lists only shrink, or grow with a recorded finding. The gate detects a deleted run and a re-run cheaply, but not every edit: no commit trailer, variable or label skips it, and the script and the workflow are code-owned paths.The same workflow then builds the image and pushes it to GHCR:
ghcr.io/mnema-xyz/mnema-talk:X.Y.Z,:X.Yand:latest(linux/amd64 and linux/arm64), with OCI labels, provenance and SBOM, and signs every tag keylessly with cosign (verify with operations.md). The image job pushes withGITHUB_TOKEN: the package grants this repository write access under "Manage Actions access". Releases up to 3.2.1 also exist at the old pathghcr.io/msch128/mnema-talk, which gets no new ones (the repository moved to the mnema-xyz organization, and GHCR does not redirect). Thedockerhub-mirrorjob copies the image to Docker Hub, one leg per account:docker.io/<DOCKERHUB_NAMESPACE>/mnema-talkwith the secretsDOCKERHUB_USERNAMEandDOCKERHUB_TOKEN, anddocker.io/mnemaxyz/mnema-talkwithDOCKERHUB_MNEMAXYZ_USERNAMEandDOCKERHUB_MNEMAXYZ_TOKEN(Docker Hub allows no hyphen in a name). Each leg copies:X.Y.Z,:X.Y,:Xand:latest, checks the digests, signs the copy the same way; the mnemaxyz leg also sets that repository's short and full description fromdeploy/dockerhub/(its token has the Read, Write, Delete scope this needs). A leg without its secrets is skipped with a notice and the release is not affected. To mirror an existing release, run the Release workflow by hand onmain(workflow_dispatch) withmirror_versionset to the version, for example2.0.0. Only the mirror job runs: it resolves the GHCR digest of that version, mirrors it only if cosign shows that this workflow signed it, and moves:X.Y,:Xand:latestonly when it is the newest release. The version and commit are baked into the binary (internal/version, via theVERSION/REVISIONbuild args) and the same version into the web app, so the app can tell browsers when a newer version is running..github/workflows/release-check.ymlchecks what a release really published, daily and by hand (Run workflow, optionally with a tag): the GHCR image must pass the same verification the updater runs (TestLiveReleaseImage), and the release'smnema-web-*.zipmust satisfy the desktop clients' archive and manifest rules (TestPublishedBundlesSatisfyDesktopLayout). A failed run is the alert.Desktop builds are published from talk-desktop; this repository's releases stopped attaching them after v0.7.3. They are unsigned Beta packages (see desktop-app.md). The server ships through Docker; releases do not publish standalone server binaries.