Architecture
From mnema-talk v3.3.13, rendered on this site.
Mnema Talk is one Go binary that serves the REST API, a WebSocket hub, a
WebRTC selective forwarding unit (SFU) and the built Vue web app. State lives
in PostgreSQL; uploads, avatars and profile banners live in S3-compatible object storage
(SeaweedFS in the default docker compose stack).
Browser (Vue 3 SPA)
HTTPS / WSS │ │ UDP (DTLS-SRTP)
▼ ▼
┌───────────────────────────────────────┐
│ mnema-talk (single Go binary) │
│ http router ─ REST handlers │
│ ws hub ───── events, presence, │
│ voice signaling │
│ sfu ──────── Pion WebRTC forwarding │
│ web.go ───── embedded web/dist │
└──────────┬─────────────────┬──────────┘
▼ ▼
PostgreSQL 18 SeaweedFS (S3 API)
Backend components
| Package | Role |
|---|---|
cmd/server | Entry point: loads the config, sets up JSON logging, connects the database and storage, starts the retention worker, the SFU, the update checker and the HTTP server, shuts down gracefully. |
internal/config | Reads every environment variable, applies defaults and validates them; the server refuses to start on a bad value. See operations.md. |
internal/server | Router wiring (chi): middleware, /api/health, /api/metrics, /api/legal, the API docs, the WebSocket endpoint, the authenticated and admin route groups, and the admin System tab (system.go, system_update.go, system_selfupdate.go). Integration tests live here too. |
internal/httpx | Middleware: request IDs, security headers (CSP, HSTS in production), CORS, CSRF origin check, body limits, rate limits (ratelimit.go), client IP from trusted proxies (request.go), JSON errors. |
internal/auth | Password hashing, signed session cookies with a per-user token_version (revocation), login lockout, invite codes, admin seeding, user administration, password re-authentication for sensitive actions (authn/reauth.go); sign-in handlers, two-factor and the e-mail flows live in internal/auth/authn. |
internal/perm, internal/roles | Roles and permissions: the permission bits, the in-memory Snapshot that resolves base, role and overwrite permissions (and the hierarchy rules), the cache with live invalidation, and the HTTP API for roles, member roles and channel overwrites. See roles-and-permissions.md. |
internal/chat | Categories, channels (text and voice), layout, messages, replies, threads, reactions, mentions (by name, @all/@here and @Role, with a fan-out cap) and the mentions inbox, read state, notification levels (notify.go), search (search.go, filters in search_filters.go), members. Since 1.1 also direct messages (dm/), blocks (blocks/blocks.go), reports (reports.go), saved messages (saved.go), forwarding (forward.go) and thread follow (thread_follow.go). The package is split by domain into layers: core (model, lookups, permission guard) and stage (turning a Talk into a Stage) at the bottom, then mentions, blocks and readstate, messages, channels, dm, search, reports, forum, saved and expiry (the end of timed messages), and talks, botapi and account on top; the root package only holds the Handler that builds each domain's small API from its fields and mounts the routes. internal/repo/boundaries_test.go enforces which package may import which. |
internal/later | Reminders and scheduled messages: private per-member lists, the sweep that delivers and sends them, and their REST routes (see "Reminders and scheduled messages"). |
internal/automod, internal/moderation | Automod (1.2): rules compiled once per configuration version and checked in the send and sign-up paths (mention spam, new accounts, raid protection, word and link lists); a hit blocks or flags a message and adds an entry to the reports queue. moderation is the REST surface of the rules and of the per-member moderation history with private moderator notes. See moderation.md. |
internal/communityevents | Community events (1.2): events, RSVPs, cover images, the scheduler that starts events and sends reminders 15 minutes ahead. See events.md. |
internal/wiki | The wiki (2.9): one tree of pages with a revision for every save, locks, redirects after renames, restore, search and pictures. Reading needs View the wiki, writing Edit the wiki, locking and deleting Manage the wiki; a save must name the revision it started from (base_rev), so a concurrent save is refused. See wiki.md. |
internal/onboarding | The first-run side of the community: name, welcome text and rules, who accepted which rules version, dismissed tips and the public invite preview. |
internal/media | Uploads with content sniffing and an allow-list (policy.go), authenticated media serving through the app, admin storage stats and pruning, and the retention worker (off unless MEDIA_RETENTION_DAYS > 0). |
internal/s3 | S3 client (AWS SDK v2); creates the bucket only when missing. |
internal/db | pgx pool and the embedded, numbered migrations in internal/db/migrations/. Each applied file's checksum is recorded in schema_migrations; an edited migration stops startup. |
internal/events | The Publisher interface between feature packages and the hub (Broadcast, SendToUsers), with an in-memory recorder for tests. |
internal/ws | WebSocket hub: one connection per browser session; chat events, presence and idle state, typing, voice rooms (voice.go), mute/deafen state, speaking indicators and WebRTC signaling for the SFU. |
internal/sfu | Pion WebRTC SFU: one room per voice channel, forwards RTP without transcoding; per-viewer subscriptions (subscriptions.go); announced addresses from WEBRTC_NAT_1TO1_IP, re-resolved every 5 minutes (announce.go). |
internal/slash, internal/slashcmd, internal/poll | The built-in slash commands: a registry of server-side commands that share the composer menu and the invoke route with bot commands (adding one is a slash.Builtin and a line in the server wiring). slashcmd holds /me, /shrug, /tableflip, /unflip, /spoiler, /nick, /thread, /msg and the moderation shortcuts /timeout, /ban and /kick; poll holds /poll. A command never decides on its own what a member may do. See built-in commands and polls. |
internal/push, internal/mailer, internal/mdplain | Web Push (VAPID keys, subscriptions, payloads in the account's language, previews cut to plain text by mdplain), and the few plain-text e-mails (address confirmation, password reset, two-factor alert, test message) sent through the operator's SMTP server. Without SMTP_HOST nothing is sent. German and English mail and push texts live in locales/ files next to the code. |
internal/emoji, internal/emojitext, internal/opwarn | Custom emoji (uploads are redrawn, never stored as sent; <:name:id> in text), and operator warnings (configuration problems registered under a stable code and shown in the admin System tab). |
internal/metrics | Process-wide counters (HTTP, realtime, SFU, storage, rate limits) rendered in the Prometheus text format without a client library; every labelled family has a series cap. Feeds /api/metrics and the live operator view in the admin System tab. See operations.md. |
internal/webbundle, cmd/webbundle | Builds, signs (Ed25519) and verifies the web app bundle that the trusted desktop client runs instead of server-served JavaScript. |
internal/linkpreview | Link cards: fetches title, description and image of public pages with an SSRF guard (http(s) on 80/443 only, every resolved address checked, also on redirects) and a cache. |
internal/imageverify, internal/updaterd | The self-update sidecar (cmd/updater): checks the cosign signature and SLSA provenance of a new release image against the pinned release workflow identity, then pulls that digest and has the recreator restart the app. |
internal/update | Release check against GitHub every 30 minutes (ETag, backoff on 403/429), image-tag reach analysis (image.go) and the client for the optional updater sidecar (updater.go). |
internal/raid, internal/mention, internal/worker, internal/userdata | Raid protection (sign-up throttle, lockdown, minimum membership age; moderation.md); the one definition of an @name mention that chat and automod share; one panic-safe iteration of a periodic maintenance job with its mnema_job_* metrics; and the registry in which each domain contributes its part of a member's data export and account erasure. |
internal/safego | Starts every goroutine, timer and callback of the server. A panic is logged and counted (mnema_goroutine_panics_total); work that belongs to one connection or one call participant then closes only that, and a process-wide loop lets the process exit. |
internal/capacity, internal/community | The planning estimate of how many people one Talk of each kind can hold (the cost model of benchmark.md, shown in Admin > Status) and the community's display name. |
internal/backupstatus | Reads the status file scripts/backup.sh writes (last backup, size, restore test) for Admin > Status. |
internal/setupwizard, internal/setupcheck, internal/portconflict, internal/stunprobe | The first-run helpers: mnema-talk setup (writes .env and docker-compose.yml), the green/amber/red setup checks of the Status page and the log, the readable "port already in use" message, and the STUN binding request that finds the public IPv4 address. |
internal/debugsrv, internal/memlimit, internal/imgproc, internal/textcut | The loopback-only pprof server (PPROF_ADDR), the soft Go memory limit at 85 % of the container limit, the image helpers that uploads share (GIF structure check, redraw, encode) and grapheme-safe shortening of user text. |
internal/fastudp | Sends the SFU's and the built-in TURN relay's UDP datagrams from many goroutines at once on one socket (sendto(2) on the descriptor without Go's write lock, Linux and IPv4 only; everything else goes through the wrapped connection). |
internal/imgstrip, internal/reportcount | Removes the metadata of uploaded JPEG, PNG and WebP files (EXIF with its GPS position, XMP, comments, thumbnails) without re-encoding them, keeping only the orientation; and counts the open reports (from members and automod) and tells the moderators when the number changes. |
internal/tools/openapifix, internal/testutil | Build and test helpers: the fixes to the generated OpenAPI document that swag cannot express (run by make), and shared test fixtures. |
internal/repo | The repository-wide checks: size limits for Go files and functions (AGENTS.md). |
internal/version | Version and commit baked in at build time (VERSION / REVISION build args). |
web/web.go | Embeds web/dist with //go:embed and serves the SPA. |
Data flow: chat
- The browser sends
POST /api/channels/{id}/messages(or/uploadfor a single attachment in one request). Files for a message with several attachments are staged first:POST /api/channels/{id}/attachmentsstores one file as a media row withstaged = TRUEand no message, visible only to its uploader; the message then lists up to 10 ids inattachment_ids, which are claimed in the message's transaction in that order (media.position), at most 100 MB together. A staged file nobody sends is deleted after six hours bymedia.SweepStaged. The request passes the CSRF origin check, the session check and the per-user rate limit. messages.InsertMessagestores the message and its mentions in one transaction (an upload's media row, or the claim of the staged files, joins the same transaction).messages.PublishMessageloads the stored message, broadcastsmessage_createthrough theevents.Publisherand answers 201. A typed message and a new forum post are read inside the transaction of step 2 instead (InsertStoredMessage,PublishStored), so an error answer means nothing was stored; an edit, a reaction, a new conversation, an event comment or answer and a new webhook answer from inside their transaction the same way.- The hub writes the event to every open WebSocket; the chat store inserts it, updates unread and mention counters and plays the notification according to the channel's notification setting.
Unread state is derived, never fanned out per recipient. A member's read marker
is one channel_reads row per channel (the time they have read up to, plus
their notification setting); a send writes no marker for anyone. Unread and
mention counts are computed on read from the marker and the indexed
(channel_id, created_at) message order, so the cost of a send does not depend
on the number of members. A marker is written only when the client reports a
read: the web app sends at most one request per channel every two seconds
(leading and trailing), and the server writes only if another member's message
lies between the old marker and the new one (or the member has no marker yet),
so a second tab, Esc on a read channel or a retry costs one index probe and no
row. The "new since" divider is placed from the marker the client saw when it
opened the channel; when the first unread message is older than the loaded page,
GET /api/channels/{id}/first-unread?since=... (one indexed lookup) names it
and the "Jump to first unread" pill loads the history around it. Numbers:
Capacity.
Every channel has a text chat, voice channels included (the chat panel under
the Talk stage). The chat of a Talk follows the channel permissions only
(View channels to read, Send messages to write), not Connect, so a
member can read it from the Talk's row in the sidebar without joining. Edits, deletes, reactions, typing and read state follow the
same pattern: REST for changes, WebSocket events for fan-out. Media is never
served from S3 directly: GET /api/media/{id} checks the session and streams
the object through the app.
Forum channels
Forum channels (channels.type = 'forum', 1.2) are structured discussion spaces. Instead of an unthreaded message stream, a forum channel holds posts displayed as cards, each with an associated thread.
- Post model: A post reuses the existing thread model. It is a root message (
parent_id IS NULL) in a forum channel with a requiredtitle(1..100 characters), an initial body (1..4000 characters), and up to 5 optional tags. Replies to the post form the thread (messages.parent_id). - Tags: Each forum channel can define up to 20 admin-curated tags (
forum_tags, name <= 20 characters, optional emoji <= 32 characters, a palette colour, a position and a moderated flag). Posts associate with tags viaforum_post_tags. Per-forum rules (a tag is required, guidelines) live inforum_settings; a post's solved mark ismessages.is_resolved; per-member read markers of posts areforum_post_reads. - Permissions: View channels gates reading posts and tags; Send messages gates creating posts and posting replies in threads. Moderation (Manage messages) allows locking posts (
messages.is_locked = TRUE), which restricts replies to moderators only, and editing tags on other members' posts. - Search and Unread: Because posts are root messages, unread tracking, mentions, search indexing, and thread follows integrate seamlessly without a secondary store.
Direct messages
Direct messages (1.1) are conversations of two to ten members. They are a deliberate reversal of migration 0003, which removed the earlier 1:1 model (recorded as a decision in the repository's decision store): the product position changed from "no direct messages" to private conversations that moderators cannot browse.
Model. A conversation is a row of channels with type = 'dm' (no
category, no name, no overwrites) with its members in dm_members
(migration 0050). Messages, reactions, attachments, polls and threads use the
one message pipeline, keyed by the conversation's id, so they inherit its
behaviour; a 1:1 conversation has a dm_key (the sorted pair), so a pair has
exactly one. Conversations have their own routes (/api/dms, see the OpenAPI
document) and events (dm_created, dm_member_added, dm_member_removed, and dm_prefs_updated to the
member's own sessions);
they appear in no channel list, layout, channel_* event, bot or webhook
payload, public message count or public message number (conversations count
down from -1 in a sequence of their own, so the public numbers show no gaps).
The member's own list (1.2, migration 0074). What a member does to their list
is stored per member and follows them to every device: dm_members.hidden_at
(a closed conversation, cleared by the next message written in it unless the
member blocked the sender), dm_members.favourite_pos (favourites, at most 25,
ordered), users.dm_mute_all and users.dm_list_collapsed. None of it touches
the conversation or its messages, none of it is visible to other members, and
every route checks membership first (a conversation you are not in answers 404,
also for the favourites list). GET /api/dms carries hidden,
favourite_position and last_message (the latest message the caller can see:
blocked authors are left out); with mute-all on and no setting of the
conversation's own, notify_level reads mute and mention_count 0, and push
sends nothing. The feature name is dm.v2.
One access rule: membership. perm.Snapshot.Channel answers for a
conversation before the Owner shortcut, from membership alone. A member may
read; writing, reacting and attaching also need Send direct messages, the
community switch direct_messages_enabled to be on, and no timeout or pending
rules. A non-member, the Owner and Administrators included, gets 0 and the
same 404 as for a missing channel, on every path that asks the snapshot (REST,
the WebSocket hub, media, push, reports). Searching a conversation also tests
membership in SQL, so a mistake in one layer does not open another. Nobody can
moderate a conversation: Manage messages never applies in it.
Fan-out. The hub delivers a conversation's events to its members' sockets
that asked for them (/api/ws?dm=1; a bundle from before direct messages
never receives one). They are never shown to the hub's Tap (outgoing
webhooks, bot gateway); only Web Push listens (DMTap), and it pushes to
members only, without a channel name. Bots and webhooks can neither open
conversations nor write in them.
Starting and writing (the rules in internal/chat/dm/*.go).
- Whatever the reason a conversation cannot be started or a message cannot be
sent because of another member (a block in either direction, their
dm_policyofnobody, a disabled or erased account, a bot, an unknown id) the answer is the same 403 text, so none can be probed. A block makes an existing 1:1 conversation unwritable in both directions, silently (new messages, edits, reactions and poll votes alike, in the chat of the conversation's Talk too; deleting one's own messages and taking a reaction or vote back stay possible, and the Talk and its seats are not touched); in a group, a block between two members does not stop the group, and nobody can add a member who blocked them. In any conversation (group or two-person), unread and mention counts and the last-activity time leave out messages by members the reader blocked, and push notifications skip blockers. - 3 tries a minute and 10 new conversations per 24 hours per member, at most
100 group conversations per member, at most 10 members. The member who
started a conversation may send 2 messages before anybody else has written
in it (counted in
channels.dm_creator_sent, so deleting messages does not reset it). - A conversation with one other person cannot be left; a group can, and when nobody is left it and its attachments are deleted. A member added to a group reads its whole history.
What is never done with conversations. They are not audited (an edit or
deletion leaves no message.* entry with text), not counted in message_count (nor is the chat of their calls, migration 0098),
not in the mentions inbox or the channels' read states (GET /api/dms carries
its own unread and mention counts), not searchable except by their members
(dm=true), not shown in the admin media list, and not listed by the
administrative channel routes (they answer 404 for a conversation id).
Moderation. The only way a moderator reads anything of a conversation is a
report by a member (POST /api/messages/{id}/report): it keeps its own copy
of the message and up to 20 around it (moderation).
Privacy model. Messages are readable by the server and by whoever holds the database or backups, like channel messages: there is no end-to-end encryption of text (see SECURITY.md). Account erasure blanks the member's words in conversations (the others keep theirs) and deletes conversations nobody is left in; the export lists the member's conversations by id and their own messages.
Calls. A conversation has no call of its own: its call button ("Call (starts a Talk)") opens a temporary Talk and posts an invitation, and the server rings the other members (dm_call_ring, dm_call_end, a "missed call" message when nobody answers); see Talks.
Blocking and reports
A block (user_blocks, migration 0049) is private: the blocked member is not
told, and nothing they can read depends on it. The server applies it where a
notification would reach the blocker: the mentions inbox (which excludes direct
messages), mention counts and push skip the blocked member's messages, decided
when the entry is listed or the push is sent. In any conversation (group direct
messages included), unread and mention counts and last activity leave out
messages by members the reader blocked. The web app collapses their messages to a
placeholder the blocker can open. A block changes nothing for anybody else. It makes
an existing 1:1 conversation unwritable in both directions (see
Direct messages). The block_updated event goes to the
blocker's own sessions only.
A report (message_reports, migration 0049, permission Manage reports,
1 << 21) keeps its own copy of the message and up to 20 around it, so it survives an
edit or a delete. It is the only way a moderator reads anything of a conversation. Rules,
retention and the audit entries are in moderation.
Anonymous polls
An anonymous poll (feature polls.anonymous) stores two facts that nothing
in the schema connects. This hides votes from every member, moderator and
administrator in the app; an operator with direct access to the database
server (its WAL, transaction ids or statement logs) could, in theory, still
correlate the timing of a voter row and a ballot. Mnema itself never stores
which option a member chose. poll_voters (poll_id, user_id) says that a member voted, which
makes a second vote impossible; it has no timestamp, so neither insert order
nor time can be matched against the other fact. The choice goes to
poll_anon_ballots (poll_id, option_id): no member, no timestamp, appended in
the same transaction but by a separate statement. Readers add the unfolded
ballots to the counter poll_options.anon_votes, and poll.RunFold sums the
ballots into it and deletes them every ten minutes. polls.anon_voters is the
participant counter. No table, log line, audit entry, event, export or data export holds
a member next to a choice; a trigger on poll_votes (which still serves polls
that name their voters) refuses a row for an anonymous poll, and an integration
test walks every table of the schema to prove that no row mentions a voter and
an option together. A running anonymous poll broadcasts no tally at each vote, since in a small audience the
last vote would follow from it, and the REST and message payloads report 0 for every option until the poll ends (only the number of voters is live); the result goes out when the poll ends. Because a choice is never known, a vote cannot be changed
or withdrawn and the data export of a member holds only that they took part
in an anonymous poll (the poll id, never the choice). Erasing an account deletes its poll_voters rows and leaves the
counters, as a tally is not personal data. Migration 0063 converted the
anonymous polls that existed before 1.2 the same way (see
upgrade.md).
Per-channel auto-delete
channels.auto_delete_hours (NULL follows instance_settings.auto_delete_default_hours,
0 is off, up to 8760) and auto_delete_scope (all or attachments; a check
constraint forbids a period on conversations and their call Talks, migration
0106) are read by channels.AutoDeleter
(internal/chat/channels/auto_delete.go), a
background loop started with the router that runs on whole multiples of ten
minutes. Each pass lists the channels with an effective period and works
through each in transactions of at most 200 messages (or files): share-lock
the channel row and the settings, read the effective setting again in a
statement of its own (a change stops the run), lock the expired rows
(FOR UPDATE ... SKIP LOCKED, which also holds off concurrent replies and
uploads, so the list of attachment keys is complete), collect their media
keys, delete the rows, then delete the objects and broadcast
message_delete. With the scope attachments the media rows are marked
removed instead (is_deleted, the message stays) and message_update is
broadcast. The audit entry per run is written by the sweeper itself.
Messages with a lifetime
messages.expires_at is set at the insert from the same clock reading as
created_at and never changes (a trigger refuses it; checks keep it within
60 seconds to 30 days and off pinned messages). expiry.Sweeper
(internal/chat/expiry/sweeper.go)
sleeps until the next end (min(expires_at) through a partial index); the
message door wakes it after the commit of a message with an earlier end
(messages.OnExpiry). It deletes each ended message in a transaction of its
own through messages.DeleteExpired, the path of an own delete
(removeLocked: the author's row, then the message's, then the end read
again), then deletes the objects, broadcasts message_delete and recounts.
Reads do not wait for it: core.LiveMessages and core.LiveSQL leave out a
row whose end has passed in every read (doc/timed-messages.md).
Forwarding and saved messages
A forward is a reference, never a copy. POST /api/messages/{id}/forward
creates an ordinary message in the target (a channel or a direct message) whose
text is only the forwarder's own comment. messages.forward_of points at the
root of what was forwarded (forwarding a forward points at the same root) and
messages.forwarded stays set when the original is later deleted
(ON DELETE SET NULL). The forwarder needs to see the original and to send in
the target, the target's slow mode and a conversation's rules apply, and every
refusal answers one 403 text.
The visibility rule. Nothing about the original is in the forwarded
message's payload, not even an id: only forward: {forwarded: true}. What a reader sees of the
original, with its ids, is returned by GET /api/messages/{id}/forward, decided for that
reader at that moment: the text, author and channel when they may see the
original (a channel they can view, or a conversation they are a member of; the
Owner has no special access to conversations), otherwise only
available: false. Lists, thread views, pins, search, the mentions inbox, the
WebSocket events, push notifications, the export and the payloads of bots and
webhooks therefore cannot show the original to someone who may not see it.
A forwarded message with an empty comment is pushed as "Forwarded message".
Edits and deletes of the original show through because the quote is read when
asked for.
Saved messages (saved_messages, migration 0051) are the same idea: a row
holds a reference and the member's note. GET /api/me/saved joins the messages
the member may see now; a conversation's messages need membership, tested in
SQL like search does. A deleted message takes its saves with it (ON DELETE CASCADE).
The web client keeps the saved list and the reminders and scheduled messages
current from the events to the member's own sessions (saved_updated,
reminder_update, scheduled_message_update). Those of a time the socket was
down are lost, so every socket open reads again the lists that were already
asked for (one never opened stays unread), and an answer older than a newer
request of the same list, or than a change made meanwhile, does not overwrite it.
The saved list is read again as far as it was shown (up to 100 entries; older
entries shown behind them stay until a page replaces them), and the saved marks
follow the list that lands last. A failed read after a reconnect keeps the
lists shown.
On message_delete the client drops the saved entry (and those of the thread's
replies) and shows a reminder on the message as unavailable, as a reload would.
Reminders and scheduled messages
Both live in internal/later (migration 0062)
and are private to their owner. A background loop (every 15 seconds, started by
the router, Service.Sweep) delivers the due reminders and sends the due
scheduled messages. Times are absolute: the web app works them out in the
member's own time zone ("tomorrow morning" is 08:00 local) and the server only
checks that they lie between 10 seconds and a year ahead.
A reminder is a reference (message_reminders): at most 100 per member. The
message may be deleted later (ON DELETE SET NULL). GET /api/me/reminders
checks the member's access on every read, as saved messages do, and answers
available: false without text for a message they cannot see. When a reminder
falls due the row is claimed first (delivered_at, at most once), then the
member's sessions get reminder_update (due) and the push pipeline gets a
push.Direct. The access check runs again at that moment against the
permission snapshot (a conversation needs membership); a message the member can
no longer see is announced as "a message you can no longer see", with no ids,
author or text. A delivered reminder stays in the list until dismissed (or for
30 days) so a member who was offline finds it.
A scheduled message is sent by the server as its author through the normal
send path (scheduled_messages, at most 50 per member, text only). Creating
one needs Send messages in the channel, but nothing is promised: at send_at
the worker loads the account (disabled, banned and deleted accounts do not
send), builds the same send context as a request (messages.SendContextFor, so a
changed role, channel overwrite, pending rules or a timeout apply), validates
the reply and thread target again and inserts the message with
messages.InsertMessage, whose transaction applies slow mode and the direct message
rules (blocks, closed conversations). The scheduled row is deleted in the same
transaction, so a message is neither sent twice nor lost half-way, and a row
edited meanwhile is not sent in its old version. Unlike a typed message, the
send leaves the author's read marker alone: they are not looking at the channel
then. A refusal turns the row into a
failed draft with a reason code (slow_mode, not_allowed, unavailable,
refused; a 403 never says which rule, because it can concern another member)
and tells the author with scheduled_message_update (failed) and a push. The
author can edit the text, set a new time (which puts it back to waiting) or
delete it. A failure of the server itself (the database) leaves the message
waiting for the next round.
Data flow: voice, cameras and screen share
Signaling runs over the existing WebSocket; media goes over UDP straight to
the SFU (WEBRTC_UDP_PORT_MIN–MAX), or through TURN when configured: the
relay built into the server (internal/turnrelay,
WEBRTC_BUILTIN_TURN) or an external one. The built-in relay opens its relayed
sockets on the server itself and forwards only to the SFU's own media sockets
(translating the announced WEBRTC_NAT_1TO1_IP address to the local socket and
back), so it needs no public relay ports and cannot reach anything else.
- Join. Clicking a voice channel sends
voice_join. The hub (voice.go) leaves any previous room, callssfu.Joinand announces the participant. The SFU is the offerer: it sendswebrtc_offerandwebrtc_candidate; the browser answers withwebrtc_answerand its own candidates. - Publish. Each peer connection has fixed receive lines, told apart by
position: the microphone (audio), the screen (first video line) and the
camera (second video line). In 0.4.0 a second audio line carries the
screen's sound separately (stream ID
screen:<user>); before that, the screen's sound was mixed into the microphone track. Audio and screen are published under the user's ID as stream ID, a camera undercam:<user>(sfu.go). The web client changes the SFU's offer before applying it so that the Opus parameters of the microphone line carryusedtx=1(discontinuous transmission, RFC 7587; the screen's sound does not). While the encoder finds nothing but silence the browser then sends about 5 small packets a second (some 16 bytes) instead of 50, around 90 % fewer; speech goes out at the full rate. The encoder decides by the signal: true silence and quiet room noise count as silence, speech never does, a steady tone or very quiet music may. So the setting "Save bandwidth when silent" (Audio, Fine-tune; on by default) turns it off, and it is off while the noise filter is Off, the setting for studio microphones and music; it applies from the next Talk. With sealed media the encoder's empty frames (one or two bytes, header only) pass the media worker, which drops them on a line that uses DTX: sealed, each would be 107 bytes of overhead plus its one or two payload bytes, and they would go out 50 times a second. Sequence numbers stay consecutive and only the RTP timestamp jumps, which the SFU forwards untouched; listeners' browsers fill the gap with comfort noise. Nothing on the server depends on a steady packet rate: speaking comes from the client's ownvoice_speaking, the ingress limits are ceilings, and the track is not timed out. The microphone line shown as "sending" in the connection panel follows the track, not the packet counters, and the no-sound warning reads the raw capture. - Forward. For every incoming track the SFU creates one local track and
copies the RTP packets to it (
forward); a write error for one subscriber never ends the track for the others. No decoding, no transcoding, no recording. Viewers' keyframe requests (PLI/FIR) go to the publisher, rate-limited per track (300 ms), so a burst of viewers costs the publisher one keyframe. This rate limit and the SFU's other stored times (the 500 ms in which late packets of a stopped track do not publish it again, a layer's last packet, resends, bitrate measurements) follow the monotonic clock, so a step of the host's wall clock moves none of them. - Subscriptions (opt-in). What each viewer receives is decided per
viewer (
subscriptions.go): voice audio is always forwarded; cameras are on by default and can be turned off per person or for everyone; screen shares are off by default and only sent after the viewer clicks "Watch" (webrtc_subscribe). The screen's sound follows the screen subscription. Changes renegotiate with that viewer only. The hub tells the room who publishes what (webrtc_media_state) and who watches each share (screen_viewers), so clients can offer "Watch" and show viewers without receiving the video. Cameras are received only while their tile is on screen. The web client reports which tiles are visible (IntersectionObserver), keeps the cameras of the others off with the same opt-out (several at once in onewebrtc_subscribewithuser_ids, so one renegotiation), and asks for a camera again about a tenth of a second after its tile comes back; a camera the member hid stays hidden. As a safety net the server forwards at mostMaxViewerCameras(24,camera_cap.go) cameras to one viewer, the lowest user IDs first (by ID alone: it ignores which camera the viewer focuses, an accepted limitation for a limit meant never to be reached), and tells the viewer which others it does not send (cameras_capped) so their tiles say so instead of showing a frozen picture. Egress per viewer scales with the cameras on screen, not with everyone who has one on. - Stop.
webrtc_screenshare_stop/webrtc_camera_stopunpublish that source; the screen opt-ins belong to one share and are dropped with it. Leaving, a failed connection or an admin kick removes the peer.
Codec pinning. The SFU pins the codec order of the first offer on each
video line (pinCodecOrder). Without it, Pion
re-offers every video line in the order of the client's screen line, where the
web client puts H.264 first; a browser would then switch a running camera
from VP8 to H.264 at the next renegotiation while the forwarded track keeps
the first codec, and viewers could not decode it. The client still reorders
its own screen line so screen shares prefer H.264 (hardware encoding) and get
a higher start bitrate (tuneScreenOffer).
Layers of a screen share (0.8). A sharer can send its screen as up to three
separate encodings of the same capture ("layers", best first), and each viewer
receives exactly one of them. The SFU offers three more receive lines for this,
after the four above: the screen's lower layers are the third and fourth
video line (screen.go,
layers.go). They are plain lines, not RTP
simulcast (rid): the SFU is the offerer and a browser cannot add simulcast to
an answer, whereas separate lines work in every browser and need no SDP
munging. A client that does not know them answers them inactive, so old and
new clients interoperate in both directions. Per viewer the SFU forwards one
layer on one outgoing track and switches layers only where a keyframe of the
new layer starts (a VP8 keyframe tag or an H.264 parameter set, both of which
the browser's media transform leaves readable), asking the publisher for one
when it switches. Sequence numbers, timestamps and VP8 picture IDs and
TL0PICIDX are rewritten so the viewer sees one continuous stream; the sealed
payload is never touched. A switch continues after the highest sequence
number, the newest timestamp and the newest VP8 picture ID and TL0PICIDX
sent, not after the last packet: a packet that arrived late (overtaken on the
way, or repaired by a retransmission) would otherwise give the new layer's
keyframe numbers the viewer already holds, and its browser would drop them as
duplicates. A late packet the new layer's sender sent before the keyframe of
the switch is dropped for the same reason. The sender declares the layers
(webrtc_layers, announced as screen_layers), a viewer picks one with quality on
webrtc_subscribe and is told what it receives (screen_quality), and the
sender is told which layers anyone needs (screen_layers_wanted) so it can
stop encoding the others. A layer nobody needs any more stays wanted for eight
seconds, so switching back and forth does not restart an encoder each time.
Auto. A viewer who leaves the choice to the server starts on layer 0 and is
moved by what its browser reports (adapt.go):
the loss fraction and round trip time of its RTCP receiver reports (about
every second), how often it asks for a keyframe, and a bandwidth estimate.
Lasting loss (5 % twice in a row, 15 % at once), a round trip time 250 ms above
its recent minimum or three keyframe requests in five seconds move it down one
layer; the reports of the three seconds after any move are ignored (a keyframe
burst loses packets on its own). After 12 seconds without trouble it tries one
layer up, and a try that fails within eight seconds makes the next one wait
twice as long (up to five minutes), so a link that carries the lower layer but
not the higher one does not flap. A viewer who picked a layer is never moved.
The SFU cannot read the media; it never estimates from it.
Size of the picture. The stage tells the server how large it shows the share
(fit on webrtc_subscribe: the index of the smallest layer that still has
the pixels the stage shows, its CSS height times devicePixelRatio, see
fitLayer in web/src/talk/lib/screenLayers.ts and useScreenFit). Auto starts on
that layer and never goes above it, so a small tile or a share that is watched
but not on the stage costs a small stream. When the picture grows (fullscreen,
a bigger window) a viewer that sat on the old limit climbs on after four quiet
seconds per layer instead of twelve; a smaller picture moves it down at once.
The viewer's menu shows what Auto chose ("Auto (1080p)"). The bandwidth
estimate of a stream the link does not limit only ever reaches about 1.5 times
what is sent, so while it is still rising after a move up it is not held
against the better layer; it counts as a limit only when it stays below the
layer's bitrate and stops rising.
Bandwidth (bwe.go). A link that is too slow for
a layer but queues instead of dropping shows neither loss nor a higher round
trip time for a long while. So each viewer connection carries Pion's Google
Congestion Control estimator (pkg/gcc of pion/interceptor): the SFU stamps
the transport-cc sequence number on every outgoing packet (beneath the
authorization gate, so a refused packet never leaves a gap the browser would
report as lost), the browser answers with transport-wide feedback (arrival
time of each packet) and the estimator turns both into a bitrate. It is only
read: the pacer is a pass-through, so nothing is delayed or dropped, and
audio, cameras and viewers who chose a layer are unaffected. The cost is the
estimator's history of 250 packets and two small goroutines per connection,
plus one writer per stream it sends, which is forgotten when the stream's
sender stops.
The estimate is compared with the bitrate each layer has at the SFU (bytes
arriving per layer, measured once a second per share, smoothed; a layer the
sender switched off keeps its last size). Delivery that keeps up makes the
estimate about 1.5 times the bitrate sent, so an estimate below 1.2 times the
current layer's bitrate for four reports in a row counts as trouble and moves
the viewer down one layer. The estimate cannot rise above 1.5 times what is
being sent, so it can never show that a better layer fits; a viewer moved down
this way therefore stays put for a minute unless the estimate is above 1.5
times the better layer's bitrate for four reports, and then tries once more
under the usual backoff. The estimates are counted in
mnema_sfu_bandwidth_estimate_reports_total{range} (seven fixed ranges) and the
moves they cause in mnema_sfu_bandwidth_layer_moves_total. Browsers
that send no transport-wide feedback leave the estimate at "unknown" and the
rules above are the only ones. BW_KBPS of the load test gives each viewer a
link with a rate and a queue to try this.
Cameras and audio are not layered (small pictures, and every layer costs the sender an encoder).
The web client (talk/lib/screenLayers.ts,
syncLayers in useWebRTC.ts) sends
the share as the streamer chose it on layer 0 and up to two smaller ones
(1080p, 720p, 480p or 360p, at most 30 fps, at least a quarter smaller than the
layer above) from the same capture with scaleResolutionDownBy on the extra
lines. The lower layers stay switched off (active=false) until the server
says a viewer needs them, and layer 0 is switched off when everybody watches
a lower one. It uses these lines only when server_info.screen_layers says the
server has them. Viewers pick Auto or a layer in a menu on the stage, labelled
with the sizes the sender declared. The sender watches its capture: when the
width, height or frame rate it delivers (a resized window) has changed and
stayed so for a second, it sizes the best layer for the new picture and
declares the layers again; the server announces them only if they differ, and
the pickers follow live. A viewer's explicit choice is a position among the
layers and survives new sizes, but not a different number of layers or other
frame rates. Each layer's frames are sealed in a stream
of their own (see SECURITY.md), so a switch needs no coordination between
sender and receiver.
Layers of a camera. A camera is sent as layers the same way, so a small
tile costs the sender and the viewer a thumbnail instead of the full picture.
After the screen's lower lines the SFU offers two more video lines for the
camera's lower layers; the camera line itself is layer 0
(camera_layers.go). The server never
requires them: a camera is forwarded as one plain track, exactly as before,
unless its sender declared more than one layer (webrtc_layers with
source: "camera") before its first video, which is when the SFU decides.
A camera that began plain stays plain until the connection is made anew: a
declaration that comes while it is live (or when it was last live) is ignored,
because a lower layer joining later would replace its best layer. A
client without the lines (an older browser or desktop client, an older server)
sends one layer, and a mixed room works in every combination: every viewer
gets one track named like a camera (camera:<id>), whether the SFU switches
layers behind it or not. The sealed lines are unchanged: a camera layer is
sealed on the camera's line with its layer only in its stream ID (the same
low bits a screen's layers use), so an older viewer opens it like any camera.
Nobody picks a layer of a camera: the viewer reports how tall its picture of
that camera is in device pixels (fit_px on webrtc_subscribe, from the tile
or the stage that shows it, the largest one if several), said again when that camera arrives and every ten seconds while it is shown, so a lost message heals; a picture that is gone is only reported after two seconds, because a tile moving from the grid to the strip is removed and shown again, the SFU gives it the
smallest layer that still has the pixels the picture shows, and Auto
adaptation can move it lower under loss as for screens. A layer its sender
switched off is attached but silent, so a camera viewer whose picture fits it
is carried by the nearest layer that delivers and moves over, at a keyframe,
when the layer is back; a viewer that has not reported a size yet gets the
best layer (older clients never report, and must not be left on a thumbnail).
Switches count in mnema_sfu_camera_layer_switches_total. The sender is told
which layers anyone needs in the same event as for a screen, marked
source: "camera" (screen_layers_wanted), and switches the others off. The
server announces the capability as server_info.camera_layers; a client sends
camera layers and size reports only where it is there. The web client
(talk/lib/cameraLayers.ts,
useCameraFit.ts) plans up to three
layers of the capture (full size, 360p and 180p thumbnail at 150 kbit/s, none
smaller than 180p) and sends them with scaleResolutionDownBy on the extra
lines.
Stream quality and volume (0.4.0). The sharer chooses mode, resolution
and frame rate; the client applies them as capture constraints and sender
parameters (maxFramerate, a bitrate cap, scaleResolutionDownBy), so the
SFU still forwards whatever arrives. Viewers play the screen's sound in its
own element with a per-sharer volume (0–100 %, default 50 %), independent of
the person's voice volume.
Media key lifecycle (0.8). The browsers of a Talk form an MLS group and
derive the media keys from it; the server only relays the handshakes and
stamps the sender. The member with the lowest id commits a fresh key about
every 30 minutes, every member renews its own leaf key, members that showed no
sign of life for 20 minutes are removed and re-admitted when they return, and
every sealed frame is signed with its sender's leaf key. The connection
details show a safety code for the current epoch so members can compare
keys out of band. The server relays the handshakes, checks their framing and that the sender is in the Talk, and never sees keys; the members validate who may commit.
The code is web/src/lib/media/. Details and limits:
SECURITY.md.
Status expiry
A custom status may carry users.status_expires_at (migration 0061). The
client works out the absolute time from the member's own time zone ("tonight"
is 23:59 and "tomorrow morning" 08:00 where they are) and sends it as
expires_at; the server only checks that it lies ahead, at most a year. Two
mechanisms make the clearing exact without a timer per member: every read of a
user (auth.UserColumns, the member list) maps a
status past its time to an empty one, so nobody sees a stale line even between
sweeps, and Handler.RunStatusExpiry (every 30 seconds, an indexed partial
scan) clears the rows and broadcasts the usual user_update. Setting another
member's status as a moderator never sets an expiry and always removes it.
Notifications, search and onboarding
Three features that cut across the chat path. All of them are enforced on the server; the client only renders the result.
Notification levels. A member sets all, mentions or mute ("Nothing") per channel
and per category, a mute optionally with an end time. The stored value
inherit (the default) follows the category, then all; the level a client
receives is always the effective one. Settings are rows in channel_reads and a
category counterpart (migration 0035) and are changed through
PUT /api/channels/{id}/notifications and PUT /api/categories/{id}/notifications.
Levels decide who is notified and what the mention badge counts. A mute stays
silent except for a message addressed to the member (by name, through a role
they hold, or as a reply); @all and @here do not break a mute. Both the
badge (mention_count in GET /api/read-state) and the push follow this rule.
"Addressed by name" means the server resolved the name under the sender's
mention policy: a name a webhook or bot left as plain text (user mentions off,
allowed_mentions without users, the recipient cap) addresses nobody, and the
web client decides from the message's mentions list, not from the text, so
the live badge, the page's notification and the sound match the badge after a
reload. The answer of GET /api/read-state and the live message_create
events travel separately and can cross. Each answer therefore says per channel
last_message_number, the number of the newest message its counts include:
a message draws its number and commits while its transaction holds the
channel row's lock (messages.insertMessageTx, channels.last_message_number,
migration 0104), so a channel's messages commit in the order of their numbers
and the counts hold exactly the messages up to that number. The client counts
a message_create on top of the answer only when its number is above the
mark, and re-adds the ones it counted while the request was on its way the
same way; a read in another session (read_state with counts) after the
request began keeps the client's state, like a read in this one. Only the
answer of the newest request is kept. This exactness has a price, accepted
for exact counters: the posts of one channel now wait for each other from the
lock to the commit. Measured on the insert alone (messages.InsertMessage, no
HTTP, no rate limit): in one busy channel with 8 concurrent writers about 230
to 480 posts a second instead of about 630 to 660, and a median insert of 15 to
27 ms instead of about 8.5 ms (independent review, loaded machine); a second
run with the erased-person guard of migration 0103 in place, on a quieter
machine, gave about 740 to 760 posts a second instead of about 870 and a
median of about 10.5 ms instead of 6.5 ms. Writers spread over channels show
no difference. A channel still takes several hundred posts a second, far more
than the per-member send limit lets members produce. The wait does not include
the disk: the post's transaction commits asynchronously (db.AsyncCommit), so
the lock is released once the commit is in PostgreSQL's WAL buffers, and only
then the post waits for the WAL flush (db.AwaitDurable), which concurrent
posts share. Nobody is told of a message (the answer, message_create,
bots, push) before it is on disk; a webhook worker claims its event with a
commit of its own, which flushes the WAL past the message first. Other
sessions (message history, read-state counts) can read the message a few
milliseconds before the flush: should PostgreSQL itself crash in that window,
the message is gone after the restart (and its number may be drawn again),
like any transaction committed asynchronously. Before 3.3.5 a post committed
synchronously: a message was on disk before anyone could read it. Acknowledged
messages are as durable as before; only that read-only glimpse is new. The
wait is a transactional logical message (pg_logical_emit_message), which
needs the database role to hold EXECUTE on that function; PostgreSQL grants it
to PUBLIC, and the server refuses to start when it has been revoked
(db.CheckDurable) rather than fail every post. The work under the lock is also
kept to few round trips to the database (seven for a message with a mention:
the lock with the channel's settings, the insert that notes its number, the
mentioned names, the mentions, the stored message and its details, each of the
last three in one round trip, and the commit), since under CPU load every
round trip waits for the scheduler on both sides. On a 4-core Intel N100 host
with a disk that takes about 5 ms per flush, 3.3.3 held the lock for 18
round trips and through the flush: a median of 13 ms, and the posts of a busy
channel waited up to 110 ms (p99) for it. GET /api/threads/following is ordered
the same way without a mark: thread_follow_updated carries the whole state
of a thread, so the changes that arrived while the list was on its way are
applied to it again.
A client counts a new message itself; a delete (also a bulk delete or
the auto-delete sweep, once per channel and run) or an edit that changes the
mentions brings nothing it could count, so the server sends the members whose
count it may have moved (online, allowed to view the channel, not read up to
the changed messages; for an edit, those it mentioned before or after) a
read_state with their unread_count, mention_count and the channel's
last_message_number, which the client applies like an answer of
GET /api/read-state for that channel (recount.go).
The request only queues the recount: one worker counts after the broadcast,
with the server's context, 500 members a statement, and merges the requests
of a channel that is already waiting (at most 256 channels wait; beyond that
a channel is skipped and logged, and its badges heal at the next read,
reconnect or reload). Nothing new starts once the server shuts down. The
worker's time grows with the members told and the channel's unread backlog:
measured, 1,000 members take 0.04 s (50 unread), 0.2 s (500) and 2 s (5,000),
10,000 members 0.3 s, 1.8 s and 21 s; the deleting request waits for none of it.
Do Not Disturb silences everything. Since 1.2 a member also has a server-wide
default (server_notify, migration 0073; PUT /api/notification-settings/server,
GET /api/notification-settings lists it under server): the precedence is
channel, then category, then that default, then all. The default has no
inherit and does not reach direct messages. Its Suppress @all and @here
switch makes those two mention kinds count for nothing, at every level: no push
and no mention badge. Unread counts and the mentions inbox
(GET /api/mentions) are independent of the level, so a muted channel still
shows what was addressed to the member.
Thread follow (1.1). A reply notifies only the thread's followers (push and
the in-page notification alike). The author of the root, everyone who replies
and everyone mentioned in a reply follow automatically, in the reply's own
transaction (thread_follows, migration 0054, backfilled for existing
threads). PUT/DELETE /api/threads/{id}/follow change it; an explicit
unfollow is kept and a later reply does not undo it, but a reply that mentions
the member by name or quotes their message still reaches them. A change, and
a reply that makes someone follow, is sent to that member's own sessions as
thread_follow_updated (a hub tap, thread_follow_sync.go);
the tap runs before the hub hands the reply to any connection (taps never
run under the hub's lock, and this one is the first tap, ahead of outgoing
webhooks and the bot gateway), so a new follower's sessions have the update
before the reply they decide it by; the lookup is bounded to a second, after
which the reply goes out without the update, and
GET /api/threads/following lists the followed threads for clients. The
in-page OS notification shows the message text only when the member's
"show message text" setting is on and the operator has not set
PUSH_PREVIEWS=off, like Web Push.
Mentions. A message stores one message_mentions row per notified member,
with a kind (by name, @all/@here, through a role) and the mentioned roles
in message_role_mentions (migration 0036), so a pill survives a role rename.
A member reached several ways gets the strongest kind: a name, then a role,
then @all/@here. A name only counts when the sender may name members (not
a webhook with allow_user_mentions off, nor a bot whose allowed_mentions
leaves users out and does not list the member; a bot, which @all never
reaches, is then not mentioned at all). A message carries every notified
member in mentions and those of a kind other than @all/@here in
direct_mentions: a muted channel's live badge, desktop notification and
sound read that list, as the badge after a reload and push read the kinds.
Webhooks and bots mention roles too, subject to allowed_mentions. An author
without the right to mention everyone notifies at most 50 members per message
(mentions_cap.go).
Search. GET /api/search runs trigram-indexed substring matching on
message text (pg_trgm, one index since migration 0038) with filters (channel,
author, mentions, attachment kind, date window; since 1.1 also has=poll,
has=edit and is=pinned). Results are limited to the
channels the caller can read, using the same permission snapshot as message
listing; a query has a length and term cap, a database statement timeout and
at most two searches in flight per member.
Onboarding and rules. The Owner (or Administrators) writes a welcome text,
rules and a start channel through /api/admin/onboarding (migration 0037).
Publishing rules creates a rules version; a member who has not accepted the
current one is pending. Enforcement is part of the permission resolver
(internal/perm): a pending member loses what
a timeout takes (posting, reacting, uploading, joining a Talk) and receives
RULES_NOT_ACCEPTED, in REST, the hub and the SFU alike. Empty rules enforce
nothing, which is why an upgrade never locks anyone out; the Owner and bots are
exempt. GET /api/invites/{code} (the one public read) serves the community
name for the sign-up page; it is rate limited and refuses weak legacy codes.
The API contract
From 1.0.0 the public surfaces are versioned with semantic versioning
(api-stability.md): the REST API as api/openapi.json, the
WebSocket event registry in internal/events,
bot and webhook payloads, the desktop bridge, environment variables and metric
names. The contract is enforced mechanically, not by convention:
make openapi-checkfails whenapi/openapi.jsonis stale andmake openapi-difffails on an incompatible change against the latest release tag, unless a commit carries aBREAKING CHANGE:footer.- Event names are constants in
internal/events; a golden file (internal/events/testdata/events.golden.json) and a comparison withweb/src/types/events.tsfail the build when the registry and the client types drift. - The alert rules in
deploy/monitoring/mnema-alerts.ymlare checked withpromtool, and an integration test fails when a rule names a metric/api/metricsdoes not export. GET /api/healthlists optional features by name (internal/server/health.go); a new feature adds a name and never bumpsweb_client_api.- Upgrade and restore drills install each supported release from GHCR, seed data through its own API, move to the build under test and check the data; a fresh-install check follows deployment.md on an empty directory.
The database schema, internal/... packages and the operator-only admin
endpoints are not part of the contract.
Capacity
Measured on a 4-core home NAS with the default APP_MEM_LIMIT=1g: one browser
sharing a 640x360 screen at 20 fps to 99 members in one Talk used about 28 %
of one core and about 500 MB of app memory, each receiver getting about
320 kbit/s (0.7.5). App memory grows with receivers x forwarded video sources,
because the server keeps a retransmission history of the video it sends each
receiver. Before this change that was Pion's responder: 1.5 KB buffers, a fixed count
sized when the connection joined (1024 packets for the first 16 members,
fewer later), so the earliest joiners kept the large history and a measured
Talk of 100 members watching a screen and ten cameras reached a live heap of
0.8 to 1.0 GB. Now internal/sfu/nack_responder.go keeps the history by bytes (buffer sizes
included) and looks at the load on every packet: a track remembers about a
second of its own bitrate (a quarter of a second beyond 64 members), within a
share of 4 MiB (512 KiB beyond 64 members) of the connection's budget, and
never less than 64 KB or, for a track with large frames such as a screen's
keyframe, a frame and a quarter (up to 768 KB), so a connection that joined
early shrinks as the server fills up. A hundred members watching three
1.5 Mbit/s cameras hold about 20 MB of history (mostly the 64 KB floor per
track) instead of about 460 MB. The ring that indexes a track's packets starts at 64 slots of 24 bytes
and grows with the track. Each connection also has a retransmission budget (nack_budget.go): a token
bucket that fills at a quarter of what the connection is sent, at least
128 KB/s, and holds half a second of that. A subscriber that keeps asking for
packets beyond it (a retransmission storm on an overloaded link) is refused the
excess, which mnema_sfu_retransmit_dropped_total counts; it recovers at the
next keyframe. Each connection answers its NACKs one at a time from a queue of
16 (the excess is counted too), and one NACK for at most 128 packets. The sizing guide gives
memory, CPU and upload for a given Talk. The price of the shorter history is that
a late retransmission request goes unanswered and the picture shows an artifact
until the next keyframe. Memory is returned after the call. With about 16 real browsers plus 84
lightweight members (100 total) the app used about 1.6 cores. Real E2EE browser
clients were verified with up to 24 viewers in one call; beyond that the single
test machine, not the server, was the limit. These are single runs, not a
qualified capacity guarantee.
Capacity table (server host)
The scenarios of loadtest/host-run.sh run on the
server host itself, without browsers (see "On the host" in
loadtest/README.md). The figures are from one run
on the reference host (a 4-core x86-64 home NAS with 16 GB, server 2.2.0, other
members online: one), so read them as orders of magnitude. CPU is the app
process in cores, averaged over the steady phase; memory is its peak resident
set. The generator runs on the same host, so its own CPU (0.5 to 1 core in the
voice scenarios) is part of what the host spends.
| Scenario | Load | Server CPU (cores) | Server memory (peak) | Result |
|---|---|---|---|---|
| Chat, 100 sessions | 10 writers x 0.2 msg/s, typing, reactions, edits | 0.02 | 47 MB | messages p50/p95/p99 12/18/26 ms, no loss |
| Chat, 300 sessions | 30 writers x 0.2 msg/s | 0.09 | 62 MB | messages p50/p95/p99 14/21/44 ms, no loss |
| Chat, 1000 sessions | 100 writers x 0.2 msg/s | 0.67 | 102 MB | messages p50/p95/p99 33/84/115 ms, no loss (2.36 million deliveries) |
| Voice, direct | 1 screen (1500 kbit/s) to 99 members | 0.56 | 312 MB | 1585 kbit/s per member, 0.00 % loss, no freezes, first video p95 374 ms |
| Voice, through the built-in TURN relay | 1 screen and 3 speakers to 100 members | 1.61 | 312 MB | all 100 members through the relay, 0.00 % loss, one member froze for 12.6 s once (one connection flap) |
| Voice, mixed | 1 screen, 10 cameras (300 kbit/s), 3 speakers, 100 members watching all | 1.87 | 1044 MB | does not hold: 82 % screen and 34 % camera packet loss, members received 200 kbit/s of the screen (audio loss 0.3 %) |
The method, caveats and per-scenario detail are in doc/benchmark.md; the sizing table (members, speakers, cameras, screens to CPU, memory and upload) is in its sizing guide.
What the host comfortably carries: chat for 1000 sessions, and one screen to about 100 members directly or through the relay. Watching all of 10 cameras plus a screen with 100 members (about 450 Mbit/s of media out of one 4-core host that also runs the generator) did not hold on 2.2.0 to 3.3: the SFU's one UDP socket let one send through at a time and capped the server at about 45,000 packets a second. Since the SFU sends from all goroutines at once (on Linux, which is where the container runs; other platforms keep the one-at-a-time path), that Talk holds with no loss at 1.91 cores and 406 MB (benchmark); the next limit is the host's CPU, at about 120 members.
The measurements above are reproducible with make load-test (see
loadtest/README.md): one sharer, real browsers and
lightweight Pion viewers, with join/reload stall checks, optional packet loss
and jitter, server heap and goroutine sampling, and thresholds that fail the
run. It can also run against a remote instance.
Unread tracking with 10,000 members
The integration tests in
internal/server/unread_scale_integration_test.go
(go test -tags=integration -run 'Unread|ReadMarkers|SendingAMessage|Mentioning' -v ./internal/server)
put 10,000 members into one community on PostgreSQL 18 and count what the
database really writes (row versions stamped by the test's own window, not
statistics). One run on an Apple M5 laptop, database in a local container,
single run, so read the figures as orders of magnitude:
| What | Result |
|---|---|
| Rows written by one message, 2 members vs 10,002 members | the same: 1 message row and 1 users row (the sender), 0 read markers for the members; the sender's own marker moves up to their message, which writes a row only if others posted since it (see below) |
Median POST /messages time, 2 vs 10,002 members | 3.2 ms vs 2.8 ms (no difference beyond noise) |
One message with @all to 10,001 members | 1 statement, 10,000 message_mentions rows, 185 ms (the only send that writes per recipient; needs the right to use @all) |
| All 10,000 members read the channel (32 concurrent workers) | 10,000 marker rows, 1.0 s in total (about 0.1 ms each) |
| The same 10,000 members read it again without news | 0 rows written, 0.28 s |
GET /api/read-state for one member, 600 unread in a 50,000-message channel | 45 ms (cold), index scan on idx_messages_channel_roots |
| Same, 50,000 unread | 29 ms (warm) |
| First-unread lookup in that channel | 0.04 ms, 6 buffers, index scan |
Sending a message reads the channel for its author: messages.MarkReadAfterSend moves the marker up to the new message (top-level messages only; thread replies are no channel unread), which writes the sender's marker only when someone else posted since the last one, and sends read_state to the author's other sessions. That holds for every way a member posts while looking at the channel: a typed message, an upload, a forward, a forum post, a built-in command such as /shrug and a poll. A scheduled message is the exception: its author is not reading the channel when the server sends it, so what others wrote since their last visit stays unread. The client does the same at once and drops the "new since" divider, so the "Jump to first unread" pill goes.
No migration was needed: the existing indexes serve every query (the plan is
printed by the test with -v). The remaining cost that grows with unread
volume is the count in GET /api/read-state, linear in the unread messages of
a member who has been away for a long time (about 30 to 45 ms for 50,000);
it is paid once per page load or reconnect, not per message.
Hot queries on a million messages
A throwaway PostgreSQL 18 container (4 CPUs, 1 GB, shared_buffers 128 MB as
in docker-compose.yml) was filled with 1,026,000 messages in 334 channels
(one with 309,000), 10,000 members, 410,000 reactions, 296,000 read markers,
51,000 mentions, 57,000 saved items, 20,000 forum posts with 80,000 replies,
31,000 attachments and 300,000 audit entries. Each hot path ran through its Go
function against it; the times are the median of five calls on an Apple
laptop, warm cache:
| Path | Before | After | What changed |
|---|---|---|---|
GET /api/read-state, member away from most channels | 780 ms | 109 ms | one index-only pass per channel (migration 0112), no JIT |
| Read state recount of the largest channel for 500 members online | 1,600 to 2,000 ms | 920 ms | same query |
| History page (50 messages) | 5.6 to 6.4 ms in the database | 0.3 to 0.4 ms | attachments of quoted messages looked up by id, not hashed |
| Forum post search | 345 ms | 6 ms | no JIT |
| Forum post list by activity | 66 to 83 ms | 8 ms | no JIT |
| Member list totals without two hidden channels | 43 to 68 ms | 31 ms | counted from the index alone |
The unread summary stays linear in the unread messages of a member (about 470,000 in the case above); a message that quotes another one costs one primary key lookup. Startup on that database takes about 40 ms from the first log line to listening; migration 0112 adds about 0.45 s once.
The permission snapshot with 10,000 members
Permission questions are answered from one in-memory snapshot
(internal/perm). Every permission change drops it and the next question
reads the whole state again: 13 queries in one read-only repeatable-read
transaction. Everyday actions are among those changes (accepting the rules, a
new conversation, a self-assigned role), so the cost of one load decides
whether that design holds. The integration test in
internal/server/perm_snapshot_scale_integration_test.go
(PERM_SNAPSHOT_SCALE=full go test -tags=integration -run TestAcceptingTheRulesTakesEffectAtOnce -v ./internal/server;
without the variable it runs a small community, as in CI) measures it with
10,000 members, 30 roles, 20,000 role assignments and 5,000 conversations
(10,000 conversation memberships), 9,500 of the members having accepted the
rules. Three runs on an Apple M5 laptop (10 cores), PostgreSQL 18 in a local
container, built without the race detector, while other work kept the machine
busy (load average 7 to 11), so read the figures as upper bounds of an order
of magnitude:
| What | Run 1 | Run 2 | Run 3 |
|---|---|---|---|
| One load, nothing else going on (median of 20; slowest) | 13.6 ms; 24.7 ms | 12.2 ms; 22.2 ms | 13.5 ms; 17.4 ms |
| Rows read by one load | 39,535 | 39,535 | 39,535 |
| 500 members accept the rules one after the other: loads | 500 | 500 | 500 |
| The same: total time | 8.4 s | 7.5 s | 7.8 s |
POST /api/onboarding/accept, p50 / p99 | 15 / 56 ms | 15 / 19 ms | 15 / 23 ms |
POST /api/channels/{id}/messages meanwhile (4 writers, about 85 messages/s), p50 / p99 | 20 / 80 ms | 19 / 34 ms | 20 / 35 ms |
| The same without permission changes, p50 / p99 | 14 / 274 ms | 12 / 25 ms | 14 / 25 ms |
Each acceptance costs exactly one full load, paid inside its own request, so 500 acceptances cause 500 full loads and one acceptance takes about 15 ms, of which about 13 ms are the load: the hub is told about the change and asks for the new snapshot at once, so a message sent meanwhile waits for at most the load in progress. Loads never run side by side, which bounds this design at about 65 permission changes per second at this size. That adds about 6 ms to the median send and about 10 ms to its p99 in the two runs without an outlier (the 274 ms in run 1 fell into the phase without any permission change, so it is the busy machine, not the snapshot). Both figures stay well below the limits set for this design, 50 ms for one load and 200 ms for the p99 of sending during such a burst; the test fails above them when it runs at full size. The snapshot therefore stays one consistent, fully reloaded view. A load grows linearly with role assignments, conversation memberships and members who accepted the rules (about 0.35 ms per 1,000 rows here), so the design is worth another look at several times this size, or when a load is seen above 50 ms.
Talks, sessions and operator status (1.2)
Lobbies, Talks and Stages. A lobby channel (channels.is_lobby, migration 0059) opens a temporary Talk
(is_temp, leader_id) for the member who joins it. talks.TalkManager creates it, removes an empty one
after DefaultTalkEmptyGrace (30 seconds) and removes every temporary Talk at start. A Stage
(channels.is_stage, migration 0068) is a Talk in which talk power decides who may send
media; the permission Manage stage (1 << 24) grants it, hands raised and reactions are relayed over
the hub and kept nowhere. See talks.md.
Sessions. user_sessions (migrations 0064 and 0065) describes each signed cookie by its jti: a coarse
device label, creation and last activity. The cookie stays the credential; a periodic sweep
(Handler.RunSessionSweep in internal/auth/authn) signs out a session that was inactive for longer than
instance_settings.session_inactivity_days (default 7), and the same condition is checked when the cookie is
used. See operations.md.
View as role. GET /api/admin/roles/{id}/view-as runs the normal permission resolver for a made-up member
that holds @everyone and the role, intersected with what the caller can view. It reads, it changes nothing.
Operator status. GET /api/admin/status (internal/server/admin_status.go, feature ops.status, Owner
and Administrators) gathers version, update state, database, storage, backup and live activity for the Status page.
Frontend structure (web/src)
- Folders: one folder per feature (
chat,talk,dm,forum,events,wiki,admin,settings), each withcomponents/,composables/andlib/.shared/holds what several features use,app/the shell that composes them (sidebar, user bar, member list, sign-in, routing).stores/,types/,i18n/anddirectives/serve all of them;sw/is the service worker (Web Push and the app shell) andcompat/holds the wire-compatibility tests.lib/media/is the media encryption of Talks and stays where it is. A feature importssharedand itself, not another feature; the allowed edges are the tables inweb/scripts/importZones.ts, checked by ESLint. - Entry:
main.ts,App.vue(routing viashared/lib/router.ts, layout, dialogs). - Addresses:
shared/lib/routePaths.tsbuilds every address (/c/:slug,/v/:slug,/c/:forum/p/:title-:shortId,/dm/:username,/dm/g/:shortId) andshared/lib/router.tsparses and resolves them client-side against the channels and conversations the member already sees: the current slug, thenprevious_slugs, then a legacy id; the last two are rewritten (replace) to the canonical address. The server has no slug-resolving endpoint. Slugs are assigned bychannels.assignSluginside the transaction that creates or renames a channel; former slugs stay reserved for the channel inchannel_slugs(up to 10 newest, older ones are freed; temporary Talks keep a fixedtalk-<8hex>slug that is never changed on rename and not kept in the history). - Stores (Pinia):
auth(current user; the session itself is an HttpOnly cookie),socket(the WebSocket transport: connect, heartbeat, reconnect backoff; it knows no event),chat(channels, messages, WebSocket events, unread state, presence; it hosts the transport and still exposes its connection state andsendWSEvent, so callers did not change),voice(call state, devices, input mode and push-to-talk key, per-person volumes, camera and stage choices),appVersion(reload banner on version mismatch),toast. - Composables:
useWebRTC(peer connection, publishing mic/screen/camera, push-to-talk keys, screen offer tuning, stats),useTalkStageanduseVideoGrid(stage and grid layout),usePictureInPicture,useSidebarReorder,useSortableDraganduseChannelLayout(drag and drop with immediate save and undo),useNavMenus(context menus),useMessageActions,useMessageKeyboard,useResizable,useDialog. - Components:
Sidebar.vue,ChatArea.vue,ThreadSidebar.vue,VoiceStage.vue(the Talk view),ParticipantTile.vue,VoiceChatPanel.vue,ScreenViewers.vue,PipHost.vue,MemberList.vue,AdminDashboard.vuewith one component per tab (users, layout, invites, media, system),UpdateBanner.vue. - Styles: Tailwind CSS 4 (
@import 'tailwindcss'instyle.css, the@tailwindcss/viteplugin; notailwind.config). The Mnema design tokens are in its@themeblock. The shared UI building blocks are plain CSS files instyles/:buttons.css(the.btnfamily and the focus ring),controls.css(checkbox, radio, switch, slider, fields, select),surfaces.css(dialogs, menus, popovers, tooltips, toasts, list rows, elevation),settings.css(the cards and groups of the account and audio settings pages),motion.css(motion durations, scales and the patterns built on them),feedback.css(message and sidebar flashes, drag feedback, scrollbars and tooltips) andcalls.css(the incoming call card and the caller's strip of direct-message calls). They use tokens only and no inline style, because the Content-Security-Policy forbids it. - 1.1 features in the UI: direct messages
(
DmCreateDialog.vue,DmInfoDialog.vue), blocking and reports (BlockedMessageRow.vue,ReportDialog.vue), saved messages and reminders in one header panel (SavedLaterPanel.vue), forwarding (ForwardDialog.vue,ForwardedQuote.vue), pinned messages with "Pinned by" (PinnedMessages.vue) and the slash bar (SlashCommandBar.vue). Each shows only whenGET /api/healthlists its feature. - i18n:
i18n/de.jsonandi18n/en.json; tests fail on hard-coded UI text and unused keys.
Key design decisions
- One binary, three containers. API, hub, SFU and web app ship together
(
web/web.go);docker composeadds only PostgreSQL and SeaweedFS (optional profiles addcoturnand the update sidecar). Pure Go (CGO_ENABLED=0), so images cross-compile for amd64 and arm64 without QEMU (Dockerfile). - Forward, never transcode. The SFU only copies RTP
(
internal/sfu); quality is decided by the sender. Transport is DTLS-SRTP. Microphone, camera, screen video and screen-audio payloads are encrypted in the browser with MLS-derived keys, so the SFU cannot read them; an active or compromised server can still subvert group membership and serves the code, and metadata stays visible. Text and attachments stay readable by the server (SECURITY.md). - Media key lifecycle. A new epoch (and so a new key) comes from every
join, leave or kick, and otherwise every 30 minutes (+-2 min jitter) from a
path commit by the committer. A member with no heartbeat or handshake for
20 minutes is removed and must be let in again. Constants in
bounds.ts, logic ingroup.ts, heartbeat format inheartbeat.ts. The e2e build (MNEMA_E2E=1) lets tests shorten them; release builds cannot. - Frame format and line binding. Each sealed unit is
M3 | epoch | stream | counter | line | nonce | AES-GCM | Ed25519 signature; thelinebyte (source in the high nibble, screen layer in the low one) is in the AAD and the signed bytes, and each receiving transform is armed with the source the SFU names for its line, so a unit moved to another line does not open (frame.ts,mediaFrameWorker.ts). Version 2 (M2, no line) was removed in 1.2.0: it is neither sent nor opened, and a unit that carries it is counted and reported to the member as coming from an outdated client. Members still sign a capability message (capability.ts,MC, next to the heartbeat) so that older clients, which choose their format by it, switch to version 3 (SECURITY.md). - Screen shares are opt-in per viewer so an idle viewer costs no
bandwidth (
subscriptions.go). - Invite-only, one Owner. No public registration; the Owner is seeded
from
ADMIN_USERNAME(admin_seed.go) and holds every permission; roles below them are managed throughinternal/permand enforced server-side (roles-and-permissions.md). - Sessions revocable by version. Password change or "log out everywhere"
bumps
token_version, invalidating every cookie and closing the user's sockets (session.go,token.go). - Forward-only, checksummed migrations run at startup
(
db.go); never edit an applied one. - A strict page policy. The web app is served with
script-src 'self' 'wasm-unsafe-eval',style-src 'self'andstyle-src-attr 'none'(SPAContentSecurityPolicy): no inline script or style, so the Vue code sets nostyleattributes and uses classes and custom properties set from script. SECURITY.md has the full header list. - Retention off by default.
MEDIA_RETENTION_DAYS=0deletes nothing (admin.go). - The app never touches Docker. Self-update goes through an opt-in
sidecar on an internal network with a token, admin password re-auth, a
5-minute cooldown and an audit log entry; the sidecar verifies the new
image's cosign signature and provenance before it pulls anything
(
internal/imageverify,internal/updaterd,system_selfupdate.go,docker-compose.yml, SECURITY.md). - Reload, never auto-reload. The server sends its version on every
WebSocket connection; a page built for another version shows a reload
banner (
appVersion.ts).