REST:Â Realtime
WebSocket and WebRTC configuration. Generated from mnema-talk v3.3.13 (cca4f0fdb723).
ICE servers for voice
GET /api/webrtc/config
STUN servers and, when configured, TURN relays with fresh time-limited credentials: the external one named by WEBRTC_TURN_URLS and the one built into the server while it runs (members who may join voice only). The list is empty when none is configured.
Authentication: Session cookie
Responses
| Status | Meaning | Body |
|---|---|---|
| 200 | ICE configuration. | ICEConfig |
| 401 | No valid session (UNAUTHORIZED): missing, expired or revoked cookie, or the account was disabled. | ErrorResponse |
| 429 | RATE_LIMITED: too many requests. | ErrorResponse |
| 500 | INTERNAL_ERROR: sanitized server failure. | ErrorResponse |
ICEConfig
| Field | Type | Description |
|---|---|---|
ice_servers (required) | IceServer[] |
Answer a connection test
POST /api/webrtc/probe
Backs "Test my connection": answers a browser's data-channel-only offer with a throwaway peer connection that uses the same media sockets and announced addresses as a Talk, so the browser can tell whether its media reaches the server directly or only through the relay. No audio, no media permission, no room and no presence; the connection closes itself after a few seconds. Signed-in members only; at most twelve per ten minutes and eight at a time. Candidates of the offer are vetted like those of a Talk member.
Authentication: Session cookie
Request body (application/json, required): ConnectionProbeRequest
| Field | Type | Description |
|---|---|---|
sdp (required) | string | SDP is the browser's offer: one data channel and no audio or video, with all of its candidates gathered. |
Responses
| Status | Meaning | Body |
|---|---|---|
| 200 | The server's answer. | ConnectionProbeAnswer |
| 400 | Invalid input (INVALID_INPUT): not a data-channel-only offer. | ErrorResponse |
| 401 | No valid session (UNAUTHORIZED): missing, expired or revoked cookie, or the account was disabled. | ErrorResponse |
| 429 | RATE_LIMITED: too many requests, or too many tests running. | ErrorResponse |
| 500 | INTERNAL_ERROR: sanitized server failure. | ErrorResponse |
| 503 | UNAVAILABLE: the voice engine is not running. | ErrorResponse |
ConnectionProbeAnswer
| Field | Type | Description |
|---|---|---|
sdp (required) | string | SDP is the server's answer with all of its candidates. |
Download speed test
GET /api/webrtc/speedtest
Backs the upload hint of "Test my connection": streams up to 16 MiB of noise or for at most three seconds, whichever ends first, so the browser can measure its download speed from the server. The speed is the one to the nearest reverse proxy or tunnel, not necessarily to the server itself. Signed-in members only; at most four per ten minutes per member and six per address, one at a time. A busy slot or an exhausted budget answers 429 and costs nothing; while a Talk has participants it answers 503 so the test does not disturb the call.
Authentication: Session cookie
Responses
| Status | Meaning | Body |
|---|---|---|
| 200 | A stream of bytes. | file |
| 401 | No valid session (UNAUTHORIZED): missing, expired or revoked cookie, or the account was disabled. | ErrorResponse |
| 429 | RATE_LIMITED: too many tests, or one is running. | ErrorResponse |
| 500 | INTERNAL_ERROR: sanitized server failure. | ErrorResponse |
| 503 | UNAVAILABLE: a Talk has participants; the test would disturb the call. | ErrorResponse |
WebSocket event stream
GET /api/ws
Upgrades to a WebSocket (requires a same-origin Origin header; a foreign origin is refused with 403 by the upgrade handler). OpenAPI cannot model WebSocket frames, so the protocol is described here.
Every frame in both directions is a JSON text message {"type": string, "payload": any}. At most a few events per second per connection (and per account across its connections) are processed; excess client events, and events whose payload is over the cap for their type (1 KiB, more only for an SDP answer, ICE candidate, diagnostic and media handshake), are dropped.
Server to client, on connect (to this connection only):
server_info:{version, screen_layers?, camera_layers?}, sent first: the server's release version (a browser offers a reload when it differs from its own) and, on a server with media, how many layers of a screen share and of a camera it takes.presence_snapshot: map of user id to live status for every online user.voice_snapshot: map of voice channel id to a map of user id to User plusjoined_at.voice_rooms:{started: {channelId: time}, now}; the server clock lets clients count up.
Server to client, broadcast to all connections:
Channel events (message_*, typing, voice_state_update, webrtc_media_state) only reach connections whose user may view that channel; the connect snapshots list only the Talks the user may view.
message_create,message_update: Message.message_delete:{id, channel_id, parent_id}.message_reaction:{message_id, channel_id, reactions}.channels_changed: no payload; refetch GET /api/channels (sent after any category/channel/layout admin change).permissions_changed: no payload; refetch GET /api/permissions and GET /api/channels. Sent to the users whose roles or permissions may have changed (everybody after a channel, category, overwrite or @everyone change).roles_changed: no payload; refetch GET /api/roles and GET /api/members (a role was created, edited, deleted, reordered or assigned).emoji_changed: no payload; refetch GET /api/emoji (a custom emoji was added, renamed or deleted).community_updated:{community_name, community_icon}after the community name or icon changed (community_iconis empty after removal).reports_open:{open}, only to sessions holding Manage reports: the number of open reports, sent whenever a report is filed, closed or raised by Automod.member_joined: public User of a newly registered member.user_update: public User after a profile, status or avatar change, or{id, disabled}when an admin disables or enables an account.user_stats:{user_id, voice_seconds}after a voice stay ends; voice_seconds is the public talk time (stays in Talks @everyone can see; restarts at 0 in 3.0.0).presence_update:{user_id, status}where status is online, away, dnd, focus or offline; for a bot that set an activity alsoactivity: {type, name}(activity type).voice_state_update:{action: "join", channel_id, user, started_at}or{action: "leave", channel_id, user_id}.typing:{channel_id, user_id}(to everyone except the typist).
Server to client, targeted:
block_updated: to the user's own sessions only;{user_id, blocked}after they blocked or unblocked a member (refetch GET /api/me/blocks). The blocked member is never told.dm_created: to every member's sessions; the direct message (DirectMessage) a conversation was started with, or the one the receiver was just added to.dm_member_added: to the other members of a group conversation;{dm_id, member}.dm_member_removed: to the remaining members and the leaver's sessions;{dm_id, member}.dm_prefs_updated: to the user's own sessions only;{dm_id}after they closed, reopened or (un)favourited a conversation (refetch GET /api/dms/{id}), or{dm_id: null}after their direct message settings changed (refetch GET /api/dms).dm_call_ring: to the members' sessions that asked for conversations (?dm=1), the caller's own included;{conversation_id, talk_id, message_id, caller_id, talk_name, ring_seconds}after a Talk was started from, or shared into, a conversation. Everyone but the caller has an incoming call forring_seconds(counted from receipt), unless adm_call_endcomes first. Additive: a client that ignores it still gets the invitation card asmessage_create.dm_call_end: to the same sessions;{conversation_id, talk_id, reason, active, user_id?, message_id?}when the call stopped ringing.reasonisanswered(a member other than the caller entered the Talk;user_idsays who, so their other devices and the other members drop the incoming call),cancelled(the caller left before anybody joined, or started another Talk in the conversation),timeout(nobody joined in time) orended(the last member left).activeis true while somebody is still in the Talk. Aftercancelledandtimeoutthe server has posted a "missed call" message (an embed withcall: "missed", written by the caller;message_id), announced bymessage_createjust before.talk_leader:{channel_id, leader_id}to everyone who may view a temporary Talk (opened from a Create-a-Talk lobby) when its leader changes;leader_idis null when nobody leads it.talk_moved:{from_channel_id, channel_id}to the connection whosevoice_joinof a Create-a-Talk lobby opened the temporary Talkchannel_id; the client then sends its ordinaryvoice_joinfor that Talk. The connection is never seated in the lobby itself.read_state: to the user's own sessions only;{channel_id, last_read_at, unread_count, mention_count},{channel_id, last_read_at, refresh: true}, or{refresh: true}alone after a notification setting changed or everything was marked read (reload read states and notification settings).webrtc_media_state:{channel_id, user_id, screen, camera}to voice-room members.voice_speaking:{channel_id, user_id, active}to voice-room members, only when the state changes.voice_mute_state:{channel_id, user_id, muted, deafened}to the connections whose user may view that Talk; deafened implies muted, and a muted member is never shown as speaking.stage_hand:{channel_id, user_id, raised, at?}to voice-room members of a Stage when a hand goes up (atis when) or down.stage_talk_power:{channel_id, user_id, granted}to voice-room members of a Stage;grantedis the effective state, so a member holding Manage stage shows as a speaker.talk_reaction:{channel_id, user_id, emoji}to voice-room members, the sender included; a live reaction is stored nowhere.screen_viewers:{channel_id, user_id, viewers}to voice-room members:viewersare the distinct users in the room watchinguser_id's screen share (sorted ids, never the sharer). Sent whenever that set changes (viewers: []once when a watched share ends or loses its last viewer), and onvoice_joinone per live share to the joining connection only.webrtc_offer(SDP offer),webrtc_candidate(ICE candidate): SFU signalling.screen_layers:{channel_id, user_id, layers}to voice-room members: the layers ofuser_id's live screen share, best first, each{width, height, fps}as the sharer declared them. Sent when they change, as an empty list once the share ends, and onvoice_joinone per live share to the joining connection only. A share of a client without layers has none.screen_quality:{channel_id, user_id, layer, auto}to the watching connection only: the layer ofuser_id's screen share it receives now (-1before the first picture) and whether the server chooses it (auto).cameras_capped:{channel_id, user_ids}to the viewing connection only: whose cameras the server does not forward because the viewer asked for more than 24 at once (the lowest user ids are forwarded). Sent when that set changes, as an empty list once nothing is capped; the client shows those tiles without a picture.screen_layers_wanted:{channel_id, layers, source?}to the sharing connection only: the indexes of its layers some viewer receives or is about to (source: "camera"for the layers of its camera, absent for its screen share). A sender may stop encoding the others; layer 0 is always listed while nobody watches.media_handshake:{channel_id, user_id, wire}to the other connections in that voice room.wireis opaque: the server checks only its framing (an MLS key package, welcome or public message, or a heartbeat or capability message) and never reads or logs the content.voice_kicked:{channel_id, reason?}when a moderator removes the user from voice,reason: "room_full"when a voice join is refused because the room is full (the channel'suser_limit; members with Disconnect members in that Talk, Administrators and the Owner may exceed it),reason: "rules_not_accepted"when the community's rules have not been accepted yet (POST /api/onboarding/accept),reason: "talk_rate_limited"or"talk_unavailable"when joining a Create-a-Talk lobby opened no Talk (5 new Talks per 10 minutes and member, 50 open Talks per lobby), orreason: "connection_lost"when the media connection of the call failed and the server dropped it (join again),reason: "media_unavailable"when the media connection of a join could not be set up (the seat is given up; join again),reason: "joined_elsewhere"to a connection whose Talk the same account joined on another connection (one connection per account and Talk holds the media),reason: "voice_taken_over"to a connection whose voice (in any Talk) the same account moved to another connection withvoice_joinandtakeover: true; only its voice ends, the WebSocket stays open, orreason: "missing_permission"when the user may not (or no longer may) view and connect to that Talk.pong:{t}echoing aping.
Client to server:
ping{t};presence_idle{idle: bool}.typing{channel_id}(text and voice channels, rate-limited; ignored without View channels and Send messages).voice_join{channel_id, takeover?, resume?, tab?}(needs View channels and Connect, otherwisevoice_kickedwithreason: "missing_permission"). Withouttakeover, the account's other connection in the same Talk is toldjoined_elsewhereand a connection in another Talk keeps its seat. Withtakeover: true(the client sends it after the user confirmed "take over here") the join is completed first and only then does every other connection of the account leave voice, each toldvoice_taken_over; of two takeovers at once the later seat wins. A client that reconnected sendsresume: trueto continue its call, andtab, an id its page keeps across reconnects (up to 64 characters): if a takeover ended that page's call and another connection of the account is in voice, it is answered withvoice_kickedvoice_taken_overinstead of taking the voice back;voice_leave{channel_id?, tab?}(ends the connection's seat, or when it holds none, ends the seat of another connection of the account seated in that Talk with that tab id or its lingering seat during the grace period, unless another connection of the account is seated there);voice_speaking{active}. The SFU drops audio without Speak and camera/screen without Video.webrtc_answer(SDP answer),webrtc_candidate(ICE candidate),webrtc_request_keyframe(optionally{source: "screen"|"camera"}: a keyframe of the sender's own video of that source, e.g. after a camera change, so its viewers get a fresh picture; without it, of the videos the sender watches).media_handshake{channel_id, wire}: opaque handshake bytes for the current voice room.webrtc_subscribe{kind: "screen"|"camera", user_id, on},{kind: "camera", all: true, on}or{kind: "camera", user_ids: [up to 16 ids], on}(several cameras with one renegotiation; a client that shows only some cameras asks for just those). A screen subscription may carryquality:"auto"or the index of a layer (0 is the best); it is applied before the first picture and again whenever sent. Without one, or with"auto", the server chooses: it starts on the best layer, moves down one layer at a time when the viewer's receiver reports show lasting loss, a rising round trip time or repeated keyframe requests, and tries one layer up again after a long stretch without trouble (a failed attempt makes the next one wait twice as long). A camera subscription may carryfit_px(0 to 16384): how tall in device pixels the viewer's picture of that camera is; the server then gives it the smallest layer of a camera sent as layers that still has the pixels, and the message changes nothing else (it is sent only to a server whoseserver_infohascamera_layers).webrtc_layers{layers: [{width, height, fps}, ...]}(1 to 3, best first, strictly shrinking): the layers of the sender's own screen share, or of its camera withsource: "camera"(declared before its first video; a camera without layers is one plain track). They are sent on the SFU's lower video lines.webrtc_screenshare_start(asks for a fresh keyframe of the sender's own screen share),webrtc_screenshare_stop,webrtc_camera_stop.webrtc_diag: the browser's own view of its media connection (up to 4 KiB); the server only logs it, at most 5 per 10 seconds per connection.voice_mute_state{muted, deafened}: the member's microphone and headphone marks in the Talk they are in.stage_hand{raised}raises or lowers the member's own hand in a Stage;{user_id, raised: false}dismisses another member's hand (needs Manage stage).stage_talk_power{user_id, granted}gives or takes talk power (needs Manage stage; the target is in the Stage). At most 30 of these per 10 seconds per member; refused commands are dropped without an answer.talk_reaction{emoji}: a live reaction to the Talk the member is in (needs Add reactions; at most 5 per 10 seconds per member over all connections; an invalid or unknown emoji is dropped).
Unknown event types are ignored. The server closes the connection when the session is revoked (sign-out of that session, sign-out everywhere, password change) or expires, or the account is disabled. A user keeps at most 8 browser connections: a newer one closes the oldest with close code 4001 ("too many connections"), after which that client should not reconnect by itself. Close code 4002 ("media connection renewed") means the server ended the media connection only to renew it (it had carried as many tracks as one connection may): the client reconnects at once and resumes its call with voice_join and resume: true; within the grace period nobody sees it leave. These codes are the member socket's; the bot gateway's 4001 and 4002 (unknown opcode, undecodable frame, doc/bots.md) mean something else. Upgrades are rate-limited per address and per user (429), and voice_join beyond about one per second (burst 3) is answered with voice_kicked.
Authentication: Session cookie
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
Upgrade | header | string | yes | Must be websocket. |
Responses
| Status | Meaning | Body |
|---|---|---|
| 101 | Switching Protocols; the connection is now a WebSocket. | |
| 401 | No valid session (UNAUTHORIZED): missing, expired or revoked cookie, or the account was disabled. | ErrorResponse |
| 403 | FORBIDDEN: cross-origin request rejected by the CSRF check. | ErrorResponse |
| 429 | RATE_LIMITED: too many requests. | ErrorResponse |
| 500 | INTERNAL_ERROR: sanitized server failure. | ErrorResponse |
| 503 | UNAVAILABLE: server shutting down. | ErrorResponse |