REST: Realtime

WebSocket and WebRTC configuration. Generated from mnema-talk v3.3.13 (cca4f0fdb723).

ICE servers for voice

GET /api/webrtc/config

STUN servers and, when configured, TURN relays with fresh time-limited credentials: the external one named by WEBRTC_TURN_URLS and the one built into the server while it runs (members who may join voice only). The list is empty when none is configured.

Authentication: Session cookie

Responses

StatusMeaningBody
200ICE configuration.ICEConfig
401No valid session (UNAUTHORIZED): missing, expired or revoked cookie, or the account was disabled.ErrorResponse
429RATE_LIMITED: too many requests.ErrorResponse
500INTERNAL_ERROR: sanitized server failure.ErrorResponse
ICEConfig
FieldTypeDescription
ice_servers (required)IceServer[]

Answer a connection test

POST /api/webrtc/probe

Backs "Test my connection": answers a browser's data-channel-only offer with a throwaway peer connection that uses the same media sockets and announced addresses as a Talk, so the browser can tell whether its media reaches the server directly or only through the relay. No audio, no media permission, no room and no presence; the connection closes itself after a few seconds. Signed-in members only; at most twelve per ten minutes and eight at a time. Candidates of the offer are vetted like those of a Talk member.

Authentication: Session cookie

Request body (application/json, required): ConnectionProbeRequest

FieldTypeDescription
sdp (required)stringSDP is the browser's offer: one data channel and no audio or video, with all of its candidates gathered.

Responses

StatusMeaningBody
200The server's answer.ConnectionProbeAnswer
400Invalid input (INVALID_INPUT): not a data-channel-only offer.ErrorResponse
401No valid session (UNAUTHORIZED): missing, expired or revoked cookie, or the account was disabled.ErrorResponse
429RATE_LIMITED: too many requests, or too many tests running.ErrorResponse
500INTERNAL_ERROR: sanitized server failure.ErrorResponse
503UNAVAILABLE: the voice engine is not running.ErrorResponse
ConnectionProbeAnswer
FieldTypeDescription
sdp (required)stringSDP is the server's answer with all of its candidates.

Download speed test

GET /api/webrtc/speedtest

Backs the upload hint of "Test my connection": streams up to 16 MiB of noise or for at most three seconds, whichever ends first, so the browser can measure its download speed from the server. The speed is the one to the nearest reverse proxy or tunnel, not necessarily to the server itself. Signed-in members only; at most four per ten minutes per member and six per address, one at a time. A busy slot or an exhausted budget answers 429 and costs nothing; while a Talk has participants it answers 503 so the test does not disturb the call.

Authentication: Session cookie

Responses

StatusMeaningBody
200A stream of bytes.file
401No valid session (UNAUTHORIZED): missing, expired or revoked cookie, or the account was disabled.ErrorResponse
429RATE_LIMITED: too many tests, or one is running.ErrorResponse
500INTERNAL_ERROR: sanitized server failure.ErrorResponse
503UNAVAILABLE: a Talk has participants; the test would disturb the call.ErrorResponse

WebSocket event stream

GET /api/ws

Upgrades to a WebSocket (requires a same-origin Origin header; a foreign origin is refused with 403 by the upgrade handler). OpenAPI cannot model WebSocket frames, so the protocol is described here.

Every frame in both directions is a JSON text message {"type": string, "payload": any}. At most a few events per second per connection (and per account across its connections) are processed; excess client events, and events whose payload is over the cap for their type (1 KiB, more only for an SDP answer, ICE candidate, diagnostic and media handshake), are dropped.

Server to client, on connect (to this connection only):

  • server_info: {version, screen_layers?, camera_layers?}, sent first: the server's release version (a browser offers a reload when it differs from its own) and, on a server with media, how many layers of a screen share and of a camera it takes.
  • presence_snapshot: map of user id to live status for every online user.
  • voice_snapshot: map of voice channel id to a map of user id to User plus joined_at.
  • voice_rooms: {started: {channelId: time}, now}; the server clock lets clients count up.

Server to client, broadcast to all connections: Channel events (message_*, typing, voice_state_update, webrtc_media_state) only reach connections whose user may view that channel; the connect snapshots list only the Talks the user may view.

  • message_create, message_update: Message.
  • message_delete: {id, channel_id, parent_id}.
  • message_reaction: {message_id, channel_id, reactions}.
  • channels_changed: no payload; refetch GET /api/channels (sent after any category/channel/layout admin change).
  • permissions_changed: no payload; refetch GET /api/permissions and GET /api/channels. Sent to the users whose roles or permissions may have changed (everybody after a channel, category, overwrite or @everyone change).
  • roles_changed: no payload; refetch GET /api/roles and GET /api/members (a role was created, edited, deleted, reordered or assigned).
  • emoji_changed: no payload; refetch GET /api/emoji (a custom emoji was added, renamed or deleted).
  • community_updated: {community_name, community_icon} after the community name or icon changed (community_icon is empty after removal).
  • reports_open: {open}, only to sessions holding Manage reports: the number of open reports, sent whenever a report is filed, closed or raised by Automod.
  • member_joined: public User of a newly registered member.
  • user_update: public User after a profile, status or avatar change, or {id, disabled} when an admin disables or enables an account.
  • user_stats: {user_id, voice_seconds} after a voice stay ends; voice_seconds is the public talk time (stays in Talks @everyone can see; restarts at 0 in 3.0.0).
  • presence_update: {user_id, status} where status is online, away, dnd, focus or offline; for a bot that set an activity also activity: {type, name} (activity type).
  • voice_state_update: {action: "join", channel_id, user, started_at} or {action: "leave", channel_id, user_id}.
  • typing: {channel_id, user_id} (to everyone except the typist).

Server to client, targeted:

  • block_updated: to the user's own sessions only; {user_id, blocked} after they blocked or unblocked a member (refetch GET /api/me/blocks). The blocked member is never told.
  • dm_created: to every member's sessions; the direct message (DirectMessage) a conversation was started with, or the one the receiver was just added to.
  • dm_member_added: to the other members of a group conversation; {dm_id, member}.
  • dm_member_removed: to the remaining members and the leaver's sessions; {dm_id, member}.
  • dm_prefs_updated: to the user's own sessions only; {dm_id} after they closed, reopened or (un)favourited a conversation (refetch GET /api/dms/{id}), or {dm_id: null} after their direct message settings changed (refetch GET /api/dms).
  • dm_call_ring: to the members' sessions that asked for conversations (?dm=1), the caller's own included; {conversation_id, talk_id, message_id, caller_id, talk_name, ring_seconds} after a Talk was started from, or shared into, a conversation. Everyone but the caller has an incoming call for ring_seconds (counted from receipt), unless a dm_call_end comes first. Additive: a client that ignores it still gets the invitation card as message_create.
  • dm_call_end: to the same sessions; {conversation_id, talk_id, reason, active, user_id?, message_id?} when the call stopped ringing. reason is answered (a member other than the caller entered the Talk; user_id says who, so their other devices and the other members drop the incoming call), cancelled (the caller left before anybody joined, or started another Talk in the conversation), timeout (nobody joined in time) or ended (the last member left). active is true while somebody is still in the Talk. After cancelled and timeout the server has posted a "missed call" message (an embed with call: "missed", written by the caller; message_id), announced by message_create just before.
  • talk_leader: {channel_id, leader_id} to everyone who may view a temporary Talk (opened from a Create-a-Talk lobby) when its leader changes; leader_id is null when nobody leads it.
  • talk_moved: {from_channel_id, channel_id} to the connection whose voice_join of a Create-a-Talk lobby opened the temporary Talk channel_id; the client then sends its ordinary voice_join for that Talk. The connection is never seated in the lobby itself.
  • read_state: to the user's own sessions only; {channel_id, last_read_at, unread_count, mention_count}, {channel_id, last_read_at, refresh: true}, or {refresh: true} alone after a notification setting changed or everything was marked read (reload read states and notification settings).
  • webrtc_media_state: {channel_id, user_id, screen, camera} to voice-room members.
  • voice_speaking: {channel_id, user_id, active} to voice-room members, only when the state changes.
  • voice_mute_state: {channel_id, user_id, muted, deafened} to the connections whose user may view that Talk; deafened implies muted, and a muted member is never shown as speaking.
  • stage_hand: {channel_id, user_id, raised, at?} to voice-room members of a Stage when a hand goes up (at is when) or down.
  • stage_talk_power: {channel_id, user_id, granted} to voice-room members of a Stage; granted is the effective state, so a member holding Manage stage shows as a speaker.
  • talk_reaction: {channel_id, user_id, emoji} to voice-room members, the sender included; a live reaction is stored nowhere.
  • screen_viewers: {channel_id, user_id, viewers} to voice-room members: viewers are the distinct users in the room watching user_id's screen share (sorted ids, never the sharer). Sent whenever that set changes (viewers: [] once when a watched share ends or loses its last viewer), and on voice_join one per live share to the joining connection only.
  • webrtc_offer (SDP offer), webrtc_candidate (ICE candidate): SFU signalling.
  • screen_layers: {channel_id, user_id, layers} to voice-room members: the layers of user_id's live screen share, best first, each {width, height, fps} as the sharer declared them. Sent when they change, as an empty list once the share ends, and on voice_join one per live share to the joining connection only. A share of a client without layers has none.
  • screen_quality: {channel_id, user_id, layer, auto} to the watching connection only: the layer of user_id's screen share it receives now (-1 before the first picture) and whether the server chooses it (auto).
  • cameras_capped: {channel_id, user_ids} to the viewing connection only: whose cameras the server does not forward because the viewer asked for more than 24 at once (the lowest user ids are forwarded). Sent when that set changes, as an empty list once nothing is capped; the client shows those tiles without a picture.
  • screen_layers_wanted: {channel_id, layers, source?} to the sharing connection only: the indexes of its layers some viewer receives or is about to (source: "camera" for the layers of its camera, absent for its screen share). A sender may stop encoding the others; layer 0 is always listed while nobody watches.
  • media_handshake: {channel_id, user_id, wire} to the other connections in that voice room. wire is opaque: the server checks only its framing (an MLS key package, welcome or public message, or a heartbeat or capability message) and never reads or logs the content.
  • voice_kicked: {channel_id, reason?} when a moderator removes the user from voice, reason: "room_full" when a voice join is refused because the room is full (the channel's user_limit; members with Disconnect members in that Talk, Administrators and the Owner may exceed it), reason: "rules_not_accepted" when the community's rules have not been accepted yet (POST /api/onboarding/accept), reason: "talk_rate_limited" or "talk_unavailable" when joining a Create-a-Talk lobby opened no Talk (5 new Talks per 10 minutes and member, 50 open Talks per lobby), or reason: "connection_lost" when the media connection of the call failed and the server dropped it (join again), reason: "media_unavailable" when the media connection of a join could not be set up (the seat is given up; join again), reason: "joined_elsewhere" to a connection whose Talk the same account joined on another connection (one connection per account and Talk holds the media), reason: "voice_taken_over" to a connection whose voice (in any Talk) the same account moved to another connection with voice_join and takeover: true; only its voice ends, the WebSocket stays open, or reason: "missing_permission" when the user may not (or no longer may) view and connect to that Talk.
  • pong: {t} echoing a ping.

Client to server:

  • ping {t}; presence_idle {idle: bool}.
  • typing {channel_id} (text and voice channels, rate-limited; ignored without View channels and Send messages).
  • voice_join {channel_id, takeover?, resume?, tab?} (needs View channels and Connect, otherwise voice_kicked with reason: "missing_permission"). Without takeover, the account's other connection in the same Talk is told joined_elsewhere and a connection in another Talk keeps its seat. With takeover: true (the client sends it after the user confirmed "take over here") the join is completed first and only then does every other connection of the account leave voice, each told voice_taken_over; of two takeovers at once the later seat wins. A client that reconnected sends resume: true to continue its call, and tab, an id its page keeps across reconnects (up to 64 characters): if a takeover ended that page's call and another connection of the account is in voice, it is answered with voice_kicked voice_taken_over instead of taking the voice back; voice_leave {channel_id?, tab?} (ends the connection's seat, or when it holds none, ends the seat of another connection of the account seated in that Talk with that tab id or its lingering seat during the grace period, unless another connection of the account is seated there); voice_speaking {active}. The SFU drops audio without Speak and camera/screen without Video.
  • webrtc_answer (SDP answer), webrtc_candidate (ICE candidate), webrtc_request_keyframe (optionally {source: "screen"|"camera"}: a keyframe of the sender's own video of that source, e.g. after a camera change, so its viewers get a fresh picture; without it, of the videos the sender watches).
  • media_handshake {channel_id, wire}: opaque handshake bytes for the current voice room.
  • webrtc_subscribe {kind: "screen"|"camera", user_id, on}, {kind: "camera", all: true, on} or {kind: "camera", user_ids: [up to 16 ids], on} (several cameras with one renegotiation; a client that shows only some cameras asks for just those). A screen subscription may carry quality: "auto" or the index of a layer (0 is the best); it is applied before the first picture and again whenever sent. Without one, or with "auto", the server chooses: it starts on the best layer, moves down one layer at a time when the viewer's receiver reports show lasting loss, a rising round trip time or repeated keyframe requests, and tries one layer up again after a long stretch without trouble (a failed attempt makes the next one wait twice as long). A camera subscription may carry fit_px (0 to 16384): how tall in device pixels the viewer's picture of that camera is; the server then gives it the smallest layer of a camera sent as layers that still has the pixels, and the message changes nothing else (it is sent only to a server whose server_info has camera_layers).
  • webrtc_layers {layers: [{width, height, fps}, ...]} (1 to 3, best first, strictly shrinking): the layers of the sender's own screen share, or of its camera with source: "camera" (declared before its first video; a camera without layers is one plain track). They are sent on the SFU's lower video lines.
  • webrtc_screenshare_start (asks for a fresh keyframe of the sender's own screen share), webrtc_screenshare_stop, webrtc_camera_stop.
  • webrtc_diag: the browser's own view of its media connection (up to 4 KiB); the server only logs it, at most 5 per 10 seconds per connection.
  • voice_mute_state {muted, deafened}: the member's microphone and headphone marks in the Talk they are in.
  • stage_hand {raised} raises or lowers the member's own hand in a Stage; {user_id, raised: false} dismisses another member's hand (needs Manage stage). stage_talk_power {user_id, granted} gives or takes talk power (needs Manage stage; the target is in the Stage). At most 30 of these per 10 seconds per member; refused commands are dropped without an answer.
  • talk_reaction {emoji}: a live reaction to the Talk the member is in (needs Add reactions; at most 5 per 10 seconds per member over all connections; an invalid or unknown emoji is dropped).

Unknown event types are ignored. The server closes the connection when the session is revoked (sign-out of that session, sign-out everywhere, password change) or expires, or the account is disabled. A user keeps at most 8 browser connections: a newer one closes the oldest with close code 4001 ("too many connections"), after which that client should not reconnect by itself. Close code 4002 ("media connection renewed") means the server ended the media connection only to renew it (it had carried as many tracks as one connection may): the client reconnects at once and resumes its call with voice_join and resume: true; within the grace period nobody sees it leave. These codes are the member socket's; the bot gateway's 4001 and 4002 (unknown opcode, undecodable frame, doc/bots.md) mean something else. Upgrades are rate-limited per address and per user (429), and voice_join beyond about one per second (burst 3) is answered with voice_kicked.

Authentication: Session cookie

Parameters

NameInTypeRequiredDescription
UpgradeheaderstringyesMust be websocket.

Responses

StatusMeaningBody
101Switching Protocols; the connection is now a WebSocket.
401No valid session (UNAUTHORIZED): missing, expired or revoked cookie, or the account was disabled.ErrorResponse
403FORBIDDEN: cross-origin request rejected by the CSRF check.ErrorResponse
429RATE_LIMITED: too many requests.ErrorResponse
500INTERNAL_ERROR: sanitized server failure.ErrorResponse
503UNAVAILABLE: server shutting down.ErrorResponse

Something wrong on this page? Tell us.