Gaming integration: how it works and what it never does
From mnema-talk-desktop v1.4.0, rendered on this site.
The gaming integration (sidepeek, overlay, global shortcuts, push-to-talk) is Beta and Windows-first. This page is the reference for the desktop client. The user-facing setup guide is in the main repository (desktop-app.md).
Anti-cheat safety statement
Online games run anti-cheat software, and a chat client must never look like a cheat. This is what the client does, and what it will never do.
It does:
- Ask the operating system to tell it when the focused application changes (a documented, passive notification), and look up the image path of that one process.
- Read read-only launcher manifests on disk (for example the list of installed Steam games) to learn which folders are game installs.
- Draw its own windows (the sidepeek and the overlay) as ordinary topmost windows of its own process.
It never:
- Enumerates processes or windows to find games. Detection starts from the focus-change notification and touches only the process that is in front.
- Injects code or a DLL into any other process, hooks another process, reads or writes another process's memory, or opens a handle with more access than "query limited information" to ask for its image path.
- Installs a driver or a kernel component.
- Hooks a game's graphics API. There is no in-game rendering; the overlay is a separate window, which is why it cannot draw over exclusive fullscreen.
- Sends synthetic input to a game or reads the game's keyboard or mouse state beyond the push-to-talk and shortcut keys the user configured.
- Sends the image path, window title or process name of what you run to any server. A matched game name stays on the device.
Game detection
Detection is event driven. There is no timer that looks at the foreground window.
| System | Source | Looks up |
|---|---|---|
| Windows | SetWinEventHook(EVENT_SYSTEM_FOREGROUND, WINEVENT_OUTOFCONTEXT) on a dedicated thread with its own message loop (out of context: nothing is loaded into other processes) | QueryFullProcessImageNameW on the foreground window's single PID, opened with PROCESS_QUERY_LIMITED_INFORMATION |
| macOS, Linux | none: the gaming integration is Windows only. Detection is off and the settings say so. | - |
The foreground source sits behind the ForegroundSource trait
(src/foreground.rs) with a fake for tests. The operating-system code is in
src/foreground_windows.rs.
A window counts as a game only when it was announced by the source and is still the foreground window when the sidepeek or shortcuts act, so an announcement that has been overtaken by a newer focus change never shows a game.
Which game is it?
Three sources, checked in this order. The first match wins and decides the name.
Games you added in the gaming settings: type an executable file name such as
MyGame.exe, or press "Add the app I was just in" and confirm the file name you are shown. That name is the last other application the foreground event announced before you opened Mnema (never Mnema itself, Java, or a helper such as a crash handler). Only the bare file name is handed to the settings page, only when you press the button; the path stays in the host and nothing leaves the device. The list is kept locally (up to 32 names) and each entry can be removed again.Launcher libraries. The install folders of the games your launchers report. A program that lives under one of those folders is that game, and the game is named as the launcher names it. Read-only, from files and registry keys the launchers keep for themselves:
Launcher Read Notes Steam steamapps/libraryfolders.vdfandsteamapps/appmanifest_*.acfof every librarySteam tools and runtimes (Steamworks Shared, SteamVR, Proton, Linux runtimes, Wallpaper Engine) are skipped Epic Games Store %ProgramData%\Epic\EpicGamesLauncher\Data\Manifests\*.itemadd-ons and unfinished installs are skipped GOG Galaxy HKLM\SOFTWARE\WOW6432Node\GOG.com\Games\*(gameName,path)EA app HKLM\SOFTWARE\WOW6432Node\EA Games\*andHKLM\SOFTWARE\EA Games\*(Install Dir)Battle.net not read its install list is a binary database without a public format; Battle.net games are found by the bundled list or a name you add Every path read from a manifest or the registry is checked before the disk is touched: only absolute local drive paths are opened. UNC (
\\host\share), device (\\?\,\\.\) and relative paths,..segments and alternate data streams are skipped silently, so a crafted manifest cannot make Windows contact another machine. A manifest folder that is a drive root, a system folder or a broad folder such asProgram Filesitself is ignored, so one bad manifest cannot make every program a game. Manifests larger than 256 KiB are ignored.The bundled list (
src/game-list.json): executable file names of widely played PC games. Matching is by file name only.
The launcher scan runs on its own thread: once at start, every 15 minutes, and
whenever the gaming settings are saved. The foreground path only reads the last
finished result. Java is never matched by name: javaw.exe counts as Minecraft
only when its window title says so, and a Java program inside a launcher's game
folder is that game. Crash handlers, uninstallers and redistributables inside a
game folder are not the game.
The bundled list
src/game-list.json is maintained by hand. It holds about 600 well-known PC
games from the last quarter century (shooters, battle royales, MOBAs, MMOs,
survival, sports, racing, simulators, strategy and indie hits): a display name,
one or more executable file names and, for some, a Steam app id kept for
reference. It is deliberately not complete and has not been checked against
installed copies. The launcher libraries (above) cover the rest: any game your
Steam, Epic, GOG or EA app reports is found there without being in the list,
and you can add a name yourself in the gaming settings. A missing or wrong name
only means that game is found through its launcher or a name you add instead.
A unit test keeps names, executables and ids unique and well formed. Corrections
and verified names are welcome under the repository licence.
Sources
The list is our own work, written from public knowledge of the executable names games ship. An earlier catalog that came from such a list was deleted; the Git history still contains that old file (names and executable names only), and it is kept as is because rewriting public history would break signed release tags and forks. No file in the current tree carries data derived from it. The header of src/game-list.json states the provenance.
Push-to-talk and shortcuts
Keys are event driven. Nothing asks "is this key down?" in a loop.
| System | Source | Notes |
|---|---|---|
| Windows | Raw Input (RegisterRawInputDevices with RIDEV_INPUTSINK, keyboard and mouse) on its own thread with a hidden window | No keyboard hook, no GetAsyncKeyState. Works while a game has the focus. A session lock or unlock, a console or remote disconnect, a logoff and a suspend or resume clear all keys, so a key cannot stay stuck down when its key-up happens where Raw Input cannot see it. |
| macOS, Linux | none | The gaming integration is Windows only. |
What is read: for each change only the key (or the mouse side button 4 or 5) and whether it went down or up. Text is never looked at, no key history is kept and nothing about keystrokes is logged or sent. The host keeps the set of keys currently down and compares it with your four chords (overlay, mute, deafen, push-to-talk); only a change in whether a chord is down wakes the rest of the app. A press that begins and ends between two looks is still delivered once.
The UI thread never waits for the system or for a game. The input thread, the
foreground-window thread and the library scan are threads of their own; the one
shell question that can take seconds while a game starts (is something in
exclusive fullscreen?) is asked on a worker thread, spaced two seconds apart, with
at most one in flight; the toast follows on that thread. A window of another
program is only ever asked with SendMessageTimeout and SMTO_ABORTIFHUNG, so a
stuck game costs the timeout, not a hang. A watchdog thread asks the UI thread to
answer a message every 100 ms; when it does not within 200 ms it writes what the
UI thread was doing to logs/desktop.log in the data folder (mnema-data\data\logs
beside the program for the portable copy, %LOCALAPPDATA%\com.mnema.desktop.shareddev\logs
for the installed one). The Windows CI keeps a full-screen stand-in game in front
while sending input and requires the host's windows to keep answering.
The source runs only while you are in a Talk with gaming on, and stops when you leave. On Windows the host asks the windowing library to release the process's own raw input registration first, because Windows allows one registration per process; if something else still holds it, the settings say so and push-to-talk works while Mnema is in front.
Push-to-talk and the other shortcuts act only while a detected game is in the
foreground, as before. The web page accepts a ptt-press for 150 ms and drops
the press 300 ms after the last one (a safeguard against a stuck microphone), so
while the key is held the host repeats the press every 100 ms. That is a
keepalive of a state known from the key event, not input polling. Leaving the
game, leaving the Talk, a session lock or a stalled host releases push-to-talk.
Known limits
- Raw Input does not see the secure desktop (UAC prompts, Ctrl+Alt+Del). A key released there is not reported; the session-lock reset and the page's 300 ms safeguard bound the damage.
- Games that read the keyboard exclusively (some old DirectInput exclusive modes) do not change what Raw Input delivers, but we cannot test every title.
- Not verified on real hardware or against any anti-cheat; see the safety statement above for what the client does and never does.
Games in exclusive fullscreen
The sidepeek and the overlay are ordinary topmost windows, so they cannot be drawn over a game that owns the display in exclusive fullscreen. Borderless windowed ("windowed fullscreen") works. The client does not try to get around this: no graphics hook, no injection.
While a detected game is in front and the hint has never been given, the host
asks the shell one cheap question per beat, SHQueryUserNotificationState
(Windows). QUNS_RUNNING_D3D_FULL_SCREEN means exclusive fullscreen; every other
answer, including a borderless full-screen window, can be drawn over. After two
looks in a row the host shows one toast (it waits in the notification centre
while the game owns the display) that suggests borderless windowed mode and says
that push-to-talk and shortcuts keep working. It is given once per installation
(gaming-hints.json in the app's config folder) and never repeated. The same
hint is in the gaming snapshot as exclusive_fullscreen_hint for the settings and
the surfaces. Nothing about it leaves the device.