diff --git a/docs/selkies/components/baseimages.md b/docs/selkies/components/baseimages.md index 77011624..22832b93 100644 --- a/docs/selkies/components/baseimages.md +++ b/docs/selkies/components/baseimages.md @@ -30,7 +30,7 @@ There is deliberately **no `latest` tag** for base images. Downstream images pin - **[Selkies Desktop](selkies-desktop.md)** at `/usr/bin/selkies-desktop`, activated by env var. - **Nginx** with the fancyindex module, serving the client, proxying the WebSocket, handling basic auth, subfolder support, and the `/files` download index. - **PulseAudio** with null sinks (`output` and `input`) wired for stream audio and microphone return. -- **Gamepad plumbing**: the joystick interposer and fake udev libraries, preloaded globally, with device nodes created at init. +- **Gamepad plumbing**: the input interposer and fake udev libraries, preloaded globally, with device nodes created at init. - **Quality of life**: passwordless sudo for the desktop user, all system locales prebuilt for `LC_ALL`, `proot-apps` synced into the user home for persistent app installs, Docker in Docker support for privileged containers, and notification support. ## The runtime in one diagram @@ -47,7 +47,7 @@ graph TD SELKIES -->|starts in process| PF[pixelflux Wayland compositor, socket wayland-1] DE -->|waits for wayland-1| LABWC[labwc or a full DE, exposes wayland-0] LABWC --> APP[autostart application] - NGINX -->|3000 / 3001| WEB[web client, /websocket proxy, /files, /pelorus] + NGINX -->|3000 / 3001| WEB[web client, /api proxy, /pelorus] ``` At startup a chain of one shot init scripts configures everything from environment variables: Nginx substitution (ports, auth, subfolder, title), Wayland or X11 mode selection, first run copy of the autostart and menu defaults into `/config`, hardening (the `HARDEN_*` and `DISABLE_*` family), GPU detection and permission fixes, and gamepad device setup. Then the long running services above come up in dependency order. diff --git a/docs/selkies/components/index.md b/docs/selkies/components/index.md index c4b2b3bb..c106ad0d 100644 --- a/docs/selkies/components/index.md +++ b/docs/selkies/components/index.md @@ -57,7 +57,7 @@ graph TD To make the layering concrete, here is the life of one frame in a Wayland mode container: 1. The application renders into a buffer belonging to **labwc** (or KWin on KDE), which is itself a client of the headless Smithay compositor that **pixelflux** hosts in process. -2. Pixelflux composites the output. If a GPU holds the framebuffer and the encoder lives on the same GPU, the frame is passed as a DMA-BUF straight into NVENC or VA-API, zero copy. Otherwise it is read back and encoded on CPU, in parallel stripes if the software encoder is in use. +2. Pixelflux composites the output. If a GPU holds the framebuffer and the encoder lives on the same GPU, the frame is passed as a DMA-BUF straight into NVENC or VA-API, zero copy, for any codec the card carries. Otherwise it is read back and encoded on CPU, in parallel stripes for H.264 and JPEG and full frame for the other codecs. 3. Only regions that changed get encoded at all; a static screen costs almost nothing, and after motion stops a high quality paint over pass restores perfect detail. 4. The encoded frame, with a small binary header, is handed to **Selkies**, which broadcasts it over the WebSocket to every connected viewer with backpressure control per client. 5. The container's **Nginx** carries that WebSocket alongside the static web client, file downloads, basic auth, and the optional Pelorus API, all on one HTTPS port. diff --git a/docs/selkies/components/pixelflux.md b/docs/selkies/components/pixelflux.md index 9f0a7dca..f180bcda 100644 --- a/docs/selkies/components/pixelflux.md +++ b/docs/selkies/components/pixelflux.md @@ -1,8 +1,8 @@ # Pixelflux and Pcmflux -**Repository:** [linuxserver/pixelflux](https://github.com/linuxserver/pixelflux) · **PyPI:** `pip install pixelflux` · **License:** MPL-2.0 +**Repositories:** [selkies-project/pixelflux](https://github.com/selkies-project/pixelflux) and [selkies-project/pcmflux](https://github.com/selkies-project/pcmflux) · **PyPI:** `pip install pixelflux pcmflux` · **License:** MPL-2.0 · **Upstream docs:** [docs.selkies.io](https://docs.selkies.io/) · **Rustdocs:** [pixelflux](https://pixelflux.selkies.io/pixelflux/index.html), [pcmflux](https://pcmflux.selkies.io/pcmflux/index.html) -Pixelflux is the rendering and capture pipeline at the core of the entire platform. It captures a Linux framebuffer, decides what changed, encodes it as JPEG or H.264, and hands the encoded frames to your code through a Python callback. Selkies is its primary consumer, but it is a standalone library you can embed in anything. +Pixelflux is the rendering and capture pipeline at the core of the entire platform. It captures a Linux framebuffer, decides what changed, encodes it as JPEG, H.264, H.265, VP8, VP9, or AV1, and hands the encoded frames to your code through a Python callback. Selkies is its primary consumer, but it is a standalone library you can embed in anything. Pcmflux is its audio sibling: it captures PulseAudio output and encodes Opus frames for delivery to the browser, and handles the microphone return path. Both ship as prebuilt wheels for x86_64 and aarch64 on glibc and musl. @@ -12,23 +12,28 @@ As of the 2.0.0 release, pixelflux is a **Rust** library exposed to Python throu - An **X11 backend**: XShm screen capture with XFixes cursor tracking, for the legacy X11 stack. - A **Wayland backend**: a full headless Wayland compositor built on [Smithay](https://github.com/Smithay/smithay), running in process. This is the key architectural point of the modern stack: **pixelflux does not capture a Wayland compositor, it is the compositor.** It synthesizes the output, seat, and clipboard itself, which is why the framebuffer can live directly on a GPU and why input is injected through its API rather than tools like xdotool. -- Four encoders and the damage tracking, paint over, and rate control logic shared between them. +- Six codecs across software and hardware encoders, and the damage tracking, paint over, and rate control logic shared between them. ## Encoders -| Encoder | Hardware | Mode | Notes | -| --- | --- | --- | --- | -| JPEG | CPU | Striped | Stateless, per stripe quality. Kept as the compatibility path for browsers that cannot decode video frames; 4:4:4 H.264 with paint over matches it visually everywhere else | -| x264 | CPU | Striped or full frame | Default software H.264, ultrafast zerolatency, 4:4:4 capable | -| OpenH264 | CPU | Full frame only | Opt in alternative software encoder, 4:2:0 only | -| NVENC | Nvidia GPU | Full frame | Direct NVENC via runtime library loading, no CUDA toolkit needed, supports High 4:4:4, zero copy from DMA-BUF | -| VA-API | Intel and AMD GPU | Full frame | Through FFmpeg's `h264_vaapi`, zero copy from DMA-BUF, no 4:4:4 (falls back to CPU) | +Every codec is selected by name (`codec = "h264"` and so on) and pixelflux resolves the encoder behind it: the GPU engine when the encoding device has one, the software encoder the build carries otherwise. `pixelflux.SOFTWARE_ENCODERS` names the software encoder per codec and `pixelflux.hardware_encoders(node)` the codecs a render node's NVENC or VA-API serves, which is how Selkies trims its menu at startup. + +| Codec | Software | NVENC (Nvidia) | VA-API (Intel and AMD) | Shape | 4:4:4 | +| --- | --- | --- | --- | --- | --- | +| JPEG | libjpeg-turbo, vendored | No | No | Striped | Always, JFIF is full color | +| H.264 | x264, or OpenH264 in a GPL free build | Yes | Yes | Striped or full frame | x264 and NVENC. VA-API has no 4:4:4 H.264 profile on current drivers and hands the request to x264. OpenH264 is 4:2:0 only | +| H.265 | x265, or kvazaar in a GPL free build | Yes | Yes | Full frame | x265 and NVENC, VA-API negotiates a 4:4:4 surface per device | +| VP8 | libvpx | No | Where the GPU has an engine | Full frame | No, and VP8 declares BT.601 because its bitstream can name nothing else | +| VP9 | libvpx | No | Yes | Full frame | Profile 1 on libvpx and VA-API | +| AV1 | SVT-AV1 | Ada and newer | Where the GPU has an engine | Full frame | No | + +Hardware encoders are loaded at runtime: NVENC through the driver's `libnvidia-encode` and `libcuda` with no CUDA toolkit, VA-API and the software H.265, VP8, VP9, and AV1 encoders through the system FFmpeg's `libavcodec`. The official wheels are GPL builds with x264 and x265; a `PIXELFLUX_ENABLE_GPL=0` source build swaps in OpenH264 and kvazaar and keeps everything else. ### Striped encoding -On the CPU paths the screen is divided into horizontal stripes, one per CPU core (each at least 64 rows). Each stripe has its own encoder instance and its own damage history, and stripes are encoded in parallel across a thread pool. Only stripes that changed are encoded and sent. This is the "hybrid VNC and video codec" idea that lets a CPU only server idle at nearly zero cost and still deliver 60fps where the screen is actually moving. +For JPEG and H.264 on the CPU the screen is divided into horizontal stripes, one per CPU core (each at least 64 rows). Each stripe has its own encoder instance and its own damage history, and stripes are encoded in parallel across a thread pool. Only stripes that changed are encoded and sent. This is the "hybrid VNC and video codec" idea that lets a CPU only server idle at nearly zero cost and still deliver 60fps where the screen is actually moving. -Hardware encoders always operate full frame, delivered as a single full height stripe, since the GPU encodes the whole surface in one shot anyway. +Hardware encoders always operate full frame, delivered as a single full height stripe, since the GPU encodes the whole surface in one shot anyway. H.265, VP8, VP9, and AV1 are full frame on the CPU too, so a software session on those codecs gets neither the per core parallelism nor the dirty stripe savings, and costs several times the CPU of striped x264. ### Damage detection @@ -38,7 +43,7 @@ Hardware encoders always operate full frame, delivered as a single full height s ### Paint over -The signature quality feature. After a region has been static for a configurable number of frames (`paint_over_trigger_frames`), it is re sent at high quality: a higher quality JPEG, or for H.264 a burst of frames at a lower CRF. Motion cancels an in flight burst. The result is video efficiency during motion and pixel perfect text the moment you stop scrolling. +The signature quality feature. After a region has been static for a configurable number of frames (`paint_over_trigger_frames`), it is re sent at high quality: a higher quality JPEG, or for the video codecs a burst of frames at a lower CRF. Motion cancels an in flight burst. The result is video efficiency during motion and pixel perfect text the moment you stop scrolling. ### Rate control @@ -47,7 +52,9 @@ The signature quality feature. After a region has been static for a configurable ## Zero copy on Wayland -When the compositor renders on a GPU and the encoder is on the same GPU, frames flow as DMA-BUF handles from the render buffer straight into NVENC or VA-API. The pixels never touch system RAM and the CPU never sees them. If the render and encode devices differ, or a software encoder is selected, pixelflux falls back to a readback path automatically and logs which decision it made. +When the compositor renders on a GPU and the encoder is on the same GPU, frames flow as DMA-BUF handles from the render buffer straight into NVENC or VA-API. The pixels never touch system RAM and the CPU never sees them. If the render and encode devices differ, or a software encoder is selected, pixelflux falls back to a readback path automatically and logs which decision it made. Zero copy is a property of the capture path, not the codec: any codec the GPU carries takes it. + +On X11 there is one zero copy path as well. When the session encodes on NVENC, capture goes through NvFBC, the Nvidia X driver composites the screen into a buffer in video memory, and that buffer is registered with the encoder in place. Every other X11 session copies each frame once into shared memory. Nothing selects this, the driver decides and the log says which path was taken. GPU selection is automatic: it walks `/sys/class/drm`, identifies cards by driver (`nvidia` goes to NVENC, `i915` and `amdgpu` to VA-API), and can be pinned by device path, index, or an `auto_gpu` token matching a driver or vendor ID. @@ -60,13 +67,13 @@ settings = CaptureSettings() settings.capture_width = 1920 settings.capture_height = 1080 settings.target_fps = 60.0 -settings.output_mode = 1 # 0 = JPEG, 1 = H.264 +settings.codec = "h264" # "jpeg", "h264", "h265", "vp8", "vp9", or "av1" settings.video_crf = 25 settings.use_paint_over_quality = True def on_frame(frame): # frame is a StripeFrame: zero copy buffer protocol object - # frame.data_type: 1 = JPEG, 2 = H.264 + # frame.data_type: 0 = JPEG, 1 = H.264, 2 = VP8, 3 = VP9, 4 = AV1, 5 = H.265 # bytes(frame) or memoryview(frame) for the payload ws.send(bytes(frame)) @@ -74,14 +81,14 @@ capture = ScreenCapture() capture.start_capture(on_frame, settings) ``` -Each encoded stripe arrives with a compact binary header (6 bytes for JPEG, 10 for H.264 carrying frame type, frame number, stripe offset, and dimensions) that the web client parses to place stripes on the canvas. Headers can be omitted for embedding in your own protocol. +Each encoded stripe arrives with a compact binary header (6 bytes for JPEG, 10 for the video codecs carrying frame type and codec id, frame number, stripe offset, and dimensions) that the web client parses to place stripes on the canvas. Headers can be omitted for embedding in your own protocol. The Wayland backend additionally exposes input injection (keyboard by scancode with a hot swappable XKB keymap, absolute and relative pointer, buttons, scroll), clipboard get and set, cursor callbacks delivering PNG cursor images out of band, and live rate updates. `example/screen_to_browser.py` in the repository is a complete working WebSocket streaming server in one file, with a matching browser client in `example/index.html`. ## Extras worth knowing about - **Computer Use API**: setting the `PIXELFLUX_CU=` environment variable starts a small HTTP server with a `POST /computer-use` endpoint accepting JSON actions (`screenshot`, `left_click`, `type`, `key`, `scroll`, `zoom`, and friends). This is the raw input and vision layer that [Pelorus](pelorus.md) builds on. -- **Recording sink**: point `recording_socket` (or `PIXELFLUX_RECORDING_SOCKET`) at a Unix socket path and pixelflux serves the raw Annex-B H.264 bitstream to any connected client, forcing a keyframe when someone connects. Requires a full frame H.264 mode. +- **Recording sink**: point `recording_socket` (or `PIXELFLUX_RECORDING_SOCKET`) at a Unix socket path and pixelflux serves the raw elementary stream to any connected client, forcing a keyframe when someone connects: Annex B for H.264 and H.265, an OBU stream for AV1, IVF for VP8 and VP9. Requires a full frame codec, striped H.264 and JPEG are not served. - **Watermarking**: composite a PNG over the stream at any corner, centered, or animated (it bounces). On the GPU path the watermark is composited before encode with no readback penalty. - **Fractional scaling**, HiDPI support, and cursor theme control on the Wayland backend. @@ -91,4 +98,4 @@ The library itself is configured purely through `CaptureSettings`. The familiar ## Building from source -`pip install .` drives a `setuptools-rust` build. On Debian or Ubuntu the build dependencies are roughly: `python3-dev cmake nasm libclang-dev libavcodec-dev libavutil-dev libx264-dev libturbojpeg0-dev libgbm-dev libdrm-dev libwayland-dev libinput-dev libxkbcommon-dev libva-dev` plus a Rust toolchain. Any system FFmpeg from 6.0 through 8.1 works (only used for VA-API). Most users should just take the prebuilt wheels. +`pip install .` drives a `setuptools-rust` build. On Debian or Ubuntu the build dependencies are roughly: `python3-dev cmake nasm libclang-dev libavcodec-dev libavutil-dev libx264-dev libturbojpeg0-dev libgbm-dev libdrm-dev libwayland-dev libinput-dev libxkbcommon-dev libva-dev` plus a Rust toolchain. Any system FFmpeg from 6.0 through 9.0 works, it supplies the VA-API encoders and the software H.265, VP8, VP9, and AV1 encoders, so a codec whose encoder that FFmpeg lacks has no software path. Most users should just take the prebuilt wheels. diff --git a/docs/selkies/components/selkies.md b/docs/selkies/components/selkies.md index 26689f5a..f152e9b8 100644 --- a/docs/selkies/components/selkies.md +++ b/docs/selkies/components/selkies.md @@ -1,6 +1,6 @@ # Selkies -**Repository:** [selkies-project/selkies](https://github.com/selkies-project/selkies) · **License:** MPL-2.0 · **Upstream docs:** [selkies-project.github.io/selkies](https://selkies-project.github.io/selkies/) +**Repository:** [selkies-project/selkies](https://github.com/selkies-project/selkies) · **License:** MPL-2.0 · **Website:** [selkies.io](https://selkies.io/) · **Upstream docs:** [docs.selkies.io](https://docs.selkies.io/) Selkies is the heart of the platform: a ground up, web native remote desktop protocol and the Python server plus web client that implement it. It began in the Selkies project founded by Dan Isla (ex Google, ex NASA) with Seungmin Kim, targeting cloud gaming class remote desktops for Linux, and it is developed today in partnership between the Selkies organization and LinuxServer.io. @@ -8,10 +8,10 @@ Selkies is the heart of the platform: a ground up, web native remote desktop pro `selkies` is a Python asyncio application (console script `selkies`, installed in the baseimages at `/lsiopy/bin/selkies`) that owns the session: -- **Video**: drives [pixelflux](pixelflux.md) capture and encoding, and broadcasts encoded frames to all connected clients over the WebSocket, with per client backpressure (frame acknowledgements, RTT smoothing, and stall detection) so one slow viewer does not degrade the rest. +- **Video**: drives [pixelflux](pixelflux.md) capture and encoding, and broadcasts encoded frames to all connected clients over the WebSocket, with per client backpressure (frame acknowledgements, RTT smoothing, and stall detection) so one slow viewer does not degrade the rest. An opt in [WebRTC transport](../user-guide/webrtc.md) carries the same video and Opus streams over UDP with a priority pacer and optional congestion control; clients can switch transports at runtime. - **Audio out**: drives pcmflux, which captures the PulseAudio `output.monitor` source and Opus encodes at up to 320kbps. - **Microphone in**: receives PCM from the browser and plays it into a virtual PulseAudio source (`SelkiesVirtualMic`) that session apps consume as a normal mic. -- **Input**: injects keyboard, mouse, touch, and scroll. On Wayland, injection goes through pixelflux's compositor APIs with an xkbcommon keymap (plus [waylandtyper](https://github.com/linuxserver/waylandtyper), our maintained fork of `wtype`, for unicode text batches). On X11, through pynput, xdotool, and python-xlib. Gamepads are handled by per slot Unix socket servers feeding the joystick interposer (below). +- **Input**: injects keyboard, mouse, touch, and scroll. On Wayland, injection goes through pixelflux's compositor APIs with an xkbcommon keymap (plus [waylandtyper](https://github.com/linuxserver/waylandtyper), our maintained fork of `wtype`, for unicode text batches). On X11, through pynput, xdotool, and python-xlib. Gamepads are handled by per slot Unix socket servers feeding the input interposer (below). - **Clipboard**: bidirectional sync via `wl-clipboard` on Wayland or `xclip` on X11, with optional binary (image) clipboard support and chunked transfer for large payloads. - **Files**: receives chunked uploads over the socket into the session; downloads are served by the container's Nginx file index. - **Settings and stats**: pushes the sanitized settings schema to the client (this is what builds the sidebar UI, including which controls are locked), and streams CPU, GPU, memory, and network stats. @@ -23,7 +23,7 @@ Configuration is uniform: every setting is simultaneously a CLI flag (`--framera The client lives in the same repository under `addons/`: -- **selkies-web-core**: the engine. Connects the WebSocket, demultiplexes binary frames, decodes H.264 with WebCodecs `VideoDecoder`, JPEG stripes with `createImageBitmap`, and Opus in a worker, renders to canvas, plays audio through an AudioWorklet, and captures all input including trackpad gestures and pointer lock. It exposes a documented `postMessage` API so any UI can be built on top of it. +- **selkies-web-core**: the engine. Connects the WebSocket, demultiplexes binary frames, decodes H.264, H.265, VP8, VP9, or AV1 with WebCodecs `VideoDecoder`, JPEG stripes with `createImageBitmap`, and Opus in a worker, renders to canvas, plays audio through an AudioWorklet, and captures all input including trackpad gestures and pointer lock. It exposes a documented `postMessage` API so any UI can be built on top of it. - **selkies-dashboard**: the standard React sidebar UI described in [Using the Web Client](../user-guide/web-client.md), fully internationalized. Variants exist (`selkies-dashboard-wish` is a newer TypeScript and shadcn based rewrite), and the container's `DASHBOARD` variable selects between built dashboards. The postMessage split means a white label or kiosk frontend is just another dashboard. - **universal-touch-gamepad**: an on screen touch gamepad that injects itself into the browser's Gamepad API with configurable layouts. @@ -31,7 +31,7 @@ The client lives in the same repository under `addons/`: Two small C addons make plug and play gamepads work in unprivileged containers with no kernel devices: -- **js-interposer**: an `LD_PRELOAD` library that intercepts `open()` calls to `/dev/input/js*` and `/dev/input/event*` and transparently redirects them to Unix sockets served by Selkies, faithfully emulating the Linux joystick and evdev APIs (ioctls included). Apps think they opened a real Xbox 360 pad. +- **input-interposer**: an `LD_PRELOAD` library that intercepts `open()` calls to `/dev/input/js*` and `/dev/input/event*` and transparently redirects them to Unix sockets served by Selkies, faithfully emulating the Linux joystick and evdev APIs (ioctls included). Apps think they opened a real Xbox 360 pad. It works in the other direction too: a preloaded application that opens `/dev/uinput` to create its own virtual device (Steam Input, a remapper) gets a socket backed device that sibling applications discover through fake udev, and with `SELKIES_PUBLISH_INPUT_DEVICES` the same pool carries a copy of the session keyboard and pointer for applications that enumerate evdev directly. - **fake-udev**: a stub `libudev` that makes device enumeration APIs report the synthetic pads with a plausible sysfs hierarchy, for apps and engines (SDL, browsers) that discover devices through udev rather than by opening device nodes. Both are preloaded automatically in the baseimages, and `NO_GAMEPAD=true` turns the whole mechanism off. @@ -40,7 +40,7 @@ Both are preloaded automatically in the baseimages, and `NO_GAMEPAD=true` turns | Port | What | | --- | --- | -| 8082 | The data WebSocket (`SELKIES_PORT`; upstream default is 8081, the baseimages set 8082), proxied by Nginx at `/websocket` | +| 8082 | The Selkies server (`SELKIES_PORT`, upstream default is 8080, the baseimages set 8082 via `CUSTOM_WS_PORT`). Nginx proxies everything under `/api` to it: the data WebSocket at `/api/websockets`, WebRTC signaling at `/api/webrtc/signaling`, the transport switch, and the file browser at `/api/files/` | | 8083 | Token control plane for secure sharing mode, never expose it | | 3000 / 3001 | Nginx HTTP and HTTPS in front of everything ([baseimage](baseimages.md) territory) | diff --git a/docs/selkies/developer-guide/architecture.md b/docs/selkies/developer-guide/architecture.md index d763a816..1ced3905 100644 --- a/docs/selkies/developer-guide/architecture.md +++ b/docs/selkies/developer-guide/architecture.md @@ -34,20 +34,21 @@ In X11 fallback mode the shape is more traditional: a patched Xvfb (with DRI3 de ### Encoding -Four encoders behind one policy layer: +Six codecs behind one policy layer, each resolved to a GPU engine or a software encoder at runtime: -| Path | Encoders | Shape | +| Path | Codecs | Shape | | --- | --- | --- | -| CPU | x264, JPEG (and optional OpenH264) | Striped: one stripe per core, parallel encode, only dirty stripes sent | -| GPU | NVENC (Nvidia), VA-API (Intel and AMD) | Full frame, zero copy from DMA-BUF when render and encode share a device | +| CPU striped | JPEG, H.264 (x264, or OpenH264 in a GPL free build) | One stripe per core, parallel encode, only dirty stripes sent | +| CPU full frame | H.265 (x265), VP8 and VP9 (libvpx), AV1 (SVT-AV1) | Whole frame per encode, no striping | +| GPU | NVENC: H.264, H.265, AV1. VA-API: all five | Full frame, zero copy from DMA-BUF when render and encode share a device | -Quality logic is shared: infinite GOP with on demand IDR frames, CRF rate control with live retuning, and the **paint over** system, after N static frames, resend at high quality (better JPEG quality, or an H.264 burst at lower CRF), cancelled instantly by motion. +Quality logic is shared: infinite GOP with on demand IDR frames, CRF rate control with live retuning on one quality scale mapped onto each codec's quantizer, and the **paint over** system, after N static frames, resend at high quality (better JPEG quality, or a video burst at lower CRF), cancelled instantly by motion. Selkies probes the encoding device at startup and only offers the codecs the host can serve. The full encoder and settings detail lives on the [Pixelflux page](../components/pixelflux.md). ### Transport and presentation -Encoded frames go to the Selkies server as callback invocations carrying a compact binary header (type, frame id, stripe geometry), and Selkies broadcasts them raw over the WebSocket, the server never re muxes or re packetizes. In the browser, WebCodecs decodes H.264, `createImageBitmap` handles JPEG stripes, and everything composites onto a canvas. Because the client acknowledges frame ids, the server maintains a per client backpressure window: slow clients get frames dropped *before* encode (keeping the H.264 reference chain valid), fast clients are never held back. Wire formats are specified in [The Streaming Protocol](protocol.md). +Encoded frames go to the Selkies server as callback invocations carrying a compact binary header (type, frame id, stripe geometry), and Selkies broadcasts them raw over the WebSocket, the server never re muxes or re packetizes. In the browser, WebCodecs decodes the video codec named in each frame's header, `createImageBitmap` handles JPEG stripes, and everything composites onto a canvas. Because the client acknowledges frame ids, the server maintains a per client backpressure window: slow clients get frames dropped *before* encode (keeping the H.264 reference chain valid), fast clients are never held back. Wire formats are specified in [The Streaming Protocol](protocol.md). ## The audio pipeline @@ -64,7 +65,7 @@ Gamepads bypass the display server entirely: Selkies serves the Linux joystick a ## The web layer -Nginx inside the container is the single front door: it serves the static client (a React dashboard over the `selkies-web-core` engine), proxies `/websocket` to the Selkies server, serves `/files` downloads with fancyindex, optionally enforces basic auth, applies the `SUBFOLDER` prefix, and proxies `/pelorus/` when the agent layer is on. The dashboard and the engine communicate over a documented `postMessage` API, which is the extension point for custom frontends. +Nginx inside the container is the single front door: it serves the static client (a React dashboard over the `selkies-web-core` engine), proxies `/api` (the data WebSocket, WebRTC signaling, and the `/api/files/` browser) to the Selkies server, optionally enforces basic auth, applies the `SUBFOLDER` prefix, and proxies `/pelorus/` when the agent layer is on. The dashboard and the engine communicate over a documented `postMessage` API, which is the extension point for custom frontends. ## Sharing and multi user diff --git a/docs/selkies/developer-guide/baseimage-internals.md b/docs/selkies/developer-guide/baseimage-internals.md index f2e06d05..be4b82b3 100644 --- a/docs/selkies/developer-guide/baseimage-internals.md +++ b/docs/selkies/developer-guide/baseimage-internals.md @@ -12,10 +12,10 @@ The baseimage Dockerfile is a multi stage build assembling, onto a LinuxServer.i - A rebuilt **wlroots** with a defensive patch that catches SIGSEGV and SIGBUS inside pixman draw calls and skips the frame instead of crashing the compositor - **selkies-desktop** and **[waylandtyper](https://github.com/linuxserver/waylandtyper)** (our maintained fork of wtype, fixing many bugs in the old codebase) built from source - The **selkies** Python package installed into the `/lsiopy` virtualenv (pulling in pixelflux and pcmflux wheels), plus **pelorus** -- The **joystick interposer** (`/usr/lib/selkies_joystick_interposer.so`) and **fake udev** (`/opt/lib/libudev.so.1.0.0-fake`) compiled from the Selkies addons +- The **input interposer** (`/usr/lib/selkies_input_interposer.so`) and **fake udev** (`/opt/lib/libudev.so.1.0.0-fake`) compiled from the Selkies addons - Nginx with fancyindex, PulseAudio, mesa and VA-API userspace, Vulkan loaders, all system locales, proot-apps, Docker in Docker machinery, and passwordless sudo for `abc` -Baked ENV defaults worth knowing: `HOME=/config`, `DISPLAY=:1`, `TITLE=Selkies`, `SELKIES_ENCODER="x264enc,jpeg"`, `START_DOCKER=true`, `DISABLE_ZINK=false`, `DISABLE_DRI3=false`, `NVIDIA_DRIVER_CAPABILITIES=all`, and the interposer path in `SELKIES_INTERPOSER`. +Baked ENV defaults worth knowing: `HOME=/config`, `DISPLAY=:1`, `TITLE=Selkies`, `SELKIES_ENCODER="h264enc,h265enc,vp8enc,vp9enc,av1enc,jpeg"`, `SELKIES_ENABLE_BASIC_AUTH=false`, `SELKIES_VIDEO_STREAMING_MODE=false`, `SELKIES_ALLOWED_ORIGINS="*"`, `START_DOCKER=true`, `DISABLE_ZINK=false`, `DISABLE_DRI3=false`, `NVIDIA_DRIVER_CAPABILITIES=all`, and the interposer path in `SELKIES_INTERPOSER`. ## Boot: the init chain @@ -34,13 +34,14 @@ init-os-end **`init-selkies-config`** is the big one: -- Chooses the mode: `PIXELFLUX_WAYLAND=true` selects labwc paths (`$HOME/.config/labwc`, `/defaults/autostart_wayland`, `/defaults/menu_wayland.xml`) and forces `SELKIES_SECOND_SCREEN=false`; otherwise Openbox paths. +- Chooses the mode: `PIXELFLUX_WAYLAND=true` selects labwc paths (`$HOME/.config/labwc`, `/defaults/autostart_wayland`, `/defaults/menu_wayland.xml`) otherwise Openbox paths. Multi monitor on Wayland is Selkies' call, not the init script's: at startup it probes the compositor, the labwc IPC socket or a KWin capability check, and enables the second screen only where it finds support. Both compositors carry LinuxServer patches for this, the labwc ones ship in the baseimage. - First run copies of `autostart` and `menu.xml` into the config dir (persistent, user editable); `rc.xml` for labwc is regenerated from the template every boot. - Recreates `$HOME/.XDG` as `XDG_RUNTIME_DIR` and clears stale PulseAudio state, so unclean shutdowns recover. - Applies every hardening variable (permission stripping, sudoers corruption, menu and keybind editing, locking rc.xml and autostart when watchdog mode is on), the details are in the [Security guide](../user-guide/security.md). - GPU env: with exactly one render node and nothing set, points `DRINODE` and `DRI_NODE` at it; wires `PIXELFLUX_CU=5000` and `ROOT_PATH=/pelorus` when `PELORUS=true`. - Creates the gamepad device nodes (`/dev/input/js0-3` and event nodes) and sets the global `LD_PRELOAD` for the interposer and fake udev, unless `NO_GAMEPAD` is set. - Syncs proot-apps into the user home and handles `LC_ALL` locale derivation. +- Steam shim: links `/usr/local/bin/steam` (marked `#SELKIESSHIM`) to `/usr/bin/steam` when nothing is there, so `steam` runs the installer in `/steam.sh` until a real launcher exists. `NO_STEAM=true` removes both. **`init-video`** fixes `/dev/dri` and `/dev/dvb` group permissions for `abc` (creating a matching group for the device GID when needed), auto enables `AUTO_GPU` on x86_64 when a render node exists and nothing was configured, probes whether older Intel hardware needs the `i965` VA-API driver, and repairs Nvidia ICD, Vulkan, EGL, and GBM plumbing inside the container. diff --git a/docs/selkies/developer-guide/development.md b/docs/selkies/developer-guide/development.md index 3d50740e..db9e37af 100644 --- a/docs/selkies/developer-guide/development.md +++ b/docs/selkies/developer-guide/development.md @@ -38,7 +38,7 @@ The application restarts on code changes to the mounted source directory and pro The same pattern gives you a pixelflux rapid development environment. Mount a pixelflux checkout instead and set `DEV_MODE=pixelflux`: ```bash -git clone https://github.com/linuxserver/pixelflux.git +git clone https://github.com/selkies-project/pixelflux.git cd pixelflux docker run --rm -it \ --shm-size=1gb \ @@ -56,7 +56,8 @@ The container builds the mounted source (all Rust build dependencies are present | Repository | What you touch there | | --- | --- | | [selkies-project/selkies](https://github.com/selkies-project/selkies) | Server, protocol, web client, dashboards, gamepad addons | -| [linuxserver/pixelflux](https://github.com/linuxserver/pixelflux) | Capture, compositor, encoders, Computer Use API | +| [selkies-project/pixelflux](https://github.com/selkies-project/pixelflux) | Capture, compositor, encoders, Computer Use API | +| [selkies-project/pcmflux](https://github.com/selkies-project/pcmflux) | Audio capture, Opus encoding, microphone return | | [linuxserver/docker-baseimage-selkies](https://github.com/linuxserver/docker-baseimage-selkies) | Packaging, init scripts, Nginx, hardening, patches. One branch per distro | | [linuxserver/docker-webtop](https://github.com/linuxserver/docker-webtop) and the app repositories | Downstream images, one branch per flavor for Webtop | | [selkies-project/sealskin](https://github.com/selkies-project/sealskin) | Orchestration server, extension, mobile | @@ -65,6 +66,8 @@ The container builds the mounted source (all Rust build dependencies are present | [selkies-project/selkies-desktop](https://github.com/selkies-project/selkies-desktop) | The minimal desktop shell | | [linuxserver/waylandtyper](https://github.com/linuxserver/waylandtyper) | Our maintained fork of wtype for unicode text injection | +Both Rust libraries publish generated API documentation: [pixelflux rustdocs](https://pixelflux.selkies.io/pixelflux/index.html) and [pcmflux rustdocs](https://pcmflux.selkies.io/pcmflux/index.html). Reach for those when you need the exact settings fields, frame types, and callbacks rather than the overview on the [Pixelflux page](../components/pixelflux.md). + The baseimage Dockerfile is the authoritative recipe for everything compiled into the images (labwc with the IPC patch, the patched wlroots, selkies-desktop, waylandtyper), so when you need to know exactly how a piece is built, read the corresponding Dockerfile stage. ## Contributing diff --git a/docs/selkies/developer-guide/protocol.md b/docs/selkies/developer-guide/protocol.md index 8b93bf36..2d2831dd 100644 --- a/docs/selkies/developer-guide/protocol.md +++ b/docs/selkies/developer-guide/protocol.md @@ -1,6 +1,6 @@ # The Streaming Protocol -The wire protocol between the Selkies server and the web client, for anyone implementing a client, embedding the engine, or debugging with a network inspector. Everything rides **one WebSocket** (proxied at `/websocket`), carrying a mix of binary frames and terse text messages. There is no negotiation dance: connect, receive settings, start receiving media. +The wire protocol between the Selkies server and the web client, for anyone implementing a client, embedding the engine, or debugging with a network inspector. Everything rides **one WebSocket** (proxied at `/api/websockets`), carrying a mix of binary frames and terse text messages. In the opt in [WebRTC mode](../user-guide/webrtc.md) the same text messages travel over a data channel and media over RTP, with signaling at `/api/webrtc/signaling`. There is no negotiation dance: connect, receive settings, start receiving media. ## Connection and roles @@ -17,12 +17,12 @@ The first byte of every binary frame is a type tag: | Tag | Payload | Header layout (big endian) | | --- | --- | --- | -| `0x00` | Full frame H.264 | `[1]` keyframe flag, `[2:4]` uint16 frame id, payload follows | +| `0x00` | Full frame video | `[1]` keyframe flag, `[2:4]` uint16 frame id, payload follows | | `0x01` | Opus audio packet | 2 byte header, then the Opus packet | | `0x03` | JPEG stripe | `[2:4]` frame id, `[4:6]` stripe Y offset, then the JPEG | -| `0x04` | H.264 stripe | `[1]` frame type (1 IDR, 2 I, 0 other), `[2:4]` frame id, `[4:6]` stripe Y offset, `[6:8]` width, `[8:10]` height, then Annex B NALs | +| `0x04` | Video stripe | `[1]` low nibble frame type (1 IDR, 2 I, 0 other), high nibble codec id (1 H.264, 2 VP8, 3 VP9, 4 AV1, 5 H.265), `[2:4]` frame id, `[4:6]` stripe Y offset, `[6:8]` width, `[8:10]` height, then the codec's own bitstream units | -These headers are produced by pixelflux itself (see the [wire format details](../components/pixelflux.md#api-sketch)); the Python server broadcasts them untouched. A full frame is just a stripe at Y offset 0 with full height. The client feeds H.264 to a WebCodecs `VideoDecoder` per stream, decodes JPEG stripes with `createImageBitmap`, and composites stripes onto the canvas at their Y offsets. The frame type byte reflects what the encoder actually emitted, clients use it to recover decoder state after drops. +These headers are produced by pixelflux itself (see the [wire format details](../components/pixelflux.md#api-sketch)); the Python server broadcasts them untouched. A full frame is just a stripe at Y offset 0 with full height. The client configures a WebCodecs `VideoDecoder` per stream from the codec id and the key frame's own parameter sets, decodes JPEG stripes with `createImageBitmap`, and composites stripes onto the canvas at their Y offsets. The frame type nibble reflects what the encoder actually emitted, clients use it to recover decoder state after drops. ## Binary messages, client to server @@ -84,8 +84,8 @@ Client sends `SETTINGS,{"framerate":60,"h264_crf":20,...}`; the server validates ## Implementing a client: a minimal path 1. Open the WebSocket with `binaryType = 'arraybuffer'`, send `START_VIDEO` and `START_AUDIO` after receiving `server_settings`. -2. Demux on byte zero. Feed `0x00` and `0x04` frames (strip the header, respect the keyframe flag) into a WebCodecs H.264 decoder configured from the stripe dimensions; paint `0x03` JPEGs at their offsets; queue `0x01` Opus into an audio decoder. +2. Demux on byte zero. Feed `0x00` and `0x04` frames (strip the header, respect the keyframe flag) into a WebCodecs decoder for the codec the header names, configured from the stripe dimensions; paint `0x03` JPEGs at their offsets; queue `0x01` Opus into an audio decoder. 3. Send `CLIENT_FRAME_ACK` with the latest presented frame id a few times per second. 4. Map your input events to the `kd`, `ku`, `m` or `m2` grammar. -That is a functioning viewer; everything else (clipboard, files, gamepads, stats) is additive. Start from the example client in the [pixelflux repository](https://github.com/linuxserver/pixelflux): `example/screen_to_browser.py` plus `example/index.html` are a complete working server and client pair, with the frame parsing in about a page of code. +That is a functioning viewer; everything else (clipboard, files, gamepads, stats) is additive. Start from the example client in the [pixelflux repository](https://github.com/selkies-project/pixelflux): `example/screen_to_browser.py` plus `example/index.html` are a complete working server and client pair, with the frame parsing in about a page of code. diff --git a/docs/selkies/index.md b/docs/selkies/index.md index 8dd2f680..02ccd6bf 100644 --- a/docs/selkies/index.md +++ b/docs/selkies/index.md @@ -37,14 +37,14 @@ A desktop is not a movie. Most of the time nothing on screen is changing, and wh Selkies is a ground up, web native remote desktop protocol designed to replace legacy VNC stacks. The core ideas: 1. **Hybrid protocol.** Damage tracking like VNC, video codecs like a streaming service. The screen is divided into horizontal stripes, only changed stripes are captured and encoded, and each stripe can be processed on a separate CPU core in parallel. -2. **Paint over quality.** H.264 handles fluid motion, and once motion stops the server repaints the static screen at high quality so text stays crisp. With FullColor 4:4:4 H.264 the painted over result is visually indistinguishable from a lossless image. A JPEG encoder remains available for older browsers that cannot decode video frames at all. -3. **WebSockets, not WebRTC.** Frames are delivered over a WebSocket connection and decoded in the browser with WebCodecs. This avoids WebRTC negotiation complexity, works cleanly through reverse proxies, and gives the server precise control over pacing and backpressure. +2. **Paint over quality.** A video codec (H.264 by default, with H.265, VP8, VP9, and AV1 available) handles fluid motion, and once motion stops the server repaints the static screen at high quality so text stays crisp. With FullColor 4:4:4 H.264 the painted over result is visually indistinguishable from a lossless image. A JPEG encoder remains available for older browsers that cannot decode video frames at all. +3. **WebSockets by default, WebRTC when you need it.** Frames are delivered over a WebSocket connection and decoded in the browser with WebCodecs. This avoids WebRTC negotiation complexity, works cleanly through reverse proxies, and gives the server precise control over pacing and backpressure. For lossy or high latency links an opt in [WebRTC transport](user-guide/webrtc.md) carries the same stream over UDP with congestion control, and users can switch between the two at runtime. 4. **Zero copy on Wayland.** In the current generation the display server is a virtual Wayland compositor built on [Smithay](https://github.com/Smithay/smithay). The framebuffer can live directly on a GPU, and frames are passed as DMA-BUF handles straight to the hardware encoder (VAAPI or NVENC) without a round trip through system RAM. 5. **Everything in one container.** Compositor, application, streaming server, audio, and web server all run inside a single OCI container built on `docker-baseimage-selkies`, managed by the s6 init system. ## What the platform gives you -- **A desktop in the browser.** Full desktop environments (KDE Plasma, XFCE, MATE, i3, and more) or single applications streamed over WebSockets with H.264 encoding. +- **A desktop in the browser.** Full desktop environments (KDE Plasma, XFCE, MATE, i3, and more) or single applications streamed over WebSockets with H.264, H.265, VP8, VP9, or AV1 encoding, each on the GPU where the card carries it. - **Zero copy GPU encoding.** On the Wayland stack, frames are rendered and encoded on the GPU without ever touching system RAM, for Intel, AMD, and Nvidia hardware. - **Runs anywhere.** The CPU encoding path is efficient enough to serve 1080p60 sessions from budget mini PCs and ARM boards. A GPU is optional, not required. - **A complete client, not just video.** Audio in both directions, clipboard sync, file upload and download, gamepad passthrough for up to four players, touch and virtual trackpad support for mobile, IME input, and multi user session sharing. @@ -61,7 +61,7 @@ From the browser down to the application: | Client | Selkies web client (dashboard) | Renders video with WebCodecs, plays Opus audio, captures input, provides the sidebar UI, file transfer, clipboard, gamepads, and sharing | | Transport | WebSockets over HTTPS | Binary video, audio, and input messages, fronted by Nginx inside the container | | Server | Selkies (Python) | Session orchestration: wires capture to the socket, injects input, manages clipboard, files, and settings | -| Video | Pixelflux (Rust with Python bindings) | Captures the framebuffer, detects damage, encodes H.264 or JPEG, CPU or GPU. In Wayland mode pixelflux itself hosts the compositor | +| Video | Pixelflux (Rust with Python bindings) | Captures the framebuffer, detects damage, encodes H.264, H.265, VP8, VP9, AV1, or JPEG, CPU or GPU. In Wayland mode pixelflux itself hosts the compositor | | Audio | Pcmflux | Captures PulseAudio output and encodes Opus for the browser, plus microphone return | | Display server | Smithay based Wayland compositor (inside pixelflux) | Virtual framebuffer in userspace, on GPU or CPU, replaces Xvfb from the X11 era | | Window management | labwc (single apps) or KDE Plasma (desktops) | Window decoration, tiling, desktop shell | @@ -117,3 +117,4 @@ The platform ships in two flavors that share the same machinery: - The **User Guide** is for people running the prebuilt containers. - The **Developer Guide** is for people building images on top of the baseimages or integrating the underlying libraries. - The **Components** section is a map of the subprojects that make up the whole platform, with a page for each. +- The upstream project lives at [selkies.io](https://selkies.io/) and its own reference documentation at [docs.selkies.io](https://docs.selkies.io/). These pages describe the platform as LinuxServer.io packages and ships it. diff --git a/docs/selkies/user-guide/.pages b/docs/selkies/user-guide/.pages index 96319286..331b45ff 100644 --- a/docs/selkies/user-guide/.pages +++ b/docs/selkies/user-guide/.pages @@ -9,4 +9,5 @@ nav: - Installing Applications: installing-apps.md - Security and Hardening: security.md - Reverse Proxy: reverse-proxy.md + - WebRTC Transport: webrtc.md - Troubleshooting: troubleshooting.md diff --git a/docs/selkies/user-guide/configuration.md b/docs/selkies/user-guide/configuration.md index 5762c143..d79ac9d3 100644 --- a/docs/selkies/user-guide/configuration.md +++ b/docs/selkies/user-guide/configuration.md @@ -3,7 +3,7 @@ Every Selkies based container is configured through environment variables. This page collects all of them in one place. They fall into three groups: 1. **Container variables**: standard LinuxServer.io conventions plus Selkies baseimage options (ports, auth, GPU, language). -2. **Selkies application settings** (`SELKIES_*`): stream, client UI, and feature toggles, with a locking syntax. +2. **Selkies application settings** (`SELKIES_*`): transport, stream, client UI, and feature toggles, with a locking syntax. 3. **Hardening variables**: lockdown options covered in detail on the [Security page](security.md). ## Standard LinuxServer variables @@ -26,23 +26,25 @@ These work in every LinuxServer.io container: | `PELORUS` | If set to true, the [Pelorus](../components/pelorus.md) agentic interface and accessibility stack are started alongside the session | | `CUSTOM_PORT` | Internal HTTP port, default `3000` | | `CUSTOM_HTTPS_PORT` | Internal HTTPS port, default `3001` | -| `CUSTOM_WS_PORT` | Internal WebSocket port, default `8082` | +| `CUSTOM_WS_PORT` | Internal port the Selkies server listens on behind Nginx, default `8082` | | `CUSTOM_USER` | HTTP basic auth username, default `abc` | | `PASSWORD` | HTTP basic auth password, default `abc`. If unset there is no auth | | `DRI_NODE` | Encoding GPU, enables VAAPI/NVENC stream encoding on the given device, e.g. `/dev/dri/renderD128` | | `DRINODE` | Rendering GPU for EGL and 3D acceleration, e.g. `/dev/dri/renderD128` | -| `AUTO_GPU` | Automatic GPU configuration when one is detected, first available GPU used for encoding and rendering. Set `false` to disable | +| `AUTO_GPU` | Automatic GPU configuration when one is detected, first available GPU used for encoding and rendering. Set `false` to disable, or a vendor or driver name such as `nvidia`, `amdgpu`, or `intel` to pick a specific GPU on multi GPU hosts | | `PIXELFLUX_CU` | Port to enable the Computer Use API server for AI agent control of the desktop, Wayland mode only | | `SUBFOLDER` | Subfolder when running behind a subfolder reverse proxy, needs both slashes, e.g. `/subfolder/` | | `TITLE` | Page title shown in the browser, default `Selkies` | -| `DASHBOARD` | Select the web client dashboard: `selkies-dashboard`, `selkies-dashboard-zinc`, or `selkies-dashboard-wish` | +| `DASHBOARD` | Select the web client dashboard: `selkies-dashboard` or `selkies-dashboard-wish` | | `FILE_MANAGER_PATH` | Change the default upload and download path, must be writable by the `abc` user | | `START_DOCKER` | If `false`, a privileged container will not automatically start the Docker in Docker setup | | `DISABLE_IPV6` | Set to `true` or any value to disable IPv6 | | `LC_ALL` | Session language, e.g. `fr_FR.UTF-8`, see [Internationalization](#internationalization) | | `NO_DECOR` | Run the application without window borders, for PWA style use. Toggle at runtime with `ctrl+shift+d` | | `NO_FULL` | Do not automatically fullscreen applications when using the single app window manager | -| `NO_GAMEPAD` | Disable the userspace gamepad interposer injection | +| `NO_GAMEPAD` | Disable the userspace gamepad interposer injection. Also turns off `SELKIES_GAMEPAD_ENABLED`, the player 2 to 4 sharing links, and hides the gamepad section of the sidebar | +| `NO_STEAM` | Remove the built in Steam installer, see [Installing Applications](installing-apps.md#steam-built-in-reinstalls-itself) | +| `NO_WEBCAM` | Disable the virtual webcam. Without it the container creates `/dev/video0`, preloads the V4L2 interposer, and turns on `SELKIES_WEBCAM_ENABLED` so the browser can forward a camera into the session | | `DISABLE_ZINK` | Do not set Zink variables when a GPU is detected, applications use CPU rendering | | `DISABLE_DRI3` | X11 mode only, disable DRI3 acceleration | | `MAX_RES` | Maximum virtual display resolution, default 16K (`15360x8640`) | @@ -51,99 +53,317 @@ These work in every LinuxServer.io container: **`WATERMARK_LOCATION` values:** `1` top left, `2` top right, `3` bottom left, `4` bottom right, `5` centered, `6` animated. +### How the baseimage feeds Selkies + +The Selkies server reads its own `SELKIES_*` variables, and the baseimage's init scripts translate the container level variables above into them. A few of those translations are worth knowing because they change the defaults you would otherwise read off the upstream project: + +| Selkies setting | Container default | Upstream default | Why | +| --- | --- | --- | --- | +| `SELKIES_ENCODER` | `h264enc,h265enc,vp8enc,vp9enc,av1enc,jpeg` | `h264enc,h265enc,vp8enc,vp9enc,av1enc,h264enc-striped,jpeg` | The striped H.264 encoder is left out of the sidebar menu. Selkies then drops any encoder this host cannot serve, see [Video encoders](#video-encoders) | +| `SELKIES_VIDEO_STREAMING_MODE` | `false` | `true` | Desktop use favors damage tracking and paint over. Turn it on for gaming and video | +| `SELKIES_ENABLE_BASIC_AUTH` | `false` | `true` | Nginx handles the login using `CUSTOM_USER` and `PASSWORD`, so the Selkies server's own basic auth stays off | +| `SELKIES_ALLOWED_ORIGINS` | `*` | same origin only | Nginx fronts the server, so the cross origin guard is relaxed inside the container | +| `SELKIES_COMMAND_ENABLED` | `true` | `false` | The sidebar apps and launcher section depends on command messages. `HARDEN_DESKTOP=true` flips it back to `false` | +| `SELKIES_ENABLE_DUAL_MODE` | `false` | `true` | The WebSocket / WebRTC switch is hidden until you configure WebRTC, see [WebRTC Transport](webrtc.md) | +| `SELKIES_MODE` | `websockets` | `websockets` | Becomes `webrtc` automatically when any WebRTC, STUN, or TURN variable is set | +| `SELKIES_WEBCAM_ENABLED` | `true` | `false` | Turned on when the virtual webcam device can be created, `NO_WEBCAM` prevents it | +| `CUSTOM_WS_PORT` | `8082` | `8080` | Port the Selkies server listens on behind Nginx | + +`HARDEN_DESKTOP=true` also sets `SELKIES_FILE_TRANSFERS` to empty, turns `SELKIES_PRINTING_ENABLED` off, and hides the files and apps sidebar sections unless you set those variables yourself. Anything you pass explicitly always wins over these defaults. + ## Selkies application settings -Every facet of the streaming application can be configured with `SELKIES_*` variables. These also drive what the user can change in the sidebar UI. +Every facet of the streaming application can be configured with `SELKIES_*` variables. These also drive what the user can change in the sidebar UI: the server sends the resolved settings schema to the client, so a locked or single valued setting simply has no control to change. + +Each setting is also a CLI flag with the same name, `SELKIES_VIDEO_CRF` is `--video-crf`. Precedence is CLI flag, then the `SELKIES_*` variable, then the legacy container variable where one exists (`PASSWORD`, `DRI_NODE`, and so on), then the built in default. ### Value syntax -- **Booleans and locking.** Booleans accept `true` or `false`. Append `|locked` to prevent the user changing the setting in the UI: `-e SELKIES_USE_CPU="true|locked"` -- **Enums and lists.** Comma separated values, the first item is the default. A single item hides the UI dropdown entirely: `-e SELKIES_ENCODER="jpeg"` -- **Ranges.** `min-max` renders a slider, a single number locks the value: `-e SELKIES_FRAMERATE="60"` -- **Manual resolution.** Setting `SELKIES_MANUAL_WIDTH` or `SELKIES_MANUAL_HEIGHT` locks the resolution to those values. +- **Booleans and locking.** `true` or `1` (case insensitive) is on, anything else is off. Append `|locked` to prevent the user changing the setting in the UI: `-e SELKIES_USE_CPU="true|locked"` +- **Enums and lists.** Comma separated values, the first item is the default and the full list is what the user may pick from. A single item hides the UI dropdown entirely: `-e SELKIES_ENCODER="jpeg"`. Matching is case insensitive and invalid items are dropped; if nothing valid is left the built in menu is kept. +- **Ranges.** Three forms. `min-max` restricts the allowed span and keeps the built in initial value: `SELKIES_FRAMERATE="8-120"`. A bare value sets the initial value and keeps the built in span: `SELKIES_FRAMERATE="60"`. Both at once: `SELKIES_FRAMERATE="60,8-120"`. A degenerate span such as `"60-60"` locks the setting. +- **Empty means default.** Setting a variable to `""` means "use the built in default", except for list type settings where `""` or `none` means disabled (for example `SELKIES_FILE_TRANSFERS`). +- **Manual resolution.** Setting `SELKIES_MANUAL_WIDTH` or `SELKIES_MANUAL_HEIGHT` to a positive value forces manual resolution mode. + +### Legacy variables + +These names are no longer Selkies settings, but the containers still accept them. At startup the init script copies a legacy value into its current equivalent when you have not set the current name yourself, so existing compose files keep working. Prefer the current names for anything new. + +| Legacy name | Current name | Notes | +| --- | --- | --- | +| `SELKIES_H264_CRF` | `SELKIES_VIDEO_CRF` | | +| `SELKIES_H264_FULLCOLOR` | `SELKIES_VIDEO_FULLCOLOR` | | +| `SELKIES_H264_STREAMING_MODE` | `SELKIES_VIDEO_STREAMING_MODE` | The legacy value overrides the container default of `false` | +| `SELKIES_H264_PAINTOVER_CRF` | `SELKIES_VIDEO_PAINTOVER_CRF` | | +| `SELKIES_H264_PAINTOVER_BURST_FRAMES` | `SELKIES_VIDEO_PAINTOVER_BURST_FRAMES` | | +| `SELKIES_IS_MANUAL_RESOLUTION_MODE` | `SELKIES_MANUAL_RESOLUTION` | | +| `SELKIES_CLIPBOARD_ENABLED` | `SELKIES_ENABLE_CLIPBOARD` | The current setting is a policy (`true`, `in`, `out`, `false`). A legacy `|locked` suffix is dropped | +| `x264enc` and `x264enc-striped` as `SELKIES_ENCODER` values | `h264enc` and `h264enc-striped` | Rewritten in place, the rest of the list is kept | +| `openh264enc` as a `SELKIES_ENCODER` value | `h264enc` | Selkies itself maps it. Software H.264 is whatever encoder the pixelflux build carries, x264 in the wheels the containers ship | + +### Transport + +| Variable | Default | Description | +| --- | --- | --- | +| `SELKIES_MODE` | `websockets` | Streaming transport, `websockets` or `webrtc`. The containers switch this to `webrtc` automatically when any WebRTC variable is set, see [WebRTC Transport](webrtc.md) | +| `SELKIES_ENABLE_DUAL_MODE` | `false` (upstream `true`) | Show the transport switch in the UI so users can move between WebSockets and WebRTC at runtime | +| `SELKIES_BACKPRESSURE_QUEUE_SIZE` | `120` | WebSockets mode only. Max frames or audio chunks buffered per stream before dropping under backpressure, `1` to `100000`. Higher tolerates larger client hiccups at the cost of latency | +| `SELKIES_WEBRTC_PACER` | `true` | WebRTC mode only. Pace outgoing packets with strict priorities (audio and RTCP, then data channel, then video) so audio and input stay responsive when video bursts on a congested link. `SELKIES_WEBRTC_PACER_STALE_MS` sets the stale GOP purge deadline in milliseconds, `0` disables | +| `SELKIES_CONGESTION_CONTROL` | `false` | WebRTC mode only. Adapt the video bitrate to the bandwidth estimate from receiver feedback. Effective in CBR rate control mode, may trade quality for responsiveness | + +### Video encoders + +`SELKIES_ENCODER` is a comma separated menu of the values below. The first item is the default and the whole list is what the sidebar offers. Every full frame codec is encoded on the GPU when the encoding device has an engine for it, and by the software encoder in the pixelflux build otherwise, so the same menu works on a CPU only host, an Intel or AMD card, and an Nvidia card. The container default is `h264enc,h265enc,vp8enc,vp9enc,av1enc,jpeg`. + +| Value | Codec | Hardware | Software | Shape | FullColor 4:4:4 | WebRTC | Browser decode | +| --- | --- | --- | --- | --- | --- | --- | --- | +| `h264enc` | H.264 | NVENC, VA-API | x264 | Full frame | NVENC and x264. VA-API has no 4:4:4 H.264 profile on current drivers, so the request is honored on the CPU instead | Yes | Every browser | +| `h265enc` | H.265 (HEVC) | NVENC, VA-API | x265 | Full frame | NVENC and x265, VA-API negotiates per device | Yes | Safari, and Chromium where the operating system supplies an HEVC decoder. Chrome and Firefox on Linux decode no HEVC | +| `vp8enc` | VP8 | VA-API, on the few GPUs that still carry a VP8 engine | libvpx | Full frame | No | Yes | Every browser | +| `vp9enc` | VP9 | VA-API | libvpx | Full frame | Profile 1 on VA-API and libvpx | Yes | Chromium, Firefox | +| `av1enc` | AV1 | NVENC on Ada and newer, VA-API on GPUs with an AV1 engine | SVT-AV1 | Full frame | No | Yes | Chromium, Firefox. WebKit refuses it at decode time | +| `h264enc-striped` | H.264 | None, always CPU | x264 | Striped, one stripe per core | x264 | No | Every browser | +| `jpeg` | JPEG | None, always CPU | libjpeg-turbo | Striped, one stripe per core | Always full color | No | Every browser, including ones without WebCodecs | + +Nvidia has no VP8 or VP9 encode engine, so those two codecs are software on every Nvidia card. AMD has no VP8 or VP9 engine either, and AV1 starts with RDNA 3. Intel carries VP9 and, on Arc and newer integrated graphics, AV1. Run `vainfo` inside the container to see the encode entry points your card exposes. + +How the menu behaves at runtime: + +- **The host trims the menu.** At startup Selkies probes the encoding device once and drops every encoder that neither the GPU nor the pixelflux software build can serve, logging `Encoders not served on this host are left off the menu`. If the default itself is not served the session falls back to `h264enc`. The wheels in the containers carry a software encoder for all five codecs, so on a CPU only host the full menu still appears, all of it in software. +- **The browser greys out what it cannot decode.** Encoders the browser has no decoder for stay in the menu disabled and labelled `(Unsupported Browser)`. A browser that cannot decode the server default steps through the allowed list in the order `h264enc`, `h264enc-striped`, `vp9enc`, `vp8enc`, `av1enc`, `h265enc`, `jpeg` until one plays. A shared viewer, or a browser facing a single valued locked encoder, gets an on page error instead. +- **Only H.264 and JPEG are striped.** H.265, VP8, VP9, and AV1 always encode the whole frame, so they never get the per core parallelism or the dirty stripe savings of the H.264 and JPEG paths. In software they also cost several times the CPU of x264. On a machine whose GPU lacks the engine, keep `h264enc` first and treat the other codecs as options for GPU hosts. +- **Software encoding toggle.** The sidebar's CPU encoding switch only appears for a codec the host serves both ways, since it moves the session between the GPU and the software encoder and would be a no-op otherwise. `SELKIES_USE_CPU` still applies to every full frame codec. +- **Demotion.** If pixelflux still cannot serve the codec once capture starts it demotes the display to H.264 and Selkies rewrites the encoder setting to match, logging `streams H.264 as 'h264enc': no encoder served`. Trust the encoder shown in the sidebar over the one you configured. +- **Colors.** VP8 cannot signal BT.709, so it is encoded and declared BT.601 and looks very slightly different from the other codecs. Firefox paints AV1 received over WebRTC as BT.601 for the same reason. + +### Video encoding + +| Variable | Default | Description | +| --- | --- | --- | +| `SELKIES_ENCODER` | `h264enc,h265enc,vp8enc,vp9enc,av1enc,jpeg` (upstream adds `h264enc-striped`) | Menu of video encoders, first is default, see [Video encoders](#video-encoders). Encoders the host cannot serve are dropped at startup, and the five full frame codecs are the ones that stream over WebRTC | +| `SELKIES_FRAMERATE` | `8-240`, initial `60` | Framerate range, initial value, or both | +| `SELKIES_RATE_CONTROL_MODE` | `crf` | Rate control for every video codec, `crf` (constant quality) or `cbr` (constant bitrate). WebRTC mode defaults to `cbr` unless you pin this, and so does a session on OpenH264 in a GPL free pixelflux build | +| `SELKIES_ENABLE_RATE_CONTROL` | `true` | Let the client pick the rate control mode. Set `false` to lock the encoder to CRF | +| `SELKIES_VIDEO_CRF` | `5-50`, initial `25` | CRF range, initial value, or both. Lower is higher quality. The value is on the H.264 QP scale and pixelflux maps it onto each codec's own quantizer range, so one number means the same quality on every encoder | +| `SELKIES_VIDEO_BITRATE` | `100-1000000`, initial `8000` | CBR bitrate in kbps: range, initial value, or both. `8000` is 8 Mbps | +| `SELKIES_VIDEO_MIN_QP` | `0` | CBR mode minimum quantizer on the H.264 QP scale, `0` to `51`, mapped onto each codec's own range. `0` is the encoder default. Raising it caps bit spend on easy content | +| `SELKIES_VIDEO_MAX_QP` | `0` | CBR mode maximum quantizer on the same scale, `0` is the encoder default. Lowering it keeps text legible under motion at the cost of overshooting the bitrate target | +| `SELKIES_KEYFRAME_INTERVAL` | `0` | Seconds between scheduled recovery keyframes, `0` to `300`. `0` keeps the GOP infinite and sends keyframes only on demand, which keeps bitrate and quality steady | +| `SELKIES_VIDEO_FULLCOLOR` | `false` | Encode 4:4:4 chroma instead of 4:2:0 where the codec and encoder carry it: H.264 and H.265 on NVENC, x264, and x265, VP9 profile 1 on VA-API and libvpx. VP8, AV1, and OpenH264 stay 4:2:0. A client whose decoder has no 4:4:4 profile turns it off for itself and keeps the codec; where it is locked, such a client steps to the next allowed encoder whose 4:4:4 it decodes, JPEG last. See the [GPU caveats](gpu.md#fullcolor-444-and-hardware-encoders) | +| `SELKIES_VIDEO_STREAMING_MODE` | `false` (upstream `true`) | Turbo mode: encode every frame like a traditional video stream instead of damage tracking. Useful for gaming and full motion video | +| `SELKIES_USE_CPU` | `false` | Force software encoding for the full frame codecs even when the encoding GPU serves them. The striped encoders are CPU regardless | +| `SELKIES_JPEG_QUALITY` | `1-100`, initial `40` | JPEG encoder quality range, initial value, or both | +| `SELKIES_USE_PAINT_OVER_QUALITY` | `true` | High quality paint over for static scenes | +| `SELKIES_PAINT_OVER_JPEG_QUALITY` | `1-100`, initial `90` | JPEG paint over quality range, initial value, or both | +| `SELKIES_VIDEO_PAINTOVER_CRF` | `5-50`, initial `18` | Paint over CRF range, initial value, or both, for every video codec. Must be lower than the CRF to trigger, and paint over is off by default under CBR | +| `SELKIES_VIDEO_PAINTOVER_BURST_FRAMES` | `1-30`, initial `5` | Paint over burst frames range, initial value, or both, for every video codec | +| `SELKIES_GPU_ID` | `''` | Hardware encoder GPU index, selects `/dev/dri/renderD{128 + n}` and the GPU stats index. Empty encodes on the first GPU or the one `AUTO_GPU` chose, `-1` disables hardware encoding. Ignored when `DRI_NODE` gives a device path | +| `SELKIES_ENCODE_DRI` (or `DRI_NODE`) | `''` | DRI render node the encoder uses for VA-API or NVENC | +| `SELKIES_RENDER_DRI` (or `DRINODE`) | `''` | DRI render node the Wayland compositor renders on, defaults to the `AUTO_GPU` pick, else software rendering | +| `SELKIES_AUTO_GPU` (or `AUTO_GPU`) | `true` | GPU auto selection for rendering: `true` picks the first GPU, `false` disables, or a vendor name, kernel driver name, devicetree prefix, or PCI vendor ID picks the first GPU it matches | +| `SELKIES_RECORDING_SOCKET` (or `PIXELFLUX_RECORDING_SOCKET`) | `''` | Unix socket path for an out of band recording tap, pixelflux multiplexes the elementary stream of the full frame codec to connected clients: Annex B for H.264 and H.265, OBU for AV1, IVF for VP8 and VP9. Not available on striped H.264 or JPEG. Empty is off | + +### Audio + +| Variable | Default | Description | +| --- | --- | --- | +| `SELKIES_AUDIO_BITRATE` | `128000` | Opus bitrate in bps. The UI offers `32000` through `510000` in steps, any value from `6000` to `510000` is accepted | +| `SELKIES_AUDIO_FRAME_DURATION_MS` | `10` | Opus frame duration: `2.5`, `5`, `10`, `20`, `40`, or `60`. Lower cuts audio latency at a small bandwidth and packet rate cost | +| `SELKIES_AUDIO_REDUNDANCY` | `true` | Opus RED (RFC 2198) redundancy to cut dropouts under packet loss. On WebSockets it only engages when every connected client supports it | +| `SELKIES_AUDIO_REDUNDANCY_DISTANCE` | `2` | Number of prior Opus frames carried as redundancy, `0` to `4`. Higher survives longer loss bursts at proportionally more bandwidth | +| `SELKIES_AUDIO_CHANNELS` | `2` | Number of audio channels | +| `SELKIES_AUDIO_DEVICE_NAME` | `output.monitor` | PulseAudio source pcmflux captures | + +### Display and input + +| Variable | Default | Description | +| --- | --- | --- | +| `SELKIES_MANUAL_RESOLUTION` | `false` | Lock the resolution to the manual width and height | +| `SELKIES_MANUAL_WIDTH` | `0` | Fixed width up to `16384`, a positive value forces manual resolution mode | +| `SELKIES_MANUAL_HEIGHT` | `0` | Fixed height up to `16384`, a positive value forces manual resolution mode | +| `SELKIES_ENABLE_RESIZE` | `true` | Dynamically resize the display to match the browser window | +| `SELKIES_SCALING_DPI` | `96` | Default DPI for UI scaling, `96` to `288` in steps of `24` | +| `SELKIES_FORCE_ALIGNED_RESOLUTION` | `false` | Forces the display resolution to be a multiple of 16 pixels | +| `SELKIES_USE_CSS_SCALING` | `false` | HiDPI when false. When true a lower resolution is sent and the canvas is stretched | +| `SELKIES_SECOND_SCREEN` | `true` | Offer the Add Screen button for a second monitor. Set `false` to hide it. On Wayland the button only appears when Selkies detects compositor support (labwc or KWin) regardless of this value | +| `SELKIES_ENABLE_CURSORS` | `true` | Send the remote application cursor to the client | +| `SELKIES_USE_BROWSER_CURSORS` | `true` | Use browser CSS cursors instead of rendering the cursor onto the canvas | +| `SELKIES_CURSOR_SIZE` (or `XCURSOR_SIZE`) | `-1` | Cursor size in points at 96 DPI, scaled with the session DPI. `-1` is the platform default, 32 on X11 and 24 on Wayland | +| `SELKIES_MAC_CMD_AS_CTRL` | `true` | macOS clients send Command chords as Control, so Cmd+C copies remotely. Set `false` when the session's window manager binds Super itself, Command then arrives as Super. Users may override unless locked | +| `SELKIES_KEYBOARD_SHORTCUTS` | `true` | Keep the client's own chords (Ctrl+Shift with F, M, X, or G, and Ctrl+Shift+click) for the sidebar instead of passing them to the session. Set `false` to pass them through; the sidebar buttons still work and three presses of Escape still leave gaming mode. Users may override unless locked | +| `SELKIES_PUBLISH_INPUT_DEVICES` | `false` | Mirror the session keyboard and pointer onto evdev input devices for applications that enumerate evdev directly, such as fullscreen games and remappers. Uses a kernel uinput device where `/dev/uinput` is writable, else the input interposer | +| `SELKIES_RAW_POINTER_MOTION` | `true` | Ask the browser for unaccelerated pointer movement under pointer lock (gaming mode). Windows and macOS honor it, Linux and Android do not. Clients on macOS leave it off unless chosen. Users may override unless locked | +| `SELKIES_DEBUG_CURSORS` | `false` | Cursor debug logging | + +### Feature toggles and session start state -### Stream settings +The `*_ENABLED` variables decide what the server offers at all. The `*_ON_START` variables decide whether a feature is already running when a client connects, or waits for the user to press its sidebar button. | Variable | Default | Description | | --- | --- | --- | -| `SELKIES_ENCODER` | `'x264enc,x264enc-striped,jpeg'` | Available video encoders, first is default | -| `SELKIES_FRAMERATE` | `'8-120'` | Framerate range or fixed value | -| `SELKIES_H264_CRF` | `'5-50'` | H.264 CRF range or fixed value, lower is higher quality | -| `SELKIES_JPEG_QUALITY` | `'1-100'` | JPEG quality range or fixed value | -| `SELKIES_H264_FULLCOLOR` | `False` | H.264 full color 4:4:4 range for pixelflux encoders | -| `SELKIES_H264_STREAMING_MODE` | `False` | H.264 streaming mode for pixelflux encoders | -| `SELKIES_FORCE_ALIGNED_RESOLUTION` | `False` | Forces the display resolution to be a multiple of 16 pixels. | -| `SELKIES_USE_CPU` | `False` | Force CPU encoding | -| `SELKIES_USE_PAINT_OVER_QUALITY` | `True` | High quality paint over for static scenes | -| `SELKIES_PAINT_OVER_JPEG_QUALITY` | `'1-100'` | JPEG paint over quality range or fixed value | -| `SELKIES_H264_PAINTOVER_CRF` | `'5-50'` | H.264 paint over CRF range or fixed value | -| `SELKIES_H264_PAINTOVER_BURST_FRAMES` | `'1-30'` | H.264 paint over burst frames range or fixed value | -| `SELKIES_SECOND_SCREEN` | `True` | Support for a second monitor | -| `SELKIES_AUDIO_BITRATE` | `'320000'` | Default audio bitrate | -| `SELKIES_IS_MANUAL_RESOLUTION_MODE` | `False` | Lock resolution to the manual width and height | -| `SELKIES_MANUAL_WIDTH` | `0` | Fixed width, setting this forces manual resolution mode | -| `SELKIES_MANUAL_HEIGHT` | `0` | Fixed height, setting this forces manual resolution mode | -| `SELKIES_SCALING_DPI` | `'96'` | Default DPI for UI scaling | -| `SELKIES_USE_BROWSER_CURSORS` | `False` | Use browser CSS cursors instead of canvas rendering | -| `SELKIES_USE_CSS_SCALING` | `False` | HiDPI when false. When true a lower resolution is sent and the canvas is stretched | +| `SELKIES_AUDIO_ENABLED` | `true` | Server to client audio streaming. Disabling it also disables the microphone | +| `SELKIES_MICROPHONE_ENABLED` | `false` | Client to server microphone forwarding | +| `SELKIES_WEBCAM_ENABLED` | `true` (upstream `false`) | Client to server webcam forwarding into the virtual V4L2 device. Set by the baseimage unless `NO_WEBCAM` is present | +| `SELKIES_GAMEPAD_ENABLED` | `true` | Gamepad support. Forced off by `NO_GAMEPAD` | +| `SELKIES_VIDEO_ON_START` | `true` | Start with video on. Off, nothing is captured for the primary display until the user turns video on; shared viewers and second screens always start their stream | +| `SELKIES_AUDIO_ON_START` | `true` | Start with audio on. Off, capture stays stopped until the user turns audio on. Unlike `SELKIES_AUDIO_ENABLED=false` nothing is torn down and the microphone keeps working | +| `SELKIES_MICROPHONE_ON_START` | `false` | Start with the microphone on, so the browser asks for the device as soon as the session connects | +| `SELKIES_WEBCAM_ON_START` | `false` | Start with the webcam on, so the browser asks for the camera as soon as the session connects | +| `SELKIES_GAMEPAD_ON_START` | `true` | Start with gamepad input on. The user's choice is remembered by the browser and takes precedence on later visits | +| `SELKIES_COMMAND_ENABLED` | `true` (upstream `false`) | Parsing of command messages from the client, which the sidebar apps section needs. `HARDEN_DESKTOP` turns it off | +| `SELKIES_DEBUG` | `false` | Debug logging | -### Feature toggles +### Clipboard and files | Variable | Default | Description | | --- | --- | --- | -| `SELKIES_AUDIO_ENABLED` | `True` | Server to client audio streaming | -| `SELKIES_MICROPHONE_ENABLED` | `True` | Client to server microphone forwarding | -| `SELKIES_GAMEPAD_ENABLED` | `True` | Gamepad support | -| `SELKIES_CLIPBOARD_ENABLED` | `True` | Clipboard synchronization | -| `SELKIES_ENABLE_BINARY_CLIPBOARD` | `False` | Allow binary data on the clipboard | -| `SELKIES_COMMAND_ENABLED` | `True` | Parsing of command websocket messages | -| `SELKIES_FILE_TRANSFERS` | `'upload,download'` | Allowed transfer directions, comma separated. Empty or `none` disables | -| `SELKIES_DEBUG` | `False` | Debug logging | +| `SELKIES_ENABLE_CLIPBOARD` | `true` | Clipboard policy: `true` both directions, `in` client to server only, `out` server to client only, `false` disabled. `out` is what stops the page reading the local clipboard at all, which is the read Firefox and Safari raise a paste prompt for | +| `SELKIES_ENABLE_BINARY_CLIPBOARD` | `true` | Allow binary data such as images on the clipboard. Rich text pasted from the browser into the session follows the same switch | +| `SELKIES_CLIPBOARD_SEAMLESS` | `true` | Sync the clipboard automatically on every copy on either side. Set `false` and the clipboard only moves through the sidebar's clipboard box. The direction policy above still bounds it. Users may override unless locked | +| `SELKIES_FILE_TRANSFERS` | `upload,download` | Allowed transfer directions, comma separated. Empty or `none` disables | +| `SELKIES_FILE_TRANSFER_LIMIT_MBPS` | `0` | Static throttle in Mbit/s shared by all uploads and downloads, for links whose rate you know. `0` disables. Transfers are already paced to protect the video stream without it | +| `SELKIES_FILE_MANAGER_PATH` (or `FILE_MANAGER_PATH`) | `~/Desktop` | Directory uploads land in and the file browser serves, created at startup if missing | -### Sharing toggles +### Webcam + +The containers create a virtual `/dev/video0` and preload a V4L2 interposer so ordinary applications see the browser's camera as a normal webcam. Set `NO_WEBCAM` to skip all of it. + +| Variable | Default | Description | +| --- | --- | --- | +| `SELKIES_WEBCAM_WIDTH` | `1280` | Width of the virtual webcam device, client frames are scaled and letterboxed to fit | +| `SELKIES_WEBCAM_HEIGHT` | `720` | Height of the virtual webcam device | +| `SELKIES_WEBCAM_PIXEL_FORMAT` | `auto` | Pixel format of the virtual device. `auto` follows the uplink (MJPEG for a browser sending JPEG, otherwise I420). Or pin `I420`, `NV12`, `YUYV`, or `MJPEG` | +| `SELKIES_WEBCAM_ENCODER` | `auto` | Codec the browser uses for the camera uplink: `auto`, `h264`, `h265`, `vp8`, `vp9`, `av1`, or `mjpeg`. Over WebSockets `auto` tries H.264, then VP8, then VP9, AV1, or H.265 where the browser encodes them, and JPEG where nothing keeps up; a codec name pins it, still dropping to JPEG when it cannot keep up. Over WebRTC the browser sends the named codec when the session negotiated it. Users may override unless locked | +| `SELKIES_WEBCAM_DEVICE` | `auto` | Also mirror the webcam into a v4l2loopback kernel device: `auto` uses the first one found (usually only on a host or privileged container), a path such as `/dev/video10` uses that device, `false` never does | +| `SELKIES_WEBCAM_SOCKET_PATH` | `/tmp` | Directory for the V4L2 interposer socket, `selkies_webcam0.sock` | + +### Gamepads + +| Variable | Default | Description | +| --- | --- | --- | +| `SELKIES_JS_SOCKET_PATH` | `/tmp` | Directory for the input interposer sockets, `selkies_js{0-3}.sock` | +| `SELKIES_UINPUT_GAMEPAD` | `auto` | Register gamepads as kernel devices through `/dev/uinput`, which Steam, Proton, and browsers inside the session find without the interposer. `auto` only does so where the interposer is not configured and `/dev/uinput` is writable, `true` always attempts it, `false` never does | +| `SELKIES_UINPUT_MOUSE_SOCKET` | `''` | Path to a uinput mouse socket, if not provided uinput is used directly | + +### Printing and audit + +The session has a printer named `Selkies`. Selkies runs the CUPS queue itself as the session user, the containers ship `cups-daemon` and `cups-filters` and point `CUPS_SERVER` at it, and every printed document is turned into a PDF and handed to the browser, which opens its own print dialog. Documents also collect in the sidebar's printing section for reprinting or saving. Selkies can also post an audit event for each clipboard transfer, file transfer, print, connect, disconnect, and recording. + +| Variable | Default | Description | +| --- | --- | --- | +| `SELKIES_PRINTING_ENABLED` | `true` | Offer the `Selkies` printer to the session and hand each document to the browser. `false` runs no queue and hides the printing section. `HARDEN_DESKTOP` turns it off unless you set it yourself. Documents go to the page holding the session, shared viewers receive none | +| `SELKIES_PRINT_SPOOL_PATH` | `~/.local/state/selkies/print` | Directory finished print jobs land in as PDFs until a page takes them | +| `SELKIES_AUDIT_WEBHOOK_URL` | `''` | URL that receives one JSON POST per audit event. Metadata only, never content. Empty is off, and a queue of 1024 pending events drops on overflow with no retry | +| `SELKIES_AUDIT_WEBHOOK_TOKEN` | `''` | Bearer token sent in the `Authorization` header of every audit POST | +| `SELKIES_AUDIT_WEBHOOK_TIMEOUT` | `2.0` | Seconds one audit POST may take before it counts as failed and the next one is sent | + +### Sharing | Variable | Default | Description | | --- | --- | --- | -| `SELKIES_ENABLE_SHARING` | `True` | Master toggle for all sharing features | -| `SELKIES_ENABLE_COLLAB` | `True` | Collaborative read write sharing link | -| `SELKIES_ENABLE_SHARED` | `True` | View only sharing links | -| `SELKIES_ENABLE_PLAYER2` | `True` | Sharing link for gamepad player 2 | -| `SELKIES_ENABLE_PLAYER3` | `True` | Sharing link for gamepad player 3 | -| `SELKIES_ENABLE_PLAYER4` | `True` | Sharing link for gamepad player 4 | +| `SELKIES_ENABLE_SHARING` | `true` | Master toggle for all sharing features | +| `SELKIES_ENABLE_COLLAB` | `true` | Let a viewer holding the session's master key token act as a read write collaborator. Secure mode only, this is not a sharing link of its own | +| `SELKIES_ENABLE_SHARED` | `true` | View only sharing links | +| `SELKIES_ENABLE_PLAYER2` | `true` | Sharing link for gamepad player 2. Forced off by `NO_GAMEPAD` | +| `SELKIES_ENABLE_PLAYER3` | `true` | Sharing link for gamepad player 3. Forced off by `NO_GAMEPAD` | +| `SELKIES_ENABLE_PLAYER4` | `true` | Sharing link for gamepad player 4. Forced off by `NO_GAMEPAD` | +| `SELKIES_MASTER_TOKEN` | `''` | Master token that enables secure mode and protects the token control plane API, used by [SealSkin](../components/sealskin.md) | +| `SELKIES_BASIC_AUTH_VIEWONLY_PASSWORD` (or `VIEWONLY_PASSWORD`) | `''` | Optional second basic auth password that grants view only access when the Selkies server's own basic auth is on. Ignored in secure mode | ### Client UI visibility | Variable | Default | Description | | --- | --- | --- | -| `SELKIES_UI_TITLE` | `'Selkies'` | Title in the top left corner of the sidebar | -| `SELKIES_UI_SHOW_LOGO` | `True` | Show the Selkies logo in the sidebar | -| `SELKIES_UI_SHOW_SIDEBAR` | `True` | Show the main sidebar UI | -| `SELKIES_UI_SHOW_CORE_BUTTONS` | `True` | Show display, audio, microphone, and gamepad core buttons | -| `SELKIES_UI_SIDEBAR_SHOW_VIDEO_SETTINGS` | `True` | Video settings section | -| `SELKIES_UI_SIDEBAR_SHOW_SCREEN_SETTINGS` | `True` | Screen settings section | -| `SELKIES_UI_SIDEBAR_SHOW_AUDIO_SETTINGS` | `True` | Audio settings section | -| `SELKIES_UI_SIDEBAR_SHOW_STATS` | `True` | Stats section | -| `SELKIES_UI_SIDEBAR_SHOW_CLIPBOARD` | `True` | Clipboard section | -| `SELKIES_UI_SIDEBAR_SHOW_FILES` | `True` | File transfer section | -| `SELKIES_UI_SIDEBAR_SHOW_APPS` | `True` | Applications section | -| `SELKIES_UI_SIDEBAR_SHOW_SHARING` | `True` | Sharing section | -| `SELKIES_UI_SIDEBAR_SHOW_GAMEPADS` | `True` | Gamepads section | -| `SELKIES_UI_SIDEBAR_SHOW_FULLSCREEN` | `True` | Fullscreen button | -| `SELKIES_UI_SIDEBAR_SHOW_GAMING_MODE` | `True` | Gaming mode button | -| `SELKIES_UI_SIDEBAR_SHOW_TRACKPAD` | `True` | Virtual trackpad button | -| `SELKIES_UI_SIDEBAR_SHOW_KEYBOARD_BUTTON` | `True` | On screen keyboard button in the display area | -| `SELKIES_UI_SIDEBAR_SHOW_SOFT_BUTTONS` | `True` | Soft buttons section | +| `SELKIES_UI_TITLE` | `Selkies` | Title in the top left corner of the sidebar | +| `SELKIES_UI_SHOW_LOGO` | `true` | Show the Selkies logo in the sidebar | +| `SELKIES_UI_SHOW_SIDEBAR` | `true` | Show the main sidebar UI | +| `SELKIES_UI_SHOW_CORE_BUTTONS` | `true` | Show the display, audio, microphone, webcam, and gamepad core buttons | +| `SELKIES_UI_SIDEBAR_SHOW_VIDEO_SETTINGS` | `true` | Video settings section | +| `SELKIES_UI_SIDEBAR_SHOW_SCREEN_SETTINGS` | `true` | Screen settings section | +| `SELKIES_UI_SIDEBAR_SHOW_AUDIO_SETTINGS` | `true` | Audio settings section | +| `SELKIES_UI_SIDEBAR_SHOW_STATS` | `true` | Stats section | +| `SELKIES_UI_SIDEBAR_SHOW_SHORTCUTS` | `true` | Keyboard shortcuts section | +| `SELKIES_UI_SIDEBAR_SHOW_CLIPBOARD` | `true` | Clipboard section | +| `SELKIES_UI_SIDEBAR_SHOW_FILES` | `true` | File transfer section. `HARDEN_DESKTOP` hides it | +| `SELKIES_UI_SIDEBAR_SHOW_APPS` | `true` | Applications section. `HARDEN_DESKTOP` hides it | +| `SELKIES_UI_SIDEBAR_SHOW_SHARING` | `true` | Sharing section | +| `SELKIES_UI_SIDEBAR_SHOW_GAMEPADS` | `true` | Gamepads section. `NO_GAMEPAD` hides it | +| `SELKIES_UI_SIDEBAR_SHOW_WEBCAM` | `true` | Webcam toggle among the core buttons. Hides the control only, `SELKIES_WEBCAM_ENABLED` governs whether the server accepts frames | +| `SELKIES_UI_SIDEBAR_SHOW_FULLSCREEN` | `true` | Fullscreen button | +| `SELKIES_UI_SIDEBAR_SHOW_GAMING_MODE` | `true` | Gaming mode button | +| `SELKIES_UI_SIDEBAR_SHOW_TRACKPAD` | `true` | Virtual trackpad button | +| `SELKIES_UI_SIDEBAR_SHOW_KEYBOARD_BUTTON` | `true` | On screen keyboard button in the display area | +| `SELKIES_UI_SIDEBAR_SHOW_SOFT_BUTTONS` | `true` | Soft buttons section | + +### Server and diagnostics + +The container's built in Nginx owns the listening ports, TLS, basic auth, and the `SUBFOLDER` prefix, and the init scripts hand the matching values to Selkies. Use the container variables (`CUSTOM_PORT`, `CUSTOM_HTTPS_PORT`, `CUSTOM_WS_PORT`, `CUSTOM_USER`, `PASSWORD`, `SUBFOLDER`) rather than the Selkies server's own listener, HTTPS, and basic auth settings, which exist for running the `selkies` binary outside these containers. + +| Variable | Default | Description | +| --- | --- | --- | +| `SELKIES_ALLOWED_ORIGINS` | `*` (upstream same origin) | Comma separated browser Origins allowed to open the streaming WebSocket, a cross site WebSocket hijacking guard. Relaxed in the containers because Nginx fronts the server | +| `SELKIES_PUBLIC` | `false` | Listen on every interface instead of loopback. Leave it alone in the containers, Nginx is the public face and the server refuses to start when both this and `SELKIES_ADDR` are set | +| `SELKIES_ENABLE_METRICS_HTTP` | `false` | Prometheus metrics endpoint on the Selkies server | +| `SELKIES_ENABLE_WEBRTC_STATISTICS` | `false` | Dump WebRTC statistics CSVs from the client | +| `SELKIES_WEBRTC_STATISTICS_DIR` | `/tmp` | Directory for those CSVs, `selkies-stats-video-[timestamp].csv` and `selkies-stats-audio-[timestamp].csv` | + +### Display backend + +| Variable | Default | Description | +| --- | --- | --- | +| `SELKIES_WAYLAND` (or `PIXELFLUX_WAYLAND`) | `false` | Run the Wayland headless compositor backend instead of X11 capture and input. The containers set `PIXELFLUX_WAYLAND` for you on supported hardware | +| `SELKIES_APP_WAYLAND_DISPLAY` | `''` | Wayland socket applications run on when it differs from the capture compositor, for a nested session. Empty auto detects | +| `SELKIES_WAYLAND_HOST_DISPLAY` | `''` | Socket of an external compositor (labwc started headless, for example) that pixelflux captures and injects into as a client instead of compositing itself. Empty keeps the built in compositor | +| `SELKIES_WAYLAND_SOCKET_INDEX` | `0` | Index for the Wayland command socket, `0` is `wayland-0` | +| `SELKIES_COMPUTER_USE_BIND` | `''` | Start pixelflux's Computer Use HTTP server on comma separated entries, a bare port listens on loopback only, `host:port` names the address. `PIXELFLUX_CU` remains the standalone fallback | + +### Lifecycle hooks + +| Variable | Default | Description | +| --- | --- | --- | +| `SELKIES_RUN_AFTER_CONNECT` | `''` | Shell command run after the first client connects, and again whenever a client connects while no others are connected | +| `SELKIES_RUN_AFTER_DISCONNECT` | `''` | Shell command run after the last client disconnects, including on server shutdown while clients are connected | +| `SELKIES_APP_WAIT_READY` | `false` | Wait for the ready file to exist before starting the stream | +| `SELKIES_APP_READY_FILE` | `/tmp/selkies-appready` | File a sidecar creates to signal the application is ready | -### Plumbing +### Watermark | Variable | Default | Description | | --- | --- | --- | -| `SELKIES_PORT` (or `CUSTOM_WS_PORT`) | `8082` | Data WebSocket server port | -| `SELKIES_DRI_NODE` (or `DRI_NODE`) | `''` | DRI render node for VA-API | -| `SELKIES_AUDIO_DEVICE_NAME` | `'output.monitor'` | Audio device for pcmflux capture | -| `SELKIES_WATERMARK_PATH` (or `WATERMARK_PNG`) | `''` | Absolute path to watermark PNG | +| `SELKIES_WATERMARK_PATH` (or `WATERMARK_PNG`) | `''` | Absolute path to a watermark PNG | | `SELKIES_WATERMARK_LOCATION` (or `WATERMARK_LOCATION`) | `-1` | Watermark location enum 0 to 6 | +### WebRTC networking, STUN, and TURN + +Setting any of these switches the container into WebRTC mode with the transport switch enabled. They are only meaningful for WebRTC; the [WebRTC Transport](webrtc.md) page explains when you need which. + +| Variable | Default | Description | +| --- | --- | --- | +| `SELKIES_WEBRTC_PUBLIC_IP` | `''` | Public IPv4 and/or IPv6 address (comma or space separated) to advertise in host ICE candidates, for a host behind static 1:1 NAT such as a cloud instance with an elastic IP. STUN and TURN candidates are left untouched | +| `SELKIES_WEBRTC_PORT_RANGE` | `''` | Inclusive UDP port range `min-max` (e.g. `50000-50100`) that sessions bind into, both bounds within `1024` to `65535`. Empty uses ephemeral OS ports | +| `SELKIES_WEBRTC_UDP_MUX_PORT` | `0` | Single UDP port every session shares for its host candidates, so one forwarded port serves any number of sessions. `0` gives each session its own sockets | +| `SELKIES_WEBRTC_TCP_MUX_PORT` | `0` | Single TCP port the server accepts ICE-TCP connections on, so clients on networks that block UDP still connect. May equal the UDP mux port. `0` offers no TCP candidates | +| `SELKIES_WEBRTC_ICE_LITE` | `false` | Run the server's ICE agent as ICE-lite, offering host candidates only. Suits a server whose host candidates are reachable as advertised: a public address, a static NAT with the public IP set, or forwarded mux ports | +| `SELKIES_STUN_HOST` | `stun.l.google.com` | STUN host for NAT hole punching, change to an internal server on networks without internet | +| `SELKIES_STUN_PORT` | `19302` | STUN port | +| `SELKIES_RTC_CONFIG_JSON` | `/tmp/rtc.json` | JSON file with a full WebRTC ICE configuration, checked periodically. When it exists it overrides every other STUN and TURN setting | +| `SELKIES_TURN_REST_URI` | `''` | URI of a TURN REST API service that hands out time limited credentials, e.g. `http://localhost:8008`. Overrides the static TURN settings below | +| `SELKIES_TURN_REST_API_KEY` | `''` | API key sent to the TURN REST API service | +| `SELKIES_TURN_REST_USERNAME` | `''` | Username sent to the TURN REST API service, empty uses `selkies` | +| `SELKIES_TURN_REST_USERNAME_AUTH_HEADER` | `x-auth-user` | Header carrying the username to the TURN REST API | +| `SELKIES_TURN_REST_PROTOCOL_HEADER` | `x-turn-protocol` | Header carrying the desired TURN protocol to the TURN REST API | +| `SELKIES_TURN_REST_TLS_HEADER` | `x-turn-tls` | Header carrying the TURN TLS preference to the TURN REST API | +| `SELKIES_TURN_HOST` | `staticauth.openrelay.metered.ca` | TURN host for shared secret or long term credentials, IPv6 addresses in square brackets | +| `SELKIES_TURN_PORT` | `443` | TURN port | +| `SELKIES_TURN_PROTOCOL` | `udp` | TURN transport the client uses, `udp` or `tcp`. Use `tcp` only if UDP is blocked | +| `SELKIES_TURN_TLS` | `false` | TURN over TLS (TCP) or DTLS (UDP), requires a valid certificate on the TURN server | +| `SELKIES_TURN_SHARED_SECRET` | `openrelayprojectsecret` | Shared secret used to generate time limited HMAC credentials, with `SELKIES_TURN_HOST` and `SELKIES_TURN_PORT` | +| `SELKIES_TURN_USERNAME` | `''` | Long term credential username, with `SELKIES_TURN_HOST` and `SELKIES_TURN_PORT` | +| `SELKIES_TURN_PASSWORD` | `''` | Long term credential password | +| `SELKIES_ENABLE_CLOUDFLARE_TURN` | `false` | Use the Cloudflare TURN service, requires the two Cloudflare variables below | +| `SELKIES_CLOUDFLARE_TURN_TOKEN_ID` | `''` | Cloudflare TURN app token ID | +| `SELKIES_CLOUDFLARE_TURN_API_TOKEN` | `''` | Cloudflare TURN API token | + ## Optional run configurations | Argument | Description | diff --git a/docs/selkies/user-guide/gpu.md b/docs/selkies/user-guide/gpu.md index 997183fc..c1789d7e 100644 --- a/docs/selkies/user-guide/gpu.md +++ b/docs/selkies/user-guide/gpu.md @@ -22,6 +22,18 @@ You usually do not need to set either variable. When a GPU is mounted into the c To mount a GPU but *not* use it, set `AUTO_GPU=false`. +## Which codecs a GPU encodes + +The encoder menu covers six codecs, described in the [Video encoders](configuration.md#video-encoders) table. Each full frame codec is encoded on the GPU when the encoding device has an engine for it, and by the software encoder in the pixelflux build otherwise. Zero copy applies to every hardware codec, not only H.264: the frame is passed as a DMA-BUF to whichever engine encodes it. + +| GPU | Hardware codecs | Notes | +| --- | --- | --- | +| Nvidia NVENC | H.264, H.265, AV1 | AV1 needs Ada (RTX 40 series) or newer. VP8 and VP9 are always software on Nvidia | +| Intel VA-API | H.264, H.265, VP9, and AV1 on Arc and recent integrated graphics | Depends on generation and media driver, `vainfo` lists the encode entry points. VP8 engines only exist on older generations | +| AMD VA-API | H.264, H.265, and AV1 from RDNA 3 | No VP8 or VP9 encode engines | + +At startup Selkies probes the encoding device once and drops every encoder that neither the GPU nor the software build can serve, and the container log says which were left off. A codec that falls back to software costs a readback plus a full frame software encode, which for H.265, VP9, and AV1 is several times heavier than x264 and never striped across cores. On a card without the engine, leave `h264enc` first in the menu and treat the other codecs as options for hosts that carry them. + ## Intel and AMD (open source drivers) The simple case. Mount the DRI devices and you are done: @@ -124,11 +136,13 @@ On Unraid, set `DRINODE` and `DRI_NODE` appropriately and add `--gpus all --runt If you notice blurry text, especially light text on dark backgrounds, enable **FullColor 4:4:4** encoding in the sidebar, or use the JPEG encoder. This sends true 8 bit color to the browser. -One caveat: only Nvidia GPUs can encode 4:4:4 in zero copy mode. Enabling FullColor on Intel or AMD falls back to CPU encoding, which forces a pixel readback from the GPU and costs significant performance. On those cards, prefer the default 4:2:0 for motion and let paint over handle static clarity. +FullColor is carried by H.264 and H.265 on NVENC, x264, and x265, and by VP9 profile 1 on VA-API and libvpx. VP8, AV1, and the OpenH264 software encoder stay 4:2:0 whatever you set, and the sidebar only shows the switch on a codec that carries it. + +The caveat is H.264 on Intel and AMD. Current VA-API drivers expose no 4:4:4 H.264 profile, so pixelflux honors the request on the CPU with x264 rather than silently downgrading, which forces a pixel readback from the GPU and costs significant performance. On those cards prefer the default 4:2:0 for motion and let paint over handle static clarity, or try H.265 or VP9, where the driver may negotiate a 4:4:4 surface and stay on the card. The log line `Colorspace:` reports what the session settled on. ## Wayland and X11 -The Wayland stack is the default and is where all GPU acceleration development happens. You can force the legacy X11 stack with `-e PIXELFLUX_WAYLAND=false`, but GPU acceleration under X11 is not currently seeing development attention. If you are on X11 and using acceleration, clamp the virtual display to avoid memory exhaustion, e.g. `-e MAX_RES=3840x2160`, and if you still have problems lock the resolution down: +The Wayland stack is the default and is where all GPU acceleration development happens. You can force the legacy X11 stack with `-e PIXELFLUX_WAYLAND=false`, but GPU acceleration under X11 is not currently seeing development attention. The one exception is Nvidia: when an X11 session encodes on NVENC, pixelflux captures through NvFBC and the frame is registered with the encoder in place, so that path is also zero copy. Every other X11 session copies each frame once through shared memory. If you are on X11 and using acceleration, clamp the virtual display to avoid memory exhaustion, e.g. `-e MAX_RES=3840x2160`, and if you still have problems lock the resolution down: ```bash -e SELKIES_MANUAL_WIDTH=1920 diff --git a/docs/selkies/user-guide/index.md b/docs/selkies/user-guide/index.md index 8aa431a2..0336b885 100644 --- a/docs/selkies/user-guide/index.md +++ b/docs/selkies/user-guide/index.md @@ -15,4 +15,5 @@ You do not need to know anything about the internals to use these containers. If 7. **[Installing Applications](installing-apps.md)**: adding software inside a container with proot-apps or Docker mods. 8. **[Security and Hardening](security.md)**: authentication, what the container can do, and lockdown variables for kiosk style deployments. 9. **[Reverse Proxy](reverse-proxy.md)**: putting containers behind SWAG, Nginx, Traefik, or a subfolder. -10. **[Troubleshooting](troubleshooting.md)**: the checklist to run before opening an issue. +10. **[WebRTC Transport](webrtc.md)**: the opt in UDP transport, when it helps, and the STUN and TURN setup it needs. +11. **[Troubleshooting](troubleshooting.md)**: the checklist to run before opening an issue. diff --git a/docs/selkies/user-guide/installing-apps.md b/docs/selkies/user-guide/installing-apps.md index 18927e43..07d353c0 100644 --- a/docs/selkies/user-guide/installing-apps.md +++ b/docs/selkies/user-guide/installing-apps.md @@ -44,6 +44,24 @@ Trade offs: - Not persistent in the image, but reinstalled automatically, so effectively stable as long as the variable stays set. - Best for libraries and CLI tools an app needs, less ideal for large GUI applications. +## Steam (built in, reinstalls itself) + +Every glibc based image carries a Steam installer. Nothing is installed until you ask for it, and this specialized installer can run steam in any docker environment without any additional container permissions (seccomp/apparmor): + +```bash +steam +``` + +The first run opens a terminal that installs the Steam launcher and its 32 bit dependencies with the distro package manager, then wraps the launcher so gamepads work through the [input interposer](web-client.md#gamepads), including inside Proton. Steam then appears in the menus and in the sidebar Apps section, and later runs of `steam` start it normally. You can also install or remove it from the Apps section, or with `selkies-proot install steam` and `selkies-proot remove steam`. + +The install lives in the container layer, so a recreation or upgrade drops it. Your game data in `$HOME/.steam` and `$HOME/.local/share/Steam` persists. Running `steam` again, or double clicking a Steam desktop icon you kept in `~/Desktop`, triggers the minimal installer again. After the reinstall your library is where you left it. + +Limits: + +- x86_64 only, and it needs passwordless sudo, so `HARDEN_DESKTOP` and `DISABLE_SUDO` block it. +- Alpine images ship a `steam` stub that reports it is unsupported. +- `NO_STEAM=true` removes the installer at startup, so `steam` is not a command in the container. + ## Building your own image (permanent) If you always need the same software, the clean solution is a small downstream Dockerfile: diff --git a/docs/selkies/user-guide/reverse-proxy.md b/docs/selkies/user-guide/reverse-proxy.md index 8081a8d1..0ac7673d 100644 --- a/docs/selkies/user-guide/reverse-proxy.md +++ b/docs/selkies/user-guide/reverse-proxy.md @@ -5,7 +5,7 @@ Putting a Selkies container behind a reverse proxy gets you real TLS certificate ## The rules 1. **Proxy to port 3000 (HTTP)**, and let your proxy terminate TLS. The container's port 3001 self signed HTTPS is for direct access; double TLS is pointless. -2. **WebSocket upgrades must pass through.** All streaming rides a WebSocket at `/websocket`. Any proxy that handles `Upgrade` and `Connection` headers works. +2. **WebSocket upgrades must pass through.** All streaming rides a WebSocket. Any proxy that handles `Upgrade` and `Connection` headers works. You never need path specific rules, the container's built in Nginx routes everything internally, so proxy the whole container as one location. 3. **Long timeouts.** Sessions are long lived connections. Set read and send timeouts to an hour (the internal Nginx uses 3600s), or idle sessions will drop. 4. **Client must still reach you over HTTPS.** The browser APIs need a secure context, so your proxy must serve HTTPS to the user. @@ -74,3 +74,4 @@ If you are heading toward many users and many apps, that is exactly what [SealSk - The container substitutes `SUBFOLDER` and auth settings into its Nginx config with simple string replacement at startup. Exotic characters in passwords or paths can break the substitution, keep them simple. - Server side events and the streaming WebSocket dislike buffering proxies, always disable response buffering (`proxy_buffering off` or your proxy's equivalent). - If the client loads but you get a black screen or no video, it is almost always a blocked WebSocket upgrade or an HTTP (not HTTPS) page context. +- Enabling the [WebRTC transport](webrtc.md) changes nothing at the proxy. Its signaling goes through the same proxied web port, and its media goes around the proxy entirely over UDP. diff --git a/docs/selkies/user-guide/security.md b/docs/selkies/user-guide/security.md index cc930d3e..036b9d5e 100644 --- a/docs/selkies/user-guide/security.md +++ b/docs/selkies/user-guide/security.md @@ -40,7 +40,7 @@ Note that the window manager level switches (`HARDEN_OPENBOX`, `DISABLE_CLOSE_BU | Variable | Description | | --- | --- | -| `HARDEN_DESKTOP` | Enables `DISABLE_OPEN_TOOLS`, `DISABLE_SUDO`, and `DISABLE_TERMINALS`. Also sets the related client settings (`SELKIES_FILE_TRANSFERS`, `SELKIES_COMMAND_ENABLED`, `SELKIES_UI_SIDEBAR_SHOW_FILES`, `SELKIES_UI_SIDEBAR_SHOW_APPS`) unless you set them explicitly yourself | +| `HARDEN_DESKTOP` | Enables `DISABLE_OPEN_TOOLS`, `DISABLE_SUDO`, and `DISABLE_TERMINALS`. Also sets the related client settings (`SELKIES_FILE_TRANSFERS`, `SELKIES_COMMAND_ENABLED`, `SELKIES_PRINTING_ENABLED`, `SELKIES_UI_SIDEBAR_SHOW_FILES`, `SELKIES_UI_SIDEBAR_SHOW_APPS`) unless you set them explicitly yourself | | `HARDEN_OPENBOX` | Window manager lockdown for single app containers. Enables `DISABLE_CLOSE_BUTTON`, `DISABLE_MOUSE_BUTTONS`, and `HARDEN_KEYBINDS`, and flags `RESTART_APP` unless you set it, so the primary application restarts automatically if closed | ### Individual switches @@ -60,7 +60,7 @@ Note that the window manager level switches (`HARDEN_OPENBOX`, `DISABLE_CLOSE_BU Any boolean `SELKIES_*` setting can be pinned so the user cannot change it in the sidebar by appending `|locked`: ```bash --e SELKIES_CLIPBOARD_ENABLED="false|locked" +-e SELKIES_ENABLE_CLIPBOARD="false|locked" -e SELKIES_FILE_TRANSFERS="none" -e SELKIES_UI_SHOW_SIDEBAR="false|locked" ``` @@ -78,7 +78,7 @@ services: environment: - HARDEN_DESKTOP=true - HARDEN_OPENBOX=true - - "SELKIES_CLIPBOARD_ENABLED=false|locked" + - "SELKIES_ENABLE_CLIPBOARD=false|locked" - SELKIES_FILE_TRANSFERS=none - "SELKIES_UI_SHOW_SIDEBAR=false|locked" - NO_DECOR=true diff --git a/docs/selkies/user-guide/troubleshooting.md b/docs/selkies/user-guide/troubleshooting.md index 228607d9..8d289609 100644 --- a/docs/selkies/user-guide/troubleshooting.md +++ b/docs/selkies/user-guide/troubleshooting.md @@ -27,6 +27,9 @@ This loop resolves the majority of reports, particularly GPU and reverse proxy i | Container restarts or app crashes on old hardware or NAS | Old kernel or libseccomp blocking syscalls | Try `--security-opt seccomp=unconfined` (understand the [security cost](security.md#container-isolation-options)) | | Blurry colored text | 4:2:0 chroma subsampling | Enable FullColor 4:4:4 in the sidebar or use the JPEG encoder, note the [Intel and AMD caveat](gpu.md#fullcolor-444-and-hardware-encoders) | | Choppy video during motion | CPU limited or bandwidth limited | Check the Stats section, lower FPS or raise CRF, try Turbo mode off, consider a [GPU](gpu.md) | +| CPU pegged after picking H.265, VP9, or AV1 | The GPU has no engine for that codec, so it encodes full frame in software | Go back to H.264, or check `vainfo` and `nvidia-smi` for what the card encodes, see [which codecs a GPU encodes](gpu.md#which-codecs-a-gpu-encodes) | +| An encoder you configured is missing from the sidebar | Neither the encoding GPU nor the software build serves it, or the browser cannot decode it | The log line `Encoders not served on this host` lists the first case, a greyed out `(Unsupported Browser)` entry is the second | +| Sidebar shows H.264 though the default is another codec | The browser cannot decode the default, or pixelflux demoted the codec at capture start | Expected. The log line `no encoder served` marks a demotion, otherwise the browser stepped to a codec it decodes | | Cursor feels laggy in games | Absolute pointer mode | Use Gaming mode (pointer lock) | | Gamepad not detected in app | App does not use the joystick API path | Some apps are incompatible with the userspace interposer; try another input mode in app, or file an issue | | Files will not upload | Proxy body size limit | Raise `client_max_body_size` at your proxy | diff --git a/docs/selkies/user-guide/web-client.md b/docs/selkies/user-guide/web-client.md index 29b551ae..618ee5d1 100644 --- a/docs/selkies/user-guide/web-client.md +++ b/docs/selkies/user-guide/web-client.md @@ -4,15 +4,17 @@ The web client is a complete workstation interface, not just a video player. Thi ## The sidebar -Toggle the sidebar with its handle on screen. At the top you get the core toggles: display, audio, microphone, and gamepad on or off, plus buttons for fullscreen, virtual trackpad, gaming mode, and the on screen keyboard. +Toggle the sidebar with its handle on screen. At the top you get the core toggles: display, audio, microphone, webcam, and gamepad on or off, plus buttons for fullscreen, virtual trackpad, gaming mode, and the on screen keyboard. ## Video settings -- **Encoder**: H.264 (`x264enc`), striped H.264, or JPEG. H.264 is the default and right for everything modern, with FullColor 4:4:4 plus paint over it is visually equal to lossless stills. JPEG exists for unsupported browsers that cannot decode video frames at all. -- **Frames per second** and **CRF** (H.264 quality, lower is better) sliders, plus separate paint over quality controls. "Paint over" is the platform's signature feature: after the screen goes still, it is repainted at high quality so text stays crisp. -- **FullColor 4:4:4**: true 8 bit color with no chroma subsampling. Fixes blurry colored text. Note the [GPU caveats](gpu.md#fullcolor-444-and-hardware-encoders). +- **Encoder**: H.264, H.265, VP8, VP9, AV1, or JPEG. Each video codec is encoded on the GPU when the card has an engine for it and in software otherwise, so the menu only lists what this host can serve. H.264 is the default and right for everything modern, with FullColor 4:4:4 plus paint over it is visually equal to lossless stills. Entries your browser cannot decode are greyed out as `(Unsupported Browser)`, H.265 outside Safari being the usual one. JPEG exists for browsers that cannot decode video frames at all. See the [Video encoders](configuration.md#video-encoders) table for what each codec needs. +- **Frames per second** and **CRF** (quality, lower is better) sliders, plus separate paint over quality controls. "Paint over" is the platform's signature feature: after the screen goes still, it is repainted at high quality so text stays crisp. The CRF slider means the same quality on every codec. +- **FullColor 4:4:4**: true 8 bit color with no chroma subsampling. Fixes blurry colored text. Offered on H.264, H.265, and VP9 only. Note the [GPU caveats](gpu.md#fullcolor-444-and-hardware-encoders). - **Turbo mode** (streaming mode): disables all the damage tracking logic and encodes every frame like a traditional video stream. Can be useful for gaming and full motion video. -- **CPU encoding** toggle to force cpu encoding. +- **CPU encoding** toggle to force software encoding. It only appears when the current codec is available both on the GPU and in software on this host. +- **Rate control**: CRF (constant quality, the default on WebSockets) or CBR with a bitrate slider (the default on WebRTC). +- **Transport**: when the administrator has enabled WebRTC, a switch lets you move between WebSockets and WebRTC at runtime, see [WebRTC Transport](webrtc.md). !!! tip "On a slow machine or a slow link, turn the stream down" The defaults favor fluidity. If your client device is low end or your bandwidth is poor, reduce **Frames per second** first and raise **CRF** if needed (higher CRF means smaller frames). Paint over still keeps static content sharp, so a lower FPS and quality setting costs far less than you would expect for desktop work. @@ -22,20 +24,24 @@ Toggle the sidebar with its handle on screen. At the top you get the core toggle - **Resolution**: by default the remote resolution follows your browser window exactly. You can instead pick a preset (720p through 4K) or type a manual width and height. - **UI scaling (DPI)** for HiDPI displays, with a choice between pixel perfect HiDPI rendering and CSS scaling (lighter on bandwidth, softer image). - **Scale locally** stretches a fixed remote resolution to fit your window. -- **Add Screen +**: opens a second browser window that becomes a second monitor for the session, positioned left, right, above, or below the primary. Arrange each window on the matching physical monitor and you have a real dual screen remote desktop. (Second screen is currently a feature of the X11 stack; in Wayland mode it is disabled.) +- **Add Screen +**: opens a second browser window that becomes a second monitor for the session, positioned left, right, above, or below the primary. Arrange each window on the matching physical monitor and you have a real dual screen remote desktop. Always available on X11. On Wayland, Selkies detects at startup whether the running compositor supports it, which today means labwc (the single app containers and the default Webtop shell) and KWin (the KDE Webtops), and only offers the button when it does. ## Audio and microphone Opus audio streams from the session to your browser, and output device selection if your browser exposes multiple sinks. The microphone button forwards your local mic into the session, where apps see it as a normal input device, video calls from inside a remote browser container work. +## Webcam + +The webcam button forwards your local camera into the session as a virtual V4L2 device, so video conferencing inside a remote browser or any camera aware application works. The administrator can size the virtual device and disable the feature entirely with `NO_WEBCAM`. + ## Clipboard -Bidirectional clipboard sync between your machine and the session, automatic in both directions for text. The sidebar shows an editable view of the server clipboard. Enabling **Image support** (binary clipboard, `SELKIES_ENABLE_BINARY_CLIPBOARD`) adds images and other binary formats. Clipboard direction and availability can be locked down by the admin. +Bidirectional clipboard sync between your machine and the session, automatic in both directions for text. The sidebar shows an editable view of the server clipboard. **Image support** (binary clipboard, `SELKIES_ENABLE_BINARY_CLIPBOARD`) adds images and other binary formats. Clipboard direction and availability can be locked down by the admin with the `SELKIES_ENABLE_CLIPBOARD` policy. ## Files - **Upload**: drag and drop files anywhere on the session window, or use the Upload button. Files land in the session's `~/Desktop` by default (configurable with `FILE_MANAGER_PATH`). -- **Download**: the Files section opens a dark themed file index of the same directory served by the container's Nginx, click to download. +- **Download**: the Files section opens a file browser of the same directory, click to download. Transfers can be restricted per direction or disabled entirely with `SELKIES_FILE_TRANSFERS`. @@ -76,6 +82,14 @@ On phones and tablets the client offers: - **Direct touch mode**: taps map straight to absolute clicks, best for touch friendly apps. - The **on screen keyboard** button raises your device keyboard, with IME input (for example Chinese Pinyin) supported end to end. +## macOS keyboards + +Command chords are sent as Control by default, so Cmd+C and Cmd+V work as they do locally. If the session's window manager binds Super (a Cmd+Return terminal shortcut, for example), turn it off in the sidebar or with `SELKIES_MAC_CMD_AS_CTRL=false` and Command arrives as Super. + +## Printing + +Every session has a printer named **Selkies**. Print to it from any application and the document arrives in your browser as a PDF, where the browser's own print dialog offers whatever printers your local machine can reach. The sidebar's printing section lists each document so it can be printed again or saved as a PDF, its **Print automatically** switch is what opens the dialog on arrival. Documents go to the page holding the session, shared viewers do not receive them. Administrators turn the feature off with `SELKIES_PRINTING_ENABLED=false`, and `HARDEN_DESKTOP` does so by default. + ## Gaming mode Gaming mode grabs the pointer (pointer lock) and sends relative mouse movement for accurate FPS style aiming, pairs well with Turbo encoding mode and gamepads. diff --git a/docs/selkies/user-guide/webrtc.md b/docs/selkies/user-guide/webrtc.md new file mode 100644 index 00000000..9f1406cb --- /dev/null +++ b/docs/selkies/user-guide/webrtc.md @@ -0,0 +1,184 @@ +# WebRTC Transport + +Selkies streams over WebSockets by default, and for most people that is the right choice: one TCP connection, works through any reverse proxy, no firewall rules beyond the web port. An opt in WebRTC transport carries the same video, Opus audio, and input over UDP instead. This page explains when it is worth turning on, how the containers enable it, and the networking it needs. + +!!! note "WebSockets stays the default" + Nothing on this page applies until you deliberately switch a container to WebRTC. If your desktop streams fine today, you do not need any of it. + +## When to use it + +WebRTC helps when the path between the browser and the container is lossy or has variable latency, typically a mobile network, a long haul link, or a wireless client on a congested network. UDP does not stall on a lost packet the way TCP does, the audio pacer keeps sound and input responsive while video is squeezed, and congestion control can adapt the bitrate to what the link actually delivers. + +The trade offs: + +- **Only the full frame codecs stream over WebRTC.** H.264, H.265, VP8, VP9, and AV1 all ride the RTP track. The striped H.264 and JPEG encoders are WebSocket only, and the encoder menu is narrowed to what WebRTC can carry while it is the active transport. The previous choice is restored when switching back. +- **The browser's RTP receiver decides the codec.** H.264 and VP8 are taken by every major browser, VP9 and AV1 by Chromium and Firefox, H.265 by Safari and by Chromium where the operating system decodes it. A browser that declines the codec is answered with H.264 when `h264enc` is in the menu, otherwise that browser gets no video. +- **Rate control defaults to CBR.** A congestion controlled transport needs the encoder holding a bandwidth target, so `SELKIES_VIDEO_BITRATE` becomes the setting you tune rather than CRF. Pin `SELKIES_RATE_CONTROL_MODE` if you want otherwise. +- **UDP has to get through.** A reverse proxy alone is not enough. Media takes a direct path from the container to the browser, and something has to make that path reachable: forwarded ports, a public address, or a TURN relay. +- **Turbo mode and paint over still apply.** Damage tracking and paint over work on both transports. + +## Enabling it in the containers + +The baseimage keeps WebRTC hidden until you configure it. Set `SELKIES_MODE=webrtc`, or any WebRTC, STUN, TURN, or Cloudflare variable from the [configuration reference](configuration.md#webrtc-networking-stun-and-turn), and the container: + +1. Sets `SELKIES_MODE=webrtc` if you have not set a mode yourself, so new clients start on WebRTC. +2. Sets `SELKIES_ENABLE_DUAL_MODE=true` so the transport switch appears in the sidebar and users can fall back to WebSockets at will. + +Without any of those variables the switch is hidden and the container is WebSocket only, exactly as before. Anything you set explicitly wins, so `SELKIES_MODE=websockets` plus a TURN host gives you a WebSocket default with WebRTC available in the menu, and `SELKIES_ENABLE_DUAL_MODE=false` plus `SELKIES_MODE=webrtc` gives you WebRTC with no way back. + +The simplest working setup forwards one extra UDP port next to the web port: + +```yaml +--- +services: + webtop: + image: lscr.io/linuxserver/webtop:latest + environment: + - PUID=1000 + - PGID=1000 + - TZ=Etc/UTC + - SELKIES_MODE=webrtc + - SELKIES_WEBRTC_UDP_MUX_PORT=59000 + ports: + - 3001:3001 + - 59000:59000/udp + shm_size: 1gb + restart: unless-stopped +``` + +That is enough for clients on the same LAN as the Docker host. The rest of this page explains why the extra port is needed and what to add when clients are further away. + +## How the connection is built + +Two connections are involved, and it helps to keep them apart when something fails: + +| Connection | Path | What it carries | +| --- | --- | --- | +| Signaling | The container's normal web port (3000 or 3001, or your reverse proxy), over the same WebSocket path the client already uses | Session description and ICE candidates, then settings, clipboard, files, and stats over the data channel once the peer connection is up | +| Media | Direct UDP (or TCP when muxing is configured) from the container to the browser | Video, audio, microphone, and webcam | + +If the web client loads and the sidebar works but the desktop never appears, the media path is what is broken. That is the case every section below is about. + +ICE, the negotiation that picks the media path, tries candidates in order of preference: a direct host address, then a server reflexive address discovered through STUN, then a TURN relay. The stats section of the sidebar shows which one won, `relay` means every packet is going through your TURN server. + +## Making the media path reachable + +Pick the first row that matches your deployment. + +| Situation | What to do | +| --- | --- | +| Client and container on the same LAN | Forward one UDP port and tell Selkies about it, see [UDP mux](#one-forwarded-port-udp-mux) | +| Container has a public IP, or sits behind static 1:1 NAT (a cloud instance with an elastic IP) | Forward one UDP port, set `SELKIES_WEBRTC_PUBLIC_IP` to the public address, optionally enable ICE-lite | +| Clients on networks that block UDP | Add a TCP mux port, ideally `443` | +| Container behind NAT you cannot forward through, or many clients on hostile networks | Run or rent a [TURN server](#turn-servers) | + +### One forwarded port (UDP mux) + +By default every WebRTC session binds its own ephemeral UDP ports, which inside a container means the browser is handed addresses it cannot reach. `SELKIES_WEBRTC_UDP_MUX_PORT` replaces that with one port that all sessions share, sessions are told apart by their ICE credentials. Forward that single port without remapping, as in the example above, and you are done. Always set it, the alternative is publishing the whole 49152 to 65535 range. + +`SELKIES_WEBRTC_PORT_RANGE=50000-50100` is the alternative for schedulers that allot each session a small window of ports. Forward the same range with `-p 50000-50100:50000-50100/udp`. A range is ignored when a mux port is set. + +A port already in use fails the container at startup rather than the first session, since a session that silently bound elsewhere would be unreachable through the forwarded port. + +### Public address and static NAT + +On a cloud instance the container gathers its private address as a host candidate, which a remote browser cannot reach, and the connection falls through to TURN or fails. `SELKIES_WEBRTC_PUBLIC_IP` substitutes your public IPv4 and/or IPv6 address into the host candidates. STUN and TURN candidates are left alone, so hole punching and relay fallback still work. + +`SELKIES_WEBRTC_ICE_LITE=true` is a good companion here. The server then offers host candidates only and answers the browser's connectivity checks instead of running its own, which is all a server with a reachable address needs. The browser still gets STUN and TURN for candidates of its own. + +```yaml + environment: + - SELKIES_MODE=webrtc + - SELKIES_WEBRTC_UDP_MUX_PORT=59000 + - SELKIES_WEBRTC_TCP_MUX_PORT=59000 + - SELKIES_WEBRTC_PUBLIC_IP=203.0.113.5 + - SELKIES_WEBRTC_ICE_LITE=true + ports: + - 3001:3001 + - 59000:59000/udp + - 59000:59000/tcp +``` + +### Clients that cannot use UDP + +`SELKIES_WEBRTC_TCP_MUX_PORT` makes the server accept ICE-TCP on one TCP port and advertise it next to the UDP candidates. Browsers prefer UDP whenever it works and only fall to TCP when it does not. It may share its number with the UDP mux port, and `443` is the usual choice on a public deployment because corporate firewalls pass it. Media over TCP costs latency under loss, so keep UDP reachable where you can. + +## STUN and TURN + +STUN lets both ends discover their public addresses, TURN relays media when no direct path exists. Selkies ships with working defaults for casual use and you should replace both for anything serious: + +- **STUN** defaults to Google's public server. On a private network with no internet access point `SELKIES_STUN_HOST` and `SELKIES_STUN_PORT` at your own STUN or TURN server or connections will hang looking for it. +- **TURN** defaults to the free [Open Relay](https://www.metered.ca/tools/openrelay) service. It has one location and a shared public secret, so any `relay` connection through it adds real latency and stutter. Treat it as a demo. + +### Choosing a TURN server + +| Option | Fit | +| --- | --- | +| [Cloudflare TURN](https://developers.cloudflare.com/calls/turn/overview/) | Easiest managed option, geodistributed, free for the first 1000 GB a month. Set `SELKIES_ENABLE_CLOUDFLARE_TURN=true`, `SELKIES_CLOUDFLARE_TURN_TOKEN_ID`, and `SELKIES_CLOUDFLARE_TURN_API_TOKEN` | +| [coturn](https://github.com/coturn/coturn) | The standard self hosted server, available as `coturn/coturn` on Docker Hub or from every distro. Place it as close to your clients as you can | +| [eturnal](https://eturnal.net), [Pion TURN](https://github.com/pion/turn), [STUNner](https://github.com/l7mp/stunner) | Alternatives, STUNner is the Kubernetes native one | + +### Pointing Selkies at your TURN server + +Four authentication methods are supported and they override each other in this order: + +1. **RTC config JSON** (`SELKIES_RTC_CONFIG_JSON`): a file with a complete ICE server configuration, re-read periodically so rotated credentials are picked up. When the file exists everything below is ignored. +2. **TURN REST API** (`SELKIES_TURN_REST_URI` and `SELKIES_TURN_REST_API_KEY`): a service that mints time limited credentials on request, the right answer for multi user deployments where users should not hold the TURN secret. +3. **Shared secret** (`SELKIES_TURN_SHARED_SECRET` with `SELKIES_TURN_HOST` and `SELKIES_TURN_PORT`): Selkies generates time limited HMAC credentials itself from coturn's `static-auth-secret`. +4. **Long term credentials** (`SELKIES_TURN_USERNAME` and `SELKIES_TURN_PASSWORD` with host and port): a fixed username and password, coturn's `lt-cred-mech`. + +The last two put the TURN secret inside the container, which is fine when you own both ends. `SELKIES_TURN_PROTOCOL=tcp` and `SELKIES_TURN_TLS=true` apply to the shared secret and long term methods and tell the browser how to reach the relay. + +A minimal self hosted pairing with coturn on the same host, using a shared secret: + +```yaml +--- +services: + coturn: + image: coturn/coturn:latest + command: > + -n --listening-ip=0.0.0.0 --listening-ip=:: + --listening-port=3478 --realm=example.org + --external-ip=203.0.113.5 + --min-port=65500 --max-port=65535 + --use-auth-secret --static-auth-secret=change-me-to-a-long-random-string + ports: + - 3478:3478 + - 3478:3478/udp + - 65500-65535:65500-65535/udp + restart: unless-stopped + + webtop: + image: lscr.io/linuxserver/webtop:latest + environment: + - PUID=1000 + - PGID=1000 + - TZ=Etc/UTC + - SELKIES_MODE=webrtc + - SELKIES_TURN_HOST=turn.example.org + - SELKIES_TURN_PORT=3478 + - SELKIES_TURN_SHARED_SECRET=change-me-to-a-long-random-string + - SELKIES_STUN_HOST=turn.example.org + - SELKIES_STUN_PORT=3478 + ports: + - 3001:3001 + shm_size: 1gb + restart: unless-stopped +``` + +Open `3478` for TCP and UDP plus the relay range `65500-65535` UDP on the coturn host's firewall. Keep the relay range small, Docker publishes each port individually and a wide range slows container startup. coturn also answers STUN, which is why the example points `SELKIES_STUN_HOST` at it too. For TURN over TLS add `--cert` and `--pkey` with a certificate from a real CA and set `SELKIES_TURN_TLS=true`. + +The upstream [WebRTC and Firewall Issues](https://docs.selkies.io/firewall) page covers coturn configuration files, Kubernetes deployments, and the TURN REST API in far more depth. + +## Reverse proxies + +Nothing changes at your reverse proxy. Signaling goes through the container's web port exactly like the WebSocket transport, so a proxy set up per the [Reverse Proxy](reverse-proxy.md) page already handles it, SWAG included. Media never touches the proxy. If your proxy is on a different host than the container, the port forwarding and public IP settings above refer to the container host, not the proxy. + +## Troubleshooting + +- **Client loads, transport switch shows WebRTC, no picture.** ICE failed. Switch to WebSockets in the sidebar to confirm everything else works, then check the media path: is the UDP mux port forwarded, is `SELKIES_WEBRTC_PUBLIC_IP` set on a NAT host, can the browser reach the TURN server? +- **Works on the LAN, fails from outside.** Host candidates are private addresses. Set `SELKIES_WEBRTC_PUBLIC_IP` or configure TURN. +- **Works but stutters, stats say `relay`.** You are on the default Open Relay TURN or a distant relay. Fix the direct path so the browser never needs the relay, or move the TURN server closer. +- **Fails on one specific network only.** That network blocks UDP. Add a TCP mux port on `443` or set `SELKIES_TURN_PROTOCOL=tcp`. +- **Container will not start after adding a mux port.** The port is taken on the host, pick another. +- **Need more detail.** `SELKIES_DEBUG=true` logs ICE candidate gathering and selection, and `SELKIES_ENABLE_WEBRTC_STATISTICS=true` dumps per session CSVs into `SELKIES_WEBRTC_STATISTICS_DIR`.