Skip to content

Fix iOS audio: app-owned manual audio session for stable playout#346

Draft
MaxHeimbrock wants to merge 1 commit into
mainfrom
max/ios-audio-session-manual-mode
Draft

Fix iOS audio: app-owned manual audio session for stable playout#346
MaxHeimbrock wants to merge 1 commit into
mainfrom
max/ios-audio-session-manual-mode

Conversation

@MaxHeimbrock

@MaxHeimbrock MaxHeimbrock commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

This is just the result of an investigation

Problem

On iOS (Meet sample with PlatformAudio + a Unity AudioSource playing background music), the WebRTC ADM runs the shared AVAudioSession in automatic mode. As a result:

  • Joining a call reroutes other app audio (ambient music) to the earpiece and attenuates it — WebRTC drops the DefaultToSpeaker option on reconfigure, and VoiceChat mode is receiver-biased.
  • Hanging up deactivates the shared session (setActive:NO) out from under Unity/FMOD, so Unity audio dies and never recovers on rejoin.
  • The session is left stuck in PlayAndRecord/VoIP forever — LiveKit_RestoreDefaultAudioSession existed but was never called.

Fix

Put WebRTC's RTCAudioSession into manual mode and have the app own the session. Entirely Unity-side (no Rust/FFI rebuild) — RTCAudioSession is reachable from the plugin (unprefixed class, already linked into liblivekit_ffi), and LiveKitAudioSession.mm is compiled by Xcode.

  • Hold one permanent activation. RTCAudioSession ref-counts activation, so WebRTC's per-call setActive:NO on hang-up only decrements the count and never deactivates the hardware session → Unity audio survives.
  • VideoChat mode routes to the loudspeaker by default (while honoring connected wired/Bluetooth headphones) → no more earpiece.
  • Gate the VPIO unit via isAudioEnabled around connect/disconnect. Defaults to enabled so call audio works out of the box; disabled on hang-up stops the unit while the session stays alive.
  • Restore the session on PlatformAudio.Dispose — balances the constructor so the session isn't left in PlayAndRecord.

Changes

  • Runtime/Plugins/iOS/LiveKitAudioSession.mm — manual-mode RTCAudioSession (reached via NSClassFromString + a protocol-typed id, with an AVAudioSession fallback), VideoChat mode, held activation, new LiveKit_SetAudioEnabled, rewritten restore.
  • Runtime/Scripts/Audio/PlatformAudio.cs — public SetSessionAudioEnabled(bool) + restore-on-Dispose.
  • Samples~/Meet/Assets/Runtime/MeetManager.cs — enable on connect, disable on hang-up / disconnect.

Validation

  • ✅ Meet sample compiles for iOS via IL2CPP (Unity 6000.3.10f1, Build SUCCEEDED, 0 error CS).
  • ⚠️ Needs on-device validation — manual-mode integration touches libwebrtc internals. Verify routing stays on the loudspeaker across join / mute / unmute, and that Unity audio survives hang-up → rejoin. Watch the LiveKit: NSLogs; activationCount should never reach 0 during a connect → hang-up → reconnect cycle.

Known trade-off (unchanged, by design)

While a call is active, the VPIO voice-processing unit imposes iOS in-call volume (louder, not hardware-mutable to zero); it reverts to normal media volume after hang-up. That's inherent to keeping hardware echo cancellation — only disabling VPIO (a Rust rebuild that sacrifices AEC) removes it.

🤖 Generated with Claude Code

On iOS the WebRTC ADM ran the shared AVAudioSession in automatic mode, so
joining a call rerouted other app audio to the earpiece and hanging up
deactivated the session out from under Unity (ambient audio died).

Put RTCAudioSession into manual mode and have the app own the session: hold
one permanent activation (so WebRTC's per-call setActive:NO never deactivates
it), set PlayAndRecord + VideoChat (loudspeaker by default), and gate the VPIO
unit via isAudioEnabled around connect/disconnect. Also restore the session on
PlatformAudio.Dispose (previously dead code).

Validated: Meet sample compiles for iOS (Unity 6000.3.10f1). Manual-mode audio
behavior needs on-device validation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant