Smooth out waveform/video playback cursor (lag, stutter, seek delay)#11627
Conversation
The playhead cursor lagged ~100-200 ms behind playback, froze and jumped when resuming after a seek, and stuttered during playback. Fixes: - Interpolate the playhead on a high-resolution wall clock at the current playback speed, reconciling with mpv's real position (snap on large discontinuities, gently correct small drift). This hides mpv's chunky time-pos updates and its ~200 ms stall-then-jump when resuming after a paused seek. - Drive the cursor from a dedicated ~60 fps DispatcherTimer created at DispatcherPriority.Render. The mpv video control posts frame requests at Render priority; a default (Background) timer was starved by them for 100-800 ms during decode bursts, which was the visible stutter. The heavy per-tick work (subtitle buffer, centering, selection) stays on the 50 ms timer. - Cache the waveform geometry and the per-paragraph shaped text so a cursor-only move replays cheap draw calls instead of re-running the per-pixel waveform loop and re-shaping FormattedText every frame (the latter caused GC pauses). Caches are keyed by view state and cleared in ResetCache().
- When the user seeks via the waveform, pin the cursor to the clicked position immediately and hold it until mpv's reported position arrives (or a short timeout). mpv applies pause+seek asynchronously, so without this the cursor lagged ~100-200 ms on the old spot before snapping. - Cap a single interpolation tick's forward advance. When the cursor timer is starved by mpv's decode burst right after pressing play, the next tick no longer advances by the whole missed span (a visible forward "rush"); it glides one small step and lets drift correction reconcile.
Matching DispatcherPriority.Render let a burst of mpv video-frame posts (same priority) still delay the cursor timer via FIFO ordering right after pressing play, so the cursor lagged and then raced to catch up. Running the timer at Normal (above Render) keeps it firing on time through the decode burst, so the line glides smoothly from the start. Per-tick work is ~0.2 ms, so it doesn't disturb video rendering.
In "center waveform on current video position while playing" mode the waveform scroll position was updated only by the 50 ms heavy timer, so it jumped in 20 fps steps while the cursor moved at 60 fps. Drive the scroll from the 60 fps cursor timer in lockstep with the cursor so it glides smoothly; the heavy timer now only refreshes the paragraph list in this mode.
|
Two things from reviewing this branch: 1. Playhead estimate isn't maintained in layouts without a waveform (functional regression)In the cursor timer, var av = AudioVisualizer;
if (av != null)
{
var est = UpdatePlayheadEstimate(vp); // only runs when the waveform exists
av.CurrentVideoPositionSeconds = est;
...
}
var mediaPlayerSeconds = _playheadEstimateSeconds;But the position timer runs whenever Suggested fix — always maintain the estimate; only the visual updates need var est = UpdatePlayheadEstimate(vp);
var av = AudioVisualizer;
if (av != null)
{
av.CurrentVideoPositionSeconds = est;
if (WaveformCenter && vp.IsPlaying && av.WavePeaks != null)
{
var halfSeconds = (av.EndPositionSeconds - av.StartPositionSeconds) / 2.0;
av.StartPositionSeconds = Math.Max(0, est - halfSeconds);
}
}2. Stale comment
Otherwise this looks great — nice diagnosis-driven comments, and the waveform geometry/text caching plus the cache-key invalidation all check out. SMPTE handling stays consistent too (the manual |
Addresses PR review: - Always call UpdatePlayheadEstimate in the cursor timer, not only when a waveform exists. The 50 ms _positionTimer reads _playheadEstimateSeconds as its source of truth (SelectCurrentSubtitleWhilePlaying, play-selection end detection, auto-scroll), and some layouts have a video player but no waveform (e.g. MakeLayout8), where AudioVisualizer is null. Previously the estimate stayed at 0 in those layouts and broke playback-driven logic. Only the visual updates (cursor position, center-mode scroll) need the waveform. - Update a stale comment referencing DrawWaveFormFancy (renamed to BuildWaveFormFancy in this PR).
Thanks for the review! Both addressed in latest commit:
|
Per-tick logging showed that after a paused seek mpv's time-pos (and the video) freezes for ~400 ms while it re-primes, then resumes at 1x from the same spot. The wall-clock interpolation extrapolated straight through that freeze, racing the cursor ~0.4 s ahead of the still-frozen frame, then a sub-1x crawl pulled it back to resync — the "runs fast then slow" the user saw. - Track when mpv's reported position last changed; if it has been frozen longer than PlayheadFreezeHoldSeconds (120 ms), hold the cursor instead of extrapolating. mpv resumes from where the cursor is held, so it's seamless. - Make the drift correction forward-only so it can close a lag but never pull the cursor backward (the sub-1x "slow").
|
Also seems to work for vlc :) |
thanks for merging :) i'd be happy to help here and there! i love subtitleedit! been working with it for years. |
2 participants
Problem
The waveform/video playback cursor (the vertical position line) had several issues during playback:
Changes
Interpolated playhead. The cursor is advanced on a high-resolution wall clock at the current playback speed and reconciled with mpv's real position (snap on a large discontinuity, gently correct small drift, cap a single tick's advance). This hides mpv's chunky
time-posupdates and its stall-then-jump when resuming after a paused seek.Dedicated cursor timer at the right priority. The cursor is driven by its own ~60 fps
DispatcherTimer. The mpv video control posts frame requests atDispatcherPriority.Render; a default (Background) timer was starved by them for 100–800 ms during decode bursts (the stutter), and even matchingRenderleft a play-start rush, so the timer runs atNormal(above Render). The heavy per-tick work (subtitle buffer, centering, selection) stays on the existing 50 ms timer.Rendering caches. The waveform geometry and per-paragraph shaped text are cached and replayed, so a cursor-only move no longer re-runs the per-pixel waveform loop or re-shapes
FormattedTextevery frame (the latter caused GC pauses). Caches are keyed by view state and cleared inResetCache().Instant waveform seek. A waveform click/drag pins the cursor to the clicked position immediately and holds it until mpv's reported position arrives, instead of lagging on the old spot while mpv applies the async pause+seek.
Smooth center-mode scrolling. In center mode the waveform scroll is driven by the 60 fps cursor timer in lockstep with the cursor instead of the 20 fps heavy timer.
Notes
MainViewModel(timers/interpolation) andAudioVisualizer(render caches); no public API/behavior changes outside the waveform.DispatcherPriorityinversion (mpv render posts starving the cursor timer), not GC or our rendering; that instrumentation has been removed.