Support the LingBot-Video MoE refiner (two-stage high-resolution refinement) - #1546
Draft
NancyFyong wants to merge 38 commits into
Draft
Support the LingBot-Video MoE refiner (two-stage high-resolution refinement)#1546NancyFyong wants to merge 38 commits into
NancyFyong wants to merge 38 commits into
Conversation
Model code: - LingBotVideoDiT (diffsynth/models/lingbot_video_dit.py) video denoiser - Qwen3-VL text-encoder wrapper (lingbot_video_text_encoder.py) - T2V/V2V pipeline (diffsynth/pipelines/lingbot_video.py) - LingBotVideoUniPCScheduler (FlowMatchScheduler subclass): UniPC multistep for inference, flow-matching schedule for training - Reuse QwenImageVAE via an additive, backward-compatible 5D-video encode/decode path in qwen_image_vae.py - Register models + state-dict converters in model_configs.py Training (issue's core deliverable): - LingBotVideoTrainingModule (examples/lingbot_video/model_training/train.py) with the flow-matching SFT objective - Attention-only LoRA launch script (to_q,to_k,to_v,to_out; MoE/FFN/router left frozen) Examples + docs: - T2V/V2V inference and low-VRAM examples - examples/lingbot_video/README.md Part of the SFT training support for modelscope#1530. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… LingBot-Video
- docs/{en,zh}/Model_Details/LingBot-Video.md + index.rst toctree entries
- two-stage prompt rewriter (structured JSON captions) + offline caption rewrite tool
- LoRA training validation script
- relocate low-VRAM inference into model_inference_low_vram/
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Register the MoE 30B-A3B DiT (128 experts, top-8 group-limited routing, 1 shared expert) and add VRAM management module maps for the LingBot-Video DiT and text encoder, which previously had no entry and fell back to whole-model wrapping. The MoE experts store their weights as grouped nn.Parameter rather than nn.Linear, so the expert container itself is wrapped -- without that the experts, which are the bulk of the model, would stay resident. Also fixes three issues surfaced while validating the MoE path: - _run_grouped_experts guarded the fast path with hasattr(torch, "_grouped_mm"), which is True on CPU even though the kernel is CUDA-only. Now device-aware. - The block and DiT forwards derived the activation dtype from .weight.dtype. Under VRAM management the wrapped layer holds its weight in the offload dtype (or on meta for disk offload), so this read the wrong dtype. Resolved from computation_dtype instead. - The fp32-pinned timestep and modulation MLPs were fed fp32 activations while VRAM management wraps every nn.Linear at computation_dtype, raising a dtype mismatch on any low-VRAM run. Resolved per-layer. The dense low-VRAM example set only vram_limit, which is a no-op without offload_dtype/offload_device, so VRAM management never activated; corrected along with the same claim in the docs. Validated against the official implementation with identical weights: router selection and gate scores are bit-exact, and the full DiT matches bit-exactly at fp32 (bf16 differs only because diffusers' .to() downcasts the fp32-pinned modules). The registered model_hash matches the released checkpoint headers across all 977 keys with no shape mismatches. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…t rewriter - Scheduler: use FlowMatchScheduler (Wan template) instead of a bespoke UniPC scheduler; flow_match.py restored byte-identical to main. - VAE: restore QwenImageVAE to upstream; the 5D-video encode/decode and latent normalisation now live in LingBotVideoPipeline (encode_video / decode_video). - Rewriter: keep only normalize_caption in diffsynth core; move the two-stage prompt rewriter + system prompts into the inference examples. - Examples: merge into one lingbot-video-dense-1.3b.py (T2V / V2V / optional rewrite); drop the separate _rewrite.py. - Training .sh: use --model_id_with_origin_paths instead of local --model_paths. - Docs: sync en/zh Model_Details + example README. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Under low-VRAM offload the AutoWrapped modules compute weights in bf16 but do not cast their inputs, and reading `.weight.dtype` from outside the wrapper returns the resident fp8 dtype. The parameter-free sinusoidal `time_proj` always returns fp32, so feeding it straight into the bf16 `time_embedder` MLP raised "mat1 and mat2 must have the same dtype, but got Float and BFloat16"; `proj_out` had the mirror bug via `self.proj_out.weight.dtype` (fp8 under offload). Cast both inputs to the running compute dtype (`joint`) instead, which is bf16 under offload and matches the weight dtype in a full-precision run. No effect on the normal (non-offload) path. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The trimmed diffsynth/pipelines/lingbot_video_prompt_rewriter.py held only
normalize_caption (caption -> compact-JSON serialisation) after the prompt
rewriter was moved to the examples. Its sole core consumer is the pipeline,
which already calls it internally, so keeping a separate one-function module
added a misleadingly-named file ("prompt_rewriter" that no longer rewrites)
for no benefit.
Move normalize_caption (and its _serialize_caption/_caption_from_sample
helpers) into diffsynth/pipelines/lingbot_video.py as module-level functions
and delete the old module. All importers now pull it from lingbot_video; the
example inference/training scripts already imported the pipeline, so this only
tidies their imports. The prompt-rewriting logic stays out of core, in
examples/lingbot_video/model_inference/prompt_rewriter.py.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Companion to the previous commit, which only recorded the deletion of lingbot_video_prompt_rewriter.py. This commit adds normalize_caption (and its _serialize_caption/_caption_from_sample helpers) into diffsynth/pipelines/lingbot_video.py as module-level functions, and repoints every importer (the pipeline itself, the model_inference / low_vram example scripts, the example prompt_rewriter, train.py and rewrite_captions.py) at diffsynth.pipelines.lingbot_video. Restores a working tree: the two commits together move the single remaining function into the pipeline module. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Collapse the AI-generated multi-paragraph banners and docstrings to concise one-line "why" comments matching DiffSynth's light comment style. No code changes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ixes to the MoE branch
Brings the reviewer-mi804 integration fixes from lingbot_sft onto the MoE-30B-A3B
branch so both variants share one conventions-aligned implementation:
- Scheduler: drop LingBotVideoUniPCScheduler, use FlowMatchScheduler (Wan template);
diffsynth/diffusion/flow_match.py restored byte-identical to main.
- VAE: restore diffsynth/models/qwen_image_vae.py byte-identical to main; the 5D-video
encode/decode lives in the pipeline (encode_video / decode_video).
- Rewriter: move the prompt-rewriting engine out of diffsynth/ into
examples/lingbot_video/model_inference/{prompt_rewriter,system_prompts}.py; the core
keeps only normalize_caption, folded into the pipeline module.
- Examples/docs/training .sh updated to model_id_with_origin_paths and the merged layout.
Conflict resolutions:
- lingbot_video_dit.py: kept the MoE resolve_bulk_dtype() dtype handling (superset of
the sft joint.dtype fix, and consistent with the rest of the file), plus sft's trimmed
comments and the router fp32 pin.
- low-VRAM dense example + docs: kept sft's corrected vram_limit wording, kept the MoE
section and MoE-specific files.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
time_embedder is fp32-pinned (LINGBOT_VIDEO_FP32_MODULES), so under a standard load its weights stay fp32 while joint (the bulk hidden state) is bf16. Casting the fp32 timestep_proj to joint.dtype fed bf16 into the fp32 Linear, raising "mat1 and mat2 must have the same dtype" on any non-offload run. Cast to the layer's own weight dtype instead: fp32 under standard load, bf16 under low-VRAM offload (where the wrapper casts these MLPs to the compute dtype). proj_out keeps joint.dtype since it is a bulk layer held in fp8 under offload. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Kept the MoE-branch resolve_bulk_dtype() helper for the time_embedder input cast (handles VRAM-wrapped Linears); the sft-branch used a plain .weight.dtype which the MoE version supersedes.
Condition generation on a first frame via a new `input_image` argument. Dense-1.3B reuses its T2V checkpoint (DiT unchanged, in_channels=16); the condition frame is used twice, matching the original LingBot-Video i2v pipeline: fed to the Qwen3-VL text encoder as a visual reference (image tokens prepended to the prompt), and VAE-encoded to a clean latent pinned into the first temporal slot before sampling and after every scheduler step. - LingBotVideoUnit_ImageEmbedder builds the cond latent (via encode_video) and the smart-resized VLM image; no-op for T2V/V2V. - LingBotVideoUnit_PromptEmbedder passes the image to the processor when present. - Ported smart_resize / cover-resize+center-crop / _vlm_image helpers. - Example lingbot-video-dense-1.3b_ti2v.py + released first frame and caption. - EN/ZH docs and example README document the TI2V path and `input_image`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
t2i is text-to-video with num_frames=1 through the same pipeline and DiT (no separate image weight), matching the official runner which sets num_frames=1 and swaps in the still-image negative prompt. - Add DEFAULT_NEGATIVE_PROMPT_IMAGE (verbatim from the official pipeline): drops the temporal/motion terms that cannot apply to a single frame. - Example lingbot-video-dense-1.3b_t2i.py + released still-image caption prompts/t2i_example.json; saves the single returned frame as PNG. - EN/ZH docs and example README document the t2i path. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- model_inference_low_vram: add ti2v (input_image) and t2i (num_frames=1 + DEFAULT_NEGATIVE_PROMPT_IMAGE) low-VRAM variants, mirroring the t2v script. - train.py: add opt-in --first_frame_as_condition for image-to-video LoRA. It conditions each clip on its own first frame; FlowMatchSFTLoss already pins the clean first-frame latent and excludes it from the loss, so no core change is needed. A distinct condition column still works via --extra_inputs input_image. - model_training: add ti2v LoRA launch script and validate_lora example. - docs (en/zh) + example README: reference the new scripts and TI2V LoRA. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… reviewer's refactor (531ba56) - TI2V + T2I inference / low-VRAM examples rewritten to the new pipeline API: read structured captions from JSON, pass pipe.default_negative_prompt(_image), drop the removed module-level normalize_caption / DEFAULT_NEGATIVE_PROMPT_IMAGE imports. - Add default_negative_prompt_image attribute to LingBotVideoPipeline (T2I variant with temporal terms removed) so t2i examples can reference it symmetrically with default_negative_prompt. - TI2V LoRA training script aligned to the reviewer's new t2v LoRA (dataset path, num_frames=81, num_epochs=5, dataset_repeat=50); validate_lora rewritten to use pipe.default_negative_prompt + json.load. - New TI2V full-parameter training script + validate_full script (parallel to the reviewer's t2v full training), toggled via --first_frame_as_condition. - Docs rewritten to match the standard Model_Details template: single-checkpoint overview covering T2V / TI2V / T2I with 3-row Examples table, input_image param documented, prompt-rewriter moved under Model Inference, VAE-internals text dropped per reviewer's "keep it about model info + usage" directive. - README.md / README_zh.md: add Update History entry and Video Synthesis series block (Quick Start + Examples table) for LingBot-Video.
Low-VRAM TI2V feeds the condition frame through the Qwen3-VL vision tower, but the LingBotVideoTextEncoder VRAM-management map did not cover its module types. The vision LayerNorm / patch-embed weights stayed on `meta` and the vision RoPE `inv_freq` (a non-persistent buffer, absent from the checkpoint) stayed on CPU, so a TI2V run under offload died with a cuda/cpu device mismatch. T2V / T2I never touch that path. Add LayerNorm, Qwen3VLVisionPatchEmbed and Qwen3VLVisionRotaryEmbedding to the map. PatchEmbed is wrapped whole (not its inner Conv3d) because its forward reads `self.proj.weight.dtype` to cast the input, which under offload would otherwise pick up the fp8 dtype and crash on the bf16 bias. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The input_image param (the TI2V condition frame) was grouped under a '# Video-to-video' comment. Relabel it '# Image-to-video (TI2V)' to match wan_video.py's param grouping and the file's own TI2V terminology. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Address reviewer comments on lingbot_video.py (:20, :185, :214): - Move the module-level TI2V constants (IMAGE_MIN/MAX_TOKEN_NUM, MAX_RATIO, SPATIAL_MERGE_SIZE) and helpers (smart_resize, _round/_ceil/_floor_by_factor, _pixel_tensor_to_pil) plus the pipeline methods preprocess_cond_image, _vision_patch_size and _vlm_image into LingBotVideoUnit_ImageEmbedder, the only consumer. Move IMG_PROMPT_TEMPLATE into LingBotVideoUnit_PromptEmbedder. - Move the pre-loop first-frame latent pin out of __call__ and into LingBotVideoUnit_ImageEmbedder.process; reorder self.units so InputVideoEmbedder (which produces `latents`) runs before ImageEmbedder (which pins into it) and before PromptEmbedder (which consumes vlm_image). __call__ keeps only the per-step re-pin inside the denoise loop. Behavior-preserving for inference and training (the loss overwrites `latents` and does its own first-frame pin, so the unit-level pin is inert there). Verified on GPU: TI2V pins frame 0 to the condition frame; T2V no-op path OK. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…g, keep MoE-30B Brings the lingbot_sft refactor and features onto the MoE branch: - Pipeline: adopt sft's unit-based LingBotVideoPipeline (model-agnostic; the MoE variant runs through it unchanged via self.dit). - New Dense-1.3B examples/scripts: TI2V and T2I inference (+ low-VRAM), full and LoRA training, validate_full/validate_lora; scripts/ relocation. Conflict resolutions: - lingbot_video_dit.py: keep MoE's resolve_bulk_dtype / fp32-pinned modules (incl. router). It is the later superset of sft's 7caae83 x.dtype fix, and the file is built on it throughout. - vram_management_module_maps.py: union both maps. The DiT map keeps MoE's fine-grained entries (GroupedExperts wrapped, NOT the whole block, so the walker doesn't keep all experts resident); the text-encoder map takes sft's Qwen3-VL vision-tower entries for TI2V low-VRAM. - docs (en/zh): integrate the MoE-30B-A3B sections with sft's TI2V/T2I/training content into one coherent doc (4-row model table, T2I-aware params). Style: align the MoE inference examples to the refactored dense layout (section banners, explicit negative_prompt=pipe.default_negative_prompt). Verified in the moe worktree with local dense-1.3b weights: the DiT parses and imports; dense-1.3b normal and low-VRAM (fp8 offload) inference both produce correct frame counts with no dtype/device errors (the 7caae83 regression case); a tiny random MoE DiT forward runs grouped_mm on CUDA with finite output. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ventions - lingbot_video_dit.py: drop unused LINGBOT_VIDEO_FP32_MODULES / should_keep_in_fp32, remove docstring and inline comments, keep norm_out_modulation call on one line - model_configs.py: keep only the ModelConfig example comment, collapse extra_kwargs onto a single line - vram_management_module_maps.py: drop explanatory comments - MoE inference examples: remove all comments and section dividers - docs: restore the standard template layout, fold the MoE description into the model overview section - README / README_zh: add the missing MoE row so the examples table matches the docs model overview table
- use the released structured-JSON caption from prompts/t2v_example_1.json instead of free-form prose, matching the Dense T2V / TI2V / T2I examples - switch the low-VRAM example to the Dense disk-offload profile (disk -> cpu fp8 -> cuda bf16, vram_limit - 0.5) - add the video-to-video block to the low-VRAM example so it covers the same tasks as the Dense low-VRAM T2V example
…example dataset - T2V examples now download the released structured caption through dataset_snapshot_download, reusing the lingbot-video-dense-1.3b dataset directory, exactly like the Dense T2V examples - add TI2V and T2I MoE examples (inference + low VRAM) mirroring the Dense templates: shared prompts/ captions, assets/ti2v_first_frame.png, default_negative_prompt_image and num_frames=1 for T2I - docs and README: split the MoE row into T2V / TI2V / T2I rows and mention the MoE variant in the release note
Disk offload only materialises weights for module types listed in the VRAM map, so the per-type MoE map left two owners of raw parameters on `meta`: `LingBotVideoBlock.scale_shift_table` and the router weight / correction bias. Both bf16 and low VRAM Dense and MoE inference crashed with "Tensor on device meta is not on the expected device cuda:0". - map `LingBotVideoBlock` and `LingBotVideoRouter` to `AutoWrappedNonRecurseModule`, which manages a module's own parameters while its children keep their individual wrappers, and drop the dead `torch.nn.LayerNorm` entry (`norm_out` has no affine parameters) - cast `scale_shift_table` to the modulation dtype/device in the block forward, following `wan_video_dit.DiTBlock` - keep `e_score_correction_bias` as a non-trainable parameter so the disk loader restores its checkpoint values instead of leaving CPU zeros, and add it in float32 Verified on Robbyant/lingbot-video-moe-30b-a3b: all six MoE examples and the Dense low VRAM examples run, low VRAM peaks at 29.5 GiB versus 74.5 GiB for bf16, and the low VRAM T2I output matches the bf16 output to 16.9/255.
…the MoE branch Upstream squash-merged the Dense-1.3B integration with review changes, so realign the MoE side with what landed: - Drop lingbot_video_text_encoder and its converter; the pipeline now loads Qwen3-VL through Krea2TextEncoder, and the Qwen3-VL vision-tower entries live in the Krea2TextEncoder VRAM map. - Keep a single LingBotVideoDiT VRAM map entry: the MoE variant needs LingBotVideoBlock / LingBotVideoRouter as AutoWrappedNonRecurseModule and LingBotVideoGroupedExperts as AutoWrappedModule so the 128 experts stay individually streamable under offload. - Rename the MoE examples to the upstream _t2v / _ti2v / _t2i scheme and regenerate them line-by-line from the merged Dense examples: Qwen3-VL text encoder + processor configs, captions and condition frames pulled from data/diffsynth_example_dataset, upstream output naming. - Re-add the MoE registry entry, the three MoE rows in both docs tables and both README tables, the MoE paragraph in Model Overview and the release note.
…rectories
Dense and MoE share one example dataset on ModelScope; only
lingbot_video/lingbot-video-dense-1.3b_{t2v,ti2v,t2i}/ exists, so the MoE
examples read the same captions and condition frame instead of a MoE-specific
directory that was never published.
The bulk dtype is already the dtype of the incoming hidden states, and the wrapped linears cast their own weights, so the helper only restated what x.dtype / joint.dtype already carry. Removing it makes the file identical to the Dense implementation except for the offload fixes; TI2V output stays bit-identical in bf16 and under low-VRAM offload.
…fload
AutoWrappedNonRecurseModule leaves parameter casting to the model, and the
grouped-expert matmuls read experts.w1/w2/w3 directly instead of going through
a forward, so an AutoWrappedModule wrapper never got the chance to cast them.
With a vram_limit that actually forces layers to stay on CPU, the router
projection and the expert matmuls therefore hit CPU weights:
RuntimeError: Expected all tensors to be on the same device,
but got mat2 is on cpu
Move the experts to AutoWrappedNonRecurseModule as well and cast the router
weight, the correction bias and the three expert tensors to the token device
and dtype at use, matching how scale_shift_table is already handled. bf16 and
default low-VRAM outputs stay bit-identical.
Contributor
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
LingBot-Video MoE Refiner (two-stage high-resolution refinement)
Summary
Support the refiner shipped with
Robbyant/lingbot-video-moe-30b-a3b, stacked on the MoE integration in #1545. The refiner is a second 30B-A3B DiT underrefiner/that performs a short second pass at a higher resolution over an already generated clip — the official setup generates at 480x832 with 40 steps, then refines at 1088x1920 with 8 steps.refiner/config.jsonmatchestransformer/config.json, and the two directories have the same key+shape hash (426db1dddf14e8e0f7522d7360745d27, 977 keys / 13 shards), so no new registry entry is needed — the existing MoE entry loads the refiner when the shard glob points atrefiner/. The weights themselves are a fine-tune, not a copy (cosine 0.998 on the attention projections, 0.083 one_score_correction_bias).How it works
The refinement pass is SDEdit at a higher resolution, so it reuses the existing
input_videopath: the base clip is read back at the target resolution, VAE-encoded, and noised tosigma=t_thresh. Two additions make the schedule match the official implementation:FlowMatchSchedulergains a"LingBot-Video"template. Witht_thresh=Noneit is value-identical to theWantemplate it replaces; witht_threshset it truncates the shifted sigma grid at the threshold, pins the first sigma exactly att_thresh, and appendssigma_tail_stepsextra low-noise steps ending atsigma_min.LingBotVideoPipeline.__call__gainst_thresh(defaultNone, i.e. plain generation) andsigma_tail_steps(default2). Whent_threshis set, TI2V re-pins the clean first-frame latent after every scheduler step and conditions on text only, matching the official refiner path.For
num_inference_steps=8, sigma_shift=3.0, t_thresh=0.85, sigma_tail_steps=2this reproduces the officialcompute_refiner_sigmasoutput to 2.4e-08:Usage is a second pipeline load with a different shard glob:
Examples
All paths are relative to
examples/lingbot_video/. Each script runs both stages end to end (base generation, then reload with the refiner shards):model_inference/lingbot-video-moe-30b-a3b_t2v_refiner.pymodel_inference_low_vram/lingbot-video-moe-30b-a3b_t2v_refiner.pymodel_inference/lingbot-video-moe-30b-a3b_ti2v_refiner.pymodel_inference_low_vram/lingbot-video-moe-30b-a3b_ti2v_refiner.pyPrompts and the TI2V condition frame come from the same published example dataset directories the other LingBot-Video examples use. Docs:
docs/en/Model_Details/LingBot-Video.md,docs/zh/Model_Details/LingBot-Video.md(new "Two-stage refinement" section,t_thresh/sigma_tail_stepsparameter entries); both README example tables gain two rows.Results
Base pass — 480x832, 40 steps (input to the refiner)
Refinement pass — 1088x1920, 8 steps
Detail comparison — same center crop, top bicubic-upscaled base, bottom refiner output
TI2V refinement — condition frame re-pinned after every step
Measured on 1 GPU with the low-VRAM configuration: the 1088x1920x81 pass runs at ~218 s/it for 8 steps and peaks at 53.5 GiB. Laplacian variance at matched resolution is 2.74x (T2V) and 2.50x (TI2V) that of a bicubic upscale of the base clip, while the layout is preserved (MAE 6.45 / 255 between the downscaled refined clip and the base clip). For TI2V, the first output frame differs from the condition image by MAE 2.14 / 255 — the VAE round trip — versus 24.09 for the base path, which pins the condition latent only once.
Dependencies
No new packages, no new model registry entries.
Notes
t_thresh=Nonereturns exactly the same sigmas and timesteps as theWantemplate for the configurations the pipeline and trainer use. End to end, the low-VRAM MoE T2V, V2V and TI2V examples produce byte-identical outputs before and after this change (md5d3017c72…,ab05029b…,67896646…respectively; T2V rerun twice to confirm the comparison is meaningful).--reuse_condition_features,--batch_cfg, context parallelism, and an FP8 fused-MoE runtime. Those are orthogonal to this PR.