feat: add TRL GRPOTrainer rollout_func for WAA environments

[abrichr]

Summary

• make_waa_rollout_func() wraps WAADesktopEnv into TRL's experimental rollout_func API
• Handles VLM multimodal generation (screenshot → action tokens → logprobs)
• Dense rewards via milestones automatically used when TaskConfig has milestones
• parse_action_json() handles common VLM quirks (thinking tokens, markdown fences, unknown types)
• Integrates with TaskConfig YAML tasks and evaluate_dense()

Usage

from trl import GRPOConfig, GRPOTrainer
from openadapt_evals.training.trl_rollout import make_waa_rollout_func

rollout_func = make_waa_rollout_func(
    adapter=WAALiveAdapter(config),
    task_configs=TaskConfig.from_dir("./tasks/"),
)

trainer = GRPOTrainer(model=model, rollout_func=rollout_func, ...)
trainer.train()

Test plan

• 15 tests passing (10 parser + 5 integration with mock adapter)
• GPU integration test with real VLM model (requires g5.xlarge)

🤖 Generated with Claude Code