migrate-osworld-agent
Migrate an agent from upstream OSWorld into this OSWorld-V2 repository, add matching evaluation entrypoints, and verify the integration.
git clone --depth 1 https://github.com/AMAP-ML/LongHorizon-Harness /tmp/migrate-osworld-agent && cp -r /tmp/migrate-osworld-agent/eval/OSWorldv2-harness/OSWorld-V2/.codex/skills/migrate-osworld-agent ~/.claude/skills/migrate-osworld-agentSKILL.md
# Migrate OSWorld Agent Use this when adding an upstream OSWorld agent to this repo. ## Workflow 1. Read the upstream agent. 2. Read local patterns before editing: - use `scripts/python/run_multienv_claude.py` as the main runner reference - use `scripts/bash/run_multienv_claude.sh` as the shell entrypoint reference - if a similar local agent exists, use it only for interface shape - if none exists, derive the interface from the runner and `DesktopEnv` 3. Copy the agent into `mm_agents/` with a clear, non-conflicting name. 4. Adapt only the repo-facing interfaces: - `predict()` return shape - action dict fields consumed by `DesktopEnv.step()` - `ASK_USER` turns and follow-up user responses - done/fail markers - task current date - platform, screen, provider, and password settings 5. Add a matching multi-env Python runner under `scripts/python/run_multienv_<agent>.py`. - start from the closest retained runner structure - keep task loading, env recreation, checkpoint args, logs, and cleanup behavior - remove provider/model checks that only apply to the source runner 6. Add a small shell entrypoint under `scripts/bash/`. - use `uv run` 7. Keep unrelated agents, scripts, results, and local dirty files out of the change. ## Verify Run fast checks first. - Compile the new Python files. - Check shell syntax. - Check staged diff for whitespace. - Run local fake tests for: - action parsing - `ASK_USER` - user response returning to the agent - checkpoint argument parsing if touched Then run one small real smoke test. - Use the intended provider. - Use a tiny step limit. - Prefer public IP for cloud providers when the local machine cannot reach private IPs. - Confirm the model call happens. - Confirm at least one environment action executes. - Confirm result files and trajectory are written. - Confirm cloud resources are cleaned up. Before opening a PR, stage only the migration files and re-run the fast checks.
Check OSWorld tasks. Validate the evaluation function, verify that the instruction is feasible given the task setup and agent-visible files, inspect setup artifacts when needed, and produce both markdown and structured JSON reports.
Analyze OSWorld-V2 agent trajectory logs and task results to produce actionable insights. Use this skill whenever the user wants to understand agent performance on OSWorld tasks — including analyzing trajectories, reviewing task results, finding error patterns, comparing code vs GUI strategies, identifying which tools/commands the agent used, or deciding which task types to scale up in the benchmark.
Provision and verify an OSWorld-V2 checkout after clone. Use when the user asks for OSWorld-V2 setup, installation, onboarding, AWS provider setup, Docker provider setup, mocked website server setup, GitLab server setup, gated task download, CUA-Harness hybrid experiment setup, or a final runnable export block. The skill should install/configure the selected supported infrastructure where possible, ask for user confirmation or credentials when required, and report what is fully configured versus still blocked.
Reproduce CUA-Harness experiments on WeaveBench from a GitHub checkout. Use when the user wants an AI coding agent to set up dependencies, download WeaveBench assets, prepare the 120G VM, configure Qwen/Anthropic-compatible APIs, run smoke tests, launch full or subset evaluations, inspect logs, or summarize scores for this repository.