Computer use

Computer use is a form of agentic AI in which the model looks at a screenshot of a real desktop or browser, decides where to click and what to type, and emits low-level actions such as click, type, and scroll. Your runtime carries out each action, usually in a sandboxed machine, takes a fresh screenshot, and the loop repeats until the task is done or the agent gives up. It is how you automate software that has no API: legacy enterprise apps, awkward web forms, anything behind a login. Anthropic launched computer use in public beta with the upgraded Claude 3.5 Sonnet on October 22, 2024. OpenAI followed with Operator on January 23, 2025, folded it into ChatGPT agent in July 2025, and has since retired that product as well, pointing users to ChatGPT Work and a cloud browser inside ChatGPT. Google shut down its Project Mariner browser agent on May 4, 2026. On August 19, 2026, computer use left beta on the Claude API as the `computer_toolset_20260801` toolset, with 17 member tools, batch actions, no beta header, and about 4,500 extra input tokens per request for declaring it. Older models such as Opus 4.7 and Sonnet 4.6 still need the beta `computer_20251124` version. The same capability runs in Anthropic's desktop app on macOS and Windows. Since March 23, 2026, Pro and Max subscribers can turn on computer use in Cowork and Claude Code, first released as a research preview. It starts switched off; once enabled on macOS 15 or later, Claude works in background windows by default and does not take over your pointer or keyboard. Screen control is the last resort: Anthropic documents that in Cowork Claude "uses the most precise tool first", trying a connector, then the browser, and only then direct screen interaction. It is still slow and unreliable next to an API call. In the OSWorld-Human study (arXiv 2506.16042), a computer-use agent took 12 minutes to double-space two paragraphs, a task a person with basic computer experience finishes in under 30 seconds, and for the Agent S2 agent, planning and reflection accounted for 76% to 96% of total task latency. Benchmark scores also depend on how you grade. On XLANG Lab's OSWorld 2.0, Anthropic reports Claude Fable 5.1 fully completing 41.7% of tasks under strict grading and scoring 77.9% with partial credit, and Opus 5 at 39.6% and 75.4%. A half-filled form counts toward the second number and is still an unfinished job in production. Where it earns its cost: back-office work where the alternative is a person re-keying data, and bridging to systems with no API or connector. Where it does not: anything time-sensitive, anything with a real API you could call instead, and anything that has to be right 99.9% of the time.