Anthropic is turning Claude Code into something that feels a lot less like a code assistant and a lot more like a junior engineer who can literally sit at your Mac, click around, and test what it just built for you. The new “computer use” feature plugs directly into Claude Code from the command line and, in its current research preview, lets Claude open apps, drive your UI, and verify changes without you ever touching the mouse.
At a basic level, computer use is Anthropic’s way of letting Claude operate your desktop when APIs, plugins, or shell commands aren’t enough. Anthropic already had a pretty rich tool story with MCP servers (for structured integrations), Bash access (for shell work), and a Chrome extension for browser automation, but those all assume the thing you want to control exposes a clean interface. When none of that exists—for example, a stubborn native GUI app, a custom internal tool, or a simulator—Claude can now fall back to full-on screen control: taking screenshots, moving the mouse, clicking, typing, and navigating like a human user.
The headline use case Anthropic is pushing is end‑to‑end development workflows straight from the CLI. In a single prompt, Claude can write or edit code, compile a SwiftUI or Electron app, launch the local build, then actually click through the UI to find a bug, patch it, and rerun the test flow to confirm the fix. Because it is operating on anything your Mac can open, it’s not limited to developer tools—it can drive simulators, in-house GUI tools with no command-line interface, or even that one ancient desktop app your team never bothered to integrate with.
From a user experience perspective, this all sits on top of Claude Code and Claude Desktop rather than replacing them. On macOS, computer use comes in as a built‑in MCP server called computer-use, which is disabled by default; you explicitly turn it on in a Claude Code session or via the desktop app’s settings. Once enabled, you can start a Claude Code session from the CLI, ask it to, say, “run my test suite, open the app, and walk through the checkout flow,” and Claude decides when it needs to invoke computer use versus sticking to tools like Bash or the browser integration.
Anthropic has clearly tried to wrap this power inside some guardrails. On the desktop, computer use is opt‑in, gated behind explicit macOS privacy prompts for accessibility and screen recording, so Claude can click, type, and see what’s on screen only after you grant permission. There’s also a machine‑wide lock: only one Claude Code session can control the computer at a time, and when that session starts, other apps are temporarily hidden so Claude’s clicks are confined to the approved context. Your terminal stays visible but is excluded from screenshots, which means you can watch the session live while Claude never sees its own output and can’t leak the transcript via screenshots.
Computer use also inherits Anthropic’s existing governance levers. Power users can fine‑tune what Claude is allowed to touch using configuration files and per‑app permissions, and IT teams can manage Claude Desktop via standard device‑management tools like Jamf or group policy to enable or disable Claude Code or computer use at scale. At the same time, the community is already starting to debate how far they’re comfortable letting an AI loose on their “actual computer,” with some developers advocating conservative permission setups or sandboxed environments for anything involving production data.
For developers and power users, the implications are pretty big. Instead of just generating code and leaving you to wire everything up, Claude Code can now participate in all the unglamorous but necessary glue work: configuring tools, clicking through wizards, filling forms, and reproducing bugs in a local UI. Early guides are already pitching it as a way to automate social media dashboards, ad campaigns, internal admin panels, and other browser‑ or app‑heavy workflows—all driven from text instructions. Combined with Anthropic’s broader push on advanced tool use and the Model Context Protocol, it nudges Claude closer to being a general “agent” that understands your codebase and can enact changes across the rest of your software stack.
Right now, there are a few clear caveats: computer use in Claude Code is labeled as a research preview, it’s limited to Pro and Max subscribers on macOS, and Anthropic is nudging people toward using more structured integrations (MCP, Bash, browser) whenever possible because computer use is slower and more general‑purpose. But even in this early form, the feature shifts the expectations around what “coding with AI” looks like—from generating patches to actually running, clicking, and validating those changes on your machine while you, as one commenter put it, grab a cup of chai and watch.
Discover more from GadgetBond
Subscribe to get the latest posts sent to your email.
