Computer Use

Allow ZeroTwo to operate desktop apps as it works

Computer Use lets ZeroTwo read and operate the windows of apps on your computer through the desktop app. It works by reading the selected window (a screenshot plus the accessibility tree) and then clicking, typing, scrolling, dragging and pressing keys. It is desktop-only: it is not available on the web, on iOS or Android, or on Linux. It is built for macOS, and the desktop app also includes a Windows version that is not yet accepted for release, so treat Windows as unverified. Keep tasks narrow, because it can change things outside your project.

Set up Computer Use

Open Settings > Computer use in the desktop app. The Any App switch ("Let ZeroTwo control apps on your computer") turns the feature on. The Computer Use plugin, version 1.1.3, also appears under Customize > Plugins.

On macOS, the system asks for two permissions the first time. Accessibility lets ZeroTwo click and type, and Screen Recording lets it see the window. If an app cannot be controlled, check both under System Settings > Privacy & Security. macOS applies a new Screen Recording grant only after a full restart of the app.

The same settings page also holds the Google Chrome switch (see Browser extension), a Messages switch on macOS, and Always hide picture in picture.

When to use Computer Use

Reach for it when a task depends on a window that files or command output cannot verify: desktop or simulator apps, GUI-only bugs, settings that exist only in a UI, data in apps with no connector, or flows that cross several apps. For a local web app, try the built-in browser first. If an app has a plugin or connector, prefer it for data and repeatable actions.

Start a Computer Use task

Describe the task and name the exact app or flow. You can also type @ and pick the Computer Use plugin.

Open Notes and draft a short summary of today's meeting.

While it works, ZeroTwo shows a live picture-in-picture view of each window it controls, with an agent cursor over the app. Switch the view off with Always hide picture in picture.

Permissions and approvals

The first time ZeroTwo wants to control an app, a dialog titled Desktop control request asks whether to Allow Once, Always Allow or Deny. Allow Once covers the current session, and Always Allow remembers the app. ZeroTwo refuses to control itself, since automating its own window could bypass its approvals.

Computer Use also follows a confirmation policy: ZeroTwo confirms right before risky actions, such as deleting data, submitting forms, sending messages, installing software, changing system settings or confirming a payment. It hands off a few steps entirely, like the final click of a password change. Text that appears in a window or document counts as untrusted, and it does not grant permission by itself.

Safety guidance

ZeroTwo can see the contents of the app window it controls, so keep tasks narrow:

  • Target one app or flow, and stay present for sensitive, account or payment steps.
  • Close apps that show data you do not want captured.
  • Stop the task if the wrong window is touched. You can take over at any time.

In a signed-in browser, actions happen as you. Consider using a separate browser or the built-in browser for them.