Your desktop

Let an agent drive the screen, mouse, and keyboard of your machine.

Run a session

The agent acts on your real machine. It can move the mouse, type, read anything on screen, run shell commands, and read or write files. Prefer a dedicated machine over your primary one.

Stop it with Esc twice, or holo local stop in any terminal, which stops every session driving this machine.

For a pre-built agent, holo -a h/holo-desktop "task" runs h/holo-desktop.

Install

pip install "hai-agents[cli,desktop]"

It works on macOS, Windows, and Linux, from the Python SDK and the CLI. Create an agent whose environment sets host to user_device, then run a session with it. In Python there is nothing else to set up. From the CLI, serve your desktop with holo local desktop and point the agent at the session_id it prints (the CLI example uses the agent the Python example creates).

CLI

# Serve the desktop and leave it running; it prints the session_id it serves.
holo local desktop

# In another terminal, route the agent's desktop environment here.
holo run "Open Notes and write a short summary of today's standup" \
  --agent local-desktop \
  -o 'agent.environments[kind=desktop].session_id=<printed id>'

The program running the agent (your terminal, or the app that launches Python) needs desktop permissions. Your first session triggers the prompts and stops with an error until you grant them. Run the program in a normal desktop session so it can reach the display.

Fields

Field	Default	Description
id	required	Catalog identifier.
kind		"desktop".
host		"user_device".
session_id	null	Command channel of this desktop. The Python SDK sets it. Set it yourself only to attach to holo local desktop.
lazy	false	Connect on the agent’s first desktop action, skipping the initial screenshot.
settle_s	0.5	Seconds the agent waits after each action. 0 to 10.
screenshot_max_width	null	Maximum screenshot width in pixels, 1 to 1920. null uses the host default. Never upscales.

Behavior and limits

When a session starts, the SDK opens a connection inside your Python process. It receives the agent’s actions, carries them out on your desktop, and ties the session to this machine. Watch and steer and changes work as in the cloud.

• A Python process drives one desktop at a time. A new session on it takes it over and cancels the earlier session. Another process that tries to drive the same desktop waits, then fails with DesktopBusyError.
• The connection closes when your Python process exits, and the sessions it served are cancelled. If the connection drops mid-session, the session is cancelled too.
• holo local desktop shrinks screenshots before upload.

Auto-connect scope

You start the session with	Connection
An agent spec or object passed to run_session, start_session, or create_session	The SDK connects your machine for you.
A registered agent referenced by name, or from the H Platform or another machine	Serve the desktop with holo local desktop, then set the printed session_id on the user_device environment (see the CLI example above).
HAI_AUTO_BRIDGE=0 set	Auto-connect is off. Serve the desktop yourself, as above.