Skip to main content
See the runtime’s screen, click and type on it, open apps, and watch it live. Use this for windows, dialogs, and pixels. For links and forms, use Browser Use.

Requirements

Seeing and controlling the screen needs desktop-small. desktop.enable returns 400 on a template without a display. Checking or restarting desktop services needs the desktop-small template. desktop.components returns 404 otherwise.
CLI and cURL examples on the following pages use $RT from the example above. Leave the runtime running until you are done. desktop.enable is safe to call first in every script — calling it again returns the display that is already running.

Errors

Coordinates are screen pixels. coordinate_space="screenshot" on a click maps a point in a scaled screenshot back to the screen. Live view links expire after an hour. role="watch" makes a link that can see the screen and cannot type or click.

Next

Screen

Start the display, capture it, and watch it live.

Control

Click, type, and use the clipboard.

Apps

Focus a window, launch an app, or open a file.

Inspect

Find a control by its name and press it.

Record

Record the screen and hand control to a person.

Session

Check desktop services and restart one that failed.