Skip to main content
Find a button, field, or menu by its name and press it, without guessing pixel coordinates. Each control gets a ref (d1, d2, …) you pass to the action. This needs the desktop-small template, and the display started. On any other template these calls return an error.

Read the interface

a11y_snapshot returns {rev, elements, truncated} — a flat digest of the tree. Each element carries: Refs are minted per snapshot revision: they are stable until the next snapshot or until the element disappears — acting on a gone element returns a stale-ref error, so re-snapshot after the UI changes. Scope the snapshot instead of walking the whole desktop:
  • scope: "app" + app (name substring) or pid narrows to one application
  • scope: "focus" narrows to the subtree holding keyboard focus
  • roles, name filter elements; max_depth (default 6) and limit (default 500, hard cap 5000 — the reply’s truncated says it hit) bound the walk
  • visible_only: false includes off-screen elements (default keeps only showing+visible)
  • diff: true + rev returns only elements changed since a prior snapshot

Find a control

a11y_find searches the tree directly — role and name (case-insensitive substring) combine, ref scopes the search below one element, max caps results (default 50, hard cap 500).

Press a control

a11y_action performs an accessibility action on a ref → {ok, settled, rev}: The action settles damage-driven like act — it waits for the screen to go quiet after the change, bounded by timeout_ms (default 1500).
Use a11y actions when the element is what matters — dialogs, menus, form fields — and fall back to pixel input for canvas, WebGL, or anything the tree cannot describe.

Endpoints