> ## Documentation Index
> Fetch the complete documentation index at: https://docs.gravixlayer.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Snapshots & Actions

> The element digest and ref-based interactions — the model loop

This is the loop agents run: **snapshot** the page to get a compact digest of
interactable elements, each with a `ref` (`e1`, `e2`, …); **act** on a ref —
click, fill, type; re-snapshot after the page changes; repeat. The digest is
the model's ground truth, and refs keep actions deterministic where
coordinates would not.

CLI and cURL examples assume `$RT` holds a `browser-small` runtime id with
browser `b1` open (see
[Browser Sessions](/documentation/agentruntime/browser-use/browser)).

## Endpoints

| Method | Path | Description |
| - | - | - |
| POST | `/v1/agents/runtime/{runtime_id}/browser/{browser_id}/snapshot` | Element digest with refs |
| POST | `/v1/agents/runtime/{runtime_id}/browser/{browser_id}/actions/{verb}` | click, dblclick, hover, fill, type, press, hotkey, scroll |
| POST | `/v1/agents/runtime/{runtime_id}/browser/{browser_id}/evaluate` | Run JavaScript in the page |
| POST | `/v1/agents/runtime/{runtime_id}/browser/{browser_id}/dialog` | Answer or dismiss a JS dialog |

## Snapshot

`snapshot` returns `{url, title, viewport, rev, elements}` — every visible
element the page offers for interaction (links, buttons, inputs, and anything
with a role, click handler, or tabindex), reduced to what a model needs: ref,
tag, role, accessible name, bounding box, enabled/visible, plus
`href`/`text`/`input_type` where they exist. Each snapshot re-issues the tab's
refs.

```python Python theme={"dark"}
from gravixlayer import GravixLayer

client = GravixLayer()  # reads GRAVIXLAYER_API_KEY
sandbox = client.runtime.create(template="browser-small")
bid = sandbox.browser.open(url="https://example.com")["browser_id"]

snap = sandbox.browser.snapshot(bid)
print(snap["url"], snap["viewport"], snap["rev"])
for el in snap["elements"]:
    print(el["ref"], el["role"], el["name"], el["rect"])

# Diff mode — only elements that are new or changed since the last snapshot
delta = sandbox.browser.snapshot(bid, diff=snap["rev"])
print(len(delta["elements"]))  # 0 — nothing changed
sandbox.kill()
```

```typescript TypeScript theme={"dark"}
import { GravixLayer } from 'gravixlayer';

const client = new GravixLayer(); // reads GRAVIXLAYER_API_KEY
const sandbox = await client.runtime.create({ template: 'browser-small' });
const bid = (await sandbox.browser.open({ url: 'https://example.com' })).browserId;

const snap = await sandbox.browser.snapshot(bid);
console.log(snap.url, snap.viewport, snap.rev);
for (const el of snap.elements) {
  console.log(el.ref, el.role, el.name, el.rect);
}

// Diff mode — only elements that are new or changed since the last snapshot
const delta = await sandbox.browser.snapshot(bid, { diff: snap.rev });
console.log(delta.elements.length); // 0 — nothing changed
await sandbox.kill();
```

```bash CLI theme={"dark"}
gravixlayer runtime browser snapshot "$RT" --browser b1

REV=$(gravixlayer runtime browser snapshot "$RT" --browser b1 --output json | jq -r .rev)
gravixlayer runtime browser snapshot "$RT" --browser b1 --rev "$REV"
```

```bash cURL theme={"dark"}
API="https://api.gravixlayer.ai/v1"

curl -sS -X POST "$API/agents/runtime/$RT/browser/b1/snapshot" \
  -H "Authorization: Bearer $GRAVIXLAYER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"viewport_only":true,"filter_role":"link"}'
```

**Snapshot parameters**

| Parameter | Type | Required | Description |
| - | - | - | - |
| `target_id` | string | No | Tab to snapshot (empty = current tab) |
| `viewport_only` | bool | No | Only elements inside the viewport — SDKs default `true`; omitted on the wire means the whole page |
| `diff` | string | No | Any non-empty value (pass the last `rev`) returns only elements that are new or changed since the tab's previous snapshot; removed elements are not listed |
| `filter_role` | string | No | Only elements whose role contains this text (case-insensitive) |
| `filter_name` | string | No | Only elements whose accessible name contains this text (case-insensitive) |

`viewport` is `{width, height}`; `rev` increments on every snapshot of the
tab. A snapshot returns at most 500 elements. Each element in `elements`:

| Field | Description |
| - | - |
| `ref` | Handle for actions — `e3` |
| `tag`, `role`, `name` | Element identity — `name` is the accessible name |
| `rect` | Bounding box `{x, y, w, h}` in viewport CSS pixels |
| `enabled`, `visible` | Actionability hints |
| `href`, `input_type`, `text` | Per-kind detail (empty when not applicable) |

## Targeting elements

Every action takes a `target`. A ref string (`"e3"`) from the latest snapshot
is the common case — fast, and it pins the exact element the model saw. The
other forms reach elements without a snapshot:

| `by` | Wire form | Resolves to |
| - | - | - |
| `ref` | `{"by":"ref", "value":"e3"}` | Snapshot ref — SDKs expand a bare string to this |
| `css` | `{"by":"css", "value":"#email"}` | CSS selector |
| `role` | `{"by":"role", "value":"button", "attrs":{"name":"Deploy"}}` | ARIA role + exact accessible name |
| `text` | `{"by":"text", "value":"Deploy", "exact":true}` | Visible text — substring unless `exact`; the shortest match wins |
| `xpath` | `{"by":"xpath", "value":"//textarea"}` | XPath |
| `coords` | `{"by":"coords", "attrs":{"x":"640","y":"420"}}` | Viewport coordinates (CSS pixels; `attrs` values are strings) |
| `frame` | `{"by":"frame", "value":"#checkout", "inner":{"by":"css","value":"#card"}}` | `inner`, resolved inside the iframe that `value` selects — `inner` must be a selector, not a ref |

Shorthand: TypeScript accepts `{x: 640, y: 420}` and Python the tuple
`(640, 420)` for `coords`. On `css` targets, `pierce: true` searches through
shadow roots; any CSS selector also supports `host >>> inner` for one
explicit shadow-root hop. The CLI takes `e3`, `css:…`, `text:…`, `xpath:…`,
`role:…`, or `x,y`.

## Actions

`POST .../actions/{verb}` where verb is one of `click`, `dblclick`, `hover`,
`fill`, `type`, `press`, `hotkey`, `scroll`. Actions wait for the element to
be actionable (visible, stable, enabled, receiving events — editable too for
`fill`/`type`), dispatch real input events, then wait for the page to settle,
and return an [`ActionResult`](/documentation/agentruntime/browser-use/overview#objects).
An element that never becomes actionable is not an HTTP error: the result
comes back `ok: false` with `reason` naming the failed check.

The examples use a small inline page so every target resolves:

```python Python theme={"dark"}
from gravixlayer import GravixLayer

client = GravixLayer()
sandbox = client.runtime.create(template="browser-small")
page = ("data:text/html,<input id=email placeholder=Email>"
        "<textarea id=note></textarea><button>Deploy</button>")
bid = sandbox.browser.open(url=page)["browser_id"]

snap = sandbox.browser.snapshot(bid)
deploy = next(e["ref"] for e in snap["elements"] if e["name"] == "Deploy")

r = sandbox.browser.click(bid, deploy)                    # by ref
print(r["ok"], r["navigated"], r["reason"])

sandbox.browser.fill(bid, {"by": "css", "value": "#email"}, text="user@example.com")
sandbox.browser.type(bid, {"by": "css", "value": "#note"}, "hello", delay_ms=20)
sandbox.browser.press(bid, "Enter")                       # to the focused element
sandbox.browser.hotkey(bid, ["ctrl", "a"])
sandbox.browser.hover(bid, {"by": "role", "value": "button", "attrs": {"name": "Deploy"}})
sandbox.browser.dblclick(bid, {"by": "text", "value": "Deploy"})
sandbox.browser.click(bid, (20, 20))                      # viewport coordinates
sandbox.browser.scroll(bid, dy=600)
sandbox.kill()
```

```typescript TypeScript theme={"dark"}
import { GravixLayer } from 'gravixlayer';

const client = new GravixLayer();
const sandbox = await client.runtime.create({ template: 'browser-small' });
const page =
  'data:text/html,<input id=email placeholder=Email>' +
  '<textarea id=note></textarea><button>Deploy</button>';
const bid = (await sandbox.browser.open({ url: page })).browserId;

const snap = await sandbox.browser.snapshot(bid);
const deploy = snap.elements.find((e) => e.name === 'Deploy')!.ref;

const r = await sandbox.browser.click(bid, deploy);       // by ref
console.log(r.ok, r.navigated, r.reason);

await sandbox.browser.fill(bid, { by: 'css', value: '#email' }, { text: 'user@example.com' });
await sandbox.browser.type(bid, { by: 'css', value: '#note' }, 'hello', { delayMs: 20 });
await sandbox.browser.press(bid, 'Enter');                // to the focused element
await sandbox.browser.hotkey(bid, ['ctrl', 'a']);
await sandbox.browser.hover(bid, { by: 'role', value: 'button', attrs: { name: 'Deploy' } });
await sandbox.browser.dblclick(bid, { by: 'text', value: 'Deploy' });
await sandbox.browser.click(bid, { x: 20, y: 20 });       // viewport coordinates
await sandbox.browser.scroll(bid, { dy: 600 });
await sandbox.kill();
```

```bash CLI theme={"dark"}
gravixlayer runtime browser navigate "$RT" --browser b1 \
  "data:text/html,<input id=email placeholder=Email><textarea id=note></textarea><button>Deploy</button>"

gravixlayer runtime browser action "$RT" --browser b1 click --target "text:Deploy"
gravixlayer runtime browser action "$RT" --browser b1 fill --target "css:#email" --value "user@example.com"
gravixlayer runtime browser action "$RT" --browser b1 type --target "css:#note" --value hello --delay-ms 20
gravixlayer runtime browser action "$RT" --browser b1 press --value Enter
gravixlayer runtime browser action "$RT" --browser b1 hotkey --value ctrl,a
gravixlayer runtime browser action "$RT" --browser b1 hover --target "role:button"
gravixlayer runtime browser action "$RT" --browser b1 click --target 20,20
gravixlayer runtime browser action "$RT" --browser b1 scroll --value 0,600
```

```bash cURL theme={"dark"}
API="https://api.gravixlayer.ai/v1"

curl -sS -X POST "$API/agents/runtime/$RT/browser/b1/navigate" \
  -H "Authorization: Bearer $GRAVIXLAYER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"data:text/html,<input id=email placeholder=Email><textarea id=note></textarea><button>Deploy</button>"}'

curl -sS -X POST "$API/agents/runtime/$RT/browser/b1/actions/click" \
  -H "Authorization: Bearer $GRAVIXLAYER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"target":{"by":"text","value":"Deploy"},"button":"left","settle":true}'

curl -sS -X POST "$API/agents/runtime/$RT/browser/b1/actions/fill" \
  -H "Authorization: Bearer $GRAVIXLAYER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"target":{"by":"css","value":"#email"},"text":"user@example.com","settle":true}'

curl -sS -X POST "$API/agents/runtime/$RT/browser/b1/actions/press" \
  -H "Authorization: Bearer $GRAVIXLAYER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"key":"Enter","settle":true}'

curl -sS -X POST "$API/agents/runtime/$RT/browser/b1/actions/hotkey" \
  -H "Authorization: Bearer $GRAVIXLAYER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"keys":["ctrl","a"],"settle":true}'
```

**Common action options** (every verb accepts them)

| Parameter | Type | Default | Description |
| - | - | - | - |
| `target` | ElementTarget | — | What to act on — `press`, `hotkey`, and `scroll` may omit it |
| `target_id` | string | current tab | Tab to act in (`--tab` in the CLI) |
| `settle` | bool | `true` | Wait for DOM/network quiescence afterward |
| `force` | bool | `false` | Skip the actionability checks |
| `settle_timeout_ms` | int | server default | Cap the post-action settle wait |
| `target_timeout_ms` | int | server default | Cap the actionability wait — `ok: false` + `reason` on expiry |

**Per-verb fields**

| Verb | Fields |
| - | - |
| `click`, `dblclick` | `button` (`left`/`right`/`middle`), `click_count` |
| `hover` | — |
| `fill` | `text` **or** `secret_ref` — replaces the field's value |
| `type` | `text`, `delay_ms` between keystrokes — types at the cursor |
| `press` | `key` — e.g. `Enter`, `Tab`, `Escape`, `Backspace`, `ArrowDown`, or a single letter or digit |
| `hotkey` | `keys` — e.g. `["ctrl","a"]`; modifiers are held while the rest press |
| `scroll` | `dx`, `dy` — with a `target` and no deltas it scrolls that element into view |

### Typing secrets

`fill` with `secret_ref` types the value of a secret without it ever reaching
your client, logs, or model context. The name is a secret key in an
[Identity provider](/documentation/agentruntime/getting-started/identity-providers)
attached to the runtime; a name that is not attached returns `400`.

```python Python theme={"dark"}
from gravixlayer import GravixLayer

client = GravixLayer()
provider = client.identity.providers.create(
    name="Login",
    provider_type="api_key",
    secrets=[{"key": "LOGIN_PASSWORD", "value": "s3cret"}],
)
sandbox = client.runtime.create(template="browser-small", providers=[provider.id])
bid = sandbox.browser.open(url="data:text/html,<input id=pw type=password>")["browser_id"]

r = sandbox.browser.fill(bid, {"by": "css", "value": "#pw"}, secret_ref="LOGIN_PASSWORD")
print(r["ok"])
sandbox.kill()
```

```typescript TypeScript theme={"dark"}
import { GravixLayer } from 'gravixlayer';

const client = new GravixLayer();
const provider = await client.identity.providers.create('Login', {
  providerType: 'api_key',
  secrets: [{ key: 'LOGIN_PASSWORD', value: 's3cret' }],
});
const sandbox = await client.runtime.create({
  template: 'browser-small',
  providers: [provider.id],
});
const bid = (await sandbox.browser.open({ url: 'data:text/html,<input id=pw type=password>' }))
  .browserId;

const r = await sandbox.browser.fill(bid, { by: 'css', value: '#pw' }, { secretRef: 'LOGIN_PASSWORD' });
console.log(r.ok);
await sandbox.kill();
```

```bash CLI theme={"dark"}
PROVIDER_ID=$(gravixlayer provider create "Login" --type api_key \
  --secret "LOGIN_PASSWORD=s3cret" --output json | jq -r '.id // .provider.id')
SRT=$(gravixlayer runtime create -t browser-small --provider "$PROVIDER_ID" \
  --wait --output json | jq -r '.runtime_id')

gravixlayer runtime browser open "$SRT" --url "data:text/html,<input id=pw type=password>"
gravixlayer runtime browser action "$SRT" --browser b1 fill --target "css:#pw" --secret-ref LOGIN_PASSWORD
gravixlayer runtime kill "$SRT" -y
```

```bash cURL theme={"dark"}
API="https://api.gravixlayer.ai/v1"

PROVIDER_ID=$(curl -sS -X POST "$API/identity/providers" \
  -H "Authorization: Bearer $GRAVIXLAYER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name":"Login","provider_type":"api_key","secrets":[{"key":"LOGIN_PASSWORD","value":"s3cret"}]}' \
  | jq -r .provider.id)

SRT=$(curl -sS -X POST "$API/agents/runtime" \
  -H "Authorization: Bearer $GRAVIXLAYER_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"template\":\"browser-small\",\"cloud\":\"aws\",\"region\":\"us-east-1\",\"providers\":[\"$PROVIDER_ID\"]}" \
  | jq -r .runtime_id)

curl -sS -X POST "$API/agents/runtime/$SRT/browser/open" \
  -H "Authorization: Bearer $GRAVIXLAYER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"data:text/html,<input id=pw type=password>"}'

curl -sS -X POST "$API/agents/runtime/$SRT/browser/b1/actions/fill" \
  -H "Authorization: Bearer $GRAVIXLAYER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"target":{"by":"css","value":"#pw"},"secret_ref":"LOGIN_PASSWORD","settle":true}'

curl -sS -X DELETE "$API/agents/runtime/$SRT" \
  -H "Authorization: Bearer $GRAVIXLAYER_API_KEY"
```

## Evaluate

`evaluate` runs a JavaScript expression in the page and returns
`{value, error, is_error}` (`isError` in TypeScript) — `value` is the result
as JSON. Use it for page data, DOM reads, or scroll positions the digest
doesn't expose. Returned promises are awaited.

```python Python theme={"dark"}
from gravixlayer import GravixLayer

client = GravixLayer()
sandbox = client.runtime.create(template="browser-small")
bid = sandbox.browser.open(url="https://example.com")["browser_id"]

print(sandbox.browser.evaluate(bid, "document.title")["value"])   # Example Domain
links = sandbox.browser.evaluate(bid, "document.querySelectorAll('a').length")
status = sandbox.browser.evaluate(bid, "fetch(location.href).then(r => r.status)")
print(links["value"], status["value"])                            # 1 200
sandbox.kill()
```

```typescript TypeScript theme={"dark"}
import { GravixLayer } from 'gravixlayer';

const client = new GravixLayer();
const sandbox = await client.runtime.create({ template: 'browser-small' });
const bid = (await sandbox.browser.open({ url: 'https://example.com' })).browserId;

console.log((await sandbox.browser.evaluate(bid, 'document.title')).value); // Example Domain
const links = await sandbox.browser.evaluate(bid, "document.querySelectorAll('a').length");
const status = await sandbox.browser.evaluate(bid, 'fetch(location.href).then(r => r.status)');
console.log(links.value, status.value); // 1 200
await sandbox.kill();
```

```bash CLI theme={"dark"}
gravixlayer runtime browser evaluate "$RT" --browser b1 "document.title"
gravixlayer runtime browser evaluate "$RT" --browser b1 "fetch(location.href).then(r => r.status)"
```

```bash cURL theme={"dark"}
API="https://api.gravixlayer.ai/v1"

curl -sS -X POST "$API/agents/runtime/$RT/browser/b1/evaluate" \
  -H "Authorization: Bearer $GRAVIXLAYER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"expression":"document.title","await_promise":true,"world":"isolated"}'
```

| Parameter | Type | Required | Description |
| - | - | - | - |
| `expression` | string | Yes | JavaScript expression |
| `target_id` | string | No | Tab (empty = current tab) |
| `await_promise` | bool | No | Await a returned promise — SDKs and CLI default `true`; omitted on the wire means no |
| `world` | string | No | `isolated` (default) sees the DOM but not the page's own globals; `main` runs in the page's JavaScript context |
| `timeout_ms` | int | No | Evaluation timeout (default 30000) |

A thrown exception comes back as `is_error: true` with the message in
`error`, not as an HTTP error.

## Dialogs

`alert`, `confirm`, and `prompt` dialogs are dismissed automatically about four
seconds after they open, so an unanswered dialog never wedges the page. To
answer one yourself, call `dialog` while it is open — the call waits briefly
for a dialog to appear and returns `404` if none does. `accept` confirms
(`true` in the SDKs; omitted on the wire means dismiss), `prompt_text` fills a
`prompt()`, and `auto_dismiss: true` dismisses any open dialog immediately.

```python Python theme={"dark"}
from gravixlayer import GravixLayer

client = GravixLayer()
sandbox = client.runtime.create(template="browser-small")
bid = sandbox.browser.open(url="https://example.com")["browser_id"]

sandbox.browser.evaluate(bid, "setTimeout(() => confirm('Deploy?'), 100)")
sandbox.browser.dialog(bid, accept=True)        # answers the confirm
sandbox.kill()
```

```typescript TypeScript theme={"dark"}
import { GravixLayer } from 'gravixlayer';

const client = new GravixLayer();
const sandbox = await client.runtime.create({ template: 'browser-small' });
const bid = (await sandbox.browser.open({ url: 'https://example.com' })).browserId;

await sandbox.browser.evaluate(bid, "setTimeout(() => confirm('Deploy?'), 100)");
await sandbox.browser.dialog(bid, { accept: true }); // answers the confirm
await sandbox.kill();
```

```bash CLI theme={"dark"}
gravixlayer runtime browser evaluate "$RT" --browser b1 "setTimeout(() => prompt('Name?'), 100)"
gravixlayer runtime browser dialog "$RT" --browser b1 --accept --prompt-text "Ada"
```

```bash cURL theme={"dark"}
API="https://api.gravixlayer.ai/v1"

curl -sS -X POST "$API/agents/runtime/$RT/browser/b1/evaluate" \
  -H "Authorization: Bearer $GRAVIXLAYER_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"expression\":\"setTimeout(() => prompt('Name?'), 100)\"}"

curl -sS -X POST "$API/agents/runtime/$RT/browser/b1/dialog" \
  -H "Authorization: Bearer $GRAVIXLAYER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"accept":true,"prompt_text":"Ada"}'
```

## Notes

* **Act on fresh refs.** Each snapshot re-issues the tab's refs, and a ref
  points at the element's position in the DOM — after a navigation or a
  re-render, snapshot again before acting. A ref the tab has never issued
  returns `404`.
* **If an action returns `ok: false`**, check `reason` — `visible`,
  `enabled`, `editable`, `receives-events`, or `stable`. The element is
  hidden, disabled, covered, or still animating; scroll it into view, wait,
  or pass `force` as a last resort.
* `settle: false` (`--no-settle`) skips the post-action wait — useful when an
  action starts a long navigation you'll wait for with `navigate` or
  `display.wait_stable` instead.
* `evaluate` is full code execution in the page — treat the expression as
  code you ship, not as user input.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.