Snapshots & Actions

This is the loop agents run: snapshot the page to get a compact digest of interactable elements, each with a ref (e1, e2, …); act on a ref — click, fill, type; re-snapshot after the page changes; repeat. The digest is the model’s ground truth, and refs keep actions deterministic where coordinates would not. CLI and cURL examples assume $RT holds a browser-small runtime id with browser b1 open (see Browser Sessions).

Endpoints

Method	Path	Description
POST	`/v1/agents/runtime/{runtime_id}/browser/{browser_id}/snapshot`	Element digest with refs
POST	`/v1/agents/runtime/{runtime_id}/browser/{browser_id}/actions/{verb}`	click, dblclick, hover, fill, type, press, hotkey, scroll
POST	`/v1/agents/runtime/{runtime_id}/browser/{browser_id}/evaluate`	Run JavaScript in the page
POST	`/v1/agents/runtime/{runtime_id}/browser/{browser_id}/dialog`	Answer or dismiss a JS dialog

Snapshot

snapshot returns {url, title, viewport, rev, elements} — every visible element the page offers for interaction (links, buttons, inputs, and anything with a role, click handler, or tabindex), reduced to what a model needs: ref, tag, role, accessible name, bounding box, enabled/visible, plus href/text/input_type where they exist. Each snapshot re-issues the tab’s refs.

from gravixlayer import GravixLayer

client = GravixLayer()  # reads GRAVIXLAYER_API_KEY
sandbox = client.runtime.create(template="browser-small")
bid = sandbox.browser.open(url="https://example.com")["browser_id"]

snap = sandbox.browser.snapshot(bid)
print(snap["url"], snap["viewport"], snap["rev"])
for el in snap["elements"]:
    print(el["ref"], el["role"], el["name"], el["rect"])

# Diff mode — only elements that are new or changed since the last snapshot
delta = sandbox.browser.snapshot(bid, diff=snap["rev"])
print(len(delta["elements"]))  # 0 — nothing changed
sandbox.kill()

import { GravixLayer } from 'gravixlayer';

const client = new GravixLayer(); // reads GRAVIXLAYER_API_KEY
const sandbox = await client.runtime.create({ template: 'browser-small' });
const bid = (await sandbox.browser.open({ url: 'https://example.com' })).browserId;

const snap = await sandbox.browser.snapshot(bid);
console.log(snap.url, snap.viewport, snap.rev);
for (const el of snap.elements) {
  console.log(el.ref, el.role, el.name, el.rect);
}

// Diff mode — only elements that are new or changed since the last snapshot
const delta = await sandbox.browser.snapshot(bid, { diff: snap.rev });
console.log(delta.elements.length); // 0 — nothing changed
await sandbox.kill();

gravixlayer runtime browser snapshot "$RT" --browser b1

REV=$(gravixlayer runtime browser snapshot "$RT" --browser b1 --output json | jq -r .rev)
gravixlayer runtime browser snapshot "$RT" --browser b1 --rev "$REV"

API="https://api.gravixlayer.ai/v1"

curl -sS -X POST "$API/agents/runtime/$RT/browser/b1/snapshot" \
  -H "Authorization: Bearer $GRAVIXLAYER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"viewport_only":true,"filter_role":"link"}'

Snapshot parameters

Parameter	Type	Required	Description
`target_id`	string	No	Tab to snapshot (empty = current tab)
`viewport_only`	bool	No	Only elements inside the viewport — SDKs default `true`; omitted on the wire means the whole page
`diff`	string	No	Any non-empty value (pass the last `rev`) returns only elements that are new or changed since the tab’s previous snapshot; removed elements are not listed
`filter_role`	string	No	Only elements whose role contains this text (case-insensitive)
`filter_name`	string	No	Only elements whose accessible name contains this text (case-insensitive)

viewport is {width, height}; rev increments on every snapshot of the tab. A snapshot returns at most 500 elements. Each element in elements:

Field	Description
`ref`	Handle for actions — `e3`
`tag`, `role`, `name`	Element identity — `name` is the accessible name
`rect`	Bounding box `{x, y, w, h}` in viewport CSS pixels
`enabled`, `visible`	Actionability hints
`href`, `input_type`, `text`	Per-kind detail (empty when not applicable)

Targeting elements

Every action takes a target. A ref string ("e3") from the latest snapshot is the common case — fast, and it pins the exact element the model saw. The other forms reach elements without a snapshot:

`by`	Wire form	Resolves to
`ref`	`{"by":"ref", "value":"e3"}`	Snapshot ref — SDKs expand a bare string to this
`css`	`{"by":"css", "value":"#email"}`	CSS selector
`role`	`{"by":"role", "value":"button", "attrs":{"name":"Deploy"}}`	ARIA role + exact accessible name
`text`	`{"by":"text", "value":"Deploy", "exact":true}`	Visible text — substring unless `exact`; the shortest match wins
`xpath`	`{"by":"xpath", "value":"//textarea"}`	XPath
`coords`	`{"by":"coords", "attrs":{"x":"640","y":"420"}}`	Viewport coordinates (CSS pixels; `attrs` values are strings)
`frame`	`{"by":"frame", "value":"#checkout", "inner":{"by":"css","value":"#card"}}`	`inner`, resolved inside the iframe that `value` selects — `inner` must be a selector, not a ref

Shorthand: TypeScript accepts {x: 640, y: 420} and Python the tuple (640, 420) for coords. On css targets, pierce: true searches through shadow roots; any CSS selector also supports host >>> inner for one explicit shadow-root hop. The CLI takes e3, css:…, text:…, xpath:…, role:…, or x,y.

Actions

POST .../actions/{verb} where verb is one of click, dblclick, hover, fill, type, press, hotkey, scroll. Actions wait for the element to be actionable (visible, stable, enabled, receiving events — editable too for fill/type), dispatch real input events, then wait for the page to settle, and return an ActionResult. An element that never becomes actionable is not an HTTP error: the result comes back ok: false with reason naming the failed check. The examples use a small inline page so every target resolves:

from gravixlayer import GravixLayer

client = GravixLayer()
sandbox = client.runtime.create(template="browser-small")
page = ("data:text/html,<input id=email placeholder=Email>"
        "<textarea id=note></textarea><button>Deploy</button>")
bid = sandbox.browser.open(url=page)["browser_id"]

snap = sandbox.browser.snapshot(bid)
deploy = next(e["ref"] for e in snap["elements"] if e["name"] == "Deploy")

r = sandbox.browser.click(bid, deploy)                    # by ref
print(r["ok"], r["navigated"], r["reason"])

sandbox.browser.fill(bid, {"by": "css", "value": "#email"}, text="user@example.com")
sandbox.browser.type(bid, {"by": "css", "value": "#note"}, "hello", delay_ms=20)
sandbox.browser.press(bid, "Enter")                       # to the focused element
sandbox.browser.hotkey(bid, ["ctrl", "a"])
sandbox.browser.hover(bid, {"by": "role", "value": "button", "attrs": {"name": "Deploy"}})
sandbox.browser.dblclick(bid, {"by": "text", "value": "Deploy"})
sandbox.browser.click(bid, (20, 20))                      # viewport coordinates
sandbox.browser.scroll(bid, dy=600)
sandbox.kill()

import { GravixLayer } from 'gravixlayer';

const client = new GravixLayer();
const sandbox = await client.runtime.create({ template: 'browser-small' });
const page =
  'data:text/html,<input id=email placeholder=Email>' +
  '<textarea id=note></textarea><button>Deploy</button>';
const bid = (await sandbox.browser.open({ url: page })).browserId;

const snap = await sandbox.browser.snapshot(bid);
const deploy = snap.elements.find((e) => e.name === 'Deploy')!.ref;

const r = await sandbox.browser.click(bid, deploy);       // by ref
console.log(r.ok, r.navigated, r.reason);

await sandbox.browser.fill(bid, { by: 'css', value: '#email' }, { text: 'user@example.com' });
await sandbox.browser.type(bid, { by: 'css', value: '#note' }, 'hello', { delayMs: 20 });
await sandbox.browser.press(bid, 'Enter');                // to the focused element
await sandbox.browser.hotkey(bid, ['ctrl', 'a']);
await sandbox.browser.hover(bid, { by: 'role', value: 'button', attrs: { name: 'Deploy' } });
await sandbox.browser.dblclick(bid, { by: 'text', value: 'Deploy' });
await sandbox.browser.click(bid, { x: 20, y: 20 });       // viewport coordinates
await sandbox.browser.scroll(bid, { dy: 600 });
await sandbox.kill();

gravixlayer runtime browser navigate "$RT" --browser b1 \
  "data:text/html,<input id=email placeholder=Email><textarea id=note></textarea><button>Deploy</button>"

gravixlayer runtime browser action "$RT" --browser b1 click --target "text:Deploy"
gravixlayer runtime browser action "$RT" --browser b1 fill --target "css:#email" --value "user@example.com"
gravixlayer runtime browser action "$RT" --browser b1 type --target "css:#note" --value hello --delay-ms 20
gravixlayer runtime browser action "$RT" --browser b1 press --value Enter
gravixlayer runtime browser action "$RT" --browser b1 hotkey --value ctrl,a
gravixlayer runtime browser action "$RT" --browser b1 hover --target "role:button"
gravixlayer runtime browser action "$RT" --browser b1 click --target 20,20
gravixlayer runtime browser action "$RT" --browser b1 scroll --value 0,600

API="https://api.gravixlayer.ai/v1"

curl -sS -X POST "$API/agents/runtime/$RT/browser/b1/navigate" \
  -H "Authorization: Bearer $GRAVIXLAYER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"data:text/html,<input id=email placeholder=Email><textarea id=note></textarea><button>Deploy</button>"}'

curl -sS -X POST "$API/agents/runtime/$RT/browser/b1/actions/click" \
  -H "Authorization: Bearer $GRAVIXLAYER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"target":{"by":"text","value":"Deploy"},"button":"left","settle":true}'

curl -sS -X POST "$API/agents/runtime/$RT/browser/b1/actions/fill" \
  -H "Authorization: Bearer $GRAVIXLAYER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"target":{"by":"css","value":"#email"},"text":"user@example.com","settle":true}'

curl -sS -X POST "$API/agents/runtime/$RT/browser/b1/actions/press" \
  -H "Authorization: Bearer $GRAVIXLAYER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"key":"Enter","settle":true}'

curl -sS -X POST "$API/agents/runtime/$RT/browser/b1/actions/hotkey" \
  -H "Authorization: Bearer $GRAVIXLAYER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"keys":["ctrl","a"],"settle":true}'

Common action options (every verb accepts them)

Parameter	Type	Default	Description
`target`	ElementTarget	—	What to act on — `press`, `hotkey`, and `scroll` may omit it
`target_id`	string	current tab	Tab to act in (`--tab` in the CLI)
`settle`	bool	`true`	Wait for DOM/network quiescence afterward
`force`	bool	`false`	Skip the actionability checks
`settle_timeout_ms`	int	server default	Cap the post-action settle wait
`target_timeout_ms`	int	server default	Cap the actionability wait — `ok: false` + `reason` on expiry

Per-verb fields

Verb	Fields
`click`, `dblclick`	`button` (`left`/`right`/`middle`), `click_count`
`hover`	—
`fill`	`text` or `secret_ref` — replaces the field’s value
`type`	`text`, `delay_ms` between keystrokes — types at the cursor
`press`	`key` — e.g. `Enter`, `Tab`, `Escape`, `Backspace`, `ArrowDown`, or a single letter or digit
`hotkey`	`keys` — e.g. `["ctrl","a"]`; modifiers are held while the rest press
`scroll`	`dx`, `dy` — with a `target` and no deltas it scrolls that element into view

Typing secrets

fill with secret_ref types the value of a secret without it ever reaching your client, logs, or model context. The name is a secret key in an Identity provider attached to the runtime; a name that is not attached returns 400.

from gravixlayer import GravixLayer

client = GravixLayer()
provider = client.identity.providers.create(
    name="Login",
    provider_type="api_key",
    secrets=[{"key": "LOGIN_PASSWORD", "value": "s3cret"}],
)
sandbox = client.runtime.create(template="browser-small", providers=[provider.id])
bid = sandbox.browser.open(url="data:text/html,<input id=pw type=password>")["browser_id"]

r = sandbox.browser.fill(bid, {"by": "css", "value": "#pw"}, secret_ref="LOGIN_PASSWORD")
print(r["ok"])
sandbox.kill()

import { GravixLayer } from 'gravixlayer';

const client = new GravixLayer();
const provider = await client.identity.providers.create('Login', {
  providerType: 'api_key',
  secrets: [{ key: 'LOGIN_PASSWORD', value: 's3cret' }],
});
const sandbox = await client.runtime.create({
  template: 'browser-small',
  providers: [provider.id],
});
const bid = (await sandbox.browser.open({ url: 'data:text/html,<input id=pw type=password>' }))
  .browserId;

const r = await sandbox.browser.fill(bid, { by: 'css', value: '#pw' }, { secretRef: 'LOGIN_PASSWORD' });
console.log(r.ok);
await sandbox.kill();

PROVIDER_ID=$(gravixlayer provider create "Login" --type api_key \
  --secret "LOGIN_PASSWORD=s3cret" --output json | jq -r '.id // .provider.id')
SRT=$(gravixlayer runtime create -t browser-small --provider "$PROVIDER_ID" \
  --wait --output json | jq -r '.runtime_id')

gravixlayer runtime browser open "$SRT" --url "data:text/html,<input id=pw type=password>"
gravixlayer runtime browser action "$SRT" --browser b1 fill --target "css:#pw" --secret-ref LOGIN_PASSWORD
gravixlayer runtime kill "$SRT" -y

API="https://api.gravixlayer.ai/v1"

PROVIDER_ID=$(curl -sS -X POST "$API/identity/providers" \
  -H "Authorization: Bearer $GRAVIXLAYER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name":"Login","provider_type":"api_key","secrets":[{"key":"LOGIN_PASSWORD","value":"s3cret"}]}' \
  | jq -r .provider.id)

SRT=$(curl -sS -X POST "$API/agents/runtime" \
  -H "Authorization: Bearer $GRAVIXLAYER_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"template\":\"browser-small\",\"cloud\":\"aws\",\"region\":\"us-east-1\",\"providers\":[\"$PROVIDER_ID\"]}" \
  | jq -r .runtime_id)

curl -sS -X POST "$API/agents/runtime/$SRT/browser/open" \
  -H "Authorization: Bearer $GRAVIXLAYER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"data:text/html,<input id=pw type=password>"}'

curl -sS -X POST "$API/agents/runtime/$SRT/browser/b1/actions/fill" \
  -H "Authorization: Bearer $GRAVIXLAYER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"target":{"by":"css","value":"#pw"},"secret_ref":"LOGIN_PASSWORD","settle":true}'

curl -sS -X DELETE "$API/agents/runtime/$SRT" \
  -H "Authorization: Bearer $GRAVIXLAYER_API_KEY"

Evaluate

evaluate runs a JavaScript expression in the page and returns {value, error, is_error} (isError in TypeScript) — value is the result as JSON. Use it for page data, DOM reads, or scroll positions the digest doesn’t expose. Returned promises are awaited.

from gravixlayer import GravixLayer

client = GravixLayer()
sandbox = client.runtime.create(template="browser-small")
bid = sandbox.browser.open(url="https://example.com")["browser_id"]

print(sandbox.browser.evaluate(bid, "document.title")["value"])   # Example Domain
links = sandbox.browser.evaluate(bid, "document.querySelectorAll('a').length")
status = sandbox.browser.evaluate(bid, "fetch(location.href).then(r => r.status)")
print(links["value"], status["value"])                            # 1 200
sandbox.kill()

import { GravixLayer } from 'gravixlayer';

const client = new GravixLayer();
const sandbox = await client.runtime.create({ template: 'browser-small' });
const bid = (await sandbox.browser.open({ url: 'https://example.com' })).browserId;

console.log((await sandbox.browser.evaluate(bid, 'document.title')).value); // Example Domain
const links = await sandbox.browser.evaluate(bid, "document.querySelectorAll('a').length");
const status = await sandbox.browser.evaluate(bid, 'fetch(location.href).then(r => r.status)');
console.log(links.value, status.value); // 1 200
await sandbox.kill();

gravixlayer runtime browser evaluate "$RT" --browser b1 "document.title"
gravixlayer runtime browser evaluate "$RT" --browser b1 "fetch(location.href).then(r => r.status)"

API="https://api.gravixlayer.ai/v1"

curl -sS -X POST "$API/agents/runtime/$RT/browser/b1/evaluate" \
  -H "Authorization: Bearer $GRAVIXLAYER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"expression":"document.title","await_promise":true,"world":"isolated"}'

Parameter	Type	Required	Description
`expression`	string	Yes	JavaScript expression
`target_id`	string	No	Tab (empty = current tab)
`await_promise`	bool	No	Await a returned promise — SDKs and CLI default `true`; omitted on the wire means no
`world`	string	No	`isolated` (default) sees the DOM but not the page’s own globals; `main` runs in the page’s JavaScript context
`timeout_ms`	int	No	Evaluation timeout (default 30000)

A thrown exception comes back as is_error: true with the message in error, not as an HTTP error.

Dialogs

alert, confirm, and prompt dialogs are dismissed automatically about four seconds after they open, so an unanswered dialog never wedges the page. To answer one yourself, call dialog while it is open — the call waits briefly for a dialog to appear and returns 404 if none does. accept confirms (true in the SDKs; omitted on the wire means dismiss), prompt_text fills a prompt(), and auto_dismiss: true dismisses any open dialog immediately.

from gravixlayer import GravixLayer

client = GravixLayer()
sandbox = client.runtime.create(template="browser-small")
bid = sandbox.browser.open(url="https://example.com")["browser_id"]

sandbox.browser.evaluate(bid, "setTimeout(() => confirm('Deploy?'), 100)")
sandbox.browser.dialog(bid, accept=True)        # answers the confirm
sandbox.kill()

import { GravixLayer } from 'gravixlayer';

const client = new GravixLayer();
const sandbox = await client.runtime.create({ template: 'browser-small' });
const bid = (await sandbox.browser.open({ url: 'https://example.com' })).browserId;

await sandbox.browser.evaluate(bid, "setTimeout(() => confirm('Deploy?'), 100)");
await sandbox.browser.dialog(bid, { accept: true }); // answers the confirm
await sandbox.kill();

gravixlayer runtime browser evaluate "$RT" --browser b1 "setTimeout(() => prompt('Name?'), 100)"
gravixlayer runtime browser dialog "$RT" --browser b1 --accept --prompt-text "Ada"

API="https://api.gravixlayer.ai/v1"

curl -sS -X POST "$API/agents/runtime/$RT/browser/b1/evaluate" \
  -H "Authorization: Bearer $GRAVIXLAYER_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"expression\":\"setTimeout(() => prompt('Name?'), 100)\"}"

curl -sS -X POST "$API/agents/runtime/$RT/browser/b1/dialog" \
  -H "Authorization: Bearer $GRAVIXLAYER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"accept":true,"prompt_text":"Ada"}'

Notes

Act on fresh refs. Each snapshot re-issues the tab’s refs, and a ref points at the element’s position in the DOM — after a navigation or a re-render, snapshot again before acting. A ref the tab has never issued returns 404.
If an action returns ok: false, check reason — visible, enabled, editable, receives-events, or stable. The element is hidden, disabled, covered, or still animating; scroll it into view, wait, or pass force as a last resort.
settle: false (--no-settle) skips the post-action wait — useful when an action starts a long navigation you’ll wait for with navigate or display.wait_stable instead.
evaluate is full code execution in the page — treat the expression as code you ship, not as user input.

Getting Started

Runtime

Support

Endpoints

Snapshot

Targeting elements

Actions

Typing secrets

Evaluate

Dialogs

Notes

​Endpoints

​Snapshot

​Targeting elements

​Actions

​Typing secrets

​Evaluate

​Dialogs

​Notes

Endpoints

Snapshot

Targeting elements

Actions

Typing secrets

Evaluate

Dialogs

Notes