Sessions & HITL

Drive a paused run’s browser yourself

Click, type, press a key, scroll or navigate on a run that is parked — in viewport fractions read off a screenshot. The run resumes by itself once the page stops looking blocked.

control_run

What it is

The job: Clear a blocker you can see on the page, when no code and no credential would have helped.

Sometimes a run parks on something that is neither a code nor a login: an interstitial, a cookie wall, a choice between two buttons. control_run is the escape hatch for exactly that. Take a screenshot, find the thing, and send an interaction.

Coordinates are fractions of the viewport — 0..1 from the top-left — not pixels, and they are hit-tested server-side against the observed element map. A button halfway across and 60% down is { x: 0.5, y: 0.6 } whatever size the browser is. Typed text is redacted from the stream and never stored.

There is no “done” call, and that is deliberate. After each interaction the run re-checks the page and continues by itself as soon as the block has cleared. You are nudging a run that is still running, not driving a session you now own.

The call

Call it exactly like this.

Copied from the tool's registration and the route handler — not paraphrased.

MCPcontrol_run.mcp.jsonjson
// MCP tool call — server "twin-browser"
{
  "tool": "control_run",
  "arguments": {
    "sessionId": "…",
    "action": { "kind": "click", "x": 0.5, "y": 0.62 }
  }
}
POST /api/v1/runs/{id}/inputrequest.shbash
curl -X POST https://twin-browser.com/api/v1/runs/RUN_ID/input \
  -H "Authorization: Bearer $TWIN_API_KEY" \
  -H "content-type: application/json" \
  -d '{"event":{"kind":"click","x":0.5,"y":0.62}}'

The MCP tool addresses the parked browser by its engine `sessionId`; the REST endpoint addresses it by the WEB run id and looks the parked session up from the run record. If you are driving a run you started over the hosted REST API, use the run id and POST /api/v1/runs/{id}/input.

Parameters accepted by control_run
ParameterTypeWhat it does
sessionId*stringThe sessionId from the paused run.
action*objectOne interaction: { kind:"click", x, y } | { kind:"fill", x, y, text } | { kind:"key", key } | { kind:"scroll" } | { kind:"goto", url }.
action.kind*"click" | "fill" | "key" | "scroll" | "goto"Which interaction to perform.
action.x / action.ynumberHorizontal / vertical fraction of the viewport, 0..1 from the top-left. Read them off a screenshot.
action.textstringText to type, for kind "fill". Redacted from the stream; never stored.
action.keystringKey to press, e.g. "Enter", for kind "key".
action.urlstringURL to open, for kind "goto".

Returns

response.jsonjson
{ "ok": true, "url": "https://…" }   // the page URL after the interaction

What it costs

Free

Driving a paused browser costs nothing — the parent run owns the metering, and it was refunded when it parked. You are charged when the run completes.

See the full rate card

Which one

When a different tool is the right call.

The honest answer is often the neighbouring tool. These are the trades.

submit_verification

If the wall is a code or an approval, submit_verification is one call instead of a screenshot plus a click plus a fill. Use control_run for the things that are not codes.

submit_verification
connect_account

If the wall is a login a human must do, hand it to them properly. Typing someone else’s password through control_run is technically possible and is the wrong shape.

connect_account
screenshot

You cannot aim without it. Screenshot the paused page, read the fraction, then act. GET /api/v1/runs/{id}/stream also mirrors the paused browser live, free of charge.

screenshot

Questions

control_run, answered.

How do I see the page I am driving?
Two ways. screenshot gives you one frame to read coordinates from. GET /api/v1/runs/{id}/stream gives you a live SSE screencast of the paused browser — it drives no agent and costs no credits, because it is mirroring a browser that is already open and waiting.
When does the run continue?
On its own. After each interaction the run re-evaluates the page, and as soon as it no longer looks blocked it carries on with the goal. There is no resume call for this path — sending the interaction is the whole interface.