Sessions & HITL
Drive a paused run’s browser yourself
Click, type, press a key, scroll or navigate on a run that is parked — in viewport fractions read off a screenshot. The run resumes by itself once the page stops looking blocked.
control_run
What it is
The job: Clear a blocker you can see on the page, when no code and no credential would have helped.
Sometimes a run parks on something that is neither a code nor a login: an interstitial, a cookie wall, a choice between two buttons. control_run is the escape hatch for exactly that. Take a screenshot, find the thing, and send an interaction.
Coordinates are fractions of the viewport — 0..1 from the top-left — not pixels, and they are hit-tested server-side against the observed element map. A button halfway across and 60% down is { x: 0.5, y: 0.6 } whatever size the browser is. Typed text is redacted from the stream and never stored.
There is no “done” call, and that is deliberate. After each interaction the run re-checks the page and continues by itself as soon as the block has cleared. You are nudging a run that is still running, not driving a session you now own.
The call
Call it exactly like this.
Copied from the tool's registration and the route handler — not paraphrased.
// MCP tool call — server "twin-browser"
{
"tool": "control_run",
"arguments": {
"sessionId": "…",
"action": { "kind": "click", "x": 0.5, "y": 0.62 }
}
}curl -X POST https://twin-browser.com/api/v1/runs/RUN_ID/input \
-H "Authorization: Bearer $TWIN_API_KEY" \
-H "content-type: application/json" \
-d '{"event":{"kind":"click","x":0.5,"y":0.62}}'The MCP tool addresses the parked browser by its engine `sessionId`; the REST endpoint addresses it by the WEB run id and looks the parked session up from the run record. If you are driving a run you started over the hosted REST API, use the run id and POST /api/v1/runs/{id}/input.
| Parameter | Type | What it does |
|---|---|---|
| sessionId* | string | The sessionId from the paused run. |
| action* | object | One interaction: { kind:"click", x, y } | { kind:"fill", x, y, text } | { kind:"key", key } | { kind:"scroll" } | { kind:"goto", url }. |
| action.kind* | "click" | "fill" | "key" | "scroll" | "goto" | Which interaction to perform. |
| action.x / action.y | number | Horizontal / vertical fraction of the viewport, 0..1 from the top-left. Read them off a screenshot. |
| action.text | string | Text to type, for kind "fill". Redacted from the stream; never stored. |
| action.key | string | Key to press, e.g. "Enter", for kind "key". |
| action.url | string | URL to open, for kind "goto". |
Returns
{ "ok": true, "url": "https://…" } // the page URL after the interactionWhat it costs
Free
Driving a paused browser costs nothing — the parent run owns the metering, and it was refunded when it parked. You are charged when the run completes.
See the full rate cardWhich one
When a different tool is the right call.
The honest answer is often the neighbouring tool. These are the trades.
submit_verificationIf the wall is a code or an approval, submit_verification is one call instead of a screenshot plus a click plus a fill. Use control_run for the things that are not codes.
submit_verificationconnect_accountIf the wall is a login a human must do, hand it to them properly. Typing someone else’s password through control_run is technically possible and is the wrong shape.
connect_accountscreenshotYou cannot aim without it. Screenshot the paused page, read the fraction, then act. GET /api/v1/runs/{id}/stream also mirrors the paused browser live, free of charge.
screenshotQuestions
control_run, answered.
- How do I see the page I am driving?
- Two ways. screenshot gives you one frame to read coordinates from. GET /api/v1/runs/{id}/stream gives you a live SSE screencast of the paused browser — it drives no agent and costs no credits, because it is mirroring a browser that is already open and waiting.
- When does the run continue?
- On its own. After each interaction the run re-evaluates the page, and as soon as it no longer looks blocked it carries on with the goal. There is no resume call for this path — sending the interaction is the whole interface.
Keep going
The rest of the tool set.
submit_verificationFinish a sign-in that got as far as the second factor.
connect_accountGet logged into a site that an agent is not going to be allowed to log into.
screenshotGet one honest image of a page, without catching it mid fade-in.
run_goalDo one thing on one site, right now, and block until you know whether it worked.