Twin Browser
Get an API key
PlatformOverviewHow it worksAll toolsREST APIMCP serverIntegrationsDocs
CapabilitiesSemantic dispatch cacheDeterministic replayCross-tenant skill corpusToken-efficient DOMHuman-in-the-loopLive view & videoCredential vaultOverviewSolutions
SolutionsAll use casesAI agentsRPA replacementQA & test automationData extraction
ResourcesGuidesGlossaryBlogCompareFAQStatus
Pricing
Log in

Use case

Data extraction at scale

Extract from authenticated, multi-step pages you're authorized to reach — the agent signs in, does the work, and replays repeat pulls at near-zero cost.

Start freeHow it works

The problem

What the status quo costs you.

Read-only scrapers handle public pages well, but break the moment data lives behind a login, a multi-step flow, or a stateful session. Agent-driven extraction fixes that — and re-pays the LLM to re-read the same site on every run, so cost scales with volume exactly when you want it to fall.

See how Twin works
app.example.com
  1. Log in via the credential vaultdone
  2. Compile the multi-step flowrunning
  3. Read the token-efficient DOMqueued
  4. Replay extraction — cache hitqueued
  5. Return rows as JSONqueued
A Twin run for data extraction at scale — compile once, then replay on a cache hit.

The wedge

Compile once. Replay forever.

Twin compiles an authenticated extraction flow into a skill once, then replays it deterministically. The token-efficient DOM map keeps extraction cheap on context, the credential vault handles the login, and the semantic dispatch cache means a re-phrased extraction request matches the skill you already compiled instead of cold-starting. For bulk read-only ingestion you can pair Twin with a dedicated scraper; for stateful, logged-in extraction, Twin is the layer that bends cost down.

  1. 1Describe the extraction as a goal; Twin logs in via the credential vault and compiles the flow into a skill.
  2. 2The DOM-to-indexed-state compiler returns a compact, numerically-indexed view, so even a 50-step flow stays within a tight token budget (illustratively ~3k tokens).
  3. 3Repeat and re-worded extractions hit the semantic cache and replay with zero LLM calls.
  4. 4The cross-tenant skill corpus means common site patterns are already compiled, raising your hit rate.
  5. 5Proxy support (IPRoyal) and session video keep large runs observable and resilient.

Outcome · Authenticated, repeated extraction that re-pays the model every run on agent-driven infra instead settles to deterministic replay — illustratively cost per 1,000 extractions falling ~5x after warmup rather than scaling with volume.

In practice

The actual call, and what it returns.

Name one row and the fields on it, and every structurally alike row on the page is read the same way — deterministically, with no model call for the reading itself.

Extracttransactions.shbash
curl https://twin-browser.com/api/v1/extract \
  -H "Authorization: Bearer $TWIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://billing.example.com/transactions",
    "rowSelector": "tr.tx",
    "rowFields": { "id": ".tx-id", "amount": ".tx-amount", "status": ".tx-status" },
    "account": "billing-portal"
  }'

# → 200
# {
#   "result": [ { "id": "TX-4471", "amount": "128.40", "status": "paid" }, … ],
#   "rowsFound": 42,
#   "url": "https://billing.example.com/transactions",
#   "credits_charged": 5
# }

What this call does

  • "account" signs the extraction in first, so this works behind a login rather than only on public pages.
  • "rowFields" reads each row by selector — an <a> yields its href, an <img> its src, anything else its text.
  • Describe the row in words with "rowPrompt" instead, and one metered call resolves the selector before the same expansion runs.
  • A per-host template (GET /api/v1/templates) can answer the whole request with no model call at all.
Every endpoint, every field

Under the hood

The machinery under every run.

Every use case runs on the same primitives: a token-efficient view of the page, deterministic replay, and a checkpoint where a person decides.

Semantic dispatch cache

Re-phrased requests fuzzy-match a skill you already compiled, so they skip the planner LLM entirely.

Read the mechanism — Semantic dispatch cache

Deterministic replay

Matched skills replay the same way every time — a pass is a pass, and the marginal cost trends toward zero.

Read the mechanism — Deterministic replay

Token-efficient DOM state

A live page becomes a compact, numerically-indexed map of interactive elements instead of raw HTML.

Read the mechanism — Token-efficient DOM state

Human-in-the-loop handoff

Blocked steps — approvals, MFA on an authorized flow — pause for a person, then resume cleanly.

Read the mechanism — Human-in-the-loop handoff

FAQ

Data extraction at scale on Twin — common questions.

Is Twin a scraping tool like Bright Data or Firecrawl?
For different jobs. Bright Data and Firecrawl excel at large-scale, read-only public extraction. Twin targets authenticated, stateful, repeated extraction — logging in, multi-step flows, human handoff — and bends cost down with a semantic skill cache instead of billing per gigabyte or per page. The two compose well.
How does Twin keep token cost low on big pages?
Instead of feeding raw HTML to the model, Twin’s DOM-to-indexed-state compiler produces a compact, numerically-indexed map of interactive elements under a token budget, so extraction stays cheap on context even on heavy pages.
Can Twin extract behind a login?
Yes. Credentials live in a per-tenant credential vault with default-deny RLS, so authenticated, multi-step extraction runs without hard-coding secrets in your code.

Related

More ways teams use Twin.

AI agents

Give your AI agent a real browser it can drive — reach any site, act under your guardrails, and let repeated work replay at near-zero cost.

Internal workflow automation

Automate the internal tools and vendor portals that have no API — with audit logging and human approval built in.

RPA replacement

Replace brittle, selector-keyed RPA bots with skills that adapt to the page — you authorize the run, the agent does the work.

Put data extraction at scale on autopilot.

Start free, hand your agent the work under your guardrails, and let the runs you repeat replay from a compiled skill.

Start freeRead the guides