CAPABILITY · EXTRACT

Structured JSON off a page, with a zero-model path

Ask for fields or a JSON schema and get typed data back — or match a per-host template, expand a row selector, or capture the page's own XHR, and pay no model cost at all.

Start free API reference

POST /api/v1/extract

Capability

Inside structured extraction

Extraction usually means paying a model to read a page. That is one of four paths here, and the most expensive. Before it, the engine tries an extraction template matched to the host; a per-row selector map you supply reads a listing deterministically; and a capture pattern can pull the page's own background JSON straight out of the network instead of parsing what it rendered. The model is the fallback, not the default.

Templates first, model second

Per-host and generic-metadata templates match most URLs and return structured data with zero LLM cost. GET /api/v1/templates lists the catalog and reports which template matches a URL you name.

Row expansion for listings

Give `rowSelector` one row of a repeating list and the engine expands it to every structurally-alike element, extracting per row — so a 50-row listing is read from its rows rather than from the whole page.

Deterministic per-row fields

With `rowFields` — a map of field name to a CSS selector relative to the row — each row is read with no model at all: an `a` yields its href, an `img` its src, anything else its trimmed text.

Read the API behind the page

`captureXhr` collects the page's own background fetch/XHR responses whose URL matches a pattern and returns them as `capturedXhr`, so you can consume a site's JSON instead of its rendered DOM.

How it works

The mechanism, in execution order

4 stages, in the order the runtime performs them — not a summary of them.

1Load with the right postureThe page is fetched in a real browser, with asset blocking, ad blocking and a settle wait you can tune per call.
2Try the deterministic pathsA matching template, a rowFields map or a captureXhr pattern resolves the result with no model call. The response names the template when one supplied the data.
3Fall back to the modelWhen you asked for `schema` or `fields` and nothing deterministic matched, one metered extraction call reads the page into your shape.
4Return typed dataThe response carries `data` (an object, or an array when rows were expanded), `rowsFound` where relevant, and credits_charged.

In practice

A call you can paste and run

One row selector plus a field map reads a whole listing deterministically — the model is never called.

extract.shbash

# Deterministic: one row selector + a field map. No model call.
curl -X POST https://twin-browser.com/api/v1/extract \
  -H "Authorization: Bearer $TWIN_API_KEY" \
  -H "content-type: application/json" \
  -d '{"url":"https://store.example.com/catalogue",
      "rowSelector":".product-card",
      "rowFields":{"name":"h3","price":".price","link":"a"},
      "maxRows":50}'

# {
#   "data": [ { "name": "…", "price": "…", "link": "https://…" }, … ],
#   "rowsFound": 50,
#   "credits_charged": 5
# }

api.twin-browser.com

Structured extraction

Load with the right posturedone
Try the deterministic pathsrunning
Fall back to the modelqueued
Return typed dataqueued

What it costs, how to switch it on

Priced from the same rate card the API serves

Every credit figure on this page is read from the rate card behind GET /api/v1/pricing — it is not typed into the copy, so it cannot drift from what you are billed.

5-credit floor, metered above it

Extraction bills higher-of(5-credit floor, metered model + compute + egress). A call resolved by a template, by a rowFields map, or from captured XHR makes NO model call, so it bills the floor. A schema-driven read of a large page bills what the model actually consumed. GET /api/v1/templates is public and free — check whether a template matches before you pay for reasoning.

Full rate card

Turning it on

1Check for a templateGET /api/v1/templates?url=… reports whether a per-host or generic template already covers your target. No key required.
2Describe the outputPass `fields` for a flat list of names, or `schema` for a JSON-schema-like object. Both are optional when a template matches.
3Add rows if it is a listingSet `rowSelector` to one row (or `rowPrompt` to describe one in words, which costs a single call to resolve a selector) and `rowFields` to read each row deterministically.
4Force or disable a templatePass `template: "name"` to force one, or `template: false` to disable the library — at which point `schema` or `fields` becomes required.

At a glance

The contract, in the fewest rows that say it

Property	Twin Browser
Endpoint	POST /api/v1/extract
Shape	`schema` or `fields` (or a template)
Rows	`rowSelector` / `rowPrompt` + `rowFields`
Row ceiling	`maxRows` default 100, hard cap 500
Network capture	`captureXhr` → `capturedXhr`
Templates	GET /api/v1/templates (public)

Limits

Where it stops, and what it deliberately does not do

A capabilities page with no limits section is a brochure. These are the ceilings, the defaults that will surprise you, and the things this capability is not.

Limits and defaults

`maxRows` defaults to 100 and is hard-capped at 500. `rowFields` accepts at most 50 fields.
`rowPrompt` costs one metered model call to resolve a selector, and is ignored entirely when `rowSelector` is given.
With `template: false` and no `schema` or `fields`, there is nothing to extract and the call is rejected rather than guessed at.
`blockAds` is off by default on purpose: on a guarded target, the absence of the usual third-party requests is itself an unusual signal.

What it does not do

It does not act on the page. Extraction reads; if you need to click through a paywall or a filter first, that is a run.
It does not paginate for you. One call reads one URL — crawl the site, or drive the pagination in a run, then extract per page.
It does not invent a schema. Without a template, a `schema` or `fields` spec is required, and the response is shaped by what you asked for.
It does not guarantee a template exists for your host. The catalog is finite and public; check it rather than assume.

FAQ

Questions about structured extraction

How do I extract without paying for a model call?

Three ways, all deterministic: let a per-host or generic-metadata template match (check with GET /api/v1/templates), give `rowSelector` plus a `rowFields` selector map, or pull the page's own background JSON with `captureXhr`. Each returns structured data with no LLM cost, so the call bills the floor.

What is row expansion actually doing?

You point `rowSelector` at ONE row or card. The engine finds every structurally-alike element on the page — same depth, tag, parent and grandparent, plus attribute similarity — and extracts per row rather than reading the page as a whole. The response reports `rowsFound`.

When should I use captureXhr instead of parsing the DOM?

When the page renders from its own JSON API. Give `captureXhr` a substring, a `*` pattern or a `/regex/`, and the matching background responses come back as `capturedXhr` — usually cleaner, more complete and more stable than the markup built from them.

Can extraction run on a page behind a login?

Yes. Extract takes the same account and session controls as a run — pass `account` to select a stored login and `persistSession` to reuse its captured session — so an authenticated listing is read the same way a public one is.

Keep reading

The rest of the platform

The capabilities this one works with, the tools that expose it, the guides that use it end to end, and the vocabulary behind it.

extract observe_page submit_crawl

Guides

Extract structured data from a page without paying for a model Crawl a website without crawling the whole website

Vocabulary

Structured extraction Extraction template Row expansion XHR capture

Put your agent to work. Keep the decision.

Start free. Hand your agent a goal on a site you authorize, set the guardrails, and let the first successful run compile the skill every run after it replays.

Start free Read the docs