Extract structured data from a page without paying for a model

First: is there already a template for this host?

GET /api/v1/templates is public, needs no key, and takes an optional ?url= that reports which template matches. Per-host templates and generic-metadata templates cover a lot of the web, and when one supplies the result the response names it and the model cost is zero.

curl "https://twin-browser.com/api/v1/templates?url=https://news.ycombinator.com"

# { "templates": [ { "name": "…", "label": "…", "generic": false }, … ],
#   "url": "https://news.ycombinator.com",
#   "matched": "…" }

Second: is the page a listing?

If it is, do not read it as one page. Point `rowSelector` at ONE row or card and the engine expands it to every structurally-alike element — same depth, tag, parent and grandparent, plus attribute similarity — and extracts per row. Add `rowFields`, a map of field name to a CSS selector relative to the row, and each row is read with no model at all: an `a` yields its href, an `img` its src, anything else its trimmed text.

• `maxRows` defaults to 100 and is hard-capped at 500.
• `rowFields` accepts at most 50 fields.
• No selector to hand? `rowPrompt` describes one row in words — it costs a single model call to resolve a selector, then expands exactly as rowSelector does.

curl -X POST https://twin-browser.com/api/v1/extract \
  -H "Authorization: Bearer $TWIN_API_KEY" \
  -H "content-type: application/json" \
  -d '{
    "url": "https://store.example.com/catalogue",
    "rowSelector": ".product-card",
    "rowFields": { "name": "h3", "price": ".price", "link": "a" },
    "maxRows": 200
  }'

# { "data": [ { "name": "…", "price": "…", "link": "https://…" }, … ],
#   "rowsFound": 200,
#   "credits_charged": 5 }

Third: does the page render from its own API?

Many do. `captureXhr` collects the background fetch/XHR responses whose URL matches a pattern — a substring, a `*` wildcard, or a `/regex/flags` — and returns them as `capturedXhr`. The site's own JSON is usually cleaner, more complete and far more stable than the markup built from it, and reading it costs no model call.

curl -X POST https://twin-browser.com/api/v1/extract \
  -H "Authorization: Bearer $TWIN_API_KEY" \
  -H "content-type: application/json" \
  -d '{
    "url": "https://app.example.com/orders",
    "captureXhr": "/api/v2/orders*",
    "waitMs": 1500
  }'

# { "capturedXhr": [ { "url": "https://app.example.com/api/v2/orders?page=1",
#                      "body": { … } } ], … }

Fourth: the model read

Only now. Pass `fields` for a flat list of names or `schema` for a JSON-schema-like object, with an optional `prompt` to steer it. This is the path that bills higher-of(5-credit floor, metered model cost), and it is the right call when the value you want is genuinely not addressable — "the delivery estimate, wherever it appears on the page".

curl -X POST https://twin-browser.com/api/v1/extract \
  -H "Authorization: Bearer $TWIN_API_KEY" \
  -H "content-type: application/json" \
  -d '{
    "url": "https://store.example.com/product/8891",
    "schema": {
      "title": "string",
      "price": "string",
      "inStock": "boolean",
      "deliveryEstimate": "string"
    },
    "prompt": "deliveryEstimate is the shipping window shown near the buy button"
  }'

Extracting from behind a login

Extract takes the same account and session controls as a run. Pass `account` to select one of your stored logins for the host and `persistSession` to reuse its captured session, and an authenticated listing reads exactly like a public one. If no session exists yet, that is a connect handoff, not an extract problem.

Two defaults worth knowing

`blockAssets` defaults ON when a proxy is set, which saves bandwidth but means images are not fetched. And `blockAds` defaults OFF on purpose: on a guarded target, the absence of the usual third-party requests is itself an unusual signal, so turning it on is a trade rather than a free saving.