For data and analytics teams
Get the numbers out of the dashboard with no export.
The vendor shows you the report and will not give you the data. Twin signs in, reads the rows off the page deterministically, and either hands you JSON or loads it into a queryable store — and a monitor pushes you a webhook when a watched number moves.
The problem
What this costs you today.
Half the numbers the business runs on live in somebody else’s dashboard. There is no API on the plan you are on, the CSV export is missing the one dimension you need, and the "integration" the vendor sells is a nightly file that arrives at ten in the morning. So an analyst signs in every Monday, screenshots a chart into a deck, and the warehouse never learns about any of it.
- “The report exists, we just cannot get it into the warehouse.”
- “API access is an enterprise upsell we cannot justify for one table.”
- “Someone re-keys these figures into a spreadsheet every week.”
- “We find out a number moved when a customer tells us.”
- Restore the vendor sessiondone
- Open the report pagerunning
- Expand rows by selectorqueued
- Return rows as JSONqueued
- Webhook on the next changequeued
How Twin solves it
The mechanism, not a promise.
A page is a data source if you can read it reliably. Twin signs in with a stored session, reads repeating rows by selector rather than by asking a model to read them, and gives you three destinations: raw JSON, a queryable content store, or a signed webhook when the value changes.
- 1Read rows, not pagesPOST /api/v1/extract with `rowSelector` expands one example row into every structurally-alike row on the page, and `rowFields` maps each field to a selector relative to that row. That reading is deterministic — an anchor yields its href, an image its src, anything else its text.
- 2Or read the JSON the page itself fetched`captureXhr` collects the page’s own background XHR and fetch responses whose URL matches a pattern and returns them as `capturedXhr` — so you can consume a vendor’s internal JSON instead of its rendered table.
- 3Sign in without a password in the requestPass `account` to use a stored login and its session. Nothing about the credential travels in the call.
- 4Load it somewhere queryablePOST /api/v1/etl extracts, cleans, chunks, embeds and loads a source into the content store; POST /api/v1/etl/query runs semantic search over what you ingested and returns the top-k matching chunks.
- 5Be told, rather than pollingPOST /api/v1/monitors watches a page on a schedule — minimum sixty seconds — and pushes an HMAC-signed webhook to your callback_url when the watched value changes. GET /api/v1/monitors/{id}/history shows what it saw.
In practice
The actual call, and what it returns.
One call reads the table behind the login. The second stops you from having to ask again — the change comes to your endpoint.
# 1 — read the report table row by row, behind the login.
curl https://twin-browser.com/api/v1/extract \
-H "Authorization: Bearer $TWIN_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://analytics.vendor.example.com/reports/weekly",
"account": "vendor-analytics",
"rowSelector": "tr.report-row",
"rowFields": { "campaign": ".name", "spend": ".spend", "conversions": ".conv" },
"maxRows": 500
}'
# → 200
# {
# "result": [ { "campaign": "…", "spend": "…", "conversions": "…" }, … ],
# "rowsFound": 214,
# "url": "https://analytics.vendor.example.com/reports/weekly",
# "credits_charged": 5
# }
# 2 — then stop polling: be pushed the change.
curl https://twin-browser.com/api/v1/monitors \
-H "Authorization: Bearer $TWIN_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"name": "weekly spend",
"check_type": "extract",
"url": "https://analytics.vendor.example.com/reports/weekly",
"watch_fields": ["spend"],
"interval_seconds": 3600,
"callback_url": "https://data.example.com/hooks/twin",
"callback_secret": "…"
}'What this call does
- `rowFields` reads each field off the row by selector, so the extraction itself costs no model call.
- Describe the row in words with `rowPrompt` instead when you do not know the selector; that costs one metered call to resolve it, then expands identically.
- `maxRows` defaults to 100 and is capped at 500 — a long report is paged, not silently truncated to whatever fit.
- A monitor’s minimum cadence is sixty seconds, and its webhook body is HMAC-signed with `callback_secret`.
What it costs
Priced per action, not per seat.
Reading a page is metered on what it actually costs to read: the higher of a flat floor and the model spend, and a deterministic read spends nothing on a model.
| Action | Credits | What you get |
|---|---|---|
| POST /extract — read a report | metered | Metered higher-of. Selector-based `rowFields` reading incurs no model cost. |
| POST /etl — load it into the store | metered | Metered per source; only schema or field extraction incurs model cost. |
| POST /monitors — watch a value | metered | Each scheduled check bills as the underlying extract or observe read. |
| POST /run — navigate a multi-step report | 10 / run | When the numbers are three clicks and a date filter away. |
How the unit works
- $1 buys 1,000 credits; the smallest pack is $5.
- A paid action bills the higher of its flat floor and what it actually spent on model, compute and egress — so a cheap run stays cheap.
- “Metered” means the action has no published flat floor on this page: GET /api/v1/pricing serves the live card.
- GET /api/v1/pricing returns the live rate card — the flat action floors and the per-model metered rates, in credits — so you can price a pipeline before you build it.
Be sure this fits
What this does not do.
Every one of these will come up in your evaluation. Here they are first, from us.
There is no general-purpose scheduler
Monitors are the scheduled primitive and they exist to fire on CHANGE, not to run your nightly batch. A recurring pull belongs in your own scheduler calling POST /api/v1/jobs — we would rather say that than pretend the monitor is a cron.
It does not reconcile or model the data
You get rows as the page presents them. Deduplication, type coercion, slowly-changing dimensions and the semantics of the vendor’s numbers are your pipeline’s problem; this is the extraction step, not the warehouse.
A page can lie to you quietly
If a vendor changes a column heading, a deterministic selector read returns the wrong field rather than an error. Assert on what you extract — the same way you would assert on a CSV you did not write.
Under the hood
The primitives this runs on.
Nothing here is specific to this problem — the same mechanisms carry every solution on the site.
Token-efficient DOM state
The live page is serialized into a numerically-indexed list of its interactive elements under a token budget, so the planner reasons over signal instead of markup.
Read the mechanism — Token-efficient DOM stateCredential vault
A write-only per-tenant vault: a stored secret is referenced in a goal as {{secret:NAME}}, resolved inside the browser at fill time, and redacted from every step, frame and log.
Read the mechanism — Credential vaultDeterministic replay
A successful run is minimized into a named, versioned skill — an ordered action path with its variable parts lifted out — and replaying it is a program, not a prompt.
Read the mechanism — Deterministic replayLive view & session video
POST /api/v1/live streams the browser back as Server-Sent Events while the agent works, and an opted-in run keeps a durable video you can fetch long afterwards.
Read the mechanism — Live view & session videoOver MCP, the same work is these tools
extractRead an authorized page and return structured JSON matching the fields or JSON schema you request.
etlRun the ETL pipeline: extract → transform → chunk → embed → load into the queryable content store (needs WEB_BASE_URL + TWIN_API_KEY).
etl_querySemantic search over content ingested with etl; returns the top-k matching chunks (needs WEB_BASE_URL + TWIN_API_KEY).
create_monitorWatch a page on a schedule and push an HMAC-signed webhook when the watched value changes (needs WEB_BASE_URL + TWIN_API_KEY).
get_monitor_historyGet a monitor’s recent check history — changed/unchanged/error plus value excerpts.
FAQ
Reporting from behind a login — common questions.
Does this work on a page that renders after login and after JavaScript?
How do we get the data into our warehouse?
Can we be notified instead of polling?
Does reading a report cost a model call every time?
Related
Other problems this layer solves.
Competitive and market data
Pricing, product-marketing and growth teams who need the public market as structured data.
Supplier and customer portals
Procurement, accounts-payable and supply-chain teams whose data is spread across dozens of external portals.
Systems with no API
Ops, finance and platform teams blocked by a vendor system that offers no programmatic access.
Try it on your hardest screen.
Start free, point a run at the system that is blocking you, and watch it happen live. If it does not work, the run tells you why — and what to do instead.