Fetch and jobs

POST /v1/fetch returns rendered page text. Long-running requests can run as jobs, with results available through GET /v1/job/{job_id}. For other output formats, see Extract. Scrape & crawl supports reads without a caller-managed session.

POST /v1/fetch

Load a URL in the session’s browser and return its text. The request attempts challenge recovery and reports whether usable content was delivered. Use /v1/extract for Markdown output. For protected pages, fetch combines block detection, challenge solving, and exit recovery with a typed response. Disconnect the main CDP client and any native Playwright or VNC client before calling it. CDP observers can remain connected; coordinate their commands with the fetch. On a custom-proxy session, recovery never switches the proxy exit. Auth: Bearer API key. Request:

Field	Type	Default	Meaning
`session_id`	string	required	A ready session you own.
`url`	string	required	Public http or https URL, up to 4096 characters. Private and loopback addresses are refused.
`expect_url`	string (regular expression)	null	The URL the delivered page must land on. A page that lands elsewhere is reported as `redirected_off_target` with `error: "expect_url_mismatch"`, and recovery keeps trying.
`expect_text`	string (regular expression)	null	Text the delivered page must contain (a price, a heading). A page without it is not accepted and recovery moves on.
`no_results_url`	string (regular expression)	null	A landing URL that means the site answered “no results”. Such a page is delivered as is with `status: "no_results"` and recovery stops.
`warm_entry`	boolean	false	Open the site’s entry pages first, then the URL with the last of them as referrer. Costs those extra page loads.
`visitor`	boolean	false	Leave human-like input on each page opened (pointer movement, scrolls, dwell). About 3 seconds per page.
`render`	boolean	true	Accepted for compatibility. Pages are always rendered.
`record`	boolean	false	Request a frame replay for this fetch. `replay_url` identifies the recording when one is available.
`include_html`	boolean	false	Also return the rendered DOM in `html`.
`async_mode`	boolean	false	Return at once with `status: "pending"` and a `job_id`; poll `GET /v1/job/{job_id}`.
`on_deadline`	`"error"` or `"continue"`	`"error"`	What happens when the page has not been delivered at the response limit (55 seconds). `error` answers 504 and stops the work. `continue` answers 202 with the job to poll and keeps working for up to 900 seconds.
`wait_ms`	integer, 0 to 60000	15000	Ceiling in milliseconds for readiness waits (DOM settle, selector, navigation). `0` uses 15000. The page is read as soon as it is ready.
`settle_ms`	integer, 0 to 30000	null	Read the page once the DOM has stopped changing for this long. Bounded by `wait_ms`.
`wait_for_selector`	string, up to 256 characters	null	CSS selector to wait for, bounded by `wait_ms`. The page is read either way; `selector_found` says whether it appeared.

expect_url, expect_text and no_results_url are limited to 512 characters each. Invalid or unsafe regular expressions answer 422. A plain request:

{"session_id": "sess_6ac4459ad3e24b8289fc20911d76374f", "url": "https://en.wikipedia.org/wiki/Web_scraping"}

Response:

Field	Type	Meaning
`session_id`	string	The session that served the page.
`status`	string	`ok` (text delivered), `incomplete` (page loaded but `expect_text` or `wait_for_selector` was not met; text returned, `content_ok: false`, action not charged), `empty` (no readable text, or the origin’s own 404 or 410), `blocked` (a challenge or refusal recovery could not clear), `error` (request failed), `pending` (async work), `no_results` (the site’s own “nothing found” page), `redirected_off_target` (landing fails `expect_url` or collapses to a shallower page; text withheld).
`content_ok`	boolean	Whether usable text was delivered and the requested content checks passed. False for `incomplete`, even though `text` is returned. Trust this over HTTP status.
`blocked_reason`	string or null	The block classification, when reported.
`data_requests`	object or null	Summary of page data requests, including blocked API/XHR requests when reported.
`replay_url`	string or null	Tokenized path to the frame replay, or null when no recording is available.
`final_url`	string	Where the navigation landed.
`requested_url`	string	What you asked for.
`redirected`	boolean	The landed host or path differs from the one asked for. A query string change or a `www` prefix does not count.
`http_status`	integer or null	Status of the main document.
`title`	string or null	Page title.
`text`	string	The page text. Empty when the page was not delivered; failure messages never land here.
`html`	string or null	The rendered DOM, only with `include_html: true`.
`html_len`	integer	Size of the rendered DOM in bytes, always present.
`captcha`	string	`none`, or the challenge family seen.
`self_healed`	boolean	A block was detected and cleared before delivery.
`recovery`	object or null	What was tried when the page did not come straight through. See below.
`held`	object or null	The origin served a page with an error status (a 403 with text) and it was not delivered: `{http_status, had_text, text_chars, recovery: "tried" or "not_possible", reason}`.
`detected`	object or null	What was found wrong with the first page: `{family, category, mechanism, source}`. Null for a healthy page.
`error`	string or null	Why the fetch did not deliver, as a slug (`time_exhausted`, `not_found`, `deadline`, `expect_url_mismatch`, …). Null on `ok`.
`failure_type`	string or null	What failed when the request itself broke (`TimeoutError`, …).
`cause`	string or null	A closing cause code (`wait_exhausted`, …).
`reason`, `origin`	string or null	Set on a job cancelled because its session ended: the session’s end reason and who ended it.
`premium_required`	string or null	A residential unlock could have reached the page but the plan or the credit balance does not allow it.
`status_reconciled`	string or null	Set when a `blocked` verdict was flipped to `ok` because real content was present.
`readiness`	string or null	Why the drive stopped waiting, when reported.
`selector_found`	boolean or null	With `wait_for_selector`: true when it appeared, false when the wait ran out. Null without one.
`settle`	object or null	What the read waited on: `{settled, waited_ms, requests, pending, busy, reason, dom}`. `reason` is `network`, `dom`, `selector`, `timeout` or `unsupported`. Null when the page was not settled before the read (no settle requested, or nothing delivered).
`job_id`	string or absent	The job that ran the fetch, when returned. A 504 `deadline` response may omit it. Poll only an id actually returned by the API.
`job`	object or null	Only on a 202: `{poll, may_run_s, ends_by}`.

The Wikipedia page (text cut):

{
 "session_id": "sess_6ac4459ad3e24b8289fc20911d76374f",
 "final_url": "https://en.wikipedia.org/wiki/Web_scraping",
 "status": "ok",
 "http_status": 200,
 "job_id": "sess_6ac4459ad3e24b8289fc20911d76374f.72289fc4d5c34f72a1fd9e7f054187d5",
 "job": null,
 "captcha": "none",
 "title": "Web scraping - Wikipedia",
 "text": "Jump to content\nMain menu\nSearch\nDonate\nCreate account\nLog in\nContents hide\n(Top)\nHistory\nTechniques\n...",
 "html_len": 316526,
 "html": null,
 "self_healed": false,
 "recovery": null,
 "held": null,
 "detected": null,
 "content_ok": true,
 "status_reconciled": null,
 "requested_url": "https://en.wikipedia.org/wiki/Web_scraping",
 "redirected": false,
 "error": null,
 "failure_type": null,
 "cause": null,
 "reason": null,
 "origin": null,
 "premium_required": null,
 "readiness": null,
 "selector_found": null,
 "settle": null
}

A settled read of a news front page, with a selector and the DOM returned. The request was

{"session_id": "...", "url": "https://www.bbc.com/news", "settle_ms": 500, "wait_ms": 8000, "wait_for_selector": "h1", "include_html": true}

(trimmed to the fields that differ from the plain example):

{
 "final_url": "https://www.bbc.com/news",
 "status": "ok",
 "http_status": 200,
 "title": "BBC News - Breaking news, video and the latest top stories from the U.S. and around the world",
 "text": "Skip to content\nWatch Live\nSubscribe\nSign In\nHome\nNews\nSport\n...",
 "html_len": 351266,
 "html": "<html lang=\"en-GB\"><head><meta charset=\"utf-8\" data-next-head=\"\">...",
 "content_ok": true,
 "selector_found": true,
 "settle": {
  "settled": true,
  "waited_ms": 1096,
  "requests": 67,
  "pending": [],
  "busy": [],
  "reason": "network",
  "dom": true
 }
}

A blocked page, with the recovery report. The fetch spent its response budget on two more exits and gave up, answering 200 with status: "blocked". settle is null because no page was delivered:

{
 "session_id": "sess_6ac4459ad3e24b8289fc20911d76374f",
 "final_url": "https://apnews.com/",
 "status": "blocked",
 "http_status": 403,
 "job_id": "sess_6ac4459ad3e24b8289fc20911d76374f.d51720ad0f2b4059b1b0ec6338f95099",
 "captcha": "none",
 "title": null,
 "text": "",
 "html_len": 0,
 "self_healed": false,
 "recovery": {
  "driver": "data_plane",
  "attempts": 0,
  "actions": [],
  "ladder": {
   "outcome": "exhausted",
   "attempts": 2,
   "rungs": [
    {"rung": "residential", "result": "budget", "elapsed_ms": 22962},
    {"rung": "isp", "result": "budget", "elapsed_ms": 23236}
   ],
   "exhausted_reason": "budget",
   "budget": {
    "kind": "sync",
    "total_s": 55.0,
    "elapsed_s": 42.3,
    "spent": {"heal": 19.04, "chain": 46.2}
   }
  }
 },
 "held": {
  "http_status": 403,
  "had_text": true,
  "text_chars": 258,
  "recovery": "not_possible",
  "reason": "time_exhausted"
 },
 "detected": {
  "family": "cloudflare",
  "category": "challenge",
  "mechanism": "challenge_page",
  "source": "data_plane"
 },
 "content_ok": false,
 "requested_url": "https://apnews.com/",
 "redirected": false,
 "error": "time_exhausted",
 "failure_type": null,
 "cause": "wait_exhausted",
 "premium_required": null,
 "selector_found": null,
 "settle": null
}

Reading recovery: attempts and actions at the top level are the in-page heals on your own session. ladder is the escalation across exits: outcome is delivered, exhausted or no_results; attempts counts the tries that ran; each entry in rungs names the tier (direct, residential, isp, mobile, …), its result, how long it took and, when it is a plain slug, a reason; delivered_by names the tier that delivered the page, exhausted_reason why the ladder stopped, and answered_by the tier whose page was the site’s own answer. budget is the clock the fetch ran on: kind is sync (55 s) or async (900 s), elapsed_s what was used, and spent where the seconds went by phase (heal, chain, warm_entry, persist, report). A budget result on a rung means the tier was started but the clock ran out before it answered; on a 900 second job clock the same page gets many more tries. Async mode and polling. {"session_id": "...", "url": "https://en.wikipedia.org/wiki/HTTP", "async_mode": true} answers at once:

{
 "session_id": "sess_6ac4459ad3e24b8289fc20911d76374f",
 "final_url": "https://en.wikipedia.org/wiki/HTTP",
 "status": "pending",
 "http_status": null,
 "job_id": "sess_6ac4459ad3e24b8289fc20911d76374f.95eb7d80eb3c47b69faa7a3befd7f3eb",
 "job": null,
 "captcha": "none",
 "title": null,
 "text": "",
 "html_len": 0,
 "content_ok": false,
 "redirected": false
}

(The other fields are present with their null or default values.) A second fetch on the same session while that job runs is refused:

HTTP 409
{
 "detail": {
  "error": "session_busy",
  "job_id": "sess_6ac4459ad3e24b8289fc20911d76374f.95eb7d80eb3c47b69faa7a3befd7f3eb",
  "kind": "fetch",
  "message": "session sess_6ac4459ad3e24b8289fc20911d76374f already has a running job ...; poll GET /v1/job/... or wait for it to end before another fetch"
 }
}

on_deadline: "continue". On a page that delivers in time the answer is a normal 200. When the page has not delivered at 55 seconds the route answers 202 instead, with status: "pending", the job_id, and job: {"poll": "/v1/job/<job_id>", "may_run_s": <seconds the job may still run>, "ends_by": <unix seconds>}; the work continues on the 900 second job clock and the page is read from GET /v1/job/{job_id}. On a session whose ttl cannot hold a 900 second job the request is refused:

HTTP 422
{
 "detail": {
  "error": "session_ttl_cannot_hold_job",
  "job_s": 900,
  "ttl_left_s": 599,
  "message": "on_deadline 'continue' runs the fetch as a job of up to 900 s, but this session's ttl ends in 599 s; create the session with a longer ttl, or use on_deadline 'error' or async_mode"
 }
}

Redirects. requested_url is what you asked for and final_url is where the page landed. https://wikipedia.org/ lands on https://www.wikipedia.org/ with redirected: false, because a www prefix is not counted as a move; a change of host or path is. Errors:

Status	`error` or message	When
400	`unsafe url: ...`	The URL is not public http or https (private or loopback address, bad scheme).
402	`insufficient_credits`, `quota_exceeded`	No credit for one action, or the monthly plan allowance is used up.
404	`No such session`	The session id does not exist or is not yours.
409	`session not ready (state=...)`, `session browser is not alive`	The session is still starting or closing, or its browser is gone.
409	`session_busy`	A call or job is still driving this session. `detail` includes `job_id` (nullable), `kind` and `held_for_s` when available.
409	`session_driven_by_client`	A main CDP, native Playwright, or VNC client holds the session. Disconnect it first. CDP observers can remain connected.
410	the session’s terminal record	The session was released or expired.
410	`session_closed_during_fetch`	The session ended while the fetch ran: `{error, message, reason, origin, cause}`.
410	`session's worker is unreachable`	The session’s server cannot be reached. Create a new session.
422	validation error, `session_ttl_cannot_hold_job`	Bad field (an invalid regular expression in `expect_url`, `expect_text` or `no_results_url` is a 422 too), or `continue` on a session whose `ttl` cannot hold the job.
429	rate limit	The per-key drive window was exceeded (`Retry-After` is set).
504	`deadline`	`on_deadline: "error"` (the default) and the page did not deliver by 55 seconds. The body has `status: "error"`, `error: "deadline"`, `failure_type: "TimeoutError"`, `cause: "wait_exhausted"`; work is cancelled and the action is not billed. Traffic and session charges can still apply.

Notes:

Timing: the response limit is 55 seconds; a job runs up to 900 seconds. Plain pages return in 0.5 to 3 seconds. A challenged page spends most of the limit on alternative exits, so a page you know is protected is better run with async_mode: true or on_deadline: "continue".
One job per session at a time. Parallel fetches need parallel sessions.
A finished job stays pollable for at least 20 minutes after it ends, even after the session is released.
Billing for credit-funded accounts: the action is charged for a successful delivery and waived on blocked, empty, incomplete or error outcomes. Egress can still be charged on undelivered pages, at the datacenter or premium rate for the exit used. Solves and session time are charged separately. Developer and Team have no per-call wallet debits.
The page stays open in the session’s current tab after a successful fetch, so a following /act, screenshot or extract works on it and the live view shows it.

GET /v1/job/{job_id}

Poll any job: a fetch (async_mode, or a continue handed over at the limit), an extract, an inspect, a search, a solve, an agent task or a template. The job id is the one the starting call returned. Auth: Bearer API key. No body. Response:

Field	Type	Meaning
`session_id`	string	The session the job ran on.
`job_id`	string	The id polled.
`kind`	string	`fetch`, `extract`, `inspect`, `search`, `solve`, `agent` or `template`. Decides the shape of `result`.
`status`	string	`pending` (still running), `done`, `error`, or `cancelled` (the session ended under it).
`result`	object or null	Null while pending. For `done`, the route’s response body. Failed jobs carry structured fields such as `status: "error"`, `error`, `cause`, `reason`, `http_status` and `detail`; fetch failures can also include fetch fields. For `cancelled`, `{status: "cancelled", error: "session_closed", reason, origin, cause}`.

While pending:

{
 "session_id": "sess_6ac4459ad3e24b8289fc20911d76374f",
 "job_id": "sess_6ac4459ad3e24b8289fc20911d76374f.95eb7d80eb3c47b69faa7a3befd7f3eb",
 "kind": "fetch",
 "status": "pending",
 "result": null
}

Done (text cut):

{
 "session_id": "sess_6ac4459ad3e24b8289fc20911d76374f",
 "job_id": "sess_6ac4459ad3e24b8289fc20911d76374f.95eb7d80eb3c47b69faa7a3befd7f3eb",
 "kind": "fetch",
 "status": "done",
 "result": {
  "final_url": "https://en.wikipedia.org/wiki/HTTP",
  "status": "ok",
  "http_status": 200,
  "title": "HTTP - Wikipedia",
  "text": "Jump to content\nMain menu\nSearch\n...",
  "html_len": 727436,
  "html": null,
  "content_ok": true,
  "self_healed": false,
  "recovery": null,
  "held": null,
  "detected": null,
  "error": null,
  "failure_type": null,
  "cause": null,
  "selector_found": null,
  "settle": null,
  "requested_url": "https://en.wikipedia.org/wiki/HTTP",
  "redirected": false
 }
}

Errors: 404 no such job for an unknown id, an id that belongs to another account, or a record that has expired. A 503 job_worker_unavailable includes Retry-After; the job may still be running, so retry the poll rather than starting duplicate work. Notes:

Poll every 1 to 2 seconds with backoff. A job that has ended can be polled for at least 20 minutes afterwards; the session may already be gone by then and the poll still answers.
GET /v1/fetch/job/{job_id} also works for fetch jobs and answers the bare fetch response instead of the envelope. Prefer /v1/job/{job_id}, which covers every kind.

Get started

Account

Sessions

Reading pages

Automation

MCP

Fetch and jobs

POST /v1/fetch

GET /v1/job/{job_id}

​POST /v1/fetch

​GET /v1/job/{job_id}

POST /v1/fetch

GET /v1/job/{job_id}