Skip to main content
Read this before choosing a route. The rules below are repeated next to the routes they affect.

​
Use POST /v1/fetch for stealth reads, not raw CDP

For any page that may be protected (anti-bot, challenge, geo wall, login wall), read it with POST /v1/fetch (or POST /v1/extract for markdown). Do not drive the navigation yourself over raw CDP or Playwright page.goto(). Why:
  • fetch detects blocks and challenges, attempts CAPTCHA solving, and escalates through exit tiers within its budget. It reports the outcome in content_ok, self_healed, recovery, detected, and premium_required. Main CDP clients receive automatic recovery with cdp_observe: true, the default; false disables it. Observers bypass recovery, and custom-proxy sessions always keep their proxy exit.
  • fetch returns a typed outcome (status, content_ok, error) so an agent can branch without guessing from HTML.
  • A vanilla CDP client sends Runtime.enable and other automation tells that protected sites fingerprint. The REST routes drive the browser through a hardened path.
  • fetch and navigate keep the session’s cookies and persona coherent across the retry ladder; a hand-rolled retry over CDP does not.
Use raw CDP or Playwright for interactive automation that needs the full browser API. Keep cdp_observe: true to check main-client page loads for challenges. CDP observers bypass this check regardless of the setting.

​
Other rules

  • One session, many pages. A session is a browser with cookies and a stable persona; reuse it across calls. Do not create a session per page. Release it with DELETE /v1/session/{id} in a finally block; it bills by the minute until it ends.
  • Coordinate browser control. A main CDP, native Playwright, or VNC client holds the session and causes REST drive calls to return 409 session_driven_by_client. CDP observers can remain connected during REST calls. A competing REST call can return 409 session_busy. Use separate sessions for independent work.
  • One job per session. A second fetch on a session with a running job is refused with 409 session_busy. Parallel reads need parallel sessions.
  • Protected pages go async. A challenged page can spend the whole 55 second synchronous budget on exit escalation. For pages you know are protected, send async_mode: true or on_deadline: "continue" and poll GET /v1/job/{job_id}.
  • Trust content_ok, not status, when deciding whether a fetch delivered real content.
  • Check GET /v1/usage and GET /v1/credits before a large batch; remaining and premium_required tell you whether residential unlock is available to you.
  • POST /v1/solve is for a page you are driving by hand with navigate and act. A fetch that came back blocked has already tried the solvers.
  • Send Idempotency-Key on every POST /v1/session retry so a lost reply never leaves you paying for two browsers.
  • Use POST /v1/session/{id}/elements to get a selector before act; never guess selectors.