Read block metadata first
Every API response includes Sf-block-type and Sf-failure-reason. Retry with the same settings only when the failure is transient (timeout). For cloudflare or captcha blocks, escalate proxy tier or session before retrying.
Proxy tier selection
- Default — most sites with render_js
- premium_proxy=true — geo restrictions, datacenter IP reputation, harder targets
- Combine with render_js, session_id, and retries for Cloudflare-heavy sites
Session persistence
Pass session_id from response metadata on follow-up requests to reuse cookies across pagination and multi-step flows.
Further reading
Anti-bot FAQ
What is a scrape bot?+
A scrape bot is automated software that visits web pages and extracts structured data. DeepFetch provides managed scrape bots in the cloud so you don't run Playwright or Puppeteer yourself.
How is DeepFetch different from running my own scrape bot?+
We handle browser pools, proxy rotation, CAPTCHA retries, scaling, and block detection. You send one API request and receive JSON — no infrastructure to maintain.
Which proxy tier should I use?+
Start with default rendering for most sites. Use premium_proxy for geo-restricted, rate-limited, or harder anti-bot targets.