One API to scrape and crawl the web
Send a URL and a prompt and get structured data back. Spidra handles rendering, proxies, CAPTCHAs, and AI extraction so you never have to.
300 free credits · No credit card required
"urls": [{ "url": "eventbrite.com/houston" }],
"prompt": "Get organizers with emails",
"output": "json",
"useProxy": true,
"proxyCountry": "us",
"screenshot": true,
"extractContentOnly": true
}
Scrape
POST /api/scrape
Turn any URL into structured data by passing a prompt to extract exactly what you need and let AI handle the rest.
Crawl
POST /api/crawl
Give a starting URL and instructions. Spidra discovers pages, crawls them, and returns transformed data. Re-run AI extraction on any existing crawl with a new transform instruction.
From URL to clean data
Whether you need one data point or entire datasets at scale, the Spidra API just works.
"event": "Houston's Funniest Comedy Show",
"organizer": "The Riot Comedy Club",
"email": "[email protected]",
"phone": "(713) 264-8664",
"website": "theriothtx.com",
"facebook": "fb.com/theriothtx"
}
Built for any use case
From lead generation to dataset curation, the API extracts structured data from even the most complex websites.
Lead Generation
Turn any directory or marketplace into a lead database. Extract verified emails, phone numbers, and business details from Eventbrite, Google Maps, or any source.
Price Monitoring
Track competitor pricing across Amazon, Shopify stores, and e-commerce platforms. Get structured product data including prices, sizes, and stock availability.
Market Research
Analyze reviews, ratings, and sentiment across G2, Trustpilot, and Reddit. Get market cap, P/E ratios, and financial metrics from any public source.
Dataset Building
Build structured datasets from any source at scale, including research papers from arXiv, job listings from LinkedIn, and property data from real estate platforms.
Real Estate
Extract property listings with pricing, photos, and agent details from Zillow, Redfin, or any MLS. Filter by location, price range, and property type.
Research & Academia
Extract clinical trial results, paper abstracts, and citation data from PubMed, Google Scholar, and academic databases for systematic reviews.
Scrape every item, not just the page
Most of the data you actually want is hidden behind clicks — product variant drawers, detail modals, accordion panels. forEach finds every matching element, interacts with each one, and returns a structured array in a single request.
One instruction.
Every item extracted.
Describe which elements to find in plain English. Spidra locates each one, interacts with it, runs your extraction prompt on whatever appears, and returns a single array.
{
"urls": [
{
"url": "https://example.com/products/wireless-headphones",
"actions": [
{
"type": "forEach",
"observe": "Find all color and size variant buttons",
"mode": "click",
}
]
}
],
"prompt": "Extract variant name, price, and stock status",
"output": "json"
}Everything built in
No external proxy subscriptions. No CAPTCHA solver accounts. No browser infrastructure to maintain. It's all in the API.
AI extraction
Describe what you want in plain English. The API uses AI to extract exactly those fields without needing any CSS selectors.
Browser rendering & actions
Full headless browser renders SPAs and executes sequences of actions like clicking, typing, scrolling, and iterating over element lists with forEach pagination.
CAPTCHA solving
Spidra automatically detects and solves popular CAPTCHAs including reCAPTCHA v2/v3, hCaptcha, and Cloudflare Turnstile.
Proxy rotation
Enterprise-grade residential proxies rotate per request. Target by region: US, EU, Asia, or global.
Cookie authentication
Pass session cookies to scrape pages behind logins like dashboards, member portals, and authenticated directories.
Multiple output formats
Get data in whatever format your pipeline needs, whether that's JSON, Markdown, or full-page screenshots.
Start using the API today
Three calls. Submit, poll, get data. Available in every language.
{
"status": "completed",
"result": {
"content": [
{
"event_name": "Houston's Funniest",
"organizer": "The Riot Comedy Club",
"organizer_url": "theriothtx.com"
},
{
"event_name": "Stand Up Comedy Night",
"organizer": "Comedy Showcase",
"organizer_url": "comedyshowcase.com"
}
]
}
}Give your AI assistant web scraping superpowers
Connect Spidra to Claude, Cursor, or any MCP-compatible AI client. Just ask it to scrape a page, and Spidra handles everything else. Build custom workflows that combine web data with AI reasoning.
FAQ
Common questions about the Spidra scraping API.
