Guides — Spidra Blog

Step-by-step guides to extract data, automate workflows, and build scrapers with Spidra. Learn how to scrape any site without writing CSS selectors.

All Trends Announcements Guides Tutorials

Spidra crawl API: how to crawl an entire website and extract data

Discover and extract data from entire websites with Python and Node.js. Covers re-extraction, authenticated crawling, and proxy routing.

Joel OlawanleJune 24, 2026 · 15 min read

Spidra browser actions: complete guide to clicking, scrolling, and interacting before scraping

Complete guide to Spidra browser actions. Learn how to click, scroll, type, and use forEach with real examples.

Joel OlawanleJune 23, 2026 · 15 min read

Spidra batch scraping API: how to scrape 50 URLs in parallel

Process up to 50 URLs in parallel with the Spidra batch scraping API. Covers AI extraction, JSON schema, retrying failures, and proxy routing.

Joel OlawanleJune 22, 2026 · 13 min read

HTML vs Markdown for AI: which format is better for LLMs?

Raw HTML wastes up to 90% of your LLM context window on boilerplate. Learn why Markdown is better for RAG and AI training, and when HTML is actually the right choice.

Joel OlawanleJune 12, 2026 · 12 min read

What is a headless browser? How it works, uses, and tools (2026)

Learn what a headless browser is, how it works, Playwright vs Puppeteer explained with real code, and when to use a managed API instead.

Joel OlawanleJune 11, 2026 · 15 min read

How to scrape cf_clearance cookies from Cloudflare-protected websites

Learn how to extract cf_clearance cookies from Cloudflare-protected sites, use them in a requests session, and why the approach breaks at scale.

Joel OlawanleJune 10, 2026 · 10 min read

Spidra API tutorial: complete guide to web scraping with the Spidra API

Complete Spidra API guide. Scrape pages, extract structured JSON, run browser actions, and crawl entire websites with working Python and cURL examples.

Joel OlawanleJune 9, 2026 · 18 min read

What is AI scraping? How it works, use cases, and how to get started

Learn what AI scraping is, how it differs from traditional web scraping, common use cases, and how to extract structured data from any website in minutes.

Joel OlawanleJune 9, 2026 · 12 min read

5 best AI web scraper tools I've tested in 2026 (free + paid)

I tested 5 AI web scraper tools that actually work. Here is what each one is good for, where each one falls short, and which one I reach for depending on the job.

Joel OlawanleJune 5, 2026 · 9 min read

What is WebGL fingerprinting and how to bypass it when scraping

Learn how WebGL fingerprinting uses GPU hardware to identify scrapers, why it is harder to bypass than canvas fingerprinting, and what actually works at scale.

Joel OlawanleJune 4, 2026 · 10 min read

How to scale web scraping with Playwright BrowserContext

Learn how Playwright BrowserContext works, how to run concurrent sessions with asyncio, and what to do when local hardware becomes the bottleneck.

Joel OlawanleJune 3, 2026 · 11 min read

How to use pyppeteer_stealth for web scraping

Learn how pyppeteer_stealth patches Pyppeteer's bot signals, how to fix the Chromium setup error, and where it falls short against modern anti-bot systems.

Joel OlawanleJune 2, 2026 · 9 min read

How to use Puppeteer Cluster to scale up web scraping

Learn how to use puppeteer-cluster to run concurrent browser workers, when to use each concurrency model, and how to scale beyond local hardware limits.

Joel OlawanleJune 1, 2026 · 11 min read

How to use Ghost Cursor for web scraping

Learn how Ghost Cursor adds human-like mouse movement to Puppeteer, how every method works, and what to do when mouse movement alone is not enough.

Joel OlawanleMay 29, 2026 · 12 min read

Puppeteer real browser: How to avoid detection when scraping

Learn how Puppeteer Real Browser patches Puppeteer for stealth scraping, where it fails against real anti-bot protection, and what to use instead.

Joel OlawanleMay 28, 2026 · 9 min read

Start scraping for free.

Get 300 free credits to explore Spidra. Build your first scraper in minutes, not hours. Upgrade anytime as you scale.

We build features around real workflows. Usually within days.