Skip to main content
All releases

forEach gets modes, pagination, and per-item extraction

The forEach action now supports three interaction modes, pagination across multiple pages, per-item AI extraction, and smarter handling of CSS vs natural language selectors.

The forEach action has been rebuilt with a lot more flexibility. You can now control exactly how it interacts with each element on the page, follow pagination to collect more items, and run a separate AI extraction per item before combining results.

Three interaction modes

{
  "type": "forEach",
  "observe": "Find all product cards",
  "mode": "inline"
}

click is the default. Spidra clicks each element and captures whatever opens, such as a modal, drawer, or accordion panel.

inline reads each element's content directly without clicking anything. This is the right choice for cards, table rows, list items, and other containers where the content is already visible.

navigate follows the link on each element to the destination page, captures the content there, and then comes back to continue with the next element.

Pagination

forEach can now follow a "next page" button to keep collecting elements across multiple pages before it stops.

{
  "type": "forEach",
  "observe": "Find all job listings",
  "mode": "inline",
  "captureSelector": ".job-card",
  "pagination": {
    "nextSelector": "button[aria-label='Next page']",
    "maxPages": 5
  },
  "maxItems": 50
}

Per-item AI extraction

For large datasets or when using navigate mode where each destination page has a lot of content, you can run a Gemini extraction on each item individually before the results are combined.

{
  "type": "forEach",
  "observe": "Find all speaker profile links",
  "mode": "navigate",
  "itemPrompt": "Extract speaker name, title, company, and bio as JSON"
}

Other new fields

  • captureSelector lets you point forEach at the exact container you want to capture, as a CSS selector, XPath, or plain English description. If you skip it, Spidra falls back to [role="dialog"] and then the full page.
  • maxItems caps how many elements are processed across all pages. The default is 50 and that is also the hard maximum.
  • waitAfterClick controls how long Spidra waits after clicking before it captures content. Default is 2500ms.
  • observe is now the preferred field for describing which elements to find. The older value and selector fields still work.

Smarter selector routing

All actions now automatically detect whether a selector is CSS or XPath vs natural language and route accordingly. CSS and XPath go straight to the browser engine, which is faster and does not use an AI call. Natural language descriptions still go through Stagehand as before. The selector field is the recommended way to specify a target for click, type, check, and uncheck actions.

Read more in the Browser Actions docs.

Start scraping for free.

Get 300 free credits to explore Spidra. Build your first scraper in minutes, not hours. Upgrade anytime as you scale.

We build features around real workflows. Usually within days.