RAiN — Rivium AI Network
Back to Help

Firecrawl Crawl

Read several pages of a website at once, not just one.

What it does

Follows a site's links and returns many pages at once. Reach for it when you need a whole section — a documentation site, a catalogue — rather than one page.

What you need

API key from Firecrawl. firecrawl.dev → Dashboard → API Keys.

Connecting it

Credentials belong to one project, not to your workspace and not to you. A system built for one client never runs on another client's account, and a package handed to a business runs on theirs.

  1. Open the project and stay on the Visual Editor tab.
  2. Open the Tools & Integrations rail on the right.
  3. Find this tool, open it, and paste the credential.

To change it later, open the same panel — you can leave the key blank to keep the stored one and change only the other field.

Using it in a step

An Action step runs one tool with the arguments you type, exactly as written, with no model involved — so it costs nothing per run and behaves the same every time. Add one from the canvas toolbar, choose Firecrawl Crawl, and fill in:

Field Required What goes in it
url yes Where to start. It follows links from there.
limit no Maximum pages. Always set one — a crawl costs per page.

Values can carry results forward: {{previous}} is the step before, and {{row:column}} is a column from the row this run is about when the trigger fans out.

Press Try it to run the step on its own before committing to a whole sequence.

Using it with an agent

Tick Firecrawl Crawl on any agent that should be able to call it, and the agent decides when to. Use an agent when the judgement matters and an Action step when the certainty does.

A worked example

An expert building a support agent crawls the client's entire FAQ section once, and the text becomes the project's knowledge base.

When it goes wrong

It is slow and it costs per page. Set a limit. A crawl with no ceiling on a large site is an expensive way to discover you wanted one page.

See also