Firecrawl with an AI Agent: Scrape and Search Websites

· DoubleX

Quick answer

To use Firecrawl with an AI agent, connect the Firecrawl connector, choose which actions the agent can run, and ask in plain language for pages to be extracted or searched. In DoubleX Agentes, the agent can then save the content to a spreadsheet, document or Brain note, manually or on a schedule.

Teams that research markets, track documentation or collect reference material spend a lot of time copying content from websites into other tools. An AI agent connected to Firecrawl can do the collecting for you and put the results where your team already works. This guide shows how to set that up and what to ask for.

What does Firecrawl do inside an AI agent?#

Firecrawl is a web data tool that extracts pages, searches the web and collects structured content from sites. When it is connected as a connector, an agent can use those tools during a task, instead of you opening pages and copying text by hand.

The agent decides when to call Firecrawl based on your request. It reads the result, cleans it up, and then does something with it: summarizes, compares, fills a table or writes a document. Each tool call is recorded in the task history, so you can see which pages were fetched and what came back.

NeedHow the agent uses FirecrawlWhere the result can go
Read one pageExtracts the content of a URLSummary in the chat or a Google Doc
Find sources on a topicSearches the web and collects the resultsTable in Google Sheets
Gather fields from many pagesCollects structured content page by pageAirtable base or spreadsheet rows
Keep reference materialExtracts documentation or articlesNote in the Brain

How do you connect Firecrawl to DoubleX Agentes?#

You connect Firecrawl from the Connectors screen, then enable it for the agents that should use it. Setup takes a few minutes and does not require code.

  1. Create your account (free, no card) and, if you have not yet, run the diagnostic so the system proposes a team of agents. The Quickstart walks through this.
  2. Open Connectors and add Firecrawl. Choose whether the connection is personal, for a team or for the whole company.
  3. Open the agent that will do the research, in Agents, and enable the Firecrawl connector for it.
  4. Review which Firecrawl actions that agent may run. Keep only what the job needs.
  5. Ask for a first task, such as extracting one page, and check the steps in Tasks.

If you want to manage connectors from code, the Connectors API reference describes how.

What kinds of requests can you make?#

You can ask for extraction, search or structured collection in plain language. Name the source, the fields you want and where the result should go.

Examples you can copy:

  • "Extract the page at this URL and give me a one-paragraph summary plus the five main headings."
  • "Search the web for recent articles about data retention rules for SaaS vendors and list the title, source and a one-line takeaway for each."
  • "Go through these 15 documentation URLs and collect the page title, the last updated date and any mention of rate limits into a table."
  • "Read the pricing and terms pages of these three vendors and write a side-by-side comparison of what each one includes."
  • "Pull the release notes from this site and list what changed since the last note saved in the Brain."

The more specific the fields, the more consistent the output. If a page does not contain a field, ask the agent to write "not found" instead of guessing.

How do you turn collected content into a spreadsheet, document or Brain note?#

Connect the destination as a connector and say where the result should go in the same request. The agent collects with Firecrawl and writes with the other connector, in one task.

  • Spreadsheet: with Google Sheets, the agent can create a spreadsheet or append rows. Define the columns first, for example URL, title, date, summary.
  • Document: with Google Docs, the agent can create a document or fill a template, useful for research briefs and source lists.
  • Brain: the Brain is your company knowledge base. Notes live in folders and link to each other, and agents consult them when answering. Saving extracted documentation there means any agent can use it later.
  • Deliverable file: the agent can also hand back a report or a page as an artifact. See Artifacts.

How do you create a scheduled routine to collect site content?#

A routine is a trigger that makes the agent work on its own at a set time. You describe the job once, and the agent repeats it, for example every Monday at 8 a.m. or once a day.

Triggers can run at a time, once on a date, when a webhook arrives, when an email arrives or when something happens in a connected app. A routine runs with the permissions of the person who created it. Details are in Triggers, and Webhooks explains how an outside system can start an agent.

A simple weekly routine:

  1. Write the request exactly as you would for a one-time task, including the list of URLs or the search topic.
  2. Say where results go: "append new rows to the Sheet called Research Log".
  3. Ask for a short summary of what is new, so you can scan it quickly.
  4. Schedule it in the agent's triggers for the day and time you want.
  5. After the first run, open the task history and adjust the request if columns or sources need fixing.

Which other connectors work well with Firecrawl, and what should you watch out for?#

Firecrawl pairs naturally with the connectors that store or share results. The agent keeps each tool's permissions separate, so you control what it can read and change.

CombinationUse
Firecrawl + Google SheetsResearch logs and comparison tables
Firecrawl + Google DocsBriefs and source summaries
Firecrawl + NotionPages and databases for team research
Firecrawl + SlackPost a summary to a channel
Firecrawl + Tavily or ExaBroader web search plus page extraction

For deeper analysis of collected data, see Hex with an AI agent. A few precautions:

  • Give the agent write access only to the files it needs. Actions that change data can stay off for read-only work.
  • The agent asks for confirmation before sending, deleting or changing things in your systems.
  • Treat web content as unverified. Ask the agent to include the source URL for every item so you can check it.
  • Your company content is not used to train models, and you can choose zero-retention providers in Models.

Frequently asked questions#

Do I need to code to use Firecrawl with an AI agent?#

No. You connect Firecrawl from the Connectors screen and describe the job in your own words. Developers who want more control can use the REST API, the CLI or the MCP server.

Can the agent scrape many pages in one request?#

Yes, you can give a list of URLs or a topic and ask for the same fields from each page. Start with a small batch, check the result in the task history, and then scale up.

Where are the results stored?#

They stay in the task history, and the agent can also write them to Google Sheets, Google Docs, Notion or the Brain, depending on which connectors you enable.

Can I run the collection automatically?#

Yes. Schedule a routine for a set time, such as every Monday, or trigger it with a webhook. The routine runs with the permissions of the person who created it.

Is it safe to give an agent access to Firecrawl?#

You choose which systems each agent uses and which actions it can run, and everything is logged in the task history. Connection credentials are encrypted, and your content is not used for model training.