Firecrawl

Firecrawl turns websites into clean, LLM-ready data through its scrape, crawl, map, search, and structured-extraction APIs. Use it to pull the content of a single page, crawl and map an entire site, run web searches, or extract structured data with natural-language prompts, then move the results into Nexla for downstream processing.

Power end-to-end data operations for your Firecrawl API with Nexla. Our bi-directional Firecrawl connector is purpose-built for Firecrawl, making it simple to ingest data, sync it across systems, and deliver it anywhere — all with no coding required. Nexla turns API-sourced data into ready-to-use, reusable data products and makes it easy to send data to Firecrawl or any other destination. With comprehensive monitoring, lineage tracking, and access controls, Nexla keeps your Firecrawl workflows fast, secure, and fully governed.

Features

Type: API

SourceDestination

• Seamless API Integration: Connect to any endpoint as source or destination without coding, with automatic data product creation
• Visual Composition & Chaining: Build complex integrations using visual templates, chain API calls, and compose workflows with data validation and filtering
• API Proxy: Expose curated slices of your data securely with a secure and customizable API proxy that validates and transforms data on the fly
• Request optimization with intelligent batching, retry, and caching to minimize API calls and costs

Prerequisites

Before creating a Firecrawl credential, you need a Firecrawl API key. Firecrawl authenticates every API request with an API key sent as a Bearer token in the Authorization header.

To obtain your Firecrawl API key, follow these steps:

1. Sign in to your Firecrawl account at firecrawl.dev, or create an account if you do not already have one.

2. Navigate to the API Keys page in your Firecrawl dashboard.

3. Copy an existing API key, or create a new one. Firecrawl API keys begin with the fc- prefix (for example, fc-123456789).

4. Store the API key securely, as you will need it to configure your Nexla credential. Treat the key as sensitive information, since it grants access to your Firecrawl team's usage and credits.

The API key is sent as Authorization: Bearer <your-api-key> on every request to the Firecrawl API. For complete information about authentication and available endpoints, see the Firecrawl API Documentation.

Authenticate

Credentials required

Field	Required	Secret	Description
API Key	Yes	Yes	Your Firecrawl API key (starts with 'fc-'), from https://www.firecrawl.dev/app/api-keys

Create a credential in Nexla

1. After selecting the data source/destination type, click the Add Credential tile to open the Add New Credential overlay.

2. Enter a name for the credential in the Credential Name field and a short, meaningful description in the Credential Description field.

3. Enter your Firecrawl API key in the API Key field. This is the fc--prefixed key you obtained in Prerequisites. Nexla sends this key as a Bearer token in the Authorization header to authenticate all requests to the Firecrawl API.

   Your Firecrawl API key grants access to your team's scrape, crawl, extract, and usage data and should be kept confidential. If a key is compromised, revoke it and generate a new one from the API Keys page in your Firecrawl dashboard.

4. Click the Save button at the bottom of the overlay. The newly added credential will now appear in a tile on the Authenticate screen during data source/destination creation.

Use as a data source

To create a new data flow, navigate to the Integrate section, and click the New Data Flow button. Select the Firecrawl connector tile, then select the credential that will be used to connect to Firecrawl, and click Next; or, create a new Firecrawl credential for use in this flow.

Endpoint templates

Nexla provides pre-built templates that can be used to rapidly configure data sources to ingest data from common Firecrawl endpoints. Select the endpoint from which this source will fetch data from the Endpoint pulldown menu. Available endpoint templates are listed in the expandable boxes below.

Scrape URL

Scrape a single URL and return its content (markdown/html/etc).

• Provide the target URL to scrape and a schedule that controls how often this source polls. Firecrawl returns the page content for the requested URL.

For detailed information about request parameters, output formats, and response structures, see the Firecrawl Scrape API Documentation.

Search

Search the web and optionally scrape the results.

• Provide a search query and an optional result limit, along with a schedule that controls how often this source polls. Firecrawl returns matching web results.

For detailed information about query parameters, result limits, and response structures, see the Firecrawl Search API Documentation.

Map Site

Discover URLs on a website (sitemap + link discovery).

• Provide the site URL to map and a schedule that controls how often this source polls. Firecrawl returns the list of URLs discovered on the site.

For detailed information about mapping options and response structures, see the Firecrawl Map API Documentation.

Get Crawl Status

Poll a crawl job's status and pull crawled pages.

• Provide the crawl job ID returned by the Start Crawl destination and a schedule that controls how often this source polls. This endpoint paginates automatically to retrieve all crawled pages.

For detailed information about crawl status responses and pagination, see the Firecrawl Crawl Status API Documentation.

Get Crawl Errors

List a crawl job's per-page errors.

• Provide the crawl job ID and a schedule that controls how often this source polls. Firecrawl returns the per-page errors recorded for that crawl job.

For detailed information about crawl error responses, see the Firecrawl Crawl Errors API Documentation.

List Active Crawls

List all active crawls for the authenticated team.

• Provide a schedule that controls how often this source polls. Firecrawl returns all currently active crawls for your team.

For detailed information about active crawl responses, see the Firecrawl Active Crawls API Documentation.

Get Batch Scrape Status

Poll a batch-scrape job's status and pull results.

• Provide the batch scrape job ID returned by the Start Batch Scrape destination and a schedule that controls how often this source polls. This endpoint paginates automatically to retrieve all results.

For detailed information about batch scrape status responses and pagination, see the Firecrawl Batch Scrape Status API Documentation.

Get Batch Scrape Errors

List a batch-scrape job's per-page errors.

• Provide the batch scrape job ID and a schedule that controls how often this source polls. Firecrawl returns the per-page errors recorded for that batch scrape job.

For detailed information about batch scrape error responses, see the Firecrawl Batch Scrape Errors API Documentation.

Get Extract Status

Poll an LLM-extract job's status and pull the extracted data.

• Provide the extract job ID returned by the Start Extract destination and a schedule that controls how often this source polls. Firecrawl returns the structured data extracted by the job.

For detailed information about extract status responses, see the Firecrawl Extract Status API Documentation.

Get Deep Research Status

Poll a deep-research job's status and pull results.

• Provide the deep research job ID returned by the Start Deep Research destination and a schedule that controls how often this source polls. Firecrawl returns the research results for the job.

For detailed information about deep research status responses, see the Firecrawl Deep Research Status API Documentation.

Get LLMs.txt Status

Poll an llms.txt generation job's status and pull the result.

• Provide the llms.txt job ID returned by the Start LLMs.txt Generation destination and a schedule that controls how often this source polls. Firecrawl returns the generated llms.txt result.

For detailed information about llms.txt status responses, see the Firecrawl LLMs.txt Status API Documentation.

Get Credit Usage

Remaining credits for the authenticated team.

• Provide a schedule that controls how often this source polls. Firecrawl returns the remaining credit balance for your team.

For detailed information about credit usage responses, see the Firecrawl Credit Usage API Documentation.

Get Historical Credit Usage

Historical credit usage by period for the authenticated team.

• Provide a schedule that controls how often this source polls. Firecrawl returns credit usage broken down by period for your team.

For detailed information about historical credit usage responses, see the Firecrawl Historical Credit Usage API Documentation.

Get Token Usage

Remaining LLM tokens for the authenticated team (Extract only).

• Provide a schedule that controls how often this source polls. Firecrawl returns the remaining LLM token balance used by Extract for your team.

For detailed information about token usage responses, see the Firecrawl Token Usage API Documentation.

Get Historical Token Usage

Historical token usage by period for the authenticated team.

• Provide a schedule that controls how often this source polls. Firecrawl returns token usage broken down by period for your team.

For detailed information about historical token usage responses, see the Firecrawl Historical Token Usage API Documentation.

Get Queue Status

Metrics about the team's scrape queue.

• Provide a schedule that controls how often this source polls. Firecrawl returns metrics describing the current state of your team's scrape queue.

For detailed information about queue status responses, see the Firecrawl Queue Status API Documentation.

Once the selected endpoint template has been configured, click the Test button to the right of the endpoint selection menu to retrieve a sample of the data that will be fetched. Sample data will be displayed in the Endpoint Test Result panel on the right, allowing you to verify that the source is configured correctly before saving.

Manual configuration

Firecrawl data sources can also be manually configured to ingest data from any valid Firecrawl API endpoint, including endpoints not covered by the pre-built templates, chained API calls, or custom request parameters. Select the Advanced tab at the top of the configuration screen, and follow the instructions in Connect to Any API to configure the API method, endpoint URL, date/time and lookup macros, path to data, metadata, and request headers.

Once all of the relevant settings have been configured, click the Create button in the upper right corner of the screen to save and create the new Firecrawl data source. Nexla will now begin ingesting data from the configured endpoint and will organize any data that it finds into one or more Nexsets.

Use as a destination

Click the + icon on the Nexset that will be sent to the Firecrawl destination, and select the Send to Destination option from the menu. Select the Firecrawl connector from the list of available destination connectors, then select the credential that will be used to connect to Firecrawl, and click Next; or, create a new Firecrawl credential for use in this flow.

Endpoint templates

Nexla provides pre-built templates that can be used to rapidly configure destinations to send data to common Firecrawl endpoints. Select the endpoint to which data will be sent from the Endpoint pulldown menu. Then, click on the template in the list below to expand it, and follow the instructions to configure additional endpoint settings.

Start Crawl

Start an asynchronous crawl job for a URL; poll the Get Crawl Status source with the returned id.

• Each record from your Nexset is sent as a request body to start a crawl job. Firecrawl returns a job ID that you can pass to the Get Crawl Status source to retrieve the crawled pages.

For detailed information about the crawl request body and response, see the Firecrawl Start Crawl API Documentation.

Start Batch Scrape

Start an asynchronous batch-scrape job for multiple URLs; poll the Get Batch Scrape Status source with the returned id.

• Each record from your Nexset is sent as a request body to start a batch scrape job. Firecrawl returns a job ID that you can pass to the Get Batch Scrape Status source to retrieve the results.

For detailed information about the batch scrape request body and response, see the Firecrawl Start Batch Scrape API Documentation.

Start Extract

Start an asynchronous LLM-extract job for one or more URLs; poll the Get Extract Status source with the returned id.

• Each record from your Nexset is sent as a request body to start an extract job. Firecrawl returns a job ID that you can pass to the Get Extract Status source to retrieve the extracted data.

For detailed information about the extract request body and response, see the Firecrawl Start Extract API Documentation.

Start Deep Research

Start an asynchronous deep-research job for a query; poll the Get Deep Research Status source with the returned id.

• Each record from your Nexset is sent as a request body to start a deep research job. Firecrawl returns a job ID that you can pass to the Get Deep Research Status source to retrieve the results.

For detailed information about the deep research request body and response, see the Firecrawl Start Deep Research API Documentation.

Start LLMs.txt Generation

Start an asynchronous llms.txt generation job for a site; poll the Get LLMs.txt Status source with the returned id.

• Each record from your Nexset is sent as a request body to start an llms.txt generation job. Firecrawl returns a job ID that you can pass to the Get LLMs.txt Status source to retrieve the generated result.

For detailed information about the llms.txt request body and response, see the Firecrawl Start LLMs.txt API Documentation.

Cancel Crawl

Cancel a running crawl job.

• Provide the job ID of the crawl to cancel. Each record from your Nexset supplies the job ID, and Firecrawl cancels the corresponding running crawl job.

For detailed information about cancelling crawl jobs, see the Firecrawl Cancel Crawl API Documentation.

Cancel Batch Scrape

Cancel a running batch-scrape job.

• Provide the job ID of the batch scrape to cancel. Each record from your Nexset supplies the job ID, and Firecrawl cancels the corresponding running batch scrape job.

For detailed information about cancelling batch scrape jobs, see the Firecrawl Cancel Batch Scrape API Documentation.

Manual configuration

Firecrawl destinations can also be manually configured to send data to any valid Firecrawl API endpoint. Select the Advanced tab at the top of the configuration screen, and follow the instructions in Connect to Any API to configure the API method, data format, endpoint URL, request headers, attribute exclusions, record batching, and response webhooks.

Firecrawl APIs expect JSON in the request body for most operations. Because crawl, batch scrape, extract, deep research, and llms.txt jobs run asynchronously, use the corresponding status endpoints on the data source side to retrieve results once a job completes.

Save & activate

Once all endpoint settings have been configured, click the Done button in the upper right corner of the screen to save and create the destination. To send the data to the configured Firecrawl endpoint, open the destination resource menu, and select Activate.

The Nexset data will not be sent to the Firecrawl endpoint until the destination is activated. Destinations can be activated immediately or at a later time, providing full control over data movement.