Skip to main content

Algolia Data Source

The Algolia connector enables you to ingest data from Algolia's Search API, allowing you to retrieve search results, list indices, and perform search queries on your Algolia indices. This connector is particularly useful for applications that need to extract search data, analyze search patterns, or sync search results to other systems.
algolia_api.png

Algolia

Follow the instructions below to create a new data flow that ingests data from an Algolia source in Nexla.

Create a New Data Flow​

  1. To create a new data flow, navigate to the Integrate section, and click the New Data Flow button. Then, select the desired flow type from the list, and click the Create button.

  2. Select the Algolia connector tile from the list of available connectors. Then, select the credential that will be used to connect to the Algolia instance, and click Next; or, create a new Algolia credential for use in this flow.

  3. In Nexla, Algolia data sources can be created using pre-built endpoint templates, which expedite source setup for common Algolia endpoints. Each template is designed specifically for the corresponding Algolia endpoint, making source configuration easy and efficient.
    • To configure this source using a template, follow the instructions in Configure Using a Template.

    Algolia sources can also be configured manually, allowing you to ingest data from Algolia endpoints not included in the pre-built templates or apply further customizations to exactly suit your needs.
    • To configure this source manually, follow the instructions in Configure Manually.

Configure Using a Template​

Nexla provides pre-built templates that can be used to rapidly configure data sources to ingest data from common Algolia endpoints. Each template is designed specifically for the corresponding Algolia endpoint, making data source setup easy and efficient.

Endpoint Settings​

  • Select the endpoint from which this source will fetch data from the Endpoint pulldown menu. Available endpoint templates are listed in the expandable boxes below. Click on an endpoint to see more information about it and how to configure your data source for this endpoint.

    List Indices

    This endpoint retrieves a list of all indices in your Algolia application. Use this endpoint when you need to discover available indices, monitor index metadata, or build dynamic workflows that operate on multiple indices.

    • This endpoint automatically retrieves all indices in your Algolia application. No additional configuration is required beyond selecting this endpoint template.
    • Enter a schedule in the Schedule field to specify when this data source should run. The schedule uses cron expression format (e.g., 0 6 * * * for daily at 6 AM). The default schedule is 0 6 * * * (daily at 6 AM).
    • The endpoint returns index metadata including index names, creation dates, number of records, and other index configuration details.

    This endpoint is useful for discovering available indices before configuring more specific search endpoints. The returned data includes index names that can be used as input for other Algolia endpoints. The endpoint uses the GET method to retrieve all indices in your application. For complete information about listing indices, see the Algolia List Indices API Documentation.

    Search Records in Index

    This endpoint performs a search query on a specific Algolia index and returns matching records. Use this endpoint when you need to retrieve search results, extract indexed data, or sync search results to other systems.

    • Enter a schedule in the Schedule field to specify when this data source should run. The schedule uses cron expression format (e.g., 0 7 * * * for daily at 7 AM). The default schedule is 0 7 * * * (daily at 7 AM).
    • Enter the name of the index to search in the Index Name field. This should match the exact name of an index in your Algolia application. You can find available index names by using the "List Indices" endpoint first.
    • Optionally, enter a search query in the Search Query field to filter the results. If left empty, the endpoint will return all records in the index (up to the API limits). The search query uses Algolia's search syntax and supports features like filtering, faceting, and typo tolerance.

    The search query parameter is optional. When left empty, the endpoint returns all records in the specified index (up to API limits). For complex search queries, you can use Algolia's advanced search syntax including filters, facets, and ranking parameters. The endpoint uses POST method to /1/indexes/{indexName}/query with the query in the request body. For complete information about Algolia search queries and parameters, see the Algolia Search API Documentation.

Endpoint Testing​

Once the selected endpoint template has been configured, Nexla can retrieve a sample of the data that will be fetched according to the current settings. This allows users to verify that the source is configured correctly before saving.

  • To test the current endpoint configuration, click the Test button to the right of the endpoint selection menu. Sample data will be fetched & displayed in the Endpoint Test Result panel on the right.

  • If the sample data is not as expected, review the selected endpoint and associated settings, and make any necessary adjustments. Then, click the Test button again, and check the sample data to ensure that the correct information is displayed.

Configure Manually​

Algolia data sources can be manually configured to ingest data from any valid Algolia API endpoint. Manual configuration provides maximum flexibility for accessing endpoints not covered by pre-built templates or when you need custom API configurations.

With manual configuration, you can also create more complex Algolia sources, such as sources that use chained API calls to fetch data from multiple endpoints or sources that require custom authentication headers or request parameters.

API Method​

  1. To manually configure this source, select the Advanced tab at the top of the configuration screen.

  2. Select the API method that will be used for calls to the Algolia API from the Method pulldown menu. The most common methods are:

    • GET: For retrieving data from the API (most common for Algolia search operations)
    • POST: For sending data to the API or performing search queries with complex parameters

API Endpoint URL​

  1. Enter the URL of the Algolia API endpoint from which this source will fetch data in the Set API URL field. This should be the complete URL including the protocol (https://) and any required path parameters.

Algolia API URLs typically follow the format: https://{ApplicationID}-dsn.algolia.net/1/indexes/{IndexName}/query for search operations, or https://{ApplicationID}-dsn.algolia.net/1/indexes for listing indices. You can also use https://{ApplicationID}.algolia.net as the base URL. Replace {ApplicationID} with your Algolia Application ID from your credentials. Algolia provides multiple DNS endpoints for high availability. Ensure the API endpoint URL is correct and accessible with your current credentials. You can test the endpoint using the Test button after configuring the URL. For complete information about Algolia API endpoints and base URLs, see the Algolia Search API Documentation.

Request Headers​

  1. If Nexla should include any additional request headers in API calls to this source, enter the headers & corresponding values as comma-separated pairs in the Request Headers field (e.g., header1:value1,header2:value2).

You do not need to include authentication headers (X-Algolia-Application-Id and X-Algolia-API-Key) as these are automatically included from your credentials. However, you may need to include additional headers for specific Algolia API features or custom configurations. The Content-Type header should be set to application/json for most Algolia API requests.

Response Data Path​

  1. Enter the JSON path expression that identifies the location of the data array in the API response in the Response Data Path field. This path tells Nexla where to find the array of records in the JSON response.

For Algolia search responses, the data path is typically $.hits[*] to extract individual search results from the hits array. The search response structure includes a hits array containing the matching records. For list indices responses, use $.items[*] to extract individual index objects from the items array. JSON path expressions use dot notation and array indexing to navigate the response structure. For complete information about Algolia API response formats, see the Algolia Search API Documentation.

Schedule​

  1. Enter a schedule in the Schedule field to specify when this data source should run. The schedule uses cron expression format to define the frequency and timing of data ingestion.

Common cron expressions include: 0 6 * * * for daily at 6 AM, 0 */6 * * * for every 6 hours, and 0 0 * * 0 for weekly on Sunday at midnight. For more information about cron expressions, see the Nexla documentation on scheduling.