Skip to main content

Apify Destination

Nexla's bi-directional connectors allow data to flow both to and from any location, making it simple to create a FlexFlow, DB-CDC, Spark ETL, or Replication data flow that sends data to an Apify location.
apify_api.png

Apify

Create an Apify Destination​

  1. Click the + icon on the Nexset that will be sent to the Apify destination, and select the Send to Destination option from the menu.

  2. Select the Apify connector from the list of available destination connectors. Then, select the credential that will be used to connect to the Apify organization, and click Next; or, create a new Apify credential for use in this flow.

  3. In Nexla, Apify destinations can be created using pre-built endpoint templates, which expedite destination setup for common Apify endpoints. Each template is designed specifically for the corresponding Apify endpoint, making destination configuration easy and efficient.
    • To configure this destination using a template, follow the instructions in Configure Using a Template.

    Apify destinations can also be configured manually, allowing you to send data to Apify endpoints not included in the pre-built templates or apply further customizations to exactly suit your needs.
    • To configure this destination manually, follow the instructions in Configure Manually.

Configure Using a Template​

Nexla provides pre-built templates that can be used to rapidly configure destinations to send data to common Apify endpoints. Each template is designed specifically for the corresponding Apify endpoint, making destination setup easy and efficient.

  • To configure this destination using a template, select the endpoint to which data will be sent from the Endpoint pulldown menu. Then, click on the template in the list below to expand it, and follow the instructions to configure additional endpoint settings.

    Create actor

    This endpoint creates a new actor in your Apify account with settings specified in an Actor object passed as JSON in the POST payload. Use this endpoint when you need to create new actors from your Nexset data.

    • The endpoint will send the Nexset data as JSON in the request body to create actors. Ensure your Nexset contains all required fields for actor creation, including actor name, description, and other actor settings.
    • Each record in the Nexset will be sent as a separate API request to create a new actor in your Apify account.

    The request body must contain valid JSON with all required fields for actor creation. For detailed information about required fields and data formats, see the Apify API documentation.

    Create dataset

    This endpoint creates a new dataset in your Apify account and returns its object. Use this endpoint when you need to create new datasets from your Nexset data.

    • The endpoint will send the Nexset data as JSON in the request body to create datasets. Ensure your Nexset contains all required fields for dataset creation, including dataset name and other dataset settings.
    • Each record in the Nexset will be sent as a separate API request to create a new dataset in your Apify account.

    The request body must contain valid JSON with all required fields for dataset creation. For detailed information about required fields and data formats, see the Apify API documentation.

Configure Manually​

Apify destinations can be manually configured to send data to any valid Apify API endpoint.

Using manual configuration, you can also configure Nexla to automatically send the response received from the Apify API after each call to a new Nexla webhook data source.

API Method​

  1. To manually configure this destination, select the Advanced tab at the top of the configuration screen.

  2. Select the API method that will be used for calls to the Apify API from the Method pulldown menu.

Data Format​

  1. Select the format in which the Nexset data will be sent to the Apify API from the Content Format pulldown menu. Nexla will automatically convert the data to the selected format for each API call. Apify API endpoints typically expect JSON format for request bodies.

API Endpoint URL​

  1. Enter the URL of the Apify API endpoint to which you want to send the Nexset data in the URL field. For update/upsert operations, include the ID of the object to be updated at the end of the URL. Apify API endpoints typically follow the pattern https://api.apify.com/v2/{resource_type} for creating resources.

Request Headers​

Optional

  • If Nexla should include any additional request headers in API calls to this destination, enter the headers & corresponding values as comma-separated pairs in the Request Headers field (e.g., header1:value1,header2:value2).

    You do not need to include any headers already present in the credentials.

Exclude Attributes from the Call​

Optional

  • If any record attributes in the Nexset should be omitted when sending data to this Apify destination, select the attributes from the Exclude Attributes pulldown menu.

  • Any number of attributes can be selected for exclusion, and all excluded attributes will be shown in the field. To remove an attribute from the list, click the X icon next to the attribute name.

Record Batching​

Optional

  1. If records should be sent to this destination in batched API calls, check the box next to Would you like to batch your records together? to enable record batching.

  2. Enter the maximum number of records that should be batched together in a single API call in the Batch Size field. By default, this value is set to 100.

  3. Select the algorithm that will be used to group records into batches from the Grouping Algorithm pulldown menu. The sample request shown in the panel on the right will be updated to reflect the current batching settings. Some algorithms require additional settings—click on an algorithm listed below to view instructions for configuring these settings.

Response Webhook​

Optional

Nexla can automatically send the response received from the Apify API after each call to a new Nexla webhook data source. This option allows you to keep track of the status of each API call and any additional information returned after each call.

  • To enable this option, check the box next to Would you like to process the API response as a Nexla Webhook source?.

Sample Request Payload​

Sample request payloads containing a portion of the Nexset data that will be sent to the Apify API endpoint based on the current settings are shown in the Sample Payload panel on the right. These samples can be referenced to ensure that the destination and request settings are correctly configured.

  • Click on a sample request payload to expand it and view the complete payload content.
  • Sample payloads are automatically updated with each setting change, making it easy to verify that changes achieve the desired effect.

Endpoint Testing (Manual Configuration)​

After all endpoint settings have been configured, Nexla can send a test payload to the Apify API to ensure that the destination is configured correctly.

  1. To send a test payload, select the Test button at the top of the Sample Payload panel, and click on a listed sample payload to expand it.

  2. If any modifications to the sample payload are needed, make the necessary changes directly within the sample window.

  3. Click the Send Test Data button at the top of a sample payload to send the test payload to the Apify API using the current settings.

Save & Activate the Destination​

  • Once all endpoint settings have been configured, click the Done button in the upper right corner of the screen to save and create the destination. To send the data to the configured Apify endpoint, open the destination resource menu, and select Activate.

    The Nexset data will not be sent to the Apify endpoint until the destination is activated. Destinations can be activated immediately or at a later time, providing full control over data movement.