Skip to main content

Datadog Data Source

Follow the instructions below to create a new data flow that ingests data from a Datadog source in Nexla.
datadog_api.png

Datadog

Create a New Data Flow​

  1. To create a new data flow, navigate to the Integrate section, and click the New Data Flow button. Then, select the desired flow type from the list, and click the Create button.

  2. Select the Datadog connector tile from the list of available connectors. Then, select the credential that will be used to connect to the Datadog instance, and click Next; or, create a new Datadog credential for use in this flow.

  3. In Nexla, Datadog data sources can be created using pre-built endpoint templates, which expedite source setup for common Datadog endpoints. Each template is designed specifically for the corresponding Datadog endpoint, making source configuration easy and efficient.
    • To configure this source using a template, follow the instructions in Configure Using a Template.

    Datadog sources can also be configured manually, allowing you to ingest data from Datadog endpoints not included in the pre-built templates or apply further customizations to exactly suit your needs.
    • To configure this source manually, follow the instructions in Configure Manually.

Configure Using a Template​

Nexla provides pre-built templates that can be used to rapidly configure data sources to ingest data from common Datadog endpoints. Each template is designed specifically for the corresponding Datadog endpoint, making data source setup easy and efficient.

Endpoint Settings​

  • Select the endpoint from which this source will fetch data from the Endpoint pulldown menu. Available endpoint templates are listed in the expandable boxes below. Click on an endpoint to see more information about it and how to configure your data source for this endpoint.

    Get your Managed Orgs

    This endpoint fetches a list of your managed organizations. Use this endpoint when you need to access organization information, managed organization data, or organization lists from your Datadog account.

    • This endpoint automatically retrieves all managed organizations from your Datadog account. No additional configuration is required beyond selecting this endpoint template.
    • The endpoint uses GET requests to https://api.datadoghq.com/api/v1/org. The endpoint URL is automatically constructed based on the Datadog API base URL.
    • The endpoint does not use pagination and returns all managed organizations in a single request.
    • The endpoint will return all managed organizations. The response data is extracted from the orgs array in the API response ($.orgs[*]), with each organization record processed individually.

    This endpoint returns information about all managed organizations in your Datadog account. The endpoint uses a static URL (iteration.type: static.url) and does not require pagination. The response data path is $.orgs[*], which extracts all items from the orgs array in the API response. For detailed information about listing managed organizations, see the Datadog API documentation.

    Query Event Stream

    This endpoint fetches events that are within a start and end time range. Use this endpoint when you need to access event data, monitoring events, or event streams from your Datadog account for a specific time period. Use the advanced tab for fetching events with other query criteria.

    • Enter the start time for events query in the Start Time (Posix Epoch in seconds) field. This must be a valid POSIX time in seconds (Unix timestamp). You can use Nexla macros like {now} for the current time or {now-1} for 24 hours ago. The default value is {now-1} (24 hours before the time of request) if not specified.

    • Enter the end time for events query in the End Time (Posix Epoch in seconds) field. This must be a valid POSIX time in seconds (Unix timestamp). You can use Nexla macros like {now} for the current time. The default value is {now} (time of query execution) if not specified.

    • The endpoint uses GET requests to https://api.datadoghq.com/api/v1/events?start={'{start}'}&end={'{end}'}&page={'{page}'} where {start} is the Start Time, {end} is the End Time, and {page} is the pagination page number. The endpoint URL is automatically constructed based on the Datadog API base URL and the time range and pagination parameters.
    • The endpoint uses incrementing pagination, automatically fetching additional pages as needed. Pagination starts from page 1 and increments by 1 for each subsequent page. Nexla will continue fetching pages until all events in the specified time range are retrieved.
    • The endpoint will return all events within the specified start and end time range. The response data is extracted from the events array in the API response ($.events[*]), with each event record processed individually.

    Time values must be valid POSIX time in seconds (Unix timestamps). You can use Nexla macros like {now} for the current time or {now-1} for 24 hours ago. The endpoint uses incrementing pagination (iteration.type: paging.incrementing) starting from page 1. The response data path is $.events[*], which extracts all items from the events array in the API response. Use the advanced tab for fetching events with other query criteria beyond the time range. For detailed information about querying the event stream, see the Datadog API documentation.

Endpoint Testing​

Once the selected endpoint template has been configured, Nexla can retrieve a sample of the data that will be fetched according to the current settings. This allows users to verify that the source is configured correctly before saving.

  • To test the current endpoint configuration, click the Test button to the right of the endpoint selection menu. Sample data will be fetched & displayed in the Endpoint Test Result panel on the right.

  • If the sample data is not as expected, review the selected endpoint and associated settings, and make any necessary adjustments. Then, click the Test button again, and check the sample data to ensure that the correct information is displayed.

Configure Manually​

Datadog data sources can be manually configured to ingest data from any valid Datadog API endpoint. Manual configuration provides maximum flexibility for accessing endpoints not covered by pre-built templates or when you need custom API configurations.

With manual configuration, you can also create more complex Datadog sources, such as sources that use chained API calls to fetch data from multiple endpoints or sources that require custom authentication headers or request parameters.

API Method​

  1. To manually configure this source, select the Advanced tab at the top of the configuration screen.

  2. Select the API method that will be used for calls to the Datadog API from the Method pulldown menu. The most common methods are:

    • GET: For retrieving data from the API
    • POST: For sending data to the API or triggering actions
    • PUT: For updating existing data
    • PATCH: For partial updates to existing data
    • DELETE: For removing data

API Endpoint URL​

  1. Enter the URL of the Datadog API endpoint from which this source will fetch data in the Set API URL field. This should be the complete URL including the protocol (https://) and any required path parameters. Datadog API endpoints typically follow the pattern https://api.datadoghq.com/api/v1/{'{endpoint_path}'} or https://api.datadoghq.com/api/v2/{'{endpoint_path}'}.

Ensure the API endpoint URL is correct and accessible with your current credentials. You can test the endpoint using the Test button after configuring the URL. The endpoint requires custom headers for authentication (DD-API-KEY and DD-APPLICATION-KEY), which are handled automatically by your credential configuration. For detailed information about Datadog API endpoints and available APIs, see the Datadog API documentation.

Path to Data​

  1. Enter the JSON path that identifies the location of the relevant data within the API response in the Path to Data field. JSON paths use dot notation to navigate through nested JSON structures.

    • For example, if your API response has the structure {"events": [...]}, you would enter $.events[*] to extract all items from the events array.
    • Use $[*] to extract all items from a root-level array.
    • Use $ to extract the entire root-level object.

JSON paths are case-sensitive and must match the exact structure of your API response. Datadog API responses may use different structures like events arrays, orgs arrays, or root-level objects depending on the endpoint. Use the Test button to verify that your JSON path correctly extracts the desired data from the API response.

Save the Data Source​

  1. Once all configuration steps have been completed, click the Save button to save your data source configuration.

  2. The data source will now be available in your data flow and will begin ingesting data according to the configured schedule and endpoint settings.