DataSet / Scalyr

DataSet (formerly Scalyr) is a log management and observability platform that provides real-time log analysis, monitoring, and alerting capabilities to help DevOps teams quickly identify and resolve issues in their applications and infrastructure.

Power end-to-end data operations for your DataSet / Scalyr API with Nexla. Our bi-directional DataSet / Scalyr connector is purpose-built for DataSet / Scalyr, making it simple to ingest data, sync it across systems, and deliver it anywhere — all with no coding required. Nexla turns API-sourced data into ready-to-use, reusable data products and makes it easy to send data to DataSet / Scalyr or any other destination. With comprehensive monitoring, lineage tracking, and access controls, Nexla keeps your DataSet / Scalyr workflows fast, secure, and fully governed.

Features

Type: API

SourceDestination

• Seamless API Integration: Connect to any endpoint as source or destination without coding, with automatic data product creation
• Visual Composition & Chaining: Build complex integrations using visual templates, chain API calls, and compose workflows with data validation and filtering
• API Proxy: Expose curated slices of your data securely with a secure and customizable API proxy that validates and transforms data on the fly
• Request optimization with intelligent batching, retry, and caching to minimize API calls and costs

Prerequisites

Before creating a DataSet / Scalyr credential, you need to obtain your API keys from your DataSet / Scalyr account. DataSet / Scalyr uses API key authentication for all API requests, with different API keys for different access levels. The API key is sent in the Authorization header with the Bearer prefix.

To obtain your DataSet / Scalyr API keys, follow these steps:

1. Sign in to your DataSet / Scalyr account using your administrator credentials.

2. Navigate to your account settings or API management section in the DataSet / Scalyr interface.

3. Look for the API Keys or API section in your account settings, or navigate to Settings > API Keys or Account > API Keys.

4. If you don't have API keys yet, look for the option to generate or view your API keys.

5. Click Generate API Key or Create API Key to create a new API key.

6. Configure your API key settings:

   • Enter a name for the API key (e.g., "Nexla Integration")
   • Select the access level for the API key:
     • Configuration Write Access Key: Provides access to all methods available to Log Read Access and Configuration Read Access, plus methods to write configuration files
     • Log Write Access Key: Provides access to all methods available to Log Write Access
   • Review and select the permissions or scopes for the key (if applicable)

7. Click Generate or Create to create the API key.

8. Copy the API key immediately after it's generated, as it may not be accessible again after you navigate away from the page.

9. Store the API key securely, as you will need it to configure your Nexla credential. The API key is sensitive information and should be kept confidential.

The API key is sent in the Authorization: Bearer {key} header for all API requests to the DataSet / Scalyr API. Different API keys provide different levels of access to DataSet / Scalyr resources. The Configuration Write Access Key is necessary for accessing all methods available to Log Read Access and Configuration Read Access, plus methods to write configuration files. The Log Write Access Key is necessary for accessing all methods available to Log Write Access. If your API key is compromised, you should immediately revoke it in your DataSet / Scalyr account settings and generate a new one. For detailed information about obtaining API keys, API authentication, and available endpoints, refer to the DataSet / Scalyr API documentation.

Authenticate

Credentials required

Configuration Access Key

This key is necessary for accessing all methods available to Log Read Access and Configuration Read/Write Access,

Field	Required	Secret	Description
Configuration Write Access Keys	Yes	Yes	Provide your Configuration Write Access Keys. This key is necessary for accessing all methods available to Log Read Access and Configuration Read Access, plus methods to write configuration files. See https://app.scalyr.com/help/api-keys for instructions.

Log Write Access Key

This key is necessary for accessing all methods available to Log Read Access and Configuration Read/Write Access,

Field	Required	Secret	Description
Log Write Access Keys	Yes	Yes	Provide your Log Write Access Keys. This key is necessary for accessing all methods available to Log Write Access. See https://app.scalyr.com/help/api-keys for instructions.

Create a credential in Nexla

1. After selecting the data source/destination type, click the Add Credential tile to open the Add New Credential overlay.

New Credential Overlay – DataSet / Scalyr

2. Enter a name for the credential in the Credential Name field and a short, meaningful description in the Credential Description field.

3. DataSet / Scalyr uses API key authentication for all API requests. Different API keys provide different levels of access to DataSet / Scalyr resources. The API key is sent in the Authorization header with the Bearer prefix to authenticate API requests to the DataSet / Scalyr API. Select the tab below that corresponds to the access level you need, and follow the associated step to configure the field.

DataSet / Scalyr Authentication Methods

Configuration Access Key

This key is necessary for accessing all methods available to Log Read Access and Configuration Read Access, plus methods to write configuration files. Use this method for data sources and destinations that require configuration access.

1. Enter your DataSet / Scalyr Configuration Write Access Key in the **Configuration Write Access Keys** field. This is the API key you obtained from your DataSet / Scalyr account settings (Settings > API Keys or Account > API Keys). The Configuration Write Access Key is sent in the `Authorization: Bearer {key}` header for all API requests to the DataSet / Scalyr API. The API key is sensitive information and must be kept confidential.

Log Write Access Key

This key is necessary for accessing all methods available to Log Write Access. Use this method for destinations that require log write access to add events to your DataSet / Scalyr account.

1. Enter your DataSet / Scalyr Log Write Access Key in the **Log Write Access Keys** field. This is the API key you obtained from your DataSet / Scalyr account settings (Settings > API Keys or Account > API Keys). The Log Write Access Key is sent in the `Authorization: Bearer {key}` header for all API requests to the DataSet / Scalyr API. The API key is sensitive information and must be kept confidential.

Your DataSet / Scalyr API keys can be found in your DataSet / Scalyr account settings under Settings > API Keys or Account > API Keys. The Configuration Write Access Key is necessary for accessing all methods available to Log Read Access and Configuration Read Access, plus methods to write configuration files. The Log Write Access Key is necessary for accessing all methods available to Log Write Access.

If your API key is compromised, you should immediately revoke it in your DataSet / Scalyr account settings and generate a new one. The API key provides access to your DataSet / Scalyr account data and should be treated as sensitive information. Keep your API key secure and do not share it publicly.

For detailed information about obtaining API keys, API authentication, and available endpoints, see the DataSet / Scalyr API documentation.

4. Click the Save button at the bottom of the overlay. The newly added credential will now appear in a tile on the Authenticate screen during data source/destination creation and can be selected for use with a new data source or destination.

Use as a data source

To create a new data flow, navigate to the Integrate section, and click the New Data Flow button. Select the DataSet / Scalyr connector tile, then select the credential that will be used to connect to the DataSet / Scalyr account, and click Next; or, create a new DataSet / Scalyr credential for use in this flow.

Endpoint templates

Nexla provides pre-built templates that can be used to rapidly configure data sources to ingest data from common DataSet / Scalyr endpoints. Select the endpoint from which this source will fetch data from the Endpoint pulldown menu. Available endpoint templates are listed in the expandable boxes below; click on an endpoint to see more information about it and how to configure your data source for this endpoint.

Execute a Query

This endpoint template retrieves events (log records) from your DataSet / Scalyr account. You can retrieve all events in a specified time range, or only events matching specific criteria. Use this template when you need to query log data from your DataSet / Scalyr account.

• Enter the query payload in the Query Payload (JSON Object) field. This should be a valid JSON object that includes a key queryType with value log. The query payload determines which events will be retrieved. You can include additional query parameters such as startTime, endTime, and filter criteria. Example: `{"queryType": "log", "startTime": "{now-1}"}`
• Select the date format from the Date Format pulldown menu. Available options include Unix Epoch Time (Seconds) (default), Unix Epoch Time (Milliseconds), ISO-8601 date format, yyyy-MM-dd, and other date formats. The date format determines how dates will be formatted in the query payload and response.

This endpoint returns events (log records) from your DataSet / Scalyr account based on the query payload. The endpoint uses continuation token-based pagination with continuationToken to handle large datasets efficiently. Nexla will automatically fetch subsequent pages of data by following the continuation token returned in the API response.

The query payload must be a valid JSON object and must include a key queryType with value log. You can include additional query parameters such as startTime, endTime, and filter criteria to customize the data retrieval. For detailed information about query syntax, API response structures, pagination, and available query parameters, see the DataSet / Scalyr API documentation.

List Users

This endpoint template lists all users in your DataSet / Scalyr account. Use this template when you need to retrieve information about users, including user IDs, names, and other user metadata.

• This endpoint automatically retrieves all users from your DataSet / Scalyr account. No additional configuration is required beyond selecting this endpoint template.

This endpoint returns all users in your DataSet / Scalyr account, including user IDs, names, and other user metadata. Use this endpoint to discover which users are available in your DataSet / Scalyr account.

For detailed information about user management, API response structures, and available user data, see the DataSet / Scalyr API documentation.

Once the selected endpoint template has been configured, click the Test button to the right of the endpoint selection menu to retrieve a sample of the data that will be fetched. Sample data will be displayed in the Endpoint Test Result panel on the right, allowing you to verify that the source is configured correctly before saving.

Manual configuration

DataSet / Scalyr data sources can also be manually configured to ingest data from any valid DataSet / Scalyr API endpoint, including endpoints not covered by the pre-built templates, chained API calls, or custom request parameters. Select the Advanced tab at the top of the configuration screen, and follow the instructions in Connect to Any API to configure the API method, endpoint URL, date/time and lookup macros, path to data, metadata, and request headers.

DataSet / Scalyr API typically uses the POST method for query endpoints (e.g., https://app.scalyr.com/api/query) and GET for other endpoints (e.g., https://app.scalyr.com/api/listUsers). For the Response Data Path, use $.matches[*] to extract matches from query endpoints, $.users[*] to extract users from user endpoints, or $ for single-record endpoints. Query endpoints use continuation token-based pagination via continuationToken, with the token JSONPath at $.continuationToken and the token parameter included in the request body.

Once all of the relevant settings have been configured, click the Create button in the upper right corner of the screen to save and create the new DataSet / Scalyr data source. Nexla will now begin ingesting data from the configured endpoint and will organize any data that it finds into one or more Nexsets.

Use as a destination

Click the + icon on the Nexset that will be sent to the DataSet / Scalyr destination, and select the Send to Destination option from the menu. Select the DataSet / Scalyr connector from the list of available destination connectors, then select the credential that will be used to connect to your DataSet / Scalyr account, and click Next; or, create a new DataSet / Scalyr credential for use in this flow.

Endpoint templates

Nexla provides pre-built templates that can be used to rapidly configure destinations to send data to common DataSet / Scalyr endpoints. Select the endpoint to which data will be sent from the Endpoint pulldown menu. Then, click on the template in the list below to expand it, and follow the instructions to configure additional endpoint settings.

Add Group

This endpoint template adds a group to your DataSet / Scalyr account using records from a Nexset. Use this template when you need to create or update groups in your DataSet / Scalyr account. Please see the Scalyr API documentation for required record structure.

• This endpoint sends data as JSON in the request body to add groups to your DataSet / Scalyr account. Each record from your Nexset will be sent as a separate API request to add a group in DataSet / Scalyr. The request body should contain the group data in the format required by the DataSet / Scalyr API.

This endpoint sends data as JSON in the request body to add groups to your DataSet / Scalyr account. Each record from your Nexset will be sent as a separate API request to add a group in DataSet / Scalyr. The request body should contain the group data in the format required by the DataSet / Scalyr API.

The group structure must match the DataSet / Scalyr API's expected format. For detailed information about group creation, request body formats, available group properties, and group management, see the DataSet / Scalyr API documentation.

Add Events

This endpoint template adds events to your DataSet / Scalyr account using records from a Nexset. Use this template when you need to send log events to your DataSet / Scalyr account. Please note that this endpoint requires a credential with Log Write Access Key.

• Enter the batch size in the Batch Size field. This should be the maximum number of events to add per batch (default: 1000). The batch size determines how many events will be included in each API request.
• Select the content compression from the Content Compression pulldown menu. Available options include None (send uncompressed data) and Compress with GZIP (enable GZIP compression). Compression reduces bandwidth usage and is recommended for large data volumes.

This endpoint sends data as JSON in the request body to add events to your DataSet / Scalyr account. The endpoint uses batch mode to efficiently send multiple events in a single request. Each record from your Nexset will be included in the batch, and events will be added in batches of up to the specified batch size per request.

This endpoint requires a credential with Log Write Access Key. The event structure must match the DataSet / Scalyr API's expected format. Each event should include the necessary event data. For detailed information about event creation, request body formats, batch processing, and available event properties, see the DataSet / Scalyr API documentation.

Manual configuration

DataSet / Scalyr destinations can also be manually configured to send data to any valid DataSet / Scalyr API endpoint, including endpoints not covered by the pre-built templates or when custom API configurations are required. Select the Advanced tab at the top of the configuration screen, and follow the instructions in Connect to Any API to configure the API method, data format, endpoint URL, request headers, attribute exclusions, record batching, and response webhooks.

DataSet / Scalyr API typically uses the POST method for creating resources and adding events, and expects JSON format for all requests (e.g., https://app.scalyr.com/api/addGroup, https://app.scalyr.com/api/addEvents). The request body format is typically {message.json} to send the entire Nexset as JSON, or a custom JSON structure with specific field mappings; batch operations may require an events array containing event data. The Add Events endpoint requires a credential with a Log Write Access Key.

Save & activate

Once all endpoint settings have been configured, click the Done button in the upper right corner of the screen to save and create the destination. To send the data to the configured DataSet / Scalyr endpoint, open the destination resource menu, and select Activate.

The Nexset data will not be sent to the DataSet / Scalyr endpoint until the destination is activated. Destinations can be activated immediately or at a later time, providing full control over data movement.