Dremio

Dremio is an agentic lakehouse platform delivering high-performance SQL analytics directly on open data lake storage. Built on Apache Arrow and Apache Iceberg, it provides a self-service semantic layer, query acceleration through Reflections (materialized views), and zero-ETL federation — letting data teams and AI agents query, curate, and share data at scale without moving it into a warehouse.

Power end-to-end data operations for your Dremio API with Nexla. Our bi-directional Dremio connector is purpose-built for Dremio, making it simple to ingest data, sync it across systems, and deliver it anywhere — all with no coding required. Nexla turns API-sourced data into ready-to-use, reusable data products and makes it easy to send data to Dremio or any other destination. With comprehensive monitoring, lineage tracking, and access controls, Nexla keeps your Dremio workflows fast, secure, and fully governed.

Features

Type: API

SourceDestination

• Seamless API Integration: Connect to any endpoint as source or destination without coding, with automatic data product creation
• Visual Composition & Chaining: Build complex integrations using visual templates, chain API calls, and compose workflows with data validation and filtering
• API Proxy: Expose curated slices of your data securely with a secure and customizable API proxy that validates and transforms data on the fly
• Request optimization with intelligent batching, retry, and caching to minimize API calls and costs

Prerequisites

To connect Nexla to Dremio, you need a Personal Access Token (PAT) generated from your Dremio account, as well as the base URL of your Dremio instance.

About Dremio Personal Access Tokens

Personal Access Tokens are randomly-generated tokens associated with a specific Dremio user account. They are used in place of a password to authenticate with the Dremio REST API and other programmatic interfaces. PATs are valid for up to 180 days (configurable at token creation time) and are the recommended authentication method for REST API integrations such as Nexla.

When Nexla connects to Dremio using a PAT, all API calls are made as the user who generated the token. Ensure the Dremio user account has the necessary permissions to access the catalog entities, datasets, spaces, and views that Nexla will read from or write to.

Obtain Your Dremio Base URL

The base URL is the root URL of your Dremio deployment. Its format depends on how your instance is hosted:

• Dremio Cloud: https://api.dremio.cloud

• Self-managed Dremio Software: The URL is typically the hostname and port where your Dremio instance is accessible, for example https://dremio.example.com or https://dremio.example.com:9047.

Contact your Dremio administrator if you are unsure of the correct base URL for your deployment.

Generate a Dremio Personal Access Token

1. Sign in to your Dremio instance using your user credentials.

2. Click your user initials or avatar in the bottom-left corner of the Dremio console navigation bar.

3. Select Account Settings from the menu.

4. In the Account Settings sidebar, select Personal Access Tokens.

5. On the Personal Access Tokens page, click the Generate Token button in the upper-right corner of the screen.

6. In the Generate Token dialog, enter a descriptive label for the token in the Label field. Choose a label that identifies the integration purpose, for example Nexla Integration.

7. Enter the number of days the token should remain valid in the Lifetime field. The default is 30 days and the maximum is 180 days.

8. Click Generate Token to create the PAT.

Important

Copy the generated token immediately and store it in a secure location. Dremio displays the token only once—it cannot be retrieved after you close the dialog. If the token is lost, you must generate a new one.

Additional details about managing Personal Access Tokens are available in the Dremio Authentication documentation.

Authenticate

Credentials required

Authenticate to Dremio using a Personal Access Token (PAT), which is transmitted as a Bearer token in the Authorization request header for all API calls.

Field	Required	Secret	Description
Base URL	Yes	No	Dremio instance base URL (e.g., https://dremio.example.com or https://api.dremio.cloud)
Personal Access Token	Yes	Yes	Dremio Personal Access Token (PAT) generated from Account Settings > Personal Access Tokens in the Dremio Console.

Create a credential in Nexla

1. After selecting the data source/destination type, click the Add Credential tile to open the Add New Credential overlay.

2. Enter a name for the credential in the Credential Name field and a short, meaningful description in the Credential Description field.

3. Enter the base URL of your Dremio instance in the Base URL field. This should be the root URL without a trailing slash (e.g., https://dremio.example.com or https://api.dremio.cloud).

   The base URL is used by Nexla to construct all Dremio API endpoint URLs, so it is important that it matches your deployment exactly. For Dremio Cloud, use https://api.dremio.cloud. For self-managed deployments, use the fully qualified hostname and port configured for your Dremio instance.

4. Enter your Dremio Personal Access Token in the Personal Access Token field. This is the PAT generated in the Dremio Console under Account Settings > Personal Access Tokens, as described in the Prerequisites section above.

5. Click the Save button at the bottom of the overlay. The newly added credential will now appear in a tile on the Authenticate screen during data source/destination creation.

Use as a data source

To create a new data flow, navigate to the Integrate section, and click the New Data Flow button. Select the Dremio connector tile, then select the credential that will be used to connect to the Dremio instance, and click Next; or, create a new Dremio credential for use in this flow.

Endpoint templates

Nexla provides pre-built templates that can be used to rapidly configure data sources to ingest data from common Dremio API endpoints. Select the endpoint from which this source will fetch data from the Endpoint pulldown menu. Available endpoint templates are listed in the expandable boxes below.

List Catalog Root

Returns the list of top-level catalog containers in the Dremio instance, including all registered sources, spaces, and the home space. Use this endpoint to discover what data assets are available at the root of the Dremio catalog—useful for building inventory reports, auditing registered data sources, or navigating the catalog structure programmatically.

• No additional configuration parameters are required for this endpoint. Select it from the Endpoint menu and proceed directly to endpoint testing.
• Each record returned by this endpoint represents a top-level catalog container and includes its entity type (SOURCE, SPACE, HOME), display name, and opaque catalog ID. The catalog ID can be used in subsequent calls to the Get Catalog Entity by ID or Get Catalog Entity Lineage endpoints.

Additional reference information is available in the Dremio Catalog API documentation.

Get Catalog Entity by ID

Retrieves a specific catalog entity—such as a source, space, folder, physical dataset, or virtual dataset (view)—using its opaque catalog ID. Use this endpoint when you have an entity's ID and need to fetch its full metadata, including schema information, access controls, and configuration details.

• Enter the opaque catalog ID of the entity you want to retrieve in the Catalog Entity ID field. This ID is a system-generated string identifier unique to each catalog object in Dremio.

  • Catalog entity IDs can be found in the response from the List Catalog Root or Get Catalog Entity by Path endpoints, or by inspecting the URL when viewing an entity in the Dremio console.
  • Example ID format: 3a4b5c6d-7e8f-9012-abcd-ef1234567890

Additional reference information is available in the Dremio Catalog API documentation.

Get Catalog Entity by Path

Looks up a catalog entity using its human-readable dot-separated path rather than its opaque ID. This is useful when you know the logical path to a dataset or space in the Dremio catalog (e.g., the path as it appears in the Dremio SQL editor) but do not have the entity's system-generated ID.

• Enter the dot-separated catalog path of the entity in the Catalog Path field. The path mirrors the hierarchy you see in the Dremio UI.

  • For a top-level source called MyS3Source, enter: MyS3Source
  • For a folder called SalesData inside a space called Analytics, enter: Analytics.SalesData
  • For a virtual dataset (view) called MonthlyRevenue inside that folder, enter: Analytics.SalesData.MonthlyRevenue

Additional reference information is available in the Dremio Catalog API documentation.

Get Catalog Entity Lineage

Retrieves the upstream and downstream data lineage for a specific dataset or virtual dataset (view) in the Dremio catalog. Use this endpoint to understand data provenance, trace dependencies between views, or audit which source datasets feed into downstream analytics views.

• Enter the opaque catalog ID of the dataset whose lineage you want to retrieve in the Catalog Entity ID field.

  • This must be the ID of a physical dataset or virtual dataset (view). Lineage is not available for spaces, folders, or sources directly.
  • You can obtain the catalog ID by first querying the List Catalog Root or Get Catalog Entity by Path endpoint.
• The response includes lists of upstream parents (datasets that feed into this entity) and downstream children (datasets or views that depend on this entity).

Additional reference information is available in the Dremio Catalog API documentation.

List Reflections

Returns all Reflections configured in the Dremio instance. Reflections are Dremio's query acceleration mechanism—they are pre-computed, Apache Arrow-formatted materializations of datasets or aggregations that dramatically speed up query execution. Use this endpoint to inventory acceleration configurations, monitor Reflection health, or audit which datasets have active materializations.

• No additional configuration parameters are required for this endpoint. Select it from the Endpoint menu and proceed directly to endpoint testing.
• The response includes all raw Reflections (full dataset materializations) and aggregation Reflections (pre-computed aggregates) defined across the Dremio instance. Each Reflection record includes its type, status, associated dataset path, and configuration details such as display fields, partition fields, and sort fields.

Additional reference information is available in the Dremio Reflections API documentation.

Get Job Status

Polls the current status of a previously submitted SQL job. Dremio processes SQL queries asynchronously—when a query is submitted via the SQL API, it returns a job ID that can be used to check the job's progress. Use this endpoint to monitor whether a job is RUNNING, COMPLETED, FAILED, or CANCELED before attempting to retrieve its results.

• Enter the job ID of the SQL job you want to check in the Job ID field. Job IDs are returned by the Submit SQL Query destination endpoint when a query is submitted to Dremio.
• The response includes the job state, submission time, start time, end time (if completed), and error messages (if the job failed).

Additional reference information is available in the Dremio Job API documentation.

Get Job Results

Retrieves the paginated result rows for a completed SQL job. After a job submitted via the Submit SQL Query endpoint reaches COMPLETED status, use this endpoint to fetch the actual query result data. Nexla automatically handles pagination using Dremio's offset-based paging (100 rows per page by default) to retrieve all available result rows.

• Enter the job ID of the completed SQL job in the Job ID field. This must be the ID of a job that has reached COMPLETED status; jobs that are still RUNNING or have FAILED will not return result rows.
• Nexla will automatically paginate through the results using the offset and limit query parameters, fetching pages of 100 rows until all results have been retrieved.
• The result rows are returned in the rows array of each paginated response. Each row is a JSON object with field names matching the columns of the executed SQL query.

Verify that your SQL job has COMPLETED before selecting this endpoint. Use the Get Job Status endpoint to check job state. Additional reference information is available in the Dremio Job API documentation.

Once the selected endpoint template has been configured, click the Test button to the right of the endpoint selection menu to retrieve a sample of the data that will be fetched. Sample data will be displayed in the Endpoint Test Result panel on the right, allowing you to verify that the source is configured correctly before saving.

Manual configuration

Dremio data sources can also be manually configured to ingest data from any valid Dremio API endpoint, including endpoints not covered by the pre-built templates, chained API calls, or custom request parameters. Select the Advanced tab at the top of the configuration screen, and follow the instructions in Connect to Any API to configure the API method, endpoint URL, date/time and lookup macros, path to data, metadata, and request headers.

All Dremio REST API v3 endpoints follow the pattern {'{base_url}'}/api/v3/{'{resource_path}'}—for example, catalog root (/api/v3/catalog), catalog entity by ID (/api/v3/catalog/{'{id}'}), catalog entity by path (/api/v3/catalog/by-path/{'{path}'}), Reflections list (/api/v3/reflection), job status (/api/v3/job/{'{job-id}'}), and job results (/api/v3/job/{'{job-id}'}/results). Set Path to Data to $.data[*] for the Catalog Root endpoint or $.rows[*] for Job Results; most other endpoints return the relevant object directly at $. You do not need to add the Authorization header manually—it is applied automatically from your saved Dremio credential.

Once all of the relevant settings have been configured, click the Create button in the upper right corner of the screen to save and create the new Dremio data source. Nexla will now begin ingesting data from the configured endpoint and will organize any data that it finds into one or more Nexsets.

Use as a destination

Click the + icon on the Nexset that will be sent to the Dremio destination, and select the Send to Destination option from the menu. Select the Dremio connector from the list of available destination connectors, then select the credential that will be used to connect to the Dremio instance, and click Next; or, create a new Dremio credential for use in this flow.

Endpoint templates

Nexla provides pre-built templates that can be used to rapidly configure destinations to send data to common Dremio API endpoints. Select the endpoint to which data will be sent from the Endpoint pulldown menu. Then, click on the template in the list below to expand it, and follow the instructions to configure additional endpoint settings.

Create Catalog Entity

Creates a new catalog entity in Dremio, such as a space, folder, virtual dataset (view), or a new registered source. Use this endpoint when you need to programmatically provision new catalog containers or virtual datasets from Nexla as part of an automated data pipeline. For example, you can use this endpoint to create a new virtual dataset (view) in Dremio that references data managed by Nexla.

• Prepare the Nexset records sent to this destination to conform to the Dremio Catalog entity schema. The body of each API call will be the JSON-serialized record, so each record in the Nexset should represent a complete, valid Dremio catalog entity object. The required fields vary by entity type:

  • Space: Requires entityType set to space and a name field.
  • Folder: Requires entityType set to folder and a path array representing its location in the catalog.
  • Virtual Dataset (View): Requires entityType set to dataset, type set to VIRTUAL_DATASET, a path array, and a virtualDataset object containing the SQL definition.
  • Source: Requires entityType set to source, a name, and a config object specific to the source type (e.g., S3, JDBC).
• No additional Nexla-side configuration parameters are required beyond selecting this endpoint template. Nexla will POST each record as a JSON body to the Dremio Catalog API at /api/v3/catalog.

Additional reference information, including detailed entity schema definitions, is available in the Dremio Catalog API documentation.

Update Catalog Entity

Updates an existing catalog entity in Dremio by its opaque catalog ID. Use this endpoint to programmatically update source metadata, modify virtual dataset SQL definitions, or refresh configuration details of existing catalog objects. This is useful for workflows that need to keep Dremio catalog entries synchronized with upstream changes managed in Nexla.

• Enter the opaque catalog ID of the entity to update in the Catalog Entity ID field. This ID is a system-generated string that uniquely identifies the catalog object in Dremio. It can be obtained from the Get Catalog Entity by ID or Get Catalog Entity by Path source endpoints.

• The Nexset records sent to this destination should each contain the complete updated representation of the catalog entity as a JSON object. The Dremio Catalog Update API uses a full-replace (PUT) model, so all required fields must be present in the body:

  • Include the entity's id, entityType, tag (optimistic concurrency version tag), and all other fields that should be preserved on the entity.
  • Only the fields you include in the body will be retained. Omitting a field may result in it being cleared on the entity.

The tag field (an optimistic concurrency control token) is required for update operations. Retrieve the current tag value from the entity's existing catalog record before updating. Additional reference information is available in the Dremio Catalog API documentation.

Delete Catalog Entity

Deletes a catalog entity from Dremio by its opaque catalog ID. Supported entity types include spaces, folders, virtual datasets (views), physical datasets, and registered sources. Use this endpoint to programmatically remove catalog objects as part of automated data lifecycle management or cleanup pipelines.

• Enter the opaque catalog ID of the entity to delete in the Catalog Entity ID field. This ID can be obtained from the Get Catalog Entity by ID or Get Catalog Entity by Path source endpoints.
• The delete operation sends an empty request body to the Dremio Catalog API. Ensure the Catalog Entity ID is correct before activating this destination, as deleted catalog entities cannot be automatically recovered.

Important

Deleting a space, folder, or source will also delete all catalog entities contained within it. Verify the correct entity ID is configured before activating this destination in a production pipeline.

Additional reference information is available in the Dremio Catalog API documentation.

Submit SQL Query

Submits a SQL query to Dremio for asynchronous execution and returns a job ID that can be used to poll for results. Use this endpoint to execute DDL or DML statements, trigger transformations in Dremio views, or run ad-hoc queries as part of a Nexla data pipeline. This is particularly useful for use cases such as refreshing a Dremio Reflection after upstream data has been updated in Nexla.

• Each Nexset record sent to this endpoint should be a JSON object with the following fields:

  • sql (required): The SQL statement to execute, for example SELECT * FROM mySpace.myView LIMIT 100 or ALTER TABLE mySource.myTable REFRESH METADATA.
  • context (optional): An array of strings representing the default catalog path context for the query, for example ["mySpace", "myFolder"]. Setting a context allows you to write unqualified table names in your SQL.
  • references (optional): A map of source names to version references for querying Iceberg table snapshots at a specific version.
• The API response includes a id field containing the job ID. Use the Get Job Status and Get Job Results source endpoints to monitor execution and retrieve results after the job completes.

Additional reference information, including query context and versioning options, is available in the Dremio SQL API documentation.

Create Reflection

Creates a new Reflection in Dremio to accelerate queries on a dataset. Reflections are Dremio's query acceleration mechanism—pre-computed, Apache Arrow-formatted materializations of datasets or aggregations that dramatically speed up query execution. Use this endpoint to programmatically provision new raw Reflections (full dataset materializations) or aggregation Reflections (pre-computed aggregates) as part of a data pipeline.

• Each Nexset record sent to this endpoint should be a JSON object conforming to the Dremio Reflection schema. Key fields include:

  • type (required): Either RAW (for full dataset materialization) or AGGREGATION (for pre-computed aggregates).
  • datasetId (required): The opaque catalog ID of the dataset this Reflection will accelerate.
  • name (required): A descriptive name for the Reflection.
  • displayFields (for RAW Reflections): An array of field name objects specifying which columns to include in the materialization.
  • dimensionFields, measureFields (for AGGREGATION Reflections): Arrays specifying grouping dimensions and aggregate measures.
  • partitionFields, sortFields (optional): Arrays for controlling partitioning and sort order of the materialization.
• After a Reflection is created, Dremio will begin materializing it in the background. The Reflection will not be used to accelerate queries until it has been fully materialized.

Additional reference information, including the full Reflection schema, is available in the Dremio Reflections API documentation.

Cancel Job

Cancels a running SQL job by its job ID. Use this endpoint to programmatically terminate long-running queries in Dremio, for example as part of a pipeline that detects stalled or timed-out jobs and cleans them up.

• Enter the ID of the SQL job to cancel in the Job ID field. Job IDs are returned by the Submit SQL Query endpoint when a query is submitted to Dremio, or can be retrieved from the Dremio console Job History view.
• The cancel operation sends an empty request body to the Dremio Job Cancel API. Only jobs that are currently in a RUNNING or QUEUED state can be canceled; jobs that have already COMPLETED, FAILED, or been CANCELED will not be affected.

Additional reference information is available in the Dremio Job API documentation.

Manual configuration

Dremio destinations can also be manually configured to send data to any valid Dremio API endpoint. Select the Advanced tab at the top of the configuration screen, and follow the instructions in Connect to Any API to configure the API method, data format, endpoint URL, request headers, attribute exclusions, record batching, and response webhooks.

All Dremio REST API v3 write endpoints follow the pattern {'{base_url}'}/api/v3/{'{resource_path}'} and accept JSON payloads—for example, create catalog entity (POST /api/v3/catalog), update catalog entity (PUT /api/v3/catalog/{'{id}'}), delete catalog entity (DELETE /api/v3/catalog/{'{id}'}), submit SQL query (POST /api/v3/sql), create Reflection (POST /api/v3/reflection), and cancel job (POST /api/v3/job/{'{id}'}/cancel). For update or delete operations, include the entity or job ID at the end of the URL. You do not need to add the Authorization or Content-Type headers here—they are automatically added by Nexla using your saved Dremio credential.

Nexla can automatically send the response received from the Dremio API after each call to a new Nexla webhook data source. This is particularly useful for capturing the job ID returned when submitting a SQL query, enabling you to route it into a follow-on Nexla flow to poll job status or retrieve results.

Save & activate

Once all endpoint settings have been configured, click the Done button in the upper right corner of the screen to save and create the destination. To send the data to the configured Dremio endpoint, open the destination resource menu, and select Activate.

The Nexset data will not be sent to the Dremio API until the destination is activated. Destinations can be activated immediately or at a later time, providing full control over data movement.