NVIDIA AI

NVIDIA AI provides access to NVIDIA's large language models (LLMs) including Llama, Mistral, Yi, and other advanced AI models through the NVIDIA Inference Microservice (NIM) API. The NVIDIA AI connector enables you to generate text completions and chat completions using NVIDIA's LLM models for various AI-powered applications. This connector is particularly useful for applications that need to integrate advanced language models, build AI-powered features, perform text generation, or leverage NVIDIA's high-performance inference infrastructure.

Power end-to-end data operations for your NVIDIA AI API with Nexla. Our bi-directional NVIDIA AI connector is purpose-built for NVIDIA AI, making it simple to ingest data, sync it across systems, and deliver it anywhere — all with no coding required. Nexla turns API-sourced data into ready-to-use, reusable data products and makes it easy to send data to NVIDIA AI or any other destination. With comprehensive monitoring, lineage tracking, and access controls, Nexla keeps your NVIDIA AI workflows fast, secure, and fully governed.

Features

Type: API

SourceDestination

• Seamless API Integration: Connect to any endpoint as source or destination without coding, with automatic data product creation
• Visual Composition & Chaining: Build complex integrations using visual templates, chain API calls, and compose workflows with data validation and filtering
• API Proxy: Expose curated slices of your data securely with a secure and customizable API proxy that validates and transforms data on the fly
• Request optimization with intelligent batching, retry, and caching to minimize API calls and costs

Prerequisites

Before creating an NVIDIA AI credential, you'll need to obtain an API key from your NVIDIA AI account. NVIDIA provides API keys for programmatic access to their LLM inference services through the NVIDIA API platform.

To obtain an NVIDIA AI API key:

1. Log in to your NVIDIA AI account at https://build.nvidia.com or the NVIDIA API platform.

2. Navigate to your account settings or API dashboard. This is typically accessible from the user menu or dashboard navigation.

3. In the API settings or API Keys section, locate the option to create or manage API keys.

4. Click Create API Key or Generate New Key to create a new API key for your application. You may need to provide a name or description for the API key.

5. Copy the API key immediately after generation, as it may only be displayed once for security purposes. Store it securely, as you'll need it to authenticate API requests.

NVIDIA AI API keys are used as Bearer tokens in the Authorization header for all API requests. The API key is sensitive information and should be kept secure. If you've lost your API key, you'll need to generate a new one in your NVIDIA AI account settings.

For detailed information about NVIDIA AI API authentication and API key management, refer to the NVIDIA AI API Documentation.

Authenticate

Credentials required

An authentication method that requires sending a unique API key as Bearer token with each API request to NVIDIA's LLM API

Field	Required	Secret	Description
API Key Value	Yes	Yes	An encoded string value used as a secret token to authenticate API requests on NVIDIA
Base URL	Yes	No	Parameter for defining the base URL for the NVIDIA LLM API
API Version	Yes	No	Parameter for defining the API version for NVIDIA LLM API requests

Create a credential in Nexla

1. After selecting the data source/destination type, click the Add Credential tile to open the Add New Credential overlay.

New Credential Overlay – NVIDIA AI

2. Enter a name for the credential in the Credential Name field and a short, meaningful description in the Credential Description field.

3. In the API Key Value field, enter the API key that you obtained from your NVIDIA AI account. This is the secret API key used to authenticate requests to the NVIDIA AI API.

4. In the Base URL field, enter the base URL for the NVIDIA AI API. The default value is https://api.nvidia.com, which is the standard base URL for NVIDIA AI API services. You can specify a different base URL if your organization uses a custom endpoint.

5. In the API Version field, enter the API version you want to use. The default value is v1, which is the current version of the NVIDIA AI API. You can specify a different version if your organization uses a specific API version.

6. Click the Save button at the bottom of the overlay. The newly added credential will now appear in a tile on the Authenticate screen during data source/destination creation.

Use as a data source

To create a new data flow, navigate to the Integrate section, and click the New Data Flow button. Select the NVIDIA AI connector tile, then select the credential that will be used to connect to the NVIDIA AI API, and click Next; or, create a new NVIDIA AI credential for use in this flow. NVIDIA AI data sources can be created using pre-built endpoint templates that expedite source setup for common NVIDIA AI API endpoints, or configured manually to ingest data from NVIDIA AI API endpoints not included in the pre-built templates.

Endpoint templates

Nexla provides pre-built templates that can be used to rapidly configure data sources to ingest data from common NVIDIA AI endpoints. Select the endpoint from which this source will fetch data from the Endpoint pulldown menu. Available endpoint templates are listed in the expandable boxes below.

Text Completions

This endpoint generates text completions using NVIDIA's LLM API. Use this endpoint when you need to generate text based on prompts, complete sentences or paragraphs, or perform text generation tasks using NVIDIA's language models.

• Enter the model name to use for generating content in the Model field. Examples include mixtral_8x7b, meta/llama-3.1-70b-instruct, or other available NVIDIA LLM models. The default value is mixtral_8x7b.
• Enter the prompt text in the Prompt field. This is the input text that the model will use to generate completions.
• Optionally, specify the temperature for text generation in the Temperature field. Temperature controls the randomness of the output. Lower values (e.g., 0.2) make the output more deterministic, while higher values (e.g., 1.0) make it more creative. The default value is typically 1.0.
• Optionally, specify the top_p (nucleus sampling) parameter in the Top P field. This controls the diversity of the output by considering only the top p probability mass. The default value is typically 1.0.
• Optionally, specify the frequency penalty in the Frequency Penalty field. This penalizes tokens based on their frequency in the text so far. The default value is typically 0.0.
• Optionally, specify the presence penalty in the Presence Penalty field. This penalizes tokens based on whether they appear in the text so far. The default value is typically 0.0.
• Optionally, specify the maximum number of tokens to generate in the Max Tokens field. The default value varies by model. This limits the length of the generated text.
• Optionally, specify stop sequences in the Stop field. This should be a JSON array of strings that will cause the model to stop generating when encountered. Example: ["\n", "END"].

The Text Completions endpoint uses POST requests to send prompts to the NVIDIA AI API and returns generated text completions. The endpoint supports various NVIDIA LLM models and provides fine-grained control over text generation parameters. For more information about the Text Completions endpoint, refer to the NVIDIA AI API Documentation.

Chat Completions

This endpoint generates chat completions using NVIDIA's LLM API with support for multiple models. Use this endpoint when you need to build conversational AI applications, create chatbots, or perform interactive text generation with context.

• Enter the model name to use for generating content in the Model field. Examples include mixtral_8x7b, meta/llama-3.1-70b-instruct, or other available NVIDIA LLM models. The default value is mixtral_8x7b.
• Enter the message content in the Message field. This is the user message that will be sent to the model for chat completion.
• Optionally, specify the temperature for text generation in the Temperature field. Temperature controls the randomness of the output. Lower values make the output more deterministic, while higher values make it more creative. The default value is typically 1.0.
• Optionally, specify the top_p (nucleus sampling) parameter in the Top P field. This controls the diversity of the output. The default value is typically 1.0.
• Optionally, specify the frequency penalty in the Frequency Penalty field. This penalizes tokens based on their frequency. The default value is typically 0.0.
• Optionally, specify the presence penalty in the Presence Penalty field. This penalizes tokens based on whether they appear in the conversation. The default value is typically 0.0.
• Optionally, specify the maximum number of tokens to generate in the Max Tokens field. This limits the length of the generated response.
• Optionally, specify stop sequences in the Stop field. This should be a JSON array of strings that will cause the model to stop generating when encountered.

The Chat Completions endpoint uses POST requests to send chat messages to the NVIDIA AI API and returns conversational responses. The endpoint supports various NVIDIA LLM models and provides fine-grained control over chat generation parameters. For more information about the Chat Completions endpoint, refer to the NVIDIA AI API Documentation.

Once the selected endpoint template has been configured, click the Test button to the right of the endpoint selection menu to retrieve a sample of the data that will be fetched. Sample data will be displayed in the Endpoint Test Result panel on the right, allowing you to verify that the source is configured correctly before saving.

Manual configuration

NVIDIA AI data sources can also be manually configured to ingest data from any valid NVIDIA AI API endpoint, including endpoints not covered by the pre-built templates, chained API calls, or custom request parameters. Select the Advanced tab at the top of the configuration screen, and follow the instructions in Connect to Any API to configure the API method, endpoint URL, date/time and lookup macros, path to data, metadata, and request headers.

NVIDIA AI API endpoints typically follow the pattern {base_url}/{api_version}/completions for text completions or {base_url}/{api_version}/chat/completions for chat completions, where {base_url} is typically https://api.nvidia.com and {api_version} is typically v1. For Path to Data, common paths include $ for the entire response or $.choices[*] for arrays of completion choices. You do not need to include the Authorization header, as the Bearer token is automatically included from your credential; Content-Type is typically set to application/json. For NVIDIA AI completion requests, the request body typically includes a model field (e.g., "mixtral_8x7b"), a prompt field for text completions or a messages array for chat completions, and optionally fields like temperature, max_tokens, top_p, frequency_penalty, presence_penalty, and stop. Refer to the NVIDIA AI API documentation for the complete list of supported parameters.

Once all of the relevant settings have been configured, click the Create button in the upper right corner of the screen to save and create the new NVIDIA AI data source. Nexla will now begin ingesting data from the configured endpoint and will organize any data that it finds into one or more Nexsets.