Hugging Face TGI

Hugging Face Text Generation Inference (TGI) is a production-grade inference engine designed for high-performance serving of open-source Large Language Models (LLMs). The Hugging Face TGI connector enables you to interact with TGI-powered inference endpoints through the Hugging Face API, allowing you to generate chat-based completions, perform text generation tasks, and leverage AI-powered capabilities in your data workflows. This connector is particularly useful for applications that need to generate text content, perform language analysis, integrate AI capabilities into data processing pipelines, or build conversational AI applications using open-source LLMs.

Power end-to-end data operations for your Hugging Face TGI API with Nexla. Our bi-directional Hugging Face TGI connector is purpose-built for Hugging Face TGI, making it simple to ingest data, sync it across systems, and deliver it anywhere — all with no coding required. Nexla turns API-sourced data into ready-to-use, reusable data products and makes it easy to send data to Hugging Face TGI or any other destination. With comprehensive monitoring, lineage tracking, and access controls, Nexla keeps your Hugging Face TGI workflows fast, secure, and fully governed.

Features

Type: API

SourceDestination

• Seamless API Integration: Connect to any endpoint as source or destination without coding, with automatic data product creation
• Visual Composition & Chaining: Build complex integrations using visual templates, chain API calls, and compose workflows with data validation and filtering
• API Proxy: Expose curated slices of your data securely with a secure and customizable API proxy that validates and transforms data on the fly
• Request optimization with intelligent batching, retry, and caching to minimize API calls and costs

Prerequisites

Before creating a Hugging Face TGI credential, you'll need to obtain an API key from your Hugging Face account. Hugging Face provides API keys for programmatic access to their Text Generation Inference (TGI) endpoints and other services.

To obtain a Hugging Face API key:

1. Log in to your Hugging Face account at https://huggingface.co or create an account if you don't have one.

2. Navigate to your account settings by clicking on your profile picture in the top right corner and selecting Settings, or by visiting https://huggingface.co/settings.

3. In the Settings page, locate the Access Tokens section in the left sidebar or scroll to find the token management section.

4. Click New token or Create token to generate a new API key. You can create tokens with different scopes (read, write) depending on your needs.

5. Give your token a descriptive name (e.g., "Nexla Integration") and select the appropriate permissions. For TGI API access, you typically need read permissions.

6. Click Generate token to create the new API key.

7. Copy the API token immediately after generation, as it may only be displayed once for security purposes. Store it securely, as you'll need it to authenticate API requests.

8. Note the base URL for your TGI endpoint (typically https://router.huggingface.co/novita or your custom TGI endpoint URL) and the API version you'll be using (typically v3/openai for OpenAI-compatible endpoints).

For detailed information about Hugging Face API authentication and token management, refer to the Hugging Face API Documentation and Text Generation Inference API Reference.

Authenticate

Credentials required

Authenticate using Hugging Face API Key

Field	Required	Secret	Description
API Key	Yes	Yes	Hugging Face-provided secret API key
Base URL	Yes	No	The base URL for the Huggingface API.
API Version	Yes	No	Parameter for defining the API version for Huggingface API requests

Create a credential in Nexla

1. After selecting the data source/destination type, click the Add Credential tile to open the Add New Credential overlay.

New Credential Overlay – Hugging Face TGI

2. Enter a name for the credential in the Credential Name field and a short, meaningful description in the Credential Description field.

3. In the API Key field, enter the Hugging Face API key that you obtained from your Hugging Face account settings. This is the secret API key used to authenticate requests to the Hugging Face TGI API.

   The API key is sensitive information and should be kept secure. If you've lost your API key, you'll need to generate a new one in your Hugging Face account settings. API keys are used in the Authorization header as a Bearer token for all API requests.

4. In the Base URL field, enter the base URL for the Hugging Face TGI API. The default value is https://router.huggingface.co/novita, which is a common TGI endpoint. You can specify a different base URL if you're using a custom TGI endpoint or inference endpoint.

5. In the API Version field, enter the API version you want to use. The default value is v3/openai, which provides OpenAI-compatible API endpoints for TGI. You can specify a different version if your TGI endpoint uses a different API version.

6. Click the Save button at the bottom of the overlay. The newly added credential will now appear in a tile on the Authenticate screen during data source/destination creation.

Use as a data source

To create a new data flow, navigate to the Integrate section, and click the New Data Flow button. Select the Hugging Face TGI connector tile, then select the credential that will be used to connect to the Hugging Face TGI endpoint, and click Next; or, create a new Hugging Face TGI credential for use in this flow.

Endpoint templates

Nexla provides pre-built templates that can be used to rapidly configure data sources to ingest data from common Hugging Face TGI endpoints. Select the endpoint from which this source will fetch data from the Endpoint pulldown menu. Available endpoint templates are listed in the expandable boxes below.

Chat Completions

This endpoint generates chat-based completions using Hugging Face's TGI Messages API. Use this endpoint when you need to generate conversational responses, perform text analysis, or leverage TGI's language understanding capabilities for your applications.

• Enter the model ID in the Model field. This is the identifier for the TGI model you want to use for generating completions. Examples include deepseek/deepseek-v3-0324, mistralai/Mistral-7B-Instruct-v0.2, gpt2, or other models supported by your TGI endpoint. The default value is deepseek/deepseek-v3-0324. Ensure the model you specify is available on your TGI endpoint.
• Enter the messages array in JSON format in the Messages field. This should be an array of message objects, where each object has a role (e.g., "user", "assistant", "system") and content (the message text). Example format: [{"role": "user", "content": "What is Deep Learning?"}]. The default value provides a sample user message. You can include multiple messages to create a conversation history.
• Optionally, enter the maximum number of tokens in the Max Tokens field to limit the length of the generated response. This helps control API costs and response length. The default value is 50 tokens. For longer responses, you can increase this value, but be aware that longer responses consume more API quota and take more time to generate.
• Optionally, enter a temperature value in the Temperature field to control the randomness and creativity of the model's output. Temperature controls the probability distribution of token selection. Lower values (e.g., 0.1-0.3) produce more focused, deterministic, and factual responses, while higher values (e.g., 0.7-1.0) produce more creative and varied responses. The default value is 0.7, which provides a balance between creativity and consistency.
• Optionally, enter a Top-P value in the Top P field to control diversity via nucleus sampling. Top-P limits token selection to those whose cumulative probability mass reaches the specified threshold. Higher values (closer to 1) increase diversity by considering more token options, while lower values make the model more conservative. The default value is 0.9, which allows good diversity while maintaining quality.
• Optionally, set the Stream field to true if you want to stream the response as it's generated, or false to receive the complete response at once. Streaming can be useful for real-time applications, but for data ingestion purposes, you typically want false to receive complete responses. The default value is false.

The Chat Completions endpoint uses POST requests to send messages to the TGI model. Adjust temperature and Top-P values based on your use case: use lower values for factual content and data extraction, and use higher values for creative writing and brainstorming. The combination of these parameters allows you to fine-tune the model's output to match your specific requirements. For more information about the Chat Completions endpoint, refer to the Hugging Face TGI API Documentation.

Once the selected endpoint template has been configured, click the Test button to the right of the endpoint selection menu to retrieve a sample of the data that will be fetched. Sample data will be displayed in the Endpoint Test Result panel on the right, allowing you to verify that the source is configured correctly before saving.

Manual configuration

Hugging Face TGI data sources can also be manually configured to ingest data from any valid TGI API endpoint not covered by the pre-built templates, including endpoints that require chained API calls or custom request parameters. TGI API endpoints typically follow the pattern https://router.huggingface.co/novita/v3/openai/chat/completions, and most TGI endpoints use the POST method for chat completions. Select the Advanced tab at the top of the configuration screen, and follow the instructions in Connect to Any API to configure the API method, endpoint URL, date/time and lookup macros, path to data, metadata, and request headers.

For TGI API responses, the generated content is typically located at the JSON path $.choices[*].message.content for chat completions. TGI API requests use Content-Type: application/json for request bodies.

Once all of the relevant settings have been configured, click the Create button in the upper right corner of the screen to save and create the new Hugging Face TGI data source. Nexla will now begin ingesting data from the configured endpoint and will organize any data that it finds into one or more Nexsets.