Pinecone API

Pinecone is a managed vector database service that provides high-performance similarity search capabilities for machine learning applications, enabling developers to build and scale AI-powered search, recommendation, and retrieval systems with ease.

Power AI-ready data operations with Pinecone API and Nexla. Our Pinecone API connector makes it simple to ingest, transform, chunk, and deliver structured or unstructured data to Pinecone API — all without coding. Nexla automatically organizes raw text and documents into reusable data products that you can easily prepare for vector search and retrieval-augmented generation (RAG) using our built-in transforms like agentic chunking and incremental loading. With real-time validation, schema checks, and comprehensive monitoring, Nexla keeps your Pinecone API workflows fast, secure, and fully governed for production AI use cases.

Features

Type: Vector Database

SourceDestination

• AI-Ready Data Preparation: Automatically chunk, vectorize, and index data from any source into your vector database for fast, contextually relevant search
• Advanced RAG Integration: Query vector databases to power retrieval-augmented generation workflows with query rewriting, re-ranking, and multi-model orchestration
• Enterprise RAG Framework: Build production-ready RAG applications with built-in access controls, evaluation grading, and NVIDIA NIM hardware acceleration

Prerequisites

Before creating a Pinecone credential, you need to obtain your API key and identify your index host URL from your Pinecone account. Pinecone uses API key authentication for all API requests, with the API key sent in the Api-Key header.

Pinecone Data Sources & Destinations in Nexla

Nexla offers two connectors that can be used with Pinecone databases: the Pinecone API connector documented here, and the Pinecone (Native) connector. The Pinecone API connector offers a streamlined process for rapid creation of Pinecone data sources & destinations, while the native Pinecone connector provides the flexibility to apply dynamic metadata filtering.

To obtain your Pinecone API credentials, follow these steps:

1. Sign in to your Pinecone account, or create a new account at Pinecone.

2. Navigate to your Pinecone dashboard or project settings in the Pinecone interface.

3. Look for the API Keys or API section in your account settings or project dashboard.

4. If you don't have an API key yet, click Create API Key or Generate API Key to create a new API key.

5. Configure your API key settings:

   • Enter a name for the API key (e.g., "Nexla Integration")
   • Review and select the permissions or scopes for the key (if applicable)

6. Click Create or Generate to create the API key.

7. Copy the API key immediately after it's generated, as it may not be accessible again after you navigate away from the page.

8. Navigate to your Pinecone index page to find your index host URL. The index host URL is typically in the format https://{index-name}-{project-id}.svc.{environment}.pinecone.io or similar. You can find this in your Pinecone index settings or on the index details page.

9. Store both the API key and index host URL securely, as you will need them to configure your Nexla credential. The API key is sensitive information and should be kept confidential.

The API key is sent in the Api-Key header for all API requests to the Pinecone API. The index host URL determines which Pinecone index your API requests will be sent to. The API key authenticates your requests and grants access to your Pinecone index based on your account permissions. If your API key is compromised, you should immediately revoke it in your Pinecone account settings and generate a new one. For detailed information about obtaining API keys, index host URLs, API authentication, and available endpoints, refer to the Pinecone API documentation.

Authenticate

Credentials required

Field	Required	Secret	Description
Index Host URL	Yes	No	The host URL for your Pinecone index, it can be found at your index page.
Pinecone API Key	Yes	Yes	API Key

Create a credential in Nexla

1. After selecting the data source/destination type, click the Add Credential tile to open the Add New Credential overlay.

New Credential Overlay – Pinecone API

2. Enter a name for the credential in the Credential Name field and a short, meaningful description in the Credential Description field.

3. Enter your Pinecone index host URL in the Index Host URL field. This should be the host URL for your Pinecone index, typically in the format https://{index-name}-{project-id}.svc.{environment}.pinecone.io or similar. The index host URL determines which Pinecone index your API requests will be sent to. You can find this on your Pinecone index page.

4. Enter your Pinecone API key in the Pinecone API Key field. This is the API key you obtained from your Pinecone account settings (API Keys section). The API key is sent in the Api-Key header for all API requests to the Pinecone API. The API key is sensitive information and must be kept confidential.

   Your Pinecone API key can be found in your Pinecone account settings under the API Keys section. The API key is sent in the Api-Key header for all API requests to the Pinecone API. The index host URL should match your Pinecone index host URL, which can be found on your Pinecone index page.

   If your API key is compromised, you should immediately revoke it in your Pinecone account settings and generate a new one. The API key provides access to your Pinecone index data and should be treated as sensitive information. Keep your API key secure and do not share it publicly.

   For detailed information about obtaining API keys, index host URLs, API authentication, and available endpoints, see the Pinecone API documentation.

5. Click the Save button at the bottom of the overlay. The newly added credential will now appear in a tile on the Authenticate screen during data source/destination creation and can be selected for use with a new data source or destination.

Use as a data source

To create a new data flow, navigate to the Integrate section, and click the New Data Flow button. Select the Pinecone API connector tile, then select the credential that will be used to connect to your Pinecone account, and click Next; or, create a new Pinecone API credential for use in this flow.

Endpoint templates

Nexla provides pre-built templates that can be used to rapidly configure data sources to ingest data from common Pinecone endpoints. Select the endpoint from which this source will fetch data from the Endpoint pulldown menu. Available endpoint templates are listed in the expandable boxes below.

Query Vectors

This endpoint template searches a namespace using a query vector from your Pinecone index. Use this template when you need to perform similarity search to find vectors that are most similar to a query vector, which is useful for recommendation systems, search functionality, and other AI-powered applications.

• Enter the query vector in the Vector field. This should be a vector array in JSON format (e.g., [0.1, 0.2, 0.3, ...]). The query vector should be the same length as the dimension of the index being queried. The query vector is used to find the most similar vectors in your Pinecone index.
• Enter the number of results to return in the Top K field. This should be the number of most similar vectors you want to retrieve (default: 100). The Top K value determines how many results will be returned for each query.
• Select whether to include vector values in the Include Values field. Available options include true (include vector values in the response) and false (do not include vector values, default: false). Including values increases response size but provides the full vector data.
• Select whether to include vector metadata in the Include Metadata field. Available options include true (include vector metadata in the response, default: true) and false (do not include vector metadata). Including metadata provides additional information associated with each vector.

This endpoint performs similarity search using a query vector to find the most similar vectors in your Pinecone index. The query vector should match the dimension of your index. The endpoint returns the top K most similar vectors based on the similarity metric configured for your index.

For detailed information about vector queries, similarity search, API response structures, and available query parameters, see the Pinecone API documentation.

List Vector IDs

This endpoint template lists the IDs of vectors in a single namespace from your Pinecone index. This is supported only for serverless indices. Use this template when you need to retrieve a list of all vector IDs in a namespace, which is useful for inventory management, data discovery, or batch operations.

• Enter the namespace in the Namespace field. This should be the namespace name for which you want to list vector IDs. The namespace determines which vectors' IDs will be listed. Leave this field empty to list vectors in the default namespace.
• Enter an optional prefix in the Prefix field. This should be a prefix to limit the results to IDs with a common prefix. The prefix is useful for filtering vector IDs by a common naming pattern. Leave this field empty to list all vector IDs in the namespace.

This endpoint lists the IDs of vectors in a single namespace from your Pinecone index. This endpoint is supported only for serverless indices. The endpoint uses token-based pagination to handle large datasets efficiently. Nexla will automatically fetch subsequent pages of data by following the pagination token returned in the API response.

For detailed information about listing vector IDs, namespace management, pagination, and serverless index support, see the Pinecone API documentation.

Query Vectors By ID

This endpoint template searches a namespace using the unique ID of a vector as a query vector from your Pinecone index. Use this template when you need to find vectors similar to a specific vector by its ID, which is useful for finding related content or recommendations based on existing vectors.

• Enter the vector ID in the Vector ID field. This should be the unique identifier of the vector you want to use as a query. The vector ID determines which vector will be used as the query vector to find similar vectors.
• Enter the number of results to return in the Top K field. This should be the number of most similar vectors you want to retrieve. The Top K value determines how many results will be returned for each query.
• Select whether to include vector values in the Include Values field. Available options include true (include vector values in the response) and false (do not include vector values). Including values increases response size but provides the full vector data.
• Select whether to include vector metadata in the Include Metadata field. Available options include true (include vector metadata in the response) and false (do not include vector metadata). Including metadata provides additional information associated with each vector.
• Enter the namespace in the Namespace field. This should be the namespace name where you want to query vectors. The namespace determines which namespace will be searched. Leave this field empty to query vectors in the default namespace.

This endpoint performs similarity search using a vector ID as a query to find the most similar vectors in your Pinecone index. The endpoint uses the vector with the specified ID as the query vector to find similar vectors. The endpoint returns the top K most similar vectors based on the similarity metric configured for your index.

For detailed information about querying vectors by ID, similarity search, API response structures, and available query parameters, see the Pinecone API documentation.

Once the selected endpoint template has been configured, click the Test button to the right of the endpoint selection menu to retrieve a sample of the data that will be fetched. Sample data will be displayed in the Endpoint Test Result panel on the right, allowing you to verify that the source is configured correctly before saving.

Manual configuration

Pinecone API data sources can also be manually configured to ingest data from any valid Pinecone API endpoint, including endpoints not covered by the pre-built templates, chained API calls, or custom request parameters. Select the Advanced tab at the top of the configuration screen, and follow the instructions in Connect to Any API to configure the API method, endpoint URL, path to data, and pagination.

Pinecone API typically uses GET for retrieving data and POST for querying vectors. For the Response Data Path, use $.matches[*] to extract all matches from the matches array for query endpoints, or $.vectors[*] to extract all vector IDs from the vectors array for list endpoints, depending on which endpoint you're configuring.

Pinecone API uses token-based pagination with paginationToken for endpoints that support it, such as List Vector IDs. When configuring pagination, set the JSONPath expression to the next token to $.pagination.next and the parameter name to paginationToken.

Once all of the relevant settings have been configured, click the Next button to proceed with the rest of the data flow configuration, or click Save to save the data source configuration for later use.

Use as a destination

Click the + icon on the Nexset that will be sent to the Pinecone destination, and select the Send to Destination option from the menu. Select the Pinecone API connector from the list of available destination connectors, then select the credential that will be used to connect to your Pinecone account, and click Next; or, create a new Pinecone API credential for use in this flow.

Endpoint templates

Nexla provides pre-built templates that can be used to rapidly configure destinations to send data to common Pinecone endpoints. Select the endpoint to which data will be sent from the Endpoint pulldown menu. Then, click on the template in the list below to expand it, and follow the instructions to configure additional endpoint settings.

Upsert Vectors

This endpoint template writes vectors into a namespace in your Pinecone index using records from a Nexset. Use this template when you need to insert or update vectors in your Pinecone index for similarity search, recommendation systems, or other AI-powered applications. Check out the documentation URL for more details on record schema.

• Enter the namespace in the Namespace field. This should be the namespace name where you want to upsert vectors. The namespace determines which namespace will receive the vectors. Leave this field empty to upsert vectors to the default namespace. Note: Namespaces are not supported by projects on the gcp-starter environment.

This endpoint sends data as JSON in the request body to upsert vectors into your Pinecone index. The endpoint uses batch mode to efficiently send multiple vectors in a single request. Each record from your Nexset will be included in the batch, and vectors will be upserted in batches of up to 20 vectors per request.

The vector structure must match the Pinecone API's expected format. Each vector should include an ID, vector values, and optional metadata. Check out the documentation URL for more details on record schema. For detailed information about vector upsert, request body formats, batch processing, and available vector properties, see the Pinecone API documentation.

Manual configuration

Pinecone API destinations can also be manually configured to send data to any valid Pinecone API endpoint. Select the Advanced tab at the top of the configuration screen, and follow the instructions in Connect to Any API to configure the API method, data format, endpoint URL, and request body.

Pinecone API typically uses POST for upserting vectors, and expects JSON format for all requests. Configure the request body as {message.json} to send the entire Nexset record as JSON, or construct a custom JSON structure with field names matching the Pinecone API's expected format. For batch operations, format the request body to include a vectors array containing multiple vectors.

Each vector must include an ID, vector values, and optional metadata. For detailed information about request body formats and available vector properties, see the Pinecone API documentation.

Save & activate

Once all endpoint settings have been configured, click the Done button in the upper right corner of the screen to save and create the destination. To send the data to the configured Pinecone endpoint, open the destination resource menu, and select Activate.

The Nexset data will not be sent to the Pinecone endpoint until the destination is activated. Destinations can be activated immediately or at a later time, providing full control over data movement.