Skip to main content

Inspect Credential Data

Credential inspection in Nexla allows you to validate authentication, explore data structures, and understand the content available through your credentials before creating data sources or destinations. These capabilities help you make informed decisions about data integration.

Inspection Capabilities​

The inspection endpoints provide several key capabilities:

  • Authentication Testing: Verify credential validity and connectivity
  • Structure Exploration: Understand the organization of data in external systems
  • Content Sampling: Preview actual data content and format
  • Schema Detection: Identify data structure and field types
  • Path Validation: Confirm correct paths and access permissions

Authenticate Credential​

You can check the validity of a data credentials resource using the /data_credentials/{credential_id}/probe or /data_credentials/{credential_id}/probe/authenticate endpoint.

Authentication Endpoint​

To test credential validity:

GET /data_credentials/{credential_id}/probe/authenticate
Authenticate Credential: Request
GET /data_credentials/5001/probe/authenticate

Authentication Response​

A successful authentication returns a 200 status code, while authentication failures return appropriate error codes with descriptive messages.

Inspect File or Database Type Credential Content Tree​

You can inspect the tree structure in the external source to a particular depth. Note that not all storage types have a natural tree structure.

Tree Structure Endpoint​

To explore the hierarchical structure of your data:

POST /data_credentials/{credential_id}/probe/tree
Inspect File/Database Credential: Request
POST /data_credentials/5001/probe/tree

{
  "region": "us-west-1",
  "bucket": "production-s3-basin",
  "prefix": "events_v2/",
  "depth": 3
}

Tree Structure Response​

The response shows the hierarchical organization of your data, helping you understand the structure before creating data sources.

Inspect File/Database Credential: Response
{
  "status": "ok",
  "connection_type": "s3",
  "output": {
    "events_v2": {
      "2015": {
        "11": {
          "1": {},
          "2": {},
          "3": {}
        },
        "12": {
          "1": {},
          "8": {}
        }
      },
      "2017": {
        "2": {
          "20": {}
        }
      }
    }
  }
}

Inspect File Type Credential Content​

Use the endpoint below to get metadata and sample records from a set of files. The POST body must contain the path of the file starting from the root of the location that the data credential points to.

File Content Endpoint​

To examine file content and metadata:

POST /data_credentials/{credential_id}/probe/files
Inspect File Credential Content: Request
POST /data_credentials/5001/probe/files

{
  "path": "demo-in.nexla.com",
  "file": "Demo/Stock.json"
}

File Content Response​

The response provides file metadata, sample content, and structure information to help you understand the data format.

Inspect Database Schema​

For database credentials, you can inspect the schema structure and available tables.

Schema Inspection Endpoint​

To explore database structure:

POST /data_credentials/{credential_id}/probe/schemas
Inspect Database Schema: Request
POST /data_credentials/5001/probe/schemas

{
  "database": "analytics_db",
  "table": "customer_data"
}

Schema Response​

The response includes table structure, field types, and sample data to help you understand the database schema.

Inspect Sample Data​

You can retrieve sample records from files or database tables to understand the data format and content.

Sample Data Endpoint​

To get sample records:

POST /data_credentials/{credential_id}/probe/sample
Inspect Sample Data: Request
POST /data_credentials/5001/probe/sample

{
  "path": "data/analytics",
  "file": "customer_metrics.csv",
  "sample_size": 100
}

Sample Data Response​

The response provides sample records with field values and data types to help you understand the content structure.

Search Paths​

You can search for specific paths or files within your credential's accessible locations.

Path Search Endpoint​

To search for specific paths:

POST /data_credentials/{credential_id}/probe/search_path
Search Paths: Request
POST /data_credentials/5001/probe/search_path

{
  "search_pattern": "*.csv",
  "path": "data/",
  "recursive": true
}

Path Search Response​

The response includes matching paths and files that meet your search criteria.

Inspection Benefits​

Credential inspection provides several key benefits:

Pre-Integration Validation​

  • Verify Connectivity: Ensure credentials work before creating resources
  • Understand Structure: Learn about data organization and format
  • Preview Content: See actual data samples and field types
  • Validate Paths: Confirm correct file or table paths

Resource Planning​

  • Data Source Design: Plan data source configuration based on structure
  • Destination Setup: Understand output format requirements
  • Schema Mapping: Plan field mappings and transformations
  • Performance Optimization: Identify optimal data access patterns

Best Practices​

To effectively use credential inspection:

  1. Test Before Use: Always validate credentials before creating resources
  2. Explore Structure: Use tree inspection to understand data organization
  3. Sample Data: Preview content to understand format and quality
  4. Document Findings: Keep records of discovered structures and paths
  5. Plan Integration: Use inspection results to plan data source/destination setup

Error Handling​

Common inspection issues and solutions:

  • Authentication Failures: Check credential validity and permissions
  • Path Errors: Verify correct paths and access rights
  • Connection Issues: Check network connectivity and firewall settings
  • Permission Denied: Ensure credentials have appropriate access rights

After inspecting credentials, you may need to:

Create Data Sources​

POST /data_sources

Create Data Destinations​

POST /data_sinks

Update Credentials​

PUT /data_credentials/{credential_id}

Test Connections​

PUT /data_credentials/{credential_id}/probe/authenticate