Skip to main content

Create Destination(s)

Create a Destination​

Creating a data destination in Nexla allows you to define where and how your processed data should be output. Destinations can write to various systems including databases, file storage, APIs, and streaming platforms.

Required Fields​

When creating a destination, you must specify these essential components:

  • name: A unique identifier for your destination
  • sink_type: The type of destination (e.g., s3, postgres, kafka)
  • sink_config: Configuration specific to the destination type
  • data_credentials_id: Authentication credentials for the target system
  • data_set_id: The Nexset whose output will be written to this destination

API Endpoint​

The primary endpoint for creating destinations is:

POST /data_sinks

Response Structure​

A successful creation returns a complete destination object with all configuration details, credentials, and metadata.

Creating with Existing Credentials​

You can reference existing credentials when creating a destination. This approach is useful when you have already configured authentication for a target system.

Create with Existing Credentials: Request
POST /data_sinks

{
  "name": "Test Destination",
  "description": "Destination for processed customer data",
  "sink_type": "dropbox",
  "sink_config": {
    "mapping": {
      "mode": "auto",
      "tracker_mode": "NONE"
    },
    "data_format": "json",
    "sink_type": "dropbox",
    "path": "/nexlatests/dataout/rel22",
    "output.dir.name.pattern": "demo/{yyyy}/{MM}/{dd}"
  },
  "data_credentials": 8342,
  "data_set_id": 22194
}

Creating with Inline Credentials​

You can also create credentials inline when setting up a destination. This approach is convenient for one-time setups or when you want to manage everything in a single request.

Create with Inline Credentials: Request
POST /data_sinks

{
  "name": "Test Destination",
  "description": "Destination with inline credentials",
  "sink_type": "dropbox",
  "sink_config": {
    "mapping": {
      "mode": "auto",
      "tracker_mode": "NONE"
    },
    "data_format": "json",
    "sink_type": "dropbox",
    "path": "/nexlatests/dataout/rel22",
    "output.dir.name.pattern": "demo/{yyyy}/{MM}/{dd}"
  },
  "data_set_id": 22194,
  "data_credentials": {
    "name": "FTP CREDS",
    "credentials_type": "ftp",
    "credentials_version": "1",
    "credentials": {
      "credentials_type": "ftp",
      "account_id": "XYZ",
      "password": "123"
    }
  }
}

Destination Configuration​

The sink_config object contains destination-specific settings that control how data is written and formatted.

File System Destinations​

For file-based destinations like S3, FTP, or Dropbox:

{
  "sink_config": {
    "mapping": {
      "mode": "manual",
      "mapping": {
        "customer_id": ["customer_id"],
        "name": ["customer_name"],
        "email": ["email_address"]
      },
      "fields_order": ["customer_id", "name", "email"],
      "tracker_mode": "NONE"
    },
    "data_format": "csv",
    "sink_type": "s3",
    "path": "bucket-name/output-folder",
    "output.dir.name.pattern": "{yyyy}-{MM}-{dd}/{HH}"
  }
}

Database Destinations​

For database destinations like PostgreSQL or Redshift:

{
  "sink_config": {
    "insert.mode": "UPSERT",
    "upsert.nulls": true,
    "sink_type": "redshift",
    "table": "customer_data",
    "database": "analytics_db",
    "primary.key": "customer_id",
    "mapping": {
      "mode": "manual",
      "mapping": {
        "customer_id": "customer_id",
        "customer_name": "name",
        "email_address": "email"
      }
    }
  }
}

Post-Creation Steps​

After successfully creating a destination, you should:

  1. Verify Configuration: Check that all settings match your requirements
  2. Test Connection: Validate connectivity to the target system
  3. Activate Destination: Start the data flow to begin writing data
  4. Monitor Performance: Track delivery success rates and performance

Best Practices​

To ensure successful destination creation:

  • Use Descriptive Names: Choose names that clearly identify the destination's purpose
  • Validate Credentials: Ensure credentials have appropriate permissions for the target system
  • Test Configurations: Verify sink configurations work with your data format
  • Plan Mapping: Design field mappings that preserve data integrity
  • Consider Scheduling: Set appropriate output frequencies based on your data volume

Error Handling​

Common creation issues and solutions:

  • Invalid Credentials: Verify credential IDs exist and are accessible
  • Missing Required Fields: Ensure all mandatory parameters are provided
  • Invalid Sink Type: Confirm the sink type is supported and properly configured
  • Mapping Errors: Validate field mappings match your Nexset schema
  • Permission Issues: Check that your account can create destinations

After creating a destination, you may need to:

Update Configuration​

PUT /data_sinks/{destination_id}

Activate Destination​

PUT /data_sinks/{destination_id}/activate

Test Connection​

PUT /data_sinks/{destination_id}/probe/authenticate

Monitor Status​

GET /data_sinks/{destination_id}/flow/status_metrics