Skip to main content

Update Destination(s)

Update a Destination​

Updating data destinations in Nexla allows you to modify configuration, credentials, and settings to adapt to changing requirements. The system provides comprehensive update capabilities while maintaining data integrity and access controls.

Update Endpoint​

To update an existing data destination, use the PUT endpoint with the destination ID:

PUT /data_sinks/{destination_id}

Updateable Fields​

Data destinations support updates to various configuration aspects:

  • Basic Information: Name, description, and tags
  • Configuration: Sink-specific settings and parameters
  • Credentials: Authentication and connection details
  • Mapping: Data transformation and field mapping rules
  • Scheduling: Output timing and frequency settings

Configuration Updates​

You can update destination configuration to optimize performance and functionality.

Update Destination Configuration: Request
PUT /data_sinks/4001

{
  "name": "Updated S3 Data Sink",
  "description": "Enhanced customer data export with improved mapping",
  "sink_config": {
    "mapping": {
      "mode": "manual",
      "mapping": {
        "customer_id": ["customer_id"],
        "customer_name": ["customer_name"],
        "email": ["email_address"],
        "store_code": ["store_code"],
        "city_code": ["city_code"],
        "item_price": ["item_price"],
        "discount": ["discount"],
        "discounted_price": ["discounted_price"]
      },
      "fields_order": [
        "customer_id",
        "customer_name",
        "email",
        "store_code",
        "city_code",
        "item_price",
        "discount",
        "discounted_price"
      ],
      "tracker_mode": "NONE"
    },
    "data_format": "csv",
    "sink_type": "s3",
    "path": "customer-solutions.example.com/echo/nexla_outputs",
    "output.dir.name.pattern": "{yyyy}-{MM}-{dd}/{HH}"
  }
}

Credential Updates​

Update destination credentials when authentication information changes.

Update Destination Credentials: Request
PUT /data_sinks/4001

{
  "data_credentials_id": 5002
}

Flow Type Updates​

Modify the flow type to optimize data processing performance.

Update Flow Type: Request
PUT /data_sinks/4001

{
  "flow_type": "in_memory"
}

Code Container Updates​

Update code containers for custom data processing logic.

Update Code Container: Request
PUT /data_sinks/4001

{
  "code_container_id": 6001
}

Partial Updates​

The API supports partial updates, allowing you to modify only specific fields without affecting others.

Partial Update: Request
PUT /data_sinks/4001

{
  "name": "New Destination Name",
  "description": "Updated description only"
}

Response Structure​

A successful update returns the complete updated destination object with all current configuration and metadata.

Update Destination: Response
{
  "id": 4001,
  "owner": {
    "id": 42,
    "full_name": "John Smith"
  },
  "org": {
    "id": 101,
    "name": "Acme Corporation",
    "email_domain": "acme.com",
    "email": null
  },
  "access_roles": [
    "owner"
  ],
  "name": "Updated S3 Data Sink",
  "description": "Enhanced customer data export with improved mapping",
  "status": "ACTIVE",
  "data_set_id": 3001,
  "data_map_id": null,
  "sink_type": "s3",
  "sink_format": null,
  "sink_config": {
    "mapping": {
      "mode": "manual",
      "mapping": {
        "customer_id": ["customer_id"],
        "customer_name": ["customer_name"],
        "email": ["email_address"],
        "store_code": ["store_code"],
        "city_code": ["city_code"],
        "item_price": ["item_price"],
        "discount": ["discount"],
        "discounted_price": ["discounted_price"]
      },
      "fields_order": [
        "customer_id",
        "customer_name",
        "email",
        "store_code",
        "city_code",
        "item_price",
        "discount",
        "discounted_price"
      ],
      "tracker_mode": "NONE"
    },
    "data_format": "csv",
    "sink_type": "s3",
    "path": "customer-solutions.example.com/echo/nexla_outputs",
    "output.dir.name.pattern": "{yyyy}-{MM}-{dd}/{HH}"
  },
  "sink_schedule": null,
  "managed": false,
  "nexset": {
    "id": 3001,
    "name": "echo"
  },
  "data_credentials": {
    "id": 5002,
    "name": "S3 Credentials",
    "credentials_type": "s3"
  },
  "updated_at": "2023-01-15T12:30:00.000Z",
  "created_at": "2023-01-15T11:56:40.000Z",
  "tags": []
}

Update Considerations​

When updating destinations, consider the impact on data processing and downstream systems.

Data Consistency​

  • Schema Changes: Ensure mapping updates maintain data integrity
  • Format Changes: Verify output format compatibility with downstream systems
  • Credential Updates: Test connectivity after credential changes

Performance Impact​

  • Flow Type Changes: Monitor performance after flow type modifications
  • Configuration Updates: Validate that new settings improve performance
  • Resource Allocation: Ensure sufficient resources for updated configurations

Access Control​

  • Permission Updates: Verify access rights after configuration changes
  • Team Access: Ensure team members maintain appropriate access levels
  • Audit Trail: Track all configuration modifications for compliance

Best Practices​

To effectively update data destinations:

  1. Test Changes: Validate updates in non-production environments first
  2. Monitor Performance: Track metrics after configuration changes
  3. Document Changes: Maintain records of all modifications and reasons
  4. Backup Configuration: Keep backup copies of working configurations
  5. Gradual Rollout: Implement changes incrementally when possible

Error Handling​

Common update issues and solutions:

  • Invalid Configuration: Verify all configuration parameters are valid
  • Permission Issues: Ensure you have appropriate update rights
  • Resource Conflicts: Check for conflicts with existing configurations
  • Validation Errors: Address any schema or format validation issues

After updating destinations, you may need to:

Test Destination​

PUT /data_sinks/{id}/probe/authenticate

Validate Configuration​

PUT /data_sinks/{id}/validate

Monitor Performance​

GET /data_sinks/{id}/metrics

Control Output​

PUT /data_sinks/{id}/activate
PUT /data_sinks/{id}/pause