Link Transforms

Linking transforms in Nexla allows you to connect transformation functions to your data processing workflows, enabling you to apply data modifications at specific points in your data pipeline and create complex transformation chains.

Transform Linking Overview

Transform linking establishes connections between transformation functions and data processing components, allowing you to orchestrate data modifications and create sophisticated data transformation workflows that can be reused across different Nexsets and data flows.

Core Linking Capabilities

The transform linking system provides several key capabilities for effective integration and orchestration of transformation functions.

Workflow Integration

Integrate transforms into data processing workflows:

• Nexset Integration: Link transforms to specific Nexsets for data processing
• Flow Integration: Integrate transforms into data flow pipelines
• Chain Creation: Create transformation chains and sequences
• Conditional Processing: Apply transforms based on data conditions

Transform Orchestration

Orchestrate multiple transforms effectively:

• Sequential Processing: Process transforms in specific order
• Parallel Processing: Execute transforms in parallel when possible
• Dependency Management: Manage transform dependencies and requirements
• Error Handling: Handle transform failures and recovery

Configuration Management

Manage transform configurations and settings:

• Parameter Binding: Bind transform parameters to workflow variables
• Environment Configuration: Configure transforms for different environments
• Performance Tuning: Optimize transform performance in workflows
• Monitoring Setup: Set up monitoring and alerting for linked transforms

Transform Linking Types

Understanding different linking types helps you choose the right approach for your data processing needs.

Direct Linking

Direct connection of transforms to processing components:

• Single Transform: Link one transform to a processing component
• Transform Chain: Link multiple transforms in sequence
• Parallel Transforms: Link transforms for parallel execution
• Conditional Transforms: Link transforms with conditional logic

Dynamic Linking

Dynamic and flexible transform connections:

• Runtime Linking: Link transforms based on runtime conditions
• Template Linking: Use transform templates for dynamic linking
• Parameter-based Linking: Link transforms based on parameter values
• Context-aware Linking: Link transforms based on data context

Integration Linking

Integration with external systems and services:

• API Integration: Link transforms to external API services
• Service Integration: Integrate with external transformation services
• Workflow Integration: Link to external workflow engines
• Event-driven Linking: Link transforms to event streams

Link Transform Endpoint

To link a transform to a processing component:

POST /transforms/{transform_id}/link

Nexla API

Link Transform: Request

POST /transforms/6001/link

Link Request Examples

Different types of transform linking for various integration needs.

Basic Transform Linking

Link a transform to a Nexset:

Nexla API

Basic Transform Linking: Request

{
  "target_type": "nexset",
  "target_id": 3001,
  "link_config": {
    "execution_order": 1,
    "enabled": true,
    "parameters": {
      "enrichment_enabled": true,
      "score_threshold": 0.7
    }
  }
}

Transform Chain Linking

Link multiple transforms in sequence:

Nexla API

Transform Chain Linking: Request

{
  "target_type": "nexset",
  "target_id": 3001,
  "transform_chain": [
    {
      "transform_id": 6001,
      "execution_order": 1,
      "enabled": true,
      "parameters": {
        "enrichment_enabled": true,
        "score_threshold": 0.7
      }
    },
    {
      "transform_id": 6002,
      "execution_order": 2,
      "enabled": true,
      "parameters": {
        "date_format": "ISO8601",
        "timezone": "UTC"
      }
    },
    {
      "transform_id": 6003,
      "execution_order": 3,
      "enabled": true,
      "parameters": {
        "calculation_type": "financial_metrics"
      }
    }
  ]
}

Conditional Transform Linking

Link transforms with conditional execution logic:

Nexla API

Conditional Transform Linking: Request

{
  "target_type": "nexset",
  "target_id": 3001,
  "conditional_links": [
    {
      "transform_id": 6001,
      "condition": {
        "field": "data_type",
        "operator": "equals",
        "value": "customer"
      },
      "execution_order": 1,
      "enabled": true,
      "parameters": {
        "enrichment_enabled": true
      }
    },
    {
      "transform_id": 6002,
      "condition": {
        "field": "data_type",
        "operator": "equals",
        "value": "financial"
      },
      "execution_order": 1,
      "enabled": true,
      "parameters": {
        "calculation_type": "financial_metrics"
      }
    }
  ]
}

Link Transform Response

A successful linking operation returns confirmation:

Nexla API

Link Transform: Response

{
  "status": "success",
  "message": "Transform linked successfully",
  "link_id": "link_7001",
  "link_info": {
    "transform_id": 6001,
    "target_type": "nexset",
    "target_id": 3001,
    "linked_at": "2023-01-16T15:00:00.000Z",
    "linked_by": 42
  },
  "link_config": {
    "execution_order": 1,
    "enabled": true,
    "parameters": {
      "enrichment_enabled": true,
      "score_threshold": 0.7
    }
  },
  "execution_plan": {
    "total_transforms": 1,
    "execution_sequence": [6001],
    "estimated_processing_time": "0.045s"
  }
}

Transform Chain Management

Manage complex transformation chains and sequences effectively.

Chain Configuration

Configure transform execution chains:

{
  "chain_config": {
    "name": "Customer Data Processing Chain",
    "description": "Complete customer data processing pipeline",
    "execution_mode": "sequential",
    "error_handling": "continue_on_error",
    "max_retries": 3,
    "timeout": 300000
  }
}

Chain Execution

Execute transform chains with proper orchestration:

Nexla API

Chain Execution: Request

{
  "chain_id": "chain_8001",
  "execution_config": {
    "mode": "sequential",
    "parallelism": 1,
    "error_handling": "stop_on_error",
    "monitoring": true
  }
}

Link Management Operations

Core operations for managing transform links and connections.

List Links

Retrieve all transform links:

GET /transforms/{transform_id}/links
GET /targets/{target_type}/{target_id}/transforms

Nexla API

List Links: Response

{
  "links": [
    {
      "link_id": "link_7001",
      "transform_id": 6001,
      "target_type": "nexset",
      "target_id": 3001,
      "execution_order": 1,
      "enabled": true,
      "linked_at": "2023-01-16T15:00:00.000Z",
      "status": "ACTIVE"
    },
    {
      "link_id": "link_7002",
      "transform_id": 6002,
      "target_type": "nexset",
      "target_id": 3001,
      "execution_order": 2,
      "enabled": true,
      "linked_at": "2023-01-16T15:30:00.000Z",
      "status": "ACTIVE"
    }
  ],
  "summary": {
    "total_links": 2,
    "active_links": 2,
    "inactive_links": 0
  }
}

Update Links

Modify existing transform links:

PUT /transforms/links/{link_id}

Nexla API

Update Link: Request

{
  "execution_order": 3,
  "enabled": false,
  "parameters": {
    "enrichment_enabled": false,
    "score_threshold": 0.5
  }
}

Unlink Transforms

Remove transform links:

DELETE /transforms/links/{link_id}

Nexla API

Unlink Transform: Response

{
  "status": "success",
  "message": "Transform unlinked successfully",
  "unlinked_at": "2023-01-16T16:00:00.000Z",
  "unlinked_by": 42
}

Link Configuration

Configure transform links for optimal performance and functionality.

Execution Configuration

Configure transform execution behavior:

{
  "execution_config": {
    "mode": "sequential",
    "parallelism": 4,
    "batch_size": 1000,
    "timeout": 300000,
    "retry_count": 3,
    "error_handling": "continue_on_error"
  }
}

Parameter Binding

Bind transform parameters to workflow variables:

{
  "parameter_binding": {
    "score_threshold": "${workflow.threshold}",
    "enrichment_enabled": "${workflow.enrichment_enabled}",
    "default_region": "${workflow.default_region}"
  }
}

Monitoring Configuration

Configure monitoring and alerting for linked transforms:

{
  "monitoring_config": {
    "metrics_enabled": true,
    "logging_level": "INFO",
    "alert_threshold": 0.95,
    "performance_monitoring": true,
    "error_tracking": true
  }
}

Link Best Practices

To effectively link transforms in your Nexla platform:

1. Plan Link Architecture: Design clear and logical transform linking structure
2. Manage Dependencies: Understand and manage transform dependencies
3. Monitor Performance: Track performance and resource usage of linked transforms
4. Handle Errors Gracefully: Implement proper error handling and recovery
5. Document Linkages: Maintain clear documentation of transform connections

Link Workflows

Implement structured workflows for effective transform linking.

Link Planning Workflow

Standard workflow for planning transform links:

1. Requirements Analysis: Analyze transformation requirements and workflow needs
2. Transform Selection: Select appropriate transforms for linking
3. Link Design: Design transform linking structure and execution order
4. Configuration Planning: Plan transform configuration and parameters
5. Testing Strategy: Plan testing approach for linked transforms

Link Implementation Workflow

Workflow for implementing transform links:

1. Link Creation: Create transform links with appropriate configuration
2. Configuration Setup: Configure transform parameters and settings
3. Testing: Test linked transforms with sample data
4. Deployment: Deploy linked transforms to production environment
5. Monitoring Setup: Set up monitoring and alerting for linked transforms

Error Handling

Common transform linking issues and solutions:

• Link Creation Failures: Verify transform and target compatibility
• Configuration Errors: Check parameter values and configuration format
• Execution Failures: Review transform logic and error handling
• Performance Issues: Optimize transform configuration and execution

Related Operations

After linking transforms, you may need to:

Monitor Links

GET /transforms/links/{link_id}/status
GET /transforms/links/{link_id}/performance

Execute Transforms

POST /transforms/{transform_id}/execute
POST /transforms/chains/{chain_id}/execute

Manage Links

PUT /transforms/links/{link_id}
DELETE /transforms/links/{link_id}