Skip to main content

Create Nexset

Creating Nexsets in Nexla allows you to build custom data processing workflows that transform, enrich, and structure your data according to your specific business requirements and processing logic.

Nexset Creation Overview​

Nexset creation is the foundation of data transformation in Nexla, enabling you to design sophisticated processing pipelines that combine schemas, transforms, and business logic to produce high-quality, structured data outputs.

Create Nexset Endpoint​

To create a new Nexset:

POST /nexsets
Create Nexset: Request
POST /nexsets

{
  "name": "Customer Data Enrichment",
  "description": "Enrich customer data with demographic and behavioral information",
  "input_schema_id": 2001,
  "output_schema_id": 2002,
  "transforms": [3001, 3002],
  "config": {
    "processing_mode": "streaming",
    "batch_size": 1000,
    "error_handling": "continue_on_error"
  }
}

Create Response​

A successful creation returns the complete Nexset configuration:

Create Nexset: Response
{
  "id": 3001,
  "name": "Customer Data Enrichment",
  "description": "Enrich customer data with demographic and behavioral information",
  "input_schema_id": 2001,
  "output_schema_id": 2002,
  "transforms": [3001, 3002],
  "config": {
    "processing_mode": "streaming",
    "batch_size": 1000,
    "error_handling": "continue_on_error"
  },
  "status": "INACTIVE",
  "created_at": "2023-01-15T10:00:00.000Z",
  "updated_at": "2023-01-15T10:00:00.000Z",
  "owner": {
    "id": 42,
    "full_name": "John Smith"
  }
}

Required Fields​

Essential fields for Nexset creation:

  • name: Unique identifier for the Nexset
  • input_schema_id: ID of the input data schema
  • output_schema_id: ID of the output data schema
  • transforms: Array of transform IDs to apply

Optional Fields​

Additional configuration options:

  • description: Human-readable description of the Nexset
  • config: Processing configuration and parameters
  • tags: Metadata tags for organization and search
  • metadata: Additional custom metadata

Nexset Configuration​

Configure processing behavior and parameters for your Nexset.

Processing Mode​

Define how the Nexset processes data:

  • streaming: Process data in real-time streams
  • batch: Process data in batches
  • hybrid: Combine streaming and batch processing

Error Handling​

Configure error handling behavior:

  • continue_on_error: Continue processing despite errors
  • stop_on_error: Stop processing when errors occur
  • retry_on_error: Retry failed processing attempts

Performance Settings​

Optimize processing performance:

  • batch_size: Number of records processed per batch
  • parallelism: Number of parallel processing threads
  • memory_limit: Maximum memory allocation for processing

Schema Configuration​

Configure input and output schemas for your Nexset.

Input Schema​

Define the structure of incoming data:

  • Schema ID: Reference to existing schema definition
  • Field Mapping: Map input fields to processing logic
  • Validation Rules: Validate incoming data structure
  • Data Types: Ensure proper data type handling

Output Schema​

Define the structure of processed data:

  • Schema ID: Reference to output schema definition
  • Field Definitions: Define output field structure
  • Data Types: Specify output data types
  • Constraints: Apply output data constraints

Transform Integration​

Integrate transforms with your Nexset processing workflow.

Transform Selection​

Choose appropriate transforms for your processing needs:

  • Data Transformation: Modify data structure and content
  • Data Enrichment: Add context and reference information
  • Data Validation: Ensure data quality and compliance
  • Business Logic: Implement business rules and calculations

Transform Order​

Configure the order of transform application:

  • Sequential Processing: Apply transforms in sequence
  • Parallel Processing: Apply transforms in parallel
  • Conditional Processing: Apply transforms based on conditions
  • Branching Logic: Implement complex processing workflows

Advanced Configuration​

Implement advanced Nexset features and capabilities.

Custom Processing Logic​

Implement custom processing behavior:

  • Business Rules: Apply domain-specific business logic
  • Custom Algorithms: Implement specialized processing algorithms
  • Integration Logic: Handle external system integrations
  • Complex Workflows: Implement multi-step processing workflows

Performance Optimization​

Optimize Nexset performance:

  • Resource Allocation: Allocate appropriate processing resources
  • Caching Strategy: Implement effective data caching
  • Parallel Processing: Use parallel processing for large datasets
  • Memory Management: Optimize memory usage and allocation

Nexset Creation Workflow​

Implement structured workflows for effective Nexset creation.

Planning Phase​

Plan your Nexset requirements and design:

  1. Requirement Analysis: Define processing requirements and logic
  2. Schema Design: Design input and output schemas
  3. Transform Selection: Choose appropriate transforms
  4. Configuration Planning: Plan processing configuration

Implementation Phase​

Implement your Nexset configuration:

  1. Schema Creation: Create or reference input and output schemas
  2. Transform Development: Develop or select required transforms
  3. Configuration Setup: Configure processing parameters
  4. Integration Testing: Test Nexset integration and functionality

Validation Phase​

Validate your Nexset configuration and functionality:

  1. Configuration Validation: Validate configuration parameters
  2. Schema Validation: Verify schema compatibility and structure
  3. Transform Validation: Test transform functionality and performance
  4. Integration Testing: Test end-to-end processing workflows

Nexset Creation Best Practices​

To effectively create Nexsets in your Nexla platform:

  1. Plan Thoroughly: Carefully plan requirements and design before implementation
  2. Design for Performance: Design Nexsets with performance and scalability in mind
  3. Implement Error Handling: Build robust error handling and recovery mechanisms
  4. Test Comprehensively: Test functionality and performance in development environments
  5. Document Clearly: Maintain clear documentation of configuration and logic

Error Handling​

Common Nexset creation issues and solutions:

  • Schema Mismatch: Ensure input and output schemas are compatible
  • Transform Errors: Verify transform configuration and functionality
  • Configuration Issues: Check configuration parameter values and formats
  • Permission Denied: Ensure you have appropriate creation rights

After creating a Nexset, you may need to:

Activate the Nexset​

PUT /nexsets/{nexset_id}/activate

Test Processing​

POST /nexsets/{nexset_id}/test
GET /nexsets/{nexset_id}/samples

Monitor Performance​

GET /nexsets/{nexset_id}/metrics
GET /nexsets/{nexset_id}/status

Update Configuration​

PUT /nexsets/{nexset_id}
PUT /nexsets/{nexset_id}/config