Activate and Pause Data

Controlling data processing in Nexla allows you to activate and pause Nexsets to manage data flow, perform maintenance, and optimize resource utilization across your data processing infrastructure.

Data Control Overview

Data control provides granular management of your data processing workflows, enabling you to start, stop, and pause data processing operations based on operational requirements, maintenance schedules, and performance optimization needs.

Core Control Capabilities

The data control system provides several key capabilities for managing data processing operations.

Flow Control

Manage data flow through your processing pipelines:

• Activation: Start data processing operations
• Pause: Temporarily stop data processing
• Resume: Restart paused processing operations
• Stop: Completely halt data processing

Resource Management

Optimize resource utilization:

• Capacity Planning: Manage processing capacity and resources
• Load Balancing: Distribute processing load across resources
• Performance Optimization: Optimize processing performance
• Maintenance Scheduling: Schedule maintenance windows

Activate Nexset

Start data processing operations for your Nexset.

Activation Endpoint

To activate a Nexset:

PUT /nexsets/{nexset_id}/activate

Nexla API

Activate Nexset: Request

PUT /nexsets/3001/activate

Activation Response

A successful activation returns confirmation:

Nexla API

Activate Nexset: Response

{
  "id": 3001,
  "name": "Customer Data Enrichment",
  "status": "ACTIVE",
  "activation_info": {
    "activated_at": "2023-01-16T17:00:00.000Z",
    "activated_by": 42,
    "previous_status": "INACTIVE"
  },
  "processing_config": {
    "processing_mode": "streaming",
    "batch_size": 1000,
    "parallelism": 4
  }
}

Pause Nexset

Temporarily stop data processing operations.

Pause Endpoint

To pause a Nexset:

PUT /nexsets/{nexset_id}/pause

Nexla API

Pause Nexset: Request

PUT /nexsets/3001/pause

Pause Response

A successful pause operation returns confirmation:

Nexla API

Pause Nexset: Response

{
  "id": 3001,
  "name": "Customer Data Enrichment",
  "status": "PAUSED",
  "pause_info": {
    "paused_at": "2023-01-16T18:00:00.000Z",
    "paused_by": 42,
    "previous_status": "ACTIVE",
    "reason": "Scheduled maintenance"
  },
  "queue_status": {
    "records_in_queue": 150,
    "processing_complete": true
  }
}

Resume Nexset

Restart paused data processing operations.

Resume Endpoint

To resume a paused Nexset:

PUT /nexsets/{nexset_id}/resume

Nexla API

Resume Nexset: Request

PUT /nexsets/3001/resume

Resume Response

A successful resume operation returns confirmation:

Nexla API

Resume Nexset: Response

{
  "id": 3001,
  "name": "Customer Data Enrichment",
  "status": "ACTIVE",
  "resume_info": {
    "resumed_at": "2023-01-16T19:00:00.000Z",
    "resumed_by": 42,
    "previous_status": "PAUSED",
    "queue_processed": true
  },
  "processing_status": {
    "processing_mode": "streaming",
    "current_batch_size": 1000,
    "processing_rate": 2500
  }
}

Stop Nexset

Completely halt data processing operations.

Stop Endpoint

To stop a Nexset:

PUT /nexsets/{nexset_id}/stop

Nexla API

Stop Nexset: Request

PUT /nexsets/3001/stop

Stop Response

A successful stop operation returns confirmation:

Nexla API

Stop Nexset: Response

{
  "id": 3001,
  "name": "Customer Data Enrichment",
  "status": "STOPPED",
  "stop_info": {
    "stopped_at": "2023-01-16T20:00:00.000Z",
    "stopped_by": 42,
    "previous_status": "ACTIVE",
    "reason": "Decommissioning"
  },
  "final_status": {
    "total_records_processed": 1500000,
    "final_processing_rate": 0,
    "queue_cleared": true
  }
}

Bulk Control Operations

Control multiple Nexsets simultaneously for efficient management.

Bulk Activation

Activate multiple Nexsets at once:

Nexla API

Bulk Activation: Request

PUT /nexsets/bulk_activate

{
  "nexset_ids": [3001, 3002, 3003],
  "activation_config": {
    "processing_mode": "streaming",
    "batch_size": 1000
  }
}

Bulk Pause

Pause multiple Nexsets simultaneously:

Nexla API

Bulk Pause: Request

PUT /nexsets/bulk_pause

{
  "nexset_ids": [3001, 3002, 3003],
  "pause_reason": "Scheduled maintenance window"
}

Bulk Response

Bulk operations return confirmation for all affected Nexsets:

Nexla API

Bulk Operation Response

{
  "status": "success",
  "message": "Bulk operation completed successfully",
  "operation": "activate",
  "affected_count": 3,
  "results": [
    {
      "id": 3001,
      "status": "ACTIVE",
      "result": "success"
    },
    {
      "id": 3002,
      "status": "ACTIVE",
      "result": "success"
    },
    {
      "id": 3003,
      "status": "ACTIVE",
      "result": "success"
    }
  ]
}

CLI Control Operations

Use the Nexla CLI to control Nexsets for efficient command-line management.

CLI Command Structure

Basic CLI commands for Nexset control:

nexla nexset activate <nexset_id>
nexla nexset pause <nexset_id>
nexla nexset resume <nexset_id>
nexla nexset stop <nexset_id>

CLI Options

Available CLI options for control operations:

• --reason: Specify reason for control operation
• --config: Provide configuration for activation
• --force: Force operation without confirmation
• --dry-run: Preview operation without executing

CLI Examples

Common CLI usage patterns for control operations:

Nexla CLI

CLI Control: Examples

# Activate a Nexset
nexla nexset activate 3001

# Pause with reason
nexla nexset pause 3001 --reason "Maintenance"

# Resume processing
nexla nexset resume 3001

# Stop processing
nexla nexset stop 3001 --reason "Decommissioning"

Control Considerations

Important considerations when controlling Nexset operations.

Data Integrity

Ensure data integrity during control operations:

• Queue Management: Handle records in processing queues
• State Persistence: Preserve processing state and progress
• Transaction Safety: Ensure safe transaction boundaries
• Recovery Planning: Plan for recovery after control operations

Impact Assessment

Assess the impact of control operations:

• Downstream Impact: Consider effects on dependent systems
• Data Flow Impact: Understand impact on data processing pipelines
• Performance Impact: Assess performance implications
• Resource Impact: Consider resource utilization effects

Control Workflows

Implement structured workflows for effective Nexset control.

Activation Workflow

Standard workflow for activating Nexsets:

1. Pre-Activation Check: Verify readiness for activation
2. Resource Verification: Ensure sufficient resources are available
3. Configuration Validation: Validate processing configuration
4. Activation Execution: Execute activation operation
5. Post-Activation Verification: Verify successful activation

Pause Workflow

Workflow for pausing Nexsets:

1. Pause Planning: Plan pause timing and duration
2. Queue Management: Handle records in processing queues
3. State Preservation: Preserve processing state and progress
4. Pause Execution: Execute pause operation
5. Status Verification: Verify successful pause

Control Best Practices

To effectively control Nexsets in your Nexla platform:

1. Plan Operations: Carefully plan control operations and timing
2. Assess Impact: Understand impact on data processing and downstream systems
3. Monitor Status: Monitor status during and after control operations
4. Document Procedures: Maintain clear control procedures and responses
5. Test Operations: Test control operations in development environments

Error Handling

Common control operation issues and solutions:

• Activation Failures: Check resource availability and configuration
• Pause Issues: Verify queue status and processing state
• Resume Problems: Check pause state and queue management
• Permission Denied: Ensure you have appropriate control rights

Related Operations

After controlling Nexsets, you may need to:

Monitor Status

GET /nexsets/{nexset_id}/status
GET /nexsets/{nexset_id}/metrics

View Processing History

GET /nexsets/{nexset_id}/history
GET /nexsets/{nexset_id}/audit

Update Configuration

PUT /nexsets/{nexset_id}/config
PUT /nexsets/{nexset_id}

Validate Processing

POST /nexsets/{nexset_id}/validate
GET /nexsets/{nexset_id}/samples