Skip to main content

Create a Lookup

Creating lookups in Nexla allows you to establish reference data mappings that can be used to enrich and validate your data. Lookups can be created as either static mappings with predefined content or dynamic mappings that automatically update based on data flows.

Lookup Creation Methods​

You can create lookups in several ways depending on your requirements:

  • Static Creation: Create lookups with manually defined content and structure
  • Dynamic Creation: Create lookups that automatically update from data flows
  • Template-Based: Use predefined templates for common lookup patterns

Static Lookup Creation​

Static lookups contain manually created and maintained reference data that provides consistent lookup values across your data processing workflows.

API Endpoint​

The primary endpoint for creating static lookups is:

POST /data_maps

Required Fields​

When creating static lookups, you must specify:

  • name: A descriptive name for the lookup
  • data_type: The data type of the lookup values
  • map_primary_key: The field used as the primary key for matching
  • data_defaults: Default values returned when lookup fails
  • data_map: The actual lookup data as an array of objects

Static Lookup Example​

Create A Static Lookup: Request
POST /data_maps

{
  "name": "Event Code Lookup",
  "data_type": "string",
  "emit_data_default": true,
  "map_primary_key": "eventId",
  "data_defaults": {
    "eventId": "Unknown",
    "description": "Unknown Event",
    "category": "Unknown Category"
  },
  "data_map": [
    {
      "eventId": "0",
      "description": "Search Event",
      "category": "Web Activity"
    },
    {
      "eventId": "1",
      "description": "Checkout Event",
      "category": "E-commerce"
    },
    {
      "eventId": "2",
      "description": "Return Event",
      "category": "E-commerce"
    }
  ]
}

Response Structure​

A successful creation returns a complete lookup object with all configuration details and metadata.

Create A Static Lookup: Response
{
  "id": 1023,
  "owner": {
    "id": 42,
    "full_name": "John Smith"
  },
  "org": {
    "id": 101,
    "name": "Acme Corporation",
    "email_domain": "acme.com"
  },
  "access_roles": ["owner"],
  "name": "Event Code Lookup",
  "description": "Mapping event codes to descriptions and categories",
  "public_map": false,
  "managed": false,
  "data_type": "string",
  "data_format": null,
  "data_sink_id": null,
  "data_set_id": null,
  "emit_data_default": true,
  "use_versioning": false,
  "map_primary_key": "eventId",
  "data_defaults": {
    "eventId": "Unknown",
    "description": "Unknown Event",
    "category": "Unknown Category"
  },
  "map_entry_schema": {
    "properties": {
      "category": {
        "type": "string"
      },
      "description": {
        "type": "string"
      },
      "eventId": {
        "format": "integer",
        "type": "string"
      }
    },
    "type": "object",
    "$schema": "http://json-schema.org/draft-04/schema#",
    "$schema-id": 169361033
  },
  "updated_at": "2023-01-15T18:32:10.567Z",
  "created_at": "2023-01-15T18:32:10.567Z",
  "tags": []
}

Dynamic Lookup Creation​

Dynamic lookups automatically update their content based on data flows, providing real-time reference data that reflects current system state.

Dynamic Lookup Configuration​

To create a dynamic lookup, include a valid data sink ID in the data_sink_id attribute of the POST request body.

Key Differences from Static Lookups​

  • Auto-Updating Content: Mapping content is automatically generated when data is written to the data sink
  • No Manual Content: The data_map attribute cannot be explicitly set with POST or PUT requests
  • Flow Integration: Content updates are driven by data processing workflows
  • Real-Time Data: Lookup content reflects current data state

Dynamic Lookup Example​

Create A Dynamic Lookup: Request
{
  "name": "Product Inventory Lookup",
  "data_type": "string",
  "emit_data_default": true,
  "map_primary_key": "productId",
  "data_defaults": {
    "productId": "Unknown",
    "stockLevel": "0",
    "lastUpdated": "Unknown"
  },
  "data_sink_id": 5014
}

Lookup Configuration Options​

Data Type and Format​

Configure the data structure and format for your lookup:

  • data_type: Specify the primary data type (string, integer, number)
  • data_format: Define any specific format requirements
  • map_entry_schema: Provide detailed schema definition for complex lookups

Default Value Handling​

Configure how the lookup behaves when matches fail:

  • emit_data_default: Whether to return default values on failed lookups
  • data_defaults: The default values to return when lookup fails
  • use_versioning: Whether to support lookup versioning

Access Control​

Configure lookup visibility and access:

  • public_map: Whether the lookup is publicly accessible
  • access_roles: User access permissions for the lookup
  • tags: Organizational tags for lookup management

Best Practices​

To effectively create and manage lookups:

  1. Use Descriptive Names: Choose clear names that identify the purpose and content
  2. Include Descriptions: Add descriptions to clarify lookup usage and purpose
  3. Design Primary Keys: Choose primary keys that provide efficient lookup performance
  4. Plan Default Values: Define meaningful default values for failed lookups
  5. Consider Performance: Design lookup structure for optimal performance

Error Handling​

Common creation issues and solutions:

  • Invalid Primary Key: Ensure the map_primary_key corresponds to a field in your data_map
  • Missing Required Fields: Verify all mandatory parameters are provided
  • Invalid Data Structure: Check that data_map format matches the expected schema
  • Permission Issues: Ensure your account can create lookups in the organization

Post-Creation Steps​

After successfully creating a lookup, you should:

  1. Test Functionality: Verify that the lookup works correctly with sample data
  2. Validate Content: Check that lookup entries contain expected data
  3. Configure Integration: Set up the lookup in your data transformation workflows
  4. Monitor Performance: Track lookup performance and usage patterns

After creating lookups, you may need to:

Update Lookup Content​

PUT /data_maps/{lookup_id}

Manage Lookup Entries​

PUT /data_maps/{lookup_id}/entries
DELETE /data_maps/{lookup_id}/entries/{entry_key}

Test Lookup Functionality​

GET /data_maps/{lookup_id}/entries/{entry_key}
POST /data_maps/{lookup_id}/probe/sample

Download Lookup Content​

GET /data_maps/{lookup_id}/download_map