Azure Table Storage Destination

Azure Table Storage
Create an Azure Table Storage Destination
-
Click the + icon on the Nexset that will be sent to the Azure Table Storage destination, and select the Send to Destination option from the menu.
-
Select the Azure Table Storage connector from the list of available destination connectors. Then, select the credential that will be used to connect to the Azure Table Storage account, and click Next; or, create a new Azure Table Storage credential for use in this flow.
-
In Nexla, Azure Table Storage destinations can be created using pre-built endpoint templates, which expedite destination setup for common Azure Table Storage endpoints. Each template is designed specifically for the corresponding Azure Table Storage endpoint, making destination configuration easy and efficient.
• To configure this destination using a template, follow the instructions in Configure Using a Template.Azure Table Storage destinations can also be configured manually, allowing you to send data to Azure Table Storage endpoints not included in the pre-built templates or apply further customizations to exactly suit your needs.
• To configure this destination manually, follow the instructions in Configure Manually.
Configure Using a Template
Nexla provides pre-built templates that can be used to rapidly configure destinations to send data to common Azure Table Storage endpoints. Each template is designed specifically for the corresponding Azure Table Storage endpoint, making destination setup easy and efficient.
-
To configure this destination using a template, select the endpoint to which data will be sent from the Endpoint pulldown menu. Then, click on the template in the list below to expand it, and follow the instructions to configure additional endpoint settings.
Configure Manually
Azure Table Storage destinations can be manually configured to send data to any valid Table service REST API endpoint on the storage account.
Using manual configuration, you can also configure Nexla to automatically send the response received from the Azure Table Storage API after each call to a new Nexla webhook data source — useful for capturing ETags or per-entity error codes returned by the Table service.
API Method
-
To manually configure this destination, select the Advanced tab at the top of the configuration screen.
-
Select the API method that will be used for calls to the Azure Table Storage API from the Method pulldown menu. The Table service uses:
-
POST: To insert a new entity or to create a table.
-
PUT: To replace an entity (Update Entity, Insert Or Replace Entity), or to set table-level resources such as ACLs and service properties.
-
PATCH: To merge properties into an entity (Merge Entity, Insert Or Merge Entity). Azure also accepts the legacy
MERGEverb for the same operation. -
DELETE: To delete an entity or a table.
-
OPTIONS: To issue a CORS preflight request against a table resource.
-
Data Format
- Select the format in which the Nexset data will be sent to the Azure Table Storage API from the Content Format pulldown menu. Most Table service endpoints expect JSON (
application/json) — entity CRUD, table creation, and entity-group batch transactions are all JSON. Table-level configuration endpoints (Set Table ACL,Set Table Service Properties) expect XML (application/xml); select the XML format when targeting those endpoints.
API Endpoint URL
-
Enter the URL of the Azure Table Storage API endpoint to which you want to send the Nexset data in the URL field. All Table service URLs follow the form
https://{'{account}'}.table.core.windows.net/{'{resource}'}. Examples:-
https://{'{account}'}.table.core.windows.net/Tables— create a table (POST). -
https://{'{account}'}.table.core.windows.net/{'{table}'}(PartitionKey='{'{pk}'}',RowKey='{'{rk}'}')— entity-level CRUD. -
https://{'{account}'}.table.core.windows.net/{'{table}'}?comp=acl— set the table ACL.
-
Request Headers
-
If Nexla should include any additional request headers in API calls to this destination, enter the headers & corresponding values as comma-separated pairs in the Request Headers field (for example,
header1:value1,header2:value2). Common Table service headers used on destinations include:-
If-Match— Set to*for unconditional update/delete, or to a specific ETag for optimistic concurrency. -
Prefer— Usereturn-no-contentto suppress the response body on insert operations and reduce egress. -
x-ms-client-request-id— Attach a custom client correlation ID that appears in storage logs.
You do not need to include any headers already present in the credentials. The
Authorization: Bearerheader, thex-ms-versionheader (set from the credential's API Version (x-ms-version) field), and the standard Table serviceAccept,DataServiceVersion, andMaxDataServiceVersionheaders are added automatically. -
Exclude Attributes from the Call
-
If any record attributes in the Nexset should be omitted when sending data to this Azure Table Storage destination, select the attributes from the Exclude Attributes pulldown menu. This is useful for stripping internal Nexla bookkeeping fields or attributes that the Table service would reject (for example, OData control properties produced by upstream sources).
-
Any number of attributes can be selected for exclusion, and all excluded attributes will be shown in the field. To remove an attribute from the list, click the X icon next to the attribute name.
Record Batching
-
If records should be sent to this destination in batched API calls, check the box next to Would you like to batch your records together? to enable record batching.
-
Enter the maximum number of records that should be batched together in a single API call in the Batch Size field. By default, this value is set to
100.The Table service's native entity-group transactions accept a maximum of 100 entities per batch, and all entities in a batch must share the same
PartitionKey. If you are using Nexla's record batching to construct entity-group transactions, keep the batch size at or below100and ensure upstream partitioning produces same-partition groups. -
Select the algorithm that will be used to group records into batches from the Grouping Algorithm pulldown menu. The sample request shown in the panel on the right will be updated to reflect the current batching settings.
Response Webhook
Nexla can automatically send the response received from the Azure Table Storage API after each call to a new Nexla webhook data source. This option allows you to keep track of the status of each API call and any additional information returned after each call, such as the new entity ETag, the x-ms-request-id, or per-row error codes in entity-group transactions.
- To enable this option, check the box next to Would you like to process the API response as a Nexla Webhook source?.
Sample Request Payload
Sample request payloads containing a portion of the Nexset data that will be sent to the Azure Table Storage API endpoint based on the current settings are shown in the Sample Payload panel on the right. These samples can be referenced to ensure that the destination and request settings are correctly configured.
- Click on a sample request payload to expand it and view the complete payload content.
- Sample payloads are automatically updated with each setting change, making it easy to verify that changes achieve the desired effect.
Endpoint Testing (Manual Configuration)
After all endpoint settings have been configured, Nexla can send a test payload to the Azure Table Storage API to ensure that the destination is configured correctly.
-
To send a test payload, select the Test button at the top of the Sample Payload panel, and click on a listed sample payload to expand it.
-
If any modifications to the sample payload are needed, make the necessary changes directly within the sample window.
-
Click the Send Test Data button at the top of a sample payload to send the test payload to the Azure Table Storage API using the current settings.
Save & Activate the Destination
-
Once all endpoint settings have been configured, click the Done button in the upper right corner of the screen to save and create the destination. To begin sending data to the configured Azure Table Storage endpoint, open the destination resource menu, and select Activate.
The Nexset data will not be sent to the Azure Table Storage endpoint until the destination is activated. Destinations can be activated immediately or at a later time, providing full control over data movement.