Skip to main content

Buildkite

Buildkite is a hybrid CI/CD platform that enables engineering teams to run fast, secure, and scalable continuous integration and continuous delivery pipelines on their own infrastructure. Unlike fully cloud-hosted solutions, Buildkite's hybrid architecture lets teams execute pipeline jobs on self-hosted agents while Buildkite's cloud service handles orchestration, scheduling, and real-time visibility. Buildkite supports parallel builds, multi-platform deployments, test analytics, and integrations with source control systems including GitHub, GitLab, and Bitbucket. It is trusted by high-scale engineering teams at organizations such as Shopify, Shopify, Canva, Lyft, and Uber.

Buildkite icon

Power end-to-end data operations for your Buildkite API with Nexla. Our bi-directional Buildkite connector is purpose-built for Buildkite, making it simple to ingest data, sync it across systems, and deliver it anywhere — all with no coding required. Nexla turns API-sourced data into ready-to-use, reusable data products and makes it easy to send data to Buildkite or any other destination. With comprehensive monitoring, lineage tracking, and access controls, Nexla keeps your Buildkite workflows fast, secure, and fully governed.

Features

Type: API

SourceDestination

  • Seamless API Integration: Connect to any endpoint as source or destination without coding, with automatic data product creation
  • Visual Composition & Chaining: Build complex integrations using visual templates, chain API calls, and compose workflows with data validation and filtering
  • API Proxy: Expose curated slices of your data securely with a secure and customizable API proxy that validates and transforms data on the fly
  • Request optimization with intelligent batching, retry, and caching to minimize API calls and costs

Prerequisites​

To connect Nexla to Buildkite, you need a Buildkite API access token with the appropriate permissions for the operations you intend to perform. Buildkite API access tokens are user-scoped personal tokens that authenticate REST API requests using Bearer token authentication.

Generate a Buildkite API Access Token​

Buildkite API access tokens control access to your organization's data and pipelines. Tokens can be scoped to specific permissions (read, write, delete) for each Buildkite platform feature, ensuring that Nexla receives only the access it requires.

  1. Sign in to your Buildkite account at buildkite.com.

  2. Select your user profile icon in the top-right corner of the page, then select Personal Settings from the dropdown menu.

  3. In the Personal Settings navigation, select API Access Tokens.

  4. On the API Access Tokens page, select New API Access Token to open the token creation form.

  5. Enter a descriptive name in the Description field to identify this token's purpose — for example, Nexla Integration.

  6. Under Organization Access, select the specific Buildkite organization that Nexla should access. A token can be scoped to one or more organizations.

  7. Set a Token Expiry duration appropriate to your organization's security policies. Buildkite recommends rotating tokens at least once a year following OWASP best practices.

  8. Under REST API Scopes, select the permissions required for your intended Nexla use case:

    • For data source (reading data): Enable read permissions for Builds, Pipelines, Organizations, Jobs, Artifacts, and Agents as needed.

    • For destination (writing data): Enable read and write permissions for Builds and Pipelines to allow Nexla to trigger builds and create pipeline resources.

    • Recommended minimum for most integrations: Read permissions on Builds, Pipelines, and Organizations cover the majority of data ingestion use cases.

    You can use the Presets feature in the token creation form to quickly select all Read Only, all Read + Write, or all Full Access (including delete) permissions. For most Nexla integrations, Read Only or Read + Write presets are appropriate. Full Access (which includes delete permissions) should be granted only when required.

  9. Select Create New API Access Token. Enter your Buildkite password again if prompted.

  10. On the confirmation page, copy your new API access token immediately — this is the only time Buildkite will display the token value.

Important

Copy and securely store your API access token before leaving the confirmation page. Buildkite does not display the token value again after this page. If the token is lost, you must revoke it and generate a new one.

For complete information about Buildkite API access token scopes and permissions, see the Buildkite Managing API Access Tokens documentation.

Authenticate​

Create a credential in Nexla​

  1. After selecting the data source/destination type, click the Add Credential tile to open the Add New Credential overlay.

  2. Enter a name for the credential in the Credential Name field and a short, meaningful description in the Credential Description field.

  3. Enter your Buildkite API access token in the API Key field. This token authenticates all Nexla requests to the Buildkite REST API using Bearer token authentication. Nexla will include this token in the Authorization: Bearer header with each API call.

    Keep your API access token secure. Treat it as you would a password — do not share it or commit it to source control. You can revoke and regenerate tokens at any time from the Buildkite Personal Settings > API Access Tokens page.

  4. Click the Save button at the bottom of the overlay. The newly added credential will now appear in a tile on the Authenticate screen during data source/destination creation.

Use as a data source​

To create a new data flow, navigate to the Integrate section, and click the New Data Flow button. Select the Buildkite connector tile, then select the credential that will be used to connect to the Buildkite instance, and click Next; or, create a new Buildkite credential for use in this flow.

Endpoint templates​

Nexla provides pre-built templates that can be used to rapidly configure data sources to ingest data from common Buildkite endpoints. Select the endpoint from which this source will fetch data from the Endpoint pulldown menu. Available endpoint templates are listed in the expandable boxes below. Click on an endpoint to see more information about it and how to configure your data source for this endpoint.

List Organizations

Returns a list of all organizations accessible to the authenticated user. Use this endpoint to enumerate the Buildkite organizations your API token has access to.

Supports pagination via page and per_page query parameters (maximum 100 results per page). This endpoint requires a valid API access token with organization read permissions.

List Test Suites Analytics

Returns analytics data for test suites within an organization. Use this endpoint to monitor test suite health and track trends in test execution across your CI/CD pipelines.

Test Analytics must be enabled for your Buildkite organization and plan. The organization slug is case-sensitive and must match the slug shown in your Buildkite dashboard URL.

List Organization Pipelines

Returns a list of all pipelines in an organization. Use this endpoint to retrieve pipeline configurations and metadata for all CI/CD pipelines defined in a Buildkite organization.

Pagination is supported via page and per_page query parameters. Results include pipeline settings, steps, and repository information.

Get Current Access Token

Returns details about the current access token and its associated permissions. Use this endpoint to verify which scopes and organizations your API token can access.

This endpoint is useful for validating credential configuration and diagnosing permission issues before setting up other endpoints.

List All Builds

Returns a list of all builds across all organizations and pipelines accessible to the authenticated token. Use this endpoint for a unified view of build activity across your entire Buildkite account.

Supports filtering by state, branch, and date range using query parameters such as state, branch, created_from, and finished_from. Use date/time macros in the URL to implement incremental ingestion.

List Organization Clusters

Returns a list of all clusters in an organization. Use this endpoint to retrieve information about agent clusters configured for your Buildkite organization.

Clusters are a Buildkite feature for grouping agents. This endpoint requires the Clusters feature to be enabled on your Buildkite plan.

List Organization Builds

Returns a list of all builds in an organization. Use this endpoint to retrieve build history for all pipelines within a specific Buildkite organization.

Supports filtering by build state, branch, and date range. Use date/time macros with created_from or finished_from query parameters for incremental data ingestion.

List Pipeline Builds

Returns a list of all builds in a specific pipeline. Use this endpoint when you need build history scoped to a single pipeline rather than across the entire organization.

The pipeline slug is found in the Buildkite dashboard URL when viewing the pipeline, e.g., https://buildkite.com/my-org/my-pipeline → pipeline slug is my-pipeline.

List Cluster Queues

Returns a list of all queues in a cluster. Use this endpoint to retrieve agent queue definitions and configurations within a specific Buildkite cluster.

Cluster IDs can be retrieved using the List Organization Clusters endpoint. This endpoint requires the Clusters feature to be enabled on your Buildkite plan.

List Cluster Tokens

Returns a list of all tokens associated with a cluster. Use this endpoint to audit agent tokens configured for a specific Buildkite cluster.

Token values themselves are redacted in API responses for security. This endpoint returns token metadata such as description and allowed IP addresses.

List Organization Emojis

Returns a list of custom emojis available in the organization. Use this endpoint to retrieve the custom emoji catalog defined for your Buildkite organization.

This endpoint returns both built-in and custom emojis. Custom emojis are defined at the organization level in Buildkite settings.

Get Current User

Returns the current authenticated user's profile information. Use this endpoint to retrieve the Buildkite user account associated with the API token in use.

This endpoint returns the user associated with the API token, including name, email, and avatar URL. It is useful for validating which user account is associated with the configured credential.

Once the selected endpoint template has been configured, click the Test button to the right of the endpoint selection menu to retrieve a sample of the data that will be fetched. Sample data will be displayed in the Endpoint Test Result panel on the right, allowing you to verify that the source is configured correctly before saving.

Manual configuration​

Buildkite data sources can also be manually configured to ingest data from any valid Buildkite REST API endpoint, including endpoints not covered by the pre-built templates, chained API calls, or custom request parameters. Select the Advanced tab at the top of the configuration screen, and follow the instructions in Connect to Any API to configure the API method, endpoint URL, date/time and lookup macros, path to data, metadata, and request headers.

The Buildkite REST API base URL is https://api.buildkite.com/v2 and provides access to builds, pipelines, organizations, jobs, agents, artifacts, and more. All endpoints are accessed over HTTPS. The API supports pagination using page and per_page query parameters, with a maximum of 100 results per page. Most list endpoints return a top-level JSON array, so use $[*] as the path to data (or $.builds[*] when the array is nested under a key). Use date/time macros with the created_from, created_to, finished_from, or finished_to query parameters on the Builds endpoint for incremental ingestion. You do not need to include the Authorization header — it is handled automatically by Nexla using the API key stored in your Buildkite credential.

Buildkite API Rate Limits

The Buildkite REST API enforces rate limits with a 60-second window. Rate limit status is returned in response headers: RateLimit-Remaining, RateLimit-Limit, and RateLimit-Reset. If your Nexla data flow encounters rate limit errors, consider increasing the polling interval or reducing the scope of each API request using pagination parameters (page and per_page, with a maximum of 100 results per page).

Once all of the relevant settings have been configured, click the Create button in the upper right corner of the screen to save and create the new Buildkite data source. Nexla will now begin ingesting data from the configured endpoint and will organize any data that it finds into one or more Nexsets.

Use as a destination​

Click the + icon on the Nexset that will be sent to the Buildkite destination, and select the Send to Destination option from the menu. Select the Buildkite connector from the list of available destination connectors, then select the credential that will be used to connect to the Buildkite organization, and click Next; or, create a new Buildkite credential for use in this flow.

Manual configuration​

Buildkite destinations can be manually configured to send data to any valid Buildkite REST API endpoint. This allows you to trigger builds, create pipelines, update pipeline configurations, or push data to any writable Buildkite resource. Select the Advanced tab at the top of the configuration screen, and follow the instructions in Connect to Any API to configure the API method, data format, endpoint URL, request headers, attribute exclusions, record batching, and response webhooks. You can also configure Nexla to automatically send the response received from the Buildkite API after each call to a new Nexla webhook data source.

The Buildkite REST API base URL is https://api.buildkite.com/v2, and the API accepts and returns data in JSON format — select JSON as the content format. All requests must be made over HTTPS. Common write operations include triggering a new build, creating or updating a pipeline, and creating a build annotation. For update and upsert operations, include the relevant identifier (such as the pipeline slug or build number) at the end of the URL. You do not need to include the Authorization or Content-Type headers — they are handled automatically by Nexla using the API key stored in your Buildkite credential. For complete documentation on available endpoints and required request body parameters, see the Buildkite REST API reference.

Test Payload Caution

Sending a test payload to write endpoints (such as triggering a build) will create real resources in your Buildkite organization. Review the sample payload carefully before sending, and consider using a non-production pipeline or organization for initial testing.

Save & activate​

Once all endpoint settings have been configured, click the Done button in the upper right corner of the screen to save and create the destination. To begin sending data to the configured Buildkite endpoint, open the destination resource menu and select Activate.

The Nexset data will not be sent to the Buildkite API until the destination is activated. Destinations can be activated immediately or at a later time, providing full control over data movement.