Skip to main content

Heroku PostgreSQL Data Source

The Heroku PostgreSQL connector enables you to access Heroku Postgres databases via the Heroku Platform API, allowing you to retrieve add-on information, provision new databases, and access database credentials programmatically. This connector is particularly useful for applications that need to manage Heroku Postgres databases, retrieve database connection information, or automate database provisioning workflows. Follow the instructions below to create a new data flow that ingests data from a Heroku PostgreSQL source in Nexla.
heroku_postgresql_api.png

Heroku PostgreSQL

Create a New Data Flow​

  1. To create a new data flow, navigate to the Integrate section, and click the New Data Flow button. Then, select the desired flow type from the list, and click the Create button.

  2. Select the Heroku PostgreSQL connector tile from the list of available connectors. Then, select the credential that will be used to connect to the Heroku Platform API, and click Next; or, create a new Heroku PostgreSQL credential for use in this flow.

  3. In Nexla, Heroku PostgreSQL data sources can be created using pre-built endpoint templates, which expedite source setup for common Heroku Platform API endpoints. Each template is designed specifically for the corresponding Heroku API endpoint, making source configuration easy and efficient.
    • To configure this source using a template, follow the instructions in Configure Using a Template.

    Heroku PostgreSQL sources can also be configured manually, allowing you to ingest data from Heroku Platform API endpoints not included in the pre-built templates or apply further customizations to exactly suit your needs.
    • To configure this source manually, follow the instructions in Configure Manually.

Configure Using a Template​

Nexla provides pre-built templates that can be used to rapidly configure data sources to ingest data from common Heroku Platform API endpoints. Each template is designed specifically for the corresponding Heroku API endpoint, making data source setup easy and efficient.

Endpoint Settings​

  • Select the endpoint from which this source will fetch data from the Endpoint pulldown menu. Available endpoint templates are listed in the expandable boxes below. Click on an endpoint to see more information about it and how to configure your data source for this endpoint.

    Get Add-ons

    This endpoint retrieves a list of Heroku add-ons for a given Heroku application. Use this endpoint when you need to discover which add-ons (including PostgreSQL databases) are attached to a specific Heroku app, retrieve add-on metadata, or monitor add-on configurations.

    • Enter the name of your Heroku application in the Heroku App Name field. This is the name of the Heroku app for which you want to retrieve add-ons. You can find your app name in the Heroku dashboard or by using the Heroku CLI command heroku apps.
    • Enter a schedule in cron format in the Schedule field to specify when this endpoint should be executed. The schedule determines how frequently Nexla will retrieve add-on information. The default value is 0 * * * *, which runs the endpoint every hour. You can customize this to match your monitoring or data collection needs.

    This endpoint returns all add-ons attached to the specified Heroku application, including PostgreSQL databases and other add-on services. The response includes add-on metadata such as add-on names, IDs, plans, and configuration details. For more information about the Get Add-ons endpoint, refer to the Heroku Platform API Documentation.

    Create Heroku Postgres DB

    This endpoint provisions a new Heroku Postgres database add-on for a specified Heroku application. Use this endpoint when you need to programmatically create new PostgreSQL databases, automate database provisioning workflows, or set up databases on a schedule.

    • Enter the name of the Heroku application in the Heroku App Name field. This is the name of the Heroku app to which the new Postgres add-on will be attached. The app must exist in your Heroku account before you can attach a database to it.

    • Enter the Postgres plan name in the Postgres Plan field. Available plan names include hobby-dev (free tier for development), standard-0, premium-0, and other Heroku Postgres plan tiers. The plan determines the database size, performance characteristics, and pricing. Common plan names include:

      • hobby-dev: Free tier for development and testing
      • standard-0: Standard tier for production workloads
      • premium-0: Premium tier with enhanced performance

      You can find the complete list of available plans in the Heroku dashboard or Heroku documentation.

    • Enter a schedule in cron format in the Schedule field to specify when this endpoint should be executed. The schedule determines how frequently Nexla will attempt to create a new Postgres database. The default value is 0 3 * * *, which runs the endpoint once per day at 3 AM. Use this carefully, as creating databases too frequently may result in unnecessary costs.

    This endpoint creates a new Heroku Postgres add-on and attaches it to the specified application. Creating databases programmatically can result in charges to your Heroku account, so ensure you understand the pricing for the selected plan. The endpoint returns information about the newly created database, including the add-on ID and configuration details. For more information about the Create Heroku Postgres DB endpoint, refer to the Heroku Platform API Documentation.

    Get Credentials

    This endpoint retrieves credentials for a Heroku Postgres add-on, including database connection information such as host, port, database name, username, and password. Use this endpoint when you need to programmatically access database connection strings, retrieve credentials for database connections, or monitor database configuration changes.

    • Enter the name of the Heroku application in the Heroku App Name field. This is the name of the Heroku app that has the Postgres add-on attached to it.
    • Enter the Postgres add-on ID in the Postgres Add-on ID field. This is the identifier for the add-on instance, which typically follows the format HEROKU_POSTGRESQL_IVORY, HEROKU_POSTGRESQL_COLOR, or similar. You can find the add-on ID by using the "Get Add-ons" endpoint first, or by checking the Heroku dashboard under your app's settings and add-ons section.
    • Enter a schedule in cron format in the Schedule field to specify when this endpoint should be executed. The schedule determines how frequently Nexla will retrieve credentials. The default value is 0 6 * * *, which runs the endpoint once per day at 6 AM. You can adjust this based on how often you need to refresh or monitor database credentials.

    This endpoint returns database credentials including connection strings, host information, and authentication details. The credentials are sensitive information and should be handled securely. Heroku may rotate credentials periodically, so retrieving them on a schedule ensures you have the most current connection information. For more information about the Get Credentials endpoint, refer to the Heroku Platform API Documentation.

Endpoint Testing​

Once the selected endpoint template has been configured, Nexla can retrieve a sample of the data that will be fetched according to the current settings. This allows users to verify that the source is configured correctly before saving.

  • To test the current endpoint configuration, click the Test button to the right of the endpoint selection menu. Sample data will be fetched & displayed in the Endpoint Test Result panel on the right.

  • If the sample data is not as expected, review the selected endpoint and associated settings, and make any necessary adjustments. Then, click the Test button again, and check the sample data to ensure that the correct information is displayed.

Configure Manually​

Heroku PostgreSQL data sources can be manually configured to ingest data from any valid Heroku Platform API endpoint. Manual configuration provides maximum flexibility for accessing endpoints not covered by pre-built templates or when you need custom API configurations.

With manual configuration, you can also create more complex Heroku sources, such as sources that use chained API calls to fetch data from multiple endpoints or sources that require custom authentication headers or request parameters.

API Method​

  1. To manually configure this source, select the Advanced tab at the top of the configuration screen.

  2. Select the API method that will be used for calls to the Heroku Platform API from the Method pulldown menu. The most common methods are:

    • GET: For retrieving data from the API (e.g., listing add-ons, retrieving credentials)
    • POST: For creating resources or triggering actions (e.g., creating new databases)

API Endpoint URL​

  1. Enter the URL of the Heroku Platform API endpoint from which this source will fetch data in the Set API URL field. This should be the complete URL including the protocol (https://) and any required path parameters. Heroku Platform API endpoints typically follow the pattern https://api.heroku.com/apps/{app-name}/addons or https://api.heroku.com/apps/{app-name}/addons/{addon-id}/config.

Ensure the API endpoint URL is correct and accessible with your current credentials. You can test the endpoint using the Test button after configuring the URL. The URL should include the app name and any required path parameters.

Request Headers​

Required

The Heroku Platform API requires specific headers for authentication and content negotiation. You must include the following headers:

  • Enter the required headers in the Request Headers field as comma-separated pairs. For Heroku Platform API requests, you typically need:

    • Accept: application/vnd.heroku+json; version=3 - Specifies the API version and response format

    • Authorization: Bearer {your-api-token} - Authentication header (this is typically handled automatically by Nexla based on your credential configuration)

      The Authorization header with your Heroku API token is typically handled automatically by Nexla based on your credential configuration. However, you may need to include the Accept header manually if it's not automatically added. The Accept header specifies that you want JSON responses in the Heroku Platform API v3 format.

Path to Data​

Optional

If only a subset of the data that will be returned by API endpoint is needed, you can designate the part(s) of the response that should be included in the Nexset(s) produced from this source by specifying the path to the relevant data within the response. This is particularly useful when API responses contain metadata, pagination information, or other data that you don't need for your analysis.

For example, when a request call is used to fetch a list of add-ons, the API will typically return an array of add-on objects, along with metadata, in the response. By entering the path to the relevant data, you can configure Nexla to treat each element of the returned array as a record.

Path to Data is essential when API responses have nested structures. Without specifying the correct path, Nexla might not be able to properly parse and organize your data into usable records. For Heroku API responses, common paths include $[*] for arrays of objects or $.data[*] if the response is wrapped in a data object.

  • To specify which data should be treated as relevant in responses from this source, enter the path to the relevant data in the Set Path to Data in Response field.

    • For responses in JSON format enter the JSON path that points to the object or array that should be treated as relevant data. JSON paths use dot notation (e.g., $[*] to access all elements in the root array).
    Path to Data Example:

    If the API response is in JSON format and includes a root-level array of add-on objects, the path to the response would be entered as $[*].

Autogenerate Path Suggestions​

Nexla can also autogenerate data path suggestions based on the response from the API endpoint. These suggested paths can be used as-is or modified to exactly suit your needs.

  • To use this feature, click the Test button next to the Set API URL field to fetch a sample response from the API endpoint. Suggested data paths generated based on the content & format of the response will be displayed in the Suggestions box below the Set Path to Data in Response field.

  • Click on a suggestion to automatically populate the Set Path to Data in Response field with the corresponding path. The populated path can be modified directly within the field if further customization is needed.

Endpoint Testing​

After configuring all settings for the selected endpoint, Nexla can retrieve a sample of the data that will be fetched according to the current configuration. This allows users to verify that the source is configured correctly before saving.

  • To test the current endpoint configuration, click the Test button to the right of the endpoint selection menu. Sample data will be fetched & displayed in the Endpoint Test Result panel on the right.

  • If the sample data is not as expected, review the selected endpoint and associated settings, and make any necessary adjustments. Then, click the Test button again, and check the sample data to ensure that the correct information is displayed.

Save & Activate the Source​

  1. Once all of the relevant steps in the above sections have been completed, click the Create button in the upper right corner of the screen to save and create the new Heroku PostgreSQL data source. Nexla will now begin ingesting data from the configured endpoint and will organize any data that it finds into one or more Nexsets.