Skip to main content

Google BigQuery

For the version of this article pertaining to the legacy Nexla UI, click here.

Nexla's bi-directional connectors can both send data to and receive data from any data system. This means that once a user has created or gained access to a credential for any data system, building any data flow to ingest data from or send data to a location within that data system requires only a few simple steps.

1. Credentials​

This section provides information about and step-by-step instructions for creating a new Google BigQuery credential in Nexla.

Important: During data movement to/from BigQuery, Nexla will write temporary data in a location within your GCS. Therefore, the user associated with the credential provided in Step 2 below will need the following roles in the Google Cloud Project:

  • Storage Object Creator
  • Storage Object Viewer

1.1 Add a New Google BigQuery Credential​

  1. After selecting the data source/destination type, in the Authenticate.png screen, click AddANewCredential.png. This will open the Add New Credential window.

      AddNewCredential.png

  2. Enter a name for the credential in the Credential Name field.

      CredName.png

  3. Optional: Enter a description for the credential in the Credential Description field.

      CredDescription.png

  4. Select the method that Nexla should use to authenticate to the BigQuery account from the Authentication Type pulldown menu.

    The System User Authentication method is recommended, as it is best-suited for accessing your own data. It is also tied to the service account instead of individual user accounts.

      AuthenticationType.png

1.2 End User Authentication​

  1. Click Authorize.png.

  2. In the pop-up window that appears, select the Google account associated with the BigQuery account.

  3. Click Allow.png to allow Nexla to access the account.

  4. Enter the project ID to the BigQuery database from which files should be read in the Project ID field.

      ProjectID.png

  5. Section 1.4 provides information about advanced settings available for Google BigQuery credentials along with step-by-step instructions for configuring each setting.

    • To configure any desired additional advanced settings for this credential, continue to Section 1.4, and complete the relevant steps.

    • To create this credential without configuring any advanced settings, continue to Section 1.5.

1.3 System User Authentication​

  1. Click Choose_Credentials_File.png.

  2. Select and upload the service account credentials JSON file generated by Google Cloud IAM.

  3. Section 1.4 provides information about advanced settings available for Google BigQuery credentials along with step-by-step instructions for configuring each setting.

    • To configure any desired additional advanced settings for this credential, continue to Section 1.4, and complete the relevant steps.

    • To create this credential without configuring any advanced settings, continue to Section 1.5.

1.4 Advanced Settings​

This section covers optional advanced credential settings. To create the Google BigQuery credential without configuring advanced settings, skip to Section 1.5.

  1. Click AdvSettings.png to access additional available settings for the BigQuery credential.

      AdvSettings2.png

  • GCS Location for Staging Data

    Before moving data to/from BigQuery, Nexla sometimes stages the data in a temporary GCS location, which is automatically created by the platform if the user account associated with the credential has bucket/path creation permissions.

    1. To override bucket creation and specify a GCS location where Nexla will create temporary staging files, enter the location in the GCS Location for Staging Data field.

        GCS_Location.png

  • Data Staging

    Before moving data to/from BigQuery, Nexla sometimes stages the data in a temporary dataset, which is automatically created by the platform if the user account associated with the credential has dataset creation permissions.

    1. To override dataset creation and specify an existing dataset for use, enter that dataset in the Temporary Dataset field.

        TempDataset.png

1.5 Save and Create the Google BigQuery Credential​

  1. Once all of the relevant steps in the above sections have been completed, click Save.png at the bottom of the Add New Credential screen to save the credential and all entered information.

      Save2.png

  2. The newly added credential will now appear in a tile on the Authenticate.png screen and can be selected for use with a new data source or destination.

      CredentialsList.png

2. Data Source​

To ingest data from a Google BigQuery location, follow the instructions in Section 2 of Common Setup for Databases & Data Warehouses.

3. Data Destination​

To send data to a Google BigQuery location, follow the instructions in Section 3 of Common Setup for Databases & Data Warehouses.