0 / 0

Google BigQuery connector

Last updated: Aug 07, 2025
Google BigQuery connector

Google BigQuery is a fully managed, server less, AI-ready data warehouse that enables scalable analysis over petabyte of data.

Prerequisite

Create the connection. For instructions, see Connecting to a data source in DataStage® and the Google BigQuery connection.

Configuring the Google BigQuery as a source

Configure the read process in Google BigQuery connector.

Table 1. Reading data from Google BigQuery
Read mode Procedure
Call procedure statement
  1. Specify the query timeout.
  2. Provide the SQL procedure statement.
General
  1. Specify the project ID.
  2. Enter a value of Dataset name and Table name.
  3. Specify the query timeout.
Select statement If you choose Use GCS staging:
  1. Specify the Google Cloud Storage bucket name.
  2. Specify the project ID.
  3. Enter a value of the Dataset name.
  4. Specify the query timeout.
  5. Provide the SQL select statement.
You can select the Enable partitioned reads option.

Configuring the Google BigQuery as a target

Configure the write process in Google BigQuery connector.

Table 2. Writing data to Google BigQuery
Write mode Procedure
Call procedure statement
  1. Specify the project ID.
  2. Choose table action: Append, Replace, or Truncate.
  3. Specify key column names.
  4. Specify the Google Cloud Storage name.
  5. Provide a file name prefix.
  6. Provide the SQL procedure statement.
Delete
  1. Specify the project ID.
  2. Enter a value of Dataset name and Table name.
  3. Specify key column names.
  4. Specify the Google Cloud Storage name.
  5. Provide a file name prefix.
Delete then Insert
  1. Specify the project ID.
  2. Enter a value of Dataset name and Table name.
  3. Choose table action: Append, Replace, or Truncate.
  4. Specify key column names.
  5. Specify the Google Cloud Storage name.
  6. Provide a file name prefix.
Insert
  1. Specify the project ID.
  2. Enter a value of Dataset name and Table name.
  3. Choose table action: Append, Replace, or Truncate.
  4. Specify the Google Cloud Storage name.
  5. Provide a file name prefix.
Merge
  1. Specify the project ID.
  2. Enter a value of Dataset name and Table name.
  3. Choose table action: Append, Replace, or Truncate.
  4. Specify key column names.
  5. Specify the Google Cloud Storage name.
  6. Provide a file name prefix.
Streaming Insert
  1. Specify the project ID.
  2. Enter a value of Dataset name and Table name.
  3. Choose table action: Append, Replace, or Truncate.
Update
  1. Specify the project ID.
  2. Enter a value of Dataset name and Table name.
  3. Specify key column names.
  4. Specify the Google Cloud Storage name.
  5. Provide a file name prefix.
Update Statement
  1. Specify the Google Cloud Storage name.
  2. Provide a file name prefix.
  3. Provide the SQL update statement.

Limitations

You can face some limitations while you use Google BigQuery connector.

Google BigQuery reaches the limit of nested view levels

While you run large DataStage flows in ELT mode, you can face the problem of reaching the limit of nested view levels.

Google BigQuery supports up to 16 levels of nested views. You can create views up to this limit, but querying is limited to 15 levels. If the limit is exceeded, Google BigQuery returns an INVALID_INPUT error.

For more information, see: Quotas and limits reference (Google BigQuery).