Bigquery

Euno's Bigquery integration supports auto-discovery of:

• Bigquery views

• Query statistics for usage information by Tableau and Thoughtspot

Setting up Bigquery integration

Overview

Bigquery is Google Cloud Platform's flagship analytical data platform. In Bigquery, data is stored in tables and views. Each table or view belongs to one Bigquery dataset, and each dataset belong to a Google Cloud Platform (GCP) Project. A GCP Project can have no datasets, one dataset, or more.

To discover Bigquery resources, Euno executes queries on the system-defined, read-only dynamic INFORMATION_SCHEMA. In Bigquery, a query runs under one Project, and can access tables and views of other Projects. In other words, there is no constraint that a query must run in some project P in order to query for tables in views in project P.

Euno utilizes this fact to allow users to configure the project under which to run queries, called execution project, and separately configure which GCP projects Euno should discover, called discovered projects.

To sum up:

• execution project - the GCP Project under which Euno's integration will execute queries

• discovered projects - the list of GCP Projects for which Euno's integration will discover resources

To set up Euno's Bigquery integration, complete the following stages:

1. Create a GCP Service Account, and a GCP Service Account key

2. Assign roles/permissions to the GCP Service Account to allow it to run queries on the execution project

3. Assign roles/permissions to the GCP Service Account to allow it access to all the discovered projects

4. Configure Euno's Bigquery integration

Step 1: Create a GCP Service Account

Step 2: Assign roles to the GCP Service Account to allow it to run queries on the execution project

1. Browser to GCP's console IAM configuration. Make sure the selected project is the execution project.

2. Click "GRANT ACCESS"

3. Under "New principals", enter the email address of the service account from stage 1

4. Under "Assign Roles", click the "select a role"

5. Locate the project "BigQuery", and pick the role "BigQuery User" (the technical identifier of this role is roles/bigquery.user)

6. Click "Save"

Step 3: Assign roles to the GCP Service Account to allow it to discover resources in the discovered projects`

For each discovered project, complete these steps

1. Browser to GCP's console IAM configuration. Make sure the selected project is the discovered project.

2. Click "GRANT ACCESS"

3. Under "New principals", enter the email address of the service account from stage 1

4. Under "Assign Roles", click the "select a role"

5. Locate the project "BigQuery", and pick the role "BigQuery Metadata Viewer" (the technical identifier of this role is roles/bigquery.metadataViewer)

6. Click "Save"

Step 4: Configure Euno's BigQuery integration

Asterik (*) means a mandatory field.

Configuration	Description
Service Account Key*	Paste the service account key here
Execution Project ID	The id of the GCP project used as execution project. If not specified, Euno will use the project under which the Service Account is defined.
Query Location	The BigQuery Location to run the queries under.
Discovered Projects	TODO - allow deny list of projects
Discover Views	Check this box to have Euno discover BigQuery views. Defaults to "true".
Discover dataset labels as meta	Check this box to have Euno discover dataset labels as meta for BigQuery datasets. Defaults to "true".
Discover table labels as meta	Check this box to have Euno discover table labels as meta for BigQuery datasets. Defaults to "true".

Step 5: Resource Cleanup

• Immediate Cleanup: Remove resources not detected in the most recent successful source integration run.

• No Cleanup: Keep all resources indefinitely, even if they are no longer detected.

To keep your data relevant and free of outdated resources, Euno provides automatic resource cleanup options. These settings determine when a resource should be removed if it is no longer detected by a source integration. For a detailed explanation on Euno's cleanup strategies, see: Resource Sponsorship in Euno.

Discovered Resources

The Euno-BigQuery integration discovers various resources including GCP projects, datasets, and views. For detailed information about discovered resources and their properties, see bigquery-integration-discovered-resources.