BigQuery

BigQuery is Google Cloud Platform's analytical data warehouse. Euno's BigQuery integration discovers resources from BigQuery metadata and query history so teams can understand warehouse structure, lineage, and usage.

Euno's BigQuery integration supports auto-discovery of:

• GCP projects

• BigQuery databases and datasets

• BigQuery tables, views, materialized views, external tables, snapshots, and clones

• BigQuery columns, including nested STRUCT fields

• Google Sheets referenced by BigQuery external tables

• BigQuery usage, cost, and lineage properties from query history

• Hex project usage and lineage from BigQuery query history

• Optional Sigma usage from BigQuery query history

Prerequisites

Before you configure the integration, make sure you have:

• A GCP project that Euno can use to run BigQuery metadata queries. This is the execution project.

• The list of GCP projects Euno should discover. These are the discovered projects.

• Permission to create a GCP service account and service account key.

• Permission to grant IAM roles on every discovered project.

• The BigQuery regions where Euno should scan query history, for example us, eu, or us-central1.

To discover BigQuery resources, Euno queries system-defined, read-only INFORMATION_SCHEMA views. A BigQuery query runs under one project and can access tables and views in other projects, so the execution project can be different from the discovered projects.

Stage 1: Configure GCP

Step 1: Create a GCP Service Account on the Execution Project

1. In GCP, choose the execution project.

2. Browse to the Service Accounts page.

3. Click Create service account.

4. Enter a service account name, for example Euno BigQuery Integration.

5. Copy the service account email address, for example [e-mail].

6. Click Create and continue.

7. Under Permissions, choose the BigQuery User role. The technical identifier is roles/bigquery.user.

8. Click Continue, then click Done.

Step 2: Create a Service Account Key

1. Go to the service account page for the service account you created.

2. On the Keys tab, click Add key, then click Create new key.

3. Choose JSON as the key type.

4. Click Create and save the file.

Step 3: Grant Access to Discovered Projects

For each discovered project:

1. Browse to the project's IAM configuration.

2. Click Grant access.

3. Under New principals, enter the service account email address from Step 1.

4. Under Assign roles, choose BigQuery Metadata Viewer. The technical identifier is roles/bigquery.metadataViewer.

5. Click Add another role and choose BigQuery Resource Viewer. The technical identifier is roles/bigquery.resourceViewer.

6. Click Save.

Stage 2: Configure New BigQuery Source in Euno

Step 1: Access the Sources Page

In Euno, open the Sources page and click Add New Source. Select BigQuery.

Step 2: General Configuration

Asterisk (*) means a mandatory field.

Configuration	Description
Service Account Key*	Copy the entire contents of the service account key JSON file and paste it here.
Query History Regions*	Comma-separated list of BigQuery regions to scan for query history, for example us-east1,us-west1.
Execution Project ID	GCP project ID used as the execution project. If not specified, Euno uses the project where the service account is defined.
Query Location	BigQuery location where Euno should run queries.

Step 3: Advanced Settings

Click Advanced to display additional configurations.

Configuration	Description
Discover dataset labels as meta	Discover BigQuery dataset labels as meta values for BigQuery dataset resources. Defaults to disabled.
Auto discover Sigma usage from BigQuery query history	Detect Sigma SQL footers in BigQuery query history and use them to populate Sigma usage.
Sigma usage lookback (days)	Number of days to look back when detecting Sigma usage from BigQuery query history. Defaults to 7.
Cost per slot per hour (USD)	Cost used for BigQuery cost calculations. Defaults to 0.04.
Project Discovery Pattern	Use regular expressions to allow or exclude specific GCP projects. .* includes or excludes all projects.
Column Observation - Dataset Pattern	Use regular expressions to allow or exclude specific datasets from column observation. .* includes or excludes all datasets.
Column Observation - Table Pattern	Use regular expressions in dataset.table format to allow or exclude specific tables from column observation. For example, bq_log\..* excludes all tables in the bq_log dataset.

Step 4: Schedule

• Enable the Schedule option.

• Choose:

  1. Weekly: Set specific days and times.

  2. Hourly: Define the interval in hours, for example every 8 hours.

Step 5: Resource Cleanup

• Immediate Cleanup: Remove resources not detected in the most recent successful source run.

• No Cleanup: Keep all resources indefinitely, even if they are no longer detected.

To keep your data relevant and free of outdated resources, Euno provides automatic Resource Cleanup options. These settings determine when a resource should be removed if it is no longer detected by a source run. For a detailed explanation, see Resource Sponsorship in Euno.

Step 6: Save Configuration

Click Test & Save to complete the setup.

What Euno Discovers

Euno discovers BigQuery resources including GCP projects, databases, datasets, tables, columns, Google Sheets referenced by external tables, and Hex projects detected from BigQuery query history. For detailed information about discovered resources and their properties, see BigQuery Integration Discovered Resources.