> For the complete documentation index, see [llms.txt](https://docs.euno.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.euno.ai/sources/data-warehouses/bigquery-integration/bigquery-integration-discovered-resources.md).

# BigQuery Integration Discovered Resources

## Overview

The BigQuery integration discovers GCP projects, BigQuery datasets, tables, views, and columns, and derives usage and lineage from BigQuery query history. It discovers the following resources:

* BigQuery project (`gcp_project`)
* Database (`database`)
* Dataset (`database_schema`)
* Table (`table`, subtypes `base_table`, `view`, `materialized_view`, `external_table`, `snapshot`, `clone`)
* Column (`column`, subtype `struct` for root STRUCT columns)
* Google Sheet (`google_sheet`)
* Hex project (`hex_project`), enriched with BigQuery usage and lineage

BigQuery warehouse resource URIs start with the prefix `bigquery`, followed by the GCP project ID. The GCP project itself uses `platform.gcp_project.<project_id>`. Generated URIs are lowercase.

## Resource hierarchy

```
BigQuery project (gcp_project)
└── Database (the BigQuery project)
    └── Dataset
        └── Table / View / Materialized view / External table / Snapshot / Clone
            └── Column
```

## BigQuery project (`gcp_project`)

The GCP project that holds the BigQuery datasets.

| Property    | Type   | Description          |
| ----------- | ------ | -------------------- |
| `name`      | string | The GCP project ID   |
| `type`      | string | Always `gcp_project` |
| `native_id` | string | The GCP project ID   |

**URI pattern**

```
platform.gcp_project.<project_id>
```

Example: `platform.gcp_project.proj_sales`

## Database (`database`)

In BigQuery, Euno represents each discovered GCP project as a `database` resource.

| Property         | Type   | Description                            |
| ---------------- | ------ | -------------------------------------- |
| `name`           | string | The GCP project ID                     |
| `type`           | string | Always `database`                      |
| `subtype`        | string | Always `database`                      |
| `native_id`      | string | The GCP project ID                     |
| `container_type` | string | Marks the database as a pure container |

**URI pattern**

```
bigquery.<project_id>
```

Example: `bigquery.proj_sales`

## Dataset (`database_schema`)

A BigQuery dataset within a project.

| Property              | Type      | Description                                                        |
| --------------------- | --------- | ------------------------------------------------------------------ |
| `name`                | string    | The BigQuery dataset name                                          |
| `type`                | string    | Always `database_schema`                                           |
| `subtype`             | string    | Always `database_schema`                                           |
| `created_at`          | timestamp | Date and time when the dataset was created                         |
| `updated_at`          | timestamp | Date and time when the dataset was last modified                   |
| `database_technology` | string    | Always `bigquery`                                                  |
| `database`            | string    | The GCP project ID. Derived.                                       |
| `container_type`      | string    | Marks the dataset as a pure container                              |
| `location`            | string    | The BigQuery dataset location                                      |
| `meta`                | object    | Dataset labels when **Discover dataset labels as meta** is enabled |

**URI pattern**

```
bigquery.<project_id>.<dataset>
```

Example: `bigquery.proj_sales.analytics`

## Table (`table`)

BigQuery tables, views, materialized views, external tables, snapshots, and clones are represented as `table` resources.

| Property                  | Type      | Description                                                                                                          |
| ------------------------- | --------- | -------------------------------------------------------------------------------------------------------------------- |
| `name`                    | string    | The BigQuery table, view, materialized view, external table, snapshot, or clone name                                 |
| `type`                    | string    | Always `table`                                                                                                       |
| `subtype`                 | string    | One of `base_table`, `view`, `materialized_view`, `external_table`, `snapshot`, or `clone`                           |
| `native_id`               | string    | Fully qualified BigQuery object ID in `project.dataset.TableName` format, preserving the casing reported by BigQuery |
| `description`             | string    | Table description when available                                                                                     |
| `created_at`              | timestamp | Date and time when the table was created                                                                             |
| `updated_at`              | timestamp | Date and time when table storage was last modified                                                                   |
| `native_last_data_update` | timestamp | Last refresh time for materialized views, snapshots, and clones when available                                       |
| `database_technology`     | string    | Always `bigquery`                                                                                                    |
| `database`                | string    | The GCP project ID. Derived.                                                                                         |
| `database_schema`         | string    | Parent **schema** (BigQuery dataset name)                                                                            |
| `table_properties`        | object    | View, materialized view, external table, or snapshot SQL when available                                              |
| `table_schema`            | object    | Structured schema with column names and data types                                                                   |
| `meta`                    | object    | Table labels when **Discover table labels as meta** is enabled                                                       |

**URI pattern**

```
bigquery.<project_id>.<dataset>.<table>
```

Example: `bigquery.proj_sales.analytics.orders_raw`

Copy the **Native ID** from the resource's **About** section. For example, `my-project.Sales.Opportunity` retains the dataset and table's original casing, while Euno's **URI** remains lowercase. When viewing a dbt source that defines the table, use the copy button beside the table's **Defines** link to copy the same BigQuery identifier.

Previously collected IDs may remain lowercase until the next successful scheduled BigQuery catalog crawl rediscovers the affected table and its metadata is processed. If a table is not successfully reobserved, its Native ID may remain unchanged. A dbt-only crawl does not refresh the BigQuery table's Native ID. Tables without a discovered Native ID have no copy control for that identifier.

### Usage and optimization properties

Usage and cost optimization properties are derived from BigQuery query history.

For base tables and materialized views, read and write usage properties are derived from `INFORMATION_SCHEMA.JOBS` and `INFORMATION_SCHEMA.JOBS_BY_PROJECT`. For `subtype = 'view'`, `total_read_queries_{14d,30d,60d}` reflects queries that referenced the view. Other read optimization properties, such as bytes, slots, and runtime, reflect underlying table processing and may differ.

#### Read Properties

| Property                                             | Type   | Description                                                                                                            |
| ---------------------------------------------------- | ------ | ---------------------------------------------------------------------------------------------------------------------- |
| `total_read_queries_14d` / `_30d` / `_60d`           | number | Total number of read queries referencing the resource over the last 14, 30, or 60 days. Applies to tables and columns. |
| `total_read_slots_14d` / `_30d` / `_60d`             | number | Total slots consumed by read queries referencing the table over the last 14, 30, or 60 days.                           |
| `average_read_slots_14d` / `_30d` / `_60d`           | number | Average slots consumed per read query referencing the table over the last 14, 30, or 60 days.                          |
| `total_read_bytes_processed_14d` / `_30d` / `_60d`   | number | Total bytes processed by read queries referencing the table over the last 14, 30, or 60 days.                          |
| `average_read_bytes_processed_14d` / `_30d` / `_60d` | number | Average bytes processed per read query referencing the table over the last 14, 30, or 60 days.                         |
| `total_read_runtime_14d` / `_30d` / `_60d`           | number | Total runtime of read queries referencing the table over the last 14, 30, or 60 days.                                  |
| `total_read_cost_14d` / `_30d` / `_60d`              | number | Estimated read cost over the last 14, 30, or 60 days, based on the configured cost per slot hour.                      |
| `average_read_cost_14d` / `_30d` / `_60d`            | number | Estimated average read cost per query over the last 14, 30, or 60 days.                                                |
| `distinct_users_14d` / `_30d` / `_60d`               | number | Number of distinct users who queried the table or column over the last 14, 30, or 60 days.                             |
| `distinct_impressions_users_14d` / `_30d` / `_60d`   | number | Number of distinct users from query-history based column impressions over the last 14, 30, or 60 days.                 |

#### Write Properties

| Property                                              | Type   | Description                                                                                        |
| ----------------------------------------------------- | ------ | -------------------------------------------------------------------------------------------------- |
| `total_write_queries_14d` / `_30d` / `_60d`           | number | Total number of write operations involving the table over the last 14, 30, or 60 days.             |
| `total_write_slots_14d` / `_30d` / `_60d`             | number | Total slots consumed by write operations involving the table over the last 14, 30, or 60 days.     |
| `average_write_slots_14d` / `_30d` / `_60d`           | number | Average slots consumed per write operation involving the table over the last 14, 30, or 60 days.   |
| `total_write_bytes_processed_14d` / `_30d` / `_60d`   | number | Total bytes processed by write operations involving the table over the last 14, 30, or 60 days.    |
| `average_write_bytes_processed_14d` / `_30d` / `_60d` | number | Average bytes processed per write operation involving the table over the last 14, 30, or 60 days.  |
| `total_write_runtime_14d` / `_30d` / `_60d`           | number | Total runtime of write operations involving the table over the last 14, 30, or 60 days.            |
| `total_write_cost_14d` / `_30d` / `_60d`              | number | Estimated write cost over the last 14, 30, or 60 days, based on the configured cost per slot hour. |

### Storage Properties

Storage properties reflect the current size of the table as stored in BigQuery.

| Property    | Type   | Description                              |
| ----------- | ------ | ---------------------------------------- |
| `row_count` | number | Number of rows in the table.             |
| `volume`    | number | Current logical storage volume in bytes. |

## Column (`column`)

A column of a BigQuery table or view. Nested STRUCT fields are also represented as `column` resources.

| Property               | Type   | Description                                                                                 |
| ---------------------- | ------ | ------------------------------------------------------------------------------------------- |
| `name`                 | string | The column name. Nested STRUCT fields use their full field path, for example `profile.city` |
| `type`                 | string | Always `column`                                                                             |
| `subtype`              | string | `struct` for root STRUCT columns                                                            |
| `description`          | string | Column description when available                                                           |
| `native_data_type`     | string | BigQuery data type                                                                          |
| `normalized_data_type` | string | Euno-normalized data type                                                                   |
| `struct_schema`        | string | STRUCT data type definition, for root STRUCT columns                                        |
| `parent_name`          | string | The table name                                                                              |
| `database_technology`  | string | Always `bigquery`                                                                           |
| `database`             | string | The GCP project ID. Derived.                                                                |
| `database_schema`      | string | Parent **schema** (BigQuery dataset name)                                                   |

Columns also receive `total_read_queries_*`, `distinct_users_*`, and `distinct_impressions_users_*` from column-level usage (see [Read Properties](#read-properties)).

**URI pattern**

```
bigquery.<project_id>.<dataset>.<table>.column.<field_path>
```

Example: `bigquery.proj_sales.analytics.orders_raw.column.profile.city`

## Google Sheet (`google_sheet`)

Google Sheets are discovered when they are referenced as external tables in BigQuery with `format = 'GOOGLE_SHEETS'`.

| Property         | Type   | Description                         |
| ---------------- | ------ | ----------------------------------- |
| `name`           | string | Always `Google Sheet`               |
| `type`           | string | Always `google_sheet`               |
| `external_links` | list   | Link to the referenced Google Sheet |

**URI pattern**

```
google_sheet.<spreadsheet_id>
```

Example: `google_sheet.1a2b3c4d5e6f7g8h9i0j`

## Hex project (`hex_project`)

Hex projects are detected from BigQuery query history when Hex query metadata is present and the corresponding Hex project already exists in Euno. The Hex integration discovers the project; the BigQuery integration enriches it with usage and lineage.

| Property                                             | Type   | Description                                                                     |
| ---------------------------------------------------- | ------ | ------------------------------------------------------------------------------- |
| `name`                                               | string | The Hex project name when present in BigQuery query metadata                    |
| `type`                                               | string | Always `hex_project`                                                            |
| `total_queries_14d` / `_30d` / `_60d`                | number | BigQuery queries attributed to the Hex project over the last 14, 30, or 60 days |
| `total_read_queries_14d` / `_30d` / `_60d`           | number | BigQuery read queries attributed to the Hex project                             |
| `total_read_slots_14d` / `_30d` / `_60d`             | number | Slots consumed by queries attributed to the Hex project                         |
| `average_read_slots_14d` / `_30d` / `_60d`           | number | Average slots per query attributed to the Hex project                           |
| `total_read_runtime_14d` / `_30d` / `_60d`           | number | Total runtime of queries attributed to the Hex project                          |
| `total_read_bytes_processed_14d` / `_30d` / `_60d`   | number | Bytes processed by queries attributed to the Hex project                        |
| `average_read_bytes_processed_14d` / `_30d` / `_60d` | number | Average bytes processed per query attributed to the Hex project                 |
| `distinct_users_14d` / `_30d` / `_60d`               | number | Distinct Hex users over the last 14, 30, or 60 days                             |
| `queries_breakdown_by_user_14d` / `_30d` / `_60d`    | list   | Query breakdown by Hex user                                                     |

**URI pattern**

```
hex.project.<project_id>
```

Example: `hex.project.0b1c2d3e-4f56-7890-abcd-ef1234567890`

{% hint style="info" %}
Hex projects also receive internal BigQuery query metadata used for lineage. Which BigQuery tables and columns a Hex project depends on appears under **Lineage** and in the **Relationships** section below, not as named fields in resource details.
{% endhint %}

## Relationships

Each row is one relationship edge, written in the canonical direction for its family. Use the predicate in each heading to search for these relationships in EQL.

### Containment (`has parent` / `has child`)

| Source type       | Relationship | Target type       | Notes                                                                                                         |
| ----------------- | ------------ | ----------------- | ------------------------------------------------------------------------------------------------------------- |
| `database`        | has parent   | `gcp_project`     | The BigQuery project is represented as a `database` under its GCP project.                                    |
| `database_schema` | has parent   | `database`        | BigQuery datasets are represented as `database_schema` resources under the BigQuery project.                  |
| `table`           | has parent   | `database_schema` | BigQuery tables, views, materialized views, external tables, snapshots, and clones are contained in datasets. |
| `column`          | has parent   | `table`           | BigQuery columns and nested STRUCT fields are contained in tables.                                            |

### Lineage (`has upstream` / `has downstream`)

| Source type   | Relationship | Target type    | Notes                                                                                                                                                                                          |
| ------------- | ------------ | -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `table`       | has upstream | `table`        | For views and materialized views, lineage is based on SQL analysis. For base tables, lineage can be inferred from BigQuery query history. Snapshots and clones have their base table upstream. |
| `table`       | has upstream | `google_sheet` | External tables with `format = 'GOOGLE_SHEETS'` depend on the referenced Google Sheet.                                                                                                         |
| `hex_project` | has upstream | `table`        | Hex project table lineage is inferred from BigQuery query history.                                                                                                                             |

### Field-level lineage (`has upstream_fields` / `has downstream_fields`)

| Source type   | Relationship         | Target type | Notes                                                                                                                                                                                                  |
| ------------- | -------------------- | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `column`      | has upstream\_fields | `column`    | Column-level lineage is based on SQL analysis and BigQuery query history. Columns of snapshots and clones have the matching base table column upstream.                                                |
| `table`       | has upstream\_fields | `column`    | Statement-level dependencies include fields used by clauses such as `JOIN`, `WHERE`, `GROUP BY`, `HAVING`, `QUALIFY`, and `ORDER BY`, even when those fields are not selected in the final projection. |
| `hex_project` | has upstream\_fields | `column`    | Hex project field lineage is inferred from BigQuery query history.                                                                                                                                     |

### Usage (`is user of` / `is used by`)

| Source type | Relationship | Target type   | Notes                                                                                        |
| ----------- | ------------ | ------------- | -------------------------------------------------------------------------------------------- |
| `user`      | is user of   | `hex_project` | Hex users whose BigQuery queries are attributed to the project, with recent activity counts. |

## What to expect from table-level field lineage

For BigQuery views, table-level lineage in Euno may include columns that are used only by SQL logic and are not selected as output columns.

Example:

```sql
SELECT o.order_id, o.customer_id
FROM `proj_sales.analytics.orders_raw` o
WHERE o.order_status = 'COMPLETE'
```

In this case, table-level lineage for the view can include the `order_status` column from `orders_raw`, even though `order_status` is not part of the projection.

## Known limitations

* ARRAY of STRUCT paths through UNNEST are supported, but element-level instance or index lineage is not tracked.
* Fields accessed via dynamic SQL or UDFs may not be captured.
* Some complex UNNEST operations with multiple levels may not fully resolve.
* Cross-project nested field lineage is most accurate when BigQuery metadata and query history are available for all referenced projects.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.euno.ai/sources/data-warehouses/bigquery-integration/bigquery-integration-discovered-resources.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.