# BigQuery Integration Discovered Resources

The Euno-BigQuery integration discovers the following resources:

### BigQuery Project

| Property   | Value                          |
| ---------- | ------------------------------ |
| name       | The GCP project's display name |
| native\_id | The GCP project's id           |
| type       | always `gcp_project`           |

### Database (BigQuery Project)

| Property    | Value                          |
| ----------- | ------------------------------ |
| name        | The GCP project's display name |
| description | The GCP project's id           |
| type        | always `database`              |

> Only GCP projects with views or tables are discovered

### Schema (BigQuery Dataset)

| Property             | Value                                              |
| -------------------- | -------------------------------------------------- |
| name                 | The GCP dataset's name                             |
| description          | The dataset's description, if available            |
| database\_technology | always `bigquery`                                  |
| created\_at          | date resource was created                          |
| updated\_at          | date resource was last updated                     |
| parent container     | The GCP project containing the dataset             |
| type                 | always `schema`                                    |
| meta                 | Any dataset label will be observed as a meta entry |

### Google Sheet

| Property | Value                 |
| -------- | --------------------- |
| name     | Always "Google Sheet" |
| type     | always `google_sheet` |

> Google Sheets are discovered when they are referenced as external tables in BigQuery with `format = 'GOOGLE_SHEETS'`

### Table

| Property            | Value                                                                                        |
| ------------------- | -------------------------------------------------------------------------------------------- |
| name                | The GCP view's name                                                                          |
| subtype             | `base table`, `view`, or `materialized view`                                                 |
| description         | The table description, if available                                                          |
| created\_at         |                                                                                              |
| updated\_at         |                                                                                              |
| parent container    | The GCP dataset containing the view                                                          |
| type                | always `table`                                                                               |
| materialized        | <p>for tables and materialized views: <code>true</code><br>for views: <code>false</code></p> |
| meta                | Any view label will be observed as a meta entry                                              |
| table properties    | The SQL definition of the view                                                               |
| table dependencies  | The list of views or tables this view depends on based on its SQL definition                 |
| database technology | always `bigquery`                                                                            |
| schema              | The name of the BigQuery schema containing the table                                         |
| database            | The name of the BigQuery dataset containing the table                                        |

### BigQuery Usage & Optimization Properties

Usage and cost optimization properties are derived from BigQuery query history.

For base tables and materialized views, read and write usage properties are derived from `INFORMATION_SCHEMA.JOBS` and `INFORMATION_SCHEMA.JOBS_BY_PROJECT`.

For `subtype = 'view'`, `total_read_queries_{14d,30d,60d}` reflects queries that referenced the view.

#### Read Properties

Read properties capture activity from SQL statements such as `SELECT`.

For view resources, `total_read_queries_{14d,30d,60d}` reflects query activity on the view itself. Other read optimization properties (bytes, slots, runtime) reflect underlying table processing and may differ.

<table><thead><tr><th width="299.70703125">Property</th><th>Value</th><th>Applicable to</th></tr></thead><tbody><tr><td>total_read_queries_14d total_read_queries_30d total_read_queries_60d</td><td>Total number of queries referencing (<code>SELECT</code> queries) the resource over the last 14/30/60 days.</td><td>tables, base_tables, temporary tables, materialized_views, columns</td></tr><tr><td>total_read_slots_14d total_read_slots_30d total_read_slots_60d</td><td>Total compute credits consumed by read operations (<code>SELECT</code> queries) referencing the resource over the last 14/30/60 days.</td><td>tables, base_tables, temporary tables, materialized_views</td></tr><tr><td>average_read_slots_14d average_read_slots_30d average_read_slots_60d</td><td>Average compute credits per read operation (<code>SELECT</code> query) referencing the resource over the last 14/30/60 days.</td><td>tables, base_tables, temporary tables, materialized_views</td></tr><tr><td>total_read_bytes_processed_14d total_read_bytes_processed_30d total_read_bytes_processed_60d</td><td>Total number of bytes processed by read queries (<code>SELECT</code> queries) referencing the resource over the last 14/30/60 days,.</td><td>tables, base_tables, temporary tables, materialized_views</td></tr><tr><td>average_read_bytes_processed_14d average_read_bytes_processed_30d average_read_bytes_processed_60d</td><td>Average number of bytes processed by read queries (<code>SELECT</code> queries) referencing the resource over the last 14/30/60 days.</td><td>tables, base_tables, temporary tables, materialized_views</td></tr><tr><td>total_read_runtime_14d total_read_runtime_30d total_read_runtime_60d</td><td>Total runtime of read queries (<code>SELECT</code> queries) referencing the resource over the last 14/30/60 days.</td><td>tables, base_tables, temporary tables, materialized_views</td></tr><tr><td>distinct_users_14d distinct_users_30d distinct_users_60d</td><td>Number of distinct users who queried (<code>SELECT</code> queries) the resource over the last 14/30/60 days.<br><br><strong>Note</strong>: Unlike the distinct-users property available in BI tools with native usage tracking, this distinct-user count does not provide a user breakdown.</td><td>tables, base_tables, temporary tables, materialized_views, columns</td></tr></tbody></table>

#### Write Properties

Write-related properties capture activity from SQL statements such as `INSERT`, `UPDATE`, `DELETE`, and `MERGE`. These metrics apply to both tables and materialized views.

<table><thead><tr><th width="300.1015625">Property</th><th>Value</th><th>Applicable to</th></tr></thead><tbody><tr><td>total_write_queries_14d total_write_queries_30d total_write_queries_60d</td><td>Total number of write operations (<code>INSERT</code>, <code>UPDATE</code>, <code>DELETE</code>, <code>MERGE</code>) involving the table in the last 14/30/60 days.</td><td>tables, base_tables, temporary tables, materialized_view</td></tr><tr><td>total_write_slots_14d total_write_slots_30d total_write_slots_60d</td><td>Total slots consumed by write operations (<code>INSERT</code>, <code>UPDATE</code>, <code>DELETE</code>, <code>MERGE</code>) involving the table in the last 14/30/60 days.</td><td>tables, base_tables, temporary tables, materialized_view</td></tr><tr><td>average_write_slots_14d average_write_slots_30d average_write_slots_60d</td><td>Average slots consumed by write operations (<code>INSERT</code>, <code>UPDATE</code>, <code>DELETE</code>, <code>MERGE</code>) involving the table in the last 14/30/60 days.</td><td>tables, base_tables, temporary tables, materialized_view</td></tr><tr><td>total_write_bytes_processed_14d total_write_bytes_processed_30d total_write_bytes_processed_60d</td><td>Total number of bytes processed by write queries (<code>INSERT</code>, <code>UPDATE</code>, <code>DELETE</code>, <code>MERGE</code>) involving the table over the last 14/30/60 days.</td><td>tables, base_tables, temporary tables, materialized_view</td></tr><tr><td>average_write_bytes_processed_14d average_write_bytes_processed_30d average_write_bytes_processed_60d</td><td>Average number of bytes processed by write queries (<code>INSERT</code>, <code>UPDATE</code>, <code>DELETE</code>, <code>MERGE</code>) involving the table over the last 14/30/60 days.</td><td>tables, base_tables, temporary tables, materialized_view</td></tr><tr><td>total_write_runtime_14d total_write_runtime_30d total_write_runtime_60d</td><td>Total runtime of write-related operations (<code>INSERT</code>, <code>UPDATE</code>, <code>DELETE</code>, <code>MERGE</code>) involving the table over the last 14/30/60 days.</td><td>tables, base_tables, temporary tables, materialized_views</td></tr></tbody></table>

#### **Storage Properties**

Storage properties reflect the current size of the table as stored in BigQuery. They are used to represent storage utilization and are applicable to all table types.

| volume     | The number of rows in the table.                           | tables, base\_tables, temporary tables, materialized\_views |
| ---------- | ---------------------------------------------------------- | ----------------------------------------------------------- |
| row\_count | Current table volume as stored in the database (in bytes). | tables, base\_tables, temporary tables, materialized\_views |

### Relationships

| Source type(s) | Relationship     | Target type(s) | Notes                                                                                                                                                                                                                                                            |
| -------------- | ---------------- | -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| database       | child            | gcp\_project   |                                                                                                                                                                                                                                                                  |
| schema         | child            | database       |                                                                                                                                                                                                                                                                  |
| table          | child            | schema         |                                                                                                                                                                                                                                                                  |
| column         | child            | table          |                                                                                                                                                                                                                                                                  |
| table          | upstream         | table          | <p>For views, this is based on analyzing the SQL defining the view.<br><br>For tables, the lineage is based on analyzing the queries in the log. Euno considers all write queries in the last 7 days to infer the lineage.</p>                                   |
| column         | upstream\_fields | column         | <p>For views, this is based on analyzing the SQL defining the view.<br><br>For tables, the lineage is based on analyzing the queries in the log. Euno considers all write queries in the last 7 days to infer the lineage.</p>                                   |
| table          | upstream\_fields | column         | For views, Euno also observes statement-level column dependencies from the SQL logic (for example: columns used in `JOIN`, `WHERE`, `GROUP BY`, `HAVING`, `QUALIFY`, or `ORDER BY`). This can include columns that are not projected in the final `SELECT` list. |
| table          | upstream         | google\_sheet  | For external tables with `format = 'GOOGLE_SHEETS'`, this relationship connects the BigQuery table to the Google Sheet it references.                                                                                                                            |

### What to expect from table-level field lineage

For BigQuery views, `upstream_fields` on the table resource may now include columns that are used only by SQL logic and are not selected as output columns.

Example:

```sql
SELECT o.order_id, o.customer_id
FROM `proj_sales.analytics.orders_raw` o
WHERE o.order_status = 'COMPLETE'
```

In this case, table-level lineage for the view can include `...orders_raw.column.order_status` in `upstream_fields`, even though `order_status` is not part of the projection.

### Known Limitations

For columns of type `STRUCT`, column-level lineage (`upstream_fields` relationships) is tracked only at the column level, from the source BigQuery column to the target BigQuery column. Lineage is not captured for individual nested keys within the `STRUCT`.

## Hex Projects

### Properties

| Property   | Value                          |
| ---------- | ------------------------------ |
| name       | The name of the Hex project    |
| type       | always `hex_project`           |
| subtype    | always `hex_project`           |
| native\_id | The unique project ID from Hex |

### Lineage Information

The BigQuery integration tracks which BigQuery tables and columns are accessed by each Hex project through analysis of query history. This enables:

* **Data Dependencies:** Understand how Hex projects depend on specific BigQuery resources.
* **Access Patterns:** Identify which tables and datasets are most frequently accessed by Hex projects.
* **Impact Analysis:** Determine which Hex projects might be affected by changes to BigQuery resources.