# BigQuery Integration Discovered Resources ### Ingested Resources & Properties #### BigQuery Project (`gcp_project`) | Property | Description | | ----------- | -------------------- | | `name` | The GCP project ID | | `type` | Always `gcp_project` | | `native_id` | The GCP project ID | #### Database (`database`) In BigQuery, Euno represents each discovered GCP project as a `database` resource. | Property | Description | | ------------------ | ------------------------------------------------- | | `name` | The GCP project ID | | `type` | Always `database` | | `native_id` | The GCP project ID | | `subtype` | Always `database` | | `parent_container` | The `gcp_project` containing the BigQuery project | #### Dataset (`database_schema`) | Property | Description | | --------------------- | ------------------------------------------------------------------ | | `name` | The BigQuery dataset name | | `type` | Always `database_schema` | | `subtype` | Always `database_schema` | | `created_at` | Date and time when the dataset was created | | `updated_at` | Date and time when the dataset was last modified | | `parent_container` | The `database` resource containing the dataset | | `database_technology` | Always `bigquery` | | `location` | The BigQuery dataset location | | `meta` | Dataset labels when **Discover dataset labels as meta** is enabled | #### Table (`table`) | Property | Description | | ------------------------- | ------------------------------------------------------------------------------------------ | | `name` | The BigQuery table, view, materialized view, external table, snapshot, or clone name | | `type` | Always `table` | | `native_id` | Fully qualified BigQuery object ID in `project.dataset.table` format | | `subtype` | One of `base_table`, `view`, `materialized_view`, `external_table`, `snapshot`, or `clone` | | `created_at` | Date and time when the table was created | | `updated_at` | Date and time when table storage was last modified | | `native_last_data_update` | Last refresh time for materialized views, snapshots, and clones when available | | `parent_container` | The `database_schema` resource containing the table | | `database_technology` | Always `bigquery` | | `database_schema` | Parent **schema** (BigQuery dataset name) | | `meta` | Reserved for labels and metadata when available | #### Column (`column`) | Property | Description | | --------------------- | ------------------------------------------------------------------------------------------- | | `name` | The column name. Nested STRUCT fields use their full field path, for example `profile.city` | | `type` | Always `column` | | `subtype` | `struct` for root STRUCT columns | | `description` | Column description when available | | `parent_name` | The table name | | `parent_container` | The `table` resource containing the column | | `database_technology` | Always `bigquery` | | `database_schema` | Parent **schema** (BigQuery dataset name) | #### Google Sheet (`google_sheet`) Google Sheets are discovered when they are referenced as external tables in BigQuery with `format = 'GOOGLE_SHEETS'`. | Property | Description | | ---------------- | ----------------------------------- | | `name` | Always `Google Sheet` | | `type` | Always `google_sheet` | | `external_links` | Link to the referenced Google Sheet | #### Hex Project (`hex_project`) Hex projects are detected from BigQuery query history when Hex query metadata is present and the corresponding Hex project already exists in Euno. | Property | Description | | -------- | ------------------------------------------------------------ | | `name` | The Hex project name when present in BigQuery query metadata | | `type` | Always `hex_project` | {% hint style="info" %} Hex projects also receive internal BigQuery query metadata used for lineage. Which BigQuery tables and columns a Hex project depends on appears under **Lineage** and in the **Relationships** section below, not as named fields in resource details. {% endhint %} ### BigQuery Usage & Optimization Properties Usage and cost optimization properties are derived from BigQuery query history. For base tables and materialized views, read and write usage properties are derived from `INFORMATION_SCHEMA.JOBS` and `INFORMATION_SCHEMA.JOBS_BY_PROJECT`. For `subtype = 'view'`, `total_read_queries_{14d,30d,60d}` reflects queries that referenced the view. Other read optimization properties, such as bytes, slots, and runtime, reflect underlying table processing and may differ. #### Read Properties | Property | Description | | ---------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | | `total_read_queries_14d`, `total_read_queries_30d`, `total_read_queries_60d` | Total number of read queries referencing the resource over the last 14, 30, or 60 days. Applies to tables and columns. | | `total_read_slots_14d`, `total_read_slots_30d`, `total_read_slots_60d` | Total slots consumed by read queries referencing the table over the last 14, 30, or 60 days. | | `average_read_slots_14d`, `average_read_slots_30d`, `average_read_slots_60d` | Average slots consumed per read query referencing the table over the last 14, 30, or 60 days. | | `total_read_bytes_processed_14d`, `total_read_bytes_processed_30d`, `total_read_bytes_processed_60d` | Total bytes processed by read queries referencing the table over the last 14, 30, or 60 days. | | `average_read_bytes_processed_14d`, `average_read_bytes_processed_30d`, `average_read_bytes_processed_60d` | Average bytes processed per read query referencing the table over the last 14, 30, or 60 days. | | `total_read_runtime_14d`, `total_read_runtime_30d`, `total_read_runtime_60d` | Total runtime of read queries referencing the table over the last 14, 30, or 60 days. | | `total_read_cost_14d`, `total_read_cost_30d`, `total_read_cost_60d` | Estimated read cost over the last 14, 30, or 60 days, based on the configured cost per slot hour. | | `average_read_cost_14d`, `average_read_cost_30d`, `average_read_cost_60d` | Estimated average read cost per query over the last 14, 30, or 60 days. | | `distinct_users_14d`, `distinct_users_30d`, `distinct_users_60d` | Number of distinct users who queried the table or column over the last 14, 30, or 60 days. | | `distinct_impressions_users_14d`, `distinct_impressions_users_30d`, `distinct_impressions_users_60d` | Number of distinct users from query-history based column impressions over the last 14, 30, or 60 days. | #### Write Properties | Property | Description | | ------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- | | `total_write_queries_14d`, `total_write_queries_30d`, `total_write_queries_60d` | Total number of write operations involving the table over the last 14, 30, or 60 days. | | `total_write_slots_14d`, `total_write_slots_30d`, `total_write_slots_60d` | Total slots consumed by write operations involving the table over the last 14, 30, or 60 days. | | `average_write_slots_14d`, `average_write_slots_30d`, `average_write_slots_60d` | Average slots consumed per write operation involving the table over the last 14, 30, or 60 days. | | `total_write_bytes_processed_14d`, `total_write_bytes_processed_30d`, `total_write_bytes_processed_60d` | Total bytes processed by write operations involving the table over the last 14, 30, or 60 days. | | `average_write_bytes_processed_14d`, `average_write_bytes_processed_30d`, `average_write_bytes_processed_60d` | Average bytes processed per write operation involving the table over the last 14, 30, or 60 days. | | `total_write_runtime_14d`, `total_write_runtime_30d`, `total_write_runtime_60d` | Total runtime of write operations involving the table over the last 14, 30, or 60 days. | | `total_write_cost_14d`, `total_write_cost_30d`, `total_write_cost_60d` | Estimated write cost over the last 14, 30, or 60 days, based on the configured cost per slot hour. | #### Storage Properties Storage properties reflect the current size of the table as stored in BigQuery. | Property | Description | | ----------- | ---------------------------------------- | | `row_count` | Number of rows in the table. | | `volume` | Current logical storage volume in bytes. | ### Relationships | Source type(s) | Relationship | Target type(s) | Notes | | ----------------- | -------------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `gcp_project` | has child | `database` | The BigQuery project is represented as a `database` under its GCP project. | | `database` | has child | `database_schema` | BigQuery datasets are represented as `database_schema` resources under the BigQuery project. | | `database_schema` | has child | `table` | BigQuery tables, views, materialized views, external tables, snapshots, and clones are contained in datasets. | | `table` | has child | `column` | BigQuery columns and nested STRUCT fields are contained in tables. | | `table` | has upstream | `table` | For views and materialized views, lineage is based on SQL analysis. For base tables, lineage can be inferred from BigQuery query history. | | `column` | has upstream\_fields | `column` | Column-level lineage is based on SQL analysis and BigQuery query history. | | `table` | has upstream\_fields | `column` | Statement-level dependencies include fields used by clauses such as `JOIN`, `WHERE`, `GROUP BY`, `HAVING`, `QUALIFY`, and `ORDER BY`, even when those fields are not selected in the final projection. | | `table` | has upstream | `google_sheet` | External tables with `format = 'GOOGLE_SHEETS'` depend on the referenced Google Sheet. | | `hex_project` | has upstream | `table` | Hex project table lineage is inferred from BigQuery query history. | | `hex_project` | has upstream\_fields | `column` | Hex project field lineage is inferred from BigQuery query history. | ### What to expect from table-level field lineage For BigQuery views, table-level lineage in Euno may include columns that are used only by SQL logic and are not selected as output columns. Example: ```sql SELECT o.order_id, o.customer_id FROM `proj_sales.analytics.orders_raw` o WHERE o.order_status = 'COMPLETE' ``` In this case, table-level lineage for the view can include the `order_status` column from `orders_raw`, even though `order_status` is not part of the projection. ### Known Limitations * ARRAY of STRUCT paths through UNNEST are supported, but element-level instance or index lineage is not tracked. * Fields accessed via dynamic SQL or UDFs may not be captured. * Some complex UNNEST operations with multiple levels may not fully resolve. * Cross-project nested field lineage is most accurate when BigQuery metadata and query history are available for all referenced projects. --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.euno.ai/sources/data-warehouses/bigquery-integration/bigquery-integration-discovered-resources.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.