# Databricks

Euno's Databricks integration connects to Unity Catalog metadata, Databricks system lineage and query history, and the Workspace API so you can discover catalogs, schemas, tables, columns, usage metrics, lineage, and notebooks in one source.

## What Euno Discovers

Euno's Databricks integration supports auto-discovery of:

* Databricks workspaces
* Unity Catalog databases (catalogs)
* Unity Catalog schemas
* Unity Catalog tables, views, materialized views, and external tables
* Table and column metadata (including tags)
* Table usage metrics for 14/30/60 day windows
* Column usage metrics for 14/30/60 day windows
* Query-history-backed column lineage
* Databricks notebooks and notebook-driven relationships

For resource properties and relationships in detail, see [Databricks Integration Discovered Resources](/sources/data-warehouses/databricks-integration/databricks-integration-discovered-resources.md).

## Prerequisites / Requirements

Before you add the source, make sure you have:

* A Databricks workspace with Unity Catalog enabled
* At least one running SQL warehouse and its HTTP path
* The workspace hostname (for example `dbc-xxxxxxxx-xxxx.cloud.databricks.com`)
* A Databricks personal access token for a user that can run metadata queries and access the objects you want to observe

The token user should have:

* Access to the Databricks workspace
* Permission to use the configured SQL warehouse
* `SELECT` access to the required `system.information_schema` and `system.access` / `system.query.history` objects used by the integration
* Access to the catalogs, schemas, and tables you want to observe
* Workspace API visibility to notebook paths you want to observe (under the default discovery roots)

## Stage 1: Configure Databricks

### Step 1: SQL warehouse and hostname

1. Verify that Unity Catalog is enabled.
2. Ensure at least one SQL warehouse is available and running.
3. Note your workspace hostname (for example `dbc-xxxxxxxx-xxxx.cloud.databricks.com`).
4. Note the SQL warehouse HTTP path (for example `/sql/1.0/warehouses/warehouse-id`).

### Step 2: Personal access token

1. In Databricks, go to **Settings** → **Developer** → **Access tokens**.
2. Click **Generate new token**.
3. Set a comment (for example `Euno integration`).
4. Set token lifetime as needed.
5. Click **Generate**.
6. Copy and store the token securely.

In Euno, the integration combines:

* Unity Catalog metadata (`system.information_schema.*`)
* Databricks system lineage and usage evidence (`system.access.*`, `system.query.history`)
* Workspace notebook discovery (Workspace API)

## Stage 2: Configure New Databricks Source in Euno

### Step 1: Access the Sources Page

1. Open the **Sources** page.
2. Click **Add New Source** and choose **Databricks**.

### Step 2: General Configuration

Asterisk (\*) means a mandatory field.

| Configuration         | Description                                                                           |
| --------------------- | ------------------------------------------------------------------------------------- |
| **Server Hostname**\* | Databricks workspace hostname (for example `dbc-xxxxxxxx-xxxx.cloud.databricks.com`). |
| **HTTP Path**\*       | Databricks SQL warehouse HTTP path (for example `/sql/1.0/warehouses/warehouse-id`).  |
| **Access Token**\*    | Databricks personal access token (stored as a secret).                                |
| **Workspace Name**    | Optional display name in Euno. If left blank, the hostname is used.                   |

### Step 3: Schedule

* Enable scheduling.
* Choose one option:
  1. **Weekly** with a specific day and time.
  2. **Hourly** with an interval.

### Step 4: Resource Cleanup

* **Immediate Cleanup**: remove resources not detected in the latest successful run.
* **No Cleanup**: keep resources even if they are no longer detected.

For cleanup semantics, see [Resource Sponsorship in Euno](/developer-reference/technical-concepts/resource-sponsorship-and-cleanup-in-euno.md).

### Step 5: Advanced Settings

Open the **Advanced** section to configure optional filters.

| Configuration         | Description                                                                                                                                                                   |
| --------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Override Base URI** | Override the hostname used when Euno generates resource URIs. Leave blank to use **Server Hostname**.                                                                         |
| **Database Pattern**  | Allow/deny regular expressions for catalog names. Defaults to include all catalogs; system catalogs such as `system` and `samples` are always excluded from metadata queries. |

**Pattern examples**

* `.*` — include all names that match.
* `production_.*` — include only names starting with `production_`.
* Allow `.*` and deny `test_.*` — include all except names starting with `test_`.

### Step 6: Save

Click **Test & Save** to validate connectivity and save the source.

## Important behavior notes

* Table read usage metrics are emitted for 14/30/60 day windows.
* Databricks read DBU metrics are emitted as `total_read_dbu_14d`, `total_read_dbu_30d`, `total_read_dbu_60d` and `average_read_dbu_14d`, `average_read_dbu_30d`, `average_read_dbu_60d`.
* Column usage metrics are emitted for 14/30/60 day windows.
* Notebook execution evidence and notebook-derived relationships use `system.query.history`, `system.access.table_lineage`, and `system.access.column_lineage` with a 30-day lookback window.
* Regular Databricks column lineage is read from `system.access.column_lineage` with a 30-day lookback; Euno represents it as column-level upstream field relationships after processing (see the discovered-resources page).
* Notebook observation runs by default; scope follows the integration’s discovery rules described in the hint under **Advanced Settings**.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.euno.ai/sources/data-warehouses/databricks-integration.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.