> For the complete documentation index, see [llms.txt](https://docs.euno.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.euno.ai/sources/transformation-etl/matillion/matillion-integration-discovered-resources.md).

# Matillion Integration Discovered Resources

## Overview

The Matillion integration discovers Matillion Data Productivity Cloud (DPC) pipelines from the uploaded Git repository, and the warehouse databases, schemas, and tables those pipelines read and write. It discovers the following resources:

* Matillion project (`matillion_project`)
* Matillion pipeline (`matillion_pipeline`, subtypes `transformation`, `orchestration`)
* Database (`database`)
* Database schema (`database_schema`)
* Table (`table`)

Warehouse resources (databases, schemas, and tables) use the platform URI prefix derived from the configured data platform account identifier, so they match the resources discovered by the warehouse integration (for example `snowflake.<account>`, `bigquery`, or `databricks.<hostname>`). Matillion resources use the URI prefix `matillion.<project>`, where `<project>` is the configured Matillion project ID, or the project name when no project ID is configured.

## Resource hierarchy

```
Matillion Project
└── Matillion Pipeline

Database
└── Database Schema
    └── Table
```

## Matillion project (`matillion_project`)

A single Matillion project, observed only when both the project name and the project ID are configured in the integration settings.

| Property         | Type   | Description                                                                                                 |
| ---------------- | ------ | ----------------------------------------------------------------------------------------------------------- |
| `name`           | string | The name of the Matillion project (from configuration).                                                     |
| `type`           | string | Always `matillion_project`.                                                                                 |
| `subtype`        | string | Always `matillion_project`.                                                                                 |
| `native_id`      | string | The Matillion project ID (from configuration). If no project ID is configured, the project is not observed. |
| `container_type` | string | Always `pure_container`.                                                                                    |

**URI pattern**

```
matillion.<project_id>
```

Example: `matillion.9a1c2b3d-4e5f-6789-abcd-ef0123456789`

## Matillion pipeline (`matillion_pipeline`)

A Matillion pipeline, discovered from the `.tran.yaml` (transformation) and `.orch.yaml` (orchestration) files in the uploaded DPC repository.

| Property            | Type   | Description                                                                                                          |
| ------------------- | ------ | -------------------------------------------------------------------------------------------------------------------- |
| `name`              | string | The file name up to its first `.` (for example `load_orders` for `pipelines/load_orders.tran.yaml`).                 |
| `type`              | string | Always `matillion_pipeline`.                                                                                         |
| `subtype`           | string | Either `transformation` or `orchestration`, from the pipeline file's `type`.                                         |
| `matillion_project` | string | The name of the Matillion project (from configuration).                                                              |
| `external_links`    | list   | Links to the pipeline definition file in Git and to the pipeline in Matillion Designer, when configured (see below). |
| `native_raw_object` | object | The parsed pipeline YAML file.                                                                                       |

**URI pattern**

```
matillion.<project_id_or_name>.pipeline.<pipeline_name>
```

Example: `matillion.9a1c2b3d-4e5f-6789-abcd-ef0123456789.pipeline.load_orders`

### External links

Pipelines can include links to:

* **Git source**: A direct link to the pipeline definition file in the source repository, when the Git repository and branch are configured. The link is built as `<repository>/blob/<branch>/<file path>`.
* **Matillion Designer**: A direct link to edit the pipeline in Matillion DPC, when all of the following are configured:
  * Project ID
  * Source Git branch
  * Default environment ID
  * Default agent ID

Designer URL format:

```
https://app.matillion.com/designer/project/{project_id}/branch/{branch}/job/{file_path}?environmentId={environment_id}&agentId={agent_id}
```

## Database (`database`)

A warehouse database referenced by a Matillion pipeline table component. Rich database metadata and lifecycle are owned by the warehouse integration.

| Property              | Type   | Description                                                                   |
| --------------------- | ------ | ----------------------------------------------------------------------------- |
| `name`                | string | Database name.                                                                |
| `type`                | string | Always `database`.                                                            |
| `subtype`             | string | Always `database`.                                                            |
| `platform_uri_prefix` | string | Warehouse platform URI prefix derived from the configured account identifier. |
| `container_type`      | string | Always `pure_container`.                                                      |

**URI pattern**

```
<platform_uri_prefix>.<database>
```

Example: `snowflake.xy12345.us-east-1.aws.analytics`

## Database schema (`database_schema`)

A warehouse schema referenced by a Matillion pipeline table component.

| Property              | Type   | Description                                                                   |
| --------------------- | ------ | ----------------------------------------------------------------------------- |
| `name`                | string | Schema name.                                                                  |
| `type`                | string | Always `database_schema`.                                                     |
| `subtype`             | string | Always `database_schema`.                                                     |
| `database_technology` | string | `snowflake`, `bigquery`, or `databricks`, from the configured data platform.  |
| `platform_uri_prefix` | string | Warehouse platform URI prefix derived from the configured account identifier. |
| `container_type`      | string | Always `pure_container`.                                                      |

**URI pattern**

```
<platform_uri_prefix>.<database>.<schema>
```

Example: `snowflake.xy12345.us-east-1.aws.analytics.staging`

## Table (`table`)

A warehouse table that a Matillion pipeline reads from or writes to. Tables are discovered from table components (Table Input, Rewrite Table, Create Table), SQL Script components, and S3 Load components. Rich table metadata and lifecycle are owned by the warehouse integration.

| Property              | Type   | Description                                                                   |
| --------------------- | ------ | ----------------------------------------------------------------------------- |
| `name`                | string | Table name.                                                                   |
| `type`                | string | Always `table`.                                                               |
| `database_database`   | string | Database name.                                                                |
| `database_schema`     | string | Schema name.                                                                  |
| `database_technology` | string | `snowflake`, `bigquery`, or `databricks`, from the configured data platform.  |
| `defined_by`          | string | `matillion` for tables written by a Matillion pipeline.                       |
| `platform_uri_prefix` | string | Warehouse platform URI prefix derived from the configured account identifier. |
| `native_raw_object`   | object | For S3 Load target tables only: the S3 Load component details.                |

**URI pattern**

```
<platform_uri_prefix>.<database>.<schema>.<table>
```

Example: `snowflake.xy12345.us-east-1.aws.analytics.staging.orders`

## Relationships

Each row is one relationship edge, written in the canonical direction for its family. Use the predicate in each heading to search for these relationships in EQL.

### Containment (`has parent` / `has child`)

| Source type          | Relationship | Target type         | Notes                                           |
| -------------------- | ------------ | ------------------- | ----------------------------------------------- |
| `matillion_pipeline` | has parent   | `matillion_project` | Only when a Matillion project ID is configured. |
| `database_schema`    | has parent   | `database`          |                                                 |
| `table`              | has parent   | `database_schema`   | Tables discovered from table components.        |

### Lineage (`has upstream` / `has downstream`)

| Source type | Relationship | Target type | Notes                                                                                                                                                                                                        |
| ----------- | ------------ | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `table`     | has upstream | `table`     | An output table depends on the input tables that feed it in the pipeline, and a SQL Script destination table depends on the tables its SQL reads. See [Lineage and dependencies](#lineage-and-dependencies). |

### Definition (`defines` / `has definer`)

| Source type          | Relationship | Target type | Notes                                                                                                                                                                                   |
| -------------------- | ------------ | ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `matillion_pipeline` | defines      | `table`     | Tables the pipeline writes (output tables of table components, SQL Script destination tables, and S3 Load target tables), and Table Input tables the first time they are seen in a run. |

## Lineage and dependencies

### Input vs output tables

For the Matillion integration, let's define:

* **Input tables**: Discovered from components that read from tables
* **Output tables**: Discovered from components that create or write to tables

The Matillion integration observes lineage between each output table and its input tables. For SQL Script components, Euno parses the SQL and links the destination table to the tables the statement reads. The lineage is used only for materialized tables. For `views`, Euno will use the lineage analysis of the SQL definition of the view, if available.

## Usage in EQL

```
# Tables written by Matillion pipelines
type = 'table' AND has definer(type = 'matillion_pipeline')

# Pipelines in the configured Matillion project
type = 'matillion_pipeline' AND has parent(type = 'matillion_project')
```

## Known limitations

* Pipelines are identified by file name only. Two pipeline files with the same name in different folders map to the same `matillion_pipeline` resource.
* Tables that a pipeline only reads through a Table Input component can also be marked as defined by that pipeline (`defined_by` = `matillion`, and the pipeline's **defines** relationship) the first time the table is seen in a run.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.euno.ai/sources/transformation-etl/matillion/matillion-integration-discovered-resources.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.