Matillion Integration Discovered Resources
Overview
The Matillion integration discovers the following resources:
Matillion Project
Matillion Pipeline
Database
Database Schema
Database Table
Matillion Project
A single resource of type "matillion_project" is observed when both project name and project ID are configured in the integration settings.
Properties
name
The name of the Matillion project (from configuration)
type
always matillion_project
subtype
always matillion_project
native_id
The Matillion project ID (from the configuration), if set
External Links
When Git repository information is configured, the project includes links to:
Source repository for pipeline definitions
Matillion Pipeline
Pipelines are discovered from .tran.yaml
(transformation) and .orch.yaml
(orchestration) files in the uploaded DPC artifacts.
Properties
name
The filename without the .yaml
, .tran.yaml
, or .orch.yaml
extension
type
always matillion_pipeline
subtype
always matillion_pipeline
native_id
Derived from the filename and file path
pipeline_type
Either "transformation" or "orchestration" based on the pipeline definition
Relationships
Parent: If a Matillion project is configured, pipelines are children of the project
Defines: Pipelines have "defines" relationships with their output tables
External Links
Pipelines can include links to:
Git Source: Direct links to the pipeline definition files in the source repository (when Git repository and branch are configured)
Matillion Designer: Direct links to edit the pipeline in Matillion DPC when all required configuration is provided:
Project ID
Source Git branch
Default environment ID
Default agent ID
Example Designer URL format:
https://app.matillion.com/designer/project/{project_id}/branch/{branch}?environmentId={environment_id}&agentId={agent_id}
Lineage and Dependencies
Input vs Output Tables
For the Matillion integration, let's define:
Input Tables: Discovered from components that read from tables
Output Tables: Discovered from components that create or write to tables
The Matillion integration observes lineage between all output tables and all input tables of a pipeline. The lineage is used only for materialized tables. For views
, Euno will use the lineage analysis of the SQL definition of the view, if available.
Last updated