Matillion Integration Discovered Resources
Overview
The Matillion integration discovers the following resources:
Matillion Project
Matillion Pipeline
Database
Database Schema
Database Table
Matillion Project
A single resource of type "matillion_project" is observed when both project name and project ID are configured in the integration settings.
Properties
name
The name of the Matillion project (from configuration)
type
always matillion_project
subtype
always matillion_project
native_id
The Matillion project ID (from the configuration), if set
External Links
When Git repository information is configured, the project includes links to:
Source repository for pipeline definitions
Matillion Pipeline
Pipelines are discovered from .tran.yaml (transformation) and .orch.yaml (orchestration) files in the uploaded DPC artifacts.
Properties
name
The filename without the .yaml, .tran.yaml, or .orch.yaml extension
type
always matillion_pipeline
subtype
either matillion_transformation or matillion_orchestration
native_id
Derived from the filename and file path
Relationships
Parent: If a Matillion project is configured, pipelines are children of the project
Defines: Pipelines have "defines" relationships with their output tables
External Links
Pipelines can include links to:
Git Source: Direct links to the pipeline definition files in the source repository (when Git repository and branch are configured)
Matillion Designer: Direct links to edit the pipeline in Matillion DPC when all required configuration is provided:
Project ID
Source Git branch
Default environment ID
Default agent ID
Example Designer URL format:
https://app.matillion.com/designer/project/{project_id}/branch/{branch}?environmentId={environment_id}&agentId={agent_id}Lineage and Dependencies
Input vs Output Tables
For the Matillion integration, let's define:
Input Tables: Discovered from components that read from tables
Output Tables: Discovered from components that create or write to tables
The Matillion integration observes lineage between all output tables and all input tables of a pipeline. The lineage is used only for materialized tables. For views, Euno will use the lineage analysis of the SQL definition of the view, if available.
Last updated