Matillion Integration Discovered Resources

Overview

The Matillion integration discovers the following resources:

  • Matillion Project

  • Matillion Pipeline

  • Database

  • Database Schema

  • Database Table

Matillion Project

A single resource of type "matillion_project" is observed when both project name and project ID are configured in the integration settings.

Properties

Property
Value

name

The name of the Matillion project (from configuration)

type

always matillion_project

subtype

always matillion_project

native_id

The Matillion project ID (from the configuration), if set

When Git repository information is configured, the project includes links to:

  • Source repository for pipeline definitions

Matillion Pipeline

Pipelines are discovered from .tran.yaml (transformation) and .orch.yaml (orchestration) files in the uploaded DPC artifacts.

Properties

Property
Value

name

The filename without the .yaml, .tran.yaml, or .orch.yaml extension

type

always matillion_pipeline

subtype

always matillion_pipeline

native_id

Derived from the filename and file path

pipeline_type

Either "transformation" or "orchestration" based on the pipeline definition

Relationships

  • Parent: If a Matillion project is configured, pipelines are children of the project

  • Defines: Pipelines have "defines" relationships with their output tables

Pipelines can include links to:

  • Git Source: Direct links to the pipeline definition files in the source repository (when Git repository and branch are configured)

  • Matillion Designer: Direct links to edit the pipeline in Matillion DPC when all required configuration is provided:

    • Project ID

    • Source Git branch

    • Default environment ID

    • Default agent ID

Example Designer URL format:

https://app.matillion.com/designer/project/{project_id}/branch/{branch}?environmentId={environment_id}&agentId={agent_id}

Lineage and Dependencies

Input vs Output Tables

For the Matillion integration, let's define:

  • Input Tables: Discovered from components that read from tables

  • Output Tables: Discovered from components that create or write to tables

The Matillion integration observes lineage between all output tables and all input tables of a pipeline. The lineage is used only for materialized tables. For views, Euno will use the lineage analysis of the SQL definition of the view, if available.

Last updated