Matillion
Euno's Matillion Data Productivity Cloud (DPC) integration supports auto-discovery of the following Matillion resources:
Matillion Projects
Matillion Pipelines (Transformations and Orchestrations)
Database Tables and Schemas
Setting up Euno's Matillion Integration
Step 1: Configure New Matillion Source in Euno
Step 1: Access the Sources Page
Go to the Sources page.
Click on Add New Source and select Matillion from the list of supported platforms.
Step 2: General Configuration
Asterisk (*) means a mandatory field.
Name*
Enter a name for your Matillion source (e.g., "Matillion")
Warehouse
Select your target warehouse: BigQuery, Snowflake, or Databricks
Account*
If Snowflake or Databricks is select, this is the account identifier
Default Database*
The default database name for your data warehouse, as configured in Matillion
Default Schema*
The default schema name for your data warehouse, as configured in Matillion
Step 3: Optional Matillion Configuration
Matillion Project Name
The name of your Matillion project
Matillion Project ID
The unique identifier for your Matillion project
Source Git Repository
The Git repository URL containing your Matillion pipeline definitions
Source Git Branch
The Git branch to reference (e.g., "main" or "develop")
Default Environment
The default environment ID for your Matillion project
Default Agent ID
The default agent ID for pipeline execution
How Optional Configuration Affects Discovery:
Project Creation: When both Matillion Project Name and Matillion Project ID are provided, Euno will create a Matillion project resource that serves as the parent container for all discovered pipelines.
Git Source Links: When Source Git Repository and Source Git Branch are configured, Euno will generate external links from each pipeline directly to its source file in the Git repository, enabling easy navigation to pipeline definitions.
Matillion Designer Links: When all four fields (Project ID, Source Git Branch, Default Environment, and Default Agent ID) are provided, Euno will create direct links to edit each pipeline in the Matillion Designer interface. These links follow the format:
https://app.matillion.com/designer/project/{project_id}/branch/{branch}?environmentId={environment_id}&agentId={agent_id}
Step 4: File Pattern Configuration
Include Patterns
File patterns to include (defaults to files ending with .tran.yaml or .orch.yaml)
Exclude Patterns
File patterns to exclude from processing
Step 5: Resource Cleanup
To keep your data relevant and free of outdated resources, Euno provides automatic resource cleanup options. These settings determine when a resource should be removed if it is no longer detected by a source integration. For a detailed explanation on Euno's cleanup strategies, see: Resource Sponsorship in Euno.
Immediate Cleanup: Remove resources not detected in the most recent successful source integration run.
No Cleanup: Keep all resources indefinitely, even if they are no longer detected.
Step 7: Save Configuration
Click the Save button to complete the setup.
How to Use the Integration
Step 1: Prepare Your Matillion DPC Artifacts
The Matillion integration works by processing DPC artifacts that you upload to Euno.
For a one-time ingestion of Matillino pipelines, you can upload the DPC artifact directly through the Euno UI.
To setup CI/CD integration with your Matillion Git repository, see Setting Up continuous ingestion of Matllion Artifacts.
Features
Project and Pipeline Discovery
The Matillion integration automatically discovers:
Matillion Projects: When project name and ID are configured
Transformation Pipelines: From
.tran.yaml
filesOrchestration Pipelines: From
.orch.yaml
filesPipeline Metadata: Including names, types, and hierarchical relationships
Database Resource Discovery
The integration maps your data infrastructure by discovering:
Databases: Target databases referenced in your pipelines
Schemas: Database schemas used by your transformations
Tables: Input and output tables with full lineage tracking
Lineage and Dependencies
Advanced dependency tracking includes:
Table Dependencies: Automatic discovery of which tables feed into each pipeline
Pipeline Relationships: Understanding how pipelines relate to each other
Cross-Platform Integration: Works with Euno's existing dependency resolution for comprehensive lineage
External Links
When Git repository information is provided, the integration creates direct links to:
Source Code: Links to pipeline definitions in your Git repository
Matillion Designer: Direct links to pipeline editing in Matillion DPC (when all required IDs are configured)
Last updated