Azure Data Factory Discovered Resources

Overview

The Azure Data Factory integration discovers the following resources:

• Data Factory (adf_factory)

• Pipeline (adf_pipeline)

• Dataset (adf_dataset)

• Mapping Data Flow (adf_dataflow)

• Power Query / Wrangling Data Flow (adf_power_query)

• Table (table) and Column (column) — the warehouse resources ADF reads and writes

Ingested Resources & Properties

Data Factory (adf_factory)

The top-level resource representing a single Azure Data Factory. All pipelines, datasets, and data flows discovered from one factory are children of it.

Property	Description
name	The data factory name
type	Always adf_factory
native_id	The factory's Azure Resource Manager identifier

Pipeline (adf_pipeline)

An ADF pipeline (an orchestration of activities such as Copy, Data Flow, Execute Pipeline, and Stored Procedure).

Property	Description
name	The pipeline name
type	Always adf_pipeline
native_id	The pipeline's Azure Resource Manager identifier
parent_container	Parent adf_factory URI

Dataset (adf_dataset)

An ADF dataset — a named pointer to a warehouse table or file.

Property	Description
name	The dataset name
type	Always adf_dataset
native_id	The dataset's Azure Resource Manager identifier
subtype	The ADF dataset type (e.g. AzureSqlTable, SnowflakeV2Table, AzureDatabricksDeltaLakeDataset)
parent_container	Parent adf_factory URI

Mapping Data Flow (adf_dataflow)

An ADF Mapping Data Flow or Flowlet (subtype). A Mapping Data Flow resolves the tables its sinks write; Flowlets are inventory only.

Property	Description
name	The data flow name
type	Always adf_dataflow
subtype	MappingDataFlow or Flowlet
native_id	The data flow's Azure Resource Manager identifier
parent_container	Parent adf_factory URI

Power Query / Wrangling Data Flow (adf_power_query)

An ADF Power Query (Wrangling Data Flow), whose transformations are authored in Power Query M. Euno parses the M to resolve the warehouse tables and columns its queries read and write, including embedded native SQL queries.

Property	Description
name	The Power Query data flow name
type	Always adf_power_query
subtype	Always WranglingDataFlow
native_id	The data flow's Azure Resource Manager identifier
parent_container	Parent adf_factory URI

Table (table)

A warehouse table referenced by an ADF dataset, data flow, or activity. When the source resolves to a supported warehouse (Snowflake, BigQuery, Databricks, Azure SQL, Azure Synapse, Azure Storage), Euno matches it to the same table discovered by your warehouse integration.

Property	Description
name	The table name
type	Always table
database_database	The database the table belongs to
database_schema	The schema the table belongs to
database_technology	The warehouse technology (e.g. snowflake, bigquery, databricks, mssql, azure_synapse, azure_storage)
defined_by	Set to adf when the table is currently written (defined) by an ADF pipeline or data flow

Column (column)

A warehouse column discovered to carry column-level lineage. parent_container links it to its table.

Property	Description
name	The column name
type	Always column
parent_container	Parent table URI

---

Relationships

Source type(s)	Relationship	Target type(s)	Notes
adf_factory	has child	adf_pipeline, adf_dataset, adf_dataflow, adf_power_query	Pipelines, datasets, and data flows are direct children of their factory
adf_pipeline	defines	table	Tables the pipeline's Copy and Script activities write
adf_pipeline	has upstream	adf_pipeline, adf_dataflow, adf_power_query	The pipelines and data flows the pipeline executes (Execute Pipeline, Execute Data Flow, Execute Wrangling Data Flow)
adf_dataflow	defines	table	Sink tables a Mapping Data Flow writes (Flowlets do not resolve lineage)
adf_power_query	defines	table	Sink tables a Power Query (Wrangling) Data Flow writes
column	has parent	table	Columns are grouped under their table
table	has definer	adf_pipeline, adf_dataflow, adf_power_query	The pipeline or data flow that writes the table (defined_by = adf)
table	has upstream	table	The source tables an ADF-written table reads, resolved from Copy, Script, Mapping Data Flow, and Power Query lineage (including native SQL)
column	has upstream_fields	column	Source columns a written column derives from when named by the transformation (Copy translator.mappings on single-source/single-sink Copies; Script SQL where destination columns are named; Power Query M-named columns and native-query columns)

Lineage

Euno resolves ADF lineage at two levels so ADF-written tables and columns appear alongside the rest of your lineage:

• Table level — for each table an ADF pipeline or data flow writes, Euno records the source tables it reads (has upstream) and the pipeline or data flow that produces it (has definer). Sources are resolved from Copy activities, Script activities (SQL analysis), Mapping Data Flows, and Power Query (Wrangling) Data Flows, including embedded Value.NativeQuery SQL. A transform that joins or appends several sources attributes the written table to all of them.

• Column level — where the transformation names its output columns, Euno records the source column each output column derives from (has upstream_fields).

Because warehouse tables and columns from ADF are matched to the same resources from your warehouse integration, a single table can show both its ADF-resolved lineage and any lineage from your warehouse or other tools.