🖥️Using the Data Model Screen
Last updated
Last updated
The Data Model screen serves as your guide to your organization's shared data model along with the integrated BI applications. This feature within our web application is more than a mere tool; it's a robust platform designed to simplify the comprehension of complex relationships and connections within your organization's data structures. The Data Model Screen stands as a comprehensive tool, empowering users to interact with and discover the intricacies of the underlying data structure intuitively.
This screen is composed of interconnected resources. Every asset in your data model, such as a table, a column in a table, a Chart in your BI tool, or a data transformation, is a resource.
Resources can connect to each other in different ways. For example, a column is a child of a table. A database view can depend on another database view, etc. To learn more about resources and the different ways they can relate to one another, see Relationships.
The Universal Resource Location is the unique identifier for every resource within the mapped data model. The URI is used to map location of resources within the data model, and their connections. To learn more about URIs, see Universal Resource Identifier (URI).
To easily access a URI , open a resource's sidebar and click the "Copy Link" button located next to the resource's name. This feature allows you to quickly obtain the URL linking to the data model screen and the specific resource. The URI can be found within the URL by removing the initial part: "https://app.getdelphi.io/data-model?view=graph&highlightedNode=".
The URI can be used to filter out the data model screen by a specific resource, using the URI filter, or it can be used to help diagnose connection mapping issues. In order to diagnose mapping issue between resources view the compiled code, to see the URI referenced in the SQL, and compare it to the parent resource's URI (from its URL). If the two URI do not match, contact customer support for assistance.
Aside from the identifier, and the relationship to other resources, each resource in Euno has properties. For example:
a table has a name
a column has a data type
a chart in a BI tool has a link
Properties are used to investigate the data model, can be used as filters to customize which resources are displayed in the Tabular or Graph view, or as columns in the Tabular view.
There are two classes of properties: built-in properties and custom properties. Built-in properties are available for all Euno accounts, and their value is set by integrations. Custom properties are account specific. To create custom properties in your account contact support.
For a complete list of built-in properties, see Built-In Properties.
The Data Model screen presents two distinct views: Tabular View and Graph View, both representing the same information. Resources within your data model are displayed on this screen.
The resources on the data model screen can be filtered using any of the available properties, built-in or custom. For a complete list of built-in properties, see Built-In Properties.
Custom properties: Through a service option, accounts can define custom properties based on the built-in properties. These custom properties will be displayed in the Tabular view and can also be used as filters. For further details, contact customer support.
If multiple filters are applied simultaneously, the operator between the two filters is “AND”, indicating that both criteria must be met for a resource to satisfy the filtered conditions.
The main difference between the Table and Graph view is that in the Graph view, users will not only see the relevant resources based on the filters set, but also their upstream and downstream connections, providing users with the resources and their broader context.
The tabular view is a table which display's all resources applicable to the selected filters. This table includes the following columns:
Data type: The type of resource (e.g. dbt model, view, etc…)
Name: The name of the resource
Description: The description of the resource
Columns: Number of columns under the resource
Dimensions: Number of dimensions under the resource
Entities: Number of entities under the resource
Related Metrics: Number of related metrics associated with the resource
Custom Fields: Number of custom/calculated fields under the resource
Proposals: If the resource has a change proposals
Users can click on a specific row to view the resource’s sidebar, which holds the contextual information for the resource (e.g. dbt model), and their components (e.g. columns).
This view allows users to delve deeper into their organization's shared data model by displaying resource as nodes on a graph. By clicking and highlighting a specific resource, users can better understand the structure of their data model and how that resource is connected within the larger model. Once a resource is clicked and highlighted, its downstream and upstream connections are also highlighted. Any resource that isn’t directly connected with the selected model will be overlaid, ensuring that only information relevant to that resource is displayed. This allows users to easily understand both upstream and downstream implications.
Note - Certain resources are not displayed as a node in the graph, but rather as a property of other resources (i.e. Tableau workbook are not displayed as a node, rather as a property of other Tableau resources)
The Graph view groups resources into four vertical swimlanes;
Sources: This includes resources that are dbt sources.
Transformation: This includes resources that are dbt models as well as orphan resources - Orphan resources refer to tables in your database that are referenced by a BI application but are not part of your dbt project, and therefore, we have limited information about these tables, other than their existence.
App Modeling: This includes transformation-like resources within the BI application (e.g., Views and Explores in Looker, or custom queries, published or embedded data sources in Tableau).
Application: This includes resources in the BI application that are related to visualizations (e.g., Looks, Tiles, and Dashboards in Looker, or views, dashboards, and stories in Tableau).
Any connection resource that does not match the filter will be displayed with lower opacity, making it easy to distinguish between connection resources and those that match the filters. Additionally, filtered resources cannot be removed from the graph view unless they are filtered out.
Users can efficiently search for resources and their components in the Graph view using the search bar located in the top right corner of the screen next to the view selector. The search functionality enables users to input free-text, instantly displaying a list of relevant resources, and their components in the shared data model. The search dynamically filters through elements as the text expands, allowing for quick and effective exploration. Users can select a resource, or a component from the list, which will highlight the resource or component in the Lineage view, and open the sidebar for the resource, or component.
The side panel is a comprehensive interface for accessing all relevant information about a given resource. Users can open it from the tabular view by clicking on the item, or from the DAG view by using the side panel icon (above the highlighted resource; see image below). Designed to enhance the user experience, it provides insights into relationships, usage, and attributes of data resources. The side panel is divided into a fixed summary bar for quick access and a right-side menu containing the following tabs: Overview, Schema, Used Fields, Contents, Relationships, Usage, and About. However, not all tabs apply to every resource type. The panel dynamically displays the relevant subset based on the type of resource, while the Overview and About tabs are consistently shown for all types. The sections below provide detailed descriptions of each tab's functionality.
The Fixed Summary Bar in the Resource Side Panel is designed to keep essential navigation and context elements visible as the user scroll through content and switch between different tabs. This bar includes key elements such as Breadcrumbs, Chips, Automations, View Code, and the Horizontal Ellipses menu, providing quick access to essential features for each resource.
The Breadcrumbs in the Fixed Summary Bar enhance usability by showing the container chain (parent hierarchy) of the resource, giving the users context and ability navigation within the data model hierarchy.
Chips in the Fixed Summary Bar provide a visual summary of key Built-in Properties for various resource types. Available chips include:
Euno Icon: Appears when a resource is managed by Euno. See Data Application Sync for more details.
dbt Materialization Strategy: Indicates the materialization strategy for dbt models as "table," "incremental," "view," "materialized view," or "ephemeral."
Derived View: Applicable to Looker derived views and denotes the derived type, either "SQL Derived View" or "Native Derived View."
Identity Transformation: Displays "Identity Transformation" when the resource has NO inherent logic, and "Contains Logic" when the resource HAS inherent logic.
Calculated: Indicates whether the resource is a calculated field (e.g., a Tableau calculated field or a custom field in Looker).
Instance: Applicable to Tableau embedded data sources, this chip shows "Instance" when the data source is an instance of a published data source.
If a Euno Automation is available for the resource, it appears in the Automations dropdown within the Fixed Summary Bar. Currently, two automations can be triggered:
Shift Left - For more information see Shift-left Proposals
Add Pre-Aggregations - For more information see Pre-Aggregate Models
The Action button (•••) in the Fixed Summary Bar offers additional actions to deepen the user’s understanding of the resource:
Open <APP> : Opens the resource in its native application.
Isolate in DAG: Opens a new tab in graph view displaying the node for the specific resource isolated in the graph.
Upstream Dependencies: Opens a new tab in tabular view displaying all upstream dependencies, helping users understand the impact of data model changes or explore the origins of a resource.
Downstream Dependencies: Opens a new tab in tabular view displaying all downstream dependencies, helping users understand the impact of data model changes or explore the origins of a resource.
By clicking on the code button (</>) situated next to the open source button, a window opens, revealing the resource’s code (if available), offering an insightful view into the code that shapes the element. The popup window with the transformation code, includes a toggle which allows for toggling between the source and compiled code.
In addition to being available at the resource level, certain components also provide access to their source code. Below is a list of resources and the corresponding components supporting the 'View Code' button:
Resources:
dbt Models
Source Code: Displays the compiled dbt code for the models.
Compiled Code: Shows the compiled dbt code for the models.
Semantic Code: If the dbt model includes a semantic representation, the semantic code tab will be available. This utilizes the dbt semantic layer, written in MetricFlow.
Views & Explores (LookML Files)
Source Code: Provides the LookML code for the views.
Euno Compiled Code: This is divided into two code blocks: one written in SQL, transpiling all dimensions and columns in the LookML into an SQL query, and the second, a semantic block, transpiling all measures into MetricFlow measures.
Tableau custom queries
Source Code: Provides the SQL used to generate the custom query
Sub-resources:
Measures & Related Metrics Under dbt Models
Source Code: Displays the MetricFlow code for the measure or related metric.
Compiled Code: Presents an SQL transpilation of the MetricFlow code, suitable for copying and execution in an SQL notebook.
Dimensions & Measures in Looker Under Views
Source Code: Provides the LookML code for the specific dimension or measure in LookML.
Custom Fields in Looker Under Looks
Source Code: Displays the Looker expression used to create the calculated field.
Custom Fields in Looker Under Tiles
Source Code: Displays the Looker expression used to create the calculated field.
Calculated Fields in Tableau in publshed data sources
Source Code: Displays the Tableau expression used to create the calculated field.
Calculated Fields in Tableau in Embedded data sources
Source Code: Displays the Tableau expression used to create the calculated field.
In the code pop-up window, there's a 'Copy' button to quickly copy the displayed code. For dbt semantic layer elements like measures and metrics, users can copy the transpiled SQL code to run in a notebook, producing the same results as the shared data model.
The Overview tab is the default tab and is relevant to all resource types. It includes the following sections:
Resource Description (if applicable)
Information Box: Contains essential details such as the owner of the resource, timestamps for creation and last update, data on Tableau Extracts, and other relevant information.
Tags, Metadata & Organizational Information (if applicable)
Usage Information: Provides a concise summary of the usage data found in the Usage tab. Refer to the Usage tab for more details.
Connections Section: Relevant for resource types with upstream and downstream connections. Displays a mini DAG showing the resource along with its direct connections. In the upper right corner, a "View in Data Model" button allows users to access the DAG view for a focused exploration of the resource's upstream and downstream connections.
Contents Section: Applies to resource types that function as containers for other resources. Displays the contents of the resource, organized and grouped by type. In the upper right corner, a "View in Data Model" button allows users to access the DAG view for a focused, detailed exploration of the resource's contents.
This tab presents the resource's schema, organized into sub-resource types: Columns, Dimensions, Measures, and Entities. Each entry (e.g., column, dimension) includes details such as type, shift-left applicability, primary key, and calculated field indicators. Users can click on specific components to open their side panel for a deeper understanding. Additionally, a global search function enables users to quickly find items by name or apply quick filters for more efficient navigation.
For visualization resources (e.g., Looker Looks, Looker Tiles, Tableau Views), the Schema tab is replaced by the Used Fields tab. This tab lists all fields, including calculated fields, actively used within the visualization and mirrors the functionality of the Schema tab, with features like grouping, global search capabilities, and navigation to a component's side panel for detailed insights. This design ensures users can efficiently explore and understand the specific fields involved in their visualizations.
This tab is designed for container-type resources that hold other resources, organizing their contents by resource type. A global search function enables users to efficiently find items by name. Users can also select a specific component to open its side panel for more detailed information.
This tab organizes the resource's direct connections into upstream and downstream groups and includes a global search by name. Users can select specific components to open their side panel for additional details. The tab also displays a count of all upstream and downstream connections, along with a button that redirects users to a comprehensive tabular view for further analysis.
The Usage tab provides comprehensive insights into the usage patterns of each resource, including views, queries, and persistent derived table (PDT) builds, helping users understand how resources are utilized and their relative importance.
Views (Looker Impressions): For specific resources (e.g., Looker Looks and Dashboards), we gather impression data directly from your BI tool to assess activity levels. This data helps determine the relevance of each resource by showing whether associated reports are actively viewed. Users can apply filters like "Total Views 7D" or "Total Views 30D" to narrow down the graph, highlighting resources within your application layer that receive significant attention.
Query Insights: Usage information for key resources and fields (e.g., Looker Looks, Tiles, Dashboards, LookML Views, and the measures and dimensions beneath them; in Tableau: Views, Dashboards, Data Sources, and Workbooks) is derived from query logs in your BI tool or Data Warehouse (DW). This data indicates the frequency of resource and field usage over the last 14, 30, or 60 days, offering insights into their relevance and recent query activity. Users can further explore queried sources to see which reports triggered specific queries and identify users who accessed the data. To streamline analysis, a "Total Queries 14D" filter allows users to focus on the most recent two weeks of activity in the side panel. For a broader overview, the 30-day and 60-day query data can be accessed in the tabular view, enabling a deeper analysis of long-term trends.
Persistent Derived Table (PDT) Builds: For Looker views with a persistent trigger, the Usage tab displays both the total number of PDT builds and the cumulative build time over the last 30 days. This data provides insight into the frequency and total duration of recent PDT refreshes, helping users monitor the processing load and efficiency of these derived tables.
This tab contains information such as the dbt version, native ID, Git hosting parameters (repository URI, branch, revision), Looker connection name, and more.
To customize the tabular view, click on the customize icon found in the top left corner of the table. Clicking the icon will expand a list of the columns found in the table by default. To remove a column, click on the remove icon found next to the specific column ("Type" and "Name" cannot be removed). To add a new column, click " Add column" to display a list of available columns; clicking on a specific column will add it to the table (a maximum of 9 columns can be displayed at once). To change the order of the columns, click and hold a column in the customize list and move it to the desired location.
If a filter has been applied to the data model screen, only the resources relevant to the filter will be displayed. Users can expand the graph by displaying the downstream or upstream connections of a filtered resource by clicking on the arrows attached to the resource. These arrows allow the user to expand the graph in either direction all the way or only to direct connections . Once clicked, all connections in that direction (upstream or downstream) will be displayed. Additionally, a tile displaying the connection direction along with the count of times that direction has been expanded will appear next to the search bar. Users can remove a tile to eliminate all connections displayed in that direction. Similar to the search bar, these tiles are only relevant to the graph view and do not affect the resources displayed in the tabular view.
