🖥️Using the Data Model Screen
Last updated
Last updated
The Data Model screen serves as your guide to your organization's shared data model along with the integrated BI applications. This feature within our web application is more than a mere tool; it's a robust platform designed to simplify the comprehension of complex relationships and connections within your organization's data structures. The Data Model Screen stands as a comprehensive tool, empowering users to interact with and discover the intricacies of the underlying data structure intuitively.
This screen is composed of interconnected resources. Every asset in your data model, such as a table, a column in a table, a Chart in your BI tool, or a data transformation, is a resource.
Resources can connect to each other in different ways. For example, a column is a child of a table. A database view can depend on another database view, etc. To learn more about resources and the different ways they can relate to one another, see Relationships.
The Universal Resource Location is the unique identifier for every resource within the mapped data model. The URI is used to map location of resources within the data model, and their connections. To learn more about URIs, see Universal Resource Identifier (URI).
To easily access a URI , open a resource's sidebar and click the "Copy Link" button located next to the resource's name. This feature allows you to quickly obtain the URL linking to the data model screen and the specific resource. The URI can be found within the URL by removing the initial part: "https://app.getdelphi.io/data-model?view=graph&highlightedNode=".
The URI can be used to filter out the data model screen by a specific resource, using the URI filter, or it can be used to help diagnose connection mapping issues. In order to diagnose mapping issue between resources view the compiled code, to see the URI referenced in the SQL, and compare it to the parent resource's URI (from its URL). If the two URI do not match, contact customer support for assistance.
Aside from the identifier, and the relationship to other resources, each resource in Euno has properties. For example:
a table has a name
a column has a data type
a chart in a BI tool has a link
Properties are used to investigate the data model, can be used as filters to customize which resources are displayed in the Tabular or Graph view, or as columns in the Tabular view.
There are two classes of properties: built-in properties and custom properties. Built-in properties are available for all Euno accounts, and their value is set by integrations. Custom properties are account specific. To create custom properties in your account contact support.
For a complete list of built-in properties, see Built-In Properties.
The Data Model screen presents two distinct views: Tabular View and Graph View, both representing the same information. Resources within your data model are displayed on this screen.
The resources on the data model screen can be filtered using any of the available properties, built-in or custom. For a complete list of built-in properties, see Built-In Properties.
Custom properties: Through a service option, accounts can define custom properties based on the built-in properties. These custom properties will be displayed in the Tabular view and can also be used as filters. For further details, contact customer support.
If multiple filters are applied simultaneously, the operator between the two filters is “AND”, indicating that both criteria must be met for a resource to satisfy the filtered conditions.
The main difference between the Table and Graph view is that in the Graph view, users will not only see the relevant resources based on the filters set, but also their upstream and downstream connections, providing users with the resources and their broader context.
The tabular view is a table which display's all resources applicable to the selected filters. This table includes the following columns:
Data type: The type of resource (e.g. dbt model, view, etc…)
Name: The name of the resource
Description: The description of the resource
Columns: Number of columns under the resource
Dimensions: Number of dimensions under the resource
Entities: Number of entities under the resource
Related Metrics: Number of related metrics associated with the resource
Custom Fields: Number of custom/calculated fields under the resource
Proposals: If the resource has a change proposals
Users can click on a specific row to view the resource’s sidebar, which holds the contextual information for the resource (e.g. dbt model), and their components (e.g. columns).
This view allows users to delve deeper into their organization's shared data model by displaying resource as nodes on a graph. By clicking and highlighting a specific resource, users can better understand the structure of their data model and how that resource is connected within the larger model. Once a resource is clicked and highlighted, its downstream and upstream connections are also highlighted. Any resource that isn’t directly connected with the selected model will be overlaid, ensuring that only information relevant to that resource is displayed. This allows users to easily understand both upstream and downstream implications.
Note - Certain resources are not displayed as a node in the graph, but rather as a property of other resources (i.e. Tableau workbook are not displayed as a node, rather as a property of other Tableau resources)
The Graph view groups resources into four vertical swimlanes;
Sources: This includes resources that are dbt sources.
Transformation: This includes resources that are dbt models as well as orphan resources - Orphan resources refer to tables in your database that are referenced by a BI application but are not part of your dbt project, and therefore, we have limited information about these tables, other than their existence.
App Modeling: This includes transformation-like resources within the BI application (e.g., Views and Explores in Looker, or custom queries, published or embedded data sources in Tableau).
Application: This includes resources in the BI application that are related to visualizations (e.g., Looks, Tiles, and Dashboards in Looker, or views, dashboards, and stories in Tableau).
Any connection resource that does not match the filter will be displayed with lower opacity, making it easy to distinguish between connection resources and those that match the filters. Additionally, filtered resources cannot be removed from the graph view unless they are filtered out.
Users can efficiently search for resources and their components in the Graph view using the search bar located in the top right corner of the screen next to the view selector. The search functionality enables users to input free-text, instantly displaying a list of relevant resources, and their components in the shared data model. The search dynamically filters through elements as the text expands, allowing for quick and effective exploration. Users can select a resource, or a component from the list, which will highlight the resource or component in the Lineage view, and open the sidebar for the resource, or component.
In order to understand additional information about a specific model, users can view the schema by clicking on the sidebar icon found above the model once highlighted (see image below). This sidebar will provide user insight and information into the specific model. Users can also directly select a specific component (e.g. column, dimension, etc…) when the model is expanded, which will open the sidebar view with the information on the specific component. The sidebar feature also works with BI applications, and provides users with information on the connected BI element.
At the top of the sidebar, beneath the resource's description, there's a button to access its source. Clicking this button redirects the user to the resource's respective source. Depending on the resource type, users will be directed to different sources. For instance, Views, Explores, Looks, Tiles and Dashboards redirect to the organization's Looker account, while sources and models redirect to their dbt code repository.
By clicking on the code button (</>) situated next to the open source button, a window opens, revealing the resource’s code (if available), offering an insightful view into the code that shapes the element. The popup window with the transformation code, includes a toggle which allows for toggling between the source and compiled code.
In addition to being available at the resource level, certain components also provide access to their source code. Below is a list of resources and the corresponding components supporting the 'View Code' button:
Resources:
dbt Models
Source Code: Displays the compiled dbt code for the models.
Compiled Code: Shows the compiled dbt code for the models.
Semantic Code: If the dbt model includes a semantic representation, the semantic code tab will be available. This utilizes the dbt semantic layer, written in MetricFlow.
Views & Explores (LookML Files)
Source Code: Provides the LookML code for the views.
Euno Compiled Code: This is divided into two code blocks: one written in SQL, transpiling all dimensions and columns in the LookML into an SQL query, and the second, a semantic block, transpiling all measures into MetricFlow measures.
Tableau custom queries
Source Code: Provides the SQL used to generate the custom query
Sub-resources:
Measures & Related Metrics Under dbt Models
Source Code: Displays the MetricFlow code for the measure or related metric.
Compiled Code: Presents an SQL transpilation of the MetricFlow code, suitable for copying and execution in an SQL notebook.
Dimensions & Measures in Looker Under Views
Source Code: Provides the LookML code for the specific dimension or measure in LookML.
Custom Fields in Looker Under Looks
Source Code: Displays the Looker expression used to create the calculated field.
Custom Fields in Looker Under Tiles
Source Code: Displays the Looker expression used to create the calculated field.
Calculated Fields in Tableau in publshed data sources
Source Code: Displays the Tableau expression used to create the calculated field.
Calculated Fields in Tableau in Embedded data sources
Source Code: Displays the Tableau expression used to create the calculated field.
In the code pop-up window, there's a 'Copy' button to quickly copy the displayed code. For dbt semantic layer elements like measures and metrics, users can copy the transpiled SQL code to run in a notebook, producing the same results as the shared data model.
For specific resources (Looker Looks and Dashboards), we crawl your BI tool to gather impression data. This process enriches the resource with valuable context, helping to provide context as to the importance and whether the associated report is actively utilized. Through the application of filters like "Total Views 7D" or "Total Views 30D," users can refine the displayed graph to discern which resources within your application layer are receiving attention.
For specific resources and fields (in Looker: LookML Views, and the measures and dimensions underneath them; in Tableau: Views, Dashboards, Data Sources, and Workbooks), we harness data derived from the query log within your BI tool (whether through a native integration, or manual upload) or your Data Warehouse (DW) to provide usage insights. This approach offers valuable information on how frequently the resource or field has been utilized in queries over the last 14 days, offering key insights into its relevance and usage patterns within recent activity.
Additionally, users can explore further details by expanding this section to view the top 5 queried sources, revealing which reports triggered the queries, as well as identifying the top 5 users who have queried this information.
To streamline analysis, users can apply the filter "Total usage 14D" to refine the displayed resources in the sidebar, focusing on the most recent two weeks of activity. For a broader view, usage over 30 days and 60 days can be found in the tabular view, allowing for deeper analysis of trends over a longer period.
To customize the tabular view, click on the customize icon found in the top left corner of the table. Clicking the icon will expand a list of the columns found in the table by default. To remove a column, click on the remove icon found next to the specific column ("Type" and "Name" cannot be removed). To add a new column, click " Add column" to display a list of available columns; clicking on a specific column will add it to the table (a maximum of 9 columns can be displayed at once). To change the order of the columns, click and hold a column in the customize list and move it to the desired location.
If a filter has been applied to the data model screen, only the resources relevant to the filter will be displayed. Users can expand the graph by displaying the downstream or upstream connections of a filtered resource by clicking on the arrows attached to the resource. These arrows allow the user to expand the graph in either direction all the way or only to direct connections . Once clicked, all connections in that direction (upstream or downstream) will be displayed. Additionally, a tile displaying the connection direction along with the count of times that direction has been expanded will appear next to the search bar. Users can remove a tile to eliminate all connections displayed in that direction. Similar to the search bar, these tiles are only relevant to the graph view and do not affect the resources displayed in the tabular view.
