〰️Model changes
Your organization's data model is in a constant state of evolution, and at the core of that are changes to the shared data model. Euno helps data users, both analysts and members of the central team to facilitate this process by identifying logical elements outside your dbt model, and suggests these elements as proposals to be pushed upstream to your dbt model.
As mentioned in the integrations section under Looker integration, Euno utilizes Looker integration to crawl Looker and identify dimensions, measures, and custom fields within Looker. Based on the identification of these elements, Euno suggests shifting them to dbt instead of keeping them exclusively within Looker for Looker users. Euno not only identifies these elements for promotion in the form of a proposal, but also automatically generates the requirements, generates the code, and syncs the changes back into the BI application if the data sync has been configured.
Euno currently supports proposals for the following change type:
Add a column to an existing dbt model from:
A dimension in a Looker View
A custom dimension in a Looker Look
A custom dimension in a Looker Tile
Add a measure to a dbt semantic model from:
A measure in a Looker View
A custom measure in a Looker Look
A custom measure in a Looker Tile
Add a model to dbt from:
A LookML derived view (only SQL based derived views are supported)
Once identified by Euno, these change proposals can be found in two locations:
The Data Model Screen
The Proposals Screen under the Model Changes section (In a specific project)
Proposals
Data Model Screen
A user can then click the node or row in the table to expand the node's sidebar. Once expanded, the icon will be located next to the components associated with the proposals. Users can then directly select a specific component with the proposal, opening the sidebar for that specific component. In the component, users will see a banner titled 'Shift to dbt Review change draft', which, if clicked, will trigger the Review Change Draft dialogue box detailed below.
Proposals Screen
The proposal tab, found under the Model Changes section of a project is where you can view all the logic identified by the Euno crawlers from your integrated BI applications that are associated with a specific project. It serves as a platform for users to review and evaluate proposed suggestions, determining their viability as potential change proposals. Users can analyze the details of each proposal, considering factors such as relevance, feasibility, and alignment with the organization's goals. This screen acts as a preliminary filter, ensuring that only well-considered and relevant ideas progress to the next stages of the change management process.
The proposals table is divided into six columns, each serving a specific purpose:
Proposal Number: An identifier assigned by Euno for the proposal, determined by the chronological order in which the change was detected.
Type: The nature of the change proposed by Euno; for instance, "Add Column," indicating a suggestion to incorporate a new column into an existing dbt data model.
Title: An automatically generated title by Euno, offering additional context regarding the suggested actions in the proposal.
Source: The origin or location where the logical element was identified.
Owner: The individual from the BI application who crafted the logic based on the suggestion; represented by their username in the BI application.
Date: The timestamp indicating when Euno generated the proposal.
In order to facilitate identifying the specific suggestions, Euno allows searching for proposals based on owner, name of the field, source (i.e. Look’s name) as well as filtering the list by time of suggestion (please note the table is sorted in chronological order from when the logical element was identified by Euno).
Clicking on a specific suggestion will trigger the Review Change Draft dialogue box, detailing all the change proposal details, allowing also editing the proposal.
Review Change Draft
In the “Review Change Draft” dialogue box, the following information will be displayed:
Proposal Title: By default Euno auto-generates a proposal title, but this can be edited.
Proposal Description: By default Euno auto-generates a proposal description, but this can be edited.
Change Type: “Add Column”/”Add Measure”/"Add Model", can’t be edited.
Description: This is the description of the new column, which will be added to the dbt model. By default Euno auto-generates a description, but this can be edited.
Name: The name of the column being added to the dbt model. By default Euno will use the name of the field pulled from the BI application. This can be edited.
Target Branch: The branch in the repository the change will be pushed to. By default Euno will select the Main branch from the repository the Euno project is configured to. This can be edited.
Tags: By default added by Euno, but can be edited by the user. This can be edited.
Meta: By default Euno will add the key owner, and the name or user_name of the source owner. Additional metadata can be entered by inputting a key in the text box, and the value next to the key.
Tests (Not supported for measures or models) : Currently Euno supports the four generic tests from dbt
Unique: By default not filled out by Euno, but can be edited by the user
Not Null: By default not filled out by Euno, but can be edited by the user
Accepted Values: A list of strings, which by default is empty, but can be edited by the user.
Relationships: By default not filled out by Euno, but can be edited by the user. The user will select from a dropdown list the table, and then from a second dropdown list the column.
Once satisfied with the proposal, clicking the “Create change draft” button will transform the proposal into a change draft.
Change Draft Screen
The Change Draft screen is a fundamental component in the change management process. It provides insight into the change, and its associated properties used to manage the workflow associated with the lifecycle of the change.
A change has five different stages:
Open - This is the first stage of change, after this is chosen to become a change draft. This is the point in the lifecycle where the change can be altered within the Euno web application.
Pending merge - Once a pull request has been created, the change will move to the “Pending merge” stage, in which a member of your organization will need to merge change from outside of Euno.
Merged - Once the pull request has been merged, and the code committed to your dbt repository, the change stage will be automatically moved to the “Merged” stage.
Deployed - Once your integrated orchestration tool has conducted a job that has built the transformation successfully, which includes the newest change on the database, the change stage will be automatically moved into the “Deployed” stage.
Published - Once the change has been synced with the configured data application. If no data application has been configured, the change will skip the deployed state and skip directly to “Published”, once the transformation has been completed.
This screen is split into four sections:
Change Details
dbt Change Overview
Implications
Code
Change Details Section
The details of the change draft should include the following information (outlined in red in the image below):
Change number - Unique identifier
Change title
Description of the change
Created by - Euno user who created the change draft
Timestamp of change draft creation
Last edited by - Euno user who last edited the change draft
Timestamp of the last edit
GitHub link - Once the pull request has been created
Build link - Once the dbt build has been deployed
In addition to the above, alongside the change details is a progress bar indicating the current stage of the change's lifecycle, specifically highlighting the critical path (Open->Pending merge->Merged->Deployed->Published). If the change is outside the critical path (i.e. “Rejected”), the general progress should not be displayed, but a message about the state.
If the change is in “Open” stage, a button labeled "Create pull request" can be found beneath the lifecycle progress bar, along with an option to reject the change. Clicking the and selecting the "Reject proposal" option will trigger a confirmation dialogue box, allowing you to confirm your decision to reject the change.
Once satisfied with the content of the change, you can click the "Create pull request" button found underneath the lifecycle progress bar. Upon clicking, a popup message will request confirmation to proceed, with options to cancel or continue. If "cancel" is chosen, the dialogue box should close without making any changes to the proposal. If "continue" is selected, the dialogue box should display the progress of creating the pull request, including three steps: Commit branch, Commit change, and Submit pull request. Once finished, the dialogue box should provide a link to the pull request in GitHub. Users can exit the dialogue box at any point by clicking "Done".
Once the change moves from the “Open” to “Pending Merge”, the “Create Pull Request” button will be replaced with a button called “View Pull Request”, which will link to the pull request on Github.
After the change has been “Merged”, and the orchestration tool has triggered the build, and the transformation has been completed successfully in the data warehouse, the “View Pull Request” button will be replaced with a button called “View Build”, which will link to the build section of the project screen. A link for viewing the pull request can be accessed at any point by clicking the Github icon on the change details upper section.
dbt Change Overview Section
The change draft details holds critical written contextual information regarding the change attributes itself. Information such as the type of change, the metadata associated with the change and tests related to the change would be found in this section.
While in the “Open” stage, and prior to creating a pull request, the above information can be edited by clicking on the “Edit” button found in the top right corner of the screen found above the “Change details” section. If clicked, a dialogue box will be displayed where the user can edit the details of the change, similar to in the proposal screen.
Implications Section
The Data model cection provides a graphical description of the change, displaying the change to the dbt model. In the graphical description a button called “View in data model” can be clicked, linking to the data model screen in order for the user to understand in a broader context where the change is being placed.
Code Section
The code section includes separate blocks of code for each of the different files being altered, both the YAML file with the contextual information, and the SQL file with the transformations. The code block will include a diff between the code prior and after the change, with whatever is being removed highlighted in red, and the additions highlighted in green.
At the moment, this screen is not editable, and once an actual pull request has been created, a link to the GitHub pull request can be found above the code block. Once the pull request has been merged, the user will no longer be able to see the code block through Euno. Instead, the user will see the following message with a link to the pull request:
In Progress Tab
The “In Progress” tab, found under the Model Changes section of a project is designed to provide insight into the current lifecycle stage each change is in. The screen includes changes which are currently “In progress”, meaning in the following stages of the change lifecycle:
Open
Pending merge
Merged
Deployed
This screen includes a table of all the changes in the above stages, and includes the following columns:
Change title
Created by - The Euno user who created the change
Last time the change was edited
Source of change
Timestamp when the change was created
Stage of lifecycle
This table can be searched, and filtered to assist the user in finding the change they were looking for. The filters include:
Change lifecycle stage
Timeframe of last edit
Timeframe of creation
Opened by
Archive Screen
The “Archive” screen, found under the Model Changes section of a project is identical to the “In Progress” screen, including functionality with the one exception, only showing changes which are in the two following states:
Published
Draft Rejected
Last updated
