Resource Sponsorship and Cleanup in Euno
Last updated
Last updated
In Euno, every resource is linked to one or more sponsors, which represent the integrations that have detected and recorded the resource. Sponsorship provides insight into a resource’s origin and whether it is still relevant.
A sponsor is an integration that has detected a resource during its execution. For example, a dbt Cloud integration that scans dbt builds will act as a sponsor for the models it detects.
A resource can have multiple sponsors if different integrations have observed it independently.
Example:
A Looker integration named looker_org_wide_scan scans all LookML projects once a week. It detects a Looker view named customer_orders_view, so looker_org_wide_scan is added as a sponsor for that resource.
Separately, another Looker source integration named looker_sales_project_scan, which scans a specific LookML project daily, also detects customer_orders_view. As a result, both integrations are listed as sponsors:["looker_org_wide_scan", "looker_sales_project_scan"].
If a source integration is deleted its sponsorship is removed from all relevant resources and this will also trigger a .
Each resource tracks when it was first and last observed by any source integration. These timestamps help determine whether a resource is still active.
sponsors: A list of integrations that have observed the resource.
first_observed: The date the resource was first detected by any integration.
last_observed: The most recent date an integration detected the resource.
To keep your data relevant and free of outdated resources, Euno provides automatic resource cleanup options. These settings determine when a resource should be removed if it is no longer detected by an integration run.
For dbt Cloud/Core, users can choose from:
Time-Based Cleanup (default): Remove resources that were last detected X days before the most recent successful source integration run (user-defined X, default is 7 days).
Immediate Cleanup: Remove resources not detected in the most recent successful source integration run.
No Cleanup: Keep all resources indefinitely, even if they are no longer detected.
For all other sources users can choose from:
Immediate Cleanup (default): Remove resources not observed in the most recent successful source integration run.
No Cleanup: Keep all resources indefinitely.
Immediate Cleanup (After Each Successful Run) After an integration completes a run, Euno removes that integration from the sponsorship of all resources except those detected in the latest run.
If a resource no longer has any sponsors, it is completely removed from the system.
Example: A Tableau Dashboard is Removed
A Tableau integration detects a dashboard annual_revenue on 2025-02-01.
The integration runs again on 2025-02-10, but annual_revenue is no longer found.
If immediate cleanup is enabled, annual_revenue is removed from the system.
Time-Based Cleanup (X Days Before the Most Recent Successful Run, dbt Only) After each successful run, Euno checks if any resources were last observed X days before the most recent run. If so, Euno removes that integration from their sponsorship.
If a resource no longer has any sponsors, it is completely removed from the system.
Example: A dbt Core Model Becomes Outdated
A dbt Core integration observes a model daily_revenue on 2025-01-15.
The integration runs regularly, and the most recent successful run occurs on 2025-03-10.
If the staleness policy is set to 30 days, any resources last observed before 2025-02-08 (30 days before 2025-03-10) will be removed.
Note: cleanup happens at the end of a successful source integration run. An integration that has a time-based cleanup will not trigger cleanup unless it is run either periodically or manually.
By configuring resource cleanup settings, users can ensure their data remains up to date while automatically removing outdated resources.