dbt Cloud

Euno's integration with dbt Cloud supports auto-discovery of dbt resources. It automates the retrieval and processing of dbt artifacts from completed runs. This allows for seamless data synchronization and analysis using the latest available dbt job runs.

How It Works

The integration follows these steps:

1. Retrieve Jobs from dbt Cloud

   • Using a dbt Cloud service token, the integration queries the dbt Cloud API to fetch all jobs associated with the single dbt Cloud project specified by the dbt Cloud Project ID in the integration configuration. The integration does not iterate over other projects that may be accessible via the service token; to cover multiple projects, create a separate Euno integration per project.

2. Find the Latest Unprocessed Run for Each Job

   • For each job, the integration identifies the latest run that has completed successfully and has not yet been processed by Euno.

3. Retrieve Artifacts from the Run

   • Once an eligible job run is found, the integration retrieves its associated artifacts:

     • run_results.json

     • manifest.json

     • semantic_manifest.json

     • catalog.json

4. Process the Artifacts

   • The integration processes the retrieved artifacts to extract relevant information and adds the discovered resources to Euno's data model.

Runs are started by a schedule, Run now in the Euno UI, or an on-demand Trigger URL (for example, wired to a dbt Cloud job.run.completed webhook). See Triggering integration runs.

Advanced Configuration

Filtering Jobs

It's possible to define patterns to include or exclude jobs based on:

• Job names

• Environments

• Branch names

Without Advanced Filtering

If no filtering patterns are set, the integration will process all jobs retrieved from dbt Cloud. This means every available job in the configured project will be considered.

With Advanced Filtering

By defining inclusion/exclusion patterns, users can:

• Focus on specific jobs that match certain naming conventions.

• Exclude jobs from non-production environments.

• Restrict processing to specific Git branches (e.g., main or production).

This provides better control over which jobs are processed, reducing unnecessary artifact downloads and processing time.

Prerequisites

• A dbt Cloud service token with sufficient Read-only permission. See dbt Cloud's documentation on how to create service tokens.

• The service token should grant access to the dbt project with the job run(s) to retrieve. A single token with access to multiple projects can be reused across separate Euno integrations.

Setting up Euno's dbt Cloud Integration

Step 1: Configure New dbt Cloud Source in Euno

Step 1: Access the Sources Page

1. Navigate to the Sources page in the Euno application.

2. Click on the Add New Source button.

Step 2: General Configuration

Asterisk (*) means a mandatory field.

Configuration	Description
Name*	Enter a name for your dbt Cloud source (e.g., "dbt - Marketing Models")
dbt Cloud Account ID*	Enter you dbt Cloud Account ID. It can be found in the URL, e.g. https://cloud.getdbt.com/settings/accounts/{account_id}/pages/projects/{project_id}
dbt Cloud Project ID*	Enter you dbt Cloud Project ID. It can be found in the URL, e.g. https://cloud.getdbt.com/settings/accounts/{account_id}/pages/projects/{project_id}
dbt Cloud Api Token*	Enter the service token created in the previous section
dbt Cloud Job ID (optional)	Specific dbt Cloud job ID to process. If not specified, all jobs under the dbt project associated with this integration will be processed.

Step 3: Resource Cleanup Options

To keep your data relevant and free of outdated resources, Euno provides automatic resource cleanup options. These settings determine when a resource should be removed if it is no longer detected by a source integration. For a detailed explanation on Euno's cleanup strategies, see: Resource Sponsorship in Euno.

• Time-Based Cleanup (default): Remove resources that were last detected X days before the most recent successful source integration run (user-defined X, default is 7 days).

• Immediate Cleanup: Remove resources not detected in the most recent successful source integration run.

• No Cleanup: Keep all resources indefinitely, even if they are no longer detected.

Step 4: Scheduling Updates

1. Enable the Schedule option.

2. Choose:

   1. Weekly: Set specific days and times.

   2. Hourly: Define the interval in hours (e.g., every 8 hours).

Regenerating the Trigger URL

Euno shows the dbt Cloud Trigger URL once when the source is created. If you need to recover or rotate it, go to Sources, open the dbt Cloud source's menu (⋯), and select Generate Trigger URL. You can also generate a new Trigger URL from the source details page. Generating a new URL immediately invalidates the previous URL, so update any dbt Cloud webhooks to use the new value.

Step 5: Advanced Settings (Optional)

Click on the 'Advanced' section to display these additional configurations.

Configuration	Description
Job Name	Define patterns to include or exclude jobs based on their names.
Job Environment	Define patterns to include or exclude jobs based on their environment.
Job Branch	Define patterns to include or exclude jobs based on their branch name.
dbt Cloud URL	If you have a dedicated URL to access dbt Cloud enter it here. Defaults to "https://cloud.getdbt.com"
Override Build Target	Override the auto-detected warehouse URI prefix. Use when the warehouse account identifier reported by dbt Cloud differs from the one used by the warehouse integration (e.g. Snowflake account locator vs organization account name). Accepts a Snowflake host (e.g. account.snowflakecomputing.com), a Databricks host, or an already-canonized prefix (e.g. snowflake.myorg-myaccount).

Step 6: Save Configuration

Click Test & Save (or Save). Euno generates a Trigger URL for the new source. Copy and store it securely — it is shown only once. You will need it if you want dbt Cloud (or another orchestrator) to start an Euno integration run immediately after a dbt job finishes.

Triggering integration runs

Euno can pick up new dbt Cloud runs in three ways:

1. Schedule — Enable scheduling in Step 4 so Euno polls dbt Cloud on a recurring interval.

2. Run now — From the Sources page, open the source menu (⋯) and choose Run now, or use Run now on the source details page.

3. Trigger URL — Send an HTTP POST to the Trigger URL to start a crawl on demand. This is the URL to configure as the target of a dbt Cloud webhook when you want Euno to sync right after a job completes.

Trigger URL format

When you create the source, Euno returns a URL in this form:

https://{euno-api-host}/accounts/{account_id}/integrations/{integration_id}/crawl?trigger_secret={secret}

Replace {euno-api-host} with your Euno API hostname (api.app.euno.ai in production). This is the API host, not the Euno web app (app.euno.ai). {account_id} and {integration_id} are the numeric IDs from the URL path when you view the source in Euno. {secret} is a one-time secret embedded in the query string.

Example:

https://api.app.euno.ai/accounts/42/integrations/17/crawl?trigger_secret=a1b2c3d4e5f6789012345678901234ab

To trigger a run, send POST to that URL. Euno ignores the request body (including JSON payloads from dbt Cloud webhooks) and authenticates using trigger_secret.

Alternative authentication (without the query parameter):

curl -X POST "https://api.app.euno.ai/accounts/42/integrations/17/crawl" \
  -H "X-Trigger-Secret: YOUR_SECRET"

If you lose the Trigger URL, you can still use Run now in the Euno UI. Rotating the secret requires generating a new Trigger URL through your Euno account administrator.

Configure a dbt Cloud webhook

To have dbt Cloud notify Euno when a job finishes, create a webhook subscription in dbt Cloud that POSTs to your Euno Trigger URL.

1. In dbt Cloud, go to Account Settings → Webhooks (or use the Webhooks API).

2. Create a subscription with:

   • Event type: job.run.completed — use this event (not job.run.errored) so job metadata and artifacts are available in the dbt Cloud API before Euno runs.

   • Client URL: your full Euno Trigger URL, including the trigger_secret query parameter.

   • Job IDs: limit to the jobs this Euno source should process, or leave empty to fire on every job in the account (Euno still only processes jobs in the configured dbt Cloud project and filters).

3. Save the subscription. dbt Cloud sends a POST request to the Client URL after each matching run completes.

Example subscription payload (API):

{
  "name": "Euno sync",
  "description": "Trigger Euno dbt Cloud source after job completion",
  "client_url": "https://api.app.euno.ai/accounts/42/integrations/17/crawl?trigger_secret=YOUR_SECRET",
  "event_types": ["job.run.completed"],
  "active": true,
  "job_ids": [123456]
}

Each webhook call starts an Euno crawl. The crawl fetches the latest unprocessed run per configured job, downloads artifacts from dbt Cloud, and updates Euno's data model. For production, combine webhooks (near-real-time sync after builds) with a schedule (fallback if a webhook delivery fails).