Dagster Upload

This guide shows how to run your dbt Core project with Dagster (using the dagster-dbt integration) and automatically upload the resulting artifacts to Euno after every successful build.

Euno's dbt Core integration is orchestrator-agnostic: it only needs your dbt artifacts (manifest.json, catalog.json, run_results.json, and optionally semantic_manifest.json) delivered through the two-step prepare-upload flow. Dagster is therefore fully compatible — it is responsible only for running dbt and ordering the upload after the build. The same upload logic used in the Python, GitHub Actions, and Prefect guides applies here.

Prerequisites

• Python 3.9+

• A dbt Core project you already run with Dagster (dagster + dagster-dbt)

• Euno trigger secret and endpoint URL for your dbt source

pip install "dagster" "dagster-dbt" "dbt-core" "dbt-<your-adapter>" "requests"

Replace <your-adapter> with your warehouse adapter, e.g. dbt-snowflake, dbt-bigquery, dbt-redshift, or dbt-databricks.

Configuration

Provide these as environment variables (recommended) so secrets stay out of code:

export EUNO_ENDPOINT_URL="https://api.app.euno.ai/accounts/YOUR_ACCOUNT_ID/integrations/YOUR_INTEGRATION_ID/prepare-upload"
export EUNO_TOKEN="your_trigger_secret_here"   # Bearer token from your dbt source settings
export DBT_PROJECT_DIR="/path/to/your/dbt/project"
export DBT_PROFILES_DIR="/path/to/your/dbt/profiles"   # optional

How it works

The example defines three things:

1. euno_dbt_assets — a @dbt_assets definition that runs dbt build (producing manifest.json and run_results.json) and then dbt docs generate (producing catalog.json, which Euno requires).

2. euno_artifacts_upload — an asset that deps on the dbt assets, so it runs only after a successful build. It zips the artifacts from target/ and uploads them to Euno.

3. A job + daily schedule wiring the two together.

The upload is the same two steps used everywhere in Euno's dbt Core integration:

• Step 1 – prepare-upload: POST the zip filename to your endpoint with the Bearer token and receive a short-lived signed URL.

• Step 2 – upload: PUT the zip to that signed URL.

The helper handles both prepare-upload response shapes Euno may return ({"body": {"upload": {"url": ...}}} and {"upload": {"url": ...}}).

Example

A complete, ready-to-adapt module is provided here: dagster_euno_upload.py.

The key parts:

from dagster import AssetExecutionContext, AssetKey, Definitions, ScheduleDefinition, asset, define_asset_job
from dagster_dbt import DbtCliResource, DbtProject, dbt_assets

dbt_project = DbtProject(project_dir=DBT_PROJECT_DIR)
dbt_project.prepare_if_dev()

@dbt_assets(manifest=dbt_project.manifest_path)
def euno_dbt_assets(context: AssetExecutionContext, dbt: DbtCliResource):
    yield from dbt.cli(["build"], context=context).stream()      # manifest.json + run_results.json
    dbt.cli(["docs", "generate"], context=context).wait()        # catalog.json

@asset(deps=[euno_dbt_assets])
def euno_artifacts_upload(context: AssetExecutionContext) -> None:
    target_dir = Path(dbt_project.project_dir) / "target"
    zip_path = _zip_artifacts(target_dir)                        # zips the 3-4 artifact files
    upload_url, _ = _get_signed_url(endpoint_url, token, Path(zip_path).name)  # step 1
    _put_file(zip_path, upload_url)                              # step 2

Integrating with an existing dagster-dbt project

If you already have a @dbt_assets definition, you don't need to rewrite it. Just:

1. Add a dbt docs generate call after your dbt build/dbt run so catalog.json is produced (Euno needs it for schema information).

2. Copy the upload helpers and the euno_artifacts_upload asset from the example, pointing deps at your dbt assets definition.

3. Add euno_artifacts_upload to your job/Definitions and select it alongside your dbt assets.

Notes

• manifest.json and catalog.json are required; run_results.json and semantic_manifest.json are included when present.

• dbt build can be replaced with dbt run (or dbt compile) depending on your pipeline — just make sure dbt docs generate still runs so catalog.json exists.

• To run on a schedule, keep the ScheduleDefinition (defaults to daily at 06:00). To upload only on demand, remove the schedule and trigger the job manually or from a Dagster sensor.

• If you split dbt models across multiple Dagster assets/jobs, run dbt docs generate and upload once per full build so Euno receives a complete, consistent set of artifacts.