Configure Matillion Project variables

If your Matillion project make use of Project Variables, consider configuring this entry in the Euno Matillion Integration.

By configuring project variables, Euno will substitue any reference to ${PROJECT_VARIABLE} with it's configured value.

For example, if you are using Project Variables to store the target schema for a Matillion pipeline, and you are making use of that Project Variable in your "Write Table" Matillion nodes, then Euno will only be able to observe the target table and it's relationship with the pipline if you configure the value of Project Variable in Euno.

This configure is done in the Matillion integration configuration screen, under "advanced".

The configuration expects a JSON object, where each entry is a project variable. For example, if your project variables are called foo_db and foo_schema and their values are bar_db and bar_schema respectively, enter:

{
    "foo_db": "bar_db",
    "foo_schema": "bar_schema"
}

Using a pipeline to extract all Project Variables

In case you have many Project Variables, it might not be feasible to manually configure each one. Instead, follow these steps to extract them all:

1. In Matillion Designer, create a new "Orchestration" pipeline

2. Add a single node of type "python script".

3. Under the "script", configure:

SYSTEM_VARIBLES={'__name__', '__doc__', '__package__', '__loader__', '__spec__', '__builtins__', '__file__', '__cached__', 'io', 'os', 'redirect_stdout', 'traceback', 'json', 'SimpleNamespace', '__loadObject', 'ScriptContext', 'processDPCUser', 'context', 'sysvar'}

local_variables = [x for x in globals().keys() if x not in SYSTEM_VARIBLES]
local_vars_with_value = {x: str(globals()[x]) for x in local_variables}
print(json.dumps(local_vars_with_value))

1. Run the pipeline

2. Open the run, and expand the message. The result is a properly parsed JSON with variable configuration, which you can then copy into the Euno configuration field.