Node_Columns

node object

The Node source of the column. It only shows direct lineage.

idstring

An autogenerated Node ID.

namestring

The Node name.

nodeTypestring

location object

Where the Node is located.

namestring

The name of the Node Storage Location

Example: WORK

descriptionstring

The Node description.

materializationType

The materialization of the Node. Table or View.

Possible values: [table, view]

isMultisourceboolean

If the Node is Multi Source

override object

If the materializationType is set to view, users can override Create SQL allows a user to override the DDL of a view, enabling greater flexibility in view creation. Override Create SQL Toggle.

create object

enabledboolean

If the user has toggled override to true or false.

scriptstring

The custom script written in the Create SQL box.

tests object[]

Any Node level tests.

Array [

namestring

Name of the test.

Example: Test

descriptionstring

Description of the test.

continueOnFailurestring

If the pipeline should keep running, even if the Node test fails.

runOrder

If the test should run before or after the DML operation.

Possible values: [Before, After]

templateStringstring

Any custom SQL that was added here.

Example: SELECT * FROM COALESCE

]

columns object[]

The columns in the source node that it maps to.

Array [

idstring

An auto-generated Coalesce ID.

Example: 4c1e4719-b72b-41fe-82f6-7b6311475fdd

namestring

The name of the column.

Example: L_PARTKEY

dataTypestring

This is the data type for the column and by default maps to the source of the Node.

Example: NUMBER(38,0)

descriptionstring

If there is a comment or description or it can be added here.

Example: Column description

nullableboolean

If a column can have null values

defaultValuestring

Default value of a column.

Example: Coalesce

tests object[]

Column level tests for null and unique values. Each test is in it's own object. If there are no tests, shows as an empty array. tests: []

Array [

name

Name of the test. Indicates if the test is Unique or "Null"

Possible values: [Unique, "Null"]

Default value: Null

Example: "Null"

descriptionstring

Description of the test.

• Null testing - checks for any null values
• Unique value check if value is unique

Example: checks for any null values

continueOnFailureboolean

Specifies if to continue running the pipeline after the related test fails.

runOrderstring

Column level tests run After.

Default value: After

Example: After

templateStringstring

The SQL template for the test that will be executed. These are examples.

Unique

        templateString: |2-

                      SELECT * FROM {{ ref(node.location.name, node.name)}} 
                      WHERE "L_ORDERKEY" IS NULL

Null

        templateString: |2-

                      SELECT "L_ORDERKEY", COUNT(*)
                      FROM {{ ref(node.location.name, node.name)}} 
                      GROUP BY "L_ORDERKEY"
                      HAVING COUNT(*) > 1

Example: \n SELECT * FROM {{ ref(node.location.name, node.name)}} \n WHERE \"L_ORDERKEY\" IS NULL

]

]

Node_Columns

{
  "node": {
    "id": "string",
    "name": "string",
    "nodeType": "string",
    "location": {
      "name": "WORK"
    },
    "description": "string",
    "isMultisource": true,
    "override": {
      "create": {
        "enabled": true,
        "script": "string"
      }
    },
    "tests": [
      {
        "name": "Test",
        "description": "string",
        "continueOnFailure": "string",
        "templateString": "SELECT * FROM COALESCE"
      }
    ]
  },
  "columns": [
    {
      "id": "4c1e4719-b72b-41fe-82f6-7b6311475fdd",
      "name": "L_PARTKEY",
      "dataType": "NUMBER(38,0)",
      "description": "Column description",
      "nullable": true,
      "defaultValue": "Coalesce",
      "tests": [
        {
          "name": "\"Null\"",
          "description": "checks for any null values",
          "continueOnFailure": true,
          "runOrder": "After",
          "templateString": "\n            SELECT * FROM {{ ref(node.location.name, node.name)}} \n            WHERE \"L_ORDERKEY\" IS NULL"
        }
      ]
    }
  ]
}