Skip to main content

Dynamic Tables

Overview

Dynamic tables simplify data engineering in Snowflake by providing a reliable, cost-effective, and automated way to transform data.

Installation

  1. Copy the Package ID: @coalesce/snowflake/dynamic-tables
  2. In Coalesce, open the Workspace where you wish to install the package.
  3. Go to the Build Setting of the Workspace, tab Packages, and click the Install button on the top right of the page.
  4. Paste the Package ID, and proceed with the installation process.

Description

Dynamic Tables Package

Package includes:


Dynamic Table Work

The Coalesce Dynamic Table Work UDN is a versatile node that allows you to develop and deploy a single Dynamic Table Work or a DAG of Dynamic Tables in Snowflake.

Dynamic tables are a new table type offered by Snowflake that allow data teams to use SQL statements to declaratively define the results of data pipelines. Dynamic tables simplify the process of creating and managing data pipelines by streamlining data transformations without having to manage Streams and Tasks.

Dynamic Table Work Node Configuration

The Dynamic Table Work has three configuration groups:

Dynamic Table Work Node Properties

PropertyDescription
Storage Location(Required) Storage Location where the Dynamic Table will be created
Node Type(Required) Name of template used to create node objects
DescriptionA description of the node's purpose
Deploy EnabledIf TRUE the node will be deployed/redeployed when changes are detected
If FALSE the node will not be deployed or will be dropped during redeployment

Dynamic Table Work Options

image
OptionDescription
Warehouse- (Required when Advance Warehouse is disabled) Name of warehouse used to refresh the Dynamic Table.
Advance Warehouse SelectionA toggle that enables size-based warehouse configuration for Dynamic Tables
- Refresh Warehouse: Selects the warehouse (by size) used for regular refresh operations.
- Initialization Warehouse: Selects the warehouse (by size) used during the initial creation or backfill of the Dynamic Table.
- When using Advanced Warehouse, the targetDynamicTableWarehouse parameter is not required.
Downstream(Required) True/False toggle:
- True: Refresh on demand when dependent tables need refresh
- False: Set Lag Specification for refresh schedule
Lag SpecificationOnly if Downstream is False. Review Snowflakes Dynamic Tables Refresh to understand how to specify the target lag. Set refresh schedule with:
- Time Value: Frequency of the refresh
- Time Period: Seconds/Minutes/Hours/Days
Refresh ModeSpecifies refresh type:
- BLANK(''): If the blank option is selected, the default behavior will trigger an INCREMENTAL refresh.- AUTO: Default incremental refresh. If the CREATE DYNAMIC TABLE statement does not support the incremental refresh mode, the dynamic table is automatically created with the full refresh mode.
- INCREMENTAL: Force incremental refresh
- FULL: Force full refresh
InitializeInitial refresh behavior:
- BLANK(''): If the blank option is selected, the default behavior will be ON_CREATE.
- ON_CREATE: Refresh synchronously at creation
- ON_SCHEDULE: Refresh at next scheduled time

Dynamic Table Work Advanced Options

OptionDescription
Copy grantsSpecifies to retain the access privileges from the original table when a new dynamic table is created.Useful during replication.More info on replication here
Immutability ConstraintTrue/False toggle:
- True: Applies an IMMUTABLE condition to the Dynamic Table, preventing changes to data that matches the defined rule
- False: No immutability is enforced; data can be updated normally
Immutable Where ExpressionVisible when Immutability Constraint is enabled.
- SQL condition used to identify rows that are considered immutable (no longer change).
- This expression must reference valid dynamic table columns and should be deterministic.
Enable Backfill- Visible only when Immutability Constraint is enabled.
- True/False toggle:
- True: Displays Backfill Options group.
Refer Snowflake Dynamic Tables to understand how Immutability and Backfill features work.

Dynamic Table Work Backfill Options

image
OptionDescription
Backfill Source Schema(Optional) Schema name of the backfill table.
- If provided, the backfill table is read from this schema.
- If left blank, the current node’s schema is used by default.
Backfill Source TableSpecifies a source table used to load historical data into the Dynamic Table.
Time TravelVisible when Backfill option in enabled.
True/False toggle to create the table with Time Travel options.
Time Travel TypeIf Time Travel parameters AT/BEFORE are specified, data from the backfill table is copied at the specified time.
Time Travel ReferenceDropdown to specify how Snowflake should time travel the backfill source data:
- OFFSET: Uses a relative time offset from the current time.
- STATEMENT: Uses a specific Snowflake query ID to time travel the data to the moment that query was executed.
Time Travel ValueValue depends on the selected Time Travel Reference:
- OFFSET: Provide a negative integer value (for example: -60, -120, -1440) representing time in minutes before the current time.
- STATEMENT: Provide a valid query ID of a completed DML from the query history that falls within the table’s time travel retention period.

Dynamic Table Work General Options

dynamic table1 1_general

OptionDescription
DistinctTrue/False toggle to return DISTINCT rows
Group By AllTrue/False toggle to add non-aggregated columns to GROUP BY
Multi SourceTrue/False toggle for combining multiple sources via UNION or UNION ALL
Create AsChoose 'dynamic table' or 'transient dynamic table'
Cluster keyTrue/False toggle for clustering:
- True: Specify clustering column and optional expressions
- False: No clustering
Allow Expressions Cluster KeyWhen cluster key is set to true. Allows to add an expression to the specified cluster key

Dynamic Table Work Deployment

Dynamic Table Work Initial Deployment Parameters

The Dynamic Table Work includes an environment parameter that allows you to specify a different warehouse to refresh a Dynamic Table in different environments.

The parameter name is targetDynamicTableWarehouse and the default value is DEV ENVIRONMENT.

When set to DEV ENVIRONMENT, the value entered in the Dynamic Table Options config "Warehouse on which to execute Dynamic Table" will be used when creating the Dynamic Table.

{
"targetDynamicTableWarehouse": "DEV ENVIRONMENT"
}

When set to any value other than DEV ENVIRONMENT the node will attempt to create the Dynamic Table using a Snowflake warehouse with the specified value.

For example, the Dynamic Table will refresh using a warehouse named compute_wh.

{
"targetDynamicTableWarehouse": "compute_wh"
}

Advance Warehouse selection(Optional)

  • Advanced Warehouse Selection allows using separate warehouses for initialization and refresh based on workload size.
  • Users can select different sizes for refresh vs initialization to optimize cost/performance

Prerequisite

  • Advanced Warehouse must be enabled.
  • The required warehouses must already exist in Snowflake.
  • Corresponding warehouse parameters must be defined in the deployment environment.

Example of parameter initialization

{
"warehouseSizesDict": {
"xsDynamicTableWarehouse": "dev_wh_xs",
"sDynamicTableWarehouse": "dev_wh_s",
"lDynamicTableWarehouse": "dev_wh_l"
},
"targetDynamicTableWarehouse": "DEV ENVIRONMENT",
}

Note: dev_wh_xs, dev_wh_s, and dev_wh_l are example warehouse names. Users must replace these values with the names of the Snowflake warehouses they have already created and want to use in their environment.

Warehouse sizes supported in node config UI and corresponding parameter details

SizeEnvironment Parameter
X-SmallxsDynamicTableWarehouse
SmallsDynamicTableWarehouse
MediummDynamicTableWarehouse
LargelDynamicTableWarehouse
X-LargexlDynamicTableWarehouse
2X-Large2xlDynamicTableWarehouse
3X-Large3xlDynamicTableWarehouse
4X-Large4xlDynamicTableWarehouse
5X-Large5xlDynamicTableWarehouse
6X Large6xlDynamicTableWarehouse

Notes

  • These parameters are required only when Advanced Warehouse is enabled.
  • When using Advanced Warehouse, the targetDynamicTableWarehouse parameter is not required.

📘 Deployment of nodes without adding parameters

This results in a WARNING stage getting executed insisting to execute the node after adding parameters

Dynamic Table Work Initial Deployment

When deployed for the first time into an environment the Dynamic Table Work node will execute the following stage:

StageDescription
Create Work Dynamic Table/Dynamic Transient TableThis stage will execute a CREATE OR REPLACE statement and create a Dynamic Table in the target environment.

Deploying a DAG of Dynamic Tables

When a DAG of related Dynamic Tables are deployed together Coalesce will deploy the Dynamic Tables in the order that the Dynamic Tables are ordered.

Dynamic Table Work Redeployment

After initial deployment, subsequent deployments may alter or recreate the Dynamic Table.

Altering the Dynamic Table Work

The following config changes trigger ALTER statements:

  1. Warehouse name
  2. Downstream setting
  3. Lag specification
  4. Immutability Constraint
  5. Advance Warehouse

These execute the two stages:

StageDescription
Alter Dynamic TableExecutes ALTER to modify parameters
Refresh Dynamic TableRefreshes table to make data available

Also if the location of the node, node name, column level description, and table level description results in an ALTER statement, whereas other column or table level changes like data type change, column name change, column addition/deletion result in a CREATE statement.

Changing Materialization Type

If the materialization type changes in dynamic table config options, the following steps gets executed:

  1. Drop table
  2. Create Work dynamic table
  3. Apply Table Clustering(if cluster key option is provided)
  4. Resume Recluster Table(if cluster key option is provided)

Recreating the Dynamic Table

If anything changes other than the configuration options specified in Altering the Dynamic Table then the Dynamic Table will be recreated by running a CREATE OR REPLACE statement.

If the changes in node results in recreating the Dynamic table,then following stages are executed:

StageDescription
Drop table/transient tableTable is dropped before recreating in case the node name or location is changed
Create Work Dynamic table/Dynamic transient tableDynamic table is created

Redeploying a DAG of Dynamic Tables Work

If an entire DAG of Dynamic Tables has been deployed and changes are made to a deployed Dynamic Table Coalesce will only redeploy Dynamic Tables that have changed metadata.

Redeployment with no changes

If the nodes are redeployed with no changes compared to previous deployment,then no stages are executed

Dynamic Tables Work Undeployment

A table will be dropped if all of these are true:

  • The Dynamic Work Node is deleted from a Workspace.
  • The Workspace is committed to Git.
  • The Workspace committed to Git is deployed to a higher level environment.
StageDescription
Drop Dynamic TableRemoves table from target environment

Dynamic Table Dimension

The Coalesce Dynamic Table Dimension UDN is a versatile node that allows you to develop and deploy a single Dynamic Table Dimension or a DAG of Dynamic Tables in Snowflake.

Dynamic tables are a new table type offered by Snowflake that allow data teams to use SQL statements to declaratively define the results of data pipelines. Dynamic tables simplify the process of creating and managing data pipelines by streamlining data transformations without having to manage Streams and Tasks.

Dimension Node Configuration

The Dynamic Table Dimension has four configuration groups:

Dimension Node Properties

PropertyDescription
Storage LocationStorage Location where table will be created
Node TypeName of template used to create node objects
DescriptionA description of the node's purpose
Deploy EnabledIf TRUE the node will be deployed/redeployed when changes are detected
If FALSE the node will not be deployed or will be dropped during redeployment

Dimension Table Options

image
OptionDescription
Warehouse(Required when Advance Warehouse is disabled) Name of warehouse used to refresh the Dynamic Table
Advance Warehouse SelectionA toggle that enables size-based warehouse configuration for Dynamic Tables
- Refresh Warehouse: Selects the warehouse (by size) used for regular refresh operations.
- Initialization Warehouse: Selects the warehouse (by size) used during the initial creation or backfill of the Dynamic Table.
- When using Advanced Warehouse, the targetDynamicTableWarehouse parameter is not required.
Downstream(Required) True/False toggle:
- True: Refresh on demand when dependent tables need refresh
- False: Set Lag Specification for refresh schedule
Lag SpecificationOnly if Downstream is False. Review Snowflakes Dynamic Tables Refresh to understand how to specify the target lag. Set refresh schedule with:
- Time Value: Frequency of refresh for a given Time Period.
- Time Period: Seconds/Minutes/Hours/Days
Refresh ModeSpecifies refresh type:
- BLANK(''): If the blank option is selected, the default behavior will trigger an INCREMENTAL refresh.
- AUTO: Default incremental refresh. If the CREATE DYNAMIC TABLE statement does not support the incremental refresh mode, the dynamic table is automatically created with the full refresh mode.
- INCREMENTAL: Force incremental refresh
- FULL: Force full refresh
InitializeInitial refresh behavior:
- BLANK(''): If the blank option is selected, the default behavior will be ON_CREATE.
- ON_CREATE: Refresh synchronously at creation
- ON_SCHEDULE: Refresh at next scheduled time

Dimension Options

Dimension options

OptionDescription
Table keys(Required) Business key columns for Dimension key formation
Record versioning(Required) Type of column for history maintenance:
- Datetime column
- Date and Time column
- Integer column
TimestampRequired if Datetime column chosen for Record versioning.Note:If multiple columns are chosen.The first timestamp column chosen is considered for versioning order
SequenceRequired if Integer column chosen for Record versioning
Timetamp-track data loadRequired if Integer column chosen for Record versioningNote:If multiple columns are chosen.The first timestamp column chosen is considered for versioning order
Date/Timestamp ColumnsRequired if Date and Time columns chosen for Record versioningNote:If multiple columns are chosen.The first timestamp column chosen is considered for versioning order

Dimension General Options

dynamictable1 1-dimgeneral

OptionDescription
Create AsChoose 'dynamic table' or 'transient dynamic table'
Cluster keyTrue/False toggle for clustering:
- True: Specify clustering column and optional expressions
- False: No clustering
Allow Expressions Cluster KeyWhen cluster key is set to true. Allows to add an expression to the specified cluster key

Dimension Advanced Options

OptionDescription
Copy grantsSpecifies to retain the access privileges from the original table when a new dynamic table is created.Useful during replication.More info on replication here
Immutability ConstraintTrue/False toggle:
- True: Applies an IMMUTABLE condition to the Dynamic Table, preventing changes to data that matches the defined rule
- False: No immutability is enforced; data can be updated normally
Immutable Where ExpressionVisible when Immutability Constraint is enabled.
- SQL condition used to identify rows that are considered immutable (no longer change).
- This expression must reference valid dynamic table columns and should be deterministic.
Enable Backfill- Visible only when Immutability Constraint is enabled.
- True/False toggle:
- True: Displays Backfill Options group.
Review Snowflake Dynamic Tables to understand how Immutability and Backfill features work.

Dynamic Table Dimension Backfill Options

image
OptionDescription
Backfill Source Schema(Optional) Schema name of the backfill table.
- If provided, the backfill table is read from this schema.
- If left blank, the current node’s schema is used by default.
Backfill Source TableSpecifies a source table used to load historical data into the Dynamic Table.
Time TravelVisible when Backfill option in enabled.
True/False toggle to create the table with Time Travel options.
Time Travel TypeIf Time Travel parameters AT/BEFORE are specified, data from the backfill table is copied at the specified time.
Time Travel ReferenceDropdown to specify how Snowflake should time travel the backfill source data:
- OFFSET: Uses a relative time offset from the current time.
- STATEMENT: Uses a Snowflake query ID to time travel data, but is not supported for Dynamic Table Dimension backfill.
Time Travel ValueValue depends on the selected Time Travel Reference:
- OFFSET: Provide a negative integer value (for example: -60, -120, -1440) representing time in minutes before the current time.
- STATEMENT: Provide a valid query ID of a completed DML from the query history that falls within the table’s time travel retention period.

Dimension Deployment

Dimension Initial Deployment Parameters

The Dynamic Table Work includes an environment parameter that allows you to specify a different warehouse to refresh a Dynamic Table in different environments.

The parameter name is targetDynamicTableWarehouse and the default value is DEV ENVIRONMENT.

When set to DEV ENVIRONMENT, the value entered in the Dynamic Table Options config "Warehouse on which to execute Dynamic Table" will be used when creating the Dynamic Table.

{
"targetDynamicTableWarehouse": "DEV ENVIRONMENT"
}

When set to any value other than DEV ENVIRONMENT the node will attempt to create the Dynamic Table using a Snowflake warehouse with the specified value.

For example, the Dynamic Table will refresh using a warehouse named compute_wh.

{
"targetDynamicTableWarehouse": "compute_wh"
}

Advance Warehouse selection(Optional)

  • Advanced Warehouse Selection allows using separate warehouses for initialization and refresh based on workload size.
  • Corresponding warehouse parameters must be defined in the deployment environment.

Prerequisite

  • Advanced Warehouse must be enabled.
  • The required warehouses must already exist in Snowflake.
  • Corresponding warehouse parameters must be defined in the deployment environment.

Example of parameter initialization

{
"warehouseSizesDict": {
"xsDynamicTableWarehouse": "dev_wh_xs",
"sDynamicTableWarehouse": "dev_wh_s",
"lDynamicTableWarehouse": "dev_wh_l"
},
"targetDynamicTableWarehouse": "DEV ENVIRONMENT",
}

Note: dev_wh_xs, dev_wh_s, and dev_wh_l are example warehouse names. Users must replace these values with the names of the Snowflake warehouses they have already created and want to use in their environment.

Warehouse sizes supported in node config UI and corresponding parameter details

SizeEnvironment Parameter
X-SmallxsDynamicTableWarehouse
SmallsDynamicTableWarehouse
MediummDynamicTableWarehouse
LargelDynamicTableWarehouse
X-LargexlDynamicTableWarehouse
2X-Large2xlDynamicTableWarehouse
3X-Large3xlDynamicTableWarehouse
4X-Large4xlDynamicTableWarehouse
5X-Large5xlDynamicTableWarehouse
6X Large6xlDynamicTableWarehouse

Notes

  • These parameters are required only when Advanced Warehouse is enabled.
  • When using Advanced Warehouse, the targetDynamicTableWarehouse parameter is not required.

📘 Deployment of nodes without adding parameters

This results in a WARNING stage getting executed insisting to execute the node after adding parameters

Dimension Initial Deployment

When deployed for the first time into an environment the Dynamic Table Work node will execute the following stage:

StageDescription
Create Dimension Dynamic Table/Dynamic Transient TableThis stage will execute a CREATE OR REPLACE statement and create a Dynamic Table in the target environment.

Deploying a DAG of Dimension Tables

When a DAG of related Dynamic Tables are deployed together Coalesce will deploy the Dynamic Tables in the order that the Dynamic Tables are ordered.

Dimension Redeployment

After initial deployment, subsequent deployments may alter or recreate the Dynamic Table.

Altering the Dimension Table

The following config changes trigger ALTER statements:

  1. Warehouse name
  2. Downstream setting
  3. Lag specification
  4. Immutability Constraint
  5. Advance Warehouse

These execute the two stages:

StageDescription
Alter Dynamic TableExecutes ALTER to modify parameters
Refresh Dynamic TableRefreshes table to make data available

Also if the location of the node, node name, column level description, and table level description results in an ALTER statement, whereas other column or table level changes like data type change, column name change, column addition/deletion result in a CREATE statement.

Changing Materialization Type

If the materialization type changes in dynamic table config options, the following steps gets executed:

  1. Drop table
  2. Create Work dimension table
  3. Apply Table Clustering(if cluster key option is provided)
  4. Resume Recluster Table(if cluster key option is provided)

Recreating the Dimension Table

If anything changes other than the configuration options specified in Altering the Dimension Table then the Dynamic Table will be recreated by running a CREATE OR REPLACEstatement.

If the changes in node results in recreating the Dynamic table, then following stages are executed:

StageDescription
Drop table/transient tableTable is dropped before recreating in case the node name or location is changed
Create Work Dynamic table/Dynamic transient tableDynamic table is created

Redeploying a DAG of Dimension

If an entire DAG of Dynamic Tables has been deployed and changes are made to a deployed Dynamic Table Coalesce will only redeploy Dynamic Tables that have changed metadata.

Redeployment with no changes

If the nodes are redeployed with no changes compared to previous deployment,then no stages are executed

Dynamic Table Dimension Undeployment

A table will be dropped if all of these are true:

  • The Dynamic Dimension Node is deleted from a Workspace.
  • The Workspace is committed to Git.
  • The Workspace committed to Git is deployed to a higher level environment.
StageDescription
Drop Dynamic TableRemoves table from target environment

Dynamic Table Latest Record Version

The Coalesce Dynamic Table Latest Record Version UDN is a versatile node that allows you to develop and deploy a single Dynamic Table Work or a DAG of Dynamic Tables with only the latest version of rows in Snowflake.

Dynamic tables are a new table type offered by Snowflake that allow data teams to use SQL statements to declaratively define the results of data pipelines. Dynamic tables simplify the process of creating and managing data pipelines by streamlining data transformations without having to manage Streams and Tasks.

Latest Record Version Node Configuration

The Dynamic Table Dimension has four configuration groups:

Latest Record Version Node Properties

PropertyDescription
Storage LocationStorage Location where table will be created
Node TypeName of template used to create node objects
DescriptionA description of the node's purpose
Deploy EnabledIf TRUE the node will be deployed/redeployed when changes are detected
If FALSE the node will not be deployed or will be dropped during redeployment

Latest Record Verion Table Options

image
OptionDescription
Warehouse(Required when Advance Warehouse is disabled) Name of warehouse used to refresh the Dynamic Table
Advance Warehouse SelectionA toggle that enables size-based warehouse configuration for Dynamic Tables
- Refresh Warehouse: Selects the warehouse (by size) used for regular refresh operations.
- Initialization Warehouse: Selects the warehouse (by size) used during the initial creation or backfill of the Dynamic Table.
- When using Advanced Warehouse, the targetDynamicTableWarehouse parameter is not required.
Downstream(Required) True/False toggle:
- True: Refresh on demand when dependent tables need refresh
- False: Set Lag Specification for refresh schedule
Lag SpecificationOnly if Downstream is False. Review Snowflakes Dynamic Tables Refresh to understand how to specify the target lag. Set refresh schedule with:
- Time Value: Frequency of refresh for a given Time Period.
- Time Period: Seconds/Minutes/Hours/Days
Refresh ModeSpecifies refresh type:
- BLANK(''): If the blank option is selected, the default behavior will trigger an INCREMENTAL refresh.
- AUTO: Default incremental refresh. If the CREATE DYNAMIC TABLE statement does not support the incremental refresh mode, the dynamic table is automatically created with the full refresh mode.
- INCREMENTAL: Force incremental refresh
- FULL: Force full refresh
InitializeInitial refresh behavior:
- BLANK(''): If the blank option is selected, the default behavior will be ON_CREATE.
- ON_CREATE: Refresh synchronously at creation
- ON_SCHEDULE: Refresh at next scheduled time

Latest Record Version Options

OptionDescription
Table keys(Required) Business key columns for Dimension key formation
Record versioning(Required) Type of column for history maintenance:
- Datetime column
- Date and Time column
- Integer column
TimestampRequired if Datetime column chosen for Record versioning.Note:If multiple columns are chosen.The first timestamp column chosen is considered for versioning order
SequenceRequired if Integer column chosen for Record versioning
Timetamp-track data loadRequired if Integer column chosen for Record versioningNote:If multiple columns are chosen.The first timestamp column chosen is considered for versioning order
Date/Timestamp ColumnsRequired if Date and Time columns chosen for Record versioningNote:If multiple columns are chosen.The first timestamp column chosen is considered for versioning order

Latest Record Version General Options

OptionDescription
Create AsChoose 'dynamic table' or 'transient dynamic table'
Cluster keyTrue/False toggle for clustering:
- True: Specify clustering column and optional expressions
- False: No clustering
Allow Expressions Cluster KeyWhen cluster key is set to true. Allows to add an expression to the specified cluster key

Latest Record Version Advanced Options

OptionDescription
Copy grantsSpecifies to retain the access privileges from the original table when a new dynamic table is created.Useful during replication.More info on replication here
Immutability ConstraintTrue/False toggle:
- True: Applies an IMMUTABLE condition to the Dynamic Table, preventing changes to data that matches the defined rule
- False: No immutability is enforced; data can be updated normally
Immutable Where ExpressionVisible when Immutability Constraint is enabled.
- SQL condition used to identify rows that are considered immutable (no longer change).
- This expression must reference valid dynamic table columns and should be deterministic.
Enable Backfill- Visible only when Immutability Constraint is enabled.
- True/False toggle:
- True: Displays Backfill Options group.
Review Snowflake Dynamic Tables to understand how Immutability and Backfill features work.

Latest Record Version Backfill Options

image
OptionDescription
Backfill Source Schema(Optional) Schema name of the backfill table.
- If provided, the backfill table is read from this schema.
- If left blank, the current node’s schema is used by default.
Backfill Source TableSpecifies a source table used to load historical data into the Dynamic Table.
Time TravelVisible when Backfill option in enabled.
True/False toggle to create the table with Time Travel options.
Time Travel TypeIf Time Travel parameters AT/BEFORE are specified, data from the backfill table is copied at the specified time.
Time Travel ReferenceDropdown to specify how Snowflake should time travel the backfill source data:
- OFFSET: Uses a relative time offset from the current time.
- STATEMENT: Uses a Snowflake query ID to time travel data, but is not supported for Dynamic Table Dimension backfill.
Time Travel ValueValue depends on the selected Time Travel Reference:
- OFFSET: Provide a negative integer value (for example: -60, -120, -1440) representing time in minutes before the current time.
- STATEMENT: Provide a valid query ID of a completed DML from the query history that falls within the table’s time travel retention period.

DAG of Dynamic Table Latest Record Version

When designing DAG of Dynamic tables, you should specify the target lag. Review Understanding dynamic table refresh - Snowflake

Latest Record Version Deployment

Latest Record Version Initial Deployment Parameters

The Dynamic Table Work includes an environment parameter that allows you to specify a different warehouse to refresh a Dynamic Table in different environments.

The parameter name is targetDynamicTableWarehouse and the default value is DEV ENVIRONMENT.

When set to DEV ENVIRONMENT, the value entered in the Dynamic Table Options config "Warehouse on which to execute Dynamic Table" will be used when creating the Dynamic Table.

{
"targetDynamicTableWarehouse": "DEV ENVIRONMENT"
}

When set to any value other than DEV ENVIRONMENT the node will attempt to create the Dynamic Table using a Snowflake warehouse with the specified value.

For example, the Dynamic Table will refresh using a warehouse named compute_wh.

{
"targetDynamicTableWarehouse": "compute_wh"
}

Advance Warehouse selection(Optional)

  • Advanced Warehouse Selection allows using separate warehouses for initialization and refresh based on workload size.
  • Corresponding warehouse parameters must be defined in the deployment environment.

Prerequisite

  • Advanced Warehouse must be enabled.
  • The required warehouses must already exist in Snowflake.
  • Corresponding warehouse parameters must be defined in the deployment environment.

Example of parameter initialization

{
"warehouseSizesDict": {
"xsDynamicTableWarehouse": "dev_wh_xs",
"sDynamicTableWarehouse": "dev_wh_s",
"lDynamicTableWarehouse": "dev_wh_l"
},
"targetDynamicTableWarehouse": "DEV ENVIRONMENT",
}

Note: dev_wh_xs, dev_wh_s, and dev_wh_l are example warehouse names. Users must replace these values with the names of the Snowflake warehouses they have already created and want to use in their environment.

Warehouse sizes supported in node config UI and corresponding parameter details

SizeEnvironment Parameter
X-SmallxsDynamicTableWarehouse
SmallsDynamicTableWarehouse
MediummDynamicTableWarehouse
LargelDynamicTableWarehouse
X-LargexlDynamicTableWarehouse
2X-Large2xlDynamicTableWarehouse
3X-Large3xlDynamicTableWarehouse
4X-Large4xlDynamicTableWarehouse
5X-Large5xlDynamicTableWarehouse
6X Large6xlDynamicTableWarehouse

Notes

  • These parameters are required only when Advanced Warehouse is enabled.
  • When using Advanced Warehouse, the targetDynamicTableWarehouse parameter is not required.

📘 Deployment of nodes without adding parameters

This results in a WARNING stage getting executed insisting to execute the node after adding parameters

Latest Record Version Initial Deployment

When deployed for the first time into an environment the Dynamic Table Work node will execute the following stage:

StageDescription
Create Dimension Dynamic Table/Dynamic Transient TableThis stage will execute a CREATE OR REPLACE statement and create a Dynamic Table in the target environment.

Deploying a DAG of Latest Record Version Tables

When a DAG of related Dynamic Tables are deployed together Coalesce will deploy the Dynamic Tables in the order that the Dynamic Tables are ordered.

Latest Record Version Redeployment

After initial deployment, subsequent deployments may alter or recreate the Dynamic Table.

Altering the Latest Record Version Table

The following config changes trigger ALTER statements:

  1. Warehouse name
  2. Downstream setting
  3. Lag specification
  4. Immutability Constraint
  5. Advance Warehouse

These execute the two stages:

StageDescription
Alter Dynamic TableExecutes ALTER to modify parameters
Refresh Dynamic TableRefreshes table to make data available

Also if the location of the node, node name, column level description, and table level description results in an ALTER statement, whereas other column or table level changes like data type change, column name change, column addition/deletion result in a CREATE statement.

Changing Materialization Type

If the materialization type changes in dynamic table config options, the following steps gets executed:

  1. Drop transient dimension table
  2. Create Work dimension table
  3. Apply Table Clustering(if cluster key option is provided)
  4. Resume Recluster Table(if cluster key option is provided)

Recreating the Latest Record Version Table

If anything changes other than the configuration options specified in Altering the Latest Record Version Table, then the Dynamic Table will be recreated by running a CREATE OR REPLACEstatement.

If the changes in node results in recreating the Dynamic table, then following stages are executed:

StageDescription
Drop table/transient tableTable is dropped before recreating in case the node name or location is changed
Create Work Dynamic table/Dynamic transient tableDynamic table is created

Redeploying a DAG of Latest Record Version

If an entire DAG of Dynamic Tables has been deployed and changes are made to a deployed Dynamic Table Coalesce will only redeploy Dynamic Tables that have changed metadata.

Redeployment with no changes

If the nodes are redeployed with no changes compared to previous deployment,then no stages are executed

Latest Record Version Undeployment

A table will be dropped if all of these are true:

  • The Dynamic Latest Record Version Node is deleted from a Workspace.
  • The Workspace is committed to Git.
  • The Workspace committed to Git is deployed to a higher level environment.
StageDescription
Drop Dynamic TableRemoves table from target environment

Code

Dynamic Table Work Code

Dynamic Table Dimension Code

Dynamic Table Latest Record Version Code

Macros

Versions

Available versions of the package.

Version #Release DateNotes
2.1.6January 05, 2026Added Advance Warehouse, Immutability Constraint and Backfill Features
2.1.5December 26, 2025Fix for Missing Handler for Incoming Node Type Transitions (View/Table → Dynamic Table)
2.1.4November 07, 2025Mocro update and blank option added to Refresh Mode and Initialize
2.1.3October 06, 2025Dynamic Iceberg Table moved to Iceberg Tables Package
2.1.2September 26, 2025Integer versioning column support, transform support for tstamp columns,multiple tstamp notsupported
2.1.0August 13, 2025Advanced option Copy grants added to all dynamic table node types
1.1.11June 06, 2025Deployment fix-single source to multi-source
1.1.10April 22, 2025Adding data type definition to dynamic table node types.
1.1.9March 27, 2025Fix done for dynamic dimension to use hash key as table key
1.1.8March 07, 2025Force Users to Enter Parameters When Required
1.1.7February 24, 2025Redeployment fixes for join clause