A graph represents a SQL pipeline. Each node is a logical representation and can be materialized as a table or a view in the database. Currently, you can add the following out-of-the-box types of nodes: Source, Stage, Persistent Stage, Dimension, and Fact. Users can also create their own Node Types!
|Source||Represents source tables (raw data)|
|Stage||Typically used for performing transformations. The data in a Stage node is transient. The data is truncated on every run.|
|Persistent Stage||Similar to the Stage node, but the data is persistent and doesn't get truncated.|
|Dimension||Based on the dimensional modeling concepts|
|Fact||Based on the dimensional modeling concepts|
Add Source Nodes
Source Nodes are the source tables already in Snowflake.
- In the Add Sources to SQL Pipeline popup, storage locations that were previously configured populate on the left-hand-side. Each storage location is mapped to a snowflake database identified in parentheses. If source tables exist in the storage location, a downward facing arrow will appear next to the storage location. When clicked, the source tables become visible.
- Select nodes to be added to the graph individually by clicking on the checkbox next to the source table, or select all source tables in a storage location by clicking on the checkbox next to the storage location itself.
- Data can be sampled by clicking on the source table's name.
- Click Add sources (i.e. Add 8 sources)
Snowflake Source Tables
Because Coalesce is a transformation tool, be sure the data is correctly stored in Snowflake before doing this step.
Once sources are added, they will appear on the left-hand-side of the Build page under the Node dropdown as well as the graph and organized by Node Type. From the dropdown, sources can be clicked on and the columns and data types populate below. The same display of columns and data types will occur when the source is selected in the graph view.
The data can be viewed by selecting a source and then clicking on Fetch Data.
Modifying Source Nodes and Tables
Source nodes and source tables cannot be modified as this is the raw data.
Updated 30 days ago