Skip to main content

Organizing Your DAG

A well-organized DAG is essential for building maintainable and scalable data pipelines in Coalesce. This guide will help you effectively organize your DAG using graphs, subgraphs, and filters to improve visibility and manageability.

Main Graph View​

The main graph shows your entire DAG with all Nodes and connections.

Subgraphs​

Subgraphs allow you to view specific parts of your Node graph in isolation, breaking down your pipeline into smaller segments.

Subgraph in the Graph view

To create a subgraph:

  1. In the Build Interface, click on Subgraph, then click the plus sign (+) to add a new subgraph.

    The image shows Coalesce's data platform displaying a Snowflake schema with a SUPPLIER table connected to four related tables (STG_SUPPLIER, PSTG_SUPPLIER, DIM_SUPPLIER, FCT_SUPPLIER). The bottom panel lists the SUPPLIER table's columns and data types, including keys, name, address, and other fields.

  2. Click on the new subgraph which will take you to an empty DAG.

  3. Update the subgraph name to something descriptive.

  4. You can add Nodes by click on Nodes and then dragging them into the subgraph, or right-click a Node and select Add to a Subgraph.

When you add a Node to a subgraph, you're creating a reference to the original Node. This means:

  • A Node can exist in multiple subgraphs.
  • Changes to a Node in one subgraph are reflected across all instances.
  • Dependencies between Nodes are maintained across subgraphs.

Adding Nodes to the Subgraph​

Nodes can be added to existing subgraphs in two different ways:

  • Using Add to Subgraph from the context menu.
  • Dragging and dropping nodes into an existing subgraph. This is supported in Graph, Node Grid, and Column Grid views.
Creating a subgraph using a filter and drag and drop Nodes into the subgraph
Using Filters

Use filters in the Node view to limit the number of Nodes and make it faster to select them for the subgraph.

Change the View​

You can also change the view in subgraphs by selecting an option from View As.

  • Graph: Shows the visual representation of Nodes and connections.
  • Node Grid: Displays nodes in a table format for bulk editing.
  • Column Grid: Shows all columns across nodes for inspecting dependencies.