Skip to main content

Using Subgraphs to Build Your Pipeline

Subgraphs help you focus on part of your Node graph without changing how Nodes are defined or how refreshes run. This page covers how to create and edit subgraphs, how they relate to Jobs and Git commits, when to add another subgraph, and naming patterns that keep large Projects manageable. When you are ready to deploy and refresh, follow Creating and Run Jobs from creating the Job through commit, deploy, and run. For a video on subgraphs with filters, extra screenshots, and the Main Graph overview, see Organizing Your DAG.

Subgraphs and Jobs​

Subgraphs and Jobs solve different problems. A Subgraph is a view in the Build Interface: you work on a subset of Nodes, and each placement is a reference to the same Node definition. A Job defines which Nodes to include when you deploy and refresh, using selector queries, dragged subgraphs, or both.

They differ in purpose:

  • Subgraphs - Organize the graph, reduce noise on screen, and make it easier to find related Nodes while you edit.
  • Jobs - Group Nodes for deployment and scheduled or on-demand refreshes. Jobs are stored in version control like the rest of your Project.

A practical pattern is to shape your graph with subgraphs first, then pull those subgraphs into Jobs when you are ready to run a consistent slice of the pipeline. The Job still refreshes the Nodes you selected, not a separate copy of the data logic.

Use Creating and Run Jobs for the end-to-end procedure: create the Job, add Nodes with selectors or subgraphs (see also Managing Jobs for drag-and-drop details), commit, configure the environment, deploy, and run the refresh. That guide is the place to start whenever you need to create and run Jobs.

How to Create a Subgraph​

You define subgraphs yourself. Creating one adds an empty canvas where you place references to Nodes that already exist on the main graph (or you add Nodes from the list, which still points at the same definitions).

  1. In the Build Interface, click the + plus sign to create a new Subgraph or right click on a Node.
  2. Give it a name.
  3. Then you can right click to add other Nodes or click and drag Nodes into the subgraph.

Adding a Node to a subgraph does not copy it. The same Node can appear in multiple subgraphs, dependencies stay consistent across views, and edits to that Node apply everywhere it appears.

For more on adding subgraphs, including a video, see Organizing Your DAG.

Large graphs

Organizing Your DAG shows how to combine filters in the Node list with drag-and-drop when the list is long. Filter syntax is in Selector Reference.

Naming Subgraphs​

Clear names make subgraphs easy to find in menus and when you add them to Jobs. Use this guidance:

  • Short, specific labels - Prefer names that state the scope, for example a domain, mart, or milestone, instead of generic titles.
  • Stable over time - Rename sparingly if teammates bookmark or reference subgraph names in runbooks.
  • One obvious name per purpose - Avoid near-duplicate names that differ only by punctuation or spelling so selectors and manual picks stay unambiguous.

How to Edit a Subgraph​

Editing a subgraph means changing which Nodes appear in that view, how you inspect them, or the Node definitions themselves. The underlying Nodes stay shared with the main graph and any other subgraphs that reference them.

  1. Open the subgraph from the Subgraph list when you want to change membership or layout on the canvas.
  2. Rename the subgraph if the scope of work changes. Apply the same patterns as in Naming Subgraphs.
  3. Add more Nodes with drag-and-drop from Nodes, or choose Add to Subgraph from a Node's context menu. If your UI shows Add to a Subgraph instead, choose that option. Drag-and-drop works in Graph, Node Grid, and Column Grid views.
  4. Switch View As inside the subgraph when you want a different layout for the same Nodes:
    • Graph - Visual DAG of Nodes and edges.
    • Node Grid - Table-oriented editing across Nodes.
    • Column Grid - Column-level view for dependency checks across Nodes.
  5. Edit a Node by double-clicking it on the canvas or opening it from the sidebar. The Node Editor opens the same way from the main graph or from within a subgraph.

For dependency lines that look wrong in the subgraph view, read When Visual Dependencies Look Wrong and Fix Un-Linked References in Sub-Graphs in Coalesce.

When to Add a Subgraph​

Use this section when you are deciding whether to create another subgraph.

You Want a Stable Bundle for Job Selection​

Use a subgraph as a reusable selection when you build or update Jobs.

You can drag subgraphs into the Job tab or express the same scope with selector queries. Either way, what runs in an environment depends on the committed Project, Job configuration, and deployment, not on which subgraph is open in the UI. See What Gets Committed for how subgraph metadata is stored in your repository.

After you pick subgraphs or selectors, open Creating and Run Jobs and work through each step so the Job is committed, deployed, and ready to refresh.

Your Main Graph Is Hard to Read or Navigate​

Use a named subgraph and filters when scrolling the full DAG slows you down.

Focus a subgraph on 1 subject area, layer, or release slice. Combine it with filters in the Node list and View As options so you can switch between graph, grid, and column views without losing the same Node set. Step-by-step UI guidance lives in Organizing Your DAG.

You Share a Project Across Teams or Parallel Work Streams​

Use Subgraphs named for ownership or feature areas when several people edit the same Workspace.

The same Node can appear in more than one subgraph. Edits always apply to the underlying Node, so everyone sees the same definition regardless of which subgraph they used to open it. Subgraphs don't create duplicate Nodes or isolated copies of logic.

How Edits, Commits, and Refreshes Interact​

A subgraph is a lens on the graph. Adding a Node to a subgraph adds a reference; it doesn't fork the Node. Changes you make to SQL, metadata, or settings apply everywhere that Node appears.

Deployments and refreshes follow the branch and commit you use for your Project, plus the Jobs and environments you configure. Organizing work in subgraphs doesn't bypass version control or change which Nodes belong to the pipeline. To run a refresh after deploy, use the flow in Creating and Run Jobs (scheduler, API, or CLI).

Filters, Views, and Workspace Tabs​

Selectors support a subgraph filter so you can target Nodes that belong to a named subgraph in queries and tooling that accept selector syntax. See Selector Reference for {subgraph:"..."} and related options.

Workspace tabs

When you use multiple tabs in the Build Interface, the workspace can retain scroll position, filters, and subgraph zoom as you move between tabs, so you spend less time resetting the canvas.

When Visual Dependencies Look Wrong​

Sometimes a subgraph doesn't draw edges between Nodes even though SQL references them. In those cases, use ref_link() and ref_no_link() or the commented-reference pattern described in Fix Un-Linked References in Sub-Graphs in Coalesce. Background on reference helpers is in Ref Functions.

What's Next?​