Skip to main content

Documentation index: llms.txt. This page is also available as markdown: append .md to this URL or send Accept: text/markdown.

What is Transform?

Coalesce Transform lets you build, automate, and govern data transformations with agents, in the web app, or directly through code. Every pipeline compiles to SQL that runs in your data platform, with column-level lineage, reusable patterns, and validation on every change.

Your Pipeline Is Files​

A Transform pipeline is a set of files in a Git repository, called a Workspace.

FileWhat it is
nodes/TARGET-STG_ORDERS.sqlA SQL Node: a SELECT statement plus annotations such as @materializationType("table"). Most new Nodes are SQL Nodes.
nodes/TARGET-DIM_CUSTOMER.ymlA YAML Node: the same idea, stored as the column mapping the web app's mapping grid edits.
nodeTypes/ and packages/Node Types define how a Node becomes DDL and DML: table or view, merge or insert, which tests run. You install them from the Marketplace or write your own.
data.yml, locations.ymlThe Workspace's platform and its Storage Locations, the logical names such as SRC and TARGET that map to real databases and schemas.

Because the pipeline is files, it works with the tools you already use: branches, pull requests, code review, and your coding agent.

How You Build​

You can run this loop three ways. They edit the same files.

  • With your agent and Coalesce Desktop. Your agent writes Nodes and runs them with the coa CLI. Coalesce Desktop shows each change in the graph as it lands, so you review the work visually instead of reading diffs. See Build Your First Pipeline With Your Agent.
  • In the web app. Build in the browser with the visual graph and the mapping grid, with Coalesce Copilot to help. See Quick Start Guides.
  • By hand, locally. Write SQL Nodes in your editor and run them with coa. See the coa reference.

What Transform Adds to Every Pipeline​

  • Column-level lineage. Built from your ref() calls and column references as you work, not reconstructed afterwards.
  • Standards as Node Types. A Dimension Node Type decides how every dimension merges and tracks history. Change the type once and every Node of that type follows.
  • Tests in the pipeline. Column and Node tests run with every load, declared as annotations like @notNull.
  • Safe promotion. Storage Mappings point the same pipeline at development, test, and production. coa plan shows exactly what a deploy will change before it runs.
  • One governed path for people and agents. Agents work through the same CLI, MCP, and Git history as engineers, with the same permissions.

Key Concepts​

ConceptIn one line
WorkspaceYour pipeline's files in a Git repository, plus your personal connection settings.
NodeOne table or view in the pipeline: a source, stage, dimension, fact, or view.
Node TypeThe template that turns a Node into DDL and DML.
Storage LocationA logical name for where data lives, mapped to a real database and schema per Environment.
EnvironmentA deployed copy of the pipeline, such as QA or production.
JobA scheduled refresh of some or all Nodes in an Environment.

Next Steps​