coalesce.ioTake documentation survey

CLI Commands

Suggest Edits

To access a list of available commands use coa --help.

$ coa [options] [command]

Usage: coa [options] [command]

Options:
  -v, --version                               output the version number
  -b, --debug                                 Output extra debugging (default: false)
  --verbose                                   Log more verbosely (default: false)
  --config <coa-config-location>              coa config file location
   --changeLogLevel <loggingArea>:<logLevel>  INTERNAL:Change logging level of logging area (default: false)
  -h, --help                                  display help for command

Commands:
  deploy [options]                            run a coalesce deployment plan
  //package [options] not available           experimental feature//no longer available
  plan [options]                              plan a Coalesce deployment
  refresh [options]                           runs a Coalesce pipeline
  rerun [options] <runID>                     re-runs a failed Coalesce pipeline. Runtime parameters from the previous run will not be preserved.
  environments                                Environments
  nodes                                       Nodes
  runs                                        Runs
  render [options]                            renders data using coa's renderer
  cancel [options] <runID>                    Cancels a running pipeline.
  help [command]                              display help for command

$ coa plan [options]

Generate a coa-plan.json file that can be deployed.

Run coa plan in the same directory as the locally cloned copy of your Coalesce project's Git repository.

Flags

coa help plan

Options:
  --profile <value>               Profile To Use (default: "")
  --environmentID <value>         Environment ID (default: "")
  --parameters <value>            Parameters (default: "")
  --snowflakeAuthType <value>     Snowflake Auth Type (Basic, KeyPair) (default: "")
  --snowflakeKeyPairPath <value>  Snowflake Key Pair Path (deprecated, please use --snowflakeKeyPairKey) (default: "")
  --snowflakeKeyPairKey <value>   Snowflake Key Pair Path (default: "")
  --snowflakeKeyPairPass <value>  Snowflake Key Pair Pass (default: "")
  --snowflakePassword <value>     Snowflake Password (default: "")
  --snowflakeRole <value>         Snowflake Role (default: "")
  --snowflakeUsername <value>     Snowflake Username (default: "")
  --snowflakeWarehouse <value>    Snowflake Warehouse (default: "")
  --token <value>                 Coalesce Refresh Token (default: "")
  --jobID <value>                 Coalesce JobID (default: "")
  --include <value>               Coalesce Node Selector (default: "")
  --exclude <value>               Coalesce Node Selector (default: "")
  --parallelism <value>           Parallelism level to use (default: "")
  -d, --dir <dir>                 Coalesce pipeline Yaml file path (default: ".")
  --out <plan-location>           Coalesce plan location (default: "./coa-plan.json")
  --gitsha <sha>                  Custom override for the gitsha to use for the plan
  -h, --help                      display help for command

$ coa deploy [options]

Run an existing coa-plan.json. You must create a plan, before you can deploy.

Deploy is used to make structural changes to the data warehouse (typically DDL commands) and save the corresponding data warehouse configuration for the environment for subsequent refreshes. See Deployment and Refresh for information on deployment.

Deploy results

The coa will return information about your run, including the following:

  • runType - Returns the type of run. deploy or refresh.
  • runStatus - The status of the run.
    • canceled
    • completed
    • failed
    • running
    • waitingToRun
  • runResults - Includes the SQL executed, any error details, and other environment information.

Flags

coa help deploy

Include and exclude are only used for refresh.

Options:
  --profile <value>               Profile To Use (default: "")
  --environmentID <value>         Environment ID (default: "")
  --parameters <value>            Parameters (default: "")
  --snowflakeAuthType <value>     Snowflake Auth Type (Basic, KeyPair) (default: "")
  --snowflakeKeyPairPath <value>  Snowflake Key Pair Path (deprecated, please use --snowflakeKeyPairKey) (default: "")
  --snowflakeKeyPairKey <value>   Snowflake Key Pair Path (default: "")
  --snowflakeKeyPairPass <value>  Snowflake Key Pair Pass (default: "")
  --snowflakePassword <value>     Snowflake Password (default: "")
  --snowflakeRole <value>         Snowflake Role (default: "")
  --snowflakeUsername <value>     Snowflake Username (default: "")
  --snowflakeWarehouse <value>    Snowflake Warehouse (default: "")
  --token <value>                 Coalesce Refresh Token (default: "")
  --jobID <value>                 Coalesce JobID (default: "")
  --include <value>               Coalesce Node Selector (default: "")
  --exclude <value>               Coalesce Node Selector (default: "")
  --parallelism <value>           Parallelism level to use (default: "")
  -p, --plan <plan-location>      Coalesce plan location (default: "./coa-plan.json")
  -o, --out <output json>         Run Results Output in Json
  -h, --help                      display help for command

$ coa refresh [options]

Refresh is used to run data transformations, typically DML commands (Merge, Insert, Truncate, etc). See Deployment and Refresh for information on refresh. That deployment must happen on a environment before refresh is possible.

To re-run a failed refresh use:
coa rerun <runID>

This only works on failed refreshes, and only the nodes that previously failed will be run.

Refresh results

The coa will return information about your run, including the following:

  • runType - Returns the type of run. deploy or refresh.
  • runStatus - The status of the run.
    • canceled
    • completed
    • failed
    • running
    • waitingToRun
  • runResults - Includes the SQL executed, any error details, and other environment information.

Flags

coa help refresh

Options:
  --profile <value>               Profile To Use (default: "")
  --environmentID <value>         Environment ID (default: "")
  --parameters <value>            Parameters (default: "")
  --snowflakeAuthType <value>     Snowflake Auth Type (Basic, KeyPair) (default: "")
  --snowflakeKeyPairPath <value>  Snowflake Key Pair Path (deprecated, please use --snowflakeKeyPairKey) (default: "")
  --snowflakeKeyPairKey <value>   Snowflake Key Pair Path (default: "")
  --snowflakeKeyPairPass <value>  Snowflake Key Pair Pass (default: "")
  --snowflakePassword <value>     Snowflake Password (default: "")
  --snowflakeRole <value>         Snowflake Role (default: "")
  --snowflakeUsername <value>     Snowflake Username (default: "")
  --snowflakeWarehouse <value>    Snowflake Warehouse (default: "")
  --token <value>                 Coalesce Refresh Token (default: "")
  --jobID <value>                 Coalesce JobID (default: "")
  --include <value>               Coalesce Node Selector (default: "")
  --exclude <value>               Coalesce Node Selector (default: "")
  --parallelism <value>           Parallelism level to use (default: "")
  -o, --out <output json>         Run Results Output in Json
  --forceIgnoreEnvironmentStatus  Ignore the environment status and refresh even if there is a failed deploy. This may cause
                                  refresh failures!
  -h, --help                      display help for command

$ coa environments [options] [command]

See information about your Environments.

Flags

coa help environments

Options:
  -h, --help      display help for command

Commands:
  list [options]  List Environments
  get [options]   Get Environment
  help [command]  display help for command

$ coa nodes [options] [command]

See information about your Nodes.

Flags

coa help nodes

Options:
  -h, --help      display help for command

Commands:
  list [options]  List Nodes
  get [options]   Get Node
  help [command]  display help for command

$ coa runs [options] [command]

See information about your Runs.

Flags

coa help runs

Runs

Options:
  -h, --help              display help for command

Commands:
  list [options]          List Runs
  get [options]           Get Run
  list-results [options]  List Run Results
  help [command]          display help for command

$ coa render [options]

Render data using the coa renderer.

Flags

coa help render

Options:
  -o, --out <output file>      Render to file
  -f, --from <input file>      Render from file
  -s, --string <input string>  Render input from a string
  -m, --macros <input string>  Extra macros to provide
  -h, --help                   display help for command

$ coa cancel [options]

Cancel a running pipeline.

Flags

coa help cancel

Arguments:
  runID                    The ID of the run to cancel.

Options:
  --environmentID <value>  Environment ID (default: "")
  -h, --help               display help for command

Parallelism Limits

The parallelism parameter determines how many nodes are processed simultaneously (in parallel) by the Snowflake compute warehouse. The API allows a maximum limit of 64, while coa is uncapped.

Overwriting Defaults

You can overwrite the default coa file information by passing in the flags to the CLI or entering the information in the API request.

Using Include and Exclude

Include and Exclude use the Selector format.

oa refresh --include '{ location: SAMPLE name: CUSTOMER } OR { location: SAMPLE name: LINEITEM } OR { location: SAMPLE name: NATION } OR { location: SAMPLE name: ORDERS } OR { location: SAMPLE name: PART } OR { location: SAMPLE name: PARTSUPP } OR { location: SAMPLE name: REGION } OR { location: SAMPLE name: SUPPLIER } OR { location: QA name: STG_PARTSUPP } OR { location: PROD name: STG_PARTSUPP }'

Updated about 1 month ago