Workspaces

A Workspace is a sandbox environment where you can complete the development of your data, refine, and perform preliminary validation before merging them into the codebase. Each workspace has its own graph, storage locations, macros, node types, connection configuration, and Git branch. You can create multiple workspaces to work on different tasks and merge them into the codebase using Environments. For example, a data engineer may be tasked with creating a new data mart. They could create a new workspace with its own git branch, separate from the main workspace. Once the feature is done, its code can be merged back into the main branch.

• All Coalesce development is completed in a Workspace.
• Objects are created and data is loaded and refreshed manually on-demand, either for individual objects or for subsets of or the full data pipeline.
• When a change is made to an object (for example, adding or dropping a column, renaming a column, etc.), the object is dropped and recreated vs. ALTERed.
• Deleted objects are not cleaned up (DROPPED) in the backend instance.

Create a Workspace

Before creating a Workspace, you should make sure you have a Project created with version control. You can choose from username and password or OAuth.

Username and Password Workspace

1. Select the Project you want to create the Workspace in.

2. Click Create Workspace.

3. Give your Workspace a Name and Description(optional).

   

4. Enter your Snowflake Account URL.

   

   1. Obtain your Snowflake URL, by opening the account selector in Snowflake.

      

5. Add your username and password. If authentication fails the first time, add in your Role and Warehouse then try again.

   

6. Workspaces let you work on a branch. You will need to select a branch and commit to make a new branch. For example, if you want to create a branch off main, select main, then select the commit in main to create your branch from.

   

7. Next, add Storage Locations and Storage Mappings to your Workspace. If you have any existing Storage Locations and Storage Mappings from Git, they will be listed here.

   

8. Click Create Workspace. You've just created your Workspace. Get started by adding some data.

OAuth Workspace

You need to be a Snowflake account admin.

1. Select the Project you want to create the Workspace in.

2. Click Create Workspace. You'll be taken to the Workspace creator.

3. Give your Workspace a Name and Description(optional).

   

4. Enter your Snowflake Account URL.

   

   1. Obtain your Snowflake URL, by opening the account selector in Snowflake.

      

5. Click the toggle under Snowflake OAuth to enable it for your Workspace. This will open a new window where you'll need to enter your Client ID and Secret from Snowflake.

   

   1. In Snowflake, create a new Worksheet.

   2. Run the first part of the code, Create Security Integration. Replace <COALESCE_APP_DOMAIN> with your Coalesce app domain. For example, app.coalescesoftware.io. Make sure it runs successfully. Then run the second part of the code to get the OAUTH_CLIENT_ID and OAUTH_CLIENT_SECRET.

      Create Security Integration - Run First

      Create Security Integration

      -- Create Security Integration 
      CREATE OR REPLACE SECURITY INTEGRATION "COALESCE_OAUTH" 
      ENABLED = TRUE 
      TYPE = OAUTH 
      OAUTH_CLIENT = CUSTOM 
      OAUTH_CLIENT_TYPE = 'CONFIDENTIAL' 
      OAUTH_REDIRECT_URI = 'https://<COALESCE_APP_DOMAIN>/oauthredirect' 
      OAUTH_ISSUE_REFRESH_TOKENS = TRUE 
      OAUTH_REFRESH_TOKEN_VALIDITY = 7776000; 

      Fetch ClientID and ClientSecret - Run Second

      Fetch ClientID and ClientSecret

          -- Fetch ClientID and ClientSecret 
          WITH INTEGRATION_SECRETS AS (
              SELECT parse_json(system$show_oauth_client_secrets('COALESCE_OAUTH')) AS SECRETS
          )
          SELECT
              SECRETS:"OAUTH_CLIENT_ID"::STRING AS CLIENT_ID,
              SECRETS:"OAUTH_CLIENT_SECRET"::STRING AS CLIENT_SECRET
          FROM
              INTEGRATION_SECRETS;

6. Copy the CLIENT_ID and CLIENT_SECRET that are returned.

7. Enter them into Coalesce. Click Next. If needed, enter the Role and Warehouse. Click Authenticate.

8. You'll be asked to sign into Snowflake and give Coalesce access. On success, you'll be taken back to the connection screen and Snowflake OAuth will be enabled. Click Next.

   

9. You'll connect a repo to this Workspace. Your repository was connected during Project setup. Workspaces let you work on a branch. You will need to select a branch and commit to make a new branch. For example, if you want to create a branch off main, select main, then select the commit in main to create your branch from.

   

10. Next, add Storage Locations and Storage Mappings to your Workspace. If you have any existing Storage Locations and Storage Mappings from Git, they will be listed here.

    

11. Click Create Workspace. You've just created your Workspace. Get started by adding some data.

If you need the Client ID and Client Secret again, you can run the following in Snowflake.

WITH INTEGRATION_SECRETS AS (
  SELECT parse_json(system$show_oauth_client_secrets('COALESCE_OAUTH')) AS SECRETS
) 
SELECT
  SECRETS:"OAUTH_CLIENT_ID"::STRING     AS CLIENT_ID,
  SECRETS:"OAUTH_CLIENT_SECRET"::STRING AS CLIENT_SECRET
FROM 
  INTEGRATION_SECRETS;

Key Pair Workspace

1. Select the Project you want to create the Workspace in.

2. Click Create Workspace. You'll be taken to the Workspace creator.

3. Give your Workspace a Name and Description(optional).

   

4. Enter your Snowflake Account URL.

   

   1. Obtain your Snowflake URL, by opening the account selector in Snowflake.

      

5. Change the Authentication Type to Key Pair.

6. Enter your Snowflake Username , Private Key, Private Key Passphrase (if applicable), Role and Warehouse into their respective fields and Save. Click Test Connection to ensure this works as expected.

   Adding Your Private Key

   When entering your private key, make sure it's formatted properly. It must include the full private key including the lines BEGIN ENCRYPTED PRIVATE KEY and END ENCRYPTED PRIVATE KEY.

   -----BEGIN ENCRYPTED PRIVATE KEY-----
   ...
   -----END ENCRYPTED PRIVATE KEY-----
   

7. You'll connect a repo to this Workspace. Your repository was connected during Project setup. Workspaces let you work on a branch. You will need to select a branch and commit to make a new branch. For example, if you want to create a branch off main, select main, then select the commit in main to create your branch from.

   

8. Next, add Storage Locations and Storage Mappings to your Workspace. If you have any existing Storage Locations and Storage Mappings from Git, they will be listed here.

   

9. Click Create Workspace. You've just created your Workspace. Get started by adding some data.

Edit a Workspace

There are two ways to edit a Workspace:

• Go to Build Settings > Workspace and click the cog.
• From the Project page, click the cog next to the Workspace.

You can update the following in a Workspace:

• Settings includes Workspace name, description, and tag color.
• User Credentials including OAuth
• Storage Mappings
• Parameters

Duplicate or Copy a Workspace

To duplicate an Workspace, go to the Projects page and click the Duplicate Settings button, . Then follow the on-screen prompts.

To copy a Workspace, click the three dots next to the Workspace and select Copy Objects from Workspace.

Review When To Use Copy Objects, Duplicate, or Create New Project to learn more about duplicating versus copying and what information will migrate to the Workspace.

Snowflake Passwords

Snowflake password aren't copied.

Delete a Workspace

To delete a Workspace, click the cog icon in Workspace Settings, then click Delete Workspace. If a user deletes all workspaces, a new blank workspace named DEV will be automatically created, as there must always be at least one development workspace.

There are two ways to delete a Workspace:

• Go to Build Settings > Workspace and click the cog.
• From the Project page, click the cog next to the Workspace.

Development Workspace

When you first start working in Coalesce, after you create a Project, a DEV Workspace is automatically created. You can create another Workspace from the Project page.