Skip to main content

Coalesce Scheduler

The Coalesce Scheduler gives you the ability to schedule data pipeline refreshes directly in the Coalesce app. You no longer need to use a third-party solution simplifying your IT dependency and allowing you to generate data insights faster.

Before You Begin​

  • You must have an Environment Admin role.
  • The Environment must have Snowflake credentials added. The Scheduler uses the Snowflake credentials of the user that created or last modified the Scheduled Job.
  • A Job must be deployed before it can be scheduled. Learn more about deploying your pipeline.

How to Schedule a Job​

  1. In your Coalesce App, go to the Deploy page.

  2. Next to any Job, click the dropdown and select Create Job Schedule. You can also click the Deploy button next to the environment.

    A dropdown menu under a job called Test Refresh Job within a development environment. The menu offers options to run the job, create a job schedule, or view existing schedules.

  3. Configuration: Fill out the Job Schedule form. If you click on the Job to create a Job schedule, the Project, Environment, and Job Name will be pre-filled.

    1. Project - You’ll be able to create Jobs for any Project you have Environment Admin access to.
    2. Environment - Select the Environment in the Project.
    3. Job Name - Select an existing Job in the Environment.
    4. Job Schedule Name - Choose a unique name to make it easy to identify later.
    5. Cron Expression - Enter a Cron expression in UTC time for the Job frequency.
    6. Retry on Failure - Toggle to retry the entire Job if it fails. This will retry the entire Job.
      1. Number of Retries - Enter the number of times a Job will attempt to run if it fails the first time.
      2. Time Between Retries - Enter the number of minutes between each retry.
      3. Retry Job From - Beginning will run the entire Job again. Point of Failure will retry the Job from when it failed.
    This image shows a web interface for editing a job schedule in the Coalesce platform. The interface includes various configuration options such as project, environment, job name, schedule frequency, and retry settings for managing automated jobs.
    Jobs Run One Time When Saved

    Jobs run one time as soon as they are saved. Then run on the normal schedule after. Be sure you want this Job to run.

  4. Notifications: Coalesce will send an email notification for successful, retries, and failed Jobs.

    1. Email on: Choose if you want emails to be sent for Success only, Errors only (includes retries), or both.
    2. Email Addresses: Each email address should be entered on a new line.
    Editing a job schedule in a development environment. It includes fields for the project, environment, job name, and job schedule name, along with a cron expression for scheduling

  5. Save the Job. You’ll be able to see the Job on the Job Schedules page.

Detailed view of a specific job schedule within a development environment, including the last execution time, next execution time, and the run status marked as Successful.
Refresh the Page

Refresh the page to see the status of a Scheduled Job.

View Scheduled Jobs​

You can view Scheduled Jobs in two ways.

  1. Select View Schedules next to the Job you want to view the schedules for.

    A dropdown menu under a job called Test Refresh Job within a development environment. The menu offers options to run the job, create a job schedule, or view existing schedules.

  2. Select View Schedules Jobs next to Deploy to see all Scheduled Jobs for the Environment.

A dropdown menu next to the deploy button offers shows options for environment settings, viewing scheduled jobs, and documentation

You’ll be able to view a list of Scheduled Jobs on the Jobs Schedule page.

Job schedules within a development environment, showing details like project name, environment, job name, schedule name, last execution, next execution, and run status. The list shows three different job schedules with their respective statuses and times

Edit Scheduled Jobs​

  1. On the Jobs Schedule page, in the Actions column, select Edit.
  2. You can edit the Job Schedule Name and Frequency. If you need to edit the Project or other information create a new Job Schedule.
Editing a job schedule in a development environment. It includes fields for the project, environment, job name, and job schedule name, along with a cron expression for scheduling
Editing Jobs Created By Another User

Editing Scheduled Jobs created by another user, will use your Coalesce and Snowflake credentials to run the job

Scheduled Jobs Notifications and Alerts​

Notifications can be sent using email. You can configure emails to be sent for Success, Error, or Both.

You can also edit the notifications at any time by editing the Job Schedule.

Editing job schedule notifications, allowing the user to configure email notifications for job successes, errors, or both. It includes fields to input multiple email addresses.

Notifications are sent for:

  • Each successful Job Schedule.
  • Each failed Job Schedule.

Delete Job Schedules​

To delete a Job Schedule, go to the Jobs Schedule page, in the Actions column, select Delete.

  • You can delete a Scheduled Job at any time.
Deleting a Job

If you delete a Job attached to a schedule, then commit the deleted Job change to the deploy branch, then try to deploy, you’ll get a warning message in the Review Plan step. The Jobs Schedules for the deleted Job will be removed at the next scheduled run and won’t be part of the deploy.

A deployment review screen with a warning message stating that certain job schedules will be deleted and will not run after deployment. A specific job schedule named Test delete is highlighted as being marked for deletion

Scheduled Job Status and Results​

On the Job Schedule page, you can see:

  • Last Execution - Last time the Scheduled Job was run displayed in system time.
  • Next Execution - Next Scheduled Job run time.
  • Run Status - Most recent status of the Scheduled Job.
    • Successful
    • Running
    • Failed

To view the results click on the Run Status of any scheduled job.

Results of a specific job schedule within the development environment, including details such as the project, environment, job name, schedule name, last execution, next execution, and run status, with a successful execution indicated. The successful text is higlighted

Scheduler Permissions​

Important Scheduling Information​

  • You can create multiple schedules for each Job.
  • Refreshed All Nodes and Ad-Hoc jobs aren’t part of the Job Scheduler and can only be run using the CLI or API.
  • Schedules use Cron to run and they are in UTC time.
  • Scheduled Jobs are run one time as soon as they are created.
  • Selecting Run Job from the Jobs menu will run the Job one-time. It will not change the Scheduled Job.
  • Scheduled Jobs can’t be paused. You’ll need to delete the Job Schedule or change the frequency.
  • Scheduled Jobs run in parallel. If more than one Job is scheduled for the same time, they will run at the same time. If there is an ad hoc refresh during a Scheduled Job, they will run at the same time.
  • Scheduled Jobs aren’t committed to your Git provider.

Resources​