Parameters

Parameters act as read-only environment variables. Parameters are defined as a JSON blob set by the user that can be accessed in the metadata during template rendering. These values can be set for a Workspace/Environment at various points to provide flexibility of template behavior during Deploying or Refreshing.

Where Parameters Can Be Used

• In Your Workspace or Environment to set default parameters.
• Overwrite default parameters at deploy.
• Overwrite default parameters at refresh .

Parameters in Git

Parameters are exclusively stored in Coalesce's metadata repository and will not be committed to your git repository.

Why Use Parameters

• Environment Configuration: Parameters allow you to configure your data pipeline for different environments (e.g., development, staging, production) without modifying the underlying code. For example, you can use parameters to specify different database credentials, file paths, or other environment-specific settings.
• Dynamic Data Filtering: Parameters can be used to dynamically filter or partition data based on certain criteria. For example, you can use a date parameter to only process data for a specific date range or a customer ID parameter to process data for a particular customer.
• Reusability and Modularity: By changing certain values or logic into parameters, you can make your data pipeline more modular and reusable. This promotes code reuse and maintainability, as you can easily swap out parameter values without modifying the core transformation logic.
• Testing and Debugging: Parameters can be helpful for testing and debugging purposes. You can use different parameter values to simulate various scenarios or edge cases, making it easier to identify and fix issues in your data pipeline.
• Scheduling and Automation: When scheduling or automating data pipeline runs, parameters can be used to pass dynamic values or configurations. This allows you to adapt the pipeline behavior without changing the underlying code, enabling greater flexibility and control.
• Separation of Concerns: By externalizing certain values or configurations as parameters, you can separate the concerns of data transformation logic from the specific values or configurations used in different contexts. This promotes better code organization and maintainability.
• Compliance and Auditing: In regulated industries or environments with strict data governance requirements, parameters can be used to enforce data access controls, data masking, or other compliance-related configurations.