Skip to main content

Databricks

info

This integration covers both Unity Catalog and non-Unity Catalog (Hive) Databricks

info

This integration is to replicate for each workspace you want to integrate with Catalog

Requirements​

  • You must be a Databricks administrator of the workspace to integrate

1. Create a Personal Access Token​

info

Personal Access Tokens are related to an account

Create a Token on an account with Databricks SQL access

If you already have a generic account with Databricks SQL access, you can go to token creation

If you want to create a dedicated user for Catalog, see how to Manage Users:

Create the token​

You can create a token as follows:

  1. In the top right menu containing the account mail address, go on User settings
  2. In the default Access tokens tab, Generate new token
  3. Name your token and set Lifetime (days) parameter on empty,
  4. Create a JSON file with following format

2. Retrieve your host​

Your host or instance name can be found in your Databricks URL https://<instance-name>.cloud.databricks.com or https://<instance-name>/

3. Retrieve your http_path​

Your http_path can be found with the following steps (more details here):

  1. Log in to your Databricks workspace.
  2. In the sidebar, click Compute.
  3. Choose a cluster to connect to.
  4. Navigate to Advanced Options.
  5. Click on the JDBC/DBC tab.
  6. Copy the http_path

4. Enable the system_tables​

To enable the system_tables , you can follow Databricks documentation here

5. Credentials​

Your credentials should look like the following one :

{
    "token": "<your-token>",
    "host": "<your-server-hostname>",
    "http_path": "sql/protocolv1/o/xxxxxx"
}

Add user's connection info into Catalog​

Paste you JSON file on your integration page

  1. Add Databricks

  2. Pick "Catalog Managed" and name your integration (should you have several ones)

  3. Paste your JSON in the Credentials window

For your first sync, it will take up to 48h and we will let you know when it is complete