Skip to main content

Databricks

KoraBridge loads through a Databricks SQL Warehouse and uses a managed Unity Catalog volume for temporary staging. An OAuth2 service principal is recommended, and an access token stays available as a fallback.

What you need​

  • Workspace hostname. The hostname only, without https:// or extra path segments.
  • SQL Warehouse HTTP path. KoraBridge loads through a SQL Warehouse.
  • Authentication. OAuth2 service principal credentials (recommended), or an access token as a fallback.
  • Unity Catalog destination. The target catalog and a managed staging volume in catalog.schema.volume format.

Where to find it​

  • Workspace URL. Copy the hostname from your workspace URL and remove https:// and any trailing path.
  • SQL Warehouse connection details. Open the target SQL Warehouse and copy the HTTP path from its connection details.
  • OAuth2 service principal. The client ID and client secret come from your Databricks service principal or app registration workflow.
  • Unity Catalog. Confirm the catalog name and the fully qualified managed volume name in Unity Catalog.

Workspace connection​

Confirm the workspace hostname and SQL Warehouse details before you test the destination.

Server hostname​

Copy only the hostname from the workspace URL, such as workspace.azuredatabricks.net.

HTTP path​

The HTTP path of the selected SQL Warehouse. KoraBridge uses it for COPY INTO loads.

OAuth2 service principal (recommended)​

Use this for shared and unattended pipelines. Make sure the service principal is assigned to the workspace before you test.

Client ID​

The OAuth2 client ID issued for the service principal.

Client secret​

The OAuth2 client secret issued for the service principal. KoraBridge stores it encrypted.

Access token (fallback)​

Use this only when you already have a valid Databricks access token and OAuth2 setup is not available yet. Provide the token alongside the shared workspace, warehouse, catalog and staging fields, and do not also fill in OAuth2 credentials. It is the only non-service-principal option in this form.

Unity Catalog and staging volume​

Catalog​

The Unity Catalog that holds the destination tables and the staging volume.

Staging volume name​

A fully qualified catalog.schema.volume name, in the same catalog as the Catalog field. This is temporary Databricks load staging, not another KoraBridge destination.

Required grants​

The identity KoraBridge connects as needs all of the following.

OnPrivileges
The workspaceAssignment to the workspace
The SQL WarehouseCAN USE
The catalogBROWSE and USE CATALOG
The catalog, for creating schemasCREATE SCHEMA
Schemas KoraBridge creates (one per pipeline) and their _staging twinsUSE SCHEMA, CREATE TABLE, SELECT, MODIFY
The staging volumeREAD VOLUME and WRITE VOLUME

KoraBridge creates a new schema for each pipeline at load time, with a generated name, so you cannot grant those schemas one by one in advance. Merge and replace loads also stage in a twin schema with a _staging suffix, and a merge pipeline fails on USE SCHEMA for that twin even when the main schema is granted. Grant the schema privileges at the catalog level, together with CREATE SCHEMA, so every current and future schema is covered. This is the normal setup.

The connection test runs a small merge load, so it exercises the _staging path. If a load fails on a missing permission, fix the grants and run it again. KoraBridge keeps the unfinished load and retries it. Do not discard pending data unless a retry still fails.

Common mistakes​

  • Including https:// or extra path segments in the workspace hostname.
  • A staging volume that is not in catalog.schema.volume format, or that uses a different catalog than the Catalog field.
  • Treating the staging volume as a second KoraBridge destination instead of temporary load staging.
  • Mixing OAuth2 and access token credentials instead of choosing one.
  • Granting the destination schema but not its _staging schema.