Skip to content

Databricks

Setup Permissions for Wherobots through Databricks

To link to your Databricks account, you need an existing Databricks account. If you don’t have a Databricks account, you can sign up at https://www.databricks.com/try-databricks.

Generate a Databricks Access Token

  1. Log in to your Databricks account.
  2. Please create a Databricks workspace if you don't have one. Databricks workspace
  3. Log in to your Databricks workspace. The URL of this workspace is dbc-XXX-XXX.cloud.databricks.com. Then your workspace ID is dbc-XXX-XXX. You will need it in the next steps.
  4. Navigate to your Databricks dashboard

  5. Click on your User profile

    User profile

  6. Click on “User Settings”

  7. Click on “Access tokens”

    Access tokens

  8. Press the button labelled “Generate new token”

  9. Set the lifetime of the token to something appropriate to your security requirements, or set it to nothing to have it never expire. Add an appropriate comment.

    Generate new token

  10. Press “Generate”

  11. Copy the Token and save it somewhere. You will need it for the next steps.

    Token generated

  1. Navigate to the Wherobots Databricks cloud provider page
  2. Enter your workspace ID and Access token from Databricks into the corresponding fields
  3. Press “Save Changes”

Create the Databricks Cluster

Note

Wherobots currently does not support Databricks 13.0+ (Spark 3.4+) and Job Compute clusters. Please choose All Purpose clusters.

  1. Log in to your Databricks workspace.
  2. Under Data Science & Engineering menu, choose compute
  3. Click the Create compute button. You can use the default setting if you just want a quick start.

Setup your Databricks Cluster with Wherobots

Warning

This will wipe out the existing Apache Sedona / Wherobots jars on DBFS, and overwrite existing Sedona / Wherobots settings of the cluster with the selected Wherobots version.

  1. Navigate to the Third Party Connect page on https://wherobots.services.

    Databricks clusters page

  2. Enter your credentials and click on 'Load Clusters'. Click 'Install Sedona' for the cluster you want to set up Sedona in.

    Set Up Wherobots Button

  3. Select your preferred Wherobots Version and press “Install”.

    Install Wherobots Popup

  4. Wait for the installation to finish, the progress can be tracked through a notification on the bell icon.

    Install in progress

  5. Once the installation has succeeded, you can use the cluster on databricks after it has started.

    Install successful notification

Import a Notebook and run spatial queries on Databricks

We provide many ready-to-use example Python Jupyter notebooks. Please try them out: Wherobots examples.

  1. Navigate to your Databricks workspace and press the kebab icon (three vertical dots) and press “Import” to import your notebook.

    Databricks console

  2. Select your notebook file. You can get one from Wherobots examples.

    Select notebook

  3. Press “Import” to complete the import of your notebook.

    Import notebook

  4. Now your notebook is imported. You can open it, select your cluster in the top right, and run spatial queries using Wherobots!

    Run spatial queries

Last update: October 17, 2023 01:49:40