Skip to content

Databricks

Setup Permissions for Wherobots through Databricks

To link to your Databricks account, you need an existing Databricks account. If you don’t have a Databricks account, you can sign up at https://www.databricks.com/try-databricks.

Generate a Databricks Access Token

  1. Log in to your Databricks account.
  2. Please create a Databricks workspace if you don't have one. 1.jpg
  3. Log in to your Databricks workspace. The URL of this workspace is dbc-XXX-XXX.cloud.databricks.com. Then your workspace ID is dbc-XXX-XXX. You will need it in the next steps.
  4. Navigate to your Databricks dashboard

  5. Click on your User profile

    Screenshot 2023-03-21 at 15.12.11.png

  6. Click on “User Settings”

  7. Click on “Access tokens”

    Screenshot 2023-03-21 at 15.12.47.png

  8. Press the button labelled “Generate new token”

  9. Set the lifetime of the token to something appropriate to your security requirements, or set it to nothing to have it never expire. Add an appropriate comment.

    Screenshot 2023-03-21 at 15.13.04.png

  10. Press “Generate”

  11. Copy the Token and save it somewhere. You will need it for the next steps.

    Screenshot 2023-03-21 at 15.13.13.png

  1. Navigate to the Wherobots Databricks cloud provider page
  2. Enter your workspace ID and Access token from Databricks into the corresponding fields
  3. Press “Save Changes”

Create the Databricks Cluster

Note

Wherobots currently does not support Databricks 13.0+ (Spark 3.4+) and Job Compute clusters. Please choose All Purpose clusters.

  1. Log in to your Databricks workspace.
  2. Under Data Science & Engineering menu, choose compute
  3. Click the Create compute button. You can use the default setting if you just want a quick start.

Setup your Databricks Cluster with Wherobots

Warning

This will wipe out the existing Apache Sedona / Wherobots jars on DBFS, and overwrite existing Sedona / Wherobots settings of the cluster with the selected Wherobots version.

  1. Navigate to the clusters page on https://wherobots.services.

    Databricks clusters page

  2. Select your cluster and click “Set Up Wherobots”.

    Set Up Wherobots Button

  3. Select your preferred Wherobots Version and press “Install”.

    Install Wherobots Popup

  4. Wait for the installation to finish and the notification to clear.

    Install in progress Install successful notification

  5. Once the notification has cleared, press “Start” to start your Databricks cluster.

    Start cluster

  6. Wait for your Databricks cluster to start.

    Cluster started

Import a Notebook and run spatial queries on Databricks

We provide many ready-to-use example Python Jupyter notebooks. Please try them out: Wherobots examples.

  1. Navigate to your Databricks workspace and press the kebab icon (three vertical dots) and press “Import” to import your notebook.

    Databricks console

  2. Select your notebook file. You can get one from Wherobots examples.

    Select notebook

  3. Press “Import” to complete the import of your notebook.

    Import notebook

  4. Now your notebook is imported. You can open it, select your cluster in the top right, and run spatial queries using Wherobots!

    Run spatial queries

Last update: August 30, 2023 18:56:20