Notebook instance management

This documentation provides instructions for creating a notebook instance using Wherobots. The steps outlined below will guide you through launching and configuring a new notebook instance that can be used for running Jupyter notebooks. Following these directions will allow you to initialize a notebook environment with your desired runtime, libraries, and resources.

Note

Before creating a notebook instance, it is recommended to consider a few key points regarding configuration and resource requirements:

• The default disk size for Executor and Driver is 20GB.
• The Executor Instance will default to the same configuration settings as the Driver Instance if a custom Executor configuration is not specified.
• The Notebook instance usually will be ready within 5 minutes after clicking Start to request it. If you observe the notebook stuck in STARTING status, you can try to destroy and re-create the notebook instance. If the problem persists, please contact Wherobots support.

Create notebook instance with minimal configuration

Wherobots provides a variety of configurable settings when creating Notebook instances.

When the notebook dashboard is opened, the configuration options will be shown as:

Quick start fields

Fields	Description
Name	Name of the Notebook instance.
Runtime	Select the proper runtime of the Notebook.
Version	Select the version of the runtime.
Driver Type	The configuration has 3 types of Driver Types. Selection of this field will affect options in Driver Instance field. The core count and memory allocation will be different depending on the Driver Type selected.
	Type	Description
	GENERAL_PURPOSE	This driver type provides an equal distribution of CPU cores and memory resources.
	COMPUTE_OPTIMIZED	This driver type provides a higher CPU core count and less memory resources than GENERAL_PURPOSE.
	MEMORY_OPTIMIZED	This diver type provides higher memory resources and less CPU core count than GENERAL_PURPOSE.
Driver Instance	Select the number of cores and memory you require for the Driver.
Executor Count	Define the number of Executors you need for your Notebook instance.

Quick start will allow you to start your notebook instance quickly with minimal configuration.

Create notebook instance with advanced configuration

Wherobots provides flexible customization options for Notebook instances. Clicking on Advanced Configuration displays the available settings:

Advanced configuration fields

Executor Type

The types are the same as Driver Types, please refer to there. If not defined then the Executor Type is the same as Driver Type.

Executor Instance

Select the number of cores and memory you require for the Executor(s). The core count and memory allocation will be different depending on the Executor Type selected.

Driver Disk Size (GB)

Define the disk size allocated to the driver. The expected input is a valid integer. If not defined then 20GB will be allocated.

Executor Disk Size (GB)

Define the disk size allocated to the Executor(s). The expected input is a valid integer. If not defined then 20GB will be allocated.

Spark Configuration

Add any custom spark configurations you want before starting the Notebook instance. The expected input is a valid JSON, the example configuration is for accessing a public S3 bucket.

{"spark.hadoop.fs.s3a.bucket.<YOUR_BUCKET_NAME>.aws.credentials.provider" : "org.apache.hadoop.fs.s3a.AnonymousAWSCredentialsProvider"}

Libraries

The configuration supports installing multiple additional libraries as needed. There are two available sources to choose packages from.

File Source

File source configuration allows you to add libraries from the Wherobots File section.

Note

Please refer to file structure page to upload the files in the correct directory.

File Type

Two accepted library types include Python Wheel and JAR files.

File Path

Specify the path to the library.

PYPI Source

The Notebook instance is configured to install Python libraries directly from the Python Package Index (PyPI). When the Notebook instance start, it will be fetched and installed from PyPI without requiring any additional configuration. This allows convenient access to the vast collection of Python packages available on PyPI for use within the Notebook environment.

Library Name

Provide the specific name of the library you wish to install.

Library Version

Specify the version of the library you intend to use.

Jupyter notebook environment

Now that your Notebook instance is ready, please refer to Jupyter notebook management for an overview of how to manage Jupyter notebooks. The Jupyter notebook management documentation provides guidance on working with notebooks, including how to run code cells. Reviewing the notebook management guide is recommended before starting to use your Notebook instance.

Deleting notebook instance

Once you have finished using the Notebook instance, you can click the Destroy button to free up resources. This will shut down the Notebook instance and release the resources it was using. It is recommended that you destroy Notebook instances when they are no longer needed.

Note

Community users can expect Notebook instances to automatically shut down after 2 hours of continuous use. Community users will need to restart the Notebook instance if additional time is required beyond the 2 hour limit.

---

Last update: October 17, 2023 07:48:30