Storage layer management
This page provides detailed information about the Wherobots file structure in cloud storage and its associated behaviors.
File Browser¶
File browser allows you to see the File structure of the Wherobots cloud.
Folder's behavior¶
The subsequent folders are immutable, indicating that it is not possible to create new folders, upload files, or delete any existing folders.
/(root)/data//notebooks/
The subsequent folders are where the user can create new folders, upload files, or delete any file within them.
/data/customer-XXXX/data/shared/notebooks/customer-XXXX/notebooks/shared/spark-logs
The data directory is accessible from within the Jupyter notebook environment via predefined environment variables:
USER_S3_PATH- that points to/data/customer-XXXXUSER_S3_SHARED_PATH- it points to/data/sharedUSER_WAREHOUSE_PATH- that points to/data/customer-XXXX/warehouse
Note
The /spark-logs folder allows read and write permissions, but it is recommended that users refrain from tampering with the logs, as it may affect their experience.
User specific folders¶
The folders with the name of customer-XXXX are unique to each user. The files and folders within them can only be accessed by the user.
Organization wide shared folders¶
Folders named shared grant access to everyone within your organization. Any member of the organization can create new folders, delete existing ones, and have the privileges to read, write, upload, and delete files.
Copy S3 Path¶
You are able to copy the S3 path for your directory and files, on the right side of your displayed folder or files, there is a copy icon for you to copy the S3 path.
One-way auto-syncing¶
Modifications made in the home directory of the Jupyter notebook within the Notebook instance are automatically synchronized with the file structure every 2 minutes. Additionally, synchronization occurs when you request to destroy the Notebook instance, guaranteeing that no data or progress is lost.
This synchronization operates in a one-way manner. Specifically, if you add any notebooks or data while a Notebook instance is in the RUNNING state, those changes will not be reflected in the Jupyter Notebook interface. To ensure visibility in the Jupyter Notebook, it is necessary to add files before creating a Notebook instance. This process guarantees the complete replication of all contents from /notebooks/customer-XXXX to the Notebook instance, enabling seamless recovery of the file structures.
Note
Synchronization will be performed to the directory path /notebooks/customer-XXX.