Test installation
Last updated
Was this helpful?
Last updated
Was this helpful?
This guide will show you how to set up a test environment for UltiHash.
You’ll set up a local environment using Docker Compose for container orchestration.
The main steps are as follows:
Install prerequisite tools
Set up UltiHash with Docker Compose
Integrate sample data + see space savings
This setup is intended for local testing - not production use.
Before you start setting up the UltiHash cluster, you need some tools installed. If you already have any of these installed, you can simply skip that step.
Done! You've successfully installed all the prerequisites for testing UltiHash.
Next, you'll set up your local cluster using Docker Compose.
Now you’ll set up the local UltiHash environment using Docker Compose. This involves authenticating with the UltiHash registry, downloading the necessary configuration file, and running the UltiHash services locally.
Set up authentication with the registry Before you can download and run UltiHash, you need to authenticate with the UltiHash registry. The registry is where the container images (required for running UltiHash) are stored.
Registry login
Registry password
Log in to the UltiHash registry with your credentials:
Download compose.yml
The compose.yml file is a Docker Compose configuration file that defines all the services, volumes, and settings needed to run UltiHash. In addition, policies.json defines the basic policies for the cluster.
Download both files:
Set up credentials and license
To enable access to UltiHash services, you need to export your credentials and license key. These environment variables will be used for authentication.
Run the following commands:
Start UltiHash services
Change the working directory to the folder where you saved compose.yml and policies.json. For example:
Start the UltiHash cluster:
If successful, Docker Compose will download the necessary images (if they’re not already cached) and start the UltiHash services.
Done!
You’ve successfully set up your local UltiHash cluster. Next, let's integrate sample data + see space savings.
Now that UltiHash is running on your local cluster, let's integrate some sample data.
Object storage systems like UltiHash use a top-level container called a bucket. To facilitate scalability, buckets don’t have a traditional hierarchical folder structure: instead, each object in a bucket has a unique key (which can resemble a file path, simulating directories).
To create a bucket, run:
Make sure to replace <bucket-name> with your chosen bucket name, e.g. test-bucket.
You can see your newly created bucket by running:
Now that you have a bucket in which to put objects, let's use the upload script to integrate your sample data.
To integrate your dataset, run:
Make sure to replace <upload-script-path> with the path to the upload script you downloaded, e.g. /home/user/Downloads/uh-upload.py.
Also replace <bucket-name> with your bucket name.
Finally, replace <dataset-path> with the path to the directory for the dataset you prepared or downloaded, e.g. /home/user/Downloads/test-dataset.
A bar should display the ongoing progress of your integration.
Once the integration is complete, you can run the following command to see your objects:
Make sure to replace <bucket-name> with your bucket name.
You can also download an entire bucket by running:
Make sure to replace <download-script-path> with the path to the upload script you downloaded, e.g. /home/user/Downloads/uh-download.py.
Also replace <destination-path> with the path to the directory you want to download the bucket to, e.g. /home/user/Downloads.
Finally, replace <bucket-name> with the name of the bucket to download.
You can see the storage space UltiHash is saving across the entire cluster by running the uh-see-space-savings script:
Make sure to replace <see-space-savings-script-path> with the path to the upload script you downloaded, e.g. /home/user/Downloads/uh-see-space-savings.py.
Done! You’ve successfully integrated a dataset to a local test cluster, and can see the space saved by UltiHash's built-in deduplication.
Please note that to test UltiHash, you need to .
If you want to test UltiHash in a Kubernetes environment, you can do so with .
You can find general instructions for installing Docker Engine at .
You can find general instructions for installing the AWS CLI at .
For this step, you'll need these credentials from your UltiHash :
Make sure to replace <registry-login> with the 'Registry login' from your .
When prompted for a password, enter the 'Registry password' from your .
Make sure to replace <customer-id> and <access-token> with the 'Customer ID' and 'Access token' from your .
DICOM files of brain MRI scans
1.51 GB
JPGs of driving scenarios
2.41 GB
PNGs of synthetic textures with defects
5.89 GB
WAVs of human speech for emotion recognition
0.33 GB
TIFF images of climate data
16.28 GB
CSV tables of symptoms
140 KB