API use
UltiHash offers a powerful, S3-compatible API that allows developers to interact with storage clusters using familiar commands and libraries designed for Amazon S3. In addition to supporting standard S3 operations, UltiHash extends the functionality by providing unique features such as deduplication metrics.
S3 compatibility can be achieved in different ways depending on your environment. For example, Python developers typically use the boto3
library, which is part of the AWS SDK for Python and provides a straightforward interface for interacting with UltiHash as if it were S3. In contrast, data processing tools like PySpark don’t use boto3
, instead, they rely on connectors like s3a
, which is part of the Hadoop ecosystem and optimized for distributed data processing. This distinction is important: boto3
is great for scripting and general-purpose workloads, while s3a
is better suited for large-scale data operations in frameworks like Spark.
While UltiHash is S3-compatible, you can use any S3-compliant SDK to interact with it. The AWS SDKs offer extensive support across various programming languages like Python, Java, Node.js, and more. These SDKs are well-documented and maintained, making them the ideal choice for most developers.
For developers looking to explore SDKs across various languages and environments, we highly recommend visiting AWS Developer Tools, which offers comprehensive support for integrating with S3-compatible APIs, including UltiHash.
For any further questions or advanced use cases, our support team is always ready to assist you.
Essential Operations
UltiHash supports a wide range of S3 API operations, enabling you to manage your data effectively. Here are the core operations you can perform:
CreateBucket: Create new buckets in your UltiHash storage.
aws s3api create-bucket --bucket your-bucket-name --endpoint-url https://your-ultihash-endpoint
PutObject: Upload files to your UltiHash buckets.
aws s3 cp local-file.txt s3://your-bucket-name/ --endpoint-url https://your-ultihash-endpoint
GetObject: Retrieve files from your UltiHash storage.
aws s3 cp s3://your-bucket-name/file.txt local-file.txt --endpoint-url https://your-ultihash-endpoint
ListObjectsV2: List the contents of your buckets.
aws s3api list-objects-v2 --bucket your-bucket-name --endpoint-url https://your-ultihash-endpoint
DeleteObject:
Manage your storage by removing one unnecessary object.
aws s3api delete-object --bucket your-bucket-name --key your-object-key --endpoint-url https://your-ultihash-endpoint
Facilitate object deletion operations by removing all objects in one bucket
aws s3 rm s3://your-bucket-name/ --recursive --endpoint-url https://your-ultihash-endpoint
DeleteBucket: Manage your storage by removing unnecessary buckets (buckets should be empty before removal).
aws s3api delete-bucket --bucket your-bucket-name --endpoint-url https://your-ultihash-endpoint
All supported API Functions
S3 Compatibility Layer
AbortMultipartUpload
CreateMultipartUpload
CompleteMultipartUpload
CopyObject
CreateBucket
DeleteBucket
DeleteBucketPolicy
DeleteObject
DeleteObjects
GetBucketPolicy
GetObject
HeadBucket
HeadObject
ListBuckets
ListMultipartUploads
ListObjects
ListObjectsV2
PutBucketPolicy
PutObject
UploadPart
IAM Compatibility Layer
CreateAccessKey
CreateUser
DeleteAccessKey
DeleteUser
DeleteUserPolicy
GetUserPolicy
ListUserPolicies
PutUserPolicy
For more details on available SDKs and language-specific guides, check out the AWS SDK hub.
Deduplication Metrics
get_effective_size.py
: https://github.com/UltiHash/scripts/tree/main/boto3/ultihash_info
UltiHash extends beyond the standard S3 API with features like deduplication metrics, which allow you to query the effective size of your data after deduplication. This unique functionality is crucial for optimizing storage and understanding your actual storage usage.
Example:
# Retrieve deduplicated data size from UltiHash
get_effective_size.py --url <https://ultihash>
Last updated
Was this helpful?