How to on-board new public cloud as back-end in OpenSDS multi-Cloud Project?

2019-04-30 : BY CLICK2CLOUD-BLOG

On boarding the new public Cloud as back-end in OpenSDS will help users to perform all object storage related operations such as “Create Bucket”, “Delete Bucket”, “Upload Object”, “Download Object”, “Delete Object” directly from OpenSDS application. Also, user will be able to migrate its bucket from any OpenSDS supported public cloud or local storage to newly on-boarded cloud and vice-versa.

This blog will help you on how to add S3 compatible Cloud as back-end in OpenSDS multi-cloud:

Add Cloud provider as object storage back-end in OpenSDS multi-cloud project:

Implement below interfaces in multi-cloud (Gelato) project for adding cloud as a back-end. More details about interfaces can be found here.


PUT(stream io.Reader, object *pb.Object, context context.Context) S3Error

GET(object *pb.Object, context context.Context) (io.ReadCloser, S3Error)

DELETE(object *pb.DeleteObjectInput, context context.Context) S3Error

GetObjectInfo(bucketName string, key string, context context.Context) (*pb.Object, S3Error)

InitMultipartUpload(object *pb.Object, context context.Context) (*pb.MultipartUpload, S3Error)

UploadPart(stream io.Reader, multipartUpload *pb.MultipartUpload, partNumber int64, upBytes int64, context context.Context) (*model.UploadPartResult, S3Error)

CompleteMultipartUpload(multipartUpload *pb.MultipartUpload, completeUpload *model.CompleteMultipartUpload, context context.Context) (*model.CompleteMultipartUploadResult, S3Error)

AbortMultipartUpload(multipartUpload *pb.MultipartUpload, context context.Context) S3Error

ListParts(listParts *pb.ListParts, context context.Context) (*model.ListPartsOutput, S3Error)


After successful implementation and integration of interfaces, perform the testing by registering cloud as back-end with or without Keystone. On boarded cloud provider name will appear in “Type” drop-down. Refer below screenshot.

Facilitate the Cloud which is added as a back-end for supporting migration in OpenSDS:

Implement Data Mover interfaces in multi-cloud (Gelato) project for newly added Cloud backend. You can refer implementation of “Amazon S3 Data Mover” interface. It is called in the migration routine.

Implement the below interfaces defined in MoveWorker.


UploadObj(objKey string, destLoca *LocationInfo, buf []byte) error

DownloadObj(objKey string, srcLoca *LocationInfo, buf []byte) (size int64, err error)

DeleteObj(objKey string, loca *LocationInfo)

MultiPartDownloadInit(srcLoca *LocationInfo) error

DownloadRange(objKey string, srcLoca *LocationInfo, buf []byte, start int64, end int64) (size int64, err error)

CompleteMultipartUpload(objKey string, destLoca *LocationInfo) error

AbortMultipartUpload(objKey string, destLoca *LocationInfo) error

UploadPart(objKey string, destLoca *LocationInfo, upBytes int64, buf []byte, partNumber int64, offset int64) error

MultiPartUploadInit(objKey string, destLoca *LocationInfo)


After successful implementation and integration of interfaces, perform the testing by migrating Bucket from AWS S3, Microsoft Azure, Huawei OBS to your on-boarded Cloud and vice-versa.