GCS native support (GCS implemented using Google Cloud Storage libraries)

[gifi-siby]

Native support for Google cloud storage using the Google Cloud Storage libraries. Authentication is performed using GCS service account credentials JSON.

Milvus supports GCS native and Milvus-Backup doesn't. Since Milvus supports GCS native, this enhancement is needed.

Related Issue: #444

PRs worked together are:
#494
#496

[mergify]

@gifi-siby Thanks for your contribution. Please submit with DCO, see the contributing guide https://github.com/milvus-io/milvus/blob/main/CONTRIBUTING.md#developer-certificate-of-origin-dco.

[huanghaoyuanhhy]

I noticed that in your PR for the GCP Native client, you implemented hierarchical deletion for files. Has GCP encountered similar deletion error issues as Azure? I’d like to understand the reasoning behind your choice of hierarchical deletion.

[gifi-siby]

In GCP, I didn't face the issue, but I adopted the same deletion method as in Azure as a precautionary measure.

The function RemoveWithPrefix removes the objects in a way that respects the hierarchy of the storage, meaning it starts by deleting the deepest files first and works its way up. It uses goroutines to delete objects in parallel.

[sre-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: gifi-siby
To complete the pull request process, please assign thomas-huwei after the PR has been reviewed.
You can assign the PR to them by writing /assign @thomas-huwei in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment