Google Cloud Platform(GCP) Immutable Bucket Retention with Veritas NetBackup
Hi Friends,
I've got some great news for you! Veritas now enables the retention policy on a Google Cloud Platform(GCP) bucket to ensure that all current and future objects in the bucket cannot be deleted or replaced until they reach the age you define in the retention policy.
Google cloud storage currently supports bucket level retention and not S3 object locks as seen in AWS S3 and Azure blob storage. Veritas has added bucket level retention to NetBackup for GCP customers that want to protect bucket level retention with immutability.
https://cloud.google.com/storage/docs/bucket-lock
Enabling the retention policy on a bucket ensures that all current and future objects in the bucket cannot be deleted or replaced until they reach the age you define in the retention policy.
- Locked retention policy - You cannot overwrite or delete the data that is protected using the locked policy for the defined retention period. Once you set a retention period for the bucket, you can extend it but cannot shorten or remove it.
- Unlocked retention policy - You cannot overwrite or delete the data that is protected using the unlocked policy for the defined retention period. Once you set a retention period for the bucket, you can extend, shorten, or delete it.
The bucket may have multiple cloud volumes. Retention policy in the bucket will protect all volumes' backup image.
Configuration
To manage Google cloud immutable storage use the Veritas msdpcldutil tool:
- Set the following environment variables
# export MSDPC_REGION=<your region>
# export MSDPC_PROVIDER=google
# export MSDPC_ACCESS_KEY=<your storage account>
# export MSDPC_SECRET_KEY=<your access key>
# export MSDPC_ENDPOINT=https://storage.googleapis.com
# export MSDPC_GCP_SAKEY=<Your Google service account key file path>
Note: To get the service account key, see Create and manage service account keys.
- IAM & Admin > Service accounts > add, create a service account and grant this service account access to GCP buckets.
- APIs & Services > Credentials, click the service account, goto KEYS tab > ADD KEY,save this JSON keyfile.
- To get the ACESS_KEY and SECRET_KEY, see HMAC keys
- Cloud Storage > Settings > interoperability tab > Service account HMAC, create a KEY for your service account.
- Store the access key and secret key
- Create a Google cloud immutable storage.
# msdpcldutil bucket create --bucket bucketname --mode ENTERPRISE --period 2D
ENTERPRISE is unlocked policy and COMPLIANCE is locked policy in Google.
- List the Google cloud immutable storage.
#/usr/openv/pdde/pdcr/bin/msdpcldutil bucket list
- Get the Google cloud immutable storage information.
#/usr/openv/pdde/pdcr/bin/msdpcldutil bucket info --bucket bucketname
- Update the Google cloud immutable storage retention period.
#/usr/openv/pdde/pdcr/bin/msdpcldutil bucket update –bucket bucketname –mode ENTERPRISE –period 3D
- Update the Google cloud immutable storage retention mode.
#/usr/openv/pdde/pdcr/bin/msdpcldutil bucket update –bucket bucketname –mode COMPLIANCE –period 3D
- If you change the retention policy through Google WebUI, you must sync the MSDP configuration file.
#/usr/openv/pdde/pdcr/bin/msdpcldutil bucket sync –bucket bucketname
Deployment
- Use the Veritas msdpcldutil command to create the Google cloud storage bucket.
- Set the following environment variables
# export MSDPC_REGION=<your region>
# export MSDPC_PROVIDER=google
# export MSDPC_ACCESS_KEY=<your storage account>
# export MSDPC_SECRET_KEY=<your access key>
# export MSDPC_ENDPOINT=https://storage.googleapis.com
# export MSDPC_GCP_SAKEY=<Your Google service account key file path>
Note: To get the service account key, see Create and manage service account keys
To get the ACESS_KEY and SECRET_KEY, see HMAC keys
- Create a Google cloud immutable storage.
# msdpcldutil bucket create --bucket bucketname --mode ENTERPRISE --period 2D ENTERPRISE is unlocked policy and COMPLIANCE is locked policy in Google.
- Use the NetBackup WebUI to create the Google storage unit.
- Use msdpcldutil command to create the cloud immutable volume. Note down the volume name, it will be used in step 4.
- On the NetBackup Web UI, navigate to Storage > Disk pools, and click Add.
- In Disk pool options, click Change to select a storage server. Enter the Disk pool name. If Limit I/O streams is left cleared, the default value is Unlimited and may cause performance issues. After all required information is added, click Next
- From the Volume drop-down list select a volume or add a new volume. Provide the name that is created in step 1 by msdpcldutil.
- In the Cloud storage provider window, select Google Storage Cloud from the list.
- Under Region, select the appropriate region.
- Enter the credentials to complete the setup. You can configure additional options here such as adding a proxy server.
- Under Cloud bucket, select Select or create cloud bucket and click Retrieve list. Select a bucket from the list. You can also provide the bucket name. If you provide the bucket name, ensure this bucket is created by msdpcldutil.
- If encryption is needed, select the data encryption option for data compression and encryption. MSDP can use KMS encryption which encrypts the data using a managed key. Using KMS requires that a KMS server has previously been configured.
- Enter all required information based on the selection and click Next.
- In Replication, click Next.
- On the Review page, verify that all settings and information are correct. Click Finish. The disk pool creation and replication configuration continue in the background if you close the window. If there is an issue with validating the credentials and configuration of the replication, you can use the Change option to adjust any settings.
- In the Storage unit tab, click Add.
- Select Media Server Deduplication Pool (MSDP) and click Start.
- In Basic properties, enter the Name of the MSDP storage unit and click Next.
- Select the disk pool that was created and click Next.
- In Media server, use the default selection of Allow NetBackup to automatically select, and click Next.
- Review the setup of the storage unit and then click Save.
Veritas NetBackup now enables immutability on a Google Cloud Platform(GCP) buckets to ensure that all current and future objects in the bucket cannot be deleted or replaced until they reach the age you define in the retention policy.