Forum Discussion

SplashMasterson's avatar
SplashMasterson
Level 4
8 years ago

Configuring Amazon S3 in China (Blog)

(This isn't a question, just documentation of how we fixed an issue I couldn't find an answer to online. I have no doubt all of this is in the Cloud Admin Guide, I just decided to take the long route and skip that part :manembarrassed: )

My company has a handful of remote offices around the world, with two of them in different sites in China. In each of those China offices we run a 7.7.3 NetBackup Master Server, (Windows 2008), to back up local data to disk, and use SLP to replicate those images across the WAN to the other office for offsite protection. This has been working successfully for 3 years, but we've slowly been adding more data to these offices and can no longer do local backups AND crosssite replication without filling up the disk targets. We're using "borrowed storage" from another server to protect our offsite copies at the moment while we decide where to go next. 

One of our new initiative tests was to replicate to S3, but AmazonAWS has a separate China environment different from the amazonaws.com that the rest of the world can access, amazonaws.com.cn. Our Cloud Admin set up a new S3 instance in China without an issue, but the problem is the default Amazon cloud instance in NetBackup 7.7.3 is not customizable, and does not include the China Amazon region. The problem is the cloudprovider.xml file (C:\Program Files\Veritas\NetBackup\db\cloud) is locked and there are no commands available to add the China region to the Amazon plugin. 

I was able to get a new cloudprovider.xml from NetBackup support, but that was only because the tech working my case had one on his desktop from helping another customer. The actual solution to this, and any other provider that isn't a default Cloud option, is to contact the vendor and request the plugin directly from them. (You may need help combining customized instances with the new .xml but I didn't experience that so I don't know the procedure). 

Also, my device mappings were about a year old so I had to update that file as well. (https://sort.veritas.com/checklist/install/nbu_device_mapping)

After replacing the cloudprovider.xml and upgrading the device mappings I was able to see the amazonaws.com.cn instance and, (having already opened the firewall), connected the very first attempt. 

Firewall: source to s3-cn-north-1.amazonaws.com.cn

  • Bidirectional TCP: (5637, 80, 443)
2008) Not Applicable
7.7
Amazon
China
Cloud
NBU
NetBackup
S3
Windows 2008

4 Replies

Sort By
  • Fleming_kris's avatar
    Fleming_kris
    Level 2
    8 years ago

    Hello,

    Thanks for the post. We have a similar issue where in our case the S3 is in IRELAND and has a different "s3-eu-west-1.amazonaws.com" URL than the standard URL s3.amazonaws.com. I am unable edit\add CloudProvider.xml or cldinstance in csconfig.

    Is getting new CloudProvider.xml is the only way to configure new storage server in S3 in Non-US region?

    • SplashMasterson's avatar
      SplashMasterson
      Level 4
      8 years ago

      Fleming_kris,

      I can't confirm every step to set this up because we don't have buckets in that region, but eu-west-1 is one of the default regions in 7.7.3

      If you run (...\bin\admincmd)

      csconfig cldinstance -i -pt amazon -at S3

      you should see all of the default regions for the "Amazon" instance, which includes Ireland (eu-west-1):

       

      C:\Program Files\Veritas\NetBackup\bin\admincmd>csconfig cldinstance -i -pt amazon -at S3

        Cloud Instance Name                     : amazon.com
        Provider Type                           : amazon
        Service API Type                        : S3
        Service Host                            : s3.amazonaws.com
        Service Endpoint                        : <empty>
        Service HTTP Port                       : 80
        Service HTTPS Port                      : 443
        Service URL Style                       : Virtual Hosted Style
        Customizable                            : No
        Region  Name                            : US Standard
                Location Constraint             : <empty>
                Service Host                    : s3.amazonaws.com
        Region  Name                            : Asia Pacific (Tokyo)
                Location Constraint             : ap-northeast-1
                Service Host                    : s3-ap-northeast-1.amazonaws.com
        Region  Name                            : Asia Pacific (Singapore)
                Location Constraint             : ap-southeast-1
                Service Host                    : s3-ap-southeast-1.amazonaws.com
        Region  Name                            : Asia Pacific (Sydney)
                Location Constraint             : ap-southeast-2
                Service Host                    : s3-ap-southeast-2.amazonaws.com
        Region  Name                            : EU (Frankfurt)
                Location Constraint             : eu-central-1
                Service Host                    : s3-eu-central-1.amazonaws.com
        Region  Name                            : EU (Ireland)
                Location Constraint             : eu-west-1
                Service Host                    : s3-eu-west-1.amazonaws.com
        Region  Name                            : South America (Sao Paulo)
                Location Constraint             : sa-east-1
                Service Host                    : s3-sa-east-1.amazonaws.com
        Region  Name                            : US West (Northern California)
                Location Constraint             : us-west-1
                Service Host                    : s3-us-west-1.amazonaws.com
        Region  Name                            : US West (Oregon)
                Location Constraint             : us-west-2
                Service Host                    : s3-us-west-2.amazonaws.com
        Storage Server                          : amazon.com
                Use SSL                         : DATA
                Use Proxy                       : NONE
                Proxy IP                        : <NA>
                Proxy Port                      : <NA>
                Credentials Broker              : CREDS_PROMPT



Successfully fetched Cloud Instance(s)

       

      csconfig  (NetBackup Command Reference Guide 7.7)

      REF: Cloud Admin Guide 7.7.3

       

      My understanding is you:

      1. Configure your Storage Server with your ID Key to service host s3.amazonaws.com
      2. Through either the Cloud Wizard or "Disk Pools > `right click` New Disk Pool"
        1. Select your Cloud Store
        2. Select "Add New Volume" 
          1. Add bucket and region through the drop down (see attachment)

       

       

      As I said, I can't confirm this, but if you're at 7.7.3 and still can't add the regional bucket, then you may need the CloudProvider.xml

      • Fleming_kris's avatar
        Fleming_kris
        Level 2
        8 years ago

        Thanks for the reply. We are at 7.7.2 with RHEL 7.2

         

        Though csconfig cldinstance shows the EU-WEST-1 as one of the default regions, while creating the storage server we get only "s3.amazonaws.com" and "add Cloud storage" button below service host is greyed out.

        We are able to telnet "s3-eu-west-1.amazonaws.com" at 443, 80 without an issue from eu-west instances and where as telent to s3.amazonaws.com fails. so, i was looking for a way to make netbackup to establish a connection manually to AWS at "s3-eu-west-1.amazonaws.com" instead of its default "s3.amazonaws.com"

        ** Telnet to s3.amazonaws.com is successful only from instances in Americas.