Forum Discussion

rVikas's avatar
rVikas
Level 5
8 years ago

Netbackup integration with S3 ...

Hi All,

I am evaluating a scenario with netbackup 8.0

i have integrated netbackup with AWS S3 as backup target

in S3, i have applied lifecycle rule where data will move to glacier after 1 day (netbackup will write data in S3, after 1 day, AWS will automatically transition data to glacier storage class)

Now while restoring from netbackup, i am unable to get that data since data is stored in glacier and netbackup cannot talk to galcier directly.

just want to verify if anyone have tested such scenario.

lifecycle
Restore
S3

3 Replies

Sort By
  • GeForce123's avatar
    GeForce123
    Level 5
    8 years ago

    I'm fairly certain you would have to recall the file from Glacier to S3, which can take some time depending on the size of the file and from there you would use NetBackup to restore the file. We do something similar with our archives.

    Also, Glacier isn't meant for storage that will be retrieved often so hopefully you have a local copy on disk and a copy that goes to S3->Glacier.

    • rVikas's avatar
      rVikas
      Level 5
      8 years ago

      From AWS documentation -

      • AWS offers object lifecycle management to transfer object to different storage classes for cost optimization.
      • Archived objects in Glacier cannot be accessed through Glacier Console or Glacier API.Since Amazon S3 stores archive objects to Glacier, access to those objects will available only from S3 API or S3 Console.
      • While retrieving archived objects, a temporary copy of objects are created and objects are restored from that temporary copy whereas actual object reside in glacier.
      • Typical time required to make temporary copy is around 5 hrs.

      My understanding was:

      • Backup application directly communicate with S3 for any backup and restore operation
      • Since archived object metadata is reside in S3, backup application can view the backed-up contents.
      • When backup application initiate restore, S3 brings object from Glacier and makes it available for restore.

      when i am restoring from Netbackup - getting this error - 

      12 Apr, 2017 8:38:38 PM - Info bpbrm (pid=32) starting bptm
      12 Apr, 2017 8:38:41 PM - Info tar32 (pid=1848) Restore started
      12 Apr, 2017 8:38:41 PM - connected; connect time: 0:00:00
      12 Apr, 2017 8:38:41 PM - Info bptm (pid=6012) start
      12 Apr, 2017 8:38:41 PM - started process bptm (pid=6012)
      12 Apr, 2017 8:38:41 PM - Info bptm (pid=6012) reading backup image
      12 Apr, 2017 8:38:42 PM - Info bptm (pid=6012) using 30 data buffers
      12 Apr, 2017 8:38:57 PM - Critical bptm (pid=6012) Storage Server Error: Error while reading data from cloud storage ( V-455-2)
      12 Apr, 2017 8:38:57 PM - Critical bptm (pid=6012) image open failed: error 2060037: access not allowed
      12 Apr, 2017 8:38:57 PM - Info bptm (pid=6012) EXITING with status 83 <----------
      12 Apr, 2017 8:39:58 PM - Info tar32 (pid=1848) done. status: 83: media open error
      12 Apr, 2017 8:39:58 PM - Error bpbrm (pid=32) client restore EXIT STATUS 83: media open error
      12 Apr, 2017 8:39:58 PM - restored from image qawin045_1491900746; restore time: 0:01:22
      12 Apr, 2017 8:39:58 PM - Warning bprd (pid=4596) Restore must be resumed prior to first image expiration on 25-04-2017 14:22:26
      12 Apr, 2017 8:39:58 PM - end Restore; elapsed time 0:01:24
      media open error (83).

       

  • Tape_Archived's avatar
    Tape_Archived
    Moderator
    8 years ago

    AWS Glacier is just like Tape. There is no direct integration or access with NetBackup for Glacier, which might come soon as other vendor have already come up with that support (not sure how exactly it works though).

    You may use "S3 - Infrequent Access" reduce your cost instead of "S3 - Standard" and use Glacier for long Term retention (you may to manage that on your own at this time, once Netbackup supports in future you may be able to duplicate directly to Glacier).