cancel
Showing results for 
Search instead for 
Did you mean: 

MSDP to MSDP duplication seeding

Ray_Downer1
Level 3

We are running NetBackup 7.6.1.1. We have several remote sites. Each remote site has a local media server and MSDP. We have one central site with another MSDP. We use VMware policies with SLPs to backup data in the remote sites to respective media servers before duplicating the data to the central MSDP. Some of our servers have large amounts of data and our SLP windows and WAN links are not adequate to complete the initial/full duplication jobs.

While I have seen lots of discussions about seeding, most of them do not seem to apply or are quite complicated. I saw one discussion about using a Basic disk to seed the central MSDP but the details were lacking. What simple options can I use to seed our central MSDP?

Something similar to this??

1. Connect a USB drive and create a storage unit of type BasicDisk
2. Create a policy to backup data to the new storage unit
3. Execute policy
4. Detach USB drive and ship it to central site
5. Attach USB drive

Then what? Do I need to expire images? Do I need to import images?

While I am not new to NetBackup, I have never had to seed MSDP so any help is appreciated.

3 ACCEPTED SOLUTIONS

Accepted Solutions

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

The idea behind BasicDisk using USB drive is just to have an easy way of transporting the backup to central site to duplicate to MSDP. This will serve as seeding from remote site and subsequent duplications from remote site will be optimised.

About explanation of Riaan's post:

  • Move the disk to other site
    - Here the disk must be disconnected at remote site and not yet connected at central site.
  • Create an empty folder matching the remote storage unit and expire the backups
    - The folder must exist on media server at remote site for expiration to succeed (see http://www.symantec.com/docs/TECH124525 ). Expire the backup associated with remote media server because it will never be needed at remote site. Disk must be disconnected to prevent backups from being deleted from disk.
  • Import the backups by attaching to central media server
    - Only connect USB drive to central media server now and import.

Next step:

  • Duplicate the imported backup to MSDP
  • Then run your dedupe job either directly or as a duplication to the central MSDP.

 

View solution in original post

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified
You now go to the next step in Riaan's post : Duplicate the backup to MSDP.

View solution in original post

15 REPLIES 15

Andrew_Madsen
Level 6
Partner

Is this a new environment? If not then I would assume that your central site would be adequately from the get go  with data from its own backups. I have heard this discussion many times and the thought process hinges on the thinking at AIR or even MSDP to MSDP duplication is the same as deduplicating client to its MSDP. It is not. MSDP to MSDP through AIR or duplication is performed with a hash table that includes all the blocks of data in the pool. A deduplicating client deduplicates against is own image or group of images. 

Only if your central MSDP is empty would you worry about seeding. If that is the case then we need a bit more information:

Do the remote sites have their own master servers thus making them their own domain?  

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Ray_Downer1
Level 3

Andrew, we recently replaced all of our backup servers and upgraded to 7.6.1.1 during the refresh. Our central MSDP has several terabytes of data in it, but it does not have images for 85 of our virtual machines in our central MSDP.

I have seen duplication jobs for larger virtual machines run for the same image for weeks. The duplication jobs always fail when our SLP window closes. One of our SLP windows spans 62 hours but the duplication jobs still do not complete in this time period.

We have a single master server with several media servers.

Ray_Downer1
Level 3

Marianne, Riaan's post is the one I referenced. I do not understand the step "Create an empty folder matching the remote storage unit and expire the backups". Can you explain this better?

Andrew_Madsen
Level 6
Partner

Riaan's post is for a deduplication client to MSDP backup which does depend on the image being in place. Ray, your question (if I understand it correctly) is for MSDP to MSDP duplication which is not the same thing and should not be facilitated by this process. Are your remote media servers using deduplication pools?

Andrew_Madsen
Level 6
Partner

Ray,

In the steps outlined by Riaan the expire is after the USB has been detached from the remote unit. The reasoning is that you do not want the images in the catalog or your import will fail. However I do not see where you stop the image cleanup from failing with a 1 from there on out unless you ship the USB back and reattach it to let the image cleanup happen normally. 

Ray_Downer1
Level 3

After reading Riaan's post again, I agree that is does not apply to our situation. Sorry for the confusion.

Our remote media servers use deduplication pools. Backups from the remote VMs to the remote media servers/deduplication pools work just fine. Our issue is the duplication of images from remote deduplication pools to our central deduplication pool. Most of our VMs have duplicated images in the central deduplication pool but 85 or so do not. I have looked through Activity Monitor and can see where duplication jobs ran for the same image over and over. The duplication jobs never complete successfully. I assume they don't complete because the complete image is not duplicated before the SLP window closes.

I only have so much bandwidth and so many hours of the day to duplication images. I am looking for suggestions/solutions. Thanks!

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

The idea behind BasicDisk using USB drive is just to have an easy way of transporting the backup to central site to duplicate to MSDP. This will serve as seeding from remote site and subsequent duplications from remote site will be optimised.

About explanation of Riaan's post:

  • Move the disk to other site
    - Here the disk must be disconnected at remote site and not yet connected at central site.
  • Create an empty folder matching the remote storage unit and expire the backups
    - The folder must exist on media server at remote site for expiration to succeed (see http://www.symantec.com/docs/TECH124525 ). Expire the backup associated with remote media server because it will never be needed at remote site. Disk must be disconnected to prevent backups from being deleted from disk.
  • Import the backups by attaching to central media server
    - Only connect USB drive to central media server now and import.

Next step:

  • Duplicate the imported backup to MSDP
  • Then run your dedupe job either directly or as a duplication to the central MSDP.

 

Ray_Downer1
Level 3

We finally got a USB drive attached to the remote server and performed backups of several servers. We shipped the drive to our central site. The images were expired at the remote site. The images were imported at the central site. From what I can tell, the images are still on the USB drive and are accessible via the basic disk unit I created, but the images are not in our MSDP. How do I get the backup images into our central MSDP?

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified
You now go to the next step in Riaan's post : Duplicate the backup to MSDP.

Ray_Downer1
Level 3

Thanks Marianne. I saw the duplicate option earlier, but the light bulb didn't go off.

I was able to import and duplicate several images. Some of the duplication jobs had pretty low deduplication rates while others had much higher rates than expected. However, when I activated images that had been suspended, the duplication jobs ran (over the WAN) for over 30 hours and failed to complete. I didn't expect the duplication jobs to finish immediate, but I didn't expect them to run for 30+ hours and not complete either.

Activity Monitor does not show the dedupe saving until a job completes successfully. Is there a way to tell if duplication jobs are taking advantage of the seeded images?

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified
Have you enabled Client side dedupe? The default is Media server dedupe. Host Properties -> Master server > Client Attributes. Add/select client name, select 'Prefer Client ' as dedupe option. Please also confirm that Client name is exactly the same in all policies - the original Basic Disk policy and the WAN policy.

Ray_Downer1
Level 3

All of the backups are for virtual machines so client side dedupe is not an option without installing agents.

We have a mix of client names. Some of them match exactly while others do not. For example, most backups have the VM display name which is the NetBIOS name in uppercase. Most of the seeding backups have the fully qualified names in lowercase. That didn't seem to matter during the duplicate phase though. As noted early, NBU reportedly "sent" far less data to the MSDP than was initially backed up (9.26 TB backed up vs 4.40 TB CR sent).

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified
Apologies - this has been going on for so long that I forgot this is VMware backups. Is CBT enabled on VM side? Accelerator selected in the policy?

Ray_Downer1
Level 3

Yes, CBT is enabled. Accelerator is enabled for the normal day to day backups. I did not use Accelerator for the seeding backups.

I suspended the largest of our images again to see if the smaller images would duplicate overnight. Two of the images did duplicate and show deduplication rates above 95%.

Another image started duplicating but appears to have had an issue. NBU shows the first 50GB chunk started processing at 8:47 PM and then nothing else was logged until the SLP window closed at 6 AM.

It looks like seeding worked. Thank you Marianne for your help!