cancel
Showing results for 
Search instead for 
Did you mean: 

Data Domain - reading data from FC and IP written storage units

Steve_Law
Level 4

Hi,

We use Netbackup 7.5 with Data Domain storage and we have SLPs that write weekly and monthly production backups to tape. 

Our lan is only 1gb (it's virtually an antique). Thanks to the DDBoost plugin our backup data is deduped and so tha LAN can cope. But since we finished our migration of the production servers to the new Netbackup infrastructure we are struggling to keep up with writes to tape. The backup images on Data Domain have to be reinflated to full size and then cross the LAN to the media server, then across the fibre (8gb) to the tape library. 

Some months ago we installed a fibre HBA into Data Domain and tested DDBoost over FC. We successfully set it up on the same server we now use to write the production backup images to tape. I am now investigating if I can use this fibre connection to read from Data Domain to the media server. Then the whole data transfer will be on fibre and not 1gb LAN. 

But there is a problem. When you configure DDBoost over FC it creates a new disk pool.  In our case the local Data Domain disk pool was called MB1_BAS, but the new FC disk pool was MB1_BAS_FC. Our fibre connection can only read data that was written by the fibre connection. We cannot see the backup images on MB1_BAS, we can only see the storage units within MB1_BAS_FC. 

We can't switch all our servers over to FC as there are many large servers that do not have fibre HBAs. The backup images we want to read are stored on the same Data Domain box in the same format. Surely there is a way to read the backup images through the fibre connection? 

Any suggestions? 

Thanks 

 

1 ACCEPTED SOLUTION

Accepted Solutions

RiaanBadenhorst
Moderator
Moderator
Partner    VIP    Accredited Certified

Hi

 

It should make no difference in the size, it goes into the same FS in the DD, you'll get dedupe.

 

About push and pull, let me know if the attached drawings explain it well.

 

If not, I can write it out :)

View solution in original post

13 REPLIES 13

Nicolai
Moderator
Moderator
Partner    VIP   

https://www.emc.com/collateral/software/white-papers/h7296-data-domain-boost-openstorage-wp.pdf

Page 31.

It does look like its not possible to share the same Mtree across IP and FC connected media servers. 

I would suggest you contact EMC support regarding the matter.

Steve_Law
Level 4

Thanks for your response Nikolai. However I can't see where in the section on Data Domain Boost Over Fibre Channel it says that you can't share mtrees between IP and FC connections. But it doesn't say you can either!

I have already raised this with EMC, they said they weren't sure, and "Accessing the same copy of data via LAN or FC is possible with NetWorker, but I don't know if it's the same for NetBackup, because you have different Storage Units." 

Seems crazy to me that the data is on Data Domain but you can only use FC to read data written by FC, and the same for IP.  :o

EMC suggested using internal Data Domain commands snapshot and fastcopy to copy data from Mtree1 to Mtree2. This would work but adds an extra level of complexity to our setup, and I don't understand how Netbackup would be made aware that the backup images were now on a different mtree and how this would work with the SLPs.....

 

 

RiaanBadenhorst
Moderator
Moderator
Partner    VIP    Accredited Certified

You can have the same mtree for both, but you cannot have both FC and IP configured on a single NetBackup media server. You have to pick one, and you have to make a separate pool for each. On the DD though it will write to the same mtree/ddboost device.

Steve_Law
Level 4

Hi Riann,

I do have both FC and IP configured on one server. Our data Domain box BASDD002 has two disk pools: MBFS_BAS for IP and MBFS_BAS_FC for FC. I can backup media server dwb004 using either, as i did earlier today: 

02/09/2015 10:38:41 - Info nbjm(pid=5860) starting backup job (jobid=1682894) for client dwb004.ourcorp.net, policy MBFS_FS_DWB004_DEV_TAPETEST, schedule datadomain  
02/09/2015 10:38:42 - estimated 0 Kbytes needed
02/09/2015 10:38:42 - Info nbjm(pid=5860) started backup (backupid=dwb004.ourcorp.net_1441186721) job for client dwb004.ourcorp.net, policy MBFS_FS_DWB004_DEV_TAPETEST, schedule datadomain on storage unit BASDD002_MBFS_DWB004
02/09/2015 10:38:43 - started process bpbrm (3292)
02/09/2015 10:38:44 - connecting
02/09/2015 10:38:44 - connected; connect time: 00:00:00
02/09/2015 10:38:48 - begin writing
02/09/2015 10:39:10 - Info bpbrm(pid=3292) dwb004.ourcorp.net is the host to backup data from     
02/09/2015 10:39:10 - Info bpbrm(pid=3292) reading file list from client        
02/09/2015 10:39:11 - Info bpbrm(pid=3292) starting bpbkar32 on client         
02/09/2015 10:39:11 - Info bpbkar32(pid=2888) Backup started           
02/09/2015 10:39:11 - Info bptm(pid=9112) start            
02/09/2015 10:39:12 - Info bptm(pid=9112) using 1048576 data buffer size        
02/09/2015 10:39:12 - Info bptm(pid=9112) using 1024 data buffers         
02/09/2015 10:39:14 - Info bptm(pid=9112) start backup           
02/09/2015 10:39:15 - Info bpbkar32(pid=2888) change journal NOT enabled for <F:\Backup>       
02/09/2015 10:48:51 - end writing; write time: 00:10:03
02/09/2015 10:49:09 - Info bpbkar32(pid=2888) bpbkar waited 2859 times for empty buffer, delayed 3357 times.   
02/09/2015 10:49:14 - Info bptm(pid=9112) waited for full buffer 270 times, delayed 1055 times    
02/09/2015 10:49:14 - Info bptm(pid=9112) EXITING with status 0 <----------        
02/09/2015 10:49:18 - Info bpbrm(pid=3292) validating image for client dwb004.ourcorp.net        
02/09/2015 10:49:18 - Info bpbkar32(pid=2888) done. status: 0: the requested operation was successfully completed    
the requested operation was successfully completed(0)

 

02/09/2015 09:38:41 - Info nbjm(pid=5860) starting backup job (jobid=1682886) for client dwb004.ourcorp.net, policy MBFS_FS_DWB004_DEV_FC, schedule FC_TEST  
02/09/2015 09:38:41 - estimated 0 Kbytes needed
02/09/2015 09:38:41 - Info nbjm(pid=5860) started backup (backupid=dwb004.ourcorp.net_1441183121) job for client dwb004.ourcorp.net, policy MBFS_FS_DWB004_DEV_FC, schedule FC_TEST on storage unit Test_MBFS_BAS_FC
02/09/2015 09:38:42 - started process bpbrm (2620)
02/09/2015 09:38:43 - connecting
02/09/2015 09:38:44 - connected; connect time: 00:00:01
02/09/2015 09:38:50 - begin writing
02/09/2015 09:39:10 - Info bpbrm(pid=2620) dwb004.ourcorp.net is the host to backup data from     
02/09/2015 09:39:10 - Info bpbrm(pid=2620) reading file list from client        
02/09/2015 09:39:10 - Info bpbrm(pid=2620) starting bpbkar32 on client         
02/09/2015 09:39:11 - Info bpbkar32(pid=7872) Backup started           
02/09/2015 09:39:11 - Info bptm(pid=6472) start            
02/09/2015 09:39:11 - Info bptm(pid=6472) using 1048576 data buffer size        
02/09/2015 09:39:11 - Info bptm(pid=6472) using 1024 data buffers         
02/09/2015 09:39:14 - Info bpbkar32(pid=7872) change journal NOT enabled for <F:\Backup>       
02/09/2015 09:39:15 - Info bptm(pid=6472) start backup           
02/09/2015 09:50:16 - end writing; write time: 00:11:26
02/09/2015 09:50:32 - Info bpbkar32(pid=7872) bpbkar waited 24308 times for empty buffer, delayed 28334 times.   
02/09/2015 09:50:38 - Info bptm(pid=6472) waited for full buffer 0 times, delayed 0 times    
02/09/2015 09:50:38 - Info bptm(pid=6472) EXITING with status 0 <----------        
02/09/2015 09:50:41 - Info bpbrm(pid=2620) validating image for client dwb004.ourcorp.net        
02/09/2015 09:50:42 - Info bpbkar32(pid=7872) done. status: 0: the requested operation was successfully completed    
the requested operation was successfully completed(0)

 

What I'd like to do is read the IP disk pool MBFS_BAS (and write to tape) using the FC connection. 

I'm not sure what you mean by "On the DD though it will write to the same mtree/ddboost device." That's what I can't do - read or write to the IP disk pool using FC, or vice versa. 

Nicolai
Moderator
Moderator
Partner    VIP   

Hi Steve

You are right, it does not say you can't nor does it say you can't, that was more or less what I was trying to indicate by say "does not look like".

I did some checking in-house, and I was able to access a IP based storage unit thru FC

Once you have configured the FC connection to the data doman, go to (using the GUI) >  Media Device Mangement -> Devices -> Disk Pool -> Right Click -> New Disk pool -> Storage Server Type "Data Domain" -> DFC-??? -> "Select storage server Volume to the disk pool".  Select the IP base STU and follow rest of the wizard.

You may run into to issues when configuring the storage units, but I do not have the time for looking at that issue right now.

 

RiaanBadenhorst
Moderator
Moderator
Partner    VIP    Accredited Certified

Hi,

 

Let me clarify what I mean. You can only have 1 connection, either IP or FC, to the same diskpool /storage server in NBU (from what I've seen in my configs).

 

EDIT {removed incorrect statements)

RiaanBadenhorst
Moderator
Moderator
Partner    VIP    Accredited Certified

Hi Steve

 

I've since ran into this myself and can now tell you the following. I have however not seen both IP and FC on the same server, but that's beside the point.

Scenario

You have one DD (storage server DD01)

You have one boost device on the DD.

You have two media servers, one connected via FC (MS01), and one via Ethernet (MS02).

NBU will create two storage serves in NBU, DD01, and DFC-DD01.

NBU will create two diskpools DD01_DISKPOOL & DFC-DD01_DISKPOOL / diskvolumes @aaaaB &@aaaaC (those are examples only)

So for all intents and purposes those are two separate diskpools/volumes (from NBU's point of view they might as well be located in two separate DDs really).

That also means that @aaaaB is only accessible via the Ethernet (IP) connection, and @aaaaC is only accessible via the FC connection.

Now this is where it gets really spicy. In my environment I had another DD (DD02) in site 2. So if @aaaaC is only accessible via the FC connection, how do we get the images replicated to DD02.

I figured out that you can only use a push configuration whereby any FC attached (source) media server in site 1 would need to be configured on an Ethernet / IP Storage Server/Diskpool in site 2. Plus it should have a STU configured for the site 2 DD. The STU requirement on both source and target is only applicable in the push configuration.

All quite tricky.

In your scenario I still don't see how you could achieve your goal.

 

 

Steve_Law
Level 4

Hi Riaan, thanks for your efforts!

Sorry but what is a push configuration? Is that an automated process using AIR on Data Domain?

 

I did some testing on this on Monday and created a sort of clumsy workaround, an SLP that does this:

1) Writes the initial backup to the usual IP pool storage unit  

2) duplicates the image to a storage unit in the FC pool on the same Data Domain box and sets it to expire on copy.

3) Writes the image from the FC pool duplicate to tape, automatically invoking fibre channel transport (because it is the FC pool).

This works. It misses our usual step of duplicating the image to our remote DD box across the WAN, but that shouldn't be an issue. Only problem with this method is that it will temporarily double our storage usage for each new backup until the Data Domain clean up command runs (which is Tues and Fridays) and removes the expired copies on the FC pool. It's not clear how big the duplicated image on the FC pool will be - presumably it also has the benefit of the Data Domain deduplication and won't be full size...? I can check this by generating a report on Data Domain.

 

 

 

 

 

 

 



 

RiaanBadenhorst
Moderator
Moderator
Partner    VIP    Accredited Certified

Hi

 

It should make no difference in the size, it goes into the same FS in the DD, you'll get dedupe.

 

About push and pull, let me know if the attached drawings explain it well.

 

If not, I can write it out :)

mnolan
Level 6
Employee Accredited Certified

Why not do a bpexpdate on the diskpool and expire everything ensuring you use the no delete option and then reconfigure the storage server and disk pool to use the fc connection and then import the images?

Steve_Law
Level 4

Hi Riaan, 

Sorry to take so long to respond, so stupidly busy here. I looked at your drawing but didn't understand it, and as you say it isn't the solution to my problem. I need something that is integrated within Netbackup.

I recently configured a new media server to act as a dedicated tape-writer through a FC connection to the local Data Domain. As per previous setup when you configure it for fibre it only has the option to address the dedicated fibre channel pool. Writes to tape from the FC media pool work great - really fast. 

As a workaround I decided to add a new step in an SLP to duplicate the backup image from the IP disk pool to the FC disk pool and then write to tape. For a backup of a server on our other site the process would be:

Step 1:  write to Data Domain MK1

Step 2:   duplicate to Data Domain Bas1 on second site

Step 3:  duplicate from Bas1 one IP pool to Bas1 FC pool

Step 4:  write to tape from Bas1 FC pool utilising fibre for transfer to media and server and from media server to tape drives. 


This works, but the duplications within the Bas1 Data Domain from the IP pool to the FC pool are by far the slowest step! Timings for a 150gb backup:

Step 1: write to local Data Domain = 37mins (deduped backup over lan)

Step 2: duplicate to secondary site Data Domain over WAN = 17mins (in logs says this is an "optimised duplication")

Step 3: duplicate from IP disk pool to FC disk pool within Data Domain = 92 mins

Step 4: write to tape over FC = 32 mins

 

All I can take from this is that transferring from IP to FC pool within the same Data Domain is NOT an optimised or deduped transfer, it is a full-reinflation of the data and copy, as if to a USB drive or similar. Even then I wouldn't expect writing 150gb to a USB drive to take 92 mins!

So this doesn't seem to be a solution. There seems to be no integration at all between FC and IP traffic on Data Domain. It's as if they are two different bits of hardware that have been welded together, with a serial cable connecting them... 

 

 

Steve_Law
Level 4

Thanks for your efforts though!

RiaanBadenhorst
Moderator
Moderator
Partner    VIP    Accredited Certified

No worries, its a bit tricky with the way NBU addresses these pools when they're IP/FC. Dont know why, and it wont really matter, we have to work with it.