VTL/LTO3 drive w/ NDMP not working, Stuck at mount

[dukbtr]

Environment:

• NetBackup 6.5.2a Master/Media
• Server: HP-UX 11.23 IA64
• EMC DL3D 3000 VTL
• EMC Celerra NAS

We are trying to backup up our NAS device with direct attached drives, be it Virtual.  We have one fibre channel coming from the NAS and 2 from the VTL.  We have zoned it so the the NAS can see both channels from the VTL.

I set up the nas host under credentials->NDMP Hosts and that worked fine.  I then created 1 of the 4 dricves that will be allocated to the NDMP backups.  I set up the tape drive with the appropriate drive path and it takes, then I created the storage unit for the drive.  All is well, I think, up till this point.

When I start a policy to backup:

1. The tape mounts according to activity monitor
2. The tape shows mounted in the EMC VTL web interface
3. The tape sdrive shows active in Device Monitor but it never shows the volume number of the tape
4. The job just sits at mounting and never starts writing

When I kill the job it never releases the drive and I have to manually eject the tape and drive.

We have opened up a case with Symantec, but the Admin for the EMC Celerra NAS doesn't know how to get logs from it that are requested from Symantec.  He doesn't know Linux:(

Any help is appreciated !!!

Thanks

[Andy_Welburn]

David - I may be wasting your time as we don't use any EMC kit, don't utilise VTL and have a Solaris 9 server running 6.5.1!

Just thought it may be worthwhile asking a few (one?) questions & making a couple of points/observations?

You say that it never starts writing - how long do you leave it?

There are several phases that the NDMP backup goes through:

Phase I and II: Build the map of files and directories and collect file history and attribute information

Phase III: Dump data to tape, specifically directory entries
Phase IV: Dump files
Phase V: Dump ACLs

Now depending on the make-up of the volume being backed up then each period could take some time.

e.g. from one of our NetApp NDMP save logs:

dmp Tue Feb 10 15:45:45 GMT /vol/data1/CS(0) Start (Level 0, NDMP)
dmp Tue Feb 10 15:45:45 GMT /vol/data1/CS(0) Options (b=63, u)
dmp Tue Feb 10 15:45:45 GMT /vol/data1/CS(0) Snapshot (weekly.0, Mon Feb  9 00:02:29 GMT)
dmp Tue Feb 10 15:46:13 GMT /vol/data1/CS(0) Tape_open (ndmp)
dmp Tue Feb 10 15:46:13 GMT /vol/data1/CS(0) Phase_change (I)
dmp Tue Feb 10 19:18:07 GMT /vol/data1/CS(0) Phase_change (II)
dmp Tue Feb 10 19:36:47 GMT /vol/data1/CS(0) Phase_change (III)
dmp Tue Feb 10 23:23:14 GMT /vol/data1/CS(0) Phase_change (IV)
dmp Wed Feb 11 19:14:48 GMT /vol/data1/CS(0) Phase_change (V)
dmp Wed Feb 11 20:44:07 GMT /vol/data1/CS(0) Tape_close (ndmp)
dmp Wed Feb 11 20:44:08 GMT /vol/data1/CS(0) End (1871102 MB)

So as you can see there are quite a few hours between the job starting & data actually being written to tape.

As far as the location of the logs - in our instance we have NFS mounted our NetApp "root" or vol0 areas on our master & for NetApp the above extract was taken from etc/log/backup

Sorry if I have wasted your time or telling you something you already know or even telling you something you didn't need to know, but if it helps then ......

Good luck!

[Darren_Dunham]

The "phase" stuff isn't part of NDMP, it's part of Netapp backups.  Any NDMP host is free to chose the backup method, and I don't think EMC is similar to netapp here.

But the suggestion to check logs on the NDMP host is good.  There may even be some method of making them more verbose to get some feedback about what's going on.

I doubt the actual NDMP backup has started.  Netbackup won't attempt it until the the tape is mounted and positioned.  Prior to that it's just doing NDMP SCSI pass-through to talk to the tape.  With verbose logging on the NDMP host, you should be able to see some of that communication.

David, in the activity monitor, does it say anything after the mount?  Nothing about positioning?

-- 

Darren

[dukbtr]

No, no positioning

I think I know what the issu might be, just not how to fix it.

When the drives were created the drive numbers are 1-15, but the drives in the VTL are like any other device and the drives start at 0... on up.  I noticed that when the tapes  mount the are alsways one drive off from where they should be.  

When i try to configure the drives though with TLD library I don't get to have a starting drive number  of 0. 

[J_H_Is_gone]

I think you have it David.

robots start counting the drives from drive 1, where you would have rmt0.

so when setting up tape drives in netbackup you really have to make sure that you are matching.

meaning the serial number of rmt0 matches the serial number of robot drive 1 ( it could be 2 or ever 5).

so look at that, it could be you are just off by 1 as you say.  try to match the serial numbers of the drives to see if that is the case.

[Dan_Ryan_2]

If it is a EMC Celerra, I think you will want the following output from the datamover:

#server_log server_2

#server_param server_2 -facility PAX -list

#server_param server_2 -facility NDMP -list

#server_devconfig server_2 -list -scsi -nondisks

I just did this for our environment, but this is for a NS702.  You server_2 may need to change depending upon which datamover you're connected to.  

The top command is probably the most important one, along with the last command.  As for verbose logging, that's one thing the NAS is pretty bad at.  However, when did have problems setting any of them up in the past, we have opened a case with EMC. 

[rsbst19]

This was basically the same issue I was having (at the same time you were, coincidentally).  New EMC VTL.  The solution, as you described, was indeed having to use TAPE0 in NBU Even though it is "drive1".  I was starting with TAPE1 to help me make more sense of the drive ordering, but it seems that perhaps the VTL does not like that.  I know that the EMC 3DL does not like you using the luns/drive out of sequence, but I didn't think it would be an issue starting with 1 instead of 0.