Backup, Archive and Restore crashes when restoring VMWare backup to alternate location

[danielrickard]

We recently upgraded from 7.5.0.5 to 7.7.1, still running on Windows Server 2008 R2, but we have a problem with restoring VMWare backups to alternate locations.

Restoring to the original location works as it always has, but when restoring to an alternate location the application crashes and puts the below in the Application log.
At the time it crashes the system is "Retrieving details of data stores..."

I have tried this from various admin accounts and our media servers, as well as the Master server, but they all have the same problem.
UAC is disabled on the media servers. 
UAC is enabled on the Master, but I am doing a 'run as administrator' and all other types of restore work fine.

Source "Application Error"; Event ID "1000"

Faulting application name: nbwin.exe, version: 7.701.15.910, time stamp: 0x55f1f618
Faulting module name: nbwin.exe, version: 7.701.15.910, time stamp: 0x55f1f618
Exception code: 0xc0000409
Fault offset: 0x00000000000f5e48
Faulting process id: 0x23ec
Faulting application start time: 0x01d107127ba99e17
Faulting application path: C:\Program Files\Veritas\NetBackup\bin\nbwin.exe
Faulting module path: C:\Program Files\Veritas\NetBackup

[Marianne · Accepted solution]

Have you perhaps tried the Java Console?

[danielrickard]

By the Java console, you mean the Backup, Archive, and Restore node that's in the Java based "Netbackup 7.7.1 Administration Console"?

I haven't, but I shall try it now...

Thank you

[Marianne]

Correct.

I was not aware that the Windows BAR (nbwin.exe) was still working on NBU 7.7.1.
I thought this was removed along with the Windows Admin Console (nbconsole.exe).

I have personally not yet seen 7.7.x - none of our customers have upgraded yet.

[danielrickard]

That's the ticket!

Thank you so much, Marianne.

[danielrickard]

So, the job has started and it's created the VM but it doesn't look like it's doing anything so far. 

It's quite a large server: two VMDks totalling 1.8TB. Would you expect to see the job 'hang' here while it reads the backup file?
Looking on the media server, I can't see the backup image file being accessed...

Thanks

The detailed stats in activity monitor says:

10/15/2015 08:59:39 - begin Restore
10/15/2015 08:59:45 - Info bprd (pid=3384) Found (2,376,233) files in (1) images for Restore Job ID 832486.xxx
10/15/2015 08:59:47 - restoring from image VMName_1438886343
10/15/2015 08:59:48 - Info bprd (pid=3384) Searched (        2) files of (2,376,233) files for Restore Job ID 832486.xxx
10/15/2015 08:59:48 - Info bprd (pid=3384) Restoring from copy 3 of image created 08/06/15 19:39:03 from policy FileServer
10/15/2015 08:59:55 - Info bprd.sfr (pid=3384) Restoring to vCenter server VCserver.mydomain.com, ESX host esxhost.mydomain.com, Datacenter /DC1, Folder /DC1/vm/, Display Name VMName_Restore, Resource Pool/vApp /DC1/host/, Datastore/Datastore Cluster DC1_BigMigrationDatastore
10/15/2015 09:00:03 - Info bpdm (pid=1504) started
10/15/2015 09:00:03 - started process bpdm (pid=1504)
10/15/2015 09:00:04 - Info bpdm (pid=1504) reading backup image
10/15/2015 09:00:05 - Info bpdm (pid=1504) using 30 data buffers
10/15/2015 09:00:05 - Info bpdm (pid=1504) spawning a child process
10/15/2015 09:00:05 - Info bpbrm (pid=1504) child pid: 8880
10/15/2015 09:00:05 - Info bpdm (pid=8880) started
10/15/2015 09:00:05 - started process bpdm (pid=8880)
10/15/2015 09:00:08 - begin reading
10/15/2015 09:00:10 - end reading; read time: 0:00:02
10/15/2015 09:00:10 - Info bpdm (pid=1504) completed reading backup image

Wheres the Task Progress tab in the Backup, Archive and Restore section says:

Progress log filename : D:\Program Files\Veritas\netbackup\logs\user_ops\DOMAIN\userid\logs\jbpC0DC.tmp.log Restore Job Id=832486
Restore started 10/15/2015 08:59:29
 
08:59:46 (832486.xxx) Found (2,376,233) files in (1) images for Restore Job ID 832486.xxx
08:59:48 (832486.xxx) Searched (        2) files of (2,376,233) files for Restore Job ID 832486.xxx
08:59:48 (832486.001) Restoring from copy 3 of image created 8/6/2015 7:39:03 PM from policy FileServer
08:59:55 (832486.xxx) Restoring to vCenter server VCserver.mydomain.com, ESX host esxhost.mydomain.com, Datacenter /DC1, Folder /DC1/vm/, Display Name vmname_Restore, Resource Pool/vApp /DC1/host/, Datastore/Datastore Cluster DC1_BigMigrationDatastore
09:00:08 INF - Beginning read-blockmap on server dc1sv099 of client SCOLO1FS01, reading file I:\NetBackup-Filesystem\vmname_1438886343_C3_F1_R3.
09:00:10 (832486.xxx) Executing Virtual machine creation on dc1sv099 to server vcserver.mydomain.com.

[danielrickard]

However, having kicked off the restore and Netbackup successfully creating the new VM, nothing has happened on the job in activity monitor for nearly an hour:

10/15/2015 08:59:39 - begin Restore
10/15/2015 08:59:45 - Info bprd (pid=3384) Found (2,376,233) files in (1) images for Restore Job ID 832486.xxx
10/15/2015 08:59:47 - restoring from image VMName_1438886343
10/15/2015 08:59:48 - Info bprd (pid=3384) Searched (        2) files of (2,376,233) files for Restore Job ID 832486.xxx
10/15/2015 08:59:48 - Info bprd (pid=3384) Restoring from copy 3 of image created 08/06/15 19:39:03 from policy Abbey-FileServer
10/15/2015 08:59:55 - Info bprd.sfr (pid=3384) Restoring to vCenter server VCserver.mydomain.com, ESX host esxhost.mydomain.com, Datacenter /DC1, Folder /DC1/vm/, Display Name VMName_Restore, Resource Pool/vApp /DC1/host/, Datastore/Datastore Cluster DC1_BigMigrationDatastore
10/15/2015 09:00:03 - Info bpdm (pid=1504) started
10/15/2015 09:00:03 - started process bpdm (pid=1504)
10/15/2015 09:00:04 - Info bpdm (pid=1504) reading backup image
10/15/2015 09:00:05 - Info bpdm (pid=1504) using 30 data buffers
10/15/2015 09:00:05 - Info bpdm (pid=1504) spawning a child process
10/15/2015 09:00:05 - Info bpbrm (pid=1504) child pid: 8880
10/15/2015 09:00:05 - Info bpdm (pid=8880) started
10/15/2015 09:00:05 - started process bpdm (pid=8880)
10/15/2015 09:00:08 - begin reading
10/15/2015 09:00:10 - end reading; read time: 0:00:02
10/15/2015 09:00:10 - Info bpdm (pid=1504) completed reading backup image

Any ideas?

[Marianne]

Seems you will need to log a Support call.

Hopefully you have all relevant logging enabled.

[sdo]

A quick stab in teh dark... are the LUNs on-line and writeable by the Windows host that is the NetBackup VMware Backup/Restore host?

https://www-secure.symantec.com/connect/forums/backing-vmware-over-san-netbackup

...because sometimes server admins forget to 'online' the LUNs after the storage admins have (co-)presented the LUNs to a backup host.

[danielrickard]

The LUNs are online and writeable.

I think I was having a bad day as, in my absence last week, a colleague has successfully restored.

Thanks for all your help.