FlashBackup Windows policy restore error (2817) for a VM

[rakesh_k]

Hi All,

I am facing an issue while restoring a VM. The detailed status shows its complete, but after 40 to 45 mins, its fails by 2817.

Detailed Status:

02/04/2013 15:37:48 - Info bptm (pid=9184) start
02/04/2013 15:37:48 - started process bptm (pid=9184)
02/04/2013 15:37:48 - Info bpdm (pid=9184) reading backup image
02/04/2013 15:37:48 - Info bptm (pid=9184) using 30 data buffers
02/04/2013 15:37:48 - Info bptm (pid=9184) spawning a child process
02/04/2013 15:37:48 - Info bptm (pid=9184) child pid: 9092
02/04/2013 15:37:48 - Info bptm (pid=9092) start
02/04/2013 15:37:50 - begin Restore
02/04/2013 15:37:51 - started process bptm (pid=9092)
02/04/2013 15:37:52 - media needed: ST0052
02/04/2013 15:37:52 - restoring from image client_1359970680
02/04/2013 15:37:55 - requesting resource ST0052
02/04/2013 15:37:59 - Info bptm (pid=9184) Waiting for mount of media id ST0052 (copy 1) on server cdcubkpvmhost.
02/04/2013 15:37:59 - started process bptm (pid=9184)
02/04/2013 15:37:59 - mounting ST0052
02/04/2013 15:37:59 - Info bptm (pid=9184) INF - Waiting for mount of media id ST0052 on server cdcubkpvmhost for reading.
02/04/2013 15:38:05 - Waiting for scan drive stop HP.ULTRIUM5-SCSI.009, Media server: bkpvmhost
02/04/2013 15:38:05 - granted resource  ST0052
02/04/2013 15:38:05 - granted resource  HP.ULTRIUM5-SCSI.009
02/04/2013 15:38:55 - mounted ST0052; mount time: 0:00:56
02/04/2013 15:38:55 - Info bptm (pid=9184) ST0052
02/04/2013 15:38:55 - Info bptm (pid=9184) INF - Waiting for positioning of media id ST0052 on server cdcubkpvmhost for reading.
02/04/2013 15:38:55 - positioning ST0052 to file 166
02/04/2013 15:41:21 - positioned ST0052; position time: 0:02:26
02/04/2013 15:41:24 - begin reading
02/04/2013 15:41:24 - Info bptm (pid=9184) waited for empty buffer 1 times, delayed 8 times
02/04/2013 15:41:24 - end reading; read time: 0:00:00
02/04/2013 15:41:24 - Info bptm (pid=9184) completed reading backup image
02/04/2013 15:48:27 - Info bpbrm (pid=5288) BKPVMHOST is the host to restore to
02/04/2013 15:48:27 - Info bpbrm (pid=5288) reading file list from client
02/04/2013 15:48:27 - connecting
02/04/2013 15:48:27 - Info bpbrm (pid=5288) starting bptm
02/04/2013 15:48:29 - Info tar32 (pid=8376) Restore started
02/04/2013 15:48:29 - connected; connect time: 0:00:00
02/04/2013 15:48:29 - Info bptm (pid=1636) start
02/04/2013 15:48:29 - started process bptm (pid=1636)
02/04/2013 15:48:30 - Info bpdm (pid=1636) reading backup image
02/04/2013 15:48:30 - Info bptm (pid=1636) using 30 data buffers
02/04/2013 15:48:30 - Info bptm (pid=1636) spawning a child process
02/04/2013 15:48:30 - Info bptm (pid=1636) child pid: 4152
02/04/2013 15:48:30 - Info bptm (pid=4152) start
02/04/2013 15:48:30 - started process bptm (pid=4152)
02/04/2013 15:48:33 - mounted ST0052
02/04/2013 15:48:33 - Info bptm (pid=1636) ST0052
02/04/2013 15:48:33 - Info bptm (pid=1636) INF - Waiting for positioning of media id ST0052 on server cdcubkpvmhost for reading.
02/04/2013 15:48:33 - positioning ST0052 to file 166
02/04/2013 15:48:33 - positioned ST0052; position time: 0:00:00
02/04/2013 15:48:36 - begin reading
02/04/2013 15:48:37 - requesting resource ST0052
02/04/2013 15:48:39 - granted resource  ST0052
02/04/2013 15:48:39 - granted resource  HP.ULTRIUM5-SCSI.009
02/04/2013 16:35:40 - Info bptm (pid=1636) waited for empty buffer 148216 times, delayed 173054 times
02/04/2013 16:35:43 - end reading; read time: 0:47:07
02/04/2013 16:35:43 - Info bptm (pid=1636) completed reading backup image
02/04/2013 16:35:48 - Info tar32 (pid=8376) done. status: 0
02/04/2013 17:52:03 - restored from image client_1359970680; restore time: 2:14:11
02/04/2013 17:52:04 - end Restore; elapsed time 2:14:14
FlashBackup Windows policy restore error  (2817)

Its a Linux Machine. Please suggest what needs to be done.

The loggin level is enabled, also shared the logs with Symantec, till now there has been no solution from their end.

Master Server Version : 7.1.0.2

VMhost : 7.1.0.4

[RamNagalla]

hi,

could you please provide the restore log form the master server /usr/openv/netbackup/logs/user_ops/logs/<log file>

and also from the tar log from the backup/restore host

[Mark_Solutions]

Is your VMWare backup host Windows 2008?

Does the restore actually work (can you boot the restored machine)?

If it is 2008 and the restored machine will not boot then it maybe that it is waiting for a final response and does not get it because the restore has not worked correctly.

With Windows 2008 VMWare backup hosts your disks ( the ones mapped to the datastores) must be set as "online"

You do not need to mount them or write a signature, just set them as online for a restore to work correctly.

Also worth upgrading your Master when you get chance as the Master and Media will be using different vSphere libraries due to the different patch levels so they may not be communicating with each other correctly

Finally keep an eye on the Late Breaking News in case there is anything relevant - if you are vSphere V5 then this may apply:

http://www.symantec.com/docs/TECH179716

Hope this helps

[rakesh_k]

@mark: thanks for your reply.

In this case I am trying to restore through nbd transport type, instead of san.

Restoration for other VM's are happening fine. No able to restore this particular VM.

Also to add one thing to this, while selecting the restore type as normal backups, i am not able to browse through the files for file level restore, which is happening fine for other backups.

@nagalla: i am attaching the tar logs. cant upload the user_ops logs, since they are very huge...

[Mark_Solutions]

What was the state of this machine when you backed it up? What version of Linux is it?

If it does not show any files when "normal" backup type is selected (I assume you do this whilst the policy type is still set as Flashbackup-Windows (or VMWare depending on what you used in the policy) then i am guessing that the client is either not supported for individual file restore or was powered off when you backed it up.

I will take a look at the logs and get back to you

[Mark_Solutions]

Logs look fine - this does tend to indicate just a timeout somewhere between, I would have thought, vCenter and the Media Server.

A little like when it takes a long time to deal with snapshots a new machine being created may also take a while to process.

Work looking at you timeouts to just allow it more time (though don't know why it should take so long to create a virtual machine unless your vCenter is very busy)

Take a look through the NetBackup VMWare Admin guide in the toubleshooting section for timeout changes that can be made.

You never did say if the restored machine did actually work after the restore?

[rakesh_k]

@mark:

as you can see... the restoration gets completed. but it waits for a long time... dont knw wht it does in tht time period... and fails after some time....

02/06/2013 18:07:31 - Info bptm (pid=9860) waited for empty buffer 184904 times, delayed 238338 times
02/06/2013 18:07:31 - end reading; read time: 1:08:04
02/06/2013 18:07:32 - Info bptm (pid=9860) completed reading backup image
02/06/2013 18:07:39 - Info tar32 (pid=6184) done. status: 0
02/06/2013 19:02:38 - restored from image client_1360065534; restore time: 2:07:01
02/06/2013 19:02:39 - end Restore; elapsed time 2:07:04
FlashBackup Windows policy restore error  (2817)

The Windows VM that I restored i working perfectly fine... no issues on that part...

i am facing this issue for only linux clients.... also tried installing the symcquiese in the linux machine... but still no luck....

[Mark_Solutions]

It seems to take a long time after a restore to get full acknoledgement from vCenter and it seems after restoration it sometime deals with snsphot(s).

As a result of this it does look like while it is doing all of this it gets a timeout, so do check the timeouts section in the troubleshooting section of the NBU VMWare Guide

[rakesh_k]

@mark: since the image is being written on the datastore, the tar process seems fine. I think there is some issue with the bpdbm process which runs on master server once the tart process in completed on the VMHOST.

the timeout values are as per the symantec engineer working on this case...

[rakesh_k · Accepted solution]

Went through the following link:

http://technet.microsoft.com/en-us/library/cc957549.aspx

http://h30499.www3.hp.com/t5/System-Administration/Configuring-TCP-KeepAlive-Parameters/td-p/4751119

made the changes in TCPKEEPALIVE setting at the OS end. Now the restoration is successful and the VM is booting properly.