11-26-2015 04:46 AM
We have seen this issue posted already but have been unable to resolve.
We are using NetBackup v7.6.1 and have a couple of policies (with identical settings) to backup a number of VMware virtual machines.
One of the policies has a problem when trying to select individual files for restore. Like the others it is type VMware and 'Enable file recovery from VM backup" is selected/ticked.
Backups are completing successfully every day for this policy (snapshot and backup), over 1.1 gb written).
We only get this error for this policy and only when selecting "Restore from Normal Backup..."
Restore from "Virtual Machine Backup" does not produce the error and displays results under "Contents" pain without any error.
Policy config:
C:\Program Files\Veritas\NetBackup\bin\admincmd>bppllist Test_VM_CBT -L
Policy Name: Test_VM_CBT
Options: 0x0
template: FALSE
audit_reason: ?
Names: (none)
Policy Type: VMware (40)
Active: yes
Effective date: 13/07/2015 11:46:28
File Restore Raw: yes
Block Incremental: yes
Application Consistent: yes
Mult. Data Stream: no
Perform Snapshot Backup: yes
Snapshot Method: VMware_v2
Snapshot Method Arguments: Virtual_machine_backup=2,disable_quiesce=0,drive_selection=0,enable_vCloud=0,exclude_swap=1,file_system_optimizat
ion=1,ignore_irvm=1,multi_org=0,nameuse=1,post_events=1,rHz=10,rLim=10,rTO=0,serverlist=,skipnodisk=0,snapact=2,trantype=nbd:nbdssl:hotadd:s
an
Perform Offhost Backup: yes
Backup Copy: 0
Use Data Mover: no
Data Mover Type: -1
Use Alternate Client: no
Alternate Client Name: dnetbackup1.clients.local
Use Virtual Machine: 1
Hyper-V Server Name: (none)
Enable Instant Recovery: no
Policy Priority: 0
Max Jobs/Policy: 8
Disaster Recovery: 0
Collect BMR Info: no
Keyword: (none specified)
Data Classification: -
Residence is Storage Lifecycle Policy: no
Client Encrypt: no
Checkpoint: no
Residence: dnetbackup1_DiskPool-stu
Volume Pool: NetBackup
Server Group: *ANY*
Granular Restore Info: no
Exchange Source attributes: no
Exchange DAG Preferred Server: (none defined)
Application Discovery: no
Discovery Lifetime: 28800 seconds
ASC Application and attributes: (none defined)
Generation: 17
Ignore Client Direct: no
Enable Metadata Indexing: no
Index server name: NULL
Use Accelerator: yes
Client/HW/OS/Pri/DMI/CIT: DACSDB.clients.local vmx-08 windows7Server64Guest 0 0 0 0 ?
Include: ALL_LOCAL_DRIVES
Schedule: Full_CBT
Type: FULL (0)
Frequency: 1 day(s) (86400 seconds)
Excluded Dates----------
No specific exclude dates entered
No exclude days of week entered
Retention Level: 0 (1 week)
u-wind/o/d: 0 0
Incr Type: DELTA (0)
Alt Read Host: (none defined)
Max Frag Size: 0 MB
PFI Recovery: 0
Maximum MPX: 1
Number Copies: 1
Fail on Error: 0
Residence: (specific storage unit not required)
Volume Pool: (same as policy volume pool)
Server Group: (same as specified for policy)
Residence is Storage Lifecycle Policy: 0
Schedule indexing: 0
Daily Windows:
Day Open Close W-Open W-Close
Sunday 000:00:00 000:00:00
Monday 018:00:00 028:05:00 042:00:00 052:05:00
Tuesday 018:00:00 028:05:00 066:00:00 076:05:00
Wednesday 018:00:00 028:05:00 090:00:00 100:05:00
Thursday 018:00:00 028:05:00 114:00:00 124:05:00
Friday 018:00:00 028:05:00 138:00:00 148:05:00
Saturday 000:00:00 000:00:00
11-26-2015 04:49 AM
We have also tried creating a new policy and stop/start of Netbackup services. Still same issue.
11-26-2015 05:10 AM
Hello,
maybe there is a filesystem in VM which is not supported for file-level recovery.
Check "Number Of Files" in the appropriate backup job. If it is <10 or so, instead of >1000 or so, then that is the case.
Review http://www.veritas.com/docs/000006177 if you have supported FS.
Michal
11-26-2015 06:04 AM
Thank you for your reply. That is interesting. In the last job under [Detailed Status] it shows 'Current Kilobytes Written: 1,162,732,583. Current Files Written : 9. (didn't spot that earlier).
No errors in the logs though as far as I can see, backups takes about 5 hours to complete, and the client is just a Windows 7 Server that same as all the other VMs being backed up. Any further ideas?
11-26-2015 06:29 AM
Going to get the client to confirm o/s and file system. Will post back with my findings.
11-26-2015 06:50 AM
Well, as it happens the o/s is not fully supported:
Windows 2008 R2 (64bit) – Notes - Supported for vStorage but not for VCB.
Not being too familiar with NetBackup - what are the implications of the above support statement?
Cheers
Jon
11-26-2015 07:09 AM
No this note is not important, because NBU 7.6.1 does not engage VCB.
Review also the table below - Supported file systems for VMware
But with Windows 2008 R2, I dont expect there is an unsupported FS. Then try to restart VM in question, and if it wont help, it is probably for Support Case opening.
Michal
11-27-2015 01:37 AM
Can confirm that client is Windows 2008 R2 (64bit) using NTFS file system so should be supported. We are requesting a restart of the VM.
11-27-2015 12:08 PM
Is VMtools installed, up to date (to match ESXi requirements), and started and working?
If VMtools is old or missing or stopped, then this call stack will probably fail... bpkar32 -> bpfis -> VADP -> VMtools -> VSS -> NTFS... and so it could be possible that a break in this chain might not necessarily completely break backups but may result in a 'VM style' backup as if 'Enable individual file restore' had not originally been selected. And the only way to verify this, would be to work through the chain of logs through each layer... which itself has been covered in this forum many times.
.
BTW - the call stack isn't exactly like that... the VSS/NTFS part is abstracted by a stream handler - so I only describe it like that to demonstate that a break in the stack may render the VM stream handler unable to unerstand and interpret the volume/folder/file boundaries in the tar stream (actually arriving via SAN/NBD/HotAdd transport).
12-02-2015 05:33 AM
Reboot of the VM machine did not resolve the issue.
Going to check VMtools as suggested.
12-02-2015 05:48 AM
VMware tools us running and current. Going to open a Support Case.