02-13-2018 06:52 AM
"File read failed" error on executing command bpimagelist to get the image count . Why this error had occured and how this can be resolved. Please advise.
Solved! Go to Solution.
02-13-2018 07:48 AM
There could be some image in bpdbm which does no have the associated .f file missing or the respective NBDB entry missing.
Could you check with the bpdbm logs when the ran the command, that should tell you which file seems to have reported that file read failed error.
Alternately you can do a bpdbm consistency check but that would take time to complete
bpdbm -consistency 2
The command should report the file read failed error that you see.
Or another thing that you could try is simply run the command without the find and redirect the output to a file. We should be able to view the backupid that reports the file read failed error
bpimagelist -idonly -d 01/01/1970 > C:\temp\bpimagelist.txt
02-15-2018 11:01 PM
I agree with @Amol_Nair - 1400000000_old does not belong there. Someone must've done it.
Is there a 1400000000 folder as well?
If so, can you compare the contents of the two?
If not - simply rename 1400000000_old to 1400000000.
The second header should be in a client folder under 1380000000.
Locate each of these header files. What is the size? Anything with file size 0 can be removed.
If the file is >0, look af the contents - there should be a reference to .f file(s). Does that exists?
If not, there is no point to keeping the header file. In order to restore from this backup, you will need to re-import the backup.
If you are nervous to delete anything, simply move to a non-netbackup folder.
When bpimagelist runs 'clean', you should be ready to upgrade.
PS:
Can we assume that the next step will be to migrate to supported OS and upgrade to supported NBU version?
02-13-2018 07:48 AM
There could be some image in bpdbm which does no have the associated .f file missing or the respective NBDB entry missing.
Could you check with the bpdbm logs when the ran the command, that should tell you which file seems to have reported that file read failed error.
Alternately you can do a bpdbm consistency check but that would take time to complete
bpdbm -consistency 2
The command should report the file read failed error that you see.
Or another thing that you could try is simply run the command without the find and redirect the output to a file. We should be able to view the backupid that reports the file read failed error
bpimagelist -idonly -d 01/01/1970 > C:\temp\bpimagelist.txt
02-13-2018 08:34 PM
02-15-2018 05:47 AM
Thank you @Marianne The command does give the output within seconds. we get the count along with the "file read failed" error.
We get the below output while we run the command:
bpimagelist -idonly -d "01/01/1970 00:00:00" | %SystemDrive%\Windows\System32\find.exe /C " ID: "
file read failed
1238
The count is accurate as we have verified by redirecting the output to a text file and taking the count. We did not get any error like "file read failed" in the text file.
We are planning for netbackup upgrade to 7.5 from 7.0.1. Can we go ahead and perfrom the upgrade, while we still receive the above error.
02-15-2018 06:01 AM
Thankyou @Amol_Nair for your reply. We directed the output to a text file. We got the exact count. We verified, there is no "file read" error, or "failed" in the output text file. no error reported in text.
However, we still receive the error in the command propmpt while we run the commnad.
D:\VeritasNBU65\NetBackup\bin\Admincmd>bpimagelist -idonly -d "01/01/1970 00:00:
00" > D:\Output99.txt
file read failed
D:\VeritasNBU65\NetBackup\bin\Admincmd>
02-15-2018 06:08 AM
Do you have bpdbm log folder on the master server?
If not, please create the folder and restart NBU.
When you run the command again, there should be an entry in the log file that contains the problematic image id. The error should be logged as a <16>.
Here is a TN that explains how to look for empty or corrupt image headers:
https://www.veritas.com/support/en_US/article.100018559
You should fix errors and issues with NBU catalog database before the upgrade.
NBU 7.5 and later moves image headers into the relation database. You want the headers to be 'clean'.
02-15-2018 07:06 AM
Hi @Marianne, We could see the below two bad image headers from the bpdbm log.
[51940.47336] <2> list_client_images: Bad image header: DCBA-XYZD-Test-Servers_1400238029_FULL, error: no entity was found (227)
43384.51588] <4> db_error_add_to_file: Bad image header: Forecaster-ORACLE-RMAN-ABC-Monthly-Full_1380524846_UBAK, error: events out of sequence - image inconsistency (229)
Could find one of them in the path: <Installpath>\NetBackup\db\images\client1\1400000000_old
can we go ahead and manually delete the bad image headers/or the folder containing it? Will it not create any inconsistency?
02-15-2018 10:39 AM
This folder "1400000000_old" shouldn't really be there.. Someone attempted some changes and that is when it seems to have been renamed. Please remove this folder or move it out of db\images location.
Do you also see any other files in the path for these 2 images you mentioned? For the image which reported a status 227, bpexpdate command may also return the same problem, so you may need to use some extra switches with it. But yes since these 2 images are corrupt you would need to clear them off. bpexpdate command would clear the .f as well as the entries from the NBDB, incase the image is only present in either the image database or only in the NBDB extra manual steps would be required accordingly
Ideally they should have been reported in the bpdbm consistency check as well, unless you searched with the key word -error or failed. Could you share the bpdbm consistency check that you ran earlier as well
02-15-2018 04:07 PM
02-15-2018 11:01 PM
I agree with @Amol_Nair - 1400000000_old does not belong there. Someone must've done it.
Is there a 1400000000 folder as well?
If so, can you compare the contents of the two?
If not - simply rename 1400000000_old to 1400000000.
The second header should be in a client folder under 1380000000.
Locate each of these header files. What is the size? Anything with file size 0 can be removed.
If the file is >0, look af the contents - there should be a reference to .f file(s). Does that exists?
If not, there is no point to keeping the header file. In order to restore from this backup, you will need to re-import the backup.
If you are nervous to delete anything, simply move to a non-netbackup folder.
When bpimagelist runs 'clean', you should be ready to upgrade.
PS:
Can we assume that the next step will be to migrate to supported OS and upgrade to supported NBU version?
02-22-2018 10:23 PM