01-17-2017 09:12 PM
Hi,
I have moved the db images to another filesystem and Created the symbolic link. Below is the process what i have done.
Stoped all netbackup services
Step1: cp -pr /usr/openv/netbackup/db/images /db/netbackup/db/images/
Step2: mv /usr/openv/netbackup/db/images /db/netbackup/db/images_bak
step3: ln -s /db/netbackup/db/images/images /usr/openv/netbackup/db/images
started all netbackup services
Started few scheduled backups and later Catalog backup started but getting error code 1 that is Partially sucessful
Below is the details of Job description
01/18/2017 10:07:13 - Error bpbrm (pid=8610) from client inbmaster.asicdesigners.com: ERR - Unable to process CATALOG_BACKUP inbmaster.asicdesigners.com hw_home_1479997800_INCR.f
01/18/2017 10:07:16 - Warning bpbrm (pid=8610) from client inbmaster.asicdesigners.com: WRN - /usr/openv/netbackup/db/error/daily_messages.log is only being backed up as a symbolic link
01/18/2017 10:07:16 - Info bpbkar (pid=8614) bpbkar waited 18266 times for empty buffer, delayed 23409 times
01/18/2017 10:07:17 - Info bptm (pid=8615) waited for full buffer 33267 times, delayed 34091 times
01/18/2017 10:07:39 - Info bptm (pid=8615) EXITING with status 0 <----------
01/18/2017 10:07:39 - Info bpbrm (pid=8610) validating image for client inbmaster.asicdesigners.com
01/18/2017 10:07:39 - Info bpbkar (pid=8614) done. status: 1: the requested operation was partially successful
01/18/2017 10:07:39 - end writing; write time: 0:13:27
the requested operation was partially successful (1)
hw_home is the policy that is disabled.
Please help to fix
Thanks,
Solved! Go to Solution.
01-19-2017 10:28 PM
"I digged more info from team got to know incidently they have deleted the few files. "
I don't have words!
Do they understand the implication of what they did?
Do they know about 'image retention'?
You should be able to restore the missing files from a previous catalog backup.
01-18-2017 01:18 AM - edited 01-18-2017 01:22 AM
I think you might need to open a support case to get your environment properly inspected and possibly fixed, if it's fixable.
.
Perhaps a procedure like this for anyone else coming across this post in future.
1) de-activate all SLPs
2) de-activate all backup policies (except the catalog backup)
3) de-activate all DSSU schedules
4) take a catalog backup
5) take an image list count: bpimagelist -idonly -d 01/01/1970 00:00:00 | wc -l
6) netbackup stop
7) check it really is stopped: bpps -x
8) do the move
....but are you sure those commands did the right thing? why no -v switch and tee capture of what was actually moved? why the double trailing "/images/images" on the link target, looks weird.
9) inspect the target structure: ls -lash MYNEWPATH
10) netbackup start
11) has it all started ok: bpps -x
12) check the image list count: bpimagelist -idonly -d 01/01/1970 00:00:00 | wc -l
13) enable one test policy, and run one small backup
14) check the image list count: bpimagelist -idonly -d 01/01/1970 00:00:00 | wc -l
15) run a catalog backup
16) If you are happy, then re-activate: SLPs, policies, DSSU.
.
With something like this - my advice is to build a test system and prove the procedure before implementing in production. But then that's easy for me to say after the horse has bolted.
01-18-2017 02:14 AM
What was the purpose of step 2?
To rename images folder under /usr/openv or to really move entire folder and contents to /db? And thus having 2 copies of images in /db?
The paths in /db look very long and 'messy'. And images folder within another images folder?
Even so - the test is to cd to /usr/openv/netbackup/db/images.
What does 'ls -l' show?
The image file that is showing the error has nothing to do with deactivated policy.
It is a .f file that was created during a backup that was done on 24 Nov 2016.
Can you locate the file by using 'normal' path from /usr/openv?
01-18-2017 03:37 AM
Hi Marianne
What was the purpose of step 2?
Root file system got filled up. So I moved to another file system that is under /db as backup
Yes it is messy i created one images directory so its looks odd
[root@inbmaster ~]# ls -l /usr/openv/netbackup/db/images
lrwxrwxrwx 1 root root 23 Jan 18 06:13 /usr/openv/netbackup/db/images -> /dbimages/images/images
Can you locate the file by using 'normal' path from /usr/openv?
I did not get your question, You mean move back images directory to same location?
01-18-2017 03:56 AM
Hi,
in this case a better solution would be to migrate the complete masterserversoftware from /usr/openv to another volumen/disk and mount them under /usr/openv
ciao
tunix2k
01-18-2017 03:56 AM
What I meant - if you change directory to /usr/openv/netbackup/db/images:
# cd /usr/openv/netbackup/db/images
and then run 'ls -l' , what do you see?
I did not mean that you should move images back.
Can you locate the .f file by navigating through default /usr/openv path?
e.g.
# cd /usr/openv/netbackup/db/images/<client-name>/1479000000
Have you tried to run bpdbm -consistency 2 ?
01-18-2017 07:11 PM
Hi Marianne,
[root@inbmaster ~]# cd /usr/openv/netbackup/db/images
[root@inbmaster images]# ls -l
total 16
-r--r--r-- 1 root bin 263 Feb 8 2012 db_marker.txt
drwxr-xr-x 27 root root 4096 Jan 10 20:05 inbmaster
drwxr-xr-x 11 root root 4096 Jan 1 02:00 Inbmaster
drwxr-xr-x 28 root root 4096 Jan 17 21:39 inbmaster.asicdesigners.com
[root@inbmaster 1470000000]# ls -l
total 3300
drwx------ 2 root root 4096 Nov 9 20:54 catstore
-rw-rw-rw- 1 root root 0 Aug 1 08:23 inbmaster-catalog_1470018600_FULL.lck
-rw------- 1 root root 4887 Aug 1 08:00 inbmaster-catalog_1470018609_FULL.f
-rw-rw-rw- 1 root root 0 Aug 1 08:03 inbmaster-catalog_1470018609_FULL.lck
-rw------- 1 root root 322841 Aug 1 08:23 inbmaster-catalog_1470018952_FULL.f
-rw-rw-rw- 1 root root 0 Aug 1 08:23 inbmaster-catalog_1470018952_FULL.lck
-rw-rw-rw- 1 root root 0 Aug 2 08:20 inbmaster-catalog_1470105000_FULL.lck
-rw------- 1 root root 4887 Aug 2 08:00 inbmaster-catalog_1470105010_FULL.f
-rw-rw-rw- 1 root root 0 Aug 2 08:02 inbmaster-catalog_1470105010_FULL.lck
-rw------- 1 root root 323860 Aug 2 08:20 inbmaster-catalog_1470105231_FULL.f
-rw-rw-rw- 1 root root 0 Aug 2 08:20 inbmaster-catalog_1470105231_FULL.lck
-rw-rw-rw- 1 root root 0 Aug 3 08:24 inbmaster-catalog_1470191400_FULL.lck
-rw------- 1 root root 4887 Aug 3 08:00 inbmaster-catalog_1470191413_FULL.f
-rw-rw-rw- 1 root root 0 Aug 3 08:03 inbmaster-catalog_1470191413_FULL.lck
-rw------- 1 root root 328270 Aug 3 08:24 inbmaster-catalog_1470191780_FULL.f
-rw-rw-rw- 1 root root 0 Aug 3 08:23 inbmaster-catalog_1470191780_FULL.lck
-rw-rw-rw- 1 root root 0 Aug 4 08:20 inbmaster-catalog_1470277800_FULL.lck
-rw------- 1 root root 4887 Aug 4 08:00 inbmaster-catalog_1470277809_FULL.f
-rw-rw-rw- 1 root root 0 Aug 4 08:02 inbmaster-catalog_1470277809_FULL.lck
-rw------- 1 root root 327653 Aug 4 08:20 inbmaster-catalog_1470278018_FULL.f
-rw-rw-rw- 1 root root 0 Aug 4 08:19 inbmaster-catalog_1470278018_FULL.lck
-rw-rw-rw- 1 root root 0 Aug 5 08:24 inbmaster-catalog_1470364200_FULL.lck
-rw------- 1 root root 4887 Aug 5 08:00 inbmaster-catalog_1470364209_FULL.f
-rw-rw-rw- 1 root root 0 Aug 5 08:03 inbmaster-catalog_1470364209_FULL.lck
-rw------- 1 root root 326753 Aug 5 08:24 inbmaster-catalog_1470364589_FULL.f
-rw-rw-rw- 1 root root 0 Aug 5 08:24 inbmaster-catalog_1470364589_FULL.lck
-rw-rw-rw- 1 root root 0 Aug 8 08:20 inbmaster-catalog_1470623400_FULL.lck
-rw------- 1 root root 4887 Aug 8 08:00 inbmaster-catalog_1470623409_FULL.f
-rw-rw-rw- 1 root root 0 Aug 8 08:02 inbmaster-catalog_1470623409_FULL.lck
-rw------- 1 root root 324606 Aug 8 08:20 inbmaster-catalog_1470623610_FULL.f
-rw-rw-rw- 1 root root 0 Aug 8 08:19 inbmaster-catalog_1470623610_FULL.lck
-rw-rw-rw- 1 root root 0 Aug 9 08:24 inbmaster-catalog_1470709800_FULL.lck
-rw------- 1 root root 4887 Aug 9 08:00 inbmaster-catalog_1470709812_FULL.f
-rw-rw-rw- 1 root root 0 Aug 9 08:03 inbmaster-catalog_1470709812_FULL.lck
-rw------- 1 root root 328037 Aug 9 08:24 inbmaster-catalog_1470710195_FULL.f
-rw-rw-rw- 1 root root 0 Aug 9 08:24 inbmaster-catalog_1470710195_FULL.lck
-rw-rw-rw- 1 root root 0 Aug 10 08:20 inbmaster-catalog_1470796200_FULL.lck
-rw------- 1 root root 4887 Aug 10 08:00 inbmaster-catalog_1470796210_FULL.f
-rw-rw-rw- 1 root root 0 Aug 10 08:02 inbmaster-catalog_1470796210_FULL.lck
-rw------- 1 root root 326620 Aug 10 08:20 inbmaster-catalog_1470796431_FULL.f
-rw-rw-rw- 1 root root 0 Aug 10 08:20 inbmaster-catalog_1470796431_FULL.lck
-rw-rw-rw- 1 root root 0 Aug 11 08:23 inbmaster-catalog_1470882600_FULL.lck
-rw------- 1 root root 4887 Aug 11 08:00 inbmaster-catalog_1470882610_FULL.f
-rw-rw-rw- 1 root root 0 Aug 11 08:03 inbmaster-catalog_1470882610_FULL.lck
-rw------- 1 root root 328035 Aug 11 08:23 inbmaster-catalog_1470882954_FULL.f
-rw-rw-rw- 1 root root 0 Aug 11 08:23 inbmaster-catalog_1470882954_FULL.lck
-rw-rw-rw- 1 root root 0 Aug 12 08:20 inbmaster-catalog_1470969000_FULL.lck
-rw------- 1 root root 4887 Aug 12 08:00 inbmaster-catalog_1470969012_FULL.f
-rw-rw-rw- 1 root root 0 Aug 12 08:02 inbmaster-catalog_1470969012_FULL.lck
-rw------- 1 root root 329575 Aug 12 08:20 inbmaster-catalog_1470969249_FULL.f
-rw-rw-rw- 1 root root 0 Aug 12 08:20 inbmaster-catalog_1470969249_FULL.lck
drwxr-xr-x 3 root root 4096 Aug 12 08:20 tmp
[root@inbmaster 1470000000]# pwd
/usr/openv/netbackup/db/images/inbmaster.asicdesigners.com/1470000000
bpdbm -consistency 2 Few of .f files saying does not exits and attached file for reference
01-18-2017 10:29 PM
Only those 3 folders? No client folders?
This means that there is something wrong with your symbolic links and actual location of images that were moved.
Have a look at your previous posts:
Here you showed where the images were copied to:
cp -pr /usr/openv/netbackup/db/images /db/netbackup/db/images/
But this is where the symbolic link is pointing to:
rwxrwxrwx 1 root root 23 Jan 18 06:13 /usr/openv/netbackup/db/images -> /dbimages/images/images
So, where exactly are the client folders located?
When we know this we can help you to fix the symlink.
01-18-2017 11:16 PM
01-18-2017 11:41 PM
Okay... I have honestly never see a NetBackup installation where the master server is the only client....
You seem to be using 3 different hostnames for the master in backup policies:
inbmaster, Inbmaster, inbmaster.asicdesigners.com.
So, the problem seems to be with missing .f files.
The one in your opening post and the others mentioned in consistency check.
Any idea why they are missing? Not just one or two - quite a large number!
Can you locate them using find command?
e.g.
find /dbimages/images/images -name hw_home_1479997800_INCR.f -print
Search images_bak folder as well to see if the .f files were present before they were copied/moved.
01-19-2017 09:10 PM
[root@inbmaster 1470000000]# find /dbimages/images/images -name hw_home_1479997800_INCR.f -print
/dbimages/images/images/inbmaster.asicdesigners.com/1479000000/hw_home_1479997800_INCR.f
Any idea why they are missing? Not just one or two - quite a large number!
It was working fine after images migration it starts giving me problem.
Search images_bak folder as well to see if the .f files were present before they were copied/moved.
I digged more info from team got to know incidently they have deleted the few files.
Is there way we can restore to last full catalog backup ?
we dont have backup for the deleted files
01-19-2017 10:28 PM
"I digged more info from team got to know incidently they have deleted the few files. "
I don't have words!
Do they understand the implication of what they did?
Do they know about 'image retention'?
You should be able to restore the missing files from a previous catalog backup.
01-20-2017 01:10 AM
Thanks a lot marriane for your setup. I have restored Catalog backup
01-20-2017 01:15 AM
01-20-2017 03:55 AM
Hi Marriene,
I have restored all the catalog for 4 days back. We have snapshots and another bakup for same clients.
Thanks,
Vidya Sagar