12-10-2011 12:14 AM
hannel d1: restoring control file
released channel: d1
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of restore command at 12/10/2011 11:41:39
ORA-19870: error while restoring backup piece /c-1255836038-20111209-02
ORA-19507: failed to retrieve sequential file, handle="/c-1255836038-20111209-02", parms=""
ORA-27029: skgfrtrv: sbtrestore returned error
ORA-19511: Error received from media manager layer, error text:
Backup file </c-1255836038-20111209-02> not found in NetBackup catalog
compare dbclient log now and before
before when restored good :
....
02:30:32.764 [23084] <2> vnet_vnetd_daemon_socket: ../../libvlibs/vnet_vnetd.c.479: VN_REQUEST_DAEMON_SOCKET
: 2 0x00000002
02:30:32.764 [23084] <2> vnet_vnetd_daemon_socket: ../../libvlibs/vnet_vnetd.c.493: ipc_string: /tmp/vnet-23
104323109819401252000000002-KZgdy0
02:30:32.845 [23084] <2> int_ReadData: INF - reading buffer # 1 of size 262144
02:30:33.755 [23084] <2> int_ReadData: INF - Got end-of-file
02:30:33.755 [23084] <8> int_ReadData: WRN - Failed to set client read timeout.
02:30:34.271 [23084] <2> int_CloseImage: INF - Restore - closing </c-1255836038-20111205-02>
02:30:53.431 [23084] <4> sbtend: INF - --- END of SESSION ---
Now failed:
02:30:06.786 [29503] <2> vnet_vnetd_service_socket: ../../libvlibs/vnet_vnetd.c.2068: service: bprd
02:30:06.827 [29503] <2> logconnections: BPRD CONNECT FROM 127.0.0.1.10896 TO 127.0.0.1.13724
02:30:06.953 [29503] <2> vnet_vnetd_service_socket: ../../libvlibs/vnet_vnetd.c.2054: VN_REQUEST_SERVICE_SOC
KET: 6 0x00000006
02:30:06.953 [29503] <2> vnet_vnetd_service_socket: ../../libvlibs/vnet_vnetd.c.2068: service: bprd
02:30:06.994 [29503] <2> logconnections: BPRD CONNECT FROM 127.0.0.1.63284 TO 127.0.0.1.13724
02:30:07.119 [29503] <4> VxBSAQueryObject: INF - No match was found for query
02:30:07.119 [29503] <2> int_FindBackupImage: INF - /c-1255836038-20111209-02 not found
02:30:07.119 [29503] <16> int_StartJob: ERR - Backup file </c-1255836038-20111209-02> not found in NetBackup
catalog
seems the key point :
vnet_vnetd_daemon_socket: ../../libvlibs/vnet_vnetd.c.493: ipc_string: /tmp/vnet-23104323109819401252000000002-KZgdy0
12-10-2011 08:37 AM
1.) Share the Activity Monitor Details.
2.) Did you have both Oracle machines on the Network, means 10G and 11G ?
If yes then are you able to restore normal / Flat file backup from 11G machine ? if not then first check the Network connectivity, DNS resolution.
12-10-2011 10:50 AM
Have you verified that the Rman catalog and the NetBackup images database are in sync?
This looks like the image might have expired in NBU:
'Backup file </c-1255836038-20111209-02> not found in NetBackup catalog'
Run the following on NBU master to list backups for this client for the period that you are looking for:
'bplist -C <client-name> -t 4 -s <start-date> -e <end-date> -R /
12-10-2011 06:29 PM
Thanks for replies!
Activity Monitor reveals none abnormal.
10G and 11G are on the same nbu server, 10g restored OK
bplist told good result for two client macines (10g & 11g)
I noticed the failed dbclient.log for 11g
<4> VxBSAQueryObject: INF - No match was found for query
wonder any tools can diagnose VxBSAQueryObject & ipc_string: /tmp/vnet-...
12-10-2011 06:52 PM
yeah i noticed the owner of bplist output is incorret
-rw-rw---- oracle oinstall 131072 Dec 10 22:36 /c-1255836038-20111210-02
the 'oracle' should be ora11,the group is ok, is it the cause ?
12-10-2011 07:02 PM
the above owner maybe not incorret or shoud be such, local is ora11, but before is good
12-10-2011 08:53 PM
bplist is not incorrect - this particular backup was done by oracle user. NetBackup has no way of knowing this other than the user id that is sent at the time of backup. Image ownership cannot be changed.
The restore can only be done by the user who performed the backup - in your case, oracle.
If the backup was done with 'group' or 'any' permissions, the restore can be done as different user.
See http://www.symantec.com/docs/TECH141983
extract:
With NetBackup 6.5.4 you can set permissions on the backup image when the backup is written. The permissions can be set to USER, GROUP or ANY.
SEND 'BKUP_IMAGE_PERM=USER|GROUP|ANY' when backup is written to set the permissions.
So, if the backup was not done this way, only the oracle user can restore this backup.
12-10-2011 10:25 PM
from the output of bplist
-rw-rw---- oracle oinstall 131072 Dec 10 22:36 /c-1255836038-20111210-02
I can restore by another user in group oinstall other than oracle before Dec 7
in fact I now even cannot restore backups that I ever can
12-11-2011 04:32 PM
bprd log said on Dec 6
02:30:18.491 [23096] <2> fileslist: begin db communication
02:30:18.491 [23096] <2> fileslist: criteria sent to db mgr
02:30:18.565 [23096] <2> process_request: EXIT STATUS 0
now
02:30:19.494 [5960] <2> fileslist: begin db communication
02:30:19.494 [5960] <2> fileslist: criteria sent to db mgr
02:30:19.545 [5960] <2> fileslist: unexpected return value from db_FLISTreceive: no entity was found 227
02:30:19.545 [5960] <2> process_request: fileslist failed - EC status = no entity was found
02:30:19.545 [5960] <2> process_request: EXIT STATUS 227
12-11-2011 04:45 PM
get_image_by_file all were OK
on Dec 6
02:30:18.328 [23088] <2> process_request: EXIT STATUS 0
now
02:30:19.344 [5952] <2> process_request: EXIT STATUS 0
yeah but gid in filelist is different
on Dec 6
02:30:18.447 [23096] <2> fileslist: client_uid = 1002
02:30:18.447 [23096] <2> fileslist: client_gid = 505
now
02:30:19.450 [5960] <2> fileslist: client_uid = 1002
02:30:19.450 [5960] <2> fileslist: client_gid = 1002
apparently this the cause , I'll confirm it soon
12-11-2011 05:02 PM
but
# id oracle
uid=505(oracle) gid=505(oinstall) groups=505(oinstall),504(dba),1002(asmadmin),1004(asmdba)
no problem
why
id ora11
uid=1002(ora11) gid=505(oinstall) groups=505(oinstall),504(dba),1002(asmadmin),1004(asmdba)
cannot
even # id ora11
uid=1002(ora11) gid=505(oinstall) groups=505(oinstall)
still
08:55:42.509 [27851] <2> fileslist: client_gid = 1002
the same even if restart netbackup
12-11-2011 05:26 PM
so how to change something somewhere that bprd can gives correct log ?
let
08:55:42.509 [27851] <2> fileslist: owner = ora11
08:55:42.509 [27851] <2> fileslist: group = asmadmin
back to be
02:30:18.447 [23096] <2> fileslist: owner = ora11
02:30:18.447 [23096] <2> fileslist: group = oinstall
12-11-2011 08:19 PM
It seems your backups are performed by more than one user that belongs to more than one group? Did you do a listing for more than one image? More info from bprd is needed to determine why there is a difference.
Permissions are written at the time of backup, it cannot be changed afterwards.
You have two choices:
12-11-2011 11:54 PM
thanks anyhow, but your choices too less, I dropped the user and then recreated it, the group's name became the hoped. Although I'm not willing it's a bug, sometime if it appears again after I adjust the policy ...
12-12-2011 12:10 AM
If you are not happy with my response, please open a support call with Symantec.
I have been assisting customers for more than 12 years with Oracle restores - that is how NBU works.
For many years we only had option 1.
Symantec has now added option 2 for us.
12-12-2011 03:37 AM
not unhappy,but only want to know why the group changed
for your choice 1 I already have the same user used so impossible
for choice 2 I hope to be still able to use previous backups, also impossible
thanks again for your kind support