β09-29-2011 10:30 AM
Last night I migrated my production database to a SFHA cluster made up two Sun T3-1B blade servers. The nodes of the cluster are: st31bbl01 (Sun T3-1B blade 01) and st31bbl02 (Sun T3-1B blade 02). The name of the virtual database server than can fail over between these nodes is st31bora01 (Sun T3-1B oracle 01).
Database backups report as successful within the Netbackup GUI but when I check the log file from the RMAN backup script I see the list of Db files successfully backed up followed by this:
Starting Control File and SPFILE Autobackup at 29-SEP-11
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03009: failure of Control File and SPFILE Autobackup command on ORA_SBT_TAPE_1 channel at 09/29/2011 01:47:42
ORA-19506: failed to create sequential file, name="PASPROD_c-1968089396-20110929-01", parms=""
ORA-27027: sbtremove2 returned error
ORA-19511: Error received from media manager layer, error text:
Failed to remove, PASPROD_c-1968089396-20110929-01, from image catalog.
When I check the log file under /usr/openv/netbackup/logs/dbclient, I find the following:
21:46:54.822 [9177] <2> sbtremove2: INF - entering 21:46:54.822 [9177] <2> int_RemoveImage: INF - entering 21:46:54.822 [9177] <2> int_RemoveImage: INF - Removing backup image => <PASPROD_c-1968089396-20110928-04> 21:46:54.822 [9177] <2> xbsa_ValidateFeatureId: INF - entering 21:46:54.822 [9177] <2> xbsa_ValidateFeatureId: INF - leaving (0) 21:46:54.822 [9177] <2> int_FindBackupImage: INF - entering 21:46:54.822 [9177] <2> int_GetBfsDateRange: INF - entering 21:46:54.823 [9177] <2> int_GetBfsDateRange: INF - RMAN file name = PASPROD_c-1968089396-20110928-04 21:46:54.823 [9177] <2> int_GetBfsDateRange: INF - probable auto controlfile 21:46:54.823 [9177] <2> int_GetBfsDateRange: INF - Start range check: bfs_time=1317232800, start=819936000, end=1317354414 21:46:54.823 [9177] <2> int_logDateRange: INF - Start Time = 09/27/11 12:00:00 21:46:54.823 [9177] <2> int_logDateRange: INF - End Time = 09/29/11 12:00:00 21:46:54.823 [9177] <2> int_logDateRange: INF - leaving 21:46:54.823 [9177] <2> xbsa_BeginTransaction: INF - entering 21:46:54.823 [9177] <4> VxBSABeginTxn: INF - entering VxBSABeginTxn. 21:46:54.823 [9177] <4> VxBSAGetEnv: INF - entering GetEnv - NBBSA_DB_TYPE 21:46:54.823 [9177] <4> VxBSAGetEnv: INF - returning - Oracle 21:46:54.823 [9177] <4> VxBSAGetEnv: INF - entering GetEnv - NBBSA_NTUPLE_RESTORE 21:46:54.823 [9177] <4> VxBSAGetEnv: INF - returning - 21:46:54.824 [9177] <4> VxBSAGetEnv: INF - entering GetEnv - NBBSA_COPY_NUMBER 21:46:54.824 [9177] <4> VxBSAGetEnv: INF - returning - 21:46:54.824 [9177] <2> xbsa_BeginTransaction: INF - leaving (0) 21:46:54.824 [9177] <2> xbsa_QueryObject: INF - entering 21:46:54.824 [9177] <4> VxBSAQueryObject: INF - entering QueryObject. 21:46:54.824 [9177] <4> dbc_GetServerClientConfig: entering dbc_GetServerClientConfig. 21:46:54.824 [9177] <4> dbc_GetServerClientConfig: ServerName: <inf-srv17.apacorp.net>, ClientName: <st31bora01> 21:46:54.824 [9177] <4> VxBSAGetEnv: INF - entering GetEnv - BSA_SERVICE_HOST 21:46:54.824 [9177] <4> VxBSAGetEnv: INF - returning - inf-srv17.apacorp.net 21:46:54.824 [9177] <4> VxBSAGetEnv: INF - entering GetEnv - NBBSA_CLIENT_HOST 21:46:54.824 [9177] <4> VxBSAGetEnv: INF - returning - st31bora01 21:46:54.824 [9177] <4> VxBSAGetEnv: INF - entering GetEnv - NBBSA_POLICY 21:46:54.824 [9177] <4> VxBSAGetEnv: INF - returning - ORA_PASPROD 21:46:54.825 [9177] <4> VxBSAGetEnv: INF - entering GetEnv - NBBSA_KEYWORD 21:46:54.825 [9177] <4> VxBSAGetEnv: INF - returning - pasprod 21:46:54.825 [9177] <4> bsa_bplist: entering bsa_bplist 21:46:54.828 [9177] <2> vnet_async_connect: vnet_vnetd.c.3983: connect in progress: 0 0x00000000 21:46:54.865 [9177] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2043: VN_REQUEST_SERVICE_SOCKET: 6 0x00000006 21:46:54.865 [9177] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2057: service: bprd 21:46:55.075 [9177] <2> vnet_async_connect: vnet_vnetd.c.4169: in progress connect: 0 0x00000000 21:46:55.076 [9177] <2> vnet_async_connect: vnet_vnetd.c.4172: connect: async CONNECT FROM 192.168.84.211.63677 TO 192.168.86.11.13724 fd = 14 21:46:55.076 [9177] <2> logconnections: BPRD CONNECT FROM 192.168.84.211.63677 TO 192.168.86.11.13724 21:46:55.076 [9177] <2> vauth_authentication_required: vauth_comm.c.749: no methods for address: no authentication required 21:46:55.076 [9177] <2> vauth_connector: vauth_comm.c.182: no methods for address: no authentication required 21:46:55.076 [9177] <2> bprd_connect: no authentication required 21:46:55.076 [9177] <2> vnet_dlopen_vxss_client_magic: vnet_vxss.c.1766: Assuming no VxSS for DB Agents: 0 0x00000000 21:46:55.076 [9177] <2> bsa_bplist: start_date = Tue Sep 27 12:00:00 2011 21:46:55.077 [9177] <2> bsa_bplist: end_date = Thu Sep 29 12:00:00 2011 21:46:55.077 [9177] <2> bsa_bplist: Request = oraprod dbaprod st31bora01 st31bora01 st31bora01 ORA_PASPROD 7 pasprod 3 999 1317146400 1317319200 4 4 1 1 1 0 4 1017 2007 4 0 C C C C C 0 2 0 0 0 21:46:55.077 [9177] <4> bsa_bplist: Filepath = /PASPROD_c-1968089396-20110928-04 21:46:55.273 [9177] <2> dbc_get_string: Output = EXIT STATUS 131 21:46:55.273 [9177] <16> VxBSAQueryObject: ERR - dbc_get_string() failed 131 21:46:55.273 [9177] <2> xbsa_ProcessError: INF - entering 21:46:55.273 [9177] <2> xbsa_ProcessError: INF - leaving 21:46:55.273 [9177] <16> xbsa_QueryObject: ERR - VxBSAQueryObject: Failed with error: Server Status: client is not validated to use the server 21:46:55.273 [9177] <2> xbsa_QueryObject: INF - leaving (3) 21:46:55.273 [9177] <2> xbsa_EndTransaction: INF - entering 21:46:55.273 [9177] <4> VxBSAEndTxn: INF - entering VxBSAEndTxn. 21:46:55.273 [9177] <4> VxBSAEndTxn: INF - Transaction being COMMITED. 21:46:55.273 [9177] <4> VxBSAGetEnv: INF - entering GetEnv - NBBSA_LOG_DIRECTORY 21:46:55.273 [9177] <4> VxBSAGetEnv: INF - returning - dbclient 21:46:55.273 [9177] <4> VxBSAEndTxn: INF - Cleaning directory: </usr/openv/netbackup/logs/dbclient> 21:46:55.274 [9177] <4> delete_old_files: entering delete_old_files. 21:46:55.274 [9177] <2> xbsa_EndTransaction: INF - leaving (0) 21:46:55.274 [9177] <2> int_FindBackupImage: INF - leaving 21:46:55.274 [9177] <16> int_RemoveImage: ERR - Failed to remove, PASPROD_c-1968089396-20110928-04, from image catalog. 21:46:55.274 [9177] <2> int_RemoveImage: INF - leaving 21:46:55.274 [9177] <2> sbtremove2: INF - leaving 21:46:55.274 [9177] <2> sbterror: INF - entering 21:46:55.274 [9177] <2> sbterror: INF - Error=7501: Failed to remove, PASPROD_c-1968089396-20110928-04, from image catalog. . 21:46:55.274 [9177] <2> sbterror: INF - leaving
I suspect the message "client is not validated to use the server" indicates that I have a mis-configuration between the node names, the database virtual server name and the cluster name. Does anyone have any recommendations on where to look to resolve this?
Thanks
Ken
Solved! Go to Solution.
β09-29-2011 01:56 PM
I specified the name of the virtual Oracle DB server (st31bora01) within the NetBackup policy on the Administration Console and updated /usr/openv/netbackup/bp.conf on both nodes of the cluster with the following:
SERVER = inf-srv17.apacorp.net
CLIENT_NAME = st31bora01
REQUIRED_INTERFACE = st31bora01
This appears to have resolved the issue and autobackups of the DB SPFILEs now runs cleanly. Nonetheless, I have a ticket open with Symantec support to confirm this is the recommended configuration.
Ken
β09-29-2011 01:56 PM
I specified the name of the virtual Oracle DB server (st31bora01) within the NetBackup policy on the Administration Console and updated /usr/openv/netbackup/bp.conf on both nodes of the cluster with the following:
SERVER = inf-srv17.apacorp.net
CLIENT_NAME = st31bora01
REQUIRED_INTERFACE = st31bora01
This appears to have resolved the issue and autobackups of the DB SPFILEs now runs cleanly. Nonetheless, I have a ticket open with Symantec support to confirm this is the recommended configuration.
Ken
β09-29-2011 08:22 PM
Double-check the script - specify the Virtual name : NB_ORA_CLIENT=<virtual-name>
See this old but still valid TN: http://www.symantec.com/docs/TECH28412
β10-07-2011 12:35 PM
I tried following the paper you recommended and confirmed the NetBackup policy referenced the name of the cluster (st31bora01) and not either of the individual nodes. I also set /usr/openv/netbackup/bp.conf to reference the actual node name as follows:
SERVER = inf-srv17.apacorp.net
CLIENT_NAME = st31bbl01
REQUIRED_INTERFACE = st31bbl01
VERBOSE = 0
ENABLE_ROBUST_LOGGING = NO
I set set the database RMAN configuration as follows:
RMAN> show all; RMAN configuration parameters are: CONFIGURE RETENTION POLICY TO REDUNDANCY 14; CONFIGURE DEFAULT DEVICE TYPE TO 'SBT_TAPE'; CONFIGURE CONTROLFILE AUTOBACKUP ON; CONFIGURE CONTROLFILE AUTOBACKUP FORMAT FOR DEVICE TYPE 'SBT_TAPE' TO '%d_%F'; CONFIGURE DEVICE TYPE 'SBT_TAPE' PARALLELISM 4 BACKUP TYPE TO BACKUPSET; CONFIGURE CHANNEL 1 DEVICE TYPE 'SBT_TAPE' MAXOPENFILES 1 SEND 'NB_ORA_POLICY=ORA_PASPROD, NB_ORA_SCHED=Oracle_Online_Backup NB_ORA_CLIENT=st31bora01'; CONFIGURE CHANNEL 2 DEVICE TYPE 'SBT_TAPE' MAXOPENFILES 1 SEND 'NB_ORA_POLICY=ORA_PASPROD, NB_ORA_SCHED=Oracle_Online_Backup NB_ORA_CLIENT=st31bora01'; CONFIGURE CHANNEL 3 DEVICE TYPE 'SBT_TAPE' MAXOPENFILES 1 SEND 'NB_ORA_POLICY=ORA_PASPROD, NB_ORA_SCHED=Oracle_Online_Backup NB_ORA_CLIENT=st31bora01'; CONFIGURE CHANNEL 4 DEVICE TYPE 'SBT_TAPE' MAXOPENFILES 1 SEND 'NB_ORA_POLICY=ORA_PASPROD, NB_ORA_SCHED=Oracle_Online_Backup NB_ORA_CLIENT=st31bora01'; <default settings removed for brevity>
Unfortunately, a test to run an archive log backup gives the following error:
RMAN-03009: failure of backup command on ORA_SBT_TAPE_1 channel at 10/07/2011 12:00:27 ORA-19506: failed to create sequential file, name="PASPROD_al26963_20111007_85945", parms="" ORA-27028: skgfqcre: sbtbackup returned error ORA-19511: Error received from media manager layer, error text: VxBSACreateObject: Failed with error: Server Status: the specified policy is not of the correct client type channel ORA_SBT_TAPE_1 disabled, job failed on it will be run on another channel RMAN-03009: failure of backup command on ORA_SBT_TAPE_2 channel at 10/07/2011 12:00:27 ORA-19506: failed to create sequential file, name="PASPROD_al26957_20111007_85946", parms="" ORA-27028: skgfqcre: sbtbackup returned error ORA-19511: Error received from media manager layer, error text: VxBSACreateObject: Failed with error: Server Status: the specified policy is not of the correct client type channel ORA_SBT_TAPE_2 disabled, job failed on it will be run on another channel RMAN-03009: failure of backup command on ORA_SBT_TAPE_3 channel at 10/07/2011 12:00:27 ORA-19506: failed to create sequential file, name="PASPROD_al26960_20111007_85947", parms="" ORA-27028: skgfqcre: sbtbackup returned error ORA-19511: Error received from media manager layer, error text: VxBSACreateObject: Failed with error: Server Status: the specified policy is not of the correct client type channel ORA_SBT_TAPE_3 disabled, job failed on it will be run on another channel RMAN-00571: =========================================================== RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS =============== RMAN-00571: =========================================================== RMAN-03009: failure of backup command on ORA_SBT_TAPE_4 channel at 10/07/2011 12:00:27 ORA-19506: failed to create sequential file, name="PASPROD_al26959_20111007_85948", parms="" ORA-27028: skgfqcre: sbtbackup returned error ORA-19511: Error received from media manager layer, error text: VxBSACreateObject: Failed with error: Server Status: the specified policy is not of the correct client type
Removing the REQUIRED_INTERFACE line from bp.conf not make any difference. It appears from my testing that TECH28412 is obsolete. I have put the name of the cluster back in bp.conf.
Ken