β10-11-2012 03:58 AM
HI, please help a Tech in distress. My full backup of some Oracle db ends with 6. so again please advice.. the dbclient log (verbose 2) says nothing to help.
channel ch00: starting piece 1 at 11-OCT-12
channel ch02: finished piece 1 at 11-OCT-12
piece handle=bk_34653_1_796353122 tag=HOT_DB_BK_LEVEL0 comment=API Version 2.0,MMS Version 5.0.0.0
channel ch02: backup set complete, elapsed time: 01:19:53
channel ch02: starting incremental level 0 datafile backup set
channel ch02: specifying datafile(s) in backup set
input datafile file number=00029 name=/db/durtvaORA/w/index/prod_idxlogg_w_1_1.dbf
input datafile file number=00054 name=/db/durtvaORA/x/index/prod_idxlogg_x_1_1.dbf
channel ch02: starting piece 1 at 11-OCT-12
RMAN-03009: failure of backup command on ch02 channel at 10/11/2012 02:49:12
ORA-27192: skgfcls: sbtclose2 returned error - failed to close file
ORA-19511: Error received from media manager layer, error text:
VxBSAEndTxn: Failed with error:
The transaction was aborted.
continuing other job steps, job failed will not be re-run
channel ch03: finished piece 1 at 11-OCT-12
piece handle=bk_34654_1_796354438 tag=HOT_DB_BK_LEVEL0 comment=API Version 2.0,MMS Version 5.0.0.0
channel ch03: backup set complete, elapsed time: 01:22:18
channel ch01: finished piece 1 at 11-OCT-12
piece handle=bk_34655_1_796354584 tag=HOT_DB_BK_LEVEL0 comment=API Version 2.0,MMS Version 5.0.0.0
channel ch01: backup set complete, elapsed time: 01:24:03
channel ch00: finished piece 1 at 11-OCT-12
piece handle=bk_34656_1_796357240 tag=HOT_DB_BK_LEVEL0 comment=API Version 2.0,MMS Version 5.0.0.0
channel ch00: backup set complete, elapsed time: 00:40:26
released channel: ch00
released channel: ch01
released channel: ch02
released channel: ch03
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03009: failure of backup command on ch02 channel at 10/11/2012 02:49:12
ORA-27192: skgfcls: sbtclose2 returned error - failed to close file
ORA-19511: Error received from media manager layer, error text:
VxBSAEndTxn: Failed with error:
The transaction was aborted.
RMAN-03009: failure of backup command on ch03 channel at 10/10/2012 19:12:30
ORA-27192: skgfcls: sbtclose2 returned error - failed to close file
ORA-19511: Error received from media manager layer, error text:
VxBSAEndTxn: Failed with error:
The transaction was aborted.
RMAN-03009: failure of backup command on ch02 channel at 10/10/2012 19:17:11
ORA-27192: skgfcls: sbtclose2 returned error - failed to close file
ORA-19511: Error received from media manager layer, error text:
VxBSAEndTxn: Failed with error:
The transaction was aborted.
RMAN-03009: failure of backup command on ch01 channel at 10/10/2012 19:16:16
ORA-27192: skgfcls: sbtclose2 returned error - failed to close file
ORA-19511: Error received from media manager layer, error text:
VxBSAEndTxn: Failed with error:
The transaction was aborted.
RMAN> RMAN>
Recovery Manager complete.
Script /db/durtvaORA/sid1/etc/netbackup/hot_db_backup_durp01.sh
==== ended in error on Thu Oct 11 03:01:13 CEST 2012 ====
Hot_Database_Backup.sh.out lines 309-378/378 (END)
Solved! Go to Solution.
β10-15-2012 10:00 PM
Except the NetBackup client version I mentioned before, there are no configuration issue here.
Have you already enable debug logging and log a case with Symantec?
To identify what's wrong, debug logs of various processes are needed. For example, similar issues are reported as below. These case require various debug logs.
http://www.symantec.com/docs/TECH189853
http://www.symantec.com/docs/TECH73130
β10-11-2012 05:43 AM
The master & media are Linux and we back it up to a basic DISK SU
β10-11-2012 07:59 AM
review this technote:
Article URL http://www.symantec.com/docs/TECH189853
The oracle.exe process serves multiple purposes. Each RMAN channel is a thread of oracle.exe, as are any extjob.exe tasks that are run from the SYS schema.
Consequently, any extjob.exe task that starts while dbclient is connected to the media server processes will inherit copies of the file descriptors for the sockets. If the extjob.exe task has not yet exited when dbclient closes it's copies of the file descriptors, the client host TCP stack will not send the TCP FINs as expected and the media server will be unaware that the backup is complete because the sockets have not yet closed.
If the extjob.exe completes within 15 minutes, a disordly shutdown of the sockets will occur (because extjob.exe does not know the sockets exist and hence can't perform an orderly close) resulting in the TCP RESETs.
If the extjob.exe does not complete for at least 15 minutes, then dbclient will timeout as noted above.
The behaviors documented above are not explicitly addressed within the Oracle SBT API and neither Oracle nor Symantec have a code level change to fix this situation at the current time. However, several workarounds are available.
1) Use a scheduler other than extjob.exe for the conflicting tasks.
2) Schedule the external tasks to initiate at a time when RMAN backups are not running.
3) Configure the external tasks to run from an Oracle schema other than SYS. When run from a non-SYS schema, the extjob.exe process will not be spawned directly from oracle.exe and will not conflict with the backup. When run from the SYS schema, the thread is spawned using the parent oracle.exe process which is required to also be used by RMAN for the backups.
Symantec Corporation has acknowledged that the above-mentioned issue (ETrack 2717058) is present in the current version(s) of the product(s) mentioned in this article. Symantec Corporation is committed to product quality and satisfied customers. This issue was scheduled to be addressed in a future maintenance release or update of the product.
β10-11-2012 08:18 AM
Hello
Are you using Windows 2008?
If yes, you need to add your netbackup user to the ora_dba group.
For more info check the oracle logs on drive:Program Files\Veritas\NetBackup\logs\user_ops\dbext\oracle
β10-11-2012 05:32 PM
Please provide more detail about your configuration like OS name and version, Oracle version, NetBackup version or so.
β10-12-2012 01:08 AM
Please tell us minor version of Oracle like 11.2.0.x. Tell us whether this is newly configured backup, or existing working backup as well.
BTW, your configuration seems irregal, version of media server is higher than master server.
Master server must have most highest(or equal) version.
β10-12-2012 01:14 AM
Master&media&client = SLES10sp4
db = Oracle 11g
NBU 7.1.0.4 media server
NBU 7.0.1 Client
NBU 7.1.0.3 Master server
β10-12-2012 06:25 AM
The Oracle is 11.2.0.1 and Yes we know that it is not standard to have master with a lower version.
but our local symantec office told us that in 7.1 it not a must to have master with a greater version than media servers and clients.
Yes this is a working policy. it just has a great tendency to fail full backups, diff works just fine
β10-15-2012 10:00 PM
Except the NetBackup client version I mentioned before, there are no configuration issue here.
Have you already enable debug logging and log a case with Symantec?
To identify what's wrong, debug logs of various processes are needed. For example, similar issues are reported as below. These case require various debug logs.
http://www.symantec.com/docs/TECH189853
http://www.symantec.com/docs/TECH73130
β10-17-2012 04:06 AM
Thanks I will look into it.
β11-09-2012 05:05 AM
Mine was Windows :
Edited the SQLNET.AUTHENTICATION_SERVICES line and set it to NTS in the file on oracle server as below :- C:\oracle\product\10.2.0\db_1\network\ADMIN\sqlnet.ora
SQLNET.AUTHENTICATION_SERVICES= (NTS)
NAMES.DIRECTORY_PATH= (TNSNAMES, EZCONNECT)
This solved my isse. This may help you.
β11-13-2012 02:35 AM
The media server is in advance of the master server re: NB version, thats a nono, as Yasuhisa says. Pick a different media server, or upgrade it. Give us some history...new , modified recently,...
Jim
β11-14-2012 07:45 AM
I am haveing the same issue. NBU master/media version 7.1 on Solaris 10 and NBU client version 6.5.6 on Solaris 8.The issue is confirmed in the tech notes http://www.symantec.com/docs/TECH189853. We have all ready implemented the change from Solution 3 by creating a new username in the DB and adding it to the RMAN backup script but no resolve. How do you implement the solution 1? My DBA's are unclear on exactly what needs to be changed on Oracle 9i.
The behaviors documented above are not explicitly addressed within the Oracle SBT API and neither Oracle nor Symantec have a code level change to fix this situation at the current time. However, several workarounds are available.
1) Use a scheduler other than extjob.exe for the conflicting tasks.
2) Schedule the external tasks to initiate at a time when RMAN backups are not running.
3) Configure the external tasks to run from an Oracle schema other than SYS. When run from a non-SYS schema, the extjob.exe process will not be spawned directly from oracle.exe and will not conflict with the backup. When run from the SYS schema, the thread is spawned using the parent oracle.exe process which is required to also be used by RMAN for the backups.
β11-14-2012 09:19 AM
@jmorehea per http://www.softpanorama.org/Admin/Job_schedulers/oracle_scheduler.shtml
There are two interfaces for the Oracle Scheduler:
so the suggestion is to not use these features, or use them only when RMAN job is not active
β11-14-2012 09:35 AM
Just noticed that this tech document is for Windows http://www.symantec.com/business/support/index?page=content&id=TECH189853. The Unix document is
http://www.symantec.com/business/support/index?page=content&id=TECH73130 and the work around does not not apply to us since it refers to ASM. Has anyone else had this issue on the Solaris environment.