Client version 6.5.6 on the server - bp.conf has IGNORE_XATTR =

[Netbackup_fan]

Hi experts.

I need help here.  the System Admin installed client 6.5.6, also the Sys Admin modified the bp.conf to add the following entries:

I notice that the bp.con has the following ownership, permissions:

-rw-r--r--   1 root     root         244 Jan 23 16:44 bp.conf

SERVER = NB-master

SERVER = NB-media

SERVER = timefinder

SERVER = amdocstest

CLIENT_NAME = centurion

ALLOW_MEDIA_OVERWRITE = TAR

ALLOW_MEDIA_OVERWRITE = ANSI

IGNORE_XATTR = YES

The backup is giving the following errors:

04:04:31 WRN - /export/home/oracle/dba/exp_pipe.dmp is a fifo special file. Back
ing up the raw fifo.
05:04:35 INF - Server status = 13
05:04:37 INF - Backup by oracle on client centurion using policy Database_Cold,
sched Logs_One_Year:file read failed

[Yasuhisa_Ishika]

This error was caused by 'file read failed'. IGNORE_XATTR is not related.
You should check Troubleshooting Guide, and follow Recommented Actions for status code 13. You might need to enable debug logging of bpbkar(on clinet), bptm and bpbrm(both on media server).

And, can you tell us what OS are you using on each host?

[Nicolai]

A fifo files does not contain data, is a pipe between two processes.

http://linux.about.com/library/cmd/blcmdl4_fifo.htm

This technote would be my suggestion, the code 13 may be related to bpbkar hang somwhere in the file system:

Usage specifics of the bpbkar_path_tr touch file to enable enhanced debug logging of the bpbkar process

http://www.symantec.com/docs/TECH31513

[Omar_Villa]

1. create bpcd and bpbkar logs, increase verbose to 5, upgrade to the latest version of NBU 2. touch /usr/openv/netbackup/bpbkar_path_tr # This will add "PrintFile" lines to the log

[MN_Pankaj]

Good responses above...

As mentioned above, these three are independent of each other:

1.   IGNORE_XATTR

        This is set in bp.conf to NOT backup any Extended Attributes. A customer may decide to do that if he already knows that he may later want to restore that data to a different OS or a different file-system. Extended Attributes may not be compatible across different OS or file-system. However, as expected, this is a metadata data-loss situation - only ok if the System Admin knows what he is doing. Even if we allow backup of Extended Attributes (which is the default), and the customer later decides to restore that backup to a different OS or filesystem, where those Extended Attributes are not compatible, we have a touchfile /usr/openv/netbackup/IGNORE_XATTR that is checked during restore to not even attempt to restore those Extended Attributes. However, as mentioned, check with the System Admin as to why he had bp.conf IGNORE_XATTR for backup - he may have a valid reason for doing that at this time (like doing Cold Oracle backup without using NBU Oracle Agent, in this case)

2.   WRN - /export/home/oracle/dba/exp_pipe.dmp is a fifo special file

        This shows that the customer has /export/home/oracle/dba/exp_pipe.dmp explicitly specified in the policy filelist, presumably to initiate a cold oracle backup, as clear from the policy name "Database_Cold"

3.   Backup by oracle on client centurion using policy Database_Cold, sched Logs_One_Year:file read failed

      This shows that the customer is trying to do a Oracle DB backup the cheap way - without NBU Oracle Agent. NBU generally doesn't support that, as it requires lots of prep work - like in bpstart and bpend notify scripts, etc. Check with the System Admin if he was ever able to get this to work on this system. Consult http://www.symantec.com/docs/TECH89805 for more information.

[Netbackup_fan]

The exlude_list I see is from root.

I'am oracle, how to have an exclude_list for me, for my backups?

root     other         86 Nov 24  2009 exclude_list

[Will_Restore]

is per client (or per client.policy), not per user

[Netbackup_fan]

But where do I put it, where do I include this exclude of this pipe.  How I will name the file?, where to configure this?

[Marianne]

Please explain to us how this Oracle backup is done.

A cold backup is more than just a File List / Backup Selection in a NetBackup Policy.

Are there any manual steps performed by the dba? Any scripts involved?  bpstart_ and bpend_notify scripts?
Please also share the Backup Selection in the Policy.

We need the step-by-step process in order to understand the 'big picture'.

Only when we fully understand your unique situation can we provide meaningful advice.

You can simply add this file to exclude_list exacly as-is:

/export/home/oracle/dba/exp_pipe.dmp

Another option is to create an exclude list for this specific policy:

exclude_list.Database_Cold 
/export/home/oracle/dba/exp_pipe.dmp

Here are 2 TNs for a better understanding of exclude lists:

http://www.symantec.com/docs/TECH32585

http://www.symantec.com/docs/TECH30603

I somehow doubt that this file is causing the status 13.
Please ensure that all of these log folders exist to troubleshoot the status 13:

On media server: bpbrm and bptm
On client: bpbkar

When you have a full set of logs after the next failure, please rename the logs to reflect the process name (bpbrm.txt, etc) and upload them as attachments.

[Netbackup_fan]

I created the exclude list as followed but the backup of only two files including the exp_pipe.dmp failed with the same status 13:

rw-rw-r--   1 root     other         37 Feb 19 05:46 exclude_list.Database_Cold
racle@centurion[ctimsd1]<deadbeef>
usr/openv/netbackup>more exclude_list.Database_Cold
export/home/oracle/dba/exp_pipe.dmp
racle@centurion[ctimsd1]<deadbeef>
usr/openv/netbackup>

This is the backup log:

Backup started 02/21/2012 13:31:13

13:31:14 Initiating backup
13:31:19 INF - Starting bpbrm
13:31:21 WRN - NetBackup configuration flag IGNORE_XATTR set for backup operatio
n.
13:31:21 INF - Processing /oradb/oravol01/exp_db_ctimsd1.log
13:31:21 /
13:31:21 /oradb/
13:31:21 /oradb/oravol01/
13:31:22 /oradb/oravol01/exp_db_ctimsd1.log
13:31:22 INF - Processing /oradb/oravol01/exp_pipe.dmp
13:31:22 /oradb/oravol01/
13:31:22 WRN - /oradb/oravol01/exp_pipe.dmp is a fifo special file. Backing up t
he raw fifo.
13:31:25 INF - Waiting for mount of media id PJR213 on server NB-master for writ
ing
13:31:32 INF - Mounted media id PJR213
13:31:32 INF - Beginning backup on server NB-master of client centurion
14:31:35 INF - Server status = 13
14:31:36 INF - Backup by oracle on client centurion using policy Database_Cold,
sched Logs_One_Year:file read failed

This is the script LOG_YEAR.sh:

/export/home/oracle/dba/scripts>more LOG_YEAR.sh
#! /bin/ksh

if [ "$#" -ne 1 ]
then
        echo "Usage : COLD_BACKUP.sh <ORACLE_SID>"
        exit 1
fi

/usr/openv/netbackup/bin/bpbackup -p Database_Cold -s Logs_One_Year -L /export/h
ome/oracle/dba/logs/LOGS.`date '+%Y-%m-%d'`.log -f /export/home/oracle/dba/lists
/LOGSlist
oracle@centurion[ctimsd1]<deadbeef>
/export/home/oracle/dba/scripts>

This is the list  LOGSlist:

-rwxr-xr-x   1 oracle   dba           64 Feb 21 13:30 LOGSlist
oracle@centurion[ctimsd1]<deadbeef>
/export/home/oracle/dba/lists>more LOGSlist
/oradb/oravol01/exp_db_ctimsd1.log
/oradb/oravol01/exp_pipe.dmp
oracle@centurion[ctimsd1]<deadbeef>
/export/home/oracle/dba/lists>

The following is the log from the Netackup console: - activity Monitor

Feb 21, 2012 1:40:23 PM - Warning bpbrm (pid=29692) from client centurion: WRN - NetBackup configuration flag IGNORE_XATTR set for backup operation.
Feb 21, 2012 1:40:23 PM - Warning bpbrm (pid=29692) from client centurion: WRN - /oradb/oravol01/exp_pipe.dmp is a fifo special file. Backing up the raw fifo.
Feb 21, 2012 1:40:19 PM - requesting resource Small_Tape_Storage_Group
Feb 21, 2012 1:40:19 PM - requesting resource NB-master.NBU_CLIENT.MAXJOBS.centurion
Feb 21, 2012 1:40:19 PM - requesting resource NB-master.NBU_POLICY.MAXJOBS.Database_Cold
Feb 21, 2012 1:40:20 PM - granted resource  NB-master.NBU_CLIENT.MAXJOBS.centurion
Feb 21, 2012 1:40:20 PM - granted resource  NB-master.NBU_POLICY.MAXJOBS.Database_Cold
Feb 21, 2012 1:40:20 PM - granted resource  PJR213
Feb 21, 2012 1:40:20 PM - granted resource  NB-master_Robot_003_Drive_18_SN-1250864358
Feb 21, 2012 1:40:20 PM - granted resource  NB-master-hcart-robot-tld-3
Feb 21, 2012 1:40:20 PM - estimated 0 kbytes needed
Feb 21, 2012 1:40:21 PM - started process bpbrm (pid=29692)
Feb 21, 2012 1:40:22 PM - connecting
Feb 21, 2012 1:40:23 PM - connected; connect time: 0:00:00
Feb 21, 2012 1:40:27 PM - mounting PJR213
Feb 21, 2012 1:40:33 PM - mounted PJR213; mount time: 0:00:06
Feb 21, 2012 1:40:33 PM - positioning PJR213 to file 5
Feb 21, 2012 1:40:33 PM - positioned PJR213; position time: 0:00:00
Feb 21, 2012 1:40:33 PM - begin writing
Feb 21, 2012 2:40:34 PM - Error bpbrm (pid=29692) socket read failed: errno = 62 - Timer expired
Feb 21, 2012 2:40:36 PM - Error bptm (pid=29705) media manager terminated by parent process
Feb 21, 2012 2:40:36 PM - end writing; write time: 1:00:03
file read failed  (13)

[Marianne]

Status 13 in this instance is a timeout - not a file read error:

Feb 21, 2012 1:40:33 PM - begin writing
Feb 21, 2012 2:40:34 PM - Error bpbrm (pid=29692) socket read failed: errno = 62 - Timer expired

You will need bpbkar log plus the log file produced by bpbackup on the client as well as bpbrm and bptm logs on the media server to see what is happening (or not!) in the hour between 1:40 and 2:40.

[Will_Restore]

13:31:22 WRN - /oradb/oravol01/exp_pipe.dmp is a fifo special file. Backing up the raw fifo.

your exclude_list has this path:

/export/home/oracle/dba/exp_pipe.dmp

how about fixing the path in the exclude_list or changing it to just

exp_pipe.dmp

[Marianne]

/oradb/oravol01/exp_pipe.dmp is explicitly listed in the backup file list. It makes no sense to exclude it again.

The message in the job details is not an error - it is a NOTICE saying that a raw backup will be done.

[Will_Restore]

thought he wanted to exclude the pipe

[Netbackup_fan]

I'am doint test to see if the exclude_list will exclude the exp_pipe.dmp though I have it listed in the list.

Because in the original list it is inside a filesystem that I want to back up.

I have modified the exclude list to include exp_pipe.dmp (alone)

[Marianne]

Exit status 13 is NOT because of exp_pipe.dmp. A notice/warning is displayed that the file will be backed up as raw, and then the backup carries on.

Then, for 1 hour exactly NOTHING seems to happen, resulting in the timeout.

You need the logs that I mentioned above to see what is happening (or NOT happening) during this period.

Please also note the following:

"In NetBackup, exclude and include lists are only supposed to be used by automatic backups and not by user backups."  (It was considered as bug when a certain NBU version applied exclude lists to User backups: http://www.symantec.com/docs/TECH137767 )

[Netbackup_fan]

Hi.

To inform that I changed the contents of the exclude_list.Database_Cold to reflect only exp_pipe.dmp and it failed also.

I thought that even though I'am listing the file it would be excluded from being backup.

-rw-rw-r--   1 root     other         13 Feb 22 12:36 exclude_list.Database_Cold
oracle@centurion[ctimsd1]<deadbeef>
/usr/openv/netbackup>more exclude_list.Database_Cold
exp_pipe.dmp
oracle@centurion[ctimsd1]<deadbeef>
/usr/openv/netbackup>

Feb 22, 2012 12:46:17 PM - Warning bpbrm (pid=18641) from client centurion: WRN - NetBackup configuration flag IGNORE_XATTR set for backup operation.
Feb 22, 2012 12:46:18 PM - Warning bpbrm (pid=18641) from client centurion: WRN - /oradb/oravol01/exp_pipe.dmp is a fifo special file. Backing up the raw fifo.
Feb 22, 2012 12:46:13 PM - requesting resource Small_Tape_Storage_Group
Feb 22, 2012 12:46:13 PM - requesting resource NB-master.NBU_CLIENT.MAXJOBS.centurion
Feb 22, 2012 12:46:13 PM - requesting resource NB-master.NBU_POLICY.MAXJOBS.Database_Cold
Feb 22, 2012 12:46:14 PM - granted resource  NB-master.NBU_CLIENT.MAXJOBS.centurion
Feb 22, 2012 12:46:14 PM - granted resource  NB-master.NBU_POLICY.MAXJOBS.Database_Cold
Feb 22, 2012 12:46:14 PM - granted resource  PJR213
Feb 22, 2012 12:46:14 PM - granted resource  NB-master_Robot_003_Drive_22_SN-1250864362
Feb 22, 2012 12:46:14 PM - granted resource  NB-master-hcart-robot-tld-3
Feb 22, 2012 12:46:14 PM - estimated 0 kbytes needed
Feb 22, 2012 12:46:15 PM - started process bpbrm (pid=18641)
Feb 22, 2012 12:46:16 PM - connecting
Feb 22, 2012 12:46:17 PM - connected; connect time: 0:00:00
Feb 22, 2012 12:46:21 PM - mounting PJR213
Feb 22, 2012 12:46:28 PM - mounted PJR213; mount time: 0:00:07
Feb 22, 2012 12:46:28 PM - positioning PJR213 to file 6
Feb 22, 2012 12:46:28 PM - positioned PJR213; position time: 0:00:00
Feb 22, 2012 12:46:28 PM - begin writing
Feb 22, 2012 1:46:28 PM - Error bpbrm (pid=18641) socket read failed: errno = 62 - Timer expired
Feb 22, 2012 1:46:30 PM - Error bptm (pid=18646) media manager terminated by parent process
Feb 22, 2012 1:46:31 PM - end writing; write time: 1:00:03
file read failed  (13)

[Will_Restore]

that confirms Marianne's post immediately above

[Marianne]

You can clearly see that the job TIIMES OUT after exactly one hour:

Feb 22, 2012 12:46:28 PM - begin writing
Feb 22, 2012 1:46:28 PM - Error bpbrm (pid=18641) socket read failed: errno = 62 - Timer expired

You need the logs to see if data is sent from the client and if any is received by the media server.

I have been trying to tell you that for 2 days now....

[Nicolai]

I have described step in this blog of how to enable bpbkar debug log. 

http://www.mass.dk/netbackup/quick-hints/68-netbackup-client-hang-on-hp-ux-1131.html

This should lead to which file or catalog that cause the status 13 errors. 

Replace this command :

/usr/openv/netbackup/bin/bpbkar -nocont /dev > /dev/null

with 

/usr/openv/netbackup/bin/bpbkar -nocont  /oradb/ > /dev/null

TN: Usage specifics of the bpbkar_path_tr touch file to enable enhanced debug logging of the bpbkar process

http://www.symantec.com/business/support/index?page=content&id=TECH31513