Forum Discussion

rocker65's avatar
rocker65
Level 4
14 years ago

listxattr() failed message

Hello all.

 

NB 7 server (Windows), NB 6.5 Linux client (32 bit). I have one schedule that handles only NFS shares, none NDMP. The schedule always fails with this error message in the client bpbkar log file:

listxattr() failed on '/qtrees/nfs_storage/'. Errno = 95: Operation not supported

 

I can't seem to find any information on this listxattr() call that is failing. Can someone shed some light on this?

 

Many thanks

7.1.x and Earlier
Backup and Recovery
Configuring
Error messages
NetBackup
Troubleshooting
  • Nicolai's avatar
    Nicolai
    14 years ago

    If the NFS share has huge amount of small files the backup may die because of timeouts. Try to run the backup of the nfs share manual like this:

    /usr/openv/netbackup/bin/bpbkar  -nocont -nofileinfo -nokeepalives  /qtrees/nfs_storage/ > /dev/null 2> /tmp/file.out 

    This command will read all files but don't send the data to the Netbackup Master/media server.

    Remember to add VERBOSE to bp.conf and create the touch file bpbkar_path_tr on the client in /usr/openv/netbackup. Inspect the log in /usr/openv/netbackup/logs/bpbkar/ while the command is running. It should list every files it reads. If the backup stop, re-run to see if it stop on the same path.

    Just watch out for thesapce used by the bpbkar debug log. It will grow very quick.

10 Replies

Sort By
  • Marianne's avatar
    Marianne
    Level 6
    14 years ago

    Long shot, but it could be 'Extended Attributes'. The message says to me that Extended attributes cannot be listed ?

    Does this cause the backup to fail, or is just a 'notice' in the log file?

    What is the 'key value' of the entry in bpbkar?

    <2> = debug
    <4> = info
    <8> = warning
    <16> = error
    <32> = severe error

    Anything other than <16> and <32> can be ignored.

     

    Have a look at this TN for explanation of Extended Attributes. (No idea why Windows is referenced in the title - it has nothing to do with Windows.)

    http://www.symantec.com/docs/HOWTO34729

     

    You could try to create the IGNORE_XATTR touch-file, but it seems that it will only apply to restores...

  • rocker65's avatar
    rocker65
    Level 4
    14 years ago

    Here's the whole entry

    11:55:31.396 [15141] <4> bpbkar: INF - BACKUP START
    11:55:31.396 [15141] <4> bpbkar: INF - Estimate:-1 -1
    11:55:31.399 [15141] <2> bpbkar add_to_filelist: starting sizeof(filelistrec) <68>
    11:55:31.399 [15141] <4> bpbkar: INF - Processing /qtrees/nfs_storage
    11:55:31.439 [15141] <4> bpbkar: INF - listxattr() failed on '/qtrees/nfs_storage/'. Errno = 95: Operation not supported
    12:17:24.475 [15141] <16> bpbkar: ERR - Cannot write to STDOUT. Errno = 104: Connection reset by peer
    12:17:24.690 [15141] <16> bpbkar: ERR - bpbkar FATAL exit status = 24: socket write failed
    12:17:24.691 [15141] <4> bpbkar: INF - EXIT STATUS 24: socket write failed
    12:17:24.691 [15141] <4> bpbkar: INF - setenv FINISHED=0

     

    Looking at the entry above, I assumed the listxattr failed causing the exit status 24... but maybe it's not the case because the listxattr() error is an info leve message. <didn't know about that>, quite handy.  So I guess maybe that isn't my problem?

    Can you point me to a document to enable MPX on the storage unit? I can't find one anywhere, all I can find is documents that explain what it does. Sighh..

     

    Many thanks

     

    Re. Windows, it's a windows server, linux client.

  • Nicolai's avatar
    Nicolai
    Moderator
    14 years ago

    See details for listxattr here : http://linux.die.net/man/2/listxattr

    My guess is that ext3 file system is assumed in the backup, but the NFS protocol does support that system call. If you enable bpbkar logging as Marianne suggest you should be able to see what filesystem type Netbackup detects.

  • Marianne's avatar
    Marianne
    Level 6
    14 years ago

    It could very well be causing your problem, since nothing is happening after that entry. It seems that bpbkar stops processing files when it hits this entry, resulting in a timeout 20 minutes later.

    11:55:31
    12:17:24

    Your Client_Read_Timeout is probably something like 20 minutes?

    I have Googled "Errno = 95: Operation not supported". Lots of hits on various Linux forums. This one suggests that it has something to do with filesystem support: http://www.fsarchiver.org/forums/viewtopic.php?f=14&t=1051

    What type of filesystem is this?

    You might be able to get more info if you increase logging level on the client.

    Add this line to client's bp.conf:

    VERBOSE = 5

     

    About your second question:

    It's documented in NBU Admin Guide I: http://www.symantec.com/docs/TECH127079

    p. 369: Changing storage unit settings
    p. 385: Maximum streams per drive setting

    All NBU 7 manuals: http://www.symantec.com/docs/TECH126327

  • rocker65's avatar
    rocker65
    Level 4
    14 years ago

    /qtree's filesystem is plain NFS. Big filesystem with 111GB of smaller files... so I fully expected it to just work away at its own speed.

    I have 2 schedules / policy to handle NFS and it works fine on the same system for another huge NFS share. But this one which comes from the same NetApp head seems to fail.

    The system in question is ext3, but the as I said, the filesystem being backed up is NFS. My bp.conf has:

    SERVER = zyzserver
    CLIENT_NAME = xyz
    ENABLE_ROBUST_LOGGING = YES

     

    Should I add the verbose flag? Seems like the enable_robust_logging way overrides the verbose flag. Will try.

  • Marianne's avatar
    Marianne
    Level 6
    14 years ago

    What is the filesystem type on the source server (NetApp)?

    Anything different about this filestem or its attributes on the filer?

  • Nicolai's avatar
    Nicolai
    Moderator
    14 years ago

    If the NFS share has huge amount of small files the backup may die because of timeouts. Try to run the backup of the nfs share manual like this:

    /usr/openv/netbackup/bin/bpbkar  -nocont -nofileinfo -nokeepalives  /qtrees/nfs_storage/ > /dev/null 2> /tmp/file.out 

    This command will read all files but don't send the data to the Netbackup Master/media server.

    Remember to add VERBOSE to bp.conf and create the touch file bpbkar_path_tr on the client in /usr/openv/netbackup. Inspect the log in /usr/openv/netbackup/logs/bpbkar/ while the command is running. It should list every files it reads. If the backup stop, re-run to see if it stop on the same path.

    Just watch out for thesapce used by the bpbkar debug log. It will grow very quick.

  • rocker65's avatar
    rocker65
    Level 4
    14 years ago

    Just to update you with where I am with this problem. Marianne, your orignal statement about the lisxattr() being informational was correct.

    The backups are working fine now. Only thing I changed was I downed one of the tape drives which was having repeated problems which I believe to be caused by NB having troubles seeing the tape unit as it was visible with multiple paths on the SAN. Not sure why because I don't have it configured that way, and other drives in the same HP MSL-6K silo are visible only on one path.... argh. So I downed it to remove confustion and with the (THANKYOU) guidance of Marianne again I figured out how to up the mpx setting on the storage unit. So now the drive is in a Drive Control: RESTART  mode, whatever that is. Will read on.

     

    Thank you for your help.laugh

     

    PS. I worked on Netbackup when it was owned by Open data then Openvision, early 90's. It's changed quite a bit. I guess I'm dating myself here.   #8D

  • Marianne's avatar
    Marianne
    Level 6
    14 years ago

    Oh My!!! I thought I was the Community 'granny'! I've worked with NBU since 3.1.1 (1999) - Veritas days...

  • rocker65's avatar
    rocker65
    Level 4
    14 years ago

    Honestly don't even remember the versions, but it worked really well on a Sparc 10 server. Early 90's, used to write to 1/4" cartridges as well as small exabyte libraries which were state of the art back then. If I remember, there were 4 & 8mm drives. Always unix based so that is why I am having so much trouble with the windows media server. Storage units used to always be single ended scsi / acs attached. But nowadays almost always fibre, which is fine. Just different, and a hell of a lot faster.

    </EndOfNostalgia>