bpbkar read_and_sort_dir_entries: WRN - Cannot lstat

[dulla]

We used to back up the file systems / and /opt on our solaris 10 netbackup clients until recently. The back up fails every time I take the backup

with status 13  file read failed. From the bpbkar log I was able to see the error below

read_and_sort_dir_entries: WRN - Cannot lstat some file name Error = 2 there is no such file

after this error the last one read the network link is brocken always displayed but I dont think there is a network problem because on the same machine the other policies( oracle policies) worked with out any problem. Any of your help will be appreciated

[Yasuhisa_Ishika]

This messages means lstat() call to the file returns ENOENT. Please check file by "ls -l file_name" first.
And also check file name(s) reported are always same or not, and any clue recorded in syslog(/var/adm/messages).

This message is warning level, so this does not cause status 13 error. You should set  higher logging level(VERBOSE = 5 in bp.conf), and reproduce this error.

[dulla]

thanks for the quick reply yasuhisa. I tried to reproduce the error by manually backing up the system but the back up finished successfully in the 2nd retry, and I cant generate the log. There is nothing reported in /var/adm/messages. I will wait till tomorrow the policy to run with the schedule and post u the log.

[Yasuhisa_Ishika]

If you can not repuroduce this error, it might be timing issue. I suppose this error occured in senario like below.

1. NetBackup read directory entry, and check what files and directories reside in this directory.
2. NetBackup check existence of the file in question by calling access()
3. The file in question is deleted by some other applications or administrator's operations.
4. NetBackup try get metadata like last modified time, access time, permissions, onwer, etc. by calling lstat(), but the file in question have been already deleted in step 3.So lstat() returns ENOENT.

[MN_Pankaj]

Good responses above, by Yasuhisa. As he mentions, more details are needed...

W.R.T. "until recently" in the orginal question, did NBU get upgraded or reconfigured recently on this system...?

The verbose "read_and_sort_dir_entries: WRN - Cannot lstat some file name Error = 2 there is no such file" by itself is NOT an issue, as the backup logs that as a WRN message, and reports it as a TRV message to NBU server, and moves on to the next file, WITHOUT incrementing the error count. What that "some file name" is would probably help somewhat, and also if there is more than one file that reports that verbose.

W.R.T. "after this error the last one read the network link is broken always displayed", good to have that "some file name" to see what happened, even though you mention that the backup is of "file systems / and /opt" - both of which are presumably local filesystems.

[dulla]

actually when the policy run at night the error appeared and after I set the VERBOSE=5, here is the out put of bpbkar at the time of failure

PrintFile: /opt/s2100/om/
18:56:34.908 [24212] <8> bpbkar read_and_sort_dir_entries: WRN - Cannot lstat PMData_5min.xml.1302826861109. Errno = 2: No such file or directory
19:00:32.769 [24212] <16> bpbkar sighandler: ERR - bpbkar killed by SIGPIPE
19:00:32.769 [24212] <2> bpbkar sighandler: INF - ignoring additional SIGPIPE signals
19:00:32.780 [24212] <16> bpbkar Exit: ERR - bpbkar FATAL exit status = 40: network connection broken
19:00:32.780 [24212] <4> bpbkar Exit: INF - EXIT STATUS 40: network connection broken
19:00:32.780 [24212] <2> bpbkar Exit: INF - Close of stdout complete
19:00:32.780 [24212] <4> bpbkar Exit: INF - setenv FINISHED=0
 

there are toomany PrintFile: xxxxxxxxxxxxxxxxx above this out put. I donot agree with the ntwk because the oracle policy is running without any error.\

Thank you!

[Yasuhisa_Ishika · Accepted solution]

• Are there any unreadable file under /opt/s2100/om? Try to check by "tar cvf /dev/null /opt/s2100/om".
• Increase Client Read Timeout
• Enable debug logging for bpbrm and bptm on your media server
• What version of NetBackup are you using? Please tell us with minor version(7.1.x.x).

[dulla]

thank you both yasuhisa and mn_ pankaj for your kind response. I added the file that lstat() can't get a return in my exclude_list file and the backup is running flawlessly. I will monitor it for some time and close the thread.