Forum Discussion

Stiva's avatar
Stiva
Level 4
12 years ago
Solved

Backup are failing with 11 error(System Call failed)

Hi All,

From last few days I am observing that the backups are failing for some policies with errro code 11 at one specific time. But after some time when I restart the backup. They are getting successfully completed.

Please find error:

06/19/2014 19:20:27 - Error bptm (pid=24055) system call failed - Resource temporarily unavailable (at tmmpx.c.1507)

Netbackup Master server version 6.5 OS - Solaris.

Please help to resolve this issue. I found there are so many reasons for error code 11 failure. but not able to decide exact resolution.

Thank You.

7.1.x and Earlier
Backing Up
Backup and Recovery
NetBackup
Solaris
  • Marianne's avatar
    Marianne
    12 years ago

    Well, it has been 6 days since you posted.

    So, All Log Entries report and messages file need to be checked for the time that the error was seen.

    I have never seen a Status 11 where NOTHING was logged in messages or NBU All Log Entries.

    Because bptm reported the error, you need to check logs for the media server that was used at the time.

10 Replies

  • Marianne's avatar
    Marianne
    Level 6
    12 years ago

    Status 11 means that something went wrong at OS level. 
    This means that it is difficult to trace from NBU point of view.

    The 1st thing to check is system resources - disk space, memory, swap, shared memory.

    Check NBU All Log Entries report as well as /var/adm/messages for clues.

    PS:
    Are you aware that NBU 6.5 ran out of support almost 2 years ago?

  • Linette_V's avatar
    Linette_V
    Level 4
    12 years ago

    Hi Stiva,

    As you've already found this can happen for lots of different reasons. The most common theme tends to be system resource issues. This can be due to an overloaded NIC, or more commonly not enough inodes/file descriptors.

    Whilst the issue is manifesting differently here, this is a clear KM article for file descriptor tuning: http://www.symantec.com/docs/TECH168846

    You may also find this useful: http://www.symantec.com/docs/TECH62633

    Give tuning a go, and see if they still fail. If so it would be worth monitoring network utilization at the time of failure.

    Good luck!

     

  • Stiva's avatar
    Stiva
    Level 4
    12 years ago

    Thanks to reply :)

    @Marianne : I am aware that 6.5 is out of service.

    I will check NBU logs and update.

  • Marianne's avatar
    Marianne
    Level 6
    12 years ago

    Not only NBU logs - OS logs as well. 
    Important to understand that some OS call is failing.

    I was hoping that you would get back to us today with section of /var/adm/messages that covers the time frame when status 11 errors were seen.

    Same with All Log Entries report - select the master server and the start and end time a bit before (+- 1 hour) the status 11 up to about 1 hour after the error was seen.

  • Stiva's avatar
    Stiva
    Level 4
    12 years ago

    Hi Marianne,

    I am observing that from last 3 days failures are not coming. I will observe it for some more days and will get back to you.

    Thank you for your help.

     

  • Marianne's avatar
    Marianne
    Level 6
    12 years ago

    Well, it has been 6 days since you posted.

    So, All Log Entries report and messages file need to be checked for the time that the error was seen.

    I have never seen a Status 11 where NOTHING was logged in messages or NBU All Log Entries.

    Because bptm reported the error, you need to check logs for the media server that was used at the time.

  • Stiva's avatar
    Stiva
    Level 4
    12 years ago

    Hi Marianne,

    We found the issue. Yes,There was a issue with media server itself.

    Media server was not rebooted from last 3.7 Years. Real memory was 14GB on the server. The kernel was using 98% which leaves almost nothing for programs, program data, and I/O caching.

    We rebooted the server and applied latest patch on it. Now its up and running fine.

    Thank you Marianne & L. Voller for your valuable comments. :)

     

  • Marianne's avatar
    Marianne
    Level 6
    12 years ago

    So, this means that nothing has been updated on this (and probably other servers) in almost 4 years?

    No OS patches, no NBU updates?

    Please shedule updates/upgrades as a matter of urgency. Especially NBU.
    If anything more serious goes wrong, you will be left without any support.

    Glad all is fine (for now).

  • Stiva's avatar
    Stiva
    Level 4
    12 years ago

    No, With only two media servers. It was a very old setup. Previously it was working on shared library. Now we are using dedicated one. We are in the process to upgrade NBU to 7.1.

    Let's hope It will work fine for 1/2 months...wink