Forum Discussion

Sisso's avatar
Sisso
Level 5
12 years ago
Solved

Application Error in Application event view log on media server

Hello,

We are running Netback 7.5.0.6 on windows server 2008 sp1 R2.

All our media servers with the same version as the master, 7.5.0.6.

Yesterday on the Backup window many backups failed at the same time on the same media server.

I tried to find what was the reason and found the following error in the event log in the media server:

 

application error 1000_0.jpg

 

please advise.

7.5
Backing Up
Backup and Recovery
NetBackup
Windows Server (2003-2008)
  • Mark_Solutions's avatar
    Mark_Solutions
    12 years ago

    I have seen this a few times and when overloaded it does not always do the graceful thing!

    If possible reduce the workload on the media server - VMware backups run faster the less you do at a time (at least from the same datastore)

    Also run a bpfis query to see if there are abandoned snapshots that need to be cleaned up.

    Finally make sure your Media Server is OK for memory and that its paged pool memory is tuned well to cope with the workload - use these registry settings (needs a reboot):

    HKLM\System\CurrentControlSet\Control\Session Manager\Memory Management\

    New DWORD - PoolUsageMaximum  - Decimal value of 40

    Edit the DWORD - PagedPoolSize Hex value of FFFFFFFF (this is 8 x F)

    Hope this helps

  • mph999's avatar
    mph999
    Level 6
    12 years ago

    Looks like bpfis crashed.

    Best way to procede is to log a call with Symantec, you will probably have to wait for it to reoccur to get all the required details.

    You will need to provide:

    Windows crash dumps

    (http://www.symantec.com/docs/TECH74145  /  http://www.symantec.com/docs/TECH128146)

    nbsu -c -t output

    Windows event logs (in txt format)

    bpfis log

     
  • mph999's avatar
    mph999
    Level 6
    12 years ago

    ... opps, forgot to add, bpfis should be at VERBOSE 5 / GENERAL 2

  • Mark_Solutions's avatar
    Mark_Solutions
    Level 6
    12 years ago

    So - as a guess your server is a VMware Backup Host - and these bpfis crashes ocurred during VMware backups.

    You say many failed at the same time .. how many VMware backups are you running at the same time to this media server? Any server can only do so much!

     

  • mph999's avatar
    mph999
    Level 6
    12 years ago

    Good point Mark, but I would suggest that even if 'overloaded' the process should exit a little more gracefully than crashing ...

  • Sisso's avatar
    Sisso
    Level 5
    12 years ago

    Hi mark,

    You are right, there were many VM backups during this crash, I dont remeber the exactly number but i think like 20 jobs of VM backups or more.

  • Sisso's avatar
    Sisso
    Level 5
    12 years ago

    Hi mph999,

    Thank you for suggesting, I will open a case to symantec about this crash to figure the reason of this crash.

  • Mark_Solutions's avatar
    Mark_Solutions
    Level 6
    12 years ago

    I have seen this a few times and when overloaded it does not always do the graceful thing!

    If possible reduce the workload on the media server - VMware backups run faster the less you do at a time (at least from the same datastore)

    Also run a bpfis query to see if there are abandoned snapshots that need to be cleaned up.

    Finally make sure your Media Server is OK for memory and that its paged pool memory is tuned well to cope with the workload - use these registry settings (needs a reboot):

    HKLM\System\CurrentControlSet\Control\Session Manager\Memory Management\

    New DWORD - PoolUsageMaximum  - Decimal value of 40

    Edit the DWORD - PagedPoolSize Hex value of FFFFFFFF (this is 8 x F)

    Hope this helps

  • Sisso's avatar
    Sisso
    Level 5
    12 years ago

    Thank you Mark,

    It was as you said too much VM backups ran together.

    I changed the policy and the amount of vm backups the run at the same time, and now it is look fine.

  • Mark_Solutions's avatar
    Mark_Solutions
    Level 6
    12 years ago

    Great - thanks for letting us know - it sounds odd but backing up just a few (or even one) at a time can be faster than backing up lots together - the datastores and vCenter snapshots don't especially like a huge workload either

    Glad your issue is resolved