cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted

nbsl.exe spawning hundreds of cmd.exe and nbproxy.exe Netbackup 8.1.2

Hi Forum Guru's

I have noticed recently that our master server over time ends up with hundreds/thousands of cmd and nbproxy that are not terminating gracefully. The only way to get them all cleared is either a server reboot or issuing bpdown then bpup.

We have OpsAnaytics 2k8R2 8.1.2

Netbackup Master 2k8R2 8.1.2

Media Servers 2k12R2 8.1.2

A bug report for knowledge base article 100007812 BUG REPORT: nbproxy policy manager processes become orpaned and are not getting cleaned up for version 7.1 decribes the issue exactly.

I have followed this article Article: 100038207 to up the logging on nbsl and server logging on the OpsCenter server but the logs are not revealing much. There is however a reocurring error on the master nbsl log

snip<>

0,51216,132,132,507364,1551791637794,83592,56404,0:,94:NB error code: 25 NB localized Error text: cannot connect on socket(NBUErrorTranslator.cpp:47),37:NBUErrorTranslator::getErrorDetails(),1
0,51216,132,132,507365,1551791637794,83592,56404,0:,170:Error:ClientQueryFacetImpl:Smiley TongueopulateOutput():#229:../ClientQueryFacetImpl.cpp(482): errNo:25 localizedStr:cannot connect on socket additionalStrSmiley SadExceptionManager.cpp:97),39:ExceptionManager::fillExceptionObject(),1
0,51216,132,132,507366,1551791637794,83592,56404,0:,63:Exception name:NBSLOpException(CORBAExceptionTranslator.cpp:47),43:CORBAExceptionTranslator::getErrorDetails(),1
0,51216,132,132,507367,1551791637794,83592,56404,0:,69:Rethrowing NBSLOpException exception(CORBAExceptionTranslator.cpp:85),43:CORBAExceptionTranslator::getErrorDetails(),1

snip<>

On the Opscenter log all looks pretty good. I hope someone can throw a different light on this as I cannot see the tree for the woods at the moment. Thanks in advance.

1 Solution

Accepted Solutions
Highlighted
Accepted Solution!

Re: nbsl.exe spawning hundreds of cmd.exe and nbproxy.exe Netbackup 8.1.2

All I think I found a solution/work around for my issue. nbsl process was spawning literally muliple hundreds of nbproxy.exe so many that I actually ran out of ephemeral ports. I did extend the ports (not my first choice but needs must). But I found when I cleared all the media servers from the "VMware Access Hosts" in the Master server Host properties the 7000+ nbproxy processes droppped back to 550 connected processes. I could be completely in the wrong park here and I am sure someone will be along to correct me. But this has worked in this environment.

View solution in original post

6 Replies
Highlighted

Re: nbsl.exe spawning hundreds of cmd.exe and nbproxy.exe Netbackup 8.1.2

I have logged a case with Veritas support. Let's see what they make of it.

Highlighted

Re: nbsl.exe spawning hundreds of cmd.exe and nbproxy.exe Netbackup 8.1.2

Good choice!
The snippet of the 'raw' log does not show source or destination IPs that's involved in the socket errors.

Highlighted

Re: nbsl.exe spawning hundreds of cmd.exe and nbproxy.exe Netbackup 8.1.2

Any updates from support on the issue ? I am facing similar issue.

Highlighted

Re: nbsl.exe spawning hundreds of cmd.exe and nbproxy.exe Netbackup 8.1.2

Wild guess... what is your IPv4 TCP dymanic ephemeral port range on master ?

netsh int ipv4 show dynamicport tcp

(FYI - NetBackup does not use UDP, and does not generally use IPv6 - so don't worry about the other available ranges - we are only intested in the IPv4 TCP range)

Highlighted

Re: nbsl.exe spawning hundreds of cmd.exe and nbproxy.exe Netbackup 8.1.2

Have seen lot of nbproxy.exe if the connection to opscenter has been unstable,

in that case we have found that killing the nbsl tree helps.

Usually there starts a new nbsl after a short while.

Not ideal, but a workaround when you can't stop netbackup

The standard questions: Have you checked: 1) What has changed. 2) The manual 3) If there are any tech notes or VOX posts regarding the issue
Highlighted
Accepted Solution!

Re: nbsl.exe spawning hundreds of cmd.exe and nbproxy.exe Netbackup 8.1.2

All I think I found a solution/work around for my issue. nbsl process was spawning literally muliple hundreds of nbproxy.exe so many that I actually ran out of ephemeral ports. I did extend the ports (not my first choice but needs must). But I found when I cleared all the media servers from the "VMware Access Hosts" in the Master server Host properties the 7000+ nbproxy processes droppped back to 550 connected processes. I could be completely in the wrong park here and I am sure someone will be along to correct me. But this has worked in this environment.

View solution in original post