Forum Discussion

Dany123's avatar
9 years ago

Backups are hung or failed with error code 196 and very less backups giving byte count

Hi Team, Master Server OS is Solaris Media server OS is Solaris Netbackup Version on master/media server  is 7.6.0.4 We have one Master server and 2 media server. we have 24 tape drives i...
  • RamNagalla's avatar
    9 years ago

    "maximum stream per drive" to 25 is Mulitplexing value and its very high ...never recomended..

    what the Mulitplexing values that is set in policy schedules.. Do not set more than 6, better value is 4

    Allow Multiple Data stream let you tirgger multiple jobs for each client and reults more processing time and more jobs and more failures... I would suggest you to remove the Allow Multiple Data stream option from policy since you have only 19 tape drives..(24 when faulty one fixed) ,this should reduce the number of jobs...(inturn number of failues)

    also you need to check if there is any slow backups, slow clients that are holding the Drives for long time, if you find those need to fix it got get the jobs running fast.

    also check the backup window and see if you have increase the backups windows times as per your requirement..

    once this is all done if you still see the 196 failues.. then that is time for vefirying the tunining and  capacity planning..

    -> howmuch of data that needs by backup every day?

    -> howmuch of data Drives can backup every day?

    > does the Drives running with optimised speed.. any think can de done to impove the drives/backup speed. etc....

    if amount of data is more than the drives can backup with good perfromance..then its time to expand the storage capacity...

     

  • RonCaplinger's avatar
    9 years ago

    This looks like your first post, so please excuse the assumption that you are new to NetBackup if that is incorrect.

    The NetBackup system enviornment is not set up properly and is causing backups to perform slowly, which is causing the error code 196 because the rest of the backup jobs have not been able to complete in a timely manner.  Adding tape drives to an environment does NOT increase how much data can be backed up in the backup window.  There must be an equivalent amount of bandwidth from all parts of the backup environment.

    First, Solaris is not the best choice for a media server.  It works fine for the Master, but media servers need higher Input/Output bandwidth that, in my experience, a 64-bit Linux or 64-bit Windows OS can handle better.  Consider changing those media servers to Linux if at all possible.

    Second, I know of no media server that can handle enough throughput to send data to more than 2 or 3 tape drives at once.  I would suggest reducing the number of concurrent write drives to just TWO drives per media server.  And as Ram suggested, change the backup policies to not allow multiple streams.  See if the tape drives are able to back up some clients faster than they have been.  If so, then consider adding more media servers, with only 2 or 3 tape drives per media server. If not, then your clients likely cannot stream data fast enough and you should investigate the client side more.

    Once you are able to see better throughput, then start allowing multiple data streams in the policy, but no higher than about 4 streams.  It will only hurt performance to increase higher than that. 

    And at all times, monitor the master and media server CPU and network I/O speeds, especially during the backup window.  If you are consistently seeing >70% CPU usage, or 100% network bandwidth, those indcate it is time to either add more powerful media servers, or look at disk-to-disk-to-tape backups, like the Symanted NetBackup appliances.