cancel
Showing results for 
Search instead for 
Did you mean: 

Netbackup appliance : Log Partition usage crossed 95%

dugga
Level 4

 

Hi all,

I'm getting the following alerts from one for the netbackup appliance , this is our media server

I have enabled logging to max, now i started getting alerts that its growing and it is almost 95% full.

From where can cleanup these logs in the cli; kindly help if there is any procedure or article on how to clean up this partition.

+-----------------------------------------------------------------------------+
|                            Partition Information                            |
|+---------------------------------------------------------------------------+|
||ID |  Partition  |  Total  | Used |  Status   |   State   |  Acknowledge   ||
||---+-------------+---------+------+-----------+-----------+----------------||
||5  |Log          |184 GB   |95 %  |Optimal    |Warning    |No              ||
||---------------------------------------------------------------------------||
|| * The value displayed here may be different from the backup space that is ||
|| available or used on the MSDP partition. The backup space statistics for  ||
||  the MSDP partition can be obtained by checking the MSDP disk pool sizes  ||
||                  from NetBackup Administration Console.                   ||
|+---------------------------------------------------------------------------+|
+-----------------------------------------------------------------------------+

1 ACCEPTED SOLUTION

Accepted Solutions

The /log/crash directory is where core dumps from programs that crash end up.   Some revs of netbackup have processes that core dump all to regularly.   You can pretty safely delete files from /log/core/ assuming you are not using them to debug a problem with netbackup.

The /log/netbackup/ directory has a whole subset of directories.  These contain the traditional netbackup logs.  You can fairly safely clean up files in the subdirectories under /log/netbackup/.   Again you may want them to debug problems,   but you can not keep them all forever.   Go to /log/netbackup/ and run the du command again and you will see which directories to focus on in that area.   You also probably should look at how verbose you have the logging set.  You may want to turn it down a bit if it is set at a high level.

Read more here on how to manage netbackup logs:  https://www.veritas.com/support/en_US/article.000108090

 

View solution in original post

8 REPLIES 8

Mouse
Moderator
Moderator
Partner    VIP    Accredited Certified

You have not specified the appliance firmware version but in any case, first steps of the technote will be quite useful https://www.veritas.com/support/en_US/article.000108180

quebek
Moderator
Moderator
   VIP    Certified

Hey

You need to clean up some NBU related logs located in /logs/netbackup directory. To do this please elevate to CLISH (https://www.veritas.com/support/en_US/article.000116864) and change to this dir and there run 

du -sh *

figure out which one is the biggest and if you dont need these logs remove them

Also can you provide output form this command

cat /usr/openv/netbackup/bp.conf  so we can see the debug levels etc...?

Our Master Server is at 7.6.0.3 and Media servers which are the 5220 Netbackup Appliance at 2.6.0.3

 

I checked the /log/upload partition in the CLI by logging to the maintenance mode and could see a lot of files generated with the name format like : cmd_out_1489659883457351.log

 

I did a vi and found this:

s65013:/log/upload # vi cmd_out_1489659883457351.log
Time Ran: Thu Mar 16 2017 11:24:43 CET
CMD: LC_ALL=C /usr/bin/w3m -dump /tmp/9oWiCQD0gC.html
RETVAL: 0
STDOUT:


Compute Node s65013

Time Monitoring Ran: Thu Mar 16 2017 11:22:10 CET

+-----------------------------------------------------------------------------+
| Partition Information |
|+---------------------------------------------------------------------------+|
||ID | Partition | Total | Used | Status | State | Acknowledge ||
||---+-------------+---------+------+-----------+-----------+----------------||
||5 |Log |184 GB |95 % |Optimal |Warning |No ||
||---------------------------------------------------------------------------||
|| * The value displayed here may be different from the backup space that is ||
|| available or used on the MSDP partition. The backup space statistics for ||
|| the MSDP partition can be obtained by checking the MSDP disk pool sizes ||
|| from NetBackup Administration Console. ||
|+---------------------------------------------------------------------------+|
+-----------------------------------------------------------------------------+

STDERR:

Not sure if these logs will be of use for any troubleshooting.

Please let me know if I can just delete them.

These version are out of support now.

Recently i had set the logging to MAX in the master server to get more details when any backup is failed.

I have attached the bp.conf file to here.

Let me know if any information is required. 

 

quebek
Moderator
Moderator
   VIP    Certified

Hi

My take is that majority of the /log is being used by netbackup subdirectory so /log/netbackup - you can go into this location and check this out by running from there

du -sh *

If I am mistaken run this command from /log this will allow you quickly figure out the 'guilty' one folder... and then take actions. Also when backups are failing only certain logs are needed - maybe you do not need all which are configured under /log/netbackup.... ask VRTS which should be enabled...

 

And as you said I see the culprit is /netbackup its using up a space of around  95GB out of 184 GB

In /log/netbackup/bptm 

the bptm logs are consuming majority of the space

media2:/log # du -sh *
1.7G app_vxul
27G crash
696K diskperflogs
12K initrd_shrink_log_062414122048.txt
32K last_sg_ses_failure.log
0 logrotate.lck
16K lost+found
84M nbappdb
94G netbackup
8.0G openv
4.0K patch
4.0K patch_copy_iso_to_tmpfs_201406161119.log
4.0K patch_log.sh
9.7M patch_log_2.6.0.1_061614101216
9.2M patch_log_2.6.0.1_061614115835
273M patch_log_2.6.0.1_061614121715
296K patch_log_2.6.0.2_062414113225
148K patch_log_2.6.0.3_112514144919
8.0K patch_output_2.6.0.1_061614115835
8.0K patch_output_2.6.0.2_062414113225
8.0K patch_output_2.6.0.3_112514144919
744K puredisk
4.0K s65013_09-23-2015_09h00m38s_logs.tar.gz
33G scsplog
4.0K selftest_report_SYM0910127_06-21-2014.txt
8.0K selftest_report_SYM0910127_06-24-2014.txt
4.0K selftest_report_SYM0910127_11-25-2014.txt
4.0K storage
1.5G upload
272K volmgr
25M webgui

 

 

 

 

quebek
Moderator
Moderator
   VIP    Certified

Well I would for sure removed the following: 27G crash - I do hope you already cleared out with VRTS possible crashes of this one appliance, than depending on the case you do run I would removed all logs if the job you are debugging did not fail in given time period. That's all - NBU appliance is just a NBU software installed on customized by VRTS linux OS with some add ons - so ...

I couldn't check with Veritas as my code is out of support life now. :(

Thanks alot for the guidance.

I'm not sure on what data does the /log/crash holds. I need to check that, is there any document where in I can find all these details, or quickly refer. 

The /log/crash directory is where core dumps from programs that crash end up.   Some revs of netbackup have processes that core dump all to regularly.   You can pretty safely delete files from /log/core/ assuming you are not using them to debug a problem with netbackup.

The /log/netbackup/ directory has a whole subset of directories.  These contain the traditional netbackup logs.  You can fairly safely clean up files in the subdirectories under /log/netbackup/.   Again you may want them to debug problems,   but you can not keep them all forever.   Go to /log/netbackup/ and run the du command again and you will see which directories to focus on in that area.   You also probably should look at how verbose you have the logging set.  You may want to turn it down a bit if it is set at a high level.

Read more here on how to manage netbackup logs:  https://www.veritas.com/support/en_US/article.000108090