5220 Appliance File System errors at Reboot - Check Forced

[DecKy]

Hi,

Had a situation last week where our 5220 running 2.5.1 as Master / Media server had file system errors detected at boot time and a check was forced.

The check failed, a message was displayed to run fsck manually and the unit was put into a maintenance mode with a root login prompt, however my normal root password didn't work - it hasn't been changed from the default. The only option on the screen was to enter CONTROL-D to reboot the appliance, after trying this a number of times I booted the unit into single user mode and was able to have a look at the file system. I eventually got the system to boot but it has raised some questions.

The /tmp folder was full of log files, after clearing it out on Friday it is back up to almost 8000 files already, is there a scheduled process to clear down the /tmp folder, my DR appliance which is only used for replication has 146,000 files in /tmp.

What root password is required if the unit is put into maintenance mode ?.

What bootable media could be used to run a file system check with the root file system unmounted.

Would it be possible to make /tmp and /log separate mount points in a future release.

Thanks,

Dec

[Mark_Solutions · Accepted solution]

If you tmp folder was full it is caused by one of two things - either Telemetry data or VMWare snapshot files left behind after failed VMWare backups

Both do the same thing and both have EEB's available so please install than as soon as possible as it will soon fill up again.

Not sure on the root password - i dare say Symantec wouldn't want us to know it anyway!

This is the telemetry patch: http://www.symantec.com/docs/TECH199364

The VMware patch is not on general release - you may need to ask Symantec for that - can't find the ETrack number off hand - maybe see if you have .vmdk file in /tmp - if so call Support

There are also AIR patches if youuse AIR that should be applied - these need to be requested:

http://www.symantec.com/docs/TECH200177

Hope this helps

[DecKy]

Thanks for the repsonse, I have the telemetry patch installed and have asked support for the AIR patch as I am getting 191 errors

I've analysed the files in /tmp and I get the following 14 files recreated every 16 mins, is this normal ?

System-Serial-Number-Log_01-25-2013_00.05.03.log

Expander-Log_01-25-2013_00.05.03.log

Raid-Card-FW-Term-Log_01-25-2013_00.05.03.log

Alarm-Log_01-25-2013_00.05.03.log

SmartInfo_01-25-2013_00.05.03.log

CPU-INFO-Log_01-25-2013_00.05.03.log

SDR-ELIST-Log_01-25-2013_00.05.03.log

Raid-Card-Enclosure-Info-Log_01-25-2013_00.05.03.log

Adp-Log_01-25-2013_00.05.03.log

Raid-Card-Config-Log_01-25-2013_00.05.03.log

LDPD-Log_01-25-2013_00.05.03.log

FRU-Log_01-25-2013_00.05.03.log

DMESG-Log_01-25-2013_00.05.03.log

BBU-Info-Log_01-25-2013_00.05.03.log

[Mark_Solutions]

It is normal if you just get the one set - the issue is if they do not get regularly cleaned up and start to fill the system up

Telementry seems to store the files in /tmp as each check runs - the EEB should make sure that they are cleaned up regularly so just keep an eye on how many there are

A couple of reboots seems to trigger a cleanup but the EEB should fix it anyway

[DecKy]

So these are the telemetry generated logs, in which case the EEB doesn't seem to be working for me, I have a case open with support, thanks again for your help

[Mark_Solutions]

It depends if you get lots of them (days worth) if there is just one or two of each then you are probably OK - if there are hundreds then it may need looking at