05-03-2011 05:43 AM
Hi,
I'm running Puredisk 6.6
I have 3 nodes (SPA, CR and MBE).
The /Storage partition on the MBE has filled despite it being scaled sufficiently. When I look I can see that the vast bulk of the data is in /Storage/tmp/quarantine - it has 172GB out of the 200GB of data.
The data is in files that start as bulkInsertPO* and prexxxxxxxxx.tmp files
I have an outstanding critical error logged with Symantec but they are deciding to not answer the phone or ring me back.
Can anyone here identify if any of these files can be deleted without causing any issues with the data.
I am happy to lose any temp data from backup jobs that have not fully completed, but cannot lose data from previous full backups.
Thanks,
Tim.
Solved! Go to Solution.
06-05-2011 03:59 PM
Hi!
Please log on as "root" and go to reports. Check if MBE is 90% full in "Capacity Report" (or something like that).
The MBE needs not only the disk space but also is limited by number of file entires. 400M is max when PureDisk becomes dead.
To make PD alive (only if you have reached the 90% of MBE utilisation and are unable to do anything): [extreeme and emergency solution's summary, data loss will occur]: try to delete minor data selection (i.e. one C: drive), run maintenance jobs. Stop not using archive bit.
It may be good to try to install MBE on other nodes and reroute it gaining more total MBE capacity.
Do not run jobs not using archive bit. Avoid running manual backup jobs (that will always not use the archive bit).
In case of disk capacity issues: all stuff under /Storage/tmp older than 2-3 days may "go bananas" ;) [logs and history may be lost but files should be restorable]
BR, Michał
05-03-2011 05:40 PM
Hi
Follow this to free up some space
http://www.symantec.com/business/support/index?page=content&id=TECH66776&key=52672&actp=LIST
Else anything in /Storage/tmp can generally be deleted. The BulkInserts are usually from importing jobs, if there is 1 particular file that is really large might be worth just deleting it. If you do delete anything just make sure there is no active jobs running else you might break one of those. Always terminate gracefully.
Checking one of my PD SPA's in /Storage/tmp/quarantine is 168MB (8TB Storage Pool)
total 171608
-rw-r--r-- 1 root root 619563 Apr 14 23:28 bulkInsertPO1723345902236611781
-rw-r--r-- 1 root root 4993418 Apr 15 07:30 bulkInsertPO4987775422992872416
-rw-r--r-- 1 root root 114501930 Apr 14 23:28 bulkInsertPO5549656096251953483
-rw-r--r-- 1 root root 55122805 Apr 15 07:30 bulkInsertPO6662917363342972757
-rwxrwxr-x 1 www-data www-data 390613 Mar 29 12:00 bulkInsertPO7705782156767067342
-rwxrwxr-x 1 www-data www-data 0 Mar 29 12:00 errsplit1305306962531784366
-rw-r--r-- 1 root root 0 Apr 15 07:30 errsplit2574260245056201682
-rwxrwxr-x 1 www-data www-data 0 Mar 29 12:01 errsplit3404573549386764255
-rw-r--r-- 1 root root 0 Apr 14 23:28 errsplit868707632069837191
-rwxrwxr-x 1 www-data www-data 45056 Oct 27 2010 pre4885860012418562252.tmp
-rwxrwxr-x 1 www-data www-data 16384 Mar 15 06:53 pre704065592789438229.tmp
Simon
05-05-2011 12:50 AM
Hi,
I have removed all data from /Storage/tmp/quarantine (moved to an archive drive just in case) so have removed the issues with space.
However my backups no longer function, nor do any data removal jobs.
They all fail on the MBE stage.
The error is a routing error and always to a file called 647.recommended. No matter what the job is
And don't even get me started on how I have a call logged with Symantec as critical and 48 hours later not a squeak. I actually spnt over an hour on hold after one of their people rang me in response to my boss moaning at them and I was promised to speak to the next available engineer. That's why I'm asking these question that should go to support.
Any help would be greatly appreciated as the support is now so poor I don't expect any Symantec help with this.
Thanks,
Tim.
05-05-2011 01:25 AM
it seems that 647.recommended was blank. I have done the folloiwng to replace it
cat 647.current > 647.recommended
and will post here what happens next.
05-05-2011 01:52 AM
it seems that 647.recommended was blank. I have done the folloiwng to replace it
cat 647.current > 647.recommended
and will post here what happens next.
05-05-2011 05:57 PM
Hi
I havent seen this issue before but looking at all the files in that folder for me, they are both the same, none of them are empty
-rwxrwxr-x 1 www-data www-data 14 Oct 12 2009 1.current
-rwxrwxr-x 1 www-data www-data 14 Oct 12 2009 1.recommended
-rwxrwxr-x 1 www-data www-data 40 Apr 20 18:48 2100.current
-rwxrwxr-x 1 www-data www-data 40 Apr 20 18:48 2100.recommended
-rwxrwxr-x 1 www-data www-data 42 Nov 12 2009 362_1310.current
-rwxrwxr-x 1 www-data www-data 42 Nov 12 2009 362_1310.recommended
-rwxrwxr-x 1 www-data www-data 42 Nov 14 2009 362_1452.current
-rwxrwxr-x 1 www-data www-data 42 Nov 14 2009 362_1452.recommended
-rwxrwxr-x 1 www-data www-data 42 Nov 21 2009 362_2056.current
-rwxrwxr-x 1 www-data www-data 42 Nov 21 2009 362_2056.recommended
-rwxrwxr-x 1 www-data www-data 42 Nov 28 2009 362_2709.current
-rwxrwxr-x 1 www-data www-data 42 Nov 28 2009 362_2709.recommended
06-04-2011 12:17 AM
After a few days Syamntec support eventually decided that they would help me with my 2-hour response critical incident.
They told me that the Server DB Maintenance workflow will clear out the /Storage/tmp directory.
I must admit I'm not convinced about that but as deleting all the files in quaratine seemed to fix the problem all seems well.
I got an apology from the support team leader who has promised me that the support issues for Puredisk will be resolved soon as they are recruiting more staff.
Thanks for your suggestions.
06-05-2011 03:59 PM
Hi!
Please log on as "root" and go to reports. Check if MBE is 90% full in "Capacity Report" (or something like that).
The MBE needs not only the disk space but also is limited by number of file entires. 400M is max when PureDisk becomes dead.
To make PD alive (only if you have reached the 90% of MBE utilisation and are unable to do anything): [extreeme and emergency solution's summary, data loss will occur]: try to delete minor data selection (i.e. one C: drive), run maintenance jobs. Stop not using archive bit.
It may be good to try to install MBE on other nodes and reroute it gaining more total MBE capacity.
Do not run jobs not using archive bit. Avoid running manual backup jobs (that will always not use the archive bit).
In case of disk capacity issues: all stuff under /Storage/tmp older than 2-3 days may "go bananas" ;) [logs and history may be lost but files should be restorable]
BR, Michał