01-12-2012 12:57 AM
Hi!
We have two Puredisk 6.6.1.2.EEB20 SPA's; we have seven content routers per SPA and each SPA sits at around 59% utilisation of 185TB usable space per SPA. We only use Puredisk as back-end storage for Netbackup PDDO backups. (Netbackup version 7.1.0.2).
My query is why does it take so long for compaction to re-start on each of the content routers after a re-boot? I'm currently sitting at around 2.5TB or more per content router of "Space Needs Compaction" from the dsstat output and rising e.g.:
01-15-2012 09:33 AM
Compaction does not run based on your crcontrol --dsstat. There is a threshold of 30% that you saw that will kick off compaction. That 30% is based on df -h output not at dsstat out. If your disk grows beyond a certain point, compaction will kick off to bring it back down to a certain point.
If EEB20 the latest binary you have? Or do you have EEB21?
01-17-2012 08:03 AM
Thanks for your reply. Certainly the 30% is already far exceeded on most CR's from a df -h perspective so I don't think that's it. As I say, compaction eventually starts on one or two of the CR's in our SPA and then starts on all of them after a week or more. The bigger the CR the longer it takes. It's almost as if there's something else running in the background that it's waiting for. We have EEB20 as the latest.
01-22-2012 01:41 PM
There is a chance that someone has modified your default values in your contentrouter.cfg. Also at EEB21, there is a known issue with compation. So you would need to upgrade to 6.6.3 to obtain an EEB to fix that issue.
03-26-2012 04:29 AM
The solution was/is:
# /opt/pdcr/bin/crcontrol --dsstat
# df -k
# /opt/pdcr/bin/crcontrol --compactstart 0 0 1
Data store compaction: ON, DeleteSpaceThreshold: 0%, CompactLBound: 0MB
Compaction busy: Yes
# watch -n 20 /opt/pdcr/bin/crcontrol --compactstate
Once the compaction completes:
Compaction busy: No
# /opt/pdcr/bin/crcontrol --dsstat
# df -k
Then the allocated space should be close to the used disk space.
Space allocated for containers : 11945175272705 bytes (10.86TB)
Space used within containers : 11903081645497 bytes (10.83TB)
Filesystem Size Used Avail Use% Mounted on
21T 11T 10T 53% /Storage/data
Good Luck!
Michał
05-09-2012 07:12 AM
Thanks Michal, I will try that :)
05-29-2012 11:43 PM
One more thing,
I recently was in need to rebuild a CR storage (/Storage/data volume), so it was necessary to "clean it".
Simple removing the Content Router role from the node in the GUI -> Topology did not solve the issue.
I had CR utilisation at 0%, but disk utilisation around 40%.
Then the parameters like:
# /opt/pdcr/bin/crcontrol --compactstart 80 0 1
helped to vacum the CR storage to "back-up-able" size. (80% of expected compaction ratio, 0, 1== true for "try hard").
Cheers!
05-30-2012 06:43 AM
Moving up to Puredisk version 6.6.3a plus EEB bundle V6 has meant that the compaction kicks in earlier after a re-start/re-boot i.e. after a few days rather than weeks. Mind you, 6.6.3a has had to have its fair share of fixes too - particularly regarding log rotation and CR performance.