cancel
Showing results for 
Search instead for 
Did you mean: 

NBU5000 filling up

smwoodcrafts
Level 6

I have a small concern. I was thrown into is situation where I am the sole support for our backup environment and I am not too sure about the dedupe solution. We have an NBU5000 with 32TB of storage. We originally had a problem in that it appeared nothing was being expired, but the engineer got that fixed. (Not telling me how it was fixed, of course). I was hesitant to point any other jobs then what was already going to it until I got a handle on things. Because of that, the used capacity was hovering between 16 and 19 TB. I went on vacation and when I got back I checked it and it is now at 24 TB. I'm afraid that we are running into the same problem we had before. wht can I do to check on this?

 

Dan Seymour

7 REPLIES 7

Sebastian_Baszc
Level 5
Partner Accredited Certified

Hey Mate,

You can try the following:

1. Log to the appliance as root
2. Run the following command crcontrol --processqueuestatus

If any of rows has yes, it means that PureDisk engine is working on the queue.
If both rows have no - please run the following command:

crcollect -v -m +1,+2

Wait till it finishes and run:

crcontrol --processqueue

followed by monitoring with crcontrol --processqueuestatus

Try also to update your appliance to 1.4.1.1.

Hope it helps.

 

Chad_Wansing2
Level 5
Employee Accredited Certified

so there could be a couple of things that could cause this:

A)  the queue processing isn't working as it should be as Sebastian mentioned to check above

B)  your retention levels on your data are keeping things for a considerable period of time and your data set has a high change rate so there's lots of new data to keep thereby filling up your storage

C) you're backing up lots of compressed/encrypted data (I see this a lot particularly with DB exports that the DBA's are doing and just not thinking about it) that doesn't dedupe well.  This could also be other types of compressed or encrypted file formats like music or videos.

 

What is your original dataset size that you're protecting and what kind of retention are you keeping things in your dedupe storage for?  As a general rule of thumb, you can keep about 30 days of dedupe retention in an equivalent quantity of sotrage to the original data set.  I generally allow an extra 50% of the original for every additional 30 days of retention.

Also, what release of code are you running on your 5020 (the 32TB version of the 5000)?  If you're not on at least 1.3.0.2, I would definitely upgrade.  There have been significant improvements in every version of code released in both performance and reliability.

 

Respectfully,

 

Chad

smwoodcrafts
Level 6

Let me give you a little more info. These are NBU5000s 2 units at 16 TB each together in one pool for 32 TB. I'm assuming the commands you want me to perform would be on the SPA and not he content router correct?

Where would this command be run? I get a bash error.

I was told that if I upgraded, there is a chance I would lose my data and I don't have anywhere to copy it to. What are the chances of data loss if I upgraded?

Dan Seymour

Chad_Wansing2
Level 5
Employee Accredited Certified

I have done probably 30 upgrades and never lost data.  What is your current version?  Particularly once you get up to at least 1.2, the builds are very good.  One thing to watch out for that I've seen is that on the original builds, the /swap and the / partition sizes got interchanged somehow.  It didn't cause a problem until the 1.3 release when folks had to make filesystem checkpoints for the upgrade and there wasn't enough space, but then the upgrade would just fail.  Easy enough to figure out and there is a technote on the situation.  Open a pro-active support case for the upgrade and you'll be fine.

 

-Chad

wboaz
Level 3

If you have 2 appliances in a pool running 1.4.1.1 there is a bug where the non-SPA nodes don't run the PDDO data removal process.

http://www.symantec.com/business/support/index?page=content&id=TECH179764

The correct process to reclaim space manually is here:

http://www.symantec.com/business/support/index?page=content&id=TECH180659

Step 5 is wrong though if you are at version 1.4.1.1. You need to run the process you added in the previous crontab entry:

/opt/pdag/bin/php /opt/pdmbe/cli/PDDODataRemoval.php >>/Storage/log/PDDODataRemoval.log 2>&1
 

I hve been fighting this for the last week, once I added the line into crontab from the first article there were multiple terabytes removed from the non-SPA node to get them back into balance. After that, the processqueue ran for a full day and many TB were removed.

I hope this helps.

 

Wayne

Chad_Wansing2
Level 5
Employee Accredited Certified

On the above note; iIt is accurrate, although only applies to multi-node installations.

 

-Chad

wboaz
Level 3

Which he, and I, have. He has two 5000s with 16TB each and I have two 5020s with 35TB each.

Also to answer his other question, all commands are run on both units except for the PDDO removal job (step 5) which is only run on the SPA. It will then run the task on both nodes.

 

Wayne