OK, so long story short the past week my 5220 appliance has been close to useless. While things hummed along nicely for the initial months (4), as expected space usage slowly began creeping up. First 60% and then 80%, but it sat in the 80's for a number of weeks, so it seemed this is where we would sort of plan to be. Well, no, within roughly 1-2 weeks, we went from this acceptable capacity to 100%. Yes, 100%, someone set highwater to 100% and in the end our appliance was cooked, Status 129 all the way, restores were not possible either, so we turned to support. After several days of cleaning things out, we managed to get to a point we could at least get the appliance online again. However, by no means did this mean we could use it. The appliance functioned as a media server, reading VM's, sending them to a data domain appliance and it seemed we were in the clear while we cleaned up more space on the 5220. But, as luck would have it, the DD also filled and we resorted to using advanced disk to stage data and then SLPs to send it to tape. Long story short, we are just getting by as things are made useful again and the only way to make the 5220 useful is to free up lots of space, so the plan is to get the largest systems off of it. I guess I would rather have several systems use the appliance and one write to tape as opposed to several to tape so that one can use the appliance.
My goal is to get a report out of opscenter that tells me what is using all of my space. I expired image after image, but each time I did I did not gain any space. In talking with support this is normal with deduplicated data and the images as each can be really small. Therefore, I need to find the ones that are really big and get them off of there until we can manage to get more appliances to increase capacity. I hunted around in opscenter, but nothing appeared to be the report I am looking for. All I want is a list of sort that says, these 5 machines use 10TB while these 300+ use 50TB and then just show me what each system is using so I can dupe those to tape and expire the images off disk. Is such a report possible? Be it SQL or with the Wizard this report would help me in a huge way.
I hope you appliances are running better since the post...
There are a few canned reports in OpsCenter which may help you pinpoint what you are looking for...
This will not show deduplication statistics...
2) Deduplication Size Factor
This will show the top client deduplicaiton rates...
Execute the report, click on ranking, then click edit.
Set you time frame, and click next
Set the Report on = clientname
Set the Display = 100
Click Next to Preview the report
Choose to use the data onscreen or save the report for later use...
Compariing 1st report to the 2nd (bottom dwellers on dedupe) you maybe able to cross reference the highest volume clients to the lowest dedupe clients that appear on the report.
Targeting these images should assist you in clearing up valuable space and potentially move poor performing dedup clients to other disk STU's or tape.
If you have any questions, please let me know.
Thanks for this information. It helps a little. I had some SQL queries that would give similar results, except yours actually seems to combine things to a single client name. With the SQL query if the client changes policies, then you have the same client twice sort of thing, so this does help.
However, the crazy thing is that it still does not give exact data. Although, maybe I am just expecting too much. Maybe that is not even possible because of deduplication. The way I see things, if I run backups for 6 months, but only have a 3 month retentions, Opscenter when it runs, still sees all the logs/data from all jobs. So, if I run a report targetting 5 months, I get a value for 5 months, even though retention is 3 months so the result would be inaccurate. So, if my assumption is right and this is the most accurate we can get, then I guess I need to run reports based on my policy values.
In other words 2 month retention, 2 month time frame. Otherwise I will screw myself up... For instance, I did deduplication ascending, so I can see the systems that dedupe the worst. I have taken like 5 bad ones and duped them to tape, however, they are still appearing in the report...
Even further, I did as you noted and things dont appear to match up. At least not well, looking at the first report, the first few systems don't seem to appear on my second report, at least I am not seeing them. Exporting them helps a little, but in the end I start relying on a lot of back and forth between 2 reports to come up with a value.
But, if we were to agree that there is no way to get real values and instead I need to rely on retention level in my policy when running a report, then the SQL query I use is as follows:
clientname as 'Client',
COALESCE(CAST(SUM(preSISSize)/1024.0/1024.0 AS NUMERIC (20,2)), 0) AS 'Pre Dedup Size (MB)',
COALESCE(CAST(SUM(preSISSize)/1024.0/1024.0/1024.0 AS NUMERIC (20,2)), 0) AS 'Pre Dedup Size (GiB)',
COALESCE(CAST(SUM(bytesWritten)/1024.0/1024.0 AS NUMERIC(20,2)), 0) AS 'Post Dedup Size (MB)',
COALESCE(CAST(SUM(bytesWritten)/1024.0/1024.0/1024.0 AS NUMERIC(20,2)), 0) AS 'Post Dedup Size (GiB)',
COALESCE(CAST((SUM(preSisSize) - SUM(bytesWritten))/1024.0/1024.0/1024.0 AS NUMERIC(20,2)), 0) AS 'Total Savings (GiB)',
COALESCE(CAST(100 - (SUM(bytesWritten) / SUM(preSISSize) * 100) AS NUMERIC (20,2)), 0) AS 'Percent'
WHERE DATEDIFF(day, UtcBigIntToNomTime(endTime), GETDATE()) <=120 and presisSize != bytesWritten and presissize !=32768
GROUP BY clientName, policyname
ORDER BY COALESCE(CAST(100 - (SUM(bytesWritten) / SUM(preSISSize) * 100) AS NUMERIC (20,2)), 0)
It gives me the size before, the size after and even the percent of dedupe. The only thing it does not do is combine multiple entries per client, so the same client can appear more than once, but overall it is the same data on one page. I think, no? You just need to set the time frame you want in the query to get the right data. I simply thought there is a way to know what is truly on the appliance, but instead it seems we can get close as long as our query is right, but we will never have the exact figure.