02-24-2016 09:41 AM
Is there a command (or series of commands) that can be run on a Windows MSDP Media Server to get something like the output that an Applince kicks out when you go to Reports / Deduplication?
In most cases I'm running Server 2k12 with the Unix tools installed so I do have access to sed/awk/grep...etc....I wish MS had kept them in 2k12 R2...oh well...
EXAMPLE:
APPLIANCE.Main_Menu> Reports
Entering NetBackup reporting view...
Deduplication Show deduplication statistics.
Exit Logout and exit from the current shell.
Process Show current process status.
Return Return to the previous menu.
Shell Shell operations.
APPLIANCE.Reports> Deduplication
Deduplication statistics
Deduplication technology MSDP
Deduplication rate 87.409%
Data Protected 119344857 MB
Space Used For Protection 15026345 MB
Physical free space 8877902 MB
Logical free space 0 MB
Total free space 8877902 MB
Maximum Disk Space 23886659 MB
Used Disk Space 15008757 MB
APPLIANCE.Reports>
Solved! Go to Solution.
02-25-2016 07:31 AM
The command i listed earlier does pipe out to an excel file and the output is quite nice really - like the appliance
If you use "crstats --verbose>c:\stats.txt" then you get this in the text file:
Storage Pool Raw Size=62672026402816Bytes
Storage Pool Reserved Space=2507954794496Bytes
Storage Pool Size=60164071608320Bytes
Storage Pool Used Space=45523747012608Bytes
Use Rate=76%
Storage Pool Available Space=14640324595712Bytes
Catalog Logical Size=632274848694179Bytes
Catalog files Count=516719
Container Number=172809
Container Average Size=263120631Bytes
Space Allocated For Containers=45469613218215Bytes
Deduplication Ratio=13.9
02-25-2016 04:00 AM
I'm not aware of any individual NetBackup CLI commands that produce this level of detail in a quick summary. Such a command may exist, but I don't know it.
The storage pool size can be collected using:
nbdevquery -listdp -stype PureDisk -U
...and the storage volume size, but also free space, and occupancy level, with:
nbdevquery -listdv -stype PureDisk -U
...but not the de-dupe rate or the quantity of (pre-dedupe) data protected.
If you could get the overall average de-dupe rate from somewhere, then using the size and occupancy (from -dv), then one could easily calculate the quantity of data protected.
So, the missing piece is the overall average de-dupe rate.
02-25-2016 04:02 AM
The qestion is... where is this quick view getting the de-dupe rate from?
See TN:
Monitoring the MSDP deduplication rate
02-25-2016 04:19 AM
I guess crcontrol --dsstat gives a lot of it .. except the dedupe ratio ..
OpsCenter has that per Master server so i guess you could edit a report or do a query in OpsCenter to get what you need
02-25-2016 04:24 AM
I couldnt resist digging
\program files\veritas\pdde\crstats --verbose
Hope that helps
#edit#
If you do crstats --help you will see the other options as it will pipe out to a file etc for you
02-25-2016 07:19 AM
I have a script which will produce something like this:
10:23:34 %s_load_images, Disk volume figures... 10:23:34 %s_load_images, Volume Media Max Free Used Contains Dedupe 10:23:34 %s_load_images, ------ ----- --- ---- ---- -------- ------ 10:23:34 %s_load_images, dp_disk_xxxpsssxxyy001 @aaaah 100,880 35,646 65,234 258,099 4.0 10:23:34 %s_load_images, ----------- ------ ------- ------- ------- -------- ------ 10:23:34 %s_load_images, Grand Total 100,880 35,646 65,234 258,099 4.0 10:23:34 %s_load_images, ...actual global dedupe max capacity: 100,880 10:23:34 %s_load_images, ...actual global dedupe reservation percentage: 20.0% 10:23:34 %s_load_images, ...actual global dedupe reservation capacity: 20,176 10:23:34 %s_load_images, ...actual global dedupe writeable capacity: 80,704 10:23:34 %s_load_images, ...actual global dedupe used capacity: 65,234 10:23:34 %s_load_images, ...actual global dedupe remaining capacity: 15,470 10:23:34 %s_load_images, ...actual global dedupe average rate: 4.0 10:23:34 %s_load_images, ...actual global raw used capacity: 258,099 80.8% 10:23:34 %s_load_images, ...projected global raw remaining capacity: 61,209 19.2% 10:23:34 %s_load_images, ...projected global raw total capacity: 319,308
...but it's in VBscript, and does rely on having a bpimagelist -most_columns for *all* images.
As it stands right now the script has a load of other junk in it, but I could see if I could strip it down to just producing the above, i.e. to provide a basic report - if there's any interest in it.
So in the above, it gives a simple summary for all MSDP pools known to a master, and then summary a breakdown of total remaining capacity (across all pools) for that master.
Again, I could amend the script to generate the same bunch of final figures (in list form in section 2), but on a per MSDP pool basis (in table form in section 1).
02-25-2016 07:21 AM
Just thinking it would be cool if it also appended to a CSV file with rows by date - so that over a period of time the numbers could be graphed in excel quite easily.
02-25-2016 07:31 AM
The command i listed earlier does pipe out to an excel file and the output is quite nice really - like the appliance
If you use "crstats --verbose>c:\stats.txt" then you get this in the text file:
Storage Pool Raw Size=62672026402816Bytes
Storage Pool Reserved Space=2507954794496Bytes
Storage Pool Size=60164071608320Bytes
Storage Pool Used Space=45523747012608Bytes
Use Rate=76%
Storage Pool Available Space=14640324595712Bytes
Catalog Logical Size=632274848694179Bytes
Catalog files Count=516719
Container Number=172809
Container Average Size=263120631Bytes
Space Allocated For Containers=45469613218215Bytes
Deduplication Ratio=13.9
02-25-2016 10:01 AM
Mark,
It does look like all they're doing is reformatting the output from CRSTATS.
Thanks to everyone for all the information!
02-25-2016 10:34 AM
I guess you could use one of the SysInternals tools to run that command, from the master server, remotely on media servers...
...or , I'm wondering if crstats accept a 'storage server' name switch?
i.e. can a master poll its own multiple MSDP storage-servers / media-servers?
02-25-2016 11:09 AM
It doesn't look like it has a pool name flag...but...
It does look like OpsCenter does run that command to a file called crstats.out on each server so that file could be retrieved. It doesn't make use of the --convert-size parm so the data is in raw KB. And there is an "output file" option so that could be directed to a UNC path...perhaps...
On the other hand, a little PowerShell scripting might allow the Master to run the command, with the desired options, on each server...hummmmmm
03-07-2016 04:31 PM
This command should give you the overall deduplication rate, and then you can easily work out how much backup data is stored:
nbdevconfig -getconfig -stype PureDisk -storage_server $MYSERVER