Netbackup Vmware Policy Deduplication Rate

[Penchy]

Hi,

Our Environment:

Master Server : 7.7 on WIn 2012 Server

Media Server : Appliance 5230 on 7.6.0.4

We are backing up almopst 400 Servers and we have around 40 SQL Servers and rest are File Servers

The Backups are working fine 

We need to find out if our Vmware Policies are able to take maximum advantage of Deduplication. Our MS Windows Policies gives us a DeDup Rate of 95% against our VMware Policies which is around 85%. Is that the Best that we can get or can we still get more out from Vmware Policies. Please check the OPsCentre Report which shows Dedup Data of Feb and for March.

Deduplication Rates by Policy Type
Feb 1, 2016 12:00:00 AM to Feb 29, 2016 11:59:00 PM (Custom)
Policy Type	Pre Dedup Size(TB)	Post Dedup Size(TB)	Deduplication Rate	Job Size Deduplication Savings(TB)
MS-SQL-Server	7.24	1.32	81.79	5.92
MS-Windows	48.13	2.85	94.09	45.28
Standard	2.66	0.03	98.68	2.62
VMware	345.98	53.6	84.51	292.37
Deduplication Rates by Policy Type
Mar 1, 2016 12:00:00 AM to Mar 22, 2016 9:47:28 AM (Custom)
Policy Type	Pre Dedup Size(TB)	Post Dedup Size(TB)	Deduplication Rate	Job Size Deduplication Savings(TB)
MS-SQL-Server	1.73	0.04	97.89	1.7
MS-Windows	30.05	0.86	97.14	29.19
Standard	1.9	0.03	98.53	1.87
VMware	252.29	34.82	86.2	217.47

[Nicolai]

Looks OK to me initially, you got the majority of the data in VMware - 345 TB pre dedupe, stored in 53 TB. Almost 100TB increase from previous month

But as always be careful what people put inside the VM's. Encrypted data or precompressed data is poison for dedupe. You need to monitor each VMware job and take action if some jobs show  really awful deduplication ratios (remember the dedupe ration in the activity monitor). This cat after mouse chase is no diffrent than having e.g Data Domains around.

There is no setting in Netbackup  that will increase dedupe ration by itself - it's all based on what type of data you put into the MSDP pool.

Also remeber that dedupe ration may increase over time when data is being backup repeatedly.

[Penchy]

Thanks for your suggestion Nicolai.

We did again have a look at our Dedup Rate for Clients and it seems we are having issues getting a Good rate on Servers which has SQL Database.

These are Sharepoint Server Database but we just backup them up using Vmware Policy and selecting Application State Capture and not using the Sharepoint Policy.

Please look at this Dedup rate for SLQ Servers

Job ID	Job Policy	Job Schedule	Client	Kilobytes	KB/Sec	Elapsed Time	Media Server	Deduplication Rate
Backup	PTC-VMWARE-BRONZE-SHAREPOINT_SQL_QA	Full	Client	1914491744	65852	8:11:06	Appliance	52%
Backup	PTC-VMWARE-BRONZE-SHAREPOINT_SQL_DEV	Full	Client	680921760	57217	3:21:05	Appliance	72.20%
Backup	PTC-VMWARE-BRONZE-60DAY_BATCH1	Daily	Client	90004416	24329	1:02:24	Appliance	31.10%
Backup	PTC-VMWARE-BRONZE-60DAY_BATCH2	Daily	Client	79931712	26240	0:51:29	Appliance	34.60%
Backup	PTC-VMWARE-BRONZE-SHAREPOINT_SQL_DEV	Full	Client	681031968	63961	2:59:27	Appliance	59%
Backup	PTC-VMWARE-BRONZE-SHAREPOINT_SQL_QA	Full	Client	1919030848	90274	6:00:58	Appliance	59.80%
Backup	PTC-VMWARE-BRONZE-SHAREPOINT_SQL_DEV	Full	Client	681076992	61477	3:07:06	Appliance	59.60%
Backup	PTC-VMWARE-BRONZE-60DAY_BATCH1	Daily	Client	282380576	53814	1:29:21	Appliance	20.70%
Backup	PTC-VMWARE-BRONZE-SHAREPOINT_SQL_QA	Full	Cient	1947966208	85117	6:33:31	Appliance	51.60%
Backup	PTC-VMWARE-BRONZE-SHAREPOINT_SQL_DEV	Full	Client	681076896	72115	2:39:47	Appliance	66.40%
Backup	PTC-VMWARE-BRONZE-60DAY_BATCH1	Daily	Client	197884544	27318	2:22:23	Appliance	19.20%
Backup	PTC-VMWARE-BRONZE-60DAY_BATCH3	Daily	Client	5076800	3344	0:28:17	Appliance	53.40%
Backup	PTC-VMWARE-BRONZE-60DAY_BATCH3	Daily	Client	20672352	8900	0:41:05	Appliance	30.80%

[Nicolai · Accepted solution]

There are some bad dedupe client in-between. The only option is to re-direct backup of those databases (or VMs) to non-dedupe storage. Alternative just ignore and use the MSDP space required. I would recommend re-directing backup of those VM's to non-dedupe storage

As mentioned before, this dilemma is not unknown in a dedupe world. It is just hitting you now, but good work you discovered it before you ran of of MSDP disk space !!

It seem all the _BATCH* suffer from bad dedupe rates. 

[Penchy]

Thanks Nicolai, I will do some more digging about SQL Database Dedup Rates