Deduplication - impacts of data the is already compressed, encrypted or large numbers of tiny files.
Is this still a significant problem? Had a quick look at the doco on deduplication and there is no mention of if. In the past vendors including Veritas/Symantec warned that data that is already compressed and/or encrypted would result in poor deduplication performance, the same with large numbers of tiny files files. Compressed data includes stuff likeaudio files (mp3, wma, ogg etc), video files (avi, mkv, mp4 etc), image files (png, bmp, jpeg etc), PDFs, zipped files. A while back I tested the rumour that of the gzip & pigz using the --rsyncable option is deupe friendly friendly. I found it had significant improvment in dedupe performance on commvault, and it there was only a very small size difference between the compressed files with/without the --rsyncable option. The amount data can be deduped can have a big impact on what can be logically stored.... example - click to see full view
Duplicate Jobs stuck in Active Queued
Veritas support has been worse than useless. We have a CAS MBS enviroment, duplicate jobs are scheduled to run every week from MBS to Deduplication storage on CAS but will get stuck in the active queued state for hours without any data transfer. No failure and no events in logs. Many weeks ago there was an Alert about Robotic Libraries but it has been cleared and not appeared since. Vertias support had me run an inventory on Dedupe storage, recreate storage, update all servers to 20.5, and generate several different logs. All to no avail. Any ideas are welcome.
crcontrol.exe --processqueue never finishes?
Hi guys. We have had the issue that our Backup Exec dedup disk(approx 14TB) was close to running out of space. So after some investigation I found the running the "crcontrol.exe --processqueue" command, could do some cleanup. First time I ran it discovered there were about 44.000 tlog files in the queue folder, so I disabled all backup jobs to let it work on the files. Now A WEEK later, there are still files created in the folder, it has cleaned the folder several times, but it constantly seems to create more tlog files to work on, between 100-2000.. running "crcontrol.exe --processqueueinfo" keeps giving Busy=yes and pending=yes A few times have i caught pending=no, but the scheduled run at 12:20 and 00:20 seems to reset pending to "yes" The "crcontrol.exe --queueinfo" reports: total queue size: 12440670369 Am i supposed to stay patient and pray to god nothing happens? Or should i just start the backup jobs again? It seems it has cleaned up to make 3TB of available space. Any advice?
Deduplication - PDVFS errors
Hi All, I'm struggling with an issue that has occured with deduplication storage (or thats what it appears to be to me) on one of our managed media servers. All of a sudden most of the jobs that are run on this server are not completing. They start and go into the "Active: Running - Backup" state, but the average job rate for 75% of the jobs is only 1-2 MB/min or just stops at 0 MB/min. I have checked the event logs today and can see that PDVFS errors have appeared as well. Here is what we are getting (oldest first): Log Name: Application Source: PDVFS Date: 15/08/2016 22:00:53 Event ID: 50000 Task Category: None Level: Error Keywords: Classic User: N/A Computer: RESBE03.RES.TSS.NHS.UK Description: An error occurred in the PDVFS driver: <ERR> PdvfsOpen: O_CREAT not set Log Name: Application Source: PDVFS Date: 15/08/2016 22:00:53 Event ID: 50000 Task Category: None Level: Error Keywords: Classic User: N/A Computer: RESBE03.RES.TSS.NHS.UK Description: An error occurred in the PDVFS driver: <ERR> PdvfsOpen: failed for "/RESBE03/2/BEData/_BE101_.TMP": No such file or directory (2) Log Name: Application Source: PDVFS Date: 15/08/2016 22:12:23 Event ID: 50000 Task Category: None Level: Error Keywords: Classic User: N/A Computer: RESBE03.RES.TSS.NHS.UK Description: An error occurred in the PDVFS driver: <ERR> PdvfsPread: pdvfs_io_pread_fake failed: No such file or directory (2) Log Name: Application Source: PDVFS Date: 15/08/2016 22:39:17 Event ID: 50000 Task Category: None Level: Error Keywords: Classic User: N/A Computer: RESBE03.RES.TSS.NHS.UK Description: An error occurred in the PDVFS driver: <ERR> PdvfsPread: fd->dep or fp->dep->stat NULL I have tried accessing the pdvfs file path and it is present, I have been able to navigate (very slowly) to the directory mentioned in the alerts, \\.pdvfs\RESBE03\2\BEdata, I'm still waiting for it to open to see if the mentioned files are present as well. While I'm confident this is pointing to a storage error, I',m confused as to he extent of the issue as we are still seing jobs completing without any issues?? The server itself is running Backup exec 2015 FP4, Windows 2012 R2. Dedupe storage is a VNX5500. Thanks in advance for any insight anyone can provide on this. P.S. Other media servers we are running are not affected by this issue.Backup Exec Upgrade 2012 to 2014 - Deduplication option error
It was mentioned that I should repost this as a Blog since it seemed to be helpful to a few of you out there, so here goes: ============================================================== Just an FYI for anyone running into this issue during an upgrade to BE-2014:Buy a NetBackup Appliance or Build My Own?
I was recently asked, “Why would I buy a NetBackup appliance when I could buy the software and cheap JBOD?” Initially, I was taken aback by the question. However, after giving it more thought, I realized it was a fair question and an opportunity to explain the benefits of NetBackup appliances.
Don't Get Duped by Dedupe
If you are evaluating dedupe solutions, the dedupe ratios claimed by dedupe vendors are bound to intrigue you. I have seen claims of dedupe rates as high as 50:1 and I am sure there are claims of even higher dedupe than that. Are such dedupe rates realistic? Speaking truthfully, yes, but one must understand the assumptions and the math behind such high dedupe rates.Protect Dedupe device against failure and recover it Easy if it fails.
We Often Loose Data. If we, there is a failure and the recovery of the dedupe becomes tedious. To protect and recover dedupe in case of failure, below information would be helpfull Protecting Dedupe: 1. Dedupe device Can be backed up as it is Using backup exec Using the Shadow Copy ComponentsMy 2cents worth. Virtualization driving data growth by as much as 8x… according to Gartner.
Not being a big fan of long blogs (particularly the dry ones), I’ll try to share with you what I’m thinking about today with a bit of brevity. As we, the collective group of industry veterans and new-comers alike who live and breathe Enterprise-class Information Technology and Management, march down the path of virtualization, we are coming face to face with what is gene
