I have a problem with the compression in BE 2012 since upgrading from BE 2010.
But first some informations about the system:
Windows Server 2008 R2, BE 2012 SP1, Hotfix 189571, 180964, 194470
It's connected to an Quantum SuperLoader3 library with a HP Ultrium-4 drive (LTO-4).
Web Interface from the SuperLoader3 says compression is disabled but it doesn't offer an option to enable it.
In BE 2012 compression is enabled for the HP drive.
HP StorageWorks Library and Tape Tools test 'Data Compression test' running successfully with this result:
Read compression ratio is: 2.61:1; Write compression ratio is: 2.61:1; compressed write transfer rate = 38.71Mb/s; compressed read transfer rate = 34.29MB/s
So I think the device is working well.
Then I used tracer.exe to check what mode_select6 and mode_sense6 says.
The last mode_select6 before writing starts says 00 00 10 00 0F 0E C0 80 00 and so on. Guess it means data compression is being turned on.
But the following mode_sense6 says 0B 00 10 08 46 00 00 00 and so on. According this tech-doc http://www.symantec.com/docs/TECH50960 I have to look for previous mode_sense6 with an 0F in the 5th byte. I did so and found one befor the last mode_select6 operation. The mode_sense6 says now 13 00 10 00 0F 0E C0 80 00 and so on.
So as a summery; event 246 says mode_sense6 with 13 00 10 00 0F 0E C0 80; event 247 says mode_select6 with 00 00 10 00 0F 0E C0 80 and event 248-256 are 4x mode_sense6 with 0B 00 10 08 46 00 00 00.
For me thats confusing and I dont know what to do now.
In BE for the backup job compression "hardware (if available, otherwise none)" is set.
But now there's the second confusing part for me.
For getting the tracer.exe log I used an extra backup-job. I also run a quick delete of the used tape before starting. It contains our IT-Service folder with 41.266 files and 6.330 folders and about 22.5gb size.
Now the jobprotocoll says it backed up 41266 files in 6330 folder and processed 24.137.683.527 bytes in 15 minutes and 51 seconds with compressiontype hardware.
But after it BE says on the tape the used capacity is 43,2GB!!! And available is just 738GB from total 781GB.
It also says compressionratio is 1:1.
According to my bad math knowledge, it's more a compressionratio about 0.5:1 and not 1:1!!
So all in all, after upgrading from BE 2010 to 2012, i have to use more than twice the ammount of tapes needed in BE2010.
I hope some of you can help me to get the compression back to work as well as putting more than 781GB of files on a LTO-4 tape.
There were some HP drivers installed for the tape drive.
Used tapeinst.exe to install Symantec drivers. They're old, from 12.02.2008 but I gonna go for a try.
Backupjob is started, but tracer.exe says still 0B 00 10 08 46 00 00 00 for the mode_sense6 operation.
mode_select6 and the mode_sense6 before the mode_select6 are correct.
Have to wait now until the job is done to tell how many GBs it backed up now.
Now with the Symantec drivers I've got the same errors with the same behavior described in first post.
It backed up now 24.138.799.362 Bytes in 12 min 40 secs but capacity used on the tape is now at 46GB!!
After deleting the tape it shows 781GB total capacity and 0 Byte used.
OK...do the following:
1. Check the tapes for Hard Write Errors. If there are, this indicates problems with either the tape drive itself or the tapes. HP LTT can assess the hardware and advise accordingly.
2. Completely remove the autoloader from both BE and Windows, and shut it down and disconnect it. Restart the server, and after booting into Windows, then shut the server down again. Reconnect the autoloader and start it up and allow to initialise. Once done, start up the server, and once booted, run tapeinst.exe again.
Lastly, is this a new issue or has it been happening for a while?
I found nothing about Hard Write Errors in HP LTT, but maybe I've searched at the wrong place.
But on the Health-view I found something for me strange looking:
Native data voluime read / written (last 4 tapes) : Wrote 38.7GB, read 15.2MB
Previous tape 0D52668800 Wrote 13.6MB, read 3MB
Previous tape 0D52668800 Wrote 19.3GB, read 4.5MB
Previous tape 0D52668800 Wrote 13.6MB, read 3MB
Previous tape 0D52668800 Wrote 19.3GB, read 4.5MB
If I think, one row = one job / action in BE, it would mean that BE compressed 22.5GB to 19.3GB but then it have displayed somehow twice the ammount of saved data.
Is there any way to check the used capacity on a tape without using BE?
Anyhow, while searching for Hard Write Errors in HP LTT, I run a Drive Assessment test with a new HP LTO-4 tape (bought some weeks ago, never used before).
It stopped with errors, saying "The LTO Drive Assessment Test has checked the history and operation of the selected drive, and problems have been reported. The drive is no longer recommended for use. Please contact hp support for further assistance." It also shows for Data Cartridge Information: Overall drive margin: -93.6%
Does it mean the tape is faulty or is this a problem with the tape drive??
Step 2 from you I'll perform when I'm sure the tape drive is working fine.
Both issues, the compression and the usage of twice the space on tapes is since we upgraded to BE 2012. (About 2 months ago, I just hadn't enough time to write here earlier.)
Ok, I'll try with another new tape.
Meanwhile I had run a backup with the tape I used for the Drive Assessment test and it completed without errors. The report says it backed up 24.138.799.362 Byte in 13 min 34 sec so at least the tape is still working. But the used capacity on the tape is now 42GB, still way to much for saving just about 24GB.
By the way, tomorrow I'm out of office so I just can worke remote on the tape drive.
Ok, the 2nd Drive Assessment test completed with errors.
Used another new HP tape for the test and now the Overall drive margin says -93.8%, 0.2% worsens than on the first run today.
Gonna make tomorrow some more tests and then I think I have to call for the now 7th drive replace within 2 years
Have you seen this discussion
and have you updated your BE 2010 to SP1a and all the latest hotfixes?
If you have done the above, you can check whether your hardware compression is working by backing a test directory which contains only text or Word files. You should be able to see compression at work.
For your regular data, try software compression and see what is the compression ratio. Hardware and software compression should give you roughly the same compression ratio for the same set of data. If you are not getting good compression ratio even with software compression, then it is your data that is the problem. If you are compressiing data which is already compressed like zipped files, movie or sound clips, then you are likely to end up with more data than what you have start off with. See my article below on this subject.
I have manually upgraded to SP1a and also installed manually Hotfix 189571.
The other two installed Hotfixes, 180964 and 194470 have to installed themselves through Live Update. Live Update also says no new Updates available. So I think and hope that BE is up to date.
At the moment I'm trying software compression of my IT_Service folder. If it doesn't work, I'll try to back up some text files.
Also you said it can be the data which cant be compressed but I can tell you, its not the data.
While using BE 2010 I allways had a compression from about 1.4:1. Ok, I have to admit that was for the whole fileserver.
Well, anyhow, the backup using software compression is finished now.
Saved 24.138.799.530 Byte with a software compression ratio of 1,1:1. (Job Protocoll says it)
But Storage => All media - details view tells for tape 000027 there's an 1,0:1 compression.
And again, details view from the tape says compression 1:1 and an used capacity of 42,4GB.
Please tell me, which view shall I believe? The Job Protocoll saying there's a compression with 1,1:1 and 24GB saved data or the tapes detail view saying there's a compression with 1:1 and 42.4GB saved data
Gonna create some text files now, about 50MB total size I've read here sometime ago, I'll report back when I've some results.
I created now 50 text files with the following cmd command:
fsutil file createnew C:\textfile.txt 1024000
It contains just space's but I think that should not matter.
Then I copied the file 50 times and backed them up using software compression. The job protocoll says now backed up 50mb in 9 secs with software compression ratio 29,8:1
Storage => All media - details view tells again 1,0:1 compression.
Details of the tape says now capacity used 19MB (!) with compression 1:1.
If that what the job protocoll said was correct, it should have used 1.67MB on tape and if that what the tape details said was correct, it should had an compression of 2.63:1.
Anyhow, it's still displaying stupid compression ratio 1:1.
Old archiv tapes, created with BE 2010 are displayed as an example with a compression of 1.2:1, so it should not be a display error from BE!
Gonna do a run now with hardware compression.
Hey at least I've got something good now.
For the run with hardware compression, job protocoll just sayd the saved size and MB/s.
But now the Storage => All media view says for tape 000027 compression ratio is 2.6:1!!
Also, the details view from the tape displays now an compression ratio of 2.6:1!
It still uses 19MB capacity but now it's the first time a backup job is looking good with BE 2012!
It is also remarkable that I did no changes to BE or to the server, just creating a new job.
I'll gonna create a new job for the IT_Service folder and report later back if compression is still working.
Ok I have some new results.
I created a new backup job for my it-service folder.
Then I started two backup jobs, the first with the option validate backed up files checked and the second with this option deactivated.
I have not deleted the tape between those two jobs.
After the first job the job protocoll said 24.138.799.530 bytes were saved and the details view of the tape showed an used capacity of 39.8GB! It also showed a compression 1:1.
After running the second backup job, the job protocoll said again 24.138.799.530 bytes were saved, so no changes there. But now the details view of the tape showed an used capacity of 77.3GB.
So just let me calculate it
First job: 39.8GB used on a new tape => capacity used by this job: 39.8GB
Second job: 77.3GB used on a new tape => capacity used by this job: 77.3GB - 39.8GB = 37.5GB
And by the way, now I discovered the backup records view for tapes.
That view says at size for both backup job each 22.5GB.
So please tell me, how the hell can BE 2012 uses total 77.3GB on this tape???
When you use software compression and then write to tape, the compression ratio is always 1:1 because the amount of data sent to the tape drive is the same as the amount of data written to tape. For example, you have 50GB of data. Software compession reduces it to 25GB and this 25GB gets written to tape. What is on the tape is 25GB, so the compression ratio is 1:1, not 2:1. If you use hardware compression and you send 50GB to the tape drive, if only 25GB is actually written to tape then you have a 2:1 compression ratio (= 50/25).
From your run with your test 50GB of data, hardware compression is working.
I don't understand your last post.
Hi pkh, I haven't perform a backup job with 50GB, just 50MB.
I'll try again to tell what's my problem.
About the data saved here's some information:
I deletet some .exe and install files so there's now most .doc/x, .ppt/x, .xls/x, .gif, .jpg, .png, .html, .php, .txt left.
The folder is now 8GB big and contains 19.091 files.
I hope you can understand it now.
Now I erase the tape again and after making sure there's 0 Bytes capacity used, I'll perform a backup of 50 textfiles with a total filesize of 50MegaByte. (The textfiles just containing spaces)
I used the same settings as those of the previous backup job.
The result now is different, because it has a compression of 2.6:1 and only needed 19MB:
You will probalby say now, the compression is working fine, but I say it dont work fine!
Because this is the only and one backup job where I get a compression better than 1.0:1!!
I have 4 backup jobs, saving two domain controllers, one mailserver and one fileserver but no job is getting a compression better 1.0:1. And also they need much more space on tape than on hard disk what I cant understand.
It is due to the type of data that you have. You have jpegs which are compressed files. When you compress them, you will probably get more than what you have started with. Read my article on compression again.
How often are you cleaning your tape drive? For an LTO drive at a minimum I would run a cleaning tape once per week. The HP LTO cleaning tape has 50~ uses and so I would at a minimum run it every Friday and replace it once a year. In very dirty environments I would run it even more often than that.
Based on years of experience with clients in various environments using LTO2 to LTO5 drives the most common cause of the drives failing, often with the exact same symptom of not being able to compress, is not running the cleaning tape every week. What you're experiencing right now feels like an instant replay of a client with internal IT support that didn't follow our recommendation to run the cleaning once a week. When he left and we came in to cover IT support, the tape drive was not compressing, dirty as a mud puddle and needing to be replaced.
Hopefully you've got your care pack up to date and current on that drive.
There are 2 schools of thought on the subject of cleaning. One is to clean frequently. The other is that cleaning is abrasive and frequent cleaning will shorten the life-span of the tape head so you set auto-clean and let the drive take care of the cleaning.
We are performing weekly one drive cleaning.
Some weeks ago BE reported the cleaning media as spent so I ordered a new cleaning media.
Also the drive is just some months old, because we got a new unit end of july this year.
Other question, have you experience with backups of MS Exchange Databases? If yes, what compression do you get there?