defragmentation of EV indexes - best practise

[goatboy]

Hi

Running EV 9.0.2

As per the performance guide:

"You must regularly defragment indexes, ideally while the indexing service is stopped so that defragmentation does not conflict with updates. This is very important if you are using the Accelerator products."

Is it OK to put the indexes in backup mode, rather than stop the indexing services?

I am concerned about the time it's going to take to defrag 1.5TB of indexes.

Any recommendations? e.g. running a daily defrag would be quick and would negate the need to do a long weekly or monthly defrag? Commercial defragmenation products vs native Windows? (I would prefer to use the native defrag on our 2008 R2 server).

Thanks

[AndrewB]

i would think that putting the indexes in backup mode has a sufficient effect for defrag purposes but you have to remember that your defrag tool wont be able to move the bits around if they are in use by EV. that's why they say to stop the service so the defrag has exclusive access to the data.

[Liam_Finn1]

Putting the indexes into backup mode only prevents writing to the indexes but reading is still permitted. Indexes are being read when trying to defrag will cause you problems - you may end up with corrupt indexes. You need to stop the services run the defrag then enable the services again.

Depending on the size of the indexes and the free space on the disk a defrag can take allot of time

From experience I found that stopping the services, doing a full backup then format the disk and do a restore is faster than defrag. Saying that I used to have a spare LUN on the SAN where I did the backup on the active LUN, Mounted the spare LUN, performed a restore to the spare LUN then switched drive letters on the disks. This way my live indexes were untouched until I know the procedure completed successfully.

Doing this without a spare LUN (as i used to do) or without being sure 100% that your backup was good and restored cleanly is risky so be careful. Plan it well if you go this route.

If you plan on Windows defrag and you have 300GB of indexes on that LUN....needless to say it will take quiet some time as windows defrag is a slow process

[goatboy]

Thanks... on one of my EV severs, I have 1.5TB of indexes.

Backing up and restoring the indexes sounds like way too much pain, I will try to see how long Windows defrag takes (this server is a fast VM) or possibly look at 3rd party solutions.

Just to clarify, is it OK to stop just the indexing service? I hope I don't have to stop all EV services.

[AndrewB]

I wouldn't do the backup and restore unless you faced a DR situation. Definitely make sure you have good backups before running the defrag and moving all the bits around. Like I said before, you can have EV in backup mode (which is read only) or sure, go ahead and stop the Indexing service.

[JesusWept3 · Accepted solution]

honestly thinking about it, i dont think you will get that much performance improvement through defragmentation anyway, if you look at the avg size of the index files, they are next to nothing, its not like they're one giant file where contiguous access would make a difference

[goatboy]

hmm, that's a tricky one, it's recommended in the best practices guide and we're trying to do everything we can to improve DA search speed.

Can anyone else share their experiences with what they've done around regular index defragmentation?

[Brian_Spooner]

I don't know anything about your environment...if you're running EV on a VM, if you storage is on DAS, SAN, etc. In our case, EV runs as a VM and our indexes are accessed via NetApp SAN storage. In this scenario, according to NetApp, it's unnecessary to perform defragmentation.

See: https://communities.netapp.com/thread/8226

"To answer your question, I'm not finding any "good canned document" on this unfortunately. I did check with <high-level NetApp engineer> who confirmed that OS-level defrag (i.e. defrag inside Windows of its NTFS volumes) isn't needed. The main reason here is that when Windows writes blocks it assumes that it's writing to a physical disk and having those blocks out of order will cause speed issues. When Windows is writing to a NetApp, WAFL (the NetApp's file system) actually controls where the blocks gets written -- Windows has no idea where the blocks are on the actual physical disks inside the NetApp (since the NetApp controls the physical disks and presents a virtual disk up to Windows, VMware, etc.).

Note: in this regard NetApp is different from other SANs/storage arrays which do actually map LUNs/volumes directly to physical disks (for those there could be a benefit to Diskeeper, etc.).

To put it another way....

• Is there any benefit in running a Windows-level defrag tool? No.
• Will it absolutely hurt anything? Not really but.....
• Will it create larger NetApp snapshots? Absolutely -- as you'll be rewriting many of the blocks inside Windows that wouldn't be touched normally.
• Will it create a LOT of disk traffic between the Windows server (or VMware server if a Windows VM) and potentially slow down other people/servers using the NetApp? Yes as well."

Just thought I'd throw that out there...depending on your setup and storage, it may not be necessary imo.

If your goal is to speed up searches, make sure your indexes are on fast disk (FC, 15k, etc). Before trying to defrag 1.5TB of indexes, you might also try the recommended SQL database maintenance if you aren't performing it already: http://www.symantec.com/business/support/index?page=content&id=TECH74666&actp=search&viewlocale=en_U....

NOTE: If you are unable to stop EV services for weekly maintenance then the maintenance can be run while the services are running. However, Symantec recommends to stop services once per month to perform maintenance. This allows for a more effective maintenance process. In some environments, it may also be necessary to run Update Statistics daily to maintain performance. The daily Update Statistics can be run while the EV services are running. 

-Brian

[goatboy]

Thanks, very interesting. Our environment is all virtual, with SAN storage (EMC's VMAX). Here's an interesting article that goes into depth around defragging VMs: http://blogs.vmware.com/vsphere/2011/09/should-i-defrag-my-guest-os.html

Ugggg, I am going to get our Storage and ESX gurus involved to see what their take is on it.

Thanks for the food for thought.