File System Archiving performance best practices (EV9)

[roaryT]

Hi,

Can anyone confirm if performance is enhanced by using archive points for all subfolders of a large (say 100 GB +) data share when selecting from the add new folder wizard - a new folder within a volume?  The performance is measured over the alternative which is to create a large share as its own volume and setting the archive point at the root of that volume.  I guess the questions is - Are multiple archive points better than 1, or can the same performance be acheived by spittig a large volume down into smaller shares each havign their own archive volumes withouth using folders?

Hope this makes sense! :)

[Darren_Locke]

While it may not get you much better archival performance, having multiple archive points is certainly desirable from the perspective of keeping your archives and indexes small, manageable and performant.

Best practise for archive points is to create one in the root of each users home folder or in the root of department, common or project folders. One archive point for a whole volume is not best practise. Try to create them on the volume roots subfolders if that makes sense.

Darren 

[roaryT]

Thanks Darren, yes that makes sense but my query is that with this method say I am targeting Users home dirs volume which is say 300 Gb or a Department Share vol over 600 Gb am I limited to having one task to schedule and also will I have in my Archives window an archive for every subfolder in  Users or Department which could be thousands. 

Sorry I could test but am limited for time but appreciate your expert advice.

Regards,

[gmackinn]

In my experience the users home drives will all have an explicit permission for the user so you will need an archive point for each unless you are set to ignore explicit permissions (not recommennded). I find we have the same with project and group shares so single archive points will not work anyway so we set an archive point usually at the level of individual shares as that is usually the level where explicit permissions are applied and everything beneath is inherited.

[roaryT]

Thanks so re the explicit permissions are you saying that if I have an archive point at the users share level and the policy is set to archive explicit permissions then it wont work as expected and that I would require an archive point at every user folder unless explicit permissions were not being archived?  Its not too clear.  Thanks,

[gmackinn]

The normal setting is to not archive where there are explicit permissions. So if you have h:\home\user1, h:\home\user2, h:\home\user3 and each user has explicit access to the root of their home drive and you set your archive point at h:\home nothing would ever archive as EV will ignore everything under the level where the explicit permissions are set. So when you look at security you only ever want to see inherited permissions from the level where you set your archive points. So to decide where to set archive points I check where explicit permissions are set in the file structure and set the archive points at that level.

Sometimes with large group shares there are sub folders that have a different explicit permission and we add additional archive points for these to get the data contained in them archived.

[roaryT]

Thanks but surely by setting to archive explicit permissoins the users folders will get archived even if the archive point is set on the top level folder?

[gmackinn]

The archive will be set with the NTFS permissions of the folder where the archive point is created. n  all the users will not be able to access the archive and if you grant them all access they wil have access to each other's da The problem is if you choose to archive explicit permissions you may give people access to data that they should not have access to.  e.g. if you archive P:\Groups\Accounting and there is a subfolder with explicit permissions P:\groups\accounting\salaries suddenly alll the users with access to accounting would also have access to the salaries data when it is archived.

[roaryT]

thanks you have been most helpful on this subject, finally can you confirm the outcome if I set the archive point at the root of a volume and I dont select to archive explicit permissions (the default)?  Like would less data get archived and would users not get permissions to view archives they shouldn't have access to?

[gmackinn · Accepted solution]

I think only your administrators will have NTFS permissions on the root of a volume and so nothing will archive. I'd expect each of the home or project or group folders beneath the root to have explicit NTFS permissions set granting individual users or groups access to the folders so EV will ignore them. It is also worth noting that even if these explicit permissions match the inherited permissions the data will not archive. We had some LUN's where the local administrator account had been explicitly granted all the way down the structure as well as being inherited, nothing archived until we removed the explicit permission.  Moreover if you only have that one archive you will need to give everyone access to it and if you do not already give them acess to the whole volume I would not give them access to the whole archive. It is really simple to add loads of archive points, we have some large LUN's with 3000 home drives and it only takes about 5 minutes to set up. In file server targets select the volume right click and select New > Folder, browse to Home and select it, deselect the "Archive this folder" and select what policy you want to use, then select either the second or third radio buttons to creat an archive point for every folder underneath "Home". I have never really used the "Auto enable" option but I've no reason to think it will not work, I guess it is useful if you move home drives about and don't want to have to manually add the archive points. We do pretty much the same with group and project shares which are generally one level down in a Data folder on or drives.

[roaryT]

Thanks really useful infromation.

[roaryT]

on further reading and consideration  the option to ignore explicit files is the default and I think maybe for a good reason - to me its saying if you archive that file which has a specific permission set then it will take the default folder permissions from inheritance and thus make available to anyone who has access the the folder.  I can see this may lead to less archived data but it makes sense to ignore those files.  I agree to creating an archive point for each  sub folder but am leaning towards leaving explicit perms as to be ingored. What do you think?

Thanks,

[gmackinn]

Yes, I'd never, ever change the explicit permission setting and have loads of archive points.

[roaryT]

One issue I see with this method is you only have one task per volume whereas I'd of thought it would be better to create more volumes for the shares below the root volume for performance, ie 10 tasks are better than 1 doing large vols of data?

[gmackinn]

The 1 task you run against a volume will be multithreaded, it will be ingesting/indexing more than 1 file at a time. We have no requirement for more tasks, initially we had issues with very large drives not completing the whole disk in the window we had for running between SQL backups but the checkpointing feature now solves that issue.  

[roaryT]

yeah that's what I expected am keeping the tasks to a minumum now and going with this approach.  Thanks again for info.

[FreKac2]

Maybe I've missed something in the descriptions here.

But it's not really necessary to create multiple archive points in order to archive explicit permissions and still keep other users from seeing what they don't have access to.

The key here is if you've changed the regkey SynchronizeFSASharePermissions or not.

If it's at its default value then yes you would need to check exactly how things are setup, since EV will synchronize the permissions on a share level by default.

If you set it to 0 Ev will synchronize permissions on a folder level rather than share level.

So even if they belong to the same archive they can't see folders they don't have access to.

The reason there is a question in the policy in regard explicit permissions is because EV don't do file-level synchronization of permissions.

So if you e.g. restore a file, they will get the folder permissions no matter what they were set as, on a file level before they were archived.

[gmackinn]

Gosh, another can of worms to chew through. I believe the default setting for the registry key SynchronizeFSASharePermissions  is now NTFS permissions not share permissions.  I just assumed this was a new install of EV, I think the default changed around V8.  This makes sense as Microsoft best practice is to have open share permissions and restrictive NTFS permissions. The critical part is that the entire archive permissions are set from the folder where the archive point is set.  If your archives are set to these open share permissions people will have access to data they should not have access to so it's worth checking.

This does not really change the need to creatine multiple archive points. I can think of many other reasons to have multiple archive points as well. Searching for users and administration of archives and index's will be better. Load balancing is much simpler if you do not have to split up an archive, eg if your home drive volume is running out of space and you have to move 100 of 600 users home drive shares to another drive if you only have one archive point you will have to recall data to move the users (not ideal if you are out of space) or move the whole lot. With individual archive points fsautility can be used to move the placeholders for just the home drive shares without restoring any data.

[FreKac2]

Nope it was not changed in EV8, I have checked :)

I've checked as well (today, using EV9 sp1) that if you don't have permissions to a folder in the archive you won't see it.

I don't disagree that having multiple archive points have advantages but it's not necessary in the sense that it doesn't work or that permissions will somehow be circumvented.

[gmackinn]

I disagree, for the security model I have described above if each user or group has an explicit permission on their own folder if you set an archive point one level above that nothing in any of the folders will archive. If you use the setting to ignore explicit permissions you risk giving people access to files in the archive that they should not have access to.

You may be right that the default has not changed, I've not done a new install recently only upgrades where I had already set the key.