Centera Migration - while it is running

[Hucky]

Hi People,

Hope you can way lay some fears...currently running the NTFSCenteraMigrator tool...

Collections enabled on the Centera Partition....I've selected small date range via the migrator tool CLI...should equate to about 20gb...

(note that I already tested to the centera with a smaller range/amount but collections were not enabled and it worked fine)

The collection folder temporary location is currently at 10gb and growing!? no change showing on the centera.

Will this not start transferring to the centera until the 20gb of data is collected?

(which may lead me on to more questions..pending answers :)

Many thanks

[Liam_Finn1]

  Hucky,

I dont believe this is normal. Are there any event warnings or errors?

Collections normally are collected into 10MB files which are then transferred to the centera.

In your vault store partition properties do you have the IP's of the Centera listed? Do you have the PEA file specified?
Did you click on the Test Settings button to validate that the connection to the Centera was working

Did you run the Centera Veify too to validate all communication to the Centera is ok

[Hucky]

Hey Scanner001,

Cheers - PEA and testing/verify all good.

Nothing directional in the logs....I'm wondering if I just kicked this off too quickly after opening the centera partition. 
As I tested first without collections, then closed centera partition and re-opened the NTFS partition for the normal daily archiving run.....then on the second test, using collections, I had a migrate job ready to run, I closed the NTFS and re-openend the centera partition and then restarted the storage service to kick off the migrator run.  The files did eventually move up to the centera, but missed whether this was at the end of the 20gb or before!  Either way they are all moved and the temp location is empty upon my return.

I'll do another, smaller, test and re-post the results.

On the 10mb front....
Not using safetly copies......will the icon not change/reflect on the mailbox item until the 10mb collection file has been transported to the centera? wondering if there's a risk of data loss that's all.

Cheers.

[Liam_Finn1]

 There is very little chnace of data loss. UNtil the data is committed to the centera the files in the collections are not deleted

Also the adding of files to the blob (10MB) is very fast so there is very little window to loose data

On the Centera you only use safety copies if you have two Centera devices replicating. when you have one you remove copies immediate after archiving.


Regarding the shortcutting, it is almost immediate because the move of the data to the Centera is very fast

Centera is a beast and it can take anything you throw at it

Some things to note regarding Centera

Centera stores data as BLOBS so it does not know what a file is.
BLOBS are stored inside a C-Clip (this is basically a storage box for the BLOBS
Each C-Clip can have more than one BLOB
When you run expiry on EV if all of the BLOBS in a C-CLIP are not expired then you dont get your space back

Example

I store 5 * 2MB files on Centera and the files are stored in the one C-Clip

I run expiry and 4 of the 5 files are expired. This means one of the items in this c-clip are not expired

Because of this one item the whole C-Clip is still on the Centera even though the items are gone from EV the data actually still resides on the Centera.
So if you know the Clip ID and the SSID you can actually retrieve the data from the Centera

This also means that the 10MB used by that C-Clip is still being used on the Centera because the Centera deletes C-Clips not blobs

When expiry runs and the last item in the clip is expired, then and only then is the C-Clip deleted and all the data contained in that C-Clip is deleted.

[Liam_Finn1]

 Dont forget if you got the answer you were looking for please mark the thread as solved in the item that answered your question

[Hucky]

Hi,

Thanks for the reassurance/info scanner001 :)

another couple of small tests...both using the migration tool...with/without the collections enabled
1) collections disabled on the centera partition....5gb flew through and shows as expected on the centera
2) collections enabled,....1.9gb using the migrator tool - temp 'collection' folder, specified on the centera partition, has the whole 1.9gb sitting in it...

Restared the storage service....nothing just sitting there for the past 30mins?

This system has been through many upgrades..to the latest 8.0 sp1 and has always been backed up via arcserve/brightstore Ntfs....anything I'm not aware of when changing to the Centera that may have been configured via registry etc...? hoping something you may be aware of.
noted that the "storage file watch" event doesn't report even looking at the centera partition?

I'm baffled and will pick this up again tomorrow.

Cheers.

[Hucky]

Well as I didn't get to leave quite as I was hoping...

The collection folder 90minutes later is down to 1.3gb from 1.9...so I guess they're trickling to the centera.

Nothing in the logs either..hmm

now it can wait!

Cheers.

[Liam_Finn1]

 how many access nodes do you have on your Centera

[Hucky]

This is a configuration between New York and London,

All migration work being carried out on the New York Site at present,

New York Centera replicates to London Centera.

EV partition set to the New York Centera via two IP addresses with three London Centera IP addresses as replica.

Checked the Centera replication status and it was always behind by 1-2 hours, but then I was pushing a fair bit of data over and it did not affect the 'non collection' test from continuing to place data on the New York Centera.

Think the collections are more dependent on the replica status then? maybe something to do with the 'scan partition every 60 minutes' (default) setting?

[Liam_Finn1]

 Maybe,

May I ask how many Centera nodes do you have in each of the Centera devices

EMC recommends that for a 16 node Centera cube that you have 4 of the nodes configured as access / storage nodes (this is you have a Gen4) If not a Gen 4 then you need 4 access nodes and the remainder as storage nodes

In the Gen4 Centera's the nodes can be both access and storage but in earlier versions this was not possible

[Hucky]

That the New York Centera is a 4 node cube, 2 nodes are configured as access/storage, 2 as storage.  Gen4 centera.

[Liam_Finn1]

 oh ok that explains it.

The reason i asked is i have 3 cubes in my cluster which is 48 nodes and i have 16 access/storage nodes in my setup

[Chris_Harrison]

By default storagefilewatch scans a Centera partition every hour, or immediately upon startup of the storage service.

So i would think your issue is related to the fact that SFW just hasn't started processing the items.

I would:

stop storage service
set a dtrace of SFW
start storage service
leave running for 10 minutes

You *should* see items clearing down and the dtrace should confirm...

[Hucky]

Cheers Scanner001/Chris,

I'll setup a dtrace of the SFW and see what's going on.

:)

[Liam_Finn1]

 Hucky

Any update on this????

[MichelZ]

Hi there

Do you need more info on this subject?
If not, could you mark a Post that suits as a Solution?

Thanks & Cheers
Michel