Information Governance

Once the decision has been make to deprecate older equipment and remove the device from service the decision needs to be made on how to migrate the data to a new device. An Administrator can decide to move shares over in a piecemeal fashion that allows for removal from the old device as they are added to the new device by deleting the old share or move the entire data structure to the new location on a cut-over date. Removing devices removes all the associated shares. Dependent on the version of Data Insight there is the ability to remove shares singularly in 4.5 or earlier and in bulk in 5.0 and newer.

There are ramifications to the data already gathered and stored on the usage, access and metadata that Data Insight maintains.

If reporting to be done is on a scope of time that is included in the past when the old device / share was still in service while being scanned and monitored by Data Insight care must be taken during the removal of the old device / share from the configuration to leave the data in the indexes for the duration of the retention period and available for reporting. Purged data is no longer eligible for reporting purposes and enabling purge on deletion will remove the accumulated data.

You can configure the duration for which you want Data Insight to retain various types of data and the duration after which you want to purge data. Automatic archiving and purging of data is not enabled by default.

To configure the data retention period

  1. Click Settings > Data Retention.

  2. On the Data Retention details page, click Edit.

  3. Enter the following information:

 
Wizard Options
Do the following:
Archive access data automatically
1. Select the check box to enable archiving of file system or SharePoint events.
2. Enter the age of the data (in months) after which the data must be archived.
3. Enter the path of the archive folder.
In case, you want to archive access data immediately, execute the following command on the Indexer node:
INSTALL_DIR/bin/configcli execute_job
DataRetentionJob
Purge access data automatically Select the check box to enable purging of file system or SharePoint access events, and enter the age of the data (in months) after which the data must be deleted.
Purge Data Insight system events automatically Select the check box to enable purging of Data Insight system events, and enter the age of the data (in months) after which the data must be deleted.
Data Insight system events are displayed on the Settings > Events page.
Purge alerts automatically Select the check box to enable purging of alerts, and enter the age of the alerts (in months) after which they must be deleted.
Automatically purge data for deleted shares or site collections Select the check box to enable purging of data pertaining to deleted shares. This option is enabled by default.

If you clear the check box , the index data for the share or site collection remains on the disk until the global archive or purge takes place. Thus, if you decide to decommission a filer, or site collection, the data continues to be available for reporting.
 

 

Click Save.

dataretention_1_0.png

Data Insight enforces the data retention policy twice a month. Archived index segments can be restored using a command line utility called indexcli.exe. The utility is also useful to enforce a more granular archiving or purging policy, if the global option is not sufficient for your needs.

See indexcli.exe syntax in the codicil below.


Index command line utility assists in managing the index segments available on an Indexer worker node.

SYNOPSIS

indexcli.exe
             --display|--archive|--purge|--restore|--rearchive|--list-jobs
             |--stop-jobs [OPTIONS]
indexcli.exe -A <name of the index segments to be archived> 
         
indexcli.exe -c 
indexcli.exe -D <name of the index segments to be purged>
         
indexcli.exe -d
indexcli.exe -h
indexcli.exe -j
indexcli.exe -r
indexcli.exe -t
indexcli.exe -u

Archive options

indexcli.exe -A -a ¦ -f <FILERS> ¦ -m 
<SHARES> ¦ -S <SITECOLLS> ¦ -w <WEBAPPS> ¦ -I
 <MONTHS>
-a

Archives all index segments older than the specified interval.

    -f <name of filer(s)>

    Archives all index segments for the specified list of filers.

      -I <interval in months>

      Archives segments older than the specified interval. The segments which have been restored earlier are not archived.

        -m <name of share(s)>

        Archives all index segments for the specified list of shares.

          -S,--sitecoll <SITECOLLS>

          Archives segments for specified list of Microsoft SharePoint site collections.

            -w, --webapp <WEBAPPS><

            Archives segments for specified list of Microsoft SharePoint Web applications.

              Purge options

              indexcli.exe -D -a ¦ -f <FILERS> ¦ 
              -m <SHARES> ¦ -S <SITECOLLS> ¦ -w <WEBAPPS> ¦ 
              -I <MONTHS>
              -a

              Purges all index segments older than the specified interval.

                -f <name of filer(s).

                Purges all index segments for the specified list of filers.

                  -I ,interval in months.

                  Purges segments older than the specified interval. The segments which have been explicitly restored earlier, for which the lease is still valid, are not purged.

                    -m <name of share(s)>

                    Purges all index segments for the specified list of shares.

                      -S,--sitecoll <SITECOLLS>

                      Purges segments for specified list of Microsoft SharePoint site collections.

                        -w, --webapp <WEBAPPS><

                        Purges segments for specified list of Microsoft SharePoint Web applications.

                          Display options

                          indexcli.exe -d -a ¦ -f <FILERS> ¦ 
                          -m <SHARES> ¦ -S <SITECOLLS> ¦ -w <WEBAPPS> ¦ 
                          -s <STATES>
                          -a

                          Displays information for all shares.

                            -f <name of filer(s)>

                            Displays information for the specified list of filers.

                              -m <name of share(s)>

                              Displays information for the specified list of shares.

                                -s <name of state>

                                Displays index segments for the given state only. Multiple stars can be separated by comma. Possible states are, ARCHIVING, RE-ARCHIVING, ARCHIVED, RESTORING, RESTORED, RESTORE, FAILED, or DELETED.

                                  -S,--sitecoll <SITECOLLS>

                                  Displays information for a specified list of Microsoft SharePoint site collections.

                                    -w, --webapp <WEBAPPS><

                                    Displays information for a specified list of Microsoft SharePoint Web applications.

                                      Restore options

                                      indexcli.exe -r -a ¦ -f <FILERS> ¦ 
                                      -m <SHARES> ¦ -S <SITECOLLS> ¦ -w <WEBAPPS> -C -F <FROM> ¦ > 
                                      ¦ -R <RANGE>
                                      [-L <MONTHS> ¦ -l <MONTHS> ¦ -y]
                                      -a

                                      Restores the index segments for all shares.

                                        -C

                                        If the continue-on-error option is not specified, the restore command fails if the segment files required to restore data for the specified parameters are not available.

                                          -f <name of filer(s)>

                                          Restores all index segments for the specified list of filers.

                                            -F <month from which the segments need to be restored>

                                            Specify the month in the format, YYYY/MM. For example, indexcli.exe -r -F 2010/01 restores segments from January 2010 till date.

                                              -L <interval in number of months>

                                              Resets lease on segments that are restored earlier using -l option. Specify the new lease interval in months. This command replaces the previous lease interval. Setting the value to 0 will make the lease permanent.

                                                -l <interval in number of months>

                                                Restores segments for a temporary lease in months. After the lease expires, restored segments are automatically re-archived. If this option is not specified, segments remain restored till you re-archive them with the -u, --rearchive, option.

                                                  -m <name of share(s)>

                                                  Restores all index segments for the specified list of shares.

                                                    -S,--sitecoll <SITECOLLS>

                                                    Restores segments for a a specified list of Microsoft SharePoint site collections.

                                                      -w, --webapp <WEBAPPS><

                                                      Restores segments for a specified list of Microsoft SharePoint Web applications.

                                                        -R <range in months>

                                                        Restore all index segments for the specified month range. Specify the month in the format, YYYY/MM-YYYY/MM. For example, indexcli.exe -r -R 2010/01-2010-03 restores segments from January 2010 to March 2010.

                                                          -y

                                                          Instead of restoring segments, this option displays the list of files that must be available before restoring the specified segments.

                                                            Re-archive options

                                                            indexcli.exe -u -a ¦ -f <FILERS> ¦ 
                                                            -m <SHARES>  ¦ -S <SITECOLLS> ¦ -w <WEBAPPS>  
                                                            -F <FROM> ¦ -R <RANGE>
                                                            
                                                            -a

                                                            Re-archives all previously restored index segments.

                                                              -f <name of filer(s)>

                                                              Re-archives previously restored index segments for the specified list of filers.

                                                                -F <month FROM which the segments need to be restored>

                                                                Specify the month in the format, YYYY/MM. For example, indexcli.exe -u -F 2010/01 restores segments from January 2010 till date.

                                                                  -m <name of share(s)>

                                                                  Re-archives previously restored index segments for specified list of shares.

                                                                    -S,--sitecoll <SITECOLLS>

                                                                    Re-archives previously restored segments for a specified list of Microsoft SharePoint site collections.

                                                                      -w, --webapp <WEBAPPS><

                                                                      Re-archives previously restored segments for a specified list of Microsoft SharePoint Web applications.

                                                                        -R <range in months>

                                                                        Restore all index segments for the specified month range. Specify the month in the format, YYYY/MM-YYYY/MM. For example, indexcli.exe -u -R 2010/01-2010-03 restores segments from January 2010 to March 2010.

                                                                          EXAMPLES

                                                                          EXAMPLE 1: The following command archives index segments for specified list of filers.

                                                                          indexcli.exe -A -f \\filer1,\\filer2,ID1,ID2

                                                                          EXAMPLE 2: The following command archives index segments for specified list of shares.

                                                                          indexcli.exe -A -m \\filer1\share1,\\filer2\shares2,ID3,ID4
                                                                                   
                                                                          

                                                                          EXAMPLE 3: The following command purges index segments for specified list of filers.

                                                                          indexcli.exe -D -f \\filer1,\\filer2,ID1,ID2
                                                                                   
                                                                          

                                                                          EXAMPLE 4: The following command purges segments for specified list of shares.

                                                                          indexcli.exe -D -m \\filer1\share1,\\filer2\shares2,ID3,ID4
                                                                                   
                                                                          

                                                                          EXAMPLE 5: The following command restores index segments for specified list of filers.

                                                                          indexcli.exe -r -f <\\filer1,\\filer2,ID1,ID2>

                                                                          EXAMPLE 6: The following command restores index segments for specified list of shares.

                                                                          indexcli.exe -r -m \\filer1\share1,\\filer2\shares2,ID3,ID4
                                                                                   
                                                                          

                                                                          EXAMPLE 7: The following command re-archives previously restored index segments for specified list of filers.

                                                                          indexcli.exe -u -f \\filer1,\\filer2,ID1,ID2
                                                                                   
                                                                          

                                                                          EXAMPLE 8: The following command re-archives previously restored index segments for specified list of shares.

                                                                          indexcli.exe -u -m \\filer1\share1,\\filer2\shares2,ID3,ID4
                                                                                   
                                                                          

                                                                          EXAMPLE 9: The following command archives segments for specified list of Microsoft SharePoint site collections.

                                                                          indexcli.exe -S,--sitecoll<http://sp_webapp:8000/sc1,ID2,ID3...>
                                                                                   
                                                                          

                                                                          EXAMPLE 10: The following command archives segments for specified list of Microsoft SharePoint Web applications.

                                                                          indexcli.exe - w,--webapp<http://sp_webapp:8000,ID2,ID3,...>
                                                                                   
                                                                          

                                                                          EXAMPLE 11: The following command purges segments for specified list of Microsoft SharePoint site collections.

                                                                          indexcli.exe -S,--sitecoll<http://sp_webapp:8000/sc1,ID2,ID3...>
                                                                                   
                                                                          

                                                                          EXAMPLE 12: The following command purges segments for specified list of Microsoft SharePoint Web applications.

                                                                          indexcli.exe - w,--webapp<http://sp_webapp:8000,ID2,ID3,...>
                                                                                   
                                                                          

                                                                          EXAMPLE 13: The following command displays information for specified list of Microsoft SharePoint site collections.

                                                                          indexcli.exe -S,--sitecoll<http://sp_webapp:8000/sc1,ID2,ID3...>
                                                                                   
                                                                          

                                                                          EXAMPLE 14: The following command displays information for specified list of Microsoft SharePoint Web applications.

                                                                          indexcli.exe - w,--webapp<http://sp_webapp:8000,ID2,ID3,...>
                                                                                   
                                                                          

                                                                          EXAMPLE 15: The following command restores segments for specified list of Microsoft SharePoint site collections.

                                                                          indexcli.exe -S,--sitecoll<http://sp_webapp:8000/sc1,ID2,ID3...>
                                                                                   
                                                                          

                                                                          EXAMPLE 16: The following command restores segments for specified list of Microsoft SharePoint Web applications.

                                                                          indexcli.exe - w,--webapp<http://sp_webapp:8000,ID2,ID3,...>
                                                                                   
                                                                          

                                                                          EXAMPLE 17: The following command re-archives previously RESTORED segments for specified list of Microsoft SharePoint site collections.

                                                                          indexcli.exe -S,--sitecoll<http://sp_webapp:8000/sc1,ID2,ID3...>
                                                                                   
                                                                          

                                                                          EXAMPLE 18: The following command re-archives previously RESTORED segments for specified list of Microsoft SharePoint Web applications.

                                                                          indexcli.exe - w,--webapp<http://sp_webapp:8000,ID2,ID3,...>