Forum Discussion

watcharapongs's avatar
9 years ago

How to extend the dedup diskpool without data migration?

Hi,

 

Here is my backup system environment:

- 1 Master server (windows, NBU 7.5.0.4)

- 2 Media servers (Windows, NBU 7.5.0.4) - Dedup disk pool configured

- 2 Media servers (Windows, NBU 7.5.0.4) - Advanced disk pool configured.

- D2D2T, SLP policies configured to migrate data from disk to tape for longer retention.

 

I found that Advanced disk is easy to extend the space, just create new disk volume from SAN storage then merge to existing disk pool. The drawback is it has no data dedup feature, the free space is running out increasingly.

 

Here is my questions:

1. Is it possible to directly convert existing advanced disk pool to dedup disk pool without to migrate date?

2. As from KB https://www.veritas.com/support/en_US/article.000083795, the dedup disk pool has to be created from a single disk volume and the extention means to recreate and migrate data. Is it a better way to extend dedup disk pool? Is it possible to increate the disk volume size on SAN storage then OS then reconfigure something on NBU to recognize the new bigger disk volume?

3. We found data transfer from dedup to tape is slighly slow when comparing to advanced disk. Is there any way to improve the data transfer performance on dedup diskpool type?

4. Is there the tool to estimate the required dedup diskpool size to support backup data ?

 

Your suggestion is appreciated, thanks.

 

 

  • 1. Is it possible to directly convert existing advanced disk pool to dedup disk pool without to migrate date?

    No.  Duplication from one storage unit to another is th eonly way.

    2. As from KB https://www.veritas.com/support/en_US/article.0000..., the dedup disk pool has to be created from a single disk volume and the extention means to recreate and migrate data. Is it a better way to extend dedup disk pool? Is it possible to increate the disk volume size on SAN storage then OS then reconfigure something on NBU to recognize the new bigger disk volume?

    If your MSDP storage is on a SAN LUN, then you may be able to grow the LUN, and then use Windows OS to grow the NTFS Volume (i.e. the drive letter) to expand in to the new free space.

    3. We found data transfer from dedup to tape is slighly slow when comparing to advanced disk. Is there any way to improve the data transfer performance on dedup diskpool type?

    There's no quick magic win.  You may be able to tune tape buffers, or tune some of the MSDP "read buffer" values.

    4. Is there the tool to estimate the required dedup diskpool size to support backup data ?

    Not public.  Vendor/channel/partner/Veritas pre-sales do have a rough sizing calculation spreadsheet, but it is not validated for customer use.  Can you not use your existing dedupe rates to get an idea?  Or use the figures from bperror CLI report and scrape the pre and post dedupe sizes.

    .

    Is your MSDP pool on an NTFS volume?  Did you set an NTFS block-cluster-factor size?  Turned off 8.3 and "atime" updates?  Is the MSDP LUN/meta-volume/spindles behind a RAID card?

  • 1. Is it possible to directly convert existing advanced disk pool to dedup disk pool without to migrate date?

    No.  Duplication from one storage unit to another is th eonly way.

    2. As from KB https://www.veritas.com/support/en_US/article.0000..., the dedup disk pool has to be created from a single disk volume and the extention means to recreate and migrate data. Is it a better way to extend dedup disk pool? Is it possible to increate the disk volume size on SAN storage then OS then reconfigure something on NBU to recognize the new bigger disk volume?

    If your MSDP storage is on a SAN LUN, then you may be able to grow the LUN, and then use Windows OS to grow the NTFS Volume (i.e. the drive letter) to expand in to the new free space.

    3. We found data transfer from dedup to tape is slighly slow when comparing to advanced disk. Is there any way to improve the data transfer performance on dedup diskpool type?

    There's no quick magic win.  You may be able to tune tape buffers, or tune some of the MSDP "read buffer" values.

    4. Is there the tool to estimate the required dedup diskpool size to support backup data ?

    Not public.  Vendor/channel/partner/Veritas pre-sales do have a rough sizing calculation spreadsheet, but it is not validated for customer use.  Can you not use your existing dedupe rates to get an idea?  Or use the figures from bperror CLI report and scrape the pre and post dedupe sizes.

    .

    Is your MSDP pool on an NTFS volume?  Did you set an NTFS block-cluster-factor size?  Turned off 8.3 and "atime" updates?  Is the MSDP LUN/meta-volume/spindles behind a RAID card?

  • Yes extending dedup disk pool is possible and descriped in the Deduplication Guide,

    basically you stop netbackup on the media server containing the dedup disk pool, extend the volume, start netbackup on the media server.

    Have done from 32 TB to 48 TB, without any problems.

    The person that has written that technote, seems to have forgotten about dynamic disk in windows. Know it is not much used, but has been a feature since Windows 2003

     

    Not the precise size, but have heard same front end data (1 full of all systems) and 60% of front end data

  • I disagree with using a simple calculation, which can easily catch you out.  The size of your dedupe pool is dependent upon several other and very different factors.  Not necessarily the size of one full plus one other factor.   It is initial uniqueness (i.e. not initial size) + rate-of uniqueness (i.e. not rate of change) + retention...    which determine the size of dedupe storage.   N.B: the longer you retain in dedupe the more likely it is that the oldest backup data becomes more unique (and apparently swells in size although you can never see it).   Dedupe requires a completely different thinking hat on.

    If you really need a tool and want to see how one should approach it - then ask your Veritas TAM whether he/she can spend an hour on-site with you to model it in their calculation tool.  If your TAM is kind, and has time, get them to dig in to the Excel cells that sit behind the calculations and show them to you - and you'll see some cells with over 100 cross references to other cells and multi-level nested IFs and Elses... all very very complicated.

    Remember, the reason this tool can never be made public is because no-one can ever swear by it... because it all comes down to how unique your data is... and this is very very dificult to model accurately - and so the tool will never be, and can never be, released... because unscrupulous customers will use it to challenge a vendor when customers get it horribly wrong and need to find someone else to blame.  This isn't Veritas being cagey.  No sensible vendor is going to publish tools which, by the very nature of the paradigm that is dedupe, can never be 100% accurate.

  • 3) Re performance:

    What does this show on one of your media servers:

    powercfg -list
    
    findstr "ContentRouter CRDataStore BufferSize" "D:\Program Files\Veritas\pdde\contentrouter.cfg"
    
    fsutil fsinfo ntfsinfo X: | find /i "per cluster"
    fsutil behavior query disable8dot3        (i.e. show system default)
    fsutil behavior query disable8dot3 X:
    fsutil behavior query disablelastaccess
    reg query "HKLM\SYSTEM\CurrentControlSet\Control\FileSystem"
    reg query "HKLM\SYSTEM\CurrentControlSet\Services\RpcXdr\Parameters"        /v "DefaultNumberofWorkerThreads"
    reg query "HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Executive" /v "AdditionalDelayedWorkerThreads"
    reg query "HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Executive" /v "AdditionalCriticalWorkerThreads"
    
    netsh int ipv4 show dynamicportrange udp
    netsh int ipv4 show dynamicportrange tcp
    netsh int ipv6 show dynamicportrange udp
    netsh int ipv6 show dynamicportrange tcp
    reg query "HKLM\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters" /v "MaxUserPort"
    
    reg query "HKLM\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters" | sort
    reg add   "HKLM\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters" /v "KeepAliveTime"
    
    netsh interface tcp show global
    reg query "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters"

    You should be able to use the above items in Gogle searches to help get you started along the road of tuning up your Windows based MSDP server.

    i.e. readup on the topics surrounding each of the technical elements above.  I can't tell you what to change in your environment.  You have to make reasoned decisions and choices.

  • Thanks for the comments.

    Do you have any suggestion on how to calculate or any NBU tool to check the total maximum backup throughput to a MSDP disk pool?

    We need that assessment information of the current existing backup system to support new data growth and new servers.

  • Again, this a very difficult to calculate.  E.g. if you were to backup only zeroes to an MSDP pool then you would see some ridiculously high transfer rates, and if you were to backup only utterly unique data then the throughput levels would be much lower.

    You can use GEN_DATA style backup policies (if you have some Linux/Unix) clients to hand... and use such test backups to see what effect best case, and worst case... levels of sameness/randomness(i.e. uniqueness) have.