Tape bottle necks & archive bit

[Jim-m]

Interesting observations when doing some stress testing on a new setup
OS - Windows SP2 x64
NBU - 6.5.1
LTO3 tape drives & tapes
Disks - external disk arrays - HP's of some sort

Bottleneck
Any ideas about the tape bottleneck ?
All test data source is SAN connected  to media server  and comes in flavours  :
- 50 GB files highly compressible
- 50 GB files incompressible
- moderate number ( ~ 1 million) random files (< 512 K ) in tree structure
- 20x the previous with files < 8k

Backing up to a tape - no problems tape averages about 100 MB/s.  If run as a single stream most averages just under 25 MB/s, exception being the 20 million files.  Conclusion the disk is a bottleneck at just under 25 MB/s.

Backing up to disk storage unit (STU) on the same box but the source and target STU are on different raid sets within the same array.  Results are interesting - Starts off on a rush  100 - 150+  then settles to about 55 - 65 MB/s will compressibility influencing rate (no compression turn on in NBU). The initial rush can be explained by the disk array cache but the disk-2-disk is > 2x to tape there fore my previous conclusion is not valid.    This test would assume the max more about the 65 MB/s  and not about 25 MB/s. I have not tested with bkbkar32 -nocont path > NUL yet

Archive  Bit
Does calendar-based backup cause a disk write for each file?  I suspect not
Rummaging through STN some people are under the belief that doing a windows backup based on calendar updates some info  in the file properties

Why should we care.  Backing up windows servers with huge numbers of files is notoriously slow.  It can be slowed down even further by causing a disk write for each file i.e updating the archive bit.

If NBU was smart it would send the last full back info to the client and compare the modification time with the last full backup.   It would be interesting to see what happens and what the mechanism is for when different data on a box is backed up (full) by different policies at different times i.e the one large box may have many full backup times for different data locations

Regards Jim 

[Darren_Dunham]

Use of the archive bit is a choice.  You can disable it on the client under Host Properties.  I'm not sure if it is still updated or not when that change is made.

--
Darren

[Jim-m]

I did that and IO went from 6.8 MBs to 8.9 MB/s superficially it appears that using calendar-based full backups are faster cause the archive bit is not being updated for each file (requires more job runs to reach a better conclusion, also we are dealing with 20+ million files in one drive letter). 

But according to some here calendar-based backups also cause a disk write which makes the argument that they are faster invalid becuase that would also a cause a disk write for every file that's read.