cancel
Showing results for 
Search instead for 
Did you mean: 

low speed when backup many small files

turguns
Level 5

Environment

netbackup 7.1 ,os : Redhat 2.6

Hello All

We have one policy with standart type which is backing up file system,

It  starts appr. 60 MB/s.  when the backup process reach to the path which are located many small files the speed decreasing till 2 MB/s.

How  to solve this issue??

With BR,

Turgunbek

1 ACCEPTED SOLUTION

Accepted Solutions

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

I agree with Nicolai - quite normal...

How long does it take to run 'ls -lR' against this filesystem?
NBU (bpbkar) needs to do the same in addition to reading each file in order to backup it up.

The alternative is to see if FlashBackup is an option.
Compatibility Guide over here: http://www.symantec.com/docs/TECH126901

Or else multi-streaming as per Nicolai's suggestion:

NEW_STREAM
/big-volume1/folder1
/big-volume1/folder2
NEW_STREAM
/big-volume1/folder3
/big-volume1/folder4
NEW_STREAM
/big-volume1/folder5
...etc

View solution in original post

11 REPLIES 11

Nicolai
Moderator
Moderator
Partner    VIP   

This is quite normal.

  • Clean up the files if possible.
  • Improve disk speed (fragmentation, kernel inode (some systems), disk cache etc etc
  • If it durable - accept it as it is.

Else deploy backup via multiple data streams

e.g: /data/data*

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

I agree with Nicolai - quite normal...

How long does it take to run 'ls -lR' against this filesystem?
NBU (bpbkar) needs to do the same in addition to reading each file in order to backup it up.

The alternative is to see if FlashBackup is an option.
Compatibility Guide over here: http://www.symantec.com/docs/TECH126901

Or else multi-streaming as per Nicolai's suggestion:

NEW_STREAM
/big-volume1/folder1
/big-volume1/folder2
NEW_STREAM
/big-volume1/folder3
/big-volume1/folder4
NEW_STREAM
/big-volume1/folder5
...etc

Nicolai
Moderator
Moderator
Partner    VIP   

You can also run bpbkar by hand to see how fast possible data can be read:

time /usr/openv/netbackup/bin/bpbkar  -nocont -nofileinfo -nokeepalives /directory_to_read  > /dev/null 2> /tmp/file.out

This command read data from file system just like a normal backup, but the data is not transferred to media server but discarded. 

 

turguns
Level 5

Thank you Marianne and Nicolai..

The netbackup version is 6.0 on  client side.

So Fashbackup will not work((..

Stream also not a variant. Because of VTL.

blind alley (((

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

The netbackup version is 6.0 on  client side.

Is this telling us about the client what I think??
6.0 because of old, slow, 32-bit OS and hardware???

By the way... FlashBackup has been supported on Linux for many versions back - depending on Volume Manager/File System.
You should really try to get this client updated to at least NBU 6.5.6.

Stream also not a variant. Because of VTL.

Multi-streaming is no problem with VTL.
As long as multiplexing on the STU is disabled to ensure that each stream backs up to a different virtual drive.

turguns
Level 5

hardware is old but strong)).. E2900 sun server with 65GB RAM and 8 CPU..populated 3 CPU MEM board

So why solaris 9? Because of application requirement on it.

My friends which are they tried (3 years ago ) to install netbackup 7 version was unsuccessful. So they told me this system is compatible only with 6 version.

 

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

So, this is the first time you are mentioning Solaris 9... 
You have selected Linux in your opening post. Probably master server, but it is important that we understand Client environment as well as this is where you are experiencing performance issues.

Solaris 9 is supported as NBU Client until NBU 7.5. Please see NBU OS SCL:  http://www.symantec.com/docs/TECH76648

The reason why installation/upgrade failed is probably because tar instead of gtar was used to extract installation binaries. See: 
http://www.symantec.com/docs/TECH154080

Another user's experience: Not able to install Solaris Client 7.1 on Solaris 9

About multistreaming with VTL - please see my post above.

FlashBackup is also perfectly supported for Solaris.

turguns
Level 5

I am afraid of multistreaming will against the dedup ,because of VTL.

I will try to install netbackup 7.1 to this machine.

can you clarify the how to create FlashBackup policy. Because I tried on test machine ,but I couldnt.

I used this document (chapter 4): 

http://kbdownload.symantec.com/resources/sites/BUSINESS/content/live/DOCUMENTATION/5000/DOC5165/en_US/NetBackup7.5_AdminGuide_SnapshotClient.pdf?__gda__=1393256291_e15962d9ff3065f379f78cb1469590b8

 

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

PLEASE do not confuse multistreaming with multiplexing.

With VTL, you can create multiple virtual tape drives with MPX = 1.
This will ensure 1 stream per virtual tape drive.

Multistreaming will generate more than on stream on the client.
Because multiplexing is OFF on the STU, each stream will go to own virtual tape drive.

FlashBackup works with Enterprise Client license and if configured correctly.

Once you have upgraded this Client to NBU 7.5, please start a new discussion for assistance with FlashBackup config.

Or try multistreaming.

 

turguns
Level 5

Thank you Marianne for your effort to help and showing way to reach the solution.

Multistreaming is not valid for us.Because  everytime application generating new folders.

Only FlashBackup is a variant.

For this I couldnt install the Netbackup 7.5.I got this messages :

           Do you wish to continue? [y,n] (y) y
 
       NetBackup 7.5 server software is not supported
      on Solaris sparc operating systems older than Solaris10.

So I installed Netbackup 7.1 version on this client.

Now as you mentioned above I am starting new discussions for FlashBackup assistance.

Thank you.

 


 

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

NetBackup 7.5 server software is not supported
      on Solaris sparc operating systems older than Solaris10.

 

You tried to install the Server installation package instead of Client software.

To get back to your original issue:

While you are battling to get FlahBackup to work, please follow advice in the meantime to enable Multistreaming on the Client.
Keep MPX on VTL STU as 1 to ensure each stream goes to different VTL tape drive.

See this discussion:  https://www-secure.symantec.com/connect/forums/multistream-and-dedup