cancel
Showing results for 
Search instead for 
Did you mean: 

efficiency problems on netbackup for exclusion

grocanar
Level 3

Hi

we have a huge problem of performance with netbackup 7.6.0.4

We have one master server , one media server and a netbackup client in the same machine

and we want to backup a netapp nfs share.

it s mounted on //env/export/nfs7/agc/

we have a  strandard policy to backup this share.

as we use netapp snapshot to backup this qhare we mounted them on

//env/export/nfs7/agc/.snapshot

we don't want to backup these snapshot then we exclude them and it worked wuite well until the share growth.

Right now it doesn't finish.

when we strace the bpbkar process we see this

open("/env/export/nfs7/agc/.snapshot/hourly.1/home/steff/diamond-0.7.4/src/boost_1_57_0/libs/random/test", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 16
getdents(16, /* 109 entries */, 32768)  = 5120
lstat("chi_squared_test.hpp", {st_mode=S_IFREG|0640, st_size=2405, ...}) = 0
lstat("concepts.hpp", {st_mode=S_IFREG|0640, st_size=4581, ...}) = 0
lstat("histogram.cpp", {st_mode=S_IFREG|0640, st_size=5378, ...}) = 0
lstat("integrate.hpp", {st_mode=S_IFREG|0640, st_size=2256, ...}) = 0
lstat("Jamfile.v2", {st_mode=S_IFREG|0640, st_size=6147, ...}) = 0
lstat("statistic_tests.cpp", {st_mode=S_IFREG|0640, st_size=15415, ...}) = 0
lstat("statistic_tests.hpp", {st_mode=S_IFREG|0640, st_size=19089, ...}) = 0
lstat("test_bernoulli.cpp", {st_mode=S_IFREG|0640, st_size=2725, ...}) = 0

Even if we exclude the .snapshot path bpbkar goes in it to check files.

as we have 20+ snapshot it multiplies by more than 20 the time to check the file and it s totally useless.

here is the file that make the exclusion. the name is exclude_list.AGC_Standard as the policy is named AGC_Standard

/env/export/nfs7/agc/alex/MOPAD/patterns
.snapshot
*/.snapshot/*
/env/export/nfs7/agc/.snapshot/

how can i say to netbackup client not to go in the /env/export/nfs/agc/.snapshot during backup?

 

8 REPLIES 8

PatS729
Level 5

Hi,

You must check bpbkar logs as to why exclude_list file is not being honored. Get bpbkar logs on VERBOSE = 5. Else, try creating a new "exclude_list" file as exsting may be gone bad.

Hi

my log file was already VERBOSE=5

in the logs file i see

20:07:59.936 [14843] <4> is_excluded: Excluded /env/export/nfs7/agc/.snapshot by
exclude_list entry .snapshot

then the exclude is honoured

After this line i have numerous line like that

20:08:02.792 [14843] <2> bpbkar SelectFile: cwd=/env/export/nfs7/agc/.snapshot/h
ourly.0 path=quota.user
20:08:02.792 [14843] <2> bpbkar SelectFile: cwd=/env/export/nfs7/agc/.snapshot/h
ourly.0 path=quota.group
20:08:02.792 [14843] <2> bpbkar SelectFile: cwd=/env/export/nfs7/agc/.snapshot/h
ourly.0 path=.profile

but at the end these files are not backuped.

if i look in the log file entries concerning files that are backuped i got

14:19:55.239 [14843] <2> bpbkar SelectFile: cwd=/env/export/nfs7/agc/Data/FTP/INTERPROSCAN/116/Panther/books/PTHR11389 path=SF340
14:19:55.240 [14843] <2> bpbkar PrintFile: /env/export/nfs7/agc/Data/FTP/INTERPROSCAN/116/Panther/books/PTHR11389/SF340/
14:19:55.241 [14843] <2> bpbkar SelectFile: cwd=/env/export/nfs7/agc/Data/FTP/INTERPROSCAN/116/Panther/books/PTHR11389/SF340 path=hmmer.hmm.gz
14:19:55.267 [14843] <2> bpbkar PrintFile: /env/export/nfs7/agc/Data/FTP/INTERPROSCAN/116/Panther/books/PTHR11389/SF340/hmmer.hmm.gz

 

I guess that the file is backuped if there s a line PrintFile in the logs.

Then the backup work like expected.

My problem is that it s made in an horriblous ineffeciency and my backup took too much time.

why the backup process is traversing the .snapshot direcoty as everything should be excluded in it?

as the .snapshot is twenty times huge the the rest of directory you can imagine the big problem it s for me.

mph999
Level 6
Employee Accredited

NBU will go into dir even though it is excluded.  I don't know why it was written like this, but there is no way to stop it.

All I can suggest, is that you log a call and request for it to be changed.

Andy_Welburn
Level 6

A *possible* workaround if this is causing issues - but may depend upon your underlying filer architecture & your specific requirements for the .snapshot directory(ies)

My experience is with NetApp & this allows you to make the .snapshot folder visible or not on a per volume basis - it is a simple tick-box within the snapshot configuration for the volume.

Obviously, your ability to perform this workaround would depend upon your requirements to have the .snapshot folder/directory visible in the first place.

watsons
Level 6

I usually avoid backing up (NAS) nfs share.. generally NAS data is huge, so performance is always the reason behind.

Why don't make use of NDMP policy type to backup your NAS data (& snapshot), you can also use exclude directive in NDMP backup selection, which specifically tells the NAS (netapp) what to exclude.

Thanks for the anwser.

I think it s not a technical question anymore.

I will contact my netbackup commercials to see how i can make this request for enhancement 

Well I need to have the .snapshot mounted for users. 

i just want netbackup client not to go in it.

it s totally useless :)

The main prblem is that NDMP license is quite expensive on netapp. 

As we have aetapp cluster we have to buy it for all the nodes

just because the nbu client goes to 

the exclude directory :(