can't backup directory with 15 million files

[sa2]

Dears ,

wish you have a nice day 

i have a directory exist on aix client , this directory contain 15 million files 

making "ls" command  on it not working ,  because the mechanism of how "ls " work because it is need to buffer output

i use "find . " to display files in it

the backup process on it hanged with status code 13 as it waits long time before it can't find files to backup

at first , i think the problem is from network , so i have to elimante the network by using this command , it also hang 

 /usr/openv/netbackup/bin/bpbkar -nocont /siebel/siebelfs/att > /dev/null
INF - Estimate:-1 -1
Edi - 0 0 0 4 1 82413 root system 0 1497351412 1497278395 1497278395 /
Edi - 0 0 1 1 8 82413 siebel siebel 0 1497351412 1497347215 1497347215 /siebel/
Edi - 0 0 2 1 17 82285 root system 0 1497351412 1486489228 1486662433 /siebel/siebelfs/
Dir - 0 1 3 1 21 82431 siebel siebel 0 1497351412 1497351398 1497351398 /siebel/siebelfs/att/

this is log file of bpbkar

12:56:52.327 [24051764] <2> mount build_mount_list: INF - Processing (local) /dev/lv01 on /siebel
12:56:52.329 [24051764] <2> bpbkar SelectFile: INF - cwd = NULL
12:56:52.329 [24051764] <2> bpbkar SelectFile: INF - path = /siebel/siebelfs/att
12:56:52.332 [24051764] <2> bpbkar resolve_path: INF - Actual mount point of /siebel/siebelfs is /siebel/siebelfs
12:56:52.332 [24051764] <2> bpbkar SelectFile: INF - Resolved_path = /siebel/siebelfs/att
12:56:52.336 [24051764] <2> bpbkar resolve_path: INF - Actual mount point of /siebel/siebelfs/att is /siebel/siebelfs/att

Thanks in advance 

[Mike_Gavrilov]

Try to add touch-file

# touch /usr/openv/netbackup/bpbkar_path_tr

add VERBOSE = 5 into bp.conf

and re-run bpbkar test

[Michael_G_Ander]

As long as the native ls does not work I don't think you can bring Netbackup to work on the full directory.

The options are as I see it:

1) Tune the file cache/OS so that ls works on this directory

2) Split up the backup of this directory, probably only makes sense if it contains sub directories

3) Reduce number of files in this directory, guess it contains some kind of attachments for siebel.

Normally I would suggest accelerator for many files, but as long the OS can't list the files I am not sure that will work.

[RiaanBadenhorst]

You can try a raw backup of the file system but since its not able to even do an "ls" this might also not work.

[Nicolai]

Forget about getting NBU to work if "ls" doesn't work. The location /siebel/siebelfs/att  is attachment folder. It looks to me that the sibel admin did not watch this directory and not its to big. 

You can't and should not be responsible to fix this, they are doing it wrong. Instead Sibel admin should find a solution.

Some reference about same issue:

http://siebel.ittoolbox.com/groups/technical-functional/siebel-admin-l/siebel-attachment-folder-is-b...

[ElGringo]

Hi

This issue can occured if you set Accelerator feature "on" and you select take checkpoiny attribute.
For the very first backup, these feature could be mutually exclusive...

If the change rate is high (<15%) I recommend that you use Flashbackup policy type.

Let me know if my advises help you !

[ElGringo]

And yes, a simple "ls" command should work !