Painfully Slow Restore

[Alex_Vasquez]

Hello all,

Here's the evironment that I have:
Master Server: win2k3 server, standard ed. sp1
Client: AIX 5
What is being restored: 230gb DB2 database

The restore is going at about 800 - 900kb per second... Obviously very slow on a GigE network. By contrast the backup, done with the db2 agent, finishes in about an hour and a half.

Here's what I have tried:
Running a cable from the backup server to the client and use that to conduct the restore. Didn't work. Tried changing the switch, router, nic settings to Full duplex. Didn't help performance. I ftp'd a file from the master server to the client. It was about 600mb and it finished in almost 3 minutes avg about 4,000kb per second... At this point I'm going to get choked out because I can't finish the restore in a timely fashion.... So any suggestions or advice is appreciated. Thank you so much.

[zippy]

Alex,

Is the backup Sched set to multiplex?

JD

[Alex_Vasquez]

Yes, actually... The multiplex setting was set to 4. The data is across two, tapes, though... How much impact does this setting have on DB2?

[Lance_Hoskins]

Multiplexing of 4 (assuming 3 other streams with a multiplexing of 4 are running at the same time) is a HUGE deal for restores. It's like trying to copy files from a highly fragmented disk, but instead manually fragmenting the file across a tape.

If you want restores to be fast, definately set multipleting on this beast to 1.

What is your tape technology? LTO1/2/3?

[Alex_Vasquez]

Thanks for the information... I think this may potentially be the issue... But I'd like to confirm it or refute it, if possible. To do this, would restoring the data to an alternate client be useful? I'd think that if the performance is the same, then that would confirm that multiplexing is at fault..

By the way, it is LTO2 technology.

[zippy]

Lance,

Nice pull on the points, your sorta feeding off of me arent you?

;(

JD

[Stumpr2]

Sounds like James was helpful in getting the dialog going. Only additional thoughts I have is for faster restores is to try disk storage or perhaps disk staging which will leave the primary copy on disk until the staging units gets full.Message was edited by:
Bob Stump

[Ankur_Kumar]

Alex
Its more of the multiplexing issue as per the discussion and suggestions by the Maestro's, well i just wanted to say did you try a direct connection between the master and the client, on the 1 Gig Full Duplex NIC on both the sides. please do try using a dedicated NIC from the master side for the restoration.

chiao
Ankur Kumar

[Alex_Vasquez]

Hey there,
So, to answer Ankur's question, I did try what you suggested. Unfortunately, this does not help with the restore time, which ultimately eliminates the network as a potential problem...

I would like to shoot that data to disk and then try to restore from there. However, the database, as it is, is corrupt, thus the need for the restore. I don't believe this database has attempted to restore before...

I'm thinking that maybe adjusting databuffers on the master server may help... However, the only documentation I saw referred to UNIX master media servers. Has anyone tried this?

[Alex_Vasquez]

Oh and thanks for the input... James gets a few points...

[Lance_Hoskins]

James,

Sorry for the points mishap. Didn't know life was all about points on an NBU forum :( If you want a few more, I'll start a post and all you have to do is say hi 4 times and 20 points are yours! ;)

Actually, on a less sinical (sp?) note, I'm sorry if you feel that I took your points. I just happened to be in the forums and was looking around and followed-up on a couple of posts. Unfortunately, I'm not the one that awarded the points (or else I'd have a lot more)! :)

Rock on,
Lance

[Stumpr2]

Lance,
James has been a valuable contributer to this forum. He has unselfishly given many hours of his time helping others. Most posters are thankful but few actually award much deserved points. James isn't looking for a handout..

But I am :)

[zippy]

Ahh bob allways the modest one

jd

[Alex_Vasquez]

Well, I've gotten this issue resolved, I believe... So here's the update on what was done with this issue since I first posted:

- We Checked a few different network settings in terms of NIC cards' duplex on the servers and switches in between. Didn't find anything wrong. We ran a network cable from the master media server to the AIX db2 client and didn't see a change in the rate of speed during the restore. Then I took a 4gb flat file backup of a database dumbp file. This took about 4min for netbackup to backup. I removed all duplex and multi stream settings for this backup. The rate of backup was about 30,000kb. Then, I tried restoring the data and the restore was still very slow, 800kb per sec again on this flat file.

Eventually, Veritas Netbackup article: 240887 was offered by a veritas engineer. I followed the recommendations and have seen a vast improvement in the speed of the restore. The 4gb file, which took over an hour to restore previously, took only 7minutes to restore... Now, I have to do the restore of the db2 database. I'll post the results. I'm expecting the restore to take about 2hrs...