cancel
Showing results for 
Search instead for 
Did you mean: 

Restoration failed with error code 5--Urgent

susindran_surul
Level 6

Hi Folks,

I am doing windows to window alternate restoration but its getting failed with error code 5.Please find the below for error code and attachement for mt media server logs.could you please alayse and let us know the solution..

 

 

08/24/2012 15:08:04 - Error bpbrm (pid=26847) socket read failed: errno = 131 - Connection reset by peer
08/24/2012 15:08:04 - Error bptm (pid=26846) cannot write data to socket, Broken pipe
08/24/2012 15:11:46 - Warning bprd (pid=17239) Restore must be resumed prior to first image expiration on Sat 17 Aug 2013 06:00:03 PM GMT
08/24/2012 15:11:46 - end Restore; elapsed time 0:07:27
termination requested by administrator (150)
 
I have attached below logs.
 

From the client, collect and send me the bpbkar and bpcd logs again.

 From the media server, collect and send me the bpbrm, bptm, and bpcd logs.

 

My master and media server is solaris 10 with 6.5.5 version.

 

Thanks,

Susi.S

 

1 ACCEPTED SOLUTION

Accepted Solutions

susindran_surul
Level 6

Hi Marianne,

We found issue with NIC card driver version very old and speed also mis matching.We are planing to upgrade driver versiona and i will keep on posted once issue is fixed.

 

I want to know how you analysing logs? using any tool?

 

Thanks,

Susi.S

View solution in original post

20 REPLIES 20

susindran_surul
Level 6

Hi All,

 

Any one help on this???????????????

 

Thanks,

Susi.S

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Bpbkar log is for backup. You need tar log on client for restore. Please also give us the restore progress log on the master server.

PS: Urgent matters should be logged with Symantec Support.
We all live in different parts of the world in different timezones..... America, Europe, Africa are all asleep right now... plus it's the weekend...

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

The PIDs and the times in media server log are different to the ones that you posted above in Job Details, but I can see that restore was running, that bptm was reading files and was sending it to the client until network connection was broken on client side.

You need to concentrate troubleshooting efforts on the client - check for anti-virus software that could terminate restore process. 
Check tar log to see if any data was actually received and written by the client.
Check Event Viewer logs for errors.
Something outside of NBU broke network connection.

susindran_surul
Level 6

Thanks Marianne..What and all i have to check on client side..i am basically for unix side and i want to check windows team to look on it..Could you please explain what and all logs i need to check with them??

I have logged call with symantec 2 days before  but its going like a journy............:(

 

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Please read through my previous post. 
I have listed all I can think of.

susindran_surul
Level 6

Thanks marianne..Let me check with windows team for below pont ..

 

You need to concentrate troubleshooting efforts on the client - check for anti-virus software that could terminate restore process. 
Check tar log to see if any data was actually received and written by the client.
Check Event Viewer logs for errors.

 

 

V4
Level 6
Partner Accredited

Most of times i got this kind of error it due to insufficient privileges on target side.

Check if that helps

Mike_Gavrilov
Moderator
Moderator
Partner    VIP    Accredited Certified

Have you collected tar log from the client side? It's really necessary.

susindran_surul
Level 6

please find the tar and bpcd logs..

susindran_surul
Level 6

i have attached below my tar logs..

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

No problem in NBU logs on client - successful connection in bpcd; tar log show files being received/restored between 09:37 and 09:53. No errors.

This confirms that "something" outside of NBU has terminated the restore/connection. 

Have you checked Event Viewer logs on client yet? I would check all of them for any hints as to what happened around the time of the last entry in tar log (9:53:58).

Check anti-virus logs as well.

Mike_Gavrilov
Moderator
Moderator
Partner    VIP    Accredited Certified

How many NIC does your clinet have? Run bplocaladdrs -verbose on your client.

 

susindran_surul
Level 6

Hi Marianne,

We found issue with NIC card driver version very old and speed also mis matching.We are planing to upgrade driver versiona and i will keep on posted once issue is fixed.

 

I want to know how you analysing logs? using any tool?

 

Thanks,

Susi.S

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

I do not have any tool.

This post explains how I am reading logs: https://www-secure.symantec.com/connect/forums/how-use-logs#comment-6505511

Not sure if Symantec Support staff have tools to read/analyse logs (especially level 5 logs), but I run a mile when someone posts level 5 log....

muhanad_daher
Level 6
Partner Accredited Certified

Dear Susindran,

 

Is this client has backup policy in your nbu system?

Is this server inserted to hosts file?

susindran_surul
Level 6

Hi Muhanas,

 

We find issue.Its will NIC card.

susindran_surul
Level 6

Good information marianne...

mph999
Level 6
Employee Accredited

I love level 5 logs ....  ;o)

"Not sure if Symantec Support staff have tools to read/analyse logs"

Yes, it is called a BackLine Engineer .... he he he ...

More seriously, yes and no.

The best tools for looking at logs are textpad and unix awk, oh, and I hate to say it but experience.

There are no magic tools, support have an internal only tool that helps when looking through the IRM logs (pem, jm, rb, emm) , but this is really the only script that I know of.

Legacy logs (found in /usr/openv/volmgr/debug and /usr/openv/netbackup/logs ) are fairly easy to read as they are single threaded.  Quite conveniently, if you have an error you quite often have a <16> on the line.

Unfortunately easy to read does not mean 'easy to find the problem', just that  it is easy to pull out the lines that refer to 'whatever' part of the job, as the lines will all have the same PID.

For support, most of us just get them at verbose 5 ( + general 2 if windows) although sometimes overkill, this ensures we definately have the details we need and avoids having to get the logs again, if we make a mistake and request them too low.

The lines contain a process ID, and as single threaded, if you pull out the lines with the same process id, you often have the details you need.  There may be a child process, with a different PID, but quite often the parent process gives the PID of the child.

The best two tools for legacy logs are the unix 'awk' command and /or the program 'textpad' which can be purchasd for about $25.

How good is textpad, well, Symantec have a license to use it.  It was once described to me as almost being written to read Netbackup logs.

If you are good with Unix awk, this can be good for pulling bit's out of logs, and other outputs from NBU commands.

You could also do simple things like a small unix script to automatically pull out all the lines for the different PIDs in a legacy log and place in a sepaare files - this could save vast amounts of time, and just makes things less stressful.

VX/ Unified logs are a nightmare because they are multi-threaded, as it happened I wrote some details about these the other day in another post which I have copied here.

 

vx logs are very difficult to read, because they are multithreaded.  From example, if we look in say the bptm log (not a vx log and single threaded) you can read it like a book.
 
[1234] xxxx
[1234] xxxx
[1234] xxxx
[1244] yyyy
[1244] yyyy
[1244] yyyy
 
Where xxxx and yyyy are lines relating to a particular 'activity'
 
So if I search all the lines containing a given 'PID' (shown in [xxx] ) then those lines relate to a single activity, eg. mounting a tape.
 
With VX logs the PID (process ID ) is shown, and TID (Thread ID)
 
nbemm 111 PID:20454 TID:6 
 
The problem, is that if xxx and yyy are lines relating to a particular activity ...
 
nbemm 111 PID:20454 TID:6  xxxx
nbemm 111 PID:20454 TID:6  yyyy
nbemm 111 PID:20454 TID:7  xxxx
 
One second, TID 6 could be dealing with the 'activity', the next moment, TID 6 is dealing with some other activity (yyyy) and TID 7 has taken over 'xxxx'.
 
That means if I search for all the TID 6 lines, I get multiple lines returned that could have nothing to do with each other,
 
Unfortunately, NBU is too complex for any one person to know exactly what the logs lines will be for every possibly thing NBU can do ... therefore the only way to read these logs:
 
1/ You need to know what NBU will do next and to be able to find the next line in the sequence.
 
or
 
2/  Run the job on a working machine and compare the logs line by line to spot the differencies.
 
 
Generally, I use both 1/ and 2/ when reading these logs.  For something 'simple' like a backup, I know the main steps thet appear in the logs, so I canfind one line and then look for the next.
 
For example, NBPEM will start a backup ,and then submit the job to nbjm
 
So, I find the line in nbpem that shows the job starting, and I then know that the next line I have to look for is where nbpem gos to contact nbjm, which may, or may not be the same TID).
 
But...
 
If I have an issue I have never seen before, I have no idea what the logs should be showing, I then usually run the job on a working system to get the logs, and then compare these to the non-working logs.
 
With legacy logs (bptm etc ...) we don't have this issue, you can just spearate the logs out by searching for the individual PIDs.

Regards,

 

martin

 

 

susindran_surul
Level 6

Very very helpful martin..