Forum Discussion

AnthonyTsang's avatar
AnthonyTsang
Level 4
11 years ago

Backup failure with error code 636

Media Server : RedHat 2.6 with NBU 7.0

Maste Server: Windows  2008 R2 wiht NBU 7.5.0.5

That policy has two schedule job

1) Full Backup ( Full Backup)

2) cumulative Incremental (Differential Backup)

The Yearly Full Backup can work , but the monthly cumulative incremental can't work.

Please help

Log

3/4/2014 5:35:10 PM - Info nbjm(pid=12396) starting backup job (jobid=143114) for client drs1estm1b.intra.whb.net, policy DRS1ESTM1B_ARCHIVE, schedule Monthly_Diff_LES 
3/4/2014 5:35:10 PM - Info nbjm(pid=12396) requesting STANDARD_RESOURCE resources from RB for backup job (jobid=143114, request id:{E5645178-5B4D-4348-998E-95600F02A8A1}) 
3/4/2014 5:35:10 PM - requesting resource drs1estm1b-hcart3-robot-tld
3/4/2014 5:35:10 PM - requesting resource whb1nbum1v-vaa.NBU_CLIENT.MAXJOBS.drs1estm1b.intra.whb.net
3/4/2014 5:35:10 PM - requesting resource whb1nbum1v-vaa.NBU_POLICY.MAXJOBS.DRS1ESTM1B_ARCHIVE
3/4/2014 5:35:11 PM - granted resource whb1nbum1v-vaa.NBU_CLIENT.MAXJOBS.drs1estm1b.intra.whb.net
3/4/2014 5:35:11 PM - granted resource whb1nbum1v-vaa.NBU_POLICY.MAXJOBS.DRS1ESTM1B_ARCHIVE
3/4/2014 5:35:11 PM - granted resource LESM15
3/4/2014 5:35:11 PM - granted resource Drive088
3/4/2014 5:35:11 PM - granted resource DRS1ESTM1B-hcart3-robot-tld-1
3/4/2014 5:35:11 PM - estimated 0 Kbytes needed
3/4/2014 5:35:11 PM - Info nbjm(pid=12396) started backup (backupid=drs1estm1b.intra.whb.net_1393925711) job for client drs1estm1b.intra.whb.net, policy DRS1ESTM1B_ARCHIVE, schedule Monthly_Diff_LES on storage unit DRS1ESTM1B-hcart3-robot-tld-1
3/4/2014 5:35:12 PM - started process bpbrm (13742)
3/4/2014 5:35:14 PM - connecting
3/4/2014 5:35:15 PM - connected; connect time: 00:00:01
3/4/2014 5:35:19 PM - mounting LESM15
3/4/2014 5:35:28 PM - mounted; mount time: 00:00:09
3/4/2014 5:35:28 PM - positioning LESM15 to file 5
3/4/2014 5:35:28 PM - positioned LESM15; position time: 00:00:00
3/4/2014 5:35:28 PM - begin writing
read from input socket failed(636)
3/4/2014 5:47:45 PM - Error bptm(pid=13765) media manager terminated by parent process      
3/4/2014 5:47:49 PM - Error bpbrm(pid=13742) could not write EXIT STATUS to stderr  

 

7.5
Backup and Recovery
Linux
NetBackup
Troubleshooting
  • mnolan's avatar
    mnolan
    11 years ago

    The last time I saw a 636 error it had nothing to do with bpsynth.

    This was the issue, http://www.symantec.com/docs/TECH214335

    Which was the same as http://www.symantec.com/docs/TECH197144

    The root cause here was that the tcp keep alive timeout on the media server and master server were higher than the firewall.

    NBJM timeouts and writes the message into the detailed status without the timestamp.

    The only reason I can think that this would fail only on a incremental schedule is if that schedule is using a different storage unit/ media server than the full backup is using.  Check to see if this pattern is there and then investigate the timeout setting on both Operating System.

17 Replies

Sort By
Replies have been turned off for this discussion
  • Nicolai's avatar
    Nicolai
    Moderator
    11 years ago

    General Symantec Recommendation regarding status code 636.

    http://www.symantec.com/docs/HOWTO90830

    This sound more like a network issues (firewall ??) than the incremental backup doesn work. How have you come to this conclusion ?

  • RamNagalla's avatar
    RamNagalla
    Moderator
    11 years ago

    so what is the client OS and netbackup version?

    are you making all local Drives backup or any specific file system?

    does it have Mulit stream enabled? if not enable once for test and see which file system is failing..

    if possible please post the bptm log from the media server with Verbose 5

     

  • Marianne's avatar
    Marianne
    Level 6
    11 years ago

    I'm battling to understand your policy config -

    2) cumulative Incremental (Differential Backup)

    Cummulative and Differential are totally different -
    see  Difference between Incremental and Differential back up

    Are there other backup policies for this client?
    Having only a yearly and monthly Incr schedule sounds weird...

    You forgot to tell us anything about the client:
    OS?
    NBU version?
    What is in Backup Selection?

    Please ensure that all of the following log folders exist:

    On media server: bptm and bpbrm
    On client: bpbkar

    After next failure, copy log files to reflect process name (e.g. bpbkar.txt) and upload as File attachments.

  • Mark_Solutions's avatar
    Mark_Solutions
    Level 6
    11 years ago

    I totally agree with Marianne - only do a full backup once a year and an incremental monthly - VERY strange!

    The file list it has to read is likely to be pretty large and what you are actually hitting looks to be a timeout issue - it fails after about 10 minutes

    Check you Media Servers Host Properties - Timeouts section and increase the client read value to 3600 and see if that helps

    But please look into why you would do your backups this way - there is such a gap in your data protection here that it doesn't make the backups very much use.. for example, I create a new file on 4th March and it gets deleted on 2nd April - based on your schedules that file would never have been backed up and so never be retrievable - it really is not best practice

  • AnthonyTsang's avatar
    AnthonyTsang
    Level 4
    11 years ago

    Client = Media server (Same machine) RedHat 2.6 with NBU 7.0

    The Backup Selection point to the folder for store the tar files

    path: /applications/esmt/archive

    The Backup type is "cumulative incremental Backup"

    By restoring two backup images ( Full backup and latest cumulative incremental backup) that can restore the all latest data in that folder.

     

  • AnthonyTsang's avatar
    AnthonyTsang
    Level 4
    11 years ago

     

    I have increased the client read time out to 3600 from 300 seconds and restart all nbu service to take effective. The backup result is the same. Error code 636

     

    The files in backup selection will not be deleted. Since that folder used to store the archive tar files. for example, March backup image contain Feb and Jan files, if the full backup run on last Dec.

  • AnthonyTsang's avatar
    AnthonyTsang
    Level 4
    11 years ago

    Thx your link

     

    I also read that KB article. But i can't find the process bpsynth running on master server. And no firewall between the master and media server=client.

    The same policy is setting in UAT and Prod.

    In UAT,however,  No backup issue.

     

  • Marianne's avatar
    Marianne
    Level 6
    11 years ago

    So, imagine this:

    Client has to 'walk' the filesystem to search for files and folders that were modified since the last full back.
    Have you verified that the Full backup is still valid? When last did it run?

    Have you tried to do the same at OS level?

    Something like: 
    # find /applications/esmt/archive -mtime -<no of days since last full backup>

    Keep an eye on system resources while command is running.

    What is the result?

    To troubleshoot from NBU point of view, create all of these folders on the media server (since it is the client as well):

    bptm
    bpbrm
    bpbkar

    The one that will hopefully be most useful is bpbkar.

    Please copy logs to .txt files (e.g. bpbkar.txt) and upload as File attachment.

     

    PS: any reason why this server is running such an old, unpatched version of NBU?

     

  • AnthonyTsang's avatar
    AnthonyTsang
    Level 4
    11 years ago

    Q:Have you verified that the Full backup is still valid? When last did it run?

    A: Yes, the full backup is still vaild. The retention period is yearly and backup in last Dec 2013.

     

    Q:ve you tried to do the same at OS level?

    A: In uat environment no backup issue with same OS level and policy setting.