Forum Discussion

RecioLola's avatar
RecioLola
Level 3
10 years ago

Netbackup 7.6.1.1 vmware problems with CBT option

Master/Media :  Symantec Appliance 5220  version 7.6.1.1 upgraded recently

Client:  Vmware client ( ESX version 5.5 ). Vmware tools installed .
 

Problem description:

 

We are added recently several vmware clients and all of them have the same problem.

They have accelerator option enabled.

They have 3 schedules defined:

1.- first full backup with forced rescan option enabled ( with frecuency repetition of 15 weeks )

2.- Incremental backups for every day

3.- Full backup every week

 

First full backup ( with rescan) and following days work properly with optimization near 100% of accelerator ratio.

But suddenly one day the backup shows this message

      "There is no complete backup image match with track journal, a regular full backup will be performed."

       and accelerator ratio is 0%, so the backup is like a full ( so CBT is not working )

 

We opened a support question and they told us a workaround to solve it, but this process is manual:

1) Disable accelerator in the attributes tab
2) Run a full backup without accelerator
3) Check accelerator and run a full backup with accelerator.
4) Run incremental.

 

This workaround works, and next backups are doing well again with an accelerator ratio near 100%.

But we had to do two backups full for each vmware server, with much volume of backup.

But after a few weeks the problem is repeated and we got again the same message:

      "There is no complete backup image match with track journal, a regular full backup will be performed." .



We are planning to add hundreds of vmware clients and if we need to apply this workaround every time is

a work very tedius.

Anybody know what is the cause of the problem and why can we to avoid these errors?

Thanks in advance,

regards.

Lola Recio.

 

Here is a complete trace of the backup:

03/18/2015 00:38:55 - Info nbjm (pid=9084) starting backup job (jobid=373438) for client ow-atp, policy xxxxx.VMware.dylan4.pol, schedule xxxxx.VMware.polincr
03/18/2015 00:38:55 - estimated 67241650 kbytes needed
03/18/2015 00:38:55 - Info nbjm (pid=9084) started backup (backupid=xxxxxx_1426635535) job for client xxxxx, policy xxxxx.VMware.pol, schedule xxxxx.VMware.polincr on storage unit stu_disk_dylan4 using backup host yyyy
03/18/2015 00:38:55 - started process bpbrm (pid=22923)
03/18/2015 00:38:55 - Info bpbrm (pid=22923) xxxxx is the host to backup data from
03/18/2015 00:38:55 - Info bpbrm (pid=22923) reading file list for client
03/18/2015 00:38:56 - Info bpbrm (pid=22923) accelerator enabled
03/18/2015 00:38:56 - Info bpbrm (pid=22923) There is no complete backup image match with track journal, a regular full backup will be performed.
03/18/2015 00:38:56 - Info bpbrm (pid=22923) starting bpbkar on client
03/18/2015 00:38:56 - Info bpbkar (pid=22936) Backup started
03/18/2015 00:38:56 - Info bpbrm (pid=22923) bptm pid: 22938
03/18/2015 00:38:56 - Info bptm (pid=22938) start
03/18/2015 00:38:56 - Info bptm (pid=22938) using 262144 data buffer size
03/18/2015 00:38:56 - Info bptm (pid=22938) using 128 data buffers
03/18/2015 00:38:56 - connecting
03/18/2015 00:38:56 - connected; connect time: 0:00:00
03/18/2015 00:38:57 - Info bptm (pid=22938) start backup
03/18/2015 00:39:00 - begin writing
03/18/2015 00:39:57 - Info bpbkar (pid=22936) INF - Transport Type =  nbd
03/18/2015 01:03:14 - Info bpbkar (pid=22936) accelerator sent 57570284544 bytes out of 57570283008 bytes to server, optimization 0.0%
03/18/2015 01:03:14 - Info bpbkar (pid=22936) bpbkar waited 1 times for empty buffer, delayed 791 times
03/18/2015 01:03:14 - Info bptm (pid=22938) waited for full buffer 65609 times, delayed 92625 times
03/18/2015 01:03:17 - Info bptm (pid=22938) EXITING with status 0 <----------
03/18/2015 01:03:17 - Info dylan4 (pid=22938) StorageServer=PureDisk:yyyy; Report=PDDO Stats (multi-threaded stream used) for (yyyy): scanned: 56226368 KB, CR sent: 16742141 KB, CR sent over FC: 0 KB, dedup: 70.2%, cache disabled
03/18/2015 01:03:17 - Info bpbrm (pid=22923) validating image for client xxxxx
03/18/2015 01:03:18 - Info bpbkar (pid=22936) done. status: 0: the requested operation was successfully completed
03/18/2015 01:03:18 - end writing; write time: 0:24:18
the requested operation was successfully completed  (0)

 

 

 

7.6
Backing Up
Backup and Recovery
NetBackup
Windows Server (2003-2008)
  • Nicolai's avatar
    Nicolai
    10 years ago

    Symante recommend not mixing frequency and calendar based backup in the same policy. I know a lot of user are saying it work fine. But for the sake of problem resolving can you change the schedule XXXXX.VMware.polfull to calender as well.

     

     

7 Replies

Sort By
  • Nicolai's avatar
    Nicolai
    Moderator
    10 years ago

    What are the retention periods ?

    If retention periods are to short for the full backups, there is no reference for the incremental backup. 

    Also be aware of Netbackup will require a new full backup is the Nebackup destination storage unit is changed. This is per design to ensure data integrity.

    Can you please clarify if this is VMware backup using Vsphere or file system backups using a agent deployed in each VM.

  • Marianne's avatar
    Marianne
    Level 6
    10 years ago
    Also verify that Incremental schedule is in same policy as the Full.
  • RecioLola's avatar
    RecioLola
    Level 3
    10 years ago

    Thanks for the answers!,

    Nicolai, Our retention period is of 30 days. We are adding our Vmware clients using vpshere, not file system backups. The vmware clients are added last week so is not possible to need data integrity so soon.

    Marianne, All of the scheduled are in the same policy  ( an unique policy with accelerator option enabled):

           They have 3 schedules defined in it:

             1.- first full backup with forced rescan option enabled ( with frecuency repetition of 15 weeks )

             2.- Incremental backups for every day

             3.- Full backup every week

    All of the Vmware policies we have added has the same problem. After first full backup with

    forced rescan the second incremental is done like a full with the warning in the activity monitor:

      "There is no complete backup image match with track journal, a regular full backup will be performed. " And accelerator ratio 0%.

    And we can fix it with this workaroung (  support has told us), but is a manual process and we are

    planning to add many vmware hosts policies and is a tedious proccess...

    Have you tested this scenario?. Are we doing something wrong?.

     

    Thanks!

    Lola

  • Nicolai's avatar
    Nicolai
    Moderator
    10 years ago

    This sound very wrong - Marianne and I provided the most universal configuration blunders that could lead to this scenario.

    I can't say what wrong right now - need more information.

    Can you please post the output from 

    bppllist {POLICY_NAME} -U

    To be honest I think you should re-open the support case with Symantec and tell them workaround is usable on 100 of servers - be firm about this. If nothing happens - call the duty manger and have him escalate the case.

  • sdo's avatar
    sdo
    Moderator
    10 years ago

    How many different storage unit targets are there defined withinthe backup policy, and/or the schedules, and/or the SLPs?

  • RecioLola's avatar
    RecioLola
    Level 3
    10 years ago

    Hi Nicolai,

    Bellow I send you my policy so you can see if we have some mistake for this behaviour.

    You are right, we can try to scale this issue at symantec they have to give me some solution about that.

    Sdo, my master server is a symantec appliance. Storage unit is a SLP to replicate our backups to another

    appliance in another location and this is working properly.

     

    Policy Name:       XXXXX.VMware.pol

      Policy Type:         VMware
      Active:              yes
      Effective date:      01/01/1970 01:00:00
      File Restore Raw:    yes
      Application Consistent:    yes
      Block Incremental:   no
      Mult. Data Streams:  no
      Client Encrypt:      no
      Checkpoint:          no
      Policy Priority:     0                                                                                                                                                       
      Max Jobs/Policy:     Unlimited                                                                                                                                               
      Disaster Recovery:   0                                                                                                                                                       
      Collect BMR info:    no
      Residence:           SLP
      Volume Pool:         NetBackup
      Server Group:        *ANY*
      Keyword:             XXXXX 2428
      Data Classification:       -
      Residence is Storage Lifecycle Policy:    yes
      Application Discovery:      no
      Discovery Lifetime:      0 seconds
      ASC Application and attributes: (none defined)

      Granular Restore Info:  no
      Ignore Client Direct:  no
      Enable Metadata Indexing:  no
      Index server name:  NULL
      Use Accelerator:  yes
      HW/OS/Client:   Windows-x64     Windows2003                   XXXXX

      Include:  ALL_LOCAL_DRIVES

      Schedule:              XXXXX.VMware.polfull
        Type:                Full Backup
        Frequency:           every 105 days
         Excluded Dates----------
            No specific exclude dates entered
            No exclude days of week entered
        PFI Recovery:        0
        Maximum MPX:         1
        Retention Level:     3 (1 month)
        Number Copies:       1
        Fail on Error:       0
        Residence:           (specific storage unit not required)
        Volume Pool:         (same as policy volume pool)
        Server Group:        (same as specified for policy)
        Residence is Storage Lifecycle Policy:         0
        Schedule indexing:   0
        Daily Windows:
              Sunday     04:00:00  -->  Monday     03:59:59
              Monday     04:00:00  -->  Tuesday    03:59:59
              Tuesday    04:00:00  -->  Wednesday  03:59:59
              Wednesday  04:00:00  -->  Thursday   03:59:59
              Thursday   04:00:00  -->  Friday     03:59:59
              Friday     04:00:00  -->  Saturday   03:59:59
              Saturday   04:00:00  -->  Sunday     03:59:59

      Schedule:              XXXXX.VMware.polzfull
        Type:                Full Backup
        Calendar sched: Enabled
          Allowed to retry after run day
         Included Dates-----------
            Monday, Week 1
            Monday, Week 2
            Monday, Week 3
            Monday, Week 4
            Monday, Week 5
         Excluded Dates----------
          EXCLUDE DATE 0 - 03/27/2015
          EXCLUDE DATE 1 - 03/28/2015
            No exclude days of week entered
        PFI Recovery:        0
        Maximum MPX:         1
        Retention Level:     3 (1 month)
        Number Copies:       1
        Fail on Error:       0
        Residence:           (specific storage unit not required)
        Volume Pool:         (same as policy volume pool)
        Server Group:        (same as specified for policy)
        Residence is Storage Lifecycle Policy:         0
        Schedule indexing:   0
        Daily Windows:
              Sunday     04:00:00  -->  Monday     03:59:59
              Monday     04:00:00  -->  Tuesday    03:59:59
              Tuesday    04:00:00  -->  Wednesday  03:59:59
              Wednesday  04:00:00  -->  Thursday   03:59:59
              Thursday   04:00:00  -->  Friday     03:59:59
              Friday     04:00:00  -->  Saturday   03:59:59
              Saturday   04:00:00  -->  Sunday     03:59:59

      Schedule:              XXXXX.VMware.polincr
        Type:                Differential Incremental Backup
        Calendar sched: Enabled
          Allowed to retry after run day
         Included Dates-----------
            Sunday, Week 1
            Tuesday, Week 1
            Wednesday, Week 1
            Thursday, Week 1
            Friday, Week 1
            Saturday, Week 1
            Sunday, Week 2
            Tuesday, Week 2
            Wednesday, Week 2
            Thursday, Week 2
            Friday, Week 2
            Saturday, Week 2
            Sunday, Week 3
            Tuesday, Week 3
            Wednesday, Week 3
            Thursday, Week 3
            Friday, Week 3
            Saturday, Week 3
            Sunday, Week 4
            Tuesday, Week 4
            Wednesday, Week 4
            Thursday, Week 4
            Friday, Week 4
            Saturday, Week 4
            Sunday, Week 5
            Tuesday, Week 5
            Wednesday, Week 5
            Thursday, Week 5
            Friday, Week 5
            Saturday, Week 5
         Excluded Dates----------
          EXCLUDE DATE 0 - 03/27/2015
          EXCLUDE DATE 1 - 03/28/2015
            No exclude days of week entered
        PFI Recovery:        0
        Maximum MPX:         1
        Retention Level:     3 (1 month)
        Number Copies:       1
        Fail on Error:       0
        Residence:           (specific storage unit not required)
        Volume Pool:         (same as policy volume pool)
        Server Group:        (same as specified for policy)
        Residence is Storage Lifecycle Policy:         0
        Schedule indexing:   0
        Daily Windows:
              Sunday     04:00:00  -->  Monday     03:59:59
              Monday     04:00:00  -->  Tuesday    03:59:59
              Tuesday    04:00:00  -->  Wednesday  03:59:59
              Wednesday  04:00:00  -->  Thursday   03:59:59
              Thursday   04:00:00  -->  Friday     03:59:59
              Friday     04:00:00  -->  Saturday   03:59:59
              Saturday   04:00:00  -->  Sunday     03:59:59

     

  • Nicolai's avatar
    Nicolai
    Moderator
    10 years ago

    Symante recommend not mixing frequency and calendar based backup in the same policy. I know a lot of user are saying it work fine. But for the sake of problem resolving can you change the schedule XXXXX.VMware.polfull to calender as well.