cancel
Showing results for 
Search instead for 
Did you mean: 

Exchange 2010 Dag without GRT backup failing

Bharath_Achar
Level 6

 

02/16/2013 22:19:49 - Info nbjm (pid=9462) starting backup job (jobid=2012098) for client DAG1.qqq.COM, policy Exchange2010DAG, schedule Full_Backup
02/16/2013 22:19:49 - Info nbjm (pid=9462) requesting STANDARD_RESOURCE resources from RB for backup job (jobid=2012098, request id:{E2DA761E-78B0-11E2-9DC4-7B1B035D8E5C})
02/16/2013 22:19:49 - requesting resource DISKPOOL5020_DE6
02/16/2013 22:19:49 - requesting resource ppp.qqq.com.NBU_CLIENT.MAXJOBS.DAG1.qqq.COM
02/16/2013 22:19:49 - requesting resource ppp.qqq.com.NBU_POLICY.MAXJOBS.Exchange2010DAG
02/16/2013 22:19:49 - granted resource  ppp.qqq.com.NBU_CLIENT.MAXJOBS.DAG1.qqq.COM
02/16/2013 22:19:49 - granted resource  ppp.qqq.com.NBU_POLICY.MAXJOBS.Exchange2010DAG
02/16/2013 22:19:49 - granted resource  MediaID=@aaaas;DiskVolume=PureDiskVolume;DiskPool=dp_disk_rrr;Path=PureDiskVolume;StorageServer=rrr-bak.qqq.com;MediaServer=slcnpde090.qqq.com
02/16/2013 22:19:49 - granted resource  DISKPOOL5020_DE6
02/16/2013 22:19:49 - estimated 43836728 kbytes needed
02/16/2013 22:19:49 - Info nbjm (pid=9462) started backup (backupid=DAG1.qqq.com_1361071189) job for client DAG1.qqq.COM, policy Exchange2010DAG, schedule Full_Backup on storage unit DISKPOOL5020_DE6
02/16/2013 22:19:49 - started process bpbrm (pid=12672)
02/16/2013 22:19:50 - connecting
02/16/2013 22:19:51 - connected; connect time: 0:00:00
02/16/2013 22:19:56 - Info bpbkar (pid=13736) Backup started
02/16/2013 22:19:56 - Info bpbrm (pid=12672) bptm pid: 13392
02/16/2013 22:19:56 - Error bpbrm (pid=12672) from client DAG1.qqq.com: ERR - Streaming backups are invalid with Exchange 2010.
02/16/2013 22:19:56 - Info bptm (pid=13392) start
02/16/2013 22:19:57 - Info bptm (pid=13392) using 262144 data buffer size
02/16/2013 22:19:57 - Info bptm (pid=13392) using 30 data buffers
02/16/2013 22:20:08 - Info bptm (pid=13392) start backup
02/16/2013 22:20:14 - Info bptm (pid=13392) backup child process is pid 13400
02/16/2013 22:20:14 - Error bptm (pid=13400) system call failed - Connection reset by peer (at child.c.1298)
02/16/2013 22:20:14 - Error bptm (pid=13400) unable to perform read from client socket, connection may have been broken
02/16/2013 22:20:15 - begin writing
02/16/2013 22:20:33 - Info bptm (pid=13392) EXITING with status 42 <----------
02/16/2013 22:20:33 - Info slcnpde090.qqq.com (pid=13392) StorageServer=PureDisk:rrr-bak.qqq.com; Report=PDDO Stats for (rrr-bak.qqq.com): scanned: 2 KB, CR sent: 0 KB, CR sent over FC: 0 KB, dedup: 100.0%
02/16/2013 22:20:33 - Error bpbrm (pid=12672) could not send server status message
02/16/2013 22:20:34 - Info bpbkar (pid=13736) done. status: 42: network read failed
02/16/2013 22:20:35 - end writing; write time: 0:00:20
network read failed  (42)
 
 
ERR - Streaming backups are invalid with Exchange 2010; What does it mean ??????????????
 
Please help me in trouble shooting this issue.
15 REPLIES 15

RamNagalla
Moderator
Moderator
Partner    VIP    Certified

1) does this ever worked fine or new configuration?

2) what is the backup selection for this exchange policy?

3) could you show the screenshot of Policy attributes tab?

Bharath_Achar
Level 6

1. This is not new configuration and was working fine in old configurationa and we have 2 DAG uner a          same policy. One DAG is working fine but other throwing some error.   

2.  Microsoft Exchange Database Availability Groups:\

3. root@ppp ~]# bppllist Exchange2010DAG -U

------------------------------------------------------------
 
Policy Name:       Exchange2010DAG
 
  Policy Type:         MS-Exchange-Server
  Active:              yes
  Effective date:      09/21/2011 18:20:49
  Mult. Data Streams:  yes
  Client Encrypt:      no
  Checkpoint:          no
  Policy Priority:     0
  Max Jobs/Policy:     Unlimited
  Disaster Recovery:   0
  Collect BMR info:    no
  Residence:           DISKPOOL5020_DE6
  Volume Pool:         NetBackup
  Server Group:        *ANY*
  Keyword:             (none specified)
  Data Classification:       -
  Residence is Storage Lifecycle Policy:    no
  Exchange Source active db:               yes
  Exchange 2010 Preferred Server: (none defined)
  Application Discovery:      no
  Discovery Lifetime:      0 seconds
ASC Application and attributes: (none defined)
 
  Granular Restore Info:  no
  Ignore Client Direct:  no
Enable Metadata Indexing:  no
Index server name:  NULL
  Use Accelerator:  no
  HW/OS/Client:  Windows-x64   Windows2008   sss.qqq.com
                 Windows-x64   Windows2008   DAG1.qqq.com
 
  Include:  Microsoft Exchange Database Availability Groups:\
 
  Schedule:              Full_Backup
    Type:                Automatic Backup
    Frequency:           every 7 days
    Maximum MPX:         1
    Synthetic:           0
    Checksum Change Detection: 0
    PFI Recovery:        0
    Retention Level:     15 (8 days)
    Number Copies:       1
    Fail on Error:       0
    Residence:           (specific storage unit not required)
    Volume Pool:         (same as policy volume pool)
    Server Group:        (same as specified for policy)
    Residence is Storage Lifecycle Policy:         0
    Schedule indexing:     0
    Daily Windows:
          Saturday   22:00:00  -->  Sunday     11:00:00
 
  Schedule:              12am_exch_incremental
    Type:                Differential Incremental Backup
    Frequency:           every 1 day
    Maximum MPX:         4
    Synthetic:           0
    Checksum Change Detection: 0
    PFI Recovery:        0
    Retention Level:     15 (8 days)
    Number Copies:       1
    Fail on Error:       0
    Residence:           (specific storage unit not required)
    Volume Pool:         (same as policy volume pool)
    Server Group:        (same as specified for policy)
    Residence is Storage Lifecycle Policy:         0
    Schedule indexing:     0
    Daily Windows:
          Sunday     00:00:00  -->  Sunday     08:00:00
          Monday     00:00:00  -->  Monday     08:00:00
          Tuesday    00:00:00  -->  Tuesday    08:00:00
          Wednesday  00:00:00  -->  Wednesday  08:00:00
          Thursday   00:00:00  -->  Thursday   08:00:00
          Friday     00:00:00  -->  Friday     08:00:00
          Saturday   00:00:00  -->  Saturday   08:00:00
 
  Schedule:              4am_exch_incremental
    Type:                Differential Incremental Backup
    Frequency:           every 1 day
    Maximum MPX:         4
    Synthetic:           0
    Checksum Change Detection: 0
    PFI Recovery:        0
    Retention Level:     15 (8 days)
    Number Copies:       1
    Fail on Error:       0
    Residence:           (specific storage unit not required)
    Volume Pool:         (same as policy volume pool)
    Server Group:        (same as specified for policy)
    Residence is Storage Lifecycle Policy:         0
    Schedule indexing:     0
    Daily Windows:
          Sunday     04:00:00  -->  Sunday     12:00:00
          Monday     04:00:00  -->  Monday     12:00:00
          Tuesday    04:00:00  -->  Tuesday    12:00:00
          Wednesday  04:00:00  -->  Wednesday  12:00:00
          Thursday   04:00:00  -->  Thursday   12:00:00
          Friday     04:00:00  -->  Friday     12:00:00
          Saturday   04:00:00  -->  Saturday   12:00:00
 
  Schedule:              8_PM_exch_incremental
    Type:                Differential Incremental Backup
    Frequency:           every 1 day
    Maximum MPX:         4
    Synthetic:           0
    Checksum Change Detection: 0
    PFI Recovery:        0
    Retention Level:     15 (8 days)
    Number Copies:       1
    Fail on Error:       0
    Residence:           (specific storage unit not required)
    Volume Pool:         (same as policy volume pool)
    Server Group:        (same as specified for policy)
    Residence is Storage Lifecycle Policy:         0
    Schedule indexing:     0
    Daily Windows:
          Sunday     20:00:00  -->  Monday     08:00:00
          Monday     20:00:00  -->  Tuesday    08:00:00
          Tuesday    20:00:00  -->  Wednesday  08:00:00
          Wednesday  20:00:00  -->  Thursday   08:00:00
          Thursday   20:00:00  -->  Friday     08:00:00
          Friday     20:00:00  -->  Saturday   08:00:00
          Saturday   20:00:00  -->  Sunday     08:00:00
 
 
 
Regards,
Bharath

RamNagalla
Moderator
Moderator
Partner    VIP    Certified

hi,

does this  DAG1.qqq.com name of the Cluster or Node name?

could you provide the bpbkar , bpfis from Client

bpbrm and bptm logs from media server.

RLeon
Level 6
   VIP   

Streaming backups are invalid with Exchange 2010

That means "Perform snapshot backups" is not enabled in the policy.

Starting from Exchange 2010, non-snapshot based backups are no longer supported by Microsoft.
Snapshot as in VSS based.

 

http://technet.microsoft.com/en-us/library/dd876874%28v=exchg.141%29.aspx

Microsoft Exchange Server 2007 and Exchange Server 2003 include two different options for data backup and recovery: Extensible Storage Engine (ESE) streaming backup APIs and support for the Volume Shadow Copy Service (VSS) backup APIs.

Exchange 2010 no longer supports the ESE streaming APIs for backup and restore of program files or data. Instead, Exchange 2010 supports only VSS-based backups.

RamNagalla
Moderator
Moderator
Partner    VIP    Certified

if you are using the node name, you can not use the backup selection as 

Microsoft Exchange Database Availability Groups:\

it should be "Microsoft Exchange Database Availability Groups:\<database name>"

 

http://www.symantec.com/business/support/index?page=content&id=HOWTO60734

imitations and conditions:

  • Use a node name (physical server name) that the NetBackup servers can contact.
  • The Backup Selection List (also known as the files list) in the policy must contain "Microsoft Exchange Database Availability Groups:\<database name>".  The Backup Selection can contain more than entry, but each database must be explicitly specified.  The use of "Microsoft Exchange Database Availability Groups:\" or "Microsoft Exchange Database Availability Groups:\*" is not permitted.

Bharath_Achar
Level 6

 

This not a single node, its a Database availablity group (DAG); contains group of nodes.

 

I ran backup second time, getting below error now :

 

on 11 streams below kind of errors:
 
02/17/2013 01:47:54 - Info nbjm (pid=9462) starting backup job (jobid=2012369) for client DAG1.qqq.COM, policy Exchange2010DAG, schedule Full_Backup
02/17/2013 01:47:54 - Info nbjm (pid=9462) requesting STANDARD_RESOURCE resources from RB for backup job (jobid=2012369, request id:{F4EC5C4C-78CD-11E2-85FF-D34C9454FBDE})
02/17/2013 01:47:54 - requesting resource DISKPOOL5020_DE6
02/17/2013 01:47:54 - requesting resource ppp.qqq.com.NBU_CLIENT.MAXJOBS.DAG1.qqq.COM
02/17/2013 01:47:54 - requesting resource ppp.qqq.com.NBU_POLICY.MAXJOBS.Exchange2010DAG
02/17/2013 01:47:55 - granted resource  ppp.qqq.com.NBU_CLIENT.MAXJOBS.DAG1.qqq.COM
02/17/2013 01:47:55 - granted resource  ppp.qqq.com.NBU_POLICY.MAXJOBS.Exchange2010DAG
02/17/2013 01:47:55 - granted resource  MediaID=@aaaas;DiskVolume=PureDiskVolume;DiskPool=dp_disk_rrr;Path=PureDiskVolume;StorageServer=rrr-bak.qqq.com;MediaServer=slcnpde091.qqq.com
02/17/2013 01:47:55 - granted resource  DISKPOOL5020_DE6
02/17/2013 01:48:11 - Info bpbrm (pid=20251) DAG1.qqq.COM is the host to backup data from
02/17/2013 01:48:11 - Info bpbrm (pid=20251) reading file list from client
02/17/2013 01:48:11 - estimated 488690185 kbytes needed
02/17/2013 01:48:11 - Info nbjm (pid=9462) started backup (backupid=DAG1.qqq.com_1361083683) job for client DAG1.qqq.COM, policy Exchange2010DAG, schedule Full_Backup on storage unit DISKPOOL5020_DE6
02/17/2013 01:48:18 - Info bpbrm (pid=20251) starting bpbkar on client
02/17/2013 01:48:18 - started process bpbrm (pid=20251)
02/17/2013 01:48:24 - Info bpbkar (pid=12436) Backup started
02/17/2013 01:48:24 - Info bpbrm (pid=20251) bptm pid: 20955
02/17/2013 01:48:24 - Error bpbrm (pid=20251) from client DAG1.qqq.com: ERR - Streaming backups are invalid with Exchange 2010.
02/17/2013 01:48:24 - Info bptm (pid=20955) start
02/17/2013 01:48:24 - connecting
02/17/2013 01:48:24 - connected; connect time: 0:00:00
02/17/2013 01:48:26 - Info bptm (pid=20955) using 262144 data buffer size
02/17/2013 01:48:26 - Info bptm (pid=20955) using 30 data buffers
02/17/2013 01:48:30 - Error bptm (pid=20955) media manager terminated by parent process
02/17/2013 01:48:32 - Error bpbrm (pid=20251) could not send server status message
02/17/2013 01:48:34 - Info bpbkar (pid=12436) done. status: 69: invalid filelist specification
02/17/2013 01:48:35 - end writing
invalid filelist specification  (69)
 
On 5 streams below kind of errors:
 
02/17/2013 01:47:54 - Info nbjm (pid=9462) starting backup job (jobid=2012366) for client DAG1.qqq.COM, policy Exchange2010DAG, schedule Full_Backup
02/17/2013 01:47:54 - Info nbjm (pid=9462) requesting STANDARD_RESOURCE resources from RB for backup job (jobid=2012366, request id:{F4BEF96E-78CD-11E2-9DCF-DD133D19CCBD})
02/17/2013 01:47:54 - requesting resource DISKPOOL5020_DE6
02/17/2013 01:47:54 - requesting resource ppp.qqq.com.NBU_CLIENT.MAXJOBS.DAG1.qqq.COM
02/17/2013 01:47:54 - requesting resource ppp.qqq.com.NBU_POLICY.MAXJOBS.Exchange2010DAG
02/17/2013 01:47:54 - granted resource  ppp.qqq.com.NBU_CLIENT.MAXJOBS.DAG1.qqq.COM
02/17/2013 01:47:54 - granted resource  ppp.qqq.com.NBU_POLICY.MAXJOBS.Exchange2010DAG
02/17/2013 01:47:54 - granted resource  MediaID=@aaaas;DiskVolume=PureDiskVolume;DiskPool=dp_disk_rrr;Path=PureDiskVolume;StorageServer=rrr-bak.qqq.com;MediaServer=slcnpde092.qqq.com
02/17/2013 01:47:54 - granted resource  DISKPOOL5020_DE6
02/17/2013 01:48:03 - Info bpbrm (pid=6155) DAG1.qqq.COM is the host to backup data from
02/17/2013 01:48:03 - Info bpbrm (pid=6155) reading file list from client
02/17/2013 01:48:03 - estimated 0 kbytes needed
02/17/2013 01:48:09 - Info nbjm (pid=9462) started backup (backupid=DAG1.qqq.com_1361083680) job for client DAG1.qqq.COM, policy Exchange2010DAG, schedule Full_Backup on storage unit DISKPOOL5020_DE6
02/17/2013 01:48:11 - Info bpbrm (pid=6155) starting bpbkar on client
02/17/2013 01:48:11 - started process bpbrm (pid=6155)
02/17/2013 01:48:11 - connecting
02/17/2013 01:48:12 - connected; connect time: 0:00:00
02/17/2013 01:48:17 - Info bpbkar (pid=9936) Backup started
02/17/2013 01:48:17 - Info bpbrm (pid=6155) bptm pid: 6894
02/17/2013 01:48:17 - Error bpbrm (pid=6155) from client DAG1.qqq.com: ERR - Streaming backups are invalid with Exchange 2010.
02/17/2013 01:48:17 - Info bptm (pid=6894) start
02/17/2013 01:48:22 - Info bptm (pid=6894) using 262144 data buffer size
02/17/2013 01:48:22 - Info bptm (pid=6894) using 30 data buffers
02/17/2013 01:48:26 - Info bptm (pid=6894) start backup
02/17/2013 01:48:33 - Info bptm (pid=6894) backup child process is pid 6924
02/17/2013 01:48:33 - Error bptm (pid=6924) system call failed - Connection reset by peer (at child.c.1298)
02/17/2013 01:48:33 - Error bptm (pid=6924) unable to perform read from client socket, connection may have been broken
02/17/2013 01:48:34 - begin writing
02/17/2013 01:48:36 - Info bptm (pid=6894) EXITING with status 42 <----------
02/17/2013 01:48:45 - Info slcnpde092.qqq.com (pid=6894) StorageServer=PureDisk:rrr-bak.qqq.com; Report=PDDO Stats for (rrr-bak.qqq.com): scanned: 2 KB, CR sent: 0 KB, CR sent over FC: 0 KB, dedup: 100.0%
02/17/2013 01:48:45 - Error bpbrm (pid=6155) could not send server status message
02/17/2013 01:48:47 - Info bpbkar (pid=9936) done. status: 42: network read failed
02/17/2013 01:48:48 - end writing; write time: 0:00:14
network read failed  (42)
 

Marianne
Level 6
Partner    VIP    Accredited Certified

As per RLeon's post - please check if Snapshot option is selected in policy attributes.
Unfortunately 'bppllist -U' does not report snapshot selection, only -L option.

Why do you have 2 client names in the policy?
Do you have 2 separate Exchange clusters?
If only one environment, specify only the virtual hostname in the policy.

Bharath_Achar
Level 6

 

[root@ppp ~]# bppllist Exchange2010DAG -L
 
Policy Name:       Exchange2010DAG
Options:           0x0
template:          FALSE
audit_reason:         ?
Names:             (none)
Policy Type:       MS-Exchange-Server (16)
Active:            yes
Effective date:    09/21/2011 18:20:49
Mult. Data Stream: yes
Perform Snapshot Backup:   yes
Snapshot Method:           VSS
Snapshot Method Arguments: prov_type=1,max_snapshots=1,snap_attr=0
Perform Offhost Backup:    no
Backup Copy:               0
Use Data Mover:            no
Data Mover Type:           -1
Use Alternate Client:      no
Alternate Client Name:     (none)
Use Virtual Machine:      0
Hyper-V Server Name:     (none)
Enable Instant Recovery:   no
Policy Priority:   0
Max Jobs/Policy:   Unlimited
Disaster Recovery: 0
Collect BMR Info:  no
Keyword:           (none specified)
Data Classification:       -
Residence is Storage Lifecycle Policy:    no
Client Encrypt:    no
Checkpoint:        no
Residence:         DISKPOOL5020_DE6
Volume Pool:       NetBackup
Server Group:      *ANY*
Granular Restore Info:  no
Exchange Source active db:               yes
Exchange 2010 Preferred Server: (none defined)
Application Discovery:      no
Discovery Lifetime:      0 seconds
ASC Application and attributes: (none defined)
Generation:      94
Ignore Client Direct:  no
Enable Metadata Indexing:  no
Index server name:  NULL
Use Accelerator:  no
Client/HW/OS/Pri/DMI:  sss.qqq.com Windows-x64 Windows2008 0 0 0 0 ?
Client/HW/OS/Pri/DMI:  DAG1.qqq.com Windows-x64 Windows2008 0 0 0 0 ?
Include:           Microsoft Exchange Database Availability Groups:\
 
 
 
 
Yes,
 
We have 2 seperate exchange cluster.
sss.qqq.com
DAG1.qqq.com
 
Backup is working fine for sss.qqq.com. But failing for DAG1.qqq.com .

RLeon
Level 6
   VIP   

Try putting DAG1.qqq.com into its own policy to better isolate the problem.
Best not to use the "copy" feature to copy from the existing policy.
Create the new Exchange policy from scratch.

Marianne
Level 6
Partner    VIP    Accredited Certified

Thanks for confirmation that Snapshot option is enabled.

The only thing that is left now is to look at logs...

As per Nagalla's post above:

 

could you provide the bpbkar , bpfis from Client

bpbrm and bptm logs from media server.

bpresolver log may also be helpful.


As per NBU for Exchange manual:
This log is written to the DAG node. To determine the active node, see the following:
See “Finding the virtual Database Availability Group (DAG) node” on page 218 of NBU for Exchange Admin Guide: http://www.symantec.com/docs/DOC5172

Please copy files to reflect process name (e.g. bpbkar.txt) and post as File attachments.
 

RamNagalla
Moderator
Moderator
Partner    VIP    Certified

 

Info bpbkar (pid=12436) done. status: 69: invalid filelist specification
02/17/2013 01:48:35 - end writing
invalid filelist specification  (69)
did you check the name DAG1.qqq.com is correctly pointing to the active node of the cluster?
there is a possilbity of pointing to passive node becuase of the name resolutsions or cluster config , make sure that..
 
 

Bharath_Achar
Level 6

 

 

I have attached bpbrm and bptm.

 

 


bpbkar , bpfis and bpresolver from which server ?

Can it be 

Any of the DAG member

or it should virtual Dag node  

 

Nagalla,

 

How can i check that ?

RamNagalla
Moderator
Moderator
Partner    VIP    Certified

Those logs should be from the active node of the cluster..

 

Marianne
Level 6
Partner    VIP    Accredited Certified

How can i check that ?

See extract from manual in my previous post:

 

To determine the active node, see the following:
See “Finding the virtual Database Availability Group (DAG) node” on page 218 of NBU for Exchange Admin Guide: http://www.symantec.com/docs/DOC5172

Bharath_Achar
Level 6

 

 

Yes is it pointing to Active node.

 

 

Issue got resolved since had changed acount to LSA account. Hence exchange backups witout GRT was failing.

 

There is an issue with exchange backup of DAG with GRT.

Issue is : once we run a full backup MICROSOFT EXCHANGE WRITER status changes to waitng for completeion sate on one of the dag memeber.

becuase of this successive backups are failig.

 

Is ther any way other than reboot ot restarting information store / replicetion service on affected server ?????????