cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Veritas.com
Support

NetBackup 6.5.2A - Exchange 2007 Weekly Mailbox job fails error 69

dluss16
Level 2

‎02-18-2009 08:41 AM

My Exchange 2007 policy only fails on the weekly backup. The daily and the monthly run successfully.

I either have the job fail because the server fails over to the 2nd cluster node or i get the following error

for each stream:

2/18/2009 9:52:06 AM - granted resource 0319L3
2/18/2009 9:52:06 AM - granted resource IBM.ULTRIUM-TD3.000
2/18/2009 9:52:06 AM - granted resource nvl-media1-hcart3-robot-tld-1
2/18/2009 9:52:11 AM - estimated 0 kbytes needed
2/18/2009 9:53:02 AM - connecting
2/18/2009 9:53:01 AM - Error bpbrm(pid=3656) from client nvl-ms9: ERR - object not found for file system backup: Microsoft Exchange Mailboxes:\[q-t]*
2/18/2009 9:53:02 AM - Error bpbrm(pid=3656) could not send server status message      
2/18/2009 9:53:03 AM - connected; connect time: 00:00:01
2/18/2009 9:53:03 AM - begin writing
2/18/2009 9:53:21 AM - end writing; write time: 00:00:18
invalid filelist specification(69)

 

The policy has the mailboxes broken into the following streams:

NEW STREAM

Microsoft Exchange Mailboxes:\[a-e]*

NEW STREAM

Microsoft Exchange Mailboxes:\[f-l]*

NEW STREAM

Microsoft Exchange Mailboxes:\[m-p]*

NEW STREAM

Microsoft Exchange Mailboxes:\[q-t]*

NEW STREAM

Microsoft Exchange Mailboxes:\[u-z]*

NEW STREAM

Microsoft Exchange Mailboxes:\[_]*

 

I have media multiplexing set at 10. I'm not sure if something is wrong with how I have the streams set up. If the streams are not set up properly they could be putting a heavy load on the cluster to make it fail over. Any Ideas?

0 Kudos
7 REPLIES 7

schmaustech
Level 5

‎02-18-2009 12:26 PM

My hunch is you do not have the rights to see the mailboxes on the Exchange server.

 

Did you follow page 2 of this document on creating a service account to run the Netbackup client service on the Exchange server:

 

http://seer.entsupport.symantec.com/docs/292587.htm

 

If so, can you log in with the service account user and run Netbackup Archive Restore and see if you can enumerate the mailboxes there?

 

Have you also installed the Exchange MAPI client from Microsoft?  Mapi is no longer part of Exchange 2007 and is required for Mailbox backups and restores.

 

Regards,

 

Benjamin Schmaus

0 Kudos

dluss16
Level 2

‎02-18-2009 01:33 PM

Thanks for your response!

 

Yes we have a specific backup account that is a domain admin, and an Exchange Organization Admin. Its used to start the netbackup client service on both of the cluster nodes. I opened the backup/archive/restore client with "run as" with that account and made sure I could click on the mail store and see all the mailboxes. I did this for each cluster node.

 

The IMAPI client is installed also. Exchange IMAPI client 1.2.1 v6.5.8022.0

 

I looked up the bprm(pid=3656) in the bprm log on the media server and this is what I found (sorry for the long insert):

 

 

09:53:00.991 [3656.4972] <2> MNG: backup_cmd = /usr/openv/netbackup/bin/bpbkar bpbkar32 -r 3024000 -ru root -dt 0 -to 0 -clnt nvl-ms9 -class Exchange_2007 -sched Weekly_Mailboxes_2007 -st FULL -bpstart_to 500 -bpend_to 500 -read_to 18000 -blks_per_buffer 127 -stream_count 5 -stream_number 4 -jobgrpid 232258 -use_otm -use_ofb -b nvl-ms9_1234968731 -kl 7 -fso -ct 16
09:53:00.991 [3656.4972] <2> bpbrm handle_backup: forking client backup
09:53:00.991 [3656.4972] <2> bpbrm send_bpsched_connected_msg: sending bpsched msg: CONNECTED TO CLIENT FOR nvl-ms9_1234968731
09:53:01.101 [3656.4972] <2> get_readline_type: Returning BPBRM_READ_INFORM_WHEN_DONE
09:53:01.319 [3656.4972] <2> bpbrm read_backup_start: from client nvl-ms9: read client start message
09:53:01.319 [3656.4972] <2> bpbrm write_continue_backup: wrote CONTINUE BACKUP on COMM_SOCK <1648>
09:53:01.319 [3656.4972] <2> write_file_names: buffering file name 'Microsoft Exchange Mailboxes:\[q-t]*' for output
09:53:01.319 [3656.4972] <2> write_file_names: successfully wrote buffer to COMM_SOCK
09:53:01.319 [3656.4972] <2> bpbrm write_filelist: wrote CONTINUE on COMM_SOCK
09:53:01.507 [3656.4972] <2> bpbrm handle_backup: ESTIMATE -1 -1 nvl-ms9_1234968731
09:53:01.507 [3656.4972] <16> bpbrm handle_backup: from client nvl-ms9: ERR - object not found for file system backup: Microsoft Exchange Mailboxes:\[q-t]*
09:53:01.507 [3656.4972] <2> vnet_cached_gethostbyname: vnet_hosts.c.307: found host in cache: nvl-bk2
09:53:01.507 [3656.4972] <2> vnet_cached_gethostbyname: vnet_hosts.c.307: found host in cache: nvl-bk2
09:53:01.507 [3656.4972] <2> vnet_async_connect: vnet_vnetd.c.4019: connect in progress: 0 0x00000000
09:53:01.522 [3656.4972] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2043: VN_REQUEST_SERVICE_SOCKET: 6 0x00000006
09:53:01.522 [3656.4972] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2057: service: bpjobd
09:53:01.694 [3656.4972] <2> vnet_async_connect: vnet_vnetd.c.4205: in progress connect: 0 0x00000000
09:53:01.694 [3656.4972] <2> vnet_async_connect: vnet_vnetd.c.4208: connect: async CONNECT FROM 10.0.5.92.3581 TO 10.0.5.83.13724 fd = 1624
09:53:01.694 [3656.4972] <2> logconnections: BPJOBD CONNECT FROM 10.0.5.92.3581 TO 10.0.5.83.13724
09:53:01.694 [3656.4972] <2> vauth_authentication_required: vauth_comm.c.749: no methods for address: no authentication required
09:53:01.694 [3656.4972] <2> vauth_connector: vauth_comm.c.182: no methods for address: no authentication required
09:53:01.694 [3656.4972] <2> job_authenticate_connection: no authentication required
09:53:01.694 [3656.4972] <2> job_authenticate_connection: ignoring VxSS authentication check for now...
09:53:01.694 [3656.4972] <2> job_connect: Connected to the host nvl-bk2 contype 10 jobid <232262> socket <1624>
09:53:01.694 [3656.4972] <2> job_connect: Connected on port 3581
09:53:01.694 [3656.4972] <2> set_job_details: Sending Tfile jobid (232262)
09:53:01.694 [3656.4972] <2> set_job_details: LOG 1234968781 16 bpbrm 3656 from client nvl-ms9: ERR - object not found for file system backup: Microsoft Exchange Mailboxes:\[q-t]*

09:53:01.694 [3656.4972] <2> set_job_details: Done
09:53:01.913 [3656.4972] <2> job_monitoring_exex: ACK disconnect
09:53:01.913 [3656.4972] <2> job_disconnect: Disconnected
09:53:01.913 [3656.4972] <2> vnet_cached_gethostbyname: vnet_hosts.c.307: found host in cache: nvl-bk2
09:53:01.913 [3656.4972] <2> vnet_cached_gethostbyname: vnet_hosts.c.307: found host in cache: nvl-bk2
09:53:01.913 [3656.4972] <2> vnet_async_connect: vnet_vnetd.c.4019: connect in progress: 0 0x00000000
09:53:01.929 [3656.4972] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2043: VN_REQUEST_SERVICE_SOCKET: 6 0x00000006
09:53:01.929 [3656.4972] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2057: service: bpdbm
09:53:02.132 [3656.4972] <2> vnet_async_connect: vnet_vnetd.c.4205: in progress connect: 0 0x00000000
09:53:02.132 [3656.4972] <2> vnet_async_connect: vnet_vnetd.c.4208: connect: async CONNECT FROM 10.0.5.92.3582 TO 10.0.5.83.13724 fd = 1624
09:53:02.132 [3656.4972] <2> logconnections: BPDBM CONNECT FROM 10.0.5.92.3582 TO 10.0.5.83.13724
09:53:02.132 [3656.4972] <2> vauth_authentication_required: vauth_comm.c.749: no methods for address: no authentication required
09:53:02.132 [3656.4972] <2> vauth_connector: vauth_comm.c.182: no methods for address: no authentication required
09:53:02.132 [3656.4972] <2> check_authentication: no authentication required
09:53:02.132 [3656.4972] <2> vnet_check_vxss_client_magic_with_info: vnet_vxss_helper.c.770: Ignoring VxSS authentication: 2 0x00000002
09:53:02.132 [3656.4972] <2> vnet_check_vxss_client_magic_with_info: vnet_vxss_helper.c.926: Not using VxSS authentication: 2 0x00000002
09:53:02.851 [3656.4972] <2> bpbrm handle_backup: client nvl-ms9 EXIT STATUS = 69: invalid filelist specification
09:53:02.851 [3656.4972] <2> inform_client_of_status: INF - Server status = 69
09:53:02.851 [3656.4972] <2> put_long: (11) network write() error: An established connection was aborted by the software in your host machine. ; socket = 1648
09:53:02.851 [3656.4972] <16> inform_client_of_status: could not send server status message
09:53:02.851 [3656.4972] <2> vnet_cached_gethostbyname: vnet_hosts.c.307: found host in cache: nvl-bk2
09:53:02.851 [3656.4972] <2> vnet_cached_gethostbyname: vnet_hosts.c.307: found host in cache: nvl-bk2
09:53:02.851 [3656.4972] <2> vnet_async_connect: vnet_vnetd.c.4019: connect in progress: 0 0x00000000
09:53:02.866 [3656.4972] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2043: VN_REQUEST_SERVICE_SOCKET: 6 0x00000006
09:53:02.866 [3656.4972] <2> vnet_vnetd_service_socket: vnet_vnetd.c.2057: service: bpjobd
09:53:02.897 [3656.4972] <2> vnet_async_connect: vnet_vnetd.c.4205: in progress connect: 0 0x00000000
09:53:02.897 [3656.4972] <2> vnet_async_connect: vnet_vnetd.c.4208: connect: async CONNECT FROM 10.0.5.92.3584 TO 10.0.5.83.13724 fd = 1620
09:53:02.897 [3656.4972] <2> logconnections: BPJOBD CONNECT FROM 10.0.5.92.3584 TO 10.0.5.83.13724
09:53:02.897 [3656.4972] <2> vauth_authentication_required: vauth_comm.c.749: no methods for address: no authentication required
09:53:02.897 [3656.4972] <2> vauth_connector: vauth_comm.c.182: no methods for address: no authentication required
09:53:02.897 [3656.4972] <2> job_authenticate_connection: no authentication required
09:53:02.897 [3656.4972] <2> job_authenticate_connection: ignoring VxSS authentication check for now...
09:53:02.897 [3656.4972] <2> job_connect: Connected to the host nvl-bk2 contype 10 jobid <232262> socket <1620>
09:53:02.897 [3656.4972] <2> job_connect: Connected on port 3584
09:53:02.897 [3656.4972] <2> set_job_details: Sending Tfile jobid (232262)
09:53:02.897 [3656.4972] <2> set_job_details: LOG 1234968782 16 bpbrm 3656 could not send server status message

 

0 Kudos

schmaustech
Level 5

‎02-20-2009 11:31 AM

What is the difference between your daily/monthly schedules and your weekly?  Are they incrementals, cumulatives or fulls?  Mailbox vs information store?

 

You mentioned this is a cluster, CCR cluster or a shared storage backend cluster?  Doing circular logging?  If CCR are you using snapshots and backing up passive node?

 

I noticed in the bpbrm log it only has errors for the mailboxes q-t.  Do you see the same errors for the other streams?

 

Depending on your answers above, do the following:

 

Create the following log on the both nodes of the cluster C:\Program Files\VERITAS\Netbackup\logs\bpbkar

 

-Optional:  Increase the logging level on the client by opening up Netbackup Archive Restore: File-> Netbackup Client Properties ->Troubleshooting.  Set General to 2 and Verbose to 5.  Restart Netbackup Client Service

 

Run your job and then paste the log that gets created.

 

Regards,

 

Benjamin Schmaus

0 Kudos

dluss16
Level 2

‎02-20-2009 12:30 PM

The cluster is a shared storage backend cluster. I'm not sure about the logging.

 

I ran this command to get the policy info for the Mailbox policy and the Info Store policy:

bppllist <exchange_policy_name> -U > c:\exchangepolicy.txt

 

------------------------------------------------------------

Policy Name:       Exchange_Information_Stores

  Policy Type:         MS-Exchange-Server
  Active:              yes
  Effective date:      02/02/2009 11:10:52
  Mult. Data Streams:  no
  Client Encrypt:      no
  Checkpoint:          no
  Policy Priority:     0
  Max Jobs/Policy:     Unlimited
  Disaster Recovery:   0
  Collect BMR info:    no
  Residence:           nvl-media1-hcart3-robot-tld-1
  Volume Pool:         NetBackup
  Server Group:        *ANY*
  Keyword:             (none specified)
  Data Classification:       Gold
  Residence is Storage Lifecycle Policy:    no

Granular Restore Info:  no
  HW/OS/Client:  PC-X64        Windows2003   nvl-ms9

  Include:  NEW_STREAM
            Microsoft Information Store:\

  Schedule:          Monthly_Info_Store
    Type:            Automatic Backup
    Maximum MPX:     1
    Synthetic:       0
    PFI Recovery:    0
    Retention Level: 8 (1 year)
    Number Copies:   1
    Fail on Error:   0
    Residence:       (specific storage unit not required)
    Volume Pool:     Vault_1_Year
    Server Group:    (same as specified for policy)
    Calendar sched: Enabled
      Monday, Week 2
    Residence is Storage Lifecycle Policy:     0
    Daily Windows:
          Monday     14:00:00  -->  Tuesday    03:00:00

  Schedule:          Weekly_Info_Store
    Type:            Automatic Backup
    Maximum MPX:     1
    Synthetic:       0
    PFI Recovery:    0
    Retention Level: 4 (5 weeks)
    Number Copies:   1
    Fail on Error:   0
    Residence:       (specific storage unit not required)
    Volume Pool:     Weekly_Backup
    Server Group:    (same as specified for policy)
    Calendar sched: Enabled
      Monday, Week 1
      Monday, Week 3
      Monday, Week 4
      Monday, Week 5
    Residence is Storage Lifecycle Policy:     0
    Daily Windows:
          Monday     14:00:00  -->  Tuesday    03:00:00

  Schedule:          Daily_Info_Stores
    Type:            Differential Incremental Backup
    Maximum MPX:     1
    Synthetic:       0
    PFI Recovery:    0
    Retention Level: 3 (8 days)
    Number Copies:   1
    Fail on Error:   0
    Residence:       (specific storage unit not required)
    Volume Pool:     Daily
    Server Group:    (same as specified for policy)
    Calendar sched: Enabled
    Residence is Storage Lifecycle Policy:     0
    Daily Windows:
          Sunday     00:00:00  -->  Sunday     06:00:00
          Tuesday    14:00:00  -->  Wednesday  06:00:00
          Wednesday  14:00:00  -->  Thursday   06:00:00
          Thursday   14:00:00  -->  Friday     06:00:00
          Friday     14:00:00  -->  Saturday   06:00:00
          Saturday   14:00:00  -->  Saturday   24:00:00

 

 

------------------------------------------------------------

Policy Name:       Exchange_2007

  Policy Type:         MS-Exchange-Server
  Active:              yes
  Effective date:      05/21/2003 09:34:48
  Mult. Data Streams:  yes
  Client Encrypt:      no
  Checkpoint:          no
  Policy Priority:     1
  Max Jobs/Policy:     Unlimited
  Disaster Recovery:   0
  Collect BMR info:    no
  Residence:           nvl-media1-hcart3-robot-tld-1
  Volume Pool:         Daily
  Server Group:        *ANY*
  Keyword:             (none specified)
  Data Classification:       Gold
  Residence is Storage Lifecycle Policy:    no

Granular Restore Info:  no
  HW/OS/Client:  PC-X64        Windows2003   nvl-ms9

  Include:  NEW_STREAM
            Microsoft Exchange Mailboxes:\[_-e]*
            NEW_STREAM
            Microsoft Exchange Mailboxes:\[f-l]*
            NEW_STREAM
            Microsoft Exchange Mailboxes:\[m-p]*
            NEW_STREAM
            Microsoft Exchange Mailboxes:\[q-t]*
            NEW_STREAM
            Microsoft Exchange Mailboxes:\[u-z]*

  Schedule:          Monthly_Mailboxes_2007
    Type:            Automatic Backup
    Maximum MPX:     10
    Synthetic:       0
    PFI Recovery:    0
    Retention Level: 8 (1 year)
    Number Copies:   1
    Fail on Error:   0
    Residence:       nvl-media1-hcart3-robot-tld-1
    Volume Pool:     Vault_1_Year
    Server Group:    (same as specified for policy)
    Calendar sched: Enabled
      Tuesday, Week 1
    Residence is Storage Lifecycle Policy:     0
    Daily Windows:
          Tuesday    18:00:00  -->  Wednesday  07:00:00

  Schedule:          User
    Type:            Application Backup
    Maximum MPX:     1
    Synthetic:       0
    PFI Recovery:    0
    Retention Level: 8 (1 year)
    Number Copies:   1
    Fail on Error:   0
    Residence:       (specific storage unit not required)
    Volume Pool:     (same as policy volume pool)
    Server Group:    (same as specified for policy)
    Residence is Storage Lifecycle Policy:     0
    Daily Windows:
          Sunday     00:00:00  -->  Saturday   23:50:00

  Schedule:          Weekly_Mailboxes_2007
    Type:            Automatic Backup
    Maximum MPX:     10
    Synthetic:       0
    PFI Recovery:    0
    Retention Level: 4 (5 weeks)
    Number Copies:   1
    Fail on Error:   0
    Residence:       nvl-media1-hcart3-robot-tld-1
    Volume Pool:     Weekly_Backup
    Server Group:    (same as specified for policy)
    Calendar sched: Enabled
      Allowed to retry after run day
      Tuesday, Week 2
      Tuesday, Week 3
      Tuesday, Week 4
      Tuesday, Week 5
    Residence is Storage Lifecycle Policy:     0
    Daily Windows:
          Tuesday    18:00:00  -->  Wednesday  07:10:00

  Schedule:          Daily_Mailboxes_2007
    Type:            Differential Incremental Backup
    Maximum MPX:     10
    Synthetic:       0
    PFI Recovery:    0
    Retention Level: 3 (8 days)
    Number Copies:   1
    Fail on Error:   0
    Residence:       (specific storage unit not required)
    Volume Pool:     (same as policy volume pool)
    Server Group:    (same as specified for policy)
    Calendar sched: Enabled
      Monday, Week 1
      Wednesday, Week 1
      Thursday, Week 1
      Friday, Week 1
      Saturday, Week 1
      Monday, Week 2
      Wednesday, Week 2
      Thursday, Week 2
      Friday, Week 2
      Saturday, Week 2
      Monday, Week 3
      Wednesday, Week 3
      Thursday, Week 3
      Friday, Week 3
      Saturday, Week 3
      Monday, Week 4
      Wednesday, Week 4
      Thursday, Week 4
      Friday, Week 4
      Saturday, Week 4
      Monday, Week 5
      Wednesday, Week 5
      Thursday, Week 5
      Friday, Week 5
      Saturday, Week 5
    Residence is Storage Lifecycle Policy:     0
    Daily Windows:
          Monday     18:00:00  -->  Tuesday    06:00:00
          Wednesday  18:00:00  -->  Thursday   06:00:00
          Thursday   18:00:00  -->  Friday     06:00:00
          Friday     18:00:00  -->  Saturday   06:00:00
          Saturday   18:00:00  -->  Sunday     06:00:00

 

----------------------------------

 

I am running another job right now with the bpkar log turned on on the client. I will post it below.

0 Kudos

schmaustech
Level 5

‎02-25-2009 10:57 AM

So your information store backups work, but your mailbox backups do not?

 

For mailbox backups you need to have the Netbackup client service run as n account that has rights to peer into the mailboxes.  You also need the MAPI client installed.  I posted a technote on this board in one of the Exchange 2007 threads.

0 Kudos

dluss16
Level 2

‎02-25-2009 11:05 AM

(from an above post...)

 

Yes we have a specific backup account that is a domain admin, and an Exchange Organization Admin. Its used to start the netbackup client service on both of the cluster nodes. I opened the backup/archive/restore client with "run as" with that account and made sure I could click on the mail store and see all the mailboxes. I did this for each cluster node.

0 Kudos

schmaustech
Level 5

‎02-25-2009 06:16 PM

How about posting that bpbkar log?
0 Kudos