Sharepoint 2007 document level backup and restore trouble with NFS

[mfh1957]

We have a dedicated Netbackup 6.5.5 Master server and a dedicated Media server both running Windows server 2003 R2 in our management domain.  We have a firewall configured between this management domain and the corporate domain and niether has any AD relationship with the other.

The Sharepoint infrastructure is a dedicated SP 2007 Web server and SQL 2008 sp1 database, WSS 3.0 sp1 on Windows server 2008 r2.

Normal file leve backups work just fine from the SP server and the Netbackup infrastructure.  We have opened the 7394 port between the Media server and SQL server, loaded NFS on both, started the portmap and configured theuser credentials on both the SP and Netbackup servers per the user guides and tech notes.  We are getting what looks like a backup of the portal database but we get no document level backups.  The backup is configured correctly for the path and granular backups.  We are stumped.

Within the Detailed Status window of the Activity monitor we get the following:
 
5/19/2010 3:19:08 PM - estimated 161974197 kbytes needed
5/19/2010 3:19:09 PM - started process bpbrm (4548)
5/19/2010 3:19:17 PM - mounting 400467
5/19/2010 3:19:18 PM - connecting
5/19/2010 3:19:19 PM - connected; connect time: 00:00:01
5/19/2010 3:20:03 PM - mounted; mount time: 00:00:46
5/19/2010 3:20:03 PM - positioning 400467 to file 1
5/19/2010 3:20:07 PM - positioned 400467; position time: 00:00:04
5/19/2010 3:20:07 PM - begin writing
5/19/2010 3:20:34 PM - Error bpbrm(pid=3992) from client mnedna-srvsps: ERR - Unable to initialize nbfsd.   
5/19/2010 3:20:35 PM - Error bpbrm(pid=3992) from client mnedna-srvsps: ERR - Granular items will not be cataloged. 

In the SQL servers application event logs we get event ID 16397:
 
Windows(R) Lightweight Directory Access Protocol (LDAP) failed a request to connect to Active Directory Domain Services(R) for Windows user <ELOYALTYCO\spnbuadmin>.
 
Without the corresponding UNIX identity of the Windows user, the user cannot access Network File System (NFS) shared resources.
 
Verify that the Windows user is in Active Directory Domain Services and has access permissions.

The nbfsd log on the SQL server:

13:48:22.114 [4468.3744] <2> logparams: C:\Program Files\Veritas\NetBackup\bin\nbfs mount -server mn2infbak002 -port 7394 -timeout 60 -cred D0BEFC5586351B242802E736F01B6704D0C7A3BFA440726B7437F95F8A28FB958EDC5BA7AF6B5745A706DC3F235B3D0DBD30246092CF1BEB6ACCD4D1256DD13D *
13:48:22.136 [4468.3744] <2> rpc_connect: connecting to mn2infbak002
13:48:22.139 [4468.3744] <2> rpc_connect: connection established to mn2infbak002
13:48:22.139 [4468.3744] <2> nbfsd_op: making call 11 (128 *)
13:48:22.153 [4468.3744] <2> nbfsd_op: completed call 11 - 0  /NBUFS_A_28650F72B844FCC4_000 
13:48:22.153 [4468.3744] <2> nbfs main: mount -o mtype=soft-o timeout=60 -o retry=1 mn2infbak002:/NBUFS_A_28650F72B844FCC4_000 *
13:48:22.155 [4468.3744] <2> _nbfs_mount_info_check: last access information does not exist, setting it
13:48:22.160 [4468.3744] <16> _nbfs_mount_info_get_name: WNetOpenEnum() failed, error = 50
13:48:22.160 [4468.3744] <16> _nbfs_mount_info_set: unable to determine last export name
13:48:22.160 [4468.3744] <2> _nbfs_connect: adding connection for mn2infbak002:/NBUFS_A_28650F72B844FCC4_000
13:48:25.586 [4468.3744] <16> _nbfs_connect: WNetAddConnection2() failed, error = 53
13:48:26.605 [4468.3744] <16> _nbfs_connect: WNetAddConnection2() failed, error = 53
13:48:31.026 [4468.3744] <16> _nbfs_connect: WNetAddConnection2() failed, error = 53
13:48:32.037 [4468.3744] <16> _nbfs_connect: WNetAddConnection2() failed, error = 53
13:48:36.454 [4468.3744] <16> _nbfs_connect: WNetAddConnection2() failed, error = 53
13:48:37.464 [4468.3744] <16> _nbfs_connect: WNetAddConnection2() failed, error = 53
13:48:41.880 [4468.3744] <16> _nbfs_connect: WNetAddConnection2() failed, error = 53
13:48:42.890 [4468.3744] <16> _nbfs_connect: WNetAddConnection2() failed, error = 53
13:48:47.307 [4468.3744] <16> _nbfs_connect: WNetAddConnection2() failed, error = 53
13:48:48.317 [4468.3744] <16> _nbfs_connect: WNetAddConnection2() failed, error = 53
13:48:52.729 [4468.3744] <16> _nbfs_connect: WNetAddConnection2() failed, error = 53
13:48:53.740 [4468.3744] <16> _nbfs_connect: WNetAddConnection2() failed, error = 53
13:48:58.147 [4468.3744] <16> _nbfs_connect: WNetAddConnection2() failed, error = 53
13:48:59.159 [4468.3744] <16> _nbfs_connect: WNetAddConnection2() failed, error = 53
13:49:03.567 [4468.3744] <16> _nbfs_connect: WNetAddConnection2() failed, error = 53
13:49:04.578 [4468.3744] <16> _nbfs_connect: WNetAddConnection2() failed, error = 53
13:49:08.995 [4468.3744] <16> _nbfs_connect: WNetAddConnection2() failed, error = 53
13:49:10.007 [4468.3744] <16> _nbfs_connect: WNetAddConnection2() failed, error = 53
13:49:14.427 [4468.3744] <16> _nbfs_connect: WNetAddConnection2() failed, error = 53
13:49:15.439 [4468.3744] <16> _nbfs_connect: WNetAddConnection2() failed, error = 53
13:49:19.855 [4468.3744] <16> _nbfs_connect: WNetAddConnection2() failed, error = 53
13:49:20.866 [4468.3744] <16> _nbfs_connect: WNetAddConnection2() failed, error = 53
13:49:25.283 [4468.3744] <16> _nbfs_connect: WNetAddConnection2() failed, error = 53
13:49:25.283 [4468.3744] <16> _nbfs_connect: aborting connection, timeout period exceeded
13:49:25.284 [4468.3744] <2> nbfsd_op: making call 12 (128 )
13:49:25.303 [4468.3744] <2> nbfsd_op: completed call 12 - 0   

Any advice would be great.

M

[stu52]

From your Activity monitor, it looks like you are using tape for the backup.  This will not work since a document level backup using GRT would need to use disk as the backup.  If you are not using GRT, then you will be able to use tape.

[mfh1957]

Well I read a technote some place saying tape can be used too.  I am configuring a disk unit now, will let you know if that is all it is.  Thanks for the response.
 

[mfh1957]

Nope that did not fix it.

Looks like its not entirly a disk storage unit.  I added 4 disks locally to the media server, raid 10'ed them and created a storage unit from them, it still fails the same way.  Normal backups work to the disk storage unit.

[stu52]

OK...I was hoping to eliminate at least that part of the problem.

I found this technote for doing a check of nbfsd:

http://seer.entsupport.symantec.com/docs/346957.htm

At least check to see if you can test out NFS manually.

[mfh1957]

No Joy. 

It gets to the mount= and then gets a flashing cursor, waits a while and times out.

What bothers me the most is that AD error on the SQL server.  I will grab the logs later and post.  Thanks for the assist.

[Backup_Guy1]

Any Joy with this?

I have the same issue.
ERR - Client for NFS service is either not installed or not running. ERR - Granular items will not be cataloged.
 
I have confirmed with Symantec support NFS is installed and configured as per TN.
As result GRT jobs finish with status 1 “Error encountered while attempting to get additional files for Microsoft SharePoint Resources”

I have open case with Symantec support it is over 4 weeks no fix every day they aske me to send logs after logs.
We are exploring to deploy other backup product for this as we had no luck from Symantec support .

[Deepak_W]

Well well well...

I faced this issue (nbfs_connect: WNetAddConnection2() failed, error = 53) on my Exchange GRT backup. I followed below technote to resolve the issue and it worked.

http://seer.entsupport.symantec.com/docs/330034.htm

Try below mentioned steps, and check if this helps you...

First make sure that the network connectivity and host name resolution between Exchange client and media server are functional. If yes, then make sure that NFS Client service on Exchange system is running asNetwork Service account. The above failure can occur, for example, if NFS Client is running as local Systemaccount. 

If the account needs to be changed; NFS Client service must be restarted after the change. On Windows 2008 system, simply restarting the NFS Client Service via graphical user interface may not be effective. A better approach is

1. net stop nfsclnt 

2. net stop nfsrdr 

3. Make the desired change for the NFS Client service 

4. net start nfsrdr 

5. net start nfsclnt 

 

[Deepak_W]

Sharepoint backup checklist 

Also have a look at the below checklist for full backup of checklist --

1.  NetBackup Client version 6.5.5 is installed on all servers in the SharePoint farm, even SQL backend. (The SQL Server is part of the SharePoint farm)
 
2.  The NetBackup Client service is being started by a Domain Account
 
3.  The domain account in Step 2 has the following privileges and permissions:
a.  "Replace a process level token" and "Debug Programs" (Administrator Tools - Local Security Policy - Local Policies - User Rights Assignment) for all servers in the Sharepoint farm, backend SQL included.
b.  Local Administrator rights on all servers in the SharePoint farm.
c.  Within Sharepoint Central Administrator, the Domain Account is specified as a SharePoint Administrator
d.  The System Administrator role on the SQL Server for the SharePoint databases. 
 
4.  At a minimum, you must have .NET Framework 3.5 installed on all members of the Sharepoint Farm. 
 
5.  Under Host Properties - Clients on the Master server, under the Windows Client - SharePoint section, make sure that the Domain Account from step 2 is specified there, with the syntax as follows:  DOMAIN\Username  (not just the Username).  Do this for every member of the SharePoint farm!
 
6.  In the policy, the client selection is a Front End Web server for the Sharepoint portal.  The backup selection for your first policy should be: "Microsoft SharePoint Resources:\"
 
7.  Create a "beds" directory under <install path>\netbackup\logs\ on each of the SharePoint farm servers (SQL included)
 
8.  From the Front End web server specified in the policy, open the NetBackup Backup, Archive, and Restore (BAR) GUI.  You should be able to see Microsoft SharePoint Resources objects there.
 
9.  Under Host Properties - Master Server, there is a Sharepoint Hosts section.  For the first field, put in the name of the Front end server in the Sharepoint policy, and in the 2nd field, the name of the SQL server hosting the Sharepoint databases.

For setting up GRT backups of SharePoint, the Media server (if it is a Windows box) and the SQL server need to be running Windows 2003 R2 (or greater) with the following hotfix applied: http://support.microsoft.com/kb/947186

This the extract from one of the connect discussion thread - https://www-secure.symantec.com/connect/forums/error-200-sharepoint-backup



Hope this helps....