Sharepoint Status 13 since upgrade to 7.1.0.1

[Marianne]

Master:  SUSE Linux Enterprise Server 11 (x86-64)

Client : 
Sharepoint: Windows  2003 SP2 (32-bit) Sharepoint version: 2007
SQL: Windows  2003 SP2 (32-bit) SQL version:  2005

Background :

Backup was successful on Monday night (NBU version 7.0.1)

Master server and all media servers as well as all clients were upgraded to 7.1.0.1 on Tuesday (yesterday).

We have double-checked Client Service logon accounts on Sharepoint and SQL servers – still the SAME ‘sharepointbackup’ account as before the upgrade.

Error in Job details as well as bpbrm and bpbkar logs:

Error bpbrm (pid=32032) from client XXX-web1: ERR - failure reading file: Microsoft SharePoint Resources:\Windows SharePoint Services Help Search\Search instance\Search-DB 1 (XXX-SQL1/WSS_Search_XXX-WEB1)\WSS_Search_XXX-WEB1 (BEDS 0xE000FEA9: The Backup Exec data store encountered a problem during the operation. See the job log for details.)

Rest of directives in this list backup successfully:

Microsoft SharePoint Resources:\ConfigurationV3-DB (XXX-SQL1/SharePoint_Config)\
Microsoft SharePoint Resources:\XXXamin Shared Services\Services-DB 1 (XXX-SQL1/XXXaminSharedServices_DB)\
Microsoft SharePoint Resources:\XXXamin Shared Services\Intranet\Content-DB 1 (XXX-SQL1/WSS_Content_Intranet)\
Microsoft SharePoint Resources:\XXXamin Shared Services\Shared Search Index\Search-DB 1 (XXX-SQL1/XXXaminSharedServices_Search_DB)\
Microsoft SharePoint Resources:\Windows SharePoint Services Help Search\Search instance\Search-DB 1 (XXX-SQL1/WSS_Search_XXX-WEB1)\
Microsoft SharePoint Resources:\WSS_Administration\WebApplication\Content-DB 1 (XXX-SQL1/SharePoint_AdminContent_81639db1-73d4-4a47-87ff-31b8481873a1)\

I have looked at all of these TN’s:

http://www.symantec.com/docs/TECH157854

http://www.symantec.com/docs/TECH143436

http://www.symantec.com/docs/TECH29781

http://www.symantec.com/docs/TECH140635

None of them seem applicable.

I am about to log a support but would like to avoid that avenue for as long as possible....

ANY IDEAS??

[CRZ]

Hi Marianne,

I feel pretty sure you'll need to open a support call for this one.

Your issue looks a lot like it may be already logged as Etrack 2405300 (for 7.1) - see if you can get your TSE to start there.  If they can confirm you have the same issue, get it escalated to backline.

Sorry I don't have better news!

[RonCaplinger]

"The Backup Exec data store encountered a problem during the operation. See the job log for details."

Backup Exec?  You are using NetBackup...or is this possibly one of those features that may have crossed over from one product to another and they didn't update the messages...?

[CRZ]

You can find a lot of BEDS (Backup Exec data store) logs in NetBackup...especially when using certain agents, such as the SharePoint one.  :)

[RonCaplinger]

Makes more sense.  Ignore me, Marianne! 

[Marianne]

I was just told by the backline engineer that this is a KNOWN ISSUE...

I just wasted an entire day of my LIFE on a KNOWN ISSUE.

KNOWN to whom?? Symantec Staff only? Does a KNOWN ISSUE not warrant a TechNote? If this was documented in LBN, I would NOT have bothered to upgrade this set of clients.

Sorry Symantec staff - I have been defending your product with my LIFE for the last 12 years. Today is simply NOT one of those days...........................

[Marianne]

Hi Chris

Would it be worth the effort to ask Symantec to clean up BEDS error messages? (I haven't experienced much joy with posting IDEAS...) It used to confuse me as much as it has confused Ron......

[RiaanBadenhorst]

7.1 seems a bit flakey to me :p

[Marianne]

We are worse off today….

 We have uninstalled 7.1 on the Sharepoint and SQL clients, installed 7.0 and patched to 7.0.1.  Updated NBU Client Service Logon account to the same SharepointBackup account that worked fine until Monday night.

 Backup of SQL server using SQL agent works 100%

 Kicked off a manual Sharepoint backup.  One backup stream now fails with Status 175 and a Status 150 can also be seen in Job details:

07/21/2011 10:47:39 - Info bpbrm (pid=22216) sending bpsched msg: CONNECTING TO CLIENT FOR XXX-web1_1311238058

07/21/2011 10:47:39 - connecting

07/21/2011 10:47:40 - Info bpbrm (pid=22216) start bpbkar on client

07/21/2011 10:47:40 - connected; connect time: 0:00:00

07/21/2011 10:47:43 - Info bpbkar (pid=0) Backup started

07/21/2011 10:47:43 - Info bpbrm (pid=22216) Sending the file list to the client

07/21/2011 10:47:44 - Info bpbrm (pid=22216) sending media manager msg: STOP BACKUP XXX-web1_1311238058

07/21/2011 10:47:44 - Info bpbrm (pid=22216) media manager for backup id XXX-web1_1311238058 exited with status 150: termination requested by administrator

07/21/2011 10:47:44 - end writing

07/21/2011 10:48:38 - Info bpbrm (pid=22216) killing bpbrm child 22226.

07/21/2011 10:48:38 - Info bpbrm (pid=22216) sending media manager msg: STOP BACKUP XXX-web1_1311238057

not all requested files were restored  (175)

[Marianne]

We managed to get the Sharepoint backup working again on version 7.0.1....

When we removed the software and reinstalled, all NBU registry entries were obviously wiped.

We changed the Client Service logon account on Sharepoint and SQL clients, added user via Host Properties -> Clients -> Windows Client -> Sharepoint.  At this point we only updated Host Properties for the Sharepoint Client and forgot about SQL...

[CRZ]

Hi Marianne,

I'm really sorry for your frustration.  Although this is a "known issue," it definitely hasn't been known for very long and even though one affected customer is too many, we hadn't seen this issue affect customers at the leve where would we have gotten it documented and into the LBN.  (I hadn't heard about this at all until your Connect post!)

A TechNote will obviously be forthcoming and this issue will be included in the next maintenance release's Release Notes, but I know that that's little consolation for you as it's too late to help you.

All I can do is promise to try to do better in the future.

[Marianne]

Symantec has identified the bug and supplied an EEB last night.

NB_7.1.0.1_eebinstaller.2526376.1.....

Customer will upgrade client again, patch, then apply EEB.

Will update again when that is done.

[Dyneshia · Accepted solution]

If you are seeing the same issue in 7.1.0.2 the eeb was ported for this release : 2589733

eebinstaller.2589733.1.AMD64.exe
eebinstaller.2589733.1.x86.exe

[Marianne]

GREAT STUFF!!!! Will let customer know!

Forgot to update as promised.... EEB for 7.1.0.1 fixed the problem, but customer was asking for a fix for 7.1.0.2 as well since the original EEB did not make it to the 7.1.0.2 cut off.