Agent crashing on Domino after upgrade to 12.

[curriertech]

We've been on 10d for quite a while, then a few weeks ago upgraded to 11d with no issues.  When the notification of 12 came out I jumped on it, and I haven't had a good Domino backup since because the agent keeps crashing.  The agents on our other non-Domino servers seem to be fine.  I recall this being a common issue with 11d when it was first released.  Here is the error that shows in the event log on the Domino server:

Faulting application beremote.exe, version 12.0.1364.0, faulting module bedsnote.dll, version 12.0.1364.0, fault address 0x00022040.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

Any ideas?  I'll go back to 11d if I have to, but I'd much rather get this one issue worked out and stick with 12. 

[Mats_Holm]

I had a case open with Symantec for three weeks without finding any answer.

Customer got tired of not having any daily backups so we skipped the incremental backup of Notes and started to do Full backup instead. This is OK for this customer cause it only differs 10 % for a full backup and an incremental.

Sorry to not being able to give any answers to the problem. We saw the same thing as David Cook regarding the UCJ files and crash of RA. Symantec wanted us to install a Microsoft tool call Debug Diag to find out why the RA crashed but as I said the customer said no.

If anyone else has any solution or has a possibility to investigate together with Symantec please update this thread.

//Mats

[Stayduro]

I have the same problem, with Domino 8.0.1 and BE-12. I used the agent for some weeks, without problems. Now I cannot backup the domino server because the agent crash. I edited the mentioned registry keys and I deleted ucj file, but I didn't solved the problem. Now I cannot browse domino database, and on Domino console, every minute, I see in the log this error: "c:\program files\symantec\belnapi.exe has terminated abnormally".

[big_trouble]

I have not had a successful backup since installing BE 12. This is a fresh install, not an upgrade from a previous version. I receive multiple errors when performing a backup of the Lotus server.

Windows 2003 SP2, Domino 7.0.2, Symantec AntiVirus Enterprise Edition 10.1.6.6000, Symantec Mail Security for Domino 4.1

Event error 1:

Event Type: Error
Event Source: Application Error
Event Category: (100)
Event ID: 1000
Date:  6/23/2008
Time:  8:46:08 PM
User:  N/A
Computer: DOMINOMAIL01
Description:
Faulting application beremote.exe, version 12.0.1364.0, faulting module bedsnote.dll, version 12.0.1364.0, fault address 0x00022040.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 41 70 70 6c 69 63 61 74   Applicat
0008: 69 6f 6e 20 46 61 69 6c   ion Fail
0010: 75 72 65 20 20 62 65 72   ure  ber
0018: 65 6d 6f 74 65 2e 65 78   emote.ex
0020: 65 20 31 32 2e 30 2e 31   e 12.0.1
0028: 33 36 34 2e 30 20 69 6e   364.0 in
0030: 20 62 65 64 73 6e 6f 74    bedsnot
0038: 65 2e 64 6c 6c 20 31 32   e.dll 12
0040: 2e 30 2e 31 33 36 34 2e   .0.1364.
0048: 30 20 61 74 20 6f 66 66   0 at off
0050: 73 65 74 20 30 30 30 32   set 0002
0058: 32 30 34 30               2040   

Event error 2:

Event Type: Error
Event Source: Backup Exec
Event Category: None
Event ID: 65215
Date:  6/18/2008
Time:  7:16:30 PM
User:  N/A
Computer: DOMINOMAIL01
Description:
The connection to the Lotus Domino server has been lost. The following error was reported: Notes Agent:CLNFileSystem::AttachToDLE - Access Violation Exception Caught. The job that was running on this server has been stopped.

 For more information, click the following link:
http://eventlookup.veritas.com/eventlookup/EventLookup.jhtml

I also attempted the Change Journal registry fix with no success. Does anyone have any suggestions or an estimate from Symantec as to when a fix will be available?

[David_Cook_5]

Is it possible to spilt this thread?

We are now listing two seperate issues in this thread.

1. The Original is a problem with the BEREMOTE.EXE crashing after upgrading to Backup Exec 12. This is a problem with the UCJ file becoming corrupted

2. The second issue is with BELNAPI in Domino 8. This is not the same issue

[GrayFog]

We have the Agent crashing problem, too. We don't even use the LD db backup functionality only doing file based backups (LD service is stopped before and restarted after backup). I'll change the registry entries and see whats happening...

[StiflingCobra]

Having the same issues here with Lotus Agent.  Initially had the problem of not being able to browse databases and select them but the registry hack works for that.

Now BackupExec has started reporting files as corrupt which are fine and have no issues (eg mailjrn.ntf / perfweb.ntf). So have had to delete and remove these only to now have the servers say they can no longer attach the remote agent even though they are backing up themselves.

This is a real problem for us as we have over 500+ domino servers out in the fiekld and this is stopping us upgrading them to v8.

Have been working with Symantec support but they aint been great so far and are being VERY tight lipped overthe problems and not commenting on any future potential hotfixes to resolve these issues.  One thing to note one of our site removed Windows 2003 SP2 (going back to just SP1) from one of it's servers having the problem and Backup Exec 12 has worked flawlessly since. Will let you know if I get anythign from Symantec.

Right off to kick our Symantec Account Managers bottom over this!

[StiflingCobra]

Ok I've just come off the phone with a Symantec Engineer and they advised the following:-

Symantec has no intentions of supplying a hotfix for this problem that can be fixed by editing the registry.  You will have to create a reg file that can be imported/run on each manchine and do it manually.  The next release of Backup Exec 12 will have this issue fixed though.

So there you go folks.

Message Edited by StiflingCobra on 07-02-2008 06:54 AM

Message Edited by StiflingCobra on 07-02-2008 06:54 AM

[big_trouble]

I was provided a solution from a Symantec tech last week and so far I have had all successful backups. Here is what we did:

1. Stop the Backup Exec remote agent service on the Domino server 

2. Go into the registry and browse to the following path:

    HKLM\SOFTWARE\Symantec\Backup Exec for Windows\Backup Exec\Engine\DOMINO

3. Change the following 2 DWORD values:

    Enable Change Journal = 0

    Enable Notes Database Filtering = 0

4. Close the registry

5. Browse to C:\Program Files\Symantec\Backup Exec\RAWS\logs and delete NOTESDB*.UCJ files

6. Restart the Backup Exec remote agent service.

Note: Do not change the registry settings back to there original settings. Every article that I found had me change the registry settings back. The Symantec tech told me that the changes will not affect the Domino backups.

[GrayFog]

Works now.

In our case the agent started the Domino backup functions (two sub processes to beremote.exe started) because not all Domino services had stopped and crashed.

For other users who wants to do an offline Domino DB backup: remember to disable the global exclusion list of BE server and set the reg key for the agent to allow offline backups. Otherwise no DB files (*.nsf) will be backed up.

[jecdk]

I did an install of the remote agent on the Notes server itself from the media.

My test backups are running with NO problem, but I'll post the result of the "normal" backup here....

[Joji]

hi, any updates on your tes backups? thanks.

[rlj]

I have the same beremote crash problem on a new Domino 7.03/Win 2003 R2 SP2 system.  I'm doing this now and will report back..  Ron

1. Stop the Backup Exec remote agent service on the Domino server

2. Go into the registry and browse to the following path:

    HKLM\SOFTWARE\Symantec\Backup Exec for Windows\Backup Exec\Engine\DOMINO

3. Change the following 2 DWORD values:

    Enable Change Journal = 0

    Enable Notes Database Filtering = 0

4. Close the registry

5. Browse to C:\Program Files\Symantec\Backup Exec\RAWS\logs and delete NOTESDB*.UCJ files

6. Restart the Backup Exec remote agent service.

Note: Do not change the registry settings back to there original settings. Every article that I found had me change the registry settings back. The Symantec tech told me that the changes will not affect the Domino backups.

[Joji]

Thanks rlj for the update. Yes, please do send an update. I am still using 11d for the RAWS and 12 for BEWS.

[rlj]

Well...  The Notes database backup appears to work, but the console messages (RAWs terminated) continue.  I have 11D running on a Windows file server, but without a Notes license there.  If I copy the 11D RAWS and install it on the Notes server, will the v12 backup server then backup the Notes server?  Ron

[Guizmo]

Hi all,

I face the same problem after migrating to version 12.5 in ordder to backup Domino 8 that is supported only since version 12 of Backupexec. The domino console was pollut by a message regarding benlapi.exe terminated abnormally and was not possible to browse the selections list for Notes databes.The backup job failed in the same time.

After reading all the post for Domino agent , i made a mix of everything. To resume, i remove the agent from the panel control/add&remove program and reboot the notes server. After reboot, i install the remote agent directly on the server. Made the change in the registry:

Changed Enable Change Journal to 0

Changed Enable Notes Databse Filtering to 0

and after that i restart the Domino server.

I turn the remote agent utility to run each time when logon to the server.

I have no more error in the Domino console. I can browse from the backup server and change the selection as i want and finaly, for the moment, backup are running and i have no more errors.

The server where the agent is installed run on W2K3 Std edition SP2 english 

Domino enterprise server 8

The version of BE is 12.5rev 2213

Hope that will help you to fix your problem if it s similar.

I will back to this discussion in one week for the status

[GrayFog]

---

@BlackFog wrote:

For other users who wants to do an offline Domino DB backup: remember to disable the global exclusion list of BE server and set the reg key for the agent to allow offline backups. Otherwise no DB files (*.nsf) will be backed up.

---

"global exclusion list" means here "Active File Exclusion". See 259152

[Cyberfed]

I just wanted to report my expirence with this same EXACT problem.

I upgraded from 11d to 12/12.5.  Around this time we installed a domino 7 server in our office as a backup server. I was getting the same error messages and event logs people listed above

Since the upgrade to 12.5 I have never gotten a 100% sucessful backup of my domino server.

I had googled the error codes and stumbled across this page.

I can confirm that the following fix WORKED for me:

 System is a 2003 server ent ed 32bit with sp2

2. Go into the registry and browse to the following path:

    HKLM\SOFTWARE\Symantec\Backup Exec for Windows\Backup Exec\Engine\DOMINO

3. Change the following 2 DWORD values:

    Enable Change Journal = 0

    Enable Notes Database Filtering = 0

4. Close the registry

5. Browse to C:\Program Files\Symantec\Backup Exec\RAWS\logs and delete NOTESDB*.UCJ

At first I only noticed the part to make the registry changes, did that and rebooted. No luck backup still failed. Then I read the part about deleting the UCJ files. Deleted them restarted the backup exec service and kicked off a backup.

I just got a 100% success backup.

[Mats_Holm]

Hi all! Looks like the problem still exist with 12.5, strang that they can't fix it....

When You say that the backup is 100% after regfix, have the agent utility in autorun do you do Full backups or any Incrementals?

We hade problems only when running incremental backups.

//Mats

[CWtechnician]

Hello everyone,

I see that this may be an old thread but I would like to resurrect it if I may.

I have done everything mentioned within this thread (RegKeys.. Etc) and at this point in time I have spoken with three different Symantec technicians with 0 resolve.

Win03 server / Domino 08/ Notes 08/ Symantec BE v12.0 

I am having the belnapi.exe is abnormally terminating error displayed on the Domino console. Within the failed full backup failure job log I am getting the error message:

Job ended: Saturday, November 29, 2008 at 12:30:53 AM
Completed status: Failed
Final error: 0xe000fe30 - A communications failure has occurred.
Final error category: Server Errors
"For additional information regarding this error refer to link V-79-57344-65072

This is occurring on 40 of our servers with the same setup described above and even the Symantec Technicians are having problems with this. I just sent a log to a technician but have heard nothing. I can increment the backups but once a full backup is performed it always ends with failure, "A communications failure has occurred". I cannot have this on these servers as the data is crucial.

We do have one server on Symantec v12.5 and this server has had zero issues but within this very thread a user has been having problems - so this may still be an issue.

Is this happening for servers that do not have Lotus Domino installed on them or is it just us?

Has anyone resolved this issue or any new advise? S.O.S.

Thanks in advance,

D

[rlj]

The 12.5 upgrade appears to have fixed the problem for me..  Ron