A few days ago, our Windows server 2008 R2 has crashed because of a Out of Memory event caused by beremote.exe process which used up to 30GB of memory...
Windows successfully diagnosed a low virtual memory condition. The following programs consumed the most virtual memory: beremote.exe (10192) consumed 31887167488 bytes, dns.exe (3084) consumed 803971072 bytes, and w3wp.exe (6416) consumed 711815168 bytes.
After investigating, a backup failed within the same period:
Job ended: Saturday, July 16, 2016 at 5:34:27 AM
Completed status: Failed
Final error: 0xe0009579 - Unable to copy the virtual machine disk using the VMware VixDiskLib.
Final error category: Resource Errors
For additional information regarding this error refer to link V-79-57344-38265
Error writing file data.
V-79-57344-65304 - The Virtual Machine resource is not responding. Backup set canceled.
V-79-57344-65072 - An error occurred while connecting to the Backup Exec Remote Agent on the media server. Ensure that the job options are correct, and then submit the job again. The backup set was not created.
So the backup failed 1h prior to the run out of memory failure and memory growth was exponential from there...
We run Backup Exec 2010 R3 on Windows Server 2008 R2
So how can we avoid beremote.exe to consume all of the memory like that? What is going on?
Ideally if I can have some logs from BEremote to see why this memory ballon in the first place, it would be very helpful.
There are 2 things you should be checking first:
1. Run LiveUpdate to make sure BE 2010 R3 is fully patched with the latest SP and subsequent patches;
2. Make sure any antivirus installed isn't perhaps actively scanning the BE services, and if so, put in an exclusion for them.