Exchange log files

[Betsey_Thurlo]

I am trying to figure out why my backup job isn't backing up the exchange folder? This is the folder that has the E*log files, the edb files, etc. It's selected (as well as the information store) but when I look at the backup job it just did not backup the exchange files on that drive. If the log files aren't being backed up, how would I restore the database to a specific point in time (I have to clear out the log files ever once in awhile or they would take up the whole exchange server hard drive free space)?
Thanks!

[Robert_Schmidt_]

Betsey - i'm not trying to tell you how to run your Exchange environment, but ...

ideally you should have your Exchange transaction logs writing to a different place than the actual stores (really, not just to different folders or logical drives, but to different physical spindles - i know, not always possible in all places).

You should never manually clear the log files - if you're running out of disk space, you need to get that infrastructure fixed. When you manually clear the transaction logs, you make any sort of recovery difficult.

You must use the BE Agent for Exchange.
The BE Agent for Exchange should take care of the log files for you. You do not select the folder from the server tree - when you are using the agent, you select the information store.

Now, when BE (and the agent) finish the information stores successfully, the agent then sends a signal to Exchange to flush the transaction logs. So, this is your "point in time" clean database for a recovery. Then, exchange starts writing a fresh set of transaction logs.

When it's working correctly, you can see all this interaction very clearly by stepping through the entries in the event log on the exchange server.

Cheers

[Betsey_Thurlo]

I understand I should move my log files to a different drive and I will do that promptly.
I have the exchange BE agent installed on the backup server.
I have the information store being backed up. But the transaction logs are not being cleared (even though the option is set to Full - clear logs in BE). I just upgraded BE to service pack 2 so I'm hoping that resolves it. There is a KB article about this problem (http://support.veritas.com/docs/280659) so I am going to try that setting change too.
I guess I'm confused...why can't you do a manual remove (after the backup job runs of course)...if BE is going to clear them anyway? I removed them by turning circular reference on and then back off again. After I made sure they were backed up somewhere else.

Betsey

[Robert_Schmidt_]

Hi - it's not SP2 - it worked correctly before that.
But, that KB article is important. your IS backup should be a separate job with AOFO turned OFF.

[Betsey_Thurlo]

Where I worked before we didn't have two separate jobs, we had one job that backed up everything including the information store. I don't know if we had different exchange log settings (i.e. circular referencing) but we never had a problem with drive space and the log files. If I have that setting unchecked referenced in that KB article, why doesn't it clear the log? This is something the job should do, I do not understand why the functionality is failing.

BDT

[Hywel_Mallett]

> Now, when BE (and the agent) finish the information
> stores successfully, the agent then sends a signal to
> Exchange to flush the transaction logs.

Just being really pedantic, it backs up the information stores, then it backs up any new transaction logs created while the IS backup was taking place, then it flushes the old logfiles.

[Betsey_Thurlo]

Just out curiosity, where does it backup the log files to? Because I cannot find them in any part of the restore, all I see is the information store. Also, again, the part that is not working, is BE is not clearing the old log files. I understand that is how it should work but it is not and I had 28 gigs of log files going back to 9/2006. This is a problem.

[Hywel_Mallett]

The log files get backed up to tape, but they just appear in the catalog as the Information Store backup - you can't select them specifically.
The removal of the transaction logs should be triggered straight away once the backup of the logfiles has finished.
If you look in Event Viewer on your Exchange server, are there any warnings or errors with the event source ESE?

[Betsey_Thurlo]

It looks like it might have started working once I unchecked that box in the open file settings area (kb article above). Last night for the first time this event appeared:
ESE: Informatino store first storage group: deleting log files e:\exchsrvr\mdbdata\E0006015.log to e:\exchsrvr\mdbdata\e000602a.log.

And if I look at the log files there are none older than 1/31/2007 7:00PM (when our backup job runs).

[Hywel_Mallett]

Well it looks like it's deleting the files correctly.
After each IS backup you should get a message like that one.

[Mark_Mills]

Betsy, I'm guessing you may have a small office your administrating, but I could be wrong. If your not being regulated by HIPAA or Sarbanes Oxley it would save you a lot of pain in the future to set up your Exchange server to do circular logging. This way it would commit files to the Exchange Database (Information Store) regardless of whether your backup job "succeeded" or not. The backup job would still be committing log files but this way if it failed then the Exchange server would be also committing files and you don't run as high of risk running out of disk space. Circular logging setting is found at: http://www.computerperformance.co.uk/images/Exchange/CircularLogging.jpg
or learn and read more at:
http://www.computerperformance.co.uk/exchange2003/exchange2003_circular_logging.htm#Where%20do%20you%20check%20the%20circular%20logging%20setting


I am not running circular logging but I do have 2 scripts that I run to that will commit my log files in the event BUexec fails. This was done to make sure I never run out of disk space due to overflowing log files.

1) the first script makes a backup of the Informattion Store using NTBackup (which commmits all log files) and logs each time it runs and if it succeeds successfully or not. The .bks file name has the date and time in it. The native NTBackup log file from the backup job is moved off the C:\ drive to another location

2) The second script deletes the data backed up by the NTbackup utility after x amount of days (you can make "x" be any amount of days you want). The script logs any file it deletes so you always have a list of all files deleted. (If you are truly interested in it, email me at mark.mills@desktop-assistance.com so I can send it to you)


For the script below to work: first use NTBackup file to create a job called ES01ExchStore.bks usually this just creates a file with the following syntax "JET ES01\Microsoft Information Store\First Storage Group\"

Here is the first script:

@echo off
rem The ExchStoreBackup.bat Script
rem
rem
echo.This script backs up the Exchange Store
echo. only to F:\ES01Store\NTBackup\
echo. The NT scheduler is currently starting this script at
echo. 3am Sunday through Thurday
echo.
echo. Sources that help me build this script are located at
echo.
echo. How to time and date stamp backup files found at
echo. http://support.microsoft.com/default.aspx?scid=kb;en-us;239892
echo.
echo. How to keep 30 days of ntbackup log files and move them to
echo. alternate location found at http://support.microsoft.com/?kbid=241162
echo.
echo. Ntbackup should overwrite any backup log files older than 30 days,
echo. so how do delete backups older than 30 days?
echo. I used a native MS command from the
echo. Windows 2003 Resource Kit called "Forfiles",
echo. Another resource on how to delete files older than 30 day
echo. can be found at http://www.michna.com/software.htm#DelOld
echo.
echo. How to use ntbackup to write a command line backup script
echo. to backup an Exchange 2003 server found at
echo. "http://groups.google.com/group/microsoft.public.exchange.admin/
echo. browse_thread/thread/c91796d23456bc0d/869ed7c96b260947%23869ed7c96b260947"
echo.
echo. PART ONE
echo. Setting the date and time stamp variables
echo.
echo. This sets the date and time stamp variable to be put in the
echo. name of the backup file
echo.
set tim=%time%
set dat=%date%
for /f "Tokens=1-4 Delims=/ " %%i in ('date /t') do set dt=%%i-%%j-%%k-%%l
for /f "Tokens=1" %%i in ('time /t') do set tm=-%%i
set tm=%tm::=-%
set dtt=%dt%%tm%
echo. PART TWO
echo. This backs up the Exchange Store and puts the date and time stamp
echo. in the name of the backup file. Note the ES01ExchStore.bks reads as
echo. JET ES01\Microsoft Information Store\First Storage Group\
echo.
ntbackup backup "@C:\Documents and Settings\Administrator.DA\Local Settings\Application Data\Microsoft\Windows NT\NTBackup\data\ES01ExchStore.bks" /j "EXch2003 Store %dtt%" /f "F:\ES01Store\NTBackup\ExchStore %dtt%.bkf" /v:no /r:no /l:s /rs:no /hc:off /SNAP:off /m normal
echo.
echo. PART THREE
echo. This copies the backup log from its native, obscure, profile, location on the C:\ drive
echo. and places the backup logs on a different and more logical location at
echo. F:\ES01Store\BackupLogs.
echo.
copy "C:\Documents and Settings\Administrator.DA\Local Settings\Application Data\Microsoft\Windows NT\NTbackup\Data\*.log" "F:\ES01Store\BackupLogs" /y
echo.
echo. PART FOUR
echo. This deletes the backup log from its native, obscure, profile, location on the C:\ drive
echo. (we wouldnt want the C: volume to get filled up with backup file logs would we)
echo. Note - a separate script purges log files after 14days on the alternate location.
echo.
echo.
del "C:\Documents and Settings\Administrator.DA\Local Settings\Application Data\Microsoft\Windows NT\NTbackup\Data\*.log"
echo.
echo. If you ever need to check to see if the database backups are successful then go to the logs at
echo. F:\ES01Store\BackupLogs
echo.
echo.
exit