cancel
Showing results for 
Search instead for 
Did you mean: 

Accelerator caused Journal log of 80 GB

V4
Level 6
Partner Accredited

Recently we migrated to 7.5 and wanted to leverage benefits of accelerator with our Dedupe jobs which happens over WAN.

Strange behaviour we noticed was if we enable Use Change Journal option in Client Host properties, Then During 1st Full backup Accelerator is causing Journal Log building at Client end whopping entire System disk ... 80 GB in system drive.. System ran out of space...affecting other application.

Had any one faced this issue.

 

Note: If Change journal option is unchecked Log is not created.

 

23 REPLIES 23

mph999
Level 6
Employee Accredited

Is this the track_journal.v1 file ?

My understanding (from Symantec internal TSE training) is that around 30MB of space should be allowed, so 80GB, from my understanding - something is wrong.

You should log a call on this - it sounds like it 'might' be a bug, (but note, I am not say it is for sure),

Regards,

Martin

V4
Level 6
Partner Accredited

yes its track_journal file...

Check the attached one...

mph999
Level 6
Employee Accredited

Hmm, found this ... maybe normal ..

 

Accelarator is NOT recommended for systems that have a high change rate.

The size of the track log in bytes can be approximated using the following formula:
(number of files x 200)+ ((Total used disk space/128KB) * 20)
For example a file system containing 1,000,000 files and using 1 TB of disk space would have a track log of:
(1000000 x 200) + ((1TB/128KB)*20) = 367772160 bytes = about 350 MB

 

Martin

V4
Level 6
Partner Accredited

350 MB or upto 1 GB of log, would be acceptable.. but it's whopping 80 GB... sounds odd...i'd better log case and report it to support for further investigation...

mph999
Level 6
Employee Accredited

Could you make te calculation and see what it comes out as.

If you do log a call, could you update the post here to help others, and also, if any of these posts have helped you, mark as solution.

Many thanks,

Martin

V4
Level 6
Partner Accredited

sure

ddm2
Level 5
Partner Accredited Certified

Similar problem here; 2 fileserver with a 25 GB track log file.

On both I had "Perform snapshot backups" enabled on the policy attributes. I unchecked that, deleted track log files and started new full backups. I'm monitoring file growth, they seem to remain stable (1.5/2 GB).

I'm not sure the snapshot used with the accelerator was the issue.  

Any feedback very apreciated.

 

Diversion
Not applicable

Having the same issue here - I'm using 7.5.0.1 to backup my remote WAN file servers (about 500,000 files on each) using Accelerator.  

In my situation however, regardless of Change Log turned on or off for the clients, the Track journal files grow to immense, astronomical sizes.  In some cases 500gb causing my file servers to crash from loss of ALL disk space.    The track journal logs are LARGER than the amount of data i'm backing up.

This is a heck of a bug/problem.  I already opened a case with Symantec and the engineer stated that he's seen this before and the way they fixed it was to uncheck "Windows open file backup" on each client but this didn't change a thing for me.    The engineer also stated that he believes the fix is coming in 7.5.0.3.  

I hope he's right, nothing has improved or fixed my situation.  Netbackup has rendered itself 100% useless for us at this point in time.   I need to use Accelerator because my circuit speeds to some of these remote WAN sites do not have the capacity to do Full Backups every weekend.  They initially took over a week for their first backup.

 

 

mph999
Level 6
Employee Accredited

OK, that is not good.  Was the call escalated to BL, (I guess not) - can you ask for it to be escalated, and see if an EEB is available.  What is the case number ?

To workaround the issue, what about synthetic backups ?

Martin

Omar_Villa
Level 6
Employee

Check bptm, bpbkar and bpcd logs and see if something is getting stuck over there, is deffenitely not normal.

cicaza
Not applicable

Updagrade 7.5.0.3  - same error  directory %%\Veritas\Netbackup\track\     250 GB

 

Marianne
Moderator
Moderator
Partner    VIP    Accredited Certified

Is this problem only seen on Windows?

We were advised by Symantec SE and PM to recommend Accelerator to customer with large filesystems on RHEL client.
Seems this might NOT be a good idea........ crying

mph999
Level 6
Employee Accredited

I think only windows, as Accelerator does not implement UNX file system change detection as it does for win ntfs change journal.

Martin

Jed_Gresham
Level 4
Employee Accredited

One of the things we'll have to get used to with accelerator is the paradigm shift it may be causing.  What you might want to try is scheduling backups to run more often on these servers where the track log is filling up because of a high change rate.  You can setup backups for every 4 hours that expire after 24 or 48, and then keep your normal daily backups at the retention you currently have.  Might be worth a test if nothing else.  Could also give you some pretty good RPO.

ddampf
Level 2
Partner

We are running 7.5.0.1 on RHEL master/media servers, and we recently deployed Accelerator in 6 of our NBU environments consisting of about 5,000 total Windows clients.  We saw an immediate increase in the performance after enabling accelerator and change journaling (faster full backups, and a shorter backup window).  After about 2 or 3 weeks with this implemented, we started getting alarms for filesystems nearing 100% on their C drive.  The accelerator track log had grown to several GB in size (some servers 20+ GB), and we had to quickly turn off change journaling and accelerator, and quickly write scripts to query and purge the track log directory on the 5,000 clients.

Is this a known bug, is there an ETRACK log, or an EEB that can be applied or that will be included with 7.5.0.3, or a future release?

My colleague logged a case a couple of weeks ago, and Symantec didn't seem to have any answers.

I really want to get accelerator running again, but I'll have to do it in a more controlled manner.  I think we'll be spending time designing scripts that can be ran weekly to check the size of the track directory on the clients.  Our current scripts are not very robust and were thrown together in the middle of the crisis.

Please let me know if anyone else has come up with a great way to monitor the size of the track directories on Windows clients.  We are using some Powershell, but it's only working on about 30% of our Windows clients.  Our current method is to use "pstools" to run the "dir" cmd for the directory and output it to a file.

 

-David

Spartacus81
Level 6
Partner Accredited

I have come across the same issue.. NBU 7.5.0.1 big track journal file..

i really dont want to disable the VSS otherwise open files wont get backed up..

has anyone found the solution of this issue yet, upgrade? EEB? can we point the track log file to another location? etc...

i have disabled the accelerator option for the time being and running inc and synthetic..

 

ddampf
Level 2
Partner

Symantec has informed us that a fix is coming with 7.5.0.4 or 7.6.

Dyneshia
Level 6
Employee

Are you using "checkpoint restart"  please see : http://www.symantec.com/docs/TECH195758

Dyneshia
Level 6
Employee

7.1.0.4 has been released.  You can download it here : http://www.symantec.com/docs/TECH194138