Backups don't complete - hangs on Active: Running - Backup

[SuperBeta]

Just upgraded from a working install of Backup Exec 2012 to BE2014 14.1 with all update installed. Since upgrading, I have not had a single successful backup. I have deployed the 2014 backup agent to my servers and rebooted them after the install. They now validate as having the latest version of the agent installed.

The backups will run and appear to backup all of the selections as expected. The issue is the backup job never ends. It just hangs up on "Backing up". Job status indicates "Active: Running - Backup". If I view the job activity, it doesn't seem to be working on any current file or directory. Judging by bytes and file count, I'm pretty sure everything has been backed up, but the job doesn't stop.

The job log xml file shows the following:

Job server: BACKUPSRV 
Job name: SRV23.DOMAIN.LOCAL Backup 00014-Full - End of Week Job started: Monday, March 30, 2015 at 4:40:37 PM 
Job type: Backup Job Log: BEX_BACKUPSRV_04279.xml Job 
Backup Method: Full   
Drive and media mount requested: 3/30/2015 4:40:39 PM  
Drive and media information from media mount: 3/30/2015 4:40:42 PM 
Drive Name: Synology DS512+ 
Media Label: B2D014498 
Media GUID: {6326d716-216c-493b-a5d8-4eeac38bbb6f}  
Job Operation - Backup Backup Set Retention Period: 3 Weeks.
Compression Type: None 
Encryption Type: None  

SRV23.DOMAIN.LOCAL
Backup Exec server is running Backup Exec version 14.1.1786.1103 with SP-2,HF-227745. 
Agent for Windows(SRV23.DOMAIN.LOCAL) is running Backup Exec version 14.1.1786.1103 with SP-2,HF-227745. 
Snapshot Technology: Started for resource: "\\SRV23.DOMAIN.LOCAL\C:". 
Snapshot technology used: Symantec Volume Snapshot Provider (VSP) for Windows Server 2000 only. 
Snapshot Technology: Symantec Volume Snapshot Provider (VSP) snapshot cache file settings for volume "C:" : 
Initial: 12376 MB. Maximum: 12376 MB. File: "\\?\Volume{dd143455-36c9-11dc-a162-806d6172696f}\Backup Exec AOFO Store\_BEVspCacheFile_1.VSP". 
Snapshot Technology: Symantec Volume Snapshot Provider (VSP) snapshot settings used: Quiet Time: 5 seconds. 
Time Out: 2000 seconds. 
Network control connection is established between 172.22.1.62:62236 <--> 172.22.1.53:10000 
Network data connection is established between 172.22.1.62:62331 <--> 172.22.1.53:3450 \\SRV23.DOMAIN.LOCAL\C: 
Family Name: "Media created 3/30/2015 4:40:39 PM" 
Backup of "\\SRV23.DOMAIN.LOCAL\C:" 
Backup set #1 on storage media #1 Backup set description: "" 
Backup Method: Full - Back up files (using modified time) Backup started on 3/30/2015 at 5:06:53 PM. 
Backup completed on 3/30/2015 at 5:11:08 PM. 
Backed up 11495 files in 1321 directories. 
Processed 1,137,817,834 bytes in 4 minutes and 15 seconds. 
Throughput rate: 255 MB/min 

Additionally, cancelling the backup manually (which requires killing the job engine process) results in the Job Status being "Cancelled". It also means the files from the backup do not show up in the restore list when attempting to restore.

I can provide additional logs as needed.

[lmosla]

Check to make sure there aren't any alerts that need to be responded to. Are you backing up to disk or tape?

[SuperBeta]

We are backing up to Disk. An iSCSI disk array in this case.

I checked all of the alerts and I don't see any that I wasn't expecting. There is always one drive that is offline since we have a storage pool with 2 usb disks that I switch out weekly. Other than that, it's just alerts that jobs were manually cancelled.

[pkh]

Somehow your job is using the Symantec Volume Snapshot provider (VSP) which was discontinued from BE 2012. Edit your job and specify the Microsoft provider for AOF. For the longer term, uninstall the remote agent, including the VSP, from the remote server and then push the remote agent out to the remote server again

[pkh]

See the screenshot above

[SuperBeta]

My initial post was a backup of a old test box that was Windows XP. I believe that is why it is using VSP. I see that this is no longer supported as of BE2014, so I understand why this may not work as expected.

However, I am having exactly the same issue on a Server 2008 box now, using VSS. Here is the log for that:

 Job server: BACKUPSRV 
 Job name: SRV15.DOMAIN.LOCAL Backup 00004-Incremental 
 Job started: Monday, March 30, 2015 at 7:18:16 PM 
 Job type: Backup Job Log: BEX_BACKUPSRV_04280.xml 
 Job Backup Method: Incremental  
 Critical Resource \\SRV15.DOMAIN.LOCAL\D: was not included in the original selections list, but has been included to insure complete protection. 
 Backup Exec will not follow mount points, junction points, and symbolic links when the Simplified Disaster Recovery indicator is ON.  
 Drive and media mount requested: 3/30/2015 7:18:36 PM  Drive and media information from media mount: 3/30/2015 7:18:38 PM 
 Drive Name: Synology DS512+ Media Label: B2D014499 Media GUID: {9aabe7d4-0ea9-4ede-b6a5-70a3325e6794}  Job Operation - Backup Backup Set Retention Period: 1 Week.Compression 
 Type: None Encryption Type: None  SRV15.DOMAIN.LOCAL
 Backup Exec server is running Backup Exec version 14.1.1786.1103 with SP-2,HF-227745. 
 Agent for Windows(SRV15.DOMAIN.LOCAL) is running Backup Exec version 14.1.1786.1103 with SP-2,HF-227745. 
 The following resources that were selected for backup can be used for future conversion-to-virtual jobs: 'C:, D:, System?State' 
 The following resources that were selected for backup cannot be used for future conversion-to-virtual jobs: 'Utility Partition' 
 Snapshot Technology: Started for resource: "\\SRV15.DOMAIN.LOCAL\C:". 
 Snapshot technology used: Microsoft Volume Shadow Copy Service (VSS). The snapshot technology used by VSS for volume C: - Microsoft Software Shadow Copy provider 1.0 (Version 1.0.0.7). 
 Network control connection is established between 172.22.1.62:50977 <--> 172.22.1.59:10000 Network data connection is established between 172.22.1.62:50985 <--> 172.22.1.59:49682 \\SRV15.DOMAIN.LOCAL\C: 
 Family Name: "Media created 3/30/2015 7:18:36 PM" Backup of "\\SRV15.DOMAIN.LOCAL\C:" Backup set #1 on storage media #1 Backup set description: "" 
 Backup Method: Incremental - Back up changed files since the last full or incremental (using modified time) 
 Backup started on 3/31/2015 at 4:06:20 AM. 
 Backup completed on 3/31/2015 at 4:08:33 AM. 
 Backed up 1615 files in 349 directories. 
 Processed 5,807,838,271 bytes in 2 minutes and 13 seconds. 
 Throughput rate: 2499 MB/min ---------------------------------------------------------------------- 
 Note: Resource: "\\SRV15.DOMAIN.LOCAL\Utility Partition" is not snappable. 
 Attempting to back up directly from the source volume.  
 Drive and media mount requested: 3/31/2015 5:58:10 AM  
 No appendable media could be mounted. 
 Switching to overwrite operation on scratch media.  
 Drive and media information from media mount: 3/31/2015 5:58:12 AM Drive Name: Synology DS512+ Media Label: B2D014500 Media GUID: {19b7dfd0-aafa-4466-ab6b-59ccca3db168} 
 Network control connection is established between 172.22.1.62:50977 <--> 172.22.1.59:10000 
 Network data connection is established between 172.22.1.62:51976 <--> 172.22.1.59:62623 \\SRV15.DOMAIN.LOCAL\Utility Partition 
 Family Name: "Media created 3/30/2015 7:18:36 PM" Backup of "\\SRV15.DOMAIN.LOCAL\Utility Partition" Backup set #1 on storage media #1 Backup set description: "" 
 Backup Method: Full - Back up files (reset archive bit) 
 Backup started on 3/31/2015 at 5:58:12 AM. 
 Backup completed on 3/31/2015 at 5:58:14 AM. 
 Backed up 1 Utility Partitions Processed 82,240,620 bytes in 2 seconds. 
 Throughput rate: 2353 MB/min ---------------------------------------------------------------------- 
 Snapshot Technology: Started for resource: "\\SRV15.DOMAIN.LOCAL\D:". 
 Snapshot technology used: Microsoft Volume Shadow Copy Service (VSS). 
 The snapshot technology used by VSS for volume D: - Microsoft Software Shadow Copy provider 1.0 (Version 1.0.0.7).  
 Drive and media mount requested: 3/31/2015 7:50:04 AM  No appendable media could be mounted. Switching to overwrite operation on scratch media.  D
 rive and media information from media mount: 3/31/2015 7:50:08 AM 
 Drive Name: Synology DS512+ Media Label: B2D014501 Media GUID: {5d140dcd-8e55-470b-881d-e6f64f98207d} 
 Network control connection is established between 172.22.1.62:50977 <--> 172.22.1.59:10000 Network data connection is established between 172.22.1.62:52460 <--> 172.22.1.59:52379 \\SRV15.DOMAIN.LOCAL\D:

[pkh]

How is your D: drive connected to the machine? It appears to be a junction point which is why it fails to back up

[SuperBeta]

D: is a local drive on the server. Direct attached via onboard raid controller. It is not a junction point. I just don't have it selected to be backed up. Simplified Disaster Recovery is turned off for this server. 

[SuperBeta]

In further testing, it seems I am having additional issues as follows:

-Restores from previous backups captured using BE2012 will not restore, even when the restore is setup to recover the files locally on the media server. The jobs just remain Queued indefinately.

-Backups also hang in "Discovering Resourses" for hours; even a backup job that only backs up the media server itself hung up in this state.

-Cancelling jobs nearly always requires opening the Services Manager to restart all servers, in addition to manually killing the bengine.exe process in Task Manager. I suppose this is the same as it was in BE2012 too.

Are these types of issues common and able to be resolved. I'm considering uninstalling Backup Exec and installing it again without any old data or catalogs, but I don't want to loose the ability to restore old data. I'm also concerned if a fresh install will be aware of my old backup sets and be able to overwrite them as necessary without me having to manually prune them overtime. I have 8TB of backups I need to keep from BE2012.

[SuperBeta]

Okay, so still trying to get a successful backup out of this program. I'm going to try to avoid reinstalling anything for now. 

After some more reading, I decided to test deleting the backup job and recreating it. So far, the results seem the same. The job starts out backing things up okay, basically just the C: drive. After it completes the C drive backup, it doesn't seem to move on from that point. It just sits there. Here is the Job Activity Window of a full backup of the media server itself going to disk.

[SuperBeta · Accepted solution]

Okay, after having all of these issues I decided to reinstall from 0. Had to catalog all of my previous backups but now everything seems to be working much better. Thanks for the suggestions everyone. Here's hoping I will continue to have a functioning backup product for a while.