Backup Exec VSS Writer
Good day, We are running Backup Exec on Windows Server 2008 R2 Standard. The backups keep failing when running a job for the Hyper-V Host server (Windows Server 2016 Standard) with 3x VM's and generates the errors below. The backup job will run once after the Host server has been restarted but after that it keeps failing. When running the VSSADMIN LIST WRITER command, no errors are generated on the Hyper-V host or the clients. Please may this get your full attention as these are critical servers that need to be backed up. Error log found on backup Exec (Backup Server) Job Log: BEX_HEBS_08814.xml<noscript> 30 October 2018 11:20:01 AM - V-79-40960-38521 - Snapshot Technology: Initialization failure on: "VRTSRV::\\****.he.local\Hyper-V?Virtual?Machine\****". Snapshot technology used: Microsoft Volume Shadow Copy Service (VSS). A virtual machine snapshot could not be created. Use the Event Viewer on the Hyper-V host to identify the cause. If the virtual machine is not properly configured, it may be necessary to use the Event Viewer in the virtual machine to identify the cause. The following volumes are dependent on resource: "C:" . The snapshot technology used by VSS for volume C: - Microsoft Software Shadow Copy provider 1.0 (Version 1.0.0.7). Please see Acronis VSS Doctor report below (Errors were generated on Hyper-V Host server): 2018-10-30 11:23:23 GMT+02:00 AcronisVSSDoctor-net45, 1.0.39.0, v4.0.30319 Microsoft Windows NT 6.2.9200.0, 4.0.30319.42000 =========================================================== VSS Services Configuration and Status Status: OK Description: All VSS-related services are configured correctly Services: Name: swprv Title: Microsoft Software Shadow Copy Provider Status: Running ReferenceStatus: NeedToFixStatus: False StartMode: Manual ReferenceStartMode: Manual NeedToFixStartMode: False IsOk: True ExpectationDescription: Exception: Name: VSS Title: Volume Shadow Copy Status: Running ReferenceStatus: NeedToFixStatus: False StartMode: Manual ReferenceStartMode: Manual NeedToFixStartMode: False IsOk: True ExpectationDescription: Exception: Name: EventSystem Title: COM+ Event System Status: Running ReferenceStatus: Running NeedToFixStatus: False StartMode: Auto ReferenceStartMode: Auto NeedToFixStartMode: False IsOk: True ExpectationDescription: Exception: =========================================================== VSS Providers Configuration Status: OK Description: Only known VSS provider(s) found Providers: Id: 89300202-3cec-4981-9171-19f59559e0f2 Path: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\VSS\Providers\{89300202-3cec-4981-9171-19f59559e0f2} Name: Microsoft File Share Shadow Copy provider IsDefault: True Id: b5946137-7b9f-4925-af80-51abd60b20d5 Path: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\VSS\Providers\{b5946137-7b9f-4925-af80-51abd60b20d5} Name: Microsoft Software Shadow Copy provider 1.0 IsDefault: True =========================================================== VSS Access Control Configuration Status: OK Description: Known VSS access control entries are configured correctly AccessControls: UserName: NT Authority\NetworkService EntryExists: True UserExists: True IsActive: True IsOk: True StatusDescription: =========================================================== VSS Components Availability Status: OK Description: VSS components are accessible =========================================================== VSS Writers Status Status: OK Description: All VSS writers are working correctly Writers: Name: Task Scheduler Writer Id: {d61d61c8-d73a-4eee-8cdd-f6f9786b7124} State: [1] Stable IsOk: True Error: No error Name: VSS Metadata Store Writer Id: {75dfb225-e2e4-4d39-9ac9-ffaff65ddf06} State: [1] Stable IsOk: True Error: No error Name: Performance Counters Writer Id: {0bada1de-01a9-4625-8278-69e735f39dd2} State: [1] Stable IsOk: True Error: No error Name: System Writer Id: {e8132975-6f93-4464-a53e-1050253ae220} State: [1] Stable IsOk: True Error: No error Name: ASR Writer Id: {be000cbe-11fe-4426-9c58-531aa6355fc4} State: [1] Stable IsOk: True Error: No error Name: Shadow Copy Optimization Writer Id: {4dc3bdd4-ab48-4d07-adb0-3bee2926fd7f} State: [5] Waiting for completion IsOk: True Error: No error Name: FRS Writer Id: {d76f5a28-3092-4589-ba48-2958fb88ce29} State: [1] Stable IsOk: True Error: No error Name: WMI Writer Id: {a6ad56c2-b509-4e6c-bb19-49d8f43532f0} State: [1] Stable IsOk: True Error: No error Name: MSSearch Service Writer Id: {cd3f2362-8bef-46c7-9181-d62844cdc0b2} State: [5] Waiting for completion IsOk: True Error: No error Name: Registry Writer Id: {afbab4a2-367d-4d15-a586-71dbb18f8485} State: [1] Stable IsOk: True Error: No error Name: NTDS Id: {b2014c9e-8711-4c5c-a5a9-3cf384484757} State: [1] Stable IsOk: True Error: No error =========================================================== VSS Shadow Storages Status Status: OK Description: All VSS shadow storages are configured correctly Storages: ForVolume: C:\ OnVolume: C:\ Used: 96 MB Allocated: 50 GB Maximum: UNBOUNDED Minimum: 320 MB VolumeSize: 3721 GB Available: 3218 GB TargetMaximum: 372 GB MaximumGreaterThenMinimum: True MaximumIsEnough: True MaximumIsNotReached: True AvailableIsEnoughForMaximum: True IsOk: True Description: =========================================================== Disk Free Space Available Status: OK Description: All disks have enough free space Volumes: Name: C:\ DeviceId: \\?\Volume{1c4947d2-e53e-e711-a993-6679abe43d90}\ Size: 3721 GB Available: 3218 GB Minimum: 320 MB IsOk: True Description: IsMounted: True Name: E:\ DeviceId: \\?\Volume{0005f107-0000-0000-0000-100000000000}\ Size: 1862 GB Available: 1520 GB Minimum: 320 MB IsOk: True Description: IsMounted: True =========================================================== Disk I/O Load Status: OK Description: All disks are currently not overloaded Disks: Name: 0 C: DiskBytesPersec: 437574 DiskReadBytesPersec: 173144 DiskReadsPersec: 9 DiskTransfersPersec: 36 DiskWriteBytesPersec: 264429 DiskWritesPersec: 27 PercentDiskReadTime: 6 PercentDiskTime: 28 PercentDiskWriteTime: 23 PercentIdleTime: 74 IsOk: True Name: 1 E: DiskBytesPersec: 0 DiskReadBytesPersec: 0 DiskReadsPersec: 0 DiskTransfersPersec: 0 DiskWriteBytesPersec: 0 DiskWritesPersec: 0 PercentDiskReadTime: 0 PercentDiskTime: 0 PercentDiskWriteTime: 0 PercentIdleTime: 97 IsOk: True =========================================================== NTFS Volume Access Control Configuration Status: OK Description: All volumes have correct access control configuration Volumes: Name: C:\ SystemHasAccess: True SystemSid: S-1-5-18 IsOk: True Description: "SYSTEM" account has "Full Control" access Name: E:\ SystemHasAccess: True SystemSid: S-1-5-18 IsOk: True Description: "SYSTEM" account has "Full Control" access =========================================================== Event Log Status: Warn Description: Some VSS-related errors or warnings occurred during the previous day Events: Timestamp: 2018/10/30 10:31:50 AM Type: Warning Source: VSS Message: A VSS writer has rejected an event with error 0x800423f3, The writer experienced a transient error. If the backup process is retried, the error may not reoccur. . Changes that the writer made to the writer components while handling the event will not be available to the requester. Check the event log for related events from the application hosting the VSS writer. Operation: PrepareForSnapshot Event Context: Execution Context: Writer Writer Class Id: {66841cd4-6ded-4f4b-8f17-fd23f8ddc3de} Writer Name: Microsoft Hyper-V VSS Writer Writer Instance ID: {51b65980-2447-4878-9507-c4a25021a881} Command Line: C:\Windows\system32\vmms.exe Process ID: 4032 InstanceId: 8229 SearchLink: https://www.google.com/#newwindow=1&q=VSS+event+id+8229 Timestamp: 2018/10/30 10:18:48 AM Type: Warning Source: VSS Message: A VSS writer has rejected an event with error 0x800423f3, The writer experienced a transient error. If the backup process is retried, the error may not reoccur. . Changes that the writer made to the writer components while handling the event will not be available to the requester. Check the event log for related events from the application hosting the VSS writer. Operation: PrepareForSnapshot Event Context: Execution Context: Writer Writer Class Id: {66841cd4-6ded-4f4b-8f17-fd23f8ddc3de} Writer Name: Microsoft Hyper-V VSS Writer Writer Instance ID: {51b65980-2447-4878-9507-c4a25021a881} Command Line: C:\Windows\system32\vmms.exe Process ID: 4032 InstanceId: 8229 SearchLink: https://www.google.com/#newwindow=1&q=VSS+event+id+8229 Timestamp: 2018/10/29 11:04:06 PM Type: Warning Source: VSS Message: A VSS writer has rejected an event with error 0x800423f3, The writer experienced a transient error. If the backup process is retried, the error may not reoccur. . Changes that the writer made to the writer components while handling the event will not be available to the requester. Check the event log for related events from the application hosting the VSS writer. Operation: PrepareForSnapshot Event Context: Execution Context: Writer Writer Class Id: {66841cd4-6ded-4f4b-8f17-fd23f8ddc3de} Writer Name: Microsoft Hyper-V VSS Writer Writer Instance ID: {51b65980-2447-4878-9507-c4a25021a881} Command Line: C:\Windows\system32\vmms.exe Process ID: 4032 InstanceId: 8229 SearchLink: https://www.google.com/#newwindow=1&q=VSS+event+id+8229