10-07-2015 06:41 PM
Hi Team,
I had enabled accelerator for few windows file server backups.
Backup used to be success when server size was small however from last few weeks, these servers size increased.
Below is specification:
Server has 4 drives : out of which 2 drives size is about 2.8 TB, and 2 TB respectively.
Backup frequently fails with error 42 and 13 :
Error for code 13 : not using change journal data for <F:\>: unable to initialize change journal usage <reason=record for requested start usn has been dropped from the change database (too many changes)>
Error for code 42: ERR - Snapshot Error while enumerating: GLOBALROOT\Device\HarddiskVolumeShadowCopy105\Profiles\
Seems this file servers experience high rate of change.
Kindly let me know how can i avoid these failures.
Do i need to disable accelerator then perform backup.
10-07-2015 09:00 PM
Try with out "allow multiple datastreams" selected. so there will be only 1 job. Also, what the deduplication setting for the client, media server or client-side? Client-side sometimes causes status 13.
10-12-2015 12:53 AM
This looks like Windows 'Change Journal' issue and not really Acceleraror.
Please see if this section: “Accelerator and the Windows change journal” on page 694 of NetBackup Administrator's Guide, Volume I helps.
10-24-2015 12:01 AM
Hello Marianne,
Sorry for checking late, as i was out of offive for few days.
I tried backup without accelerator, though backup speed was fast, still received error with code 24/13 and backup failed.
Now a days, i am experienceing backkup failure for all file servers with drive size around TBs failing woth code 13 and 24.
Not able to figure out cause of issue as other file servers with less drives size is working fine.
On both the servers, i have installed Netbackup 7.6.0.1 client version and storage target is DataDomain.
Multistreaming is allowed.
Backup runs fine for some time a then after wrting around 1TB or more it fails it incomplete with code 13/24.
Kindly assist me.
10-24-2015 06:34 AM
It would seem to me that your file servers have a high rate of change. This is borne out by the NTFS Change Journal running out of room to track all file changes:
Error for code 13 : not using change journal data for <F:\>:
unable to initialize change journal usage
<reason=record for requested start usn has been dropped
from the change database (too many changes)>
Can I ask whether the NTFS volumes are quite full too?
If so, then there's probably not a lot you can do from NetBackup side. Before doing anythign else, I would suggest monitoring both free disk space and VSS disk space consumption and which volumes (drive letters) are being used by which other volumes (drive letters) as VSS delta snap space... on the nodes with failing backup... do this during the actual backups, because this may give you the evidence that you need to present to management to prove that you may need to look at providing some additional storage space, not for files, and not for allplcations - but purely sitting there unused (most of the time) to provide nothing other than delta snap space for VSS. i.e. you may need to give more space to VSS and to manually configure VSS so as not to leave VSS to make its own mind up about what it should do.
Remember: NetBackup uses bpfis to call VSS and if VSS fails, then it is most definitely not a NetBackup software issue per-se - in fact, it is a OS and/or server administration and/or storage administration issue. If VSS cannot do its job, then NetBackup Client cannot do its job.
.
Have you considered FlashBackup instead? FlashBackup is a backup method which is sometimes a useful alternative (to walking the file system) for volumes which are large and quite full and contain millions of folders and files.
10-29-2015 06:47 PM
Hi Sdo
While using accelerator i also received same error "
<reason=record for requested start usn has been dropped
from the change database (too many changes)>
i disabled "use change journal option" and tried backing up without accelerator, still seeing failure with code 13/42.
I could not understand troubleshooting steps given by you. kindly simplify.
10-29-2015 07:10 PM
10-30-2015 02:25 AM
Also, please show us output from the problem client:
wmic logicaldisk list brief vssadmin list providers vssadmin list volumes vssadmin list writers vssadmin list shadowstorage vssadmin list shadows
10-30-2015 02:34 AM
sure ..i will provide the output
one more behaviour , the clients are configured as cluster say client1A and client1B and are windows2012 servers
have configured both the clients in backup.
while backing up one client , its total size for one drive is 5 TB used space is 2 TB. however my backup stream for that drive is running more than3.5 TB dont know how..
have anyone came across this issue.
10-30-2015 02:50 AM
1) Is NTFS native de-duplication being used within the NTFS volumes?
see:
http://blogs.technet.com/b/filecab/archive/2012/05/21/introduction-to-data-deduplication-in-windows-server-2012.aspx
.
2) Is Enterprise Vault configured to archive the contents of the file system? Maybe something is missing from your server configuration which is causing stubbed/offline files to be recalled?
.
3) Or maybe you have a different archiving solution, which is causing files to be fully recalled/retrieved from the archive store, i.e. the 'offline-ness' or 'stubby-ness' of the stubbed files is being ignored when NetBackup walks down through the file system looking for files to backup.
e.g.:
https://www-secure.symantec.com/connect/forums/ignoring-stub-files
10-30-2015 05:26 AM
we have DataDomain system as target device.
We dont have archviing solution , its a standard file system backup.
10-30-2015 06:06 AM
1) To be clear, I wasn't asking about your de-dupe target. I was aksing whether you have a de-duped source. I'll restate the question: Is Windows NTFS native internal file-system de-duplication being used on the NTFS volumes/drive-letters?
10-30-2015 12:17 PM
Hello Sdo,
below is requested output:
C:\Windows\system32>fsutil usn queryjournal F:
Usn Journal ID : 0x01d0fc9fe1f625cc
First Usn : 0x000000243f4c0000
Next Usn : 0x000000245f4eb730
Lowest Valid Usn : 0x0000000000000000
Max Usn : 0x7fffffffffff0000
Maximum Size : 0x0000000020000000
Allocation Delta : 0x0000000000040000
Minimum record version supported : 2
Maximum record version supported : 4
Write range tracking: Disabled
C:\Windows\system32>wmic logicaldisk list brief
DeviceID DriveType FreeSpace ProviderName Size VolumeName
C: 3 388435955712 449694396416 System
E: 3 7337857327104 8589798273024 PDW203PAFS002-V
ol01
F: 3 2383314305024 4397910192128 PDW203PAFS004-V
ol01
G: 3 5477908062208 5497421819904 PDW203PAFS002-V
ol02
Q: 3 241692672 771616768 QuorumW
C:\Windows\system32>vssadmin list providers
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line t
(C) Copyright 2001-2013 Microsoft Corp.
Provider name: 'Microsoft CSV Shadow Copy Helper Provider'
Provider type: Software
Provider Id: {26d02d81-6aac-4275-8504-b9c6edc5261d}
Version: 1.0.0.1
Provider name: 'Microsoft iSCSI Target VSS Hardware Provider'
Provider type: Hardware
Provider Id: {3f900f90-00e9-440e-873a-96ca5eb079e5}
Version: 6.3.9600.16384
Provider name: 'Microsoft CSV Shadow Copy Provider'
Provider type: Software
Provider Id: {400a2ff4-5eb1-44b0-8a05-1fcac0bcf9ff}
Version: 1.0.0.1
Provider name: 'Microsoft File Share Shadow Copy provider'
Provider type: Fileshare
Provider Id: {89300202-3cec-4981-9171-19f59559e0f2}
Version: 1.0.0.1
Provider name: 'Microsoft Software Shadow Copy provider 1.0'
Provider type: System
Provider Id: {b5946137-7b9f-4925-af80-51abd60b20d5}
Version: 1.0.0.7
C:\Windows\system32>vssadmin list volumes
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2013 Microsoft Corp.
Volume path: \\?\Volume{0425fd85-79b8-11e4-80e5-806e6f6e6963}\
Volume name: \\?\Volume{0425fd85-79b8-11e4-80e5-806e6f6e6963}\
Volume path: E:\
Volume name: \\?\Volume{9e366a4b-2674-4bd4-ba1a-c4eb96cb9839}\
Volume path: Q:\
Volume name: \\?\Volume{a201ae40-434a-456f-bf5a-48c24bfd308e}\
Volume path: G:\
Volume name: \\?\Volume{fbca6175-df00-4ba9-8e99-58ac71cd16ac}\
Volume path: F:\
Volume name: \\?\Volume{72ba8ebd-03c9-4b7f-b132-c5b1e09ce0cd}\
Volume path: C:\
Volume name: \\?\Volume{0425fd86-79b8-11e4-80e5-806e6f6e6963}\
C:\Windows\system32>vssadmin list shadowstorage
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2013 Microsoft Corp.
Shadow Copy Storage association
For volume: (E:)\\?\Volume{9e366a4b-2674-4bd4-ba1a-c4eb96cb9839}\
Shadow Copy Storage volume: (E:)\\?\Volume{9e366a4b-2674-4bd4-ba1a-c4eb96cb98
39}\
Used Shadow Copy Storage space: 15.9 GB (0%)
Allocated Shadow Copy Storage space: 18.8 GB (0%)
Maximum Shadow Copy Storage space: 500 GB (6%)
Shadow Copy Storage association
For volume: (Q:)\\?\Volume{a201ae40-434a-456f-bf5a-48c24bfd308e}\
Shadow Copy Storage volume: (Q:)\\?\Volume{a201ae40-434a-456f-bf5a-48c24bfd30
8e}\
Used Shadow Copy Storage space: 24.0 MB (3%)
Allocated Shadow Copy Storage space: 87.4 MB (11%)
Maximum Shadow Copy Storage space: UNBOUNDED (100%)
Shadow Copy Storage association
For volume: (G:)\\?\Volume{fbca6175-df00-4ba9-8e99-58ac71cd16ac}\
Shadow Copy Storage volume: (G:)\\?\Volume{fbca6175-df00-4ba9-8e99-58ac71cd16
ac}\
Used Shadow Copy Storage space: 13.2 GB (0%)
Allocated Shadow Copy Storage space: 16.1 GB (0%)
Maximum Shadow Copy Storage space: 512 GB (10%)
Shadow Copy Storage association
For volume: (F:)\\?\Volume{72ba8ebd-03c9-4b7f-b132-c5b1e09ce0cd}\
Shadow Copy Storage volume: (F:)\\?\Volume{72ba8ebd-03c9-4b7f-b132-c5b1e09ce0
cd}\
Used Shadow Copy Storage space: 221 GB (5%)
Allocated Shadow Copy Storage space: 223 GB (5%)
Maximum Shadow Copy Storage space: 750 GB (18%)
Shadow Copy Storage association
For volume: (C:)\\?\Volume{0425fd86-79b8-11e4-80e5-806e6f6e6963}\
Shadow Copy Storage volume: (C:)\\?\Volume{0425fd86-79b8-11e4-80e5-806e6f6e69
63}\
Used Shadow Copy Storage space: 5.43 GB (1%)
Allocated Shadow Copy Storage space: 8.25 GB (1%)
Maximum Shadow Copy Storage space: UNBOUNDED (4102062249%)
C:\Windows\system32>vssadmin list shadows
Original Volume: (E:)\\?\Volume{9e366a4b-2674-4bd4-ba1a-c4eb96cb9839}\
Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy688
Originating Machine: PDW203PAFS001B.ca.bestbuy.com
Service Machine: PDW203PAFS001B.ca.bestbuy.com
Provider: 'Microsoft Software Shadow Copy provider 1.0'
Type: ClientAccessible
Attributes: Persistent, Client-accessible, No auto release, No writers,
Differential
Contents of shadow copy set ID: {83a4671a-b4d7-4e5c-82ee-48c174971e66}
Contained 1 shadow copies at creation time: 10/23/2015 7:00:13 AM
Shadow Copy ID: {fff7f210-0099-4b58-a17a-e1fee9ab2e66}
Original Volume: (E:)\\?\Volume{9e366a4b-2674-4bd4-ba1a-c4eb96cb9839}\
Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy708
Originating Machine: PDW203PAFS001B.ca.bestbuy.com
Service Machine: PDW203PAFS001B.ca.bestbuy.com
Provider: 'Microsoft Software Shadow Copy provider 1.0'
Type: ClientAccessible
Attributes: Persistent, Client-accessible, No auto release, No writers,
Differential
Contents of shadow copy set ID: {3a458d64-583e-4180-9eae-4deb5e70ccfa}
Contained 1 shadow copies at creation time: 10/23/2015 12:00:12 PM
Shadow Copy ID: {0cf2f7ad-1b03-4b3e-a656-7a0c0295c3a7}
Original Volume: (E:)\\?\Volume{9e366a4b-2674-4bd4-ba1a-c4eb96cb9839}\
Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy743
Originating Machine: PDW203PAFS001B.ca.bestbuy.com
Service Machine: PDW203PAFS001B.ca.bestbuy.com
Provider: 'Microsoft Software Shadow Copy provider 1.0'
Type: ClientAccessible
Attributes: Persistent, Client-accessible, No auto release, No writers,
Differential
Contents of shadow copy set ID: {9dd5fcbf-9545-4752-a175-4b842ec30c76}
Contained 5 shadow copies at creation time: 10/24/2015 10:53:31 PM
Shadow Copy ID: {af5b2d92-a68c-45b9-81da-fc2a064e8cc0}
Original Volume: (E:)\\?\Volume{9e366a4b-2674-4bd4-ba1a-c4eb96cb9839}\
Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy780
Originating Machine: PDW203PAFS001B.ca.bestbuy.com
Service Machine: PDW203PAFS001B.ca.bestbuy.com
Provider: 'Microsoft Software Shadow Copy provider 1.0'
Type: ApplicationRollback
Attributes: Persistent, No auto release, Differential
Shadow Copy ID: {d42cec66-27a3-42c1-aa11-d606e817514c}
Original Volume: (Q:)\\?\Volume{a201ae40-434a-456f-bf5a-48c24bfd308e}\
Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy777
Originating Machine: PDW203PAFS001B.ca.bestbuy.com
Service Machine: PDW203PAFS001B.ca.bestbuy.com
Provider: 'Microsoft Software Shadow Copy provider 1.0'
Type: ApplicationRollback
Attributes: Persistent, No auto release, Differential
Shadow Copy ID: {c67571c0-23ad-4d72-91be-70da34a55166}
Original Volume: (G:)\\?\Volume{fbca6175-df00-4ba9-8e99-58ac71cd16ac}\
Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy778
Originating Machine: PDW203PAFS001B.ca.bestbuy.com
Service Machine: PDW203PAFS001B.ca.bestbuy.com
Provider: 'Microsoft Software Shadow Copy provider 1.0'
Type: ApplicationRollback
Attributes: Persistent, No auto release, Differential
Shadow Copy ID: {668a14fc-ed5b-4c69-ab70-8aa65a624e32}
Original Volume: (F:)\\?\Volume{72ba8ebd-03c9-4b7f-b132-c5b1e09ce0cd}\
Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy779
Originating Machine: PDW203PAFS001B.ca.bestbuy.com
Service Machine: PDW203PAFS001B.ca.bestbuy.com
Provider: 'Microsoft Software Shadow Copy provider 1.0'
Type: ApplicationRollback
Attributes: Persistent, No auto release, Differential
Shadow Copy ID: {86de38c4-185a-414a-87c6-2ea0de550549}
Original Volume: (C:)\\?\Volume{0425fd86-79b8-11e4-80e5-806e6f6e6963}\
Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy776
Originating Machine: PDW203PAFS001B.ca.bestbuy.com
Service Machine: PDW203PAFS001B.ca.bestbuy.com
Provider: 'Microsoft Software Shadow Copy provider 1.0'
Type: ApplicationRollback
Attributes: Persistent, No auto release, Differential
Contents of shadow copy set ID: {48a8e552-4879-466c-886c-62cc25da5f2d}
Contained 1 shadow copies at creation time: 10/26/2015 7:00:11 AM
Shadow Copy ID: {9df667a5-6b05-4b91-951a-79b8585a61f8}
Original Volume: (E:)\\?\Volume{9e366a4b-2674-4bd4-ba1a-c4eb96cb9839}\
Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy813
Originating Machine: PDW203PAFS001B.ca.bestbuy.com
Service Machine: PDW203PAFS001B.ca.bestbuy.com
Provider: 'Microsoft Software Shadow Copy provider 1.0'
Type: ClientAccessible
Attributes: Persistent, Client-accessible, No auto release, No writers,
Differential
Contents of shadow copy set ID: {f6958a24-55e8-4dc9-88f8-488b7954d8c4}
Contained 1 shadow copies at creation time: 10/26/2015 12:00:11 PM
Shadow Copy ID: {f885326f-393e-4565-91ad-52134e01f899}
Original Volume: (E:)\\?\Volume{9e366a4b-2674-4bd4-ba1a-c4eb96cb9839}\
Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy815
Originating Machine: PDW203PAFS001B.ca.bestbuy.com
Service Machine: PDW203PAFS001B.ca.bestbuy.com
Provider: 'Microsoft Software Shadow Copy provider 1.0'
Type: ClientAccessible
Attributes: Persistent, Client-accessible, No auto release, No writers,
Differential
Contents of shadow copy set ID: {6a6174e8-4147-4c7b-b848-082193696334}
Contained 1 shadow copies at creation time: 10/27/2015 7:00:12 AM
Shadow Copy ID: {5a3a1c75-151f-461e-b89a-135ea6475366}
Original Volume: (E:)\\?\Volume{9e366a4b-2674-4bd4-ba1a-c4eb96cb9839}\
Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy841
Originating Machine: PDW203PAFS001B.ca.bestbuy.com
Service Machine: PDW203PAFS001B.ca.bestbuy.com
Provider: 'Microsoft Software Shadow Copy provider 1.0'
Type: ClientAccessible
Attributes: Persistent, Client-accessible, No auto release, No writers,
Differential
Contents of shadow copy set ID: {d36d74bf-e9e2-4b03-965d-c43d7e8477e8}
Contained 1 shadow copies at creation time: 10/27/2015 12:00:56 PM
Shadow Copy ID: {b039443a-b3d4-48f1-af39-71ac51126aec}
Original Volume: (E:)\\?\Volume{9e366a4b-2674-4bd4-ba1a-c4eb96cb9839}\
Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy844
Originating Machine: PDW203PAFS001B.ca.bestbuy.com
Service Machine: PDW203PAFS001B.ca.bestbuy.com
Provider: 'Microsoft Software Shadow Copy provider 1.0'
Type: ClientAccessible
Attributes: Persistent, Client-accessible, No auto release, No writers,
Differential
Contents of shadow copy set ID: {8e55400f-588d-4d3b-8274-b053c72456c4}
Contained 1 shadow copies at creation time: 10/28/2015 7:01:07 AM
Shadow Copy ID: {c4f07ce7-ba93-46fd-b4a5-ab999e4b8b50}
Original Volume: (E:)\\?\Volume{9e366a4b-2674-4bd4-ba1a-c4eb96cb9839}\
Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy869
Originating Machine: PDW203PAFS001B.ca.bestbuy.com
Service Machine: PDW203PAFS001B.ca.bestbuy.com
Provider: 'Microsoft Software Shadow Copy provider 1.0'
Type: ClientAccessible
Attributes: Persistent, Client-accessible, No auto release, No writers,
Differential
Contents of shadow copy set ID: {e339aa74-3722-4ff0-9930-4f31964adda7}
Contained 1 shadow copies at creation time: 10/28/2015 12:00:12 PM
Shadow Copy ID: {6bae1004-a411-44e3-b839-055c8c31acf0}
Original Volume: (E:)\\?\Volume{9e366a4b-2674-4bd4-ba1a-c4eb96cb9839}\
Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy870
Originating Machine: PDW203PAFS001B.ca.bestbuy.com
Service Machine: PDW203PAFS001B.ca.bestbuy.com
Provider: 'Microsoft Software Shadow Copy provider 1.0'
Type: ClientAccessible
Attributes: Persistent, Client-accessible, No auto release, No writers,
Differential
Contents of shadow copy set ID: {33c751a7-6d88-4073-970d-5c700c5babb7}
Contained 1 shadow copies at creation time: 10/29/2015 7:00:58 AM
Shadow Copy ID: {3e67c370-f55f-424f-8b69-44e619b85640}
Original Volume: (E:)\\?\Volume{9e366a4b-2674-4bd4-ba1a-c4eb96cb9839}\
Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy891
Originating Machine: PDW203PAFS001B.ca.bestbuy.com
Service Machine: PDW203PAFS001B.ca.bestbuy.com
Provider: 'Microsoft Software Shadow Copy provider 1.0'
Type: ClientAccessible
Attributes: Persistent, Client-accessible, No auto release, No writers,
Differential
Contents of shadow copy set ID: {42d19586-45a0-4d42-b81d-cc5b2bed190f}
Contained 1 shadow copies at creation time: 10/29/2015 12:00:12 PM
Shadow Copy ID: {42dc1d58-63d8-4fc1-8656-186a67c2218d}
Original Volume: (E:)\\?\Volume{9e366a4b-2674-4bd4-ba1a-c4eb96cb9839}\
Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy892
Originating Machine: PDW203PAFS001B.ca.bestbuy.com
Service Machine: PDW203PAFS001B.ca.bestbuy.com
Provider: 'Microsoft Software Shadow Copy provider 1.0'
Type: ClientAccessible
Attributes: Persistent, Client-accessible, No auto release, No writers,
Differential
Contents of shadow copy set ID: {504e3f06-f462-489d-a55f-33a4c73b6cf7}
Contained 6 shadow copies at creation time: 9/25/2015 9:01:51 PM
Shadow Copy ID: {c9309c30-4eb4-4f8d-9d17-448398e2f9e2}
Original Volume: (G:)\\?\Volume{fbca6175-df00-4ba9-8e99-58ac71cd16ac}\
Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy103
Originating Machine: PDW203PAFS001B.ca.bestbuy.com
Service Machine: PDW203PAFS001B.ca.bestbuy.com
Provider: 'Microsoft Software Shadow Copy provider 1.0'
Type: ApplicationRollback
Attributes: Persistent, No auto release, Differential
Contents of shadow copy set ID: {9aaf28af-aa33-48ea-b9cd-21d922add4b0}
Contained 5 shadow copies at creation time: 9/29/2015 4:32:18 AM
Shadow Copy ID: {4ae9d35e-7cf2-43ff-846f-3e67dd4f0f87}
Original Volume: (G:)\\?\Volume{fbca6175-df00-4ba9-8e99-58ac71cd16ac}\
Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy116
Originating Machine: PDW203PAFS001A.ca.bestbuy.com
Service Machine: PDW203PAFS001A.ca.bestbuy.com
Provider: 'Microsoft Software Shadow Copy provider 1.0'
Type: ApplicationRollback
Attributes: Persistent, No auto release, Differential
Contents of shadow copy set ID: {aab6b3a0-7eba-44da-bfa3-ebc58a894c29}
Contained 1 shadow copies at creation time: 10/26/2015 7:00:37 AM
Shadow Copy ID: {926e618e-74ad-4c47-8ab9-f49191f97f1c}
Original Volume: (F:)\\?\Volume{72ba8ebd-03c9-4b7f-b132-c5b1e09ce0cd}\
Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy814
Originating Machine: PDW203PAFS001B.ca.bestbuy.com
Service Machine: PDW203PAFS001B.ca.bestbuy.com
Provider: 'Microsoft Software Shadow Copy provider 1.0'
Type: ClientAccessible
Attributes: Persistent, Client-accessible, No auto release, No writers,
Differential
Contents of shadow copy set ID: {20464ab3-3498-4753-9dae-9fbfca8582ae}
Contained 1 shadow copies at creation time: 10/26/2015 12:00:34 PM
Shadow Copy ID: {74eaa624-71a9-4f34-a7e0-918e611b3e4f}
Original Volume: (F:)\\?\Volume{72ba8ebd-03c9-4b7f-b132-c5b1e09ce0cd}\
Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy816
Originating Machine: PDW203PAFS001B.ca.bestbuy.com
Service Machine: PDW203PAFS001B.ca.bestbuy.com
Provider: 'Microsoft Software Shadow Copy provider 1.0'
Type: ClientAccessible
Attributes: Persistent, Client-accessible, No auto release, No writers,
Differential
Contents of shadow copy set ID: {bc444d19-a4ae-4db9-81da-2ff36751a779}
Contained 1 shadow copies at creation time: 10/27/2015 7:00:35 AM
Shadow Copy ID: {aeb3a3df-4c3e-4a74-905e-652e2ab5d084}
Original Volume: (F:)\\?\Volume{72ba8ebd-03c9-4b7f-b132-c5b1e09ce0cd}\
Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy842
Originating Machine: PDW203PAFS001B.ca.bestbuy.com
Service Machine: PDW203PAFS001B.ca.bestbuy.com
Provider: 'Microsoft Software Shadow Copy provider 1.0'
Type: ClientAccessible
Attributes: Persistent, Client-accessible, No auto release, No writers,
Differential
Contents of shadow copy set ID: {57552ba6-0e5c-4e53-a7c6-418fd6c04850}
Contained 1 shadow copies at creation time: 10/27/2015 12:00:06 PM
Shadow Copy ID: {674a5014-90e9-4516-9eb2-3f78bae886e9}
Original Volume: (F:)\\?\Volume{72ba8ebd-03c9-4b7f-b132-c5b1e09ce0cd}\
Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy843
Originating Machine: PDW203PAFS001B.ca.bestbuy.com
Service Machine: PDW203PAFS001B.ca.bestbuy.com
Provider: 'Microsoft Software Shadow Copy provider 1.0'
Type: ClientAccessible
Attributes: Persistent, Client-accessible, No auto release, No writers,
Differential
Contents of shadow copy set ID: {eb515261-d819-4e00-a2b0-f7c5070e8913}
Contained 1 shadow copies at creation time: 10/28/2015 7:00:13 AM
Shadow Copy ID: {65d2d62d-4cb6-43c3-a6e8-1ea26d23fe41}
Original Volume: (F:)\\?\Volume{72ba8ebd-03c9-4b7f-b132-c5b1e09ce0cd}\
Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy868
Originating Machine: PDW203PAFS001B.ca.bestbuy.com
Service Machine: PDW203PAFS001B.ca.bestbuy.com
Provider: 'Microsoft Software Shadow Copy provider 1.0'
Type: ClientAccessible
Attributes: Persistent, Client-accessible, No auto release, No writers,
Differential
Contents of shadow copy set ID: {f3bb0b77-b473-45e5-97f4-82b84a651f0c}
Contained 1 shadow copies at creation time: 10/28/2015 12:00:41 PM
Shadow Copy ID: {98112b68-efa4-4897-94a6-202f7f227228}
Original Volume: (F:)\\?\Volume{72ba8ebd-03c9-4b7f-b132-c5b1e09ce0cd}\
Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy871
Originating Machine: PDW203PAFS001B.ca.bestbuy.com
Service Machine: PDW203PAFS001B.ca.bestbuy.com
Provider: 'Microsoft Software Shadow Copy provider 1.0'
Type: ClientAccessible
Attributes: Persistent, Client-accessible, No auto release, No writers,
Differential
Contents of shadow copy set ID: {f97c9d17-765c-472d-9187-b746adfecaa5}
Contained 1 shadow copies at creation time: 10/29/2015 7:00:09 AM
Shadow Copy ID: {df1a7346-7a88-464d-8306-b81aec9a76e4}
Original Volume: (F:)\\?\Volume{72ba8ebd-03c9-4b7f-b132-c5b1e09ce0cd}\
Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy890
Originating Machine: PDW203PAFS001B.ca.bestbuy.com
Service Machine: PDW203PAFS001B.ca.bestbuy.com
Provider: 'Microsoft Software Shadow Copy provider 1.0'
Type: ClientAccessible
Attributes: Persistent, Client-accessible, No auto release, No writers,
Differential
Contents of shadow copy set ID: {2eface3b-36b0-4642-bb6b-4865c37246c8}
Contained 1 shadow copies at creation time: 10/29/2015 12:00:44 PM
Shadow Copy ID: {8df573ee-2d76-4140-b710-8ce5584d7e6f}
Original Volume: (F:)\\?\Volume{72ba8ebd-03c9-4b7f-b132-c5b1e09ce0cd}\
Shadow Copy Volume: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy893
Originating Machine: PDW203PAFS001B.ca.bestbuy.com
Service Machine: PDW203PAFS001B.ca.bestbuy.com
Provider: 'Microsoft Software Shadow Copy provider 1.0'
Type: ClientAccessible
Attributes: Persistent, Client-accessible, No auto release, No writers,
Differential
C:\Windows\system32>.
'.' is not recognized as an internal or external command,
operable program or batch file.
10-30-2015 01:06 PM
My initial reaction is that this is a fairly complex situation:
- Clustered, using iSCSI.
- There appears to be some 'server admin' or 'application/service admins' scheduled VSS snapshots in place.
- Oddly the Quorum drive also has VSS configured - and I wouldn't expect to see this.
- The NetBackup Client at 7.6.0.1 is not good.
- The actual VSS output above is messed up - clearly you have not pasted everything, or you have pasted in between other text - thus it is unreliable.
- The USN of the F: drive is clearly wrapping:
First Usn : 0x000000243f4c0000 Next Usn : 0x000000245f4eb730 Difference: : 0x000000002002b730 Maximum Size : 0x0000000020000000
.
1) Were the vssadmin commands run whilst the backup was running?
2) Why is the NetBackup Client at v7.6.0.1 and not v7.6.0.4?
.
I think you're going to need to open a Veritas Support case to get to the bottom of this one.
.
Somethings to ask internally, i.e. discuss with your server admins - you shouldn't need to discuss these with Veritas Support - but finding out the answers to these will help you understand the nature and usage of the data:
- If you've disabled Accelerator and no longer want to use this feature... then how to disable MS Change Journal tracking? (..unless perhaps NTFS Change Journal is required for another reason.)
- Can you upgrade the NetBackup Client to v7.6.0.4?
- How many resources groups exist on the cluster? Are all of E: F: G: all in the same resource group?
- This probably isn't really relevant, but might have some clue as the server's intended behaviour... why is the C: drive so large?
- Why are the VSS snapshots of the G: drive timed differently to those of the E: and F: drives?
- Each of the large drives are using themselves for their shadow space. This can lead to horrible performance and timeouts. Why not rotate? i.e. E use F, F use G, G use E? Or if the volume of data and churn/change rate is going to increase notably, then perhaps look at providing additional LUNs for volumes to act purely as VSS snap space - e.g. add four additional LUNs or volumes (I,J,K,L), and configure VSS to use I for C, and J for E, and K for F, and L for G - to both ease IO load and spread IO load both during normal use and during backups.
10-30-2015 01:13 PM
You can expand the size of the change journal, see here:
https://technet.microsoft.com/en-gb/library/cc788042.aspx
.
Just out of curiosity (and it may help others):
3) What OS is the problem client running?
4) Which drive is NetBackup Client installed upon?
5) What is the client buffer size? (found in: client properties, windows client, client settings, buffer size)
.
But if it was me, I'd open an official support case.
10-31-2015 11:41 AM
Hello sdo,
Even sysadmins in our team are not able to figure out what causing failure as the server is not namaged by them and they are not supposed to access any drive as it is used to store users data.
Servers are running on - Windows 2012
Two servers are there say client1A and client1B and are configured as cluster.
Netbackup version 7.6.0.1 is installed
Both the servers have : Communication buffer size = 128 kb / Raw partition read buffer size - 32 kb
I tried backup even withput accelerator , it is failing.
I have logged a case with symantec support.
11-01-2015 03:16 PM
On both nodes say client1A and client1B, in event viewer, i could see below VSS error:
Error on Cleint1A
Volume Shadow Copy Service error: Unexpected error querying for the IVssWriterCallback interface. hr = 0x80070005, Access is denied.
. This is often caused by incorrect security settings in either the writer or requestor process.
Operation:
Gathering Writer Data
Context:
Writer Class Id: {d46bf321-fdba-4a35-8ec3-454df03bc86a}
Writer Name: Sync Share Service Writer
Writer Instance ID: {07bd3903-3982-4e3a-a1d6-8a8df0455dbf}
Error on Client1B:
Volume Shadow Copy Service error: Unexpected error querying for the IVssWriterCallback interface. hr = 0x80070005, Access is denied.
. This is often caused by incorrect security settings in either the writer or requestor process.
Operation:
Gathering Writer Data
Context:
Writer Class Id: {d46bf321-fdba-4a35-8ec3-454df03bc86a}
Writer Name: Sync Share Service Writer
Writer Instance ID: {53f6f316-8bb9-4c06-8fa1-d1750ed50f53}
Also i had checked with server owner regarding configuration:
The cluster name is Client1 being hosted on servers Cleint1A and Client2B. There are 4 cluster resources on the cluster.
11-24-2015 12:52 PM
Hello Sdo,
One query, does successfull backup delete a VSS snapshot.