cancel
Showing results for 
Search instead for 
Did you mean: 

Data inconsistency on Puredisk storage

Michele_Nicosia
Level 5

good morning all,

   yesterday i've found a warning in the logs:

 

### Reported problems

10/06/2013 12:21:30 V1 S:ph469702 C:? J:0 (U:0,0)

  Warning(0x8) Disk(0x8200) nbemm

  A data inconsistency has been detected and corrected automatically. Please

  contact support to investigate the root cause. (Search storaged.log on

  server PureDisk:ph469702 for inconsistency in database)

 

 

I do not see any issues, on the backup side at least, but this warning worry me a lot , maybe it could lead us any restore to a failure.

Attached the storaged.log, do i have to open a case, or can i do something on my side?

 

Thank you.

 

Kind Regards,

 

Michele

5 REPLIES 5

Michele_Nicosia
Level 5

Hi all,

  no one is aware of the issue, or simply i should open a case?

I would prefer understand before do any actions, because it looks like no impact on the system (at least which i'm aware of)

 

Thank you.

 

Regards,

 

Michele

arctics2
Level 5

I'm getting a similar issue on of our Appliances

"A  data inconsistency has been detected and corrected automatically."

Is repeated daily, but backups succeed. 

 

 

StefanosM
Level 6
Partner    VIP    Accredited Certified

If you want to check the hashes of the deduplication and be sure that you can restore your backups, you can run the following command

dcscan --verify -q -H -a > crcerrors.txt

the command location is at the pdde directory and it will calculate and check all hashes in your deduplication database. It will take time (a lot ) to finish and it is best (but not required ) not to run any backup during the test. You can stop the command any time.

The output of the command is the crcerrors.txt file but will print all errors to screen. If you need the errors to the crcerrors.txt file and not to the screen, run the command:
dcscan --verify -q -H -a > crcerrors.txt 2>&1

 

I suggest you to open a case. If you check the storage log, every day and the same time you have  the same error

October 07 12:24:29 WARNING [47462661798208]: 25002: processDoTRefCheck: Data loss was found. Could not find DO digest[f9f147ab275c07d1778b6f3f536f6e52] from CRDB
....
....
....
October 07 12:29:59 INFO [47462661798208]: Transaction log 4428249-4431931 Completed. Expect: 9820757 (661.70MB) Commit: 9820757 (75251.76MB) Retry: 0 Log: from /DSU/queue/partsorted-4428249-4431931-0.tlog to /DSU/queue/partsorted-4428249-4431931-5.tlog SO: Add 827502, Ref Add 6426641, Ref Add Fail: 0, Ref Del 755408 DO: Add 828, Ref Add 786, Ref Add Fail: 0, Ref Del 941 TASK: Add 851, End 851, End All 0, Del 719 DCID: SO 1803314, SO Fail 0, DO 1457, DO Fail 0 MARKER: 0, Fail 0

October 07 12:29:59 INFO [47462661798208]: Finished a time of Data loss checking, 1459 DOs and 0 SOs were checked, 472 DOs and 0 SOs were lost.

 

 

 

arctics2
Level 5

I opened a ticket with support and we worked through a few things including running crchk on the appliance, which made some repairs, but also pointed to a single corrupt image, which we then expired.  Based on the results of the crchk, it appeared as though our issues were resolved, however, the next morning, I'm seeing more data inconsistency errors in the problems log.  So while I'm working with support, I thought I'd check here as well to see if anyone had any input.

 

Thanks

arctics2
Level 5

When looking at the storaged.log, the following are some of the details:

 

January 31 00:43:43 WARNING [1080080704]: 25072: CheckTRef: The type of reference P2:01a3bbd77df519f0d8ba21a56990a02b is not found
January 31 00:43:43 WARNING [1080080704]: 25072: processDoTRefCheck: The reference [P2:01a3bbd77df519f0d8ba21a56990a02b] is not found in the DO[f3a558a33ec2e65689cdbb6cb2169a80]. It's a potential d
ata loss.
January 31 00:43:43 INFO [1080080704]: processDoTRefCheck: This potential data loss was fixed.
January 31 00:43:43 INFO [1080080704]: WSRequestExt: submitting &request=4&login=agent_3_266370&passwd=********************************&action=newevent&data=EVENT%7Bversion%3A2%3Btype%3A1%3Bid%3A0%
3Bdate%3A1391147023%3B%7BLEGACYEVENT%7Bpayload%3Asev%3D4%3Btype%3D1058%3Bmsg%3Dinconsistency%20in%20database%3B%7D%7D
January 31 00:43:43 WARNING [1080080704]: 25072: CheckTRef: The type of reference P2:01550c0bd75c6087fce6d3161e831922 is not found
January 31 00:43:43 WARNING [1080080704]: 25072: processDoTRefCheck: The reference [P2:01550c0bd75c6087fce6d3161e831922] is not found in the DO[f3a558a33ec2e65689cdbb6cb2169a80]. It's a potential d
ata loss.
January 31 00:43:43 INFO [1080080704]: processDoTRefCheck: This potential data loss was fixed.
January 31 00:43:43 INFO [1080080704]: WSRequestExt: submitting &request=4&login=agent_3_266370&passwd=********************************&action=newevent&data=EVENT%7Bversion%3A2%3Btype%3A1%3Bid%3A0%
3Bdate%3A1391147023%3B%7BLEGACYEVENT%7Bpayload%3Asev%3D4%3Btype%3D1058%3Bmsg%3Dinconsistency%20in%20database%3B%7D%7D
January 31 00:43:43 WARNING [1080080704]: 25072: CheckTRef: The type of reference P2:0110f35d0a7a50aaf5e3569b4393d817 is not found
January 31 00:43:43 WARNING [1080080704]: 25072: processDoTRefCheck: The reference [P2:0110f35d0a7a50aaf5e3569b4393d817] is not found in the DO[f3a558a33ec2e65689cdbb6cb2169a80]. It's a potential d
ata loss.
January 31 00:43:43 INFO [1080080704]: processDoTRefCheck: This potential data loss was fixed.
January 31 00:43:43 INFO [1080080704]: WSRequestExt: submitting &request=4&login=agent_3_266370&passwd=********************************&action=newevent&data=EVENT%7Bversion%3A2%3Btype%3A1%3Bid%3A0%
3Bdate%3A1391147023%3B%7BLEGACYEVENT%7Bpayload%3Asev%3D4%3Btype%3D1058%3Bmsg%3Dinconsistency%20in%20database%3B%7D%7D