November 4, 2012 at 11:23 am
The night of oct-30 we upgraded our production A/P cluster from 2k8 sp3 to 2012 on new HW.
We shutdown sql, moved the luns to the new HW, and attached all the DB's. The attach/upgrade went smoothly, with no errors.
The next CHECKDB found some corruption in our 2.0 TB archival DB.
Full output is in attachment.
DateTime: 2012-11-03 04:06:09
Command: DBCC CHECKDB ([GB_PurgedData]) WITH NO_INFOMSGS, ALL_ERRORMSGS, DATA_PURITY
Msg 8909, Level 16, State 1, Server SQL2VCLUSTER, Line 1
Table error: Object ID 0, index ID -1, partition ID 0, alloc unit ID 0 (type Unknown), page ID (1:156655154) contains an incorrect page ID in its page header. The PageId in the page header = (0:0).
Msg 8909, Level 16, State 1, Server SQL2VCLUSTER, Line 1
Table error: Object ID 0, index ID -1, partition ID 0, alloc unit ID 0 (type Unknown), page ID (1:156655155) contains an incorrect page ID in its page header. The PageId in the page header = (0:0).
Msg 8939, Level 16, State 98, Server SQL2VCLUSTER, Line 1
Table error: Object ID 1605580758, index ID 0, partition ID 72057594040418304, alloc unit ID 71881342401773568 (type LOB data), page (1:156655153). Test (IS_OFF (BUF_IOERR, pBUF->bstat)) failed. Values are 133129 and -1.
Msg 8928, Level 16, State 1, Server SQL2VCLUSTER, Line 1
Object ID 1605580758, index ID 0, partition ID 72057594040418304, alloc unit ID 71881342401773568 (type LOB data): Page (1:156655153) could not be processed. See other errors for details.
Msg 8928, Level 16, State 1, Server SQL2VCLUSTER, Line 1
Object ID 1605580758, index ID 0, partition ID 72057594040418304, alloc unit ID 71881342401773568 (type LOB data): Page (1:156655154) could not be processed. See other errors for details.
Msg 8928, Level 16, State 1, Server SQL2VCLUSTER, Line 1
Object ID 1605580758, index ID 0, partition ID 72057594040418304, alloc unit ID 71881342401773568 (type LOB data): Page (1:156655155) could not be processed. See other errors for details.
Msg 8965, Level 16, State 1, Server SQL2VCLUSTER, Line 1
Table error: Object ID 1605580758, index ID 0, partition ID 72057594040418304, alloc unit ID 71881342401773568 (type LOB data). The off-row data node at page (1:156655154), slot 0, text ID 1867186176 is referenced by page (1:146589647), slot 7, but was not seen in the scan.
Msg 8965, Level 16, State 1, Server SQL2VCLUSTER, Line 1
Table error: Object ID 1605580758, index ID 0, partition ID 72057594040418304, alloc unit ID 71881342401773568 (type LOB data). The off-row data node at page (1:156655154), slot 1, text ID 1617690624 is referenced by page (1:146589647), slot 8, but was not seen in the scan.
Msg 8965, Level 16, State 1, Server SQL2VCLUSTER, Line 1
Table error: Object ID 1605580758, index ID 0, partition ID 72057594040418304, alloc unit ID 71881342401773568 (type LOB data). The off-row data node at page (1:156655153), slot 0, text ID 1867186176 is referenced by page (1:146589647), slot 7, but was not seen in the scan.
Msg 8965, Level 16, State 1, Server SQL2VCLUSTER, Line 1
Table error: Object ID 1605580758, index ID 0, partition ID 72057594040418304, alloc unit ID 71881342401773568 (type LOB data). The off-row data node at page (1:156655155), slot 0, text ID 1617690624 is referenced by page (1:146589647), slot 8, but was not seen in the scan.
Msg 8929, Level 16, State 1, Server SQL2VCLUSTER, Line 1
Object ID 1605580758, index ID 0, partition ID 72057594040418304, alloc unit ID 72057594043236352 (type In-row data): Errors found in off-row data with ID 1867186176 owned by data record identified by RID = (1:20493:10)
Msg 8929, Level 16, State 1, Server SQL2VCLUSTER, Line 1
Object ID 1605580758, index ID 0, partition ID 72057594040418304, alloc unit ID 72057594043236352 (type In-row data): Errors found in off-row data with ID 1867186176 owned by data record identified by RID = (1:9595580:23)
Msg 8929, Level 16, State 1, Server SQL2VCLUSTER, Line 1
<<several thousand similiar rows omitted...>>
CHECKDB found 0 allocation errors and 2 consistency errors not associated with any single object.
CHECKDB found 0 allocation errors and 6916 consistency errors in table 'PurgedItems_old' (object ID 1605580758).
CHECKDB found 0 allocation errors and 6918 consistency errors in database 'GB_PurgedData'.
repair_allow_data_loss is the minimum repair level for the errors found by DBCC CHECKDB (GB_PurgedData).
Outcome: Failed
Duration: 04:12:50
DateTime: 2012-11-03 08:18:59
The referenced table [PurgedItems_old] is a 1.5TB 300 mil row heap. (No criticism please -- I inherited it. New data goes into a partitioned table with a clustered index. I've been slowly moving data. )
Timeline (I have all these backups):
10/24 -- FULL Backup with checksum, restore verifyonly successful
10/27 -- DBCC CHECKDB completes clean with NO_INFOMSGS, ALL_ERRORMSGS, DATA_PURITY
10/30 -- DIFF Backup with checksum, restore verifyonly successful
10/30 -- DIFF Backup with checksum, restore verifyonly successful -- right before upgrade
10/30 -- Upgrade to SQL 2012
11/1 -- DIFF Backup with checksum, restore verifyonly successful
11/2 -- DIFF Backup with checksum, restore verifyonly successful
11/3 -- DIFF Backup with checksum, restore verifyonly successful
11/3 -- DBCC CHECKDB reports errors
11/4 -- DIFF Backup with checksum, restore verifyonly successful
I don't have disk space to restore another copy of the DB. I think that page restore of page ID (1:156655154) and
page ID (1:156655155) is my next step, but which backup to use? That table shouldn't have changed since the last FULL.
both pages seem to have been zero'd out...
DBCC PAGE (Gb_purgeddata, 1, 156655154, 2) returns:
PAGE: (0:0)
BUFFER:
BUF @0x0000008195083000
bpage = 0x0000002196574000 bhash = 0x0000000000000000 bpageno = (1:156655154)
bdbid = 16 breferences = 0 bcputicks = 0
bsampleCount = 0 bUse1 = 64433 bstat = 0x809
blog = 0x2121215a bnext = 0x0000000000000000
PAGE HEADER:
Page @0x0000002196574000
m_pageId = (0:0) m_headerVersion = 0 m_type = 0
m_typeFlagBits = 0x0 m_level = 0 m_flagBits = 0x0
m_objId (AllocUnitId.idObj) = 0 m_indexId (AllocUnitId.idInd) = 0 Metadata: AllocUnitId = 0
Metadata: PartitionId = 0 Metadata: IndexId = -1 Metadata: ObjectId = 0
m_prevPage = (0:0) m_nextPage = (0:0) pminlen = 0
m_slotCnt = 0 m_freeCnt = 0 m_freeData = 0
m_reservedCnt = 0 m_lsn = (0:0:0) m_xactReserved = 0
m_xdesId = (0:0) m_ghostRecCnt = 0 m_tornBits = 0
DB Frag ID = 1
Allocation Status
GAM (1:156436992) = ALLOCATED SGAM (1:156436993) = NOT ALLOCATED
PFS (1:156648384) = 0x42 ALLOCATED 80_PCT_FULL DIFF (1:156436998) = NOT CHANGED
ML (1:156436999) = NOT MIN_LOGGED
DATA:
Memory Dump @0x000000B6C24A8000
000000B6C24A8000: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8014: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8028: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A803C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8050: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8064: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8078: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A808C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A80A0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A80B4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A80C8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A80DC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A80F0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8104: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8118: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A812C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8140: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8154: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8168: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A817C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8190: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A81A4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A81B8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A81CC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A81E0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A81F4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8208: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A821C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8230: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8244: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8258: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A826C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8280: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8294: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A82A8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A82BC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A82D0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A82E4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A82F8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A830C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8320: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8334: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8348: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A835C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8370: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8384: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8398: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A83AC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A83C0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A83D4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A83E8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A83FC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8410: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8424: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8438: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A844C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8460: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8474: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8488: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A849C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A84B0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A84C4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A84D8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A84EC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8500: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8514: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8528: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A853C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8550: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8564: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8578: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A858C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A85A0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A85B4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A85C8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A85DC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A85F0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8604: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8618: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A862C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8640: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8654: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8668: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A867C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8690: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A86A4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A86B8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A86CC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A86E0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A86F4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8708: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A871C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8730: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8744: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8758: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A876C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8780: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8794: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A87A8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A87BC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A87D0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A87E4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A87F8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A880C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8820: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8834: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8848: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A885C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8870: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8884: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8898: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A88AC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A88C0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A88D4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A88E8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A88FC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8910: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8924: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8938: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A894C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8960: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8974: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8988: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A899C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A89B0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A89C4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A89D8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A89EC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8A00: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8A14: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8A28: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8A3C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8A50: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8A64: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8A78: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8A8C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8AA0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8AB4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8AC8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8ADC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8AF0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8B04: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8B18: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8B2C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8B40: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8B54: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8B68: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8B7C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8B90: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8BA4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8BB8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8BCC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8BE0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8BF4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8C08: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8C1C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8C30: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8C44: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8C58: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8C6C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8C80: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8C94: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8CA8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8CBC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8CD0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8CE4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8CF8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8D0C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8D20: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8D34: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8D48: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8D5C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8D70: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8D84: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8D98: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8DAC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8DC0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8DD4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8DE8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8DFC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8E10: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8E24: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8E38: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8E4C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8E60: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8E74: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8E88: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8E9C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8EB0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8EC4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8ED8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8EEC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8F00: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8F14: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8F28: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8F3C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8F50: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8F64: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8F78: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8F8C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8FA0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8FB4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8FC8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8FDC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A8FF0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9004: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9018: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A902C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9040: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9054: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9068: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A907C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9090: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A90A4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A90B8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A90CC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A90E0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A90F4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9108: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A911C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9130: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9144: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9158: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A916C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9180: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9194: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A91A8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A91BC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A91D0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A91E4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A91F8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A920C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9220: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9234: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9248: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A925C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9270: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9284: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9298: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A92AC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A92C0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A92D4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A92E8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A92FC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9310: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9324: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9338: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A934C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9360: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9374: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9388: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A939C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A93B0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A93C4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A93D8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A93EC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9400: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9414: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9428: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A943C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9450: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9464: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9478: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A948C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A94A0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A94B4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A94C8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A94DC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A94F0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9504: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9518: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A952C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9540: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9554: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9568: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A957C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9590: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A95A4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A95B8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A95CC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A95E0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A95F4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9608: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A961C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9630: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9644: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9658: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A966C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9680: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9694: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A96A8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A96BC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A96D0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A96E4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A96F8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A970C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9720: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9734: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9748: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A975C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9770: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9784: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9798: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A97AC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A97C0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A97D4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A97E8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A97FC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9810: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9824: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9838: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A984C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9860: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9874: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9888: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A989C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A98B0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A98C4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A98D8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A98EC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9900: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9914: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9928: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A993C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9950: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9964: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9978: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A998C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A99A0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A99B4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A99C8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A99DC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A99F0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9A04: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9A18: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9A2C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9A40: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9A54: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9A68: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9A7C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9A90: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9AA4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9AB8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9ACC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9AE0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9AF4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9B08: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9B1C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9B30: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9B44: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9B58: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9B6C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9B80: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9B94: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9BA8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9BBC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9BD0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9BE4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9BF8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9C0C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9C20: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9C34: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9C48: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9C5C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9C70: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9C84: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9C98: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9CAC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9CC0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9CD4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9CE8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9CFC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9D10: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9D24: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9D38: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9D4C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9D60: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9D74: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9D88: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9D9C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9DB0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9DC4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9DD8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9DEC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9E00: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9E14: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9E28: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9E3C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9E50: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9E64: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9E78: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9E8C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9EA0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9EB4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9EC8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9EDC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9EF0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9F04: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9F18: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9F2C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9F40: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9F54: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9F68: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9F7C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9F90: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9FA4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9FB8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9FCC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9FE0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6C24A9FF4: 00000000 00000000 00000000 ............
DBCC execution completed. If DBCC printed error messages, contact your system administrator.
November 4, 2012 at 1:58 pm
What is your status for log backups? In order to do a page restore, you will need to bring those pages current with the rest of the database, and that means restoring log files as well. If there have truly been no changes to those pages since the backup you use, you may be able to get away without log backups. Either way, you are going to want to restore the most recent backup with a valid copy fo those pages.
November 4, 2012 at 2:34 pm
Simple recovery model...
CRAP!
ok, no page restore then.
My options just got limited.
option 1, restore full, restore diffs until i find the corruption, then reload in missing data. This is the long option.
option 2, create snapshot, run repair allow data loss, hope it's limited to the 6900 rows identified in the checkdb output, reload those rows later from an older backups of the source system.
I'm leaning toward option 2, then falling back to option 1 if needed.
thoughts?
November 4, 2012 at 3:04 pm
How quickly do you need to have it fixed? If there is enough time, I'd restore a backup to a test server and run repair there first to see how much data is lost.
November 4, 2012 at 3:12 pm
Robert Davis (11/4/2012)
How quickly do you need to have it fixed? If there is enough time, I'd restore a backup to a test server and run repair there first to see how much data is lost.
No rush to resolve this. Would taking a snapshot be sufficient before running repair?
Test servers are in different DC. Royal pain to get backups between them.
November 4, 2012 at 4:20 pm
There's no guarantee that reverting to the snapshot will work. I just tested it on a corrupt database (I have several laying around) where I created a snapshot, ran repair_allow_data_loss, and then reverted to the snapshot successfully. But it's not guaranteed. You may want to take a backup first.
November 5, 2012 at 1:52 pm
consecutive pageids, you may want to run a storage health check too.
-----------------------------------------------------------------------------------------------------------
"Ya can't make an omelette without breaking just a few eggs" 😉
November 5, 2012 at 3:40 pm
Good idea, thanks.
November 5, 2012 at 5:45 pm
ok, things just got wierd.
I went back one more page, and dbcc paged the contents:
.. valid data omitted ...
000000B6B9AC9E00: a9db97db 4008ae4b 97f0ffb9 a327c7e1 8ee50f2b ©Û?Û@.®K?ðÿ¹£'Çá?å.+
000000B6B9AC9E14: 6c6b5949 b9812ed2 147d00c1 22000000 00000000 lkYI¹..Ò.}.Á".......
000000B6B9AC9E28: 23000000 00000000 00000000 00000000 43006c00 #...............C.l.
000000B6B9AC9E3C: 75007300 74006500 72002000 50006100 72007400 u.s.t.e.r. .P.a.r.t.
000000B6B9AC9E50: 69007400 69006f00 6e000000 00000000 00000000 i.t.i.o.n...........
000000B6B9AC9E64: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6B9AC9E78: 00000000 00000000 16e3c9e3 5c0bb84d 817df92d .........ãÉã\.¸M.}ù-
000000B6B9AC9E8C: f00215ae 1617f00d 31dc904b 83abb8c1 53ba83f6 ð..®..ð.1Ü.K?«¸ÁSº?ö
000000B6B9AC9EA0: 24000000 00000000 21000400 00000000 00000000 $.......!...........
000000B6B9AC9EB4: 00000000 4d006900 63007200 6f007300 6f006600 ....M.i.c.r.o.s.o.f.
000000B6B9AC9EC8: 74002000 72006500 73006500 72007600 65006400 t. .r.e.s.e.r.v.e.d.
000000B6B9AC9EDC: 20007000 61007200 74006900 74006900 6f006e00 .p.a.r.t.i.t.i.o.n.
000000B6B9AC9EF0: 00000000 00000000 00000000 00000000 a2a0d0eb ................¢ Ðë
000000B6B9AC9F04: e5b93344 87c068b6 b72699c7 f5004ff4 6ab4cf44 å¹3D?Àh¶·&.Çõ.Oôj´ÏD
000000B6B9AC9F18: b4924435 539c4558 00080400 00000000 1edb55b4 ´?D5S?EX.........ÛU´
000000B6B9AC9F2C: 01000000 00000000 00000000 42006100 73006900 ............B.a.s.i.
000000B6B9AC9F40: 63002000 64006100 74006100 20007000 61007200 c. .d.a.t.a. .p.a.r.
000000B6B9AC9F54: 74006900 74006900 6f006e00 00000000 00000000 t.i.t.i.o.n.........
000000B6B9AC9F68: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6B9AC9F7C: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6B9AC9F90: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6B9AC9FA4: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6B9AC9FB8: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6B9AC9FCC: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6B9AC9FE0: 00000000 00000000 00000000 00000000 00000000 ....................
000000B6B9AC9FF4: 00000000 00000000 00000000 ............
"Microsoft reserved partition", "basic data partition", that looks like low level disk MBR or GPT data...
Has anyone ever seen this before?
November 5, 2012 at 7:43 pm
Ok, i think it's related to: http://support.microsoft.com/kb/2581408
it seems the that bad data was written to the disk at the 2TB mark.
Why remains a mystery. I'll update this thread when I have a definitive answer.
November 5, 2012 at 11:50 pm
What disk type are you using, what does Windows think the type is?
-----------------------------------------------------------------------------------------------------------
"Ya can't make an omelette without breaking just a few eggs" 😉
November 6, 2012 at 8:35 pm
Gpt partitioned lun on ibm ds4200 san. Windows sees it as such. the vol was created on a win 2k3 clus, detached, moved, re-attached on w2k8 then again on w2k8r2. a couple months ago the lun was extended from 2t to 3.4 tb. It does seem that the most recent attachment caused the corruption.
November 27, 2012 at 9:19 pm
status update: worked through ms support, verified that win wouldn't have caused this corruption as the detach/attach of the lun wouldn't have caused the partition data to be touched at all.
also found a known bios bug that affected these machines and would cause this corruption. we had already patched our bios to a version that included the fix, so it seems there was a regression. still working with hw vendor...
I fixed the corruption by cheating. I used a hex editor to overwrite the corrupt 20k from a staging copy of the db. The affected pages hadn't changed, and a checkdb afterwards came up clear, so I feel pretty good about the fix. it it hadn't worked, restoring full plus diff would have been the next step. Not the recommend approach, but effective.
Viewing 13 posts - 1 through 12 (of 12 total)
You must be logged in to reply to this topic. Login to reply