SQL Clone
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


SQL Server has encountered occurrence(s) of I/O requests taking longer than 15 seconds to complete


SQL Server has encountered occurrence(s) of I/O requests taking longer than 15 seconds to complete

Author
Message
Summer90
Summer90
SSCertifiable
SSCertifiable (6.9K reputation)SSCertifiable (6.9K reputation)SSCertifiable (6.9K reputation)SSCertifiable (6.9K reputation)SSCertifiable (6.9K reputation)SSCertifiable (6.9K reputation)SSCertifiable (6.9K reputation)SSCertifiable (6.9K reputation)

Group: General Forum Members
Points: 6885 Visits: 3831
I have a few SQL Server 2008R2 SP1 and SP2 as well as a couple SQL Server 2008 SP3 getting this error when I run Update Stats or DBCC CHECKDB. This started a few weeks ago and is getting worse and worse. Before the issue started we did not have any large dbs on this one specific SQL Server and now that we do we are getting these. I know in my research that this can be a very difficult issue to pinpoint what is causing this. I know it isn't AV as the SQL Server files are excluded and we also stopped the service on two of the servers. Anyone ever dealt with this issue? I am going to open a case with Microsoft in the morning.

SQL Server has encountered 22953 occurrence(s) of I/O requests taking longer than 15 seconds to complete on file [H:\MSSQL10_50.MSSQLSERVER\MSSQL\Data\xxxxxxxxx.MDF] in database [CORETABLES_PROD] (6). The OS file handle is 0x0000000000000DE4. The offset of the latest long I/O is: 0x000000d152a000
SpringTownDBA
SpringTownDBA
Ten Centuries
Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)

Group: General Forum Members
Points: 1408 Visits: 1499
checkdb is very IO intensive, and your IO subsystem isn't keeping up.

What are you using for storage?
vivekkumar341
vivekkumar341
SSC Veteran
SSC Veteran (244 reputation)SSC Veteran (244 reputation)SSC Veteran (244 reputation)SSC Veteran (244 reputation)SSC Veteran (244 reputation)SSC Veteran (244 reputation)SSC Veteran (244 reputation)SSC Veteran (244 reputation)

Group: General Forum Members
Points: 244 Visits: 93
SQL Server has encountered 22953 occurrence(s) of I/O requests taking longer than 15 seconds to complete on file [H:\MSSQL10_50.MSSQLSERVER\MSSQL\Data\xxxxxxxxx.MDF] in database [CORETABLES_PROD] (6). The OS file handle is 0x0000000000000DE4. The offset of the latest long I/O is: 0x000000d152a000



22953 code is for "Could not remove Change Data Capture metadata for capture instance ‘%s’ and source table ‘%s.%s’. Refer to previous errors in the current session to identify the cause and correct any associated problems."

for more detail please check below mentioned url:

http://sqlserverlearner.com/sql-server-error-list/sql-server-error-messages-list-22000-to-39000

Regards,
SpringTownDBA
SpringTownDBA
Ten Centuries
Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)

Group: General Forum Members
Points: 1408 Visits: 1499
vivekkumar341 (9/30/2012)
SQL Server has encountered 22953 occurrence(s) of I/O requests taking longer than 15 seconds to complete on file [H:\MSSQL10_50.MSSQLSERVER\MSSQL\Data\xxxxxxxxx.MDF] in database [CORETABLES_PROD] (6). The OS file handle is 0x0000000000000DE4. The offset of the latest long I/O is: 0x000000d152a000



22953 code is for "Could not remove Change Data Capture metadata for capture instance ‘%s’ and source table ‘%s.%s’. Refer to previous errors in the current session to identify the cause and correct any associated problems."

for more detail please check below mentioned url:

http://sqlserverlearner.com/sql-server-error-list/sql-server-error-messages-list-22000-to-39000

Regards,


22953 isn't the error code, it's the number of times the error occured...
vivekkumar341
vivekkumar341
SSC Veteran
SSC Veteran (244 reputation)SSC Veteran (244 reputation)SSC Veteran (244 reputation)SSC Veteran (244 reputation)SSC Veteran (244 reputation)SSC Veteran (244 reputation)SSC Veteran (244 reputation)SSC Veteran (244 reputation)

Group: General Forum Members
Points: 244 Visits: 93
Please go through the url and my mistake its error code.
SpringTownDBA
SpringTownDBA
Ten Centuries
Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)

Group: General Forum Members
Points: 1408 Visits: 1499
vivekkumar341 (9/30/2012)
Please go through the url and my mistake its error code.


No thanks, I prefer this url: http://support.microsoft.com/kb/2137408
SQL Server logs "Msg 833" when I/O delay problems occur
mssqlsrv
mssqlsrv
Mr or Mrs. 500
Mr or Mrs. 500 (505 reputation)Mr or Mrs. 500 (505 reputation)Mr or Mrs. 500 (505 reputation)Mr or Mrs. 500 (505 reputation)Mr or Mrs. 500 (505 reputation)Mr or Mrs. 500 (505 reputation)Mr or Mrs. 500 (505 reputation)Mr or Mrs. 500 (505 reputation)

Group: General Forum Members
Points: 505 Visits: 581
I also had the same issue.
It was due to the storage which was used for replication. During the storage replication
we used to get lots I/O messages,even sql services went down.

My windows and storage team had resolved this issue somehow.
Summer90
Summer90
SSCertifiable
SSCertifiable (6.9K reputation)SSCertifiable (6.9K reputation)SSCertifiable (6.9K reputation)SSCertifiable (6.9K reputation)SSCertifiable (6.9K reputation)SSCertifiable (6.9K reputation)SSCertifiable (6.9K reputation)SSCertifiable (6.9K reputation)

Group: General Forum Members
Points: 6885 Visits: 3831
Thanks for the link SpringTown... I hadn't found that article. I have opened an incident with MSFT and have yet another meeting with the storage gang today. The pressure is on ME to find what is causing this since users come to me complaining about performance issues.
SpringTownDBA
SpringTownDBA
Ten Centuries
Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)Ten Centuries (1.4K reputation)

Group: General Forum Members
Points: 1408 Visits: 1499
Markus (10/1/2012)
Thanks for the link SpringTown... I hadn't found that article. I have opened an incident with MSFT and have yet another meeting with the storage gang today. The pressure is on ME to find what is causing this since users come to me complaining about performance issues.


Ah, the joys of dealing with storage people....

I highly recommend Brent Ozar's webcast: "How to Prove It's a SAN Problem" http://www.brentozar.com/archive/2011/08/how-prove-its-san-problem-webcast-video/

Also, I'd recommend capturing some windows perfmon counters while the problem is occuring.

This is my list for troubleshooting IO issues:

(N: and P: are the SAN volumes that host DB files)
\LogicalDisk(NSmile\*
\LogicalDisk(PSmile\*
\SQLServer:Buffer Manager\Buffer cache hit ratio
\SQLServer:Buffer Manager\Checkpoint pages/sec
\SQLServer:Buffer Manager\Database pages
\SQLServer:Buffer Manager\Free list stalls/sec
\SQLServer:Buffer Manager\Free pages
\SQLServer:Buffer Manager\Lazy writes/sec
\SQLServer:Buffer Manager\Page life expectancy
\SQLServer:Buffer Manager\Page lookups/sec
\SQLServer:Buffer Manager\Page reads/sec
\SQLServer:Buffer Manager\Page writes/sec
\SQLServer:Buffer Manager\Readahead pages/sec
\SQLServer:Buffer Manager\Reserved pages
\SQLServer:Buffer Manager\Stolen pages
\SQLServer:Buffer Manager\Target pages
\SQLServer:Buffer Manager\Total pages
\Memory\*

Capture at 1 second intervals, (capture only while needed).

Major things to look for, Disk Reads/Sec, Disk Writes/Sec (IOPS) and how that affects latency ( Avg Sec/Write, Avg Sec/Read )
And, what is driving those IOPS (Lazy writes, checkpoints, etc)
TheSQLGuru
TheSQLGuru
SSC-Dedicated
SSC-Dedicated (31K reputation)SSC-Dedicated (31K reputation)SSC-Dedicated (31K reputation)SSC-Dedicated (31K reputation)SSC-Dedicated (31K reputation)SSC-Dedicated (31K reputation)SSC-Dedicated (31K reputation)SSC-Dedicated (31K reputation)

Group: General Forum Members
Points: 31035 Visits: 8669
Markus (10/1/2012)
Thanks for the link SpringTown... I hadn't found that article. I have opened an incident with MSFT and have yet another meeting with the storage gang today. The pressure is on ME to find what is causing this since users come to me complaining about performance issues.


There are a BUNCH of things that can cause IO problems. A ticket with MSFT on this is a non-starter. The problem is your system - period. Hire a good performance consultant and let them efficiently find/fix issues (there are ALWAYS a bunch of low-hanging-fruit issues whenever I do a first review at a new client) and mentor you along the way. Just a few hours or days will likely give you a system that is WAY different from what you are experiencing now!

Best,
Kevin G. Boles
SQL Server Consultant
SQL MVP 2007-2012
TheSQLGuru on googles mail service
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum

































































































































































SQLServerCentral


Search