Click here to monitor SSC
SQLServerCentral is supported by Red Gate Software Ltd.
 
Log in  ::  Register  ::  Not logged in
 
 
 
        
Home       Members    Calendar    Who's On


Add to briefcase 1234»»»

The semaphore timeout period has expired Expand / Collapse
Author
Message
Posted Tuesday, September 16, 2008 12:42 AM
SSC Rookie

SSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC Rookie

Group: General Forum Members
Last Login: Monday, January 17, 2011 6:18 AM
Points: 32, Visits: 73
Hi to all,

Just moving over to a new firm I was asked to find out the reason why most of the clustered servers where prompting "SEMAPHORE TIMEOUT" every now and than so I digged in.
This message occurs when using all kinds of I/O to the subsystem (which resides on the SAN).

The exact error message is;

Executed as user: . TCP Provider: The semaphore timeout period has expired. [SQLSTATE 08S01]
(Error 121) Communication link failure [SQLSTATE 08S01] (Error 121). The step failed.


Hardly any info is to be found in the Internet so I need your help guys!!

As far as I found out this is an OS error which is written in the SQL server 2005 logs just before losing contact with one of the parts of the subsystems. This can be Hardware response timeout or an Autodetect setting on cluster network interface card. Others say this could be a Memory issue on windows 2003 SP1 cluster node with sql 2005 Ent. 64 bits SP1 or this is a bug/failure to properly release the cached memory. . . . .

Who delt with this issue before and /or can help me out on this?

Regards,
GKramer
The Netherlands
Post #569962
Posted Tuesday, September 16, 2008 1:54 AM


SSCrazy

SSCrazySSCrazySSCrazySSCrazySSCrazySSCrazySSCrazySSCrazy

Group: General Forum Members
Last Login: Yesterday @ 9:00 AM
Points: 2,707, Visits: 2,317
We had exactly the same problem with a cluster using SAN disks. However only one node showed this behaviour and it happened when high I/O occurred and especially around full backups. We changed the firmware etc and put the latest drivers on but still had the problem on the one node so in the end we replaced the node.
Post #569987
Posted Tuesday, September 16, 2008 2:02 AM
SSC Rookie

SSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC Rookie

Group: General Forum Members
Last Login: Monday, January 17, 2011 6:18 AM
Points: 32, Visits: 73
DNA,

Thanks for the comforting thought.....we have several hundreds of nodes showing this problem (randomly).......

GKramer
Post #569993
Posted Tuesday, September 16, 2008 2:53 AM


SSCrazy

SSCrazySSCrazySSCrazySSCrazySSCrazySSCrazySSCrazySSCrazy

Group: General Forum Members
Last Login: Yesterday @ 9:00 AM
Points: 2,707, Visits: 2,317
Hope you find out the root cause and don't have to start swapping out servers! We don't have hundreds of clusters but 20 or so and only one node on one cluster had the problem
Post #570017
Posted Friday, October 3, 2008 1:42 AM
Forum Newbie

Forum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum NewbieForum Newbie

Group: General Forum Members
Last Login: Friday, June 4, 2010 6:43 PM
Points: 8, Visits: 20
Hi!

I encountered same problem also.

ODBCQuery: SQLSTATE: 08S01. Error: (121). Msg: [Microsoft][SQL Native Client]TCP Provider: The semaphore timeout period has expired.

ODBCQuery: SQLSTATE: 08S01. Error: (121). Msg: [Microsoft][SQL Native Client]Communication link failure.

ODBCQuery: SQLSTATE: 08S01. Error: (10054). Msg: [Microsoft][SQL Native Client]Communication link failure.

Please if there's anybody might share what's the cause of these errors... and Hope to hear any solutions that you might suggests.

Thanks.

Post #580115
Posted Sunday, May 24, 2009 9:51 PM


Ten Centuries

Ten CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen Centuries

Group: General Forum Members
Last Login: Today @ 2:02 AM
Points: 1,251, Visits: 1,737
Sorry to bump an old post, but was this ever resolved? We have the same issue on a cluster and can't pinpoint where the connection is being dropped.
Post #722835
Posted Monday, May 25, 2009 12:10 AM
SSC Rookie

SSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC Rookie

Group: General Forum Members
Last Login: Monday, January 17, 2011 6:18 AM
Points: 32, Visits: 73
Foxxo,

This problem still excists and no cause can be found.
I Entered this on the Microsoft Helpdesk (we've got platinum support) but they cannot solve this either.
Recentley we where supported by a microsoft engineer for a project and aksed him if he was familiar with this problem but he also was scratching his head again and again.
As mentioned in the opening text this is still a random issue which occurs every now and then. The Timeout could be related to several parts on the hardware and/or probably to some software as well. . . . .

No sollution yet !!

Hope this will be sollved when we move over to MS Windows 2008 (running SQL Server 2008) within several months.

I can give you one small hint though;
examin your HBA's queue depth length and set it at least to 128.
We're hosting our SAN and they once set this queue depth length to 4 . . . .
by upgrading this setting we encounter a 8x speed up of our most heavly used systems and probably lose the semaphore issue. . .(I hope)

Regards,
Guus Kramer
Post #722852
Posted Tuesday, June 9, 2009 12:32 AM
SSC Rookie

SSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC Rookie

Group: General Forum Members
Last Login: Thursday, April 18, 2013 3:06 AM
Points: 25, Visits: 131
Hello Guus,

Did changing that setting ever resolve the problem?
Post #731167
Posted Tuesday, June 9, 2009 3:13 AM
SSC Rookie

SSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC RookieSSC Rookie

Group: General Forum Members
Last Login: Monday, January 17, 2011 6:18 AM
Points: 32, Visits: 73
Grasshopper,

Unfortinately this didn't solve the semaphore issue. . . . .
it is just increasing on some of our servers cuasing a lot of networktimeouts resulting in processes ending / dataloss (and a lot of extra work). This is driving us to the edge

Still no permanent solution yet even though we re-entred this at microsofts support-team.

Guus
Post #731252
Posted Monday, June 22, 2009 8:32 AM


SSC-Addicted

SSC-AddictedSSC-AddictedSSC-AddictedSSC-AddictedSSC-AddictedSSC-AddictedSSC-AddictedSSC-Addicted

Group: General Forum Members
Last Login: Monday, October 20, 2014 12:11 PM
Points: 454, Visits: 885
We just encountered the same problem yesterday during reindexing on a SQL 2005 SP2 EE 64bit cluster running on Windows Server 2003 EE SP2. We have three reindexing jobs running at the same time on three drives. One of the three jobs failed with the error below.

Message
Executed as user: BLAIRNET\sqlsaservice. TCP Provider: The semaphore timeout period has expired. [SQLSTATE 08S01] (Error 121) Communication link failure [SQLSTATE 08S01] (Error 121). The step failed.


We replaced the motherboard on this server a few days ago to address a memory issue. The following KB article [url=http://support.microsoft.com/kb/325487][/url] indicates this can be a firmware/motheboard issue, but the firmware is not that old and the KB references 32bit. We are 64bit.

Dave





Post #739428
« Prev Topic | Next Topic »

Add to briefcase 1234»»»

Permissions Expand / Collapse