Click here to monitor SSC
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


SQL Cluster shutdown issue due to network issue


SQL Cluster shutdown issue due to network issue

Author
Message
govindarajan69
govindarajan69
Grasshopper
Grasshopper (20 reputation)Grasshopper (20 reputation)Grasshopper (20 reputation)Grasshopper (20 reputation)Grasshopper (20 reputation)Grasshopper (20 reputation)Grasshopper (20 reputation)Grasshopper (20 reputation)

Group: General Forum Members
Points: 20 Visits: 215
Hi All,

I have faced an issue yesterday on our SQL Server 2008 SP1 cluster. It got shut down due to network issue. In HA, the cluster should failover to another node if there is any issue. When the private network is not down, why the cluster service went down. The error reported in Event Viewer and Cluster events are 1205, 1069, 1077, 1126, 1127, 1129 and 1130.

Please help me to understand.

Thanks in advance.

Regards
S Govindarajan
Perry Whittle
Perry Whittle
SSCrazy Eights
SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)

Group: General Forum Members
Points: 8839 Visits: 16579
Check your windows application and system logs, collate this info with the cluster events log. Can you post details of any errors reported, failure of the public network will initiate a failover.

Public network will usually allow cluster and user traffic.
Private network will usually only allow cluster communications.

If the private network goes down it can still communicate on the public network.

-----------------------------------------------------------------------------------------------------------

"Ya can't make an omelette without breaking just a few eggs" ;-)
dan-572483
dan-572483
SSChasing Mays
SSChasing Mays (647 reputation)SSChasing Mays (647 reputation)SSChasing Mays (647 reputation)SSChasing Mays (647 reputation)SSChasing Mays (647 reputation)SSChasing Mays (647 reputation)SSChasing Mays (647 reputation)SSChasing Mays (647 reputation)

Group: General Forum Members
Points: 647 Visits: 1958
Network issues can make clusters inaccessible, prevent failover, or trigger failovers than shouldn't happen, but they shouln't cuase the cluster instance or SQL server to completely shut down.
govindarajan69
govindarajan69
Grasshopper
Grasshopper (20 reputation)Grasshopper (20 reputation)Grasshopper (20 reputation)Grasshopper (20 reputation)Grasshopper (20 reputation)Grasshopper (20 reputation)Grasshopper (20 reputation)Grasshopper (20 reputation)

Group: General Forum Members
Points: 20 Visits: 215
Hi All,

Thanks for your reply.

Even i have the same understanding that the sql server should not go down due to network issues whether it is private or public. When i was going thru the event logs, i found event id 1077

- System
- Provider
[ Name] Microsoft-Windows-FailoverClustering
[ Guid] {baf908ea-3421-4ca9-9b84-6689b8c6f85f}
EventID 1077
Version 0
Level 2
Task 20
Opcode 0
Keywords 0x8000000000000000
- TimeCreated
[ SystemTime] 2012-11-07T09:25:16.053Z
EventRecordID 45227
Correlation
- Execution
[ ProcessID] 5232
[ ThreadID] 28880
Channel System
Computer NODE1.COMPANY.COM
- Security
[ UserID] S-1-5-18
- EventData
ResourceName SQL IP Address 1 (SQLCLUSTER01)
IPAddress 192.168.101.154
Status 1117
Health check for IP interface 'SQL IP Address 1 (SQLCLUSTER01)' (address '192.168.101.154 ') failed (status is '1117'). Run the Validate a Configuration wizard to ensure that the network adapter is functioning properly.

When i searched in google, i found that "failed (status is '1117')." is ERROR_IO_DEVICE error. On the other hand, i could not find any logs related to this in system logs and applications logs.

Help me to find out how i can find the root cause for this message.

Thanks in advance.

Regards
S Govindarajan
Perry Whittle
Perry Whittle
SSCrazy Eights
SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)

Group: General Forum Members
Points: 8839 Visits: 16579
govindarajan69 (11/8/2012)
Even i have the same understanding that the sql server should not go down due to network issues whether it is private or public.

No, stop, this is incorrect.
Kindly re read my post above, failure of the public network will initiate a failover of the instance.

-----------------------------------------------------------------------------------------------------------

"Ya can't make an omelette without breaking just a few eggs" ;-)
govindarajan69
govindarajan69
Grasshopper
Grasshopper (20 reputation)Grasshopper (20 reputation)Grasshopper (20 reputation)Grasshopper (20 reputation)Grasshopper (20 reputation)Grasshopper (20 reputation)Grasshopper (20 reputation)Grasshopper (20 reputation)

Group: General Forum Members
Points: 20 Visits: 215
Hi Perry,

if you are saying the public network will initiate a failover of the instance, will it shut down the cluster itself ? or will it failover to the available node ? When connection to the SAN Storage & Quorum disk are intact, then why it is happening.

Regards
Govind
Perry Whittle
Perry Whittle
SSCrazy Eights
SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)

Group: General Forum Members
Points: 8839 Visits: 16579
It should try to start on an available node, if theres an issue with the available node too then the group will stay offline after a certain amount of retries

-----------------------------------------------------------------------------------------------------------

"Ya can't make an omelette without breaking just a few eggs" ;-)
govindarajan69
govindarajan69
Grasshopper
Grasshopper (20 reputation)Grasshopper (20 reputation)Grasshopper (20 reputation)Grasshopper (20 reputation)Grasshopper (20 reputation)Grasshopper (20 reputation)Grasshopper (20 reputation)Grasshopper (20 reputation)

Group: General Forum Members
Points: 20 Visits: 215
Thanks Perrry. Infact thats what my understanding after this incident. It might have been costlier though.

Regards
Govind
Perry Whittle
Perry Whittle
SSCrazy Eights
SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)SSCrazy Eights (8.8K reputation)

Group: General Forum Members
Points: 8839 Visits: 16579
You can change the actions that are taken during a failover but the default is to try online locally then fail over and try 3 times then offline if unsuccessful.

It sounds like you had a public network outage affecting both nodes, ensure they're not plugged into the same switch ;-)

-----------------------------------------------------------------------------------------------------------

"Ya can't make an omelette without breaking just a few eggs" ;-)
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum

































































































































































SQLServerCentral


Search