Thanks george, in my case. It seems that the clustered windows O/S is taking up the connection. When I tried to run the server in normal mode and ran a trace on the connection. I found that the window o/s is connect to sql (may be due to clusters).
Also, FYI we use Veritas Netback. Could that be the issue?