Hi Neeraj, unless I oversight something important this is not a credentials related issue. If I restart the application when the Secondary server is the Principal - it connects with no problem. I was running TCPView on the client machine and saw that the application attempted to connect to SQL2.domain.com,1433 where the server listens on 2444. The root cause seems to be that initially the Principal returned "SQL2.domain.com " to the client rather than "SQL2.domain.com,2444" and this returned value got cached and used then.
Also I can't see any login errors on the Secondary server (neither in SQL Error log nor in Security log).