Thank you very much Lynn for asking me to write the step by step script for you to replicate issue on your side.

While I was doing that I just realized that the default database for the Application user being as "Master".

So anytime the failover was happening the user was able to connect to Master but not the User DB.

I corrected it and it started working as expected.

Thanks you very much again.