I know this is weird but I have faced situations where the Advanced tab of the jOb step had the option selected as :

On Success -> Quit the Job reporting Failure and then there were False alerts.. 🙂 This was done by some Junior DBA to check if the job really fires an alert on failure (Although there were other better ways that could be used)

I Hope this is not the case here as well 😉

Cheers!!!!