The issue isn't the use of <> or != (which I prefer the later). The pitfall is in presumption of the data and our understanding of the question.

The question being asked that fits your final solution is

"Which students haven't taken the %s exam?"

In your first solution the question is more to the effect

"List the students who have taken exams each time but do not show the times the %s exam was taken."

If you had used left join

"List the students who have taken exams each time but do not show the times the %s exam was taken and make sure to show those who haven't taken an exam at all at least once."

The biggest issue with generating data for reports is always making sure you understand the intended result fully and structure the query correctly. Some syntax is more prone than others to creating the structure that represents the question you want to pose. As always thou there can be dozens of ways to ask the same question and get the correct answer, then it is a matter of most effective way.