The rule of thumb is that you always go to 3rd Normal Form (or preferably to Elementary Key Normal Form) and then look at whether you need to go to a higher normal form. But that's because you want to be sure that you don't have buggy or overcomplicated code, not a performance issue.

And usually, going to 3NF (or EKNF) instead of staying at some lower normal form will improve performance as well as reducing code complexity, because it will reduce the size of your data.

As a general rule, any redundancy in data costs both performance and code complexity so that you end up with buggy and non-performant code. But there may be cases where redundancy can be justified - but you need to verify the justification by seeing what actually happens when you normalise the redundancy out.

Obviously if you build a data warehouse which doesn't permit any update it can safely be in a much lower normal form (hence more redundancy so greater data size than if it were properly normalised) and it will be worth doing some performance tests to see whether the normalised or the unnormalised form gives better performance in this no update case.