How are your waits? Are you seeing lags in tempdb? If not, leave it alone. If so, make a change. Guidance on the number of files for tempdb is all over the map. I generally fall to "more than one is good, more than eight is WAY too many". It's simplistic, but keeping things simple and only adjusting as evidence suggests adjustments are needed has worked really well in the past. Others suggest one file per core. Others say that, but a limit of 8. It varies. You have more than one per core. PANIC! Kidding. I doubt you're seeing anything major. Plus, is tempdb management your worst performance headache? For most, it's not. It's all the bad code, horrid structures, and out of date statistics that are causing issues.
Measure, then decide.