The default value was set in 1998 on SQL Server 7. It's utterly and completely out of date. It bears no relation to the modern world. It should be a higher default, but Microsoft, being extremely conservative when it comes to their standards, won't change it. So, you should.
The core issue is simple. A query that crosses the threshold in terms of estimated cost can be run parallel. Not that it will be run parallel, just that it can. The problem is, simple queries without much complexity at all can easily pass an estimated cost of 5. What happens then is that you see queries that are only moving small amounts of data go parallel, taking up CPU, memory and I/O, but especially CPU, in order to split the data into multiple streams, process multiple streams, and then bring the streams together. This negatively impacts performance of those queries. Further, it starves resources of other queries, again, especially CPU, causing them to run slow, causing blocking, more resource consumption, more waits, etc.. You get the picture.
Follow the link that Johan gave you. It goes into more detail on this and also links to places that can help you suggest a better value.
Personally, I change this on every single server I've ever managed since about 2003. I even use this in demos of Redgate SQL Monitor as a thing to use the tool to help you solve. It's a silly default and should be changed everywhere.