Using the advanced editor does work but I have quite few packages with this problem.
I'm pretty sure the problem is related to the MSSQLToSSIS10.XML mapping file but nothing I seem to do affects the way SSIS maps the columns. I think I am missing something here.
However I have found that changing the OLEDB provider for my source connections from 'Microsoft OLE DB Provider for SQL Server' to 'SQL Server Native Client 11.0' fixes the problems as SSIS maps the columns to the right data types.
The problem is not fixed but the work arround isn't too bad.