Just to cover, there are no triggers firing on the destination table correct?

Also from this page https://msdn.microsoft.com/en-us/library/ms141237(v=sql.105).aspx"> https://msdn.microsoft.com/en-us/library/ms141237(v=sql.105).aspx

I can see where it reads "Performance can be improved if you use the ORDER option to sort the input data according to the clustered index on the table. " Perhaps that can offer some benefits with using the Fast Load option with a SSIS data flow destination