I'm guessing your error rows destination is preceded by an OLE_DB destination pointing to the table?
What I've done in this kind of case is to do a two-stage insert. The first OLE_DB destination uses the Fast Load data access mode (which attempts to insert in batches.) The problem with this mode is that a single bad row in the batch causes the whole batch to be thrown away. However, it's MUCH faster than Table or view, which attempts to insert each row one at a time. So, redirect error rows from the Fast Load into another OLE_DB destination in Table or view mode, then redirect error rows to your flat file for examination. That should at least limit the error records you need to look at.
[Flat File Source]
| (green arrow)
[OLE DB Dest, Fast Load]
| (red arrow)
[OLE DB Dest, Slow Load]
| (red arrow)
[Error Flat File Destination]
As an aside, it would probably be best to select the SECOND OLE DB Destination's errorcode/errorcolumn to go into the file. It may or may not tell you what happened, but it should be more accurate than the first error.