If the ACE OLE DB driver thinks the column is a date column, it will try converting the numbers to a date.

Internally, dates in Excel are represented by a number (the number of (milli)seconds after a certain date).