When the SELECT statement is compiled for the INSERT, the first value is implicitly taken as an INT. In the UNIONs, as each SELECT is added, they must match this data type.
If that is the correct explanation, would the next statement work?
Insert into test Select 'A' union Select 1;
(Answer: No)