Why did you create a new master key on instance2? :w00t: This is probably the cause of the problem.
The correct sequence to restore an encrypted database from one instance to another is:
Inst1
Run SetupTDE.sql
Backup database to db.bak
Copy db.bak, TDEServerMasterKey.key, TDEServerCertificate.cer and TDEServerCertificate.key to Inst2
Inst2
Run RestoreTDE.sql
Run StatusOfTDE.sql to check all OK
Restore the database
I have done this many times and it works fine.
To then remove TDE from Inst2:
Run RemoveTDE.sql
Possibly necessary now to restart SQL service so that tempdb is rebuilt definitly without TDE
You will not now be able to restore db.bak until you run RestoreTDE.sql again