Restoring a 2008 R2 backup to a 2012 instance means the restore process has to do an in-place upgrade of the involved database files which can add time to the restore process.

Are you watching the restore process, or do you have access to the output as it runs? If you supplied WITH STATUS then you'll see once it reaches 100% that upgrade messages start to be written. See if you can isolate which part is taking so long, the restore of the data files to disk, or the upgrade process that starts after that is done.