Recovery model has no effect at all on recovery times.

Recovery time is a factor of how much needs to be undone or redone at startup. If it's taking forever, you likely had a large uncommitted transaction at time of shutdown or a large transaction that had committed but not yet written to the data file, possibly also VLF fragmentation.