A few questions:

1. Ballpark how many transaction log backups are you restoring to get your database up to date?

2. How big are these transaction log backup files?

3. When each individual transaction log backup restored are you using the READ ONLY recovery option with the tuf file?

If the answer to point 1 is more than 1, and the answer to point 3 is yes, then this is the reason your restore takes so long, as when restoring the log backup with the READ ONLY recovery option it requires the creation of the tuf file so that it can be returned to a no recovery state when the next transaction log backup is restored. If you have lots of files (like we do) you only want to recover the database to read only when the last transaction log backup in your set is restored.