norecovery option would have to kick off connections as well.

You would need a lot of uncommitted transactions I would say before getting a noticeable degradation in restore time.

If by latency you mean there is a delay before the logs are restored that is a setting you can specify in log shipping for that - check it has not been set.