Maybe. Maybe not. Depends on the rows involved, the join type, other filters, complexity of the query, data types and a few other things. Add indexes, see if they help.
Temp tables are not necessarily memory resident, but anything that they query processor operates on will be in memory, temp or permanent, the query processor doesn't know what a disk is.
Ignoring joins, the permanent table needs a clustered index.
Microsoft Certified Master: SQL Server, MVP, M.Sc (Comp Sci)SQL In The Wild
: Discussions on DB performance with occasional diversions into recoverability
We walk in the dark places no others will enter
We stand on the bridge and no one may pass