It may be frustrating, but often it's best to materialize something that otherwise can "get lost in the plan", so to speak. I can't begin to tell you how often I've had CTEs that pull a rather small number of rows end up choking the rest of the query until I put the results into a temp table instead and join to that temp table instead. It just seems sometimes that there are some queries that the optimizer simply can not fathom a good row-count guess for, until it's "written in stone", so to speak.