It might be easier to follow if we could see the actual code, but....

Remember that a CTE does not produce a temporary table. It is just a symbolic representation of a query (like a view). Every time you reference the CTE, it runs the CTE query all over again, or possibly does a lazy spool. If I need to reference a result from a cte multiple times, I usually instantiate it by writing it out to a temporary table (and possibly indexing it) before proceeding to the next step.