........ Long story short, we need to rewrite our aggregation routines, and it looks like you're pointing us in the right direction.
I would entirely agree with your comment and suggest you read and reread the article in question.
I am sure that you will be able to increase performance.
ps....whilst the solution I provided may work...I wouldnt be looking to implement in production :-)...was only suggested to get the discussion going some more...both to help you and to assist my own learning.
________________________________________________________________you can lead a user to data....but you cannot make them think
and remember....every day is a school day