I understand that the goal is to translate voice commands into web search or iPhone commands, but I don't see why it's necessary to steam the actual voice over to one of Apple's servers. Not only is that a privacy issue, but I would expect it to result in latency or loss of service issues as well.

A 3rd party could implement a client side voice recognition solution that does the same thing but only better. They wouldn't have to invest in server or operational infrastructure; just sell the widget for $5 a pop.