FYI, just got an answer from MS Azure support forum and you can also use Azure with an OLEDB connection string, unofficially. Here it is:

Provider=SQLNCLI10;Server=tcp:yourServer.database.windows.net;Database=yourDb;Uid=yourUser@yourServer;Pwd=yourPassword;

However, there seems to be better features/support with ODBC. Then again, OLEDB is not officially supported right now, only ODBC and ADO.NET. What I noticed with ODBC is that using it with QueryExPlus for Azure, I am able to see a list view of the tables and the table's fields in the left pane "primitive" object explorer of QueryExPlus. With an OLEDB connection for QueryExPlus, I don't get that list view.