ODBC-it is designed for connecting to relational databases.
However, OLE DB can access relational databases as well as nonrelational databases.
There is data in your mail servers, directory services, spreadsheets, and text files. OLE DB allows SQL Server to link to these nonrelational database systems. For instance, if you want to query, through SQL Server, the Active Directory on the domain controller, you couldn't do this with ODBC, because it's not a relational database. However, you could use an OLE DB provider to accomplish that.
In summary, The key difference between OLE DB and ODBC is that OLE DB can provide connection to data stored in non-relational format.