i've played with Microsoft.SqlServer.Management.Smo.Scripter that is part of the microsoft.sqlserver.managment.sdk.sfc ;

using that, you can tokenize any SQL statement.

i did it when i was playing with making my own SQL reformatter/prettifier, so if you had a suite of strings you wanted to abstract/tokenize out, that's the way to do it;

you can do it in a programming language, and there's a brief example on an SSC blog here:

http://www.sqlservercentral.com/blogs/dave_ballantynes_blog/2012/03/13/parsing-t-sql-the-easy-way/

here's the master list of tokens that can appear in a given SQL statement:

http://technet.microsoft.com/en-us/library/microsoft.sqlserver.management.sqlparser.parser.tokens(v=sql.105).aspx