I use pandas and various other Python tools to do this sort of analysis. Like Buck Woody, I'm building a toolbox that will probably include R in the future. This is in addition to Excel, SQL Server and various other utilities.
As for data sets, I'd rather grab some of the various climate, topographic, astronomy, census or demographic data available from the government and other organizations. I can usually find data that's interesting and applicable to similar work data.