Data Driven

Listen to the data.

There are any number of phrases that implore us to use bits and bytes, pieces of information that lead us to better decisions. However, can the data ensure we make the best decisions? Do the models we use get better and better over time? It's hard to tell.

This has certainly been debated in the wake of Moneyball, the book that ocuses on the use of data over experience to drive decisions for baseball teams. The Oakland As were the first team to do this, without winning a championship, but the Boston Red Sox also followed the formula and won three championships in the last eleven years.

However they've also had some abysmal years, including the current one. Does that mean that the principles of data driven decisions work, don't work, or something in between? Personally, I think that the ideas of predictive analytics does work, but it's not magic. It's also not a guarantee of reaching some level of performance.

In sports there's a strong human element involved. While many players do perform at a similar level from one year to the next (slightly higher or lower), there are also times when a player dramatically diverges from the past. There's also the notion that a sports team is a very small sample size for statistical analysis. Trends tend to be easier to predict when there are thousands of people's behavior involved.

There's also some variance from transaction to transaction. Even in retail, where I might be able to predict today's sales fairly accurately, I couldn't necessarily determine the volume of sales for a particular product or the total on any transaction. Statistics are generated over time, and they'll be accurate over time as well. Just understand that in the more a human is involved and the more detailed your granularity, the more actual results might deviate from your predictions.

Even the card counters in Bringing Down the House, for all the millions they earned, still expected to lose some hands, and occasionally some big ones. Keep that in mind when making data driven decisions.