• We're mostly CI and being pushed strongly towards CD, and I am just starting to think through some of the challenges. DB changes are seen as a major block to CD, and it does create some friction with some developers. Unfortunately databases are pretty much the anti-pattern for CD, they are shared across multiple teams, require specialized knowledge to code for properly, etc. A couple of issues I'm thinking about in particular are...

    Performance: Databases in most systems are single point bottlenecks, so a single poorly performing query can have system wide effect in a way that most app code can't. Performance tests in general are a tricky part of CD, and database performance testing is trickier than most. This isn't just about queries. Adding a not null column to a table that has 10,000 rows in a developers environment probably won't even flag as an issue, but will turn into an unpleasant evening when the same script is deployed to the production system with 100 million rows.

    Rollback: Rolling back code is easy. Rolling back data changes is really not. I was just reading a very well known book on the subject where they suggested "always make sure to backup your database" as a way to handle rollbacks. I almost threw the book down because of how fundamentally wrong headed that is. If I have to take a downtime long enough to restore one of our big db's from backup, I'm going to be unemployed. Rollback strategies for destructive data changes require careful planning and often keep both versions available for some period of time, using triggers, hiding through views, etc.

    Not to suggest that there aren't others. I think my first bite at the agile apple will probably be to instrument db deployments in a performance environment in order to capture slow scripts.