Click here to monitor SSC
SQLServerCentral is supported by Red Gate Software Ltd.
 
Log in  ::  Register  ::  Not logged in
 
 
 

Bill Nicolich

A big fan of SQLServerCentral, I hope to add good posts for the general enjoyment of other readers at SQLServer Central. Also check my personal blog for additional comments, recommendations and such at SQLFave.blogspot.com.

Agile Practices Meet Database Development: Intro

I think the database developer role is on the rise and that it's better suited for agile practices - but guidance is needed make that infusion of agile practices a success. Here's why.

My impression is that fusing agile practices into the DBA role is inherently awkward since administration differs significantly from development. A DBA might be involved in development occasionally or might bundle together a series of items into a project where a SCRUM sprint could apply - but commonly a DBA is frequently interrupted by a broad spectrum of issues and must also attend to routines of ongoing maintenance and support where they're taken out of development.

That said, there are signals that database development is becoming more formalized into its own role unique from that of the DBA - with both roles overlapping each other to some extent of course. One signal is that Database development now has its own separate Microsoft certification with the 70-433 test offering the MCTS certification in Database Development.

Mind you, I'm talking about the broad-sweeping trends. No need to argue about individual cases. I'm suggesting that out in the wild, more places are cropping up for a separate database developer role - and that role could be more suited for the infusion of agile practices.

Another signal is Oslo - an upcoming approach to development involving collaborative modeling. The goal is to get the various IT professionals to work together more effectively which includes getting someone from the database side of the equation into the collaborative development space. That person I suspect is the database developer primarily and the DBA secondarily.

I think that adoption of agile practices are coming to the database world later then the world of software development at large. That's hard to prove. One hint though is that articles relating to test-driven development practices (one area of agile practices) on SQLServerCentral are not frequent and are more recent. It's somewhat of a new conversation. The question is, if agile practices are late in coming to database development, is that an advantage or a disadvantage? An important question is will we learn from the mistakes already made, or will we be doomed to repeat them? It depends, of course, but on what?

To help answer that, let me take a quick step back. I've been talking so far about agile "practices," lumping things together whereas Alistair Cockburn, one of the very founders of the entire Agile movement keenly separates out the "procedures" used in agile from the "properties" of an agile environment. In his influential book Crystal Clear: A Human-Powered Methodoligy for Small Teams, Cockburn suggests that if a small team ensures that their environment has certain properties such as "reflective improvement," then the practices used to foster that property will follow.

That said, we can clarify here that it's the specific practices like teaming up programmers to run in dual-code mode, setting up a walking skeleton and using incremental rearchitecture and so forth, might be more awkward for the database world. The properties, however, are not as awkward.

For instance, one of Cockburn's properties is "personal safety" - (which could end up being one of the most important agile team properties of all) I think can be set up in the database world with no inherent impedance. Personal safety in a nutshell is "being able to speak about something bothering you, without fear of reprisal" (Cockburn). It can lead a developer for instance to admit that an aspect of a project is beyond their ability. They'll get help sooner and the project will move forward.

At this point I think I should lay down another impression about the whole agile movement that is taking hold of software development - because I think it can serve as a warning signal for the database world. I think there are forces acting on the agile movement from the business world to change it into a sort of magical "get something for nothing" proposition. In order to sell an agile methodology, the technical personnel and others emphasize speed and success and tend to leave out everything else.

I think developers are being talked about as if they were processors on a motherboard. Extreme programming is like working in dual-processor mode. The discussions are about efficiency, throughput and keeping developers at maximum productivity.

What doesn't get mentioned or emphasized is the human-powered side of the equation. The result can be project-seizing turnover and morale drain. Set up a series of intense sprints with little time to come up for air, adjust the methodology so that the developers aren't coming up with the timelines, and what do you think will happen?

Here's where I think the database world can sidestep some thorny issues by going back to the beginning and embracing some of the values that were discussed at the conception of the agile movement.

The history, some of which is kept at agilemanifesto.org, is that some experienced developers set up a meeting one late winter day in 2001 at a ski lodge in the Wasatch mountains of Utah. One of the original signatories of the movement, Jim Highsmith, made a statement that I think we should take particular notice of. Highsmith says "I believe Agile Methodologists are really about the 'mushy stuff' about delivering good products to customers by operating in an environment that does more than talk about 'people as our most important asset' but actually 'acts' as if people were the most important, and lose the word 'asset'."

From the beginning, the agile movement was concerned about not just the success of software projects but also about the human sustainability of the software development profession. There is a concern for the developer community. There are costs and trade-offs to be made. So to create a sustainable and humane development environment, an agile methodology can't be sold to the business segment as a cost-free trade-off free proposition. From the get-go, the signatories of the agile manifesto laid out the key trade-offs that they perceived should be made, which can be reviewed here. One of them is "individuals and interactions over processes and tools" where individuals and interactions are valued more.

I'd like to venture one last speculative point in this entry - the validity of which I think doesn't negate the other points I've made. Non-database developers seem to wield suprisingly little political power in general. They're struggling to get control over setting their own sprint timelines. I think the database developers and DBAs might have a little stronger bargaining chip. They preside over not just an RBDMS these days - but with SQL Server and the likes, they've got the reigns on an entire data-centric business platform that handles a gambit of technological needs. If that added leverage exists, perhaps database professionals will become more successful at creating the agile environment that was and is actually intended by the founders.

Lastly, I recommend Cockburn's book, which reviews both the ideal properties of small agile teams, but also reviews some of the successful practices and techniques like walking skeleton, process miniature and so forth.

Comments

Posted by kcarlin on 6 August 2009

Good post.  Thanks.

Posted by David.Poole on 6 August 2009

My personal experience is that "normal" development does not recognise the value of data, or the fact that it has a long life.

If you could throw away the data then there would be no issue with database development being agile.  The more data there is the less agile any database developer/DBA is able to be.

An agile project may decide to refactor a table, but in a production environment they are talking about a 1 billion record table with partitioning and replication.  In development you can drop and recreate the table to suit your needs but in production this is going to breach 99.999% uptime SLAs.

What I am seeing is that data is becoming a more valuable resource, not a more disposable one.  If data is a valuable and long-lived resource then the inability to change a massiveley loaded structure has to be fed back into the design process so a suitable strategy can be derived.

Posted by Sushila on 7 August 2009

Well said David. I personally have trouble understanding the scalability factor of an "agile database" - what about (e.g.) performance tuning - altering/adding/dropping indexes etc. - in a production environment? Also, each sprint is (likely) a joined-at-the-hip relationship between application interface and database development - what does this do for sprint timelines if the database end requires more integrated testing and it is not feasible to have an exact simulation of this in the development/test environment? Am I being too naive in my perception and understanding of this subject? Bill - it'd be great if you could write (for instance) about how your team is using agile methodologies for database development - in theory, I love the agile concept - it is the practice of it that I do not have a handle on. Thx

Posted by Bill Nicolich on 7 August 2009

suesheila - I do want to write a post about specific "procedures" (at least what I know) to use in agile database development - and what techniques are available to make changes safely.

In this post/article I wanted to make some points about the "properties" - or how the environment should feel as developers work and interact - how agile collaboration can be improved and sustained over the long haul.

I think you bring up a good issue about performance tuning. This is an area where certain practices don't apply well - and perhaps shouldn't - such as automated tests. We might forego the automated test practice, but increase safety by maintaining version control and testing change scripts on a test database before applying to production and so forth.

One thing I'll say is that one can increase agility right now. Sure, pursue training and resources - but don't think you have to become certified in something before you make a move. Agility I think has to do with getting further into the safety zone when making changes and removing barriers to productivity (not just immediate but sustained) and even job satisfaction.

I also want to post about some of the common dysfunctions that I've seen happen between developer teams and management that prevent agility.

Leave a Comment

Please register or log in to leave a comment.