Click here to monitor SSC
SQLServerCentral is supported by Red Gate Software Ltd.
 
Log in  ::  Register  ::  Not logged in
 
 
 
        
Home       Members    Calendar    Who's On


Add to briefcase ««12

On Database Migrations and Agility Expand / Collapse
Author
Message
Posted Sunday, January 22, 2012 11:41 AM


Grasshopper

GrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopper

Group: General Forum Members
Last Login: Sunday, May 19, 2013 8:22 AM
Points: 17, Visits: 148
I can't really concur with Dreamweaver's analogy with human development. But Evil Craig F (Ferguson?) is spot on with:

The powers that be must understand that you WILL require entire sprints for re-design of the core module to integrate new work. It's not an optional choice. This will require downtime when the fixes are put into place.


Continuing the building analogy, it's like responding (to the customer's request for a bedroom after seeing the dynamite kitchen): "Sure, but we've got to shut down the kitchen, and tear down some of the existing work to rebuild this into a 2 room module. You won't be able to use the kitchen for a while."

Sure, it's not an exact analogy. More like you're building modular housing that you can crane into position. But you still need time to extend or replace the foundation, and move all the owner's stuff (data) from the one room into the new 2 room module, etc. etc.

Of course, I think most people would concur with Peter Schott that:

I'd much rather start small enough to show something and get feedback as we go. That helps us make course corrections and tackle more important needs as they come up instead of delaying them further.


In my more cynical moments, I feel like I'm one of the workers on the (in)famous Winchester Mystery House...
http://en.wikipedia.org/wiki/Winchester_Mystery_House
http://www.prairieghosts.com/winchester.html
Post #1239940
Posted Sunday, January 22, 2012 12:46 PM


SSCertifiable

SSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiable

Group: General Forum Members
Last Login: Today @ 1:55 AM
Points: 6,142, Visits: 7,195
ganotedp (1/22/2012)
I can't really concur with Dreamweaver's analogy with human development. But Evil Craig F (Ferguson?) is spot on with:


Craig Farrell. :) Pleasure to meet ya. It's in the sig. The nickname's a joke that I happened to like.



- Craig Farrell

Never stop learning, even if it hurts. Ego bruises are practically mandatory as you learn unless you've never risked enough to make a mistake.

For better assistance in answering your questions | Forum Netiquette
For index/tuning help, follow these directions. |Tally Tables

Twitter: @AnyWayDBA
Post #1239950
Posted Sunday, January 22, 2012 1:20 PM


Grasshopper

GrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopperGrasshopper

Group: General Forum Members
Last Login: Sunday, May 19, 2013 8:22 AM
Points: 17, Visits: 148
Good to meet you! Just had this vague notion that Craig Ferguson was more than just a late night TV host... what if he were a database guy by day!
Post #1239958
Posted Sunday, January 22, 2012 3:18 PM
SSC Eights!

SSC Eights!SSC Eights!SSC Eights!SSC Eights!SSC Eights!SSC Eights!SSC Eights!SSC Eights!

Group: General Forum Members
Last Login: Tuesday, July 22, 2014 3:47 PM
Points: 837, Visits: 1,259
Evil Kraig F (1/22/2012)
With that caveat, I agree you can modularly build databases within the agile structure. However, I also agree with Gus. I've seen too many shops that got their prod powers removed and suddenly Agile is 'problematic'. It's all to often a Cowboy mentality.


Along those lines, ones you have a decent Continuous Delivery system in place, this shouldn't be as much of a problem. You have your downstream environment(s) that are exact copies of your production ones, gen up the change script(s) using your automated tools, then just hand them off to the team doing the changes in Prod. They should have sufficient permissions to do the changes needed and ideally you have at least one downstream environment that's a duplicate of your production environment so the testing and notes about other pieces of the equation can be handled. We've been using the VS2010 DB Projects to store and push our changes and are pretty close to having the ability to just build/deploy all the way up the chain.

As for complete rebuilds, ideally the project hasn't changed too much along the way and/or you've done the appropriate research before starting that you have a pretty good idea of where this is going in the long run so total redesigns aren't needed. :) If not then you have a lot of other problems to deal with in data conversions, rollbacks, new structures, and all sorts of other things. It's necessary sometimes, but it can often be avoided with some good discussions and planning up front. (not to the waterfall point, but enough where you've got a good enough design for most of the objects)



Post #1239964
Posted Sunday, January 22, 2012 4:50 PM


SSCertifiable

SSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiableSSCertifiable

Group: General Forum Members
Last Login: Today @ 1:55 AM
Points: 6,142, Visits: 7,195
Peter Schott (1/22/2012)

Along those lines, ones you have a decent Continuous Delivery system in place, this shouldn't be as much of a problem. You have your downstream environment(s) that are exact copies of your production ones, gen up the change script(s) using your automated tools, then just hand them off to the team doing the changes in Prod. They should have sufficient permissions to do the changes needed and ideally you have at least one downstream environment that's a duplicate of your production environment so the testing and notes about other pieces of the equation can be handled. We've been using the VS2010 DB Projects to store and push our changes and are pretty close to having the ability to just build/deploy all the way up the chain.

Build and deployment of new modules aren't really my concern there. The equivalent of adding a scoring system to the existing inventory system isn't where Agile starts to breakdown, at least in my mind, from a database perspective.

However, allow me an actual example that I walked headfirst into a few years ago.

A simple e-sales system had started with a solid core scope and understanding. Products were included, carts were built, etc etc. It worked very well. Attached to this were additional modules. Eventually, a catalog system was built in for custom client pricing. Certain deals were to be offered to certain clients. For example, students at particular schools could order things cheaper by being a student.

So overrides were put in place in certain places, the cart was modified, etc etc. Eventually this cart proc became about 30 pages long, and was so convoluted that processing a cart to completion could take up to 10 seconds. This was unacceptable due to ever increasing volume order.

A full redesign of the structure, incorporating the new modules into the core structure, allowed for a much fuller and less intricate system of maintaining the data. This took 3 months of pure dev work, no new features, no ... you get the drift. It wasn't possible to do this in small doses, we had to redesign basically from scratch for all the new modular components that had been included.

I'm all for agile as long as the time is taken every few years to clean up the mess it leaves behind.

It's not because you can't plan for the scope, it's because the very NATURE of Agile is planned scope creep.



- Craig Farrell

Never stop learning, even if it hurts. Ego bruises are practically mandatory as you learn unless you've never risked enough to make a mistake.

For better assistance in answering your questions | Forum Netiquette
For index/tuning help, follow these directions. |Tally Tables

Twitter: @AnyWayDBA
Post #1239969
Posted Sunday, February 5, 2012 5:20 AM


SSCrazy

SSCrazySSCrazySSCrazySSCrazySSCrazySSCrazySSCrazySSCrazy

Group: General Forum Members
Last Login: Tuesday, July 22, 2014 5:56 AM
Points: 2,013, Visits: 1,584
The biggest strength of Agile Development is its biggest weakness as well i.e. ‘narrowed vision’. You don’t see the problem as a whole but as bunch of problems jumbled together, with assumption that each piece of problem is independent (or less dependent) of another. This assumption may & may not be true in real life.

I like agile development but my style is not Pure-Agile. I had limited the scope of agile development to SQL coding only. It doesn’t include Data Modeling. The reason is simple (and to follow the building analogy) to make the solid foundation & skeleton for the building first. Once I am convinced that the foundation can handle few floors weights without much adjustment, I don’t care which room of the building or which floor user wants to see first.

I would like to add here that few of my colleagues read about Agile DEV from BOL or some BLOG and blindly started following it. The projects been delivered to customers and customer was happy as well. But the delivery involved too much integration costs and over-time of DEV team. It’s not feasible IMHO.


Dev

Devendra Shirbad | BIG Data Architect / DBA | Ex-Microsoft CSS (SQL 3T) | Open Network for Data Professionals...
LinkedIn: http://www.linkedin.com/in/devendrashirbad
Post #1246982
Posted Monday, February 6, 2012 2:50 PM
Valued Member

Valued MemberValued MemberValued MemberValued MemberValued MemberValued MemberValued MemberValued Member

Group: General Forum Members
Last Login: Wednesday, July 16, 2014 6:25 PM
Points: 60, Visits: 248
Evil Kraig F (1/22/2012)
[quote]Peter Schott (1/22/2012)


So overrides were put in place in certain places, the cart was modified, etc etc. Eventually this cart proc became about 30 pages long, and was so convoluted that processing a cart to completion could take up to 10 seconds. This was unacceptable due to ever increasing volume order.

A full redesign of the structure, incorporating the new modules into the core structure, allowed for a much fuller and less intricate system of maintaining the data. This took 3 months of pure dev work, no new features, no ... you get the drift. It wasn't possible to do this in small doses, we had to redesign basically from scratch for all the new modular components that had been included.

I'm all for agile as long as the time is taken every few years to clean up the mess it leaves behind.

It's not because you can't plan for the scope, it's because the very NATURE of Agile is planned scope creep.



This also applies to DEV's comment:
I would like to add here that few of my colleagues read about Agile DEV from BOL or some BLOG and blindly started following it.


Agile development is hard. I can't see how anyone could do it without a lot of study and intense effort, hopefully with someone who knows the ropes and can guide you along. (No, I'm not a consultant. But I wish we could hire one; it would help a lot.)

A wealth of Agile "best practices" have been amply documented. Not studying these or applying them to your project is a recipe for disaster. I find that most people who think that Agile can't work, or have had a poor Agile experience, did not apply these best practices and often apparently don't even know what they are.

It's the same as if a DBA failed to learn and follow best database practices (which are out there to be learned, and many people can help you learn them), then blamed SQL Server for their project disasters. Understanding SQL Server best practices takes a lot of time, study, effort, and experience. But if you're not following them, it's not SQL Server's fault. If you think that implementing the practices takes too much time for your situation, well, good luck to you--but don't blame SQL Server when things go poorly.

And no, if you end up with a 30-page code module (be it SQL or c# or any other code in your project), that performs poorly and is difficult to understand, you're not following Agile best practices. "Refactoring" constantly, aggresively, is the best practice that avoids this situation. If you get good at it, which takes study and experience, it will speed up rather than slow down development and keep code quality high. If you think that refactoring and other best Agile practices takes too much time for your situation, well, good luck to you--but don't blame Agile development when things go poorly.

It's easy to find good sources of information for Agile development in general, but I know of only one person, Scott Ambler, who specifically addresses the database portion in depth. He even has a book specifically about refactoring databases. There have been large, complex projects that have had success using his ideas.



Post #1247720
Posted Tuesday, February 7, 2012 8:37 AM


Ten Centuries

Ten CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen CenturiesTen Centuries

Group: General Forum Members
Last Login: Wednesday, June 11, 2014 2:38 PM
Points: 1,426, Visits: 3,223
GSquared (1/9/2012)

But what most developers mean by "Agile" is actually "Cowboy". Database development can be quite Agile. It creates a horrible mess when it's done Cowboy-Coder style.


Eloquently stated sir.

I've seen a few of these cowboy designs. The thing that amazes me is it really doesn't take much longer to come up with a real design Vs. a hack. Any disparity reverses itself when you factor in the countless man hours needed to support the cowboy design (fixing bugs & hacking it even more to add features).




The probability of survival is inversely proportional to the angle of arrival.
Post #1248189
« Prev Topic | Next Topic »

Add to briefcase ««12

Permissions Expand / Collapse