The Amazon Way

  • The Amazon Way

    This is rather interesting, the philosophy of Amazon: you build it; you run it. At least that's the philosophy of Wener Vogels, the CTO up there. He basically feels that if someone builds a piece of software, they should be responsible for running it. For fixing it, for dealing with the problems they didn't think of when it was engineered (term used loosely), etc.

    It's not a bad philosophy. I've worked in this type of atmosphere where the developers went ahead and supported their code in production. I've also worked in places where there was a dedicated Operations team and a dedicated development/engineering team and neither worked on either's systems.

    Both have their problems. In a separate world, the engineers and developers don't have to live with their mistakes and they feel less pressure to fix things. Some people are extremely professional, take it personally, and live the pain with their operational counterparts. Some say deal with it and I'll look at it when I get the chance, and most are somewhere in between. It's a hard system.

    On the other hand, the Amazon way encourages people to "patch things up" instead of truly doing a good job re-architecting them. There's pressure to move forward and so you tend to spackle, touch-up, and cover-up your old mistakes so you can move forward onto the new projects.

    In a way, it makes the shrinkwrap guys seem enviable. They often "end of life" things and can forget about them, moving to the next version. And the company is happy to do that because it forces everyone to upgrade.

    Personally I think you need a hybrid environment. At one of my jobs I tried to get the Operations group to rotate someone every quarter to engineering and vice versa. However engineering never wants that because face it, it's a cushy job, less deadline pressure, and certainly less pressure when something you're working on doesn't work. Sure you get pushed to complete projects, but delays in engineering are nowhere near as problematic as failures in production.

    And don't get me started on the whole security aspect of this...

    Steve Jones

  • Different strokes for different folks. 

    In the first company in which I worked, 80% of the team left in the 6 months after version 1.0 was completed and they realised the thrill of building something new was gone and they'd be supporting it for as long as they worked there.  It wasn't what they wanted to do; it wasn't what they could get the best rate for; they moved.

    Since then, the developers I've known would frequently leave an organisation rather than support what they'd written.  I'm typically on bleeding-edge development, and the staff you recruit for such a team are often ex-researchers, with a good architect and some good engineers to keep some feet on the ground.  These people get their kicks from building something new, not supporting it.

    If Amazon are able to get their team to develop and support, I suspect they've been very careful about their recruitment process.  Equally, not having a bunch of bleeding-edge developers put your line-of-business systems together can be an advantage!

  • I enjoyed reading your short article and I would like to make two statements:

    First, I'm amazed that companies would even consider NOT having the original developers maintain the application.  In every company that I've worked for the last 15 years, the developers always maintain the application - never heard of the other scenario you describe.

    Second, supporting an application after its initial release can actually be enjoyable.  For example, you don't have to attend countless meetings with the business team, the sales team, the executive team, the team of teams...you get the idea.  After the initial release, a developer tends to spend much more time in development and less time trying to figure out how to please everyone.  Deadlines are more relaxed and holidays are a given, not a privilege.


    Kindest Regards,

    Tim Hodgson
    exclamationosft.com

  • Here the way it is done is there is a development team and a support team. After a new app is developed and in production for a month it is turned over to the support team and then the development team tackles another project. This is good for the developers but I think the product and the customers suffer as there is a learning curve for the support team since they don't know the system and have to learn it as they debug it. I don't agree with this process but that is how it is done here.

  • I'm curious how they deal with SOX segregation of duties requirements? SOX auditors prefer developers not having access to production systems.

  • I worked for a company where the development manager worked for the support manager.  Each, and every, developer was required to work so many hours per week on the support desk.  Even the VP's worked some hours each week on the support desk.

    Innovation and enhancements came directly from the needs of the customers.  We had a policy that encouraged the customer to call support with the slighest question.  We knew, minute by minute, the needs of the users.

    I often say, "The company pays my salary and the customers pay us, but the people that I work for are the ones actually using the software."

    You have to support what you write.  Also I have to be able to take vacations and do other things.  If no one else knows my code then I'm trapped.  Coding standards and style guides go a long way to reducing this.

     

    ATBCharles Kincaid

  • In the shop I operate in we have been steadily moving from a .com environment where devs are actively involved in the support role (mainly cos they are the only ones that know how things are supposed to work) towards a truly segregated dev / test / operate model where everything is split up (you can thank sox for that).

    As devs we get to do the complex stuff we had problems with before cos we've lost the distractions, but on the other hand the complex stuff is harder to define because most of the dev's have lost sight of the front line.

    I think overall I prefer the former approach with the caveat that you have to have very very good devs who are well rewarded, well motivated (and hence don't mind the 3am calls so much) and highly empowered to enact change. It doesn't work with your average corporate developer at all.

    A favourite philosophy of mine, which may be relavant to this, is that if you have a boring operational task give it to the best guy. And I think this is what the amazon guy is getting at. If I have daily issue (being data-oriented my teams issues tend to be data quality, ref data stuff etc.) and I give it to an average dev they will do the job.

    My best guys are busy on interesting stuff so it seems like a good allocation of resource.

    However the average joe will keep doing the task I've given them until I tell them to stop. Ad infinitum.

    A well motivated superstar won't do the job. Maybe they'll do it on the first day. Maybe even a few days. At this point they'll say 'I can make the computer do this job for me!' (after all thats what they were built for).

    This is how you get nicely architected seamless operations. The clever guy will find a way for anything boring and distracting to go away, or at least make the apps so supportable that issues are resolved in near-zero time.

    In my past days as a developer I've often had operational problems that have caused me to add some logging here, better error handling there as and when a problem shows itself. If you have seperation between dev and ops this doesn't happen. Instead you have the ops guys saying 'log everything' so they can fathom problems without realising that 99.9% of the log will be garbage.

    Anyway, it at least keeps the support guys busy and out of the way while someone in architecture gets in there and solves the actual problem.

    Ramble over.

    JZN.

  • There is a problem with the same developer supports the application all the time.  If anything happens, there is no backup.  It happened to my last company, one developer supported one system, when that person on vacation or left, it would take some one a while to pick it up.  

    I always feel cross training is important.  When I worked for a software development company, I developed the software, I supported the software for beta release.  Then when it finally released, I turned over to supporting team.  In the beginning they would come and ask me question, but eventually they could support the system.  Even within their group, they cross trained a couple people to support the same system.

    Don't let one person take over one system, the developer calls it 'JOB SECURITY', also it is 'COMPANY DISASTER'.

    my 2 cents.

  • Right on, Loner.  I've been in half a dozen meetings today, as typical, and did not express that part of my thought well.

    ATBCharles Kincaid

Viewing 9 posts - 1 through 8 (of 8 total)

You must be logged in to reply to this topic. Login to reply