Steve Jones - SSC Editor wrote: David.Poole wrote:
I think the important thing is to reflect on the lessons learned and ask what we would do better with the benefit of hindsight.
... The point is these are both addressable issues and in a no-blame culture these can be assigned priority appropriate to their level of risk.
Priority also based on cost/resources. I'm working on that latter part, because it's not just the risk and priority, but adding resources to better ensure these things are covered.
Heh... "Priority"... "Cost/Resources". These are things that should be a part of the culture of what people are claiming to be DevOps.
The best way to solve "Priority" issues is to not create issues by promoting bad code and breaking things that used to work. Period. That will also resolve "Cost/Resources" issues because you won't have unplanned for issues. 😉
Before you get in a huff, yes, I agree that no one is perfect and that's also why you also need to plan for rollbacks. Of course, the best rollback plans should never be needed but still have to be at the ready.
A part of "DevOps" IS having a proper test environment. You don't have one and that means that you're not a DevOps shop. Sorry, but, period.
I'm tickled that they finally fixed the issues that came about in the last several weeks (months?) and also understand the stress that it all caused. Learn to avoid the stress by doing it right 99.9% right the first time and be at the ready with a plan for the 0.1% that goes wrong. And, you folks haven't been. Sorry, but, period.
Even the first release of the new site was a complete joke. Many of us tested the hell out of the new site at your request (and we WERE seriously happy and honored to do so) before the release and many of us compared notes afterwards. Most of the faults and missing features that we cited were still released, right on bloody schedule :(. It was a waste of our time and a waste of your developers' time. The cite lost a lot of good, strong, members because of it.
Adopt some of what I call the basics of development regardless of what too-cool-for-school term you apply to the SDLC. Rule number one is do it right so you don't have to do it again, and again, and again. It will also help keep you from getting a black eye by your customers. It just doesn't take that much longer to do things right but it takes a hell of a lot longer to do things more than once.
Rule number 2 is that you have to have a test environment that very, very closely resembles the production database in order to make Rule 1 actually happen. You don't happen to know of a company that offers any database cloning software, DO YA? 😀
Rule number 3... if management wants it real bad, that's the way they'll get it. You been through this many times on this site and have seen many forum posts that cite the same thing. Fight the schedule push if something is wrong or going wrong. Heh... and only Rule 2 will tell you that.
And, please... don't YOU take any of this personally. You're not the one in control of any of this. You're only the messenger (although an important, credible messenger) and I hope you take this message back to management. This site isn't important just to the community... it's important to RedGate as a company. It generates awareness of RedGate, has many trusted members like yourself, Grant, Kathi, Kendra, and more, and all of that helps generate revenue because... you're all motivated to "do it right".