Steve Jones - SSC Editor (11/24/2014)
Iwas Bornready (11/24/2014)
I'm a little confused. If you are talking about changes to production outside of a normal release, then this certainly is high risk, unless it goes through the same testing that a normal release would have. There will always be those emergencies where production updates can't wait until next month. But those off-the-cuff updates still have to be tested. The reality is that they are never tested enough.Yes, ad hoc changes that haven't gone through a testing cycle.
We have an abbreviated testing cycle for hot-fixes and/or emergency updates. Rather than going through the typical lifecycle, they are built up and deplyoed to MO (our pre-production environment, where we perform UAT and perf testing). This is one environment that is always at the same code level as production and hasn't been "polluted" with later builds that are still underway. An abbreviated regression, supplemented with specific tests to check the effectiveness of the patch against whatever is "burning", and an abbreviated stress test occurs before it heads to prod.
The challenge is then to merge those changes back into QA and dev once it goes up to prod. One lesson we've learned is to make sure you mergge BEFORE anything else deploys (so that the next "conventional" build doesn't reintroduce the production issue you're trying to fix); not so hard to do, as long as you remember the communication to the "main branch" team.
(edited for typos)
----------------------------------------------------------------------------------
Your lack of planning does not constitute an emergency on my part...unless you're my manager...or a director and above...or a really loud-spoken end-user..All right - what was my emergency again?