Deployment. Deployment. Deployment!

A good deployment scenario can mean the difference between a successful product and a failing one. The same goes when you are talking about the QA effort needed to verify and stabilize product during development. Let me paint you a picture of a product which had a bad deployment scenario.

It’s an enterprise product which has a 3 server setup: A database server, a webserver and a client installation. 3 machines are needed for every build and almost every verification of bugs, except super focused  bug fixes only occuring client side. Those were just very rare considering it was a 30 developer team. The procedure was that the QA team had a room with a secluded set of client machines, a DB server and a webserver. Every morning a build engineer would copy the daily build to a USB stick, move it to the QA room and install the nightly build on all the machines for QA to start work. This procedure would not be all that long, maybe an hour every morning, but QA would be tied to this build all day, because the previous days build was wiped in this process. And further, if there was a blocking bug being introduced during the night, the entire QA team would be unable to use the room for the day and the build from the previous day was also gone.

So why would a build engineer be needed to install this product? Well, yours truly decided to make an attempt at installing this beast of a system using the installation manual. 107 pages of detailed installing, configurating and tuning was necessary, which meant only very few people in the company were actually able to do it. And woe on anyone if something messed up during this procedure, because the ensuing debugging effort required detailed knowledge of the entire system, Oracle, Windows and a direct line to a few deities would probably also be needed.

All of this had led the previous management to conclude that the proper way of streamlining the department was to make a bottleneck out of the build engineers (those guys also built the installers for this) and make QA “efficient” by having it all setup so they didn’t have to know about all this. Which obviously is to make another flawed process to circumvent the real problem:

DEPLOYMENT!

What needed to be done here was to empower the QA engineers to install new builds by themselves, or at least in teams sharing some servers, so they wouldn’t rely so heavily on one build all the time. 

And the REAL issue of making an installer which preinstalled everything through commandline parameters was one I howled and screamed about, which led to each team having their own deployment developer, building installers in the very first sprint of a product. This was a costly move in the short term, because many developers suddenly had to learn WiX, but the long term value of having an agile team which had no dependency on a single build engineer was much, much higher.

This is not the only time I have encountered this deployment issue. In my time as business intelligence consultant I went ahead and produced WiX installers for the entire deployment and build pipeline for several projects. Nobody asked for it, because they didn’t know how bad they needed it, but once done they could all see the inherent increase in quality just because the turnaround from fix to verification became lower. And being able to remove the mess a manual deployment can incur was a very big quality bonus too.

While writing this I realize I have been involved in deployment in virtually every job I have had, because I find it to be so incredibly important for the software process as a whole and quality in particular. So if you find yourself working on a product with a complicated deployment scenario which is prone for user failure (“this is not a bug. The user needs to RTFM when installing it.” sounds familiar?), then you need to be very, very vocal about the need to simplify your installers.