Deploy Every Iteration?

How would you change your software development process if you had to deploy to production at the end of every iteration? Do you see yourself requiring "hardening iterations"?

Deploying every iteration is not for the faint of heart. You need to ensure that the software you are releasing is solid and integrates well with the other pieces. It requires you to have your engineering practices really down, reducing the time you spend in doing the release (a.k.a. hardening iterations) as much as possible.

If you look at the activities that happen in a hardening iteration to enable a release, you generally have building the software, reviewing code, regression plus load testing and at last deploying it. This set of activities will probably grow or shrink depending on the regulatory environment you are in.

To reduce the time you spend building the software you need to automate your build process as much as you can. In the Java space you can use tools like Ant and Maven, for .Net Nant is an alternative. Maven is useful because besides just automating the build it generates a nice web site with reports about the software you're building.

Once you have a good build, i.e. no compilation issues, you need to check it to make sure it has no regressions or bugs. This is probably where automation will give you the most benefits. The shorter you can get your regression tests to take, the more time you will have to actually write code. There are plenty of tools out there that will let you automate your tests. If you are writing web applications you might want to look at Selenium. Regression tests are not the only requisite to deployment in some shops, you might also need to load test your software to make sure you still satisfy your performance service level agreements.

Finally you need a fast and easy way of deploying your application. Here I have no tool advice for you. On the bright side, if you are releasing every iteration you won't have a lot to deploy and thus it should be quick and easy. The longer you code, the more complex your deployment might be.

As you add the three automation pieces above, consider implementing a continuous integration process. It would allow you to eventually have code travel from the software developer mind to production in no time! A few free tools in this space are Hudson, CruiseControl, Continuum, etc.

In an Utopian world, every time a software developer commit a change to your Source Code Management system (you have one of those, right?), you would repeat the whole process automatically and send the software to production. I do not recommend you to do this, except the simplest of the cases!

As you can see, if you inspect your deployment process constantly, you will be able to eventually spend little to no time on "hardening" your software for deployment. Repetitive tasks should be left to a computer, not to a highly paid professional ;)

People Over Process and Tools

People and Interactions Over Processes and Tools. In other words: Brain over Fingers.

If Agile values people over processes and tools, why does it require so much automation? Reality is that people is good at creative tasks, so in order to really use their potential we need to free people from tedious and sequential tasks. People: We need your brains not your fingers!

Perhaps my biggest challenge implementing Agile so far has been convincing people to let go of their repetitive tasks and automate their processes. This is not always the case with software development professionals, but in general it is the case with the other "delivery functions" (QA, Operations, Project Management).

I'm not sure if it is related to people's background, but I find that QA analysts and System Engineers that were once developers are more receptive to automation and have greater success with it.

It is not hard to convince a developer to automate his build process for example. In fact, I haven't seen a manual build in a long time now. That's definitely not the case with QA Analysts and automated test cases or System Engineers and their deployment or monitoring processes.

The way I normally approach this issue is by selling the benefits to the professional in question. I show engineers and analysts how much easier and pleasurable their lives would be if they didn't have to do the same repetitive task over and over again. For example, I tell QA analysts how much more challenging their job would be if instead of executing their tests if they spent most of their time coming up with new test cases. In this case, we need QA analysts to use their brain to come up with test cases and not fingers to type or click the mouse.

By shifting the work of the QA analyst to test case development and automation we are now using their brains and not only their fingers. 

It is true that automating test cases might require as much software development skill as coding the system being tested. In cases like this, I'd have the QA analyst define the test cases and have developers automate them. I found that QA analysts are sometimes comfortable reading the test code and are willing to "audit" the automated tests to make sure they are indeed covering their intended cases.

Deployments is another area of the software development process that is ripe for automation. How much better life would be if instead of spending a few hours deploying an application in the middle of the night, System Engineers spent those hours coming up with scripts that would deploy the application in minutes? What if they spent time figuring out how to break the deployment apart enough to be able to have all of the components in production inactive ahead of the deployment? These are tasks that require brains and not fingers.

Once again, we need your brains and not your fingers!

First Cut at Metrics

As I’ve written before, we have been trying to identify metrics that will give us an idea of how much under control our process is. For that reason we decided to collect velocity, number of stories with definition of done, number of impediments per day, number of “distractions” and total number of team hours available.

Even though velocity is not really comparable across teams and is easily gamed, it is what we would like to “control”. Velocity is a measure of capacity and we would like it to improve it or at least keep it constant.

The team feels that the other metrics would be good control variables.

Total number of team hours available: the team cannot produce more if they are having fun on the beach ;). This variable is calculated by adding up the number of hours that was worked by each team member regardless if in the project or not.

Ratio of tasks or stories with definition of done: We have experienced the joy of having user stories with great acceptance tests once, and the productivity gains that comes from it (thanks Rhonda Dillon!).

I strongly believe if there’s something that positively influences productivity it is good acceptance tests, or the power of knowing what is expected of you. One problem though is that using the acceptance test moniker, might be limiting and people might get confused (What is the acceptance test for deploying the application to X environment?). That’s why we settled on definition of done instead. I could possibly spend hours talking about the benefits of creating test cases before starting the work.

This metric is calculated by dividing the number of stories with definition of done by the total number of stories.

So far we discussed the “positive” control variables, now let’s get to the “negative” ones.

The number of “distractions” is supposed to measure the time that team members spend chasing issues unrelated to the iteration goals like production problems, supporting other developers using the platform, etc. We deliberately stayed away from tracking time, since we believed it would be not as accurate and would be redundant to the company’s official tracking system. The problem is that we don’t control the time tracking system, Finance does.

The last control variable I listed above is number of impediments per day. It measures stories that might have had a clear definition of done, but were held up for some reason or another. This will tell us if we had “idle” resources (never really true) for some kind of reason.

Next challenge: Where/How to collect this information and how to present it properly?

Metrics and Productivity are not the same thing!

In my quest to satisfy a management request of coming up with metrics I came across Ron Jeffries fabulous article about increasing productivity.

I couldn't agree more with his perspective that increasing productivity and metrics are not exactly coupled and that it is more important to adapt what you inspect to the problems you are trying to address. In other words, the metrics that you are watching now are probably different from the metrics you’ll watch for next iteration. That is, of course, if you fixed the problem.

What brings up the question, should we put monitors on metrics that we collect? In other words, if we decided that the number of hours spent on supporting deployments is too high, should we continuously monitor it to prevent it from ever being a problem again? If the metric in question is derived from some automated process, I’m absolutely in favor of it.

Be aware though that metrics should not hinder productivity. In other words, don’t bring down your team velocity so that you can collect . Whatever you decide to measure should be derived from some automated process you run (your build process preferably).

If you are in a Java shop, take a look at Maven and Sonar. These will provide you a lot of information in a single web page.

Agile Metrics

We all know that metrics can be bad, specially if used for performance evaluations, but we need to be accountable as well.

So here I start my journey on how and what to measure for an agile team.

Here are a few links to start with:

NCSS to Function Points conversion table

Measuring Agility: Top 5 Metrics and Myths or the stream can be found here