Monday, April 20, 2009

First Cut at Metrics

As I’ve written before, we have been trying to identify metrics that will give us an idea of how much under control our process is. For that reason we decided to collect velocity, number of stories with definition of done, number of impediments per day, number of “distractions” and total number of team hours available.

Even though velocity is not really comparable across teams and is easily gamed, it is what we would like to “control”. Velocity is a measure of capacity and we would like it to improve it or at least keep it constant.

The team feels that the other metrics would be good control variables.

Total number of team hours available: the team cannot produce more if they are having fun on the beach ;). This variable is calculated by adding up the number of hours that was worked by each team member regardless if in the project or not.

Ratio of tasks or stories with definition of done: We have experienced the joy of having user stories with great acceptance tests once, and the productivity gains that comes from it (thanks Rhonda Dillon!).

I strongly believe if there’s something that positively influences productivity it is good acceptance tests, or the power of knowing what is expected of you. One problem though is that using the acceptance test moniker, might be limiting and people might get confused (What is the acceptance test for deploying the application to X environment?). That’s why we settled on definition of done instead. I could possibly spend hours talking about the benefits of creating test cases before starting the work.

This metric is calculated by dividing the number of stories with definition of done by the total number of stories.

So far we discussed the “positive” control variables, now let’s get to the “negative” ones.

The number of “distractions” is supposed to measure the time that team members spend chasing issues unrelated to the iteration goals like production problems, supporting other developers using the platform, etc. We deliberately stayed away from tracking time, since we believed it would be not as accurate and would be redundant to the company’s official tracking system. The problem is that we don’t control the time tracking system, Finance does.

The last control variable I listed above is number of impediments per day. It measures stories that might have had a clear definition of done, but were held up for some reason or another. This will tell us if we had “idle” resources (never really true) for some kind of reason.

Next challenge: Where/How to collect this information and how to present it properly?

Wednesday, April 8, 2009

Metrics and Productivity are not the same thing!

In my quest to satisfy a management request of coming up with metrics I came across Ron Jeffries fabulous article about increasing productivity.

I couldn't agree more with his perspective that increasing productivity and metrics are not exactly coupled and that it is more important to adapt what you inspect to the problems you are trying to address. In other words, the metrics that you are watching now are probably different from the metrics you’ll watch for next iteration. That is, of course, if you fixed the problem.

What brings up the question, should we put monitors on metrics that we collect? In other words, if we decided that the number of hours spent on supporting deployments is too high, should we continuously monitor it to prevent it from ever being a problem again? If the metric in question is derived from some automated process, I’m absolutely in favor of it.

Be aware though that metrics should not hinder productivity. In other words, don’t bring down your team velocity so that you can collect . Whatever you decide to measure should be derived from some automated process you run (your build process preferably).

If you are in a Java shop, take a look at Maven and Sonar. These will provide you a lot of information in a single web page.

Tuesday, April 7, 2009

Agile Metrics

We all know that metrics can be bad, specially if used for performance evaluations, but we need to be accountable as well.

So here I start my journey on how and what to measure for an agile team.

Here are a few links to start with: