First Cut at Metrics

As I’ve written before, we have been trying to identify metrics that will give us an idea of how much under control our process is. For that reason we decided to collect velocity, number of stories with definition of done, number of impediments per day, number of “distractions” and total number of team hours available.

Even though velocity is not really comparable across teams and is easily gamed, it is what we would like to “control”. Velocity is a measure of capacity and we would like it to improve it or at least keep it constant.

The team feels that the other metrics would be good control variables.

Total number of team hours available: the team cannot produce more if they are having fun on the beach ;). This variable is calculated by adding up the number of hours that was worked by each team member regardless if in the project or not.

Ratio of tasks or stories with definition of done: We have experienced the joy of having user stories with great acceptance tests once, and the productivity gains that comes from it (thanks Rhonda Dillon!).

I strongly believe if there’s something that positively influences productivity it is good acceptance tests, or the power of knowing what is expected of you. One problem though is that using the acceptance test moniker, might be limiting and people might get confused (What is the acceptance test for deploying the application to X environment?). That’s why we settled on definition of done instead. I could possibly spend hours talking about the benefits of creating test cases before starting the work.

This metric is calculated by dividing the number of stories with definition of done by the total number of stories.

So far we discussed the “positive” control variables, now let’s get to the “negative” ones.

The number of “distractions” is supposed to measure the time that team members spend chasing issues unrelated to the iteration goals like production problems, supporting other developers using the platform, etc. We deliberately stayed away from tracking time, since we believed it would be not as accurate and would be redundant to the company’s official tracking system. The problem is that we don’t control the time tracking system, Finance does.

The last control variable I listed above is number of impediments per day. It measures stories that might have had a clear definition of done, but were held up for some reason or another. This will tell us if we had “idle” resources (never really true) for some kind of reason.

Next challenge: Where/How to collect this information and how to present it properly?