Thursday, 13 December 2012

The Challenge of Measurement

- Mr.Jagadish Shri

Mr. Jagadish manages our program in Mysore. He has over 20 years of experience in the IT sector.

In any project the question of measuring progress comes up at one time or another. There is an interesting book on this topic by Robert D. Austin which talks about how the goals which are set tend to distort the behavior of individuals and teams.

The lower level attributes are always easier to measure. For eg: In our program it is easy to see if a school is logging into WebBox or not. This metric is convenient to measure and follow up with schools at the early stage of the project. But once all schools have started logging into the WebBox on a regular basis this does not have the same value. It becomes more important to understand what they are doing after logging in. Should we look at the amount of time that they have logged in? Should we look at what all sections within the V-Class the teacher has accessed and so on. As you can see things become more complicated both in terms of defining what to measure, setting systems in place to measure it with the least effort and subsequently analyzing the same to derive meaningful results which can help make systemic corrections. There is no end in sight if we take this path of measurements.

Finally one needs to be conscious of the fact that all metrics however well thought out are only indicators. They cannot be substitutes to what in 'Lean' software development is known as the "Genchi Genbutsu" principle or "go see yourself". This simply means that one has to visit the actual place of action where the value is getting created and be a silent observer. In case of 'Learn, Out of the Box' it would be the classroom where learning is supposed to happen and see what happens when the teacher uses the WebBox. The insights derived from this will go far beyond what any metrics will capture. This definitely involves effort but it is worth it.