Umesh Vyas

Home     BPO Perspectives    Other Articles    About Me    Contact Me


Management by Rolling Dice or Tyranny of Numbers

An obsession has gripped the Indian BPO Industry. It is the obsession of managing individual performance through metrics. Although measurement per se is useful, some of the individual measures used are as reliable as rolling dice. Unwisely used, numbers can create a tyranny, without improvement. Unless the focus is quickly shifted to the process and the system, current performance management is likely to turn into a specter that will haunt the industry for years to come.

Let us consider the measurement of Quality, and examine what is happening. Quality is measured through monitoring. Each individual is monitored about 1 to 10 times a month. Each transaction monitored is scored. Individuals are given feedback based on these scores. These scores are used to evaluate performance. Rewards or punishment follows.

So what is wrong with this? Well, let us listen to some voices.

Agent: ‘Everything was fine. I was doing very well. And then I got a bad transaction. And that was the one they picked to monitor. What do I do?’

Agent: ‘This new monitor is very strict. Holier than thou! I miss the old one. She was nice and lenient with the scores. I do not know how to improve my score with this new monitor. What do I do?’

TL: ‘We have to be careful with the scores these monitors give. They do not know the practical challenges and intricacies of the process. We have to watch out to ensure that scoring is fair.’

Manager: ‘Of course we focus on Quality. We have now linked incentives to the Quality scores. Everyone knows that they will lose money if they make defects. And even after this if they fail to perform we classify them as low performing outliers and put them through remedial training.’

As we are talking about Quality, let us also listen to one of the undisputed Gurus – Edward Deming.

Deming Principle 11: ‘Eliminate work standards (quotas) on the factory floor. Substitute leadership. Eliminate management by objectives. Eliminate management by numbers, numerical goals. Substitute leadership.’

Let us now put it all together.

What the agents are complaining about is the reliability of the measurement system being used to evaluate them. The first statement is questioning sampling. And the second is questioning calibration. There may be an element of defensiveness, but fundamentally they are right. Current monitoring methods do not produce reliable results to measure individual performance.

Even if we consider an individual sample of 10 transactions, the sampling error is likely to be about 20%. And this is small compared to some of the calibration errors we have seen, up to about 50%. Combine these two errors, and you can see what the agent is questioning. There is little statistical reliability of the sample.

What is interesting, however, is that when the measure is rolled up at the process level, the sampling error reduces to less than 3%. What this implies is that current sampling is unreliable for individual measures but reliable for measuring at the process level. No wonder Deming said what he did. But there are other reasons why he wanted to banish individual measures. Let us examine the other voices to understand him better.

The TL is simultaneously highlighting lack of calibration and focus on individual evaluation. So individual evaluation has overpowered process improvement. The Manager has, of course, given up on leadership. He has substituted leadership with incentives. Despite strong historic evidence on intrinsic motivation and quality, the manager believes that quality can be produced through incentives. This is the biggest harm that individual measurement and evaluation is doing. If we take this to its logical conclusion, company’s share price can be put on the agent’s scorecard and management can relax! Or be declared redundant!

So what is the way out.

The first step is recognize the measurement for what it is. It is a measure of Process Capability. Nothing less and nothing more. Individual performances may vary, but the fundamental variation is the system variation. Hence, rather then looking at the rolling dice that evaluate individuals, look at the control charts that evaluate the process. The process will have natural variation. However, it will continue to operate within the limits of its capability, unless some systemic steps are taken to reduce the variation and improve performance.

So the second step is to find systemic root causes. And then work on them. Whether you use PDCA or PSP or DMAIC, the answers are in the process. And managers who only manage outliers, should be made to lie out.

Hence, the biggest challenge is to shift the focus from individual measurement to process improvement. This is what will yield benefits.