My criticisms of variance analysis are directed at its use to analyse performance. This shouldn’t be taken to mean that I believe all forms of comparison are invalid. Quite the contrary. A number in isolation signifies nothing. It has to be compared to something else to mean something – the issue is what should it be compared to and what conclusions can be drawn from this?
Indeed, Part 2 of my book advocates methods based on comparing numbers from the same data series to:
- Expose patterns of behaviour
- Help spot changes in the level and nature of performance
- Quantify the level of noise and so by exception detect signals buried in the data Also, in my previous book (Future Ready) I strongly advocated systematically comparing actuals with forecasts, not to judge performance but as a means of testing and improving the models and assumptions on which the forecasts were based.
Variances between actual and forecast are a reflection of the reliability of the forecast not the quality of performance. I have devoted a lot of space to decrying the way in which targets are set and how they are compared to actual data, but there are ways to do both of these things that avoid most of the problems associated with simple variance analysis, as we will discover later.
Variance analyses are seductive
Variance analyses have gripped the corporate imagination because they are simple to calculate and to understand. They are also a seductive tool for senior executives since they offer the prospect of being able to direct performance without having to get involved in the management of the business: you just set targets once a year and administer systems of rewards and punishments to encourage compliance.
You can see why that might be popular! But perhaps another reason why variance analysis took hold in the 1920s was that the results could easily be communicated using the technology available at that time – paper and typewriters.
Tables of numbers are easy to produce and compress a lot of information into a relatively small space whereas charts had to be drawn manually and take up a lot of real estate on the page.
The problem with tables
Tables are an efficient way of cramming pieces of numerical data, like variances, onto a single sheet of paper, and all other things being equal the less space that you use the better. But does that make them an effective device for communicating information about performance?
Table 1.4 gives a summary of the simulated performance of a simple one-product company XYZ Ltd, using variances shown in a conventional table format. How easy is it to make sense of the analysis of performance in this table?
Put yourself in the position of the recipient of this report and use it to analyse the performance of XYZ Ltd. Try to answer these questions:
- Is this good or bad performance?
- Is it getting better or worse?
- Has anything significant just happened?
- How credible is the budget for next year?
How did you find it? If you are used to analysing numerical information you will probably find it easy to come up with some answers, but you might be less sure in them than you would have been before you read my criticisms of variance analysis.
And I’m confident that if you share this table with someone you will find that they come to slightly different conclusions than your own.
The other thing that I’m sure that you will have noticed is that trying to make sense of all this is hard work. You will have had to really concentrate in order to answer the questions I posed, and you might have noticed your eyes jumping around the table comparing one number with another as you tried to build up a picture of what was going on.
If you weren’t aware of this take another look at the table and retrace the path that your eyes took. Any lack of confidence you might have in your conclusions and the sense of effort you experienced is not because the task I set you is inherently difficult.
Our brains don’t work that way
My ‘toy’ company is hugely less complex than anything that you deal with in real life. These uncomfortable sensations are a sure sign that I chose a very poor way to communicate information. It simply isn’t a good fit with the way that our brains work.
The human brain has evolved over millions of years to efficiently assimilate information from our natural environment. This is why our visual perception is so much better developed than any other of our senses and why it is particularly good at spotting patterns and movement – provided information is presented in a way that appeals to our eyes.
In contrast, the symbolic systems we use in the West for communicating numbers emerged only 500 years ago when Leonardo Fibonacci introduced Arabic numbering systems into Europe.
Two conclusions, same numbers
We are born with highly-developed visual circuitry, but we have to laboriously train our brain to use numbers, which is why so much of our formal education is devoted to it, and – even then – some ‘well-educated’ people fail to acquire more than the most basic level of numeracy.
But even if you are highly numerically literate you will find it much more effortful to assimilate the numbers in the tables than if the same data was presented visually. And while two people may come to different conclusions about the meaning of a set of numbers there is much less chance of them perceiving the same shape in a different way, for the same reason: it is ‘easy’ and ‘more natural’.
And yet, despite our unsatisfactory personal experiences with tables and the ubiquity of computing power at our fingertips, performance reporting in business – particularly that produced by finance people – is still hugely reliant on tabular presentations and on decks of paper.
Thanks to technological advances our capability to collect data, to analyse it and to communicate information to an audience has increased enormously over the last few decades. But our chronic addiction to approaches created to solve the problems faced by the first professional managers nearly a century ago severely limits our ability to exploit this potential. This is wasteful and it imprisons our minds.
Our world is colourful, rich and full of life and ambiguity, but the ‘pictures’ of it that we create in our head are no better than bad caricatures. The problem is clear. The challenge is to work out what to do differently.
This is an extract from Steve Morlidge’s latest book Present Sense.