Is return on investment analysis really worth the effort? Donald H Taylor does not think so, and his article cautions fellow trainers and their clients to steer clear of this troublesome issue.
Wouldn’t it be wonderful if we could prove training’s value in money terms? Sure, really wonderful. So why don’t we do it?
A lack of technique isn’t stopping us. The approach of Jack Phillips at the ROI Institute extends Donald Kirkpatrick’s 1959 four levels of evaluation and is a robust ROI method. There are other techniques, too. So why aren’t they more widely used?
There are two reasons. Firstly, a practical reason – the ROI of training is time consuming and complicated to calculate. The second reason is a question of attitude and a fundamental show stopper. We’ll come to it later.
ROI basics
First, the basics: what exactly is Return on Investment? It is absolutely not a demonstration of cost saving, although many studies claiming to show ROI evaluations are nothing more than that – especially in the e-learning field. Rather, it is a way of showing what you get out for what you’ve put in.
Suppose you have installed some new equipment in a factory. The ROI calculation is clear: the plant cost you something, and your output increased after the installation. The ratio of the benefit to the cost is your ROI. ROI = incremental value / investment made. If your new piece of plant cost €10,000, and over a year produces an extra €2,000 worth of value, it will yield a ROI of 20% per annum.
Splendid isolation
People, though, aren’t machines. Making the same calculations for training is complex. Even for workers whose output you can measure, it is hard to prove that a change in their output is a direct result of training. There is too much else going on to easily isolate the effects of training.
If, after training, a group of employees shows an increase in productivity, is that down to the training? Not necessarily. Maybe the change would have happened anyway.
Suppose they are salespeople. Were their sales flat, or already on an upward path? If the latter, did the training result in an increase beyond the existing upward trend? Something other than training may be responsible for an increase in performance. Perhaps the employer made internal systems more efficient, or a competitor went bankrupt increasing demand.
To see whether these exogenous factors caused any increase in productivity, we need to divide our sales force into two groups, with only one receiving training. The second group should be as similar as possible in every way to the trained group. This includes factors such as their age and educational background to their locations, experience and knowledge. If we can set up a control group like this, and be sure what the historical productivity trend was, we can be fairly sure to identify any productivity increases stemming from training.
I say fairly sure, because there is still the “Hawthorne Effect” to consider – people increase productivity as a result of change or observation.
Neil Rackham and Huthwaite
If we can account for the Hawthorne effect, plus we have historical data and a control group, then ROI should be calculable.
It was only after several years of false starts that Neil Rackham, a scientifically rigorous research psychologist and founder of sales training company Huthwaite, was able to find an organisation willing to undergo a trial of whether a training programme could affect behaviour. This company was Motorola, in Canada.
The results of this trial did indeed show a significant increase in sales as a result of training. Even so, Rackham warns that “there are even more tests I’d like to carry out before I’ll be totally satisfied that the ideas I’ve described in this book will significantly improve the results of major sales.”
Rackham stood to gain considerably by proclaiming the validity of the training programme he had devised. If he felt less than totally convinced by his study, this is surely a lesson for the rest of us.
It is difficult to isolate the effects of training to a rigorous, scientifically defendable point. In addition, it is time consuming and it may not in the end be convincing to an expert in the field. But there’s a more important reason why training ROI is just not worth the effort.
The wrong answer
At the beginning of this piece, I said that there was a second, fundamental reason why people don’t carry out ROI studies on training. It is this – apart from training professionals, nobody really cares about training ROI.
The reason Rackham had trouble finding a company willing to be the guinea pig for his study was not just the complexity of setting up a representative control group, or the political difficulty of explaining to half a sales force why they weren’t getting trained. It was the constant response he received: “Look we’re pretty sure that the training works well enough. Skip the ROI and just get on with it”. In other words ROI is the wrong answer to the right question.
The question from managers and directors is: “How do I know this training is valuable to the organisation?” The answer to this question, from the training department, need not be “because I can show you, through this exhaustive ROI study, the increased monetary value of productivity that will result.”
So what do they want if they don’t want ROI? They want value. They want training (where training is the best solution) to make the organisation more effective.
The value proposition
Every organisation has operational pinch points. Training cannot improve all of them. However, where it will help there is usually a clear value proposition attached.
If there is a series of delayed project starts because of a lack of staff with key skills, targeted training can solve that. The value proposition: faster project starts. The time saved has a monetary value. That isn’t rigorous enough to be an ROI study, but it’s the sort of calculation that will justify most training spends.
Justifying ongoing training for staff, so that they can do their jobs, is harder. This is because the benefits are further down the line. The best answer is to have already engaged in a series of exercises, like those above, that demonstrate value. More important than any ROI study, that will establish in the minds of the organisation that you understand the business issues, and are using that understanding in building your training programme.
There are plenty of good proprietary methods that give a structure to such business-value based calculations for training impact. However, before beginning to justify your training, you may want to question why you are being asked for an ROI study.
Good and bad requests for ROI
There are two reasons why someone asks for an ROI justification for training. Usually, they don’t understand the complexity of a real ROI study as described above. Put them right on that, and then discuss the business problem they’re trying to solve. That will root out the value proposition and get you talking on their wavelength.
The other reason for an ROI request is more sinister – they know exactly how difficult it is. Asking for a study ensures a long period of quiet while that pesky training person goes away and tries to sort it out.
Don’t stand for it. Explain that a full study will need funding, and offer an alternative, value-based approach. Given the choice, anyone serious will opt for the latter. A business value proposition that relates directly to a perceived organisational issue is where the training department can make a real, clear difference. Identify and solve these business issues and I guarantee that nobody will ask you for an ROI analysis.
Reproduced by kind permission of TrainingZone, the UK’s largest training community
[…] Return on investment: Why ROI for training does not matter […]