‘Putting a value on training’ by Jenny Cermak and Monica McGurk, McKinsey Quarterly, March 2010

I have not featured McKinsey’s respected publication before because it is unusual for their topics to include anything connected with management learning. Evaluation of training is a topic that I always try to feature because so many Learning & Development professionals are grappling with the issue. However, I have to confess that, because of the superficial nature of many of the articles on the topic, my enthusiasm and belief in this ‘holy grail’ is waning. This article did not do much to restore my faith.

The article starts promisingly by quoting some statistics from McKinsey’s research; that only 25% of companies believe that their training programmes increase performance in measurable ways and only 8% track ROI. My initial reaction was that, based on our own experiences, the level of 8% seems high, assuming that we are talking about a quantified financial calculation. On the other hand the 25% seems low because most companies believe in what they are doing, however difficult it may be to justify and quantify.

It was disappointing that the main case study of the article is based on a not for profit organisation – the Boys and Girls Clubs of America. This is not to suggest that such organisations do not have good management practices that we can learn from; it is just that you would have expected a consultancy with the reputation of McKinsey to produce an example from one of the world’s top companies.

As is usually the case with evaluation projects, the starting point was good and the intentions sound. They developed a training program for their local leaders following a 360 degree assessment of each person’s performance. They then used regression analysis to produce correlations between each leader’s 360 result and the KPIs used to measure their performance. This resulted in four key areas being isolated as fundamental to success and therefore the main topics of a leadership development programme.

I would like to have learnt more about the methodology of this analysis but at least it showed that the starting point was sound and that there were clear measurable objectives, always the necessary basis for any measurement process. However, it is the post programme measurement that is the difficult bit and this was less convincing.

The authors state that the next stage was ‘straightforward’ – which worried me a great deal because evaluating training is never straightforward – and involved ‘comparison of pre and post training results’. However they were very vague about how this crucial process was carried out. They looked to measure improvement in the relevant KPIs but they inevitably came up against the classic question; what other factors could have had an impact on performance improvement? They claim to have compared the post training results against a ‘control set of organisations’ which had similar characteristics but which had not gone through the training; they also factored in the ‘impact of external factors’. Finally they used interviews to make more qualitative assessments of behaviour changes.

Apart from a feeling that the authors were glossing over a highly complex process, I also began to wonder about the cost of the evaluation process itself and how this could be justified by the benefits. As is usually the case with attempts at evaluation, there is an assumption that it is an end in itself, with little attention to what will be done with the results once they have been evaluated.. The claimed outcome was that the ROI showed a return that was four times the investment but would this knowledge justify the time and cost of the process?

One interesting and positive feature of McKinsey articles is that they show on their website the feedback from readers, both favourable and otherwise. The responses generally shared my reservations and there was criticism of the ‘dated’ approach of talking about training rather than learning. One comment was, I thought, quite profound and applicable to many companies who do not think through a learning strategy; ‘Management is ready to spend millions on training but is not able to give its time’ …

Click here to read the article in full;
http://www.mckinseyquarterly.com/Organization/Talent/Putting_a_value_on_training_2634?gp=1#