I always look out for articles on evaluation of learning for two connected reasons. First because our clients like to know if there is anything new on this topic; second because we are all looking for that Holy Grail that takes us beyond the work of Kirkpatrick and provides better answers to the much asked question; is this training providing a return on investment?
I would like to say that the author helps us in our search but sadly he offers little that is new. Nevertheless he does add some substance to those sceptics who say that the above question is so difficult to answer that maybe we should stop asking it. Mr Lovell achieves this by spending the first page of his article telling us that measuring training effectiveness is difficult, expressing this obvious point in a number of different ways.
He also mentions that there are a range of different evaluation methods, 'from NPS to the Kirkpatrick model'. This caused me immense irritation because, though I have written about and debated training evaluation for many years, I do not know what NPS is. My financial background took me immediately to Net Proceeds of Sales which did not seem to fit. So I Googled and found the Nigerian Police Service and the National Philatelic Society but nothing to do with training. It may show my ignorance but I am unlikely to be the only one who is flummoxed and annoyed.
The author then goes on to make the interesting claim that 'many Learning & Development professionals believe that learning delivers more value than they can demonstrate'. This is an interesting proposition which is probably true though it begs the question as to whether they are right. It may be that most of us are born optimists or feel that we need to behave that way to convince others. Unfortunately the author does not go on to explain why this happens or why it is important.
Too much of the first half of the article is made up of statements of the blindingly obvious; evaluation is desirable but difficult, budgets are likely to be cut if value is not proved, there are many other uncontrollable variables that impact the transfer to job performance. The only positive suggestion is that training objectives should be linked to business goals, hardly a new breakthrough in thinking.
The author then moves on to suggest three steps; the first is that you have to make a case for investment in training and assemble all the information required, again not exactly earth shattering stuff. The only interesting recommendation is that we should look positively for anecdotal evidence and share it around. This may seem obvious but I can relate to it; when we worked with the old ICI business, we ran a follow up to a finance programme and asked participants for ‘critical incidents’ when they had used course content to add value. We were favourably surprised with what they came up with, including several benefits that far exceeded the cost of the course. It is of course difficult to separate the genuine direct benefits from the contrived ones but such anecdotes can be highly useful with sceptical senior managers who want to cut budgets.
Another important point made by the author is that any attempts to use the methods of 'Kirkpatrick, Phillips and others' (I would be interested to know who the 'others' are) are always likely to be too late to create interest and impact decisions; senior managers and sponsors want to know about last week’s course not last year’s. But he is less convincing when suggesting what you can do to create immediate impact, apart from some rather obvious questions about the business issues and strategic goals.
The other two steps are similarly obvious; adopt a holistic approach when justifying training and get your priorities straight. This means looking outside the one course you are assessing and comparing the investment and the impact with alternatives, including those managed by other claimants to training budgets. There is an interesting classification of training priorities between Mandatory, Vital, Important and Elective which is a useful framework for making choices; I might have added a fifth category that is ‘nice to have’ or ‘could be delayed’.
The article concludes by restating the three steps in a way that is a good summary but only confirms the obvious nature of the author’s recommendations:
- Show that learning is integral to organisational goals
- Have a comprehensive understanding of training in the organisation
- Identify areas of importance to strategic goals.
For those who, like, me, have been looking for the Holy Grail for many years, this article only confirms the extent of the challenge. For those who are just starting out on their search, it represents a useful summary of the key issues and the barriers to be overcome.
To read the article in full click here
http://www.trainingjournal.com/feature/2012-08-01-evaluating-learning/
