Building an evaluation strategy, by Martin Schmalenbach, Training Journal, April 2009

I have included this article because I know how much anything on the Holy Grail of learning evaluation is always appreciated by learning professionals, even if articles usually only remind us of the major challenges involved. This article is different from others because it focuses on the need for a strategy on evaluation - the direction to be taken - and therefore focuses very much on the questions that need to be asked and that are often overlooked.

One interesting question comes early in the article - should evaluation be pre or post the training? I have always assumed that it has to be post but, if you compare training evaluation with other types of investment appraisal such as capital expenditure, you could argue that it has to be before the training to fully support the decision. Of course any
pre-evaluation requires assumptions about future benefit but that is exactly the issue that should be addressed during the early stages. Also pre-evaluation makes post-evaluation that much more effective because it provides a basis for comparison, which is fundamental to good evaluation.

The author also poses another valid question that must be asked as part of the strategy - are you sure you really want to do this and why? To address evaluation as a broad issue before looking at individual programmes must be the right approach, rather than assuming that evaluation is a good thing per se. It allows an organisation to say that maybe we prefer to rely on our business judgment to assess priorities, rather than get into detailed analysis that does not produce reliable information. Or maybe we have to do so for certain types of programme and levels of management.

The article’s coverage of what you evaluate is less original and helpful; the author is really only repeating the Kirkpatrick framework when he suggests that a decision be made on whether to ask for reaction, learning, behaviour or organisational impact. He is however on more original ground when he asks the question - who should do the evaluation? At MTP we have often been interested that most organisations want us to evaluate ourselves - which of course we do fairly! - rather than having an internal process that ensures objectivity and independence.

Having made all these valid and interesting points, the article then rambles on rather too long by asking the same questions in a different way and produces a checklist as a framework for ensuring everything is covered - stakeholders, purpose, coverage, responsibilities, process and product, reporting and communicating.

However the author does end with another powerful point and a good reason for pushing forward an evaluation strategy - the impression that it creates for learning people as business partners with other functions, helping to ensure that they are seen to add value.

He also makes the point that pre-evaluation will often allow this business partnering to take place in such a way that mistakes are avoided. Too often, it is suggested, the training department takes criticism for supplying non-value adding training that partners in other functions have asked for!


To access this article go to http://www.trainingjournal.com/tj/2040.html