The MTP Business Learning Blog

This blog is produced by MTP for senior professionals highlighting relevant and interesting books and articles on business, finance and strategy, and the opportunity to comment on them. It also contains news of MTP and its clients and, from time to time, extracts from MTP publications.

Friday 8 May 2009

Building an evaluation strategy, by Martin Schmalenbach, Training Journal, April 2009

I have included this article because I know how much anything on the Holy Grail of learning evaluation is always appreciated by learning professionals, even if articles usually only remind us of the major challenges involved. This article is different from others because it focuses on the need for a strategy on evaluation - the direction to be taken - and therefore focuses very much on the questions that need to be asked and that are often overlooked.

One interesting question comes early in the article - should evaluation be pre or post the training? I have always assumed that it has to be post but, if you compare training evaluation with other types of investment appraisal such as capital expenditure, you could argue that it has to be before the training to fully support the decision. Of course any
pre-evaluation requires assumptions about future benefit but that is exactly the issue that should be addressed during the early stages. Also pre-evaluation makes post-evaluation that much more effective because it provides a basis for comparison, which is fundamental to good evaluation.

The author also poses another valid question that must be asked as part of the strategy - are you sure you really want to do this and why? To address evaluation as a broad issue before looking at individual programmes must be the right approach, rather than assuming that evaluation is a good thing per se. It allows an organisation to say that maybe we prefer to rely on our business judgment to assess priorities, rather than get into detailed analysis that does not produce reliable information. Or maybe we have to do so for certain types of programme and levels of management.

The article’s coverage of what you evaluate is less original and helpful; the author is really only repeating the Kirkpatrick framework when he suggests that a decision be made on whether to ask for reaction, learning, behaviour or organisational impact. He is however on more original ground when he asks the question - who should do the evaluation? At MTP we have often been interested that most organisations want us to evaluate ourselves - which of course we do fairly! - rather than having an internal process that ensures objectivity and independence.

Having made all these valid and interesting points, the article then rambles on rather too long by asking the same questions in a different way and produces a checklist as a framework for ensuring everything is covered - stakeholders, purpose, coverage, responsibilities, process and product, reporting and communicating.

However the author does end with another powerful point and a good reason for pushing forward an evaluation strategy - the impression that it creates for learning people as business partners with other functions, helping to ensure that they are seen to add value.

He also makes the point that pre-evaluation will often allow this business partnering to take place in such a way that mistakes are avoided. Too often, it is suggested, the training department takes criticism for supplying non-value adding training that partners in other functions have asked for!


To access this article go to http://www.trainingjournal.com/tj/2040.html

Predicting your competitor's reaction, by Ken Favaro, Tim Romberger and David Meer, Harvard Business Review, April 2009

This article covers some fascinating research by McKinsey which challenges some of the conventional wisdom around the ways that managers assess competitor reaction to new innovations and initiatives. It starts by quantifying what we have often observed in our work with a range of companies - that very few of the managers surveyed recalled having brought competitor reactions into their decision-making processes.

Many of the more sophisticated companies would respond by pointing out their use of game theory and War Games as evidence of their practice of predicting competitor reaction. However the authors are sceptical of both these approaches; in the case of game theory this is because they say it becomes unmanageable in the real world, particularly when there are multiple competitors with multiple options.

The criticism of War Games is that the assumptions are inevitably arbitrary and are likely to miss out on the subtleties of politics and personalities involved in real-life decisions. A more valid point in my view is the fact that War Games may over-estimate the likelihood of significant response, because there is an implied assumption that the innovation will be noticed and reacted to.

Maybe this is because there is a tendency for companies to think that they are bigger players than they really are. McKinsey’s research confirms that, despite what companies expect, many innovations are not noticed at all by competitors, which would of course make for a short and boring War Game session!

The article does disappoint by being less good at putting forward solutions compared to knocking down other approaches. It does however suggest four fundamental questions that may seem blindingly obvious but which are a good discipline and structure:

- Will the competitors see your actions?
- Will they feel threatened?
- Will mounting a response be a priority?
- Will they overcome organisational inertia?

It then goes on to suggest that the analyst should try to assess the options open to the major competitors, the extent to which they look ahead and the metrics they use. They stress that most competitors are likely to see the world quite simply and to restrict the number of options being considered. There is a danger that sophisticated companies think that every other company looks to maximise net present value, just because they think that way.

I know that our strategy and War Game specialists will look at this article and say that it is merely repeating the thinking that should go into all strategic assessments and War Game designs. They would nevertheless benefit - as every reader will - from the challenges to conventional assumptions and the survey evidence that the decisions of many companies are often less logical and rational than we think.


To access this article go to http://harvardbusinessonline.hbsp.harvard.edu/b01/en/common/item_detail.jhtml?id=R0904H&_requestid=13084

Roundtable discussion on training and qualifications, Director Magazine, April 2009

I chose this article because I know that many of our clients and contacts have interests in qualifications and because it provides new thoughts on the perennial issue of their value to managers. It also updated me on government involvement in learning, skill development and qualifications; for instance I had never previously heard of the ‘Qualification and Curriculum Authority’ whose ‘Director of Qualification and Skills’ was represented on the panel.

The article is made up of the separately expressed views of this panel which was supposed to represent ‘directors, educators and employee representatives’. Though the views expressed come over as generally informed and sensible, it would have been nice to have seen a rather more high calibre group; it is not the first time that a good feature in the Director Magazine has been adversely impacted by the low quality of the names. Apart from the Head of Skills and Economic Affairs of Microsoft, there is no other household name and it is surprising that the Director could find no educators other than the Principal of Norwich City College and some hardly known private sector suppliers.

Nevertheless, there is knowledge to be gained. Readers may already know this but apparently there is now an ‘Employer Recognition Programme’ (ERP) and a ‘Qualifications and Credit Framework’ (QCF), both sponsored by the above mentioned ‘Qualification and Curriculum Authority’ (QCA). Apparently employers can bundle up in-house training into qualifications to have national recognition and McDonalds and Flybe are among the companies who have done this, though it was interesting that no other names could be quoted. Also quoted was the ‘Training Quality Standard’ (TQS, obviously all designed to be TLAs!) which is awarded by the Learning and Skills Council (LSC) and needs to prove business gain before it can be granted.

One of the panellists mentioned that the ERP is reminiscent of attempts to build management qualifications around the NVQ framework some years ago; another questioned whether a future employer would be attracted by a qualification earned through a specialist company like McDonalds. Yet another point - which was going through my mind as I read the article - was that, for high calibre managers, it is difficult to turn them on unless the qualification is an MBA, and they haven’t the time for that. Clearly some of the panellists were sceptical about the government initiatives, particularly in a management context and during a recession when coping with or keeping a job become the main priorities.

Though the representative of the QCA was selling very hard the benefits of a flexible and integrated qualifications strategy, the answers to the final question - would you employ someone with a McDonalds qualification? - were interesting. The article finished with these three responses from employers:

- ‘I’m interested in ability not qualification’.
- ‘You look at attitude to learning and development’.
- ‘You can develop skills but you can’t train values’.

Maybe it’s because my focus is tailored management training but I was left wondering about the cost effectiveness of the taxpayers’ money that is being spent on maintaining all these TLAs.


To access this article go to http://www.director.co.uk/MAGAZINE/2009/4%20April/round_table_62_9.html

Learning Integration; can informal learning be formalised? Training Journal March 2009, by Gareth Walters

When I saw the title of this article, my initial reaction was that the real question is not can it be formalised but should it be formalised? Isn’t the whole question a contradiction? But after reading what the authors have to say, I became convinced that they have a point and that this topic is a valid issue for the learning professional. Perhaps a better question is - how can the two types of learning, formal and informal, best be integrated?

Informal learning is defined by the authors as the ‘unofficial, unscheduled, impromptu way most people learn to do their jobs’. They make the same point as several articles which I have reviewed on the blog recently; the different ways in which people - particularly the younger population - are now learning, mainly as a result of the technology available and the increased opportunities that it provides via Facebook, YouTube, Wikipedia, Google etc. The question the article faces is - can you, should you try to embrace this trend through adaptations of learning strategies and learning management systems?

The point is made that to ignore these trends would be wrong, not least because many of the ‘new media’ may produce data that is wrong or misleading, notably Wikipedia. There is also the point that the younger learners, who use these media the most, are the ones most in need of formal structures to replace their lack of experience.

The author sees great benefit in building in access to blogs, wikis and podcasts through learning management systems, thus showing a willingness to embrace them and allowing some element of control. There are opportunities to provide a structure - for example a series of Podcasts - and encourage the sharing of data and opportunities among users. They also suggest that the LMS can track usage, which is perhaps more arguable and rather depends on the purpose and the uses of the tracking information.

Another excellent idea which this blog has discussed before is the integration of informal learning into course design, for example, using on-line access to Google et al as part of a case study or exercise. Previous discussion of this type of integration encouraged me to develop an exercise for my grandchildren over Easter - ‘answer these sport questions via Google and the first to finish can start the Easter Egg hunt’. I was amazed at the speed and motivation (largely due to a cash reward at the end) and it confirmed to me what can be done by some structuring of informal learning!

So I agree with the thrust of the article if not the title question. You cannot formalise the informal but you can integrate the two to get the best of both worlds.


To access this article go to http://www.trainingjournal.com/tj/1987.html

A War of Words, by Kate Halpern, Personnel Today 7th April 2009

This article is to some extent tongue in cheek but it raises some interesting issues. It makes fun of the increasing tendency for businesses in general, and the HR function in particular, to use jargon, and the extent to which this is a problem. There is also the question - when does a word or a phrase become jargon? ‘Stakeholder’ and ‘incentivising’ are mentioned as examples but aren’t these just words that succinctly express a concept?

There is also a suggestion that the HR function is more guilty of jargon than others and Sales and Marketing are also mentioned as guilty parties, but my reaction was to say that neither are in the same league as Finance or IT. It is even suggested that the term ‘Human Resources’ is jargon and that simple ‘Personnel’ expresses the role much better. I can see that one word is better than two but my recollection is that HR was coined to make it clear that the role was more than managing canteens and hygiene, which came to be associated with Personnel. If this change of title has helped to change the image, it surely was worthwhile.

The article does produce some quite silly examples which I had never heard of - like ‘strategic staircase’ and ‘re-baselining’ - but it also makes fun of other terms that have become part of business and general language like ‘in the loop’ and ‘singing from the same hymn sheet’, both of which I confess to using quite regularly.

My well known defensive instincts therefore drive me to ask the question - why not? If there is a phrase which easily encapsulates a state or a concept, isn’t this just a natural development of language? In fairness the article does ask this question at the end and says that maybe we should be celebrating rather than criticising our ability to ‘jargonise’.

In the end it comes down to why it’s used and whether it helps or hinders communication - that should be the test. If it is being used to make others feel excluded or to make the user feel clever, it should be discouraged. But if it helps us all to express our ideas clearly and succinctly, go for it!


To access this article go to http://www.personneltoday.com/articles/2009/04/01/50111/jargon-war-of-words.html

Sir Alan Sugar - The biography, by Charlie Burden, published by John Blake

The fact that I was disappointed by this book may be something to do with my own rather negative views of ‘Sir Alan’. I had hopes and expectations that, as the book was written by a journalist rather than by the man himself, there might be some challenge to the sycophantic attitude of everyone involved in the Apprentice series. But challenge and questioning was kept to a minimum and always answered with a rationalisation or a ‘he was right all along’ explanation.

The book takes on a rather conventional structure by starting with his birth - ‘you’re sired’ is the chapter heading - and emphasising the rags to riches story that the man himself loves to tell us about. And it certainly was impressive during the seventies and eighties when he developed technology products that were cleverly geared to consumer needs and price points. But somewhere along the way he seemed to lose his spark and his flair for product development, perhaps when he became one of the first rich businessmen to be tempted to invest in the glamour of football clubs.

It was at this stage that I began to suspect that the author was being less than objective: I had hoped that a journalist might at least be open about the mistakes that were made, and investing in Tottenham Hotspur with Terry Venables as a partner was about as big a mistake as it was possible to make. But even this is glossed over as a triumph, with not a suggestion that perhaps this was a distraction that helped to derail his progress as an innovator in the electronics market.

The final part of the book covers in some detail perhaps his greatest triumph, his ability to ‘out-trump’ Donald Trump by making the Apprentice so much more entertaining and compulsive than its US equivalent. Yet this part of the book follows the misleading line of the TV programme by continuing to talk about his ‘sprawling business empire’ while admitting elsewhere that it is now not much more than a property portfolio based on past successes.

Whilst accepting the brilliance of Sugar’s early innovative drive and his undoubted TV charisma, I found the book to be so one sided and sycophantic that I wanted someone to write a ‘hatchet job’ to provide some kind of balance. This might also have raised the issue that the book avoids - whether the management style represented by his manner and the way in which conflict is encouraged on the Apprentice, give out all the wrong signals about the way in which business success is achieved.

One interesting insight into the Apprentice is that the two grey haired sidekicks who appear to work for him on the programme, are not his employees at all, but are retired legal and public relations executives whom Sugar used to do business with. It perhaps explains something that has always puzzled me - how two such shrewd individuals could have possibly worked for ‘Sir Alan’.

Business Nightmares by Rachel Elnaugh, published by Crimson

Whereas the Sugar biography disappointed me, this book exceeded my expectations. I was not a ‘Dragon’s Den’ fan when Rachel was on the programme for the first two series but I had heard about the bankruptcy of her firm ‘Red Letter Days’ and the inevitable flak that she took from those who expressed schadenfreude at her fall from grace.

The book is well written and is structured around a number of other business people - some well known, others less so - who experienced crises during their business careers. Most of them overcame the problems and achieved great success; Elnaugh’s argument is that the crises strengthened their resolve and enabled them to learn lessons that led to later triumphs.

There are also, at the end of each chapter, some practical bullet points summarising the lessons learned and providing advice for budding entrepreneurs. Though these are sometimes repetitive and not always related to the content of the chapter, I thought they were valuable and down to earth, so much so that I am making sure to hand on my copy to a young relative who has founded his own business. Clearly the advice to be careful whom you appoint as Financial Director and the warning of the dangers involved in bringing in someone from outside to run your business, were heartfelt but no less valuable for that.

Rachel weaves her own rise and fall story into the chapters at a number of stages and you begin to realise why she is so bitter. It is not just around financial directors and imported CEOs, it is also about wicked banks who failed to support her when she needed it. It was at this point that I wanted to hear the other side of the story though, in contrast to the Sugar book, it is easier to accept a one sided account from the person involved. And it is not all one sided; there are times when she admits fault, in particular her naivety and inexperience as the business grew rapidly in the early stages.

The book suffers from one or two chapters where the stories are week and where more research could have been done. One felt that some names - Jeffrey Archer,
Donald Trump and James Dyson - were there because of their fame rather than the lessons their stories provided. Rachel also seems to have some questionable ideas about men and women in business - women go into it for passion but men to make money for instance - but the short and content filled chapters manage to keep your interest pretty well.

The author should be congratulated for turning defeat into a victory of sorts, she now claims to be happier ‘speaking, writing, consultancy and mentoring… inspiring others to achieve business success’.

MTP News

Summer Workshop

We would like to remind our clients and contacts about the latest in our series of highly successful workshops for learning professionals on 26th June, on the highly topical subject of Reducing the Cost of Learning Delivery. This session will focus on the alternatives to face-to-face training now being used by some of our major clients, including the Virtual Classroom for global on-line delivery and Train the Trainer solutions. Invitations have gone out but you can reserve a place any time by e-mailing learn@mtpplc.com.

See below for more about these activities.

Economic Position

Earlier in the year we reported that, despite the gloomy news on the economy, MTP’s programmes and forward bookings were looking good. Since then the external gloom has clearly increased but we are pleased to report that the present and forward position are both strong.

We believe that this is partly because we work with companies and managers who believe that continued investment in management training is essential to long-term success and should not be sacrificed for short-term reasons. Another factor is that we now work in a wide range of industrial sectors, most of which are resilient to global downturn. It is also a lot to do with our quality and flexibility of delivery method, with a lot of our new business coming from innovative non-classroom solutions, see examples below.

We would be interested to hear from clients and contacts about the impact of the current economic environment on spending this year.

Virtual Classroom Training

Our innovative work in this area - the delivery of live, interactive, on-line learning - has been steadily developing over the last few years and we have already delivered virtual sessions for Barclays, Financial Times and Unilever. Our most ambitious project yet was started in October when we launched a virtual version of a two-day learning programme in Business Acumen for the HR professionals of one of the world’s biggest high tech companies, a household name throughout the world.

This programme went much further than our previous virtual classroom projects, extending to four sessions of three hours and including many of the features that make our ‘face-to-face’ courses so successful - quizzes, polls, case studies and even a role-play exercise. The session also includes a video covering the company’s investor relations strategy and it is all brought together by an MTP tutor working from our offices, using the most up-to-date on-line learning software.

The following quote is typical of the highly favourable response so far:

“I was very surprised at the level of interaction and engagement with the use of the VC and teleconferencing technology. I’ve been in L&D roles and have really struggled with how to best facilitate this type of programme at low cost to an audience that is spread across such a vast geography. This is definitely a big leap forward and I see huge opportunity for using this technology. Overall, the programme was fantastic, delivery and use of technology was great; I hope we can do more of this in the future.”

The success has been confirmed by the booking of more than 20 programmes this year, with around 90% delivered via virtual classroom methodology and an audience extending for the first time beyond the HR function. We have also taken on a number of new projects for Barclays, including the development of virtual role-plays on business partnering and innovative sessions on managing risk and uncertainty.

Business Simulation

MTP’s activities in this increasingly popular area of management training continue to develop. More and more companies are beginning to realise that a well designed simulation is the best way of tackling that difficult to define ‘holy grail’ of management training - business acumen.

Until about 18 months ago, our approach was to offer three outstanding, generic simulations to meet learning needs in the areas of finance, strategy and marketing, and these are still available to meet many learning needs. However, we then took on a project for BP which was to prove a watershed - the development of a tailored simulation based on the running of petrol stations and bringing in behavioural as well as financial and strategic aspects. It was also based in Excel and was thus much easier to operate and to tailor to specific needs.

The success of this project has led to exciting further developments; the conversion of our most popular generic simulation - 2020 Business - to Excel and the creation of a version that can be run virtually. We are also in the middle of a major new project for Unilever Foodsolutions; the development of a new tailored simulation based on the management of a team of sales people in the food service business. The design is based on a similar concept to BP and opens up possibilities that have never previously been available; tailored business simulations that are practical, realistic and cost effective, and can be run by either our own team or by internal facilitators after a Train the Trainer programme.