The Trainer’s Handbook

Chapter 5 - Evaluating Your Effectiveness

Methods for Long-term Evaluations

1. Mandate posttraining action plans.
2. Set key variables to follow.
3. Send posttraining surveys.
4. Offer segmented training over months or years.
5. Prepare a follow-up needs assessment.
6. Check data on performance impacted by training–scrap. sales and so on.
7. Convert data to percentages whenver possible.

1. Mandate posttraining action plans. Sun Life of Canada currently practices one of the best evaluation techniques I’ve run across. At the completion of each lesson, trainees are given five minutes to create an action plan for how and when they will use what they’ve just learned when they return to the job. Upon completion of the entire course, they are required to spend about thirty minutes consolidating these plans into a final action plan for integrating the course into their daily work. Trainees are paired up to exchange their action plans with their partners for discussion, commitment, and a reality check. Finally, two to three weeks after the training, participants are required to make an appointment with their individual immediate supervisor or manager to go over the action plan to assess which elements of it they have already accomplished and where they may be having problems. This system guarantees transfer of learning, but it requires full management support and training as well as commitment.

2. Set key variables. Plant a method or a word in the midst of your program–some particularly clear example or acronym. Even years later, as you talk to those you’ve trained, listen for those key terms, techniques, or descriptions. They may have forgotten how they learned them, but you’ll know. Of course, this technique assumes that you’ll keep in touch with at least some of those you trained.

3. Take posttraining surveys. Let your trainees know that at some future date they will be asked to respond to a survey. Send out a simple survey no sooner than three to six weeks after training, asking which skills they are using and which have benefited them. Have them describe an instance in which they feel their new skills helped them do a better job. Ask for comments on the validity of the training now that they have had a chance to put it into practice. While they are still in training explain that the future survey is an aid to your performance rather than an evaluation of theirs and that you need their feedback. You should then get a reasonable response when you send out the survey. Do the survey three months to a year after the training, if you can.

4. Hold follow-up sessions. Structure your training in well-spaced intervals. If you get the same group every six months or even once a year, evaluate how well they have used what they were taught last time. Use pretests and posttests as well as assessment sessions, projects, case histories, and role-playing. I’ve had particular success with this technique in sales training. Often in unsupervised work such as sales, the trainees enjoy the session but fail to put into practice much of what they are taught. One or two of them do, however, and their sales performance soars. At follow-up sessions, I use those who’ve succeeded as examples to motivate the others.

5. Perform another (or a continuous) needs analysis. Six months after training, collect data in whatever way you used to establish the need for the training (see Chapter 4). Then compare that data to your earlier results. Don’t forget that walking around is one the best and most available tools for gathering data both before and after training. As long as your learning objectives are clear, you will be able to easily gauge how effective your training has been.

6. Monitor company records. If your training impacts directly on measurable data like scrap rates, sales, or customer complaints, track these data over the years and see what the accumulated information reveals.

7. Convert your data to percentages. As the population shrinks, fewer and fewer trainees can respond. Straight data would erroneously imply a decline in training effectiveness. By converting the results to percentages, you can report such facts as 98 percent of those responding to a recent survey felt that training significantly impacted on their work–even though only six or seven former trainees were left to respond.

Long-Term Evaluation of Affective Learning

• Word of mouth. It is possible to gauge how employees feel about a subject, a type of work, an impending or recent change, their training, and other trainees by tapping the company grapevine. Find someone who always knows what’s going on. You won’t get hard data, but you’ll find out how well you’ve achieved your affective training objectives.
• Surveys. The in-house magazine or newsletter is a good place to run a survey. Surveys are usually anonymous so that they invite honest answers. If you are not perceived in a hostile light, you might circulate among workers and ask for their responses to key attitude questions. This technique was used in the famous Mayo-Hawthorne studies,1 which discovered that people work harder and better when they know others are watching them. Dr. Mayo and his team interviewed employees and asked them to tell how they felt.
• Participation. If you have been doing motivational training, monitor the figures on voluntary participation in such things as blood drives, the United Way, or toy collections for needy children. In a similar vein, watch for employee activity in company-sponsored events such as Little League or the annual picnic. Participation in such activities usually reflects a positive attitude.
• Confrontations. Look at the company records on confrontations. A reduction in the number or types of arguments in the workplace reflects shifts in attitude.
• Absenteeism and turnover rates. High absenteeism is related to employee attitude, as are turnover rates. Employees who stay with the company usually like something about it.
• Safety-related accidents. The number of accidents tends to decrease with increasingly positive attitudes toward safety, safety training, and the company as a whole.
• Scrap (reject) and error rates. Happy workers make fewer errors. People who care about quality take the time to do a job right. On the other hand, excessive errors and wasted materials point to poor work attitudes.
  
  

> Bottom-line Evaluation

Excerpts from Chapter 5, The Trainer's Handbook

1. Evaluating Effectiveness
2. Short-term Evaluation
3. Project Sessions
4. Case Histories and Practice Sessions
5. Examinations
6. Types of Exam Questions
7. Assessment Sessions
8. Self-evaluation
9. On-the-Job Evaluation
10. Long-term Evaluations
11. Bottom-line Evaluation

More learning articles:

• The miracle of learning
• Orienting new employees
• More about the role of the trainer
• Learning Organizations
• Knowledge Management
• The value of lectures