The Trainer’s Handbook

Chapter 5 - Evaluating Your Effectiveness

TRAINING is the business of bringing about change. To know whether you have achieved change, you must be able to evaluate the effects of your instruction. You’ve defined your objectives and determined what the change should be. You’ve assessed the present status and have a program for changing it to meet those objectives. Now you need a means of measuring the success you’ve achieved. This last step is evaluation.

There are three levels of change in performance that must be monitored and evaluated. Level one is the measurement of how well the trainees can perform the skills you have been communicating to them. It takes place during and upon completion of training.

Level two is the observation of the trainees’ performance when they return to the job. It is a measure of what theorists call the “transfer” of learning to the workplace. You know from level one evaluation how well each of the trainees is able to perform the skills you’ve taught. Now you must follow up and observe whether or not they are actually transferring those skills to their jobs. If they are not doing so, you will need to perform a narrow-focused needs analysis in order to troubleshoot the problem, diagnose it, and propose a solution.

Level three is a measurement of the impact of the training on the operations of the department for which you have performed the training or, indeed, on the entire organization. It is a measurement of the dollar return on the money investment by management in training. Level three is the bottom-line evaluation. It answers the question “Are we getting our money’s worth out of training?”

As you can see, evaluation is an important ongoing function for the trainer. It is also a vital function for the trainees. If you remember, one of Thorndike’s principles of learning is the law of effect: nothing succeeds like success. Trainees must get constant feedback to develop the motivation to continue. Constant evaluation not only lets you know where you are, but it also does the same for your trainees. For convenience, the evaluation function has been divided into several operations, but there is considerable overlap and much mixing of technique. It is likely that you will be using several methods at the same time.

Level One: Short-Term Evaluations

Level one evaluation consists mainly of short-term projects with which we are most familiar. Homework assignments, class projects, term papers, and tests are the forms of evaluation we all remember from school. This level of evaluation usually consists of some challenging task set by the instructor and performed by the student. How well the student performs the task is the measure of his or her success at learning it. The task provides the learner with necessary feedback on how well he or she is learning. At the same time, it provides the instructor with feedback on how well the learner is mastering the skills being taught and what coaching, if any, will be required. Incidentally, it also provides feedback on the instructor’s success at teaching the material.

Exams and Tests

Many people dread exams because, when they were in school, the results were associated with passing or failing. Since exams were the basis for vital judgments affecting our future, it isn’t surprising that exams are thought of as almost punitive to some of us. You will have to reposition the purpose and function of exams to show your trainees how tests let them know how they are doing. Let exams be a service to them, a diagnostic tool to point out strengths and weaknesses. The fact that you can also use them to evaluate yourself is really immaterial. Tests exist solely for them, to provide important feedback. We’ll talk about how to structure those tests later in this chapter.

Alternative Types of Evaluation

There are, however, several other types of short-term evaluations. They range from observational techniques, such as eye contact, to various types of projects and reviews. Let’s take a quick look at them, then discuss in detail how to create and use each one as an evaluation tool.

• Socratic questioning. In Chapter 2, we established the power and teaching value of asking questions. But one of the other major benefits of asking questions is that it lets you monitor the state of mind of the learners and assess the degree of learning taking place. To develop your questioning technique, see Chapter 3.

Methods for Short-Term Evaluations

1. Examinations
2. Socratic questioning
3. Eye contact and observations
4. Spot quizzes and reviews
5. Project Sessions
6. Case histories
7. Practice sessions
8. formal and informal assessment sessions

• Eye contact. Also in Chapter 3, we mentioned the importance of eye contact as positive nonverbal communication between instructor and learner. As with most communication, eye contact is a two-way exchange. Contact is initiated and maintained by the instructor, but the learner sends back a message as well. What do your trainees’ eyes tell you? Eyes that stare or glare at you are challenging you or disagree with what you say. Eyes that frown are expressing challenge and doubt. Eyes that are glassy and expressionless have had enough. It’s time to change the subject. Eyes that shine are challenged and interested, while eyes that droop are sleepy. Ask questions or change the topic. Eyes that blink rapidly or wander about are nervous; the person may be holding something back.

Interpreting Eye Contact

• Glare or stare–challenge or disagreement
• Frown–doubt or deep thought
• Glassy or blank–had enough
• Shining eyes–challenged and interested
• Droopy or sleepy–tuned out or bored
• Blinking or wandering–nervousness or hiding something

• Spot quizzes and reviews. By reviewing a topic using a question format, in either a written or oral quiz, you can take the pulse of the group and find out how well they understand the material. Formal testing and how to structure questions for tests are discussed later in this chapter.
• Project sessions. Assigning work to be done in class allows you to circulate and check their understanding as they work. In a project session, you are looking for how well the trainees can use what you’ve taught them. You also have the opportunity later on to respond in writing to their projects, expressing your evaluation of them. I usually follow up by discussing the project with the group, using several of their efforts as examples.
• Case histories. The case history is a more involved, practical project. It challenges the learners to use what they’ve learned. while it allows you to see how well they are doing. Case histories are discussed in greater detail in Chapter 7.
• Practice sessions. Hands-on practice serves to lock in learning. It also provides you with an excellent opportunity for evaluation and correction.
• Assessment sessions. Assessments usually take place at the end of a program. When technical training has been involved, it might be a troubleshooting session in which equipment has been intentionally maladjusted. Trainees are evaluated on their speed and accuracy in correcting the situation. In “soft” skill areas, you can create hypothetical crises and assign members of the class to respond appropriately. Role plays are useful simulations.

> Short-term Evaluation

Excerpts from Chapter 5, The Trainer's Handbook

1. Evaluating Effectiveness
2. Short-term Evaluation
3. Project Sessions
4. Case Histories and Practice Sessions
5. Examinations
6. Types of Exam Questions
7. Assessment Sessions
8. Self-evaluation
9. On-the-Job Evaluation
10. Long-term Evaluations
11. Bottom-line Evaluation

More learning articles:

• The miracle of learning
• Orienting new employees
• More about the role of the trainer
• Learning Organizations
• Knowledge Management
• The value of lectures