The Trainer’s Handbook

Chapter 5 - Evaluating Your Effectiveness

Types of Exam Questions. Various specific types of questions are used in tests. See Figure 4-7 in Chapter 4 for further examples.

Most of us are so familiar with multiple-choice questions that we turn automatically to this format.

Sample: Which of the following is a demonstrated principle of learning:

     a. Trainees learn best what they hear.

     b. Trainees learn best by making and correcting mistakes.

     c. Trainees learn best by watching role models perform.

     d. All of the above*

*The answer is b. we learn best by making and correcting mistakes. a. is wrong becasue we forget 50 percent o what we hear within twenty-four hours. c. is wrong because, although we may get ideas by observing others, we will learn little from watching wihtout actual practice. If you doubt this, try letting someone learn to drive a car by merely watching you do it!

It is effective for evaluating straight recall of facts and for testing recognition of principles, but otherwise this question type is rather limited. If you do use it, offer at least four choices. If you wish to test the ability to discriminate among alternatives, make several choices correct but only one perfect. Use all of the above and none of the above sparingly. They encourage guessing.

The multiple-choice format allows humor. Lighten the pressure from time to time by making one choice absurd. This can be particularly effective if the humor comes from some event or situation shared by the group. Try to vary the format with other types of questions, too. Use only twenty multiple-choice questions in a set (several sets are possible in a longer test). This gives a built-in break.

If you give regular exams and worry about having different questions for different groups, write and store about one hundred questions in a computer and then program it to select at random and print the quantity of questions you need. This is a great time-saver.

Fill-in-the-blank questions serve much the same purpose as multiple-choice but are not as easy to guess right. Include enough information in the sentences so they can be understood. Few questions are more frustrating than ambiguous fill-in-the-blank questions.

Sample: The first step in planning a training lesson is to formulate your ____________.*

*“Learning objectives,” because what is learned is much more important that what is taught.

Minimize your use of true-false questions. These are temptingly easy to write, but psychologically they reenforce the false statements as strongly as the true ones. Frequently, trainees come away remembering the negative statement rather than the true one.

Sample: The trainer must always be the most assertive person in the room.

*True. It is part of the mantle of leadership that is given to us to wear.

Matching questions are excellent for evaluating recall and discriminating between choices. Labeling diagrams are perfect for subjects in which trainees must recall the names of parts.

Sample: Match the following methods with the learning principle they utilize:

*1,d;2,c;3,a;4,e;5b. If you got any of these wrong, think them through again carefully.

Tests also include short and long essays. If you are evaluating judgment–that is, a trainee’s ability to use the material taught–then the essay question is hands-on problem solving. It forces thought and involves the test-taker far more than any other question format. The essay question is the type from which the Harvard Business School developed the concept of case histories. Short essays (a sentence or two) draw specific responses; long essays probe thinking processes. Also, the essay format exposes those who don’t understand and need extra help.

Sample: Describe how you would respond to a participant who openly challenged something you have said to the group.*

*The answer would need to include: smile, move toward the difficult participant, and ask open-ended questions to debrief him or her.

When making essay questions, keep your objectives in mind. Be sure the question asks for a specific response. This makes the question easier to evaluate and more valid. Mini-case histories work nicely, too.

> Assessment Sessions

Excerpts from Chapter 5, The Trainer's Handbook

1. Evaluating Effectiveness
2. Short-term Evaluation
3. Project Sessions
4. Case Histories and Practice Sessions
5. Examinations
6. Types of Exam Questions
7. Assessment Sessions
8. Self-evaluation
9. On-the-Job Evaluation
10. Long-term Evaluations
11. Bottom-line Evaluation

More learning articles:

• The miracle of learning
• Orienting new employees
• More about the role of the trainer
• Learning Organizations
• Knowledge Management
• The value of lectures