Formative Evaluation of ICW (revise-as-you-go)
Whenever the Coast Guard develops training, or, for that matter, any
Overview
intervention to improve performance, there must be some way to determine if
the training (intervention) works. Is the ICW what students need to perform
the tasks that make up their jobs proficiently Does the ICW contain too much
or too little information Did course designers hit the mark
Old
The Coast Guard uses a "revise-as-you-go" (formative) evaluation approach to
Method
resident instruction. This approach may use a "murder board" of experts to try
out instruction. Or, it may use a small group of students (pilot evaluation) to
"test" new courses. Either way, course designers fix the deficiencies experts or
students find and then field test the revised course. Research shows that the
"revise-as-you-go" method has been very effective. However, there are better
ways to "revise-as-you-go" (formatively) evaluate ICW.
"New"
Methods"
Alternatives to
How it Works
1 to 1 Methods
Two learners review instruction. As they work
through the program, they discuss with each other
Two-on-One
(and the evaluator) errors and problems that arise.
Learners describe their thoughts (reactions, plans,
ideas and confusions) to the evaluators as they
proceed through the materials.
Think aloud
Protocols
Interviewers send questions via electronic mail to
experts or learners. Or, computer-assisted data
Computer
collection (CASAC) programs present questions on
Interviewing
screens and register the answers, with or without an
evaluator present (See illustration)
48
Previous Page
