Course completion is a milestone. What happened afterward?

Start with the decision the data will support

Before collecting another metric, ask what you will do with the result. Are you deciding whether to revise a confusing activity, extend a pilot, or replace a costly intervention? Each question needs different evidence.

Course completion is useful for understanding participation and access. Satisfaction can tell you how people experienced the learning. Neither answers whether someone can perform the target task correctly in a realistic situation.

Define the performance standard

Write a task and a rubric before you build the assessment. Specify what an acceptable response includes, which errors matter most, and whether references or tools are permitted. Ask the team responsible for the work to verify that the standard reflects actual practice.

For a customer conversation, the rubric might address identifying the issue, selecting an appropriate next step and explaining it accurately. For a technical procedure, it may include sequence, accuracy and recovery from a mistake. Do not substitute a recall quiz merely because it is easy to score.

Collect evidence at more than one moment

Use formative checks during practice to guide feedback. Use a new task afterward to examine independent performance. Later, look for evidence of application in the work environment, with appropriate permission and privacy safeguards.

Choose timing based on the task. If a skill is used every day, there may be an early opportunity to observe transfer. If the event is rare, a realistic follow-up simulation may be more practical. There is no universal interval that fits every project.

• Before: what is the baseline and how reliable is it?
• During: where do learners need help?
• After: can they perform a new task independently?
• At work: does the behavior appear when it is needed?

Name the alternative explanations

Suppose errors decline after a training launch. That is encouraging, but the software may also have changed, managers may have increased coaching, or the team may be handling different cases. Describe those conditions before attributing the result to the course.

A comparison group or a more rigorous evaluation design may be justified when the decision is consequential. For a small exploratory pilot, an honest account of the sample, timing and limitations is still useful.

Close the loop with a design decision

Turn findings into action. If learners can explain the rule but struggle under time pressure, add practice under realistic conditions or improve point-of-work support. If they cannot find the resource, fix access before producing more content.

Your evaluation canvas should end with a decision: improve, expand, replace or stop. A dashboard becomes valuable when it changes what the team does next.