What you will learn
- Process metrics
- Version changes
- Exceptions and reasons
- Evidence required before increasing complexity
Process metrics
Process metrics measure behaviors such as complete records, rule compliance and reconciliation errors. Outcome metrics measure results such as net expectancy and drawdown. Track both; neither substitutes for the other.
Version changes
Version the plan when rules change. Record the date, reason and expected effect so later results can be assigned to the rules actually used. Mixing several versions without labels makes conclusions unreliable.
Exceptions and reasons
Exceptions should state who decided, what information was available and why the normal rule was not followed. An exception log is not permission to improvise indefinitely; recurring exceptions may indicate that the rule is poorly specified.
Evidence required before increasing complexity
Increase complexity only when it solves an identified problem and can be evaluated. Adding indicators or instruments after a small losing sample may obscure the original question. A simpler unchanged baseline helps judge whether the addition contributes useful evidence.
Worked example
A learner changes entry, stop and session rules together after three losses, then attributes the next win to a new indicator. The simultaneous changes and tiny sample cannot isolate the indicator's contribution.
Try it yourself
Design a bounded comparison for one proposed change while preserving an unchanged baseline and a record of all qualifying cases.
Show the worked solution
Choose one change, freeze both definitions, use a predefined sample and compare net outcomes and compliance under consistent assumptions. Report uncertainty and avoid claiming that a few observations establish durable improvement.
Apply this to your course project
Submit a one-page demo plan with eligibility, invalidation and review rules.
Keep the calculation inputs, assumptions and decisions with your work. Practical exercises are self-reviewed; the scored knowledge checks assess the questions shown, not an independent certification of practical competence.