What you will learn
- Track all trials
- Freeze selection rules
- Compare against simple baselines
- Document why complexity is justified
Track all trials
Track every material trial, including objective revisions and constraint changes. The final candidate is selected from that history. Omitting failed trials can exaggerate the apparent quality of the chosen solution.
Freeze selection rules
Freeze selection rules before the final evaluation. If the evaluation is used to choose another candidate, acknowledge that it has become development information and obtain separate evidence for the new selection.
Compare against simple baselines
Compare against simple baselines under the same information, costs and constraints. Complexity should contribute enough practical value to justify additional estimation and operational burden.
Document why complexity is justified
Document why the selected complexity is necessary and how it will be monitored. A research result can justify retaining a simple baseline when optimized alternatives are fragile or indistinguishable within uncertainty.
Worked example
After seeing the final test, a researcher changes the objective from return to a ratio that makes the candidate look better. The test has influenced the criterion and cannot remain an untouched confirmation of that revised choice.
Try it yourself
Write a governance note that records the change and proposes an honest next evaluation step.
Show the worked solution
Label the revised objective as a new development choice, preserve the earlier result and freeze the new procedure before evaluating on independent later evidence. Report the search rather than retroactively rewriting the protocol.
Apply this to your course project
Compare a simple rule with an optimized alternative under perturbation and held-out scenarios.
Keep the calculation inputs, assumptions and decisions with your work. Practical exercises are self-reviewed; the scored knowledge checks assess the questions shown, not an independent certification of practical competence.