What you will learn
- Temporal splits
- Purging overlapping labels where applicable
- Economic costs
- Drift and model retirement
Temporal splits
Temporal splits should reflect how the model would be updated and used. Avoid training on future periods for a prospective earlier prediction. Store split definitions so the evaluation can be reproduced.
Purging overlapping labels where applicable
Purging or gaps may be needed when labels overlap across split boundaries. Determine the overlap from target intervals rather than applying a universal arbitrary gap. Record removed observations and the reason.
Economic costs
Convert predictions into a specified decision policy with costs, delays and risk. A statistical score improvement may not survive trading frictions. Compare the full policy with the baseline, not only the classifier metric.
Drift and model retirement
Monitor drift and define review or retirement conditions. A historical model can become inappropriate when inputs, target relationships or implementation change. Monitoring is evidence gathering, not a guarantee of detecting failure before losses.
Worked example
A model improves classification score but generates many more trades whose costs exceed the added gross gains. The statistical improvement does not establish a better net decision policy.
Try it yourself
Write a final model report containing prediction metrics, policy results, costs, temporal design and limitations.
Show the worked solution
Report the frozen model and decision mapping, baseline comparison, net outcomes, uncertainty, split and overlap treatment, search history and monitoring assumptions. Keep unproven generalization claims out of the conclusion.
Apply this to your course project
Compare a simple baseline with one model using a frozen temporal evaluation plan.
Keep the calculation inputs, assumptions and decisions with your work. Practical exercises are self-reviewed; the scored knowledge checks assess the questions shown, not an independent certification of practical competence.