# Optimization and Decision-Making Under Uncertainty Evaluate allocation and parameter choices without disguising estimation error. Use this workbook alongside the course. Write your answers before opening the solutions. Practical work is self-reviewed; scored knowledge checks are in the Academy. ## 1. Objectives ### Return versus utility An optimization objective defines what is being maximized or minimized. Return, drawdown, variance and utility are different quantities. A solution can be optimal for a poorly chosen objective and still be unsuitable for the actual decision. ### Risk constraints Constraints define permitted exposure, turnover, concentration or other limits. They should reflect the problem rather than be adjusted retrospectively to make a favorite result feasible. Check whether a proposed solution satisfies every constraint. ### Turnover costs Turnover and transaction costs can change the optimum. A gross-return objective may prefer frequent reallocations whose gains disappear after costs. Include the implementation model in the objective or evaluation consistently. ### Multiple objectives Multiple objectives require explicit trade-offs. A frontier describes alternatives under assumptions, not one universally best point. Explain the preference or constraint that selects a candidate from the alternatives. ### Worked example An optimizer chooses a portfolio with the highest estimated return but twice the allowed concentration. Its objective value does not make it a valid solution to the constrained problem. ### Independent exercise Define an objective and two constraints for a hypothetical research allocation, then state how infeasible solutions should be reported. My inputs and assumptions: My calculation or decision: Evidence that would change my conclusion: ## 2. Methods ### Grid search Grid search evaluates a specified set of candidates. The grid's range and resolution are research choices. A best point at a boundary may indicate an incomplete search or an unstable objective, not a trustworthy universal optimum. ### Constrained optimization Constrained optimization uses mathematical procedures whose assumptions, convergence criteria and scaling matter. A solver reporting success does not prove that inputs are valid or that the economic model is correct. ### Regularization Regularization discourages certain forms of complexity or extreme choices. It can improve stability under some conditions but introduces another parameter and trade-off. Evaluate it rather than treating it as automatic protection from overfitting. ### Sensitivity to initial conditions Initial conditions and local optima can affect nonlinear procedures. Repeat under justified starting conditions and inspect feasibility and objective values. Report sensitivity instead of selecting the most attractive run silently. ### Worked example A nonlinear solver returns different feasible weights from different starting points with similar objective values. Reporting only one weight vector to many decimals conceals uncertainty in the solution. ### Independent exercise Write a results table that reports starting condition, convergence, feasibility, objective and allocation stability. My inputs and assumptions: My calculation or decision: Evidence that would change my conclusion: ## 3. Uncertainty ### Parameter instability Parameter instability occurs when small input changes produce large changes in the selected solution. Measure this directly by perturbing assumptions or resampling under a justified design. A precise output can still be unstable. ### Scenario perturbation Scenario perturbation varies returns, costs, correlations or other inputs within plausible ranges. State why the changes are meaningful. A stress selected only because the candidate survives it is not an informative robustness check. ### Estimation error Estimation error affects expected returns and risk inputs. Optimizers can amplify small noisy differences by assigning extreme weights. Compare against simple baselines that require fewer estimated quantities. ### Robust alternatives Robust alternatives seek acceptable behavior across a set of assumptions rather than the best value under one estimate. The uncertainty set is itself a modeling choice and should be reviewed for relevance and conservatism. ### Worked example Two strategies have nearly equal estimated means, but the optimizer allocates almost everything to the slightly higher one. A small change in the sample flips the allocation to the other strategy. ### Independent exercise Describe a comparison that tests whether this sensitivity materially affects out-of-sample decisions. My inputs and assumptions: My calculation or decision: Evidence that would change my conclusion: ## 4. Governance ### Track all trials Track every material trial, including objective revisions and constraint changes. The final candidate is selected from that history. Omitting failed trials can exaggerate the apparent quality of the chosen solution. ### Freeze selection rules Freeze selection rules before the final evaluation. If the evaluation is used to choose another candidate, acknowledge that it has become development information and obtain separate evidence for the new selection. ### Compare against simple baselines Compare against simple baselines under the same information, costs and constraints. Complexity should contribute enough practical value to justify additional estimation and operational burden. ### Document why complexity is justified Document why the selected complexity is necessary and how it will be monitored. A research result can justify retaining a simple baseline when optimized alternatives are fragile or indistinguishable within uncertainty. ### Worked example After seeing the final test, a researcher changes the objective from return to a ratio that makes the candidate look better. The test has influenced the criterion and cannot remain an untouched confirmation of that revised choice. ### Independent exercise Write a governance note that records the change and proposes an honest next evaluation step. My inputs and assumptions: My calculation or decision: Evidence that would change my conclusion: ## Course project Compare a simple rule with an optimized alternative under perturbation and held-out scenarios. ### Self-review rubric - Concepts and reasoning: 25% - Calculations, data and evidence: 30% - Process and risk controls: 25% - Limitations and communication: 20% Record one correction and one next practice task. This rubric is not automatically graded. ## Worked solutions ### Exercise 1 For example, compare net expected benefit under stated assumptions subject to exposure and turnover limits. An infeasible candidate should be flagged or rejected, not presented as the winner without its violations. ### Exercise 2 The table should expose all justified runs and their constraints. Similar objectives with very different weights suggest the allocation is weakly identified by the model; a simpler or more stable choice may deserve comparison. ### Exercise 3 Compare the optimized policy with capped or simple allocations under predefined input perturbations and later data, including costs. Report both performance and weight turnover; do not assume the most stable policy is automatically best without the decision context. ### Exercise 4 Label the revised objective as a new development choice, preserve the earlier result and freeze the new procedure before evaluating on independent later evidence. Report the search rather than retroactively rewriting the protocol. ## Further reading - https://www.itl.nist.gov/div898/handbook/ - https://scikit-learn.org/stable/modules/cross_validation.html