# Model Governance, Security and Auditability Create controls that make research and execution changes traceable. Use this workbook alongside the course. Write your answers before opening the solutions. Practical work is self-reviewed; scored knowledge checks are in the Academy. ## 1. Governance ### Ownership Ownership identifies who maintains a model, approves its use and responds to failures. A system with no named responsibility can accumulate changes that nobody can explain. Record both primary ownership and escalation arrangements. ### Model inventory A model inventory lists purpose, version, dependencies, data and operating status. Include simple rules and manual overrides where they affect decisions, not only machine-learning components. Unregistered behavior can be as consequential as a complex model. ### Change classification Classify changes by their effect on behavior, data, execution or presentation. A harmless label change and a new sizing rule require different evidence. The classification should follow actual consequences rather than the developer's preferred description. ### Independent review Independent review challenges material assumptions and checks evidence before acceptance. It does not mean a reviewer guarantees outcomes. Record findings, unresolved issues and the decision boundary under which use is permitted. ### Worked example A change described as display cleanup alters a default order quantity. Its behavioral effect makes it an execution change regardless of the label in the ticket. ### Independent exercise Write a change record identifying the actual effect, required evidence, owner and rollback plan for that case. My inputs and assumptions: My calculation or decision: Evidence that would change my conclusion: ## 2. Security ### Least privilege Least privilege grants only the access needed for a task. A research process generally does not need account-trading authority to read historical data. Separating permissions limits the consequences of an error or compromised component. ### Credential rotation Credential rotation and revocation need a tested procedure. Know which services use a secret and how to replace it without exposing values in logs. Lost or suspected-compromised credentials require a documented response. ### Dependency risk Dependencies introduce supply-chain and maintenance risks. Track versions and relevant changes, and avoid executing unreviewed code merely because it appears in a tutorial. Reproducibility and security both benefit from explicit dependency management. ### Data access boundaries Data access boundaries separate public research, account records and sensitive credentials. Export only what a reviewer needs, with personal identifiers minimized where appropriate. A private repository is not a reason to place secrets in source files. ### Worked example A backtest notebook contains a live trading key even though it only reads a local CSV. Removing the unused privilege reduces the harm possible from an accidental call or leaked notebook. ### Independent exercise Design separate permissions for data ingestion, research and execution, and identify how a reviewer can reproduce a study without receiving trading credentials. My inputs and assumptions: My calculation or decision: Evidence that would change my conclusion: ## 3. Audit evidence ### Configuration provenance Configuration provenance connects each decision with the settings actually used. A timestamp alone is insufficient if several versions were active. Store stable version identifiers and relevant effective times. ### Immutable event records Event records should preserve what happened without silent rewriting. Corrections can be appended with explanations. The objective is traceability, not pretending that a log cannot ever contain an error. ### Decision traceability Decision traceability links inputs, model version, output and resulting action. It should distinguish a recommendation from an executed order and record failures or withheld actions as well as successful ones. ### Retention policy Retention policy defines what is kept, for how long and under which access rules. Balance reproduction and operational needs with applicable privacy and contractual constraints. Do not retain sensitive data indefinitely simply because storage is cheap. ### Worked example An incident log records an order but not the configuration version that generated its size. The account state may be reconciled, yet the cause remains difficult to reproduce. ### Independent exercise Specify the minimum trace needed to connect a decision to an execution while avoiding unnecessary personal data. My inputs and assumptions: My calculation or decision: Evidence that would change my conclusion: ## 4. Incident response ### Triage and containment Triage identifies the affected systems, current exposure and evidence needed. Containment limits further harm while preserving the ability to reconcile. An immediate compensating trade without state knowledge can make an incident worse. ### Trading state reconciliation Trading-state reconciliation compares positions, working orders and executions with the authoritative account record. Local logs may be incomplete after a failure. Preserve disagreements and resolve them before assuming the system is flat. ### Root-cause analysis Root-cause analysis distinguishes the triggering event from the underlying defect and missing controls. Blaming an operator without examining ambiguous interfaces or unsafe defaults can leave the same failure mechanism intact. ### Remediation verification Remediation is complete only when the correction is verified against the failure case and relevant neighboring cases. Document residual limitations and ownership. A code change alone is not evidence that the incident cannot recur. ### Worked example A timeout causes an operator to retry, producing duplicate exposure. The timeout is a trigger; missing request identity and unclear pending state may be underlying contributors. ### Independent exercise Write a remediation plan covering interface state, request identity, reconciliation and a regression scenario. My inputs and assumptions: My calculation or decision: Evidence that would change my conclusion: ## Course project Submit an approval, rollback and incident-evidence package for a hypothetical model change. ### Self-review rubric - Concepts and reasoning: 25% - Calculations, data and evidence: 30% - Process and risk controls: 25% - Limitations and communication: 20% Record one correction and one next practice task. This rubric is not automatically graded. ## Worked solutions ### Exercise 1 Record the quantity change explicitly, require sizing and execution tests plus authorization appropriate to the system, identify the responsible owner and preserve a known configuration for rollback. Do not approve based on the cosmetic label. ### Exercise 2 Give each component only its required capability. Provide approved data access or reproducible files and configuration descriptions to the reviewer; keep execution authority and secret values outside the research package. ### Exercise 3 Record decision identity, input references, model/configuration versions, timestamps, requested action, order and execution identities, and outcome. Use references rather than copying sensitive material everywhere, and apply an explicit access policy. ### Exercise 4 Show pending status, preserve stable request identity where supported, reconcile uncertain outcomes before retry and test a successful request with a lost response. Verify that the repeated message does not create a second action. ## Further reading - https://owasp.org/www-project-top-ten/ - https://www.nist.gov/cyberframework