Control Packs · AI-003 · v1.0.0
High-Consequence Model Review
Keep high-consequence model suggestions visibly provisional until an authorized person confirms the action boundary.
Status: review · Review: not independent. This is design guidance; review status does not establish independent verification or compliance.
What this safeguard addresses
Separate model suggestion from authorization and require consequence-aware human review before irreversible effects.
- A model suggestion becomes an irreversible decisionA model's confidence or fluent output is treated as authorization for an action that is difficult or impossible to undo.
Versioned applicability rule
{
"all": [
{
"characteristic": "AI_CONTROLLED",
"equals": true
},
{
"characteristic": "IRREVERSIBLE_ACTION",
"equals": true
}
]
}Decisions for the project owner
Leave a decision open when its value is unknown. Suggested values become confirmed only through an explicit user decision.
- Which model-assisted actions require explicit human confirmation before execution?A model cannot choose its own level of authority for a high-consequence action.AI-003-Q1 · confirmation
Requirements for the coding agent
- AI-003-R1Keep model output provisional, require explicit human confirmation for irreversible effects, record decision provenance, and provide a stop path.
Tests and evidence to retain
- MODEL_REVIEW_BOUNDARY_TESTVerify suggestion, rejection, confirmation, stale decision, duplicate, stop, and audit behavior.
- configuration_or_policy
- implementation_location
- test_result
- telemetry_definition
Passing a published example shows that example's behavior. A coding agent's implementation report remains a claim until its evidence is independently checked.
Related guidance
- AC-6 · Least PrivilegeNIST_SP_800_53_5_2_0 · partially addressesThe provisional-output and explicit-confirmation boundary constrains automated authority; it does not cover the full access-control program.
- SA-8 · Security and Privacy Engineering PrinciplesNIST_SP_800_53_5_2_0 · partially addressesThe stop path and decision provenance apply engineering principles to a model-assisted action; they are not a complete engineering review.
- MAP 3.5 · Processes for human oversight are defined, assessed, and documented in accordance with organizational policies from the Govern functionNIST_AI_RMF_1_0 · partially addressesExplicit confirmation and a stop path operationalize human oversight for this use case; they are not an AI RMF assessment.
Mappings indicate contextual relevance or partial support. They do not establish equivalence, certification, government endorsement, or complete framework implementation.
- OWASP Cheat Sheet Series — LLM Prompt Injection Prevention Cheat SheetGUIDANCE_AI_CONTENT_BOUNDARY
- National Institute of Standards and Technology — Artificial Intelligence Risk Management FrameworkGUIDANCE_AUTOMATION
- National Institute of Standards and Technology — Security and Privacy Controls for Information Systems and OrganizationsGUIDANCE_NIST_SP_800_53_5_2_0
- National Institute of Standards and Technology — Artificial Intelligence Risk Management FrameworkGUIDANCE_NIST_AI_RMF_1_0