Control Packs · AUT-004 · v1.0.0
Automated Work Completeness and Accuracy
Decide what a complete and correct automated run looks like before relying on one, so a run that skipped, duplicated, or mishandled work is distinguishable from a run that did the job.
Status: review · Review: not independent. This is design guidance; review status does not establish independent verification or compliance.
What this safeguard addresses
Ensure every automated run states what it was expected to process, reports what it actually processed and what it did not, checks its output against a defined expectation or source of truth before the result is relied on, and surfaces discrepancies to a person instead of absorbing them.
- An automated run is treated as finished without being checkedAn automated job, workflow, or agent finishes without raising an error, and its result is accepted as complete and correct. Nothing states what completeness means for a run, nothing compares the result against a source of truth, and skipped, duplicated, or incorrect items look identical to correctly handled ones.
Versioned applicability rule
{
"any": [
{
"characteristic": "CONSEQUENTIAL_ACTION",
"equals": true
},
{
"characteristic": "TOOL_ACCESS",
"equals": true
}
]
}Decisions for the project owner
Leave a decision open when its value is unknown. Suggested values become confirmed only through an explicit user decision.
- What has to be true for one run to count as complete and correct?"It finished without an error" is not the same as "it did the whole job correctly". Only you can say what the run was supposed to cover and what it should be checked against — a record count, a control total, a source system, a reconciliation.AUT-004-Q1 · scope
- What should happen when a run finishes but some items could not be processed?Partial completion is the normal case at scale. Continuing silently, holding the whole run, and reporting-and-proceeding are all defensible, but they lead to different implementations and cannot be chosen for you.AUT-004-Q2 · choice
Requirements for the coding agent
- AUT-004-R1For each automated run, record the intended work set, the items processed, the items skipped or failed with a reason, and the outcome of the confirmed completeness and accuracy check. Make a re-run safe against duplicates, and do not report a run as successful when its own check did not pass.
- AUT-004-R2Implement the confirmed behaviour for a run that could not process every item, surface the exceptions to a person with enough detail to act on them, and never discard an unprocessed item without recording it.
Tests and evidence to retain
- RUN_COMPLETENESS_TESTRun against a known work set with a deliberately missing, duplicated, and malformed item, and confirm the completeness and accuracy check fails rather than reporting success.
- PARTIAL_RUN_TESTInterrupt a run mid-way and re-run it, and confirm the confirmed partial-completion behaviour, no duplicate effects, and an exception record for every unprocessed item.
- configuration_or_policy
- implementation_location
- test_result
- telemetry_definition
- operational_procedure
Passing a published example shows that example's behavior. A coding agent's implementation report remains a claim until its evidence is independently checked.
Related guidance
- SI-7 · Software, Firmware, and Information IntegrityNIST_SP_800_53_5_2_0 · partially addressesChecking automated output against a defined expectation is an integrity check on information the system produced; it does not cover the full control family.
- CA-7 · Continuous MonitoringNIST_SP_800_53_5_2_0 · informsPer-run completeness and exception reporting supplies monitorable evidence for this automation; it is not a continuous-monitoring programme.
- AU-2 · Event LoggingNIST_SP_800_53_5_2_0 · informsRecording expected, processed, and exception counts per run supports later reconstruction of what an automated run did; it is not a complete audit programme.
Mappings indicate contextual relevance or partial support. They do not establish equivalence, certification, government endorsement, or complete framework implementation.
- National Institute of Standards and Technology — Artificial Intelligence Risk Management FrameworkGUIDANCE_AUTOMATION
- National Institute of Standards and Technology — Security and Privacy Controls for Information Systems and OrganizationsGUIDANCE_NIST_SP_800_53_5_2_0
- Internet Engineering Task Force — HTTP SemanticsRESEARCH_IDEMPOTENCY