Human Oversight
Part of Automation programme reviews
Identifying automations that no longer match the process
Compare current business rules, automation versions and recent cases to identify where a live workflow no longer fits its process.
An automation stops matching its process when its trigger, rule, handoff or completion check diverges from the route the business now requires. Compare the owner-confirmed current process, the automation version and recent cases to find the difference. A successful run can still apply an outdated rule.
Start with a specific signal
Repeated corrections, overrides or unexpected routing can prompt a review. A changed form, approval rule, customer requirement or connected application can do the same. None proves a mismatch alone.
Choose the affected case type and establish what result should now occur.
Bring together the rule confirmed by the process owner, the configured automation version and recent case histories. Note when each rule took effect. Assess a past case against the rule applicable when it was processed before calling it a defect.
Steps to Identify Mismatched Automations
- Identify the signalLook for repeated corrections, overrides or unexpected routing
- Confirm the intended outcomeDefine what result should occur for the affected case type
- Gather evidenceCollect process owner’s rule, automation version, and recent case histories
- Assess past casesReview cases processed before the change; avoid labelling as defects prematurely
Compare the route point by point
| Point | Question |
|---|---|
| Entry | Are the same case types eligible, and is the required input available? |
| Decision | Does the configured rule match the currently authorised rule? |
| Handoff | Does the next team receive the information and responsibility it needs? |
| Exception | Can a case outside the routine route be recognised and held? |
| Finish | Does the recorded completion state still represent the intended business result? |
Inspect ordinary cases and cases near a changed boundary. Follow them to the destination or receiving team.
An event log may reveal a path or interval, but it may miss work done by phone, email or a separate register. Confirm what a surprising event means with the people who handled the case.
Current Process vs Automation Configuration
- Entry
- Check if same case types are eligible and required input is available
- Decision
- Verify configured rule matches current authorised rule
- Handoff
- Ensure next team receives correct information and responsibility
- Exception
- Confirm cases outside routine route can be recognised and held
- Finish
- Validate recorded completion state reflects intended business result
Classify the difference
The cause may be an outdated business rule, an upstream input change, a new receiving-team requirement or a technical mapping that implements the agreed rule incorrectly. Record what the cases support and what remains uncertain.
If the business process legitimately changed, its owner should approve the intended route before technical changes. If staff use an informal workaround, establish why before copying it into the automation. If records do not retain decisions and handoffs, use a bounded case review rather than presenting a process map as complete evidence.
For a consequential mismatch, identify cases processed since the relevant change and give them an owner for outcome review. Hold new affected cases or use an authorised route where continuing the old rule could produce an unacceptable result. Then specify the proposed change and examples of cases that should pass, stop or reach a person.
Record the mismatch with its date, affected cases, current and configured rules, observed outcomes, owner and next action. Whether to redesign or retire the automation is a later decision.
Key Questions When Classifying a Difference
- Was the business rule updated?Check if the process legitimately changed and was approved by the owner
- Is there an informal workaround?Investigate why staff use alternative methods before automating them
- Are records incomplete?Use bounded case reviews if decisions and handoffs aren’t documented
- What’s the impact?Identify cases processed since the change; assign owners for outcome review
Action Required After Identifying a Mismatch
- Cases Affected
- Identify all cases processed since the relevant change
- Owner Assigned
- Assign an owner for outcome review of affected cases
- Next Step
- Decide whether to redesign or retire the automation
