Loading insights
Loading insight article
Insights / Production Engineering
Design AI workflow automation around clear states, approved actions, human review and recovery, then test whether it improves the whole task.
An automation demo usually follows the happy path: a request arrives, AI prepares something useful and the next system accepts it. Real work includes missing documents, duplicate requests, unavailable reviewers and connections that stop responding halfway through an update.
AI workflow automation uses model capabilities inside a business process. A model can classify an unfamiliar message, extract information or prepare a draft. Application rules still need to decide who may see the information, whether the next action is allowed and what to do when the process cannot continue.
The practical goal is a completed task that people can inspect and recover. This guide uses a fictional document-request workflow to show how to select the right automation, design its handoffs and measure the work left for people.
Start by following a real item from arrival to completion. Ask the people doing the work where it waits, which facts they check and what makes them send it backwards.
For a document request, the visible job may be drafting a message. The harder work could be confirming the matter, checking which documents are already present and identifying the authorised recipient. Automating the wording alone would leave those delays intact.
A useful process map records the trigger, source of truth, responsible person, allowed output and common exceptions for each step. Include work outside the main application, such as a clarification obtained by phone. Otherwise the new workflow may assume facts that only exist in someone’s memory.
Define completion in operational terms. “Draft generated” is different from “reviewed request delivered to the correct person.” Use the state that matters to the workflow owner when evaluating the pilot.
Different parts of a process need different methods. Fixed rules are suitable when the decision is explicit. An integration moves information between systems. Retrieval finds relevant material. A model helps interpret or produce text. An agent is useful where selecting the next step requires some flexibility within boundaries.
Scroll horizontally to view all columns.
| Work to be done | A starting approach | A reason to add more |
|---|---|---|
| Route a known form by its selected department | Rules | Free-text requests need interpretation |
| Copy an approved status between systems | API integration | The source needs extraction or reconciliation |
| Find a policy paragraph for a reviewer | Permission-aware retrieval | The user also needs a supported explanation |
| Draft a request from approved facts | A model inside a fixed workflow | The task needs different permitted follow-up steps |
| Investigate why a request is incomplete | A bounded agent | The next useful information source varies by case |
These approaches can coexist. A model may interpret a message, while code validates the extracted fields and routes an uncertain result to a person. The model does not need to control the whole process.
This follows the workflow-versus-agent distinction in Anthropic’s architecture guidance. Choose flexibility where the task needs it, then test its cost and reliability against a simpler implementation.
An item waiting for review should look different from one that failed to send. Clear states let users understand what happened and let the software choose a safe next step.
In the fictional document workflow, a small state model could be:
Scroll horizontally to view all columns.
| State | Meaning | Who or what moves it forward |
|---|---|---|
| Received | The request has an identity and a recorded source | Validation checks |
| Needs information | Required facts are missing or inconsistent | A staff member supplies the missing facts |
| Preparing | Approved records are being used to draft the request | The preparation step finishes or fails |
| Awaiting review | A specific draft is ready for inspection | An authorised reviewer |
| Ready to send | The exact action has been approved | Final permission and freshness checks |
| Sent | The destination has confirmed acceptance | Follow-up process, if required |
| Outcome unknown | The attempt may have succeeded, but confirmation is missing | Reconciliation with the destination |
| Closed without sending | The request was cancelled or rejected | An explicit new request if work resumes |
Choose names that users can understand. Record when and why a state changed. Do not represent “outcome unknown” as an ordinary failure that automatically starts again.
A human review may take longer than an HTTP request or a browser session. Save the workflow’s state so closing a tab, restarting a worker or deploying the application does not lose the pending decision.
A durable workflow engine is one option for processes with long waits and many failure paths. Temporal’s workflow documentation describes persisted execution that can continue across failures. A simpler system can use a database-backed state machine and job queue when that fits the scope. Both need deliberate rules for retries, cancellation and changed inputs.
Keep external effects such as sending messages separate from deciding the next state. Make jobs identify the work they belong to, and prevent two workers from claiming the same operation without coordination.
Define what cancellation means for queued, running and already completed work. A cancellation received after delivery cannot unsend a message. The interface should communicate that limitation rather than marking everything as undone.
When an API request times out, the destination might have completed the action before the response was lost. Repeating it immediately could create a second request or message.
Where supported, use a stable operation identifier for retries and have the receiving system recognise repeat attempts. Check the destination’s actual contract: an identifier in your own logs does not make an external API idempotent. AWS explains this distinction through caller-supplied request identifiers and consistent handling of repeated intent.
If the destination offers no duplicate protection, design another check before repeating a consequential action. You may need reconciliation, a unique external reference or manual resolution. Bound retry attempts and delays so an outage does not become a flood of repeated actions.
Also check event delivery between your own systems. If a database update succeeds but its notification is lost, another part of the workflow may never continue. The transactional outbox pattern records the intended event with the local change, then publishes it separately. Consumers still need to handle repeated messages.
A review queue needs enough information for a decision: the proposed action, relevant evidence, unresolved issues and the result of accepting it. Include an assigned reviewer or team, escalation rules and a way to return the item for correction.
For the sample document request, approval covers a particular message, recipient and attachment set. If those change materially, check approval again. A generic “looks good” on a previous draft should not authorise a different outgoing message.
Design the process for reviewer absence and sudden volume. If the queue grows, pause eligible work, narrow the pilot or assign additional authorised reviewers. Do not use unattended approval as a queue-management shortcut.
The review interface should capture useful correction reasons: wrong source, missing fact, inappropriate wording or incorrect destination. Keep sensitive free text out of analytics unless its collection is necessary and approved.
An overall readiness score can conceal a critical gap. Good sample outputs cannot compensate for missing access controls or no owner for failed requests.
Scroll horizontally to view all columns.
| Area | Evidence to bring to a pilot review |
|---|---|
| Process | A workflow map with completion and exception definitions |
| Data | Approved sources, access boundaries and freshness rules |
| Connections | Tested reads and writes with known failure responses |
| Review | A working queue with ownership and escalation |
| Quality | Representative examples with expected results and failure cases |
| Recovery | Demonstrated pause, reconciliation and resumption procedures |
For sensitive workflows, test with sample or appropriately approved data first. Minimise what is sent to the model and what remains in execution records. Follow the same access restrictions during retrieval and downstream actions. OWASP’s agent security guidance is a useful reference where tool-using agents are part of the design.
Capture a baseline before introducing automation. Use comparable types of work and record the exceptions, rather than comparing easy automated items with every difficult manual item.
Measure elapsed time separately from hands-on time. A faster draft may have little effect on completion if the review queue remains the main delay. Conversely, better collection of missing information could improve completion without a dramatic change in drafting speed.
Include model calls, connectors, infrastructure, review, support and correction work in the operating cost. Separate one-time build costs from recurring costs. Then compare cost per accepted result, not cost per model response.
Useful pilot measures include:
Agree the acceptance criteria before reviewing results. A pilot should be allowed to conclude that better source data or a simpler integration is the next investment.
Start with one owner, one defined process and a limited user group. Keep a manual path available for items the automation cannot handle. Introduce live actions only after preparation quality, access checks and review behaviour have been tested.
Before expanding, deliberately interrupt the system: disconnect a tool, submit the same request twice, change a source after review and remove a reviewer’s access. Confirm that each item lands in an understandable state and someone receives the failure.
Connect this work to the existing AI automation service or, for a legal workflow, legal AI engineering. The agent governance checklist is useful when the workflow gains permission to act through tools.
The first useful release is one people can use, review and recover. Broader automation comes after that evidence exists.
Share the process and where work gets stuck. I can help assess the connections, review steps and recovery needed for a scoped pilot.
Discuss your workflow(opens in new tab)It can help with inputs that are difficult to express as fixed rules, such as varied wording or document content. Keep explicit business rules in code where possible and test whether the model improves the task enough to justify its uncertainty and cost.
Sometimes, but the missing connection changes the design and maintenance burden. Consider a file exchange, an approved manual handoff or supported user-interface automation. Test credentials, change detection and failure recovery before relying on it.
Keep the item recorded, show that preparation is delayed and use the agreed retry or manual route. An unavailable model should not cause the workflow to skip required checks or approve an action.
No. Validate any score against representative outcomes and combine it with action risk, missing information and policy rules. A confident output can still contain a factual error or target the wrong record.
Identify each operation, coordinate workers, use the destination’s duplicate protection where available and reconcile uncertain results. Test the failure where the action succeeds but its confirmation is lost.
When the current process meets its agreed quality, operating cost and recovery criteria at the tested volume, and the team can support more work. Reassess data, permissions and review capacity for each additional workflow.
Tell me about the system or workflow you are working on.
30-minute intro call on Calendly
