Insights / Production Engineering
Task-Specific AI Agents: Build Around One Job
Plan a focused AI agent inside your app: choose its job, connect approved data, design review, test actions and measure useful outcomes.
10 min read
Loading insights
Loading insights
Loading insight article
Insights / Production Engineering
Plan a focused AI agent inside your app: choose its job, connect approved data, design review, test actions and measure useful outcomes.
An AI agent becomes easier to build and judge when its job fits into a sentence. “Prepare a matter-status update for the responsible lawyer” gives a product team something concrete to design. “Help everyone with everything” leaves the data, permissions and definition of success unresolved.
A task-specific agent works within a particular part of an application. It can gather relevant information, choose among permitted steps and prepare an outcome. The surrounding software decides which records it may use and which actions need a person. That division matters as much as the model.
This guide explains how to turn a useful task into a product feature: select the work, define its boundaries, design the review experience and earn permission to do more. The legal example is fictional. The same decisions apply to other enterprise applications that handle sensitive information.
Before choosing tools, describe what the user should have at the end. A usable draft? An updated internal field? A message sent to someone outside the organisation? Those outcomes carry different obligations.
For a matter-status assistant, a sensible first outcome might be an editable draft that shows the records behind each factual statement. A responsible lawyer decides whether to send it. The initial release does not need permission to email clients, change deadlines or interpret the legal significance of a document.
Write a short task agreement that product, engineering and the workflow owner can read together:
Scroll horizontally to view all columns.
| Decision | Fictional matter-status example |
|---|---|
| User and starting point | A staff member requests an update from a matter page |
| Finished work | An editable summary with links to the supporting records |
| Allowed information | Records the requesting user may access for that matter |
| Missing information | Mark it as missing and ask for clarification |
| Permitted action | Save a draft for review |
| Reserved decision | A lawyer approves the wording and any external communication |
| Operating owner | A named product or operations owner receives failures and feedback |
Keep exclusions beside the task, where future contributors can see them. Otherwise, a small improvement such as “also send the update” can quietly turn a preparation feature into a delegated action system.
Repetition helps, but frequent work is not automatically a good agent candidate. You also need a way to tell whether the result is useful and whether the system stayed within its authority.
Compare candidate tasks against concrete questions:
A summarisation task may fail the review test if its source material is so scattered that checking every statement takes longer than writing the summary. Fixing document access and evidence links could be the first useful project.
Do not add an agent just because a task involves text. Anthropic distinguishes predefined workflows from agents that choose their own steps. If the route is known, ordinary application code with a small model-assisted step may be sufficient. That distinction helps keep the first release understandable.
A separate chat window can be useful for exploration. For recurring operational work, the current record often provides a clearer starting point: the matter, support request, document or internal approval item.
The interface should make three things visible before a user accepts an output:
For the fictional matter assistant, place the draft beside the supporting records. Mark unsupported statements as unresolved rather than filling gaps with plausible text. Let the reviewer edit, reject or request more information. A “save draft” action and a “send update” action should have separate labels and permissions.
Review needs an owner and a queue. If the reviewer is unavailable, the work should remain pending or follow an agreed escalation route. A waiting item should never become approved merely because time has passed.
An agent needs tools that express the task in application terms. “Save this approved draft to this matter” is easier to constrain than unrestricted database access or a general command executor.
The backend should independently verify the requesting user, organisation, record and operation. Passing a record ID through the model does not establish permission to access it. Retrieval, cached results and tool calls all need to respect the same access boundary.
Scroll horizontally to view all columns.
| Capability | Useful first boundary | Evidence to retain |
|---|---|---|
| Retrieve records | Read authorised records for the selected matter | Source IDs, versions and retrieval time |
| Prepare text | Produce a draft with unresolved facts marked | Draft version and source references |
| Save work | Save to the permitted draft location | Request ID, destination and result |
| Communicate externally | Require approval for the exact message and recipients | Approved version, reviewer and delivery result |
OWASP’s agent security guidance covers scoped tools, input handling and human controls. In a product implementation, those controls belong in the application and tool layer. Instructions inside a prompt are useful context, but they are not an authorisation mechanism.
Treat retrieved documents and messages as information that may contain hostile instructions. A paragraph in a document must not grant a tool new permissions. Keep credentials outside model-visible content and validate tool arguments against the allowed operation.
An approval should identify the thing a person reviewed. If a recipient, attachment or material fact changes afterwards, the application needs to check whether approval still applies.
In the sample workflow, the review could be tied to a draft version, matter ID, recipient list and relevant source versions. The system checks those values again immediately before sending. A material change returns the action to review.
This is particularly useful when several people can work on the same record. A reviewer may approve a correct summary while another user changes the underlying matter status. The app needs a defined response to that conflict; the model cannot resolve it by sounding confident.
Also distinguish “approved” from “completed.” A tool timeout may leave the result unknown. Display that uncertainty and reconcile with the destination before repeating an external action. A second attempt must not create a duplicate message or record.
An answer can look good while the system has accessed the wrong matter or changed an unapproved field. Test the visible result, the application state and the permitted actions separately.
Start with representative examples agreed with the workflow owner. Keep development examples separate from a held-out set used to check changes. Include cases that should end without a completed draft.
Scroll horizontally to view all columns.
| Test case | Expected behaviour |
|---|---|
| Complete, consistent source records | Produces a supported draft without adding facts |
| Missing update date | Shows the gap and requests clarification |
| Conflicting source records | Makes the conflict visible for review |
| Record from another organisation | Denies access without revealing the record |
| Instruction hidden in a retrieved document | Does not treat the instruction as authority |
| Source changes after approval | Rechecks or returns the action to review |
| Destination times out | Records an uncertain result and checks before retrying |
Anthropic’s evaluation guidance explains why an agent’s final state and execution record both matter. A useful product test suite combines deterministic checks for permissions and state with human review of content quality. A second model can assist assessment, but should not be the only judge of sensitive factual work.
Repeat important cases because model behaviour can vary. Track failures by type so an improvement to wording does not hide a regression in access control or tool use.
Increase the agent’s scope only when the evidence supports the next permission. Test mode, shadow operation and draft preparation answer different questions; none proves that unattended action is ready.
For shadow operation, compare the agent’s suggestions with normal work without allowing its output to affect live decisions. For draft release, check whether reviewers can detect mistakes and whether the queue stays manageable. Before limited actions, test cancellation, duplicate handling, permission changes and recovery.
A multi-agent design adds coordination and trust boundaries. Use it when the task needs genuinely separate responsibilities and the benefit survives testing. A single bounded agent or a fixed workflow is often easier to inspect at the beginning.
Measure the whole task rather than the number of generated responses. Record preparation time, review time, correction reasons, unresolved items and downstream rework alongside model and infrastructure costs.
Useful questions include:
Avoid counting all saved minutes as cash savings. Released capacity, shorter waits and reduced rework are different benefits. State which one the pilot actually demonstrates.
A native platform feature may be sufficient when the task stays in one system and its access, review and recordkeeping meet your needs. A custom integration becomes more useful when the task crosses systems or requires a review experience the platform cannot express.
Before committing either way, run the same sample cases against the option. Check data handling, exportable records, permission changes and the ability to stop it. Include maintenance when comparing costs: source changes, model changes, failed connections and review feedback all need an owner.
For a product team, the next useful deliverable is a short task agreement, a working review screen and an evaluation set. Those give AI agent development a concrete scope. The governance launch checklist covers the controls to check before expanding authority.
Tell me about the task, the systems involved and who needs to review the work. I can help define a useful first release.
Discuss your agent(opens in new tab)Yes. A button on a record, an inbox action or a queued job can start the task. The interface should show the prepared work, its evidence and the next decision in a place users already understand.
Usually the task boundary comes from the application, instructions, authorised data and tools. A separately trained model is a different decision and needs evidence that the existing approach cannot meet the task’s requirements.
Set review according to the consequences of an error. A first release can prepare work while a person approves consequential actions. Test whether reviewers can actually spot problems and handle the volume before reducing oversight.
It should leave a clear status, preserve useful work where permitted and route the item to a named person or queue. Include the reason and the last confirmed action so someone can continue without guessing.
Only if separate responsibilities improve the result enough to justify additional coordination, latency and failure paths. Prove the simpler design first, then compare alternatives using the same cases and operating costs.
It can support scoped preparation and coordination, with access controls, evidence and review. Lawyers retain responsibility for professional judgement, advice and consequential legal actions. This article describes engineering choices, not legal advice.
Tell me about the system or workflow you are working on.
30-minute intro call on Calendly