03, Agents
An agent at work, step by step
A simulation with an invented company and invented data. It shows the three places where a demo and production part ways: reading only inside the guardrails, writing only after a person approves, and every correction flowing back into evaluation.
03.1
Read, write, learn
The run starts on its own as soon as the panel is in view. At the approval it stops and waits. You can decide there yourself, but you do not have to: when the time runs out the demo takes the rejection branch.
-
An email arrives in the sales inbox
The agent listens to a mailbox. It answers nothing before it is settled who is writing and what that sender is allowed to see.
Hello, when is the delivery for order 4711 arriving? And what would express shipping cost? Best regards, M. Krause
m.krause@stahlbau-krause.example to sales@nordwerk.example -
Five checks before the first record is opened
Guardrails are not a sentence in the prompt that the model may or may not follow. They sit between the agent and the system and decide on every single call.
- Identity sender address is mapped to customer number K-10482
- Read scope orders of this customer only, no third party records
- Fields order header and shipment status, no costing, no margin
- Personal data names and addresses are replaced before the model call
- Call budget four tool calls at most, then hand over to a person
-
Three tool calls, one of them aborted
The second question about express pricing leads into an area this channel may not read. The agent gets no data and no workaround, it gets a refusal. So it does not invent what it does not know.
- erp.read_order(4711) 128 ms in production, promised 12 Sep 2026, delay reported ERP, order header
- logistics.shipment_status("S-88213") 210 ms 40 of 120 units ready to ship, remainder in production Logistics, shipment tracking
- pricing.read("express") aborted field outside the scope released for this channel blocked, Guardrail, field scope
-
A draft reply, every sentence tied to a source
Anything without a source does not make it into the draft. The blocked question does not disappear, it goes to a person.
Order 4711 is in production. 40 of 120 units are ready to ship. The promised date of 12 Sep 2026 cannot be met, we currently expect 26 Sep 2026. Sales will get back to you about express delivery, that is not something I am allowed to answer here.
Draft, not sent yet -
Up to here the agent has only read
The next step changes data in three systems and triggers an email to a customer. From here the agent no longer decides alone, not because it could not, but because a mistake at this point reaches a customer.
- So far
- 3 calls, all read only, one of them aborted
- Now planned
- 2 writes in the ERP, 1 note in the CRM, 1 email
- Approval required
- yes, writes with an outside effect
-
Approval required
The widget is not a notice you dismiss, it is the switch itself. Without a decision the agent does not write. It waits, and if nobody decides, nothing happens.
Planned changes
- Delivery date, order 4711 before12 Sep 2026 after26 Sep 2026
- Order status beforein production afterin production, date corrected
- CRM note beforenone aftercustomer informed about the delay, express request open
Side effects of this approval
- The customer receives the draft as an email
- Planning reschedules calendar week 39
- The request expires after 20 minutes without a decision
Waiting for a decisionWith no decision the demo takes the rejection branch -
Executed and logged
What the agent does is afterwards in the log with a timestamp, a person and the previous state. There is no change without a name next to it, and none that cannot be taken back.
- Transaction
- TX-2026-0912-0447
- Approved by
- S. Ritter, sales support, 09:41
- Written
- 2 fields in the ERP, 1 note in the CRM, 1 email sent
- Undo
- one click, the previous state is stored
Executed
-
Rejected, with a reason
The agent writes nothing. The correction is not a wasted interaction, it is the most valuable record of the day: this is the gap between what the system proposed and what would have been right.
26 Sep is too pessimistic. The 40 units ship on 22 Sep, the rest follows later. Please set this up as a partial delivery and tell the customer accordingly.
S. Ritter, sales support Nothing written
-
The episode turns into three entries
Self improvement does not start with the model, it starts with keeping books on where the model was wrong. Every approval and every rejection is a measurement, including the one from the other branch.
- New test case
- delay with a shippable partial quantity, expected output: propose a partial delivery instead of moving the whole date
- Gap in the tools
- partial deliveries cannot be created through the existing interface, added to the backlog
- Recurring refusal
- express pricing was requested four times in 30 days and blocked four times, put forward as a decision about scope
-
The comparison runs overnight
A candidate that is better in one place and worse in another does not get rolled out. That is the difference between learning and drifting, and it is why self improvement needs a test set rather than optimism.
- Test set
- 412 cases, 37 of them from real corrections over recent months
- Candidate
- the partial delivery rule, extended by the shippable partial quantity case
- Comparison
- candidate against the running version on the same set, same conditions
- Promotion rule
- better on the target metric and no regression across the remaining cases
412 test cases scored, 9 Candidate promoted
-
Version 8 goes live, version 7 stays as the fallback
The metrics keep running in production. If they diverge, the system falls back to the previous version without anyone having to step in at night.
Metric Version 7 Version 8 Verdict Answered correctly on the facts 88.4 % 91.6 % better Proposed date accepted without correction 61 % 78 % better Blocked accesses that got through 0 0 unchanged Cost per case 0.031 euro 0.024 euro better Regressions in the test set none none unchanged Version 8 live
End of the run. What plays out here in a minute is the work of the first few weeks in a real project.
03.2
Four mechanisms, and where they come from in a project
None of these is a property of the model. All four are parts of the architecture and are built in the first weeks, not after the prototype.
- Reading with guardrails
- Permissions, field scope, personal data and the call budget are checked on every single tool call, not agreed once in the instructions. A call outside the released scope is aborted, never replaced by a guess.
- Writing with a widget
- Every write with an outside effect is presented as a comparison of before and after, side effects included. Without approval nothing happens, not even when the clock runs out.
- A log instead of trust
- Every executed change sits in the log with a timestamp, a person and the previous state, and can be taken back. That is what makes it reasonable to grant the agent more room later.
- Self improvement
- Approvals and rejections become test cases. Candidates are scored against the running version on the same set and are only rolled out when they are better without becoming worse elsewhere.
03.3
What this demo is not
This is a script, not a running agent. Names, numbers and systems are invented, nothing is queried, nothing is sent and nothing is stored. What the demo shows is the structure: where an agent in production is stopped, who decides there, and what becomes of that decision afterwards.
What this looks like with your systems, your permissions and your data can be clarified in thirty minutes, far enough for you to know whether it is worth the effort.