createif-labs:/agenten$
02, Agents
How we build and run agents
The approach in ten steps, the five deployment models compared, and the reasons agent projects typically fail.
Ten steps to building an agent
Here the order carries the content. Starting at step three means paying a multiple for step two later.
- 01 Use case before technology
- We start with the process and a measurable target, not with the model.
- 02 Evaluation driven development
- How success is measured is settled before implementation. Test sets and automated evaluations run from the start and accompany every change.
- 03 Context engineering
- Deliberate control over what the agent sees at which point: data connections, retrieval of relevant information, limits on context.
- 04 Tool design
- Agents are only as good as their tools. Tightly scoped, robust, well documented, with defined failure behaviour.
- 05 A narrow scope of action
- Permissions, guardrails, output validation, defined abort conditions. Autonomy is widened step by step, never assumed.
- 06 Human in the loop
- Sign off where mistakes are expensive. Full automation where they are not.
- 07 Observability
- Full tracing of decisions, tool calls and cost. Without observability there is no operation.
- 08 Cost and latency
- Model choice per subtask, caching, smaller models where possible.
- 09 Operation and evolution
- Versioning, regression tests on model changes, a defined process for model updates.
- 10 Iterative scope
- A small, usable stage first, then extend. No twelve month project without an interim result.
Deployment and hosting models compared
We run our own GPU infrastructure and know these models from daily operation. None of them is better in general.
| Model | Data control | Cost | Model quality | Latency | Operating effort | Typical case |
|---|---|---|---|---|---|---|
| Local, at your site | complete | hardware upfront | open weights | very low | with you | Data must not leave the building |
| On premise, your data centre | complete | hardware upfront | open weights | low | shared | Integration into existing operations |
| Hosted in Germany | contractually clear | usage based | open weights | low | with us | GDPR clarity without owning hardware |
| Cloud or API | with the provider | per request | highest available | medium | minimal | Speed and model quality come first |
| Hybrid | by data class | mixed | per subtask | mixed | shared | Sensitive work local, uncritical load external |
Choosing the model is part of the advice: open weights against proprietary, size against cost, fine tuning against context control.
Agents that get better in production
An agent that is exactly as good after a year as it was on day one has learned nothing for a year. Self improvement here is not a model rewriting itself, it is a closed loop of measurement, candidate, comparison and fallback.
- 01 Every episode becomes a record
- Approvals, rejections, corrections and blocked accesses are stored with the previous state, the reason and the outcome. A rejected action is the most valuable record, because that is exactly where the distance between the proposal and the right answer sits.
- 02 Corrections become test cases
- Every correction turns into a case with an expected output. The test set grows with operation instead of freezing at handover.
- 03 A candidate, not a change in the running system
- Changes to instructions, tools, rules or the model are built as a versioned candidate next to the running version, never inside it.
- 04 Comparison on the same set
- Candidate and running version are scored under identical conditions. Only what is better on the target metric and worse nowhere else is rolled out.
- 05 Fallback and observation
- The previous version stays runnable. If the metrics diverge in production, the system falls back automatically, with nobody stepping in at night.
- 06 Gaps surface instead of being worked around
- Recurring aborts and missing tools land on the table as a decision, instead of being creatively bypassed by the model.
Self improvement without a test set is drift. A system that changes without measurement does not get better, it only gets different, and nobody can say in which direction.
Why agent projects fail
Six causes we see regularly and what we do instead. All six can be addressed before the first line of code.
Where it goes wrong
No success measure, nobody can say whether it is improving
Scope too wide, no usable interim result
No real data connection, only sample files
No observability, faults go unnoticed
Uncontrolled cost until the first invoice
No operating concept, nobody owns it after launch
What we do instead
The success measure and test set are settled before implementation
The smallest usable stage first, then extend
Connecting the real system is part of the first iteration
Tracing of decisions, tool calls and cost from the start
Cost per request is a metric in monitoring
Versioning, regression tests and a defined update process