AI-Native Series · Control for Agentic Systems
The Order Ticket
1-minute takeaway — what you'll walk away with
Prompt engineering is the one layer of the four with real survey coverage, and the season starts here to be fair to it. Its unit of work is a single model call, which is also its ceiling: no rewording lets a prompt observe what happened after the answer was returned. The most interesting result in the prompt literature is not a technique — it is Promptware Engineering's argument that prompts need versioning, tests and owners. That is not advice about wording. It is advice about the machinery around the wording, which is the next episode.
Cold open
A busy kitchen. You cannot enter it. You cannot touch a pan, taste a sauce, or tell the cook to try again. You have exactly one instrument of control: a paper slip that goes through a slot.
Table 7. Chicken. No peanuts. Medium spicy. Twenty minutes.
Everything you are able to influence has to fit on that slip. This is prompt engineering, and the constraint is not a metaphor — it is the literal interface. One model invocation, one slip.
The reason this episode is first and short is that this is the mature layer. It is the one with a real literature, real surveys, and real methods. The rest of the season is about what happens when the slip stops being enough.
The story world
The kitchen has one rule that governs the whole season: you are outside it. You do not control the cook's process, only what is written down and what comes back through the window. Every later episode is an attempt to get more control without being allowed inside — first by rebuilding the kitchen (Episode 2), then by watching the plates come back and re-sending slips (Episode 3), then by running many kitchens at once (Episode 4).
1:1 technical map
| Kitchen | Technical name | What you actually control |
|---|---|---|
| The paper slip | the prompt | instructions, examples, output format, role framing |
| "No peanuts" | a constraint | a hard requirement stated in-band |
| "Medium spicy" | an under-specified field | the biggest single source of variance |
| Two example plates on the pass | few-shot exemplars | in-context learning |
| "Twenty minutes" | a budget | the only stop condition available at this layer |
| The window the plate comes back through | one completion | your entire observation channel |
The claims, classed
[DEF] Prompt engineering is the design of the instruction, examples, output contract and role framing for one model invocation or a short interaction (arXiv:2401.14423, Prompt Design and Engineering). Its unit of work is a single call.
[EVIDENCE] This is the layer with genuine survey coverage. Prompt Design and Engineering (arXiv:2401.14423) covers the core methods and reaches into agent-adjacent techniques including ReAct, ReWOO and DERA — which is worth noticing, because it means the "agent loop" material was already being treated as part of the prompt literature before it acquired its own label.
[DESIGN] Promptware Engineering (arXiv:2503.02400) makes the move that matters for this season: it argues prompts should get the ordinary apparatus of software — requirements, testing, versioning, maintenance. Note the shape of that argument. It does not say prompts need better wording. It says prompts need process discipline around them. That is already a step out of the kitchen and toward the harness.
[OPEN] How much of what we attribute to prompt quality is actually harness quality — retrieval, context assembly, tool availability — is not cleanly separated in the practitioner literature. When a "better prompt" ships alongside a changed context window, the attribution is unresolved.
Prediction Gate
Before you read on, commit to an answer. Write it down; the point is that it can be wrong.
A team improves an agent's task success from 61% to 78%. They changed two things in the same week: they rewrote the system prompt, and they added a repository-file retrieval step.
Bet: which change carried most of the gain, and what single experiment would settle it?
Most people bet on the prompt, because the prompt is the thing they can read. The experiment that settles it is an ablation — ship the retrieval with the old prompt — and the reason it is rarely run is that nobody wants to spend a week proving their own prose did not matter.
Failure Room
Break it deliberately: remove the output contract and keep everything else.
Take a prompt that reliably returns {"status": ..., "retryable": ...} and delete the schema line, leaving the task description intact. The model still does the task. Prose still describes it. What breaks is everything downstream — the parser, the retry logic, the evaluator that reads retryable.
The diagnosis matters more than the breakage: at this layer, a failure of format is indistinguishable from a failure of capability until something downstream tries to read the output. That is the first symptom of the missing layer this season is about. The slip was fine. The kitchen was fine. There was no instrument watching the window.
Lab — 15 minutes
Treat one prompt as a specification under test.
- Take a prompt you rely on. Write down, as assertions, what its output must satisfy — fields present, types, enum values, length bounds.
- Run it 20 times at temperature 0 and count assertion failures. Then 20 at your production temperature.
- Now change exactly one word of the instruction (not the schema) and repeat.
You are measuring what most prompt work never measures: variance, not quality. A prompt with a better mean and a worse variance is usually a worse prompt for a system that has to parse it.
Reality Mission
Find the highest-traffic prompt in something you own. Give it a version number, a test file, and an owner. That is the entire recommendation of Promptware Engineering (arXiv:2503.02400) reduced to one afternoon, and it is the cheapest durable win at this layer.
Research Challenge
Use an agent to do the attribution study nobody runs. Take a public agent benchmark harness, hold the prompt fixed and ablate the context-assembly step; then hold context fixed and ablate the prompt. Report the two deltas side by side. The claim to test: in a system with retrieval and tools, prompt edits explain less of the variance than practitioners assume. That is currently [OPEN], and it is measurable.
Exit test
You can leave this episode when you can answer all three without looking:
- What is the unit of work at this layer? (one model invocation)
- Name the one thing this layer cannot do, no matter how well written the slip is. (observe what happened after the plate left the window)
- What did Promptware Engineering actually argue for — better wording, or surrounding process?
Cliffhanger
The slip is as good as a slip can be. The schema is enforced, the constraints are explicit, the budget is stated. And the cook still cannot read the file you meant, cannot run the test you care about, and cannot tell you why the plate came back wrong.
So you stop writing better slips and start building the kitchen. That is where the vocabulary begins to slip, too — because the next word everyone reaches for is harness, and its most-cited definition says something that quietly breaks the tidy diagram we all draw.
→ Episode 2 — The Body You Give It
Read next
Episode 1 of Intelligence Engineering Adventures, Season 6 — The Governor Engine. Claims in the series source are tagged by class — definition, derivation, evidence, engineering choice, open question — and a metaphor may introduce a claim but never serves as evidence for it. The season takes the four-label diagram seriously enough to check it against its own primary sources, and it does not survive: Anthropic's definition makes the harness the loop rather than a layer above it. Every claim is tagged by class, and the one genuinely open problem is stated as open — nobody has published a stability criterion for an LLM control loop. This article contains no material from any employer or client. — Paul Jialiang Wu · agentic-portfolio-lovat.vercel.app
References
- Prompt Design and Engineering: Introduction and Advanced Methods — arXiv:2401.14423. Survey of prompt methods, including ReAct, ReWOO and DERA. Verified 2026-08-24 via arXiv.
- Promptware Engineering: Software Engineering for Prompt-Enabled Systems — arXiv:2503.02400. Note: the brief that prompted this season cited this as "Software Engineering for LLM Prompt Development"; the ID is right and the title is not. Verified title as printed above.
- Anthropic, Building Effective AI Agents — separates predefined orchestration ("workflows") from systems where the model dynamically controls process and tool use ("agents"). https://www.anthropic.com/engineering/building-effective-agents