AI-NATIVE SERIES · HUMAN DYNAMICS IN AGENTIC AI, PART 2 OF 4
Best Practices That Evolve Slowly: Working With Agents Today
1-minute takeaway
What this is. Part 2 of the series: six practices for people working with AI agents, each tied to a principle from part 1 and to its evidence, from Anthropic and OpenAI to NIST, the EU AI Act and field experiments.
Why it matters. The same AI tool that raised consultants' output quality by more than 40% on one task made them 19 points less likely to be correct on a task just outside its frontier. Knowing which side you're on is the practice to remember.
What you can do after reading. List every action your agent can take, mark each reversible or irreversible, and count how many irreversible ones run without a person seeing them first. If the count isn't zero, that's this week's fix.
Human Dynamics in Agentic AI — the series spine
- Principles that barely move episode 1
- Best practices that evolve slowly this episode
- Emergent trends that move fast episode 3
- coming episode 4
How to read the labels: EVIDENCE is sourced, DESIGN is how the practice is designed, OPEN is not yet tested, and LENS is my own conviction.
You're putting agents to work beside people. Six practices, from labs, regulators and field experiments, turn part 1's principles into rules for this quarter. The surprising one: AI's frontier is jagged, so the same tool that makes consultants 40% better on one task makes them worse on the next. Start simple, gate the irreversible, and know which side you're on.
What this episode covers: best practices for people working with AI agents
Part 1 named six principles that barely move. This layer moves, but slowly: practices that took years of deployment to settle and will probably hold for a few more. It's for the people writing the rollout plan: what to do, in what order, and what the evidence behind each step actually says.
Make a bet first
In a 2023 field experiment, 758 consultants at a top strategy firm worked on realistic tasks with or without GPT-4. On a task deliberately chosen to be just outside what the AI could do well, the consultants using AI were:
- A. About as accurate as those without it.
- B. A little more accurate.
- C. Much less accurate.
Practice 1: start with the simplest thing that works
EVIDENCE Anthropic's guidance on agents: "we recommend finding the simplest solution possible, and only increasing complexity when needed," because "agentic systems often trade latency and cost for better task performance" (Anthropic, Building effective agents, 2024). OpenAI's guide, published a few months later, says the same in its own words: "Our general recommendation is to maximize a single agent's capabilities first" (OpenAI, A practical guide to building agents, 2025).
LENS When two competitors give the same advice, it's usually because they've both watched the alternative fail. A single agent with good tools is easier to check, which is part 1's Principle 2 turned into an architecture.
Practice 2: gate the irreversible
EVIDENCE The same OpenAI guide names when a person should step in: "Actions that are sensitive, irreversible, or have high stakes should trigger human oversight until confidence in the agent's reliability grows," with examples like "canceling user orders, authorizing large refunds, or making payments." Anthropic describes agents that "pause for human feedback at checkpoints or when encountering blockers."
DESIGN The rule that falls out is simple. Sort every action an agent can take into reversible and irreversible. Let the reversible run. Put a person, with time to actually think, in front of the irreversible. Then let the gate loosen only as the agent earns it, on evidence, not on enthusiasm. "It's been fine so far" is what everyone says, right up until the refund that wasn't.
Practice 3: say what it can do, and narrow it when unsure
EVIDENCE Microsoft's research team proposed "18 generally applicable design guidelines for human-AI interaction," checked by 49 design practitioners against 20 AI products. The first is "Make clear what the system can do." The tenth is "Scope services when in doubt" (Amershi et al., CHI 2019).
These sound obvious, and that's the point. Introducing an agent as able to "help with anything" is like introducing a new hire as "good at stuff." LENS In my experience, failed rollouts often skip them: the agent is introduced as able to "help with anything," people try it on the wrong thing, and trust collapses before it was ever calibrated.
Practice 4: design against over-reliance, on purpose
EVIDENCE Standards bodies and regulators now name the failure explicitly. NIST's voluntary generative AI profile warns that "humans may over-rely on GAI systems… This phenomenon is an example of automation bias, or excessive deference to automated systems" (NIST AI 600-1, 2024). The EU AI Act requires high-risk systems to be designed so "that they can be effectively overseen by natural persons," and that overseers "remain aware of the possible tendency of automatically relying or over-relying on the output" (EU AI Act, Article 14).
That is part 1's "misuse," written into law. DESIGN In practice: show the agent's confidence and sources, make "I disagree" as easy as "approve," and measure how often people override. If the answer is never, either the agent is perfect or nobody is reading. Bet on the second.
Practice 5: know where the frontier is, and work it as a centaur or a cyborg
EVIDENCE Back to the bet. In that experiment, consultants with AI "completed 12.2% more tasks on average, and completed tasks 25.1% more quickly," with results "more than 40% higher quality." But on the task outside the AI's frontier, consultants using AI "were 19 percentage points less likely to produce correct solutions" (Dell'Acqua et al., HBS working paper, 2023). The answer is C. If you remember one practice from this part, make it this one: know which side of the frontier you're on.
Part 1's autopilot, again: the danger is the handback you didn't see coming. The authors called the boundary jagged: AI is excellent at some tasks and quietly wrong at neighbouring ones that look similar. They saw two good ways to work. Centaurs split the work, "dividing and delegating" between person and AI. Cyborgs "completely integrating their task flow with the AI." LENS Both work. What fails is not knowing which side of the frontier you're on, which is why practice 3 matters so much.
Practice 6: expect novices to gain most, and protect how they learn
EVIDENCE In a study of 5,179 customer-support agents (2023 working paper, published in the Quarterly Journal of Economics in 2025), AI assistance raised "issues resolved per hour, by 14% on average," with "a 34% improvement for novice and low-skilled workers but with minimal impact on experienced and highly skilled workers" (Brynjolfsson, Li & Raymond, NBER).
This is the best news in the field and a warning at the same time. The AI was, in effect, passing experts' habits to novices. But part 1's Principle 1 says the novice who never does the work unaided may never become the expert the next system learns from. DESIGN So the practice is two-sided: give juniors the assistance, and also give them deliberate, unassisted practice and explanations, not just answers.
How the practices map to the principles
Each practice is a principle with an implementation:
| Practice | Implements | Evidence | |---|---|---| | Start simple | reliance you can calibrate (principle 2) | Anthropic 2024; OpenAI 2025 | | Gate the irreversible | judgment where it matters (part 1's frame) | OpenAI 2025; Anthropic 2024 | | Say what it can do | calibrated trust from day one (principle 2) | Amershi et al. 2019 | | Design against over-reliance | misuse, prevented (principle 2) | NIST AI 600-1, 2024; EU AI Act Art. 14 | | Know the frontier | the ironies, managed (principle 1) | Dell'Acqua et al. 2023 | | Protect how novices learn | the ironies, prevented (principle 1) | Brynjolfsson, Li & Raymond 2023/2025 |
OPEN Notice what's missing: practices for principles 3 to 6 (contracts, Conway, Goodhart, motivation) are far thinner in the published guidance. That gap is an open question the field hasn't closed. Part 4 takes up two of them, the delegation contract and motivation.
How this could fail
- Practices as a compliance checklist. Ticking "human oversight" without measuring overrides is part 1's misuse in a new form.
- Freezing the gate. If the irreversible gate never loosens, people learn to route around it.
- Mistaking one study for a law. The consulting and call-centre studies are strong, but each is one setting. Retest in yours.
Your move this week
List every action your agent can take. Mark each reversible or irreversible. Count how many irreversible actions run without a person seeing them first. If the count isn't zero, you've found this week's fix.
Next
Practices move slowly. Part 3 is about what's moving fast right now: junior hiring, deskilling, agents with their own identities, and why several headline numbers from 2025 have already gone stale.
Sources
- Anthropic (2024). Building effective agents. https://www.anthropic.com/engineering/building-effective-agents
- OpenAI (2025). A practical guide to building agents. https://cdn.openai.com/business-guides-and-resources/a-practical-guide-to-building-agents.pdf
- Amershi, S. et al. (2019). Guidelines for human-AI interaction. CHI 2019. https://www.microsoft.com/en-us/research/uploads/prod/2019/01/Guidelines-for-Human-AI-Interaction-camera-ready.pdf
- NIST (2024). AI 600-1: Generative AI Profile. https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf
- NIST (2023). AI Risk Management Framework 1.0. https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf
- European Union (2024). AI Act, Article 14: Human oversight. https://artificialintelligenceact.eu/article/14/
- Dell'Acqua, F. et al. (2023). Navigating the jagged technological frontier. HBS working paper 24-013. https://mitsloan.mit.edu/sites/default/files/2023-10/SSRN-id4573321.pdf
- Brynjolfsson, E., Li, D. & Raymond, L. (2023; QJE 2025). Generative AI at Work. NBER w31161. https://www.nber.org/papers/w31161