Most agentic AI engineer job descriptions are a list of framework names. That screens for people who have read the documentation and against people who have been paged at 3am by a loop that would not stop. Here is a version that does the opposite.
The template
Agentic AI Engineer
You will design and ship systems where a language model plans, calls tools, and takes real actions against real data - and you will make those systems measurable, safe and cheap enough to run at our volume.
What you will do
- Design agent control flow: state, transitions, budgets, retries, escalation to a human.
- Build and harden tool integrations against our internal systems, with validated, idempotent contracts.
- Own the evaluation harness: golden sets from real traffic, regression gates in CI, quality reported per release.
- Model and manage cost and latency per completed task, not per token.
- Build guardrails against prompt injection, tool abuse and data exfiltration, and test them adversarially.
- Instrument everything: traces you can replay, failure taxonomies, dashboards our on-call can read.
- Work with product and operations to decide what the agent must never decide alone.
What we look for
- You have shipped a non-deterministic system to production and kept it there.
- You reason about a language model as an unreliable component in a distributed system, not as an oracle.
- You measure before you tune, and you can show us the eval set you built.
- You are comfortable deleting an abstraction that is not paying for itself.
- Strong general engineering: Python or Go, queues, APIs, databases, observability.
Nice to have
- Retrieval systems at scale, including permission-aware retrieval.
- Open-weight model deployment, self-hosted inference, or on-premise constraints.
- Security background, or experience red-teaming model-driven systems.
- Public writing or open source in this space.
Cut freely. Every line you cannot honestly offer costs you candidates.
The skills that actually predict success
| Signal | Why it predicts | How to test it |
|---|---|---|
| Has built an eval set | Everything downstream depends on being able to measure | Ask to see one. Ask what they would label and who would label it |
| Thinks in budgets | Loops and cost are the two failure modes that reach the CFO | Ask how they stop an agent that will not terminate |
| Designs tool contracts | Most “the model is dumb” bugs are interface bugs | Give them a bad tool signature and ask what is wrong with it |
| Has a threat model | Prompt injection is a design concern, not a filter | Ask how a retrieved document could take over the agent |
| Deletes things | Agent codebases rot into layers of unused abstraction | Ask what they removed from their last system and why |
| Reads traces | Diagnosis beats intuition after week two | Hand them a real trace with a failure in it |
An interview loop that works
- Portfolio walkthrough, 45 minutes. A system they shipped: architecture, what broke, what they measured, what they would do differently. Push until you hit something specific. People who have only demoed cannot get past the second follow-up.
- Design exercise, 60 minutes. “An agent that processes incoming invoices against our policy and pays the ones under $500.” Watch for: where they put the human, how they scope tool permissions, what they refuse to automate, what budget they set, and how they would know it works.
- Debugging exercise, 60 minutes. Give them a real trace where an agent looped, or retrieved the wrong document, or spent forty dollars on one task. Diagnosis quality is the single most transferable signal in the loop.
- Code review, 45 minutes. A tool implementation with an ambiguous contract, an unbounded retry and a missing idempotency key. Do they find all three, and do they explain why each matters?
- Values and collaboration, 30 minutes. This role says “no” to product a lot. Make sure they can do it without being insufferable about it.
Skip the algorithm screen. It tests nothing this job requires.
Five ways this hire goes wrong
- Hiring a researcher for a production role. Impressive background, no instinct for on-call. Ask what they have operated, not what they have read.
- Screening on framework names. The frameworks in your JD will be replaced within eighteen months. The judgement will not.
- No counterpart on the product side. Agentic systems make policy decisions - what to automate, what to escalate. Without a product owner, the engineer ends up guessing at your risk tolerance.
- Hiring before the use case is proven. You will spend a quarter recruiting for a system whose requirements change in month two. Contract first if you are still learning.
- No budget for evals. If the first roadmap has no line item for an evaluation harness, the hire will spend a year tuning prompts by feel and you will never know whether it is getting better.
If you would rather not run this search
Contracting an agentic AI engineer gets a system into production while the search runs - and leaves your eventual hire with a working codebase, evals and runbooks instead of a blank repository. Twenty-minute call to talk it through.