METAHEURISTIC
When an AI system fails in production, the cause is often not the model. The cause is the process. People did the work with rules that were never written: exceptions for some customers, verbal instructions from years ago, judgment calls that nobody explained. The AI followed the written process exactly. The written process was incomplete.
A short video from 2026 described a typical case. A medical practice put an AI voice system on the front desk. The AI answered every call. Then the front desk work broke. The routing rules were verbal instructions from years ago. Nothing was documented. The video listed four gaps. Scheduling needed judgment. Nobody wrote down the intake exceptions. There was no rule for urgent handoffs. Calls after hours had no escalation. The AI performed. The undocumented workflow under it failed.
This page explains how to find the unwritten rules before the launch.
Why does AI expose undocumented work?
A human worker learns the unwritten rules over months. A new colleague explains them. A customer complains, and the worker remembers. A manager says “for this client, always call first.” None of this goes into a document.
An AI system does not learn this way. It only knows what you give it: the prompt, the documents, the data and the tools. If a rule is not in these sources, the rule does not exist for the AI.
A human worker
- Learns rules from colleagues, mistakes and memory
- Handles exceptions with judgment
- Asks when not sure
An AI system
- Knows only the prompt, the documents and the data
- Handles exceptions only if they are written
- Often does not ask, unless designed to
This is not a weakness of AI only. The same problem occurs when you outsource a process or hire a new team. AI makes it faster and more visible.
What types of unwritten rules are there?
Each type needs a different method to find it.
How do I find the unwritten rules?
Use these five methods. Each one finds different rules. Use at least three of them.
Method 1: Shadow the work
Sit with the people who do the task for half a day. Watch, do not interview. Write down each decision that they make. Ask “why?” after each decision.
People cannot always describe their rules in an interview. They forget the rules that they use without thinking. Observation finds these rules.
Method 2: Collect the exceptions
Ask the team: “When does the normal process not work?” Then ask for examples from the last month. Write each example down with the action that the person took.
Ask about these situations specifically:
- Customers or cases that get special treatment.
- Cases where they ask a manager.
- Cases where they break the written rule.
- Cases that came back as complaints.
Method 3: Read the history
Look at the real records: tickets, emails, notes in the CRM, call logs. Take a random sample of 100 cases. For each case, check if the written process explains the action. Each case that the written process does not explain shows a hidden rule.
Method 4: Ask the new people
People who joined the team recently remember what they had to learn. Ask them: “What did you learn in your first months that is not in any document?”
Method 5: Run the AI in shadow mode
Run the AI system on real cases, but do not use its output. Compare its output with what the human did. Each difference is either an AI error or a hidden rule. Review each difference with the team.
Shadow mode is the most reliable method. It finds rules that nobody remembers. It also gives you real data for the eval set.
How do I write the rules down?
Write each rule in a simple, testable form. Use this structure:
| Field | Example |
|---|---|
| When | A caller asks for an appointment on the same day. |
| And | The caller is an existing patient with a chronic condition. |
| Then | Offer the first slot with their own doctor, or transfer to the nurse. |
| Unless | It is after 4 PM. Then transfer to the nurse line. |
| Owner | Front desk manager |
| Source | Interview with the front desk team, 3 March |
A rule in this form can go into the prompt, into the system logic or into the eval set. A rule in the form “use good judgment” cannot.
Which rules should the AI follow, and which need a human?
Do not try to give all rules to the AI. Sort them into three groups.
Rules for the AI. Clear, frequent and low risk. Example: “Send the standard form for new patients.”
Rules for a human. Rare, high risk or based on judgment that you cannot write down. Example: “Decide if a caller’s symptoms are urgent.” The AI must recognize these cases and hand them to a person.
Rules to change. Some unwritten rules are bad. They exist because of an old system or a person who left. Use the project to remove them.
The second group is the most important. The AI must know when to stop and give the task to a human. Write the conditions for the handoff as clearly as the other rules.
How do I test the rules?
Put each rule into the eval set. For each rule, add at least three real or realistic examples:
- One normal case where the rule applies.
- One case where the “unless” applies.
- One case that looks similar, but where the rule does not apply.
Run the eval set before the launch and after each change. When a new hidden rule appears in production, add it to the document and to the eval set in the same week.
What does a launch plan look like?
- Map the process. Write the current process, with the help of the team.
- Find the hidden rules. Use at least three of the five methods above.
- Sort the rules. AI, human or change.
- Write the handoff conditions. When does the AI give the task to a human?
- Build the eval set from the rules and from real cases.
- Run in shadow mode for one to two weeks. Compare AI and human results.
- Fix the differences. Add rules, change the prompt or add handoffs.
- Launch to a small group. One team, one shift or one type of request.
- Review each week. Collect new exceptions. Add them to the document and the eval set.
- Expand. Add more users when the error rate is stable.
What if the process is too complex to write down?
Some processes are very complex. Each case needs judgment. Writing all rules is not realistic.
In this case, change the role of the AI. Do not let it make the decision. Let it prepare the decision for a human:
- Collect the information that the human needs.
- Draft a recommendation with the reasons.
- Show the similar past cases.
- Let the human approve, change or reject.
This design is slower than full automation. But it is much safer, and it still saves time. It also collects data. Over time, the human decisions show which rules you can give to the AI later.
Who must own the written process?
The written process needs an owner, like the AI system. Without an owner, the document gets old. New exceptions appear in the work, but nobody adds them to the document. After six months, the AI follows a process that no longer exists.
The owner must be a person in the department that does the work, not in IT. This person:
- Approves each new rule before it goes into the prompt or the system.
- Reviews the list of handoffs to humans each month. Many handoffs for the same reason show a missing rule.
- Removes rules that are no longer correct.
- Tells the technical owner when the process changes, for example a new product or a new law.
Put the process document and the eval set in the same place. When the owner changes a rule, the technical owner adds the test in the same week.
What warning signs show a hidden rule in production?
After the launch, watch for these signs. Each one can show a rule that you missed.
- The same correction many times. People change the AI output in the same way again and again.
- Complaints from one group. One customer type, one location or one product gets more errors than the others.
- Workarounds by users. People skip the AI for some cases and do them by hand.
- Escalations without a clear reason. Cases go to a manager, and the AI output looked correct.
Ask the team about each sign. Usually, the answer is a rule that “everybody knows.”
A hypothetical example
A property management company used AI to answer tenant repair requests. The written process was simple: classify the request, create a work order, send a confirmation.
In the first week, three problems occurred:
- Water leaks in some old buildings went to an outside contractor, not the internal team. This was a verbal rule from a manager who left.
- Requests from tenants in a legal dispute had to go to the legal team first. Nobody told the AI.
- Heating requests in winter were urgent, even when the tenant wrote “no rush.”
The company stopped the automatic work orders and ran the AI in shadow mode for two weeks. The team compared 400 AI classifications with the human decisions. They found eleven unwritten rules. Seven went into the prompt. Three became handoffs to a human. One was an old rule that the company removed. After the second launch, the error rate was low enough, and each new exception went into the eval set.
Next step
Before your next AI launch, spend half a day watching the people who do the work. Write down each decision. If your system is already live and failing in strange ways, run it in shadow mode against the human process. For the other common causes of failure, read why AI pilots never reach production and the pilot to production checklist. For help with the design of a human handoff, read the article on design bugs in human-in-the-loop systems.
