How do always-on AI agents work?

How do always-on AI agents work? An event wakes the agent. The agent loads its state, does the work, saves its progress and goes back to sleep. Three parts do this. Triggers start the agent. An agent server keeps the reasoning loop, the task queue and the memory. A cloud computer gives the agent a browser, a shell and files. Lee Robinson of the SpaceX AI team explained this design in a 45-minute lecture for Stanford CS146S. He is an engineer on Grok Bot. The timestamps come from notes by Peter Piekarczyk. I add my opinion after each point.

How do always-on AI agents work? The short answer

The lecture has one central thesis. AI changes from a tool that you prompt again and again into a persistent digital employee. This employee remembers, plans, delegates, does the work and tells you about the result. The notes put it in one sentence:

I agree with this. I wrote the same thing in October, after OpenAI shipped Dots: the model is swappable and the harness is the moat. The lecture gives the engineering details for that opinion.

What changes when an agent is always on?

The first point is at 0:51. Agents change from reactive to proactive. A usual agent stops when you close your laptop. An always-on agent continues to work without you. It wakes up from emails, Slack messages, schedules or other events. It remembers its previous tasks. It tells you only when something important occurs.

Robinson calls this a change in your job. You stop prompting an assistant. You start to manage a digital colleague. I said the same thing in my Dots post. An agent is a process that runs in the background. For two years, we put that process into a chat window. If you know the agent loop, this is the same loop. The difference is that nobody watches it turn.

Does an always-on agent run all the time?

No. The point at 10:11 is that “always on” does not mean “always running.” The agent sleeps for most of the time, and this saves compute. A trigger wakes it. It restores its previous state, does the work, saves its progress and goes back to sleep.

This cycle makes persistent agents economical at scale. The agent uses compute only when there is work to do.

What are the three parts of an always-on agent?

At 13:15 Robinson shows the architecture. It has three components. The diagram below is my version of his conceptual drawing.

The important design decision is to keep the reasoning loop on the server. The loop is separate from the cloud computer. Thus you can restart or replace the computer, and the agent keeps its task state.

In my post on agent sandboxes and microVMs, I said that the isolation is real, but the VM is not the harness. This architecture agrees. The VM is a tool that the agent uses. The agent does not live in it.

What happens when the agent crashes in the middle of a task?

At 16:17 the lecture says that persistent workflows are more important than simple task queues. An agent can crash halfway through a complex task. Then it must resume from its last checkpoint. It must not do all the work again.

Robinson’s team uses Temporal for workflow persistence and recovery. This is necessary for tasks that continue for days or weeks. A plain queue only knows that a job started and that a job finished. It does not know which steps are done. Use the figure above to see the difference. Set the crash to “on” and compare checkpoints with plain queue.

This is the boring software engineering that I mentioned in the Dots post. Queues, retries, permissions and tests are the work that we did five years ago. LLM output alone is not reliable enough for an agent system. The workflow engine gives the reliability.

Why use one orchestrator and many subagents?

At 23:30 Robinson describes the agent as an organization. One model does not do all the work. A main agent coordinates the work. Specialized subagents do the individual tasks. Each subagent has a separate context window. Only the relevant results go back to the main agent.

The main agent is a manager. It is not an individual contributor that does every task. In my post on the blackboard pattern, several agents share state on one board and do not need to know about each other. Both designs keep each agent’s context small. The main agent sees only the results.

Where does agent memory live?

At 32:50 the lecture says that memory must live in files. Robinson describes four layers of memory:

These layers can be Markdown files. The agent searches them when it needs them. Thus the agent does not load all of the conversation history again and again.

How do you keep the context small and cheap?

At 27:26 Robinson says that context engineering is a large competitive advantage. More context is not always better. Effective agents do five things. They keep the system prompts and the tool definitions stable, which gives the most prompt cache hits. They load tool details dynamically. They put long outputs in files. They send context-heavy operations to subagents. They compress the conversation before the context becomes too large.

One optimization is very useful. Summarize the conversation while the prompt cache is still warm. Do this even when the user is away. This can decrease inference costs by a large amount.

The figure shows the tool-loading part. When the agent loads all tool schemas at the start, the context fills before the first instruction. When it loads tool details on demand, the context stays small. I wrote about the same problem in how to learn AI agents. The smallest tool set that does the job is the best tool set.

Which method should the agent try first?

At 21:32 the lecture gives an escalation order. The agent uses the cheapest reliable method first. It goes up one step only when the lower step does not work.

The principle is to get the most autonomy for the fewest tokens, the least compute and the least human attention. Sensitive actions still need safeguards and approval.

Where does safety go in the architecture?

At 26:30 the lecture says that safety must be a part of the architecture. Robinson lists four controls. The system reviews shell commands automatically. It protects against prompt injection. It runs the agent in an isolated environment. A human approves irreversible actions, for example sending a message.

Autonomy is useful only when you can trust the permissions and the actions of the agent. I agree, and I add one control. Put the checks in the request path and make them fast. In my guardrails post I put the classifiers in front of the model, at the gateway. An agent that runs while you sleep needs the same gate on each tool call. Also, when an agent repeats the same call, stop it with a loop guard. A tool call cycle is the most frequent way for an agent to turn a task into a runaway bill.

Will always-on agents need a user interface?

At 37:45 Robinson says that the best AI product can have almost no interface. Models become more capable, so complex interfaces become less necessary. He tells teams to build for the capability that models will have. Do not engineer too much around the limits of the current models.

He expects three changes in the next 6 to 12 months. Agents will operate for weeks or months. Long-term memory will become better. Conversational interfaces will become simpler. These are forecasts. They are not established results.

I made a similar forecast after Dots. I expect the always-on agent to become a new form of app that lives on your phone next to the other apps, within a few years.

What does this mean for an AI product?

The notes give three conclusions for product and business.

The third conclusion is the one that I tell CEOs. Do not ask AI to do isolated steps. Give it an outcome and a process to run in the background. Reconcile the invoices. Chase the follow-up. Keep the CRM correct. Nobody needs a new app for this work. They need a process that does it.

What is my take as a fractional AI CTO?

I work with teams as a fractional AI CTO. The lecture describes the system that I see teams try to build, often without a plan. Three parts of it decide whether a client project succeeds.

The first part is the step from prototype to production. In my post on when to hire a fractional AI CTO, the frequent sign is a product that works for a small group of users. It does not scale to the full user base, the data volume, the latency, the cost or the security that production needs. An always-on agent makes this step harder. The agent runs without a user, so nobody sees the failure at the time it occurs. Checkpoints, file memory and cost per task are the parts that make the failure visible.

The second part is the model. A CEO often asks me which model to choose. My answer is the same as in my Dots post. You can make almost any model compatible with the OpenAI API and change it in an afternoon. Your proprietary data, your process knowledge and your integrations are the difficult part. The lecture puts the same value on the infrastructure around the model.

The third part is security. An always-on agent acts while nobody watches. Before a team adds more autonomy, I inspect the pipeline against the frequent gaps for its use case. This is a first pass. It tells the team if it also needs a penetration test of the agent.

What would I build first?

The notes give five priorities for a team that builds an AI product. I agree with all five, in this order, and I add my reason for each.

So, how do always-on AI agents work?

How do always-on AI agents work? A trigger, such as an email, a message or a schedule, wakes the agent. The reasoning loop on the agent server loads the task state and the memory files. Then it does the work through subagents and a replaceable cloud computer. It tries the cheapest method first and asks a person only for irreversible actions. A workflow engine saves a checkpoint after each step, so a crash resumes where it stopped. Then the agent saves its progress and goes back to sleep until the next trigger. If you build one, start with the checkpoints and the memory files. If you want help with the design, talk to me.