Agentic Software vs Software That Uses AI

The Line Nobody Draws Clearly

Most software described as “AI-powered” is not agentic. A spell-checker that calls an LLM, a search bar that reranks results using embeddings, a form that auto-fills a draft — these are software that uses AI. The AI is a subroutine. The developer wrote the loop; the model executes one step inside it and returns. The programmer knows what happens next.

Agentic software inverts that. The model owns the loop. You give it a goal and a set of tools — read a file, run a query, call an API, write output — and it decides which tools to call, in what order, based on what it sees. The programmer defines the action space and the success condition. The path through them is the model’s business.

The distinction is who controls the flow.

AI as Component vs AI as Orchestrator

In “software that uses AI”, the model is a hammer. Accurate description: a function call with a smarter return value. GPT-4 summarizes the text, nomic-embed-text turns it into a vector, Whisper transcribes the audio. These are useful and important — but architecturally, they’re no different from any other library call. The developer remains in charge of sequencing.

In agentic software, the model is more like a contractor. You describe what needs to get done. It figures out the steps — searches for context, reads relevant files, drafts output, checks its own work, revises. The ReAct loop (Reason → Act → Observe → repeat) is the pattern underneath most agent frameworks. The model reasons about what to do, does it, observes the result, reasons again.

That shift changes everything downstream: failure modes, observability, trust, testing.

What Changes When the Model Owns the Loop

Failure modes — AI-as-component fails predictably. Bad output, wrong classification, API timeout. You can write tests. Agents fail in stranger ways: reasoning errors that compound over five tool calls, infinite loops because the model misread an observation, hallucinated tool arguments that corrupt state. The bug is often not in any single step but in the sequence the model chose.

Observability — you can trace a call stack. You can’t easily trace why an agent took a particular path through a problem. You can log each tool call, but the reasoning between calls lives inside the model’s context window — not in your system. Frameworks like LangSmith and Langfuse exist specifically because standard observability tooling doesn’t map onto this.

Non-determinism at the path level — individual LLM calls are already probabilistic. Agents amplify this: the path through the tool graph is non-deterministic, and different paths produce genuinely different outcomes, not just different phrasings of the same outcome. Two runs of the same agent on the same goal can write different files, query different APIs, and produce structurally different results.

Trust and authorization — when the AI is a component, you authorize what the program can do and the model inherits those permissions implicitly. When the AI is the orchestrator, you have to think about what it’s allowed to decide independently versus what requires a human in the loop. An agent that can write files, send emails, and modify database records is a different trust surface than a model that generates text you review before acting on.

Where the Line Gets Blurry

A pipeline can be mostly deterministic but call an AI at one decision node — that’s not agentic. A single LLM call that returns JSON you parse and route on — also not agentic. The loop has to be model-driven: the model reads an observation, decides what to do next, takes an action, and that action’s result feeds back into the model’s reasoning.

By that definition, most RAG systems are not agentic. You retrieve, you inject into context, you generate — the retrieval strategy is predetermined. An agent that decides whether to search, what to search for, and whether the results are good enough to answer the question — that’s agentic. The boundary is whether the model is making architectural decisions at runtime or just performing a predetermined step.

What This Means in Practice

The frameworks (Strands, LangGraph, CrewAI, AutoGen) are all essentially about giving structure to the model-driven loop — tool registration, history management, parallel execution, handoffs between specialized agents. The engineering problem is no longer “how do I call the model” but “how do I constrain an autonomous decision-maker to behave reliably inside my system.”

That’s a meaningful shift in what software development looks like. You’re not specifying steps anymore — you’re specifying constraints, goals, and trust boundaries. Closer to managing a process than writing a function.