It's Tokens All the Way Down: How Recursive Language Models Differ from Coding Agents
Disclaimer: This blog written by AI 🤖
Recursive Language Models represent an intuitive but distinctively important approach to how LLMs handle context—and the practical implications are bigger than they first appear. In an AI Engineer talk, Kevin Madura of AlixPartners argues that tasks which traditionally require careful prompt engineering, custom agent scaffolding, or multi-step orchestration can collapse into surprisingly simple, composable programs when you treat the model as a recursive component inside a real programming environment rather than as the center of an agent loop.
Madura’s distinction between an RLM and a coding agent is architectural, not marketing. A coding agent wraps the model in external harnesses—tool routers, memory systems, retry policies—to manage long-horizon work. An RLM embeds the model inside a program that can inspect dataframes, navigate long documents, delegate nested model calls, and audit its own source code within a loop the developer controls directly. He builds these systems with typed, testable Python using DSPy signatures and Pydantic schemas, treating AI applications as modular software with explicit evaluation metrics instead of prompt soup. Side-by-side comparisons in the talk focus on trade-offs in complexity, wall-clock performance, and token usage—what Madura calls token capital efficiency: business value generated per dollar spent on model tokens.
The abstraction shines when context can be handled recursively inside the program rather than flattened into a single giant prompt. It breaks down when the environment is too messy to encode cleanly or when production requires durable orchestration across many independent actors. For harness engineers, the talk is a useful counterweight to default agent-first design: sometimes the right move is not another planner-evaluator loop but a smaller recursive program that keeps the model where it belongs—as a callable function inside code you can test, version, and benchmark.