7 Reasons the AI Scaffolding Layer Is Collapsing – And What Comes Next

For years, developers building applications on large language models (LLMs) have relied on a thick scaffolding layer: indexing pipelines, retrieval engines, query planners, and carefully orchestrated agent loops. But according to Jerry Liu, co-founder and CEO of LlamaIndex, that layer is rapidly becoming obsolete. In a recent interview on VentureBeat's Beyond the Pilot podcast, Liu explains that this collapse isn’t a bug—it’s a feature. The future of AI development hinges on simpler primitives, massive context, and a new kind of competitive advantage. Here are seven things you need to know about the shifting landscape.

1. Frameworks Are Fading as Models Get Smarter

Liu, whose company built one of the most popular retrieval-augmented generation (RAG) frameworks, openly admits that even LlamaIndex is becoming less essential. Each new generation of LLMs improves its ability to reason over huge, unstructured datasets—often outperforming humans. Models now self-correct, plan multi-step tasks, and discover tools without explicit integration. As a result, the need for deterministic, framework-guided workflows shrinks. Developers can skip heavy orchestration layers and let the model figure out the path.

2. Context Is the New Moat

When the scaffolding collapses, what remains? For Liu, it’s context. Whether you use OpenAI Codex, Claude Code, or another model, the primary requirement is the same: high-quality, well-parsed data. LlamaIndex is betting big on agentic document processing—using optical character recognition (OCR) to extract information from locked-in file formats like PDFs, images, and legacy documents. “The thing that they all need is context,” Liu says. Companies that provide cheaper, more accurate parsing will own the next competitive edge.

3. Agent Patterns Converge on a “Managed Agent Diagram”

Liu describes a new, consolidated architecture for AI agents: a managed agent diagram. This consists of a lightweight harness layer combined with tools, Model Context Protocol (MCP) connectors, and skill plug-ins—rather than custom-built orchestration for each workflow. Standardized interfaces (like MCP) allow agents to discover and use tools on the fly, dramatically reducing integration effort. The era of wiring every tool manually is ending.

4. Code Agents Write Most of the Code – Even for LlamaIndex

One startling data point: about 95% of LlamaIndex’s own code is now generated by AI. Liu notes that engineers aren’t writing code in the traditional sense—they’re typing natural language prompts. “The new programming language is essentially English,” he says. This shift blurs the line between programmers and non-programmers. Instead of debugging API integrations or reading dense documentation, developers can point a coding agent like Claude Code at a problem and let it write the solution.

5. The Gap Between Devs and Non-Devs Is Collapsing

Because natural language is the new programming interface, the barrier to building AI applications is falling. Tasks that were “extremely inefficient” or would break an agent three years ago are now straightforward. Liu explains that pointing a tool at a complex file format and extracting the right information used to require deep technical knowledge. Now, a few sentences in plain English suffice. This democratization means more people can build—and the scaffolding layers that previously separated experts from novices are melting away.

6. OCR and Document Parsing Become Critical Infrastructure

With agents handling reasoning and task planning, the bottleneck shifts to data extraction. Many valuable datasets are locked in bespoke file formats—scanned PDFs, legacy documents, proprietary images. LlamaIndex’s approach uses intelligent OCR and agentic parsing to convert those formats into structured context that any model can consume. Liu sees this as a core differentiator: “There’s a core set of data that has been locked up in all these file format containers.” Solving that unlocks the next wave of AI capabilities.

7. Modular, Not Monolithic – The Future Stack

Despite the collapse of heavy scaffolding, Liu doesn’t advocate for a single monolithic platform. Instead, he envisions a modular stack where context providers, agent harnesses, and model interfaces remain loosely coupled. The concern that builders like Anthropic (maker of Claude) might “lock in” users is real, but Liu believes open connectors like MCP and skill plug-ins will preserve choice. The winning approach isn’t a Swiss‑army‑knife framework—it’s a small, flexible core that lets you swap out models, tools, and data sources as needed.

Conclusion: Embrace the Unbundling

The AI scaffolding layer is indeed collapsing, but that’s not a crisis—it’s an evolution. As models grow more capable, developers can shed bloated orchestration layers and focus on what truly matters: providing rich, accurate context in a modular, agent-friendly way. Companies like LlamaIndex are pivoting from being a framework to being a context infrastructure provider. For builders, the lesson is clear: invest in high-quality data plumbing, use natural language as your primary tool, and keep your stack light. The layers that survive won’t be the ones that control the workflow—they’ll be the ones that feed the agent.