AI is a new computer
How I'm thinking about AI (2025) - Part 2
AI is a new kind of computer.
A traditional computer processes structured data with deterministic instructions.
AI processes unstructured data with natural-language nondeterministic instructions.
A more formal definition:
A traditional computer is a universal structured information processor that can simulate any effective procedure (algorithm) given sufficient resources and the proper program.
AI is a universal unstructured information processor that can simulate any intuitive procedure (reasoning) given sufficient resources and the proper context.
Like a traditional computer, AI will organize storage based on access speed, capacity, and cost.
Primitives of a traditional computer:
CPU registers/cache: Fastest access, smallest capacity
RAM: Fast access, moderate capacity
Disk storage: Slower access, large capacity
Network storage: Slowest access, potentially unlimited capacity
Primitives of an AI computer:
Transformer attention heads
Context window / KV cache
Retrieval data
Tool use data
CPU -> RAM -> Disk/DB -> Network
LLM -> Context -> Retrieval/DB -> Tool useTo build the great production AI systems of the future - these components must work well together. Memory hierarchy enables software that makes the best tradeoffs of speed, cost and reliability.
This stack is in its infancy today - there is a lot of work to do.
For example, today, retrieval is modeled more like a tool - whereas in the future it will sit directly next to the GPU in the same cluster (including speaking embeddings directly instead of natural language).
Today LLMs do retrieval and tool-use one time before the generation - and in the near future (already happening with fine-tunes of Deepseek R1) models will continuously use tools and retrieval.
Today context is manually managed by the developer - in the future a memory management system (an AI kernel running inside an AI operating system) will manage it and the developer will meta-program the memory management systems (AI compilers).
This all begs the question - if AI is a new kind of computer - then what kind of programs can we create and how do we program them? And how is that different from software as we know it today?
Your insights are always very interesting, and I look forward to more thoughts on how the idea of memory is changing, and the role of vector dbs (and other building blocks like embeddings).
I'm hearing more talk about defining ai-systems (agent-runtimes) as new types of computers.
https://www.wwt.com/blog/llms-as-intent-interpreters-in-agentic-systems
Maybe your next post could expand into MCP as next gen tool-use but also "networking" or communications; or, mem0 and other memory systems that are smarter about what and when to remember.