As AI models get larger and more impressive, the systems that feed them data quietly determine whether they work reliably or fall apart under real-world load. Memory-safe, high-performance infrastructure is no longer optional; it has become a critical layer for anyone who wants AI that truly scales. As a Rust architect and AI researcher working on high-speed graph databases, I use this column to share my thoughts on the role of data infrastructure in the future of AI. Why does so much depend on the storage layer as much as on the models themselves? And, most importantly, what do teams need to understand if they want to be ready today?
Models vs. Reality: The Missing Conversation About AI Infrastructure
Many talk about AI models; almost no one talks about infrastructure. From the perspective of a Rust architect working in AI research, this is what's missing from the usual narrative about "AI progress." There are three important directions of AI progress that should get more attention: building better harnesses, Claude Code & Cowork being the best examples of a harness done right; real-time communication, such as voice modes; and, of course, scaling.
At the same time, while developing newer, better, stronger models is important, end users want a reliable solution that doesn't stutter. In that sense, Rust shines when it comes to real-time communication and predictable scaling.
When Graphs and Genomics Break "Normal" Databases
AI research workloads are not like typical web apps. In fact, workloads such as graph-based reasoning, symbolic computation, or genomics place very different demands on data infrastructure compared to traditional applications. Ultimately, the main demand is the scale and complexity of queries. Most of the time, when people work with large datasets, they use database queries (such as SQL) or some form of Map/Reduce. By contrast, we're doing graph transformations on terabyte-sized datasets.
When Memory Bugs Turn into Wrong Answers
Memory safety sounds abstract. Until something breaks. In simple terms, this is why memory safety and low-level performance matter so much when you design the storage layer for AI systems. Memory correctness matters because, without considering it, you get unreliable responses, database crashes, and hard-to-debug issues. Building a reliable database without considering memory safety is impossible. Decades of human effort went into making databases work concurrently, and people who make databases are very aware of shared mutability rules, under different names.
So, the reason memory safety specifically is important is that we're building something that works really close to the hardware, and we care about performance. Being this close to hardware requires additional scrutiny to make things right.
From Toy Experiments to Terabyte-Scale Reality
Scaling from "toy" datasets to real research scale requires a different set of architectural decisions than small experiments. In my recent work with high-performance graph databases, these are the decisions that allowed us to go from small experiments to handling truly large datasets. The main one was paying attention to memory usage at every step. The second most important decision was offloading data to the hard drive. It turns out that it's easy to forget about memory usage when you're working on typical problems, and these experiments require a lot of extra effort.
Letting Researchers Move Fast without Breaking the Foundation
Balancing flexibility for researchers with reliability in production is often framed as a trade-off between freedom to experiment and guarantees of stability, reproducibility, and performance. However, I think the trade-off is never between freedom and stability. You get to choose parts of the system that are reliable, and parts of the system that you can change (and subsequently break). You want the entire foundation under you to be stable, with as much flexibility as possible in what you do. Rust is an excellent language for the purpose of making a foundation. On top of that, you get something that is as flexible as you can make it.
Rust's Superpower and the Price You Pay for It
No language excels at everything, and Rust is no exception. But when it comes to AI and data infrastructure, its strengths matter most. On the positive side, the main advantage of Rust that we've observed is fearless concurrency. You can always know that your multithreaded code will work correctly. We've implemented a system of permissions for shared database modification, similar to the borrow checker, allowing as much parallelism as possible. As for the drawbacks, the main one is iteration speed. If you make a change in a large Rust codebase, it can take up to a minute to see the change take effect. For experimentation and testing, this is a drawback.
Native Code, Edge Devices, and the Next Wave of AI
Here are a few predictions about how the data infrastructure behind AI labs and companies might change over the next 3–5 years, and what will separate teams that are ready for the next wave of AI from those that are not. I don't expect the infrastructure of AI labs to change much, because people seem to have settled on a solution (Python / Torch), and it seems to work. At the same time, I expect AI inference providers to move towards native code for the sake of consistent timing and throughput reliability. In particular, I anticipate end-user devices having excellent real-time ML inference, enabled by native code such as Rust, Swift, or C++.
It might be a gradual shift or a sudden one; we'll see. But I am sure that teams who start treating infrastructure as a core part of their AI strategy now will be in a significantly better position when it happens.
