“The long run is already right here,” science fiction author William Gibson as soon as stated. “It’s simply not evenly distributed but.” One one that’s trying to deliver information storage into the long run and make it extensively distributed is David Flynn, who’s the CEO and founding father of Hammerspace in addition to a BigDATAwire Particular person to Look ahead to 2025.
Even earlier than founding Hammerspace in 2018, Flynn had an eventful profession in IT, together with creating solid-state information storage platforms at Fusion-iO and dealing with Linux-based HPC programs. However now as Hammerspace positive aspects traction, Flynn is raring to construct the subsequent technology of distributed file programs and hopefully clear up among the hardest information issues on this planet.
Right here’s our current dialog with Flynn:
BigDATAwire: First, congratulations in your choice as a 2025 BigDATAwire Particular person to Watch! Earlier than Hammerspace, you had been the CEO and founding father of Fusion-io, which SanDisk purchased in 2014. Earlier than that, you had been chief architect at Linux Networx, the place you designed a number of of the world’s largest supercomputers. How did these experiences lead you to discovered Hammerspace in 2018?
David Flynn: It’s a extremely attention-grabbing trajectory, I believe, that led to the creation of Hammerspace. Early on in my profession, I used to be embedding alternate open-source software program like Linux into tiny programs like TV set-top containers, company sensible terminals and the like. After which I got here to design lots of the world’s largest supercomputers within the high-performance computing business that leveraged applied sciences like Linux clustering, InfiniBand, RDMA-based applied sciences.
These two extremes – small embedded programs versus large supercomputers – won’t appear to have a ton in frequent, however they share the necessity to extract absolutely the most efficiency from the {hardware}.
This led to the creation of Fusion-io, which pioneered using flash for enterprise software acceleration, which till that time was typically used for embedded programs in client electronics — for instance, the flash on gadgets like iPods and early cell telephones. I noticed a possibility to take that innovation from the patron electronics world and translate into the information middle, which created a shift away from mechanical exhausting drives in the direction of solid-state storage. The difficulty then turned that this transition in the direction of solid-state drives wanted extraordinarily quick efficiency; the information wanted to be bodily distributed throughout a set of servers or throughout third celebration storage programs.
(ALPAL-images/Shutterstock)
The introduction of ultra-high-performance flash was instrumental in addressing this problem of decentralized information, and abstracting information from the underlying infrastructure. Most information in enterprises at present is unstructured, and it’s exhausting for these organizations to seek out and extract the worth inside it.
This realization in the end led to the creation of Hammerspace, with the imaginative and prescient to make all enterprise information globally accessible, helpful, and indispensable, fully eliminating information entry delays for AI and high-performance computing.
BDW: We’re 20 years into the Large Knowledge increase now, nevertheless it feels as if we’re at an inflection level in relation to storage. What do you see as the primary drivers this time round, and the way is Hammerspace positioned to capitalize on them?
DF: To essentially thrive on this subsequent information cycle, we’ve received to repair the damaged relationship between the information and the information infrastructure the place it’s saved. Enterprises have to assume past storage and relatively how they will remodel information entry and administration in fashionable AI environments.
Distributors are all competing to supply the efficiency and scale that’s wanted to help AI workloads. Besides it’s not nearly accelerating information throughput to GPU servers – the core downside is that information pathways between exterior storage and GPU servers get bottlenecked by pointless and inefficient hops within the information path throughout the server node and on the community, whatever the exterior shared storage in use.
The answer right here, which is addressed by Hammerspace’s Tier 0, is using the native NVMe storage which is already included inside GPU servers to speed up AI workloads and enhance GPU utilization. By leveraging the prevailing infrastructure and built-in Linux capabilities, we’re eradicating that bottleneck with out including complexity.
We do that by leveraging the intelligence that’s constructed into the Linux kernel which permits our prospects to make the most of the prevailing storage infrastructure they’re already utilizing, with out proprietary shopper software program or different level options. That is along with offering international multi-protocol file/object entry, information orchestration, information safety, and information companies throughout a worldwide namespace.
BDW: You said on the HPC + AI on Wall Avenue 2023 occasion that we had been all duped by S3 and object storage to surrender the advantages of native entry inherent with NFS. Isn’t the combat in opposition to S3 and object storage destined to fail, or do you see a resurgence in NFS?
(whiteMocca/Shutterstock)
DF: Let’s be clear—its not about object or file, nor, S3 or NFS. Storage interfaces wanted to evolve to perform scale. S3 took place and have become the default for cloud-scale storage for a very good motive: older variations of NFS merely couldn’t scale or carry out on the ranges wanted for early HPC and AI workloads.
However that was then. Right now, NFSv4.2 with pNFS is a special animal—totally matured, built-in into the Linux kernel, and able to delivering large scale and native efficiency with out proprietary purchasers or advanced overhead. In reality, it’s turn out to be a typical for organizations that demand excessive efficiency and environment friendly entry throughout giant, distributed environments.
So this isn’t about selecting sides in a file vs. object debate. That framing is outdated. The actual breakthrough is enabling each file and object entry inside a single, standards-based information platform—the place information might be orchestrated, accessed natively, and served by way of whichever interface a given software or AI mannequin requires.
S3 isn’t going away—many apps are written for it. But it surely’s now not the one possibility for scalable information entry. With the rise of clever information orchestration, Tier 0 storage, and fashionable file protocols like pNFS, we are able to now ship efficiency and suppleness with out forcing a selection between paradigms.
The long run isn’t about preventing S3—it’s about transcending the boundaries of each file and object storage with a unified information layer that speaks each languages natively, and places the information the place it must be, when it must be there.
BDW: How do you see the AI revolution of the 2020s impacting the earlier decade’s large advance, which was separating compute and storage? Can we afford to deliver large GPU compute to the information, or are we destined to return to shifting information to compute?
DF: The separation of compute and storage made sense when bandwidth was low-cost, workloads had been batch-oriented, and efficiency wasn’t tied to GPU utilization. However within the AI period, the place idle GPUs imply wasted {dollars} and misplaced alternatives, that mannequin is beginning to crack.
The problem now isn’t nearly the place the compute or information lives—it’s about how briskly and intelligently you may bridge the 2. At Hammerspace, we imagine the reply is to not return to outdated habits, however to evolve past inflexible infrastructure with a worldwide, clever information layer.
We make all information seen and accessible in a worldwide file system—irrespective of the place it bodily resides. Whether or not your software speaks S3, SMB, or NFS (together with fashionable pNFS), the information seems native. And that’s the place the magic occurs: our metadata-driven orchestration engine can transfer information with excessive granularity—file by file—to the place the compute is, with out disrupting entry or requiring rewrites.
So the true reply isn’t selecting between shifting compute to information or vice versa. The actual reply is dynamic, policy-driven orchestration that locations information precisely the place it must be, simply in time, throughout any storage infrastructure, so AI and HPC workloads keep fed, quick, and environment friendly.
The AI revolution doesn’t undo the separation of compute and storage—it calls for we unify them with orchestration that’s smarter than both alone.
BDW: What are you able to inform us about your self exterior of the skilled sphere – distinctive hobbies, favourite locations, and so forth.? Is there something about you that your colleagues is perhaps shocked to be taught?
DF: Outdoors of labor, I spend as a lot time as I can with my children and household—often on skis or grime bikes. There’s nothing higher than getting out on a mountain or a path and simply having fun with the journey. It’s quick, technical, and a little bit chaotic—just about my preferrred weekend.
That stated, I’ve by no means actually separated work from play within the conventional sense. For me, writing software program and inventing new methods to unravel powerful issues is what I’ve at all times cherished to do. I’ve been constructing programs since I used to be a child, and that curiosity by no means actually went away. Even after I’m off the clock, I’m typically deep in code or sketching out the subsequent thought.
Individuals is perhaps shocked to be taught that I genuinely benefit from the artistic course of behind tech—whether or not that’s low-level system design or rethinking how infrastructure ought to work within the AI period. Some of us unwind with hobbies. I unwind by fixing exhausting issues.
You may learn the remainder of our conversations with BigDATAwire Individuals to Watch 2025 honorees right here.
