Storage Is Cool

by Kiran Bhageshpur – May 29, 2015


When you think of ‘cool’ tech market segments, which ones come to mind? Cloud, of course. Big data. The Internet of Things. But storage? For decades, storage has been a necessary, massive, but boring market.

But that has been changing lately. Everywhere you look, there are exciting new storage start-ups: Nimble, Pure, SolidFire, Tegile, Qumulo, Data Gravity,  Primary Data and many more. According to a recent article in Fortune, cumulative investment in storage start-ups has been running in excess of a billion dollars a year for the last few years. That’s pretty cool.

The question is, why all the sudden interest in storage? What’s changed? A couple of things, in my opinion. You might think the massive growth in the amount of information being stored is key, but I don’t think the growth in information by itself is the reason storage is suddenly cool.

Growth Alone Doesn’t Create New Markets

We all know aggregate storage is growing. IDC, in its recent Digital Universe report, forecasts a growth in aggregate worldwide storage from 4.4 zettabytes to 44 zettabytes by 2020. That’s an annual growth rate of 41 percent. What is driving this growth are new uses cases that few even thought about a few years ago. Machine-generated log file, digital medical imaging files, high-resolution imaging of our planets and so on.

Now, there is no question that robust data growth is a fantastic thing for storage vendors, but why wouldn’t EMC, IBM, HP and the other established leaders be able to service this growth? Why the need for all these new start-ups?

New Use Cases Create New Segments

What does create new markets are new use cases. In The  Innovators Dilemma, Clayton Christensen writes about portable computing upending the storage market by creating a market for smaller drives. We saw that a decade or so ago when the need for low-power, tiny storage for consumer devices (e.g., iPods) created the Flash market.

Today, new use cases are once again driving new storage architectures. It started with Amazon’s S3, an online file storage web. Amazon made a bet that businesses would benefit from a simple to use storage solution as they scaled their business. That bet paid off, as Amazon S3 now stores more than two trillion storage objects. A more recent example is SolidFire, which sees a world of multi-tenant, cloud-based infrastructure and rethought storage from the ground up. They are betting scale-out storage with predicable performance and fully automated management is the best choice for that use case.

You can see the pattern … spot a new use case, rethink how storage needs to behave, build a new segment.

Among those dozens of recently funded storage companies, there are some amazingly cool ideas. And that’s what’s shaking-up the storage  market.

A Really Cool Use Case

I have been thinking about these new uses cases a lot lately.  The one that has really caught my attention is driven by the simple fact that the traffic dedicated to sharing data is growing substantially faster than bandwidth. 

Here is a simple example.  Several times in the past few years, Dropbox has had hiccups that cause every user to have to re-download all of their shared files from the cloud – simultaneously.  The effect of such a hiccup wreaks havoc in an Enterprise.  Imagine the traffic jam when thousands of users all engage in a sustained, multi-day, massive download from the cloud.

The simple fact is that enterprise bandwidth isn’t keeping up with the growth of shared data traffic.  Let’s start with the growth rate for bandwidth.  Nielsen’s Law which predicts that user bandwidth will grow by 50 percent per year.  That’s how we got from frame-relay in 1983 to 10GE today.

At first blush, this seems like more than enough to keep up with the 41 percent growth rate in storage.  And it would be, except for one thing: the number of people sharing data is also growing (by 32 percent annually, according to industry figures). 

When you compound these growth rates, you get 86 percent more data-sharing traffic each year.  Okay, it’s probably not as simple as that … not all new users share all new data with all the other new users, but it is clear the data-sharing traffic growth rate will outstrip the bandwidth available for that purpose. 

Where do you go with that?  More about what that means in a future post, but that is a perfect example of the kind of new storage use case I am speaking about.

So, is storage ‘cool’?  Investors seem to think so.  At Igneous we think so, but we’re storage geeks at heart. 

What really matters is what you think?  Let me know, is storage cool? Are new use cases straining your existing infrastructure?  Do you feel the pressure to go to the cloud…