Where does storage go from here?

by Kiran Bhageshpur – June 23, 2015

Trends are hard to spot. Sometimes it helps to step back and take a look from a distance at where we’ve been to get a better view to what is coming.

When I do that with storage, I see that the dominant long-term storage trend has traditionally been speed. Since the earliest days, the industry has focused on making sure storage was not the bottleneck in computing. Take RAID for example. RAID came along in the late 1980s when users needed higher performance than current technology could provide. RAID 0 delivered a higher performance than normal single drives could supply. Other “exotic” techniques, such as short-stroking drives so as to have more spindles, were developed for the same reasons.

Then the Web came along and a second trend emerged: Capacity. For certain applications, speed was no longer the driving force. At that point, we saw a bifurcation within the industry. The performance camp embraced Serial Attached SCSI (SAS) drives while the capacity camp gravitated towards SATA. Fiber channel is really just SCSI at a distance, so I’ll include them in the SAS crowd. Besides FC drives are “officially” dead, right?

Today flash has arrived, and SSDs are shipping in large quantities. Now what? 

With all of that as a backdrop, here are my five predictions for what comes next …

Prediction #1:  SCSI is Dead

My, that’s bold!  Let me start with something a bit less controversial:  SAS (Serial Attached SCSI) drives are dead. Why?  The better question is why is anyone still shipping SAS drives?  SATA kills SAS when it comes to price per GB. SSD’s kill SAS hard drives on price per IOPS. Additionally, flash-based storage systems are already looking past SAS and towards NVMe.

So why would anyone buy a SAS drive today?  In fact, my guess is if you ask any major drive vendor they’ll tell you privately that SAS hard drive sales are down and SSD sales are up.

The real answer to ‘why buy SAS’ is likely inertia. All those legacy systems with 15K and 10K rpm SAS drives need to be refreshed, and it takes a long time for an older technology to be flushed from the market. I know for a fact that there are still functional VAX VMS systems out there along with a robust secondary market for ‘scavenged parts.’

As the unit of storage changes in form and function (PCIe Flash, SSD’s, SMR, HAMR drives), SCSI and by extension SAS / SATA as a mechanism for aggregating ‘drives’ is showing its age. Note the attempts by Seagate to get away from the ancient SCSI command sets with their attempts to introduce Kinetic DrivesHGST and Toshiba are not far behind.

Prediction #2: The Storage Tiering Problem Finally Gets Fixed

We have known for years that tiering data made sense. Put the data that users required fast access to on a ‘performance tier,’ and the rest on a ‘capacity tier.’  Easy to say, but how do you implement that?  File movers (Acopia, Rainfinity, etc.) have never achieved broad adoption. Why?  They were too intrusive:  Customers had to significantly change their workflows to reap the benefits these products offered.

Their fundamental flaw, in my opinion, was trying to mask the tiering. They constructed highly complex solutions to automatically tier data based on factors such as how often data is accessed, how important it is, and so on.

By contrast, organizations with hyper-scale storage don’t mask tiering. Instead, they make it very easy (think self-service) and fast for users and applications to move data back and forth between tiers. It is just a matter of time before these solutions trickle down to enterprises.

Prediction #3: Home Directories Will Migrate to the Cloud (Private or public)

The local home directory has always been sacrosanct. In a technology world increasingly hard to recognize, there was always a simple, familiar, reassuring little oasis of local storage that users could call home.

The problem is users themselves stopping being local. Now they want access to their data wherever they are: On the road, at home, a client’s office – wherever. Local home directories may be familiar and reassuring, but they are a pain-in-the-ass for a mobile worker.

So, the initial pain of moving user data to the cloud and getting used to that model is well worth it when you weigh the benefits of simple, consistent access to information regardless of location.

One need only look at the success and proliferation of solutions like Box and Dropbox to see the writing on the wall.

All but the most conservative will move user’s home directories to the cloud, and soon. Are you listening, NetApp?  The one area safe from this ‘move to the cloud’ will be engineering home directories. Even Google & Facebook, I am told, use NTAP filers for developer home directories.

Prediction #4: Infrastructure Failure Goes from Being Feared to Being Embraced

For years, we’ve treated disk failures as a disaster scenario. Yes, we back-up, but human error during back-up or problems restoring from tape are all too common, and we live in fear of losing our most precious computing commodity: Our information.

This model of praying to the disk gods and treating the rare failure as a disaster is not sustainable for much longer. Why?  Because as the size of our storage continues to grow geometrically, along with the number of drives we use, failures will become more and more common. In hyper-scale installations, a drive fails within their massive battery of disks every 60 seconds. Far from being a disaster, a drive failure here is like a small itch… something unremarkable that causes no real harm.

How does one take the disaster out of drive failures?  By building storage systems that expect failure and are built to withstand it without human intervention.  That’s where we are headed – to a world where failure is an option – and, in fact, expected.

Prediction #5: The Rise of the “DevOps” 

I remember a customer from a prior life that refers to his team as ‘storage monkeys.’  It seemed pretty derogatory, but also apt. They did little more than apply patches, replace failed drives and sit on hold for long periods of time waiting for vendor support.

That model won’t work in the brave new world I have outlined in this post. The model going forward is of a much more muscular, proactive infrastructure team. Think Dev Ops or Site Reliability Engineers (SREs). Instead of waiting for disaster to strike and then cleaning it up, these teams will work on designing infrastructure that is more reliable and even self-healing.

Team members will need to be more than proactive in designing architecture and working with API’s to create data pipelines that offer more value to the business. They will have to bring a skill set that is part infrastructure management, part coder, part architect as the business continues to expand with new application paradigms.

It is a very different model, but one in keeping with the move towards data center agility. Here is an excellent deck from Chris Dagdigian, from the BioTeam in Boston, which dives into greater detail on this exact point. Take a look at slides 29-44 in particular. Chris extols the value of scripting and getting on top of orchestration and points out that you’ll need more than a Cisco certification in the future.

So, that’s my take on where we are going. As always, I am interested in your view. Am I off base?  Where do you see storage going in the next few years?