Blog

Subscribe to Email Updates

Data Growth Is Unpredictable — but Here's How You Can Plan for It

by Catherine Chiang – January 30, 2018

According to Gartner, 70% of organizations list managing data growth as their top concern. With data growing exponentially while legacy storage systems struggle to handle petabyte-scale data, it’s no easy feat for enterprise IT to balance modern data management needs with budgets that aren’t growing as quickly as their data.

Even if you believe that your enterprise has a long-term plan for managing your data growth, changes in the way we generate and use data are already resulting in unpredictable and explosive data growth that’s harder and harder to manage.

Data Growth is Unpredictable

Moore’s Law, named after Intel cofounder Gordon Moore, predicted that chip performance could double about every two years. However, the industry is reaching the limits of Moore’s Law while 63% of enterprises report 20% or higher annual data growth rates—meaning that data is still growing, but storage density isn’t keeping pace.

In addition, enterprises need to not only take into account how their data is growing today, but changes in how data is generated and used. Advancements in machine learning and artificial intelligence are likely to improve the way businesses use data, but also create data storage and management problems for businesses that may not yet struggle with large amounts of unstructured data.

For example, unpredictable changes in how data is generated in the life sciences industry resulted in an explosion of genomic data. As technological developments made sequencing equipment more affordable and widespread, labs began generating genomic data faster than IT could add capacity.

The way we generate and use data is changing across all industries, making it imperative for enterprises to plan for the unpredictable when it comes to data growth.

Scale-Out, Not Up

So how do you plan for data growth that's both unpredictable and potentially explosive?

One way to future-proof your storage and data management systems against unpredictable data growth is to incorporate scale-out architecture into your infrastructure. Rather than the traditional way of scaling vertically (“scale-up”), which was to increase capacity by adding more devices to your existing system, scale-out, or scaling horizontally, increases capacity by connecting multiple nodes that work as a single system.

While the emphasis has been on scale-out primary systems, scale-out secondary storage systems also have many benefits over scale-up for large unstructured data.

With traditional scale-up storage, once the system reaches its limit, the only way to add capacity and performance is to add a new, separately managed system. This results in silos that add unnecessary complexity to your infrastructure.

Scaling out means that as storage capacity increases, so does performance. Since all the data can stay in one system as it scales, enterprises can eliminate their backup silos and streamline their secondary storage even as data grows.

If you’d like to learn more about how we’ve built Igneous Hybrid Storage Cloud to handle unstructured data at scale, check out our technical whitepaper, “Igneous Scale-Out Architecture.”

Read whitepaper

Related Content

Moving to Cloud? Here's How a Hybrid Approach Can Help

May 8, 2018

Today, the growth of unstructured data is scaling beyond what traditional legacy storage and backup softwares were designed for. While advances in cloud architecture from AWS, Microsoft Azure, and Google Cloud present solutions for managing enterprise data, many enterprises face obstacles to adopting cloud and stick with legacy storage and backup software as a result.

read more

Archive First, Backup Should Be Boring, and Other Insights: A Conversation with Life Sciences Technologist Chris Dwan

April 24, 2018

Chris Dwan is a leading consultant and technologist specializing in scientific computing and data architecture for the life sciences. Previously, he directed the research computing team at the Broad Institute, and was the first technologist at the New York Genome Center.

Chris joined us for a conversation on data management challenges and trends in the life sciences. Read on to discover Chris’ insights from over a decade of experience in life sciences IT!

 

read more

How Much is Tape Really Costing Your Business?

April 17, 2018

Tape remains a popular medium of data storage due to its advantages in cost and ease of use. Often as little as a penny per gig per month to cover just the hardware, tape seems to be a great, affordable option for backup. Unlike other backup solutions, scaling tape is simply a matter of adding physical inventory, and tape doesn’t require network bandwidth or expertise to move offsite, or power when storing long-term archives.

read more

Comments