Bottleneck elimination. Seamless horizontal scalability. Cost predictability. Capacity without logical limits. Forklift upgrade avoidance. IT professionals offer different takes on the benefits of scale-out architecture.
At its core, scale-out architecture involves horizontal scaling – adding more nodes, such as by adding a new computer to a distributed software application. In contract, scale-up architecture involves vertical scaling – adding resources to a single node in a system.
What drives the need for new architectures? Microservices. Hybrid cloud. Internet of Things applications. Infrastructure growth. Machine learning. Unstructured data growth.
“Data explosion demands new architectures,” said Patrick Rogers, Cohesity Vice President Marketing & Product Management. “Six zettabytes today, growing to 90+ zettabytes in 2025.”
“Businesses are generating a ton of new data and are afraid to delete any of the old,” said Expedient Vice President of Product Strategy John White. “All are looking to a world where they might get answers from big data one day.”
Thanks to everyone who participated in our Aug. 8 CrowdChat conversation on scale-out architecture. Led by host John Furrier, participants discussed current scale-out technologies and predicted the future of scale-out architecture.
Before I share the insights on scale-out architecture from our Aug. 8 CrowdChat, let’s briefly explore the difference between scale-out architecture and something it’s sometimes confused with – clustering.
Scale-out architecture vs. clustering
A computer cluster involves connected computers that work together, typically through local area networks. A couple CrowdChat participants believe there’s a close link between clustering and scale-out architecture, with Cohesity’s Patrick suggesting that clustering is one way to achieve scale-out.
Here are three suggestions for distinguishing between scale-out architecture and clustering.
- Focus: “Clustering focus = keep my services online,” said Rubrik Technical Marketing Manager Andrew Miller. “Scale-out focus = grow forever.”
- Resources: While clusters can have passive resources, scale-out generally has all active resources, said P1 Technologies Chief Technology Officer Jeff DiNisco. Igneous Chief Technology Officer Jeff Hughes agrees that resources are a major differentiator. “Clustering usually means tightly coupled,” he said. “All the resources are pooled together. Scale-out many times just means dividing up the work.”
- Purpose: Historically, clustering increased resiliency and removed single points of failure, said Rubrik’s Andrew, while scale-out architecture focused on “very different things.” One of those things? Adding linear available resources, said Expedient’s John.
What's the litmus test for whether a product is scale-out?
Crowd-Chat participants offered a few scale-out test stations. Does the node fail but the service remains online? Can you power on a new node and it automatically joins a cluster? Does is have a consistent namespace across all nodes or components in the architecture?
“How they answer what the limit is,” added Igneous’ Jeff.
But at least one participant - Chris Dwan, technologist to the life sciences industry - questioned the question.
“I would rather deal with technical features than buzzword compliance,” he said.
And another participant - Rubrik’s Andrew - said that a non-scale-out architecture might make the most sense financially if your organization can predict your growth and environment.
“I’d argue that almost no companies can accurately predict that in a three-year window, though,” he said.
What isn't scale-out?
CrowdChat participants offered several examples of what isn’t scale-out architecture. Workloads that are very latency-sensitive “simply don’t need it,” said P1 Technologies’ Jeff.
“The cost of coordination between nodes isn’t worth it,” he said.
Rubrik Technical Marketing Engineer Kenneth Hui had a take on this question, too.
“It’s not scale-out if your workload has to be sharded and mapped to specific nodes so that each time a node fails or is added, you have to manually remap everything,” Kenneth said. “It’s not true scale-out if adding nodes does not provide linear or near-linear increase in performance.”
- “Anything where you say, ‘I loved my first; I hated my ninth,” such as with NetApp or Data Domain, said Igneous Systems Chief Marketing Officer Steve Pao.
- “Adding more disk shelves,” said Rubrik’s Andrew.
- “My network closet,” said life science technologist Chris.
Where's scale-out architecture going?
What’s the ultimate capability of scale-out architecture? Where does it all lead? And what should companies do to prepare? Just a few of the questions that CrowdChat participants explored while predicting the future of scale-out architecture.
Scale-out architecture is so important that Rubrik’s Andrew believes it’ll become table stakes.
“If a solution isn’t scale-out, it won’t make the short list,” he said. “Even if the buyers don’t know exactly what scale-out is, they’ll know it provides certain characteristics they care about.”
Within five years, scale-out architecture will be “a fundamental requirement” for secondary storage, and most enterprise data will be stored, protected and accessed via scale-out systems, predicts Cohesity’s Patrick.
Igneous Systems' Steve agreed on the importance of scale-out.
"At the edge, everything is converging as endpoints get more powerful. Look at the power of an iPhone," he said. "In the datacenter, everything has to scale out."
Expedient’s John believes scale-out will turn the traditional backup industry on its head, offering “an easy entry into the enterprise.” But CyberNorth Chief Technology Officer Chris Harney said not all organizations understand the role of scale-out in data storage.
“I think scale-out architecture has confused the industry,” he said. “If you can scale-out backup infinitely, is it backup or is it archive?”
Check out the entire CrowdChat conversation on scale-out architecture for more insights from industry experts. To lean more about Igneous and our modern approach to secondary storage, download the Igneous whitepaper “Secondary Storage for the Cloud Era” and visit our Services page.