GitLab brings Storage In-House

by Steve Pao – November 15, 2016

GitLab's decision to move from the public cloud was covered well in their blog post "How We Knew It Was Time to Leave the Cloud."  We understand, and much of their actual experience was consistent with what we predicted when we started our company.


GitLab has built of wealth of experience for running their applications on the public cloud, and they shared it on their blog this past week.  The experiences they documented were consistent what we expected for customers with data-centric computing applications.

Here are some key takeaways that GitLab observed that we expected and that motivated us to create Igneous Data Service:

  1. Impact of shared tenant architecture on storage performance.

    One of my favorite quotes from GitLab's blog was:

    "With this performance capacity, we became the 'noisy neighbors' on the shared machines, using all of the resources. We became the neighbor who plays their music loud and really late."

    While public cloud has been proven sufficient for compute, storage performance is a different matter.  We've designed our service for those that want to operate a dedicated performant data plane, particularly for the new class of data-centric applications operating on large, unstructured data.

  2. Management costs of building in-house storage.

    We appreciated that GitLab knew what they were getting into:

    "Of course hardware comes with it's upfront costs: components will fail and need to be replaced. This requires services and support that we currently don't have today. You have to know the hardware you are getting into and put a lot more effort into keeping it alive."

    While some organizations like GitLab are prepared to take on the challenges of building and maintaining their own hardware at scale, we developed the Igneous Data Service from the ground up for those that didn't have either the expertise or the appetite for these tasks.

  3. The need for observability and to metric everything.

    As we built the Igneous Cloud management platform, we understood the importance of being able to monitor our fleet and tune our service based on the workloads across our fleet of equipment.

    "The bottom line is that once you have moved beyond a handful of systems it is no longer feasible to run one-off commands to try and understand what is happening within your infrastructure. True insight can only be gained by having enough data to make informed decisions with."

    Like GitLab, we've designed our offering with software observability at both a level of granularity and a breadth beyond what humans can generally observe.  We've designed our service for those who also don't have the expertise or appetite to build and maintain sophisticated fleet management platforms.

While we embrace the agility of the public cloud, we also understand that it's not right for all data-centric workloads.  That said, we also understand what's involved in managing a dedicated data plane, on your network and behind your firewall.  We invite a discussion with you!

Read more here on the GitLab blog.