Why Automating Snowflake is a Huge Deal for Agile Data Warehousing

Sometimes a major technological advance is accelerated by another complementary innovation. It seems data automation and a Cloud-native database that separates storage from compute go together just like peas and carrots. Two huge steps forward in data management and democratization have just become one giant leap, and here’s why…

Ten years ago, could you imagine a giant 3D dragon jumping out of your bedroom wall? The rate of technological advance means you can buy a 3D projector the size of a handbag for $500 these days, whereas a few years ago this technology was only accessible in a cinema. First, you had big standard definition home projectors that made lots of noise and got very hot; then smaller, quieter HD; then 3D and now Ultra HD and so on. The gradual nature of this feels natural as everyone adjusts to the latest model, then suddenly you have a dragon leaping out of your wall and that feels normal.

Sometimes technology lurches forward in a giant step, skipping those granular stages. This shift precedes the understanding and acceptance of it, requiring people to retrospectively figure out how they can use its power. Breakthroughs like Big Data, which exploded with Hadoop’s ability to process huge streams of data in parallel, are solutions looking for problems: many people want Big Data strategies and invest in it without knowing exactly what data they want and why they really need it.

Scratching a 30-Year Itch

Snowflake is different, as it’s an answer to a problem IT has suffered for too long. In the ‘90s data warehouses and the resultant business insight were supposed to be the holy grail that would transform how we do business. However, the logistics of building and managing a data warehouse that worked were more complex than we had first hoped. So, the power of data resided with a few specialists, its extraction in any meaningful form was slow, and this created political struggles within organizations that disabled rather than enabled the majority.

In the beginning, on-premises data warehouses were hugely expensive and inhibited agility. Building a data warehouse took years and cost millions, meaning only the richest companies could afford one. We had to estimate how much storage and compute power we needed three years in advance. Buy too little and we ran out of space and lost the ability to do our job, buy too much and we wasted huge amounts of money on unused memory.

The Cloud changed this by allowing us to only pay for what we need. Now with increased security and trust, we are shifting more critical workloads on to the Cloud, but we can still often spend more money than is needed. While storage is cheap, compute power is expensive and often VMs sit unused, racking up huge bills, money that can now be spent on growing the business elsewhere. People often talk of the Cloud being expandable to grow with a company based on their needs, but its ability to contract so we only spend what we need to is equally, if not more, important.

The Snowflake Effect

Snowflake has separated storage and compute, and the bar has been raised again. As Snowflake’s VP of Product and Partner Marketing, Jon Bock, explains on this podcast, if we just want to drive to the shops and back, we don’t need to buy a Ferrari, but if we want to go for a long drive on the open road, it would be nice to rent one cheap. This means whole teams only have to pay for the specific amount of computing power each individual needs at the time they need it, while also allowing all team members to access and collaborate on a huge pool of shared storage data.

Snowflake is a Cloud native database. Whether you’re a Cloud native company or not, Snowflake enables you to take advantage of this latest technology and run like a digital native, only paying for the storage and compute you need at any time. When we heard about what they were doing, we knew it complemented our offering perfectly. WhereScape automation makes the design, development, deployment, and operation of data warehouses and other structures quicker and cheaper, so teams can deliver projects in hours and days, not months and years. Snowflake allows IT teams to spin up new data environments in the Cloud in seconds.

WhereScape® automation for Snowflake represents one of those shifts that disrupts the whole geology of IT, a seismic shift that answers the new holy grail of business requirements – doing more with less and delivering faster. It also makes the availability of cheap, efficient data warehouses possible to companies of all sizes, and takes them out of the hands of experts to democratize data in a way we haven’t yet experienced. It’s time to make dragons leap out of your wall.

This blog is based on an Inside Analysis podcast in which Snowflake VP of Product and Partner Marketing Jon Bock and WhereScape CEO Mark Budzinski discuss the potential of data warehouse automation for the Cloud. Click here to listen to the podcast.