Modernizing analytics with an open data stack

About:

The leading European online closed shopping community, faced challenges with their monolithic data warehouse. The increasing complexities of onboarding teams, rising costs with their cloud data warehouse, and a lack of flexibility prompted a rethink of their data strategy. This case study explores how the company partnered with Starburst to overcome these challenges and achieve cost-effective, open data architecture.

Challenge: 

In 2019, the online retailer embarked on a cloud data warehouse journey, but issues arose with rising Looker costs and the effort needed to onboard teams. The monolithic nature of the cloud data warehouse required extensive ETL efforts for every new dataset, leading to inefficiencies and increased workload for the central data warehousing team. The need for a flexible solution became apparent.

Solution: 

Adopting a modern data lakehouse with Starburst and Iceberg

The company initially experimented with Trino, a fast, distributed SQL query engine designed for big data analytics, to address Looker costs and facilitate easier access to data stored in Azure Blob Storage. The proof of concept with Starburst fueled further excitement, leading to the decision to adopt Starburst Enterprise, a full-featured data lake analytics platform. The decision also included using Apache Iceberg as their table format. 

The combination of enterprise-grade Trino and Iceberg provides an open data stack that enables seamless access to various data sources without vendor lock-in. The retailer’s new data stack, built on commodity storage and compute, offers unparalleled optionality and ownership.    

Results: 

The integration of Starburst Enterprise and Apache Iceberg reduces data transformation efforts. The European retailer experienced a 70% reduction in costs, thanks to the separation of compute and storage and the ability to leverage Azure Blob Storage. 

Moreover, Starburst’s federated query engine enhances data discovery and analysis. Analysts and data scientists at the company can now directly access Kafka data for instant data exploration. This empowers teams to ideate and drive innovative ideas faster.

“With this change, we’re able to support many more projects at the same time, so we can focus on our core initiatives,” shares the Director of Engineering at the company. “And all the emerging projects that are growing, and may be important, can be onboarded on the data platform without needing much of our attention.” 

The retailer’s journey with Starburst and Iceberg showcases the evolution of a modern data strategy. The strategy enables the company to not only achieve significant cost savings and accelerated analytics, but also establish a foundation for future scalability.