Optimizing query performance, reducing failure rates, and simplifying Iceberg adoption with Starburst Galaxy

About

Talkdesk is a leading AI-powered contact center as a service (CCaaS) company on a mission to transform customer service worldwide. Recognized for its innovation, Talkdesk delivers critical real-time analytics to its global customer base via an open lakehouse built on Trino and Apache Iceberg. To handle growing customer demand across regions, Talkdesk needed to improve the reliability and stability of its analytics products. By partnering with Starburst, Talkdesk optimized its lakehouse architecture to deliver enhanced performance, improve customer experience, and future-proof scalability.

Challenge

Talkdesk’s rapid business growth strained their Apache Iceberg-based lakehouse, powered by Trino. High query failure rates in Talkdesk Explore, their embedded analytics solution, impacted customer satisfaction and increased the workload on SRE and on-call engineering teams.

Additionally, growing data volumes in Apache Iceberg required Talkdesk to integrate multiple services (e.g., AWS EMR Spark) and develop bespoke internal applications to manage Iceberg table maintenance effectively.

Solution

Talkdesk replaced Open Source Trino with Starburst Galaxy to power its Iceberg-based lakehouse platform. Starburst Galaxy improved query stability and performance, automated data maintenance, and unlocked consolidation of previously disparate AWS services and bespoke applications. Starburst Galaxy’s autoscaling, fault-tolerant execution mode, and enhanced implementation of Trino—an open-source MPP SQL engine—allowed Talkdesk to meet stringent performance and compliance requirements for its analytics product, Talkdesk Explore. Finally, Starburst Galaxy’s automated data optimization streamlines Iceberg table management by handling data compaction, profiling, statistics, vacuuming, and retention policies.

Results

With Starburst Galaxy, Talkdesk achieved significant improvements in query performance, error rates, and operational efficiency, enabling them to deliver faster, more reliable insights to their customers at scale. These advancements have transformed the performance of key analytics products, both internal and external-facing:

• 85% faster query performance: Talkdesk reduced its P99 query times from 20 minutes to 3 minutes for Talkdesk Explore, an externally-facing historical analytics platform. This improvement ensures customers can analyze trends and performance metrics faster and more reliably.
• 150X lower error rate: For Talkdesk Explore, Starburst Galaxy reduced query error rates from 1.5% to 0.01%, ensuring accurate and reliable insights into historical data.
• Streamlined Iceberg maintenance: Centralizing Iceberg data maintenance within Starburst Galaxy eliminated the need for 14 separate applications and six AWS services. This consolidation not only simplified operations but also significantly reduced operational costs by decreasing maintenance overhead and infrastructure complexity.

With its new lakehouse architecture built with Starburst Galaxy, Talkdesk is now positioned to scale a key analytics product—Talkdesk Explore—with greater reliability and performance. Looking forward, Talkdesk aims to further leverage its lakehouse by using Starburst Galaxy fine-grained access controls to allow internal business users to analyze customer data.

More Resources: From Beeland to the Icehouse with Starburst Galaxy | Datanova 2024