Introducing Fast Warm-Up and Autoscaling to Warp Speed in Starburst Galaxy

Today, we’re thrilled to announce significant enhancements to Starburst’s Warp Speed technology, a fully automated acceleration layer that delivers a 3-5x boost in performance for interactive workloads while reducing CPU costs by up to 40% and object storage costs by up to 50%.

These latest updates introduce new features that make Warp Speed even more powerful, including Fast Warm Up (Cache Resiliency) and Autoscaling, further optimizing your data lake queries for cost-efficient performance.

What’s new in Warp Speed?

1. Fast Warm Up (Cache Resiliency)

Warm-up is the process of building cache and resiliency, a CPU-intensive operation that can take anywhere from minutes to hours depending on cluster resources and data size. This process is crucial for ensuring that queries run efficiently, but it presents a major challenge in cloud-based analytics, particularly during “cold start” scenarios. When clusters restart or scale, the lack of pre-built caches can lead to slower query performance, falling short of the low-latency expectations for interactive analytics.

To tackle this, our Fast Warm-Up feature introduces a resilient three-tier storage system:

• Hot tier: Persistent caching that retains data across cluster restarts and scaling events.
• Warm tier: A secondary layer that accelerates cache rebuilds, reducing warm-up times by 3x for more consistent, high-performance query execution.
• Cold tier: Backup storage for long-term data retention, enhancing disaster recovery scenarios and simplifying the spin-up of new environments.

These improvements ensure enhanced reliability and responsiveness for mission-critical analytics workloads, meeting the demands of interactive query latency SLAs.

2. Autoscaling

To complement Fast Warm-Up, we’re excited to roll out autoscaling for accelerated clusters in Galaxy!

This new feature automatically adjusts cluster size based on your workload needs, ensuring optimal resource use and cost efficiency. Galaxy’s autoscaler maintains consistent performance even during peak times without the hassle of overprovisioning. 

Together with Warp Speed’s capabilities, these updates let you scale with confidence, ensuring that performance and costs are consistently optimized. Enjoy fewer manual adjustments and more reliable, cost-effective analytics, allowing you to focus on getting the most out of your data.

Learn more.

Performance that speaks for itself

A global airline recently conducted a benchmarking project comparing Databricks, Teradata, and Starburst Warp Speed, and the results were nothing short of impressive:

1. Improvements of up to 99.61% in some test cases compared to existing solutions.
2. A specific query that cost $24 on Databricks was reduced to just $1 with Starburst Warp Speed.
3. Starburst empowered gate agents with quicker access to crucial data, enabling them to make better voucher decisions for high-priority customers.

The impact on query costs was substantial. The airline expects to migrate 45%+ of existing workloads from existing solutions and reduce spending on queries by $6.9M over the next three years. With the addition of Fast Warm-Up and Autoscaling to Warp Speed, even greater improvements in performance and cost savings are anticipated for this airline and all our customers.

What this means for you

These enhancements to Warp Speed offer several key benefits:

1. With Fast Warm Up, your analytics are consistently fast, even after cluster restarts or scaling events.
2. Autoscaling ensures you’re only using (and paying for) the resources you need, when you need them.
3. Persistent caching means more stable performance for your critical analytics workloads.
4. The autonomous nature of Warp Speed, now extended to scaling, reduces the operational burden on your data teams.

Getting Started with Warp Speed

Enabling Warp Speed in your Starburst Galaxy environment is easy! Log in to your Galaxy account or start for free today. You’ll be running accelerated queries in minutes! 

Follow these steps:

1. When creating a new cluster, select “accelerated” as the execution mode.
2. The new Fast Warm Up and Autoscaling features will be automatically enabled for your accelerated clusters.
3. Start running your queries and experience the benefits immediately.

Interested in learning more? Schedule a demo with our team or check out our Warp Speed hub.