Starburst Galaxy May 2023 roundup: Optimizing for price-performance

At Starburst, we are hyper-focused on helping you maximize price-performance for individual data workloads. That’s why we’re excited to announce the release of cluster execution modes in Starburst Galaxy. Now, it’s easier than ever to match your compute resources to the exact needs of your job.

Cluster execution modes

A Galaxy cluster is a set of computation resources and configurations on which you run data engineering and data analytics workloads, such as production ETL pipelines, ad-hoc analytics, and BI reports.

With this latest release, you now have the ability to choose between three execution modes – Standard, Fault Tolerant, and Accelerated.

You can make this designation either when creating a cluster or when editing an existing cluster. We will use the rest of this article to explore when to use each mode.

1) Standard mode

All new clusters default to Standard execution mode. This mode is recommended for ad-hoc and exploratory analytics workloads. If you’re not sure which mode fits your workload, best practice is to start with Standard.

2) Fault tolerant mode (public preview)

Fault tolerant mode allows a cluster to retry queries or parts of query processing in the event of failures without having to restart the query. This is especially useful for long-running queries that are typical with data transformation. This includes batch processing and Extract Transform Load (ETL) queries.

💡 Pro Tip: If you’re running data transformation workloads in Galaxy, we also recommend you check out the new Starburst connector in dbt Cloud. 

3) Accelerated (public preview)

Accelerated mode allows you to take advantage of Starburst’s proprietary smart indexing and caching layer – Starburst Warp Speed.

Starburst Warp Speed automatically optimizes data lake queries with smart indexing and smart caching technology. We’ve seen Starburst Warp Speed increase query performance up to 7x.

Accelerated execution mode is in public preview. At the time of this blog, accelerated clusters are only available on four AWS regions, but we are actively working on adding support for more regions and clouds.

To check if your region is supported, navigate to the Galaxy UI, select “accelerated” as the cluster type, and then check the region drop-down. If you don’t see the region you’re looking for, use the chat function to submit a region request.

To learn more about Starburst Warp Speed’s use of nanoblock indexing and why it produces better results than micro-partitioning, check out this blog from our Director of Cache Strategy, Roman Vainbrand.

Get started with cluster execution modes today

To choose your cluster execution mode, log into Starburst Galaxy and access the execution mode drop-down menu by either creating a new cluster or editing an existing one.