What’s New in Starburst Galaxy – August 2022

Over the last two months, we’ve added a robust set of updates to Starburst Galaxy. These new updates include additional data catalog connectivity, improved cluster management capabilities, additional security features, as well as upgrades to Starburst Galaxy’s built-in query editor. These additions come together to help you access more of your data faster, optimize your cluster costs, and maximize analytics flexibility, all leading to higher productivity and greater peace of mind for your analytics environment. This article provides a quick overview of the improvements and some insight into our upcoming roadmap.

Connectivity

On the connectivity front, Starburst Galaxy now has the ability to query Snowflake on AWS and Azure. In addition to Snowflake, we’ve added connectivity to other popular cloud data warehouses including Azure Synapse and Google BigQuery. This connectivity will now provide Starburst Galaxy users with the ability to enrich their data lake analytics with data in the warehouse opening new possibilities in their analytics. The Snowflake and Google BigQuery catalogs are now in preview. In addition, we’ve added MongoDB as a catalog which is also now in preview. You can find out more information about how to connect and query a catalog in the video below:

 

Cluster management

We’re really excited about the enhancements we’ve made to clusters and cluster management in Starburst Galaxy enabling you to more reliably execute long-running workloads like ELT. As you may have read in June, we added a new cluster type to Starburst Galaxy called “Batch”. These clusters have fault-tolerant execution enabled which reduces query failures for long-running workloads and provides more predictable landing times for these types of queries. The combination of Great Lakes connectivity to open table formats like Iceberg and Delta Lake on object storage (S3, GCS and ADLS), and the new Batch cluster type allows you to build an open lakehouse architecture with Starburst Galaxy. Batch clusters enable your data and analytics engineers to create new data pipelines for business analysts and these same analysts can use the platform’s standard cluster mode for data discovery directly on the lake.

If you’ve ever tried to build autoscaling in Trino or big data engines, you know how much of a pain it can be to set-up and tune correctly. That’s why we’re excited to release autoscaling as a feature in Starburst Galaxy. Using the platform, you can now create custom autoscaling configurations by building a new cluster profile and choosing the desired minimum/maximum workers on the cluster. As more or heavier workloads hit the cluster, it scales to meet demand based on your pre-defined tolerances. This new flexibility allows you to optimize for your desired price/performance ratio while hitting your SLAs.

In addition to Batch-enabled clusters and autoscaling, we’ve enabled access to the Starburst Galaxy REST API. This initial release of API management within Starburst Galaxy includes the ability to start and stop a cluster as well as obtain a list of clusters in your environment. Over the next few months, we’ll be working to add more options to the Starburst Galaxy REST API to provide you with a greater degree of programmatic flexibility on the platform. This will allow Starburst Galaxy users to  create more sophisticated data applications leveraging the best-in-class MPP SQL engine.

Security

On the security-side, we’ve added two new mission critical features for our Starburst Galaxy users: Table-level role-based access control and single sign-on (SSO). Role-based access control is critical to define who in your organization can see what data. With the release of the updated access control, Starburst Galaxy users can now manage privileges and access down to the table-level. In addition, we’ve made it even easier to use access controls with a more intuitive UI. We’ve also made logging into the platform even more secure by leveraging SSO with some of the most popular providers like Okta, Google, and Azure AD. This new capability provides you with yet another layer of confidence in the platform and makes adding new users even easier.

Query editor

On top of the additions to connectivity, cluster management, and security, we’ve been adding more features to Starburst Galaxy’s query editor tool. Schema discovery allows users to easily discover schemas and tables in object storage using the query editor (now in preview). With no need to manually define a schema, the time to analytics on Starburst Galaxy is even shorter. The new schema discovery functionality is also a powerful tool for business analysts while exploring data during discovery. Finally, we’ve added support for selecting multiple SQL statements in the query editor and made UX enhancements to improve the user experience.

What’s next?

Over the next few months, we will continue to add additional functionality including more connectivity, cluster optimizations, more granular security features, and enhanced performance. In the fall, we’re also hoping to add more usability features to enhance the in-platform experience in Starburst Galaxy while also providing critical integrations to more of your data ecosystem.

You can try Starburst Galaxy for free with up to $500 in usage credits today!