Introducing new SQL enhancements in Starburst Galaxy

Data professionals often grapple with challenges such as code complexity, efficiency, and readability issues in SQL queries. Manually handling repetitive or resource-intensive tasks not only impedes productivity but also introduces the risk of errors in critical data processing.

Today, we are excited to announce two new features in Starburst Galaxy to streamline SQL workflows and reduce operational overhead— Jobs and SQL routines.

Jobs

Consistency and predictability are of paramount importance to data engineers handling high-value workloads. Central to achieving these goals is the pivotal role played by scheduling and orchestration functionality, empowering engineers to reliably automate and streamline their workload processes.

The Jobs feature within Starburst Galaxy empowers users to effortlessly schedule and automate the execution of SQL jobs. This feature offers users the ability to define specific SQL queries within Galaxy and set up a recurring schedule for their execution.

Furthermore, Galaxy offers monitoring and logging capabilities, allowing users to track job progress, review execution history, and receive alerts in case of failures or anomalies.

Learn more on scheduling and executing jobs in Galaxy here.

SQL Routines

A SQL routine is a custom, user-defined function authored by a Starburst Galaxy user, also known as a SQL UDF. These user-defined functions can encapsulate a series of SQL statements and calculations, which can be reused within SQL queries just like a built-in function.

The support for SQL routines in Galaxy is limited to scalar UDFs – functions that return a single value based on input parameters rather than an entire result set. This scoping helps ensure efficiency and precision in your SQL operations.

Consider the simplicity of a scalar SQL routine with this example function that calculates the square of a given number:

CREATE FUNCTION square (x double) 
RETURNS double RETURN (x * x);