Consolidating data sources, enhancing analytics efficiency, and accelerating insights

About

Hundreds of thousands of businesses worldwide rely on this software company every day to automate bill and invoice payments and sync accounting software. The cloud-based software provider helps small and midsize businesses simplify, digitize, and automate back-office financial processes.

Challenge

Seeking to support new acquisitions with a better data platform 

This data-driven organization uses invoice, payment history, and other customer data to gain new business insights. “Our data scientists and product analytics teams need data to track business metrics and find out more about our operations,” says the Senior Director of Engineering for Data Platform. “We also process a lot of payments every day, so we need to make sure our customers are getting the full value from our products and features.”

Data analytics became challenging, though, as the company started to grow rapidly and acquire new businesses over the past several years. “We have a small data team, and we had two acquisitions in six months, which brought a bunch of new, disparate data platforms and technology stacks to our company,” the Senior Director of Engineering for Data Platform says. These technologies include Amazon Web Services (AWS)—an Amazon Redshift data lake and another data lake based on AWS Glue—as well as a Snowflake data warehouse. “Our challenge was making all this new company data available quickly. Rather than having our analysts spend their time fetching the data from different systems on their own, we knew we had to bring all the data together in one place, so they can access it easily and focus on the actual analysis.”

Solution

Connecting and consolidating data with Starburst Enterprise on AWS

In early 2022, the financial software company began evaluating Trino, an open-source engine for querying large datasets. That evaluation led them to Starburst, which offers a full-featured data lakehouse platform built on top of Trino. “Starburst was exactly what we were looking for, because immediately after deployment and setup it would provide blended analytics across different sources in a single query,” says the Senior Director of Engineering for Data Platform. 

Starburst Enterprise is an end-to-end analytics platform that helps teams discover, organize, and consume data from one source. The software company uses the platform to connect an existing Amazon Simple Storage Service (Amazon S3) data lake with the Snowflake and Amazon Redshift data solutions from recent acquisitions. “Starburst Enterprise allows us to federate data across multiple data sources into a consolidated place for data analysis and visualization,” says the Data Platform Leader at the financial software company.

Without using Starburst Enterprise, the customer would likely have needed to build separate solutions for extracting data from multiple sources and physically copy the data from each source. “That really would have required a lot of engineering time and effort,” says the Senior Director of Engineering for Data Platform. Starburst technical teams worked closely with them to connect the different data sources from newly acquired companies. “This was a problem that was difficult to solve, but the Starburst team helped us easily resolve any issues we had,” says the customer. “We didn’t have to create duplicate datasets among different repositories to run analytics.”

Results 

Now, more than 200 of the company’s data scientists and product analysts have immediate access to a consolidated data lake that holds close to 50 terabytes of financial and customer data.

Scaling data pipelines on demand

With Starburst Enterprise, the financial solutions provider can scale its data pipelines as needed to support the company’s rapid business and data growth. “Using Starburst Enterprise on AWS, we can manage heavy data usage across all these different data solutions from a single place. That helps us optimize the entire platform to scale, versus having to scale each platform separately,” the Data Platform Leader says. The financial software company can also dynamically fit its new data analytics and visualization solution by providing easy access to everyone, not just a handful of data scientists. “Through federation, we were able to get anyone access to the data quickly, so our engineers can spend their time on more productive tasks.”

Reducing time to insight from days to minutes 

Data teams joining the company via a merger or acquisition now have access to data immediately, helping speed time to insight and supporting business goals. The Senior Director of Engineering for Data Platform says, “We can seamlessly run analytical queries across multiple data solutions with Starburst Enterprise and empower our data teams to perform self-service analytics. By making data available on day one, analysts get to the data in minutes instead of days, which means we accelerate time to insight and our customers can gain more value from the data as well.” 

As the next step in its data analytics journey, the customer will explore the use of AI to support machine learning use cases. “Being able to quickly access data from different places for AI models is important,” says the Data Platform Leader. “We’re specifically interested in using data to support cross-sell initiatives following acquisitions. All companies are looking for synergies from companies they acquire, and we expect to use Starburst and AWS to help us do that.”