Improving data access for faster time-to-insight and a superior customer experience

About

Bank Hapoalim is one of the largest banks in Israel with locations all over the world. The bank has developed an extensive network of branches and commercial banking activities, servicing more than 2.5 million accounts and providing a broad range of financial services to its customers.

After 20 years of storing data in a traditional relational data warehouse, Bank Hapoalim’s Information and Intelligence team decided to adopt a data lake architecture to allow data consumers to access the data regardless of infrastructure limitations.

Challenge

Migrating data from the bank’s legacy system to the data lake in Hadoop required the Big Data team to monitor and maintain data pipelines to perform ETL. This took up valuable engineering time and resources. 

At the same time, the bank was experiencing very slow query turnaround times from its Apache Hive infrastructure. The resources needed to build reports and dashboards created a data access bottleneck. 

These performance issues led the Big Data team to evaluate another solution to allow analysts to run queries faster. 

Solution

After testing Starburst against Hive and Spark, the Big Data team deployed Starburst as the query engine for Bank Hapoalim’s data lake. To assist with the deployment, BigData Boutique, a premium consulting firm based out of Israel, provided Trino subject matter expertise and hands-on support to the bank. The consulting firm helped the bank install Starburst Enterprise at the bank in less than three months.

With Starburst, legacy data coming from sources such as bank accounts or ATMs could remain in its original form. Data consumers could then query and explore the data quicker than ever before. Business intelligence (BI) analysts now have access to real-time data from various tools in the bank, including Bank Hapoalim’s machine learning platform. 

Starburst enables Bank Hapoalim to grow and scale. The bank can migrate its data to the cloud through the Starburst engine without disrupting business-as-usual analytics. Additionally, data scientists can run SQL queries across distributed data that connects across sources, regions, and clouds. 

With 50+ supported enterprise connectors, Starburst connects to Apache Hadoop, Apache Ranger, Apache Hive, and Dataiku. 

Results

With the deployment of Starburst, Bank Hapoalim enjoys the following advantages: 

Greater query performance. Starburst’s performance enhancements to Trino — such as dynamic filtering, cost-based optimizer, query pushdown, and cached views — improve query performance.  

Quicker time-to-insight. Data scientists and analysts have access to reports and dashboards, enabling them to get new models and applications into the hands of business decision-makers faster.

Enhanced self-service analytics. Starburst connects to Bank Hapoalim’s BI tool and serves as the data consumption layer, supplementing dashboards and reports with a secure single point of access to data. Starburst’s integration with SAS and Qlik unlocks new possibilities for data discovery and insight generation. 

Automated scenario testing. As the bank implements and tests new applications, it can streamline the experience by leveraging Starburst to query and find customers that meet the parameters for the scenarios it wants to run. 

The Big Data team chose Starburst because the solution enables end users and analysts access to data for rapid ad-hoc interactive queries in addition to business intelligence reporting. By deploying Starburst on its data lake, Bank Hapoalim has easy access to more data sources and has ensured the organization’s future readiness with scalable technology.