Making data from any location available and discoverable in real-time

About

This Fortune 500 oil and gas company runs three primary business operations: upstream oil and gas exploration and production, midstream storage and transportation, and downstream refining and distribution. Each operation requires that data domains be standardized for the major business processes. This company must make timely decisions using data that’s spread across multiple, disparate systems.

Challenge

For almost three years, the oil and gas company used Denodo for data querying and data virtualization across disparate sources, despite continual limitations from slow and failed queries. After a data re-platform, Denodo query performance worsened to the point that it would fail if the query was long or even slightly complex. Even when the query worked, it took five minutes to refresh data reports.

The data virtualization platform couldn’t handle the scale and concurrency required for reporting, leaving the engineers to manually join data. This cost the business in overhead expenses and delayed insights while taking the engineers away from performing other tasks. On top of this, Denodo was expensive, and the data engineering team didn’t like the way the catalog was organized.

The engineering department recognized it needed a high-performing, secure MPP query engine with the ability to connect and query all data sources natively. This functionality would provide a complete view of oil well information for use in reporting, analytics, and workflow optimization. 

Solution

After running on cross-platform joins with Starburst Enterprise, the oil and gas company was blown away by the performance and efficiency. It selected Starburst to unlock visibility across all of the company’s data. 

“Starburst has a great reputation at our company,” shares the Data Engineering Manager. “The cross-platform joins makes it seem like Starburst is breaking the laws of physics. We appreciate the open-source approach and the fact that Starburst actually contributes to the open-source nature of the standards.”

Data analysts and geoscientists can now view data from all of the company’s oil and gas reservoirs. Starburst integrates with the company’s data stack and analytics tools, powering business intelligence (BI) tasks at scale without sacrificing performance. 

Starburst Enterprise keeps data secure with enterprise-grade security features like fine-grained access control, data masking and encryption, column and row-level security, and query auditing. Dedicated resources and 24×7 support from Starburst’s team of Trino experts was another deciding factor. According to the data engineering manager, “Having Starburst and open source technology is the best of both worlds.” 

Results

The oil and gas producer can now provide quick, easy access to data across multiple levels within the organization while keeping data secure. Starburst has helped the company achieve the following results:

Holistic view of well master data. With a single query, end users can pull data from anywhere. This enables the organization to maintain a consistent and accurate view of the entire lifecycle of the wells. 

Near real-time reporting. Executives can now go into their reporting tool and query data themselves, instead of waiting on the engineering team to build pipelines for their reports. 

30% less data movement. The oil and gas producer streamlines and enhances existing data pipelines with SQL transformations. This reduces data movement, data copying, and ETL by 30% while ensuring the quality of its data with ad-hoc queries. 

Increased productivity.  The engineering department has freed up time to focus on more strategic initiatives, instead of managing data pipelines and copying data. This allowed the team to increase its workload significantly without increasing staff or engineering resources.

Overall, having well-managed master data is critical to supporting the oil and gas company’s business initiatives. The oil and gas producer now has visual access to exploration, drilling, and production operations through the Starburst query engine, allowing it to make better decisions that reduce risk, accelerate decision-making, and streamline operations.