Querying across borders to
advance data-driven medicine

About:

SOPHiA GENETICS is advancing and democratizing data-driven medicine through a pioneering global network of healthcare institutions. Working with more than 780 hospitals and research institutions in over 70 countries, SOPHiA GENETICS enables its customers to outsource their bioinformatics operations by providing them with both a cloud-based, Software-as-a-Service analytics platform and unprecedented insights from the global network. This way, SOPHiA GENETICS’ customers can focus on what they do best — advancing research, treatment decisions, and drug development efforts.    

Challenge: 

Over the years, SOPHiA GENETICS has come to rely on a mix of different backend storage systems. Cataloging data and collecting business metrics were becoming increasingly difficult, since application data is distributed globally to comply with various regional and national data security and compliance requirements. Ultimately, the data services team wanted to be able to catalog its data and allow creating business insights in a secure, controllable, and demonstrably compliant way.

Solution: 

Starburst Enterprise, the fully supported, production-tested distribution of open source Trino, improves performance while making it easy to deploy, connect, and manage your cluster. It includes additional connectors for commercial database systems along with query optimization, cluster management tools, and enhanced security – an especially important feature to SOPHiA GENETICS.

“Starburst is creating the infrastructure to realize our business metrics demands, while tightly controlling access to source systems,” Alexander Seeholzer, Director, Data Services at SOPHiA GENETICS explains, “and it offers enterprise-grade extensions, auditability, and more at an affordable price. After our evaluation, we realized it was the most logical choice.”

Today, the data SOPHiA GENETICS manages resides in data warehouses within specific regions or countries. SOPHiA GENETICS deploys Starburst via Kubernetes and operates numerous instances, including in-country or in-region clusters that ensure compliance with local data regulations. 

Key features:

Fine-grained access control:

• Managing data consumers and gathering insights into their activity has improved significantly. “We basically have one entry point that we can use to serve access to different users and automated systems, whereas before, access had to be done on a per-resource basis. It was a service overhead to maintain that,” notes Seeholzer. “Starburst allows us to specify on a per user basis, in a very fine-grained manner, who is allowed access to what, and it gives us an auditable trail of that activity.”

Regional compliance:

• SOPHiA GENETICS adheres to strict requirements to secure data within specific regions or countries, as regulations demand. “Due to compliance constraints, we simply can not deploy any system that accesses all data from one central point,” Seeholzer says. “One advantage of Starburst is that we can deploy Starburst distributed too, in each region, and make it so the source data used to generate business metrics never leaves the region.”

Exploration & discovery:

• Starburst has also made it easier for  SOPHIA’s data services team to explore and catalog data. In the past, this would have been done semi-automatically, but now the process can be fully automated. The team can easily build, maintain, and update catalogs of data across different storage systems, which in turn makes discovery much easier.

Starburst Stargate

• SOPHiA GENETICS relies on Starburst Stargate, a cluster-to-cluster connector, designed to analyze distributed data while remaining compliant. “We don’t have one Starburst cluster that queries everything,” explains Seeholzer. “We have Starburst clusters everywhere, and another Starburst-to-Starburst connector that queries all clusters, in a secure and compliant fashion.” Starburst’s optimizer reduces the amount of data transferred over the network by executing aggregation operations in source systems when possible. This allows the team to collect the necessary business metrics.

Results: 

SOPHiA GENETICS sees various strategic and operational advantages of the solution.

10-15X more data available

By unlocking siloed data, SOPHiA GENETICS has 10-15X more data available that is leveraged to improve their product offering. Additionally, the company expanded the number of users able to query data from 3 to 30, a 900% increase in data access. 

Centralized data access 

Data activities that had been dispersed are now accessed through a single point of secure access, making them more controllable and observable. 

Compliance

With Starburst, it’s easier for the data services team to demonstrate to auditors, the QA department, and others that they are adhering to policies and regulations.

Time-to-insight 

Business analysts can explore data faster because the datasets, columns, and rows they’re permitted to access have already been established, and they don’t need to appeal to data services. This cuts down on the turnaround time and, ultimately, accelerates time-to-insight.

Accelerating the data mesh initiative
As SOPHiA GENETICS advances its mission, Starburst Enterprise will continue to be an important piece of its infrastructure. In addition to the benefits outlined above, the platform is advancing one of the larger strategic goals of the data services team – moving toward the increasingly popular data mesh architecture now being adopted by many forward-thinking enterprises.

More resources: Alexander Seeholzer’s Voyager profile