Secure and scalable mobility intelligence with faster data insights and lower costs

Arity is a mobility data and analytics company focused on enhancing safety and efficiency for drivers, communities, and businesses. Leveraging one of the world’s most extensive driving behavior datasets connected to insurance claims, Arity provides advanced insights to clients across insurance, public sector, and advertising. As Arity scaled its operations and data processing needs, its existing Spark-based cloud solution began showing significant performance and cost limitations. Seeking a faster, more accessible, and secure data analytics platform, Arity selected Starburst to help optimize costs, accelerate data processing, and make their insights accessible to a broader range of teams with their new lakehouse.

Arity’s operations generate an immense volume of driving data, providing rich insights into traffic patterns, safety, and driver behavior. However, they faced several barriers to effectively analyzing this data at scale:

• High costs: The Spark setup was resource-intensive and costly to scale.
• Complexity and accessibility: Arity’s existing technical complexity limited data access to fewer than half of Arity’s engineers. This restriction created bottlenecks for Arity’s business teams, preventing departments like sales engineering and professional services from accessing data independently.
• Speed and efficiency: For mobility-focused applications, such as analyzing traffic patterns to inform city planning or aggregating driver behavior for safety insights, Arity’s system often requires long processing times. Queries on large datasets could take up to 45 minutes, slowing time-to-insight and impacting timely decision-making.
• Data security needs: Arity needed a solution that allowed data to remain securely within its infrastructure, meeting stringent data privacy and strict requirements.

To meet these challenges, Arity adopted Starburst Galaxy, a fully managed, highly performant, secure, and user-friendly data lakehouse platform. Shifting from Spark to Starburst Galaxy, a Trino-based MPP SQL-based engine, enabled Arity to benefit from:

• Enhanced processing speeds: Query times were reduced by 10X, enabling Arity to process data in minutes instead of hours. This was essential for time-sensitive use cases like mobility insights and real-time traffic and road planning analytics.
• Lowered costs: Arity reduced its data processing costs to 33% of what it previously spent on Spark services. This reduction freed up the budget and allowed for expanding analytics into new areas, such as targeted advertising and usage-based insurance models.
• Broader data accessibility: Starburst Galaxy’s SQL interface allowed a broader range of employees—including non-engineering teams and analysts within sales engineering and professional services—to directly access and analyze data, doubling the number of internal data users.
• Secured data infrastructure: Starburst Galaxy’s architecture ensured that Arity’s data remained within their controlled Apache Iceberg environment, meeting Arity’s security and data ownership requirements without compromising operational efficiency.

Arity’s transition to Starburst Galaxy resulted in:

• Significant cost savings with processing costs reduced by approximately 66%.
• 10x faster data insights, enabling teams to analyze data in a fraction of the time.
• Increased data accessibility across departments, empowering non-engineering teams to run queries and generate insights independently.
• Enhanced data governance with secure, in-place data processing, ensuring compliance with stringent data security policies.

Looking forward, Arity plans to further integrate Starburst across additional business areas, including advertising and insurance. By leveraging Starburst’s data lakehouse capabilities, Arity aims to introduce enhanced data governance and achieve greater real-time analytics. Future potential expansions include using data to create targeted audiences in advertising and developing fairer, usage-based insurance policies based on actual driving behaviors. This continued partnership will support Arity’s strategic growth, and their ability to drive data-driven decisions and deliver secure, scalable analytics across all core sectors.

More resources:

• BigDATAwire article: Spark-to-Starburst Engine Swap Speeds Big Driving Data for Arity
• Press release: Arity Boosts Data Processing Speed 10X and Cuts Costs by 66% with Starburst Galaxy