Starburst Galaxy allows users to run Trino at scale to query their data where it resides. AWS PrivateLink has had strong adoption by our user base and continues to be fundamental to enabling users to query their data from Trino without any security concerns.
Announcement on AWS Cross-VPC resources
Today, AWS announced that Cross-VPC resource access will now be available via AWS PrivateLink and VPC Lattice. It’s a major announcement that drastically simplifies the infrastructure we have to provision and maintain on behalf of our customers. Before this, if either we or our customer has to make any changes to the PrivateLink endpoint service or VPC endpoint, it could result in a manual coordination of changes between us.
Cross-VPC resource access leverages AWS RAM to simplify the authorization between the two parties allowing us to make changes without impacting customers. This gives us far more flexibility in our infrastructure. Specifically, it allows us to deprecate some of the complexity we had to introduce into our data planes to enable us to re-provision our AWS EKS clusters if needed.
What this means for users
For our users, cross-VPC resource access will allow you to easily leverage Amazon VPC Lattice service networks to reach customer data sources. Amazon VPC Lattice service networks allow Starburst Galaxy to secure connectivity to your data at a finer granularity than with just having AWS PrivateLink alone. Today we have a good deal of instrumentation in our connectors to ensure we don’t have any potential cross-account communication between Trino instances and VPC endpoints. AWS Cross-VPC resource access via AWS PrivateLink and VPC Lattice will allow us to route traffic from specific host instances directly into provisioned service networks for specific customers.
AWS Cross-VPC support and Streaming Ingest
Beyond this, AWS cross-VPC resource access will also help with streaming ingestion. Before this, managing streaming ingestion using PrivateLink required a manual connection to MSK. With cross VPC resource access this should be a much simpler and easier process.
If you would like to learn more about streaming ingest, check out the video below.
