Last Updated: 2024-03-20

Background

AWS PrivateLink allows private connectivity between virtual private clouds (VPC), supported AWS services, and on-premises networks. This connection does not expose traffic to the public internet, making it a great choice for data federation across cloud and on-prem networks.

Starburst Galaxy extends support for AWS PrivateLink across certain catalogs. This tutorial will guide you through the process of configuring PrivateLink for an on-premises data source.

Scope of tutorial

In this tutorial, you will learn how to configure AWS PrivateLink for an on-premises data source.

Learning objectives

Once you've completed this tutorial, you will be able to:

• Configure AWS PrivateLink for connectivity from Starburst Galaxy to your on-premises data source.
• Use PrivateLink to securely connect Starburst Galaxy to your on-premises data source.

Prerequisites

• You need a Starburst Galaxy account to complete this tutorial. Please see Starburst Galaxy: Getting started for instructions on setting up a free account.
• This tutorial comes with a bring your own storage requirement. Before proceeding with this lesson, you must already have an on-prem data source set up.

About Starburst tutorials

Starburst tutorials are designed to get you up and running quickly by providing bite-sized, hands-on educational resources. Each tutorial explores a single feature or topic through a series of guided, step-by-step instructions.

As you navigate through the tutorial you should follow along using your own Starburst Galaxy account. This will help consolidate the learning process by mixing theory and practice.

Background

If you are configuring PrivateLink for the first time you are encouraged to work with a Starburst technical resource. This individual will work with you to set up the environment needed to complete the tutorial.

Contacting your technical resource

To be assigned this resource, you should reach out to your regular Starburst account team for assistance.

Working together

Once assigned, your Starburst technical resource will work with you to set up an environment where you can complete the tutorial.

Please review the following overview of this process before beginning the tutorial.

Your responsibilities:

• Record the IP address and port for your data source.
• Create a target group.
• Create a network load balancer.
• Create an endpoint service.
• Allow the Starburst Galaxy AWS account principal to use the endpoint service.
• Submit a support request via Starburst Galaxy to have an endpoint connection created.
• Accept the endpoint connection in your AWS account.

Background

Understanding the AWS PrivateLink on-prem architecture is important when completing the steps in this tutorial.

In this section you will learn about this architecture and the way that Starburst Galaxy uses it to securely connect private clouds.

Reference architecture

The following diagram illustrates a PrivateLink connection between the Starburst Galaxy VPC, your VPC, and your on-prem data source.

Review the diagram below for more information.

Background

It's time to get started. In this section, you'll begin by obtaining some key information about your data source, including:

• IP address
• Port number

You'll need this information to create a target group and load balancer in the next sections of this tutorial.

Step 1: Record data source IP address and Port number

For Starburst Galaxy to connect to your data source using PrivateLink, you will need to know both the IP address and Port number for your data source.

How you do this will vary depending on the implementation and underlying technology used to construct your on-prem data source.

• Locate and record the IP address for your data source.
• Locate and record the Port number for your data source.

The following are examples of each:

• IP address: 172.28.12.4
• Port number: 3306

Background

Now it's time to set up a target group. In the context of AWS, a target group is responsible for directing incoming traffic from a load balancer to designated targets, such as cloud instances, containers, or IP addresses.

In this tutorial, the target group you create will play a crucial role in routing traffic to your data source's IP address. This ensures efficient communication between the load balancer and data source, optimizing performance and reliability.

Step 1: Start the target group wizard

AWS makes creating target groups easy with a creation wizard, accessed through the EC2 dashboard. You are going to use that wizard to create your target group.

• Navigate to the EC2 dashboard in the AWS console. This can be done by searching for EC2 and clicking EC2 in the results list.
• Using the left-hand navigation bar, expand the Load Balancing menu and select Target Groups.
• Click the Create target group button on the right.

Step 2: Provide a target group name

Now it's time to configure your new target group.

AWS will ask you to select a target type and provide a meaningful name.

• In the Basic configuration section, select IP addresses.
• In theTarget group name field, enter a meaningful name .

Step 3: Configure the target group

Next, you're going to configure your target group for use with your data source. To do this, you're going to use some of the details that you copied into your text editor earlier in this tutorial.

• Using the Protocol drop-down menu, select TCP.
• Enter the port number used by your data source.
• Select IPv4.
• Select the VPC for your data source.
• Using the Health check protocol drop-down menu, select TCP.
• Click the Next button.

Step 4: Complete configuration process

Almost there! For the final step, you're going to finish the configuration process and create the target group.

• Expand the Network drop-down menu and select Other private IP address.
• Expand the Availability Zone drop-down menu and select All.
• In the Enter a private IP address field, enter the IP address of your data source.
• In the Ports section, click the Include as pending below button.
• Confirm that your data source IP is now listed under Targets and that its Health status is shown as Pending.
• Click the Create target group button.

Background

Now it's time to create a network load balancer. In AWS, a Network Load Balancer (NLB) is a service that automatically distributes incoming network traffic across multiple targets based on IP protocol data. This includes Amazon EC2 instances, containers, and IP addresses. Load balancers are also configurable across either a single AWS Availability Zone or multiple Availability Zones.

After configuring PrivateLink, an endpoint in the Starburst Galaxy VPC will connect to your Network Load Balancer using a service located in your VPC.

Step 1: Start the load balancer wizard

Once again, AWS makes the process of creating a load balancer easy by providing a wizard.

• Using the left-hand navigation menu, in the Load balancing section, select Load Balancers.
• Click the Create load balancer button on the right side of the dashboard.

Step 2: Select load balancer type

AWS load balancers come in several different types. These include Application Load Balancers, Network Load Balancers, and Gateway Load Balancers.

For this tutorial, you're going to select the Network Load Balancer.

• Select the Network Load Balancer by clicking the corresponding Create button.

Step 3: Name your load balancer

It's time to start configuring your new load balancer, starting with a name.

• Enter your Load balancer name in the field provided.

Step 4: Configure the load balancer

Next, you're going to configure your load balancer for use with your on-prem data source.

• In the Scheme section, select Internal.
• In the IP address type field, select IPv4.
• In the VPC section, select your VPC using the drop-down menu.

Step 5: Configure AWS availability zone mappings

Now it's time to map AWS availability zones to subnets for your load balancer. You will need to map each listed availability zone to a private subnet capable of routing traffic to your on-prem data source.

• Select the first Availability Zone from the list.
• Select an associated private Subnet capable of routing traffic to your data source.
• Leave the Private IPv4 address field unchanged.
• Repeat this process for the remaining availability zones.

Step 6: Configure security group IP CIDR

Next, it's time to select a security group to control access between your load balancer and data source. Without this security group, your network load balancer will accept all connections, creating a security risk in production environments.

• In the Security groups section, select a Security Group with inbound rules allowing the IP CIDR 10.0.0.0/8 for your database port.

Step 7: Configure port number and target group

• Enter the Port number of your data source.
• Using the Forward to drop-down menu, select the target group that you created earlier in this tutorial.
• Click the Create load balancer button.

Step 8: Wait for load balancer to activate

That's it! Your load balancer is now being created. This process takes between three to five minutes.

• Wait for the State to change from Provisioning to Active before moving to the next step.
• Click the Refresh button to view status updates.

Background

Now it's time to create an endpoint service.

In the context of AWS PrivateLink, an endpoint service allows you to expose services running in your VPC to other accounts within the same AWS region using a private connection.

Step 1: Start the endpoint service wizard

Just like target groups and load balancers, AWS includes a wizard to help you create an endpoint service.

• Navigate to the VPC dashboard in the AWS console. This can be done by searching for VPC and clicking on VPC in the results list.
• Using the left-hand navigation menu, expand the Virtual private cloud menu, and select Endpoint services.
• Click the Create endpoint service button on the right side of the dashboard.

Step 2: Name your endpoint service

It's time to start configuring your new endpoint service, starting with a name.

• In the Name field, enter your endpoint service name.
• In the Load balancer type section, select Network.

Step 3: Configure endpoint service

Now it's time to configure your endpoint service. To do this, you're going to make sure that it connects with your network load balancer and uses the correct IP address.

• In the Available load balancers section, select your network load balancer.
• In the Supported IP address type field, select IPv4.
• Click the Create button.

Background

Time to switch gears. You've completed all of the steps required on your own. Now it's time to contact the Starburst support team to finish the last steps.

Step 1: Enter the Starburst Galaxy ARN

In the last section of this tutorial, you created your endpoint service. At the end of that process, you are directed to a page that displays the details of that service.

You're going to use this section to input the Starburst Galaxy Amazon Resource Name (ARN).

• Select the Allow principals tab under the Details box.
• Select the Allow principals button.
• Enter the following ARN in the ARN field: arn:aws:iam::179619298502:root
• Select the Allow principals button.

Step 2: Record Service name

Now it's time to locate and copy the service name for your endpoint service. This is one of the endpoint service details listed in AWS.

The Starburst support team will use it to create the endpoint in Starburst Galaxy.

• Scroll up and copy the Service name.

Step 3: Open support ticket

You are going to use the automated assistant in Starburst Galaxy to open a support ticket and provide support with the Service name that you just copied. You will also need to provide the port your database is listening on and your preferred Starburst Galaxy PrivateLink configuration name.

• Log in to Starburst Galaxy.
• Click the support icon located at the bottom right of the screen.
• Select Chat with technical support.
• Select Submit a Support Ticket.
• The automated assistant will ask you to provide your email address, first name, and last name.
• When you receive the prompt to describe your issue, note that you would like support to create a private endpoint connection for you. Be sure to include the Service name you just copied, the port your database is listening on, and your preferred Starburst Galaxy PrivateLink connection name.

• Wait for Starburst support to confirm that they have created the Endpoint in Starburst Galaxy. This should take no longer than 24 - 48 hours.

Step 4: Select the Starburst Galaxy endpoint

Do not begin this step until you receive confirmation that the Starburst Galaxy endpoint has been created successfully.

• Scroll down and select the Endpoint connections tab.
• Wait to see the connection listed.
  
  Note: You may need to click the Refresh button.
• Select the endpoint from the list.

Step 5: Accept the endpoint connection request

Now that you've selected the Starburst Galaxy endpoint, it's time to accept the connection request.

• Expand the Actions drop-down menu.
• Select Accept endpoint connection request.
• Manually enter accept in the field.
• Click the Accept button.

Step 6: Confirm endpoint connection

That's it. The connection is now being created. This process takes between 1 to 3 minutes to complete.

When this process is complete, you are finished and ready to start using PrivateLink.

• Wait for the State to change from Pending to Available.
• Click the Refresh button to view status updates.

Tutorial complete

Congratulations! You have reached the end of this tutorial, and the end of this stage of your journey.

You're all set! Now you can use PrivateLink to configure access to your organization's on-prem data.

Continuous learning

At Starburst, we believe in continuous learning. This tutorial provides the foundation for further training available on this platform, and you can return to it as many times as you like. Future tutorials will make use of the concepts used here.

Next steps

Starburst has lots of other tutorials to help you get up and running quickly. Each one breaks down an individual problem and guides you to a solution using a step-by-step approach to learning.

Tutorials available

Visit the Tutorials section to view the full list of tutorials and keep moving forward on your journey!