Last Updated: 2024-01-19

Background

Object storage is a type of data storage system that organizes data as objects rather than files. It is commonly used in data lakes and is one of the key reasons that data lakes are able to efficiently store large volumes of unstructured and semi-structured data.

Starburst Galaxy offers two methods for connecting to object storage in AWS.

Configuring a cross-account IAM role
Using an AWS AccessKey and SecretKey pair

This tutorial will follow the first approach, and will show you how to create a cross account IAM role using a step-by-step process.

Scope of tutorial

In this tutorial, you will learn how to configure a cross account IAM role. To do this, you will work in both the AWS UI and Starburst Galaxy UI.

Learning objectives

Once you've completed this tutorial, you will be able to:

Configure a cross account IAM role in AWS.
Use the cross account IAM role to securely connect Starburst Galaxy to your Amazon S3 bucket.

Prerequisites

You need a Starburst Galaxy account to complete this tutorial. Please see Starburst Galaxy: Getting started for instructions on setting up a free account.
This tutorial comes with a bring your own storage requirement. Before proceeding with this lesson, you must already have an Amazon S3 data lake set up. You must also have an IAM policy set up for use with Galaxy.

If this is not the case, please set this up first then return to this tutorial.

About Starburst tutorials

Starburst tutorials are designed to get you up and running quickly by providing bite-sized, hands-on educational resources. Each tutorial explores a single feature or topic through a series of guided, step-by-step instructions.

As you navigate through the tutorial you should follow along using your own Starburst Galaxy account. This will help consolidate the learning process by mixing theory and practice.

Background

Starburst Galaxy separates users by role. Configuring a new catalog will require access to a role with appropriate privileges. Today, you'll be using the accountadmin role.

This is a quick step, but an important one.

Step 1: Set your role

First, you're going to start by confirming that you are using the accountadmin role. Your current role is listed in the top right-hand corner of the screen.

Check your role to ensure that it is set to accountadmin.
If it is set to anything else, use the drop-down menu to select the correct role.

Step 1: Configure a new IAM role in Starburst Galaxy

Now it's time to begin creating a new IAM role. You're going to start creating this in Starburst Galaxy, then finish creating it in the AWS console.

Using the left-hand navigation menu, select Admin>>Cloud settings.
Select th AWS tab.
Click the Configure IAM role button.

Step 2: Locate Starburst AWS account ID and External ID

Starburst Galaxy simplifies the process of obtaining your Starburst AWS account ID and External ID.

Both IDs will be used in the next step. For now, you're just going to locate them and confirm that they are present.

Locate your Starburst AWS account ID.
Locate your External ID.
Continue to the next step.

Step 1: Create a new IAM role

Now it's time to create a new IAM role. To do this, you're going to move to the AWS console to continue the setup process.

Open a new window and navigate to the AWS console.
Sign in to your AWS account.
In the search field, enter IAM.
Select IAM.
Using the left-hand navigation menu, select Roles.
Click the Create role button.

Step 2: Configure the role

Next, it's time to begin configuring the new role. This will require you to switch between the AWS console and the Starburst Galaxy UI.

In the AWS console, set the Trusted entity type to AWS account.
In the An AWS account section, select Another AWS account.
In the Starburst Galaxy UI, copy the Starburst AWS Account ID.
In the AWS console, paste the Starburst AWS Account ID into the Account ID field.
In the AWS console, select Require external ID.
Note: This is considered best practice when a third party will assume the role.
In the Starburst Galaxy UI, copy the External ID.
In the AWS console, paste the ID into the External ID field.
Click the Next button.

Step 3: Add permissions to the role

Next, it's time to set the correct permissions for the new role. This will require you to locate the policy created by your cloud administrator for usage with Starburst Galaxy.

In the Permissions policies section, enter the name of the IAM policy used by Starburst Galaxy in the filter field.
Select the policy from the list.
Click the Next button.

Step 4: Complete the role configuration

Next, it's time to create a meaningful name for your role. You will complete this step in the AWS console.

Provide a meaningful Role name.
In the Description field, enter a description of the new role's scope.
Scroll down.
Click the Add Tag button.
Add tags according to your organization's guidelines.

Note: This will vary depending on guidance and practices at your organization.
Click the Create role button.

Step 5: Retrieve the IAM role ARN

Now it's time to retrieve the ARN for your IAM role from the AWS console. Later in this tutorial, you will use this in Starburst Galaxy.

In the filter box, enter the name of the role that you just created.
Select the role.
Copy the ARN for the role.

Step 6: Complete the cross account IAM role configuration in Starburst Galaxy

Now it's time to complete the configuration process in Starburst Galaxy.

Return to your Starburst Galaxy UI.
In the AWS IAM ARN field, enter ARN.
In the Cross account IAM role alias field, enter a meaningful name.
Click the Validate cross account IAM role button.
Confirm you see the following message:
Hooray! Your cross account role has been set up, you can now use it in catalogs.
Click the Close button.
Your configuration is now complete!

Tutorial complete

Congratulations! You have reached the end of this tutorial, and the end of this stage of your journey.

You're all set! Now you can use your cross account IAM role to configure access to data in your AWS catalogs.

Continuous learning

At Starburst, we believe in continuous learning. This tutorial provides the foundation for further training available on this platform, and you can return to it as many times as you like. Future tutorials will make use of the concepts used here.

Next steps

Starburst has lots of other tutorials to help you get up and running quickly. Each one breaks down an individual problem and guides you to a solution using a step-by-step approach to learning.

Tutorials available

Visit the Tutorials section to view the full list of tutorials and keep moving forward on your journey!