Last Updated: 2024-02-26

Background

Azure service principals are service identities used by applications and automation tools to access Azure resources securely. They have a number of different characteristics.

Role-based permissions

Service principals do not control individual user logins. Instead, they are used by applications, services, or automation tools to access Azure resources on behalf of users by providing the specific permissions needed to access resources in Azure. The roles assigned to service principals define which actions those applications or services can perform using Azure resources.

Automation

Service principals are primarily used to automate tasks. This typically includes several key automation use cases:

Deployment of applications using Azure Resource Manager templates.
Accessing resources from a pipeline in Azure DevOps.
Granting permissions to an application to access specific Azure resources.

Secure connection

Service principals ensure secure interaction between applications and Azure resources without manual intervention. Starburst Galaxy supports using Azure service principal as a means of securely connecting to your Azure Data Lake Storage (ADLS).

In this tutorial, you will learn how to use Starburst Galaxy, ADLS, and Azure service principals together. .

Scope of tutorial

In this tutorial, you will learn how to configure an Azure service principal. You will work in both the Azure portal and Starburst Galaxy UI.

Learning objectives

Once you've completed this tutorial, you will be able to:

Configure an Azure service principal.
Use the Azure service principal to securely connect Starburst Galaxy to your Azure data lake storage.

Prerequisites

You need a Starburst Galaxy account to complete this tutorial. Please see Starburst Galaxy: Getting started for instructions on setting up a free account.
This tutorial comes with a bring your own storage requirement. Before proceeding with this lesson, you must already have an Azure data lake storage account set up.

If this is not the case, please set this up first then return to this tutorial.

About Starburst tutorials

Starburst tutorials are designed to get you up and running quickly by providing bite-sized, hands-on educational resources. Each tutorial explores a single feature or topic through a series of guided, step-by-step instructions.

As you navigate through the tutorial you should follow along using your own Starburst Galaxy account. This will help consolidate the learning process by mixing theory and practice.

Background

Azure allows you to to register your application or service, integrating it with Microsoft Entra ID. This is the first step towards using Azure Service Principals.

After registration, your application will be able to sign in users, request access to Microsoft Entra ID-protected resources like APIs, and perform other authentication-related tasks. This will be used to allow Azure Service Principals.

Step 1: Sign in to Azure portal

You'll begin in the Azure portal. If you use multiple Azure accounts, make sure to log in to the account that has access to the ADLS that you want to use for this tutorial.

Step 2: Create a new application registration

Now it's time to create a new Application registration using the Azure portal.

Using the search bar at the top of the screen, enter app registrations.
Select App registrations from the results list.
Select + New registration.

Step 3: Register an application

To register a new application, you must provide a name and supported account type.

In the Register an application field, enter a meaningful Name.

Note: We suggest including Starburst Galaxy for clarity.
Click the Register button.

Step 4: Record application registration details

Now it's time to capture some details about your application registration. Later in this tutorial, you'll need these to set up your Azure Service Principal.

At this point, you should be on the Overview page for your new app registration. In the Essentials section, copy the following details, and save them for future use.

Record the Display name value.
Record the Application (client) ID value.
Record the Directory (tenant) ID value.

Background

An Azure client secret is a credential used by an application to authenticate its identity when requesting access to resources from Microsoft Entra ID. Similar to a user's username and password, it serves as a form of authentication for the application.

In this section, you'll create a client secret for authentication between Azure and Starburst Galaxy. Later in the tutorial, this will be used to help set up your Azure Service Principal.

Step 1: Create a new client secret

You're going to start by creating a new client secret in the Azure portal.

Using the left-hand navigation menu, select Certificates & secrets.
Click the + New client secret button.

Step 2: Configure secret

It's time to begin configuring the new secret. To do this, you're going to add a description and expiration date.

In the Description field, enter a meaningful description of the new client secret.
Set the expiration to 24 months.
Click the Add button.

Step 3: Copy and store secret

Azure will have created a new secret. Next, you need to save it and use it to configure the Azure service principal authentication in Starburst Galaxy.

In the Client secrets tab, confirm that the new secret is listed.
Copy the secret Value field.
Save the value in a safe place.

Background

It's time to switch over to the Starburst Galaxy UI. In this next section you will configure a new Azure service principal using the information you just obtained from your application registration.

Step 1: Sign into Starburst Galaxy

Sign into Starburst Galaxy in the usual way. If you have not already set up an account, you can do that here.
Input your Email and Password.
Click the Sign in to Starburst Galaxy button.

Step 2: Set your role

Starburst Galaxy separates users by role. Your current role is listed in the top right-hand corner of the screen.

Setting up Azure Service Principal authentication will require access to a role with appropriate privileges. Today, you'll be using the accountadmin role.

Check your role, to ensure that it is set to accountadmin.
If it is set to anything else, use the drop-down menu to select the correct role.

Step 3: Select Azure cloud settings

Starburst Galaxy supports all three major cloud providers: AWS, Azure, and Google Cloud. The Starburst Galaxy web UI lets you configure access to each cloud provider using the Cloud settings menu.

Using the left-hand navigation bar, expand the Cloud settings menu.
Select Azure.
Click the Configure service principal button.

Step 4: Configure Azure service principal

Now it's time to use the information that you copied from the Azure portal to configure the Azure service principal. To do this, you're going to use the Starburst Galaxy web UI.

In the Azure tenant id field, paste the Directory (tenant) id from the Azure app registration.
In the OAuth service principal client id field, paste the Application (client) ID from the Azure app registration.
In the OAuth service principal secret field, paste the Value of the client secret from the Azure app registration.
In the Azure service principal alias field, paste the Display name from the Azure app registration.
Click the Validate Azure service principal button.
Click the Close button.

Background

Azure data lake storage (ADLS) requires sufficient permissions to allow Starburst Galaxy to access your data sources in Azure. In particular, you will need to grant both the Contributor and Storage Blob Data Owner roles to the service principal you're using for authentication.

This section will walk you through the process of granting these roles in the Azure portal. You'll begin by navigating to the Storage accounts section.

Step 1: Access Azure storage accounts service

You're going to begin in the Azure portal. To grant permissions to your ADLS account, you need to locate it from a list of storage accounts.

In your browser, return to the Azure portal.
In the Azure search field at the top of the screen, enter storage accounts.
Select Storage accounts from the list of services.

Step 2: Locate storage account

Now it's time to locate the ADLS account that you'd like to connect to Starburst Galaxy.

In the filter field, enter your ADLS account (AKA blob storage account) name.
Select yourADLS account from the list below.

Step 3: Add first role assignment

It's time to add the first of two new role assignments to your ADLS account. This one will allow your service principal to have Contributor permissions to your storage account.

Using the left-hand menu, select Access Control (IAM).
Click the + Add button.
Select Add role assignment from the drop-down menu.
Select the Privileged administrator roles tab.
Select Contributor from the list below.

Step 4: Select role members

Now you need to add your Azure service principal as a member of this role. This will complete the connection between the two.

Select the Members tab at the top of the screen.
Select User, group, or service principal.
Click the + Select members button.
In the filter field, enter your Azure service principal name.

Note: This name will be identical to the App registration name that you created earlier. For example, erin-rosas-starburst-galaxy.
Select your service principal name from the list.
Click the Select button.

Step 5: Confirm role assignment

Now that you've added the new role, it's time to confirm that it has been assigned properly.

Confirm your service principal name is now listed under Members.
Click the Review + assign button.
Click the Review + assign button a second time.

Step 6: Add second role assignment

Next, it's time to add the second role assignment. This one will allow your service principal to have Storage Blob Data Owner permissions to access your storage account.

Click the + Add button.
Using the drop-down menu, select Add role assignment.
Using the search bar, enter Storage Blob Data Owner.
Select Storage Blob Data Owner from the list.

Step 7: Select role members

Now it's time to add your Azure service principal as a member of this role. This will complete the connection between the two.

Select the Members tab at the top of the screen.
Select User, group, or service principal.
Click the + Select members button.
In the search field, enter your Azure service principal name.

Note: This name will be identical to the Application registration name that you created earlier. For example, erin-rosas-starburst-galaxy.
Select your service principal name.
Click the Select button.

Step 8: Confirm role assignment

Now that you've added the second role, it's time to confirm that it has been assigned properly. This step is similar to the check you did on the first role.

Confirm your service principal name is now listed under Members.
Click the Review + assign button.
Click the Review + assign button a second time.

Tutorial complete

Congratulations! You have reached the end of this tutorial, and the end of this stage of your journey.

You're all set! Now you can use your Azure service principal to configure access to data in your ADLS catalogs.

Continuous learning

At Starburst, we believe in continuous learning. This tutorial provides the foundation for further training available on this platform, and you can return to it as many times as you like. Future tutorials will make use of the concepts used here.

Next steps

Starburst has lots of other tutorials to help you get up and running quickly. Each one breaks down an individual problem and guides you to a solution using a step-by-step approach to learning.

Tutorials available

Visit the Tutorials section to view the full list of tutorials and keep moving forward on your journey!