Combine Amazon MWAA with Microsoft Entra ID utilizing SAML authentication

Amazon Managed Workflows for Apache Airflow (Amazon MWAA) offers a totally managed answer for orchestrating and automating complicated workflows within the cloud. Amazon MWAA affords two community entry modes for accessing the Apache Airflow net UI in your environments: private and non-private. Clients typically deploy Amazon MWAA in non-public mode and wish to use present login authentication mechanisms and single sign-on (SSO) options to have seamless integration with the company Lively Listing (AD). Additionally, the end-users don’t have to log in to the AWS Administration Console to entry the Airflow UI.

On this submit, we illustrate the way to configure an Amazon MWAA setting deployed in non-public community entry mode with buyer managed VPC endpoints and authenticate customers utilizing SAML federated identification utilizing Microsoft Entra ID and Utility Load Balancer (ALB). Customers can seamlessly log in to the Airflow UI with their company credentials and entry the DAGs. This answer may be modified for Amazon MWAA public community entry mode as properly.

Resolution overview

The architectural parts concerned in authenticating the Amazon MWAA setting utilizing SAML SSO are depicted within the following diagram. The infrastructure parts embrace two public subnets and three non-public subnets. The general public subnets are required for the internet-facing ALB. Two non-public subnets are used to arrange the Amazon MWAA setting, and the third non-public subnet is used to host the AWS Lambda authorizer operate. This subnet may have a NAT gateway hooked up to it, as a result of the operate must confirm the signer to verify the JWT header has the anticipated LoadBalancer ARN.

The workflow consists of the next steps:

1. For SAML configuration, Microsoft Entra ID serves because the identification supplier (IdP).
2. Amazon Cognito serves because the service supplier (SP).
3. ALB has built-in assist for Amazon Cognito and authenticates requests.
4. Publish-authentication, ALB forwards the requests to the Lambda authorizer operate. The Lambda operate decodes the person’s JWT token and validates whether or not the person’s AD group is mapped to the related AWS Identification and Entry Administration (IAM) function.
5. If legitimate, the operate creates an internet login token and redirects to the Amazon MWAA setting for profitable login.

The next are the high-level steps to deploy the answer:

1. Create an Amazon Easy Storage Service (Amazon S3) bucket for artifacts.
2. Create an SSL certificates and add it to AWS Certificates Supervisor (ACM).
3. Deploy the Amazon MWAA infrastructure stack utilizing AWS CloudFormation.
4. Configure Microsoft Entra ID companies and combine the Amazon Cognito person pool.
5. Deploy the ALB CloudFormation stack.
6. Log in to Amazon MWAA utilizing Microsoft Entra ID person credentials.

Conditions

Earlier than you get began, ensure you have the next stipulations:

• An AWS account
• Acceptable IAM permissions to deploy AWS CloudFormation stack sources
• A Microsoft Azure account is required for creating the Microsoft Entra ID app (IdP config) and Microsoft Entra ID P2.
• A public certificates for the ALB within the AWS Area the place the infrastructure is being deployed and a customized area title related to the certificates.

Create an S3 bucket

On this step, we create an S3 bucket to retailer your Airflow DAGs, customized plugins in a plugins.zip file, and Python dependencies in a necessities.txt file. This bucket is utilized by the Amazon MWAA setting to fetch DAGs and dependency recordsdata.

1. On the Amazon S3 console, select the Area the place you wish to create a bucket.
2. Within the navigation pane, select Buckets.
3. Select Create bucket.
4. For Bucket kind, choose Basic objective.
5. For Bucket title, enter a reputation in your bucket (for this submit, mwaa-sso-blog-<your-aws-account-number>).
6. Select Create bucket. 

   

7. Navigate to the bucket and select Create folder.
8. For Folder title, enter a reputation (for this submit, we title the folder dags).
9. Select Create folder.

   

Import certificates into ACM

ACM is built-in with Elastic Load Balancing (ALB). On this step,  you’ll be able to request a public certificates utilizing ACM or import a certificates into ACM. To import group certificates linked to a customized DNS into ACM, you have to present the certificates and its non-public key. To import a certificates signed by a non-AWS Certificates Authority (CA), you have to additionally embrace the non-public and public keys of the certificates.

1. On the ACM console, select Import certificates within the navigation pane.
2. For Certificates physique, enter the contents of the cert.pem file.
3. For Certificates non-public key, enter the contents of the privatekey.pem file.
4. Select Subsequent.

   

5. Select Assessment and import.
6. Assessment the metadata about your certificates and select Import.

After the import is profitable, the standing of the imported certificates will present as Issued.

Create the Azure AD service, customers, teams, and enterprise software

For the SSO integration with Azure, an enterprise software is required, which acts because the IdP for the SAML stream. We add related customers and teams to the appliance and configure the SP (Amazon Cognito) particulars.

Airflow comes with 5 default roles: Public, Admin, Op, Person, Viewer. On this submit, we concentrate on three: Admin , Person and Viewer. We create three roles and three corresponding customers and assign memberships appropriately.

1. Log in to the Azure portal.
2. Navigate to Enterprise purposes and select New software.

   

3. Enter a reputation in your software (for instance, mwaa-environment) and select Create.

   

   
   Now you can view the small print of your software.
   
   Now you create two teams.

4. Within the search bar, seek for Microsoft Entra ID.
   
   
5. On the Add menu, select Group.
   
   
6. For Group kind, select a sort (for this submit, Safety).
7. Enter a bunch title (for instance, airflow-admins) and outline.
8. Select Create.
   
   
   
9. Repeat these steps to create two extra teams, named airflow-users and airflow-viewers.
10. Be aware the item IDs for every group (these are required in a later step).
    
    
    Subsequent, you create customers.
11. On the Overview web page, on the Add menu, select Person and Create new person.
    
12. Enter a reputation in your person (for instance, mwaa-user), show title, and password.
13. Select Assessment + create.
    
    
    
14. Repeat these steps to create a person referred to as mwaa-admin.
15. In your airflow-users group particulars web page, select Members within the navigation pane.
16. Select Add members.
17. Seek for and choose the customers you created and select Choose.
    
    
    

18. Repeat these steps so as to add the customers to every group.

19. Navigate to your software and select Assign customers and teams.
    
    

20. Select Add person/group.
    
    

21. Seek for and choose the teams you created, then select Choose.
    
    

 

Deploy the Amazon MWAA setting stack

For this answer, we offer two CloudFormation templates that arrange the companies illustrated within the structure. Deploying the CloudFormation stacks in your account incurs AWS utilization prices.

The primary CloudFormation stack creates the next sources:

• A VPC with two public subnets and three non-public subnets and related route tables, NAT gateway, web gateway, and safety group
• VPC endpoints required for the Amazon MWAA setting
• An Amazon Cognito person pool and person pool area
• Utility Load Balancer

Deploy the stack by finishing the next steps:

1. Select Launch Stack to launch the CloudFormation stack.
   

2. For Stack title, enter a reputation (for instance, sso-blog-mwaa-infra-stack).

3.  Enter the next parameters:

   1. For MWAAEnvironmentName, enter the setting title.

   2. For MwaaS3Bucket, enter the S3 artifacts bucket you created.

   3. For VpcCIDR, enter the specify IP vary (CIDR notation) for this VPC.

   4. For PrivateSubnet1CIDR, enter the IP vary (CIDR notation) for the non-public subnet within the first Availability Zone.

   5.  For PrivateSubnet2CIDR, enter the IP vary (CIDR notation) for the non-public subnet within the second Availability Zone.

   6. For PrivateSubnet3CIDR, enter the IP vary (CIDR notation) for the non-public subnet within the third Availability Zone.

   7. For PublicSubnet1CIDR, enter the IP vary (CIDR notation) for the general public subnet within the first Availability Zone.

   8. For PublicSubnet2CIDR, enter the IP vary (CIDR notation) for the general public subnet within the second Availability Zone.

4. Select Subsequent
   
   
5. Assessment the template and select Create stack.

After the stack is deployed efficiently, you’ll be able to view the sources on the stack’s Outputs tab on the AWS CloudFormation console. Be aware the ALB URL, Amazon Cognito person pool ID, and area.

 

Combine the Amazon MWAA software with the Azure enterprise software

Subsequent, you configure the SAML configuration within the enterprise software by including the SP particulars and redirect URLs (on this case, the Amazon Cognito particulars and ALB URL).

1. Within the Azure portal, navigate to your setting.
2. Select Arrange single signal on.
   
3. For Identifier, enter urn:amazon:cognito:sp:<your cognito user_id>.
4. For Reply URL, enter https://<Your person pool area>/saml2/idpresponse.
5. For Signal on URL, enter https://<Your software load balancer DNS>.
   
6. Within the Attributes & Claims part, select Add a bunch declare.
7. Choose Safety teams.
8. For Supply attribute, select Group ID.
9. Select Save.
   
10. Be aware the values for App Federation Metadata Url and Login URL.

    

Deploy the ALB stack

When the SAML configuration is full on the Azure finish, the IdP particulars must be configured in Amazon Cognito. When customers entry the ALB URL, they are going to be authenticated in opposition to the company identification utilizing SAML by means of Amazon Cognito. After they’re authenticated, they’re redirected to the Lambda operate for authorization in opposition to the group they belong to. The person’s group is then validated in opposition to matching IAM function. If it’s legitimate, the Lambda operate provides the online login token to the URL, and the person will achieve entry to the Amazon MWAA setting.

This CloudFormation stack creates the next sources:

• Two goal teams: the Lambda goal group and Amazon MWAA goal group
• Listener guidelines for the ALB to redirect URL requests to the related goal teams
• A person pool consumer and SAML supplier (Azure) particulars to the Amazon Cognito person pool
• IAM roles for Admin, Person, and Viewer personas required for Airflow
• The Lambda authorizer operate to validate the JWT token and map Azure teams to IAM roles for acceptable Airflow UI entry

Deploy the stack by finishing the next steps:

1. Select Launch Stack to launch the CloudFormation stack:
   

2. For Stack title, enter a reputation (for instance, sso-blog-mwaa-alb-stack).

3. Enter the next parameters:

   1. For MWAAEnvironmentName, enter your setting title.

   2. For ALBCertificateArn, enter the certificates ARN required for ALB. 

   3. For AzureAdminGroupID, enter the group title for the Azure Admin persona.

   4. For AzureUserGroupID, enter the group title for the Azure Person persona.

   5. For AzureViewerGroupID, enter the group title for the Azure Viewer persona.

   6. For EntraIDLoginURL, enter the Azure IdP URI.

   7. For AppFederationMetadataURL, enter the URL of the metadata file for the SAML supplier. 

4. Select Subsequent.
   
   
5. Assessment the template and select Create stack.

Check the answer

Now that the SAML configuration and related AWS companies are created, it’s time to entry the Amazon MWAA setting.

1. Open your net browser and enter the ALB DNS title.
   The SP initiates the sign-in request course of and the browser redirects you to the Microsoft login web page for credentials.
2. Enter the Admin person credentials.
   
   The SAML request sign-in course of completes and the SAML response is redirected to the Amazon Cognito person pool hooked up to the ALB.
   
   The listener guidelines will validate the question URL and go the requests to the Lambda authorizer to validate the JWT and assign the suitable group (Azure) to function (AWS) mapping.
   
   
   
3. Repeat the steps to log in with Person and Viewer credentials and observe the variations in entry.

Clear up

Once you’re completed experimenting with this answer, it’s important to scrub up your sources to keep away from incurring AWS prices.

1. On the AWS CloudFormation console, delete the stacks you created.
2. Take away the SSM parameters and personal webserver and database VPC endpoints (created by the Lambda occasions operate):

   aws ssm delete-parameters --names "MyFirstParameter" "MySecondParameter"
   aws ec2 delete-vpc-endpoints --vpc-endpoint-ids "Endpoint1" "Endpoint2"

3. Delete the customers, teams, and enterprise software within the Azure setting.