Identity & Access Management (IAM)

Introduction

This document outlines an approach for managing and controlling access to Azure Resources, AKS, and Kubernetes resources in a unified way.

AKS provides three different methods for authentication and authorization:

• Local accounts with Kubernetes RBAC
• Azure AD Authentication with Kubernetes RBAC
• Azure AD authentication with Azure RBAC

By default, we deploy the first option, where Azure AD is only used to retrieve the Kubernetes Cluster admin certificate. Authentication and authorization are handled by Kubernetes.

However, this approach requires managing identities and permissions in multiple systems. Users and admins receive a local X509 client certificate specific to the cluster they’re accessing. This certificate has a common name of “masterclient,” belongs to the “system:masters” group, and is bound to a cluster-admin role that grants full AKS cluster admin access. To add additional users, admins must manually create a certificate for each user signed by the cluster’s certificate authority.

The second option combines Azure AD Authentication with Kubernetes RBAC, allowing customers to use their Azure AD identity to authenticate to Kubernetes. Customers need to define Kubernetes roles and map them to Azure AD identities and security groups.

The third option is similar to the second, but now you can utilize Azure RBAC roles to authenticate en authorize identities accessing the Kubernetes cluster. With this option, customers can easily manage identity permissions within Kubernetes using Azure AD. Setting up an admin group is not necessary, as Azure RBAC can be used to grant identities admin access to a cluster.

Prerequisites

• Ensure you have the Azure CLI version 2.24.0 or later
• Ensure you have installed kubectl v1.18.3+

Limitations

• Requires Managed Azure AD integration.
• AKS-managed Azure AD integration can’t be disabled.
• Use kubectl v1.18.3+.
• If you have CRDs and are making custom role definitions, the only way to cover CRDs today is to provide Microsoft.ContainerService/managedClusters/*/read. AKS is working on providing more granular permissions for CRDs. For the remaining objects you can use the specific API Groups, for example: Microsoft.ContainerService/apps/deployments/read.
• New role assignments can take up to 5min to propagate and be updated by the authorization server.
• Requires the Azure AD tenant configured for authentication to be the same as the tenant for the subscription that holds the AKS cluster.
• if your organization uses conditional access policies to allow only compliant devices to authenticate with the company identities, make sure your administrator adds the kubernetes Azure AAD Server to the app exclusion list.

Getting Started

The remainder of this document outlines the steps required to integrate Azure AD with your AKS cluster and enable Azure RBAC. Following these instructions will enable you to manage Kubernetes identity access and permissions using Azure AD. You’ll also be able to create custom Kubernetes RBAC roles and map them to Azure AD users and security groups.

This guide begins by deploying the AD integration using option 2: “Azure AD Authentication with Kubernetes RBAC.” This option requires customers to create some necessary security groups. In the background, a Kubernetes admin role is created, which can be easily customized further.

Next, the guide enables Azure RBAC integration, which permits customers to use Azure RBAC roles alongside Kubernetes RBAC roles, offering greater flexibility in managing identity access and permissions within Kubernetes.

Enable Managed Azure AD integration

Prepare environment

Before enabling Azure AD authorization, you need to prepare your AD environment.

• Create user groups (AKS Administrators, AKS Readers):
  You will need two Azure AD groups for your cluster. To create these groups, navigate to your Azure Directory in the portal and follow these steps:

  1. Create Administrator Group: The first group will be registered as an admin group in the cluster.
  2. Create Reader Group: The second group must be configured with reader permissions on the AKS cluster, using the following Azure AD roles:
     • Reader Get access to subscription and see resources.
     • Azure Kubernetes Service Cluster User Role Get permissions to retrieve kubeconfig
     • Azure Kubernetes Service RBAC Reader Get reader permissions within kubernetes

• Create an App registration with AKS reader permissions:
  After deploying the Azure AD integration, CMaaS engineers will no longer be able to access the cluster. To provide customers with support, it is necessary for us to have at least reader permissions within the cluster. To grant access, create a new app registration and add it as a member of the AKS Reader group you just created.

Enable Azure AD integration

The next step is to connect the AKS cluster to Azure AD:

• Enable Azure AD Authentication for the AKS CLuster: (This action cannot be reversed)
  Enabling this step will allow you to use your company identities to access the AKS cluster.
  Be sure to set your admin group to maintain access to your cluster.

  az aks update -g MyResourceGroup -n MyManagedCluster --enable-aad --aad-admin-group-object-ids <id-1> [--aad-tenant-id <id>]
  

  Info

  Behind the scenes, to make the group an admin for the cluster, AKS will create a cluster role binding that assigns it a cluster role providing full admin access to the cluster.

• Enable Azure RBAC Authorization:
  Enabling this step will allow you to use Azure AD to authorize company identities within your AKS cluster.

  az aks update --resource-group <resource-group> --name <cluster-name> --enable-azure-rbac
  

Disable Local Accounts

When deploying the AKS cluster, local accounts are enabled by default, even when enabling RBAC or Azure AD integration, --admin access still exists as a non-auditable backdoor option. To address this, you can use the disable-local-accounts parameter to disable local accounts.

az aks update -g <resource-group> -n <cluster-name> --disable-local-accounts

Use Azure AD RBAC to access the AKS cluster

• To ensure your Kubernetes management tools are up-to-date, run the following command:

  az aks install-cli
  

• Retrieve kubernetes kubeconfig:

  az aks get-credentials --resource-group myResourceGroup --name myAKSCluster --overwrite-existing
  

• After running a command, such as kubectl get pods, you should receive an authentication prompt with Azure, similar to the following:

  kubectl get pods
  To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code B24ZD6FP8 to authenticate.
  

If your company sign-in flow is successful, and you have appropriate permissions, you should see the result of the command.

Troubleshoot sign-in

• If you encounter difficulties during authentication or notice that permissions are not being propagated, try cleaning your .kube folder and try to authenticate again.
  You can find the .kube folder located at C:\Users<USERNAME>\.kube.
  However, be aware that by cleaning your .kube folder, you will lose access to any other Kubernetes instances that you have defined within that folder.

Use Azure RBAC for Kubernetes authorization

Roles can be assigned for the entire AKS cluster through either the Access Control (IAM) section of the cluster resource in the Azure portal or by using Azure CLI commands, as demonstrated below:

# Get your AKS Resource ID
AKS_ID=$(az aks show -g MyResourceGroup -n MyManagedCluster --query id -o tsv)

az role assignment create --role "Azure Kubernetes Service RBAC Admin" --assignee <AAD-ENTITY-ID> --scope $AKS_ID

where <AAD-ENTITY-ID> could be a username (for example, user@contoso.com ) or even the ClientID of a service principal.

You can also create role assignments scoped to a specific namespace within the cluster:

az role assignment create --role "Azure Kubernetes Service RBAC Reader" --assignee <AAD-ENTITY-ID> --scope $AKS_ID/namespaces/<namespace-name>

Today, role assignments scoped to namespaces need to be configured via Azure CLI.

Built-in Roles

AKS provides the following four built-in roles:

Create custom role definitions

Optionally, you may choose to create your own role definition, which you can then assign as shown above.

For example, here is a sample role definition that allows a user to only read deployments and nothing else. You can check the full list of possible actions here.

To create this custom role definition, copy the JSON below into a file called deploy-view.json.

{
    "Name": "AKS Deployment Reader",
    "Description": "Lets you view all deployments in cluster/namespace.",
    "Actions": [],
    "NotActions": [],
    "DataActions": [
        "Microsoft.ContainerService/managedClusters/apps/deployments/read"
    ],
    "NotDataActions": [],
    "assignableScopes": [
        "/subscriptions/<YOUR SUBSCRIPTION ID>"
    ]
}

Replace <YOUR SUBSCRIPTION ID> by the ID from your subscription, which you can get by running:

az account show --query id -o tsv

Now we can create the role definition by running the below command from the folder where you saved deploy-view.json:

az role definition create --role-definition @deploy-view.json

Now that you have your role definition, you can assign it to a user or other identity by running:

az role assignment create --role "AKS Deployment Reader" --assignee <AAD-ENTITY-ID> --scope $AKS_ID

Use Kubernetes RBAC with Azure AD integration

In Kubernetes, Roles define the permissions to grant, and RoleBindings apply them to desired users or groups. These assignments can be applied to a given namespace, or across the entire cluster. If you need more control over access management, you can leverage Kubernetes RBAC and combine it with Azure RBAC. By creating custom Kubernetes roles and applying them to the desired Azure AD security groups, you can define fine-grained permissions, providing more flexibility and security.

Create Role

To create a Role for the dev namespace that grants full permissions to the namespace, you can use the following example. In production environments, you may want to specify more granular permissions for different users or groups.

Create a file named role-dev-namespace.yaml and paste the following YAML manifest:

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: dev-user-full-access
  namespace: dev
rules:
- apiGroups: ["", "extensions", "apps"]
  resources: ["*"]
  verbs: ["*"]
- apiGroups: ["batch"]
  resources:
  - jobs
  - cronjobs
  verbs: ["*"]

Create the Role using the kubectl apply command and specify the filename of your YAML manifest.

kubectl apply -f role-dev-namespace.yaml

Create RoleBinding

To utilize the previously created Role, we must create a RoleBinding.
Create a file named rolebinding-dev-namespace.yaml and paste the YAML manifest shown below.
On the last line, replace groupObjectId with the group object ID of the desired Azure AD security group. You can use an existing or create a new security group.

If you want to create the RoleBinding for a single user, specify kind: User and replace groupObjectId with the user principal name (UPN) in the sample below.

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: dev-user-access
  namespace: dev
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: dev-user-full-access
subjects:
- kind: Group
  namespace: dev
  name: <groupObjectId>

Create the RoleBinding using the kubectl apply command and specify the filename of your YAML manifest:

kubectl apply -f rolebinding-dev-namespace.yaml