Managing how individuals and applications can interact with your Kubernetes cluster is critical to keeping your application secure. Catalyst Cloud Kubernetes Service integrates with the Identity and Access Management service to make it easy to manage access using role-based access control (RBAC).
In this section we will discuss how to grant Kubernetes RBAC roles to your Catalyst Cloud account, and use them to interact with your cluster.
Role-Based Access Control (RBAC) is one of the key pillars of Kubernetes security. RBAC determines who or what can interact with your clusters. It provides an easy way to assign or revoke permissions to groups of users. Permissions can be easily audited which is extremely valuable for ensuring compliance with strict regulatory and security requirements.
Note
Kubernetes RBAC roles only grant you permissions within Kubernetes. These roles are separate from the standard user roles.
To create a Kubernetes cluster in your project, your user must be granted the Project Member role.
To interact with Kubernetes after your cluster is created, even Project Admins need to grant themselves the appropriate Kubernetes RBAC roles.
Catalyst Cloud provides the following roles which can be used to control access to Kubernetes clusters.
Role |
CLI Name |
Permissions |
|---|---|---|
Kubernetes Admin |
|
Privileged users with maximum rights. Full admin access is granted for Catalyst Cloud Kubernetes Service cluster CRUD operations and all Kubernetes namespaces in a cluster. |
Kubernetes Developer |
|
Privileged users with restricted rights.
Kubernetes CRUD operation access is granted to any namespace
other than the managed ( |
Kubernetes Viewer |
|
Non-privileged users able to perform read actions in a Catalyst Cloud Kubernetes Service cluster. Has read-only access to all namespaces, excluding the managed namespaces. |
When inviting a new user to your Catalyst Cloud project, you can assign them the required roles for interacting with Kubernetes, so that when they create their account they will automatically have the required access.
A good practice is to limit the roles you grant users to the minimum that are needed for the tasks they will perform.
Note
Only Project Admins or Project Moderators can invite users to a Catalyst Cloud project.
Run the following command to invite a new user to your project,
replacing jondoe+k8s_admin@example.com with the email address
you wish to send the invite to.
Note
If you would like this user to be able to create and delete Kubernetes
clusters, as well as manage other Catalyst Cloud resources,
add the _member_ (Project Member) role
as an additional parameter to the command.
openstack project user invite jondoe+k8s_admin@example.com k8s_admin
An invite email will be sent to the provided email address. Once the user accepts, their Catalyst Cloud account will be automatically granted the requested roles in your project.
Navigate to the Management -> Access Control -> Project Users page, and press the + Invite User button in the top right of the page.
The Invite User window will open. Type in the email address of the user to invite, and grant the new user the k8s_admin role by ticking it in the role list.
Note
If you would like this user to be able to create and delete Kubernetes clusters, as well as manage other Catalyst Cloud resources, also assign them the Project Member role.
Once you are done, press Invite to send the invite. Once the user accepts, their Catalyst Cloud account will be automatically granted the requested roles in your project.
Run the following command to invite a new user to your project,
replacing jondoe+k8s_dev@example.com with the email address
you wish to send the invite to.
openstack project user invite jondoe+k8s_dev@example.com k8s_developer
An invite email will be sent to the provided email address. Once the user accepts, their Catalyst Cloud account will be automatically granted the requested roles in your project.
Navigate to the Management -> Access Control -> Project Users page, and press the + Invite User button in the top right of the page.
The Invite User window will open. Type in the email address of the user to invite, and grant the new user the k8s_developer role by ticking it in the role list.
Once you are done, press Invite to send the invite. Once the user accepts, their Catalyst Cloud account will be automatically granted the requested roles in your project.
Run the following command to invite a new user to your project,
replacing jondoe+k8s_viewer@example.com with the email address
you wish to send the invite to.
openstack project user invite jondoe+k8s_dev@example.com k8s_viewer
An invite email will be sent to the provided email address. Once the user accepts, their Catalyst Cloud account will be automatically granted the requested roles in your project.
Navigate to the Management -> Access Control -> Project Users page, and press the + Invite User button in the top right of the page.
The Invite User window will open. Type in the email address of the user to invite, and grant the new user the k8s_viewer role by ticking it in the role list.
Once you are done, press Invite to send the invite. Once the user accepts, their Catalyst Cloud account will be automatically granted the requested roles in your project.
Project Admins and Project Moderators can grant Kubernetes RBAC roles to existing users.
Run the following command to grant the Kubernetes Admin role
to a Catalyst Cloud user (replacing jondoe+k8s_admin@example.com
with the email address of the user).
openstack project user role add jondoe+k8s_admin@example.com k8s_admin
Navigate to the Management -> Access Control -> Project Users page.
Find the user you wish to grant the role to in the list, and press the Update User button to open the Update User window.
Grant the Kubernetes Admin role by ticking k8s_admin in the role list, and press Update to save your changes.
Run the following command to grant the Kubernetes Developer role
to a Catalyst Cloud user (replacing jondoe+k8s_dev@example.com
with the email address of the user).
openstack project user role add jondoe+k8s_dev@example.com k8s_developer
Navigate to the Management -> Access Control -> Project Users page.
Find the user you wish to grant the role to in the list, and press the Update User button to open the Update User window.
Grant the Kubernetes Developer role by ticking k8s_developer in the role list, and press Update to save your changes.
Run the following command to grant the Kubernetes Viewer role
to a Catalyst Cloud user (replacing jondoe+k8s_viewer@example.com
with the email address of the user).
openstack project user role add jondoe+k8s_viewer@example.com k8s_viewer
Navigate to the Management -> Access Control -> Project Users page.
Find the user you wish to grant the role to in the list, and press the Update User button to open the Update User window.
Grant the Kubernetes Viewer role by ticking k8s_viewer in the role list, and press Update to save your changes.
A kubeconfig file is required for kubectl to interact with a Kubernetes cluster.
On Catalyst Cloud there are two types of kubeconfig file, both of which can be downloaded via the API:
RBAC kubeconfig: Provides access to the Kubernetes cluster based on user roles assigned in Catalyst Cloud.
This is recommended for most interaction with a managed Kubernetes cluster.
Admin kubeconfig: Allows unrestricted access to a Kubernetes cluster using an admin token provided with the kubeconfig file.
Not recommended for general access. For most use cases, RBAC kubeconfig files should be used to interact with the cluster. Refer to Retrieving the admin kubeconfig for more information.
Note
Retrieving the kubeconfig file from Catalyst Cloud requires Kubernetes Admin permissions.
Kubernetes Developers and Kubernetes Viewers cannot retrieve their own kubeconfig, but an RBAC kubeconfig retrieved by a Kubernetes Admin can be shared with these users to give them access to the cluster.
Currently, the only way to retreve the kubeconfig file is to use the Catalyst Cloud CLI.
First, open a terminal window and source your OpenRC file to authenticate with Catalyst Cloud.
To retrieve the RBAC kubeconfig file, run the openstack coe cluster config command
with the --use-keystone option. The exact usage is as follows:
openstack coe cluster config <CLUSTER-NAME> --use-keystone
This will save the kubeconfig file in the current directory under the name config.
To configure kubectl to use the kubeconfig, run the following command
to set the KUBECONFIG environment variable to the current directory:
export KUBECONFIG=$(pwd)/config
First, open a terminal window and source your OpenRC file to authenticate with Catalyst Cloud.
To retrieve the RBAC kubeconfig file, run the openstack coe cluster config command
with the --use-keystone option. The exact usage is as follows:
openstack coe cluster config <CLUSTER-NAME> --use-keystone
This will save the kubeconfig file in the current directory under the name config.
To configure kubectl to use the kubeconfig, run the following command
to set the KUBECONFIG environment variable to the current directory:
$Env:KUBECONFIG = $pwd\config
First, open a terminal window and source your OpenRC file to authenticate with Catalyst Cloud.
To retrieve the RBAC kubeconfig file, run the openstack coe cluster config command
with the --use-keystone option. The exact usage is as follows:
openstack coe cluster config <CLUSTER-NAME> --use-keystone
This will save the kubeconfig file in the current directory under the name config.
To configure kubectl to use the kubeconfig, run the following command
to set the KUBECONFIG environment variable to the current directory:
set KUBECONFIG=%cd%\config
This config file can now be shared with other team members that need
access to this cluster. This file provides the following levels of access:
Users with the Kubernetes Admin, Kubernetes Developer or Kubernetes Viewer role will have access specified by that role (see Role-Based Access Control).
Users without RBAC roles will not be able to access the cluster.
Warning
The admin kubeconfig should not be used directly in most use cases.
Unlike the RBAC kubeconfig, which requires authenticating with Catalyst Cloud before providing cluster access, the admin kubeconfig allows for unrestricted access without authentication. Using this kubeconfig file makes it impossible to audit who or what is making changes to a cluster.
A Kubernetes Admin can perform any task that the admin kubeconfig allows, so this file is not necessary for everyday usage. Only use the admin kubeconfig file when required.
Currently, the only way to retreve the kubeconfig file is to use the Catalyst Cloud CLI.
First, open a terminal window and source your OpenRC file to authenticate with Catalyst Cloud.
To retrieve the RBAC kubeconfig file, run the openstack coe cluster config command,
without the --use-keystone option. The exact usage is as follows:
openstack coe cluster config <CLUSTER-NAME>
This will save the kubeconfig file in the current directory under the name config.
To configure kubectl to use the kubeconfig, run the following command
to set the KUBECONFIG environment variable to the current directory:
export KUBECONFIG=$(pwd)/config
First, open a terminal window and source your OpenRC file to authenticate with Catalyst Cloud.
To retrieve the RBAC kubeconfig file, run the openstack coe cluster config command,
without the --use-keystone option. The exact usage is as follows:
openstack coe cluster config <CLUSTER-NAME>
This will save the kubeconfig file in the current directory under the name config.
To configure kubectl to use the kubeconfig, run the following command
to set the KUBECONFIG environment variable to the current directory:
$Env:KUBECONFIG = $pwd\config
First, open a terminal window and source your OpenRC file to authenticate with Catalyst Cloud.
To retrieve the RBAC kubeconfig file, run the openstack coe cluster config command,
without the --use-keystone option. The exact usage is as follows:
openstack coe cluster config <CLUSTER-NAME>
This will save the kubeconfig file in the current directory under the name config.
To configure kubectl to use the kubeconfig, run the following command
to set the KUBECONFIG environment variable to the current directory:
set KUBECONFIG=%cd%\config
Note
The admin kubeconfig file contains an admin token that provides unrestricted access to your cluster.
Make sure it is stored on your system with the appropriate permissions, e.g. removing read/write access to anyone except your system user.
The openstack coe cluster config command can optionally take a few
additional arguments:
--dir <path> - Specify an alternative directory to save the kubeconfig file to.
By default, the kubeconfig file will be saved to the current directory.
--force - Recreate the config file in the specified directory if it already exists.
By default, the command will fail if the config already exists in the specified directory.
Once the kubeconfig file has been downloaded, kubectl can be configured to use the
config file using one of the following methods:
Passing it using the --kubeconfig command line argument.
Setting the KUBECONFIG environment variable to the full path of the kubeconfig file.
Saving the kubeconfig file to $HOME/.kube/config.
kubectl looks for the kubeconfig file in the above order,
with $HOME/.kube/config being checked last.
With the kubeconfig file in place, it is now possible to interact with a Kubernetes cluster. Depending on the level of access granted, the user will be able to query and/or create resources on the cluster.
You can verify what your user can and cannot do using the kubectl auth can-i command.
A user with the Kubernetes Admin (k8s_admin) role has the ability to perform any actions on the cluster.
Check if your user can create pods in the default namespace:
$ kubectl auth can-i create pods -n default
yes
Check if your user can create pods in all namespaces:
$ kubectl auth can-i create pods -A
yes
Check if your user can delete secrets in all namespaces:
$ kubectl auth can-i delete secrets -A
yes
The Kubernetes Developer (k8s_developer) role allows a user to perform
most everyday operations within all non-privileged namespaces.
$ kubectl auth can-i create pods -n default
yes
However, a Kubernetes Developer is not allowed to perform any actions
in the admin (kube-system) namespace.
$ kubectl auth can-i create pods -n kube-system
no
Kubernetes Developers are also not allowed to perform certain cluster-level operations.
$ kubectl auth can-i patch clusterrolebinding
no
$ kubectl auth can-i create clusterrolebinding
no
The privileged roles deserve special attention when deploying Kubernetes clusters.
The RBAC permissions that grant the ability to launch a pod in the cluster is a powerful privilege. Use of a more restrictive Admission Controller may be appropriate to meet specific customer security needs.
Any user with the Kubernetes Admin or Kubernetes Developer role must be a trusted individual.
There are a number of other tools available for managing RBAC to Kubernetes clusters.
Integrated policy solutions:
Third-party policy solutions:
This is a comprehensive list of the exact RBAC permissions that each role gives a user:
+----------------------+-----------------------------------------------------+
| Role | Permissions |
+======================+=====================================================+
| k8s_admin | resourcemanager.projects.* |
+----------------------+-----------------------------------------------------+
| k8s_developer | container.apiServices.* |
| | container.bindings.* |
| | container.certificateSigningRequests.create |
| | container.certificateSigningRequests.delete |
| | container.certificateSigningRequests.get |
| | container.certificateSigningRequests.list |
| | container.certificateSigningRequests.update |
| | container.certificateSigningRequests.watch |
| | container.clusterRoleBindings.get |
| | container.clusterRoleBindings.list |
| | container.clusterRoleBindings.watch |
| | container.clusterRoles.get |
| | container.clusterRoles.list |
| | container.clusterRoles.watch |
| | container.componentStatuses.* |
| | container.configMaps.* |
| | container.controllerRevisions.get |
| | container.controllerRevisions.list |
| | container.controllerRevisions.watch |
| | container.cronJobs.* |
| | container.customResourceDefinitions.* |
| | container.deployments.* |
| | container.endpoints.* |
| | container.events.* |
| | container.horizontalPodAutoscalers.* |
| | container.ingresses.* |
| | container.initializerConfigurations.* |
| | container.jobs.* |
| | container.limitRanges.* |
| | container.localSubjectAccessReviews.* |
| | container.namespaces.* |
| | container.networkPolicies.* |
| | container.nodes.get |
| | container.nodes.list |
| | container.nodes.watch |
| | container.persistentVolumeClaims.* |
| | container.persistentVolumes.* |
| | container.podDisruptionBudgets.* |
| | container.podPresets.* |
| | container.podSecurityPolicies.get |
| | container.podSecurityPolicies.list |
| | container.podSecurityPolicies.watch |
| | container.podTemplates.* |
| | container.pods.* |
| | container.replicaSets.* |
| | container.replicationControllers.* |
| | container.resourceQuotas.* |
| | container.roleBindings.get |
| | container.roleBindings.list |
| | container.roleBindings.watch |
| | container.roles.get |
| | container.roles.list |
| | container.roles.watch |
| | container.secrets.* |
| | container.selfSubjectAccessReviews.* |
| | container.serviceAccounts.* |
| | container.services.* |
| | container.statefulSets.* |
| | container.storageClasses.* |
| | container.subjectAccessReviews.* |
| | container.tokenReviews.* |
+----------------------+-----------------------------------------------------+
| k8s_viewer | container.apiServices.get |
| | container.apiServices.list |
| | container.apiServices.watch |
| | container.binding.get |
| | container.binding.list |
| | container.binding.watch |
| | container.clusterRoleBindings.get |
| | container.clusterRoleBindings.list |
| | container.clusterRoleBindings.watch |
| | container.clusterRoles.get |
| | container.clusterRoles.list |
| | container.clusterRoles.watch |
| | container.componentStatuses.get |
| | container.componentStatuses.list |
| | container.componentStatuses.watch |
| | container.configMaps.get |
| | container.configMaps.list |
| | container.configMaps.watch |
| | container.controllerRevisions.get |
| | container.controllerRevisions.list |
| | container.controllerRevisions.watch |
| | container.cronJobs.get |
| | container.cronJobs.list |
| | container.cronJobs.watch |
| | container.customResourceDefinitions.get |
| | container.customResourceDefinitions.list |
| | container.customResourceDefinitions.watch |
| | container.deployments.get |
| | container.deployments.list |
| | container.deployments.watch |
| | container.endpoints.get |
| | container.endpoints.list |
| | container.endpoints.watch |
| | container.events.get |
| | container.events.list |
| | container.events.watch |
| | container.horizontalPodAutoscalers.get |
| | container.horizontalPodAutoscalers.list |
| | container.horizontalPodAutoscalers.watch |
| | container.ingresses.get |
| | container.ingresses.list |
| | container.ingresses.watch |
| | container.initializerConfigurations.get |
| | container.initializerConfigurations.list |
| | container.initializerConfigurations.watch |
| | container.jobs.get |
| | container.jobs.list |
| | container.jobs.watch |
| | container.limitRanges.get |
| | container.limitRanges.list |
| | container.limitRanges.watch |
| | container.localSubjectAccessReviews.get |
| | container.localSubjectAccessReviews.list |
| | container.localSubjectAccessReviews.watch |
| | container.namespaces.get |
| | container.namespaces.list |
| | container.namespaces.watch |
| | container.networkPolicies.get |
| | container.networkPolicies.list |
| | container.networkPolicies.watch |
| | container.nodes.get |
| | container.nodes.list |
| | container.nodes.watch |
| | container.persistentVolumeClaims.get |
| | container.persistentVolumeClaims.list |
| | container.persistentVolumeClaims.watch |
| | container.persistentVolumes.get |
| | container.persistentVolumes.list |
| | container.persistentVolumes.watch |
| | container.podDisruptionBudgets.get |
| | container.podDisruptionBudgets.list |
| | container.podDisruptionBudgets.watch |
| | container.podPresets.get |
| | container.podPresets.list |
| | container.podPresets.watch |
| | container.podTemplates.get |
| | container.podTemplates.list |
| | container.podTemplates.watch |
| | container.podSecurityPolicies.get |
| | container.podSecurityPolicies.list |
| | container.podSecurityPolicies.watch |
| | container.pods.get |
| | container.pods.list |
| | container.pods.watch |
| | container.replicaSets.get |
| | container.replicaSets.list |
| | container.replicaSets.watch |
| | container.replicationControllers.get |
| | container.replicationControllers.list |
| | container.replicationControllers.watch |
| | container.resourceQuotas.get |
| | container.resourceQuotas.list |
| | container.resourceQuotas.watch |
| | container.roleBindings.get |
| | container.roleBindings.list |
| | container.roleBindings.watch |
| | container.roles.get |
| | container.roles.list |
| | container.roles.watch |
| | container.secrets.get |
| | container.secrets.list |
| | container.secrets.watch |
| | container.selfSubjectAccessReviews.get |
| | container.selfSubjectAccessReviews.list |
| | container.selfSubjectAccessReviews.watch |
| | container.serviceAccounts.get |
| | container.serviceAccounts.list |
| | container.serviceAccounts.watch |
| | container.services.get |
| | container.services.list |
| | container.services.watch |
| | container.statefulSets.get |
| | container.statefulSets.list |
| | container.statefulSets.watch |
| | container.storageClasses.get |
| | container.storageClasses.list |
| | container.storageClasses.watch |
| | container.subjectAccessReviews.get |
| | container.subjectAccessReviews.list |
| | container.subjectAccessReviews.watch |
+----------------------+-----------------------------------------------------+