Kubernetes Pivoting to Clouds
Last updated
Last updated
Learn & practice AWS Hacking:HackTricks Training AWS Red Team Expert (ARTE) Learn & practice GCP Hacking: HackTricks Training GCP Red Team Expert (GRTE)
If you are running a k8s cluster inside GCP you will probably want that some application running inside the cluster has some access to GCP. There are 2 common ways of doing that:
A common way to give access to a kubernetes application to GCP is to:
Create a GCP Service Account
Bind on it the desired permissions
Download a json key of the created SA
Mount it as a secret inside the pod
Set the GOOGLE_APPLICATION_CREDENTIALS environment variable pointing to the path where the json is.
Therefore, as an attacker, if you compromise a container inside a pod, you should check for that env variable and json files with GCP credentials.
A way to give access to a GSA to a GKE cluser is by binding them in this way:
Create a Kubernetes service account in the same namespace as your GKE cluster using the following command:
Create a Kubernetes Secret that contains the credentials of the GCP service account you want to grant access to the GKE cluster. You can do this using the gcloud
command-line tool, as shown in the following example:
Bind the Kubernetes Secret to the Kubernetes service account using the following command:
In the second step it was set the credentials of the GSA as secret of the KSA. Then, if you can read that secret from inside the GKE cluster, you can escalate to that GCP service account.
With Workload Identity, we can configure a Kubernetes service account to act as a Google service account. Pods running with the Kubernetes service account will automatically authenticate as the Google service account when accessing Google Cloud APIs.
The first series of steps to enable this behaviour is to enable Workload Identity in GCP (steps) and create the GCP SA you want k8s to impersonate.
Enable Workload Identity on a new cluster
Create/Update a new nodepool (Autopilot clusters don't need this)
Create the GCP Service Account to impersonate from K8s with GCP permissions:
Connect to the cluster and create the service account to use
Bind the GSA with the KSA
Run a pod with the KSA and check the access to GSA:
Check the following command to authenticate in case needed:
As an attacker inside K8s you should search for SAs with the iam.gke.io/gcp-service-account
annotation as that indicates that the SA can access something in GCP. Another option would be to try to abuse each KSA in the cluster and check if it has access.
From GCP is always interesting to enumerate the bindings and know which access are you giving to SAs inside Kubernetes.
This is a script to easily iterate over the all the pods definitions looking for that annotation:
An (outdated) way to give IAM Roles to Pods is to use a Kiam or a Kube2IAM server. Basically you will need to run a daemonset in your cluster with a kind of privileged IAM role. This daemonset will be the one that will give access to IAM roles to the pods that need it.
First of all you need to configure which roles can be accessed inside the namespace, and you do that with an annotation inside the namespace object:
Once the namespace is configured with the IAM roles the Pods can have you can indicate the role you want on each pod definition with something like:
As an attacker, if you find these annotations in pods or namespaces or a kiam/kube2iam server running (in kube-system probably) you can impersonate every role that is already used by pods and more (if you have access to AWS account enumerate the roles).
The IAM role to indicate must be in the same AWS account as the kiam/kube2iam role and that role must be able to access it.
This is the recommended way by AWS.
First of all you need to create an OIDC provider for the cluster.
Then you create an IAM role with the permissions the SA will require.
Create a trust relationship between the IAM role and the SA name (or the namespaces giving access to the role to all the SAs of the namespace). The trust relationship will mainly check the OIDC provider name, the namespace name and the SA name.
Finally, create a SA with an annotation indicating the ARN of the role, and the pods running with that SA will have access to the token of the role. The token is written inside a file and the path is specified in AWS_WEB_IDENTITY_TOKEN_FILE
(default: /var/run/secrets/eks.amazonaws.com/serviceaccount/token
)
To get aws using the token from /var/run/secrets/eks.amazonaws.com/serviceaccount/token
run:
As an attacker, if you can enumerate a K8s cluster, check for service accounts with that annotation to escalate to AWS. To do so, just exec/create a pod using one of the IAM privileged service accounts and steal the token.
Moreover, if you are inside a pod, check for env variables like AWS_ROLE_ARN and AWS_WEB_IDENTITY_TOKEN.
Sometimes the Turst Policy of a role might be bad configured and instead of giving AssumeRole access to the expected service account, it gives it to all the service accounts. Therefore, if you are capable of write an annotation on a controlled service account, you can access the role.
Check the following page for more information:
This is a script to easily iterate over the all the pods and sas definitions looking for that annotation:
The previos section was about how to steal IAM Roles with pods, but note that a Node of the K8s cluster is going to be an instance inside the cloud. This means that the Node is highly probable going to have a new IAM role you can steal (note that usually all the nodes of a K8s cluster will have the same IAM role, so it might not be worth it to try to check on each node).
There is however an important requirement to access the metadata endpoint from the node, you need to be in the node (ssh session?) or at least have the same network:
Previously we have discussed how to attach IAM Roles to Pods or even how to escape to the Node to steal the IAM Role the instance has attached to it.
You can use the following script to steal your new hard worked IAM role credentials:
Learn & practice AWS Hacking:HackTricks Training AWS Red Team Expert (ARTE) Learn & practice GCP Hacking: HackTricks Training GCP Red Team Expert (GRTE)