Kubernetes Cheatsheet

Kubectl

List all applications and services

kubectl get all

Resource: https://coreos.com/tectonic/docs/latest/tutorials/sandbox/deleting-deployment.html#:~:text=Go to Workloads > Deployments.,Go to Routing > Services.

List all pods

kubectl get pods

List all containers in all pods

kubectl get pods -o='custom-columns=NameSpace:.metadata.namespace,NAME:.metadata.name,CONTAINERS:.spec.containers[*].name'

List all containers in a pod

kubectl get pods $POD_NAME -o='custom-columns=NameSpace:.metadata.namespace,NAME:.metadata.name,CONTAINERS:.spec.containers[*].name'

Resource: https://serverfault.com/questions/873490/how-to-list-all-containers-in-kubernetes

List pods in a namespace

kubectl get pods -n <namespace>

Get all pods running in all namespaces

kubectl get pods --all-namespaces

Get all container images

kubectl get pods --all-namespaces -o=jsonpath="{..image}
Get all container images filtered by pod label
kubectl get pods --all-namespaces -o=jsonpath="{..image}" -l app=<name>

For example, for a label with the app name 'nginx' created with this tutorial:

kubectl get pods --all-namespaces -o=jsonpath="{..image}" -l app=nginx

Resource: https://kubernetes.io/docs/tasks/access-application-cluster/list-all-running-container-images/#list-container-images-filtering-by-pod-label

Get Pod IP Address

kubectl get pods -l app=<app name> -o yaml |grep podIP

Delete a pod

kubectl delete pod <pod name>

Resource: https://www.fairwinds.com/blog/how-to-create-view-and-destroy-a-pod-in-kubernetes

List all nodes

kubectl get nodes

Get more information about a node

This will have things like the pods that are running on a node.

kubectl describe nodes <node name>

Get information about all deployments

kubectl describe deployments

Get information about a deployment

kubectl describe deployment nginx-deployment

Get shell to first container in a pod

kubectl exec -it $pod_name -- bash

Get shell to specific container in a pod

kubectl exec -it $pod_name --container $container_name -- sh

Secrets

Get a list of secrets

kubectl get secrets

Describe a secret

kubectl describe secret $SECRET_NAME

View a secret

kubectl get secret $SECRET_NAME -o json | jq .

You can also opt to output as yaml as well:

kubectl get secret $SECRET_NAME -o yaml

If a secret is a json blob with multiple key/value pairs (like a kubernetes.io/service-account-token type for example), you can get the associated value from one of the keys like so:

kubectl get secret $SECRET_NAME -o jsonpath='{.data.keyname}' | base64 -d

Resource: https://kubernetes.io/docs/concepts/configuration/secret/

List all services

kubectl get services

Get names of services

kubectl get services --sort-by=.metadata.name

Resource: https://kubernetes.io/docs/reference/kubectl/cheatsheet/

Get more information about a pod

kubectl describe pod $POD_NAME

Alternatively, you can use:

kubectl describe pods/$POD_NAME

Resources:
https://stackoverflow.com/questions/34848422/how-to-debug-imagepullbackoff
https://github.com/cloudnativelabs/kube-router/issues/711

Delete an application

Find the deployment and service beforehand:

kubectl delete deployment.apps/<name> service/<name>

For example, for a service and deployment called 'app':

kubectl delete deployment.apps/app service/app

Resource: https://coreos.com/tectonic/docs/latest/tutorials/sandbox/deleting-deployment.html#:~:text=Go to Workloads > Deployments.,Go to Routing > Services.

Alternatively, you could just run this in the directory with all of the files:

kubectl delete -k ./

Resource: https://kubernetes.io/docs/tutorials/stateful-application/mysql-wordpress-persistent-volume/

Get external IP addresses of all nodes

kubectl get nodes -o jsonpath='{.items[*].status.addresses[?(@.type=="ExternalIP")].address}'

Resource: https://kubernetes.io/docs/reference/kubectl/cheatsheet/

List all namespaces

kubectl get namespace

Create new namespace

NAMESPACE_NAME=blablabla
kubectl create namespace $NAMESPACE_NAME

Resource: https://jhooq.com/helm-chart-wordpress-installation/

Delete namespace

NAMESPACE_NAME=blablabla
kubectl delete namespace $NAMESPACE_NAME

Show Persistent Volumes

kubctl get pvc

Delete Persistent Volume

PVC_NAME=somepvc
kubectl delete pvc $PVC_NAME

Troubleshooting

This will sort the output for you as well based on when something was created:

kubectl get events --all-namespaces  --sort-by='.metadata.creationTimestamp'

Alternatively, you can just run:

kubectl get events

Resources:
https://serverfault.com/questions/728727/kubernetes-stuck-on-containercreating
https://stackoverflow.com/questions/36377784/pod-in-kubernetes-always-in-pending-state

Copy file from pod to system

kubectl cp $POD_NAME:/run/secrets/kubernetes.io/serviceaccount .

Resource: https://stackoverflow.com/questions/52407277/how-to-copy-files-from-kubernetes-pods-to-local-system/52408599

Copy file from system to pod

kubectl cp file $POD_NAME:/tmp

Resource: https://medium.com/@nnilesh7756/copy-directories-and-files-to-and-from-kubernetes-container-pod-19612fa74660

Get all clusters

kubectl config view -o jsonpath='{"Cluster name\tServer\n"}{range .clusters[*]}{.name}{"\t"}{.cluster.server}{"\n"}{end}'

Resource: https://kubernetes.io/docs/tasks/administer-cluster/access-cluster-api/

List all service accounts

kubectl get serviceaccounts

List all clusterroles

kubectl get clusterrole

Get yaml file from running pod

kubectl get po $POD_NAME -o yaml | tee file.yaml

Resource: https://stackoverflow.com/questions/43941772/get-yaml-for-deployed-kubernetes-services

Check for insecure kubelet API access

From a pod

curl -k https://localhost:10250/pods

Remotely

curl -k https://<target system running kub>:10250/pods

Resource: https://sysdig.com/blog/kubernetes-security-kubelet-etcd/

Kubernetes config file location

env |grep KUBECONFIG

View config

kubectl config view

Use config file

# run before setting the env var
kubectl config view
export KUBECONFIG=/path/to/config/file
# run view config again to see the changes
kubectl config view

You can also run it like this if you don't want to export the environment variable for whatever weird reason you have:

KUBECONFIG=/path/to/config/file kubectl config view

Resource: https://ahmet.im/blog/mastering-kubeconfig/

Access K8s API from inside a Pod

Setup

Set these variables to start:

K8S=https://$KUBERNETES_SERVICE_HOST:$KUBERNETES_SERVICE_PORT
TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)
CACERT=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt
NAMESPACE=$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace)

Health Check

curl -s -H "Authorization: Bearer $TOKEN" --cacert $CACERT $K8S/healthz

Show pods

curl -s -H "Authorization: Bearer $TOKEN" \
--cacert $CACERT $K8S/api/v1/namespaces/$NAMESPACE/pods/

Resource:
https://medium.com/@pczarkowski/the-kubernetes-api-call-is-coming-from-inside-the-cluster-f1a115bd2066

Get network policies

kubectl get networkpolicy

Get information about a particular policy

kubectl describe networkpolicy $NETWORK_POLICY_NAME

Resource: https://www.stackrox.com/post/2020/02/azure-kubernetes-aks-security-best-practices-part-2-of-4/

Default network access policies

If network policies are defined, the Kubernetes default policy is allow. Subsequently, you can talk to networked assets from within a container. To test this, exec into a container:

kubectl exec -it <container name> sh

Once inside, you can try things like querying the AWS metadata service (or another networked resource):

wget -O - -q http://169.254.169.254/latest/meta-data/identity-credentials/ec2/security-credentials/ec2-instance/

If this works, you can ascertain that an attacker that gains access to a running container gains unfettered network access.

Resolution

Specify network access configurations that minimize ingress and egress access for each pod.

Resources:
https://medium.com/@reuvenharrison/an-introduction-to-kubernetes-network-policies-for-security-people-ba92dd4c809d

Port forwarding

Service to localhost

This will forward the service running on 8443 to localhost:1234

kubectl port-forward service/<service name> 1234:8443 -n <namespace>

Pod to localhost

This will forward the service running on $pod-port to localhost:$pod-port

kubectl port-forward $POD_NAME $pod-port

Pod to the network

This will expose the service running on $pod-port in $POD_NAME to the system running the kubectl command on $localhost-port to other systems on the network:

kubectl port-forward --address <hostname or IP of system> $POD_NAME $localhost-port:$pod-port

Resource: https://stackoverflow.com/questions/51468491/how-kubectl-port-forward-works

Set cluster

kubectl config use-context <cluster> 

Get list of everything a service account can do

kubectl auth can-i --list

Use local image

Add this line to your pod yaml file:

imagePullPolicy: Never

Resource: https://stackoverflow.com/questions/55392014/kubectl-get-pods-shows-errimagepull

Run docker in docker

This is a very bad thing to do from a security standpoint, but when you need it, this is how you do it:

apiVersion: v1
kind: Pod
metadata:
  name: test-pd
spec:
  containers:
  - image: dev
    imagePullPolicy: Never
    name: dev
    volumeMounts:
    - name: docker-sock-volume
      mountPath: "/var/run/docker.sock"
  volumes:
  - name: docker-sock-volume
    hostPath:
      # location on host
      path: /var/run/docker.sock

Resources:
https://stackoverflow.com/questions/56462126/how-to-add-v-var-run-docker-sock-var-run-docker-sock-when-running-container
https://devops.stackexchange.com/questions/2506/docker-in-kubernetes-deployment

Helm

Debug template

helm template g <template name> -f ./values.yaml --debug

For example, if we deployed the bitnami/ghost template, we could use this command to debug it:

helm template g bitnami/ghost -f ./values.yaml --debug

Resource: https://www.reddit.com/r/kubernetes/comments/j3j3ox/ghost_helm_no_pod_showing_up/

Search for repository by name

This example will search for ghost repositories:

helm search hub ghost --max-col-width=0 

Open the desired link to get the helm repo add command to use.

Show local repo list

helm repo list

Remove repo from local repo list

This example will remove the bitnami repo:

helm repo rm bitnami

Add repo to local repo list

This particular example will add the bitnami repo:

helm repo add bitnami https://charts.bitnami.com/bitnami

Search for versions of a chart in local repo

This will return a list of ghost charts:

helm search repo ghost --versions

Get latest version of a chart in local repo

helm search repo ghost --versions | sed -n 2p | awk '{print $2}'

List Releases in all namespaces

helm ls --all-namespaces
# Shorthand:
helm ls -a

Resources:
https://github.com/helm/helm/issues/7527
https://jhooq.com/helm-chart-wordpress-installation/ - tutorial with some great helm examples
https://helm.sh/docs/intro/using_helm/ - helm docs
https://github.com/helm/hub/issues/208 - explanation of how to add a repo found with the search command

Install Plugin

This example will install the helm diff plugin:

helm plugin install https://github.com/databus23/helm-diff

Resources:
https://jhooq.com/helm-chart-plugin/ - tutorial
https://github.com/helm/helm/issues/3156 - initial suggestion

Uninstall Plugin

This example will uninstall the helm diff plugin:

helm plugin uninstall diff