Introduction
Today I compare 2 security tools that enable to assess the security of your Kubernetes clusters. This is a critical step to achieve before releasing a cluster for production usage.
Disclaimer: I’m by no means a security expert so this post will be light on that aspect. I will mainly focus on usability of the tools.
The two competitors are Kubestriker and Kubescape.
I will use both of them against my very own Scaleway Kapsule cluster.
Kubestriker
Kubestriker is a platform agnostic open source tool designed to tackle Kubernetes cluster security issues due to misconfigurations (from the Github README page).
Kubestriker is available in 2 flavours: command line and web application. I will only test the command line version.
Installation
Kubestriker CLI is available as a Docker container, can be built from source or installed using python pip. I will use the Docker container version as it allows to keep my system clean.
Remark: only version
1.0.0is available on Docker hub whenever the latest version on Github is1.2.0.
docker pull cloudsecguy/kubestriker:v1.0.0Configuration
Before running Kubestriker CLI in Docker, you need some preparation.
Remark: I tried to first test Kubestriker with
k3dbut it did not provide any interesting output and only performed some basic tests. Not surek3dis supported though.
Create a service account kubestriker
apiVersion: v1
kind: ServiceAccount
metadata:
name: kubestriker
namespace: defaultCreate a read only cluster role
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: cluster-read-all
rules:
- apiGroups:
- ""
- apps
- autoscaling
- batch
- extensions
- policy
- rbac.authorization.k8s.io
resources:
- componentstatuses
- configmaps
- daemonsets
- deployments
- events
- endpoints
- horizontalpodautoscalers
- ingress
- jobs
- limitranges
- namespaces
- nodes
- pods
- persistentvolumes
- persistentvolumeclaims
- resourcequotas
- replicasets
- replicationcontrollers
- serviceaccounts
- services
verbs:
- get
- watch
- list
- nonResourceURLs: ["*"]
verbs:
- get
- watch
- listBind the cluster role to the service account
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: cluster-read-all
subjects:
- kind: ServiceAccount
name: kubestriker
namespace: default
roleRef:
kind: ClusterRole
name: cluster-read-all
apiGroup: rbac.authorization.k8s.ioNow get the token of the service account kubestriker
TOKEN=$(kubectl get secrets -o jsonpath="{.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='kubestriker')].data.token}"|base64 --decode)
echo $TOKENCopy the content of $TOKEN. We’re ready to run the tool.
Executing the tests
docker run -it --rm -v $HOME/.kube/config:/root/.kube/config -v "$(pwd)":/kubestriker --name kubestriker cloudsecguy/kubestriker:v1.0.0
# Inside the container
python -m kubestrikerKubestriker can connect to Kubernetes cluster using your kubectl config file, by URL/ip address of your Kubernetes API server or using CIDR range. I used kubectl config file.
Whenever Kubestriker asks to choose between authenticated or anonymous checks, select Authenticated and paste the content of the $TOKEN variable.
At the end of its process, Kubestriker creates a log file named {url of your k8s api server}.txt.
Tests content
Kubestriker tests the following points :
- checks various services and /or open ports like
- enumerates misconfigured insecure services (if any found)
- scans IAM misconfigurations
- detects variety pf misconfigured containers
- detects broad range of misconfigured pod security policies
- check various misconfigured network policies
- assess excessive privileges of subjects in the cluster
Sample report
Performing Service Discovery on host {redacted}.scw.cloud..........
[+] Scanning for KubeServer Secure Service................................
[+] Scanning for KubeServer Insecure Service..............................
[+] Scanning for Kubelet ReadWrite Service................................
[+] Scanning for Kubelet Readonly Service.................................
[+] Scanning for kubecontroller Service................................
[+] Scanning for ETCD Client..............................................
[+] Scanning for ETCD Server..............................................
[+] Scanning for Kube proxy Healthcheck...................................
[+] Scanning for Kube Scheduler Service...................................
[+] Scanning for Kube proxy ..............................................
[+] Scanning for known Open Ports.........................................
[+] Scanning for Kubernetes Dashboard.....................................
KubeServer Secure Identified Services ........................................
[+] KubeServer Secure Endpoint Identified
Gearing up for Api Server Secure Scan................................
Scanning for IAM Misconfigurations ........................................
Validating namespace rolesdefault
Validating namespace roleskube-node-lease
Validating namespace roleskube-public
Validating namespace roleskube-system
Validating namespace rolesdefault
Scanning for Misonfigured containers ......................................
[+] Scanning for Privileged Containers .................................
[+] Scanning for livenessProbe .........................................
[+] Scanning for readinessProbe ........................................
[+] Scanning for CPU Limit .............................................
[+] Scanning for Memory Limit ..........................................
[+] Scanning for Priorityclassname .....................................
[+] Scanning for ServiceAccount Mount ..................................
[+] Scanning for Secrets Mounted .......................................
[+] Scanning for docker Socket Mount .................................Each category details the security issues found in a detailed section of the report.
╔══════════════════════════════════════════════════════╗
║ ######## CPU Limit not set below containers ######## ║
╚══════════════════════════════════════════════════════╝
kube-proxy
{... the list goes on}My feelings
The tool looked promising and it’s a work in progress. It enables to quick scan a cluster against various misconfigurations.My main concern is that we don’t know which misconfigurations are checked (technically you can but you need to dig into the code for that) and the tool does not offer contextual help on those that failed and what to do to correct the reported issue. I guess the corrective actions are well known to people versed into Kubernetes security but for a security noob like me, a bit more guidance will be much appreciated. These are certainly future enhancements that could make it more attractive to non security aware people.
Kubescape
Kubescape is also an open source tool to test if a Kubernetes cluster is deployed securely. It offers multiple frameworks gathering different security tests like NSA-CISA,MITRE ATT&CK.
It can scans not only Kubernetes clusters but also YAML files and HELM charts to detect misconfigurations and software vulnerabilities.
Installation
Installing the tool is very simple. Copy and paste the following command.
curl -s https://raw.githubusercontent.com/armosec/kubescape/master/install.sh | /bin/bashCaution: NEVER run such installation scripts without having first a look in the code.
Running the tests
Kubescape uses your current kubectl context to run the tests. Run the following command to test your cluster against the NSA-CISA framework.
kubescape scan framework nsaThe tests are executed immediately and complete quickly.
Tests content
The content of the tests depends on the framework used. Each framework is composed of controls. Controls define the check executed. All controls are documented on KubeScape documentation site that not only describes it but also give info about how to remediate a failed control.
Sample report
The result of each control is output to stdout.
[control: Privileged container] passed 👍
Description: Potential attackers may gain access to privileged containers and inherit access to the host resources. Therefore, it is not recommended to deploy privileged containers unless it is absolutely necessary. This control identifies all the privileged Pods.
Summary - Passed:22 Excluded:0 Failed:0 Total:22At the end of the framework tests, a summary table is displayed
NSA FRAMEWORK
+-----------------------------------------------------------------------+------------------+--------------------+---------------+-----------+
| CONTROL NAME | FAILED RESOURCES | EXCLUDED RESOURCES | ALL RESOURCES | % SUCCESS |
+-----------------------------------------------------------------------+------------------+--------------------+---------------+-----------+
| Allowed hostPath | 5 | 0 | 22 | 77% |
| Applications credentials in configuration files | 0 | 0 | 50 | 100% |
| CVE-2021-25741 - Using symlink for arbitrary host file system access. | 0 | 0 | 23 | 100% |
| CVE-2021-25742-nginx-ingress-snippet-annotation-vulnerability | 0 | 0 | 40 | 100% |
| Cluster-admin binding | 3 | 0 | 173 | 98% |
| Container hostPort | 0 | 0 | 22 | 100% |
etc ...My feelings
I really like the experience of KubeScape. Running the tool is easy, the checks are well documented and you can also find actions to remediate. The output of the KubeScape scan can be formatted in json or junit and then be used in a CI/CD pipeline to be integrated immediately.
Last but not least, KubeScape also provides a web interface as SAAS product with its portal. You can submit the results of your scans to it and then explore them in a nice web interface, compare them over time, customize the controls and define exceptions. I did not explore that part of the service but will sure do in the near future.
Conclusion
I must admit that KubeScape looks very promising. The usage is easy, tests are well documented, results are clear and contains remediation information. It is also more active than Kubestriker if we compare commits on both project’s repository.
Kubestriker does work ok but in its current iteration, it looks more geared towards security enthousiasts that know what to check, how to remediate and want an automated way of running some security tests. The documentation is not always very clear (but it’s open source so we can contribute), there are some typos here and there in the outputs and the fact that the Docker container version is not aligned with the latest tag in Github is a bit puzzling to me.
My choice goes thus to KubeScape but hey, that’s my personal feeling.
I hope this post could help you to try those tools because securing Kubernetes clusters is a must and all tools that can help us with that task are more than welcome!