yogeshkk/k8spurger

Hunt Unused Resources In Kubernetes

k8spurger's Issues

Failed to run

python K8sPurger.py

This script is created to find unused resource in Kubernetes

Not able to read Kubernetes cluster check Kubeconfig
Traceback (most recent call last):
File "/Users/Shariq.Mustquim/DevOps/poc/k8s-misc/K8sPurger/K8sPurger.py", line 25, in main
v1beta1Api = client.ExtensionsV1beta1Api()
AttributeError: module 'kubernetes.client' has no attribute 'ExtensionsV1beta1Api'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/Shariq.Mustquim/DevOps/poc/k8s-misc/K8sPurger/K8sPurger.py", line 332, in
main("standalone")
File "/Users/Shariq.Mustquim/DevOps/poc/k8s-misc/K8sPurger/K8sPurger.py", line 30, in main
raise RuntimeError(e)
RuntimeError: module 'kubernetes.client' has no attribute 'ExtensionsV1beta1Api'

Handaling false positive resources

Need to think about false positive resources.

There are many resources which will be marked as unused but not necessary unused. It might be ok for ad-hoc script run but it might be problem when running in K8S as services and specially when alerts are configured. One might get alert which are not genuine.

In order to tackle above a user can provide a list of resources which getting marked as false positive and before sending output we can remove such resources.

Unused Services Bug

All of my services are returned as unused, as well as ingresses. The issue is this:

When the GetUsedServices(v1) function is run, it updates the local variable UsedEP, not the global variable of UsedEP. The global variable remains an empty list and so when the difference is calculated between EP and UsedEP, the entire EP list is returned.

A simple fix to this is to set UsedEP when GetUsedServices(v1) is called here:
https://github.com/yogeshkk/K8sPurger/blob/main/K8sPurger.py#L44

As in, update it to this:
UsedEP = GetUsedServices(v1)

I can create a PR if you want, but because the change is so small, you might want to just fix this yourself.

Great project btw

Add option for json output

Cool tool!!

This will enable running K8Spurger as a Cronjob (or Jenkins a job) periodically and do some additional processing of the output. E.g. Send notifications to namespace owners about unused resources in their namespace.

Removing ability to delete resources

Hello All,

I have created this script to find clean old unused config map. Then I started adding feature to it and now it can find and clean up secret, PVC and recently added service.

I am thinking about removing deleting ability as this is not much used feature also this help us to improve script further. Also in case of newly added service it might be marked as unused because of wrong selector or any miss-configuration. I will just comment the deletion part after 7 days so if anyone want to use it they can but if any new resources added won't have delete enabled (I am thinking about adding ingress next).

I know this is not popular that I receive reply but as this is in opensource it is my obligation to you all to share my view on upcoming development.

Finally, I never thought this will receive this much appreciation. Thanks for that. Let me know your thoughts on above.

Thanks,
Yogesh(Yogi)

Replace with list comprehension

K8sPurger/K8sPurger.py

Lines 77 to 82 in f91e1d7

	def Diffrance(listA, listB):
	listC = []
	for i in listA:
	if i not in listB:
	listC.append(i)
	return listC

Could be:

def Diffrance(listA, listB):
    return [i for i in listA if i not in listB]

External Name services incorrectly listed

The script lists external names in the unused services.

showing default service accounts

We use Openshift clusters. Every project in Openshift has 4 default service accounts. More importantly, default service account is being used at run time. The script is showing they are used or not referred to. This showing large output although they are used by PODS at run time. It is better to filter these default service accounts

Example

for service accounts
| builder |velero |
| default |velero |
| deployer |velero

for secretes
| builder-dockercfg-f875f |velero |
| default-dockercfg-f6bzm |velero |
| deployer-dockercfg-qjwc7 |velero |
|
Even in a small cluster, the output is too verbose and showing lot of objects

CrashLoopBackOff

Awesome stuff, but I have the below issue. So the pod is in crash loop.

kubectl apply -f deploy/manifest.yaml
deployment.apps/k8spurger created
service/k8spurger-svc created
serviceaccount/k8spurger-sa created
Warning: rbac.authorization.k8s.io/v1beta1 ClusterRole is deprecated in v1.17+, unavailable in v1.22+; use rbac.authorization.k8s.io/v1 ClusterRole
clusterrole.rbac.authorization.k8s.io/k8spurger-cluster-role created
Warning: rbac.authorization.k8s.io/v1beta1 ClusterRoleBinding is deprecated in v 1.17+, unavailable in v1.22+; use rbac.authorization.k8s.io/v1 ClusterRoleBindin g
clusterrolebinding.rbac.authorization.k8s.io/k8spurger-rb created

The error:

Getting unused secret it may take couple of minute..
Traceback (most recent call last):
File "K8sPurger.py", line 324, in
main("svc")
File "K8sPurger.py", line 32, in main
GetUsedResources(v1)
File "K8sPurger.py", line 125, in GetUsedResources
UsedConfigMap.append([volume.config_map_ref.name, i.metadata.namespace])

Add container environment variables use case

https://kubernetes.io/docs/concepts/configuration/secret/#use-cases

Need to take into account below scenario and have it append to UsedSecret/UsedConfigMap function.

    spec:
      containers:
      - env:
        envFrom:
        - configMapRef:
            name: configmap-test
        - secretRef:
            name: secrets-test

below is the block that will get you all the names of those secrets

                if item.env_from is not None:
                    for envfrom in item.env_from:
                        if envfrom.secret_ref is not None:
                            UsedSecret.append(
                                    [envfrom.secret_ref.name, i.metadata.namespace])

If you know a short way

@yogeshkk We're using a combination of ArgoCD and kube-janitor to achieve this.

The ArgoCD application resource gets a timestamp through CI/CD:

---
apiVersion: argoproj.io/v1alpha1
metadata:
  name: 'foobar-{{ getenv "GIT_BRANCH" | strings.Slug }}'
  annotations:
    janitor/expires: '{{ ((time.Now).Add (time.Hour 24)).UTC.Format "2006-01-02" }}T12:00:00Z'
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:

Janitor will then clean up resources after 24h.

Need to get just one type resource

This script is very useful. I been looking for quite some to list unused PVC, not being used in any pods. I don't find any solution so far. I found this script is super helpful in what am looking for. However, It would be nice if script takes 1 or more arguments to print just a specific type of resource.
For example, I want to just print unused PVC, not interested in other resources. This helps

1. Script doesn't have to fetch every resource which is not interested. fewer API calls to cluster
2. Easy to get information rather than searching in big list of all the resources

Srinivas Kotaru

ModuleNotFoundError: No module named 'kubernetes'

ModuleNotFoundError: No module named 'kubernetes'