yogeshkk / k8spurger Goto Github PK
View Code? Open in Web Editor NEWHunt Unused Resources In Kubernetes
License: Apache License 2.0
Hunt Unused Resources In Kubernetes
License: Apache License 2.0
python K8sPurger.py
This script is created to find unused resource in Kubernetes
Not able to read Kubernetes cluster check Kubeconfig
Traceback (most recent call last):
File "/Users/Shariq.Mustquim/DevOps/poc/k8s-misc/K8sPurger/K8sPurger.py", line 25, in main
v1beta1Api = client.ExtensionsV1beta1Api()
AttributeError: module 'kubernetes.client' has no attribute 'ExtensionsV1beta1Api'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/Shariq.Mustquim/DevOps/poc/k8s-misc/K8sPurger/K8sPurger.py", line 332, in
main("standalone")
File "/Users/Shariq.Mustquim/DevOps/poc/k8s-misc/K8sPurger/K8sPurger.py", line 30, in main
raise RuntimeError(e)
RuntimeError: module 'kubernetes.client' has no attribute 'ExtensionsV1beta1Api'
Need to think about false positive resources.
There are many resources which will be marked as unused but not necessary unused. It might be ok for ad-hoc script run but it might be problem when running in K8S as services and specially when alerts are configured. One might get alert which are not genuine.
In order to tackle above a user can provide a list of resources which getting marked as false positive and before sending output we can remove such resources.
All of my services are returned as unused, as well as ingresses. The issue is this:
When the GetUsedServices(v1)
function is run, it updates the local variable UsedEP
, not the global variable of UsedEP
. The global variable remains an empty list and so when the difference is calculated between EP
and UsedEP
, the entire EP
list is returned.
A simple fix to this is to set UsedEP
when GetUsedServices(v1)
is called here:
https://github.com/yogeshkk/K8sPurger/blob/main/K8sPurger.py#L44
As in, update it to this:
UsedEP = GetUsedServices(v1)
I can create a PR if you want, but because the change is so small, you might want to just fix this yourself.
Great project btw
Cool tool!!
This will enable running K8Spurger as a Cronjob (or Jenkins a job) periodically and do some additional processing of the output. E.g. Send notifications to namespace owners about unused resources in their namespace.
Hello All,
I have created this script to find clean old unused config map. Then I started adding feature to it and now it can find and clean up secret, PVC and recently added service.
I am thinking about removing deleting ability as this is not much used feature also this help us to improve script further. Also in case of newly added service it might be marked as unused because of wrong selector or any miss-configuration. I will just comment the deletion part after 7 days so if anyone want to use it they can but if any new resources added won't have delete enabled (I am thinking about adding ingress next).
I know this is not popular that I receive reply but as this is in opensource it is my obligation to you all to share my view on upcoming development.
Finally, I never thought this will receive this much appreciation. Thanks for that. Let me know your thoughts on above.
Thanks,
Yogesh(Yogi)
Lines 77 to 82 in f91e1d7
Could be:
def Diffrance(listA, listB):
return [i for i in listA if i not in listB]
The script lists external names in the unused services.
We use Openshift clusters. Every project in Openshift has 4 default service accounts. More importantly, default service account is being used at run time. The script is showing they are used or not referred to. This showing large output although they are used by PODS at run time. It is better to filter these default service accounts
Example
for service accounts
| builder |velero |
| default |velero |
| deployer |velero
for secretes
| builder-dockercfg-f875f |velero |
| default-dockercfg-f6bzm |velero |
| deployer-dockercfg-qjwc7 |velero |
|
Even in a small cluster, the output is too verbose and showing lot of objects
Awesome stuff, but I have the below issue. So the pod is in crash loop.
kubectl apply -f deploy/manifest.yaml
deployment.apps/k8spurger created
service/k8spurger-svc created
serviceaccount/k8spurger-sa created
Warning: rbac.authorization.k8s.io/v1beta1 ClusterRole is deprecated in v1.17+, unavailable in v1.22+; use rbac.authorization.k8s.io/v1 ClusterRole
clusterrole.rbac.authorization.k8s.io/k8spurger-cluster-role created
Warning: rbac.authorization.k8s.io/v1beta1 ClusterRoleBinding is deprecated in v 1.17+, unavailable in v1.22+; use rbac.authorization.k8s.io/v1 ClusterRoleBindin g
clusterrolebinding.rbac.authorization.k8s.io/k8spurger-rb created
The error:
Getting unused secret it may take couple of minute..
Traceback (most recent call last):
File "K8sPurger.py", line 324, in
main("svc")
File "K8sPurger.py", line 32, in main
GetUsedResources(v1)
File "K8sPurger.py", line 125, in GetUsedResources
UsedConfigMap.append([volume.config_map_ref.name, i.metadata.namespace])
https://kubernetes.io/docs/concepts/configuration/secret/#use-cases
Need to take into account below scenario and have it append to UsedSecret/UsedConfigMap function.
spec:
containers:
- env:
envFrom:
- configMapRef:
name: configmap-test
- secretRef:
name: secrets-test
below is the block that will get you all the names of those secrets
if item.env_from is not None:
for envfrom in item.env_from:
if envfrom.secret_ref is not None:
UsedSecret.append(
[envfrom.secret_ref.name, i.metadata.namespace])
@yogeshkk We're using a combination of ArgoCD and kube-janitor to achieve this.
The ArgoCD application resource gets a timestamp through CI/CD:
---
apiVersion: argoproj.io/v1alpha1
metadata:
name: 'foobar-{{ getenv "GIT_BRANCH" | strings.Slug }}'
annotations:
janitor/expires: '{{ ((time.Now).Add (time.Hour 24)).UTC.Format "2006-01-02" }}T12:00:00Z'
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
Janitor will then clean up resources after 24h.
This script is very useful. I been looking for quite some to list unused PVC, not being used in any pods. I don't find any solution so far. I found this script is super helpful in what am looking for. However, It would be nice if script takes 1 or more arguments to print just a specific type of resource.
For example, I want to just print unused PVC, not interested in other resources. This helps
Srinivas Kotaru
ModuleNotFoundError: No module named 'kubernetes'
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.