Automatically deleting resourcereservation object when spark-driver completed

[palantir]

Automatically deleting resourcereservation object when spark-driver completed

[onursatici]

Hey @boryssemeniuk thanks for the flag. Yes you are right that as long as the resource reservation object exists it will be considered for scheduling decisions. They have an owner reference to the driver pod object so we expect the pods to be deleted, which in turn will delete the resource reservation objects and free up space.

One possible solution I can think of would be to filter resource reservations that point to driver pods that are either Failed or Completed, so we only accumulate usages of resource reservations that point to running driver pods here: https://github.com/palantir/k8s-spark-scheduler/blob/master/internal/extender/resource.go#L442