[Enhancement] Watch for certificates in subdirectories for path specified in "watchDirectories" attribute.

[enix]

[Enhancement] Watch for certificates in subdirectories for path specified in "watchDirectories" attribute.

[porwalameet]

If this enhancement is acceptable, I can pick it up and contribute for same.

[npdgm]

Hi @porwalameet

Sorry we didn't get back to you sooner. There have been other requests for this enhancement already, but your issue certainly revived the discussion on how we could implement that. Work is in progress.

Actually there was a good reason why this directory watch option never had recursion. It comes to the fact there is no identification of certificate files based on filenames, such as extensions: .crt, .pem, ... Crawling a large filesystem would instantiate the PEM parser for every single file, consuming memory and cpu resources, and probably taking too much time to answer scrape queries before the cache gets fed. And since we do not maintain the parser code which is from Golang, that's also an open door for regressions in performance or behaviour, should we rely too much on it for scanning content in tens of thousands of files likely found in a /var/lib/kubelet/pods/.
So it was a safeguard against configurations that would make the exporter behave poorly and likely far bellow user expectations.

Ultimately we'll need an optional configuration file to add many options to each file or directory path.
In the meantime this can be implemented in a limited form with CLI arguments. Such as giving a list of file extensions, or whole path globbing.

Let me sync up with my colleague who has been looking at that feature already, and I'll tell you what to expect.

[porwalameet]

Thanks @npdgm. I do agree there will lot of files and parsing might take lot of memory based on pods running on a node. We can filter volumes directory specifically - like /var/lib/kubelet/pods/*/volumes, since such secrets/certificates will be volume mounted, so the scanning target is limited now, we can have such optimizations to narrow it down further. Just a thought.

[porwalameet]

Hi @npdgm , just checking did you hear back anything on this feature/Enhancement.

[arcln]

Hello, I am almost done implementing this into the exporter. Then we'll need to update the helm chart to be compatible with the changes, and then we'll release this feature. You can expect to see it released within 1-2 weeks.