GitPython and Cloning Down 7k+ repositories

[mattmiller87]

GitPython and Cloning Down 7k+ repositories

[itdependsnetworks]

For Golden Config, we should approach it this way. nautobot/nautobot-app-golden-config#683

For Nautobot Core in general, I am not sure what the best path forward is.

[glennmatthews]

As far as I understand the original issue, the current biggest problem is gitpython-developers/GitPython#718, which may be mitigated by ensuring that we explicitly delete any git.Repo objects when done with them, most likely either by explicitly calling Repo.__del__() at the end of our ensure_git_repository() utility function, or by calling Repo.__del__() when a nautobot.core.utils.git.GitRepo instance is deleted.

Shallow clone might be a nice feature to mitigate disk space requirements but to the best of my knowledge the memory leak is the current problem to solve.

[glennmatthews]

Thus far in some manual testing I'm unable to trigger the GitPython memory leak. Even after re-cloning a repository 1000 times, or cloning 100 separate repos, there are no extraneous open file handles and no apparent increase in memory consumption. This leads me to believe that my previous understanding was perhaps incorrect, and perhaps the "OOM" referenced in the issue was due to disk space exhaustion rather than memory exhaustion -- @mattmiller87 can you provide any additional clarity?