Memory leak from client objects

[googleapis]

Memory leak from client objects

[crwilcox]

It seems possible it is part of setting up auth.

❯ # Add @profile to inner method to get tracing.
❯ pip install memory-profiler
❯ python -m memory_profiler profile.py
Filename: venv/lib/python3.7/site-packages/google/auth/transport/requests.py

Line #    Mem usage    Increment   Line Contents
================================================
   242   93.113 MiB   81.426 MiB       @profile
   243                                 def request(self, method, url, data=None, headers=None, timeout=None, **kwargs):
   244                                     """Implementation of Requests' request.
   245
   246                                     Args:
   247                                         timeout (Optional[Union[float, Tuple[float, float]]]): The number
   248                                             of seconds to wait before raising a ``Timeout`` exception. If
   249                                             multiple requests are made under the hood, ``timeout`` is
   250                                             interpreted as the approximate total time of **all** requests.
   251
   252                                             If passed as a tuple ``(connect_timeout, read_timeout)``, the
   253                                             smaller of the values is taken as the total allowed time across
   254                                             all requests.
   255                                     """
   256                                     # pylint: disable=arguments-differ
   257                                     # Requests has a ton of arguments to request, but only two
   258                                     # (method, url) are required. We pass through all of the other
   259                                     # arguments to super, so no need to exhaustively list them here.
   260
   261                                     # Use a kwarg for this instead of an attribute to maintain
   262                                     # thread-safety.
   263   93.113 MiB    0.012 MiB           _credential_refresh_attempt = kwargs.pop("_credential_refresh_attempt", 0)
   264
   265                                     # Make a copy of the headers. They will be modified by the credentials
   266                                     # and we want to pass the original headers if we recurse.
   267   93.113 MiB    0.016 MiB           request_headers = headers.copy() if headers is not None else {}
   268
   269                                     # Do not apply the timeout unconditionally in order to not override the
   270                                     # _auth_request's default timeout.
   271                                     auth_request = (
   272                                         self._auth_request
   273   93.113 MiB    0.008 MiB               if timeout is None
   274   93.113 MiB    0.008 MiB               else functools.partial(self._auth_request, timeout=timeout)
   275                                     )
   276
   277   93.113 MiB    0.008 MiB           with TimeoutGuard(timeout) as guard:
   278   93.234 MiB    3.078 MiB               self.credentials.before_request(auth_request, method, url, request_headers)
   279
   280   93.234 MiB    0.008 MiB           timeout = guard.remaining_timeout
   281
   282   93.234 MiB    0.004 MiB           with TimeoutGuard(timeout) as guard:
   283   93.234 MiB    0.008 MiB               response = super(AuthorizedSession, self).request(
   284   93.234 MiB    0.008 MiB                   method,
   285   93.234 MiB    0.012 MiB                   url,
   286   93.234 MiB    0.008 MiB                   data=data,
   287   93.234 MiB    0.008 MiB                   headers=request_headers,
   288   93.234 MiB    0.008 MiB                   timeout=timeout,
   289   93.453 MiB    1.102 MiB                   **kwargs
   290                                         )
   291   93.453 MiB    0.008 MiB           timeout = guard.remaining_timeout
   292
   293                                     # If the response indicated that the credentials needed to be
   294                                     # refreshed, then refresh the credentials and re-attempt the
   295                                     # request.
   296                                     # A stored token may expire between the time it is retrieved and
   297                                     # the time the request is made, so we may need to try twice.
   298                                     if (
   299   93.453 MiB    0.008 MiB               response.status_code in self._refresh_status_codes
   300                                         and _credential_refresh_attempt < self._max_refresh_attempts
   301                                     ):
   302
   303                                         _LOGGER.info(
   304                                             "Refreshing credentials due to a %s response. Attempt %s/%s.",
   305                                             response.status_code,
   306                                             _credential_refresh_attempt + 1,
   307                                             self._max_refresh_attempts,
   308                                         )
   309
   310                                         if self._refresh_timeout is not None:
   311                                             if timeout is None:
   312                                                 timeout = self._refresh_timeout
   313                                             elif isinstance(timeout, numbers.Number):
   314                                                 timeout = min(timeout, self._refresh_timeout)
   315                                             else:
   316                                                 timeout = tuple(min(x, self._refresh_timeout) for x in timeout)
   317
   318                                         # Do not apply the timeout unconditionally in order to not override the
   319                                         # _auth_request's default timeout.
   320                                         auth_request = (
   321                                             self._auth_request
   322                                             if timeout is None
   323                                             else functools.partial(self._auth_request, timeout=timeout)
   324                                         )
   325
   326                                         with TimeoutGuard(timeout) as guard:
   327                                             self.credentials.refresh(auth_request)
   328                                         timeout = guard.remaining_timeout
   329
   330                                         # Recurse. Pass in the original headers, not our modified set, but
   331                                         # do pass the adjusted timeout (i.e. the remaining time).
   332                                         return self.request(
   333                                             method,
   334                                             url,
   335                                             data=data,
   336                                             headers=headers,
   337                                             timeout=timeout,
   338                                             _credential_refresh_attempt=_credential_refresh_attempt + 1,
   339                                             **kwargs
   340                                         )
   341
   342   93.453 MiB    0.008 MiB           return response

[tseaver]

To investigate, try calling client._http.close() at the bottom of the loop.

[tseaver]

@crwilcox The Python memory allocator only returns stuff sporadically to the OS, and holds on to blocks of 256Kb for small objects expected to have short lifetimes. That first graph looks pretty much like what I would expect, and doesn't seem to indicate a leak to me.