Django integration: You cannot access body after reading from request's data stream

[Audiopolis]

Django integration: You cannot access body after reading from request's data stream

[szokeasaurusrex]

Hi @Audiopolis, I attempted to reproduce your issue using this code, but I did not observe the error message that you reported.

Can you please clarify how exactly you encountered this issue? If possible, please share a reproduction with me (you can also do so by opening a PR on my reproduction repo and sharing a link to the PR here).

[Audiopolis]

@szokeasaurusrex Thanks for trying. I will try to create a reproduction. In the meantime, I'll clarify how I encountered the issue:

We send an email from a Django view using Resend. In this environment, Resend does not recognize the domain, so it throws an error. We catch that error and use logger.exception(...) instead, where logger = logging.getLogger(__name__). The view returns a 201 response. Then, the exact traceback I added to this issue description is printed.

The original request is automatically converted from an ASGIRequest (no data attribute, and can only read body once) to a RES framework Request (has a cached data property) by REST framework. I don't know how Sentry gets the original ASGIRequest instead of the DRF Request, but since it does, it makes sense that 'ASGIRequest' object has no attribute 'data' and You cannot access body after reading from request's data stream. I will have to analyze the situation more closely and provide a reproduction.

[antonpirker]

Same error is also happening inside Sentry with SDK 2.1.1:

  File "/usr/local/lib/python3.11/site-packages/sentry_sdk/integrations/django/__init__.py", line 555, in parsed_body
    return self.request.data
           ^^^^^^^^^^^^^^^^^
AttributeError: 'WSGIRequest' object has no attribute 'data'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/sentry_sdk/integrations/django/__init__.py", line 486, in wsgi_request_event_processor
    DjangoRequestExtractor(request).extract_into_event(event)
  File "/usr/local/lib/python3.11/site-packages/sentry_sdk/integrations/_wsgi_common.py", line 96, in extract_into_event
    parsed_body = self.parsed_body()
                  ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/sentry_sdk/integrations/django/__init__.py", line 557, in parsed_body
    return RequestExtractor.parsed_body(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/sentry_sdk/integrations/_wsgi_common.py", line 142, in parsed_body
    return self.json()
           ^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/sentry_sdk/integrations/_wsgi_common.py", line 154, in json
    raw_data = self.raw_data()
               ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/sentry_sdk/integrations/django/__init__.py", line 538, in raw_data
    return self.request.body
           ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/http/request.py", line 328, in body
    raise RawPostDataException(
django.http.request.RawPostDataException: You cannot access body after reading from request's data stream", "level":"error", "name":"sentry_sdk.errors"}

Can be seen here: https://cloudlogging.app.goo.gl/yfAdqBdmA8iER8ZQ8 (retricted link)

[sentrivana]

So just to expand on what was already written here, the way DRF works is that it wraps the native Django WSGIRequest (or ASGIRequest) in its own Request. The wrapper class adds some stuff on top and proxies everything else to the underlying Django request.

We capture the data here. Basically, self.request.data is only expected to be there if this is a DRF Request; otherwise we essentially fall back to self.request.body (that's what RequestExtractor.parsed_body(self) eventually looks at). Reading the body on a Django request should be fine but in some cases the body has already started being read and we hit this.

The quick "fix" would be to capture RawPostDataException when we try to access request.body and just give up on reading the body ourselves and return None. However this doesn't address the underlying problem of why request.data is not there in the first place (it indeed looks like we're working with the Django request instead of the DRF one).

Wasn't able to repro this yet -- @Audiopolis What does the request look like in your app? I assume it's a POST? Does it contain any form data, uploaded files, etc.? Do you access anything on the request in the view?

[Audiopolis]

So just to expand on what was already written here, the way DRF works is that it wraps the native Django WSGIRequest (or ASGIRequest) in its own Request. The wrapper class adds some stuff on top and proxies everything else to the underlying Django request.

We capture the data here. Basically, self.request.data is only expected to be there if this is a DRF Request; otherwise we essentially fall back to self.request.body (that's what RequestExtractor.parsed_body(self) eventually looks at). Reading the body on a Django request should be fine but in some cases the body has already started being read and we hit this.

The quick "fix" would be to capture RawPostDataException when we try to access request.body and just give up on reading the body ourselves and return None. However this doesn't address the underlying problem of why request.data is not there in the first place (it indeed looks like we're working with the Django request instead of the DRF one).

Wasn't able to repro this yet -- @Audiopolis What does the request look like in your app? I assume it's a POST? Does it contain any form data, uploaded files, etc.? Do you access anything on the request in the view?

Thanks for the additional context @sentrivana. I don't have the full body anymore, but I believe it was indeed a POST request containing a JSON payload. All the logic succeeded (including feeding the payload into a serializer that validated the data and used it to create an object) until it attempted to send an email through a third-party service, which failed, triggering an exception log. In other words, the DRF request has been read multiple times, and at some point, it seems to stop being a DRF request.

We are using this middleware:

MIDDLEWARE = [
   "django.middleware.security.SecurityMiddleware",
   "django.contrib.sessions.middleware.SessionMiddleware",
   "corsheaders.middleware.CorsMiddleware",
   "django.middleware.common.CommonMiddleware",
   "django.middleware.csrf.CsrfViewMiddleware",
   "django.contrib.auth.middleware.AuthenticationMiddleware",
   "django.contrib.messages.middleware.MessageMiddleware",
   "django.middleware.clickjacking.XFrameOptionsMiddleware",
   "nplusone.ext.django.NPlusOneMiddleware",
]

In this environment, we are also using debug_toolbar.middleware.DebugToolbarMiddleware.

I have not had time yet to try to create a reproduction.