Using a component inside a loop will duplicate the database query (or rendering the entire page) for each element

[medycynka]

Using a component inside a loop will duplicate the database query (or rendering the entire page) for each element

[adamghill]

Hello, I'm sorry you are running into this issue. Off the top of my head, I wonder if this is an n+1 problem? You don't show the query for filtered_object_list, but you might be able to put a select_related or prefetch_related which would reduce subsequent foreign-key lookups. Although based on your statement that you have tried "changing Django models to simple objects" this might not solve your problem.

One thing to see what might be going on is to change the Unicorn component to a regular Django include and output the same model data. If it's still slow, it's an ORM issue. If it's super fast, then there might be something going on in Unicorn.

I render a component for every row in a table in this example: https://github.com/adamghill/django-unicorn/blob/main/example/unicorn/templates/unicorn/nested/table.html#L38. The backend code is here: https://github.com/adamghill/django-unicorn/blob/main/example/unicorn/components/nested/table.py#L31. You can see I have a select_related to prevent duplicate database queries. I just tested it with 100 rows and it still renders very quickly, so hopefully tweaking the ORM query will work for you as well!

There should be minimal overhead when rendering components with Unicorn, but If the above doesn't work, please let me know.

[medycynka]

Hi,
Thank you very much for your quick reply. Forgive me, I forgot to include that filtered_object_list is basically a converted QuerySet of projects to a list of objects of type "AvailableProjectPresenter" in a way:

context['filtered_object_list'] = [
    AvailableProjectPresenter(project, ...) for project in Project.objects.select_related(...).prefetch_related(...).filter(...).annotate(...)
]

Unfortunately, we are sure that this is not an N+1 problem because:

• we are already using select_related and prefetch_related and we use it on the correct fields
• we tested this problem on simplified queries without using any related models
• in the template we only changed {% include ... %} to {% unicorn ... %} and this increased the number of queries from 55 to 685 (of which almost 95% were duplicates) and the wait time increased from ~800ms to ~11s
• according to django-debug-toolbar every database query called in our template is duplicated, not only the Project.objects... query, but also other queries used in the template

We suspect it may be a problem with the way components render themselves and the use of beautifulsoup4 to find a component's parent, but we haven't been able to test that yet.

[adamghill]

Ah, thanks for the further clarification here -- that is frustrating! I'm going to try to replicate your set up in the example app in this repo to see if I get the same results. Or let me know if there is a minimal way to reproduce, thanks.