Performance Bottleneck

[giangianoulas]

Performance Bottleneck

[hvbtup]

Hmm, I don't think changing the type to a LinkedList instead of ArrayList would help. In both cases removing elements takes O(n).
Shouldn't it be a HashMap or something like that instead?

If you are profiling anyway, it would also be interesting to see the average element count of the structure.

Maybe the size is more of a problem. Looking at the code, it seems to me that registering/deregistering only takes place in the methods of DataSourceQuery class.

Maybe there is some kind of memory leak like forgetting to call deregister somehow or delaying the call too long? That would cause the structure to grow and grow, if a big report uses several thousands or millions of queries. (Well, with a classical remote RDBMS, such a report would be a candidate for refactoring anyway, because it'd run forever and slow the DB down)

Because otherwise I cannot understand why a few (or a few dozens) queries which are open at the same time can cause the structure to grow to a significant size n.

[speckyspooky]

Yes, I confirm the idea of Henning. I have also some reports which use many queries in combination with loops and with large row count (1Mio.) but I have never seen this problem.

Do you got the problem with the latest BIRT version 4.15?

[giangianoulas]

There was indeed a memory leak. In DataSourceQuery#execute elements were added but never removed.
Also there was a unnecessary big syncronized block.

I agree that this report needs a redesign, but it is not that easy to be done.
I am using Birt 4.12.0 but it seems to be ancient problem

[hvbtup]

Can you provide a PR?

[giangianoulas]

giangianoulas@3e2e587

[hvbtup]

Hmm, the code looks OK at first sight, but you need to create a pull request and sign the ECA.

I'm not sure about the call to super.equals in the equals method. Is that correct and is it necessary at all?

BTW did you test it with your huge report and did it reduce runtime?

[giangianoulas]

Ok all done
equals and hashCode was not necessary so i removed them.

Sadly i can produce this amount of data because it is quite complex and i don’t have the business knowledge for this.

I will come back to this and comment when i will have something at hand

[merks]

@giangianoulas

FYI, a build with your changes is available here:

https://download.eclipse.org/birt/updates/nightly/latest