OutOfMemory during building workspace after upgrade to 2023-03

[AlBundy33]

OutOfMemory during building workspace after upgrade to 2023-03

[trancexpress]

Are you able to take memory snapshots with 2022-06 and 2023-03, compare and list the differences? If not, making the workspace available as an archive somewhere would help with analysis.

[AlBundy33]

"Unfortunately" these are only internal projects.
But the good news are that I can reproduce it with an attached JProfiler.
With 2023-03, a fresh workspace and an imported mid-size-project it looks like this.

In the screenshot the GC Activity goes down after some time - this does not happen in my large workspace (or at least I didn't wait that long).

I've also added -XX:+HeapDumpOnOutOfMemoryError to my eclipse.ini - the file is over 5GB.
I can check with MAT what are the leak suspects.

[AlBundy33]

the instance is currently showing this messages:

this is the end of the logfile

!ENTRY org.eclipse.lsp4e 4 0 2023-03-20 17:55:13.980
!MESSAGE TimeoutException
!STACK 0
java.util.concurrent.TimeoutException
	at java.base/java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1960)
	at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2095)
	at org.eclipse.lsp4e.LanguageServerWrapper.lambda$10(LanguageServerWrapper.java:519)
	at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804)
	at java.base/java.util.concurrent.CompletableFuture$AsyncRun.exec(CompletableFuture.java:1796)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)

!ENTRY org.eclipse.core.jobs 4 2 2023-03-20 17:55:15.312
!MESSAGE An internal error occurred during: "Initialize Language Servers for MANIFEST.MF".
!STACK 0
java.lang.OutOfMemoryError: Java heap space

!ENTRY org.eclipse.ui.monitoring 4 0 2023-03-20 17:55:15.315
!MESSAGE UI freeze of 1,6s at 17:52:05.653
(no stack samples were collected because the monitoring thread starved for CPU)

!ENTRY org.eclipse.ui.monitoring 4 0 2023-03-20 17:55:15.702
!MESSAGE UI freeze of 1,5s at 17:52:55.461

!ENTRY org.eclipse.core.jobs 4 2 2023-03-20 17:55:16.403
!MESSAGE An internal error occurred during: "Building".
!STACK 0
java.lang.OutOfMemoryError: Java heap space

!ENTRY org.eclipse.lsp4e 4 0 2023-03-20 17:55:23.650
!MESSAGE TimeoutException
!STACK 0
java.util.concurrent.TimeoutException
	at java.base/java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1960)
	at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2095)
	at org.eclipse.lsp4e.LanguageServerWrapper.lambda$10(LanguageServerWrapper.java:519)
	at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804)
	at java.base/java.util.concurrent.CompletableFuture$AsyncRun.exec(CompletableFuture.java:1796)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)

My project has also a lot of errors and warnings

but this is fine because Errors came from a wrong target platform (test-project is a rcp-application) and warnings a because of missing generics, deprecated methods and so on (because the application exists since java 7)

[trancexpress]

It would be great if you can diff 2 dumps (when the build still worked with 3GB heap and now), that should show where consumption increased.

[AlBundy33]

Tested with this version https://www.eclipse.org/downloads/download.php?file=/technology/epp/downloads/release/2023-03/R/eclipse-rcp-2023-03-R-win32-x86_64.zip and fresh workspace

[AlBundy33]

I've loaded the heapdump into MAT and the leak-suspects are

The class "org.eclipse.lsp4e.ConnectDocumentToLanguageServerSetupParticipant", loaded by "org.eclipse.lsp4e", occupies 1.678.750.376 (52,42 %) bytes. The memory is accumulated in one instance of "java.util.HashMap$Node[]", loaded by "<system class loader>", which occupies 1.678.750.184 (52,42 %) bytes.

Keywords
org.eclipse.lsp4e.ConnectDocumentToLanguageServerSetupParticipant
org.eclipse.lsp4e
java.util.HashMap$Node[]

and

The thread org.eclipse.core.internal.jobs.Worker @ 0x7498a6f18 Worker-3: Building keeps local variables with total size 1.202.832.664 (37,56 %) bytes.

The memory is accumulated in one instance of "org.eclipse.jdt.internal.compiler.Compiler", loaded by "org.eclipse.jdt.core.compiler.batch", which occupies 1.178.283.192 (36,79 %) bytes.
The stacktrace of this Thread is available. [See stacktrace](https://github.com/eclipse-jdt/eclipse.jdt.core/issues/pages/41.html). [See stacktrace with involved local variables](https://github.com/eclipse-jdt/eclipse.jdt.core/issues/pages/42.html).



Keywords
org.eclipse.jdt.internal.compiler.Compiler
org.eclipse.jdt.core.compiler.batch
org.eclipse.jdt.internal.core.builder.AbstractImageBuilder.cleanUp()V
AbstractImageBuilder.java:276
org.eclipse.jdt.internal.core.builder.BatchImageBuilder.cleanUp()V
BatchImageBuilder.java:207
org.eclipse.jdt.internal.core.builder.BatchImageBuilder.build()V
BatchImageBuilder.java:95
org.eclipse.jdt.internal.core.builder.JavaBuilder.buildAll()V
JavaBuilder.java:273
[Details »](https://github.com/eclipse-jdt/eclipse.jdt.core/issues/pages/35.html)

I did also a leak suspect analysis with the 2022-06 but created the heapdump "somewhere" during build.
In this dump the first leak suspects are

One instance of "org.eclipse.core.internal.dtree.DeltaDataTree" loaded by "org.eclipse.core.resources" occupies 533.999.000 (33,54 %) bytes. The memory is accumulated in one instance of "org.eclipse.core.internal.dtree.AbstractDataTreeNode[]", loaded by "org.eclipse.core.resources", which occupies 533.998.952 (33,54 %) bytes.

Keywords
org.eclipse.core.internal.dtree.DeltaDataTree
org.eclipse.core.resources
org.eclipse.core.internal.dtree.AbstractDataTreeNode[]

[Details »](https://github.com/eclipse-jdt/eclipse.jdt.core/issues/pages/21.html)

and

One instance of "org.eclipse.jdt.internal.core.search.indexing.IndexManager" loaded by "org.eclipse.jdt.core" occupies 165.611.792 (10,40 %) bytes. The memory is accumulated in one instance of "java.lang.Object[]", loaded by "<system class loader>", which occupies 155.021.032 (9,74 %) bytes.

Keywords
org.eclipse.jdt.internal.core.search.indexing.IndexManager
org.eclipse.jdt.core
java.lang.Object[]

[Details »](https://github.com/eclipse-jdt/eclipse.jdt.core/issues/pages/37.html)

But bot with much less memory.

[AlBundy33]

and JProfiler shows this

old.hprof is from my default-workspace with all projects and eclipse 2022-06 where I've started a full build for my test-project

eclipse.bin is the heapdump of eclipse 2023-03 after the OutOfMemoryError.
-> The imported projects is the same as in the step above

[trancexpress]

Does JProfiler show which classes are holding the top 6 in this list?

[AlBundy33]

JProfiler unfortunetely not but EclipseMemoryAnalyzer

Class Name	Shallow Heap	Retained Heap
java.util.HashMap$Node[4096] @ 0x7c2b771d0	16.400	1.678.750.184
table java.util.HashMap @ 0x7c743c380	48	1.678.750.232
map java.util.HashSet @ 0x7c743c370	16	1.678.750.248
SUBMITTED_JOBS class org.eclipse.lsp4e.ConnectDocumentToLanguageServerSetupParticipant @ 0x7c743c2f8	8	1.678.750.376
org.eclipse.lsp4e.ConnectDocumentToLanguageServerSetupParticipant @ 0x7c7c14c88	16	144
this$0 org.eclipse.lsp4e.ConnectDocumentToLanguageServerSetupParticipant$2 @ 0x7fe6db8b8	112	288
, currentJob org.eclipse.core.internal.jobs.Worker @ 0x74f2c0940 Worker-4: Initialize Language Servers for plugin.xml Thread	120	6.384
job org.eclipse.ui.internal.progress.ProgressManager$JobMonitor @ 0x7fe6db638 »	32	32
Total: 2 entries
this$0 org.eclipse.lsp4e.ConnectDocumentToLanguageServerSetupParticipant$2 @ 0x7fe9f79d8 »	112	288
this$0 org.eclipse.lsp4e.ConnectDocumentToLanguageServerSetupParticipant$2 @ 0x7fee8fd60 »	112	288
this$0 org.eclipse.lsp4e.ConnectDocumentToLanguageServerSetupParticipant$2 @ 0x7ff5fa718 »	112	288
this$0 org.eclipse.lsp4e.ConnectDocumentToLanguageServerSetupParticipant$2 @ 0x7ff988ec8 »	112	288
this$0 org.eclipse.lsp4e.ConnectDocumentToLanguageServerSetupParticipant$2 @ 0x7ffb9a920 »	112	376
this$0 org.eclipse.lsp4e.ConnectDocumentToLanguageServerSetupParticipant$2 @ 0x7ffbb7a50 »	112	288
this$0 org.eclipse.lsp4e.ConnectDocumentToLanguageServerSetupParticipant$2 @ 0x7ffc1b298 »	112	288
this$0 org.eclipse.lsp4e.ConnectDocumentToLanguageServerSetupParticipant$2 @ 0x7ffebeae8 »	112	288
Total: 9 entries
[0] java.lang.Object[49] @ 0x793fdbbe0 »	216	5.976

[trancexpress]

Its odd that the diff shows an increase in TreeMap objects and TreeMap nodes... not the HashMap nodes seen in the dump from the newer Eclipse version. Just to be sure both versions are the same type of IDE, can you paste the "header" contents of the about dialog? Main menu -> Help -> About ...

E.g. for the SDK this is e.g.:

Eclipse SDK
Version: 2023-03 (4.27)
Build id: I20230211-0330

You do see lsp4e objects also in the heap dump from the older build you are using? If you can make the dumps available somewhere, that might simplify things. I'm not sure whether the dumps would reveal anything about your private code though.

Anyway, @gayanper are you aware of any changes in lsp4e that could lead to more memory consumption?

[trancexpress]

Seems like this commit fits the time frame: 8775fa82e

That code is now reworked, but I assume the respective commit has not made it in 2023-03 (somehow): 8f9220cc

[AlBundy33]

Luckily I did some screenshots for a colleague today 😄

I can test a nightly build if you think this is already fixed.
-> but I would need a link where I can find the downloads

[trancexpress]

I can test a nightly build if you think this is already fixed. -> but I would need a link where I can find the downloads

No idea, I'm not involved with lsp4e.

The 2022-06 IDE has this version: 0.13.12.202206011407
The 2023-03 IDE has this version: 0.15.0.202211292024

The update site (if you want you can try updating from it, but I have no idea if it will help): https://download.eclipse.org/lsp4e/releases/latest/

The version there is: https://www.eclipse.org/downloads/download.php?file=/lsp4e/releases/latest/plugins/org.eclipse.lsp4e_0.16.0.202303141217.jar

I do see ConnectDocumentToLanguageServerSetupParticipant.PENDING_CONNECTIONS in that jar (specifically in the source jar). So maybe it would help to update. Considering the statistics you listed show 50% memory reserved from this class and the map where a lot of memory went was replaced by a WeakHashMap (which probably helps with the memory allocation for the map).

[AlBundy33]

Downloaded eclipse-SDK-I20230320-1800-win32-x86_64.zip from here
https://download.eclipse.org/eclipse/downloads/index.html

added this at the beginning of the eclipse.ini
-data
C:\temp\home\workspace
-configuration
C:\temp\home\configuration

this at the end
-Duser.home=C:\temp\home

and change Xmx to -Xmx3G

After that I've startet Eclipse and installed lsp4e from this update site
https://download.eclipse.org/lsp4e/releases/latest/

Now I've imported my test-project -> compiled without OutOfMemory
Added another large project -> compiled without OutOfMemory

Next I'll try 2023-03 mit updated lsp4e

[AlBundy33]

At first I've used a clean eclipse again (without updating lsp4e)

[AlBundy33]

Updated lsp4e looks promising...


-> Here I've already added two large projects

I think I'll try to update my main installation again now...

[trancexpress]

Should we move this to the lsp4e tracker and close as fixed by eclipse/lsp4e@8f9220c#diff-8e21697c4a1f40f67124b21910baf0e9e3729e144bca3215c7e8732cfa34f0e2 then? I assume. Also the regression is probably coming from: eclipse/lsp4e@8775fa8#diff-8e21697c4a1f40f67124b21910baf0e9e3729e144bca3215c7e8732cfa34f0e2

[AlBundy33]

I didn't switch because all other informations are already here and I hope that this issue is solved and I can just add success-message in some minutes.

If there is still an issue I would move to lsp4e

[AlBundy33]

But the diff would explain why the build didn't finish on a colleagues PC.
-> he has more RAM and works on Linux so instead of getting an OutOfMemory his builds may wait for the submitted jobs.

[AlBundy33]

Updating lsp4e seems to fix the OutOfMemory

-> This is the workspace with all of my projects

Thanks for your help and sorry for blaming jdt 😄

[trancexpress]

Alright, thanks for also opening eclipse/lsp4e#563.