Cause eclipse freeze

[angelozerr]

Cause eclipse freeze

[angelozerr]

@gamerson pay attention, it seems that there is a big problem with WTP WebResources (memory, slow problem). I will study that.

[paulvi]

👍 sign up

[jabby]

Just some information about UI freeze with this plugin. For me, the main freeze appear when I write content in class attribute on html tags.

[kaloyan-raev]

@jabby If you reproduce it easily could you attach a thread dump?

[angelozerr]

@kaloyan-raev I'm very busy now because I'm preparing a presentation of tern.java for JUG Paris http://www.parisjug.org/xwiki/bin/view/Meeting/20141118 but I have planned to improve WTP Web resources after.

Today I scan the whole files of project for each type of web resources (js, css, img) and I store in memory each IFile which are web resources. More I have seen a problem for some web resources which are not found although they exist?

I think to improve WTP Web resources performance, we should :

• do the scan files in a Thread.
• don't store each IFile but only IFolder which contains web resources. We should manage an observer of IFile to update those list of IFolder (and not the list of IFile)
• provide UI preferences to set IFolder(s) which contain web resources files. It will avoid parsing the whole files of IProject.

[kaloyan-raev]

@angelozerr Sounds like a good plan. Good luck with your presentation!

[angelozerr]

Thank's @kaloyan-raev !

[angelozerr]

@Tony-Proum @jabby I have tried to fix performance problem whit a lot of mean (I have an idea to improve again CSS class validation).

Could you please install the 0.70-SNAPSHOT http://oss.opensagres.fr/eclipse-wtp-webresources/0.7.0-SNAPSHOT/ and tell me if you UI freeze problem disappear?

Thank's!

Just for your info, you can install just WTP WebResources without installing AngularJS Eclipse.

[angelozerr]

@gamerson I have done a lot of refactoring, please test in your side if I have not broken something. Thank's!

[Tony-Proum]

@angelozerr Thank's, I will try that as soon as possible.

[Tony-Proum]

@angelozerr
Seem's to work : the memory usage stay stable when editing html files, and eclipse don't freeze.
It's seems not very quick but do the job

Good work!

[Tony-Proum]

Hi,
@angelozerr
the feeling of slowness I had yesterday seem's to be due to my computer's lags. Today, everything work quickly and the issue may be close in my opinion.

[angelozerr]

@Tony-Proum thank a lot for having tested this issue!

I think I have improved a lot performance, but I think we could improve a little again but not sure that it's very important. Today when for each CSS class which must be validated, we parse every time the full CSS files. My idea is when validation is done for HTML file :

• collect the CSS class of the HTML file on the first
• parse one time the full CSS files and check for each collected CSS class if there exists.

the feeling of slowness I had yesterday seem's to be due to my computer's lags

Have you this problem with uninstalled wtp webresources?

[gamerson]

Hey Angelo,

What exact refactoring did you do? Is the API the same?

On Sun, Nov 16, 2014 at 3:58 PM, Angelo [email protected] wrote:

@gamerson https://github.com/gamerson I have done a lot of refactoring,
please test in your side if I have not broken something. Thank's!

—
Reply to this email directly or view it on GitHub
#20 (comment)
.

Greg Amerson
Liferay Developer Tools
Liferay, Inc. www.liferay.com

[jabby]

I tested the new version yesterday and it was not so fast but eclipse didn't freeze. I will try with a fresh install as soon as possible.
Maybe it's a problem with my project configuration.

[angelozerr]

What exact refactoring did you do?

• validation of JS, CSS files scanned before the full files of the project to check if files existed. Today I use getFile().exists. See 1a24dcc
• when user changed the value of img/@src, script/@src, link/@href and */@Class, the full HTML was validated every time. Now it's just the attribute value which is validated (total -> partiabl validator). See 6a2c1d7
• before my refactoring, I stored the whole IFile (CSS, img, js) in a list. Now I store just folder which contains css, img, js files. The compute of this list is done whith an implementation of WTP AbstractIndexManager which is executed on background. See 2b2bee1

Is the API the same?

I think I have not changed the API, but please check that it doesn't break something in your side.

I tested the new version yesterday and it was not so fast but eclipse didn't freeze.

@jabby it should be very cool if you could share your project. Thank's!

[kaloyan-raev]

I have a memory dump from Zend Studio with some huge projects in the workspace. The version of the eclipse-wtp-webresources plugins is from Nov 15. I will try to have a look at the dump, but wanted to share it if anyone else is interested too.

The memory dump is available here: https://www.dropbox.com/s/x1sx72k7vlwu2a6/zsdump.zip?dl=0

More details about the situation are available here: http://forums.zend.com/viewtopic.php?f=59&t=122403

[angelozerr]

@kaloyan-raev it should be really interesting to do the same test with last version of wtp webresources.

The basic idea os to loop for each files of the workkspace to discover folder which host web resources.

To do that, I have implemented the WTP AbytractIndexManager.

Please tell me if you have problem again with memory or performance. Thank's!

[kaloyan-raev]

@angelozerr if you mean 2b2bee1, it should be already included in the version of the plugin used by this user. The version number of the plugins is "0.7.0.201411150452" installed from the 0.7.0-SNAPSHOT update site.

[angelozerr]

@kaloyan-raev if you are sure that 2b2bee1 is included it's very annoying -(

But perhaps 8a50a27 that it i snot included can improve too memory (if CSS and other files are in the same folder).

As I have explained, the basic idea is to loop for each files of the workkspace to discover folder which host web resources. @kaloyan-raev have you already done that (loop for files of workspace to do something).

The issue #24 will fix this problem, but it means that user must configure the project to benefit with WTP web resources.

[kaloyan-raev]

Perhaps, we are looking in the wrong direction. I opened the memory dump in Yourkit Profiler. Here is are the top objects by retained size:

We can see that some CSS model object are holding 50% of the heap.

On this screen we can see that these objects are collected in a Vector in CSSHyperlinkTraverser. The traversedSheets vector is actually a member of the org.eclipse.wst.css.core.internal.util.AbstractCssTraverser class.

As far as I can understand, the user has projects more than 3,000 CSS files. For each of them there is a CSSStyleSheetImpl object stored in a Vector that blows the available memory. In the memory dump taken a minute later, the number of CSSStyleSheetImpl object grows from 3,080 to 3,218 - eating even more memory.

Can this be optimized?

[angelozerr]

Many thank's @kaloyan-raev for your analyse.

It seems problems comes from WTP CSS plugin, so it's a bad news -(

Can this be optimized?

I'm afraid, no-( And I think WTP guys will reject this issue, because you open never in a CSS editor (which loads CSSModel) 3000 CSS files in the same time.

To fix this problem, we should perhaps provide a preferences settings to ignore some folder to avoid parsing 3000 CSS files which could be not used in a HTML file.

I think to fix this problem, is not to use WTP CSS model to retrieve CSS classes but implement a SAC Parser (it will not work for CSS content which is not valid) to collec CSS class names. A SAC Parser is the same thing than SAX Parsre but for CSS.

[kaloyan-raev]

BTW, why do you need to traverse all files in the workspace?

If I need code completion for CSS classes in the HTML editor, I need only the CSS files, referenced in the edited HTML file, to be parsed, not all CSS files in my workspace.

[Tony-Proum]

Bad news for me : the problem reappears since i use eclipse more intensively. it's seems due to css files parsing too : when i remove this specific validation it works a little better.

In order to be reactive I had to remove the plugin : eclipse and my computer became really slow.
(certainly due to excessive files access)

[gamerson]

Hey Angelo, this reminds me of the memory bug we fixed in the
eclipse-wtp-xmlsearch were loading the full model class for big files was
expensive. Here it isn't big files but lots of them.

Is there anyway to get the CSSMOdelImpl to just load the fault-tolerate
parser but not load all of the model adapters and queries, extensions, etc?

When we analyzed our bug in xmlsearch there was only one DOMModel but there
were tons of JTreeLine Trackers, etc. If you look in Kaloyan's list you
can see there are 300Mb+ of ITextRegion type classes. Those are used only
if the CSSDOMModel is planning on every displaying that data in an editor.

Is there a way that we could get CSSModelImpl to do just a basic DOM load
and not load all the secondary objects that are created as a part of the
dom like text regions line style trackers and providers, etc?

On Thu, Nov 20, 2014 at 9:54 PM, Angelo [email protected] wrote:

Many thank's @kaloyan-raev https://github.com/kaloyan-raev for your
analyse.

It seems problems comes from WTP CSS plugin, so it's a bad news -(

Can this be optimized?

I'm afraid, no-( And I think WTP guys will reject this issue, because you
open never in a CSS editor (which loads CSSModel) 3000 CSS files in the
same time.

To fix this problem, we should perhaps provide a preferences settings to
ignore some folder to avoid parsing 3000 CSS files which could be not used
in a HTML file.

I think to fix this problem, is not to use WTP CSS model to retrieve CSS
classes but implement a SAC Parser (it will not work for CSS content which
is not valid) to collec CSS class names. A SAC Parser is the same thing
than SAX Parsre but for CSS.

—
Reply to this email directly or view it on GitHub
#20 (comment)
.

Greg Amerson
Liferay Developer Tools
Liferay, Inc. www.liferay.com

[angelozerr]

If I need code completion for CSS classes in the HTML editor, I need only the CSS files, referenced in the edited HTML file, to be parsed, not all CSS files in my workspace.

Sorry it's not whole files from workspace but from the project. But this rule is applyied only when HTML page has none declaration of link element OR if link has not rel="stylesheet" (is it normal?)

Indeed for PHP or JSP page which include some other PHP/JSP, we must search from css files coming from the project (the better thing is that include is managed correctly, but I think it's hard).

In other words.:

• if your HTML page contains only one link with rel="stylesheet", it searchs only inside CSS files declared with link with rel="stylesheet"
• if your HTML page doesn't contains link or contains link without rel="stylesheet", it search from CSS files coming from the project.

It seems that searching CSS class name from the whole CSS files of the project give memory problem (in my case, no problem even with dojo project which contains a lot of CSS, but my computer is powerfull)

So I think to fix memory problem, we should have a preference setting to enable/disable search from the whole CSS files of the project or not when HTML page doesn't contains link (or link without rel="stylesheet")

By default this preference will be disable, so JSP page whithout link declaration but which include some other JSP which declare link, will not work. @gamerson is it a problem?

Hey Angelo, this reminds me of the memory bug we fixed in the
eclipse-wtp-xmlsearch were loading the full model class for big files was
expensive

It's not the same problem here : for WTP XML search it was when we check XML content type for each file. Here we need to load CSSDOMModel to retrieve CSS class names.

[kaloyan-raev]

It would be great if we can switch on/off this behavior via a preference. If we have a preference then, in my case, the default value is not important, because I can set it in the plugin_customization.ini file of my product. Even an UI for the preference is not necessary.

[angelozerr]

Ok @kaloyan-raev I will do it ASAP.

[angelozerr]

@kaloyan-raev I have added a new preferences Search CSS id and class in all CSS files (if no link)? :

By default this option is disable, so the CSS ID, class are searched (for compeltion, hyperlink, validation, hover) only for CSS files declared in the HTML page :

• with link/@href element
• and this link element MUST contains rel="stylesheet"

The second rule is very important otherwise WTP considers that it's not a valid CSS link (see org.eclipse.wst.html.core.internal.htmlcss.LinkElementAdapter#isValidAttribute :

if (!element.hasAttribute("rel") || !"stylesheet".equalsIgnoreCase(element.getAttribute("rel")))
  return false;

@gamerson hope this new feature will not break something inside Liferay IDE.

[angelozerr]

@Tony-Proum @jabby could you please retry to install and test 0.7.0-SNAPSHOT.

Pay attention to have rel="stylesheet" in your link element, otherwise stylesheet cannot be retrieved.

Thank's!

[kaloyan-raev]

@angelozerr Thank you for providing a solution so quickly. I will give it a spin and will let you know the results.

[jabby]

@angelozerr I update with the latest version of this plugin, but I can't see the new preference.
The version is : Eclipse WTP HTML-Web resources 0.7.0.201411191530

In fact I haven't "Web Resources Section". You can see my actual project preferences wizard.

Did I miss something ?

[kaloyan-raev]

@jabby are you updating from http://oss.opensagres.fr/eclipse-wtp-webresources/0.7.0-SNAPSHOT/ ?

I get an up-to-date version from the above update site: 0.7.0.201411211705

[jabby]

@kaloyan-raev In fact no. I did a mistake. I use http://oss.opensagres.fr/angularjs-eclipse/0.7.0-SNAPSHOT/
I will update and try the new preference.

[jabby]

I tried the new version. In fact I had some problems with my project, but because Eclipse didn't completely freeze, I saw what was the main problem of my project.

In fact, I use npm and bower to manage my dependencies. So I have 2 folders with lots of files (JS files and CSS files). I deal with JS files with tern path. For CSS the only way I found is to exclude in project settings all folders named bower_components or node_modules. So with this new configuration and the new version of the plugin, autocomplete on CSS don't freeze Eclipse.

[jabby]

@angelozerr Thank you for this good job 👍

[angelozerr]

Many thank's @jabby for your test.

@kaloyan-raev could you retest please?

[kaloyan-raev]

I provided the user who gave us the memory dump with a new build. I am waiting to hear back from him.

[Tony-Proum]

@angelozerr
Eclipse freeze when i activate css class and id search. It may be due to the number of files opened in my workspace. it may be a good id to let user specify : where are the project's specifics css files and js libs.

[angelozerr]

Eclipse freeze when i activate css class and id search

@Tony-Proum you have the problem again with new CSS preferences ?

By default it is unchecked and it should search only in CSS files which are decalred in the link/@href.

it may be a good id to let user specify : where are the project's specifics css files and js libs.

Yes issue was created for that #24

[kaloyan-raev]

@angelozerr My user informs that it is definitely better with this fix. He will let me know if there is an issue after using it for a longer time.

[angelozerr]

Thank's @kaloyan-raev for your feedback.

I think is there is again problem, I will implement #24 to avoid using index manager.

[Tony-Proum]

@angelozerr It was better with the fix, but eclipse still very slow and after two or three hours, it freeze, I can't see where is the problem, but it seems eclipse work faster/better without wtp plugin.

[angelozerr]

It was better with the fix, but eclipse still very slow and after two or three hours, it freeze, I can't see where is the problem, but it seems eclipse work faster/better without wtp plugin.

Ok -(

We will create a new release this week. I will try to improve it again when I will have more time.

[kaloyan-raev]

@Tony-Proum can you take a memory dump when it freezes?

[Tony-Proum]

@angelozerr Ok, sorry for this news, I think i will have more time to test it more deeply on 15th of December.

@kaloyan-raev ok, I need to re-install it, but will try when I have time

[angelozerr]

@Tony-Proum have you a problem again? Can I close this issue?

[Tony-Proum]

I had do a quick test, and everything seems Ok. No freeze, with the default configuration.
We can close this issue
@angelozerr & @kaloyan-raev sorry for the time it take, but I were not able to test it earlier.

[angelozerr]

Thanks @Tony-Proum for having tested. Great news.

[danielsokolowski]

Hello there, may I recommend to adjust the setting to always search all CSS with a suboption '- if no link element found' - both disabled initially.

Reason I am asking is that my link tags use server side functionality which resolve to proper locations when deployed but are useless when developing.

<link rel="stylesheet" type="text/css" href="{!URLFOR($Resource.ocms_mli_pageSupport, 'css/ocms-reset+support-v2-edit-template.css')}" />