Run jQuery unit tests in XHTML

[jquery]

Run jQuery unit tests in XHTML

[mgol]

Comment author: dmethvin

Still searching for a good solution, but this isn't a blocker.

[mgol]

Comment author: dmethvin

Ref #12359

[mgol]

Comment author: tomgrohl

Is it a case of copying all the tests and changing the doctype to xhtml, or is it limited to certain test.

I might be able to help on this one.

[mgol]

Comment author: dmethvin

That's part of the investigation to be done.

For example, we support Firefox and you can serve XHTML with the right docttype and HTTP Content-Type to ensure it's interpreted as XHTML, but it just spits out console warnings if you get it wrong according to #12359. Our automated tests wouldn't catch that type of problem.

On the other hand, in #12357 it seems a Firefox XUL application demands correct XHTML to run at all. We don't have XUL on our automated tests so there's no formal support for it, but we've been able to keep it running except when we break XHTML by accident as we've done a couple of times now. There have been similar reports related to Chrome plugins.

Perhaps we can just provide some sort of XUL test in Testswarm, since that seems to be the most sensitive environment.

[mgol]

Comment author: tomgrohl

Thanks for the info.

I'll have a look into it, see what else I can find.

[mgol]

Comment author: tomgrohl

@dmethvin

As a starting point, I've wrote some code to check for content type and doctype.

It detects if closing tags / quoted attributes are required based on the following:

1 - content type is application/xhtml+xml 2 - content type is text/html and doctype is XHTML 3 - document is xul (Needs more testing though)

Generally is a doctype and/or content type is omitted it detaults to text/html.

This logic will be good to use and probably fix #12359 as well.

I'll now work on creating some tests for xhtml (based on the existing ones) and see where I get.

[mgol]

Comment author: dmethvin

Bulk change from enhancement to feature.

[mgol]

Comment author: dmethvin

Add IE10 to the list of browsers that will generate console warnings when there are doctype-related errors. Those happen too early for a shim to console.warn to see them.

[mgol]

Comment author: dmethvin

Although I wonder whether the shim would see them in a delay-loaded iframe.

Anway,  https://github.com/jquery/jquery/commit/c6a694e1c262acb941b7fd86663f186656496bbc#commitcomment-2975014

[mgol]

Comment author: anthonyryan1@…

Firstly a note: The majority of the regressions in XHTML compatibility over the past few years have been in feature / bug detection code included in jQuery.

There are two approaches we can take to solving this problem, neither of which is particularly ideal:

We can run the entire test suite in XHTML mode:

Pros:

• No duplicate tests required in the test suite.
• All code paths are properly tested

Cons:

• Looking through the QUnit code history, they are extremely regression prone with XHTML compatibility. We could expect regular breakage when updating QUnit unless they are similarly encouraged to test XHTML support.
• The test running interface uses document.write() and would need to be re-implemented in a better way.
• Some unit tests may not be possible in XHTML mode, they would require some form of indicator that they should not be run in XHTML mode.
• As a result of the previous, some tests may be marked incompatible with XHTML when they could work just fine out of developer laziness or misunderstanding.
• Increases the test-running time by requiring the entire suite be run mulitple times
• iframes tests will be immune to XHTML mode setup in this way, since the iframe has it's own context. This would result is tests passing when they should fail.

Alternatively, we can duplicate specific tests to be run in XHTML iframes:

Pros:

• None

Cons:

• Code coverage will be poor
• Tests will be duplicated from their HTML counterparts
• Maintaining a list of what should be duplicated will be a maintenance burden on jQuery.

[mgol]

Comment author: dmethvin

@anthonyryan, thanks for taking another look at this. I agree that none of the solutions are ideal, and it does get harder now that we are doing the feature detects on an as-needed basis. I'm thinking we may still want to do some sort of basic-functionality test with XHTML, at least it would give us some hope of avoiding these kind of recurring regressions.

[mgol]

Comment author: gibson042

True, just loading jQuery in an iframe and invoking the parent callback on successful load would go a long way. Per-module tests can come later, and need not be much more complicated either.

[mgol]

Comment author: m_gol

Replying to dmethvin:

@anthonyryan, thanks for taking another look at this. I agree that none of the solutions are ideal, and it does get harder now that we are doing the feature detects on an as-needed basis.

In test/support.js we force-compute all support tests to check if they match our expected matrix so that's not a problem.

[dmethvin]

Per discussion in the meeting today, we might do an occasional run in XHTML mime type.

[dmethvin]

Ref gh-1969

[anthonyryan1]

With the issue of XHTML regressions coming up again, it seems like an appropriate time to revisit this.

I think the optimal course of action would be:

1. Fixing QUnit for XHTML and ensuring they start regularly testing in an XHTML mode. We can't expect to keep the XHTML tests green if our test harness keeps breaking.
2. Requiring a webserver to serve the test suite so everything can be served with appropriate headers (we can't ensure all of the various file:/// URLs are served with the correct content-type).
3. Conditionally skip based on specific criteria (in this case if (document.contentType === 'application/xhtml+xml') { /* skip */ }). This would be a very significant effort as each XHTML failing test would need to be manually checked to understand why it's failing. Whether it's because of how it is being used (for example, appending an unclosed tag), or because of a bug in jQuery itself.
4. Get all of the tests passing or skipping under XHTML. Because while people might notice the difference between 0 and 1 failure. The difference between 17 and 18 will lead to people just ignoring these failures and we'll see no fewer regressions.

[mgol]

Rescheduling to 3.0.1 as it's non-trivial and we still have a lot to do before 3.0.0.

[timmywil]

Moving to Roadmap. We're not sure when we'll get to this.

[mgol]

@timmywil When you're closing an issue while not fixing it please always make sure to remove a milestone, otherwise it will get picked up in a changelog totally incorrectly.

[timmywil]

@mgol Good call, thanks for the reminder.