Enable tests for GitHub Actions

[eclipse-birt]

Enable tests for GitHub Actions

[wimjongman]

How far are we with this?

[ruspl-afed]

How far are we with this?

Somewhere in the first part of this story. Currently org.eclipse.birt.chart.tests are failing for me.

[hvbargen]

I think that the failing tests (4 for me) are entirely caused by different locale settings.
E.g. the first failing test on my machine says:

expected:<13[.]1> but was:<13[,]1>

I'm pretty sure that this is because my machine has German settings, e.g. a decimal comma is used instead of a decimal point as in US and GB.
But I don't know how this can be fixed.

[hvbargen]

There are also test failures caused by DocArchiveLockManagerTest.java.
I guess that's because the lock file is hardcoded as

static final String LOCK_FILE_NAME = "./utest/lock.lck";

and probably the directory "utest" does not exist. It would be safer to just use "lock.lck" in the current working directory.

[hvbargen]

The first test failure (the one I mentioned a few days ago) can be fixed by changing the file chart/org.eclipse.birt.chart.tests/src/org/eclipse/birt/chart/tests/engine/computation/ValueFormatterTest.java like this:

public void testFormat2( ) throws ChartException
{
	NumberFormat us_decimal_format = NumberFormat.getInstance(ULocale.US);
	assertEquals( "13.1", ValueFormatter.format( new Double( 13.1 ), null, null, us_decimal_format ) );//$NON-NLS-1$ 
	assertEquals( "13.1", ValueFormatter.format( NumberDataElementImpl.create( 13.1 ), null, null, us_decimal_format ) );//$NON-NLS-1$
}

I recommend to temporarily disable the 3 other failing tests (all 3 are locale-dependent):

MonthDateFormatTest.testFormat:56 Locale=en_CA expected:<Jun[] 2013> but was:<Jun[.] 2013>
ChartUtilTest.testFormatDouble:136 expected:<0[.]000000000> but was:<0[,]000000000>
ChartUtilTest.testStringValue:152 expected:<...5-01-22 11:56:10.125[-08]00> but was:<...5-01-22 11:56:10.125[+01]00>

Then let's see if there are still other tests failing.
I think it's important to have green light for the build. Only then the unit tests are useful for verifying new PRs.

[wimjongman]

The tests are not failing on the server so I don't think we should disable them.

[ruspl-afed]

we have only 2 test modules active in the pipeline
if you know how to fix others please create a PR

For the current failures in master I created #633

[hvbargen]

Hmm...
It should be possible to run the unit tests on our developer machines - just to have a good feeling before creating a PR.
And our developer machines are using Dutch, Russian or German locale settings, so these tests fail.

And the CI is still red. It seems like there are two different errors which I cannot imagine to be caused by one of the PRs. Instead they seem more or less random to me:

• DocArchiveLockManagerTest.testThread (fails for #633 and for #630 and others).
• ContentCommandTest.testCopyAndPaste (Fails for #624)

[wimjongman]

Hmm...
It should be possible to run the unit tests on our developer machines - just to have a good feeling before creating a PR.

Totally agree. But disabling tests is not giving a good feeling ;)

Fixing these tests should be quite easy?

[hvbargen]

I tried to understand what's going on in MonthDateFormatTest (resp. in MonthDateFormat) yesterday, but I didn't understand the code and looking at the git log didn't help either, so unfortunately I could not fix it.

The failure in ChartUtilTest.testFormatDouble should indeed be easy to fix (similar to ValueFormatterTest).

The failure in ChartUtilTest.testStringValue probably cannot be fixed (unless you fly with your test machine into the expected time zone).

Regarding the other two failures which occur in the CI build (see my comment from 2h ago), I have no idea.

Alexander, can you fix them?

[ruspl-afed]

Alexander, can you fix them?

both DocArchiveLockManagerTest and ContentCommandTest works for me locally

[ruspl-afed]

Now I have PropertyValueExceptionTest failing in server environment. It cannot find property definition for "bookmark" of TableItem. No idea why.

[xavierdpt]

On my side, I ran mvn install -nf and I see a large number of test failures.

I summed up all the test failures and errors here
https://gist.github.com/xavierdpt/93d198c2e684a5d330eb04b2c01e6a80

I think it would be quite difficult to fix all them at once, and it would be simpler to fix them one after another.

ChartUtilTest.testStringValue can be fixed by setting the default timezone in the unit test, I can make a PR for this one.

On a side note, the classes in com.ibm.icu.util.* should probably be migrated to java.util.* (Calendar, TimeZone, etc...), but this will be easier to do after all the unit tests have been fixed and a separated issue should be created to track this.

[wimjongman]

Closing. We are working on this story step by step and we have succeeded in reviving a great number of tests.