thaliproject / jxcore Goto Github PK
View Code? Open in Web Editor NEWThis project forked from jxcore/jxcore
Evented IO for Chakra, SpiderMonkey & V8 JavaScript
Home Page: http://jxcore.com
License: Other
This project forked from jxcore/jxcore
Evented IO for Chakra, SpiderMonkey & V8 JavaScript
Home Page: http://jxcore.com
License: Other
We currently have a set of scripts to build all the components needed to do a jxcore release but is still not a fully automated environment.
We need script(s) for jxcore-cordova too.
Running the Thali coordinated test testLocalSeqManager.js in a loop will eventually make the SM's GC crash.
In debug mode the following assert fails:
Assertion failure: (ptrBits & 0x7) == 0, at ../deps/mozjs/incs/js/Value.h:837
Currently the proof of concept to run jxcore tests using a jxcore-cordova app doesn't create a report log on the host machine, it only shows the results in the jxcore-cordova app running on the simulator.
Can we use Travis for all there desktop platforms? Is there a free service we can use for Mobile for iOS and Android (since we are an open source project)?
Marek noticed that if he uses 'jx install' he doesn't have to set various environment variables with our paths and such but with 'jx npm install' he does. Did we set the paths for one but not the other?
The 'externalized paths' commits in our repo haven't been pushed to mainline jxcore because we use our own URLs and in order to push to mainline jxcore we would need to change them. That, in turn, would cause conflicts when we merge back from mainline jxcore.
In the Node test suite there are tests that were failing with the last release of JXcore from Nubisa (e.g. we didn't make them fail) as well as a bunch of tests that are being skipped. Unfortunately the CI system used by Nubisa is very old and it doesn't output a log and it doesn't know how to count skipped versus passing tests. So we need to go investigate which tests are being skipped and see if we want to actually run them and we also need to investigate the known failures to see if we want to fix them.
The investigation of #22 has shown that SM in some cases doesn't properly catch exceptions.
At this stage we don't know if this is a bug in SM or if JXcore doesn't initialize the SM engine properly.
We have a workaround to catch the exception in JXcore, that should be enough to ship New Infra, but this need further investigation for V1.
ios_compile.sh and android_compile.sh don't accept a --debug
flag as input parameter.
Adding the --debug
flag in the scripts when configure
is called doesn't work.
We'll work around this for now but why if I just want to support PSK do I get an error saying "Missing PFX or certificate + private key."? I only want to support PSK connections, I don't want to support any certs.
Repo:
Expected Result: Happily running tests
Actual Result: Segmentation fault: 11
When I repeat the above but us the latest drop of JX with v8 everything works fine.
What's going on?
Taken from @enricogior's explanation:
Found a problem with the logic used by some tests to verify if the system's OpenSSL version is 1.0.0 or greater. Some tests were skipped erroneously, the fix is simple but it takes time and we have more urgent issues.
The code that doesn't work is like this one:
https://github.com/thaliproject/jxcore/blob/master/test/simple/test-tls-psk-client.js
function checkOpenSSL() {
exec('openssl version', function(err, data) {;
if (err) throw err;
if (/OpenSSL 0\./.test(data)) {
console.error('Skipping due to old OpenSSL version.');
return false;
}
else {
return true;
}
});
}
if (! checkOpenSSL()){
console.error("Skipping because OpenSSL version < 1.0.0.");
process.exit(0)
}
if (! checkOpenSSL())
always succeeds, since checkOpenSSL()
is not returning the result of the 'exec' command.
That test fails now that we run it, so beside the fix to verify the OpenSSL version we need to investigate why it fails.
Did we get all of our changes accepted to JXcore?
A proof of concept has been tested on the iOS simulator.
test/simple/test-stream-pipe-multi.js
fails on macOS when run using the test/test.js
script.
It passes when executed directly jx test/simple/test-stream-pipe-multi.js
.
On macOS and Linux these two OpenSSL-related tests are failing:
test/pummel/test-https-ci-reneg-attack.js
test/pummel/test-tls-ci-reneg-attack.js
They fail in the current master branch and also in the private branch where we are testing OpenSSL 1.0.1t.
When we generate npmjx311 (this lives in jxcore\tools\npmjx) we actually have the path /Users/cicoriasmbp13/ all over the place. This means that folks who use JXcore NPM have to create that literal path in their system. If nothing else that is a possible attack on Windows since it requires creating a directory under Users which is just a bad idea.
I believe this is all being set in deps/npm/lib/config/defaults.js as a result of the value of setting the home variable.
Now the obvious question is - o.k. so what path should it be set too? It's pretty obvious that it wants to be set to the user's local home directory. I'm not sure why this isn't being set automatically.
I'm probably sending you on a wild goose chase but this smells like a bug in https://github.com/thaliproject/jxcore/blob/master/tools/npmjx/index.js#L156, that function should figure out where the user's local directory is and update the npmrc to change the paths.
Although the more I think about it the more likely I think it is that putting npmrc into the tar.gz is the actual bug. So maybe we just change the packager logic so that file isn't included?
I noticed in the logs:
[email protected] (git+https://github.com/ktrzeciaknubisa/jxtools.git#644cf31ea259cb65a97e5c3ed5ea1236dba298a3)
So do we depend in JXC on some random depot in GitHub?
The work consits of:
The following script shows the bug in the process.sendToMain()
method.
jxcore.tasks.setThreadCount(2);
var count = 0;
function runTest() {
// make sure there are no tasks still running
jxcore.tasks.killThread(0)
jxcore.tasks.killThread(1)
count++;
if (count > 5) return;
console.log("---- Count " + count + " -----");
var subTask = function (param) {
console.log("sending msg to main");
process.sendToMain({result: "msg " + param.count});
return 'ok';
};
jxcore.tasks.on("message", function(threadId, obj) {
console.log("thread " + threadId + ": " + obj.result);
});
jxcore.tasks.addTask(subTask, {count: count}, function (err, result) {
console.log("subTask done");
runTest();
});
}
runTest();
The espected result should be:
---- Count 1 -----
sending msg to main
thread 0: msg 1
subTask done
---- Count 2 -----
sending msg to main
thread 0: msg 2
subTask done
---- Count 3 -----
sending msg to main
thread 0: msg 3
subTask done
---- Count 4 -----
sending msg to main
thread 0: msg 4
subTask done
Instead the result is:
---- Count 1 -----
sending msg to main
thread 0: msg 1
subTask done
---- Count 2 -----
sending msg to main
thread 0: msg 2
thread 0: msg 2 <---- unexpected
subTask done
---- Count 3 -----
sending msg to main
thread 0: msg 3
thread 0: msg 3 <---- unexpected
thread 0: msg 3 <---- unexpected
subTask done
---- Count 4 -----
sending msg to main
thread 0: msg 4
thread 0: msg 4 <---- unexpected
thread 0: msg 4 <---- unexpected
thread 0: msg 4 <---- unexpected
subTask done
Build and upload the jxcore and jxcore-cordova bits.
Target arch x64
CXX(target) Debug/obj.target/fs-ext/fs-ext.o
In file included from ../fs-ext.cc:20:
In file included from /Users/hiqouser/.node-gyp/jxb311/src/node.h:46:
In file included from /Users/hiqouser/.node-gyp/jxb311/src/jx/Proxy/JSEngine.h:51:
In file included from /Users/hiqouser/.node-gyp/jxb311/src/jx/Proxy/Mozilla_340/JXString.h:19:
In file included from /Users/hiqouser/.node-gyp/jxb311/src/jx/Proxy/Mozilla_340/MozJS/MozJS.h:6:
In file included from /Users/hiqouser/.node-gyp/jxb311/src/jx/Proxy/Mozilla_340/MozJS/Isolate.h:17:
In file included from /Users/hiqouser/.node-gyp/jxb311/src/jsapi.h:12:
In file included from /Users/hiqouser/.node-gyp/jxb311/src/mozilla/FloatingPoint.h:12:
In file included from /Users/hiqouser/.node-gyp/jxb311/src/mozilla/Assertions.h:16:
In file included from /Users/hiqouser/.node-gyp/jxb311/src/mozilla/Attributes.h:12:
In file included from /Users/hiqouser/.node-gyp/jxb311/src/mozilla/Compiler.h:12:
/Users/hiqouser/.node-gyp/jxb311/src/mozilla/Char16.h:173:15: error: unknown type name 'char16_t'
typedef const char16_t* char16ptr_t;
I am not familiar with osx. It looks like it is using custom apple clang version. I am not sure but it looks like this clang doesn't want to support char16_t
11 tests fail on Android Nougat because binding to a file pipe doesn't work.
The code that handles the pipe binding is in libuv:
https://github.com/jxcore/jxcore/blob/166784821f30e62503f04eb267ea26cf7c2c58d7/deps/uv/src/win/pipe.c#L363
This is a sub task of #16
I've tried to update trevis.xml to use the latest XCode version previously, but it was failing because the log size exceeded the max length, the solution maybe to redirect the stderr and/or the stdout to a log file. I need to verify if we can later access the log file for investigation if needed.
The test/simple/test-child-process-fork-net2.js test failed on Travis during a build, we also observed it on some Linux VMs, it occurs randomly.
We run Spidermonkey on both iOS and Android. So does it make any sense for us to build with v8 for any of the desktop builds? Isn't it just going to hide bugs? Maybe we shouldn't build for v8 at all so no one accidentally installs it?
Right now we hardcode in paths pointing to our distribution points during the build. That would be what is known as 'bad'. :) We need to submit a PR to JXcore that lets us set the paths in some kind of external configuration file so that different people who download JXcore can choose to point at whatever location they want. I found at least three files in JXcore forexample that point directly at azureedge.net. But that is an endpoint owned by Microsoft. It's not something that should be in the general JXcore repo. Instead those values should point at configuration files that someone building JXcore can set up and then do a compile and produce an output that points where ever they want.
By externalizing this we can keep the JXcore build agonist from Microsoft which is what we want. We are absolutely NOT promising the world to host JXcore! :)
Right now if we ask JXcore (especially when we are using spidermonkey on desktop) to build an optional dependency it can't handle (think bufferutil that is brought in by socket.io in TestServer) then it fails the whole build. In perfect land it would detect that the node-gyp build failed and it would see that the dependency is optional and realize that the failure really doesn't matter. So it would clean up any entries in node_modules and continue.
Error: EACCES, link '/data/local/tmp/jxcore/test/fixtures/cycles/root.js' (node.js 138:7)
hTe symbolic link is created correctly and it works if tested in the shell.
Please, see the log from samsung-SM-A300FU and search for Fatal signal 11
.
You can also check the logs from some other devices that failed with Fatal signal 11
:
samsung-SM-A500FU
samsung-SM-G935F
AssertionError: "" == "asdfasdf" ( 0:0)
at equal (assert.js:136:11)
at grandparent/< (/data/local/tmp/jxcore/test/simple/test-child-process-stdio-inherit.js:30:5)
at emit (events.js
child.stdout.on('data'...) doesn't get called because the child process dies before the event is emitted or captured.
How npmjx works is not documented.
There are actually two npmjx modules, one is just a proxy module that points to the real npmjx module.
When you run the "jx npm install" command, jx.exe executes the lib/jx/_jx_install.js script (that is embedded in jx.exe) that downloads the proxy module using a hardcoded url.
The proxy module in turn downloads the real npmjx module (using another hardcoded url). The proxy module contains the tar.gz module that is required to expand the real npmjx module.
There are some problems with the current way this mechanism work:
We tried to use Travis with the latest XCode but it didn't work.
Let's find an alternative service and if none will fit our need let's try to setup Jenkins on our own machine.
function childProcessExecPromise(commandString, currentWorkingDirectory) {
return new Promise(function (resolve, reject) {
var commandSplit = commandString.split(' ');
var command = commandSplit.shift();
var stdErrData = false;
var theProcess = spawn(command, commandSplit,
{ cwd: currentWorkingDirectory});
theProcess.stdout.on('data', function (data) {
console.log('' + data);
});
theProcess.stderr.on('data', function (data) {
//stdErrData = true;
console.log('' + data);
});
theProcess.on('close', function (code) {
if (code !== 0) {
return reject(code);
}
if (stdErrData) {
return reject(-1);
}
return resolve();
});
});
If I uncomment stdErrData = true; then JXcore faills.
The test/test.js script's main purpose is to run tests on platform that don't have python, such iOS and Android.
On desktop it should also work and return the same results returned by running the tests using the python script tools/test.py, but it currently doesn't.
Several tests fails on macOS when run using test.js.
Test test-regress-GH-4015.js fails on Ubuntu 16.04 using SpiderMonkey.
The test forces a stack overflow and then checks the error signal.
The test passes on Travis running Ubuntu 12.04 and it fails on our dev machines running Ubuntu 16.04.
In our environment the err.signal is set to 'null' instead of the expected 'SIGSEGV'.
=== release test-regress-GH-4015.js ===
Path: _auto_simple_single/test-regress-GH-4015.js
AssertionError: false == true ( 0:0)
at ok (assert.js:126:5)
at /home/enrico/github/thaliproject/jxcore/test/_auto_simple_single/test-regress-GH-4015.js:21:5
at exithandler (child_process.js:
SyntaxError: JSON.parse: unexpected end of data at line 1 column 1 of the JSON data (/data/local/tmp/jxcore/test/simple/test-child-process-fork-exec-argv.js 28:21)
at (/data/local/tmp/jxcore/test/simple/test-child-process-fork-exec-argv.js:28:22)
at emit (events.js:85:1)
at $0h/this._handle.onexit (child_process.js:826:1)
The fork process doesn't start, maybe because the child process dies first.
test/simple/test-timer-match.js fails randomly when verifying the first timer:
if (time == last_time) {
throw new Error("Fail!");
}
We are getting the following error during tests:
jx(1068,0x105751000) malloc: *** error for object 0xa6e00786a2e65: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
To repro:
It will core dump with the error above pretty quickly.
Some tests assume that openssl is not installed on Windows and return true (success).
We should change the current behavior and adopt the same behavior of other tests that first check if openssl is installed.
Adding a new method to the system Array type causes unpredictable errors in jxcore.
The error is not occuring immediately and the error message that may appear doesn't provide any indication that the problem is caused by adding a new method to the Array type.
Other system types may also be effected by the same issue.
Investigate the failing tests based on platforms priority and open specific issues based on the investigation results.
Currently ios_compile.sh and android_compile.sh have the hardcoded -j 2
flag, it would be better to pass the flag value as input parameter to the script in order to utilize the full CPU power.
A regression was introduced with the latest changes.
Travis uses Ubuntu 12.04, we tested on macOS and Linux 16.04, so this is something we didn't encounter and we will have to investigate. It may be caused by an old version of OpenSSL on the Travis machine. The test failed should be able to detect it and skip.
=== release test-tls-psk-client.js ===
Path: simple/test-tls-psk-client.js
exists
Using default temp DH parameters
Using default temp ECDH parameters
ACCEPT
client connected
secure error: Error: socket hang up
ERROR
47323619255072:error:140F2044:SSL routines:SSL3_SEND_SERVER_HELLO:internal error:s3_srvr.c:1495:
client close
killed
WAIT-ACCEPT
AssertionError: 0 == -1 ( 0:0)
at equal (assert.js:136:27)
at /home/travis/build/thaliproject/jxcore/test/simple/test-tls-psk-client.js:212:5
at emit (events.js
Command: out/Release/jx /home/travis/build/thaliproject/jxcore/test/simple/test-tls-psk-client.js
[02:58|% 100|+ 624|- 1|- 16]: Done
make: *** [test-simple] Error 1
We have a timer failure. I don't know why and I ran out of time today to investigate. Since this blocks everything please sync to the latest vNext_yarong_417_812, investigate and repair the issue.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.