dworkin/lpc-ext

Extension modules for Hydra and DGD

lpc-ext's Issues

Communication with spawned child doesn't quite work on Windows

For some reason, the child process (for instance, the JIT compiler) fails to read from the pipe after a while. Read returns 0, even though the pipe is still open on the other side and new things are being written to it.

JIT clang crash

I've been able to create a relatively small code example (from an originally much longer program) that causes clang to crash:

void t(string a)
{
    int loop;
    string b;

    if (sscanf(a, "%s", b)) {
        a = b;
    }

    loop = TRUE; /* crashes with or without this line */
    switch (a[0]) {
        case 0:
            while (loop) {
                switch (a[0]) {
                    case 0:
                        break;

                    default:
                        loop = FALSE;
                }
            }
    }
}

This actually causes clang to give a seg fault, so in my opinion, it seems like a clang issue rather than just a dgd issue (even if it's malformed, it shouldn't seg fault...), but you mentioned wanting such reports.

Let me know if you have trouble reproducing it, and I can try to give more details.

Alternate JIT compiler based on cc?

The current JIT compiler, as of version 1.2.5, runs clang as an independent program with IR input. As an alternative, it could instead produce C source code, to be compiled by the host's C compiler. This would make JIT compilation possible on platforms where clang is not supported.

All it would take is a replacement for jit/genclang.cpp.

JIT compiler can upgrade calls to builtin kfuns

The JIT compiler performs a more thorough analysis of the VM code than DGD does, and it could use the knowledge obtained to upgrade kfun calls, for example from KF_ADD to KF_ADD_INT if the arguments are certain to be integers.

JIT .ll code error

While testing the other issue, I noticed after cleaning the JIT cache, another program was failing. After a whole lot of trial and error, this seems to be the minimum function to cause the problem:

atomic void t()
{
    int a, b;

    for (a=1; a--;) {
        sscanf("3", "%d", b);
        catch(b = 3);
    }
}

This results in (after compiling twice and calling f() on it, for some reason?) an error much like the following:

cache/05/05843706b1e0936d55143796c287a630.ll:51:42: error: use of undefined value '%l1m34'
        %l1r24 = call i32 %c9(i8* %f, i8 2, i32 %l1m34)
                                                ^
1 error generated.

Clang crashes on IR input created by JIT compiler

Clang will crash compiling some IR. At present this can be seen when enabling the JIT compiler for SkotOS.

The cause is that the JIT compiler will put at most one entry for any predecessor block in a PHI node, while clang expects multiple entries in the case of a switch with more than one case jumping to the PHI node's block. The JIT compiler could be made to comply with clang's requirements by reversing commit fd41729.

The LLVM team is considering changing over to having at most a single PHI node entry per predecessor block, which would make the JIT compiler's current behaviour correct. The issue is tracked here.

'use of undefined value' with JIT

I see an error message in the DGD console using latest from dgd and lpc-ext repositories (only JIT extension is used) and my cloud-server fork (also occurs in base cloud-server repository with the same changes) using my mac. The message appears only once with a new cache.

I'm unable to try on another computer for a while.

Procedure:

• create cache/ in jit extension
• boot mud
• log in as admin, grant a user access
• log in as that user
• type foo
• message in DGD console

cache/ca/ca5af742ad29b8c4074b06877db045b8.ll:157:41: error: use of undefined value '%t10'
        %t12 = call double %c13(i8* %f, double %t10, double %t11)
                                               ^
1 error generated.

Files:

• https://github.com/november-kilo/cloud-server/blob/master/src/include/Foo.h
• https://github.com/november-kilo/cloud-server/blob/master/src/lib/Foo.c
• https://github.com/november-kilo/cloud-server/blob/master/src/usr/System/initd.c (line 62)
• https://github.com/november-kilo/cloud-server/blob/master/src/usr/System/obj/user.c (lines 9, 608, 955-957)

Compiler:

$ uname -s
Darwin
$ which cc
/usr/bin/cc
$ cc -v
Apple LLVM version 10.0.1 (clang-1001.0.46.4)
Target: x86_64-apple-darwin18.7.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
$ which g++
/usr/bin/g++
$ g++ -v
Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 10.0.1 (clang-1001.0.46.4)
Target: x86_64-apple-darwin18.7.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

Cannot use upgrade command with JIT module loaded

After adding the jit module, cloud-server's upgrade command does not complete and I have to kill the mud.

Save only relevant variables in caught code

The JIT compiler ensures that all local variables are preserved when an exception is thrown in caught code. However, it only needs to preserve those variables that could have been changed.

The relevant code is marked with XXX comments in typed.cpp and flow.cpp.

JIT error

Hi, I'm not sure if this belongs here or in lpc-ext, but with JIT enabled, this code will eventually fail:

#include <limits.h>

int test(int prev, int amt) {
    int next;
    next = prev + amt;
    if (next < prev) {
	    next = INT_MAX;
    }

    return next;
}

mixed t() {
    int i;

    if (test(1, INT_MAX) < 0) {
	    error("Should not be able to go negative");
    }
}

More precisely... the first time you compile it, the t() function above will work indefinitely (or at least seems to), but then if you compile it again, it will work once before failing. Likewise, if you destruct the object and recompile it, it works indefinitely again. Recompiling it will allow it to work one more time before failing again.

It never seems to fail without JIT.

Specifically, it's the next < prev condition that fails despite next in fact being less than prev. It seems to fail for numbers greater than the given 1 too, so I don't think it's some funny business around INT_MIN.

JIT fails to compile LPC

The JIT extension seems to fail to compile x = y[0] = z[0] when x/y/z are int or float types, respectively showing error messages like:

int error

cache/b8/b88ec35eebcbedf1df275572df197e79.ll:92:19: error: use of undefined value '%t15'
        %l1r30 = xor i32 %t15, 0
                         ^
1 error generated.

float error

cache/cf/cf3f625685caef596c6b995f7dec9a5c.ll:92:28: error: use of undefined value '%t15'
        %l1r35 = fadd fast double %t15, 0x0000000000000000
                                  ^
1 error generated.

Example code to reproduce this:

mixed test()
{
    mixed x;
    mixed *a, *b;

    a = allocate(11);
    b = allocate(21);
    
    x = a[0] = b[0];

    return ({ x, a, b });
}

Alternate JIT compiler based on libgccjit?

The current JIT compiler, as of version 1.2.5, makes use of clang. It might be nice to have a GCC-based alternative.

The problem here is that GCC is licensed under the GPLv3. This includes libgccjit.h, which must be #included. GPLv3 is compatible with DGD, but not with Hydra. Since the JIT compiler for DGD/Hydra is actually a separate program (released into the public domain), and only the object file that it produces is linked with Hydra at runtime, this may technically not be in conflict with the GPLv3, but the Free Software Foundation could well take the view that it goes against the spirit of the license, regardless.

Furthermore, the JIT-compiled code produced might also be subject to the GPLv3. This may actually be the cause that libgccjit has seen little uptake and is still in alpha.

For these reasons, I am not going to touch it, but perhaps someone else wants to have a go at it.

Tracing JIT compiler

The current JIT compiler, as of 1.2.5, compiles every LPC function separately. However, the function call overhead in LPC is quite large, so a tracing JIT compiler would generate much faster code.