The genjvm from simn

constructors

We have to do the super-before-everything-else dance. Also have to make sure that the verifier is happy.

Last `case` does not need a goto

Reminder issue for me to fix that at some point. Control flow can just fall through from the last case.

Boolop generation isn't ideal

I don't know what I'm doing but I know it's not the correct way to handle this.

super.calls()

This currently doesn't compile:

class Base {
	function test() {
		trace("Base.test");
	}
}

class Main extends Base {
	static public function main() { }

	function new() { }

	override function test() {
		trace("Main.test");
		super.test();
	}
}

I have to check which instruction to use for this. I think it's invokespecial.

Signature vs. native type parameters

This currently errors:

import java.jvm.annotation.EnumReflectionInformation;

class Main {
	static public function main() {
		var exprDefClass = Type.resolveClass("haxe.macro.ExprDef");
		var exprDefNative = exprDefClass.native(); // same value but typed as java.lang.Class
		var annotation:java.lang.annotation.Annotation = exprDefNative.getAnnotation(cast EnumReflectionInformation);
	}
}

java.lang.NoSuchMethodError: java.lang.Class.getAnnotation(Ljava/lang/Class;)Ljava/lang/Object;

I think it's because of the return type being `Ljava/lang/Object;``.

The Haxe dump looks fine:

[Var annotation(1388):java.lang.annotation.Annotation]
	[Call:java.lang.annotation.Annotation]
		[Field:(param1 : java.lang.Class<java.lang.annotation.Annotation>) -> java.lang.annotation.Annotation]
			[Local exprDefNative(1387):java.lang.Class<Dynamic>:java.lang.Class<Dynamic>]
			[FInstance:(param1 : java.lang.Class<java.lang.annotation.Annotation>) -> java.lang.annotation.Annotation]
				java.lang.Class<Dynamic>
				getAnnotation:(param1 : java.lang.Class<getAnnotation.A>) -> getAnnotation.A
		[Cast:java.lang.Class<java.lang.annotation.Annotation>] [TypeExpr haxe.jvm.annotation.EnumReflectionInformation:Class<java.jvm.annotation.EnumReflectionInformation>]

I have to check that the generator uses the correct type for this.

StringBuf.add vs. Int

This broke in my overload changes:

class Main {
	static public function main() {
		var buf = new StringBuf();
		buf.add(12);
		trace(buf.toString());
	}
}

I really want to blame the StringBuf.add implementation for that though:

	public function add<T>( x : T ) : Void {
		if (Std.is(x, Int))
		{
			var x:Int = cast x;
			var xd:Dynamic = x;
			b.append(xd);
		} else {
			b.append(x);
		}
	}

[quest] Haxe unit test compilation

src/unit/TestOverloads.hx:194: characters 6-11 : Could not find overload

eq(child.someField(ChildJava2), 51);

No idea why this is so special.

src/unit/issues/Issue3024.hx:20: characters 2-32 : Recursive signature?

That's #30

src/unit/issues/Issue7335.hx:8: characters 17-25 : Unuspported unop on Ljava/lang/Object;

That's #29 (comment)

Getting close!

Std.hx

is (I think this can just call Jvm.instanceof, but the genjava version does some other stuff too)
string
int (this just casts which emits a d2i instruction and returns the result)
parseInt (the genjava version is huge... surely there's something in the Java API for this)
parseFloat (same)
instance (uuhm... how did this one work again...?)
random (Math.random + cast I guess)

operands in the wrong order when comparing with zero

class Main {
	static function main() {
		var a = 1;
		var x = 0 < a;
	}
}

  public static void main(java.lang.String[]);
    Code:
       0: iconst_1
       1: istore_1
       2: iload_1
       3: ifge          10
       6: iconst_1
       7: goto          11
      10: iconst_0
      11: istore_2
      12: return

    public static void main(String[] args) {
        int a = 1;
        boolean x = a < 0;
    }

interesting, even tho instruction is ifge, intellij decompiler says <...

Assertion on nested native arrays

class Main {
	static public function main() {
		var a = new java.NativeArray<java.NativeArray<Int>>(0);
	}
}

This hits the assertion in NativeArray.create. I have to check what we're supposed to do with this.

Method name clashes

Since all classes implicitly inherit from java.lang.Object at run-time, we have to deal with name clashes. The following methods are final and have to be escaped:

Class<?> getClass()
void notify()
void notifyAll()
void wait(long timeout)
void wait(long timeout, int nanos)
void wait()

Note that the signature is relevant and we only have to escape if a method matches in both name and signature.

The following methods are not final and might be overridden:

int hashCode()
boolean equals(Object obj)
Object clone()
String toString()
void finalize()

The question here is what we want to do with those. We can treat toString as a separate case because it's defined in Haxe with the same semantics, so we always want to allow overriding it in classes. For the others, there are two options:

Escape by default, allow to not-escape (and thus override) with metadata.
Don't escape (and thus override) by default, allow to to escape with metadata.

Not sure which option is better. From a pragmatic point of view I'm leaning towards 2.

As for how to handle the escaping, I propose that we prefix with _hx_ and make sure reflection respects that. This should be a matter of stripping a leading _hx_ in some places, and adding it in some others. That's pretty annoying, but I don't think we can avoid it.

Top-down-casting

In #1 (comment) I've found a case where casting is not ideal. Instead of casting to the expression type Haxe gives us, it would be better to cast to the type we actually expect.

This requires piping the expected type through the texpr-to-instruction converted. We already have RValue to track if we're in a value place or not, so I think we can simply attach a jsignature option argument to that and use it where appropriate.

Exceptions

The straightforward approach is to always catch Throwable and do a series of Std.is checks. I wonder if it could be worth to create exception wrappers classes for thrown classes that are not aliased (no parent class or interface). This way we could have StringException and others and directly catch them through the JVM instruction.

Reflection

It has to be done eventually. This requires generating some auxiliary type information in a way that it doesn't bother the run-time too much. I don't really want to think about this though.

Missing super call on field-init-generated constructor

class Base {
	public function new() { }
}

class Main extends Base {
	static public function main() {
		new Main();
	}

	var s:String = "foo";
}

java.lang.VerifyError: Constructor must call super() or this() before return
Exception Details:
  Location:
    Main.<init>()V @6: return
  Reason:
    Error exists in the bytecode

I probably shouldn't use add_field_inits at all anyway...

OpAdd

All the variants of + have to be supported (with conversion to string).

Emojis

class Main {
	static public function main() {
		Sys.println("😂");
	}
}

java.lang.ClassFormatError: Illegal UTF8 string in constant pool in class file Main

I thought I didn't have to worry about this kind of stuff but... looks like I do.

javac synchronized equivalent

There are two instructions for this:

https://docs.oracle.com/javase/specs/jvms/se7/html/jvms-6.html#jvms-6.5.monitorenter
https://docs.oracle.com/javase/specs/jvms/se7/html/jvms-6.html#jvms-6.5.monitorexit

Sounds surprisingly straightforward actually.

Type.hx

switch on string

Let's do something smart here.

[quest] Haxe unit test verification endgame

This commit shows what currently doesn't verify:

TestJava.hx

This sometimes fails with this error:

Exception in thread "main" java.lang.NoClassDefFoundError: haxe/test/Base$InnerClass
        at unit.TestMain.main(src/unit/TestMain.hx:81)
Caused by: java.lang.ClassNotFoundException: haxe.test.Base$InnerClass
        at java.net.URLClassLoader.findClass(Unknown Source)
        at java.lang.ClassLoader.loadClass(Unknown Source)
        at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
        at java.lang.ClassLoader.loadClass(Unknown Source)

And sometimes with this:

Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
Exception Details:
  Location:
    unit/TestJava.testIssue2964()V @313: checkcast
  Reason:
    Type integer (current frame, stack[2]) is not assignable to 'java/lang/Object'
  Current Frame:
    bci: @313
    flags: { }
    locals: { 'unit/TestJava', 'haxe/test/MyClass', 'java/lang/Boolean', null }
    stack: { 'unit/TestJava', 'haxe/test/MyClass', integer }

The latter is a cast(1, Integer) case which I don't think should be a thing... but we should be able to support it regardless.

The first one I have no idea.

Issue2281.hx

Something still going wrong with crement ops. This is likely missing a cast to some expected type.

Issue3118.hx

Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
Exception Details:
  Location:
    unit/issues/Issue3118.test()V @27: invokevirtual
  Reason:
    Type 'java/lang/Object' (current frame, stack[0]) is not assignable to 'Array'
  Current Frame:
    bci: @27
    flags: { }
    locals: { 'unit/issues/Issue3118', 'unit/issues/Final' }
    stack: { 'java/lang/Object', 'java/lang/Integer' }

Another missing cast, probably after field access.

Issue4002.hx

Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
Exception Details:
  Location:
    unit/issues/Issue4002.hx_closure$0(Ljava/lang/Object;)I @1: getfield
  Reason:
    Type 'java/lang/Object' (current frame, stack[0]) is not assignable to 'unit/issues/_Issue4002/A'
  Current Frame:
    bci: @1
    flags: { }
    locals: { 'java/lang/Object' }
    stack: { 'java/lang/Object' }

That could be bad, or it could be another missing cast.

Issue5124.hx

Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
Exception Details:
  Location:
    unit/issues/Issue5124.<init>()V @4: invokevirtual
  Reason:
    Type uninitializedThis (current frame, stack[2]) is not assignable to 'java/lang/Object'
  Current Frame:
    bci: @4
    flags: { flagThisUninit }
    locals: { uninitializedThis }
    stack: { uninitializedThis, 'java/lang/invoke/MethodHandle', uninitializedThis }

Something wrong with constructor logic vs. field inits. Oh no...

the rest

#42

`@:strict`

I just found out from the unit tests that genjava supports Java annotations through @:strict:

@:strict(MyClass_MyAnnotation({ author:"author", currentRevision: 2 }))
public function testAnnotations()

It's probably not worth inventing something new here, so we should just adhere to this. Requires writing a function that transforms expressions to https://docs.oracle.com/javase/specs/jvms/se7/html/jvms-4.html#jvms-4.7.16.1.

P.S.: This is the last error in the unit tests before we can move on to failures.

Enum value printing

Enum values are currently not printed properly. This is related to #6 and we should make sure to generate the necessary information in a way that doesn't waste memory unnecessarily.

Generated constructors

typeloadFields.ml has this:

		(* add_constructor does not deal with overloads correctly *)
		if not ctx.com.config.pf_overload then TypeloadFunction.add_constructor ctx c cctx.force_constructor p;

That means that constructors are not auto-generated, which leads to problems. This is handles for genjava through overloadingConstructor.ml, but that thing is pretty gencommon.

I'll probably just enable add_constructor for the time being because constructors with wrong overloads are slightly better than no constructors.

haxe.CallStack

yeah...

Branching in constructor arguments doesn't work

class Main {
	static public function main() {
		new Main(f ? 1 : 2);
	}

	function new(_) {}

	static var f = false;
}

java.lang.VerifyError: Inconsistent stackmap frames at branch target 13
Exception Details:
  Location:
    Main.main([Ljava/lang/String;)V @13: iconst_2
  Reason:
    Type uninitialized 0 (current frame, stack[0]) is not assignable to 'Main' (stack map, stack[0])

The problem is that in the JVM a new instruction causes the verification type of the stack item to be Uninitialized until the invokespecial instruction happens. My own stack handling doesn't respect that and sets the type to the actual verification type immediately.

I'll have to see how to handle this because my entire stack handling is based on signatures, and there's no signature for "uninitialized". I might have to encode this in the signature structure. There's TObjectInner which is unused and I don't think could appear on the stack, so that would be an option.

Exhaustive switches on compile-time-finite values

This is currently not generated correctly:

enum E {
	A;
	B;
}

class Main {
	static public function main() {
		switch (a) {
			case A:
				return 1;
			case B:
				return 2;
		}
	}

	static var a = A;
}

I think that in cases like this we should treat the last case as default to make JVM happy.

Something is wrong with closures

The unit tests throw this at me:

Error: Expr function(c2:Int) {
        return a26 + b17 + c2;
}
Stack error
        ops      :
                 121 ldc_w 329 [(Lunit/TestArrowFunctions$2;I)I]
                 121   new 318 [, (Lunit/TestArrowFunctions$2;I)I]
                 121       dup [, , (Lunit/TestArrowFunctions$2;I)I]
                 121   aload_0 [Ljava/lang/Integer;, , , (Lunit/TestArrowFunctions$2;I)I]
                 121   iload_1 [I, Ljava/lang/Integer;, , , (Lunit/TestArrowFunctions$2;I)I]
             line: 121
        operation: invokespecial unit/TestArrowFunctions$2.<init>:(II)V
        expected : [I, I, Lunit/TestArrowFunctions$2;]
        actual   : [I, Ljava/lang/Integer;, , , (Lunit/TestArrowFunctions$2;I)I]
Method unit.TestArrowFunctions.testSyntax

This probably means that there's some disconnect between types somewhere. Relying on v_type is bad because the run-time type could be something else if captured.

std/java

The standard library is a bit of a mess. There's some __java__ which have to be replaced, as well as @:functionCode. I'm not sure how to go about this without making a huge #if-fest.

Invalid boxed primitve comparison

class Main {
	static function main() {
		var b:Null<Bool> = null;
		trace(b == false);

		var i:Null<Int> = null;
		trace(i == 0);
	}
}

traces true x2, however it must be false.
This is because we call Jvm.toBoolean/toInt which converts null to default values, but we should not do that for comparisons.

Stack overflow on recursive signatures

E.g.

typedef T = haxe.ds.Vector<T -> Void>;

class Main {
	static public function main() {

	}

	static function x(t:T) { }
}

This probably affects all compound signature types.

enums again

Something I didn't realize is that genjava allows using java-native enums as Haxe enums. The only way to support that would be to make our enums extend java.lang.Enum so there's a common API.

This isn't entirely free because the java.lang.Enum constructor wants a string of the enum name. It would also mean dropping (parts of) the annotation-based reflection approach.

Box-variance

HaxeFoundation/haxe#4342 is still a big problem. This code causes a NullPointerException:

class Main {
    static function main() {
        var n:Null<Int> = null;
        var closure:Null<Int>->Void = printA;
        closure(n);
    }

    static function printA(a:Int) {
        trace(a);
    }
}

This has to be solved when we detect the assignment of printA to closure, but I'm not sure what exactly we should do here. The notion of "adapter methods" has come up and I think MethodHandle supports that somehow, but my brain can't handle any more Java documentation for the moment.

Note that this fails on genjava too.

Reflect.hx

Regarding isObject: The problem with this is that we always box primitive values when assigning to Dynamic, as is the case here. This means that isObject(12) and isObject((12 : Null<Int>)) are indistinguishable within the function body.

The genjava implementation solves this by doing a whole bunch of Std.is checks against java.lang.Number, java.lang.Boolean etc. The question here is if we're talking about "object" in the run-time sense or the Haxe-sense.

API patches (like String methods)

Ideally, we want to support both "foo".substr and ("foo" : Dynamic).substr. There are several options for the first one, but the second one is a bit harder.

An idea would be to collect all these methods at compile-time and create a map from the accessed field to the implementation field. Something along the lines of:

["String" => [ "substr" => ["StringExt", "substr"], "substring" => ["StringExt", "substr"]]]

The dynamic lookup can then (either eagerly or on failure) index into that map with the type and field name, and rewrite the field accordingly.

This could be done explicitly through some @:jvm.reroute("StringExt", "substr") metadata on the field, or we do it implicitly for all extern fields that have expressions.

Debugging

We should check what we need to make our code debuggable properly. There's good chance that this already works quite well given how nice the decompiled code looks.

I don't know how to use Java debuggers though. :P

Bytecode-level tests

So the reason I worked on format.jvm.Reader was because I wanted to add support for bytecode-level tests in order to catch regressions there. The basic idea is to get the jvm-tests to read their own .jar file and then compare it to some specific input.

This requires writing a printer for https://github.com/Simn/format/blob/jvm/format/jvm/Data.hx#L47 which fetches items from the constant pool. Once we have that, we can come up with some nice way to define the expected bytecode in-line. I imagine a macro like this would work nicely:

		bytecodeEquals(<>
			iconst_0
			istore_1
			iinc 1, 1
		</>, {
			var i = 0;
			i++;
		});

Comparison operator flipping

The generator currently flips some comparison operators, which is not safe to do because of NaN stuff.

The reason it does that is because I initially got mega-confused by the if instructions. They branch on comparison with 0, so if_eq with 0/false on the stack branches.

VerifyError: Method expects a return value

class Main {
	static function main() {
		f();
	}

	static function f():Int {
		while (true) {
			// throw "oops";
			return 1;
		}
	}
}

  public static int f();
    Code:
       0: iconst_1
       1: ifeq          6
       4: iconst_1
       5: ireturn
       6: return

looks like we always add a final return in cases like this, but we really should generate a proper instruction based on the return type

[boss battle] invokedynamic

We really have to understand invokedynamic. The idea would be to use that whenever we call something but aren't sure what we're calling.

My concern is that this is not enough to cover our semantics. For every unknown.call() there's also (unknown.call)(). However, maybe these are just cascading problems: The dynamic field access would return "something", and then the invokedynamic would figure out how to call it.

Relevant documentation:

What I understood so far is that this is a two-step process:

When the JVM finds invokedynamic while jitting, it calls a bootstrap method which returns something (I think CallSite?) that is associated with the instruction.
At run-time, that returned something is called, somehow.

haxe-checkstyle failures

    FAIL: massive.munit.AssertionException: Value [null] was not equal to expected value [eol] at checkstyle.detect.DetectCodingStyleTest#testDetectOperatorWrap (486)
    FAIL: massive.munit.AssertionException: Value [null] was not equal to expected value [^[A-Z][A-Z0-9]*(_[A-Z0-9_]+)*$] at checkstyle.detect.DetectCodingStyleTest#testDetectConstantName (322)
    FAIL: massive.munit.AssertionException: Value [null] was not equal to expected value [eol] at checkstyle.detect.DetectCodingStyleTest#testDetectSeparatorWrap (506)

@AlexHaxe's hint:

might it be an issue, that all three of these checks are child classes and the failing property comes from base class?
at least that's what all three have in common and (I think) all others don't

Increment issues

class Main {
	static public function main() {
		x++;
	}

	static var x:Null<Int> = 0;
}

Fails in the generator with:

Stack error
        ops      :
                   3                  getstatic Main.x:Ljava/lang/Integer; [Ljava/lang/Integer;]
                   3 invokestatic haxe/jvm/Jvm.toInt:(Ljava/lang/Object;)I [I]
                   3                                              iconst_1 [I, I]
                   3                                                  iadd [I]
             line: 3
        operation: putstatic Main.x:Ljava/lang/Integer;
        expected : [Ljava/lang/Integer;]
        actual   : [I]
Method Main.main

I think it's because the TUnop itself is typed as Int:

			[Unop:Int]
				++
				Postfix
				[Field:Null<Int>]
					[TypeExpr Main:Class<Main>]
					[FStatic:Null<Int>]
						Main
						x:Null<Int>

interface fields

The JVM doesn't allow non-method non-statics on interfaces. Is there something clever we can do here, or do we need fully dynamic lookup?

TObjectDecl

We need a good way to generate anonymous objects.

Haxe extern vs. native signatures

Math.floor is a good example:

class Main {
	@:analyzer(ignore)
	static public function main() {
		var d:Dynamic = Math;
		Sys.println(d.floor(f)); // 12.0
		Sys.println(Math.floor(f)); // Exception in thread "main" java.lang.NoSuchMethodError: java.lang.Math.floor(D)I
	}

	static var f = 12.5;
}

For the first one, the result has to be cast to Int. We cannot detect this at compile-time in the general case, so the dynamic resolution has to deal with this. This may be a good use-case for annotations.

In the second one we have to patch the signature at compile-time, but still make sure the result is cast back to Int. This is hardcoded in genjava, but I would like look into having a more flexible solution (metadata?).

Improve StackMapTable

Currently, all stack maps are generated as StackFull because the documentation went a bit over my head. We should

read the documentation until we understand it and
figure out the diff between the last and current stack frame and use one of the other options.

NativeArray wrapping

We currently always create java.NativeArray (should move that btw...) as a reference type and wrap the values, mostly because "that's easy". We should use the array specializations for basic types here and generate the instructions accordingly.

The problem with this is that Haxe's type system is retarded and allow this kind of stuff:

class Main {
	static public function main() {
		var ia = new java.NativeArray<Int>(1);
		var fa = new java.NativeArray<Float>(1);
		var da = new java.NativeArray<Dynamic>(1);
		da = ia;
		da = fa;
		f(ia);
		f(fa);
	}

	static function f<T>(_:java.NativeArray<T>) { }
}

I think we have three options:

Introduce some @:absolutelyNoVariance metadata in Haxe core which does not allow this. That's somewhat straightforward for the assignment to Dynamic, but could be a huge mess for type parameters. I think we would basically have to disable type inference for this, like for the old @:generic implementation.
Generate reference arrays by default (like now), but do some escape analysis to detect if the array is never assigned and aliased. In these cases we can safely use the unwrapped types. This is quite similar to the inline constructor analysis, but less local because it should work for fields as well.
Don't care and simply let the JVM verifier fail.

We'll start with 3 either way, but the question is how much nicer we can/should make this and with what priority...

Closures

Both field and local closures don't work at the moment.

Interfaces vs. type parameters

interface I<T> {
	public function get(t:T):Void;
}

class Main implements I<String> {
	static public function main() {
		var i:I<String> = new Main();
		i.get("foo");
	}

	function new() { }

	public function get(s:String) { }
}

Exception in thread "main" java.lang.AbstractMethodError: Method Main.get(Ljava/lang/Object;)V is abstract
        at Main.get(src/Main.hx)
        at Main.main(src/Main.hx:8)

The problem is this invokeinterface instruction:

14: invokeinterface #17,  2           // InterfaceMethod I.get:(Ljava/lang/Object;)V

I wonder if this is supposed to use the concrete type...

simn / genjvm Goto Github PK

genjvm's People

Contributors

Stargazers

Watchers

Forkers

genjvm's Issues

Recommend Projects

Recommend Topics

Recommend Org