smarr/somns

SOMns: A Newspeak for Concurrency Research

SOMns - A Simple Newspeak Implementation

Introduction

Newspeak is a dynamic, class-based, object-oriented language in the tradition of Smalltalk and Self. SOMns is an implementation of the Newspeak Specification Version 0.0.95 derived from the SOM (Simple Object Machine) class libraries, and based on TruffleSOM. It is implemented using the Truffle framework and runs on the JVM platform.

Truffle provides just-in-time compilation based on the Graal compiler, which enables SOMns to reach performance that is on par with state-of-the-art VMs for dynamic languages, including V8.

A simple Hello World program looks like:

class Hello usingPlatform: platform = (
  public main: platform args: args = (
    'Hello World!' println.
    ^ 0
  )
)

Obtaining and Running SOMns

The basic requirements for SOMns are a system with Java 9 or later, git, ant, and Python. Windows is currently not supported, but we test on Linux and macOS.

To checkout the code:

git clone https://github.com/smarr/SOMns.git

Then, SOMns can be build with Ant:

ant compile

Afterwards, the simple Hello World program is executed with:

./som core-lib/Hello.ns

To get an impression of the benefit o

For testing on the command line, the full command is ./som core-lib/Benchmarks/Harness.ns Mandelbrot 500 0 500

Additionally, there are JUnit tests and ant test for executing the test suite.

A more comprehensive setup guide is available in the docs folder and on ReadTheDocs.

Implementation and Deviations from the Specification

Compared to other Newspeaks and Smalltalks, it is completely file-based and does not have support for images. Instead of using customary bytecodes, SOMns is implemented as self-optimizing AST interpreter using the Truffle framework.

The overall goal is to be compliant with the specification, but include only absolutely necessary features. The current list of intended deviations from the specifications are as follows:

• the mixin support of slots is not yet complete, see deactivate tests in core-lib/TestSuite/MixinTests.ns

• simultaneous slots clauses are not fully supported (spec. 6.3.2)

• object literals currently require a keyword prefix objL, to work around parser limitations

License and Author Information

This code is distributed under the MIT License. Please see the LICENSE file for details. All contributions to the project are implicitly assumed to be under the MIT License. If this is not desired, we ask that it is stated explicitly. Information on previous authors are included in the AUTHORS file.

Setup Development Environment with Eclipse and VS Code

SOMns code is best written using our VS Code plugin, which provides support for typical IDE features such as code navigation and compilation, as well as a debugger. The SOMns support can then be installed via the Marketplace.

For the development of SOMns itself, we typically use Eclipse. A complete guide on how to setup a workspace is available in the docs folder and on ReadTheDocs.

Development Status

Active development of SOMns happens on the dev branch .

The latest release is reflected by the release branch .

Changes and releases are documented in our CHANGELOG.md.

Academic Work

SOMns is designed as platform for research with a special interest for concurrent programming models, their interactions, and tooling for debugging.

Related papers:

• Transient Typechecks are (Almost) Free, R. Roberts, S. Marr, M. Homer, J. Noble; ECOOP'19.

• Efficient and Deterministic Record & Replay for Actor Languages, D. Aumayr, S. Marr, C. Béra, E. Gonzalez Boix, H. Mössenböck; ManLang'18.

• Newspeak and Truffle: A Platform for Grace?, S. Marr, R. Roberts, J. Noble; Grace'18.

• Few Versatile vs. Many Specialized Collections: How to design a collection library for exploratory programming? S. Marr, B. Daloze; Programming Experience Workshop, PX/18.

• A Concurrency-Agnostic Protocol for Multi-Paradigm Concurrent Debugging Tools, S. Marr, C. Torres Lopez, D. Aumayr, E. Gonzalez Boix, H. Mössenböck; Dynamic Language Symposium'17.

• Kómpos: A Platform for Debugging Complex Concurrent Applications, S. Marr, C. Torres Lopez, D. Aumayr, E. Gonzalez Boix, H. Mössenböck; Demonstration at the <Programming>'17 conference.

• Toward Virtual Machine Adaption Rather than Reimplementation: Adapting SOMns for Grace, R. Roberts, S. Marr, M. Homer, J. Noble; Presentation at the MoreVMs'17 workshop at the <Programming>'17 conference.

• Optimizing Communicating Event-Loop Languages with Truffle, S. Marr, H. Mössenböck; Presentation at the AGERE!’15 Workshop, co-located with SPLASH’15.

• Cross-Language Compiler Benchmarking: Are We Fast Yet? S. Marr, B. Daloze, H. Mössenböck at the 12th Symposium on Dynamic Languages co-located with SPLASH'16.

somns's Issues

Add support for Object Literals

Object Literals are defined in the Newspeak spec sec. 5.1.10.

Depending on the context where they are defined, they either have the current activation (i.e. Truffle frame) as context, or nil. From that I derive that object literals can only be defined either on the module level or within a method.

The proposed syntax is objectLiteral = (identifier, keywordMsg opt) opt, classBody.

This means, we could have the following code:

public foo = (
  | local obj val |
  local := 1.
  obj := ( | public field = local. |
               local println.
         )(
           public class Baz = ()()
           public println = ( field println. local := local + 1. )
         ).

  obj println. (* => 1 *)
  local println. (* => 2 *)

 val := Value ()() (* => Signals a NotAValue exception *)
)

For the implementation, beside the parser changes, there is at least one change needed:
SClass is defined with final SObjectWithClass enclosingObject to represent the outer scope. This needs to be changed so that the outer scope also can be a MaterializedFrame. This should be a straightforward change, there are only two things that depend on it: reading of the outer object which is either done with the outer keyword for an outer send, or for an implicit send. This does likely not need any adaptation, beside potential casts, because, access to local variables, i.e., the sends to the current activation are not done as 'sends' but are directly done as local variable accesses. I am not entirely clear on whether there are any issues in the parser's lookup logic for determining whether it is a variable read or a message send, though. Currently, it first checks for local variables, and then does a send. That might be more involved with the additional nesting logic.

The second thing that needs the enclosing object is the check whether a class is a value. That might be a bit more involved. Currently it should be simply returning false for activations, I presume. Except perhaps if they don't have locals. That gets more complicated if we actually start supporting lazy slots (promise slots, not sure what the spec term is) and immutable slots, but, we currently don't have that.

I am not sure what else interferes with it, there might be more.

Add support for message cascades

See charig/TruffleMATE@cb28ee4

Debugger is currently not breaking on breakpoints

AST inlining breaks capture semantics

@eregon found a bug in the AST inlining. The following test does not result in the correct output:

class Test usingPlatform: platform = Value (
| private Array = platform kernel Array. |
)(
  public main: args = (
    | arr b |
    arr := Array new: 10.
    args from: 2 to: args size do: [ :arg | arg print. ' ' print ].

    'Correct Semantics' println.
    b := [ :i  |
        arr at: i put: [ i ]].
    1 to: 10 do: b.

    1 to: 10 do: [ :i  |
        (arr at: i) value println ].

    'Broken Semantics' println.
    1 to: 10 do: [ :i  |
        arr at: i put: [ i ]].

    1 to: 10 do: [ :i  |
        (arr at: i) value println ].
    ^ 0
  ) 
)

Possible performance regression in TreeSort caused by #50

The changes in #50 might have caused a performance regression:

http://somns-speed.stefan-marr.de/timeline/?exe=14%2C16%2C15&base=none&ben=%5Bpeak%5D+TreeSort&env=1&revs=50&equid=off

Promise resolution/synchronization

Currently, promises synchronize on the potentially wrapped value: https://github.com/smarr/SOMns/blob/master/src/som/interpreter/actors/SPromise.java#L343

Since these values can also be primitives, it could be highly problematic, and potentially deadlock prone, in case primitives like ints map to the same boxed object for promises that are to be resolved in parallel.

Need to properly characterize the synchronization dependencies here again, and figure out whether this is an actual issue.

This was identified as part of https://github.com/graalvm/graal-core/issues/201

Simplify codespeed.conf

It is already >600LOC, there are various options to simplify the configuration.
For instance by using variables or input sizes to generate the different tracing configurations. Also unclear why we need all the separate experiment definitions, rebench allows to filter by vm names e.g. with vm:SOMns-interp.

Blocks of multiple arities are not handled correctly at same send site

See charig/TruffleMATE@da10f5f#commitcomment-18420330

Reinstate coverage tracking with coveralls

For the core-lib we used to have coverage tracking with https://github.com/MetaConc/CoverallsTruffle

See https://github.com/smarr/SOMns-corelib/blob/master/.travis.yml#L12

We should enable that again, and also enable it for java code and kompos.

Add support for literal arrays

I really miss the ability to simply note down a literal array.

As far as I can see, the spec only talks about arrays in very few places, and uses the {2. 4. 5} notation. That is a list of expressions, of which their return value defines the value of the corresponding array index.
So, {self foo. 4 + 3. a <-: hello: 'ww'} is perfectly legal, and might evaluate to {self. 7. a Promise}.

What's unclear is how we can have value arrays supported as well.
Perhaps one option would be to use the # sign as an indicator, and make #{2. 4. 5} a value array. That's somewhat in Smalltalk tradition of #(foo bar 42) evaluating to {#foo. #bar. 42}, except that Smalltalk doesn't have ValueArrays, and this seems merely a convenience to note down a list of symbols without too much syntactical noise. Not sure I really care for such symbol arrays.

[DYM] Sort output in CVS files

Output should be sorted by a stable criterion to allow change easy tracking with diff tools.

Lookup of outer sends in `#dnu` handler seems broken

The following code does not work as expected, but gives a #dnu for vmMirror

class T usingPlatform: platform = Value (
| private actors = platform actors. |
)(
  public class Sub = Value (  ) (
    public do = (
      1 halt.
      error: 'foo'
    )
  )

  public main: args = (
    | s |
    s := actors createActorFromValue: Sub.
    s foo.
  )
)

./som -G core-lib/T.som

The expected outcome would be an error message from the far reference implementation.

WebDebugger Features

Features to be added

• adding of breakpoints via line numbers
• removing of breakpoints via line numbers
• display configure breakpoints in list
• set all know breakpoints on reconnect

Reinstate support for updating source sections in Kompos frontend

The #38 removes sending of updated source section info from suspended events.

That's good for debugger performance, but means that we don't get the semantic information and highlighting for field accesses and so on anymore.

Should do a sync from time to time to avoid performance issues, but get best information displayed.

Failing MinitestTests in JUnit Harness

The MinittestTests fail with a MessageNotUnderstood Exception not being properly caught.
The main issue is that the harness reinitializes the Kernel object, but reuses existing ASTs. The AST do value caching, for instance for the outer object, which is used to access class slots. The profile holds on to the old Kernel, and thus also to the old Exception object, which then in turn does not compare identical with the new Exception object.

Setup weekly benchmark run for precise results

For better performance tracking, we need a more reliable but resource intensive performance tracking setup.

Results should be almost paper ready, i.e. include enough warmup.
It should all be on the somns-speed for easy access.

• AWFY vs. Java
• Savina vs. Scala
• full batch of language feature microbenchmarks

Ideally, it only is triggered if there are new commits in the repo.

SOMns does not support Newspeaks setter sends, but uses Smalltalk's `:=` assignment

To be spec. compliant, this should be fixed.

Reading of uninitialized local variables is broken

Reading of uninitialized variables in methods is currently broken.

The following test fails with an assertion, because the read evaluates to null:

class Test usingPlatform: platform = Value ()(
  protected foo = (
    | a |
    a println.
    a := 1.
    a println.
  )

  public main: args = (
    foo.
    foo.
    ^ 0
  )
)

A fix could be:

diff --git a/src/som/compiler/MethodBuilder.java b/src/som/compiler/MethodBuilder.java
index 1598a2b..20e8757 100644
--- a/src/som/compiler/MethodBuilder.java
+++ b/src/som/compiler/MethodBuilder.java
@@ -101,7 +102,8 @@ public final class MethodBuilder {
     MethodScope outer = (outerBuilder != null)
         ? outerBuilder.getCurrentMethodScope()
         : null;
-    this.currentScope   = new MethodScope(new FrameDescriptor(), outer, clsScope);
+    assert Nil.nilObject != null;
+    this.currentScope   = new MethodScope(new FrameDescriptor(Nil.nilObject), outer, clsScope);

     accessesVariablesOfOuterScope = false;
     throwsNonLocalReturn          = false;

Remove support for method categories

They were removed from the Newspeak spec, and aren't useful in either case.

Update License and Author info

The info is still based on SOM, TruffleSOM, and does not reflect SOMns at all.

Supporting Transfer of Far References Between Actors

Currently, SOMns and Newspeak do only support the transfer of far references between actors as direct message parameters.

Since we do not have variable-length argument lists, the only way to transfer an arbitrary amount of references is by sending a corresponding amount of messages. Furthermore, there is currently no way to send far references as part of structured data. So, any semantic associated to references needs to be decomposed on the sender side and reassembled at the receiver side.

There are two ways to fix this:

1. make far references proper first class objects which are themselves values, and can be part of values
   • we cannot allow there storage in values without that, because we would have to treat them transparently as direct references in the owning actor, which would cause values not to be deeply immutable anymore
2. introduce a new kind of object that is passed by value: isolates/transfer objects

Currently, I am thinking that option 2 is the least problematic one, because having far references as fully first-class objects leads to all kind of compilation with their handling (identity, reflection, how to treat them in the owning actor, etc).

The only drawback of transfer objects/isolates is that they are yet another kind of object one has to be aware off.

Resolving a promise with another already resolved promise does not trigger callbacks/messages

When resolving a promise with a resolved promise like this:

| pp ppChained |
pp := actors createPromisePair.
ppChained := actors createPromisePair.

(* first resolve *)
pp resolver resolve: 11.
(* then chain *)
ppChained resolver resolve: pp promise.

Then any action scheduled on ppChained promise is not triggered currently.
Full example below.

Note, a solution for this might also think about fixing issue #29.

class Hello usingPlatform: platform = Value (
| private actors = platform actors.
|)(

  public testFirstResolvedThenChained = (
    | pp ppChained ppCompletion callbackPG |
    pp := actors createPromisePair.
    ppChained := actors createPromisePair.
    ppCompletion := actors createPromisePair.

    (* first resolve *)
    pp resolver resolve: 11.
    (* then chain *)
    ppChained resolver resolve: pp promise.

    callbackPG := pp promise whenResolved: [:v |
      '[testFirstResolvedThenChained] whenResolved on pp: ' print.
      'v was resolved to ' print.
      v print.
      '. Expected: 11' println.
    ].

    callbackPG := callbackPG, (ppChained promise whenResolved: [:v |
      '[testFirstResolvedThenChained] whenResolved on ppChained: ' print.
      'v was resolved to ' print.
      v print.
      '. Expected: 11' println.
    ]).

    callbackPG whenResolved: [:v |
      ppCompletion resolver resolve: #done.
    ].

    ^ ppCompletion promise.
  )

  public testFirstChainedThenResolved = (
    | pp ppChained ppCompletion callbackPG |
    pp := actors createPromisePair.
    ppChained := actors createPromisePair.
    ppCompletion := actors createPromisePair.

    (* first chain *)
    ppChained resolver resolve: pp promise.
    
    (* then resolve *)
    pp resolver resolve: 11.

    callbackPG := pp promise whenResolved: [:v |
      '[testFirstChainedThenResolved] whenResolved on pp: ' print.
      'v was resolved to ' print.
      v print.
      '. Expected: 11' println.
    ].

    callbackPG := callbackPG, (ppChained promise whenResolved: [:v |
      '[testFirstChainedThenResolved] whenResolved on ppChained: ' print.
      'v was resolved to ' print.
      v print.
      '. Expected: 11' println.
    ]).

    callbackPG whenResolved: [:v |
      ppCompletion resolver resolve: #done.
    ].

    ^ ppCompletion promise.
  )

  public main: args = (
    | pp |
    pp := actors createPromisePair.

    testFirstChainedThenResolved whenResolved: [:v |
      testFirstResolvedThenChained whenResolved: [:vv |
        pp resolver resolve: 0 ] 
    ].
    
    ^ pp promise
  )
)

Investigate Failing Actor benchmarks

At least the dictionary benchmarks are failing, probably, because I changed the dict implementation:

https://gitlab.soft.vub.ac.be/stefan.marr/SOMns/builds/738

Use JSON library to reduce boilerplate code

Gson, which is already used in the language server might be a good alternative to manually serializing the data.

Goal is to remove the code and cleanup the related code structure. At the moment, it seems to be quite a mess.
One problem is the different tools that have some dependencies now.

Simplify Specializations, especially around loop nodes, merge where possible

We got too many specialization nodes. There should be quite a bit of potential to merge stuff together. And refactor the implementation to remove code duplication.

Handle Array out of bounds access with exception

Currently, we just crash when there is an array out of bounds exception from Java.
We need fallback specializations that throw a proper SOM error on out-of-bounds.

Add a timeout primitive for executing blocks after a set time interval

Should probably look something like this in the actor module:

(* Executes aBlock after waiting for `milliseconds`.
   Returns a promise, which is going to be resolved to the
   return value of the block. *)
public after: milliseconds do: aBlock = (
  ^ vmMirror actorsDo: aBlock after: milliseconds
)

Revise Promise/Resolver Design: Ensure promises can only be resolved once, and ensure that state change is not observable

Currently, promise resolution more than once triggers and assertion, that should be changed and trigger a proper SOMns exception.
Promise cannot be resolved multiple times, and we should not allow silent erroneous resolution and potential unmitigated race conditions.

Doing this with the current promise and resolver design might be broken.

We need to make sure that a loop inside an actor cannot observe the promise being resolved by another actor in a racy manner.

In vats, it is possible that a resolution is triggered in the same actor multiple times, so, it is natural to have the desire to check whether a promise has been resolved. However, this might be racy.
I think, I don't remember exactly, that resolvers a shared as values between actors, so multiple actors might be able to resolve the same promise.

This means, we might need to make promise resolution asynchronous. So, we might need to have something like resolver resolve: [:alreadyResolved | #valueToBeUsedIfNotAlreadyResolved ].
This smells like headache...

Needs further thought and perhaps a different design. Perhaps resolvers should always be far references. And it should be (resolver <-: resolve: value) whenResolved: [:resolutionSuccessful | ... ]. Still not very convincing. In AT, everything was done via messages and futures are a library anyway. Hm...

Race on object transition

To reproduce:

for i in 1 2 3 4 5 6; do ./som -G -at -TF core-lib/Benchmarks/AsyncHarness.som Savina.CigaretteSmokers 50 0  1000:200; done

Originally, it resulted in a endless recursion on transitioning the object in InitializerFieldWrite.updateObject:

Run-time Error: EventualMessage failed with Exception.
java.lang.AssertionError: After transitioning the object to a new shape, we expect the layout to be valid. But, this is racy...
	at som.interpreter.objectstorage.InitializerFieldWrite.updateObject(InitializerFieldWrite.java:248)
	at som.interpreter.objectstorage.InitializerFieldWriteNodeGen$UpdateObjectNode_.execute_(InitializerFieldWriteNodeGen.java:1317)
	at som.interpreter.objectstorage.InitializerFieldWriteNodeGen$BaseNode_.acceptAndExecute(InitializerFieldWriteNodeGen.java:123)
	at com.oracle.truffle.api.dsl.internal.SpecializationNode.uninitialized(SpecializationNode.java:407)
	at som.interpreter.objectstorage.InitializerFieldWriteNodeGen$UninitializedNode_.execute_(InitializerFieldWriteNodeGen.java:379)
	at som.interpreter.objectstorage.InitializerFieldWriteNodeGen$BaseNode_.acceptAndExecute(InitializerFieldWriteNodeGen.java:123)
	at com.oracle.truffle.api.dsl.internal.SpecializationNode.removeThis(SpecializationNode.java:247)
	at som.interpreter.objectstorage.InitializerFieldWriteNodeGen$UnwrittenOrGeneralizingValueNode_.execute_(InitializerFieldWriteNodeGen.java:1280)
	at som.interpreter.objectstorage.InitializerFieldWriteNodeGen$BaseNode_.execute0(InitializerFieldWriteNodeGen.java:135)

There are however other seemingly related assertion failures:

java.lang.AssertionError
	at som.interpreter.objectstorage.StorageLocation$ObjectDirectStorageLocation.isSet(StorageLocation.java:163)
	at som.interpreter.objectstorage.FieldReadNode.createRead(FieldReadNode.java:27)
	at som.compiler.MixinDefinition$SlotDefinition.createNode(MixinDefinition.java:540)
	at som.compiler.MixinDefinition$SlotDefinition.getDispatchNode(MixinDefinition.java:517)
	at som.interpreter.nodes.dispatch.UninitializedDispatchNode$AbstractUninitialized.createSomDispatchNode(UninitializedDispatchNode.java:92)
	at som.interpreter.nodes.dispatch.UninitializedDispatchNode$AbstractUninitialized.insertSpecialization(UninitializedDispatchNode.java:65)
	at som.interpreter.nodes.dispatch.UninitializedDispatchNode$AbstractUninitialized.specialize(UninitializedDispatchNode.java:51)
	at som.interpreter.nodes.dispatch.UninitializedDispatchNode$AbstractUninitialized.specialize(UninitializedDispatchNode.java:151)
	at som.interpreter.nodes.dispatch.UninitializedDispatchNode$AbstractUninitialized.executeDispatch(UninitializedDispatchNode.java:112)
	at som.interpreter.nodes.dispatch.CachedSlotAccessNode$CachedSlotRead.executeDispatch(CachedSlotAccessNode.java:57)
	at som.interpreter.nodes.dispatch.UninitializedDispatchNode$AbstractUninitialized.executeDispatch(UninitializedDispatchNode.java:113)
	at som.interpreter.nodes.dispatch.CachedSlotAccessNode$CachedSlotRead.executeDispatch(CachedSlotAccessNode.java:57)
	at som.interpreter.nodes.dispatch.UninitializedDispatchNode$AbstractUninitialized.executeDispatch(UninitializedDispatchNode.java:113)
	at som.interpreter.nodes.dispatch.CachedSlotAccessNode$CachedSlotRead.executeDispatch(CachedSlotAccessNode.java:57)
	at som.interpreter.nodes.MessageSendNode$GenericMessageSendNode.doPreEvaluated(MessageSendNode.java:319)
	at som.interpreter.nodes.MessageSendNode$AbstractMessageSendNode.executeGeneric(MessageSendNode.java:95)
	at som.interpreter.nodes.nary.EagerBinaryPrimitiveNode.executeGeneric(EagerBinaryPrimitiveNode.java:82)
	at som.interpreter.nodes.InternalObjectArrayNode.executeObjectArray(InternalObjectArrayNode.java:26)

Issue with Basic Interpreter Test "Lookup", "testInner18"

Test fails, currently disabled. If I remember correctly, this relates to lookup issues. Might be related to issue #5.

Fix fallout after making parser more strict

See https://gitlab.soft.vub.ac.be/stefan.marr/SOMns/builds/740

Parser fails on inlining nested blocks

The following code snippet fails with the NullPointerException:

class VacationSuite = () (
  private class Manager = ()(
    public deleteCustomer: customerId = (
      critical: [
        [ nil ] whileFalse: [
          | resrv |
          critical: [ resrv ] ] ].
    )
  )
) : (
  public test = ( ^ 33 )
)

There is a problem with the resrv variable. The lookup for it fails during inlining of whileFalse:.

This is likely an issue with ResolvingImplicitReceiverSend not having a replaceWithLexicallyEmbeddedNode(.) method, but having a lexical scope that's not adapted.
There could be more issues lurking here, because the usage of the implicit receiver send was likely introduced when minimizing the code snippet.

Once this is solved, it might be opportune to also look into #37.

[DYM] Measure lines of code

• loaded lines of code
• lines of code with executable statements
• lines of code executed

Move eager specialization into the parser where possible

We need some parse-time support for the debugger. Specifically, we want the promise creation and whenResolved: primitives visible in the code (with tags) as early as possible, even if potentially incorrect.

So, from there, it's only a small step to also have + etc specialized in the parser already.

Should be possible to adapt the @Primitive annotation and specializer infrastructure for that.

SOMns does not supported cascaded sends

The classic message cascades are currently not supported, i.e., ; does not allow repeated self sends. This is also called method chaining.

FarReference has issues with lookup of #dnu. Fails with lookup of vmMirror

Master>>#stop                                  core-lib/Benchmarks/Savina.som:1781
Master>>#validate                              core-lib/Benchmarks/Savina.som:1788
FarReference>>#doesNotUnderstand:arguments:    core-lib/Actors.som:37
Thing>>#doesNotUnderstand:arguments:           core-lib/Kernel.som:32

Method vmMirror not found in class Kernel

Ensure module name conforms to file name

Currently, we don't have support for any top-level expressions.
And it happens from time to time that the top-level module definition (i.e. the main class) in a file does not have the same name as the file.

This can be confusing, because of copy/paste mistakes for instance.
We should check that. Don't think there is any use for allowing something else.
In Java, it is a little annoying, but gives a good consistency.

This is a restriction that's not useful if we would have multiple modules or other expressions, for instance for scripting support of course.

Possible regression with #perform:* because of PR #30

The MessageSendNode used to use a different node implementation for while:

    @Override
    protected PreevaluatedExpression specializeBinary(final Object[] arguments) {
      switch (selector.getString()) {
        case "whileTrue:": {
          if (arguments[1] instanceof SBlock && arguments[0] instanceof SBlock) {
            SBlock argBlock = (SBlock) arguments[1];
            return replace(new WhileWithDynamicBlocksNode((SBlock) arguments[0],
                argBlock, true, getSourceSection()));
          }
          break;
        }
        case "whileFalse:":
          if (arguments[1] instanceof SBlock && arguments[0] instanceof SBlock) {
            SBlock    argBlock     = (SBlock)    arguments[1];
            return replace(new WhileWithDynamicBlocksNode(
                (SBlock) arguments[0], argBlock, false, getSourceSection()));
          }
          break; // use normal send
      }

      return super.specializeBinary(arguments);
    }

In #30, we removed that specialization. Think, we need to change the while implementation. Should simplify implementation, don't think, the differences should have an impact on performance.

Stack printed in DNU misses top frame

There seems to be a missing frame on the stack printed on DNU. I remember messing with the stack printing before, perhaps there is some context specific issue that is relevant here.

Should Double>>#sqrt return an integer when the value fits into it?

This implicit conversion is somewhat unexpected. It is an issue because Double and Integer are not perfectly polymorphic. I am not sure they need to be. I think, I would prefer a strong separation and explicit conversion when desired.

See also: smarr/SOMns-corelib@df8ca72

Start a changelog

Goal is a changelog on merge/fix level that tracks a bit the progress/changes to get a better overview of what is happening.

Currently, it seems hard for people to keep up with the changes.

Add fallback node for `super` sends

The following test fails currently:

  (* Megamorphic Super send *)
  public class MegamorphicSuperSend = TestContext ()(
    private class A = ()(
      public aNumber = ( ^ 41 )
    )
    private class B = A () (
      public aNumber = ( ^ super aNumber + 1 )
    )
    private class C1 = B ()()
    private class C2 = B ()()
    private class C3 = B ()()
    private class C4 = B ()()
    private class C5 = B ()()
    private class C6 = B ()()
    private class C7 = B ()()
    private class C8 = B ()()
    private class C9 = B ()()

    public testMegamorphicSuper = (
      | a |
      a := Array new: 9.
      a at: 1 put: C1 new.
      a at: 2 put: C2 new.
      a at: 3 put: C3 new.
      a at: 4 put: C4 new.
      a at: 5 put: C5 new.
      a at: 6 put: C6 new.
      a at: 7 put: C7 new.
      a at: 8 put: C8 new.
      a at: 9 put: C9 new.
      1 to: 7 do: [:i |
        self assert: 42 equals: (a at: i) aNumber ]
    )
  ) : ( TEST_CONTEXT = () )

The problem is that we don't yet have a proper fallback node for super sends.
The first symptom is that there are two methods that are not yet implemented, and should looks like this:

    @Override
    protected AccessModifier getMinimalVisibility() {
      // Super sends can only access public or protected methods
      // see spec sec. 5.10
      return AccessModifier.PROTECTED;
    }

    @Override
    protected MixinDefinitionId getMixinForPrivateLockupOrNull() {
      return null;
    }

But the root cause is the missing fallback node, which can replace the chain. At the moment, the generic node is used, but it, of course, does not have the right semantics for super sends.

Testing method breakpoint fails

When adding the following code to the test file basic-protocol.ts throws this error: Error: timeout of 5000ms exceeded. Ensure the done() callback is being called in this test.

Code for setting AsyncMethodRcvBreakpoint:

describe('setting a source section asynchronous method receiver breakpoint', () => {
    // Capture first suspended event for testing
    let firstSuspendCaptured = false;
    let getSuspendEvent: (event: OnMessageEvent) => void;
    let suspendP = new Promise<SuspendEventMessage>((resolve, reject) => {
      getSuspendEvent = (event: OnMessageEvent) => {
        if (firstSuspendCaptured) { return; }
        const data = JSON.parse(event.data);
        if (data.type == "suspendEvent") {
          firstSuspendCaptured = true;
          resolve(data);
        }
      }
    });

    before('Start SOMns and Connect', () => {
      const breakpoint: AsyncMethodRcvBreakpoint = {
        type: "asyncMsgRcvBreakpoint",
        sourceUri: 'file:' + resolve('tests/pingpong.som'),
        enabled: true,
        sectionId:   'ss-7361',
        startLine:   19,
        startColumn: 7,
        charLength:  118
      };
      connectionP = startSomAndConnect(getSuspendEvent, [breakpoint]);
    });

    after(closeConnectionAfterSuite);

    it('should accept async method receiver breakpoint, and halt on expected source section', onlyWithConnection(() => {
      return suspendP.then(msg => {
        expect(msg.stack).lengthOf(2);
        expect(msg.stack[0].methodName).to.equal("Ping>>#ping:");
        expect(msg.stack[0].sourceSection.line).to.equal(21);
      });
    }));
  });

It seems the issue is related to the exception caused when reconnecting the web frontend debugger after setting a method breakpoint:

Exception: java.lang.NullPointerException thrown from the UncaughtExceptionHandler in thread "WebSocketWorker-10"

The exception occurs because there are not elements in the Map rootNodes after Reconnect the debugger, specifically in the WebDebugger class, method getSource(final URI sourceUri).
I think we have three options to solve this issue: make the Map persistent somehow to be able to access its elements again, or load the source again, or change the way of getting the Source object in the getAsyncMessageRcvBreakpoint method in Breakpoints class.

AST Inlining and temp vars: Improve naming, and ensure we don't have to many slots used

When debugging, I noticed that the vars for this method look strange:

private checkTables: manager = (
      | numRelation customers tables percentQuery queryRange maxCustomerId |
      numRelation := ParamDefaultRelations.
      customers   := manager customers.

      tables := Array with: manager cars with: manager flights with: manager rooms.

      (* Check for unique customer IDs *)
      percentQuery  := ParamDefaultQueries.
      queryRange    := (percentQuery // 100.0 * numRelation + 0.5) asInteger.
      maxCustomerId := queryRange + 1.
      1 to: maxCustomerId do: [:i |
        (customers at: i) ~= nil ifTrue: [
          (customers remove: i) == nil ifTrue: [
            ^ false ] ] ].

      (* Check reservation tables for consistency and unique ids *)
      1 to: tables size do: [:t |
        | table |
        table := tables at: t.
        2 to: numRelation + 1 do: [:i |
          (table at: i) ~= nil ifTrue: [
            t = ReservationCar ifTrue: [
              (manager addCar: i num: 0 price: 0) ifFalse: [
                ^ false ]
            ] ifFalse: [
              t = ReservationFlight ifTrue: [
                (manager addFlight: i num: 0 price: 0) ifFalse: [
                  ^ false ]
              ] ifFalse: [
                t = ReservationRoom ifTrue: [
                  (manager addRoom: i num: 0 price: 0) ifFalse: [
                    ^ false ]
            ] ] ].

            (table remove: i) ~= nil ifTrue: [
              ^ false ]
          ]
        ]
      ].
      ^ true
    )

t is not shown in the debugger, which is strange.
And, there is an i too much. Not sure what's happening there.

And, the names for i's are annoyingly unreadable.

There is an issue with failing asynchronous tests. Harness fails with SomException instead of reporting the error

Add tests for all not yet tested breakpoints to kompos tests

With #58 we introduce new breakpoints.
These and all other that have not yet tests in https://github.com/smarr/SOMns/blob/master/tools/kompos/tests/basic-protocol.ts should be tested.

Dynamic wrapping and specialization issue: tracking at least 1 activation to few

Currently, the specialization code in MessageSendNode does not return the wrapper node for the new node but the node itself. This means, when it is executed, the wrapper is not executed, and the instrumentation is only set up the next time the node is activated.

Move AST inlining checks also into the Specializer logic

Move AST inlining checks also into the Specializer logic once we got parse-time specializations.
This seems to be a good unification, and would avoid having another checking/specialization mechanism for them. Might also open the door for other useful inlining.

`1 halt` crashes with NPE when using debugger

Test

class Hello usingPlatform: platform = Value ()(
  public main: args = (
    'Hello World!' println.
    1 halt.     
    ^ 0
) )

Command: ./som -G -wd -dnu core-lib/Hello.som

Add support for blocks with arbitrary number of arguments

Currently, we only support blocks with up to two arguments (in addition to self).
So, the best we can do is [:a :b | #foo ] value: 1 with: 3.

Thus, we can't have blocks with 3 or more arguments. The implementation is currently also based on Smalltalk classes Block1, Block2, Block3 with the corresponding interface.

It would be good to lift that restriction and enable blocks with more arguments.
I am not sure what the best way forward is.
I am also not sure whether blocks should support currying or 'culling' (I think that's the term used in Pharo). So, basically the question is whether a block should handle activation with insufficient amount of arguments.

We should look into Newspeak, Pharo, TruffleMate, Grace, etc to see options on how to handle this.