skx/monkey

An interpreted language written in Go

monkey's Issues

We should allow binary and hex literals.

The following code should be valid:

   let a = 0xff;
   let b = 0b10101010;

We'll need to update the lexer to recognize these values as numbers, and the parse to convert them to decimal as processed.

Move more of our built-in functions to monkey.

We should be able to port some internals from go to monkey, with a little rejigging.

Functions should allow default values.

This should work:

  function foo( name = "Nobody") {
     puts( "Hello " + name  + "\n" );
   }

   foo();

Add monkey-tests

If we added an assert method we could write some tests in the standard-library, and run them every time we launched.

(We used to test some methods in the evaluator-test-cases, but because they're loaded in package main, via static.go we had to remove them..)

Add regular expression support

This should work:

 if ( "Steve" ~= /steve/i ) {
       ..

We should also see:

  print( type( /foo/ ) ) ;   -> "regexp"

(So it is a distinct type, not a string.)

Error handling of builtins could be improved

This fails, silently:

 let a = { "Steve": "Kemp" };
 puts( len( a ) );

Allow bare (let-free) assignments.

This program works:

   let x = 10;
   let x = x * 10;

However this does not:

    let x = 10;
    x = x * 10;

We should allow bare (let-free assignments).

Our file handling is "unusual"

At the moment you open a file via file.open() then use file.read or file.lines.

I think it would be better to have a file object. You'd receive that via calling open. From there you could call methods on the object:

let p = open( "/etc/passwd", "r" ); 
let contents = p.readlines();
p.close();

We should also implement stat to return a hash of meta-data about the given path.

For directory handling we could do something similar for example:

// implemented in golang.
let d = opendir( "/etc" );
d.readdir()
d.closedir();

// all entries, including "." and ".." - implement in monkey, via readdir:
let entries = d.entries(); 

// entries matching the pattern -> implement in monkey, via readdir.
let matched = d.glob( "pas*" );

This would break old code/examples, but I think the consistency would be worth it.

Convert github actions to new format.

Convert the github actions from the old HCL-based approach, to the new YAML-based configuration.

Implement the ternary-operator.

Sample usage is the obvious:

  function max(a, b) {
      return( a > b ? a : b );
  }

Implement the range operator

Related to #61, we should port the range operator from the evalfilter 1..10 returns an array of the values 1, 2 .. 9, 10.

Constants are accidentally global

The following code fails:

  const a =3;

  function foo() {
      let a = "Hello";
      puts( "A:" , a, "\n" );
  }
  puts( "A:", a, "\n");
  foo();

The output is:

  A:3
  Attempting to modify constant denied - a

stdlib: Implement array.min, array.max, and array.includes?

These are useful and will save users having to write them themselves.

Length calculation of unicode strings is wrong

This should output 2:

  puts( len("狐犬") );

Add more file/directory primitives.

#47 covers a reworking of file/directory handling.

In addition to the work described there we should add:

• file.stat()
  • As mentioned.
• unlink
  • Delete a file/directory.
• mkdir
  • Make a directory.
  • mkdir( "/tmp/foo/bar", { "mode": "0755" } )

Allow definition of class-methods.

We now have support for calling things like:

 "Steve".count("e");

We should allow the user to define similar methods in pure Monkey, rather than in go.

The initial case should allow:

 function TYPE.name( self );

Where TYPE is string, array, hash, etc. It might be I have to cheat and use ":" instead of "." to simplify life, but I could live with that. The important thing is that it is possible at all.

We could rethink our environmental variables.

At the moment all environmental variables are imported as variables with a $-prefix.

For example this shows your current username, assuming a Unix system:

  puts( "Hello ", $USER , "\n");

or:

  puts( "Hello " + $USER + "\n");

However this importing is read-only. So when you execute processes you cannot change the values.

It might make more sense to clone environmental variables to a hash:

  puts( "The PATH is:" ,ENV["PATH"], " You are :" , ENV["USER"], "\n" );

Using a hash would still allow iteration, via keys, but we could add special handling such that writing ot the ENV-hash would allow updating the contents of an environmental variable.

I'm not 100% sure if this would be a useful change, but it feels like it might be useful?

Some of our standard library could be implemented in monkey ..

Specifically the first and rest functions could be implemented in pure-monkey, rather than C.

 function first( array ) {
   puts( "first() implemented in monkey\n");
   return( array[0] );
 }
 function rest( array ) {
   let result = [];

   if ( len(array) > 1 ) {
     let i = 1;
     for( i < len(a) ) {
        result = push(result, array[i]);
        i++;
     }
   }
   puts( "rest() is implemented in monkey!!!!\n");
   return result;
 }


 //
 // Test-code
 //
 let a = [ "Steve", "Kemp", "Kirsi", "Kemp" ];
 puts( "First element is ", first(a), "\n");

 a = rest(a);
 let i = 0;
 for( i < len(a) ) {
   puts( "Element ", i, " is " , a[i], "\n");
   i++;
 }

The output of this code is as-expected:

  frodo ~/go/src/github.com/skx/monkey $ ./monkey t.mon 
  first() implemented in monkey
  First element is Steve
  rest() is implemented in monkey!!!!
  Element 0 is Kemp
  Element 1 is Kirsi
  Element 2 is Kemp

Sadly it seems the rest of our functions are required to be implemented in C, but we could write and embed a standard-library of monkey-functions..

.methods doesn't return the full set of results.

When we allowed object-methods to be defined they were originally written 100% in golang.

That meant that the golang-implemented methods call could work, outputting the names of all known methods. However now there are methods implemented in monkey.

We should use reflection/introspection to get all methods that work on a given-type, or we should remove the facility entirely. Having wrong results is the worst possible compromise!

Allow a "strict" mode.

Currently built-in functions can return errors - primarily when they're invoked on objects of the wrong type.

For example calling len against a hash is an error:

 let a = { "Name": "Steve" };
 puts( len(a), "\n");

This will output:

 Error calling `len` : ERROR: argument to `len` not supported, got=HASH

If we had a strict-mode we could terminate the process at this point, similarly on the use of an undefined variable:

 let steve = "Foo";
 puts( steven );

I think we'll add use( strict ); or pragma( "strict" ); to enable this.

Our string matching should be improved.

Right now you can't write this:

 let str = "\"";

(i.e. Trying to write a quote in a string will cause a syntax error.)

This will become important if we want to allow quotes in regular-expressions, as per #8.

Allow file-globbing.

We should allow something like:

 let entries = file.glob( "/etc/*.conf" );

Allow "+=", "-=", etc.

Once #13 is implemented the following will work:

     let x = 4;
     x = x + 3;

However this will not:

        let x = 4;
        x += 3;

We need to handle +=, -=, *= and /=. No doubt there are other cases we could consider but they will be sufficient to make me happy.

String equality testing is broken.

Consider this code:

  let a = 4;
  let b = 4;

  if ( a == b ) {
    puts( "Integers are identical.\n");
  } else {
    puts( "Integers are different - this is a bug!!\n");
  }

  

  let a = "Steve";
  let b = "Steve";

  if ( a == b ) {
    puts( "Strings are identical.\n");
  } else {
    puts( "Strings are different - this is a bug!!\n");
  }

The output shows:

    deagol ~/go/src/github.com/skx/monkey $ ./monkey e.mon 
    Integers are identical.
    Strings are different - this is a bug!!

Allow method-calls on objects.

I suspect I went down the wrong-path when I started adding the abliity to run things like this:

 puts( strings.toupper( "steve" ) );

The idea of a standard-library is appealing, but really when we want to turn a string into a modified version we should probably allow that to happen via the object itself. So I'd prefer users to write:

  puts( "steve".toupper() );

Of course not everything is an object, but a similar approach could be applied to at least:

• arrays
• hashes
• strings

Integers/Floats/Booleans will probably be left alone, and we'd keep the "standard library" for files/etc.

This will be a reasonably large change:

• Allow parsing method-calls via a "."
  • e.g. "foo.bar()" is a method call to function "bar" with implicit arg "foo".
• Remove much of the standard library.
  • e.g. strings.toupper, strings.tolower(), etc.

Fun times :)

Allow declaring constants

Related to #19 we could move PI, & E, into the monkey-based init-file(s) if we had the ability to define constants, which couldn't be modified.

Something like:

const PI=3.1.4..

Instead of:

let PI=3.14..

This should be a simple matter of updating our environment.

Allow command execution.

It might be nice to allow fork/exec/popen, etc, but they're a bit of work. (popen can probably be handled like fopen.)

For the moment I'd suggest we just implement backtick-execution:

  let out = `uptime`

  puts( out['STDOUT'] );
  puts( out['STDERR'] );
  puts( out['ERROR'] );

Allow our mutation operators to work on mixed types

We recently added mutation-operators in #14, allowing:

 let x = 4;
 x += 44;

However type-mixing isn't permitted, meaning this code fails:

  let y = 3;
  y += 3.14;

This fails because we only implemented +=, -=, *= and /= for integer types. We should allow floats, and float/int pairs to work. It probably also makes sense to allow string concatenation via "+=":

let s = "Steve";
s += " Kemp";
s += "\n";

Implement eval(str);

Current we have :

assert( bool, str );

Would be nice to evaluate the expression:

assert( str, str );

We could do that if we had eval

String index operation is wrong for unicode

This produces the wrong output:

  puts("狐犬"[0]);

(We expect "狐".)

We need some kind of string-matching function.

Right now if we want to do a string-find / string-match we must use the index-operator to do it manually.

There are two ways to go here:

• We could implement various string matching functions:
  • string.has_prefix( "Steve", "test" ) -> "false"
  • string.has_suffix "Steve", "eve" ) -> "true"
  • string.contains( "Hello, world!", "world" ) -> true
• We could implement a regular expression matcher.
  • let result = steve =~ /e[vV]e/;

The regular expression method is probably cleaner, but we'd need to think about what it would return:

• NIL on failure to match
• A hash with captures, if any were present:

I'd expect:

 let ip = "10.1.2.234";
 let result = ip =~ /^([0-9]+)\.([0-9]+)\.([0-9]+)\.([0-9]+)$/;
 puts(result[0]) -> "10";
 puts(result[1]) -> "1";
 puts(result[2]) -> "2";
 puts(result[3]) -> "234";

But the use of the result there is a bit nasty. The only alternative would be to create a regexp-type so we could say:

     let regexp = "^([0-9]+)\.([0-9]+)\.([0-9]+)\.([0-9]+)$";
     let result = match(regexp, "string input" );
     puts( result[0] );

That might make the most sense, but feedback welcome.

Implement `foreach`

My evalfilter project uses foreach to iterate over:

• arrays
• characters in strings.

We can port that easily, and once done we'll also support iterating over hash key/value pairs.

pragma function could be improved

Right now if you call pragma(), with no arguments, it returns the pragmas in-use.

If you give it an argument pragma("strict"); it returns true.

It should always return the enabled pragmas.

Literal regular expressions should allow captures

We shouldn't need to use the match function to access captures, we can just populate $1, $2, .. etc, as perl does.

Implement array.sort

Should be simple.

Implement switch-statements

I want to allow this to succeed:

function test( name ) {
  switch( name ) {
    case /^steve$/i {
       printf("I'm a regexp\n");
    }
    case "Steven" {
	printf("I know you!\n" );
    }
    default {
	printf("I don't know who you are\n" );
    }
  }
}

test( "Steve" );    // Regexp match
test( "Steven" );   // Literal match
test( "Bob" );      // Unhandled case

NOTE: I explicitly use blocks here, because fall-throughs are evil :)

String comparisions should handle <= >=

Right now you can't run :

• if ( "a" >= "A" ) ...
• if ( "a" <= "A" ) ...

Rewrite string.interpolate in 100% monkey.

Given input like this:

string.interpolate("Steve ${surname}", { "surname": "Kemp" } );
-> "Steve Kemp"

Crash with array.find

The following program contains a crash, which seems to be as a result of #61:

   a = [ "steve", "kemp" ];
   let out = a.find( "steve" );

Functions/identifiers should allow digits in their names.

e.g. This should work:

function max(a,b) {
   return( a > b ? a : b );
}

function max2(a,b) {
    if ( a > b ) {
        return a;
    } else {
        return b;
    }
}

puts( "max(1,2) -> ", max(1, 2), "\n" );
puts( "max2(-1,-2) -> ", max2(-1, -2), "\n" );

Implement array.reverse

Should be trivial.

We need to parse numbers better, to handle periods.

The following code works as you'd expect:

   let a = 3;
   let b = 3.312;

   puts( a.type() );
   puts( b.type() );

It outputs integer, then float. However this code fails:

  puts( 3.type() );
  puts( 3.14.type() );

The reason for the failure is that the lexer thinks we're seeing an invalid number, rather than stopping when it become obvious that is not the case.

   puts( 3.type() );
   -> no prefix parse function for ILLEGAL found

Conditionals should handle and + or

Right now you cannot run:

• if ( a && b ) ..
• if ( a || b ) ...

Implement more string-methods in monkey

At least

• string.split
• string.substring
• string.replace

Copies of constant objects should be modifiable.

This code fails:

 let a = PI;
 let a = a.to_i();
 > Error: Attempting to modify 'a' denied; it was defined as a constant.

However this works:

let a = PI;
puts( a.to_i() );

The reason for this is the assignment-operation copies the const-value of the source variable (PI in this case).

Nested for-loops contain an error

This code works:

function array.sort() {
   let pass = 0;

   for( ! self.sorted?() ) {
      let i = 1;
      let l = len(self);
      for( i < l ) {
         if ( self[i] < self[i-1] ) {
           self = self.swap( i-1, i);
         }
         i++;
      }
      pass +=1;
   }
   return self;
 }

However remove the use of pass and it panics. Seems to be the return-value of the for looks is causing isseus - suspect it doesn't return a value and it should.

A language isn't real until it has an emacs mode.

We need to cover:

• functions in the standard-library
• strings
• comments
  • single and multi-line
• keywords

If function parameters are missing we need to handle that.

This causes an ugly crash:

    function foo( name ) {
      puts( "Hello " + name  + "\n" );
    }
    foo();

(i.e. A parameter is expected but not found.)

The crash is here:

    panic: runtime error: index out of range

    goroutine 1 [running]:
    github.com/skx/monkey/evaluator.extendFunctionEnv(0xc42007a600, 0x0, 0x0, 0x0, 0x44013f)
    	/home/skx/go/src/github.com/skx/monkey/evaluator/evaluator.go:797 +0x19f
    github.com/skx/monkey/evaluator.applyFunction(0x548260, 0xc42007a600, 0x0, 0x0, 0x0, 0x0, 0x0)

We should just call len on the two parameter arrays and abort if one is too short. (Though that might need more care when implementing #16 )

Sorting an array of booleans fails ..

Calling this code led me on a chase:

 [ true, false, true ].uniq();

array.uniq does everything else, up to the point where it sorts the keys of the (temporary) hash it used.

The comparison makes no sense:

if ( true < false ) { ... }

So either :

• Disallow this.
• Fake an order (false comes first?)
• Don't sort the array when the types are mixed/bogus.

Suspect there's no real right/wrong answer here. Just a choice to be made.

Allow file writing.

We can replicate read as write to allow writing to an arbitrary open handle.