bakpakin/fennel

Lua Lisp Language

fennel's Issues

document macros

is there any intention to implement macros in Fennel

Allow some way of loading macros that doesn't go through io.* functions

I'm using Fennel inside love2d. Although the environment has the io module, it doesn't work very well and cannot be used with a packaged game (as io can't read from inside the love archive). For now I've hacked the compiler to use love.filesystem instead of io in my own fork, but this solution can't really be contributed back here. It would be good if there was some way of using macros that didn't need to go through io.

Either that, or I can think about maintaining a fork that replaces all usage of io with love.filesystem. But I'd rather not do that...

Write compiler errors to stderr instead of stdout

Currently, the ahead-of-time compiler writes error messages to stdout instead of stderr, meaning if you're redirecting output to a file, the error message will be written to the file rather than to terminal output as would likely be desired.

Quasiquote?

Once we get some more people writing macros it would probably be good to have quasiquote; this would allow us to remove the list and sym specials that are only visible inside eval-compiler.

Source is in one File

The majority of the source code is all in one file. For someone familiar with the codebase, this isn't a huge problem, but I think it makes the code base seem more complex than it really is. Most of the code could easily be refactored into separate files/modules, such as ones for streams, parsing, common data structures, core compilation and compiler helpers, and various individual files for special forms.

As I am familiar with the code, I am wondering if people think this would be helpful for making the code more readable.

provide a binary build?

I'm able to build a binary executable using luastatic. The step is actually very simple. Just three steps (on macos).

./fennel --compile fennelview.fnl > fennelview.lua
ln -s fennel main.lua
luastatic main.lua fennel.lua fennelview.lua $HOME/.local/lib/libluajit-5.1.a  -I$HOME/.local/include/luajit-2.1 -pagezero_size 10000 -image_base 100000000 

Maybe a useful thing to document. Another thing I wish for is to make the executable work as shebang #! interpreter like lua. I don't know how.

Compile error for `.` with inline table

This Fennel:

(. {:a 1} :a)

Produces this Lua:

return {a = 1}.a

Trying to evaluate this with Lua gives: lua: stdin:1: <eof> expected near '.'

I think it should produce:

return ({a = 1}).a

or perhaps:

local _0_ = {a = 1}
return _0_.a

Rename?

Do you want to go with "Fennel", or should we brainstorm a bit more about potential names?

In Tutorial: include/require/use other fnl files

In the tutorial, it's not clear how to compile one's project if it were to consist of multiple .fnl files (for example, a ./main.fnl file, plus some ./lib/foo.fnl and ./lib/bar.fnl files).

I think it would be very useful if, in the Tutorial :: "Modules and multiple files" section there were a simple Fennel-only example of how to include/require/use a couple of other .fnl library/module files from your main my-proj.fnl file.

Improve consistency of multi-valued set?

The current way to set multiple values at once is like: (set a b c (values 1 2 3))

It seems to me it would be more consistent to support it as (set a 1 b 2 c 3) to make it more similar to let.

Or another option would be (set [a b c] [1 2 3]) to make them both support destructuring.

What do you think?

Confusing interaction between parse errors and multiline input at the REPL

See #96

`set` always shadows when destructuring multi value returns

The semantics of set change when destructuring a multi value return. Here is some fennel code with the compiled output to illustrate my point:

(do
  (var a nil)
  (var b nil)
  (while true
    (set a 1)
    (set (a b) (value 1 2))
    (set [a b] [1 2])))

local function _15_()
  local a = nil
  local b = nil
  while true do
    a = 1
    local a, b = value(1, 2)
    local _16_ = ({1, 2})
    a = _16_[1]
    b = _16_[2]
  end
  return nil
end
_15_()

I think the correct compilation here would be for this line:

     local a, b = value(1, 2)

To actually be:

     a, b = value(1, 2)

In the tutorial, tables: how to push, pop, etc for sequential tables

In the tutorial, section "Ok, so how do you do things?" :: "Tables", there should be a section specifically for sequential tables, with info on how to do the following:

• check if an item is present in the sequential table
• get an item using its index
• push/pop (to/from end of sequential table)
• shift/unshift (to/from front of sequential table)
• get an item's index using its value

Macros in the repl

What I tried:

$ fennel --repl
Welcome to fennel!
>> ((lambda [] (require-macros "test-macros") (->1 9 (+ 2) (* 11))))
Compile error: Compile error in 'ipairs' ./test-macros.fnl:4: unknown global in strict mode: ipairs

What should happen:

$ fennel --repl
Welcome to fennel!
>> ((lambda [] (require-macros "test-macros") (->1 9 (+ 2) (* 11))))
121

support multiline string?

Hi,

Is it possible to support multiline string? It seems [[...]] can't be used for that anymore, use """ ... """ instead?

Thanks!

"Base Library" of standard macros

Is there any appetite for curating a few useful macros in to a standard module? (Not sure if this is really an "issue", but don't know where else to ask.)

I've taken the macros from test-macros.fnl and tweaked them slightly and also added thread last aka ->>, and am finding them very productive.

I then wanted the correct syntax highlighting and indentation for defn so added it to @technomancy 's fennel-mode. And this got me thinking that a "standard" set of macros would be nice for tooling.

REPL fails to compile (if 1 2)

I was trying different expressions in REPL and noticed that it failed to evaluate expression below with a cryptic message:

> (if 1 2)

./fennel.lua:56: [string "if 1 then..."]:3: 'end' expected (to close 'if' at line 1) near 'local

As far as I can tell, it compiles to:

if 1 then
    return 2
    -- saveSource goes here
end

However, spliceSaveLocals() starts adding block for capturing locals after return statement.

Lua: 5.3.5
Fennel: 0.1.0 (commit ed22cc4)

Can't put side-effect inside if branch

>> (print (fnl.compile "(if true (set x 1))"))
local 
do
  if true then
    x = 1
     = 
  end
end

Current workaround is to wrap the branches in a do.

Unexpected behaviour for `lambda`

The tutorial mentions this code:

(global print-calculation
        (lambda [x ?y z] (print (- x (* (or ?y 1) z)))))
(print-calculation 5) ; -> error: Missing argument z

I had expected that this would mean that you could invoke this function with only two arguments, but that isn't the case:

(print-calculation 5 6) ; -> error: missing argument z

I think it was this sentence from the tutorial that confused me:

Note that the second argument ?y is allowed to be omitted, but z is not.

Is the expected usage more like this:

(print-calculation 5 nil 6)

Is this clarification that is required in the documentation or a bug in lambda?

Destructuring fn, each locals?

Right now destructuring only applies to local, let, var, global, and set. It would be convenient if locals introduced by fn could also have destructuring applied. @bakpakin is this something you were planning on implementing, or was there a reason to leave it out?

Behavior of the Product Function (*)

This might be a little bit of a minor complaint, but (*) returns 0 in Fennel, but it returns 1 in Scheme, Common Lisp, Emacs Lisp, Clojure, and every other Lisp I know of. And 1 is a more sensible answer in some sense, since (+) returns 0, which is the identity element for +, and the identity for * is 1.

However, this behaviour might be there for some compatibility issue with Lua. But even in that case, I think it should be documented more explicitly.

P.S.: I didn't know what to tag this, that's why I didn't.

Table iteration form

We have the *for form, but it only works with "numeric for", not with pairs and ipairs. Should we introduce a new form for table iteration; maybe each? Is the star in the name of *for intended to indicate an internal implementation detail, or is there some other meaning?

I noticed that *for takes its arguments in an unusual way: (*for i [begin end] body) instead of keeping i inside the vector like most lisps do. If it were up to me I would implement table iteration as (each [k v (pairs tbl)] (print k v)) where the last element in the vector is the iterator call, and you can bind as many loop locals as the iterator will return. But that's different from the existing *for so I thought I'd ask before implementing.

Better tail calls

The compiler does not properly detect all tail calls.

For example, the fnl code

(fn [x y]
    (if x (+ x 1) y))

Does not properly detect the tail call.

_0_ = function(x, y)
  local _1_
  do
    if x then
      _1_ = (x + 1)
    else
      _1_ = y
    end
  end
  return _1_
end
return _0_

The correct output should be

return function(x, y)
  do
    if x then
      return (x + 1)
    else
      return y
    end
  end
end

This requires a small change to the compiler. Special form compilers will need to take one more argument, options, that will describe attributes about the form they are compiling, like if the form is in tail call position. If a form is in tail-call position, it should not generate a local binding, and prefix the resulting expression with a return. Forms that evaluate to no values (block, set, etc.) should just emit a single 'return' in the tail call position.

`.` form can't handle repeated indexing

It would be nice if the . form could not only turn (. foo bar) into foo[bar], but turn (. foo bar baz) into foo[bar][baz]. Similar functionality is provided by other forms, such as .. turning (.. foo bar baz) into foo .. bar .. baz so you don't have to write long chains by hand.

An implementation of this as a macro isn't hard, but it seems useful enough that it should be built in:

{".>" (fn [val ...]
        (var val val)
        (each [_ key (ipairs [...])]
          (set val (list (sym ".") val key)))
        val)}

(Some discussion on Mastodon.)

Add online demo

Easy to do with fengari.

For fun, try on https://fengari.io/:

For those that want to copy/paste:

package.preload.fnl = loadfile("https://raw.githubusercontent.com/bakpakin/fnl/master/fnl.lua")
require "fnl".eval([[  (print 'hello, world!')  ]])

Repl printing

While I agree that Fennel shouldn't require a pretty-printer, I wonder if it would be helpful to set up the default repl that you get with the executable fennel script with something better than table: 0x55a8c7165b00.

I wrote a fennel-specific pretty-printer that's 128 lines; and I've been using it in the repl via a .fennelrc of such:

(set options.pp ((. (require "fennel") "dofile") "/home/phil/src/fennelview/fennelview.fnl"))

Should we include this and load it up in the default repl? https://gitlab.com/technomancy/fennelview

(Also, I have some generative testing in that repo that might be handy for other fennel-related tests if anyone is curious.)

Locals don't work in REPL

Hi! Thank you for your great work on Fennel!
I'm facing an issue where locals don't seem to register in the repl:

>> (local x 1)

>> x

>> (global x 1)

>> x
1

I'm on a Mac and installed via brew.
Thanks again! Can't wait to mess around with the LOVE engine!

What is the use of `do local _ = <function-name> end`?

Hi all and thanks to the contributors for this awesome project!

I tried to convert some Hammerspoon script to Fennel and noticed something strange in my compiled output. Why does each function I declare get an additional do local _ = <function-name> end?

From the issues I read, there seem to have been similar problems in the past but it looks like they were fixed. #6

So maybe it is done to my code... But after reading this question on SO, I really don't see why these do end would be useful (as they set a local that is never used).

Fennel code:

(fn appToggleScanSnap [data]
  (if (= data.eventType "added")
        (hs.application.launchOrFocus "ScanSnap Manager")
      (= data.eventType "removed")
        (let [app (hs.appfinder.appFromName "ScanSnap Manager")]
          (app.kill))))

(fn settingToggleKinesis [data]
  (if (= data.eventType "added")
        (when true
	  (hs.keycodes.setLayout "U.S. International - PC")
          (hs.execute "/Applications/Karabiner.app/Contents/Library/bin/karabiner select 1"))
      (= data.eventType "removed")
        (when true
          (hs.keycodes.setLayout "Swiss French")
          (hs.execute "/Applications/Karabiner.app/Contents/Library/bin/karabiner select 0"))))

(fn usbDeviceCallback [data]
  (if (= data.productName "ScanSnap S1300i")
        (appToggleScanSnap data)
      (= data.productName "Kinesis Keyboard Hub")
        (settingToggleKinesis data)))

(local usbWatcher (hs.usb.watcher.new usbDeviceCallback))
(usbWatcher.start)

Compiled code:

local function appToggleScanSnap(data)
  if (data.eventType == "added") then
    return hs.application.launchOrFocus("ScanSnap Manager")
  elseif (data.eventType == "removed") then
    local app = hs.appfinder.appFromName("ScanSnap Manager")
    return app.kill()
  end
end
do local _ = appToggleScanSnap end
local function settingToggleKinesis(data)
  if (data.eventType == "added") then
    if true then
      hs.keycodes.setLayout("U.S. International - PC")
      return hs.execute("/Applications/Karabiner.app/Contents/Library/bin/karabiner select 1")
    end
  elseif (data.eventType == "removed") then
    if true then
      hs.keycodes.setLayout("Swiss French")
      return hs.execute("/Applications/Karabiner.app/Contents/Library/bin/karabiner select 0")
    end
  end
end
do local _ = settingToggleKinesis end
local function usbDeviceCallback(data)
  if (data.productName == "ScanSnap S1300i") then
    return appToggleScanSnap(data)
  elseif (data.productName == "Kinesis Keyboard Hub") then
    return settingToggleKinesis(data)
  end
end
do local _ = usbDeviceCallback end
local usbWatcher = hs.usb.watcher.new(usbDeviceCallback)
return usbWatcher.start()

Question extra: When doing multiple actions as the result of a matched if condition, is it the best practice to use let (even without setting local variables)/when or is there another way (i.e. a do a-la-clojure)?

Tables not printed correctly in repl

Welcome to fennel!
>> [2]
table: 0x55b14945f880

It appears the metatable of ret[1] is not set in this case; not sure why.

Runtime arguments

In standard Lua, you can use {...} to get the runtime arguments that the script was executed with. Is there a way to do this in Fennel?
Right now, we're using a makefile to patch the compiled file and add local arguments = {...}, but it doesn't feel quite right.
Thanks a lot.

How does this compare to similar projects?

There are a lot of projects that seem similar in scope to this one. What makes this one stand out (aside from having the most recent contributions)? Just curious.

EDIT:

Here is yet another such project, although it also has recent contributions.

incorrect output on let binding

this fennel file:

(let [x 1
      r nil
      l 3]
      (print ""))

generates:

do
  local x = 1
  local r = nil
  return print((""))
end

this does not generate a local l var, leaking any later assignments to the global namespace.

with some testing i noticed that all let assignments after a nil assignment are ignored. So in the above example x is emitted but l is not.

Changing the assigment to r false fixes the problem

`fennel` and `fennel --repl` both provide a repl. Same?

For me, fennel and fennel --repl both get me the repl.

Is there a difference between the two?

Improve cosmetics of compilation output

A few things that would be nicer:

• Fixing a bug with side-effects in 9acf9cd caused spurious do _ = nil end lines to be emitted in compilation output.
• Functions are always compiled and set to gensym'd locals; this is a bit untidy when they could be emitted directly as expressions.
• Can we detect cases where mangling dashes to _2d is unnecessary? If dashes could mangle to underscores when there's no conflict then the code emitted would be a fair bit more readable.

versioned releases?

Hey @bakpakin do you have any thoughts about starting to use version numbers so we can have an easier time referring to when a specific change in Fennel happened? I can throw up a changelog and call current master 0.1.0, and we can move on from there, or if you've got something you'd rather do I'd like to hear your thoughts.

let block inside loop discards last body form

I found a bug where the last form of a let block inside a loop (both each and for) is discarded:

(for [_ 1 3] (let [] (form1) (this-form-gets-discarded)))

I think it has to do with an in correct nval setting when let calls doImpl but I'm having trouble tracing it beyond that.

There's a failing test case on this branch: https://github.com/bakpakin/Fennel/tree/let-inside-loop

Float parsing curiously working

Hello,

It seems the float parsing is broken in some way:

(do
    (set a 0.3)
    a
)

fennel --compile test.fnl                                                                                                                                   do
  a = 0.29999999999999999
  return a
end

Is this intended?

Confusing interaction between parse errors and multiline input at the REPL

I was confused earlier by the fact that a parse error could put the REPL in a state where it appears that no output is being printed. Here is an example REPL session, I have marked the moment of my confusion!

Welcome to fennel!
>> (1x 1)
Parse error: could not read token "1x" in unknown:1
>> 1  ;; <--- confusion here
>> )
Compile error: Compile error in '1' unknown:1: cannot call literal value
>> 1
1

I'm not sure what the fix is as I think there are actually two issues:

1. Parse errors should probably "reset" the parser state, in particular I think the issue in this case is that the stack isn't reset.
2. Multiline input "mode" at the REPL is indistinguishable from the first line of input, for reference Lua doesn't appear to allow multiline input, Python changes from >>> to ..., Clojure changes from namespace=> to #=>. (I had a quick look as to whether this could easily be added to the Fennel REPL, but the point that the prompt is printed doesn't currently have access to the parser state.)

luarocks packaging doesn't include fennelview

Would be nice to cut a 0.1.1 bugfix release with the most recent fixes where luarocks users could have fennelview in their repl by default.

@bakpakin what kind of problems were you running into when you tried this previously?

Fennel should mangle all symbols

While discussing #53 on IRC, I did some tests and realized that symbol mangling doesn't apply to non-head symbols in multi-symbols. @technomancy and I discussed this, and I think this is the wrong decision.

For reference, on master (1101e67), the multi-symbol a.b/c translates to a["b/c"], but a/b.c translates to a_2fb.c.

In a normal Lua script, you can be sure that evaluating function () a = 5; return _G.a will create a global variable a and return 5. The equivalent Fennel code would be (fn [] (global a 5) _G.a), which also creates a global a and returns 5. However, if your global is an illegal Lua identifier, it will undergo mangling, as shown with (fn [] (global a/b 5) _G.a/b), which creates a global a_2fb and returns the value of the global _G["a/b"], which is unknown. Following the principle of least surprise would mean that this instead returns the value of the global _G.a_2fb and continues to work.

@technomancy suggested the current behavior of mangling only when absolutely necessary could be preserved by making (global sym val) translate into _G["sym"] = val and making access of an unknown binding g translate into _G["g"]. However, this has a couple of problems.

_G is not always going to be your normal environment. Lua 5.2+ introduces _ENV to more accurately make dynamic access of the current environment available. We would now want to have knowledge of what Lua version you'll be running your compiled code with at the end, and certain execution environments may not even expose _G or _ENV. We'd also have to refactor the code to more comprehensively track bindings to know when you're reaching for a global (this should probably be done anyways, but it's not what we do right now). This is too high of a cost IMHO for avoiding some mangling.

If we do decide to consistently mangle across multisyms, some other things become easier. If a user is using the new defn macro to set functions to keys of a table that they return at the end (a common Lua idiom), then you could pair this with a prospective smarter table destructuring syntax (brought up in #53 (comment)) to provide nice looking selective requiring of functions. If a file foo.fnl contains (local foo {}) (defn foo.f/a [] 1) (defn foo.f/b [] 2) foo, then the first function from there could be required like this: (local { f/a } (require :foo)).

As for exposing manglings in APIs like that, my current view is that manglings are an implementation detail of the compiler and not to be relied on directly. We currently don't document mangling at all, so I would suggest that the tutorial or reference states that Fennel allows for a wider range of characters in symbols than Lua does, but that you shouldn't rely on specific manglings outside of code compiled with the same version of the compiler. This allows for situations like the selective require to be implemented cleanly within a project and PRs like #60 that clean up generated output while also warning against exposing mangled symbols in your public APIs. This also allows for specific mangling behaviors to be stabilized as desired, such as the prospective - to _ conversion in #53, which would probably make the public API limitations easier to deal with in idiomatic Fennel code (as you could continue to use dashes instead of underscores or CamelCase while still making your API easy to consume for Lua users).

This would have no effect on code that only uses valid Lua identifiers as symbols.

require-macros causes compile fail in project using love2d

While working on a small project that uses love2d i noticed that (require-macro) causes the compile to fail on code that depends on the global love

Summary

Fennel treats globals like love as "unknown global in strict mode" when the code calls (require macros)

Reproducing

I tested with a fresh git clone from master.

to narrow down the issue i created this minimal setup to reproduce this issue:

test.fnl

(require-macros :macros)

(local foo (love.graphics.newFont))

macros.fnl

{}

I compiled the file with fennel --compile ./test.fnl
The output i got is the following:

Compile error in 'love.graphics.newFont' ./test.fnl:3: unknown global in strict mode: love
stack traceback:
  [C]: in function 'error'
  /home/jamestk/git/Fennel/fennel.lua:370: in function 'assertCompile'
  /home/jamestk/git/Fennel/fennel.lua:541: in function 'symbolToExpression'
  /home/jamestk/git/Fennel/fennel.lua:815: in function 'compile1'
  /home/jamestk/git/Fennel/fennel.lua:781: in function 'compile1'
  /home/jamestk/git/Fennel/fennel.lua:954: in function 'destructure'
  /home/jamestk/git/Fennel/fennel.lua:1169: in function 'special'
  /home/jamestk/git/Fennel/fennel.lua:764: in function 'compile1'
  /home/jamestk/git/Fennel/fennel.lua:1519: in function ?
  [C]: in function 'xpcall'
  /home/jamestk/git/Fennel/fennel:73: in main chunk
  [C]: in ?

Obmitting (require-macros :macros) in test.fnl lets the file compile without errors.

Workaround

After looking into some code i found that the error is generated around line 541 in fennel.lua

As a temporary hack i commented the lines:

    assertCompile(not isReference or isLocal or globalAllowed(parts[1]),
                  'unknown global in strict mode: ' .. parts[1], symbol)

After removing the lines no further errors are raised, and the program and macros work like intended.

Comments

It might be that the failure on unknown globals is desired, but the inconsistency of it happening only when (require-macros) is called makes me believe that this behaviour is unintentional.

I also found a variable called allowedGlobals, but iam unsure how i would add love to that.

Prevent typo'd global references

We already prevent accidental global setting with set, but there's currently no way to prevent typos from accidentally referring to a global. I would like to add a strict mode which would prevent this from happening unless the it's in a list of allowed globals.

This is pretty simple to do with one catch: passing the list of globals thru from the compiler entry point reliably to compile1 which needs to check it. Originally I thought the opts map would be the right place for this, but this table seems to be used exclusively for compiler internals, and storing a user-provided value in there not only feels wrong but doesn't really work anyway since opts tables are frequently replaced and not propagated downstream.

As far as I can tell the only way to support this would be to introduce a new top-level mutable local for user-provided compiler settings or to start storing those settings in the existing opts table which is already an argument to many functions, and modify all those calls to propagate the user-provided settings thru. Since there are over 30 places this happens that would need to be modified, I'm inclined towards the top-level mutable local, but maybe there's another way I haven't thought of.

Parsing with escape sequences

Here's a failing test case:

["\"abc\\\"def\""]="abc\"def"

Results in Error: ./fennel.lua:64: [string "return "abc\""]:1: unfinished string near <eof> in: return ("\"abc\\\"def\"")

Cannot use mangled identifiers in a fennel.eval environment

When evaluating fennel code at runtime, it's impossible to use identifiers that require mangling.

Test case:

>> ((. (require "fennel") :eval) "testEnv" {:env {:testEnv "hello"}})
"hello"
>> ((. (require "fennel") :eval) "test-env" {:env {:test-env "hello"}})
 

The test-env identifier in the evaluated code is mangled, but the :test-env key in the :env table is not, so the lookup fails.

:keyword syntax allowed characters

Based on the following line in the tutorial:

Strings that don't have spaces in them can use the :keyword syntax instead

I would expect any string that does not have spaces in it to be valid with the keyword colon syntax. However, it seems that some characters other than spaces can make such strings evaluate to nil.

For example:
>> :dog
;; => dog

>> :adj*
;; => nil

>> :string?
;; => nil

>> :butter+jam
;; => nil

I'm not sure if this is intended behavior (in which case the tutorial or other documentation should be updated to better indicate which characters are allowed in a :keyword string).

Looking at line 294 in fennel.lua, the pattern seems to match only alphanumeric characters (A-Z, a-z, 0-9), underscores, and hyphens.

My personal preference would be to allow more characters than those in the keyword syntax, since in my exploration of Fennel I've been using keywords as something analogous to symbols in other Lisps. I don't know if it's feasible to define the pattern to simply be "anything other than space or nonprinting characters," (and I understand not wanting to make Fennel into a clone of other Lisps) but maybe it would be possible to include some commonly used characters in other Lisps.

Alternately, at least getting some error when an invalid keyword literal is used would be good.

Scripting Terra

Is there a possibility for fennel to also work as an extension/scripter/metalang for Terra?

The only form/macro to be added would be terra, which works like fn or defn, the only complication being that it requires static types for the arguments of such function.

This is the most basic example on Terra's page:

C = terralib.includec("stdio.h")
terra hello(argc : int, argv : &rawstring)
    C.printf("Hello, Terra!\n")
    return 0
end

Which I believe could be the result of calling something like this in fennel:

(global C (terralib.incudec "stdio.h"))
(global hello 
    (terra [[argc int] [argv &rawstring]] 
        (do (C.printf "Hello, Fennel!\n") 0)))

Tutorial should mention varargs, ie Lua's `...`

While looking to see if Fennel supported Lua's ... varargs syntax, I noticed it's not in the tutorial. That would be a handy addition, especially if it mentions vararg to make it easy to find.

Trouble compiling code with require-macros

Trying to compile a short code snippet with macros from #37

Just as in #37 I have this two files (Note I changed defn to defn1 and -> to ->1 as in test-macros.fnl):

;; main.fnl
(require-macros "mymacros")

(defn1 add-seven-then-multiply-by-a-hundred [x]
  (->1 x (+ 7) (* 100)))

(print add-seven-then-multiply-by-a-hundred 93)

;; mymacros.fnl
{"->1" (fn [val ...]
         (each [_ elt (pairs [...])]
           (table.insert elt 2 val)
           (set elt.n (+ 1 elt.n))
           (set val elt))
         val)
 "defn1" (fn [name args ...]
           (list (sym "set") name
                 (list (sym "fn") args ...)))}

My folder structure, therefore, is such:

.             # I'm here
./Fennel/     # cloned git repository as of 09-07-2018 (latest commit: f2a3d3b)
./main.fnl
./mymacros.fnl

Compiling with ./Fennel/fennel --compile main.fnl fails with this error:

Compile error in 'set' ./mymacros.fnl:6: expected local var
stack traceback:
  [C]: in function 'error'
  ./Fennel/fennel.lua:368: in function 'assertCompile'
  ./Fennel/fennel.lua:903: in function 'getname'
  ./Fennel/fennel.lua:913: in function 'destructure1'
  ./Fennel/fennel.lua:953: in function 'destructure'
  ./Fennel/fennel.lua:1153: in function 'special'
  ./Fennel/fennel.lua:762: in function 'compile1'
  ./Fennel/fennel.lua:985: in function 'compileDo'
  ./Fennel/fennel.lua:1313: in function 'special'
  ./Fennel/fennel.lua:762: in function 'compile1'
  ./Fennel/fennel.lua:1095: in function 'special'
  ./Fennel/fennel.lua:762: in function 'compile1'
  ./Fennel/fennel.lua:846: in function 'compile1'
  ./Fennel/fennel.lua:1517: in function ?
  ./Fennel/fennel.lua:1592: in function ?
  ./Fennel/fennel.lua:1806: in function 'special'
  ./Fennel/fennel.lua:762: in function 'compile1'
  ./Fennel/fennel.lua:1517: in function ?
  [C]: in function 'xpcall'
  ./Fennel/fennel:66: in main chunk
  [C]: in ?

---

I've tried cutting out ->1 macro and replacing it with built-in ->.

(Showing changed files for clarity)

;; main.fnl
(require-macros "mymacros")

(defn1 add-seven-then-multiply-by-a-hundred [x]
  (-> x (+ 7) (* 100)))

(print add-seven-then-multiply-by-a-hundred 93)

;; mymacros.fnl
{ "defn1" (fn [name args ...]
           (list (sym "set") name
                 (list (sym "fn") args ...)))}

This gave me a following error:

Compile error in 'list' ./mymacros.fnl:3: unknown global in strict mode: list
stack traceback:
  [C]: in function 'error'
  ./Fennel/fennel.lua:368: in function 'assertCompile'
  ./Fennel/fennel.lua:539: in function 'symbolToExpression'
  ./Fennel/fennel.lua:813: in function 'compile1'
  ./Fennel/fennel.lua:779: in function 'compile1'
  ./Fennel/fennel.lua:1095: in function 'special'
  ./Fennel/fennel.lua:762: in function 'compile1'
  ./Fennel/fennel.lua:846: in function 'compile1'
  ./Fennel/fennel.lua:1517: in function ?
  ./Fennel/fennel.lua:1592: in function ?
  ./Fennel/fennel.lua:1806: in function 'special'
  ./Fennel/fennel.lua:762: in function 'compile1'
  ./Fennel/fennel.lua:1517: in function ?
  [C]: in function 'xpcall'
  ./Fennel/fennel:66: in main chunk
  [C]: in ?

Also, I've run test.lua in the repository and all tests passed.

fennelview only loads when in project root directory

If I start the fennel repl in my git checkout then fennelview loads fine. But if I start the repl from another directory fennelview isn't loaded.

I believe this is because fennelview is loaded with a local filename, and eventually ends up in a call to io.open which will open the file relative to the current working directory.

I'd be happy to put together a PR for a fix, but I'll need some pointers as to the right way to solve this. I noticed the fennel.path variable, perhaps this should include a notion of the "install directory" that works across a git checkout or a luarocks install?

fennel.searcher "fennel.lua:1366: attempt to index a string value (local 'options')"

My environment: Lua 5.3.4 on MacOS 10.13.3

fennel.searcher returns a function that takes an options table, then calls fennel_dofile with the filename and the options. However, require passes the name of the module rather than options.

Fennel/fennel.lua

Lines 1361 to 1368 in 1c46182

	local function dofile_fennel(filename, options)
	options = options or { accurate = true }
	local f = assert(io.open(filename, "rb"))
	local source = f:read("*all")
	f:close()
	options.filename = options.filename or filename
	return eval(source, options)
	end

So for example, when I run:

;; File: require-test.fnl
;; Env: fennel.lua and helpers.fnl are in same directory

(set fennel (require :fennel))
(table.insert package.searchers fennel.searcher)
(set helpers (require :helpers))

fennel_dofile ends up being called like fennel_dofile("./helpers.fnl", "helpers"). This results in an error when the code tries to set options.filename, so the require fails. This may be due to a difference in the way require works in Lua 5.1 and in 5.2+.

From the Lua 5.1 manual:

Once a loader is found, require calls the loader with a single argument, modname.

From the Lua 5.2 manual (same in 5.3):

Once a loader is found, require calls the loader with two arguments: modname and an extra value dependent on how it got the loader. (If the loader came from a file, this extra value is the file name.)

That said, I'm a bit puzzled because it seems from those descriptions that the loader will always be called with a string value, so it should also error on 5.1. (Just assuming that it works on 5.1 because the package.loaders used in the documentation becomes package.searchers in 5.2+)