lfe-rebar3/compile

The LFE rebar3 compiler plugin

compile's Issues

Add and maintain a CHANGELOG

As rebar3 and LFE progress, we sometimes see some radical changes here. It'd be nice to document them concisely in one place.

Don't ignore extra_src_dirs

This will make testing LFE code work "out of the box."

See this comment and the rest of the thread for details.

Add lfe-rebar3 to hex.pm?

I could write a patch if required. Improves visibility of the plugin.

Uncaught error when exporting undefined function

When you try to use lfe-rebar3 to compile an LFE module that tries to export undefined functions, it crashes rebar3. This means we're not matching the resulting error pattern there is was a bug in rebar3.

Environment

Name	Version
macOS	10.11.5
erlang	18.3.4
lfe	1.1.1
rebar3	3.2.0

Minimum Failing Example

rebar.config

%% -*- erlang -*-

{plugins, [
  {'lfe-compile',
   {git, "git://github.com/lfe-rebar3/compile.git",
    {tag, "0.4.0"}}}
]}.

{provider_hooks, [{pre, [{compile, {lfe, compile}}]}]}.

{overrides, [
  {override, 'lfe-compile', [
    {deps, [{lfe, {git, "git://github.com/rvirding/lfe.git", {tag, "1.1.1"}}}]}
  ]}
]}.

src/example.app.src

{application,example,[{vsn,"0.0.0"}]}.

src/example.lfe

(defmodule example (export (nothing 0)))

Compile results

{error,
 [{error,
   [{"/path/to/src/example.lfe",
     [{9999,lfe_lint,
       {unbound_func,{nothing,0}}}]}],
   []}],
 [],[]}

Apps in apps/* are ignored

I attempted to use lfe-compile with some LFE source located under apps/* instead of directly in src/. It does not appear to know how to look here. I started trying to add support, but then ran into the problems described in Issue #1.

(include-lib "lfe/include/clj.lfe") fails

error expanding (include-lib "lfe/include/clj.lfe"): #(none file enoent)

It seems the include path is not being set properly.

If I add the following to my rebar.config, it seems to work as desired:

{lfe_opts, [{i,"_build/default/lib"}]}.

post compile hook and {vsn, git} in .app.src file => relx release not generated

I'm not sure if this is related to this specific provider or to the rebar3 behaviour in general, as I didn't investigate it yet. However I'm posting in here since you might have an idea what this is related to.

I have got an LFE application that is built with rebar3 and this plugin.
It's all working perfectly fine. However, when I'm using a combination of {vsn, git} in the <app>.app.src file AND running this provider as a post compile hook, the git version is not translated to the git tag. Also the generated .app file remains with {vsn, git}.

When I'm changing the hook to run pre compile, everything is generated correctly while creating a relx release.

I assume this is related to rebar3 rather than lfe-compile plugin.
I thought it's worth mentioning it here anyway (and the fact that changing to pre helps).

EDIT: adding versions I have tested the above with:

lfe-compile: 0.7.0, 0.8.0-rc3
rebar3: 3.3.2, 3.3.3

Compiler output and configuration issue

Some year ago I created an LFE project to learning the language, here the GitHub link.

Original rebar.config didn't worked in its original state, so I've re-configured dependencies. Reading documentation I understood the need of having this rebar3 plugin properly configured. So I've added instructions presented here to the file.

Please be patient... But too much time is passed, I lost confidence with environment and I can't see the artifact of compilation. I would expect it on _build/default.

This how I modified my original rebar.config:

{erl_opts, [debug_info, {src_dirs, ["test"]}]}.
{lfe_first_files, []}.
{deps_dir, ["deps"]}.
{eunit_compile_opts, [
   {src_dirs, ["test"]}
  ]}.
{deps, [
   {lfe, {git, "https://github.com/rvirding/lfe.git", {"tag", "v1.3"}}},
   {ljson, {git, "https://github.com/lfex/ljson.git", "master"}},
   {ltest, {git, "https://github.com/lfex/ltest.git", {tag, "0.9.0"}}}
  ]}.
{plugins, [
   {'lfe-compile',
    {git, "git://github.com/lfe-rebar3/compile.git",
      {tag, "0.7.0"}}}
  ]}.
{provider_hooks, [
   {post, [{compile, {lfe, compile}}]}
  ]}.

This the output of rebar3 compile (or rebar3 lfe compile, the same):

===> Verifying dependencies...
===> Compiling ltest
 ~~>    Finding .lfe files ...
 ~~>    Compiling ./_build/default/plugins/ltest/src/ltest-runner.lfe ...
===> Plugin {'lfe-test',{git,"https://github.com/lfe-rebar3/test.git",
                                    {branch,"master"}}} not available. It will not be used.
===> Compiling lfe-clean
 ~~>    Finding .lfe files ...
 ~~>    Compiling ./_build/default/plugins/lfe-clean/src/lr3-cln-util.lfe ...
===> Plugin {'lfe-clean',{git,"https://github.com/lfe-rebar3/clean.git",
                                     {branch,"master"}}} not available. It will not be used.

If I need to post more info, please let me known.

Any help really appreciated! Thanks in advance.

lfe-compile needs updating for newest rebar3

I get stack traces when trying to use lfe-compile with a recent rebar3. I did not have time to port it myself, but I got far enough to notice the following API changes:

• rebar_app_info:state_or_new/2 no longer exists.
• rebar_dir:all_src_dirs/3 now takes a rebar_dict() instead of a rebar_app_info:t(). @tsloughter says that some info was moved from rebar_state to app_info. rebar_app_info:opts/1 can be used to pull the field out of the app_info.

There may be more. I hope this info helps to bootstrap the effort!

Add support for module hierarchies

Note that this is not true package support, since Erlang/LFE don't have such a concept.

Instead, this is about finding modules in a directory structure and creating a beam file name that is composed of the path elements of the module files parent directories.

This will mean that you can have a path structure like src/a/b/c/d.lfe and, once compiled, there would be a .beam file in ./ebin created with the name a.b.c.d.beam. Calling this module's functions from LFE code would be done in the usual manner: (a.b.c.d:some-func ...). The onus will of course be on the developer to ensure that the module is given the proper name: (defmodule a.b.c.d ...).

Add Duncan as owner on Hex

... once he makes an account.

"Unable to run pre hooks for 'compile', command 'compile' in namespace 'lfe' not found."

I am a newbie at Erlang, but am trying to set up a proper release with apps, which are lfe.

I think this is the simplest setup, it doesn't even have an lfe file, but I still get an error:

I use rebar3 to set up a release and an app:

$ rebar3 new release testrel
===> Writing testrel/apps/testrel/src/testrel_app.erl
===> Writing testrel/apps/testrel/src/testrel_sup.erl
===> Writing testrel/apps/testrel/src/testrel.app.src
===> Writing testrel/rebar.config
===> Writing testrel/config/sys.config
===> Writing testrel/config/vm.args
===> Writing testrel/.gitignore
===> Writing testrel/LICENSE
===> Writing testrel/README.md
$ cd testrel/apps
$ rm -rf testrel
$ rebar3 new app testapp
===> Writing testapp/src/testapp_app.erl
===> Writing testapp/src/testapp_sup.erl
===> Writing testapp/src/testapp.app.src
===> Writing testapp/rebar.config
===> Writing testapp/.gitignore
===> Writing testapp/LICENSE
===> Writing testapp/README.md
$ cd testapp
$ cat > rebar.config <<EOF
> {erl_opts, [debug_info]}.
> 
> {deps, [
>         {lfe, {git, "https://github.com/rvirding/lfe.git", {tag, "v0.10.1"}}}
>        ]}.
> 
> {plugins, [
>            {'lfe-compile', {git, "https://github.com/lfe-rebar3/compile.git", {tag, "0.2.1"}}}
>           ]}.
> 
> {provider_hooks, [
>    {pre, [{compile, {lfe, compile}}]}
>   ]}.
> 
> EOF
$ cd ../..

But now if I try to compile this release with skeleton app it doesn't work:

$ rebar3 compile
rebar3 compile
===> Verifying dependencies...
===> Fetching lfe ({git,"https://github.com/rvirding/lfe.git",{tag,"v0.10.1"}})
===> Compiling lfe
/home/dir/projects/testrel/_build/default/lib/lfe/src/lfe_eval.erl:128: Warning: variable 'M' shadowed in 'fun'
  ...
/home/dir/projects/testrel/_build/default/lib/lfe/src/lfe_ms.erl:302: Warning: function fetch_binding/2 is unused

===> Compiling testapp
===> Unable to run pre hooks for 'compile', command 'compile' in namespace 'lfe' not found.

(this exits with an error)
Note that it compile lfe once, and doesn't compile lfe-compile.

But note that it does work in the app:

cd apps/testapp/
$ rebar3 compile
rebar3 compile
===> Fetching lfe-compile ({git,"https://github.com/lfe-rebar3/compile.git",
                                {tag,"0.2.1"}})
===> Fetching lfe ({git,"https://github.com/rvirding/lfe.git",{tag,"v0.10.1"}})
===> Compiling lfe
/home/dir/projects/testrel/apps/testapp/_build/default/plugins/lfe/src/lfe_eval.erl:128: Warning: variable 'M' shadowed in 'fun'
 ...
/home/dir/projects/testrel/apps/testapp/_build/default/plugins/lfe/src/lfe_ms.erl:302: Warning: function fetch_binding/2 is unused

===> Compiling lfe-compile
===> Verifying dependencies...
===> Fetching lfe ({git,"https://github.com/rvirding/lfe.git",{tag,"v0.10.1"}})
===> Compiling lfe
/home/dir/projects/testrel/apps/testapp/_build/default/lib/lfe/src/lfe_eval.erl:128: Warning: variable 'M' shadowed in 'fun'
  ...
/home/dir/projects/testrel/apps/testapp/_build/default/lib/lfe/src/lfe_ms.erl:302: Warning: function fetch_binding/2 is unused

===> Compiling testapp
 ~~>    Finding .lfe files ...

(no error)
Note that this compiles lfe twice and does compile lfe-compile.

Am I doing something wrong? How is this meant to work?

Add support for generating app.src files from lfe.config files

Tasks:

• Update lcfg library
  • Provide definition of Hex.pm metadata
  • Standardize other configuration values on Hex.pm metadata values
  • Provide means of writing Hex.pm data to app.src
• Update compile plugin
  • Make the lcfg library a dependency (things may get tricky here ...)
  • Compile rebar.config file from lfe.config
  • Compile app.src file from lfe.config
  • Compile relx.config file from lfe.config (or maybe do this in an LFE relx plugin?)
  • Only with all of these done, proceed to compile .erl and .lfe files

For context on this issue, see the following tickets:

• lfeutre/lcfg#16
• lfeutre/lcfg#14

Add Hex badge

Make user 'oubiwann' the Hex.pm maintainer

@yurrriq: I've finally added myself to hex.pm; can you make me the owner now?

Thanks!

.beam files end up in the wrong place

Rather than ./ebin, they should be in ./_build/{{profile}}/lib/{{name}}/ebin.

---

Cross post from quasiquoting/lfe-rebar3-plugin#10.

rebar3 lfe compile fails when using (include-lib)

Running rebar3 lfe compile as standalone fails if you include libraries from dependecies.

How to reproduce

$ rebar3 version
rebar 3.0.0+build.272.ref7d29b74 on Erlang/OTP 18 Erts 7.2
$ rebar3 new lib includebug
$ cd includebug
$ rm src/includebug.erl
$ cat rebar.config                                                               
{erl_opts, [debug_info]}.
{deps, [
    {exemplar, {git, "[email protected]:lfex/exemplar.git", {branch, "master"}}}
]}.

{provider_hooks, [
  {post, [{compile, {lfe, compile}}]}
 ]}.

{plugins, [
  {'lfe-compile',
    {git, "https://github.com/lfe-rebar3/compile.git", {tag, "0.3.0"}}}
]}.
$ cat src/includebug.lfe
(defmodule includebug
  (export (one 1)))

(include-lib "exemplar/include/html-macros.lfe")

(defun one (x) (+ 1 x))
$ rebar3 lfe compile
===> Uncaught error in rebar_core. Run with DEBUG=1 to see stacktrace

Error is:

{4,lfe_macro,
                                  {expand_macro,
                                   ['include-lib',
                                    "exemplar/include/html-macros.lfe"],
                                   badarg}}]}

Compiling with rebar3 compile works.

Add tests

I would imagine we can add rebar3 as a test dependency and take cues from rebar_compile_SUITE. We should make sure to cover some of the bad error cases that have come up.

Once erlang/rebar3#1331 gets sorted, we should ensure test/*.lfe get compiled and the .beam files end up in the right place.

We should also test a multi-module .lfe file and make sure that works correctly. I don't do this personally, but lfe-compile needs to handle it.

Tag 0.3.0

I don't have permissions, so I can't do it. Maybe you could add me, @oubiwann 😁

N.B. here's a hack I'm using in the mean time:

{overrides,
 [{override, 'lfe-compile',
   [{deps,
     [{lfe, {git, "git://github.com/rvirding/lfe.git", {tag, "1.0"}}}]}]}]}.

Handle syntax errors

Update the case expression in lr3_comp:compile/3 to handle more failure patterns, e.g.

{error,[],
 [{"bad-syntax.lfe",
   [{42,lfe_parse,{illegal,')'}}]}],
 []}}