hoelzro/lua-repl

A Lua REPL implemented in Lua for embedding in other programs

REPL.lua - a reusable Lua REPL written in Lua, and an alternative to /usr/bin/lua

This project has two uses:

• An alternative to the standalone interpreter included with Lua, one that supports things like plugins, tab completion, and automatic insertion of return in front of expressions.

• A REPL library you may embed in your application, to provide all of the niceties of the standalone interpreter included with Lua and then some.

Many software projects have made the choice to embed Lua in their projects to allow their users some extra flexibility. Some of these projects would also like to provide a Lua REPL in their programs for debugging or rapid development. Most Lua programmers are familiar with the standalone Lua interpreter as a Lua REPL; however, it is bound to the command line. Until now, Lua programmers would have to implement their own REPL from scratch if they wanted to include one in their programs. This project aims to provide a REPL implemented in pure Lua that almost any project can make use of.

This library also includes an example application (rep.lua), which serves as an alternative to the standalone interpreter included with Lua. If the lua-linenoise library is installed, it uses linenoise for history and tab completion; otherwise, it tries to use rlwrap for basic line editing. If you would like the arrow keys to work as expected rather than printing things like ^[[A, please install the lua-linenoise library or the rlwrap program.

Project Goals

• Provide REPL logic to Lua programs that include this module.

• Be extensible through polymorphism and plugins.

• Abstract away I/O, so you can run this REPL on the command line or in your own event loop and expect the same behavior.

Building

• You need Luadoc (http://keplerproject.github.com/luadoc/) installed to build the documentation.

• You need Test.More (http://fperrad.github.com/lua-TestMore/testmore.html) installed to run the tests.

Compatibility

The current version of the software runs on Lua 5.1, LuaJIT ?.? etc. A port to Lua 5.2 is envisaged, but is not at this stage a priority. Since it is written purely in Lua, it should work on any platform that has one of those versions of Lua installed.

XXX Check which version of LuaJIT this works with XXX Check that it works with other Lua interpreters

Installation

You can install lua-repl via LuaRocks:

luarocks install luarepl

You can also install it by hand by copying the repl directory to a location in your package.path, and copying rep.lua to somewhere in your PATH.

Recommended packages

rep.lua works best if you also have linenoise installed, available from https://github.com/hoelzro/lua-linenoise. rep.lua will fallback to using rlwrap if you have that as well; without either of these, you will have command editing, history, or other features generally provided by readline.

Features

rep.lua prints the results of simple expressions without requiring a return or a = in front of it. If linenoise is installed, it also offers persistent history and tab completion. It also offers a number of plugins; see plugins.md for a list of plugins that come with lua-repl.

Backwards Compatibility Changes

Removal of default plugins in 0.8

Lua REPL 0.8 breaks backwards compatability by disabling the loading of the default plugins (currently linenoise, rlwrap, history, completion, and autoreturn) if an rcfile is found for a user. This is so that plugins may not be forced onto a user if they don't want them, or play tricks with their setup (see issue #47). If you would like to continue using these plugins, please put the following code into your ~/.rep.lua:

if repl.VERSION >= 0.8 then
  -- default plugins
  repl:loadplugin 'linenoise'
  repl:loadplugin 'history'
  repl:loadplugin 'completion'
  repl:loadplugin 'autoreturn'
end

-- suppress warning message
repl.quiet_default_plugins = true

As mentioned in the code snippet, repl.quiet_default_plugins suppresses the warning. You can remove this after upgrading to Lua REPL 0.8.

lua-repl's Issues

Loading the history file fails if no such file exists

Continuous integration on Travis

Hi,

Would be interested in adding Travis integration for this project? I have implemented it on a branch and would open a pull request if you are interested

Thanks

Add color plugin

Something similar to pry's/bpython's/fish's coloration would be cool, for both entered code and things like stack traces.

Add __complete support for the completion plugin

Some objects may like to offer support for custom completion results (for example, objects may not want "private" methods displayed in a REPL, or objects using an __index metamethod). We should allow objects to provide a __complete metamethod to customize this behavior.

Should we prevent global assignment in plugins?

Users new to Lua may accidentally contaminate the global namespace when writing plugins; we should probably prevent common mistakes such as this.

Add --help to rep.lua

Add --help option to help new users discover how to use the REPL.

Write a luaish plugin

luaish is another advanced Lua REPL; it would be nice to provide a plugin for lua-repl that provides its features.

`repl.utils` module is missing

Hello! The new repl.utils module is missing in the 0.6 rockspec, which causes repl.lua to crash with a message like this:

/usr/local/bin/lua: /usr/local/share/lua/5.2/repl/init.lua:32: module 'repl.utils' not found:No LuaRocks module found for repl.utils
    no field package.preload['repl.utils']
    no file '/home/mpeterv/.luarocks/share/lua/5.2/repl/utils.lua'
    no file '/home/mpeterv/.luarocks/share/lua/5.2/repl/utils/init.lua'
    no file '/usr/local/share/lua/5.2/repl/utils.lua'
    no file '/usr/local/share/lua/5.2/repl/utils/init.lua'
    no file '/usr/local/lib/lua/5.2/repl/utils.lua'
    no file '/usr/local/lib/lua/5.2/repl/utils/init.lua'
    no file './repl/utils.lua'
    no file '/home/mpeterv/.luarocks/lib/lua/5.2/repl/utils.so'
    no file '/usr/local/lib/lua/5.2/repl/utils.so'
    no file '/usr/local/lib/lua/5.2/loadall.so'
    no file './repl/utils.so'
    no file '/home/mpeterv/.luarocks/lib/lua/5.2/repl.so'
    no file '/usr/local/lib/lua/5.2/repl.so'
    no file '/usr/local/lib/lua/5.2/loadall.so'
    no file './repl.so'
stack traceback:
    [C]: in function 'require'
    /usr/local/share/lua/5.2/repl/init.lua:32: in main chunk
    [C]: in function 'require'
    /usr/local/share/lua/5.2/repl/sync.lua:19: in main chunk
    [C]: in function 'require'
    /usr/local/share/lua/5.2/repl/console.lua:23: in main chunk
    [C]: in function 'require'
    /usr/local/lib/luarocks/rocks/luarepl/0.6-1/bin/rep.lua:23: in main chunk
    [C]: in ?

Trying to load a plugin that previously failed will fail

...with the message "plugin has already been loaded"

Allow REPL "subclasses" to use method advice

Plugins currently benefit from the before, after, around, and override objects. It would be nice to provide these to objects that extend the core REPL object.

Pretty print cannot handle function

Thanks for working on the repl!

You're probably aware of this but I'm opening this issue just for the record.
The pretty_print plugin throws an error when trying to pretty print a function:

...plugins/pretty_print.lua:184: Cannot print type 'function'

There should at least be some sort of stub for this so the repl is usable for debugging, since many real-world tables would contain functions...

Add test for iffeature

Suppress readline when TERM=dumb

I'm trying to use lua-repl as a subprocess from within Emacs, but it crashes:

rlwrap: Oops, crashed (caught SIGFPE) - this should not have happened!
If you need a core dump, re-configure with --enable-debug and rebuild
Resetting terminal and cleaning up...

Process lua-repl finished

Emacs handles all the niceties of rlwrap, so using it is redundant. Using rlwrap is unlikely to work when TERM=dumb, so we should check for that before attempting to use it.

Add tests for pretty print plugin

See also #41.

The pretty_print plugin currently just prints to standard error; we need to either override io.stderr before the plugin is loaded (which is quick and easy, but I don't care for), or devise a mechanism through which the output handle is passed to the plugin.

pretty_print plugin blows up on tables with functions as keys

For example:

> f = { [pairs] = 1, [print] = 2 }
> f
lua5.1: ./repl/plugins/pretty_print.lua:114: attempt to compare two function values
stack traceback:
    ./repl/plugins/pretty_print.lua:114: in function <./repl/plugins/pretty_print.lua:109>
    [C]: in function 'tsort'
    ./repl/plugins/pretty_print.lua:134: in function 'sortedpairs'
    ./repl/plugins/pretty_print.lua:204: in function 'dump'
    ./repl/plugins/pretty_print.lua:251: in function 'displayresults'
    repl/init.lua:103: in function 'old_value'
    repl/init.lua:227: in function 'handleline'
    ./repl/sync.lua:33: in function 'run'
    rep.lua:51: in main chunk
    [C]: ?

This is because Lua only defines less than for numbers, strings, or anything with an __lt metamethod. We could check for this explicitly, or we could wrap the check in a pcall; I don't know how poorly the latter would perform.

Can't run tests: test-utils missing

Uhm .. Lua can't find it and I can neither find it in luarocks nor in your github repositories.
Any ideas where I could obtain it?

Figure out how plugins should store values

Plugins currently store data that must persist between method calls on self. This works for now, but in the future, plugins will have to be extra careful that they don't clobber other plugins' data. We can't store data in local variables in the plugin script, because a plugin may be loaded by multiple REPLs in the same interpreter. (However, we could always re-load the plugin code for each individual REPL). We could encourage plugin authors to use local tables with weak keys and use the REPL object as a lookup key to get at the plugin's data, but that seems overly complicated, and I'd like the barrier to entry for plugin development to be as low as possible. So we need a way to do this that is safe and easy to use.

Pretty-printer doesn't cope with mixed integer/string keyed tables

> a = { thing = "this" }
> a[1] = 2
> a
lua: attempt to compare string with number

A quick and dirty fix is to replace the start of sortedpairs in pretty_print.lua with

  for k in pairs(t) do
    if type(k) == "string" then
       keys[#keys + 1] = k
    end
  end
  tsort(keys)
  for k in ipairs(t) do
    keys[#keys + 1] = k
  end

Ctrl-C/os.exit() prevent the history plugin from saving

testing

Feature Request for plugin testing

Use HOMEPATH (not HOME) on Windows

Add moonscript plugin

Moonscript is a language that compiles to Lua (much like Coffeescript -> Javascript); it would be neat to have a plugin to allow the REPL to understand it.

Allow for easier loading of plugins from a running instance/prototyping plugins from a running instance

Steal ideas from ilua

ilua is another alternative Lua REPL. We should steal, er, borrow, ideas from it.

Some terms that don't appear in _G appear in completions

Ex. before & after

Plugin to allow access to the REPL object from within the REPL

Add test for using advice from within ifplugin/iffeature

Handy debug interface plugin

 <       rjek> | hoelzro: Also for bonus points, use the debug interface to discover where a function is defined, fetch the source, extract the function prototpe, display it below the prompt ;)

Create an interactive debugger plugin

From the standalone REPL, it might be nice to be able to have an interactive debugger on exceptions. Also, it might be nice to have a "drop to lua-repl" hook function for use with xpcall.

Process command line options with rep.lua

rep.lua currently does not process any command line options (for example, -l in the standalone interpreter). We should add this so the command line interface feels more like lua.

Allow clearing of input buffer

Sometimes, you'll end up typing something like this:

print("foo')

and lua-repl will confusingly ask you for more input, and it's not always clear how to complete the input. There should be a sort of "reset input buffer" command to get around this.

Steal ideas from other REPLs

A lot of REPLs exist, some more advanced than others. We should steal their ideas.

• http://en.wikipedia.org/wiki/REPL
• pry
• ipython
• bpython
• Devel::REPL
• Factor REPL
• Dr. Racket
• https://github.com/tpope/vim-foreplay
• Xiki

Plugin reloading support

Figure out how plugins should be configured

Plugins should have the ability to be configured in a user's .rep.lua file. Default plugins should also allow their default configurations to be overriden in .rep.lua. In addition, some plugins may want to allow the user to change settings during use of the REPL itself. If an environment using lua-repl doesn't use a user RC file, that environment should still be able to provide custom configuration to the plugins it provides its users on a case-by-case basis.

Plugin that preserves locals between lines

Ideas for how this could work:

• Use debug.sethook to catch the return of our compiled chunk and grab its locals
• Rewrite source code/bytecode before evaluation
• Custom interpreter patch to "pcall and get locals"
• Custom module that dips into internals to "pcall and get locals"
• Add a chunk of source code to the end of the chunk to grab the locals

Offer alternate implementations

Currently, only the example implementation exists. It would be nice to offer other implementations, especially if they demonstrated capabilities that aren't available on the command line.

• GTK
• IRC bot
• Vim
• AwesomeWM (I started something here: https://github.com/hoelzro/luarepl-evlib)

Fancier completion support

luaprompt (another library with goals similar to lua-repl) has custom completion when you're requireing something: https://github.com/dpapavas/luaprompt/blob/3c81ad3b0b003b206aba01688259c0c49c6fea82/ChangeLog#L58

This would be really nice to have for lua-repl; I was thinking of adding a fancier completion plugin that actually parses the code and provides intelligent completions based on the current parser state.

Verify that the REPL works with different versions of Lua

I've only tested this module against Lua 5.1. It should be tested against (at least) the following versions:

• Lua 5.2
• LuaJIT
• LuaJ

pretty_print

Hi there,

Not really an issue (but there's no good place to chat on github, it's a shame).

I tried to load the pretty_print plugin, and it seems to be crashing on quite a few types (functions, ...). It wasn't loaded by default, so I suspect that it's being developed. Do you need a hand on that? What's your plan/timeline with this repl project?

So far it's the most promising repl I found for Lua, and having the full pretty print support with colors, and table expansion would be really awesome.

Cheers,
Clement.

Add terra plugin

So someone wrote this neat low level language that integrates well with Lua:

https://github.com/zdevito/terra

A plugin should exist to evaluate terra expressions rather than Lua ones!

Improve inter-plugin communication

Plugins may currently provide services to others by putting custom methods on the REPL object and advertising features. Plugins may consume services provided by others by checking for a particular plugin or feature's presence, and calling the methods provided by that plugin. This may cause problems in the future if plugins can't decide on unique names.

Support multi line pastes with linenoise

It appears that multi-line pastes are not supported with linenoise. Whenever I past several lines, only the first one shows up in lua-repl.

With the readline backend (i.e. linenoise.so not installed in ~/.luarocks/lib/lua/5.1), I can paste multiple lines.

Create a formatting plugin

Sometimes, when I'm working with the REPL, I would like specific formatting to applied to different types. For example, instead of seeing this:

> math.pi
3.1415926535898

I might prefer this:

> math.pi
3.14

Or instead of this:

> 0x1000 - 1
4095

I might prefer this:

> 0x1000 - 1
0xFFF

Another example:

> 2 ^ 32
4,294,967,296

ilua offers something like this, and I definitely think this functionality should be available in lua-repl. Of course, the implementation of this plugin raises some questions:

• We need to detect number separator (, vs .) from a user's locale, perhaps falling back to a suitable default if we can't.
• We need to detect if numbers are typically separated into thousands, hundreds, ten-thousands.
• We need to provide a way for a user to configure this plugin (in case he/she would like to have numbers rounded to the tens, or the thousands, etc)
• We should provide a way to toggle this at runtime (in case you want to turn it on/off temporarily)
• We should be able to alter the configuration at runtime (in case you want to change the precision, or stop printing numbers in hexidecimal, etc)

Other nice-to-haves:

• Allow the user to specify rules to figure out which formatting options to use. For example, if I detect that a user entered an expression involving math on a literal specified in hexidecimal (ex. 0x1000 - 1), they should be able to tell the REPL to display the result in hexidecimal format.
• Such a plugin could possibly interfere with other printing plugins (ex. pretty_print), so maybe we should have a "stack" of formatters so they can leverage each other. We might even want a "format level" on formatting plugins so that their relative order doesn't screw things up. Instead of an explicit stack, we could provide a format_result method that must return a string; formatting plugins can use the around plugin object to fallback on "lower level" formatters.

5.3 support

Drop the setfenv dependency

I know that this probably isn't easy, as lua-repl makes heavy use of modified function environments.

• There are some pitfalls when emulating setfenv() in newer versions of Lua.
  E.g. functions which are called in a modified environment doesn't inherit it, but instead they use the original environment. (Try calling dofile() in your .rep.lua.)
• Also I personally think that the environment should be as clean as possible as this might confuse programmers.

Support for loading a file and opening a repl in its lexical scope

One thing that makes the repl significantly less useful is that the only way I can load a file into the repl and have access to its internal functions is by making them globals. So I'm forced to choose between convenience and correctness, or more often, I just delete the "local" before each definition before loading up the file. Obviously this is less-than-ideal.

What I would like to see is a feature that allows you to load up a file but run a repl inside the lexical scope of the file. Sort of like if you replaced the final return of the file with debug.debug(), except debug.debug doesn't actually honor lexical scope, so it's kinda useless.

Is there a better way to do this that already exists? I have a hard time believing that there's no way to do this yet, but I haven't found anything that supports convenient repl programming yet, though of course this project is a huge improvement over the hilairously-crippled repl that ships with stock lua.

Make rep.lua only load default plugins if no rcfile is found

To address #47:
#0.7

• Move rlwrap functionality into its own plugin
• Print a warning if an rcfile is found and repl.quiet_default_plugins is not set to true
  • Allow plugins to return values, and have those values be propagated through loadplugin
  • Print the warning
• Add a method to prevent the loading of a particular plugin
  #0.8
• Change rep.lua to only load plugins if no rcfile is found
• Deprecate plugin suppression
  #0.9
• Remove plugin suppression

arrow keys compatibility?

arrow keys doesn't work in ubuntu 16.04 + gnome-terminal for me

for example, when I run rep.lua and press up arrow it becomes ^[[A.

Documentation Improvements

I've been told by some that the documentation could use some work. Here's what I have in mind:

• More clearly reference PLUGINS.md from README.md
• More clearly reference rep.lua from README.md
• Make sure that autocompletion is talked up in plugins.md (and mention in readme that many default/optional behaviors are present there)
• Make sure documentation on ~/.rep.lua is clear
• Move docs into doc/

The rule of thumb is that a first-time user of Lua should be able to fairly quickly understand how to install, run, and customize the example REPL (rep.lua), and seasoned users should be able to write plugins and contribute to the project with ease.

Add __pretty support for the pretty_print plugin

Values currently have no control over how they are printed; I think that the pretty print plugin should use a metamethod named __pretty to allow custom printing logic.

On that train of thought, having rawprint may be a good idea.