Giter Site home page Giter Site logo

replmaker.jl's Introduction

Build Status

REPLMaker

The idea behind ReplMaker.jl is to make a tool for building (domain specific) languages in julia.

Suppose you've invented some language called MyLang and you've implemented a parser that turns MyLang code into julia code which is then supposed to be executed by the julia runtime. With ReplMaker.jl, you can simply hook your parser into the package and ReplMaker will then create a REPL mode where end users just type MyLang code and have it be executed automatically.

My hope is for this to be useful to someone who implements a full language or DSL in Julia that uses syntax not supported by Julia's parser and doesn't want to deal with the headache of making their own REPL mode.

To download ReplMaker, simply do

pkg> add ReplMaker

Examples

Example 1: Expr Mode

Click me!

Suppose we want to make a very simple REPL mode which just takes whatever input we provide and returns its quoted Expr form. We first make a parsing function,

julia> using ReplMaker

julia> function parse_to_expr(s)
           quote Meta.parse($s) end
       end
test_parser (generic function with 1 method)

Now, we can simply provide that parser to the initrepl function

julia> initrepl(parse_to_expr, 
                prompt_text="Expr> ",
                prompt_color = :blue, 
                start_key=')', 
                mode_name="Expr_mode")
REPL mode Expr_mode initialized. Press ) to enter and backspace to exit.

As instructed, we simply press the ) key and the julia> prompt is replaced

Expr>  

and as desired, we now can enter Julia code and be shown its quoted version.

Expr> 1 + 1
:(1 + 1)

Expr> x ^ 2 + 5
:(x ^ 2 + 5)

Next, we might notice that if we try to do a multiline expression, the REPL mode craps out on us:

Expr> function f(x)
:($(Expr(:incomplete, "incomplete: premature end of input")))

This is because we haven't told our REPL mode what constitutes a valid, complete line. ReplMaker.jl exports a function complete_julia that will tell you if a given expression is a complete julia-expression. If you are using ReplMaker.jl for a DSL that has different parsing semantics from julia, you may need to roll your own analogous function if you want to have multi-line inputs.

To use complete_julia to check if an expression is complete, we just pass it as a keyword argument to to initrepl:

julia> initrepl(parse_to_expr,
                prompt_text="Expr> ",
                prompt_color = :blue,
                start_key=')',
                mode_name="Expr_mode",
                valid_input_checker=complete_julia)
┌ Warning: REPL key ')' overwritten.
└ @ ReplMaker ~/.julia/packages/ReplMaker/pwo5w/src/ReplMaker.jl:86
REPL mode Expr_mode initialized. Press ) to enter and backspace to exit.

Expr> function f(x)
          x + 1
      end
:(function f(x)
      #= none:2 =#
      x + 1
  end)

Example 2: Reverse Mode

Click me!

This is an example of using a custom REPL mode to not change the meaning of the input code but instead of how results are shown. Suppose we have our own show-like function which is just Base.show, but will print Vectors and Tuples backwards

backwards_show(io, M, x) = (show(io, M, x); println(io))
backwards_show(io, M, v::Union{Vector, Tuple}) = (show(io, M, reverse(v)); println(io))

We can make a quick and dirty REPL mode that uses this rather than Base.show directly:

julia> initrepl(Meta.parse,
                show_function = backwards_show,
                prompt_text = "reverse_julia> ",
                start_key = ')',
                mode_name = "reverse mode")
REPL mode reverse mode initialized. Press ) to enter and backspace to exit.

reverse_julia> x = [1, 2, 3, 4]
4-element Array{Int64,1}:
 4
 3
 2
 1

The printing was reversed, but we can check to make sure the variable itself was not:

julia> x
4-element Array{Int64,1}:
 1
 2
 3
 4

Example 3: Big Mode

Click me!

For performance reasons, Julia defaults to 64 bit precision but sometimes you don't care about speed and you don't want to juggle the limitations of 64 bit precision in your head. You could just start wrapping every number in your code with big but that sounds like something the REPL should be able to do for you. Fortunately, it is!

using ReplMaker 

function Big_parse(str)
    Meta.parse(replace(str, r"[\+\-]?\d+(?:\.\d+)?(?:[ef][\+\-]?\d+)?" => x -> "big\"$x\""))
end

julia> initrepl(Big_parse, 
                prompt_text="BigJulia> ",
                prompt_color = :red, 
                start_key='>', 
                mode_name="Big-Mode")
REPL mode Big-Mode initialized. Press > to enter and backspace to exit.

Now press > at the repl to enter Big-Mode

BigJulia>  factorial(100)
93326215443944152681699238856266700490715968264381621468592963895217599993229915608941463976156518286253697920827223758251185210916864000000000000000000000000

BigJulia>  factorial(100.0)
9.332621544394415268169923885626670049071596826438162146859296389521759999323012e+157

BigJulia>  factorial(100.0)^2
8.709782489089480079416590161944485865569720643940840134215932536243379996346655e+315

Example 4: Lisp Mode

Click me!

The package LispSyntax.jl provides a string macro for parsing lisp-style code into julia code which is then evaluated, essentially creating a lispy language embedded in julia.

julia> lisp"(defn fib [a] (if (< a 2) a (+ (fib (- a 1)) (fib (- a 2)))))"
fib (generic function with 1 method)

julia> lisp"(fib 30)"
832040

Awesome! To make this really feel like its own language, it'd be nice if it had a special REPL mode, so let's make one. For this, we're going need a helper function valid_sexpr to tell ReplMaker if we pressed return because we were done writing our input or if we wanted to write a multi-line S-expression.

using LispSyntax, ReplMaker
using REPL: REPL, LineEdit; using LispSyntax: ParserCombinator

lisp_parser = LispSyntax.lisp_eval_helper

function valid_sexpr(s)
  try
    LispSyntax.read(String(take!(copy(LineEdit.buffer(s)))))
    true
  catch err
    isa(err, ParserCombinator.ParserException) || rethrow(err)
    false
  end
end

Great, now we can create our repl mode using the function LispSyntax.lisp_eval_helper to parse input text and we'll use valid_sexpr as our valid_input_checker.

julia> initrepl(LispSyntax.lisp_eval_helper,
                valid_input_checker=valid_sexpr,
                prompt_text="λ> ",
                prompt_color=:red,
                start_key=")",
                mode_name="Lisp Mode")
REPL mode Lisp Mode initialized. Press ) to enter and backspace to exit.

λ> (defn fib [a] 
    (if (< a 2) 
      a 
      (+ (fib (- a 1)) (fib (- a 2)))))
fib (generic function with 1 method)

λ> (fib 10)
55

Tada!

Modifier keys

Click me!

Arbitrary key combinations to enter REPL modes are not yet allowed, but you can currently use the CTRL and ALT (also known as META) keys as modifiers for entering REPL modes. For example, passing the keyword argument start_key="\\C-g" to the initrepl function will make it so that holding down on the CTRL key and pressing g enters the specified mode.

Likewise, specifying start_key="\\M-u" will make it so that holding ALT (aka META) and pressing u will enter the desired mode.

Creating a REPL mode at startup time

To add a custom REPL mode whenever Julia starts, add to ~/.julia/config/startup.jl code like:

atreplinit() do repl
    try
        @eval using ReplMaker
        @async initrepl(
            apropos;
            prompt_text="search> ",
            prompt_color=:magenta,
            start_key=')',
            mode_name="search_mode"
        )
        catch
        end
    end
end

Packages using ReplMaker.jl

replmaker.jl's People

Contributors

brenhinkeller avatar c42f avatar dilumaluthge avatar fingolfin avatar harrisongrodin avatar ilyaorson avatar masonprotter avatar nhdaly avatar nickrobinson251 avatar thautwarm avatar tuckertwo avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

replmaker.jl's Issues

ReplMaker and OhMyREPL conflict

Problem

OhMyREPL is a package that enhances the base Julia REPL, in part by making various keys do useful things, like making { insert a closing curly bracket (and making it such that when one presses } while in front of to the } character, it moves over the latter and inserts nothing). To do so, it rebinds various keys, many of which are used (by other packages) to enter REPLs (for example, } is used by Reduce.jl). If ReplMaker detects that a key is in use by another package, it spits out a warning and does nothing else, causing the former keybind to break.

Examples

Note: a Unicode combining underscore (U+0332, ̲) follows the current cursor position in these examples.

Errant behavior

julia> using OhMyREPL, ReplMaker
[ Info: Precompiling OhMyREPL [5fb14364-9ced-5910-84b2-373655c76a03]
[ Info: Precompiling ReplMaker [b873ce64-0db9-51f5-a568-4457d8e49576]

julia>  ̲

Press {:

julia> { ̲}

Press }:

julia> {} ̲

Throw another keybind in:

julia> initrepl(identity, start_key='}')
┌ Warning: REPL key '}' overwritten.
└ @ ReplMaker ~/.julia/packages/ReplMaker/SZ5Aa/src/ReplMaker.jl:96
REPL mode mylang initialized. Press } to enter and backspace to exit.
"Prompt(\"myrepl> \",...)"

(The REPL mode works)

Mess with braces again
Press {:

julia> { ̲}

Press }:

julia> {} ̲}

Expected behavior

julia> using OhMyREPL, ReplMaker
[ Info: Precompiling OhMyREPL [5fb14364-9ced-5910-84b2-373655c76a03]
[ Info: Precompiling ReplMaker [b873ce64-0db9-51f5-a568-4457d8e49576]

julia> initrepl(identity, start_key='}')
┌ Warning: REPL key '}' overwritten.
└ @ ReplMaker ~/.julia/packages/ReplMaker/SZ5Aa/src/ReplMaker.jl:96
REPL mode mylang initialized. Press } to enter and backspace to exit.
"Prompt(\"myrepl> \",...)"

julia> 

Mess with braces again
Press {:

julia> { ̲}

Press }:

julia> {} ̲

(The REPL mode still works, but the keybind is triggered)

Overwriting REPL mode

I may missing it, but is there straight-forward way of overwriting the current REPL instance by changing the prompt_text or mode_name? Looking for something similar to the Pkg REPL mode where the name changes after you activate an environment.

Lisp Syntax Example Fails

Upon:

initrepl(LispSyntax.lisp_eval_helper,
valid_input_checker=valid_sexpr,
prompt_text="λ> ",
prompt_color=:red,
start_key=")",
mode_name="Lisp Mode")

I get:

ERROR: MethodError: no method matching setup_interface(::REPL.BasicREPL)
Closest candidates are:
setup_interface(!Matched::REPL.LineEditREPL; hascolor, extra_repl_keymap) at /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/st
dlib/v1.8/REPL/src/REPL.jl:908
setup_interface(!Matched::REPL.LineEditREPL, !Matched::Bool, !Matched::Any) at /Applications/Julia-1.8.app/Contents/Resources/julia/share/julia/
stdlib/v1.8/REPL/src/REPL.jl:911
Stacktrace:
[1] initrepl(parser::typeof(LispSyntax.lisp_eval_helper); prompt_text::String, prompt_color::Symbol, start_key::String, repl::REPL.BasicREPL, mod
e_name::String, show_function::Nothing, show_function_io::Base.TTY, valid_input_checker::Function, keymap::Dict{Char, Any}, completion_provider::R
EPL.REPLCompletionProvider, sticky_mode::Bool, startup_text::Bool)
@ ReplMaker ~/.julia/packages/ReplMaker/ViCwA/src/ReplMaker.jl:64
[2] top-level scope
@ none:1

feature request: enter repl when called

Sometimes I want to give a user the opportunity to intervene in some otherwise-automated process if a certain condition is satisfied.

As a toy example, consider an AbstractDict object where upon inserting a key-value, if the key exists and isinteractive(), the user is dropped into a REPL mode that lets them inspect the contents of the dictionary, the pair being added, and decide what to do.

Currently I don't think ReplMaker can be used for this case, because it can only add REPL modes which are triggered from a key-press in the default Julia REPL. What would be needed is something that returns a REPL object and runs it on call. I don't think this would require too much on top of the REPL stdlib, however the stdlib does not have a very good (or well documented) API for this sort of thing, so I think it would be worth doing here.

Unable to rewrite REPL key

An error appears when trying to overwrite the shell model key with julia 1.8.

julia> using ReplMaker

julia> function parse_to_expr(s)
                  quote Meta.parse($s) end
              end
parse_to_expr (generic function with 1 method)

julia> initrepl(parse_to_expr,
                       prompt_text="Expr> ",
                       prompt_color = :blue,
                       start_key=';',
                       mode_name="Expr_mode")
┌ Warning: REPL key ';' overwritten.
└ @ ReplMaker C:\Users\ilyao\.julia\packages\ReplMaker\ViCwA\src\ReplMaker.jl:96
ERROR: MethodError: no method matching deepcopy_internal(::Base.Threads.SpinLock)
Closest candidates are:
  deepcopy_internal(::Base.AbstractLock, ::IdDict) at deepcopy.jl:130
  deepcopy_internal(::Any, ::IdDict) at deepcopy.jl:53
  deepcopy_internal(::BigInt, ::IdDict) at gmp.jl:794
  ...
Stacktrace:
  [1] deepcopy_internal(x::Base.GenericCondition{Base.Threads.SpinLock}, stackdict::IdDict{Any, Any})
    @ Base .\deepcopy.jl:143
  [2] deepcopy_internal(x::Any, stackdict::IdDict{Any, Any}) (repeats 4 times)
    @ Base .\deepcopy.jl:65
  [3] deepcopy_internal(x::Any, stackdict::IdDict{Any, Any})
    @ Base .\deepcopy.jl:76
  [4] deepcopy_internal(x::Dict{Char, Any}, stackdict::IdDict{Any, Any})
    @ Base .\deepcopy.jl:125
  [5] deepcopy_internal(x::Any, stackdict::IdDict{Any, Any})
    @ Base .\deepcopy.jl:65
  [6] deepcopy_internal(x::Dict{Symbol, REPL.LineEdit.Prompt}, stackdict::IdDict{Any, Any})
    @ Base .\deepcopy.jl:125
  [7] deepcopy_internal(x::Any, stackdict::IdDict{Any, Any}) (repeats 2 times)
    @ Base .\deepcopy.jl:65
  [8] deepcopy_internal(x::Any, stackdict::IdDict{Any, Any})
    @ Base .\deepcopy.jl:76
  [9] deepcopy_internal(x::Dict{Char, Any}, stackdict::IdDict{Any, Any})
    @ Base .\deepcopy.jl:125
 [10] deepcopy_internal(x::Any, stackdict::IdDict{Any, Any})
    @ Base .\deepcopy.jl:65
 [11] deepcopy_internal(x::Any, stackdict::IdDict{Any, Any})
    @ Base .\deepcopy.jl:76
 [12] deepcopy(x::Function)
    @ Base .\deepcopy.jl:26
 [13] initrepl(parser::typeof(parse_to_expr); prompt_text::String, prompt_color::Symbol, start_key::Char, repl::REPL.LineEditREPL, mode_name::String, show_function::Nothing, show_function_io::Base.TTY, valid_input_checker::Function, keymap::Dict{Char, Any}, completion_provider::REPL.REPLCompletionProvider, sticky_mode::Bool, startup_text::Bool)
    @ ReplMaker C:\Users\ilyao\.julia\packages\ReplMaker\ViCwA\src\ReplMaker.jl:97
 [14] top-level scope
    @ REPL[3]:1

Implement repl menu for selecting modes

I think that having fun the package default to binding a repl mode to a key is probably a bad idea.

One idea that may be preferable is if the package hid all new repl modes behind a menu so one has to press one key to bring up a menu of modes and then another key to select which mode they want.

Another possibility is requiring that the key be accompanied by a modifier key. Ie. one must press alt+} to open the repl mode.

I’m definitely open to suggestions and PRs here. I don’t know how to implement that first idea but the second one is doable.

Remove/disable REPL mode after initrepl()?

Is there a way to remove or disable a REPL mode once installed with initrepl()?

For example, I'm trying to import a package which initiates its own REPL mode via ReplMaker.jl that I don't want to expose through my package.

How does the suggestion for including in startup.jl work ?

the suggested method is to add

atreplinit() do repl
    try
        @eval using ReplMaker
        @async initrepl(
            apropos;
            prompt_text="search> ",
            prompt_color=:magenta,
            start_key=')',
            mode_name="search_mode"
        )
        catch
        end
    end

now if we look at line 393 in client.jl we see

373  function run_main_repl(interacti
 ...
 ...
393 │   │   │   _atreplinit(active_repl)                                                       
394 │   │   │   REPL.run_repl(active_repl, backend->(global active_repl_backend = backend))    

_atreplinit gets called before REPL.run_repl ,
now there are lines inside initrepl like .. julia_mode = active_repl.interface.modes[1] will only work after
REPL.run_repl has run (right?) .

My question is .. how is @async making this happen ?
is this a "hack" such that run_repl is able to setup_interface and all other initializations that initrepl needs before @async initrepl is accessing them , what if the async reaches something(for eg line 68 julia_mode = ...) that has not been initialized by REPL.run yet ?

Rename package to REPLMaker?

Feel free to close this issue, and I totally get that people opening such issues is usually very annoying. In any case, thought I'd mention the following packages

REPLHistory     REPLTreeViews    REPLCompletions  REPLTetris       

and the REPL stdlib, which makes me wonder if it would be a good idea to alias the package name to REPLMaker

Allow custom `show`

It'd be good if the user could supply a custom function other than show, or thier own IO / MIME type to be used when show is called in a custom REPL mode.

How to disable auto completion?

The initrepl function accepts a completion_provider keyword which, by default, provides the Julia REPL auto completion feature.

Which value should I pass to completion_provider to disable auto completion for a custom repl?

Can't get REPL modes on startup to work?

I haven't tried on older julias, but at least 1.9 I can't get it to work. I had someone else test it on a different OS and confirmed.

Is this related to a Base issue?

Thanks

How to start repl from repl?

Lets say I want everything in the REPL to modify a variable x.

I would like to do

mutable struct X
    a
end
x = X(0)
@mod x # enter repl maker
mod> add!(2) # x.a = x.a +1,  

How would I accomplish this in ReplMaker? Reading through the source code, it's not obvious.

[ Feature Request ]: CTRL|ALT|SHIFT+KEY for REPL

I would like to suggest being able to invoke a REPL, with a shortcut combo, instead of just one key, give Julia expressiveness it seems only ) and } are available > is legal:

julia> )
ERROR: syntax: unexpected ")"
Stacktrace:
 [1] top-level scope
   @ none:1

julia> }
ERROR: syntax: unexpected "}"
Stacktrace:
 [1] top-level scope
   @ none:1

julia> >
> (generic function with 3 methods)

The current setup only allows for a couple of "non-legal" julia AKA, available for one key REPL command. I have become addicted to creating REPLs but there is no way to have multiple open, without taking away legal characters from Julia.

Best regards.

no method matching keymap(::Vector{Dict}}

(Reduce.jl traced this issue to ReplMaker.jl)

julia> using Reduce
[ Info: Precompiling Reduce [93e0c654-6965-5f22-aba9-9c1ae6b3c259]
ERROR: InitError: MethodError: no method matching keymap(::Vector{Dict})
Closest candidates are:
keymap(::Any, ::Union{REPL.LineEdit.HistoryPrompt, REPL.LineEdit.PrefixHistoryPrompt}) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\REPL\src\LineEdit.jl:2007
keymap(::Union{Vector{Dict{Any, Any}}, Vector{Dict{Char, Any}}}) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\REPL\src\LineEdit.jl:1610
keymap(::REPL.LineEdit.PromptState, ::REPL.LineEdit.Prompt) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\REPL\src\LineEdit.jl:2505
...
Stacktrace:
[1] initrepl(parser::typeof(Reduce.parserepl); prompt_text::String, prompt_color::Symbol, start_key::Char, repl::REPL.LineEditREPL, mode_name::String, valid_input_checker::Function, keymap::Dict{Char, Any}, completion_provider::REPL.REPLCompletionProvider, sticky_mode::Bool, startup_text::Bool)
@ ReplMaker ~.julia\packages\ReplMaker\pwo5w\src\ReplMaker.jl:78
[2] repl_init(repl::REPL.LineEditREPL)
@ Reduce ~.julia\packages\Reduce\1n3pM\src\repl.jl:45
[3] repl_init
@ ~.julia\packages\Reduce\1n3pM\src\repl.jl:60 [inlined]
[4] init()
@ Reduce ~.julia\packages\Reduce\1n3pM\src\Reduce.jl:322
[5] _include_from_serialized(path::String, depmods::Vector{Any})
@ Base .\loading.jl:597
[6] _require_from_serialized(path::String, cache::Base.TOMLCache)
@ Base .\loading.jl:649
[7] _require(pkg::Base.PkgId, cache::Base.TOMLCache)
@ Base .\loading.jl:952
[8] require(uuidkey::Base.PkgId, cache::Base.TOMLCache)
@ Base .\loading.jl:836
[9] require(into::Module, mod::Symbol)
@ Base .\loading.jl:824
during initialization of module Reduce
julia> versioninfo()
Julia Version 1.6.0-DEV.913
Commit 4c805d2310 (2020-09-14 14:07 UTC)
Platform Info:
OS: Windows (x86_64-w64-mingw32)
CPU: Intel(R) Xeon(R) E-2176M CPU @ 2.70GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-9.0.1 (ORCJIT, skylake)
Environment:
JULIA_NUM_THREADS = 8

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.