c42f/underscores.jl

Underscore placeholders for convenient closure syntax

Underscores

Underscores provides a macro @_ for passing closures to functions by interpreting _ placeholders as anonymous function arguments. For example @_ map(_+1, xs) means map(x->x+1, xs).

Underscores is useful for writing anonymous functions succinctly and without naming the arguments. This is particular useful for data processing pipelines such as

@_ people |> filter(_.age > 40, __) |> map(_.name, __)

Read the documentation for more information, or see the online help for the @_ macro.

underscores.jl's Issues

Assignment gets read as a keyword

Something which ought to work, I think:

mutable struct Bar x end; bars = [Bar(i) for i in 1:3];

@_ map(_.x, bars) # fine

@_ foreach(_.x = 2, bars) # ERROR: syntax: invalid keyword argument name "(_1,) -> begin

Originally from broadcasting setproperty!, here: JuliaLang/julia#36608

Consider capturing bindings by value (FastClosures.jl)

As was briefly mentioned on on Zulip https://julialang.zulipchat.com/#narrow/stream/235161-compiler-frontend/topic/capture-by-value.20closures.3F.20(lowering.20in.20Julia) it might be useful if all _-based closures captured a separate binding.

Alas this would departs in significant but very subtle ways from the lexical scoping of the language which is not great. But on the upside it could be considered because

• _ syntax is for very short expressions and one wouldn't normally want to mutate the bindings captured within
• _ syntax is for calling higher order functions, and in typical uses it would be unusual for the lifetime of the closure to outlive the function it's passed to.
• There's a (small?) chance that the language as a whole will eventually go in this direction.

CC @tkf

Behavior of `_` is confusing with infix operators

Not sure whether it's a bug or wrong usage of Underscores.jl, but following example fails

v = [(x = rand(), y = rand()) for _ in 1:10]
@_ v |> sum(_.x, __) / length(__)

ERROR: MethodError: no method matching /(::var"#56#58"{Array{NamedTuple{(:x, :y),Tuple{Float64,Float64}},1}}, ::Int64)
Closest candidates are:
  /(::Missing, ::Number) at missing.jl:115
  /(::BigInt, ::Union{Int16, Int32, Int64, Int8, UInt16, UInt32, UInt64, UInt8}) at gmp.jl:537
  /(::BigFloat, ::Union{Int16, Int32, Int64, Int8}) at mpfr.jl:441

Broadcasted piping .|>

Seems like this ought to work:

julia> @_ 1:4 .|> float |> filter(iseven(Int(_)), __) .|> sqrt              
#27 (generic function with 1 method)

julia> 1:4 .|> float |> xs -> filter(x -> iseven(Int(x)), xs) .|> sqrt 
2-element Array{Float64,1}:
 1.4142135623730951
 2.0

See also https://discourse.julialang.org/t/in-pipe-pipe-followed-by-gives-an-error/47174 for other examples.

Feature Request: if no `__` is specified then insert into first argument

Currently

@_ a |> f(_, b) is needed but in other tools like Lazy.jl and R's magrittr which allows the first argument to be the previous one without _ e.g.

@_ a |> f(b) would be equivalent to f(a,b) and @_ a |> f(_, b)

Doesn't with ternary operators

The below doesn't work

@_ _ == "X" ? 0 : 1 #doesn't work

but this works

x-> x == "X" ? 0 : 1

Underscore replacement should recurse into quasiquote

Expr(:quote) is actually a quasiquote, so the following should be equivalent:

julia> map(i->:(a[$i]), 1:4)
4-element Array{Expr,1}:
 :(a[1])
 :(a[2])
 :(a[3])
 :(a[4])

julia> @_ map(:(a[$_]), 1:4)
ERROR: syntax: all-underscore identifier used as rvalue
Stacktrace:
 [1] top-level scope at REPL[2]:1

Release a new version to make new bugfixes avaiable

Hello! I really enjoy using your package in my personal project, but one thing keeps bothering me i s that pipe operators are not handled, so I have to avoid it explicitly. This has already been fixed in #21 but is not included in a release.
It has been one year since the last release (2.0.0), can you release a new version to make the new bugfixes available?

Audit _ expansion rules to only expand into "things which look like function calls"

We've had a couple of specific cases where _ expansion is confusing - opened in #6 and #8 and solved by #10.

But because #10 is breaking, I'd like to make sure we have a single consistent-as-possible syntax rule before the next release. Something like:

_ expands to a closure which is passed to the outermost "thing which looks like a function call"

See #6 and #8 for extensive discussion.

A way to allow for `filter`, `map` piping...

I like it! (Better than I like my own package LambdaFn). Thanks for putting this together!

It occurred to me that, since this is a macro, you should be able to rewrite piping expressions to treat functions as if they have a curried version.

That is:

@_  data  |>  filter(_>10) |>  map(_+1)

Could be re-written as:

data |> x -> filter(a -> a > 10, x) |> x -> map(a -> a +1,x)

The rule could be that if the top-level expression after |> is a function call, rewrite the expression to be a function that takes a single argument and passes it as the final argument of this top-level expression.

Maybe that should be a separate macro (@_> @|>, @|>_ ??). It is essentially a combination of your macro @_ and the @>> macro from Lazy.jl.

Switch to more local `@_`

So I realized here: JuliaLang/julia#38713 (comment) that we actually can have map(@_ 1 + _[1], v) or whatever do the right thing. Maybe this should be the canonical usage of this macro instead of being outside the map.

TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Interaction with indexing brackets

This is very handy:

g = @_ gradient(fun(_,y,z), x)   # gradient(x -> fun(x,y,z), x)

but this was a bit of a surprise:

g ≈ @_ gradient(fun(_,y,z), x)[1]   # (x_ -> gradient(fun(x_, y, z), x))[1]

Is it obvious that this should be treated as the outermost call? It's visually last not first, it's very unlikely that you want to call getindex with a function, and the expression :((gradient(fun(_, y, z), x))[1]) is not equal to :(getindex(gradient(fun(_, y, z), x), 1)).

One slightly awkward work-around is:

g ≈ (@_ gradient(fun(_,y,z), x) |> __[1])

@_ within the selector Fun part of DataFrames.select

It seems currently @_ not working in the following:

using DataFrames
df = DataFrame(a=1:3, b=4:6, c=7:9);

df1 = select(df, Not(:b) => ByRow(@_(_1 + _2)))

MethodError: no method matching +(::var"#3#5", ::var"#4#6")
Closest candidates are:
  +(::Any, ::Any, !Matched::Any, !Matched::Any...) at operators.jl:529
Stacktrace:
 [1] top-level scope at In[18]:1

df2 = select(df, :a => @_(_ .^2))

MethodError: no method matching ^(::var"#20#21", ::Int64)
Closest candidates are:
  ^(!Matched::Float16, ::Integer) at math.jl:885
  ^(!Matched::Regex, ::Integer) at regex.jl:712
  ^(!Matched::Missing, ::Integer) at missing.jl:155
  ...
Stacktrace:
 [1] macro expansion at ./none:0 [inlined]
 [2] literal_pow at ./none:0 [inlined]
 [3] _broadcast_getindex_evalf at ./broadcast.jl:631 [inlined]
 [4] _broadcast_getindex at ./broadcast.jl:604 [inlined]
 [5] getindex at ./broadcast.jl:564 [inlined]
 [6] copy at ./broadcast.jl:830 [inlined]
 [7] materialize(::Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{0},Nothing,typeof(Base.literal_pow),Tuple{Base.RefValue{typeof(^)},Base.RefValue{var"#20#21"},Base.RefValue{Val{2}}}}) at ./broadcast.jl:820
 [8] top-level scope at In[23]:1

Redesign _ -> identity transforms

Suggestion: Whenever some underscore tends to be replaced by an identity function, make the anonymous function escape a single layer.
That will automatically enable cases like that:
1:10 |> filter(_<5, _)
while we don't lose a relevant feature. Using _ instead of identity shouldn't be considered good style anyway.