mauro3/unpack.jl

`@pack!` and `@unpack` macros

UnPack

It is often convenient to unpack some or all of the fields of a type, and pack, in the case of mutable datatypes (for immutables use Setfield.jl). This is often the case when a struct is passed into a function.

The @unpack and @pack! macros work to unpack types, modules, and dictionaries (and can be customized for other types too, see next section).

using UnPack

mutable struct Para
    a::Float64
    b::Int
end

function f!(var, pa::Para)
    @unpack a, b = pa # equivalent to: a,b = pa.a,pa.b
    out = var + a + b
    b = 77
    @pack! pa = b # equivalent to: pa.b = b
    return out, pa
end

out, pa = f!(7, Para(1,2)) # -> 10.0, Para(1.0, 77)

Example with a dictionary:

d = Dict{Symbol,Any}(:a=>5.0, :b=>2, :c=>"Hi!")
@unpack a, c = d
a == 5.0 #true
c == "Hi!" #true

d = Dict{String,Any}()
@pack! d = a, c
d # -> Dict{String,Any}("a"=>5.0,"c"=>"Hi!")

Customization of @unpack and @pack!

What happens during the (un-)packing of a particular datatype is determined by the functions UnPack.unpack and UnPack.pack!.

The UnPack.unpack function is invoked to unpack one entity of some DataType and has signature:

unpack(dt::Any, ::Val{property}) -> value of property

Note that unpack (and pack!) works with Base.getproperty. By default this means that all the fields of a type are unpacked but if getproperty is overloaded, then it will unpack accordingly.

Three method definitions are included in the package to unpack a composite type/module/NamedTuple, or a dictionary with Symbol or string keys:

@inline unpack(x, ::Val{f}) where {f} = getproperty(x, f)
@inline unpack(x::AbstractDict{Symbol}, ::Val{k}) where {k} = x[k]
@inline unpack(x::AbstractDict{<:AbstractString}, ::Val{k}) where {k} = x[string(k)]

The UnPack.pack! function is invoked to pack one entity into some DataType and has signature:

pack!(dt::Any, ::Val{field}, value) -> value

Three definitions are included in the package to pack into a mutable composite type or into a dictionary with Symbol or string keys:

@inline pack!(x, ::Val{f}, val) where {f} = setproperty!(x, f, val)
@inline pack!(x::AbstractDict{Symbol}, ::Val{k}, val) where {k} = x[k]=val
@inline pack!(x::AbstractDict{<:AbstractString}, ::Val{k}, val) where {k} = x[string(k)]=val

More methods can be added to unpack and pack! to allow for specialized unpacking/packing of datatypes. Here is a MWE of customizing unpack, so that it multiplies the values by 2:

using UnPack
struct Foo
    a
    b
end
p = Foo(1, 2)
@unpack a, b = p
a, b # gives (1, 2)

# Now we specialize unpack for our custom type, `Foo`
@inline UnPack.unpack(x::Foo, ::Val{f}) where {f} = 2 * getproperty(x, f)
@unpack a, b = p
a, b # now gives (2, 4)

Related

• Julia issue on unpacking of function arguments JuliaLang/julia#28579
• setting immutables https://github.com/jw3126/Setfield.jl

unpack.jl's Issues

Feature request: type annotation on variables in `@unpack`

E.g.

julia> d1 = Dict(:a=>3, :b=>2.0)
Dict{Symbol, Real} with 2 entries:
  :a => 3
  :b => 2.0

julia> function f(d1)
           @unpack a = d1
           a
       end
f (generic function with 1 method)

julia> @code_typed f(d1)
CodeInfo(
1 ─ %1 = invoke Base.getindex(d1::Dict{Symbol, Real}, :a::Symbol)::Real
└──      return %1
) => Real

In this case, if I know that a must be an Int, I might want to specify this as a::Int in the @unpack, but the following doesn't work:

julia> function f(d1)
           @unpack a::Int = d1
           a
       end
f (generic function with 1 method)

julia> f(d1)
ERROR: KeyError: key :Int not found
Stacktrace:
 [1] getindex(h::Dict{Symbol, Real}, key::Symbol)
   @ Base ./dict.jl:484
 [2] unpack
   @ ~/.julia/packages/UnPack/EkESO/src/UnPack.jl:35 [inlined]
 [3] macro expansion
   @ ~/.julia/packages/UnPack/EkESO/src/UnPack.jl:101 [inlined]
 [4] f(d1::Dict{Symbol, Real})
   @ Main ./REPL[290]:2
 [5] top-level scope
   @ REPL[291]:1

links for @unpack and @pack! in the readme are broken

@unpack everthing?

UnPack.jl/src/UnPack.jl

Lines 92 to 93 in 91296e0

	macro unpack(args)
	args.head!=:(=) && error("Expression needs to be of form `a, b = c`")

I was wondering whether it would make sense to extend this macro such that @unpack A unpacks all fieldnames in A? At the moment line 93 triggers that as an error

struct ABC
    a
    b
    c
end

then

julia> A = ABC(1,2,3)
ABC(1, 2, 3)

julia> @unpack A
ERROR: LoadError: type Symbol has no field head
Stacktrace:
 [1] getproperty(x::Symbol, f::Symbol)
   @ Base ./Base.jl:42
 [2] var"@unpack"(__source__::LineNumberNode, __module__::Module, args::Any)
   @ UnPack ~/.julia/packages/UnPack/EkESO/src/UnPack.jl:93

I don't know whether such a method would have undesired consequences, but I'm thinking about an application where one deals with structs that contain many lengthy names (because naming is hard and to avoid naming conflicts when unpacking), such that changing everytime all the @unpack lenghty_name_1, lengthy_name_2, lengthy_name3 = A in all the functions where A is used is annoying and typo-prone.

@white-alistair

unpack into renamed local variables

Sometimes one wants to unpack values of two similar (or the same) types in a local scope and then variable names clash. A typical example is implementing an operator or some types, eg

function Base.==(x::MyAB, y::MyAB)
    x.a == y.a && x.b == y.b
end

Cf common-lisp:with-slots. I wonder if there is some syntax we could co-opt for this purpose.

[FR] unpack into a let

Nothing crucial, but thought I'd mention, I find myself often wanting to do something like:

@unpack let (x,y) = foo
   ...
end

Seems possible with no fundamental changes, just some extra expression manipulation.

UnPack has no known versions...?

I've just deved DrWatson in a fresh Julia 1.5 installation, and I get:

  Resolving package versions...
ERROR: Unsatisfiable requirements detected for package UnPack [3a884ed6]:
 UnPack [3a884ed6] log:
 ├─UnPack [3a884ed6] has no known versions!
 └─restricted to versions 1.0.1-1 by DrWatson [634d3b9d] — no versions left
   └─DrWatson [634d3b9d] log:
     ├─possible versions are: 1.14.7 or uninstalled
     └─DrWatson [634d3b9d] is fixed to version 1.14.7

Can you install UnPack.jl in Julia 1.5.? That's a weird message... :D

Request - unpack with default values

Would it be possible to add a feature to allow unpacking with optional default values?

struct Foo
    a
    b
    c
end

p = Foo(10, 20,30)

@unpack { a=1, b,  x=1 }  =  p

a, b, x # gives (10, 20,  1)

@unpack { a=1, b, x }  =  p       #gives Error - x not present in p and no default value set.

Tag 1.0.1?

Hi, could you please tag the PR with the doc updates? Thanks!

Return nothing if field doesn't exist

I wonder if it's practical to add one, or extend the existing macro that, if the field doesn't exist in the dict I'm tring to unpack, then just return a nothing, instead of raise an error like:

julia> @unpack ll = a.kwargs
ERROR: type NamedTuple has no field ll
Stacktrace:
 [1] getindex
   @ ./namedtuple.jl:118 [inlined]
 [2] getindex(v::Base.Iterators.Pairs{Symbol, Int64, Tuple{Symbol}, NamedTuple{(:ooo,), Tuple{Int64}}}, key::Symbol)
   @ Base.Iterators ./iterators.jl:270
 [3] unpack(x::Base.Iterators.Pairs{Symbol, Int64, Tuple{Symbol}, NamedTuple{(:ooo,), Tuple{Int64}}}, #unused#::Val{:ll})
   @ UnPack ~/.julia/packages/UnPack/EkESO/src/UnPack.jl:35
 [4] top-level scope
   @ ~/.julia/packages/UnPack/EkESO/src/UnPack.jl:101

Because when writing my code, I found that will be helpful, Thanks!

Match new Base destructuring syntax (aka the fate of this package)

Property destructuring was just merged into master 🎉 JuliaLang/julia#39285

Would be nice if this package allowed you to match that syntax, i.e. if in addition to the current form, it also allowed:

@unpack (;x, y) = foo

such that in the future one can upgrade by just dropping the @unpack.

Register the package?

Unless there is a reason to wait?

Unpack for JLD2File

I would be nice to be able to use @pack! and @unpack to quickly store and load values from JLD2File instances. This can be achieved by adding the methods

@inline UnPack.unpack(x::JLDFile{MmapIO}, ::Val{f}) where {f} =  x[string(f)]

@inline function UnPack.pack!(file::JLDFile{MmapIO}, ::Val{f}, val) where {f}
    file[string(f)] = val
end

These methods can be used as

using JLD2
using Unpack

x = rand(10)
path = "temp.jld2"
file = jldopen(path, "w")
@pack! file = x # Creates entry with name x
close(file)

file = jldopen("temp.jld2") # contains fields :x
@unpack x = file # Extracts x from the struct

Maybe this could be added (with JLD2 as on optional dependency; not sure how that works though)? Happy to try and make a PR if you are open to doing this.

feature request: version of unpack using getfield

Using @unpack inside a Base.getproperty method leads to a stack overflow. A convenience form, possibly called @unpackf, could use getfield to get around this.

Happy to make a PR if you are open to doing this.