fluxml / metalhead.jl Goto Github PK

View Code? Open in Web Editor NEW

328.0 18.0 66.0 11.01 MB

Computer vision models for Flux

Home Page: https://fluxml.ai/Metalhead.jl

License: Other

Julia 100.00%

machine-learning computer-vision deep-learning flux julia

metalhead.jl's Introduction

Metalhead

Metalhead.jl provides standard machine learning vision models for use with Flux.jl. The architectures in this package make use of pure Flux layers, and they represent the best-practices for creating modules like residual blocks, inception blocks, etc. in Flux. Metalhead also provides some building blocks for more complex models in the Layers module.

Installation

julia> ]add Metalhead

Getting Started

You can find the Metalhead.jl getting started guide here.

Available models

To contribute new models, see our contributing docs.

Image Classification

Model Name	Constructor	Pre-trained?
AlexNet	`AlexNet`	N
ConvMixer	`ConvMixer`	N
ConvNeXt	`ConvNeXt`	N
DenseNet	`DenseNet`	N
EfficientNet	`EfficientNet`	N
EfficientNetv2	`EfficientNetv2`	N
gMLP	`gMLP`	N
GoogLeNet	`GoogLeNet`	N
Inception-v3	`Inceptionv3`	N
Inception-v4	`Inceptionv4`	N
InceptionResNet-v2	`InceptionResNetv2`	N
MLPMixer	`MLPMixer`	N
MobileNetv1	`MobileNetv1`	N
MobileNetv2	`MobileNetv2`	N
MobileNetv3	`MobileNetv3`	N
MNASNet	`MNASNet`	N
ResMLP	`ResMLP`	N
ResNet	`ResNet`	Y
ResNeXt	`ResNeXt`	Y
SqueezeNet	`SqueezeNet`	Y
Xception	`Xception`	N
WideResNet	`WideResNet`	Y
VGG	`VGG`	Y
Vision Transformer	`ViT`	Y

Other Models

Model Name	Constructor	Pre-trained?
UNet	`UNet`	N

metalhead.jl's People

Contributors

Stargazers

Watchers

Forkers

ayush1999 avik-pal shubhamvajpayee staticfloat wookay zgornel dhairyalgandhi mbrukman ditva605 kristofferc shreyas-kowshik bemoregt yuehhua dasudian xzbh5 stjordanis ayushk4 vballoli passysosysmas smsinks eslgastal chrisrackauckas juliatagbot ven-k gartangh djpasseyjr arthasyou paethon moelf darsnack zeta1999 mlh-fellowship jeremiedb rishabhvarshney14 hsugawa8651 mloubout varlad standardgalactic lvyitian somtambe naturalog-space playfloor jonalm c8pan iblislin alexander-barth theabhirath rutvij-25 adrhill ceferisbarov pri1311 saransh-cpp ashwani-rathee mcognetta shivance christiangnrd eternal-night-archer natema nickkeepfer mohamed82008 rafaelt00 farhad-phd ianbutterworth dmolina vinayakjeet linecode

metalhead.jl's Issues

Finetune pretrained models

I'm trying to write a computer vision, graph convolutional neural network project to help others quickly use julia's deep learning template code.

I found that using the model provided by Metalhead, finetune can not achieve good results on the specified data set. Have you frozen the relevant parameters?

This is the code using a custom model, the loss can be reduced normally.

But when I replace it with the model in metalhead and want finetune, the loss is always above 2.30. I only replaced the model part with:

vgg = VGG19()
model = Chain(vgg.layers[1:end-6],
              Dense(512, 4096, relu),
              Dropout(0.5),
              Dense(4096, 4096, relu),
              Dropout(0.5),
              Dense(4096, 10)) |> device
Flux.trainmode!(model, true)

trainimgs not defined

Not sure if this is the right place to report. When I use

using Metalhead: trainimgs, valimgs

I got

julia> using Metalhead: trainimgs, valimgs
ERROR: UndefVarError: trainimgs not defined
Stacktrace:
 [1] top-level scope
   @ ~/.julia/packages/CUDA/YpW0k/src/initialization.jl:52

This occurs in both Julia 1.6.2 and 1.7.

Add new Models and provide user with an option to fetch untrained models

List of models that I propose to add are:-

The above ones just need a few tweeks before being ported from model-zoo. Also models like:

should be added, but they are not ready in model zoo currently.

The proposed API is something like VGG19 would fetch an untrained model to the user. To get a trained model we use something like trained(VGG19). We will throw an error for trying to fetch models for which we donot have the trained weights (atleast all the above models for now).
cc @MikeInnes

Add support for miniImagenet

In meta-learning, miniimagenet is a common benchmark derived from the imagenet dataset. A library that does this (arguably the standard library for this) is https://github.com/yaoyao-liu/mini-imagenet-tools.

It would be beneficial to have support for this dataset in Metalhead.jl.

ResNet on MNIST

How would one apply ResNet on Kaggle's MNIST dataset? I shaved off the final dense layer and the softmax layer ( ResNet()[:20] ). I get a dataset of (42000, 2048). From there on train the following net.

Dense(2048, 1024, relu),
BatchNorm(1024),
Dense(1024, 512, relu),
BatchNorm(512),
Dense(512, 256, relu),
BatchNorm(256),
Dense(256, 10),
softmax

This does not surpass %88 training accuracy even after training for 50 epochs whereas a regular convolutional network I train from scratch for a few epoch easily gets %99.5 train accuracy. More generally, how would you apply ResNet(or any other) net to a custom dataset? Can I get a minimal example.

Thanks in advance.

[GSoC 2021 Proposal] Segmentation models for Metalhead

Hi!

I intend to contribute to this repo as GSoC 2021 applicant. This repo seem to miss segmentation models as well as object detection and recognition models.

Are those models needed? If yes, please let me know so that I can contribute.

Cheers!
Rakesh

Won't load into Julia 1.3.1


julia> Pkg.add("Metalhead")
 Resolving package versions...
ERROR: Unsatisfiable requirements detected for package Metalhead [dbeba491]:
 Metalhead [dbeba491] log:
 ├─possible versions are: [0.3.0, 0.4.0, 0.5.0] or uninstalled
 ├─restricted to versions * by an explicit requirement, leaving only versions [0.3.0, 0.4.0, 0.5.0]
 ├─restricted by compatibility requirements with ColorTypes [3da002f7] to versions: [0.3.0, 0.5.0] or uninstalled, leaving only versions: [0.3.0, 0.5.0]
 │ └─ColorTypes [3da002f7] log:
 │   ├─possible versions are: [0.7.0-0.7.5, 0.8.0-0.8.1, 0.9.0-0.9.1] or uninstalled
 │   └─restricted to versions 0.9.1 by an explicit requirement, leaving only versions 0.9.1
 ├─restricted by compatibility requirements with Images [916415d5] to versions: [0.4.0, 0.5.0] or uninstalled, leaving only versions: 0.5.0
 │ └─Images [916415d5] log:
 │   ├─possible versions are: [0.17.3, 0.18.0, 0.19.0-0.19.3, 0.20.0-0.20.2, 0.21.0] or uninstalled
 │   └─restricted to versions 0.20.2 by an explicit requirement, leaving only versions 0.20.2
 └─restricted by compatibility requirements with Flux [587475ba] to versions: [0.3.0, 0.4.0] or uninstalled — no versions left
   └─Flux [587475ba] log:
     ├─possible versions are: [0.4.1, 0.5.0-0.5.4, 0.6.0-0.6.10, 0.7.0-0.7.3, 0.8.0-0.8.3, 0.9.0, 0.10.0-0.10.1] or uninstalled
     └─restricted to versions 0.8.3 by an explicit requirement, leaving only versions 0.8.3
Stacktrace:
 [1] #propagate_constraints!#61(::Bool, ::typeof(Pkg.GraphType.propagate_constraints!), ::Pkg.GraphType.Graph, ::Set{Int64}) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/GraphType.jl:1007
 [2] propagate_constraints! at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/GraphType.jl:948 [inlined]
 [3] #simplify_graph!#121(::Bool, ::typeof(Pkg.GraphType.simplify_graph!), ::Pkg.GraphType.Graph, ::Set{Int64}) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/GraphType.jl:1462
 [4] simplify_graph! at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/GraphType.jl:1462 [inlined] (repeats 2 times)
 [5] resolve_versions!(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/Operations.jl:321
 [6] #add#112(::Bool, ::Pkg.BinaryPlatforms.MacOS, ::typeof(Pkg.Operations.add), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}, ::Array{Base.UUID,1}) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/Operations.jl:1010
 [7] #add at ./none:0 [inlined]
 [8] #add#25(::Bool, ::Pkg.BinaryPlatforms.MacOS, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Pkg.API.add), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:102
 [9] add(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:72
 [10] #add#24 at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:69 [inlined]
 [11] add at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:69 [inlined]
 [12] #add#21 at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:67 [inlined]
 [13] add at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:67 [inlined]
 [14] #add#20(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Pkg.API.add), ::String) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:66
 [15] add(::String) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:66
 [16] top-level scope at REPL[81]:1

julia>

Metalhead fails on M1 Mac with Julia 1.7 Beta 3

See here:

julia> using Metalhead
[ Info: Precompiling Metalhead [dbeba491-748d-5e0e-a39e-b530a07fa0cc]
ERROR: LoadError: UndefVarError: libfftw3_path not defined
Stacktrace:
  [1] getproperty(x::Module, f::Symbol)
    @ Base ./Base.jl:35
  [2] top-level scope
    @ ~/.julia/packages/FFTW/kKdEk/src/providers.jl:51
  [3] include(mod::Module, _path::String)
    @ Base ./Base.jl:417
  [4] include(x::String)
    @ FFTW ~/.julia/packages/FFTW/kKdEk/src/FFTW.jl:1
  [5] top-level scope
    @ ~/.julia/packages/FFTW/kKdEk/src/FFTW.jl:17
  [6] include
    @ ./Base.jl:417 [inlined]
  [7] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::String)
    @ Base ./loading.jl:1318
  [8] top-level scope
    @ none:1
  [9] eval
    @ ./boot.jl:373 [inlined]
 [10] eval(x::Expr)
    @ Base.MainInclude ./client.jl:453
 [11] top-level scope
    @ none:1
in expression starting at /Users/logankilpatrick/.julia/packages/FFTW/kKdEk/src/providers.jl:49
in expression starting at /Users/logankilpatrick/.julia/packages/FFTW/kKdEk/src/FFTW.jl:1
ERROR: LoadError: Failed to precompile FFTW [7a1cc6ca-52ef-59f5-83cd-3a7055c09341] to /Users/logankilpatrick/.julia/compiled/v1.7/FFTW/jl_oKjNHY.
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:33
  [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, ignore_loaded_modules::Bool)
    @ Base ./loading.jl:1466
  [3] compilecache(pkg::Base.PkgId, path::String)
    @ Base ./loading.jl:1410
  [4] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:1120
  [5] require(uuidkey::Base.PkgId)
    @ Base ./loading.jl:1013
  [6] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:997
  [7] include
    @ ./Base.jl:417 [inlined]
  [8] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::String)
    @ Base ./loading.jl:1318
  [9] top-level scope
    @ none:1
 [10] eval
    @ ./boot.jl:373 [inlined]
 [11] eval(x::Expr)
    @ Base.MainInclude ./client.jl:453
 [12] top-level scope
    @ none:1
in expression starting at /Users/logankilpatrick/.julia/packages/ImageFiltering/bTMw9/src/ImageFiltering.jl:1
ERROR: LoadError: Failed to precompile ImageFiltering [6a3955dd-da59-5b1f-98d4-e7296123deb5] to /Users/logankilpatrick/.julia/compiled/v1.7/ImageFiltering/jl_O2Z5FU.
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:33
  [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, ignore_loaded_modules::Bool)
    @ Base ./loading.jl:1466
  [3] compilecache(pkg::Base.PkgId, path::String)
    @ Base ./loading.jl:1410
  [4] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:1120
  [5] require(uuidkey::Base.PkgId)
    @ Base ./loading.jl:1013
  [6] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:997
  [7] include
    @ ./Base.jl:417 [inlined]
  [8] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::String)
    @ Base ./loading.jl:1318
  [9] top-level scope
    @ none:1
 [10] eval
    @ ./boot.jl:373 [inlined]
 [11] eval(x::Expr)
    @ Base.MainInclude ./client.jl:453
 [12] top-level scope
    @ none:1
in expression starting at /Users/logankilpatrick/.julia/packages/Images/ILl34/src/Images.jl:3
ERROR: LoadError: Failed to precompile Images [916415d5-f1e6-5110-898d-aaa5f9f070e0] to /Users/logankilpatrick/.julia/compiled/v1.7/Images/jl_r9PS6j.
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:33
  [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, ignore_loaded_modules::Bool)
    @ Base ./loading.jl:1466
  [3] compilecache(pkg::Base.PkgId, path::String)
    @ Base ./loading.jl:1410
  [4] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:1120
  [5] require(uuidkey::Base.PkgId)
    @ Base ./loading.jl:1013
  [6] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:997
  [7] include
    @ ./Base.jl:417 [inlined]
  [8] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
    @ Base ./loading.jl:1318
  [9] top-level scope
    @ none:1
 [10] eval
    @ ./boot.jl:373 [inlined]
 [11] eval(x::Expr)
    @ Base.MainInclude ./client.jl:453
 [12] top-level scope
    @ none:1
in expression starting at /Users/logankilpatrick/.julia/packages/Metalhead/PBZE5/src/Metalhead.jl:2
ERROR: Failed to precompile Metalhead [dbeba491-748d-5e0e-a39e-b530a07fa0cc] to /Users/logankilpatrick/.julia/compiled/v1.7/Metalhead/jl_kBPZwT.
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:33
 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, ignore_loaded_modules::Bool)
   @ Base ./loading.jl:1466
 [3] compilecache(pkg::Base.PkgId, path::String)
   @ Base ./loading.jl:1410
 [4] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1120
 [5] require(uuidkey::Base.PkgId)
   @ Base ./loading.jl:1013
 [6] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:997

I am guessing this will be fixed when the 1.7 PR's are merged for Flux?

Add model implementations

Below are models that we still need to implement

Will implement

These are models for which we are actively seeking implementations/weights.

Model implementations

Pre-trained weights

Will consider implementing

These are models that aren't necessarily must haves, but we are happy to accept contributions that add them.

ESRGAN (see #118).

`flatten` layers appear as closures

When using Metalhead models, layers using flatten appear as closures, e.g. Metalhead.var"#44#45"() in the following example.

This makes it impossible to dispatch on these layers or infer what they are without looking into the Metalhead source.

Example

julia> model = VGG19();

julia> model.layers
Chain(
  Conv((3, 3), 3 => 64, relu, pad=1),   # 1_792 parameters
  Conv((3, 3), 64 => 64, relu, pad=1),  # 36_928 parameters
  MaxPool((2, 2)),
  Conv((3, 3), 64 => 128, relu, pad=1),  # 73_856 parameters
  Conv((3, 3), 128 => 128, relu, pad=1),  # 147_584 parameters
  MaxPool((2, 2)),
  Conv((3, 3), 128 => 256, relu, pad=1),  # 295_168 parameters
  Conv((3, 3), 256 => 256, relu, pad=1),  # 590_080 parameters
  Conv((3, 3), 256 => 256, relu, pad=1),  # 590_080 parameters
  Conv((3, 3), 256 => 256, relu, pad=1),  # 590_080 parameters
  MaxPool((2, 2)),
  Conv((3, 3), 256 => 512, relu, pad=1),  # 1_180_160 parameters
  Conv((3, 3), 512 => 512, relu, pad=1),  # 2_359_808 parameters
  Conv((3, 3), 512 => 512, relu, pad=1),  # 2_359_808 parameters
  Conv((3, 3), 512 => 512, relu, pad=1),  # 2_359_808 parameters
  MaxPool((2, 2)),
  Conv((3, 3), 512 => 512, relu, pad=1),  # 2_359_808 parameters
  Conv((3, 3), 512 => 512, relu, pad=1),  # 2_359_808 parameters
  Conv((3, 3), 512 => 512, relu, pad=1),  # 2_359_808 parameters
  Conv((3, 3), 512 => 512, relu, pad=1),  # 2_359_808 parameters
  MaxPool((2, 2)),
  Metalhead.var"#44#45"(),
  Dense(25088, 4096, relu),             # 102_764_544 parameters
  Dropout(0.5),
  Dense(4096, 4096, relu),              # 16_781_312 parameters
  Dropout(0.5),
  Dense(4096, 1000),                    # 4_097_000 parameters
  NNlib.softmax,
)                   # Total: 38 arrays, 143_667_240 parameters, 548.053 MiB.

(jl_qLymP0) pkg> status Metalhead
      Status `/private/var/folders/74/wcz8c9qs5dzc8wgkk7839k5c0000gn/T/jl_qLymP0/Project.toml`
  [dbeba491] Metalhead v0.5.3

Precompiling fails on julia 1.0

I'm sure this is known but since I like the package I would love to be able to use it on 1.0. 👍

Version:

Julia Version 1.0.0
Commit 5d4eaca0c9 (2018-08-08 20:58 UTC)
Platform Info:
OS: Linux (x86_64-pc-linux-gnu)
CPU: Intel(R) Core(TM) i7-6650U CPU @ 2.20GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-6.0.0 (ORCJIT, skylake)

Error:

julia> using Metalhead
[ Info: Precompiling Metalhead [dbeba491-748d-5e0e-a39e-b530a07fa0cc]
ERROR: LoadError: LoadError: UndefVarError: Void not defined
Stacktrace:
 [1] top-level scope at none:0
 [2] include at ./boot.jl:317 [inlined]
 [3] include_relative(::Module, ::String) at ./loading.jl:1038
 [4] include at ./sysimg.jl:29 [inlined]
 [5] include(::String) at /home/michael/.julia/packages/Metalhead/8ZFyX/src/Metalhead.jl:2
 [6] top-level scope at none:0
 [7] include at ./boot.jl:317 [inlined]
 [8] include_relative(::Module, ::String) at ./loading.jl:1038
 [9] include(::Module, ::String) at ./sysimg.jl:29
 [10] top-level scope at none:2
 [11] eval at ./boot.jl:319 [inlined]
 [12] eval(::Expr) at ./client.jl:389
 [13] top-level scope at ./none:3
in expression starting at /home/michael/.julia/packages/Metalhead/8ZFyX/src/utils.jl:75
in expression starting at /home/michael/.julia/packages/Metalhead/8ZFyX/src/Metalhead.jl:23
ERROR: Failed to precompile Metalhead [dbeba491-748d-5e0e-a39e-b530a07fa0cc] to /home/michael/.julia/compiled/v1.0/Metalhead/OYscp.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] macro expansion at ./logging.jl:313 [inlined]
 [3] compilecache(::Base.PkgId, ::String) at ./loading.jl:1184
 [4] _require(::Base.PkgId) at ./logging.jl:311
 [5] require(::Base.PkgId) at ./loading.jl:852
 [6] macro expansion at ./logging.jl:311 [inlined]
 [7] require(::Module, ::Symbol) at ./loading.jl:834

Feature request: ESRGAN, upconv7, upcunet

Maybe too ambitious but it would be nice to have ESRGAN, upconv7 and upcunet (used by waifu2x)

I met this error when run "feature extraction with MetalHead".

Hi, @wookay @staticfloat @ViralBShah @Keno @MikeInnes

I met this error when run "feature extraction with MetalHead".

julia> using Metalhead
julia> model = ResNet()
julia> model.layers = model.layers[1:5]
julia> img = rand(224, 224, 3, 1)
julia> model(img)

My error message

julia> model.layers = model.layers[1:5]
ERROR: setfield! immutable struct of type ResNet cannot be changed
Stacktrace:
[1] setproperty!(::ResNet, ::Symbol, ::Chain{Tuple{Conv{2,typeof(identity),TrackedArray{…,Array{Float32,4}},TrackedArray{…,Array{Float32,1}}},getfield(Metalhead, Symbol("##113#116")),Metalhead.ResidualBlock,Metalhead.ResidualBlock,Metalhead.ResidualBlock}}) at ./sysimg.jl:19
[2] top-level scope at none:0

What's wrong to me?

Thanks.

while training, get InvalidIRError

julia> train_set = ((gpu(cat(train[batch]/255f0..., dims=4)), gpu(onehotbatch(label[batch], scores))) for batch in partition(randperm(length(train)), batch_size))
Base.Generator{Base.Iterators.PartitionIterator{Array{Int64,1}},getfield(Main, Symbol("##33#34"))}(getfield(Main, Symbol("##33#34"))(), Base.Iterators.PartitionIterator{Array{Int64,1}}([22, 19, 1, 37, 13, 10, 21, 15, 33, 6 … 26, 8, 11, 7, 5, 24, 20, 35, 16, 31], 3))

julia> Flux.train!(loss, params(model), train_set, opt, cb=evalcb)
ERROR: InvalidIRError: compiling mapreducedim_kernel_parallel(typeof(identity), typeof(Base.add_sum), CUDAnative.CuDeviceArray{Float32,2,CUDAnative.AS.Global}, CUDAnative.CuDeviceArray{Float32,2,CUDAnative.AS.Global}, CartesianIndices{2,Tuple{Base.OneTo{Int64},Base.OneTo{Int64}}}, Int64, Int64) resulted in invalid LLVM IR
Reason: unsupported dynamic function invocation (call to throw_inexacterror(f::Symbol, T, val) in Core at boot.jl:560)
Stacktrace:
[1] check_top_bit at boot.jl:574
[2] toInt64 at boot.jl:635
[3] Type at boot.jl:710
[4] convert at number.jl:7
[5] steprange_last at range.jl:225
[6] multiple call sites at unknown:0
Stacktrace:
[1] check_ir(::CUDAnative.CompilerJob, ::LLVM.Module) at /home/dsd/.julia/packages/CUDAnative/RcKr9/src/compiler/validation.jl:114
[2] macro expansion at /home/dsd/.julia/packages/TimerOutputs/7zSea/src/TimerOutput.jl:216 [inlined]
[3] #codegen#119(::Bool, ::Bool, ::Bool, ::Bool, ::Bool, ::typeof(CUDAnative.codegen), ::Symbol, ::CUDAnative.CompilerJob) at /home/dsd/.julia/packages/CUDAnative/RcKr9/src/compiler/driver.jl:186
[4] #codegen at /home/dsd/.julia/packages/CUDAnative/RcKr9/src/compiler/driver.jl:0 [inlined]
[5] #compile#118(::Bool, ::Bool, ::Bool, ::Bool, ::Bool, ::typeof(CUDAnative.compile), ::Symbol, ::CUDAnative.CompilerJob) at /home/dsd/.julia/packages/CUDAnative/RcKr9/src/compiler/driver.jl:47
[6] #compile#117 at ./none:0 [inlined]
[7] compile at /home/dsd/.julia/packages/CUDAnative/RcKr9/src/compiler/driver.jl:28 [inlined] (repeats 2 times)
[8] macro expansion at /home/dsd/.julia/packages/CUDAnative/RcKr9/src/execution.jl:378 [inlined]
[9] #cufunction#159(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(CUDAnative.cufunction), ::typeof(CuArrays.mapreducedim_kernel_parallel), ::Type{Tuple{typeof(identity),typeof(Base.add_sum),CUDAnative.CuDeviceArray{Float32,2,CUDAnative.AS.Global},CUDAnative.CuDeviceArray{Float32,2,CUDAnative.AS.Global},CartesianIndices{2,Tuple{Base.OneTo{Int64},Base.OneTo{Int64}}},Int64,Int64}}) at /home/dsd/.julia/packages/CUDAnative/RcKr9/src/execution.jl:347
[10] cufunction(::Function, ::Type) at /home/dsd/.julia/packages/CUDAnative/RcKr9/src/execution.jl:347
[11] macro expansion at /home/dsd/.julia/packages/CuArrays/R5hMd/src/mapreduce.jl:68 [inlined]
[12] macro expansion at ./gcutils.jl:87 [inlined]
[13] mapreducedim!(::Function, ::Function, ::CuArray{Float32,2}, ::CuArray{Float32,2}) at /home/dsd/.julia/packages/CuArrays/R5hMd/src/mapreduce.jl:65
[14] mapreducedim! at ./reducedim.jl:274 [inlined]
[15] mapreduce_dim at ./reducedim.jl:317 [inlined]
[16] mapreduce_impl at /home/dsd/.julia/packages/GPUArrays/CjRPU/src/mapreduce.jl:73 [inlined]
[17] #mapreduce#48 at /home/dsd/.julia/packages/GPUArrays/CjRPU/src/mapreduce.jl:59 [inlined]
[18] #mapreduce at ./none:0 [inlined]
[19] sum at ./reducedim.jl:679 [inlined]
[20] sum at ./reducedim.jl:678 [inlined]
[21] #sum#559 at ./reducedim.jl:652 [inlined]
[22] #sum at ./none:0 [inlined]
[23] #forward#481 at /home/dsd/.julia/packages/Tracker/RRYy6/src/lib/array.jl:315 [inlined]
[24] #forward at ./none:0 [inlined]
[25] #track#1 at /home/dsd/.julia/packages/Tracker/RRYy6/src/Tracker.jl:51 [inlined]
[26] #track at ./none:0 [inlined]
[27] #sum#480 at /home/dsd/.julia/packages/Tracker/RRYy6/src/lib/array.jl:312 [inlined]
[28] #sum at ./none:0 [inlined]
[29] unbroadcast(::TrackedArray{…,CuArray{Float32,1}}, ::TrackedArray{…,CuArray{Float32,2}}) at /home/dsd/.julia/packages/Tracker/RRYy6/src/lib/array.jl:467
[30] back(::Tracker.Call{getfield(Tracker, Symbol("#back#548")){2,getfield(Base.Broadcast, Symbol("##2#4")){getfield(Base.Broadcast, Symbol("##8#10")){getfield(Base.Broadcast, Symbol("##1#3")),getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##5#6")){getfield(Base.Broadcast, Symbol("##7#9"))}},getfield(Base.Broadcast, Symbol("##11#12")){getfield(Base.Broadcast, Symbol("##11#12")){getfield(Base.Broadcast, Symbol("##13#14"))}},getfield(Base.Broadcast, Symbol("##15#16")){getfield(Base.Broadcast, Symbol("##15#16")){getfield(Base.Broadcast, Symbol("##17#18"))}},typeof(+)},typeof(identity)},Tuple{TrackedArray{…,CuArray{Float32,2}},TrackedArray{…,CuArray{Float32,1}}}},Tuple{Tracker.Tracked{CuArray{Float32,2}},Tracker.Tracked{CuArray{Float32,1}}}}, ::CuArray{Float32,2}, ::Bool) at ./tuple.jl:159
[31] back(::Tracker.Tracked{CuArray{Float32,2}}, ::CuArray{Float32,2}, ::Bool) at /home/dsd/.julia/packages/Tracker/RRYy6/src/back.jl:58
[32] foreach at /home/dsd/.julia/packages/Tracker/RRYy6/src/back.jl:38 [inlined]
[33] back(::Tracker.Call{getfield(Tracker, Symbol("##511#512")){TrackedArray{…,CuArray{Float32,2}}},Tuple{Tracker.Tracked{CuArray{Float32,2}}}}, ::CuArray{Float32,2}, ::Bool) at /home/dsd/.julia/packages/Tracker/RRYy6/src/back.jl:38
[34] back(::Tracker.Tracked{CuArray{Float32,2}}, ::CuArray{Float32,2}, ::Bool) at /home/dsd/.julia/packages/Tracker/RRYy6/src/back.jl:58
[35] #13 at /home/dsd/.julia/packages/Tracker/RRYy6/src/back.jl:38 [inlined]
[36] foreach at ./abstractarray.jl:1921 [inlined]
... (the last 4 lines are repeated 1 more time)
[41] back(::Tracker.Call{getfield(Tracker, Symbol("##482#483")){TrackedArray{…,CuArray{Float32,2}}},Tuple{Tracker.Tracked{CuArray{Float32,2}}}}, ::Float32, ::Bool) at /home/dsd/.julia/packages/Tracker/RRYy6/src/back.jl:38
[42] back(::Tracker.Tracked{Float32}, ::Float32, ::Bool) at /home/dsd/.julia/packages/Tracker/RRYy6/src/back.jl:58
[43] foreach at /home/dsd/.julia/packages/Tracker/RRYy6/src/back.jl:38 [inlined]
[44] back(::Tracker.Call{getfield(Tracker, Symbol("##199#200")),Tuple{Tracker.Tracked{Float32}}}, ::Float32, ::Bool) at /home/dsd/.julia/packages/Tracker/RRYy6/src/back.jl:38
[45] back(::Tracker.Tracked{Float32}, ::Float32, ::Bool) at /home/dsd/.julia/packages/Tracker/RRYy6/src/back.jl:58
[46] #13 at /home/dsd/.julia/packages/Tracker/RRYy6/src/back.jl:38 [inlined]
[47] foreach at ./abstractarray.jl:1921 [inlined]
[48] back(::Tracker.Call{getfield(Tracker, Symbol("##278#281")){Rational{Int64}},Tuple{Tracker.Tracked{Float32},Nothing}}, ::Float32, ::Bool) at /home/dsd/.julia/packages/Tracker/RRYy6/src/back.jl:38
[49] back(::Tracker.Tracked{Float32}, ::Int64, ::Bool) at /home/dsd/.julia/packages/Tracker/RRYy6/src/back.jl:58
[50] #back!#15 at /home/dsd/.julia/packages/Tracker/RRYy6/src/back.jl:77 [inlined]
[51] #back! at ./none:0 [inlined]
[52] #back!#32 at /home/dsd/.julia/packages/Tracker/RRYy6/src/lib/real.jl:16 [inlined]
[53] back!(::Tracker.TrackedReal{Float32}) at /home/dsd/.julia/packages/Tracker/RRYy6/src/lib/real.jl:14
[54] gradient(::getfield(Flux.Optimise, Symbol("##15#21")){typeof(loss),Tuple{CuArray{Float32,4},Flux.OneHotMatrix{CuArray{Flux.OneHotVector,1}}}}, ::Tracker.Params) at /home/dsd/.julia/packages/Tracker/RRYy6/src/back.jl:4
[55] #gradient#24(::Bool, ::typeof(Tracker.gradient), ::Function, ::Tracker.Params) at /home/dsd/.julia/packages/Tracker/RRYy6/src/back.jl:164
[56] gradient at /home/dsd/.julia/packages/Tracker/RRYy6/src/back.jl:164 [inlined]
[57] macro expansion at /home/dsd/.julia/packages/Flux/3GyGP/src/optimise/train.jl:71 [inlined]
[58] macro expansion at /home/dsd/.julia/packages/Juno/TfNYn/src/progress.jl:133 [inlined]
[59] #train!#12(::getfield(Flux, Symbol("#throttled#14")){getfield(Flux, Symbol("##throttled#10#15")){Bool,Bool,getfield(Main, Symbol("##7#9")),Int64}}, ::typeof(Flux.Optimise.train!), ::Function, ::Tracker.Params, ::Base.Generator{Base.Iterators.PartitionIterator{Array{Int64,1}},getfield(Main, Symbol("##33#34"))}, ::ADAM) at /home/dsd/.julia/packages/Flux/3GyGP/src/optimise/train.jl:69
[60] (::getfield(Flux.Optimise, Symbol("#kw##train!")))(::NamedTuple{(:cb,),Tuple{getfield(Flux, Symbol("#throttled#14")){getfield(Flux, Symbol("##throttled#10#15")){Bool,Bool,getfield(Main, Symbol("##7#9")),Int64}}}}, ::typeof(Flux.Optimise.train!), ::Function, ::Tracker.Params, ::Base.Generator{Base.Iterators.PartitionIterator{Array{Int64,1}},getfield(Main, Symbol("##33#34"))}, ::ADAM) at ./none:0

load custom dataset

Does Flux or Metalhead have a way to serially load large datasets (too large to be fully loaded into memory as numpy arrays)? I saw that Metalhead.datasets() supports reading in standard datasets, but could it be modified to load custom datasets?

Pretrained VGG is very bad

It gets most things quite wrong. On slack it was suggested that we may have to look at the way we're doing preprocessing to see if we got something wrong there.

MethodError: no method matching maxpool(::Array{Float32,4}, ::Tuple{Int64,Int64})

When running the test, GoogleNet() model, all get this error.

Here is the runtest result:

basic Model Tests: Error During Test at /home/dsd/Downloads/Metalhead.jl/test/runtests.jl:4
Got exception outside of a @test
MethodError: no method matching maxpool(::Array{Float32,4}, ::Tuple{Int64,Int64})
Closest candidates are:
maxpool(::AbstractArray{xT,N}, !Matched::NNlib.PoolDims; kwargs...) where {xT, N} at /home/dsd/.julia/packages/TimerOutputs/7zSea/src/TimerOutput.jl:198
Stacktrace:
[1] (::getfield(Metalhead, Symbol("##44#50")))(::Array{Float32,4}) at /home/dsd/.julia/packages/Metalhead/4zI8b/src/vgg19.jl:6
[2] applychain(::Tuple{getfield(Metalhead, Symbol("##44#50")),Flux.Conv{2,2,typeof(NNlib.relu),Array{Float32,4},Array{Float32,1}},Flux.Conv{2,2,typeof(NNlib.relu),Array{Float32,4},Array{Float32,1}},getfield(Metalhead, Symbol("##45#51")),Flux.Conv{2,2,typeof(NNlib.relu),Array{Float32,4},Array{Float32,1}},Flux.Conv{2,2,typeof(NNlib.relu),Array{Float32,4},Array{Float32,1}},Flux.Conv{2,2,typeof(NNlib.relu),Array{Float32,4},Array{Float32,1}},Flux.Conv{2,2,typeof(NNlib.relu),Array{Float32,4},Array{Float32,1}},getfield(Metalhead, Symbol("##46#52")),Flux.Conv{2,2,typeof(NNlib.relu),Array{Float32,4},Array{Float32,1}},Flux.Conv{2,2,typeof(NNlib.relu),Array{Float32,4},Array{Float32,1}},Flux.Conv{2,2,typeof(NNlib.relu),Array{Float32,4},Array{Float32,1}},Flux.Conv{2,2,typeof(NNlib.relu),Array{Float32,4},Array{Float32,1}},getfield(Metalhead, Symbol("##47#53")),Flux.Conv{2,2,typeof(NNlib.relu),Array{Float32,4},Array{Float32,1}},Flux.Conv{2,2,typeof(NNlib.relu),Array{Float32,4},Array{Float32,1}},Flux.Conv{2,2,typeof(NNlib.relu),Array{Float32,4},Array{Float32,1}},Flux.Conv{2,2,typeof(NNlib.relu),Array{Float32,4},Array{Float32,1}},getfield(Metalhead, Symbol("##48#54")),getfield(Metalhead, Symbol("##49#55")),Flux.Dense{typeof(NNlib.relu),LinearAlgebra.Adjoint{Float32,Array{Float32,2}},Array{Float32,1}},Flux.Dropout{Float32},Flux.Dense{typeof(NNlib.relu),LinearAlgebra.Adjoint{Float32,Array{Float32,2}},Array{Float32,1}},Flux.Dropout{Float32},Flux.Dense{typeof(identity),LinearAlgebra.Adjoint{Float32,Array{Float32,2}},Array{Float32,1}},typeof(NNlib.softmax)}, ::Array{Float32,4}) at /home/dsd/.julia/packages/Flux/3GyGP/src/layers/basic.jl:31 (repeats 3 times)
[3] (::Flux.Chain{Tuple{Flux.Conv{2,2,typeof(NNlib.relu),Array{Float32,4},Array{Float32,1}},Flux.Conv{2,2,typeof(NNlib.relu),Array{Float32,4},Array{Float32,1}},getfield(Metalhead, Symbol("##44#50")),Flux.Conv{2,2,typeof(NNlib.relu),Array{Float32,4},Array{Float32,1}},Flux.Conv{2,2,typeof(NNlib.relu),Array{Float32,4},Array{Float32,1}},getfield(Metalhead, Symbol("##45#51")),Flux.Conv{2,2,typeof(NNlib.relu),Array{Float32,4},Array{Float32,1}},Flux.Conv{2,2,typeof(NNlib.relu),Array{Float32,4},Array{Float32,1}},Flux.Conv{2,2,typeof(NNlib.relu),Array{Float32,4},Array{Float32,1}},Flux.Conv{2,2,typeof(NNlib.relu),Array{Float32,4},Array{Float32,1}},getfield(Metalhead, Symbol("##46#52")),Flux.Conv{2,2,typeof(NNlib.relu),Array{Float32,4},Array{Float32,1}},Flux.Conv{2,2,typeof(NNlib.relu),Array{Float32,4},Array{Float32,1}},Flux.Conv{2,2,typeof(NNlib.relu),Array{Float32,4},Array{Float32,1}},Flux.Conv{2,2,typeof(NNlib.relu),Array{Float32,4},Array{Float32,1}},getfield(Metalhead, Symbol("##47#53")),Flux.Conv{2,2,typeof(NNlib.relu),Array{Float32,4},Array{Float32,1}},Flux.Conv{2,2,typeof(NNlib.relu),Array{Float32,4},Array{Float32,1}},Flux.Conv{2,2,typeof(NNlib.relu),Array{Float32,4},Array{Float32,1}},Flux.Conv{2,2,typeof(NNlib.relu),Array{Float32,4},Array{Float32,1}},getfield(Metalhead, Symbol("##48#54")),getfield(Metalhead, Symbol("##49#55")),Flux.Dense{typeof(NNlib.relu),LinearAlgebra.Adjoint{Float32,Array{Float32,2}},Array{Float32,1}},Flux.Dropout{Float32},Flux.Dense{typeof(NNlib.relu),LinearAlgebra.Adjoint{Float32,Array{Float32,2}},Array{Float32,1}},Flux.Dropout{Float32},Flux.Dense{typeof(identity),LinearAlgebra.Adjoint{Float32,Array{Float32,2}},Array{Float32,1}},typeof(NNlib.softmax)}})(::Array{Float32,4}) at /home/dsd/.julia/packages/Flux/3GyGP/src/layers/basic.jl:33
[4] (::VGG19)(::Array{Float32,4}) at /home/dsd/.julia/packages/Metalhead/4zI8b/src/vgg19.jl:46
[5] top-level scope at /home/dsd/Downloads/Metalhead.jl/test/runtests.jl:14
[6] top-level scope at /home/dsd/Downloads/julia/usr/share/julia/stdlib/v1.3/Test/src/Test.jl:1113
[7] top-level scope at /home/dsd/Downloads/Metalhead.jl/test/runtests.jl:5
[8] include at ./boot.jl:328 [inlined]
[9] include_relative(::Module, ::String) at ./loading.jl:1094
[10] include(::Module, ::String) at ./Base.jl:31
[11] exec_options(::Base.JLOptions) at ./client.jl:295
[12] _start() at ./client.jl:464

Test Summary: | Error Total
Basic Model Tests | 1 1
ERROR: Error while loading expression starting at /home/dsd/Downloads/Metalhead.jl/test/runtests.jl:4
caused by [exception 1]
Some tests did not pass: 0 passed, 0 failed, 1 errored, 0 broken

Download CIFAR-10

We should be downloading this automatically when needed, otherwise the model-zoo example does not actually run through.

cc @avik-pal

Install

Can the Manifest.toml be removed? It messes up a lot of compatibility due to old version numbers in it (like CUDA is fixed to 1.2.1 while it's now at 2.4)

VGG example fails on GPU

The code does run successfully on CPU, but as soon as I get CuArrays involved I get this issue. Same problem if I try x = rand(Float32, 224, 224, 3, 1); vgg(x);

julia> using CuArrays, Metalhead, Flux
julia> using Metalhead: classify
julia> vgg = VGG19() |> gpu
VGG19()
julia> img = load("/home/tyler/Downloads/elephant.jpg");
julia> classify(vgg,img)
ERROR: conversion to pointer not defined for CuArray{Float32,4}
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] unsafe_convert(::Type{Ptr{Float32}}, ::CuArray{Float32,4}) at ./pointer.jl:67
 [3] pointer(::CuArray{Float32,4}) at ./abstractarray.jl:880
 [4] #conv2d!#39(::Float32, ::Function, ::Array{Float32,4}, ::Array{Float32,4}, ::CuArray{Float32,4}, ::NNlib.ConvDims{(224, 224),(3, 3),3,(1, 1),(1, 1, 1, 1),(1, 1),true}) at /home/tyler/.julia/packages/NNlib/UpABH/src/impl/conv.jl:350
 [5] (::getfield(NNlib, Symbol("#kw##conv2d!")))(::NamedTuple{(:alpha,),Tuple{Float32}}, ::typeof(NNlib.conv2d!), ::Array{Float32,4}, ::Array{Float32,4}, ::CuArray{Float32,4}, ::NNlib.ConvDims{(224, 224),(3, 3),3,(1, 1),(1, 1, 1, 1),(1, 1),true}) at ./none:0
 [6] #conv2d!#40(::Tuple{Int64,Int64}, ::Tuple{Int64,Int64}, ::Tuple{Int64,Int64}, ::Int64, ::Float32, ::Function, ::Array{Float32,4}, ::Array{Float32,4}, ::CuArray{Float32,4}) at /home/tyler/.julia/packages/NNlib/UpABH/src/impl/conv.jl:373
 [7] #conv2d! at ./none:0 [inlined]
 [8] #conv!#68 at /home/tyler/.julia/packages/NNlib/UpABH/src/conv.jl:118 [inlined]
 [9] #conv! at ./none:0 [inlined]
 [10] #conv#54(::Nothing, ::Tuple{Int64,Int64}, ::Tuple{Int64,Int64}, ::Tuple{Int64,Int64}, ::Function, ::Array{Float32,4}, ::CuArray{Float32,4}) at /home/tyler/.julia/packages/NNlib/UpABH/src/conv.jl:62
 [11] #conv at ./none:0 [inlined]
 [12] (::Conv{2,typeof(relu),CuArray{Float32,4},CuArray{Float32,1}})(::Array{Float32,4}) at /home/tyler/.julia/packages/Flux/lz7S9/src/layers/conv.jl:53
 [13] Conv at /home/tyler/.julia/packages/Flux/lz7S9/src/layers/conv.jl:63 [inlined]
 [14] applychain at /home/tyler/.julia/packages/Flux/lz7S9/src/layers/basic.jl:31 [inlined]
 [15] (::Chain{Tuple{Conv{2,typeof(relu),CuArray{Float32,4},CuArray{Float32,1}},Conv{2,typeof(relu),CuArray{Float32,4},CuArray{Float32,1}},getfield(Metalhead, Symbol("##42#48")),Conv{2,typeof(relu),CuArray{Float32,4},CuArray{Float32,1}},Conv{2,typeof(relu),CuArray{Float32,4},CuArray{Float32,1}},getfield(Metalhead, Symbol("##43#49")),Conv{2,typeof(relu),CuArray{Float32,4},CuArray{Float32,1}},Conv{2,typeof(relu),CuArray{Float32,4},CuArray{Float32,1}},Conv{2,typeof(relu),CuArray{Float32,4},CuArray{Float32,1}},Conv{2,typeof(relu),CuArray{Float32,4},CuArray{Float32,1}},getfield(Metalhead, Symbol("##44#50")),Conv{2,typeof(relu),CuArray{Float32,4},CuArray{Float32,1}},Conv{2,typeof(relu),CuArray{Float32,4},CuArray{Float32,1}},Conv{2,typeof(relu),CuArray{Float32,4},CuArray{Float32,1}},Conv{2,typeof(relu),CuArray{Float32,4},CuArray{Float32,1}},getfield(Metalhead, Symbol("##45#51")),Conv{2,typeof(relu),CuArray{Float32,4},CuArray{Float32,1}},Conv{2,typeof(relu),CuArray{Float32,4},CuArray{Float32,1}},Conv{2,typeof(relu),CuArray{Float32,4},CuArray{Float32,1}},Conv{2,typeof(relu),CuArray{Float32,4},CuArray{Float32,1}},getfield(Metalhead, Symbol("##46#52")),getfield(Metalhead, Symbol("##47#53")),Dense{typeof(relu),LinearAlgebra.Adjoint{Float32,CuArray{Float32,2}},CuArray{Float32,1}},Dropout{Float32},Dense{typeof(relu),LinearAlgebra.Adjoint{Float32,CuArray{Float32,2}},CuArray{Float32,1}},Dropout{Float32},Dense{typeof(identity),LinearAlgebra.Adjoint{Float32,CuArray{Float32,2}},CuArray{Float32,1}},typeof(softmax)}})(::Array{Float32,4}) at /home/tyler/.julia/packages/Flux/lz7S9/src/layers/basic.jl:33
 [16] (::VGG19)(::Array{Float32,4}) at /home/tyler/.julia/packages/Metalhead/fYeSU/src/vgg19.jl:46
 [17] forward(::VGG19, ::Array{ColorTypes.RGB{FixedPointNumbers.Normed{UInt8,8}},2}) at /home/tyler/.julia/packages/Metalhead/fYeSU/src/utils.jl:70
 [18] classify(::VGG19, ::Array{ColorTypes.RGB{FixedPointNumbers.Normed{UInt8,8}},2}) at /home/tyler/.julia/packages/Metalhead/fYeSU/src/utils.jl:98
 [19] top-level scope at none:0

Edit: here are my manifest & project files: https://gist.github.com/tbenst/f370a5baae6e120c5869f591d5006794 for Julia 1.1

Loader For Pascal VOC dataset

I think we Metalhead.jl also needs to have a loader for PASCAL-VOC dataset as it is commonly required for object detection and localisation tasks. If required, I cango forward towards adding it...

Incompatible with Flux 0.11

PLease upgrade dependencies that Flux 0.11 can be used. I'm experimenting with the CIFAR10 example and including Metalhead downgrades a lot of libraries.

MethodError: no method matching conv(::Array{Float32,4}, ::Array{Float64,4}; ...)

When I was playing around with Metalhead I came across this error and I'm not sure how to fix it. I think it has to do with one of the arguments being an Array{Float32,4} and the other an Array{Float64,4} The same error happens with ResNet and DenseNet, but VGG19 and SqueezeNet work fine.

julia> x = rand(Float32, 224, 224, 3, 1)
224×224×3×1 Array{Float32,4}:
...

julia> gn = GoogleNet()
GoogleNet()

julia> gn(x)
ERROR: MethodError: no method matching conv(::Array{Float32,4}, ::Array{Float64,4}; stride=(2, 2), pad=(3, 3), dilation=(1, 1))
Closest candidates are:
  conv(::AbstractArray, ::TrackedArray; kw...) at /Users/christophermurphy/.julia/packages/Flux/UHjNa/src/tracker/array.jl:305
  conv(::TrackedArray, ::AbstractArray; kw...) at /Users/christophermurphy/.julia/packages/Flux/UHjNa/src/tracker/array.jl:306
  conv(::A<:AbstractArray, ::A<:AbstractArray; pad, stride, dilation) where A<:AbstractArray at /Users/christophermurphy/.julia/packages/NNlib/0EAe7/src/conv.jl:28
Stacktrace:
 [1] #_forward#335(::Base.Iterators.Pairs{Symbol,Tuple{Int64,Int64},Tuple{Symbol,Symbol,Symbol},NamedTuple{(:stride, :pad, :dilation),Tuple{Tuple{Int64,Int64},Tuple{Int64,Int64},Tuple{Int64,Int64}}}}, ::Function, ::typeof(NNlib.conv), ::Array{Float32,4}, ::TrackedArray{…,Array{Float64,4}}) at /Users/christophermurphy/.julia/packages/Flux/UHjNa/src/tracker/array.jl:308
 [2] (::getfield(Flux.Tracker, Symbol("#kw##_forward")))(::NamedTuple{(:stride, :pad, :dilation),Tuple{Tuple{Int64,Int64},Tuple{Int64,Int64},Tuple{Int64,Int64}}}, ::typeof(Flux.Tracker._forward), ::typeof(NNlib.conv), ::Array{Float32,4}, ::TrackedArray{…,Array{Float64,4}}) at ./none:0
 [3] #track#1(::Base.Iterators.Pairs{Symbol,Tuple{Int64,Int64},Tuple{Symbol,Symbol,Symbol},NamedTuple{(:stride, :pad, :dilation),Tuple{Tuple{Int64,Int64},Tuple{Int64,Int64},Tuple{Int64,Int64}}}}, ::Function, ::typeof(NNlib.conv), ::Array{Float32,4}, ::Vararg{Any,N} where N) at /Users/christophermurphy/.julia/packages/Flux/UHjNa/src/tracker/Tracker.jl:50
 [4] (::getfield(Flux.Tracker, Symbol("#kw##track")))(::NamedTuple{(:stride, :pad, :dilation),Tuple{Tuple{Int64,Int64},Tuple{Int64,Int64},Tuple{Int64,Int64}}}, ::typeof(Flux.Tracker.track), ::typeof(NNlib.conv), ::Array{Float32,4}, ::Vararg{Any,N} where N) at ./none:0
 [5] #conv#333(::Base.Iterators.Pairs{Symbol,Tuple{Int64,Int64},Tuple{Symbol,Symbol,Symbol},NamedTuple{(:stride, :pad, :dilation),Tuple{Tuple{Int64,Int64},Tuple{Int64,Int64},Tuple{Int64,Int64}}}}, ::Function, ::Array{Float32,4}, ::TrackedArray{…,Array{Float64,4}}) at /Users/christophermurphy/.julia/packages/Flux/UHjNa/src/tracker/array.jl:305
 [6] (::getfield(NNlib, Symbol("#kw##conv")))(::NamedTuple{(:stride, :pad, :dilation),Tuple{Tuple{Int64,Int64},Tuple{Int64,Int64},Tuple{Int64,Int64}}}, ::typeof(NNlib.conv), ::Array{Float32,4}, ::TrackedArray{…,Array{Float64,4}}) at ./none:0
 [7] (::Conv{2,typeof(relu),TrackedArray{…,Array{Float64,4}},TrackedArray{…,Array{Float64,1}}})(::Array{Float32,4}) at /Users/christophermurphy/.julia/packages/Flux/UHjNa/src/layers/conv.jl:44
 [8] (::getfield(Flux, Symbol("##60#61")))(::Array{Float32,4}, ::Conv{2,typeof(relu),TrackedArray{…,Array{Float64,4}},TrackedArray{…,Array{Float64,1}}}) at /Users/christophermurphy/.julia/packages/Flux/UHjNa/src/layers/basic.jl:31
 [9] mapfoldl_impl(::typeof(identity), ::getfield(Flux, Symbol("##60#61")), ::NamedTuple{(:init,),Tuple{Array{Float32,4}}}, ::Array{Any,1}) at ./reduce.jl:43
 [10] #mapfoldl#170 at ./reduce.jl:70 [inlined]
 [11] #mapfoldl at ./none:0 [inlined]
 [12] #foldl#171 at ./reduce.jl:88 [inlined]
 [13] #foldl at ./none:0 [inlined]
 [14] Chain at /Users/christophermurphy/.julia/packages/Flux/UHjNa/src/layers/basic.jl:31 [inlined]
 [15] (::GoogleNet)(::Array{Float32,4}) at /Users/christophermurphy/.julia/packages/Metalhead/fYeSU/src/googlenet.jl:83
 [16] top-level scope at none:0

When I make x an Array{Float64,4} the opposite thing happens (GoogleNet, ResNet and DenseNet work fine, but VGG19 and SqueezeNet stop working).

Any thoughts?

Metalhead APIs used in README and datasets/README seem to no longer exist

For example - Metalhead.datasets() does not work, nor does the download(CIFAR10).

It would be good to update the README with updated APIs.

Resnet50 doesn't seem to work

I see from the tests performed when @ayush1999 added resnet50 to Metalhead, that the network takes as input Float32 224x224x3x1 arrays, but when I feed cat images in that format to it I don't get meaningful results. Any example of making it work successfully?

Add steps to create new pre-trained models

It would be nice to document how folks can add pre-trained models to this repo

ResNet skip connection definition seems wrong

Hi, the definition of the ResNet skip connection via 1x1 MaxPool with stride, concatenated with zeros seems very strange to me. In the paper they mention a matrix multiply, which is closer to a 1x1 conv. PyTorch uses 1x1 conv indeed, together with a stride of 2 (or optional dilation) where necessary. Should be easy to fix, maybe I can provide a PR.

Where is the Metalhead.jl documentation?

Hey @DhairyaLGandhi can we get a link both in the Readme to the docs for this package as well as on the main repo page?

It also looks like there may not be docs right now, if this is the case, is there any plan to start going through this process? I can kick off that process if that would help.

Addition of classic LetNet-5 model

Just seeing around so many amazing and state of the art models implemented, was wondering whether the classic LeNet 5 model ( still has some use cases as the initial testing architecture for most of the ML developers ) could also be added.

Prediction/preprocessing helper

Is there interest in a prediction or preprocessing helper that helps people perform on (batches) of data for (Image-Net) pretrained models?

Thinking something like

function preprocess(img::AbstractArray{U, N}, sz = (224, 224); T = Float32, C = RGB{N0f8} )
    item = DataAugmentation.Image{N}(img)
    tfm = ToEltype(C) |> CenterResizeCrop(sz) |> ToTensor(T)  |> Normalize(IMAGENET_STATS)
    return apply(tfm, item) |> itemdata
end

preprocess(imgs::AbstractVector{<:AbstractArray}) = ...

This would require adding a dep to DataAugmentation.jl, though.

Enable code coverage

Can someone with admin rights to this repo turn on codecov.io or coveralls or something so we can get those sweet sweet coverage metrics?

Add config parameters to network constructors: n_channel_input and output_softmax?

Hi there,

I'm trying out the image networks for classification of data with some imagy-properities (e.g. x-y-locality) but not 3 channels like conventional images. As far as I can tell, the 3 input channels are hard coded in the network implementation. Would you consider a PR adding a n_channel_input = 3 keyword argument to the networks?

Also, would you consider a PR for an optional output_softmax = false parameter, which (if true) adds a final softmax layer?

best
Jon

ResNet50 not defined

julia> using Flux, Metalhead

julia> model = ResNet50()
ERROR: UndefVarError: ResNet50 not defined
Stacktrace:
 [1] top-level scope
   @ REPL[1]:1
 [2] top-level scope
   @ ~/.julia/packages/CUDA/YpW0k/src/initialization.jl:52

Any idea what I am missing here?

Readme example fails

Trying to execute the example in the readme.md file:

But when running the classify command, I get the following error:

ERROR: MethodError: no method matching maxpool(::Array{Float32,4}, ::Tuple{Int64,Int64})

I am running Julia version 1.1.1 on Ubuntu 16.04

Float32/Float64 Dilemma

Hello,

Here, you force weights to be Float64:

Metalhead.jl/src/resnet.jl

Line 70 in 076d4b7

weights[string(ele)] = convert(Array{Float64, N} where N, weight[ele])

Then, you force input to be Float32:

Metalhead.jl/src/utils.jl

Line 65 in 076d4b7

return Float32.(permutedims(im, (3, 2, 1))[:,:,:,:].*255)

I believe, this makes ResNet crash:

julia> classify(resnet,img)
ERROR: MethodError: no method matching conv(::Array{Float32,4}, ::Array{Float64,4}; stride=(2, 2), pad=(3, 3), dilation=(1, 1))
Closest candidates are:
  conv(::AbstractArray, ::Flux.Tracker.TrackedArray; kw...) at /kuacc/users/eakyurek13/.julia/packages/Flux/oN61x/src/tracker/array.jl:344
  conv(::Flux.Tracker.TrackedArray, ::AbstractArray; kw...) at /kuacc/users/eakyurek13/.julia/packages/Flux/oN61x/src/tracker/array.jl:345
  conv(::A<:AbstractArray, ::A<:AbstractArray; pad, stride, dilation) where A<:AbstractArray at /kuacc/users/eakyurek13/.julia/packages/NNlib/x0XUf/src/conv.jl:28
Stacktrace:
 [1] #_forward#433(::Base.Iterators.Pairs{Symbol,Tuple{Int64,Int64},Tuple{Symbol,Symbol,Symbol},NamedTuple{(:stride, :pad, :dilation),Tuple{Tuple{Int64,Int64},Tuple{Int64,Int64},Tuple{Int64,Int64}}}}, ::Function, ::typeof(NNlib.conv), ::Array{Float32,4}, ::TrackedArray{…,Array{Float64,4}}) at /kuacc/users/eakyurek13/.julia/packages/Flux/oN61x/src/tracker/array.jl:347
 [2] (::getfield(Flux.Tracker, Symbol("#kw##_forward")))(::NamedTuple{(:stride, :pad, :dilation),Tuple{Tuple{Int64,Int64},Tuple{Int64,Int64},Tuple{Int64,Int64}}}, ::typeof(Flux.Tracker._forward), ::typeof(NNlib.conv), ::Array{Float32,4}, ::TrackedArray{…,Array{Float64,4}}) at ./none:0
 [3] #track#1(::Base.Iterators.Pairs{Symbol,Tuple{Int64,Int64},Tuple{Symbol,Symbol,Symbol},NamedTuple{(:stride, :pad, :dilation),Tuple{Tuple{Int64,Int64},Tuple{Int64,Int64},Tuple{Int64,Int64}}}}, ::Function, ::typeof(NNlib.conv), ::Array{Float32,4}, ::Vararg{Any,N} where N) at /kuacc/users/eakyurek13/.julia/packages/Flux/oN61x/src/tracker/Tracker.jl:50
 [4] (::getfield(Flux.Tracker, Symbol("#kw##track")))(::NamedTuple{(:stride, :pad, :dilation),Tuple{Tuple{Int64,Int64},Tuple{Int64,Int64},Tuple{Int64,Int64}}}, ::typeof(Flux.Tracker.track), ::typeof(NNlib.conv), ::Array{Float32,4}, ::Vararg{Any,N} where N) at ./none:0
 [5] #conv#431(::Base.Iterators.Pairs{Symbol,Tuple{Int64,Int64},Tuple{Symbol,Symbol,Symbol},NamedTuple{(:stride, :pad, :dilation),Tuple{Tuple{Int64,Int64},Tuple{Int64,Int64},Tuple{Int64,Int64}}}}, ::Function, ::Array{Float32,4}, ::TrackedArray{…,Array{Float64,4}}) at /kuacc/users/eakyurek13/.julia/packages/Flux/oN61x/src/tracker/array.jl:344
 [6] (::getfield(NNlib, Symbol("#kw##conv")))(::NamedTuple{(:stride, :pad, :dilation),Tuple{Tuple{Int64,Int64},Tuple{Int64,Int64},Tuple{Int64,Int64}}}, ::typeof(NNlib.conv), ::Array{Float32,4}, ::TrackedArray{…,Array{Float64,4}}) at ./none:0
 [7] (::Flux.Conv{2,typeof(identity),TrackedArray{…,Array{Float64,4}},TrackedArray{…,Array{Float64,1}}})(::Array{Float32,4}) at /kuacc/users/eakyurek13/.julia/packages/Flux/oN61x/src/layers/conv.jl:44
 [8] (::getfield(Flux, Symbol("##60#61")))(::Array{Float32,4}, ::Flux.Conv{2,typeof(identity),TrackedArray{…,Array{Float64,4}},TrackedArray{…,Array{Float64,1}}}) at /kuacc/users/eakyurek13/.julia/packages/Flux/oN61x/src/layers/basic.jl:31
 [9] mapfoldl_impl(::typeof(identity), ::getfield(Flux, Symbol("##60#61")), ::NamedTuple{(:init,),Tuple{Array{Float32,4}}}, ::Array{Any,1}) at ./reduce.jl:43
 [10] Chain at ./reduce.jl:70 [inlined]
 [11] ResNet at /kuacc/users/eakyurek13/.julia/packages/Metalhead/fYeSU/src/resnet.jl:98 [inlined]
 [12] forward(::ResNet, ::Array{ColorTypes.RGB{FixedPointNumbers.Normed{UInt8,8}},2}) at /kuacc/users/eakyurek13/.julia/packages/Metalhead/fYeSU/src/utils.jl:70
 [13] classify(::ResNet, ::Array{ColorTypes.RGB{FixedPointNumbers.Normed{UInt8,8}},2}) at /kuacc/users/eakyurek13/.julia/packages/Metalhead/fYeSU/src/utils.jl:98
 [14] top-level scope at none:0

ResNet producing non deterministic and wrong predictions

Predictions from the ResNet pre-trained models seems to have an issue. The predicted probabilities are changing at each call and essentially look like random. VGG19 does work fine however.

using Flux
using Metalhead

img = Metalhead.load("data/cats/cats_00001.jpg");

julia> classify(VGG19(), img)
"tiger cat"

julia> classify(VGG19(), img)
"tiger cat"

julia> classify(ResNet(), img)
"reel"

julia> classify(ResNet(), img)
"abacus"

julia> classify(ResNet(), img)
"spotlight, spot"

julia> classify(ResNet(), img)
"fountain"

I first though it might be an issue with a different preprocessing required for ResNet, thought from the randomness in the output of the model, I'd guess some layers didn't get properly defined, possibly the BatchNorm?

If using testmode!, resnet will then produces the same predictions after each run. However, a different set of weights seem to be initialized right at each call to ``

resnet = ResNet().layers
testmode!(resnet)
cat_pred_resnet = resnet(preprocess(RGB.(img)))
julia> findmax(cat_pred_resnet)
(1.0f0, CartesianIndex(900, 1))

cat_pred_resnet = resnet(preprocess(RGB.(img)))
julia> findmax(cat_pred_resnet)
(1.0f0, CartesianIndex(900, 1))

resnet = ResNet().layers
testmode!(resnet)
cat_pred_resnet = resnet(preprocess(RGB.(img)))
julia> findmax(cat_pred_resnet)
(0.99877006f0, CartesianIndex(413, 1))

TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Does `Metalhead.classify` support pretrained models other than VGG19 ?

Hi, I'm trying to use pretrained model stored in this repository

VGG,
ResNet,
GoogleNet,
DenseNet
and so on.

I wrote the following code to see what happens.

using Metalhead
using Images
vgg = VGG19()
classify(vgg, "elephant.jpeg")

This works fine showing a kind ofelephant as result.

How about the other model e.g. DenseNet ?

dense = DenseNet()
classify(dense,"elephant.jpeg")
ERROR: MethodError: no method matching conv(::Array{Float32,4}, ::Array{Float64,4}; stride=(2, 2), pad=(3, 3), dilation=(1, 1))
Closest candidates are:
  conv(::AbstractArray, ::Flux.Tracker.TrackedArray; kw...) at /home/terasaki/.julia/packages/Flux/WgSRx/src/tracker/array.jl:342
  conv(::Flux.Tracker.TrackedArray, ::AbstractArray; kw...) at /home/terasaki/.julia/packages/Flux/WgSRx/src/tracker/array.jl:343
  conv(::A<:AbstractArray, ::A<:AbstractArray; pad, stride, dilation) where A<:AbstractArray at /home/terasaki/.julia/packages/NNlib/0EAe7/src/conv.jl:28
Stacktrace:
 [1] #_forward#433(::Base.Iterators.Pairs{Symbol,Tuple{Int64,Int64},Tuple{Symbol,Symbol,Symbol},NamedTuple{(:stride, :pad, :dilation),Tuple{Tuple{Int64,Int64},Tuple{Int64,Int64},Tuple{Int64,Int64}}}}, ::Function, ::typeof(NNlib.conv), ::Array{Float32,4}, ::TrackedArray{…,Array{Float64,4}}) at /home/terasaki/.julia/packages/Flux/WgSRx/src/tracker/array.jl:345
 [2] (::getfield(Flux.Tracker, Symbol("#kw##_forward")))(::NamedTuple{(:stride, :pad, :dilation),Tuple{Tuple{Int64,Int64},Tuple{Int64,Int64},Tuple{Int64,Int64}}}, ::typeof(Flux.Tracker._forward), ::typeof(NNlib.conv), ::Array{Float32,4}, ::TrackedArray{…,Array{Float64,4}}) at ./none:0
 [3] #track#1(::Base.Iterators.Pairs{Symbol,Tuple{Int64,Int64},Tuple{Symbol,Symbol,Symbol},NamedTuple{(:stride, :pad, :dilation),Tuple{Tuple{Int64,Int64},Tuple{Int64,Int64},Tuple{Int64,Int64}}}}, ::Function, ::typeof(NNlib.conv), ::Array{Float32,4}, ::Vararg{Any,N} where N) at /home/terasaki/.julia/packages/Flux/WgSRx/src/tracker/Tracker.jl:50
 [4] (::getfield(Flux.Tracker, Symbol("#kw##track")))(::NamedTuple{(:stride, :pad, :dilation),Tuple{Tuple{Int64,Int64},Tuple{Int64,Int64},Tuple{Int64,Int64}}}, ::typeof(Flux.Tracker.track), ::typeof(NNlib.conv), ::Array{Float32,4}, ::Vararg{Any,N} where N) at ./none:0
 [5] #conv#431(::Base.Iterators.Pairs{Symbol,Tuple{Int64,Int64},Tuple{Symbol,Symbol,Symbol},NamedTuple{(:stride, :pad, :dilation),Tuple{Tuple{Int64,Int64},Tuple{Int64,Int64},Tuple{Int64,Int64}}}}, ::Function, ::Array{Float32,4}, ::TrackedArray{…,Array{Float64,4}}) at /home/terasaki/.julia/packages/Flux/WgSRx/src/tracker/array.jl:342
 [6] (::getfield(NNlib, Symbol("#kw##conv")))(::NamedTuple{(:stride, :pad, :dilation),Tuple{Tuple{Int64,Int64},Tuple{Int64,Int64},Tuple{Int64,Int64}}}, ::typeof(NNlib.conv), ::Array{Float32,4}, ::TrackedArray{…,Array{Float64,4}}) at ./none:0
 [7] (::Flux.Conv{2,typeof(identity),TrackedArray{…,Array{Float64,4}},TrackedArray{…,Array{Float64,1}}})(::Array{Float32,4}) at /home/terasaki/.julia/packages/Flux/WgSRx/src/layers/conv.jl:44
 [8] (::getfield(Flux, Symbol("##60#61")))(::Array{Float32,4}, ::Flux.Conv{2,typeof(identity),TrackedArray{…,Array{Float64,4}},TrackedArray{…,Array{Float64,1}}}) at /home/terasaki/.julia/packages/Flux/WgSRx/src/layers/basic.jl:31
 [9] mapfoldl_impl(::typeof(identity), ::getfield(Flux, Symbol("##60#61")), ::NamedTuple{(:init,),Tuple{Array{Float32,4}}}, ::Array{Any,1}) at ./reduce.jl:43
 [10] Chain at ./reduce.jl:70 [inlined]
 [11] DenseNet at /home/terasaki/.julia/packages/Metalhead/fYeSU/src/densenet.jl:94 [inlined]
 [12] forward(::DenseNet, ::Array{ColorTypes.RGB{FixedPointNumbers.Normed{UInt8,8}},2}) at /home/terasaki/.julia/packages/Metalhead/fYeSU/src/utils.jl:70
 [13] classify(::DenseNet, ::String) at /home/terasaki/.julia/packages/Metalhead/fYeSU/src/utils.jl:98
 [14] top-level scope at none:0

also ResNet, GoogleNet show similar error.

ERROR: MethodError: no method matching conv(::Array{Float32,4}, ::Array{Float64,4}; stride=(2, 2), pad=(3, 3), dilation=(1, 1))

Does Metalhead.classify support pretrained models other than VGG19 ?

Error tagging new release

The tag name "Models" is not of the appropriate SemVer form (vX.Y.Z).
cc: @MikeInnes

ResNet does not work properly

Hi,

I am trying to classify CIFAR-10 using ResNet-Metalhead. The code is given below. As I find out that the size of the input should be 224x224x3x1 for Metalhead ResNet( ), I used the preprocess function of Metalhead to reshape the image size of CIFAR-10. I am getting an error "out of memory error" if I want to train the model for the whole data set. Another problem is if I want to train the model using a very small number of images like 100, I am not getting the error "out of memory" but I find out that the "loss" is not decreasing in each epoch, it has no change.
As CIFAR-10 has 10 classes instead of 1000, I slightly modified the source code of resnet.jl of Metalhead Package like below:

line 58: push!(layer_arr, MeanPool((7,7)))
line 59: push!(layer_arr, x -> reshape(x, :, size(x,4)))
line 60: push!(layer_arr, (Dense(2048, 1000)))
push!(layer_arr, (Dense(1000, 10)))
push!(layer_arr, softmax)

Chain(layer_arr...)
end

I make changes after line 60.

My code:
using Flux, Flux.Optimise
using Metalhead, Images
using Metalhead: trainimgs
using Images.ImageCore
using Flux: onehotbatch, onecold, throttle
using Base.Iterators: partition
using Flux: onehotbatch, onecold, logitcrossentropy, @epochs, @Treelike
using Metalhead: preprocess
using Metalhead
using Statistics
using Zygote

Metalhead.download(CIFAR10)
X = trainimgs(CIFAR10)
labels = onehotbatch([X[i].ground_truth.class for i in 1:100],1:10)

image(x) = x.img
ground_truth(x) = x.ground_truth
image.(X[rand(1:end, 10)])
imgs = [preprocess((X[i].img)) for i in 1:100]

train = ([(cat(imgs[i]..., dims = 4), labels[:,i]) for i in partition(1:90, 10)])
test = ([(cat(imgs[i]..., dims = 4), labels[:,i]) for i in partition(91:100, 10)])

m=ResNet( )

opt = ADAM(params(m))

loss(x, y) =Flux.logitcrossentropy(m(x), y)

cb = throttle(() -> (@show(loss(train[1][1],train[1][2])) ),
10)
@time(@epochs 2 Flux.train!(loss, train, opt,cb=cb))

let n = 0
for (x, y) in test
n += sum(onecold(m(x)) .== onecold(y))
end
println("Accuracy: $(n/10)")
end

CIFAR10 download not working

julia> Metalhead.download(CIFAR10)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  162M  100  162M    0     0  1835k      0  0:01:30  0:01:30 --:--:-- 1294k
tar: /home/mike/.julia/v0.6/Metalhead/src/datasets/../../datasets/cifar-10-batches-bin: Not found in archive
tar: Exiting with failure status due to previous errors
ERROR: failed process: Process(`tar xzf /home/mike/.julia/v0.6/Metalhead/src/datasets/../../datasets/cifar-10-binary.tar.gz /home/mike/.julia/v0.6/Metalhead/src/datasets/../../datasets/cifar-10-batches-bin`, ProcessExited(2)) [2]

@avik-pal

VGG19 not loading

julia> using Metalhead

julia> VGG19()
ERROR: MethodError: no method matching Flux.Conv2D(::Array{Float32,4}, ::Array{Float32,1}, ::NNlib.#relu; pad=1, stride=1)
Closest candidates are:
  Flux.Conv2D(::F, ::A, ::Int64, ::Int64) where {F, A} at /home/ranjanan/.julia/v0.6/Flux/src/layers/conv.jl:14 got unsupported keyword arguments "pad", "stride"
  Flux.Conv2D(::Tuple{Integer,Integer}, ::Pair{#s51,#s50} where #s50<:Integer where #s51<:Integer, ::Any; init, stride, pad) at /home/ranjanan/.julia/v0.6/Flux/src/layers/conv.jl:21
Stacktrace:
 [1] vgg19_layers() at /home/ranjanan/.julia/v0.6/Metalhead/src/vgg19.jl:3
 [2] Metalhead.VGG19() at /home/ranjanan/.julia/v0.6/Metalhead/src/vgg19.jl:40

This is on Flux 0.4.1 and Metalhead 0.1.0. Checking out Metalhead doesn't help. Checking out Flux master gives me this error:

julia> VGG19()
WARNING: Conv2D(args...; kw...) is deprecated, use Conv(args...; kw...) instead.
Stacktrace:
 [1] depwarn(::String, ::Symbol) at ./deprecated.jl:70
 [2] #Conv2D#107(::Array{Any,1}, ::Function, ::Array{Float32,4}, ::Vararg{Any,N} where N) at ./deprecated.jl:57
 [3] (::Flux.#kw##Conv2D)(::Array{Any,1}, ::Flux.#Conv2D, ::Array{Float32,4}, ::Vararg{Any,N} where N) at ./<missing>:0
 [4] vgg19_layers() at /home/ranjanan/.julia/v0.6/Metalhead/src/vgg19.jl:3
 [5] Metalhead.VGG19() at /home/ranjanan/.julia/v0.6/Metalhead/src/vgg19.jl:40
 [6] eval(::Module, ::Any) at ./boot.jl:235
 [7] eval_user_input(::Any, ::Base.REPL.REPLBackend) at ./REPL.jl:66
 [8] macro expansion at ./REPL.jl:97 [inlined]
 [9] (::Base.REPL.##1#2{Base.REPL.REPLBackend})() at ./event.jl:73
while loading no file, in expression starting on line 0
ERROR: MethodError: no method matching Flux.Conv(::NNlib.#relu, ::Array{Float32,4}, ::Array{Float32,1}, ::Int64, ::Int64)
Closest candidates are:
  Flux.Conv(::F, ::A, ::V, ::Tuple{Vararg{Int64,N}}, ::Tuple{Vararg{Int64,N}}) where {N, F, A, V} at /home/ranjanan/.julia/v0.6/Flux/src/layers/conv.jl:16
  Flux.Conv(::AbstractArray{T,N} where N, ::AbstractArray{T,1}, ::Any; stride, pad) where T at /home/ranjanan/.julia/v0.6/Flux/src/layers/conv.jl:24
  Flux.Conv(::Tuple{Vararg{Integer,N}}, ::Pair{#s94,#s93} where #s93<:Integer where #s94<:Integer, ::Any; init, stride, pad) where N at /home/ranjanan/.julia/v0.6/Flux/src/layers/conv.jl:29
Stacktrace:
 [1] (::Core.#kw#Type)(::Array{Any,1}, ::Type{Flux.Conv}, ::Array{Float32,4}, ::Array{Float32,1}, ::Function) at ./<missing>:0
 [2] #Conv2D#107(::Array{Any,1}, ::Function, ::Array{Float32,4}, ::Vararg{Any,N} where N) at ./deprecated.jl:59
 [3] (::Flux.#kw##Conv2D)(::Array{Any,1}, ::Flux.#Conv2D, ::Array{Float32,4}, ::Vararg{Any,N} where N) at ./<missing>:0
 [4] vgg19_layers() at /home/ranjanan/.julia/v0.6/Metalhead/src/vgg19.jl:3
 [5] Metalhead.VGG19() at /home/ranjanan/.julia/v0.6/Metalhead/src/vgg19.jl:40

predict not working

julia> predict(vgg, img)
Error showing value of type Metalhead.PredictionFrame:
ERROR: MethodError: no method matching print_frame_table(::Metalhead.##9#10, ::Metalhead.PredictionFrame)

Part of the models cannot be accessed

In the following models, we cannot extract a part of the model hence usage as feature extractor is not currently possible

Google Net
ResNet
Densenet

This would essentially be refactoring the code into something similar to that in model-zoo.

This issue is just to track the progess

VGG19 model fails to load with EOFError

I am trying to load the VGG19 model on CPU, however in the end it throws me an End of File Error.
It shows following error:

ERROR: EOFError: read end of file
Stacktrace:
 [1] read at ./iostream.jl:350 [inlined]
 [2] read(::IOStream, ::Type{BSON.BSONType}) at ./Enums.jl:22
 [3] parse_doc(::IOStream) at /home/blackforest/.julia/packages/BSON/lY9i8/src/read.jl:51
 [4] parse_tag(::IOStream, ::BSON.BSONType) at /home/blackforest/.julia/packages/BSON/lY9i8/src/read.jl:15
 [5] parse_doc(::IOStream) at /home/blackforest/.julia/packages/BSON/lY9i8/src/read.jl:53
 [6] parse at /home/blackforest/.julia/packages/BSON/lY9i8/src/read.jl:101 [inlined]
 [7] #open#271(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(open), ::typeof(BSON.parse), ::String) at ./io.jl:298
 [8] open at ./io.jl:296 [inlined]
 [9] parse at /home/blackforest/.julia/packages/BSON/lY9i8/src/read.jl:102 [inlined]
 [10] load at /home/blackforest/.julia/packages/BSON/lY9i8/src/read.jl:104 [inlined]
 [11] weights(::String) at /home/blackforest/.julia/packages/Metalhead/RZn9O/src/utils.jl:21
 [12] vgg19_layers() at /home/blackforest/.julia/packages/Metalhead/RZn9O/src/vgg19.jl:2
 [13] VGG19() at /home/blackforest/.julia/packages/Metalhead/RZn9O/src/vgg19.jl:39
 [14] top-level scope at REPL[2]:1

Add a MaxPool object

We should add a MaxPool object, so that displaying the layers in eg. VGG19 looks better:

julia> vgg.layers.layers
28-element Array{Any,1}:
 Conv((3, 3), 3=>64, NNlib.relu)   
 Conv((3, 3), 64=>64, NNlib.relu)  
 Metalhead.#52                     
 Conv((3, 3), 64=>128, NNlib.relu) 
 Conv((3, 3), 128=>128, NNlib.relu)
 Metalhead.#53                     
 Conv((3, 3), 128=>256, NNlib.relu)
 Conv((3, 3), 256=>256, NNlib.relu)
 Conv((3, 3), 256=>256, NNlib.relu)
 Conv((3, 3), 256=>256, NNlib.relu)
 Metalhead.#54                     
 Conv((3, 3), 256=>512, NNlib.relu)
 Conv((3, 3), 512=>512, NNlib.relu)
 ⋮                                 
 Conv((3, 3), 512=>512, NNlib.relu)
 Conv((3, 3), 512=>512, NNlib.relu)
 Conv((3, 3), 512=>512, NNlib.relu)
 Conv((3, 3), 512=>512, NNlib.relu)
 Metalhead.#56                     
 Metalhead.#57                     
 Dense(25088, 4096, NNlib.relu)    
 Flux.Dropout{Float32}(0.5, false) 
 Dense(4096, 4096, NNlib.relu)     
 Flux.Dropout{Float32}(0.5, false) 
 Dense(4096, 1000)                 
 NNlib.softmax

ImageNet dataset not available

In the dataset folder, there are some instructions on how to get the ImageNet dataset via the Kaggle API.
On the linked Kaggle page, an error message is reported:

I tried to follow the instructions, setting up the API and when I issue the command

kaggle competitions download -c imagenet-object-localization-challenge

I get a 403 - Forbidden message.

Different Models have different preprocessing needs

Here is a gist for the preprocessing needed for Resnet50 (use the updated model from #17).

The current preprocess function handles VGG and SqueezeNet fine, but ResNet50 needs the inputs scaled between -1 and 1 and NOT -255 to 255.

What would be the proper way to handle this? (Once we add more pretrained models they will have custom preprocessing needs)

Add EfficientNet with pretrained weights

@pxl-th has implemented EfficientNet models in his repo https://github.com/pxl-th/EfficientNet.jl and also implemented the infrastructure to load the weights from pytorch's checkpoints. If the author agrees, we could copy or move the code here and possibly extend the weight loading to other models.

ResNet50: difference between PyTorch and default Metalhead implementation

I noticed some differences between the PyTorch and the default Metalhead implementation for ResNet50.

The connectiondefaults to connection = (x, y) -> @. relu(x) + relu(y) while in PyTorch we have essentially connection = (x, y) -> @. relu(x + y) Unfortunately, the ResNet50 constructor does not give access to the connection parameter.

https://github.com/pytorch/vision/blob/6db1569c89094cf23f3bc41f79275c45e9fcb3f3/torchvision/models/resnet.py#L119

If we follow Pytorch, then the first Conv((7,7)) layer should have no bias I believe:
https://github.com/pytorch/vision/blob/6db1569c89094cf23f3bc41f79275c45e9fcb3f3/torchvision/models/resnet.py#L146

Are there any particular reasons for this difference?

If we would follow the same structure than PyTorch, then we could (more easily) use the same weight as the PyTorch pretrained model. I have made a test with the PyTorch's ResNet50 and a custom Flux' ResNet50 model by reading all the parameters using the PyTorch model state_dict() method and this approach works quite well and I get exactly the same results from PyTorch than from Flux.