Using AutoGrad with CuArray

[ngphuoc]

Using AutoGrad with CuArray

[ngphuoc]

I have checked that AutoGrad can work with ArrayFire: JuliaGPU/ArrayFire.jl#183

PS: if my benchmark was correct then KnetArray is about 3-4 times faster.

[CarloLucibello]

autograd gradient is working now (julia 0.7 and CuArrays master) but it is of the wrong type (standard array)

julia> using Knet,CuArrays, Statistics

julia> x,y = rand(13,10), randn(1,10)
([0.774427 0.275647 … 0.452615 0.09355; 0.411376 0.765637 … 0.527263 0.839156; … ; 0.650629 0.404082 … 0.520125 0.279594; 0.743382 0.192372 … 0.756798 0.457104], [-0.816243 -0.606755 … 1.3648 -0.116751])

julia> w = Any[ 0.1f0*cu(randn(Float32,1,13)), 0.0f0 ]
2-element Array{Any,1}:
  Float32[-0.092651 -0.0714096 … -0.102493 0.080609]
 0.0f0                                              

julia> w[1] |> typeof
CuArray{Float32,2}

julia> predict(w,x) = w[1]*x .+ w[2]
predict (generic function with 1 method)

julia> loss(w,x,y) = mean(abs2, y.- predict(w,x))
loss (generic function with 1 method)

julia> loss(w,x,y)
0.7325428950542855

julia> grad(loss)(w,x,y)  
2-element Array{Any,1}:
  [0.215699 0.272719 … 0.196543 0.123026]
 0.4017888307995154                      

julia> grad(loss)(w,x,y)[1] |> typeof
Array{Float64,2}