graident didn't work

[markusai]

graident didn't work

[Dongyeongkim]

The problem is when we try to use grad for optimizing weights, it has different point.
for grad to get another grad-> just through it
for grad to optimize -> it is quite different. need to get delta W for it. delta W is equal as deltaEXl(t)/deltaWgrad

[Dongyeongkim]

[jepetolee]

LInear layer was well-worked, but after pooling layers, it did'nt works. So, I'll try to fix this error

[Dongyeongkim]

there has a problem in linear layer too. need to review

[Dongyeongkim]

ValueError: Incompatible shapes for broadcasting: (300, 784) and requested shape (64, 784)

might be the problem of transpose

[Dongyeongkim]

loss.backward() didn't work due to dimension error

[Dongyeongkim]

it is resolved

but need to test gradient

[Dongyeongkim]

might be the problem of encoding or somehow?

[Dongyeongkim]

It didn't work in fully-connected and xcnn_mnist, just swing away in range +- 1.x

[Dongyeongkim]

Engine has problem. When you see engine in output, you need to elementwise multifiplying Time range(T in paper) but you matrix multiplicating grad so it went grad square indeed. need to change engine

[Dongyeongkim]

#100 detailed-description

[Dongyeongkim]

Not really. the real problem is the position of grad and gamma, matrix multiplication has directional factor so @jepetolee and @Dongyeongkim will fix it

[Dongyeongkim]

#101 has been tested. and the solution is confirmed. this case is signed a die

[Dongyeongkim]

the case which is opened: Convolution and pooling backward

[Dongyeongkim]

need to test #102

[Dongyeongkim]

mini-Batch algorithm need to be understood, it is likely to get an average of error gradient

[Dongyeongkim]

Well the states of neurons which is based on batch data is different.

ex) 1,2,3 -> have different membrane potentials because data is different

grad is not equal, weight is equal, LIF' is not equal.

so the grad has different in regard to an element of batch.

[Dongyeongkim]

need test but not now