mikhailkravets/neuroflow

Awesome deep learning crate

Hello

Welcome to my GitHub space.

---

My Publications

Building BERT with PyTorch from scratch

Add inline PlantUML to your MkDocs documentation

Build and Train Vision Transformer from Scratch

Level Up Your NestJS Testing by Automating Test Database Creation

neuroflow's Issues

Does Relu not work?

So I was playing around the with example script for training a basic neural network that uses a Tanh activation function but I saw in the git repo, the implementation of the relu in activators. I wanted to try out Relu on the problem but it threw this error:

error[E0432]: unresolved import `neuroflow::activators::Type::Relu`
 --> src\main.rs:3:5
  |
3 | use neuroflow::activators::Type::Relu;
  |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ no `Relu` in `activators::Type`

For more information about this error, try `rustc --explain E0432`.
error: could not compile `hello_cargo` (bin "hello_cargo") due to previous error

example I used: here

panic: index out of bounds

thread 'main' panicked at 'index out of bounds: the len is 1 but the index is 1', /home/moth/.cargo/registry/src/github.com-1ecc6299db9ec823/neuroflow-0.1.3/src/lib.rs:356:48
stack backtrace:
   0: rust_begin_unwind
             at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/panicking.rs:584:5
   1: core::panicking::panic_fmt
             at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/panicking.rs:143:14
   2: core::panicking::panic_bounds_check
             at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/panicking.rs:85:5
   3: <usize as core::slice::index::SliceIndex<[T]>>::index
             at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/slice/index.rs:189:10
   4: core::slice::index::<impl core::ops::index::Index<I> for [T]>::index
             at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/slice/index.rs:15:9
   5: <alloc::vec::Vec<T,A> as core::ops::index::Index<I>>::index
             at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/alloc/src/vec/mod.rs:2531:9
   6: neuroflow::FeedForward::backward
             at /home/moth/.cargo/registry/src/github.com-1ecc6299db9ec823/neuroflow-0.1.3/src/lib.rs:356:48
   7: neuroflow::FeedForward::fit
             at /home/moth/.cargo/registry/src/github.com-1ecc6299db9ec823/neuroflow-0.1.3/src/lib.rs:463:9
   8: neuroflow::FeedForward::train
             at /home/moth/.cargo/registry/src/github.com-1ecc6299db9ec823/neuroflow-0.1.3/src/lib.rs:439:13
   9: neuralnetrust::main
             at ./src/main.rs:41:5
  10: core::ops::function::FnOnce::call_once
             at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
The terminal process "cargo 'run', '--package', 'neuralnetrust', '--bin', 'neuralnetrust'" terminated with exit code: 101.

when using the lib, I get this error

help with iteration argument

Hi, I was wondering if you could explain the iterations argument and the solver. Is the iterations, the number of times the dataset is sampled with an individual row, or is iterations the number of pass through the whole data set? So if the dataset has 10_000 rows, iterations = 10_000 would imply the dataset would be sampled 10000 times with batchsize = 1 row?

From what I understood in the code, the rand() call in train() only selected one row per iteration.

Also I wanted to confirm that the error is cumulative for all iterations, especially if you call train() multiple times in a loop.

Thanks

Support for Stochastic Gradient Descent and Minibatches

I see your code is running through the whole dataset for each training iteration. For many applications, it is quicker to split them up in random smaller batches and run gradient descent on each "mini batch" (see https://en.wikipedia.org/wiki/Stochastic_gradient_descent): it may take more iterations to converge but each iteration becomes much quicker.

How high is this features in your priority list?

example not compiles(((

error[E0277]: the trait bound `Result<DataSet, Box<dyn std::error::Error>>: Extractable` is not satisfied

 |     .train(&data, 20_000);
 |            ^^^^^ the trait `Extractable` is not implemented for `Result<DataSet, Box<dyn std::error::Error>>`
...

rust:
stable-x86_64-unknown-linux-gnu installed - rustc 1.52.1 (9bc8c42bb 2021-05-09)