yrlu/teaism

A full-fledged yet minimalistic CUDA-based convolutional neural network library from scratch in C++

teaism's Issues

Memory allocation scheme

When and how to allocate GPU/CPU memory?

Global weights update

Global Tensors' garbage collection

put a reference of Tensors into a list in the global session whenever Tensor::CreateTensorCPU/GPU are called.

call session->free_tensors() to free all the memory allocated to the Tensors and set references to NULLs.

Update Const Initializer

Sessons and Graphs

Discuss and decide if to implement sessions and graphs

Save and load network models

Select a Unit Test Framework

Discuss and select a Unit Test Framework

Discuss and select a unit test framework

Discuss and select a unit test framework

Implement parameters importer

Backprop

Copying DataArray from GPU to CPU

SGD Optimizer

Implement ReLU layer

Creating Tensor GPU should also copy size_t [] dims_ from CPU to GPU

Current implementation seems buggy. Should look into it.

Build Scripts: CMakelists

Build Scripts: CMakelists

Activation layer tanh

Implement fully connected layer

Unified Memory

Might be of interest?

https://devblogs.nvidia.com/parallelforall/unified-memory-in-cuda-6/
http://stackoverflow.com/questions/21986542/is-cudamallocmanaged-slower-than-cudamalloc

Implement losses

Setup project directory structure

Setup project directory structure

Working Conv2D Layer (Gaussian blur demo)

Working Conv2D Layer (Gaussian blur demo)

Implement Softmax layer

Paddings (zero/reflection)

Clean up Tensor GPU pointer

Basic data structures

Discuss, design and implement basic data structures:

Tensor
Packet
Nodes(?)

Time per layer forward/backward

Implement data layer

Test CNN BP on cifar dataset

Time per training iteration

Parameters Initialization

Discuss and finalize the parameter initialization design: When and how to initialize the model parameters?

Gaussian Blur Bug

CMakelists

CMakelists

Improve data layer by prefetching

Two things to do:

1. Prefetch as many data as possible in CPU memory.
2. Launch another CPU thread to prefetch next-batch data in the background.

Shared activation diff

To implement a shared data holder for gradients of activations for efficient GPU memory usage.