mmlind/mnist-dnn

Deep neural network for handling MNIST image recognition

mnist-dnn's Issues

How to add an Pooling layer?

I'm trying to learn how to implement DNN in C. I took LeNet as an example. It contains pooling layer for max pooling. How do I add pooling layer with your existing code? Below is the prototxt

name: "LeNet"
layer {
  name: "data"
  type: "Input"
  top: "data"
  input_param { 
  shape: { 
      dim :10 
      dim : 1 
      dim : 28 
      dim : 28 
   } 
  }
  transform_param {
    scale: 0.00390625
  }
}
layer {
  name: "conv1"
  type: "Convolution"
  bottom: "data"
  top: "conv1"
  convolution_param {
    num_output: 20
    kernel_size: 5
    stride: 1
  }
}
layer {
  name: "pool1"
  type: "Pooling"
  bottom: "conv1"
  top: "pool1"
  pooling_param {
    pool: MAX
    kernel_size: 2
    stride: 2
  }
}
layer {
  name: "conv2"
  type: "Convolution"
  bottom: "pool1"
  top: "conv2"
  convolution_param {
    num_output: 50
    kernel_size: 5
    stride: 1
  }
}
layer {
  name: "pool2"
  type: "Pooling"
  bottom: "conv2"
  top: "pool2"
  pooling_param {
    pool: MAX
    kernel_size: 2
    stride: 2
  }
}
layer {
  name: "ip1"
  type: "InnerProduct"
  bottom: "pool2"
  top: "ip1"
  inner_product_param {
    num_output: 500
  }
}
layer {
  name: "relu1"
  type: "ReLU"
  bottom: "ip1"
  top: "ip1"
}
layer {
  name: "ip2"
  type: "InnerProduct"
  bottom: "ip1"
  top: "ip2"
  inner_product_param {
    num_output: 10
  }
}
layer {
  name: "prob"
  type: "Softmax"
  bottom: "ip2"
  top: "prob"
}
layer {
  name: "sigx"
  bottom: "ip2"
  top: "sigx"
  type: "Sigmoid"
}

How to get 88.40% accuracy with ConvNet network?

Hi, I have a question but, dont know how to contact you, so posted the issue here.

Firstly, I really appreciate your work as I'm studying neural network, and try to implement in C.
it's really helpful to study by your blog and code, thankyou.

My question is how did you construct your network to get high performance by using convolution network, like you showed on the blog (like below).

==========================================================
| Network | # of Nodes | Act.Fct | L-Rate | # Trains | Speed* | Accuracy || Compare** |
| ----------- |----------------|---------|--------|----------|--------|----------||-----------|
| 1-l FC NN | 10 output | SIGMOID | 0.0125 | 900,000 | 228s | 91.09% || 88.00% |
| 2-l FC NN | 300 hidden | SIGMOID | 0.0700 | 120,000 | 697s | 94.67% || 96.40% |
| 3-l FC NN | 500+150 hidden | SIGMOID | 0.0050 | 180,000 | 2,510s | 93.64% || 97.05% |
| 3-l ConvNet | 13x13x5, 6x6x5 | RELU | 0.0004 | 525,000 | 908s | 88.40% || 98.90% |

==========================================================

When I tried the same 13x13x5, 6x6x5 convolutional network, but keep getting 9.87%.

I cloned your code and simply run as your default main.c has 13x13x5, 6x6x5 convolutional network.

and one more thing,
when try to use relu as activation function, it also do not work, keep getting around 9% accuracy
for just simply 1-hidden layer with relu neurons, I'm keep varifing you code but dont get why.

Can you please help me with usage of your convolutional network code?