Giter Site home page Giter Site logo

mnist-from-scratch's People

Contributors

markkraay avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mnist-from-scratch's Issues

Is the multiplication implemented properly?

When you use matrix_multiply how it's implemented in this project the result is:
image

However, when you use a matrix calculator or multiply by hand the result is:
image

Is this supposed to be like that? Or is that just a small mistake?

Some Mods

Fantastic YT video! It was very helpful. I just did a little experimenting with your code.

  1. Get it working in Microsoft Visual Studio C++.
  2. It gave very poor results, till a suggestion in your responses mentioned needing to convert malloc to calloc to assure zero'd arrays. That fixed my problem. Went from 0.3% to 90%+.
  3. I'm working on versions to go on a microprocessor (ESP32/ESP8266). May even do a multi-threaded one to use dual cores. Anyway, I needed to get rid of the huge data reads and converted your code to at least remove the input side for training. I have it training on the larger 60,000 row mnist_train.csv by reading one line at a time.
#include <stdlib.h>
#include <string.h>
#include <time.h>
#include "matrix.h"
#include "ops.h"
#include "img.h"
#include "activations.h"
#include "nn.h"

#define MAXCHAR 10000

int main()
{
    printf("STARTED...\n");
    clock_t tStart = clock();
    srand(3);

    // Change process to minimize memory allocation... 
    // Thoughts of putting on MPU.
    
    // Process one image at a time out of the "train" file.
    int cnt = 0;
    char row[MAXCHAR];
    FILE* fp = fopen("data/mnist_train.csv", "r");
    NeuralNetwork* net = network_create(784, 300, 10, 0.1);

    // Read the first line - Header
	fgets(row, MAXCHAR, fp);
    while (fgets(row, MAXCHAR, fp))
    {
        cnt++;
        Img* img = (Img*)calloc(1, sizeof(Img));
        int j = 0;

        // Get image data        
        char* token = strtok(row, ",");
		img->img_data = matrix_create(28, 28);
		while (token != NULL) 
        {
			if (j == 0) 
				img->label = atoi(token);
			else
				img->img_data->entries[(j-1) / 28][(j-1) % 28] = 
                    atoi(token) / 256.0;
			token = strtok(NULL, ",");
			j++;
		}
        
        // Train image
        if (cnt % 100 == 0) printf(".");
		Matrix* img_data = matrix_flatten(img->img_data, 0); // 0 = flatten to column vector
		Matrix* output = matrix_create(10, 1);
		output->entries[img->label][0] = 1;     // Setting the result
		network_train(net, img_data, output);
		matrix_free(output);
		matrix_free(img_data);

        // Predict after every 1000 trained
        if (cnt % 1000 == 0)
        {
            clock_t t = clock();
            printf("Trained %d in %f sec", cnt, (double)(t - tStart)/CLOCKS_PER_SEC);
	        int number_imgs = 10000;    // mnist_test.csv has 10,000 samples
	        Img** imgs = csv_to_imgs("data/mnist_test.csv", number_imgs);
	        double score = network_predict_imgs(net, imgs, number_imgs);
	        printf(" score: %1.5f in %f sec\n", score, (double)(clock() - t)/CLOCKS_PER_SEC);
            imgs_free(imgs, number_imgs);
        }
        img_free(img);  // Also frees internal matrix.
    }
    network_free(net);
    getchar();
    return 0;
}

I also wanted to see the convergence and have it doing the full 10,000 items in the mnist_test.csv file. Here are the results. It is interesting that it converges pretty quickly, but then starts oscillating as it slowly continues to converge.

..........Trained 1000 in 95.693000 sec score: 0.77460 in 224.097000 sec
..........Trained 2000 in 422.487000 sec score: 0.87330 in 224.211000 sec
..........Trained 3000 in 742.162000 sec score: 0.89050 in 224.068000 sec
..........Trained 4000 in 1057.758000 sec score: 0.88830 in 223.550000 sec
..........Trained 5000 in 1383.860000 sec score: 0.90450 in 228.994000 sec
..........Trained 6000 in 1704.406000 sec score: 0.90650 in 224.825000 sec
..........Trained 7000 in 2027.219000 sec score: 0.91010 in 224.461000 sec
..........Trained 8000 in 2343.858000 sec score: 0.90440 in 224.118000 sec
..........Trained 9000 in 2665.948000 sec score: 0.91470 in 223.544000 sec
..........Trained 10000 in 2977.347000 sec score: 0.91830 in 222.331000 sec
..........Trained 11000 in 3297.136000 sec score: 0.90810 in 223.352000 sec
..........Trained 12000 in 3610.186000 sec score: 0.91910 in 224.895000 sec
..........Trained 13000 in 3931.188000 sec score: 0.92140 in 223.995000 sec
..........Trained 14000 in 4251.436000 sec score: 0.91040 in 224.078000 sec
..........Trained 15000 in 4573.185000 sec score: 0.93010 in 224.855000 sec
..........Trained 16000 in 4894.231000 sec score: 0.93330 in 224.952000 sec
..........Trained 17000 in 5206.893000 sec score: 0.93150 in 223.884000 sec
..........Trained 18000 in 5522.631000 sec score: 0.93030 in 223.146000 sec
..........Trained 19000 in 5840.735000 sec score: 0.93920 in 223.543000 sec
..........Trained 20000 in 6158.867000 sec score: 0.93620 in 225.025000 sec
..........Trained 21000 in 6478.698000 sec score: 0.93620 in 223.282000 sec
..........Trained 22000 in 6794.564000 sec score: 0.93950 in 223.005000 sec
..........Trained 23000 in 7122.286000 sec score: 0.94220 in 223.723000 sec
..........Trained 24000 in 7435.284000 sec score: 0.94780 in 223.621000 sec
..........Trained 25000 in 7759.393000 sec score: 0.94200 in 223.376000 sec
..........Trained 26000 in 8079.515000 sec score: 0.94040 in 223.514000 sec
..........Trained 27000 in 8398.266000 sec score: 0.94750 in 223.871000 sec
..........Trained 28000 in 8719.247000 sec score: 0.94370 in 223.606000 sec
..........Trained 29000 in 9043.682000 sec score: 0.94570 in 223.529000 sec
..........Trained 30000 in 9356.831000 sec score: 0.94120 in 231.249000 sec
..........Trained 31000 in 9685.170000 sec score: 0.94530 in 223.141000 sec
..........Trained 32000 in 10011.075000 sec score: 0.94610 in 224.848000 sec
..........Trained 33000 in 10333.140000 sec score: 0.95280 in 225.202000 sec
..........Trained 34000 in 10669.506000 sec score: 0.94950 in 226.308000 sec
..........Trained 35000 in 10998.972000 sec score: 0.94820 in 229.636000 sec
..........Trained 36000 in 11331.394000 sec score: 0.94370 in 227.744000 sec
..........Trained 37000 in 11656.317000 sec score: 0.94480 in 242.930000 sec
..........Trained 38000 in 11997.447000 sec score: 0.95220 in 235.433000 sec
..........Trained 39000 in 12336.421000 sec score: 0.95200 in 228.207000 sec
..........Trained 40000 in 12683.384000 sec score: 0.94620 in 243.553000 sec
..........Trained 41000 in 13025.438000 sec score: 0.95110 in 242.420000 sec
..........Trained 42000 in 13368.053000 sec score: 0.95510 in 233.444000 sec
..........Trained 43000 in 13705.663000 sec score: 0.95620 in 227.866000 sec
..........Trained 44000 in 14030.657000 sec score: 0.95390 in 230.856000 sec
..........Trained 45000 in 14352.970000 sec score: 0.95240 in 226.875000 sec
..........Trained 46000 in 14680.714000 sec score: 0.94890 in 233.112000 sec
..........Trained 47000 in 15007.209000 sec score: 0.95680 in 233.219000 sec
..........Trained 48000 in 15336.794000 sec score: 0.95870 in 229.185000 sec
..........Trained 49000 in 15667.930000 sec score: 0.94940 in 279.895000 sec
..........Trained 50000 in 16064.643000 sec score: 0.96070 in 209.766000 sec
..........Trained 51000 in 16385.168000 sec score: 0.96010 in 267.372000 sec
..........Trained 52000 in 16763.328000 sec score: 0.96260 in 240.102000 sec
..........Trained 53000 in 17095.186000 sec score: 0.95860 in 226.821000 sec
..........Trained 54000 in 17417.910000 sec score: 0.95870 in 224.317000 sec
..........Trained 55000 in 17733.573000 sec score: 0.96090 in 223.759000 sec
..........Trained 56000 in 18049.304000 sec score: 0.96030 in 223.131000 sec
..........Trained 57000 in 18364.107000 sec score: 0.96070 in 224.214000 sec
..........Trained 58000 in 18680.472000 sec score: 0.95490 in 224.037000 sec
..........Trained 59000 in 18997.095000 sec score: 0.95790 in 224.001000 sec
..........Trained 60000 in 19311.704000 sec score: 0.96000 in 223.618000 sec

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.