Light

skyer19 / digit_recognizer_for_mnist Goto Github PK

View Code? Open in Web Editor NEW

0.0 1.0 0.0 303 KB

2020年fzu西二在线python第三轮考核

License: GNU Affero General Public License v3.0

Jupyter Notebook 59.22% Python 40.78%

digit_recognizer_for_mnist's Introduction

Digit_Recognizer_For_MNIST

2020年fzu西二在线python三轮考核

目前准确率: 97.307%(02/8/2021)

本项目操作方法见operation.md

相关概念

正向传播：在神经网络中，我们使用权重，将信号从输入向前传播到输出层
反向传播：在神经网络中，我们使用权重，将误差从输出向后传播到网络中

矩阵的运用

$X = W \cdot I$

这里的W是权重矩阵(比如说W[1,2]表示第一层第一个节点与第二层第二个节点之间的权重)，I是输入矩阵，X是组合调节后的信号，即输入到第二层的结果矩阵
通过激活函数，来自第二层的最终输出为:

这里我们用三层神经网络(输入层，一层隐藏层，输出层)来举例

正向传递的过程

$X{_{hidden}}= W{_{input_hiddden}}\cdot I$

$O{_{hidden}} = sigmoid(X)$

$X{_{output}}= W{_{hidden_output}}\cdot O{_{hidden}}$

$O{_{output}} = sigmoid(X{_{output}})$

反向传递

反向传播误差

$error_{hidden}=W_{hidden_output}^{T}\cdot error_{output}$

更新权重

以梯度下降的方式，即计算出误差函数相当于权重的斜率，这里采用的误差函数是 $\left ( target-actual \right )^{2}$
目标值
实际值

以隐藏层与输出层之间的链接权重为例：

$\frac{\partial E}{\partial W_{i,j}} =\frac{\partial }{\partial W_{i,j}}\left ( t_{k} - o_{k} \right )^{2}$

$=\frac{\partial E}{\partial O_{k}}\cdot \frac{\partial O_{k}}{\partial W_{j,k}}$

$=-2\left ( t_{k} -o_{k} \right )\cdot \frac{\partial O_{k}}{\partial W_{j,k}}$

$=-2\left ( t_{k} -o_{k} \right )\cdot \frac{\partial }{\partial W_{j,k}}\cdot sigmoid\left ( \sum _{j}W_{j,k} \cdot O_{j}\right )$

又根据求导法则： $\frac{\partial }{\partial x}sigmoid \left ( x \right )=sigmoid \left ( x \right )\left ( 1-sigmoid \left ( x \right ) \right )$

最终结果： $\frac{\partial E}{\partial W_{i,j}} =-\left ( t_{k}-o_{k} \right )\cdot sigmoid\left ( \sum _{j}W_{j,k}\cdot O_{j}\right)\left ( 1-sigmoid\left ( \sum _{j} W_{j,k}\cdot O_{j}\right ) \right )O_{j}$

同理，输入层与隐藏层之间的链接权重为：

$\frac{\partial E}{\partial W_{i,j}} =-\left ( e_{j} \right )\cdot sigmoid\left ( \sum _{i}W_{i,j}\cdot O_{i} \right )\left( 1-sigmoid\left ( \sum _{i}W_{i,j}\cdot O_{i} \right ) \right ) O_{i}$

更新权重 : $new W_{j,k}=old W_{j,k} -\alpha \cdot \frac{\partial E}{\partial W_{j,k}}$

$\partial =learning\_rate$ 学习率☺️

注意⚠️初始权重可以随机选取，但要尽可能的小，但不能为零

digit_recognizer_for_mnist's People

Contributors

Watchers

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.