xiaohu2015 / deeplearning_tutorials Goto Github PK

View Code? Open in Web Editor NEW

1.8K 1.8K 768.0 11.74 MB

The deeplearning algorithms implemented by tensorflow

Python 17.85% Jupyter Notebook 82.15%

cnn dbn deep-learning-algorithms deep-learning-tutorial lstm mlp rbm sda tensorflow

deeplearning_tutorials's Introduction

Deep Learning Tutorials with Tensorflow

The deeplearning algorithms are carefully implemented by tensorflow.

Environment

Python 3.5
tensorflow 1.4
pytorch 0.2.0

The deeplearning algorithms includes (now):

Logistic Regression logisticRegression.py
Multi-Layer Perceptron (MLP) mlp.py
Convolution Neural Network (CNN) cnn.py
Denoising Aotoencoder (DA) da.py
Stacked Denoising Autoencoder (SDA) sda.py
Restricted Boltzmann Machine (RBM) [rbm.py gbrbm.py]
Deep Belief Network (DBN) dbn.py

Note: the project aims at imitating the well-implemented algorithms in Deep Learning Tutorials (coded by Theano).

CNN Models

MobileNet [self paper ref]
MobileNetv2 [self paper ref]
SqueezeNet [self paper]
ResNet [self caffe ref paper1 paper2]
ShuffleNet [self by pytorch paper]
ShuffleNetv2 [self ref paper]
DenseNet [self pytorch_ref paper]

Object detection

YOLOv1 [self paper ref]
SSD [self paper slides cafe TF pytorch ]
YOLOv2 [self paper ref]

Practical examples

You can find more practical examples with tensorflow here:

CNN for setence classification [self] [blog] [paper]
RNN for language model [self] [blog] [blog_cn]
LSTM for language model (PTB data) [self] [tutorial] [paper]
VGG model for image classification (object recongnition) [self] [source]
Residual network for cifar10_dataset [self] [source] [paper]
LSTM for time series prediction [self] [source]
Generative adversarial network (GAN) [self]
Variational autoencoder (VAE) [self]

Results

Fun Blogs

Chatbots with Seq2Seq

Personal Notes

Tensorflow for RNNs [tf_rnn.ipynb]
Tensorflow for Autoencoder [tf_autoencoder.ipynb]

If you benefit from the tutorial, please make a small donation by WeChat sweep.

微信号：xiaoxiaohu1994

欢迎关注微信公众号：机器学习算法全栈工程师(Jeemy110)

deeplearning_tutorials's People

Contributors

Stargazers

Watchers

Forkers

zhangxujinsh aliushn aitechnology algpower liuminmin1202 flying0er bikong2 zhuguangqiang hitum-dev johnson-yue keithyin niuzehai lihao0214 xiangyongcao hellowst giantstonex7 parety styanddty cristidruta gcwk snowman828 qingaidexin yuanxianfeng huangyankun khoink94 saadmahboob andyliu93 notlaughinggirl hunkyu bibongo qinyan328 tracy-king iron-fe zbn123 callingwisdom 601madman 17112 zhiqiang-zhu tanbendong ichdream okcing vishnuravi3186 makeling tony32769 qiujkx roy-algoritm lji72 zzyyxxal syx2zju graphic9 bayalla leonlzj alex891 quliuwuyihmy jsmilemsj zhx-main shubhampachori12110095 deeplearningzhy deelmind eternalshallow stevewsw venus024 sawyer7246 zhaozhengcoder blackjaxx chaconez shileone haoxinl timedcy quxiaofeng shuxjweb yyb1995 gbusr coderdj01 rosefun qingwang12138 jackson08 ooooowenxu chuangdang zhongzhengyu hunnudl dltensor leichangqing sjl421 hxy201801 roachsinai ewenwan biu-chuizi michaelodum l02i08u barbecacov huangeryu bailu0713 yky090703 migossss chenmo874300 stoooner jeankeim huaqiaouniversity piaopiaodedudou

deeplearning_tutorials's Issues

Does your yolov2 have no training？

RBM visible unit type--gaussin

Hi! I' m reading this project and I find you only implement binary rbm, any idea about GBRBM?

肺结节分类

您好！我想问您，能不能用28X28大小的灰度肺结节图像代替mnist数据集，跑您的代码呢？

Where can I find weights_file?

if name == "main":
yolo_net = Yolo("./weights/YOLO_small.ckpt")
yolo_net.detect_from_file("./test/car.jpg")

GBRBM is not useful in pretraining considering the result.

When I use GBRBM with continuous data ,I find that the performance of GBRBM without pretraining is the same as it with pretraining.

The model has one hidden layer. Input=[None,52] hidden=[100,], output=[5]

When using pretraining , the cost is as follow:

tarting pretraining...

        Pretraing layer 0 Epoch 0 cost: 260.81867837905884
        Pretraing layer 0 Epoch 1 cost: 152.01383924484253
        Pretraing layer 0 Epoch 2 cost: 127.39883494377136
        Pretraing layer 0 Epoch 3 cost: 112.40048933029175
        Pretraing layer 0 Epoch 4 cost: 100.92978549003601
        Pretraing layer 0 Epoch 5 cost: 85.16272974014282
        Pretraing layer 0 Epoch 6 cost: 77.07024431228638
        Pretraing layer 0 Epoch 7 cost: 68.50090146064758
        Pretraing layer 0 Epoch 8 cost: 58.285604596138
        Pretraing layer 0 Epoch 9 cost: 55.684627294540405
        Pretraing layer 0 Epoch 10 cost: 48.3922621011734
        Pretraing layer 0 Epoch 11 cost: 41.95375835895538
        Pretraing layer 0 Epoch 12 cost: 37.9025719165802
        Pretraing layer 0 Epoch 13 cost: 33.71773016452789
        Pretraing layer 0 Epoch 14 cost: 30.611300826072693
        Pretraing layer 0 Epoch 15 cost: 26.8545463681221
        Pretraing layer 0 Epoch 16 cost: 25.058630347251892
        Pretraing layer 0 Epoch 17 cost: 22.55670392513275
        Pretraing layer 0 Epoch 18 cost: 20.109949052333832
        Pretraing layer 0 Epoch 19 cost: 18.671690464019775
        Pretraing layer 0 Epoch 20 cost: 17.622432470321655
        Pretraing layer 0 Epoch 21 cost: 15.93561115860939
        Pretraing layer 0 Epoch 22 cost: 13.794037371873856
        Pretraing layer 0 Epoch 23 cost: 12.636497765779495
        Pretraing layer 0 Epoch 24 cost: 11.682598501443863
        Pretraing layer 0 Epoch 25 cost: 10.461727529764175
        Pretraing layer 0 Epoch 26 cost: 9.38357800245285
        Pretraing layer 0 Epoch 27 cost: 9.149504691362381
        Pretraing layer 0 Epoch 28 cost: 7.691398337483406
        Pretraing layer 0 Epoch 29 cost: 8.177620634436607
        Pretraing layer 0 Epoch 30 cost: 8.684246331453323
        Pretraing layer 0 Epoch 31 cost: 6.862771183252335
        Pretraing layer 0 Epoch 32 cost: 5.897482261061668
        Pretraing layer 0 Epoch 33 cost: 5.523693576455116
        Pretraing layer 0 Epoch 34 cost: 5.841872453689575
        Pretraing layer 0 Epoch 35 cost: 6.781636029481888
        Pretraing layer 0 Epoch 36 cost: 5.291902661323547
        Pretraing layer 0 Epoch 37 cost: 5.066544055938721
        Pretraing layer 0 Epoch 38 cost: 5.890164598822594
        Pretraing layer 0 Epoch 39 cost: 7.395811066031456
        Pretraing layer 0 Epoch 40 cost: 5.971683979034424
        Pretraing layer 0 Epoch 41 cost: 7.291851326823235
        Pretraing layer 0 Epoch 42 cost: 6.261465802788734
        Pretraing layer 0 Epoch 43 cost: 5.761327639222145
        Pretraing layer 0 Epoch 44 cost: 7.164529412984848
        Pretraing layer 0 Epoch 45 cost: 6.151705488562584
        Pretraing layer 0 Epoch 46 cost: 5.703673854470253
        Pretraing layer 0 Epoch 47 cost: 5.60428760945797
        Pretraing layer 0 Epoch 48 cost: 5.398236528038979
        Pretraing layer 0 Epoch 49 cost: 6.732670620083809
        Pretraing layer 0 Epoch 50 cost: 3.3295760229229927
        Pretraing layer 0 Epoch 51 cost: 6.440319612622261
        Pretraing layer 0 Epoch 52 cost: 7.721085608005524
        Pretraing layer 0 Epoch 53 cost: 7.5792941600084305
        Pretraing layer 0 Epoch 54 cost: 6.913594916462898
        Pretraing layer 0 Epoch 55 cost: 5.567243799567223
        Pretraing layer 0 Epoch 56 cost: 8.98297393321991
        Pretraing layer 0 Epoch 57 cost: 6.2041324228048325
        Pretraing layer 0 Epoch 58 cost: 7.210628360509872
        Pretraing layer 0 Epoch 59 cost: 7.620965078473091
        Pretraing layer 0 Epoch 60 cost: 7.292994022369385
        Pretraing layer 0 Epoch 61 cost: 7.524974316358566
        Pretraing layer 0 Epoch 62 cost: 8.054834097623825
        Pretraing layer 0 Epoch 63 cost: 8.462488144636154
        Pretraing layer 0 Epoch 64 cost: 12.607556283473969
        Pretraing layer 0 Epoch 65 cost: 9.048019409179688
        Pretraing layer 0 Epoch 66 cost: 11.15686920285225
        Pretraing layer 0 Epoch 67 cost: 9.296072781085968
        Pretraing layer 0 Epoch 68 cost: 9.096844345331192
        Pretraing layer 0 Epoch 69 cost: 10.515856593847275
        Pretraing layer 0 Epoch 70 cost: 13.93978425860405
        Pretraing layer 0 Epoch 71 cost: 11.779045760631561
        Pretraing layer 0 Epoch 72 cost: 8.926943451166153
        Pretraing layer 0 Epoch 73 cost: 9.065500438213348
        Pretraing layer 0 Epoch 74 cost: 11.144863158464432
        Pretraing layer 0 Epoch 75 cost: 10.3612519800663
        Pretraing layer 0 Epoch 76 cost: 13.402012079954147
        Pretraing layer 0 Epoch 77 cost: 10.36564788222313
        Pretraing layer 0 Epoch 78 cost: 8.519263058900833
        Pretraing layer 0 Epoch 79 cost: 10.797200798988342
        Pretraing layer 0 Epoch 80 cost: 11.911772519350052
        Pretraing layer 0 Epoch 81 cost: 10.96516764163971
        Pretraing layer 0 Epoch 82 cost: 11.286803781986237
        Pretraing layer 0 Epoch 83 cost: 11.852731883525848
        Pretraing layer 0 Epoch 84 cost: 12.5465706884861
        Pretraing layer 0 Epoch 85 cost: 9.86420676112175
        Pretraing layer 0 Epoch 86 cost: 14.13658493757248
        Pretraing layer 0 Epoch 87 cost: 13.595908015966415
        Pretraing layer 0 Epoch 88 cost: 14.341294139623642
        Pretraing layer 0 Epoch 89 cost: 16.262001037597656
        Pretraing layer 0 Epoch 90 cost: 13.679453998804092
        Pretraing layer 0 Epoch 91 cost: 12.280019283294678
        Pretraing layer 0 Epoch 92 cost: 14.773049354553223
        Pretraing layer 0 Epoch 93 cost: 13.022569745779037
        Pretraing layer 0 Epoch 94 cost: 15.778692752122879
        Pretraing layer 0 Epoch 95 cost: 14.72546523809433
        Pretraing layer 0 Epoch 96 cost: 14.121755450963974
        Pretraing layer 0 Epoch 97 cost: 15.314739406108856
        Pretraing layer 0 Epoch 98 cost: 15.002620220184326
        Pretraing layer 0 Epoch 99 cost: 11.205422580242157
        Pretraing layer 0 Epoch 100 cost: 14.114984184503555
        Pretraing layer 0 Epoch 101 cost: 14.444316059350967
        Pretraing layer 0 Epoch 102 cost: 15.220130860805511
        Pretraing layer 0 Epoch 103 cost: 14.989370942115784
        Pretraing layer 0 Epoch 104 cost: 14.847556620836258
        Pretraing layer 0 Epoch 105 cost: 13.437550753355026
        Pretraing layer 0 Epoch 106 cost: 13.943797200918198
        Pretraing layer 0 Epoch 107 cost: 16.37880104780197
        Pretraing layer 0 Epoch 108 cost: 16.04702916741371
        Pretraing layer 0 Epoch 109 cost: 16.412404358386993
        Pretraing layer 0 Epoch 110 cost: 14.767323642969131
        Pretraing layer 0 Epoch 111 cost: 15.578109204769135
        Pretraing layer 0 Epoch 112 cost: 15.451245158910751
        Pretraing layer 0 Epoch 113 cost: 14.151467323303223
        Pretraing layer 0 Epoch 114 cost: 19.20454603433609
        Pretraing layer 0 Epoch 115 cost: 16.46894198656082
        Pretraing layer 0 Epoch 116 cost: 15.836542934179306
        Pretraing layer 0 Epoch 117 cost: 14.759043753147125
        Pretraing layer 0 Epoch 118 cost: 16.109388828277588
        Pretraing layer 0 Epoch 119 cost: 17.626535832881927
        Pretraing layer 0 Epoch 120 cost: 17.100443065166473
        Pretraing layer 0 Epoch 121 cost: 16.598965406417847
        Pretraing layer 0 Epoch 122 cost: 17.256347060203552
        Pretraing layer 0 Epoch 123 cost: 16.284021019935608
        Pretraing layer 0 Epoch 124 cost: 18.254932701587677
        Pretraing layer 0 Epoch 125 cost: 16.10324251651764
        Pretraing layer 0 Epoch 126 cost: 17.299164533615112
        Pretraing layer 0 Epoch 127 cost: 17.293877601623535
        Pretraing layer 0 Epoch 128 cost: 15.657768249511719
        Pretraing layer 0 Epoch 129 cost: 19.645508229732513
        Pretraing layer 0 Epoch 130 cost: 17.034938156604767
        Pretraing layer 0 Epoch 131 cost: 17.83082640171051
        Pretraing layer 0 Epoch 132 cost: 21.059911489486694
        Pretraing layer 0 Epoch 133 cost: 18.482269823551178
        Pretraing layer 0 Epoch 134 cost: 17.50355952978134
        Pretraing layer 0 Epoch 135 cost: 16.21581470966339
        Pretraing layer 0 Epoch 136 cost: 16.477513074874878
        Pretraing layer 0 Epoch 137 cost: 20.97428208589554
        Pretraing layer 0 Epoch 138 cost: 17.389583349227905
        Pretraing layer 0 Epoch 139 cost: 17.950942933559418
        Pretraing layer 0 Epoch 140 cost: 17.601406574249268
        Pretraing layer 0 Epoch 141 cost: 16.904861629009247
        Pretraing layer 0 Epoch 142 cost: 18.673050582408905
        Pretraing layer 0 Epoch 143 cost: 18.947972238063812
        Pretraing layer 0 Epoch 144 cost: 19.574856102466583
        Pretraing layer 0 Epoch 145 cost: 16.0682530105114
        Pretraing layer 0 Epoch 146 cost: 20.08187049627304
        Pretraing layer 0 Epoch 147 cost: 19.900906026363373
        Pretraing layer 0 Epoch 148 cost: 17.040996372699738
        Pretraing layer 0 Epoch 149 cost: 19.30685704946518
        Pretraing layer 0 Epoch 150 cost: 18.25774449110031
        Pretraing layer 0 Epoch 151 cost: 17.65548074245453
        Pretraing layer 0 Epoch 152 cost: 19.521827280521393
        Pretraing layer 0 Epoch 153 cost: 19.103311359882355
        Pretraing layer 0 Epoch 154 cost: 19.3931422829628
        Pretraing layer 0 Epoch 155 cost: 18.972823798656464
        Pretraing layer 0 Epoch 156 cost: 19.003403961658478
        Pretraing layer 0 Epoch 157 cost: 16.435042321681976
        Pretraing layer 0 Epoch 158 cost: 19.063817381858826
        Pretraing layer 0 Epoch 159 cost: 20.21192181110382
        Pretraing layer 0 Epoch 160 cost: 20.54459011554718
        Pretraing layer 0 Epoch 161 cost: 18.282789707183838
        Pretraing layer 0 Epoch 162 cost: 18.70405673980713
        Pretraing layer 0 Epoch 163 cost: 20.19399231672287
        Pretraing layer 0 Epoch 164 cost: 20.028922975063324
        Pretraing layer 0 Epoch 165 cost: 19.401988685131073
        Pretraing layer 0 Epoch 166 cost: 17.234430015087128
        Pretraing layer 0 Epoch 167 cost: 19.9911989569664
        Pretraing layer 0 Epoch 168 cost: 20.31811535358429
        Pretraing layer 0 Epoch 169 cost: 20.43706387281418
        Pretraing layer 0 Epoch 170 cost: 18.249056935310364
        Pretraing layer 0 Epoch 171 cost: 20.580980956554413
        Pretraing layer 0 Epoch 172 cost: 21.461483418941498
        Pretraing layer 0 Epoch 173 cost: 16.45180654525757
        Pretraing layer 0 Epoch 174 cost: 20.502806186676025
        Pretraing layer 0 Epoch 175 cost: 20.594980120658875
        Pretraing layer 0 Epoch 176 cost: 23.16861391067505
        Pretraing layer 0 Epoch 177 cost: 22.76139211654663
        Pretraing layer 0 Epoch 178 cost: 21.105785071849823
        Pretraing layer 0 Epoch 179 cost: 17.824710726737976
        Pretraing layer 0 Epoch 180 cost: 23.199245810508728
        Pretraing layer 0 Epoch 181 cost: 20.686965882778168
        Pretraing layer 0 Epoch 182 cost: 20.624701142311096
        Pretraing layer 0 Epoch 183 cost: 18.183343946933746
        Pretraing layer 0 Epoch 184 cost: 23.277929306030273
        Pretraing layer 0 Epoch 185 cost: 20.454329133033752
        Pretraing layer 0 Epoch 186 cost: 22.396900296211243
        Pretraing layer 0 Epoch 187 cost: 20.49606192111969
        Pretraing layer 0 Epoch 188 cost: 19.786279380321503
        Pretraing layer 0 Epoch 189 cost: 21.41702824831009
        Pretraing layer 0 Epoch 190 cost: 19.006182372570038
        Pretraing layer 0 Epoch 191 cost: 20.892439544200897
        Pretraing layer 0 Epoch 192 cost: 19.629565596580505
        Pretraing layer 0 Epoch 193 cost: 18.15945839881897
        Pretraing layer 0 Epoch 194 cost: 21.30398714542389
        Pretraing layer 0 Epoch 195 cost: 19.657002866268158
        Pretraing layer 0 Epoch 196 cost: 20.744249880313873
        Pretraing layer 0 Epoch 197 cost: 21.55553847551346
        Pretraing layer 0 Epoch 198 cost: 21.639089286327362
        Pretraing layer 0 Epoch 199 cost: 22.62739485502243

The accuacy is as follow:

       Epoch 0 cost: 1.599144771695137, validation accuacy: 0.320416659116745
        Epoch 1 cost: 1.5342369079589844, validation accuacy: 0.40708333253860474
        Epoch 2 cost: 1.4703260958194733, validation accuacy: 0.42625001072883606
        Epoch 3 cost: 1.417160913348198, validation accuacy: 0.4645833373069763
        Epoch 4 cost: 1.3573171943426132, validation accuacy: 0.4662500023841858
        Epoch 5 cost: 1.3089560568332672, validation accuacy: 0.4883333444595337
        Epoch 6 cost: 1.2544884979724884, validation accuacy: 0.4808333218097687
        Epoch 7 cost: 1.1992419213056564, validation accuacy: 0.4891666769981384
        Epoch 8 cost: 1.1435130536556244, validation accuacy: 0.5299999713897705
        Epoch 9 cost: 1.091205969452858, validation accuacy: 0.5099999904632568
        Epoch 10 cost: 1.019949048757553, validation accuacy: 0.5375000238418579
        Epoch 11 cost: 0.9569952860474586, validation accuacy: 0.5679166913032532
        Epoch 12 cost: 0.8921251818537712, validation accuacy: 0.5799999833106995
        Epoch 13 cost: 0.8314656317234039, validation accuacy: 0.6208333373069763
        Epoch 14 cost: 0.7727493420243263, validation accuacy: 0.6449999809265137
        Epoch 15 cost: 0.7197526395320892, validation accuacy: 0.6379166841506958
        Epoch 16 cost: 0.6689451262354851, validation accuacy: 0.6970833539962769
        Epoch 17 cost: 0.626779094338417, validation accuacy: 0.6541666388511658
        Epoch 18 cost: 0.5854205265641212, validation accuacy: 0.7037500143051147
        Epoch 19 cost: 0.5414643362164497, validation accuacy: 0.7179166674613953
        Epoch 20 cost: 0.5085277706384659, validation accuacy: 0.7250000238418579
        Epoch 21 cost: 0.4744126796722412, validation accuacy: 0.7358333468437195
        Epoch 22 cost: 0.44661156460642815, validation accuacy: 0.7450000047683716
        Epoch 23 cost: 0.41354047879576683, validation accuacy: 0.75
        Epoch 24 cost: 0.3862645551562309, validation accuacy: 0.7583333253860474
        Epoch 25 cost: 0.36239999905228615, validation accuacy: 0.7620833516120911
        Epoch 26 cost: 0.33988894522190094, validation accuacy: 0.7804166674613953
        Epoch 27 cost: 0.3189098574221134, validation accuacy: 0.7766666412353516
        Epoch 28 cost: 0.2983291782438755, validation accuacy: 0.7799999713897705
        Epoch 29 cost: 0.28129274770617485, validation accuacy: 0.7908333539962769
        Epoch 30 cost: 0.26463982462882996, validation accuacy: 0.7900000214576721
        Epoch 31 cost: 0.2480302732437849, validation accuacy: 0.7962499856948853
        Epoch 32 cost: 0.23482228443026543, validation accuacy: 0.7979166507720947
        Epoch 33 cost: 0.22149690613150597, validation accuacy: 0.8029166460037231
        Epoch 34 cost: 0.2110046800225973, validation accuacy: 0.8020833134651184
        Epoch 35 cost: 0.20022606290876865, validation accuacy: 0.8108333349227905
        Epoch 36 cost: 0.1901907715946436, validation accuacy: 0.8012499809265137
        Epoch 37 cost: 0.17880531400442123, validation accuacy: 0.8224999904632568
        Epoch 38 cost: 0.17111288383603096, validation accuacy: 0.8041666746139526
        Epoch 39 cost: 0.1606860663741827, validation accuacy: 0.8195833563804626
        Epoch 40 cost: 0.1532617285847664, validation accuacy: 0.8162500262260437
        Epoch 41 cost: 0.1459305752068758, validation accuacy: 0.815416693687439
        Epoch 42 cost: 0.13906187005341053, validation accuacy: 0.8183333277702332
        Epoch 43 cost: 0.13391140196472406, validation accuacy: 0.815833330154419
        Epoch 44 cost: 0.12733093742281199, validation accuacy: 0.82791668176651
        Epoch 45 cost: 0.12294845096766949, validation accuacy: 0.8199999928474426
        Epoch 46 cost: 0.11710410192608833, validation accuacy: 0.8245833516120911
        Epoch 47 cost: 0.11352478619664907, validation accuacy: 0.8224999904632568
        Epoch 48 cost: 0.10902078729122877, validation accuacy: 0.8287500143051147
        Epoch 49 cost: 0.10526353027671576, validation accuacy: 0.8266666531562805
        Epoch 50 cost: 0.10194918513298035, validation accuacy: 0.8299999833106995
        Epoch 51 cost: 0.09868921339511871, validation accuacy: 0.8295833468437195
        Epoch 52 cost: 0.09561303723603487, validation accuacy: 0.8258333206176758
        Epoch 53 cost: 0.09238159283995628, validation accuacy: 0.8337500095367432
        Epoch 54 cost: 0.09002083167433739, validation accuacy: 0.8287500143051147
        Epoch 55 cost: 0.08807656541466713, validation accuacy: 0.8391666412353516
        Epoch 56 cost: 0.08474058657884598, validation accuacy: 0.8291666507720947
        Epoch 57 cost: 0.08264455012977123, validation accuacy: 0.8329166769981384
        Epoch 58 cost: 0.08023085445165634, validation accuacy: 0.8345833420753479
        Epoch 59 cost: 0.07879686448723078, validation accuacy: 0.8366666436195374
        Epoch 60 cost: 0.07659853622317314, validation accuacy: 0.8329166769981384
        Epoch 61 cost: 0.07497597113251686, validation accuacy: 0.8324999809265137
        Epoch 62 cost: 0.07262766547501087, validation accuacy: 0.8320833444595337
        Epoch 63 cost: 0.07120804116129875, validation accuacy: 0.8337500095367432
        Epoch 64 cost: 0.06945535307750106, validation accuacy: 0.8383333086967468
        Epoch 65 cost: 0.06761945877224207, validation accuacy: 0.8341666460037231
        Epoch 66 cost: 0.06673267763108015, validation accuacy: 0.8362500071525574
        Epoch 67 cost: 0.06520892912521958, validation accuacy: 0.8362500071525574
        Epoch 68 cost: 0.06398493563756347, validation accuacy: 0.8387500047683716
        Epoch 69 cost: 0.06267190864309669, validation accuacy: 0.8387500047683716
        Epoch 70 cost: 0.061722288839519024, validation accuacy: 0.8379166722297668
        Epoch 71 cost: 0.061079361476004124, validation accuacy: 0.8354166746139526
        Epoch 72 cost: 0.05982930492609739, validation accuacy: 0.8358333110809326
        Epoch 73 cost: 0.058997164480388165, validation accuacy: 0.8358333110809326
        Epoch 74 cost: 0.057740384247154, validation accuacy: 0.8374999761581421
        Epoch 75 cost: 0.05751983216032386, validation accuacy: 0.840833306312561
        Epoch 76 cost: 0.05612770328298211, validation accuacy: 0.8366666436195374
        Epoch 77 cost: 0.05509936297312379, validation accuacy: 0.8441666960716248
        Epoch 78 cost: 0.054794141091406345, validation accuacy: 0.8387500047683716
        Epoch 79 cost: 0.054244800470769405, validation accuacy: 0.84375
        Epoch 80 cost: 0.0528476913459599, validation accuacy: 0.8370833396911621
        Epoch 81 cost: 0.05232694000005722, validation accuacy: 0.8450000286102295
        Epoch 82 cost: 0.05144566623494029, validation accuacy: 0.8416666388511658
        Epoch 83 cost: 0.05101228225976229, validation accuacy: 0.8454166650772095
        Epoch 84 cost: 0.050138473976403475, validation accuacy: 0.8412500023841858
        Epoch 85 cost: 0.04985665902495384, validation accuacy: 0.8462499976158142
        Epoch 86 cost: 0.04876406816765666, validation accuacy: 0.840833306312561
        Epoch 87 cost: 0.04847849393263459, validation accuacy: 0.8454166650772095
        Epoch 88 cost: 0.04757313011214137, validation accuacy: 0.8483333587646484
        Epoch 89 cost: 0.04724666755646467, validation accuacy: 0.8445833325386047
        Epoch 90 cost: 0.04665342625230551, validation accuacy: 0.8475000262260437
        Epoch 91 cost: 0.046287189703434706, validation accuacy: 0.8416666388511658
        Epoch 92 cost: 0.04613017896190286, validation accuacy: 0.8479166626930237
        Epoch 93 cost: 0.0450873957015574, validation accuacy: 0.8450000286102295
        Epoch 94 cost: 0.04515309818089008, validation accuacy: 0.8479166626930237
        Epoch 95 cost: 0.04421996045857668, validation accuacy: 0.8433333039283752
        Epoch 96 cost: 0.043764406349509954, validation accuacy: 0.8495833277702332
        Epoch 97 cost: 0.0434781271032989, validation accuacy: 0.8433333039283752
        Epoch 98 cost: 0.04330826457589865, validation accuacy: 0.8495833277702332
        Epoch 99 cost: 0.04279144247993827, validation accuacy: 0.8495833277702332
        Epoch 100 cost: 0.0424912734888494, validation accuacy: 0.8462499976158142
        Epoch 101 cost: 0.041634741239249706, validation accuacy: 0.8512499928474426
        Epoch 102 cost: 0.04162588668987155, validation accuacy: 0.8462499976158142
        Epoch 103 cost: 0.042154546827077866, validation accuacy: 0.8550000190734863
        Epoch 104 cost: 0.04116798425093293, validation accuacy: 0.846666693687439
        Epoch 105 cost: 0.04042100580409169, validation accuacy: 0.8545833230018616
        Epoch 106 cost: 0.04001117032021284, validation accuacy: 0.8520833253860474
        Epoch 107 cost: 0.040376252960413694, validation accuacy: 0.8520833253860474
        Epoch 108 cost: 0.03914386313408613, validation accuacy: 0.8500000238418579
        Epoch 109 cost: 0.03898645052686334, validation accuacy: 0.8529166579246521
        Epoch 110 cost: 0.03879777109250426, validation accuacy: 0.847083330154419
        Epoch 111 cost: 0.03865752322599292, validation accuacy: 0.8545833230018616
        Epoch 112 cost: 0.03780481405556202, validation accuacy: 0.8508333563804626
        Epoch 113 cost: 0.03760525770485401, validation accuacy: 0.856249988079071
        Epoch 114 cost: 0.03732246905565262, validation accuacy: 0.8533333539962769
        Epoch 115 cost: 0.03717274637892842, validation accuacy: 0.8504166603088379
        Epoch 116 cost: 0.036945600993931293, validation accuacy: 0.8508333563804626
        Epoch 117 cost: 0.03644209634512663, validation accuacy: 0.8537499904632568
        Epoch 118 cost: 0.03633446618914604, validation accuacy: 0.8550000190734863
        Epoch 119 cost: 0.03610427677631378, validation accuacy: 0.8545833230018616
        Epoch 120 cost: 0.03599788388237357, validation accuacy: 0.8525000214576721
        Epoch 121 cost: 0.035328594502061605, validation accuacy: 0.8554166555404663
        Epoch 122 cost: 0.03549408772960305, validation accuacy: 0.8545833230018616
        Epoch 123 cost: 0.035058862529695034, validation accuacy: 0.8550000190734863
        Epoch 124 cost: 0.03508830210193992, validation accuacy: 0.8550000190734863
        Epoch 125 cost: 0.03491163905709982, validation accuacy: 0.8579166531562805
        Epoch 126 cost: 0.03467117249965668, validation accuacy: 0.8545833230018616
        Epoch 127 cost: 0.0342645556665957, validation accuacy: 0.8575000166893005
        Epoch 128 cost: 0.03402972803451121, validation accuacy: 0.8558333516120911
        Epoch 129 cost: 0.03371947351843119, validation accuacy: 0.8583333492279053
        Epoch 130 cost: 0.033859983552247286, validation accuacy: 0.8550000190734863
        Epoch 131 cost: 0.03343882039189339, validation accuacy: 0.8566666841506958
        Epoch 132 cost: 0.0335479979403317, validation accuacy: 0.8541666865348816
        Epoch 133 cost: 0.033064888091757894, validation accuacy: 0.8575000166893005
        Epoch 134 cost: 0.032786168390884995, validation accuacy: 0.8566666841506958
        Epoch 135 cost: 0.03298648726195097, validation accuacy: 0.8587499856948853
        Epoch 136 cost: 0.0328899547457695, validation accuacy: 0.8545833230018616
        Epoch 137 cost: 0.0319041155744344, validation accuacy: 0.8587499856948853
        Epoch 138 cost: 0.03270832262933254, validation accuacy: 0.856249988079071
        Epoch 139 cost: 0.03230214770883322, validation accuacy: 0.8583333492279053
        Epoch 140 cost: 0.03184652398340404, validation accuacy: 0.8558333516120911
        Epoch 141 cost: 0.031775945564731956, validation accuacy: 0.8629166483879089
        Epoch 142 cost: 0.03114823787473142, validation accuacy: 0.8554166555404663
        Epoch 143 cost: 0.03106176294386387, validation accuacy: 0.8579166531562805
        Epoch 144 cost: 0.031080533750355244, validation accuacy: 0.8629166483879089
        Epoch 145 cost: 0.031149546382948756, validation accuacy: 0.85916668176651
        Epoch 146 cost: 0.030730641214177012, validation accuacy: 0.8583333492279053
        Epoch 147 cost: 0.031022229697555304, validation accuacy: 0.8600000143051147
        Epoch 148 cost: 0.03047758643515408, validation accuacy: 0.8604166507720947
        Epoch 149 cost: 0.03051623492501676, validation accuacy: 0.8612499833106995
        Epoch 150 cost: 0.030879850732162595, validation accuacy: 0.85958331823349
        Epoch 151 cost: 0.030698991613462567, validation accuacy: 0.8633333444595337
        Epoch 152 cost: 0.03005589870736003, validation accuacy: 0.8600000143051147
        Epoch 153 cost: 0.030076920986175537, validation accuacy: 0.8600000143051147
        Epoch 154 cost: 0.029436665354296565, validation accuacy: 0.8604166507720947
        Epoch 155 cost: 0.029918714426457882, validation accuacy: 0.85916668176651
        Epoch 156 cost: 0.02957465872168541, validation accuacy: 0.8616666793823242
        Epoch 157 cost: 0.02939280867576599, validation accuacy: 0.8600000143051147
        Epoch 158 cost: 0.029297769768163562, validation accuacy: 0.8679166436195374
        Epoch 159 cost: 0.02921509649604559, validation accuacy: 0.8650000095367432
        Epoch 160 cost: 0.028888638596981764, validation accuacy: 0.85958331823349
        Epoch 161 cost: 0.029241600539535284, validation accuacy: 0.8654166460037231
        Epoch 162 cost: 0.02863740478642285, validation accuacy: 0.8587499856948853
        Epoch 163 cost: 0.028719150694087148, validation accuacy: 0.862500011920929
        Epoch 164 cost: 0.028049648739397526, validation accuacy: 0.8658333420753479
        Epoch 165 cost: 0.028501437976956367, validation accuacy: 0.85958331823349
        Epoch 166 cost: 0.02838796377182007, validation accuacy: 0.8608333468437195
        Epoch 167 cost: 0.027961534913629293, validation accuacy: 0.8633333444595337
        Epoch 168 cost: 0.027980234939604998, validation accuacy: 0.8666666746139526
        Epoch 169 cost: 0.02858111821115017, validation accuacy: 0.8620833158493042
        Epoch 170 cost: 0.027681350708007812, validation accuacy: 0.8620833158493042
        Epoch 171 cost: 0.027637448394671082, validation accuacy: 0.8683333396911621
        Epoch 172 cost: 0.027665867935866117, validation accuacy: 0.8650000095367432
        Epoch 173 cost: 0.02696719695813954, validation accuacy: 0.8654166460037231
        Epoch 174 cost: 0.027587547432631254, validation accuacy: 0.8679166436195374
        Epoch 175 cost: 0.027610703138634562, validation accuacy: 0.8575000166893005
        Epoch 176 cost: 0.02786984504200518, validation accuacy: 0.8662499785423279
        Epoch 177 cost: 0.027614354621618986, validation accuacy: 0.8679166436195374
        Epoch 178 cost: 0.02786106802523136, validation accuacy: 0.8570833206176758
        Epoch 179 cost: 0.027468198677524924, validation accuacy: 0.8666666746139526
        Epoch 180 cost: 0.026900691678747535, validation accuacy: 0.8616666793823242
        Epoch 181 cost: 0.02687909221276641, validation accuacy: 0.8712499737739563
        Epoch 182 cost: 0.02644415944814682, validation accuacy: 0.8687499761581421
        Epoch 183 cost: 0.02639101562090218, validation accuacy: 0.8600000143051147
        Epoch 184 cost: 0.026477640960365534, validation accuacy: 0.8670833110809326
        Epoch 185 cost: 0.02620858047157526, validation accuacy: 0.8666666746139526
        Epoch 186 cost: 0.026387503370642662, validation accuacy: 0.8608333468437195
        Epoch 187 cost: 0.02596354717388749, validation accuacy: 0.8637499809265137
        Epoch 188 cost: 0.025927416048943996, validation accuacy: 0.8691666722297668
        Epoch 189 cost: 0.0258642400149256, validation accuacy: 0.8666666746139526
        Epoch 190 cost: 0.02555061038583517, validation accuacy: 0.8629166483879089
        Epoch 191 cost: 0.025708104483783245, validation accuacy: 0.8675000071525574
        Epoch 192 cost: 0.026142134331166744, validation accuacy: 0.8679166436195374
        Epoch 193 cost: 0.025782877579331398, validation accuacy: 0.8620833158493042
        Epoch 194 cost: 0.025935868034139276, validation accuacy: 0.8645833134651184
        Epoch 195 cost: 0.02556067844852805, validation accuacy: 0.8683333396911621
        Epoch 196 cost: 0.025618890533223748, validation accuacy: 0.8691666722297668
        Epoch 197 cost: 0.025104233296588063, validation accuacy: 0.8679166436195374
        Epoch 198 cost: 0.025146273896098137, validation accuacy: 0.8620833158493042
        Epoch 199 cost: 0.02496644575148821, validation accuacy: 0.8700000047683716
        Epoch 200 cost: 0.02496408554725349, validation accuacy: 0.8654166460037231
        Epoch 201 cost: 0.02518424578011036, validation accuacy: 0.8670833110809326
        Epoch 202 cost: 0.025277751265093684, validation accuacy: 0.8666666746139526
        Epoch 203 cost: 0.025358373299241066, validation accuacy: 0.8650000095367432
        Epoch 204 cost: 0.024800084764137864, validation accuacy: 0.8637499809265137
        Epoch 205 cost: 0.025115350261330605, validation accuacy: 0.8704166412353516
        Epoch 206 cost: 0.024695909349247813, validation accuacy: 0.872083306312561
        Epoch 207 cost: 0.02438581408932805, validation accuacy: 0.8620833158493042
        Epoch 208 cost: 0.024343715514987707, validation accuacy: 0.8700000047683716
        Epoch 209 cost: 0.02438173140399158, validation accuacy: 0.8679166436195374
        Epoch 210 cost: 0.024524677777662873, validation accuacy: 0.8641666769981384
        Epoch 211 cost: 0.024733782978728414, validation accuacy: 0.8691666722297668
        Epoch 212 cost: 0.02488864865154028, validation accuacy: 0.8700000047683716
        Epoch 213 cost: 0.02499176631681621, validation accuacy: 0.8670833110809326
        Epoch 214 cost: 0.025176796363666654, validation accuacy: 0.8608333468437195
        Epoch 215 cost: 0.024175592698156834, validation accuacy: 0.8737499713897705
        Epoch 216 cost: 0.024023185949772596, validation accuacy: 0.8654166460037231
        Epoch 217 cost: 0.024129138328135014, validation accuacy: 0.8587499856948853
        Epoch 218 cost: 0.023924462730064988, validation accuacy: 0.8687499761581421
        Epoch 219 cost: 0.024083816446363926, validation accuacy: 0.8662499785423279
        Epoch 220 cost: 0.023836814565584064, validation accuacy: 0.8687499761581421
        Epoch 221 cost: 0.02329368912614882, validation accuacy: 0.8633333444595337
        Epoch 222 cost: 0.024108682060614228, validation accuacy: 0.875
        Epoch 223 cost: 0.024336645612493157, validation accuacy: 0.8650000095367432
        Epoch 224 cost: 0.024561248486861587, validation accuacy: 0.8700000047683716
        Epoch 225 cost: 0.02446965826675296, validation accuacy: 0.862500011920929
        Epoch 226 cost: 0.023849876364693046, validation accuacy: 0.8670833110809326
        Epoch 227 cost: 0.02322399360127747, validation accuacy: 0.8729166388511658
        Epoch 228 cost: 0.02296071103774011, validation accuacy: 0.8675000071525574
        Epoch 229 cost: 0.023172719171270728, validation accuacy: 0.8691666722297668
        Epoch 230 cost: 0.023500098614022136, validation accuacy: 0.8654166460037231
        Epoch 231 cost: 0.023654921911656857, validation accuacy: 0.8662499785423279
        Epoch 232 cost: 0.023205066565424204, validation accuacy: 0.8700000047683716
        Epoch 233 cost: 0.023192581487819552, validation accuacy: 0.8695833086967468
        Epoch 234 cost: 0.02288910956121981, validation accuacy: 0.8762500286102295
        Epoch 235 cost: 0.023136466275900602, validation accuacy: 0.8616666793823242
        Epoch 236 cost: 0.023054796969518065, validation accuacy: 0.8679166436195374
        Epoch 237 cost: 0.022603526711463928, validation accuacy: 0.8700000047683716
        Epoch 238 cost: 0.02298583439551294, validation accuacy: 0.8737499713897705
        Epoch 239 cost: 0.023453518515452743, validation accuacy: 0.8666666746139526
        Epoch 240 cost: 0.02276132651604712, validation accuacy: 0.872083306312561
        Epoch 241 cost: 0.02272284240461886, validation accuacy: 0.871666669845581
        Epoch 242 cost: 0.02275794348679483, validation accuacy: 0.8725000023841858
        Epoch 243 cost: 0.02256550290621817, validation accuacy: 0.8662499785423279
        Epoch 244 cost: 0.023362674051895738, validation accuacy: 0.8729166388511658
        Epoch 245 cost: 0.022831915644928813, validation accuacy: 0.8687499761581421
        Epoch 246 cost: 0.022588559426367283, validation accuacy: 0.8700000047683716
        Epoch 247 cost: 0.022342158248648047, validation accuacy: 0.8700000047683716
        Epoch 248 cost: 0.022576353745535016, validation accuacy: 0.8691666722297668
        Epoch 249 cost: 0.022891954518854618, validation accuacy: 0.8708333373069763
        Epoch 250 cost: 0.022032494889572263, validation accuacy: 0.8708333373069763

Without pretraing,the accuacy is as follows:

      Epoch 0 cost: 1.5948470532894135, validation accuacy: 0.3487499952316284
        Epoch 1 cost: 1.5230149328708649, validation accuacy: 0.42250001430511475
        Epoch 2 cost: 1.4591684490442276, validation accuacy: 0.4325000047683716
        Epoch 3 cost: 1.4064775556325912, validation accuacy: 0.4566666781902313
        Epoch 4 cost: 1.3472682535648346, validation accuacy: 0.4754166603088379
        Epoch 5 cost: 1.2923140823841095, validation accuacy: 0.4987500011920929
        Epoch 6 cost: 1.2350990772247314, validation accuacy: 0.4870833456516266
        Epoch 7 cost: 1.1798710227012634, validation accuacy: 0.5274999737739563
        Epoch 8 cost: 1.1101636588573456, validation accuacy: 0.5450000166893005
        Epoch 9 cost: 1.0479509681463242, validation accuacy: 0.528333306312561
        Epoch 10 cost: 0.9865358620882034, validation accuacy: 0.581250011920929
        Epoch 11 cost: 0.9273795261979103, validation accuacy: 0.6020833253860474
        Epoch 12 cost: 0.8755166083574295, validation accuacy: 0.6045833230018616
        Epoch 13 cost: 0.8264763876795769, validation accuacy: 0.6399999856948853
        Epoch 14 cost: 0.7711481600999832, validation accuacy: 0.64083331823349
        Epoch 15 cost: 0.728942483663559, validation accuacy: 0.6700000166893005
        Epoch 16 cost: 0.6845641508698463, validation accuacy: 0.6937500238418579
        Epoch 17 cost: 0.6428006887435913, validation accuacy: 0.7054166793823242
        Epoch 18 cost: 0.6060236766934395, validation accuacy: 0.7266666889190674
        Epoch 19 cost: 0.5691065788269043, validation accuacy: 0.7266666889190674
        Epoch 20 cost: 0.5370697900652885, validation accuacy: 0.7387499809265137
        Epoch 21 cost: 0.5028078593313694, validation accuacy: 0.7541666626930237
        Epoch 22 cost: 0.47495392709970474, validation accuacy: 0.7549999952316284
        Epoch 23 cost: 0.4462144002318382, validation accuacy: 0.7637500166893005
        Epoch 24 cost: 0.417914517223835, validation accuacy: 0.7720833420753479
        Epoch 25 cost: 0.39300087094306946, validation accuacy: 0.7770833373069763
        Epoch 26 cost: 0.37085535377264023, validation accuacy: 0.7837499976158142
        Epoch 27 cost: 0.3481658846139908, validation accuacy: 0.7879166603088379
        Epoch 28 cost: 0.3265502117574215, validation accuacy: 0.7920833230018616
        Epoch 29 cost: 0.3053378090262413, validation accuacy: 0.7916666865348816
        Epoch 30 cost: 0.2918824516236782, validation accuacy: 0.7929166555404663
        Epoch 31 cost: 0.2732721343636513, validation accuacy: 0.7941666841506958
        Epoch 32 cost: 0.25857139378786087, validation accuacy: 0.7983333468437195
        Epoch 33 cost: 0.243555199354887, validation accuacy: 0.8083333373069763
        Epoch 34 cost: 0.22972501628100872, validation accuacy: 0.8050000071525574
        Epoch 35 cost: 0.21797134913504124, validation accuacy: 0.8058333396911621
        Epoch 36 cost: 0.2086473274976015, validation accuacy: 0.8112499713897705
        Epoch 37 cost: 0.1952453199774027, validation accuacy: 0.8079166412353516
        Epoch 38 cost: 0.18381572514772415, validation accuacy: 0.815416693687439
        Epoch 39 cost: 0.17515884339809418, validation accuacy: 0.8083333373069763
        Epoch 40 cost: 0.16655678674578667, validation accuacy: 0.8179166913032532
        Epoch 41 cost: 0.1580547820776701, validation accuacy: 0.815833330154419
        Epoch 42 cost: 0.15243693441152573, validation accuacy: 0.8220833539962769
        Epoch 43 cost: 0.14568585716187954, validation accuacy: 0.8137500286102295
        Epoch 44 cost: 0.13941716775298119, validation accuacy: 0.8224999904632568
        Epoch 45 cost: 0.13557153474539518, validation accuacy: 0.8174999952316284
        Epoch 46 cost: 0.1298879273235798, validation accuacy: 0.824999988079071
        Epoch 47 cost: 0.12345707323402166, validation accuacy: 0.8245833516120911
        Epoch 48 cost: 0.11850620526820421, validation accuacy: 0.8254166841506958
        Epoch 49 cost: 0.11471707373857498, validation accuacy: 0.8224999904632568
        Epoch 50 cost: 0.11105895694345236, validation accuacy: 0.8199999928474426
        Epoch 51 cost: 0.1058640955016017, validation accuacy: 0.8304166793823242
        Epoch 52 cost: 0.10323112364858389, validation accuacy: 0.8254166841506958
        Epoch 53 cost: 0.09991796407848597, validation accuacy: 0.82791668176651
        Epoch 54 cost: 0.09642103873193264, validation accuacy: 0.8258333206176758
        Epoch 55 cost: 0.09395183622837067, validation accuacy: 0.8299999833106995
        Epoch 56 cost: 0.09211174305528402, validation accuacy: 0.82791668176651
        Epoch 57 cost: 0.08896698709577322, validation accuacy: 0.8295833468437195
        Epoch 58 cost: 0.08611858636140823, validation accuacy: 0.8374999761581421
        Epoch 59 cost: 0.08317329920828342, validation accuacy: 0.8358333110809326
        Epoch 60 cost: 0.08251206297427416, validation accuacy: 0.8308333158493042
        Epoch 61 cost: 0.080216852016747, validation accuacy: 0.831250011920929
        Epoch 62 cost: 0.07853884436190128, validation accuacy: 0.8379166722297668
        Epoch 63 cost: 0.07546081393957138, validation accuacy: 0.8337500095367432
        Epoch 64 cost: 0.07411287445574999, validation accuacy: 0.8383333086967468
        Epoch 65 cost: 0.07225224189460278, validation accuacy: 0.8366666436195374
        Epoch 66 cost: 0.07058872934430838, validation accuacy: 0.8383333086967468
        Epoch 67 cost: 0.06947430362924933, validation accuacy: 0.8374999761581421
        Epoch 68 cost: 0.0678966511040926, validation accuacy: 0.8387500047683716
        Epoch 69 cost: 0.06630135700106621, validation accuacy: 0.8379166722297668
        Epoch 70 cost: 0.06579450704157352, validation accuacy: 0.8420833349227905
        Epoch 71 cost: 0.06498471135273576, validation accuacy: 0.8412500023841858
        Epoch 72 cost: 0.06376851396635175, validation accuacy: 0.8391666412353516
        Epoch 73 cost: 0.062186496797949076, validation accuacy: 0.8454166650772095
        Epoch 74 cost: 0.060896712355315685, validation accuacy: 0.8424999713897705
        Epoch 75 cost: 0.06003121193498373, validation accuacy: 0.8420833349227905
        Epoch 76 cost: 0.059580888133496046, validation accuacy: 0.8412500023841858
        Epoch 77 cost: 0.05834292434155941, validation accuacy: 0.84375
        Epoch 78 cost: 0.05766892060637474, validation accuacy: 0.8424999713897705
        Epoch 79 cost: 0.056662678718566895, validation accuacy: 0.8445833325386047
        Epoch 80 cost: 0.055378401186317205, validation accuacy: 0.8433333039283752
        Epoch 81 cost: 0.05462424596771598, validation accuacy: 0.8441666960716248
        Epoch 82 cost: 0.054245189763605595, validation accuacy: 0.8504166603088379
        Epoch 83 cost: 0.05326728755608201, validation accuacy: 0.84375
        Epoch 84 cost: 0.052566798869520426, validation accuacy: 0.8487499952316284
        Epoch 85 cost: 0.051548016257584095, validation accuacy: 0.846666693687439
        Epoch 86 cost: 0.051346291322261095, validation accuacy: 0.846666693687439
        Epoch 87 cost: 0.050982801243662834, validation accuacy: 0.8495833277702332
        Epoch 88 cost: 0.05027435626834631, validation accuacy: 0.8483333587646484
        Epoch 89 cost: 0.0495265144854784, validation accuacy: 0.8454166650772095
        Epoch 90 cost: 0.04869308229535818, validation accuacy: 0.8525000214576721
        Epoch 91 cost: 0.047801044303923845, validation accuacy: 0.846666693687439
        Epoch 92 cost: 0.04766810964792967, validation accuacy: 0.8508333563804626
        Epoch 93 cost: 0.047308159060776234, validation accuacy: 0.8533333539962769
        Epoch 94 cost: 0.046373565681278706, validation accuacy: 0.8529166579246521
        Epoch 95 cost: 0.046361485961824656, validation accuacy: 0.8550000190734863
        Epoch 96 cost: 0.04597476543858647, validation accuacy: 0.8516666889190674
        Epoch 97 cost: 0.04536283342167735, validation accuacy: 0.8550000190734863
        Epoch 98 cost: 0.04484028788283467, validation accuacy: 0.8500000238418579
        Epoch 99 cost: 0.044349951669573784, validation accuacy: 0.856249988079071
        Epoch 100 cost: 0.043807382229715586, validation accuacy: 0.8520833253860474
        Epoch 101 cost: 0.04354841122403741, validation accuacy: 0.8487499952316284
        Epoch 102 cost: 0.042985789477825165, validation accuacy: 0.8504166603088379
        Epoch 103 cost: 0.043160802219063044, validation accuacy: 0.8537499904632568
        Epoch 104 cost: 0.042508754413574934, validation accuacy: 0.8525000214576721
        Epoch 105 cost: 0.04230228718370199, validation accuacy: 0.8520833253860474
        Epoch 106 cost: 0.041812360752373934, validation accuacy: 0.8529166579246521
        Epoch 107 cost: 0.04143759422004223, validation accuacy: 0.8495833277702332
        Epoch 108 cost: 0.04087428329512477, validation accuacy: 0.8520833253860474
        Epoch 109 cost: 0.04064552718773484, validation accuacy: 0.8537499904632568
        Epoch 110 cost: 0.040115274488925934, validation accuacy: 0.8533333539962769
        Epoch 111 cost: 0.04015572601929307, validation accuacy: 0.8529166579246521
        Epoch 112 cost: 0.039079861249774694, validation accuacy: 0.8495833277702332
        Epoch 113 cost: 0.038785810116678476, validation accuacy: 0.8545833230018616
        Epoch 114 cost: 0.038991001434624195, validation accuacy: 0.8525000214576721
        Epoch 115 cost: 0.03867336828261614, validation accuacy: 0.8520833253860474
        Epoch 116 cost: 0.03887261403724551, validation accuacy: 0.8525000214576721
        Epoch 117 cost: 0.03850983502343297, validation accuacy: 0.8512499928474426
        Epoch 118 cost: 0.038013359531760216, validation accuacy: 0.8541666865348816
        Epoch 119 cost: 0.03771372605115175, validation accuacy: 0.8575000166893005
        Epoch 120 cost: 0.037450458854436874, validation accuacy: 0.8566666841506958
        Epoch 121 cost: 0.036676598247140646, validation accuacy: 0.8520833253860474
        Epoch 122 cost: 0.036349484231323004, validation accuacy: 0.8604166507720947
        Epoch 123 cost: 0.035717427264899015, validation accuacy: 0.8541666865348816
        Epoch 124 cost: 0.03596598748117685, validation accuacy: 0.8554166555404663
        Epoch 125 cost: 0.035260996548458934, validation accuacy: 0.8558333516120911
        Epoch 126 cost: 0.03510179603472352, validation accuacy: 0.8620833158493042
        Epoch 127 cost: 0.035005740355700254, validation accuacy: 0.8583333492279053
        Epoch 128 cost: 0.03473871620371938, validation accuacy: 0.8529166579246521
        Epoch 129 cost: 0.034871046897023916, validation accuacy: 0.8566666841506958
        Epoch 130 cost: 0.03429740364663303, validation accuacy: 0.8566666841506958
        Epoch 131 cost: 0.033950616139918566, validation accuacy: 0.8570833206176758
        Epoch 132 cost: 0.03395309648476541, validation accuacy: 0.8612499833106995
        Epoch 133 cost: 0.03370762593112886, validation accuacy: 0.8616666793823242
        Epoch 134 cost: 0.03410334000363946, validation accuacy: 0.8554166555404663
        Epoch 135 cost: 0.03389162756502628, validation accuacy: 0.8529166579246521
        Epoch 136 cost: 0.03332185884937644, validation accuacy: 0.862500011920929
        Epoch 137 cost: 0.033479629550129175, validation accuacy: 0.8537499904632568
        Epoch 138 cost: 0.03299224004149437, validation accuacy: 0.8583333492279053
        Epoch 139 cost: 0.032412752509117126, validation accuacy: 0.8629166483879089
        Epoch 140 cost: 0.032303908839821815, validation accuacy: 0.862500011920929
        Epoch 141 cost: 0.032865619054064155, validation accuacy: 0.8570833206176758
        Epoch 142 cost: 0.03250659490004182, validation accuacy: 0.862500011920929
        Epoch 143 cost: 0.03199642710387707, validation accuacy: 0.8612499833106995
        Epoch 144 cost: 0.031720547238364816, validation accuacy: 0.85958331823349
        Epoch 145 cost: 0.03155840141698718, validation accuacy: 0.8575000166893005
        Epoch 146 cost: 0.03134618094190955, validation accuacy: 0.8600000143051147
        Epoch 147 cost: 0.031253730645403266, validation accuacy: 0.8633333444595337
        Epoch 148 cost: 0.030949541134759784, validation accuacy: 0.85916668176651
        Epoch 149 cost: 0.030803375644609332, validation accuacy: 0.85958331823349
        Epoch 150 cost: 0.03082563914358616, validation accuacy: 0.8650000095367432
        Epoch 151 cost: 0.03045227867551148, validation accuacy: 0.8612499833106995
        Epoch 152 cost: 0.030277827754616737, validation accuacy: 0.8600000143051147
        Epoch 153 cost: 0.030178937362506986, validation accuacy: 0.8629166483879089
        Epoch 154 cost: 0.030420024413615465, validation accuacy: 0.8575000166893005
        Epoch 155 cost: 0.030241081956773996, validation accuacy: 0.8600000143051147
        Epoch 156 cost: 0.030443538911640644, validation accuacy: 0.8612499833106995
        Epoch 157 cost: 0.030203335219994187, validation accuacy: 0.8654166460037231
        Epoch 158 cost: 0.029693310847505927, validation accuacy: 0.85958331823349
        Epoch 159 cost: 0.029612133745104074, validation accuacy: 0.862500011920929
        Epoch 160 cost: 0.02905465243384242, validation accuacy: 0.8583333492279053
        Epoch 161 cost: 0.029471824411302805, validation accuacy: 0.8645833134651184
        Epoch 162 cost: 0.029229101026430726, validation accuacy: 0.8629166483879089
        Epoch 163 cost: 0.029210385866463184, validation accuacy: 0.8566666841506958
        Epoch 164 cost: 0.02867829240858555, validation accuacy: 0.8645833134651184
        Epoch 165 cost: 0.02894262271001935, validation accuacy: 0.8616666793823242
        Epoch 166 cost: 0.028549196664243937, validation accuacy: 0.862500011920929
        Epoch 167 cost: 0.028198069892823696, validation accuacy: 0.8633333444595337
        Epoch 168 cost: 0.028224345995113254, validation accuacy: 0.8650000095367432
        Epoch 169 cost: 0.02786267828196287, validation accuacy: 0.8637499809265137
        Epoch 170 cost: 0.027785869548097253, validation accuacy: 0.8645833134651184
        Epoch 171 cost: 0.027668529422953725, validation accuacy: 0.8637499809265137
        Epoch 172 cost: 0.02796720713376999, validation accuacy: 0.8645833134651184
        Epoch 173 cost: 0.02777830953709781, validation accuacy: 0.8658333420753479
        Epoch 174 cost: 0.02768245874904096, validation accuacy: 0.8662499785423279
        Epoch 175 cost: 0.027570075821131468, validation accuacy: 0.862500011920929
        Epoch 176 cost: 0.027949515730142593, validation accuacy: 0.8645833134651184
        Epoch 177 cost: 0.027168207801878452, validation accuacy: 0.8654166460037231
        Epoch 178 cost: 0.027316154912114143, validation accuacy: 0.8670833110809326
        Epoch 179 cost: 0.02698341105133295, validation accuacy: 0.8637499809265137
        Epoch 180 cost: 0.027266510762274265, validation accuacy: 0.8641666769981384
        Epoch 181 cost: 0.02679858892224729, validation accuacy: 0.8700000047683716
        Epoch 182 cost: 0.026499778497964144, validation accuacy: 0.85958331823349
        Epoch 183 cost: 0.027162353275343776, validation accuacy: 0.8670833110809326
        Epoch 184 cost: 0.026833327254280448, validation accuacy: 0.8570833206176758
        Epoch 185 cost: 0.026706188917160034, validation accuacy: 0.8608333468437195
        Epoch 186 cost: 0.026760772801935673, validation accuacy: 0.862500011920929
        Epoch 187 cost: 0.026588062290102243, validation accuacy: 0.8637499809265137
        Epoch 188 cost: 0.026761379558593035, validation accuacy: 0.8608333468437195
        Epoch 189 cost: 0.026879476848989725, validation accuacy: 0.85916668176651
        Epoch 190 cost: 0.026468936819583178, validation accuacy: 0.8691666722297668
        Epoch 191 cost: 0.025985796935856342, validation accuacy: 0.8658333420753479
        Epoch 192 cost: 0.025718713644891977, validation accuacy: 0.8608333468437195
        Epoch 193 cost: 0.025847869459539652, validation accuacy: 0.8645833134651184
        Epoch 194 cost: 0.025624479167163372, validation accuacy: 0.8654166460037231
        Epoch 195 cost: 0.02584512601606548, validation accuacy: 0.8620833158493042
        Epoch 196 cost: 0.025915567530319095, validation accuacy: 0.8654166460037231
        Epoch 197 cost: 0.025406068423762918, validation accuacy: 0.8658333420753479
        Epoch 198 cost: 0.025431879330426455, validation accuacy: 0.8654166460037231
        Epoch 199 cost: 0.025284649804234505, validation accuacy: 0.8670833110809326
        Epoch 200 cost: 0.025524573866277933, validation accuacy: 0.8612499833106995
        Epoch 201 cost: 0.02541975793428719, validation accuacy: 0.8650000095367432
        Epoch 202 cost: 0.02520107408054173, validation accuacy: 0.8608333468437195
        Epoch 203 cost: 0.02510644681751728, validation accuacy: 0.8633333444595337
        Epoch 204 cost: 0.025094154058024287, validation accuacy: 0.8675000071525574
        Epoch 205 cost: 0.024846867192536592, validation accuacy: 0.8708333373069763
        Epoch 206 cost: 0.02487553795799613, validation accuacy: 0.8608333468437195
        Epoch 207 cost: 0.025365040404722095, validation accuacy: 0.8691666722297668
        Epoch 208 cost: 0.02472447999753058, validation accuacy: 0.8629166483879089
        Epoch 209 cost: 0.024491376942023635, validation accuacy: 0.8616666793823242
        Epoch 210 cost: 0.024949782295152545, validation accuacy: 0.8637499809265137
        Epoch 211 cost: 0.024694591062143445, validation accuacy: 0.862500011920929
        Epoch 212 cost: 0.025122939376160502, validation accuacy: 0.862500011920929
        Epoch 213 cost: 0.02474158932454884, validation accuacy: 0.8641666769981384
        Epoch 214 cost: 0.024254578398540616, validation accuacy: 0.8658333420753479
        Epoch 215 cost: 0.024276438169181347, validation accuacy: 0.856249988079071
        Epoch 216 cost: 0.024587939027696848, validation accuacy: 0.8679166436195374
        Epoch 217 cost: 0.02450160845182836, validation accuacy: 0.8616666793823242
        Epoch 218 cost: 0.025139920646324754, validation accuacy: 0.8637499809265137
        Epoch 219 cost: 0.0243537409696728, validation accuacy: 0.8637499809265137
        Epoch 220 cost: 0.023927964735776186, validation accuacy: 0.8654166460037231
        Epoch 221 cost: 0.02407812001183629, validation accuacy: 0.862500011920929
        Epoch 222 cost: 0.024083650205284357, validation accuacy: 0.8654166460037231
        Epoch 223 cost: 0.023731961147859693, validation accuacy: 0.8633333444595337
        Epoch 224 cost: 0.023697555530816317, validation accuacy: 0.8658333420753479
        Epoch 225 cost: 0.023745218059048057, validation accuacy: 0.8641666769981384
        Epoch 226 cost: 0.02358192764222622, validation accuacy: 0.8658333420753479
        Epoch 227 cost: 0.023542637238278985, validation accuacy: 0.8650000095367432
        Epoch 228 cost: 0.02339624147862196, validation accuacy: 0.8637499809265137
        Epoch 229 cost: 0.023445755010470748, validation accuacy: 0.8662499785423279
        Epoch 230 cost: 0.023490641033276916, validation accuacy: 0.8683333396911621
        Epoch 231 cost: 0.024044006131589413, validation accuacy: 0.8641666769981384
        Epoch 232 cost: 0.02384304185397923, validation accuacy: 0.8670833110809326
        Epoch 233 cost: 0.023534270701929927, validation accuacy: 0.8633333444595337
        Epoch 234 cost: 0.023052413249388337, validation accuacy: 0.8637499809265137
        Epoch 235 cost: 0.02333670947700739, validation accuacy: 0.8650000095367432
        Epoch 236 cost: 0.023080905433744192, validation accuacy: 0.8658333420753479
        Epoch 237 cost: 0.023180547170341015, validation accuacy: 0.8675000071525574
        Epoch 238 cost: 0.023183240089565516, validation accuacy: 0.8650000095367432
        Epoch 239 cost: 0.023036152822896838, validation accuacy: 0.8654166460037231
        Epoch 240 cost: 0.022997285006567836, validation accuacy: 0.8645833134651184
        Epoch 241 cost: 0.02287099207751453, validation accuacy: 0.8658333420753479
        Epoch 242 cost: 0.023251622915267944, validation accuacy: 0.8620833158493042
        Epoch 243 cost: 0.023508684942498803, validation accuacy: 0.8662499785423279
        Epoch 244 cost: 0.023060139967128634, validation accuacy: 0.8587499856948853
        Epoch 245 cost: 0.02292914711870253, validation accuacy: 0.8645833134651184
        Epoch 246 cost: 0.02258813986554742, validation accuacy: 0.8658333420753479
        Epoch 247 cost: 0.02246262668631971, validation accuacy: 0.8658333420753479
        Epoch 248 cost: 0.022471339907497168, validation accuacy: 0.8612499833106995
        Epoch 249 cost: 0.022544224513694644, validation accuacy: 0.8645833134651184
        Epoch 250 cost: 0.02263373089954257, validation accuacy: 0.8650000095367432

Actually, when the epoch is larger than 300,both of them has a result of about 0.87.
I don't think the GBRBM work in pretraining considering the same result of them.

Anyone knows the reason of it?

Where Can I get the weights file of ShuffleNetV2?

In the file of ShuffleNet V2 :
saver.restore(sess, "./models/shufflene_v2_1.0.ckpt"),
Where can I get the weights file?

question about loss function

I have one question. Why don't i see the loss function for training in your code?

train file

@xiaohu2015 Hi, thanks for your share,but I can't find shuffleNetV2 training files!Where could i find it?
looking for your reply soon.
Best!

RBM预训练完参数作为DBN微调初值赋值

您好，作者。对于DBN.py中RBM的预训练完参数作为DBN微调初值，在代码哪里体现，怎么感觉没看到这一步骤，谢谢

您好，我用您的squeezenet训练我的分类数据，验证集准确率一直保持不变是什么原因？

这个模型能不能只输出一个目标？

比如我想只对车进行画框，该如何操作？我把class里面除了车的去掉后会报错

InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [539] rhs shape= [1470]

da.py cost equals nan after epoch 1

Start training...
Epoch 0 cost: 71.95003230701798
Epoch 1 cost: nan
Epoch 2 cost: nan
Epoch 3 cost: nan
Epoch 4 cost: nan
Epoch 5 cost: nan
Epoch 6 cost: nan
Epoch 7 cost: nan
Epoch 8 cost: nan
Epoch 9 cost: nan

how much data is needed

@xiaohu2015 If the category is 400,how much data is needed for shfflenet-v2.
Looking forward your reply!

can you tell me the difference between yolo.py and yolo_tf.py in the directory yolo

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.