resnet中引入BN问题

来源:5-6 fine-tune-实战

怀瑜

2018-09-07

老师,你好。
我利用课下时间尝试将BN的思想引入resnet中,但是导致的问题就是整个模型在测试集上的acc只有不足10%,而在训练集上acc最后可以高达100%。首先模型肯定存在过拟合,但是我不懂的是为什么在测试集上表现如此的差?另外在训练过程中梯度下降没有问题,loss总体趋势是一直下降的,应该可以排除网络结构和代码编写的错误。我尝试了增大batch至200,结果并没有太大的区别。请问老师这块可能是什么问题?
[Train] Step: 100, loss: 1.60082, acc: 0.42000
[Train] Step: 200, loss: 1.40043, acc: 0.51000
[Train] Step: 300, loss: 1.27248, acc: 0.45000
[Train] Step: 400, loss: 1.30240, acc: 0.57000
[Train] Step: 500, loss: 1.27574, acc: 0.52000
[Train] Step: 600, loss: 1.27000, acc: 0.57000
[Train] Step: 700, loss: 0.95671, acc: 0.65000
[Train] Step: 800, loss: 1.02072, acc: 0.62000
[Train] Step: 900, loss: 1.06775, acc: 0.64000
[Train] Step: 1000, loss: 1.01954, acc: 0.66000
(10000, 3072)
(10000,)
[Test ] Step: 1000, acc: 0.14080
[Train] Step: 1100, loss: 0.76067, acc: 0.72000
[Train] Step: 1200, loss: 0.64366, acc: 0.75000
[Train] Step: 1300, loss: 0.65334, acc: 0.78000
[Train] Step: 1400, loss: 0.69480, acc: 0.77000
[Train] Step: 1500, loss: 0.71311, acc: 0.72000
[Train] Step: 1600, loss: 0.66507, acc: 0.79000
[Train] Step: 1700, loss: 0.58207, acc: 0.83000
[Train] Step: 1800, loss: 0.47070, acc: 0.81000
[Train] Step: 1900, loss: 0.61272, acc: 0.79000
[Train] Step: 2000, loss: 0.61040, acc: 0.78000
(10000, 3072)
(10000,)
[Test ] Step: 2000, acc: 0.11090
[Train] Step: 2100, loss: 0.45111, acc: 0.84000
[Train] Step: 2200, loss: 0.43788, acc: 0.88000
[Train] Step: 2300, loss: 0.31984, acc: 0.89000
[Train] Step: 2400, loss: 0.38695, acc: 0.85000
[Train] Step: 2500, loss: 0.35876, acc: 0.88000
[Train] Step: 2600, loss: 0.22183, acc: 0.93000
[Train] Step: 2700, loss: 0.15740, acc: 0.95000
[Train] Step: 2800, loss: 0.36373, acc: 0.87000
[Train] Step: 2900, loss: 0.21417, acc: 0.93000
[Train] Step: 3000, loss: 0.25152, acc: 0.94000
(10000, 3072)
(10000,)
[Test ] Step: 3000, acc: 0.13450
[Train] Step: 3100, loss: 0.09470, acc: 0.98000
[Train] Step: 3200, loss: 0.10025, acc: 0.98000
[Train] Step: 3300, loss: 0.09875, acc: 0.98000
[Train] Step: 3400, loss: 0.10897, acc: 0.98000
[Train] Step: 3500, loss: 0.11012, acc: 0.97000
[Train] Step: 3600, loss: 0.07915, acc: 0.98000
[Train] Step: 3700, loss: 0.10166, acc: 0.97000
[Train] Step: 3800, loss: 0.05757, acc: 0.99000
[Train] Step: 3900, loss: 0.16726, acc: 0.94000
[Train] Step: 4000, loss: 0.15187, acc: 0.96000
(10000, 3072)
(10000,)
[Test ] Step: 4000, acc: 0.12060

写回答

1回答

正十七

2018-09-14

同学你好,这个可能是因为咱们的代码中批归一化的实现上有点小瑕疵,是因为我记错了一个API的实现。你可以先跳过这个,我稍后会更新代码和课程讲解。

给你带来不便,抱歉啦。学习细心,给你点赞。

0
0

深度学习之神经网络(CNN/RNN/GAN)算法原理+实战

深度学习算法工程师必学,深入理解深度学习核心算法CNN RNN GAN

2617 学习 · 935 问题

查看课程