devbox
我家小花儿
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

基于Tensorflow的Resnet程序实现（CIFAR10准确率为91.5%）_tensorflow resnet18 cifar10测试集准确率90%

作者：我家小花儿 | 2024-03-23 14:12:18

赞

踩

tensorflow resnet18 cifar10测试集准确率90%

       在上一篇博文中我重写了Tensorflow中的CNN的实现，对于CIFAR10的测试集的准确率为85%左右。在这个实现中，用到了2个卷积层和2个全连接层。具体的模型架构如下：

       为了进一步提高准确率，我们可以采用一些更先进的模型架构，其中一种很出名的架构就是RESNET，残差网络。这是Kaiming大神在2015年的论文"Deep Residual Learning for Image Recognition"中提到的一种网络架构，其思想是观察到一般的神经网络结构随着层数的加深，训练的误差反而会增大，因此引入了残差这个概念，把上一层的输出直接和下一层的输出相加，如下图所示。这样理论上随着网络层数的加深，引入这个结构并不会使得误差比浅层的网络更大，因为随着参数的优化，如果浅层网络已经逼近了最优值，那么之后的网络层相当于一个恒等式，即每一层的输入和输出相等，因此更深的层数不会额外增加训练误差。

       在2016年，Kaiming大神发布了另一篇论文“Identity Mappings in Deep Residual Networks”，在这个论文中对Resnet的网络结构作了进一步的改进。改进前和改进后的resnet网络结构如下图所示，按照论文的解释，改进后的结构可以在前向和后向更好的传递残差，因此能取得更好的优化效果：

       在Tensorflow的官方模型中，已经带了一个Resnet的实现，用这个模型训练，在110层的深度下，可以达到CIFAR10测试集92%左右的准确率。但是，这个代码实在是写的比较难读，做了很多辅助功能的封装，每次看代码都是要跳来跳去的看，实在是很不方便。为此我也再次改写了这个代码，按照Kaiming论文介绍的方式来进行模型的构建，在110层的网络层数下，可以达到91%左右的准确率，和官方模型的很接近。

       具体的代码分为两部分，我把构建Resnet模型的代码单独封装在一个文件中。如以下的代码，这个代码里面的_resnet_block_v1和_resnet_block_v2分别对应了上图的两种不同的resnet结构：

  1. import tensorflow as tf
  2.  
  3. def _resnet_block_v1(inputs, filters, stride, projection, stage, blockname, TRAINING):
  4.     # defining name basis
  5.     conv_name_base = 'res' + str(stage) + blockname + '_branch'
  6.     bn_name_base = 'bn' + str(stage) + blockname + '_branch'
  7.  
  8.     with tf.name_scope("conv_block_stage" + str(stage)):
  9.         if projection:
  10.             shortcut = tf.layers.conv2d(inputs, filters, (1,1), 
  11.                                         strides=(stride, stride), 
  12.                                         name=conv_name_base + '1', 
  13.                                         kernel_initializer=tf.contrib.layers.variance_scaling_initializer(), 
  14.                                         reuse=tf.AUTO_REUSE, padding='same', 
  15.                                         data_format='channels_first')
  16.             shortcut = tf.layers.batch_normalization(shortcut, axis=1, name=bn_name_base + '1', 
  17.                                                      training=TRAINING, reuse=tf.AUTO_REUSE)
  18.         else:
  19.             shortcut = inputs
  20.  
  21.         outputs = tf.layers.conv2d(inputs, filters,
  22.                                   kernel_size=(3, 3),
  23.                                   strides=(stride, stride), 
  24.                                   kernel_initializer=tf.contrib.layers.variance_scaling_initializer(), 
  25.                                   name=conv_name_base+'2a', reuse=tf.AUTO_REUSE, padding='same', 
  26.                                   data_format='channels_first')
  27.         outputs = tf.layers.batch_normalization(outputs, axis=1, name=bn_name_base+'2a', 
  28.                                                 training=TRAINING, reuse=tf.AUTO_REUSE)
  29.         outputs = tf.nn.relu(outputs)
  30. 	
  31.         outputs = tf.layers.conv2d(outputs, filters,
  32.                                   kernel_size=(3, 3),
  33.                                   strides=(1, 1), 
  34.                                   kernel_initializer=tf.contrib.layers.variance_scaling_initializer(), 
  35.                                   name=conv_name_base+'2b', reuse=tf.AUTO_REUSE, padding='same', 
  36.                                   data_format='channels_first')
  37.         outputs = tf.layers.batch_normalization(outputs, axis=1, name=bn_name_base+'2b', 
  38.                                                 training=TRAINING, reuse=tf.AUTO_REUSE)
  39.         outputs = tf.add(shortcut, outputs)
  40.         outputs = tf.nn.relu(outputs)								  
  41.     return outputs
  42. 	
  43. def _resnet_block_v2(inputs, filters, stride, projection, stage, blockname, TRAINING):
  44.     # defining name basis
  45.     conv_name_base = 'res' + str(stage) + blockname + '_branch'
  46.     bn_name_base = 'bn' + str(stage) + blockname + '_branch'
  47.  
  48.     with tf.name_scope("conv_block_stage" + str(stage)):
  49.         shortcut = inputs
  50.         outputs = tf.layers.batch_normalization(inputs, axis=1, name=bn_name_base+'2a', 
  51.                                                 training=TRAINING, reuse=tf.AUTO_REUSE)
  52.         outputs = tf.nn.relu(outputs)		
  53.         if projection:
  54.             shortcut = tf.layers.conv2d(outputs, filters, (1,1), 
  55.                                         strides=(stride, stride), 
  56.                                         name=conv_name_base + '1', 
  57.                                         kernel_initializer=tf.contrib.layers.variance_scaling_initializer(), 
  58.                                         reuse=tf.AUTO_REUSE, padding='same', 
  59.                                         data_format='channels_first')
  60.             shortcut = tf.layers.batch_normalization(shortcut, axis=1, name=bn_name_base + '1', 
  61.                                                      training=TRAINING, reuse=tf.AUTO_REUSE)
  62. 								
  63.         outputs = tf.layers.conv2d(outputs, filters,
  64.                                   kernel_size=(3, 3),
  65.                                   strides=(stride, stride), 
  66.                                   kernel_initializer=tf.contrib.layers.variance_scaling_initializer(), 
  67.                                   name=conv_name_base+'2a', reuse=tf.AUTO_REUSE, padding='same', 
  68.                                   data_format='channels_first')
  69.         
  70.         outputs = tf.layers.batch_normalization(outputs, axis=1, name=bn_name_base+'2b', 
  71.                                                 training=TRAINING, reuse=tf.AUTO_REUSE)
  72.         outputs = tf.nn.relu(outputs)
  73.         outputs = tf.layers.conv2d(outputs, filters,
  74.                                   kernel_size=(3, 3),
  75.                                   strides=(1, 1),
  76.                                   kernel_initializer=tf.contrib.layers.variance_scaling_initializer(),
  77.                                   name=conv_name_base+'2b', reuse=tf.AUTO_REUSE, padding='same', 
  78.                                   data_format='channels_first')
  79.  
  80.         outputs = tf.add(shortcut, outputs)
  81.     return outputs
  82.  
  83. def inference(images, training, filters, n, ver):
  84.     """Construct the resnet model
  86.     Args:
  87.       images: [batch*channel*height*width]
  88. 	  training: boolean
  89. 	  filters: integer, the filters of the first resnet stage, the next stage will have filters*2
  90. 	  n: integer, how many resnet blocks in each stage, the total layers number is 6n+2
  91. 	  ver: integer, can be 1 or 2, for resnet v1 or v2
  92.     Returns:
  93.       Tensor, model inference output
  94.     """
  95.     #Layer1 is a 3*3 conv layer, input channels are 3, output channels are 16
  96.     inputs = tf.layers.conv2d(images, filters=16, kernel_size=(3, 3), strides=(1, 1), 
  97.                               name='conv1', reuse=tf.AUTO_REUSE, padding='same', data_format='channels_first')
  98.  
  99.     #no need to batch normal and activate for version 2 resnet.
  100.     if ver==1:
  101.         inputs = tf.layers.batch_normalization(inputs, axis=1, name='bn_conv1',
  102.                                                training=training, reuse=tf.AUTO_REUSE)
  103.         inputs = tf.nn.relu(inputs)
  104.  
  105.     for stage in range(3):
  106.         stage_filter = filters*(2**stage)
  107.         for i in range(n):
  108.             stride = 1
  109.             projection = False
  110.             if i==0 and stage>0:
  111.                 stride = 2
  112.                 projection = True
  113.             if ver==1:
  114.                 inputs = _resnet_block_v1(inputs, stage_filter, stride, projection, 
  115. 				                          stage, blockname=str(i), TRAINING=training)
  116.             else:
  117.                 inputs = _resnet_block_v2(inputs, stage_filter, stride, projection, 
  118. 				                          stage, blockname=str(i), TRAINING=training)
  119.  
  120.     #only need for version 2 resnet.
  121.     if ver==2:
  122.         inputs = tf.layers.batch_normalization(inputs, axis=1, name='pre_activation_final_norm', 
  123.                                                training=training, reuse=tf.AUTO_REUSE)
  124.         inputs = tf.nn.relu(inputs)
  125.  
  126.     axes = [2, 3]
  127.     inputs = tf.reduce_mean(inputs, axes, keep_dims=True)
  128.     inputs = tf.identity(inputs, 'final_reduce_mean')
  129.  
  130.     inputs = tf.reshape(inputs, [-1, filters*(2**2)])
  131.     inputs = tf.layers.dense(inputs=inputs, units=10, name='dense1', reuse=tf.AUTO_REUSE)
  132.     return inputs

       另外一部分的代码就是和Cifar10的处理相关的，其中Cifar10的50000张图片中选取45000张作为训练集，另外5000张作为验证集，测试的10000张图片都作为测试集。在98层的网络深度下，测试集的准确度可以达到92%左右.

  1. import tensorflow as tf
  2. import numpy as np
  3. import os
  4. import resnet_model
  5.  
  6. #Construct the filenames that include the train cifar10 images
  7. folderPath = 'cifar-10-batches-bin/'
  8. filenames = [os.path.join(folderPath, 'data_batch_%d.bin' % i) for i in xrange(1,6)]
  9.  
  10. #Define the parameters of the cifar10 image
  11. imageWidth = 32
  12. imageHeight = 32
  13. imageDepth = 3
  14. label_bytes = 1
  15.  
  16. #Define the train and test batch size
  17. batch_size = 100
  18. test_batch_size = 100
  19. valid_batch_size = 100
  20.  
  21. #Calulate the per image bytes and record bytes
  22. image_bytes = imageWidth * imageHeight * imageDepth
  23. record_bytes = label_bytes + image_bytes
  24.  
  25. #Construct the dataset to read the train images
  26. dataset = tf.data.FixedLengthRecordDataset(filenames, record_bytes)
  27. dataset = dataset.shuffle(50000)
  28.  
  29. #Get the first 45000 records as train dataset records
  30. train_dataset = dataset.take(45000)
  31. train_dataset = train_dataset.batch(batch_size)
  32. train_dataset = train_dataset.repeat(300)
  33. iterator = train_dataset.make_initializable_iterator()
  34.  
  35. #Get the remain 5000 records as valid dataset records
  36. valid_dataset = dataset.skip(45000)
  37. valid_dataset = valid_dataset.batch(valid_batch_size)
  38. validiterator = valid_dataset.make_initializable_iterator()
  39.  
  40. #Construct the dataset to read the test images
  41. testfilename = os.path.join(folderPath, 'test_batch.bin')
  42. testdataset = tf.data.FixedLengthRecordDataset(testfilename, record_bytes)
  43. testdataset = testdataset.batch(test_batch_size)
  44. testiterator = testdataset.make_initializable_iterator()
  45.  
  46. #Decode the train records from the iterator
  47. record = iterator.get_next()
  48. record_decoded_bytes = tf.decode_raw(record, tf.uint8)
  49.  
  50. #Get the labels from the records
  51. record_labels = tf.slice(record_decoded_bytes, [0, 0], [batch_size, 1])
  52. record_labels = tf.cast(record_labels, tf.int32)
  53.  
  54. #Get the images from the records
  55. record_images = tf.slice(record_decoded_bytes, [0, 1], [batch_size, image_bytes])
  56. record_images = tf.reshape(record_images, [batch_size, imageDepth, imageHeight, imageWidth])
  57. record_images = tf.transpose(record_images, [0, 2, 3, 1])
  58. record_images = tf.cast(record_images, tf.float32)
  59.  
  60. #Decode the records from the valid iterator
  61. validrecord = validiterator.get_next()
  62. validrecord_decoded_bytes = tf.decode_raw(validrecord, tf.uint8)
  63.  
  64. #Get the labels from the records
  65. validrecord_labels = tf.slice(validrecord_decoded_bytes, [0, 0], [valid_batch_size, 1])
  66. validrecord_labels = tf.cast(validrecord_labels, tf.int32)
  67. validrecord_labels = tf.reshape(validrecord_labels, [-1])
  68.  
  69. #Get the images from the records
  70. validrecord_images = tf.slice(validrecord_decoded_bytes, [0, 1], [valid_batch_size, image_bytes])
  71. validrecord_images = tf.cast(validrecord_images, tf.float32)
  72. validrecord_images = tf.reshape(validrecord_images, 
  73.                                [valid_batch_size, imageDepth, imageHeight, imageWidth])
  74. validrecord_images = tf.transpose(validrecord_images, [0, 2, 3, 1])
  75.  
  76. #Decode the test records from the iterator
  77. testrecord = testiterator.get_next()
  78. testrecord_decoded_bytes = tf.decode_raw(testrecord, tf.uint8)
  79.  
  80. #Get the labels from the records
  81. testrecord_labels = tf.slice(testrecord_decoded_bytes, [0, 0], [test_batch_size, 1])
  82. testrecord_labels = tf.cast(testrecord_labels, tf.int32)
  83. testrecord_labels = tf.reshape(testrecord_labels, [-1])
  84.  
  85. #Get the images from the records
  86. testrecord_images = tf.slice(testrecord_decoded_bytes, [0, 1], [test_batch_size, image_bytes])
  87. testrecord_images = tf.cast(testrecord_images, tf.float32)
  88. testrecord_images = tf.reshape(testrecord_images, 
  89.                                [test_batch_size, imageDepth, imageHeight, imageWidth])
  90. testrecord_images = tf.transpose(testrecord_images, [0, 2, 3, 1])
  91.  
  92. #Random crop the images after pad each side with 4 pixels
  93. distorted_images = tf.image.resize_image_with_crop_or_pad(record_images, 
  94.                                                           imageHeight+8, imageWidth+8)
  95. distorted_images = tf.random_crop(distorted_images, size = [batch_size, imageHeight, imageHeight, 3])
  96.  
  97. #Unstack the images as the follow up operation are on single train image
  98. distorted_images = tf.unstack(distorted_images)
  99. for i in xrange(len(distorted_images)):
  100.     distorted_images[i] = tf.image.random_flip_left_right(distorted_images[i])
  101.     distorted_images[i] = tf.image.random_brightness(distorted_images[i], max_delta=63)
  102.     distorted_images[i] = tf.image.random_contrast(distorted_images[i], lower=0.2, upper=1.8)
  103.     distorted_images[i] = tf.image.per_image_standardization(distorted_images[i])
  104.     
  105. #Stack the images
  106. distorted_images = tf.stack(distorted_images)
  107.  
  108. #transpose to set the channel first
  109. distorted_images = tf.transpose(distorted_images, perm=[0, 3, 1, 2])
  110.  
  111. #Unstack the images as the follow up operation are on single image
  112. validrecord_images = tf.unstack(validrecord_images)
  113. for i in xrange(len(validrecord_images)):
  114.     validrecord_images[i] = tf.image.per_image_standardization(validrecord_images[i])
  115.     
  116. #Stack the images
  117. validrecord_images = tf.stack(validrecord_images)
  118.  
  119. #transpose to set the channel first
  120. validrecord_images = tf.transpose(validrecord_images, perm=[0, 3, 1, 2])
  121.  
  122. #Unstack the images as the follow up operation are on single image
  123. testrecord_images = tf.unstack(testrecord_images)
  124. for i in xrange(len(testrecord_images)):
  125.     testrecord_images[i] = tf.image.per_image_standardization(testrecord_images[i])
  126.     
  127. #Stack the images
  128. testrecord_images = tf.stack(testrecord_images)
  129.  
  130. #transpose to set the channel first
  131. testrecord_images = tf.transpose(testrecord_images, perm=[0, 3, 1, 2])
  132.  
  133. global_step = tf.Variable(0, trainable=False)
  134. boundaries = [10000, 15000, 20000, 25000]
  135. values = [0.1, 0.05, 0.01, 0.005, 0.001]
  136. learning_rate = tf.train.piecewise_constant(global_step, boundaries, values)
  137. weight_decay = 2e-4
  138. filters = 16  #the first resnet block filter number
  139. n = 5  #the basic resnet block number, total network layers are 6n+2
  140. ver = 2   #the resnet block version
  141.  
  142. #Get the inference logits by the model
  143. result = resnet_model.inference(distorted_images, True, filters, n, ver)
  144.  
  145. #Calculate the cross entropy loss
  146. cross_entropy = tf.losses.sparse_softmax_cross_entropy(labels=record_labels, logits=result)
  147. cross_entropy_mean = tf.reduce_mean(cross_entropy, name='cross_entropy')
  148.  
  149. #Add the l2 weights to the loss
  150. #Add weight decay to the loss.
  151. l2_loss = weight_decay * tf.add_n(
  152.     # loss is computed using fp32 for numerical stability.
  153.     [tf.nn.l2_loss(tf.cast(v, tf.float32)) for v in tf.trainable_variables()])
  154. tf.summary.scalar('l2_loss', l2_loss)
  155. loss = cross_entropy_mean + l2_loss
  156.  
  157. #Define the optimizer
  158. optimizer = tf.train.MomentumOptimizer(learning_rate, momentum=0.9)
  159.  
  160. #Relate to the batch normalization
  161. update_ops = tf.get_collection(tf.GraphKeys.UPDATE_OPS)
  162. with tf.control_dependencies(update_ops):
  163.     opt_op = optimizer.minimize(loss, global_step)
  164.  
  165. valid_accuracy = tf.placeholder(tf.float32)
  166. test_accuracy = tf.placeholder(tf.float32)
  167. tf.summary.scalar("valid_accuracy", valid_accuracy)
  168. tf.summary.scalar("test_accuracy", test_accuracy)
  169. tf.summary.scalar("learning_rate", learning_rate)
  170.  
  171. validresult = tf.argmax(resnet_model.inference(validrecord_images, False, filters, n, ver), axis=1)
  172. testresult = tf.argmax(resnet_model.inference(testrecord_images, False, filters, n, ver), axis=1)
  173.  
  174. #Create the session and run the graph
  175. sess = tf.Session()
  176. sess.run(tf.global_variables_initializer())
  177. sess.run(iterator.initializer)
  178.  
  179. #Merge all the summary and write
  180. summary_op = tf.summary.merge_all()
  181. train_filewriter = tf.summary.FileWriter('train/', sess.graph)
  182.  
  183. step = 0
  184. while(True):
  185.     try:
  186.         lossValue, lr, _ = sess.run([loss, learning_rate, opt_op])
  187.         if step % 100 == 0:
  188.             print "step %i: Learning_rate: %f Loss: %f" %(step, lr, lossValue)
  189.         if step % 1000 == 0:
  190.             saver.save(sess, 'model/my-model', global_step=step)
  191.             truepredictNum = 0
  192.             sess.run([testiterator.initializer, validiterator.initializer])
  193.             accuracy1 = 0.0
  194.             accuracy2 = 0.0
  195.             while(True):
  196.                 try:
  197.                     predictValue, testValue = sess.run([validresult, validrecord_labels])
  198.                     truepredictNum += np.sum(predictValue==testValue)
  199.                 except tf.errors.OutOfRangeError:
  200.                     print "valid correct num: %i" %(truepredictNum)
  201.                     accuracy1 = truepredictNum / 5000.0
  202.                     break
  203.             truepredictNum = 0
  204.             while(True):
  205.                 try:
  206.                     predictValue, testValue = sess.run([testresult, testrecord_labels])
  207.                     truepredictNum += np.sum(predictValue==testValue)
  208.                 except tf.errors.OutOfRangeError:
  209.                     print "test correct num: %i" %(truepredictNum)
  210.                     accuracy2 = truepredictNum / 10000.0
  211.                     break
  212.             summary = sess.run(summary_op, feed_dict={valid_accuracy: accuracy1, test_accuracy: accuracy2})
  213.             train_filewriter.add_summary(summary, step)
  214.         step += 1
  215.     except tf.errors.OutOfRangeError:
  216.         break

 

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/我家小花儿/article/detail/295659
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号