devbox
Cpp五条
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

从零开始构建SSD网络实现目标识别项目_ssd项目

作者：Cpp五条 | 2024-03-22 21:56:40

赞

踩

ssd项目

前言：

-- 由于此项目全过程过于繁杂，我前后做了三四个月，无法把所有内容融入这一篇文章之中，所以本文以逻辑串联为主，记录了我从零开始，构建vgg 300网络 -> 构建 SSD模型 -> 数据标注 -> 训练以及训练模型保存 -> 恢复模型进行预测 的全过程
-- 本文提供了一些核心代码，需要完整的代码，或者有任何问题的，可以私信我或者等候后续有时间再上传到github
-- 想读懂本文需要的基础：神经网络基础、CNN基础、VGG基础、语义分割基础、实例分割基础、YOLO原理等

一、SSD原理介绍

1、怎么理解SSD（此方法属于一步法）：
--two-stage方法，如R-CNN系算法，即是两步法，第一步选取候选框，第二步对这些候选框分类或者回归
--one-stage方法，如Yolo和SSD，即是一步法，其主要思路是均匀地在图片的不同位置进行密集抽样，抽样时可以采用不同尺度和长宽比，然后利用CNN提取特征后直接进行分类与回归，整个过程只需要一步，所以其优势是速度快
--one-stage的一个重要缺点是训练比较困难，这主要是因为正样本与负样本（背景）极其不均衡（参见Focal Loss），导致模型准确度稍低。

2、SSD与YOLO
-- SSD好过YOLO（此论点参考文章：https://zhuanlan.zhihu.com/p/68151917）
-- Yolo和SSD，其主要思路是均匀地在图片的不同位置进行密集抽样，抽样时可以采用不同尺度和长宽比，然后利用CNN提取特征后直接进行分类与回归，整个过程只需要一步，所以其优势是速度快
-- SSD和YOLO一样都是直接使用CNN进行分类和回归，但其比YOLO的优点在于：
   -- 其使用用不同stride的卷积核生成了多尺度的feature map来进行检测，使用大尺度的feature map来检测小物体，使用小尺度的feature map来检测大物体
   -- 提取了feature map之后，SSD直接使用conv进行检测，而YOLO是进过特征提取之后，生成了一个特征集合的张量，使用全连接层进行检测
   -- 设置先验框为基准框，每个单元会设置多个先验框，其尺度和长宽比存在差异，在一定程度上减少训练难度
   -- SSD原理与实现(参考文章：https://zhuanlan.zhihu.com/p/68151917)

3、SSD架构图
架构图如下：


关于网络后面部分逐渐减小的feature map
-- 网络后面大小不一的卷积层，其实都是作为分类器
-- 大分类器检测小目标，小分类器检测大目标

二、数据标注部分

1、数据标注工具
-- labelimg
-- 使用参考文档：https://blog.csdn.net/wsp_1138886114/article/details/85017498

2、目前主流的标注方式是以xml格式存储
其中重要的信息有：
-- filename：图片的文件名
-- name：标注的物体名称
-- xmin、ymin、xmax、ymax：物体位置的左上角、右下角坐标

3、自动划分数据的脚本
此脚本功能为：shuffle+训练集划分+测试集划分+验证集划分
脚本内容如下：

  1. import os
  2. import random
  3.  
  4. trainval_percent = 0.66
  5. train_percent = 0.5
  6. xmlfilepath = 'face_data\Annotations'
  7. txtsavepath = 'face_data\ImageSets\Main'
  8. total_xml = os.listdir(xmlfilepath)
  9.  
  10. num=len(total_xml)
  11. list=range(num)
  12. tv=int(num*trainval_percent)
  13. tr=int(tv*train_percent)
  14. trainval= random.sample(list,tv)
  15. train=random.sample(trainval,tr)
  16.  
  17. ftrainval = open('face_data/ImageSets/Main/trainval.txt', 'w')
  18. ftest = open('face_data/ImageSets/Main/test.txt', 'w')
  19. ftrain = open('face_data/ImageSets/Main/train.txt', 'w')
  20. fval = open('face_data/ImageSets/Main/val.txt', 'w')
  21.  
  22. for i  in list:
  23.     name=total_xml[i][:-4]+'\n'
  24.     if i in trainval:
  25.         ftrainval.write(name)
  26.         if i in train:
  27.             ftrain.write(name)
  28.         else:
  29.             fval.write(name)
  30.     else:
  31.         ftest.write(name)
  32.  
  33. ftrainval.close()
  34. ftrain.close()
  35. fval.close()
  36. ftest .close()

三、使用vgg网络作为主体结构搭建SSD模型

1、VGG网络原理

其最核心逻辑是使用卷积核分解的逻辑：
-- 用多个小卷积核分解一个大卷积核，对这部分内容的理解请参考我之前关于卷积核分解的文章；

采用连续的几个3x3的卷积核代替网络中的较大卷积核（11x11，7x7，5x5）。
-- 对于给定的感受野（与输出有关的输入图片的局部大小），采用堆积的小卷积核是优于采用大的卷积核，因为多层非线性层可以增加网络深度来保证学习更复杂的模式，而且代价还比较小（参数更少）。

2、构建vgg_300网络主体架构


3、使用vgg_300网络构建SSD模型

图中的VGG每一层主要实现：
-- 1、一张原始图片被resize到(224,224,3)。
-- 2、conv1两次[3,3]卷积网络，输出的特征层为64，输出为(224,224,64)，再2X2最大池化，输出net为(112,112,64)。
-- 3、conv2两次[3,3]卷积网络，输出的特征层为128，输出net为(112,112,128)，再2X2最大池化，输出net为(56,56,128)。
-- 4、conv3三次[3,3]卷积网络，输出的特征层为256，输出net为(56,56,256)，再2X2最大池化，输出net为(28,28,256)。
-- 5、conv3三次[3,3]卷积网络，输出的特征层为256，输出net为(28,28,512)，再2X2最大池化，输出net为(14,14,512)。
-- 6、conv3三次[3,3]卷积网络，输出的特征层为256，输出net为(14,14,512)，再2X2最大池化，输出net为(7,7,512)。
-- 7、利用卷积的方式模拟全连接层，效果等同，输出net为(1,1,4096)。共进行两次。
-- 8、利用卷积的方式模拟全连接层，效果等同，输出net为(1,1,1000)。

最后输出的就是每个类的预测。

主干网络代码：

  1. import math
  2. from collections import namedtuple
  3.  
  4. import numpy as np
  5. import tensorflow as tf
  6.  
  7. import tf_extended as tfe
  8. from nets import custom_layers
  9. from nets import ssd_common
  10.  
  11. slim = tf.contrib.slim
  12.  
  13.  
  14. # =========================================================================== #
  15. # SSD class definition.
  16. # =========================================================================== #
  17. SSDParams = namedtuple('SSDParameters', ['img_shape',
  18.                                          'num_classes',
  19.                                          'no_annotation_label',
  20.                                          'feat_layers',
  21.                                          'feat_shapes',
  22.                                          'anchor_size_bounds',
  23.                                          'anchor_sizes',
  24.                                          'anchor_ratios',
  25.                                          'anchor_steps',
  26.                                          'anchor_offset',
  27.                                          'normalizations',
  28.                                          'prior_scaling'
  29.                                          ])
  30.  
  31.  
  32. class SSDNet(object):
  33.     """Implementation of the SSD VGG-based 300 network.
  35.     The default features layers with 300x300 image input are:
  36.       conv4 ==> 38 x 38
  37.       conv7 ==> 19 x 19
  38.       conv8 ==> 10 x 10
  39.       conv9 ==> 5 x 5
  40.       conv10 ==> 3 x 3
  41.       conv11 ==> 1 x 1
  42.     The default image size used to train this network is 300x300.
  43.     """
  44.     default_params = SSDParams(
  45.         img_shape=(300, 300),
  46.         num_classes=21,
  47.         # num_classes=2,
  48.         no_annotation_label=21,
  49.         # no_annotation_label=2,
  50.         feat_layers=['block4', 'block7', 'block8', 'block9', 'block10', 'block11'],
  51.         feat_shapes=[(38, 38), (19, 19), (10, 10), (5, 5), (3, 3), (1, 1)],
  52.         anchor_size_bounds=[0.15, 0.90],
  53.         # anchor_size_bounds=[0.20, 0.90],
  54.         anchor_sizes=[(21., 45.),
  55.                       (45., 99.),
  56.                       (99., 153.),
  57.                       (153., 207.),
  58.                       (207., 261.),
  59.                       (261., 315.)],
  60.         # anchor_sizes=[(30., 60.),
  61.         #               (60., 111.),
  62.         #               (111., 162.),
  63.         #               (162., 213.),
  64.         #               (213., 264.),
  65.         #               (264., 315.)],
  66.         anchor_ratios=[[2, .5],
  67.                        [2, .5, 3, 1./3],
  68.                        [2, .5, 3, 1./3],
  69.                        [2, .5, 3, 1./3],
  70.                        [2, .5],
  71.                        [2, .5]],
  72.         anchor_steps=[8, 16, 32, 64, 100, 300],
  73.         anchor_offset=0.5,
  74.         normalizations=[20, -1, -1, -1, -1, -1],
  75.         prior_scaling=[0.1, 0.1, 0.2, 0.2]
  76.         )
  77.  
  78.     def __init__(self, params=None):
  79.         """Init the SSD net with some parameters. Use the default ones
  80.         if none provided.
  81.         """
  82.         if isinstance(params, SSDParams):
  83.             self.params = params
  84.         else:
  85.             self.params = SSDNet.default_params
  86.  
  87.     # ======================================================================= #
  88.     def net(self, inputs,
  89.             is_training=True,
  90.             update_feat_shapes=True,
  91.             dropout_keep_prob=0.5,
  92.             prediction_fn=slim.softmax,
  93.             reuse=None,
  94.             scope='ssd_300_vgg'):
  95.         """SSD network definition.
  96.         """
  97.         r = ssd_net(inputs,
  98.                     num_classes=self.params.num_classes,
  99.                     feat_layers=self.params.feat_layers,
  100.                     anchor_sizes=self.params.anchor_sizes,
  101.                     anchor_ratios=self.params.anchor_ratios,
  102.                     normalizations=self.params.normalizations,
  103.                     is_training=is_training,
  104.                     dropout_keep_prob=dropout_keep_prob,
  105.                     prediction_fn=prediction_fn,
  106.                     reuse=reuse,
  107.                     scope=scope)
  108.         # Update feature shapes (try at least!)
  109.         if update_feat_shapes:
  110.             shapes = ssd_feat_shapes_from_net(r[0], self.params.feat_shapes)
  111.             self.params = self.params._replace(feat_shapes=shapes)
  112.         return r
  113.  
  114.     def arg_scope(self, weight_decay=0.0005, data_format='NHWC'):
  115.         """Network arg_scope.
  116.         """
  117.         return ssd_arg_scope(weight_decay, data_format=data_format)
  118.  
  119.     def arg_scope_caffe(self, caffe_scope):
  120.         """Caffe arg_scope used for weights importing.
  121.         """
  122.         return ssd_arg_scope_caffe(caffe_scope)
  123.  
  124.     # ======================================================================= #
  125.     def update_feature_shapes(self, predictions):
  126.         """Update feature shapes from predictions collection (Tensor or Numpy
  127.         array).
  128.         """
  129.         shapes = ssd_feat_shapes_from_net(predictions, self.params.feat_shapes)
  130.         self.params = self.params._replace(feat_shapes=shapes)
  131.  
  132.     def anchors(self, img_shape, dtype=np.float32):
  133.         """Compute the default anchor boxes, given an image shape.
  134.         """
  135.         return ssd_anchors_all_layers(img_shape,
  136.                                       self.params.feat_shapes,
  137.                                       self.params.anchor_sizes,
  138.                                       self.params.anchor_ratios,
  139.                                       self.params.anchor_steps,
  140.                                       self.params.anchor_offset,
  141.                                       dtype)
  142.  
  143.     def bboxes_encode(self, labels, bboxes, anchors,
  144.                       scope=None):
  145.         """Encode labels and bounding boxes.
  146.         """
  147.         return ssd_common.tf_ssd_bboxes_encode(
  148.             labels, bboxes, anchors,
  149.             self.params.num_classes,
  150.             self.params.no_annotation_label,
  151.             ignore_threshold=0.5,
  152.             prior_scaling=self.params.prior_scaling,
  153.             scope=scope)
  154.  
  155.     def bboxes_decode(self, feat_localizations, anchors,
  156.                       scope='ssd_bboxes_decode'):
  157.         """Encode labels and bounding boxes.
  158.         """
  159.         return ssd_common.tf_ssd_bboxes_decode(
  160.             feat_localizations, anchors,
  161.             prior_scaling=self.params.prior_scaling,
  162.             scope=scope)
  163.  
  164.     def detected_bboxes(self, predictions, localisations,
  165.                         select_threshold=None, nms_threshold=0.5,
  166.                         clipping_bbox=None, top_k=400, keep_top_k=200):
  167.         """Get the detected bounding boxes from the SSD network output.
  168.         """
  169.         # Select top_k bboxes from predictions, and clip
  170.         rscores, rbboxes = \
  171.             ssd_common.tf_ssd_bboxes_select(predictions, localisations,
  172.                                             select_threshold=select_threshold,
  173.                                             num_classes=self.params.num_classes)
  174.         rscores, rbboxes = \
  175.             tfe.bboxes_sort(rscores, rbboxes, top_k=top_k)
  176.         # Apply NMS algorithm.
  177.         rscores, rbboxes = \
  178.             tfe.bboxes_nms_batch(rscores, rbboxes,
  179.                                  nms_threshold=nms_threshold,
  180.                                  keep_top_k=keep_top_k)
  181.         if clipping_bbox is not None:
  182.             rbboxes = tfe.bboxes_clip(clipping_bbox, rbboxes)
  183.         return rscores, rbboxes
  184.  
  185.     def losses(self, logits, localisations,
  186.                gclasses, glocalisations, gscores,
  187.                match_threshold=0.5,
  188.                negative_ratio=3.,
  189.                alpha=1.,
  190.                label_smoothing=0.,
  191.                scope='ssd_losses'):
  192.         """Define the SSD network losses.
  193.         """
  194.         return ssd_losses(logits, localisations,
  195.                           gclasses, glocalisations, gscores,
  196.                           match_threshold=match_threshold,
  197.                           negative_ratio=negative_ratio,
  198.                           alpha=alpha,
  199.                           label_smoothing=label_smoothing,
  200.                           scope=scope)
  201.  
  202.  
  203. # =========================================================================== #
  204. # SSD tools...
  205. # =========================================================================== #
  206. def ssd_size_bounds_to_values(size_bounds,
  207.                               n_feat_layers,
  208.                               img_shape=(300, 300)):
  209.     """Compute the reference sizes of the anchor boxes from relative bounds.
  210.     The absolute values are measured in pixels, based on the network
  211.     default size (300 pixels).
  213.     This function follows the computation performed in the original
  214.     implementation of SSD in Caffe.
  216.     Return:
  217.       list of list containing the absolute sizes at each scale. For each scale,
  218.       the ratios only apply to the first value.
  219.     """
  220.     assert img_shape[0] == img_shape[1]
  221.  
  222.     img_size = img_shape[0]
  223.     min_ratio = int(size_bounds[0] * 100)
  224.     max_ratio = int(size_bounds[1] * 100)
  225.     step = int(math.floor((max_ratio - min_ratio) / (n_feat_layers - 2)))
  226.     # Start with the following smallest sizes.
  227.     sizes = [[img_size * size_bounds[0] / 2, img_size * size_bounds[0]]]
  228.     for ratio in range(min_ratio, max_ratio + 1, step):
  229.         sizes.append((img_size * ratio / 100.,
  230.                       img_size * (ratio + step) / 100.))
  231.     return sizes
  232.  
  233.  
  234. def ssd_feat_shapes_from_net(predictions, default_shapes=None):
  235.     """Try to obtain the feature shapes from the prediction layers. The latter
  236.     can be either a Tensor or Numpy ndarray.
  238.     Return:
  239.       list of feature shapes. Default values if predictions shape not fully
  240.       determined.
  241.     """
  242.     feat_shapes = []
  243.     for l in predictions:
  244.         # Get the shape, from either a np array or a tensor.
  245.         if isinstance(l, np.ndarray):
  246.             shape = l.shape
  247.         else:
  248.             shape = l.get_shape().as_list()
  249.         shape = shape[1:4]
  250.         # Problem: undetermined shape...
  251.         if None in shape:
  252.             return default_shapes
  253.         else:
  254.             feat_shapes.append(shape)
  255.     return feat_shapes
  256.  
  257.  
  258. def ssd_anchor_one_layer(img_shape,
  259.                          feat_shape,
  260.                          sizes,
  261.                          ratios,
  262.                          step,
  263.                          offset=0.5,
  264.                          dtype=np.float32):
  265.     """Computer SSD default anchor boxes for one feature layer.
  267.     Determine the relative position grid of the centers, and the relative
  268.     width and height.
  270.     Arguments:
  271.       feat_shape: Feature shape, used for computing relative position grids;
  272.       size: Absolute reference sizes;
  273.       ratios: Ratios to use on these features;
  274.       img_shape: Image shape, used for computing height, width relatively to the
  275.         former;
  276.       offset: Grid offset.
  278.     Return:
  279.       y, x, h, w: Relative x and y grids, and height and width.
  280.     """
  281.     # Compute the position grid: simple way.
  282.     # y, x = np.mgrid[0:feat_shape[0], 0:feat_shape[1]]
  283.     # y = (y.astype(dtype) + offset) / feat_shape[0]
  284.     # x = (x.astype(dtype) + offset) / feat_shape[1]
  285.     # Weird SSD-Caffe computation using steps values...
  286.     y, x = np.mgrid[0:feat_shape[0], 0:feat_shape[1]]
  287.     y = (y.astype(dtype) + offset) * step / img_shape[0]
  288.     x = (x.astype(dtype) + offset) * step / img_shape[1]
  289.  
  290.     # Expand dims to support easy broadcasting.
  291.     y = np.expand_dims(y, axis=-1)
  292.     x = np.expand_dims(x, axis=-1)
  293.  
  294.     # Compute relative height and width.
  295.     # Tries to follow the original implementation of SSD for the order.
  296.     num_anchors = len(sizes) + len(ratios)
  297.     h = np.zeros((num_anchors, ), dtype=dtype)
  298.     w = np.zeros((num_anchors, ), dtype=dtype)
  299.     # Add first anchor boxes with ratio=1.
  300.     h[0] = sizes[0] / img_shape[0]
  301.     w[0] = sizes[0] / img_shape[1]
  302.     di = 1
  303.     if len(sizes) > 1:
  304.         h[1] = math.sqrt(sizes[0] * sizes[1]) / img_shape[0]
  305.         w[1] = math.sqrt(sizes[0] * sizes[1]) / img_shape[1]
  306.         di += 1
  307.     for i, r in enumerate(ratios):
  308.         h[i+di] = sizes[0] / img_shape[0] / math.sqrt(r)
  309.         w[i+di] = sizes[0] / img_shape[1] * math.sqrt(r)
  310.     return y, x, h, w
  311.  
  312.  
  313. def ssd_anchors_all_layers(img_shape,
  314.                            layers_shape,
  315.                            anchor_sizes,
  316.                            anchor_ratios,
  317.                            anchor_steps,
  318.                            offset=0.5,
  319.                            dtype=np.float32):
  320.     """Compute anchor boxes for all feature layers.
  321.     """
  322.     layers_anchors = []
  323.     for i, s in enumerate(layers_shape):
  324.         anchor_bboxes = ssd_anchor_one_layer(img_shape, s,
  325.                                              anchor_sizes[i],
  326.                                              anchor_ratios[i],
  327.                                              anchor_steps[i],
  328.                                              offset=offset, dtype=dtype)
  329.         layers_anchors.append(anchor_bboxes)
  330.     return layers_anchors
  331.  
  332.  
  333. # =========================================================================== #
  334. # Functional definition of VGG-based SSD 300.
  335. # =========================================================================== #
  336. def tensor_shape(x, rank=3):
  337.     """Returns the dimensions of a tensor.
  338.     Args:
  339.       image: A N-D Tensor of shape.
  340.     Returns:
  341.       A list of dimensions. Dimensions that are statically known are python
  342.         integers,otherwise they are integer scalar tensors.
  343.     """
  344.     if x.get_shape().is_fully_defined():
  345.         return x.get_shape().as_list()
  346.     else:
  347.         static_shape = x.get_shape().with_rank(rank).as_list()
  348.         dynamic_shape = tf.unstack(tf.shape(x), rank)
  349.         return [s if s is not None else d
  350.                 for s, d in zip(static_shape, dynamic_shape)]
  351.  
  352.  
  353. def ssd_multibox_layer(inputs,
  354.                        num_classes,
  355.                        sizes,
  356.                        ratios=[1],
  357.                        normalization=-1,
  358.                        bn_normalization=False):
  359.     """Construct a multibox layer, return a class and localization predictions.
  360.     """
  361.     net = inputs
  362.     if normalization > 0:
  363.         net = custom_layers.l2_normalization(net, scaling=True)
  364.     # Number of anchors.
  365.     num_anchors = len(sizes) + len(ratios)
  366.  
  367.     # Location.
  368.     num_loc_pred = num_anchors * 4
  369.     loc_pred = slim.conv2d(net, num_loc_pred, [3, 3], activation_fn=None,
  370.                            scope='conv_loc')
  371.     loc_pred = custom_layers.channel_to_last(loc_pred)
  372.     loc_pred = tf.reshape(loc_pred,
  373.                           tensor_shape(loc_pred, 4)[:-1]+[num_anchors, 4])
  374.     # Class prediction.
  375.     num_cls_pred = num_anchors * num_classes
  376.     cls_pred = slim.conv2d(net, num_cls_pred, [3, 3], activation_fn=None,
  377.                            scope='conv_cls')
  378.     cls_pred = custom_layers.channel_to_last(cls_pred)
  379.     cls_pred = tf.reshape(cls_pred,
  380.                           tensor_shape(cls_pred, 4)[:-1]+[num_anchors, num_classes])
  381.     return cls_pred, loc_pred
  382.  
  383.  
  384. def ssd_net(inputs,
  385.             num_classes=SSDNet.default_params.num_classes,
  386.             feat_layers=SSDNet.default_params.feat_layers,
  387.             anchor_sizes=SSDNet.default_params.anchor_sizes,
  388.             anchor_ratios=SSDNet.default_params.anchor_ratios,
  389.             normalizations=SSDNet.default_params.normalizations,
  390.             is_training=True,
  391.             dropout_keep_prob=0.5,
  392.             prediction_fn=slim.softmax,
  393.             reuse=None,
  394.             scope='ssd_300_vgg'):
  395.     """SSD net definition.
  396.     """
  397.     # if data_format == 'NCHW':
  398.     #     inputs = tf.transpose(inputs, perm=(0, 3, 1, 2))
  399.  
  400.     # End_points collect relevant activations for external use.
  401.     end_points = {}
  402.     with tf.variable_scope(scope, 'ssd_300_vgg', [inputs], reuse=reuse):
  403.         # Original VGG-16 blocks.
  404.         net = slim.repeat(inputs, 2, slim.conv2d, 64, [3, 3], scope='conv1')
  405.         end_points['block1'] = net
  406.         net = slim.max_pool2d(net, [2, 2], scope='pool1')
  407.         # Block 2.
  408.         net = slim.repeat(net, 2, slim.conv2d, 128, [3, 3], scope='conv2')
  409.         end_points['block2'] = net
  410.         net = slim.max_pool2d(net, [2, 2], scope='pool2')
  411.         # Block 3.
  412.         net = slim.repeat(net, 3, slim.conv2d, 256, [3, 3], scope='conv3')
  413.         end_points['block3'] = net
  414.         net = slim.max_pool2d(net, [2, 2], scope='pool3')
  415.         # Block 4.
  416.         net = slim.repeat(net, 3, slim.conv2d, 512, [3, 3], scope='conv4')
  417.         end_points['block4'] = net
  418.         net = slim.max_pool2d(net, [2, 2], scope='pool4')
  419.         # Block 5.
  420.         net = slim.repeat(net, 3, slim.conv2d, 512, [3, 3], scope='conv5')
  421.         end_points['block5'] = net
  422.         net = slim.max_pool2d(net, [3, 3], stride=1, scope='pool5')
  423.  
  424.         # Additional SSD blocks.
  425.         # Block 6: let's dilate the hell out of it!
  426.         net = slim.conv2d(net, 1024, [3, 3], rate=6, scope='conv6')
  427.         end_points['block6'] = net
  428.         net = tf.layers.dropout(net, rate=dropout_keep_prob, training=is_training)
  429.         # Block 7: 1x1 conv. Because the fuck.
  430.         net = slim.conv2d(net, 1024, [1, 1], scope='conv7')
  431.         end_points['block7'] = net
  432.         net = tf.layers.dropout(net, rate=dropout_keep_prob, training=is_training)
  433.  
  434.         # Block 8/9/10/11: 1x1 and 3x3 convolutions stride 2 (except lasts).
  435.         end_point = 'block8'
  436.         with tf.variable_scope(end_point):
  437.             net = slim.conv2d(net, 256, [1, 1], scope='conv1x1')
  438.             net = custom_layers.pad2d(net, pad=(1, 1))
  439.             net = slim.conv2d(net, 512, [3, 3], stride=2, scope='conv3x3', padding='VALID')
  440.         end_points[end_point] = net
  441.         end_point = 'block9'
  442.         with tf.variable_scope(end_point):
  443.             net = slim.conv2d(net, 128, [1, 1], scope='conv1x1')
  444.             net = custom_layers.pad2d(net, pad=(1, 1))
  445.             net = slim.conv2d(net, 256, [3, 3], stride=2, scope='conv3x3', padding='VALID')
  446.         end_points[end_point] = net
  447.         end_point = 'block10'
  448.         with tf.variable_scope(end_point):
  449.             net = slim.conv2d(net, 128, [1, 1], scope='conv1x1')
  450.             net = slim.conv2d(net, 256, [3, 3], scope='conv3x3', padding='VALID')
  451.         end_points[end_point] = net
  452.         end_point = 'block11'
  453.         with tf.variable_scope(end_point):
  454.             net = slim.conv2d(net, 128, [1, 1], scope='conv1x1')
  455.             net = slim.conv2d(net, 256, [3, 3], scope='conv3x3', padding='VALID')
  456.         end_points[end_point] = net
  457.  
  458.         # Prediction and localisations layers.
  459.         predictions = []
  460.         logits = []
  461.         localisations = []
  462.         for i, layer in enumerate(feat_layers):
  463.             with tf.variable_scope(layer + '_box'):
  464.                 p, l = ssd_multibox_layer(end_points[layer],
  465.                                           num_classes,
  466.                                           anchor_sizes[i],
  467.                                           anchor_ratios[i],
  468.                                           normalizations[i])
  469.             predictions.append(prediction_fn(p))
  470.             logits.append(p)
  471.             localisations.append(l)
  472.  
  473.         return predictions, localisations, logits, end_points
  474. ssd_net.default_image_size = 300
  475.  
  476.  
  477. def ssd_arg_scope(weight_decay=0.0005, data_format='NHWC'):
  478.     """Defines the VGG arg scope.
  480.     Args:
  481.       weight_decay: The l2 regularization coefficient.
  483.     Returns:
  484.       An arg_scope.
  485.     """
  486.     with slim.arg_scope([slim.conv2d, slim.fully_connected],
  487.                         activation_fn=tf.nn.relu,
  488.                         weights_regularizer=slim.l2_regularizer(weight_decay),
  489.                         weights_initializer=tf.contrib.layers.xavier_initializer(),
  490.                         biases_initializer=tf.zeros_initializer()):
  491.         with slim.arg_scope([slim.conv2d, slim.max_pool2d],
  492.                             padding='SAME',
  493.                             data_format=data_format):
  494.             with slim.arg_scope([custom_layers.pad2d,
  495.                                  custom_layers.l2_normalization,
  496.                                  custom_layers.channel_to_last],
  497.                                 data_format=data_format) as sc:
  498.                 return sc
  499.  
  500.  
  501. # =========================================================================== #
  502. # Caffe scope: importing weights at initialization.
  503. # =========================================================================== #
  504. def ssd_arg_scope_caffe(caffe_scope):
  505.     """Caffe scope definition.
  507.     Args:
  508.       caffe_scope: Caffe scope object with loaded weights.
  510.     Returns:
  511.       An arg_scope.
  512.     """
  513.     # Default network arg scope.
  514.     with slim.arg_scope([slim.conv2d],
  515.                         activation_fn=tf.nn.relu,
  516.                         weights_initializer=caffe_scope.conv_weights_init(),
  517.                         biases_initializer=caffe_scope.conv_biases_init()):
  518.         with slim.arg_scope([slim.fully_connected],
  519.                             activation_fn=tf.nn.relu):
  520.             with slim.arg_scope([custom_layers.l2_normalization],
  521.                                 scale_initializer=caffe_scope.l2_norm_scale_init()):
  522.                 with slim.arg_scope([slim.conv2d, slim.max_pool2d],
  523.                                     padding='SAME') as sc:
  524.                     return sc
  525.  
  526.  
  527. # =========================================================================== #
  528. # SSD loss function.
  529. # =========================================================================== #
  530. def ssd_losses(logits, localisations,
  531.                gclasses, glocalisations, gscores,
  532.                match_threshold=0.5,
  533.                negative_ratio=3.,
  534.                alpha=1.,
  535.                label_smoothing=0.,
  536.                device='/cpu:0',
  537.                scope=None):
  538.     with tf.name_scope(scope, 'ssd_losses'):
  539.         lshape = tfe.get_shape(logits[0], 5)
  540.         num_classes = lshape[-1]
  541.         batch_size = lshape[0]
  542.  
  543.         # Flatten out all vectors!
  544.         flogits = []
  545.         fgclasses = []
  546.         fgscores = []
  547.         flocalisations = []
  548.         fglocalisations = []
  549.         for i in range(len(logits)):
  550.             flogits.append(tf.reshape(logits[i], [-1, num_classes]))
  551.             fgclasses.append(tf.reshape(gclasses[i], [-1]))
  552.             fgscores.append(tf.reshape(gscores[i], [-1]))
  553.             flocalisations.append(tf.reshape(localisations[i], [-1, 4]))
  554.             fglocalisations.append(tf.reshape(glocalisations[i], [-1, 4]))
  555.         # And concat the crap!
  556.         logits = tf.concat(flogits, axis=0)
  557.         gclasses = tf.concat(fgclasses, axis=0)
  558.         gscores = tf.concat(fgscores, axis=0)
  559.         localisations = tf.concat(flocalisations, axis=0)
  560.         glocalisations = tf.concat(fglocalisations, axis=0)
  561.         dtype = logits.dtype
  562.  
  563.         # Compute positive matching mask...
  564.         pmask = gscores > match_threshold
  565.         fpmask = tf.cast(pmask, dtype)
  566.         n_positives = tf.reduce_sum(fpmask)
  567.  
  568.         # Hard negative mining...
  569.         no_classes = tf.cast(pmask, tf.int32)
  570.         predictions = slim.softmax(logits)
  571.         nmask = tf.logical_and(tf.logical_not(pmask),
  572.                                gscores > -0.5)
  573.         fnmask = tf.cast(nmask, dtype)
  574.         nvalues = tf.where(nmask,
  575.                            predictions[:, 0],
  576.                            1. - fnmask)
  577.         nvalues_flat = tf.reshape(nvalues, [-1])
  578.         # Number of negative entries to select.
  579.         max_neg_entries = tf.cast(tf.reduce_sum(fnmask), tf.int32)
  580.         n_neg = tf.cast(negative_ratio * n_positives, tf.int32) + batch_size
  581.         n_neg = tf.minimum(n_neg, max_neg_entries)
  582.  
  583.         val, idxes = tf.nn.top_k(-nvalues_flat, k=n_neg)
  584.         max_hard_pred = -val[-1]
  585.         # Final negative mask.
  586.         nmask = tf.logical_and(nmask, nvalues < max_hard_pred)
  587.         fnmask = tf.cast(nmask, dtype)
  588.  
  589.         # Add cross-entropy loss.
  590.         with tf.name_scope('cross_entropy_pos'):
  591.             loss = tf.nn.sparse_softmax_cross_entropy_with_logits(logits=logits,
  592.                                                                   labels=gclasses)
  593.             loss = tf.div(tf.reduce_sum(loss * fpmask), batch_size, name='value')
  594.             tf.losses.add_loss(loss)
  595.  
  596.         with tf.name_scope('cross_entropy_neg'):
  597.             loss = tf.nn.sparse_softmax_cross_entropy_with_logits(logits=logits,
  598.                                                                   labels=no_classes)
  599.             loss = tf.div(tf.reduce_sum(loss * fnmask), batch_size, name='value')
  600.             tf.losses.add_loss(loss)
  601.  
  602.         # Add localization loss: smooth L1, L2, ...
  603.         with tf.name_scope('localization'):
  604.             # Weights Tensor: positive mask + random negative.
  605.             weights = tf.expand_dims(alpha * fpmask, axis=-1)
  606.             loss = custom_layers.abs_smooth(localisations - glocalisations)
  607.             loss = tf.div(tf.reduce_sum(loss * weights), batch_size, name='value')
  608.             tf.losses.add_loss(loss)
  609.  
  610.  
  611. def ssd_losses_old(logits, localisations,
  612.                    gclasses, glocalisations, gscores,
  613.                    match_threshold=0.5,
  614.                    negative_ratio=3.,
  615.                    alpha=1.,
  616.                    label_smoothing=0.,
  617.                    device='/cpu:0',
  618.                    scope=None):
  619.     """Loss functions for training the SSD 300 VGG network.
  621.     This function defines the different loss components of the SSD, and
  622.     adds them to the TF loss collection.
  624.     Arguments:
  625.       logits: (list of) predictions logits Tensors;
  626.       localisations: (list of) localisations Tensors;
  627.       gclasses: (list of) groundtruth labels Tensors;
  628.       glocalisations: (list of) groundtruth localisations Tensors;
  629.       gscores: (list of) groundtruth score Tensors;
  630.     """
  631.     with tf.device(device):
  632.         with tf.name_scope(scope, 'ssd_losses'):
  633.             l_cross_pos = []
  634.             l_cross_neg = []
  635.             l_loc = []
  636.             for i in range(len(logits)):
  637.                 dtype = logits[i].dtype
  638.                 with tf.name_scope('block_%i' % i):
  639.                     # Sizing weight...
  640.                     wsize = tfe.get_shape(logits[i], rank=5)
  641.                     wsize = wsize[1] * wsize[2] * wsize[3]
  642.  
  643.                     # Positive mask.
  644.                     pmask = gscores[i] > match_threshold
  645.                     fpmask = tf.cast(pmask, dtype)
  646.                     n_positives = tf.reduce_sum(fpmask)
  647.  
  648.                     # Select some random negative entries.
  649.                     # n_entries = np.prod(gclasses[i].get_shape().as_list())
  650.                     # r_positive = n_positives / n_entries
  651.                     # r_negative = negative_ratio * n_positives / (n_entries - n_positives)
  652.  
  653.                     # Negative mask.
  654.                     no_classes = tf.cast(pmask, tf.int32)
  655.                     predictions = slim.softmax(logits[i])
  656.                     nmask = tf.logical_and(tf.logical_not(pmask),
  657.                                            gscores[i] > -0.5)
  658.                     fnmask = tf.cast(nmask, dtype)
  659.                     nvalues = tf.where(nmask,
  660.                                        predictions[:, :, :, :, 0],
  661.                                        1. - fnmask)
  662.                     nvalues_flat = tf.reshape(nvalues, [-1])
  663.                     # Number of negative entries to select.
  664.                     n_neg = tf.cast(negative_ratio * n_positives, tf.int32)
  665.                     n_neg = tf.maximum(n_neg, tf.size(nvalues_flat) // 8)
  666.                     n_neg = tf.maximum(n_neg, tf.shape(nvalues)[0] * 4)
  667.                     max_neg_entries = 1 + tf.cast(tf.reduce_sum(fnmask), tf.int32)
  668.                     n_neg = tf.minimum(n_neg, max_neg_entries)
  669.  
  670.                     val, idxes = tf.nn.top_k(-nvalues_flat, k=n_neg)
  671.                     max_hard_pred = -val[-1]
  672.                     # Final negative mask.
  673.                     nmask = tf.logical_and(nmask, nvalues < max_hard_pred)
  674.                     fnmask = tf.cast(nmask, dtype)
  675.  
  676.                     # Add cross-entropy loss.
  677.                     with tf.name_scope('cross_entropy_pos'):
  678.                         fpmask = wsize * fpmask
  679.                         loss = tf.nn.sparse_softmax_cross_entropy_with_logits(logits=logits[i],
  680.                                                                               labels=gclasses[i])
  681.                         loss = tf.losses.compute_weighted_loss(loss, fpmask)
  682.                         l_cross_pos.append(loss)
  683.  
  684.                     with tf.name_scope('cross_entropy_neg'):
  685.                         fnmask = wsize * fnmask
  686.                         loss = tf.nn.sparse_softmax_cross_entropy_with_logits(logits=logits[i],
  687.                                                                               labels=no_classes)
  688.                         loss = tf.losses.compute_weighted_loss(loss, fnmask)
  689.                         l_cross_neg.append(loss)
  690.  
  691.                     # Add localization loss: smooth L1, L2, ...
  692.                     with tf.name_scope('localization'):
  693.                         # Weights Tensor: positive mask + random negative.
  694.                         weights = tf.expand_dims(alpha * fpmask, axis=-1)
  695.                         loss = custom_layers.abs_smooth(localisations[i] - glocalisations[i])
  696.                         loss = tf.losses.compute_weighted_loss(loss, weights)
  697.                         l_loc.append(loss)
  698.  
  699.             # Additional total losses...
  700.             with tf.name_scope('total'):
  701.                 total_cross_pos = tf.add_n(l_cross_pos, 'cross_entropy_pos')
  702.                 total_cross_neg = tf.add_n(l_cross_neg, 'cross_entropy_neg')
  703.                 total_cross = tf.add(total_cross_pos, total_cross_neg, 'cross_entropy')
  704.                 total_loc = tf.add_n(l_loc, 'localization')
  705.  
  706.                 # Add to EXTRA LOSSES TF.collection
  707.                 tf.add_to_collection('EXTRA_LOSSES', total_cross_pos)
  708.                 tf.add_to_collection('EXTRA_LOSSES', total_cross_neg)
  709.                 tf.add_to_collection('EXTRA_LOSSES', total_cross)
  710.                 tf.add_to_collection('EXTRA_LOSSES', total_loc)

四、将标注完的数据类型转换为网络可接受的格式

转换代码如下：

  1. import tensorflow as tf
  2.  
  3. from datasets import pascalvoc_to_tfrecords
  4.  
  5. FLAGS = tf.app.flags.FLAGS
  6.  
  7. tf.app.flags.DEFINE_string(
  8.     'dataset_name', 'pascalvoc',
  9.     'The name of the dataset to convert.')
  10. tf.app.flags.DEFINE_string(
  11.     'dataset_dir', './face_data/',
  12.     'Directory where the original dataset is stored.')
  13. tf.app.flags.DEFINE_string(
  14.     'output_name', 'voc_2007_train',
  15.     'Basename used for TFRecords output files.')
  16. tf.app.flags.DEFINE_string(
  17.     'output_dir', './face_tfrecord/',
  18.     'Output directory where to store TFRecords files.')
  19.  
  20.  
  21. def main(_):
  22.     if not FLAGS.dataset_dir:
  23.         raise ValueError('You must supply the dataset directory with --dataset_dir')
  24.     print('Dataset directory:', FLAGS.dataset_dir)
  25.     print('Output directory:', FLAGS.output_dir)
  26.  
  27.     if FLAGS.dataset_name == 'pascalvoc':
  28.         pascalvoc_to_tfrecords.run(FLAGS.dataset_dir, FLAGS.output_dir, FLAGS.output_name)
  29.     else:
  30.         raise ValueError('Dataset [%s] was not recognized.' % FLAGS.dataset_name)
  31.  
  32. if __name__ == '__main__':
  33.     tf.app.run()
  34.

五、训练程序设计

训练代码如下：

  1. import tensorflow as tf
  2. from tensorflow.python.ops import control_flow_ops
  3.  
  4. from datasets import dataset_factory
  5. from deployment import model_deploy
  6. from nets import nets_factory
  7. from preprocessing import preprocessing_factory
  8. import tf_utils
  9. tf.reset_default_graph()
  10. import os
  11. # os.environ['CUDA_VISIBLE_DEVICES']='0'
  12. slim = tf.contrib.slim
  13.  
  14. DATA_FORMAT = 'NHWC'
  15. # DATA_FORMAT = 'NCHW'
  16.  
  17. # =========================================================================== #
  18. # SSD Network flags.
  19. # =========================================================================== #
  20. tf.app.flags.DEFINE_float(
  21.     'loss_alpha', 1., 'Alpha parameter in the loss function.')
  22. tf.app.flags.DEFINE_float(
  23.     'negative_ratio', 3., 'Negative ratio in the loss function.')
  24. tf.app.flags.DEFINE_float(
  25.     'match_threshold', 0.5, 'Matching threshold in the loss function.')
  26.  
  27. # =========================================================================== #
  28. # General Flags.
  29. # =========================================================================== #
  30. tf.app.flags.DEFINE_string(
  31.     'train_dir', './wuyang_model/',
  32.     'Directory where checkpoints and event logs are written to.')
  33. tf.app.flags.DEFINE_integer('num_clones', 1,
  34.                             'Number of model clones to deploy.')
  35. tf.app.flags.DEFINE_boolean('clone_on_cpu', False,#True
  36.                             'Use CPUs to deploy clones.')
  37. tf.app.flags.DEFINE_integer(
  38.     'num_readers', 4,
  39.     'The number of parallel readers that read data from the dataset.')
  40. tf.app.flags.DEFINE_integer(
  41.     'num_preprocessing_threads', 4,
  42.     'The number of threads used to create the batches.')
  43.  
  44. tf.app.flags.DEFINE_integer(
  45.     'log_every_n_steps', 1,#10
  46.     'The frequency with which logs are print.')
  47. tf.app.flags.DEFINE_integer(
  48.     'save_summaries_secs', 600,
  49.     'The frequency with which summaries are saved, in seconds.')
  50. tf.app.flags.DEFINE_integer(
  51.     'save_interval_secs', 600,
  52.     'The frequency with which the model is saved, in seconds.')
  53. tf.app.flags.DEFINE_float(
  54.     'gpu_memory_fraction', 0.8, 'GPU memory fraction to use.')
  55.  
  56. # =========================================================================== #
  57. # Optimization Flags.
  58. # =========================================================================== #
  59. tf.app.flags.DEFINE_float(
  60.     'weight_decay', 0.4, 'The weight decay on the model weights.')#0.00004
  61. tf.app.flags.DEFINE_string(
  62.     'optimizer', 'rmsprop',
  63.     'The name of the optimizer, one of "adadelta", "adagrad", "adam",'
  64.     '"ftrl", "momentum", "sgd" or "rmsprop".')
  65. tf.app.flags.DEFINE_float(
  66.     'adadelta_rho', 0.95,
  67.     'The decay rate for adadelta.')
  68. tf.app.flags.DEFINE_float(
  69.     'adagrad_initial_accumulator_value', 0.01,
  70.     'Starting value for the AdaGrad accumulators.')
  71. tf.app.flags.DEFINE_float(
  72.     'adam_beta1', 0.9,
  73.     'The exponential decay rate for the 1st moment estimates.')
  74. tf.app.flags.DEFINE_float(
  75.     'adam_beta2', 0.999,
  76.     'The exponential decay rate for the 2nd moment estimates.')
  77. tf.app.flags.DEFINE_float('opt_epsilon', 1.0, 'Epsilon term for the optimizer.')
  78. tf.app.flags.DEFINE_float('ftrl_learning_rate_power', -0.5,
  79.                           'The learning rate power.')
  80. tf.app.flags.DEFINE_float(
  81.     'ftrl_initial_accumulator_value', 0.1,
  82.     'Starting value for the FTRL accumulators.')
  83. tf.app.flags.DEFINE_float(
  84.     'ftrl_l1', 0.0, 'The FTRL l1 regularization strength.')
  85. tf.app.flags.DEFINE_float(
  86.     'ftrl_l2', 0.0, 'The FTRL l2 regularization strength.')
  87. tf.app.flags.DEFINE_float(
  88.     'momentum', 0.9,
  89.     'The momentum for the MomentumOptimizer and RMSPropOptimizer.')
  90. tf.app.flags.DEFINE_float('rmsprop_momentum', 0.9, 'Momentum.')
  91. tf.app.flags.DEFINE_float('rmsprop_decay', 0.9, 'Decay term for RMSProp.')
  92.  
  93. # =========================================================================== #
  94. # Learning Rate Flags.
  95. # =========================================================================== #
  96. tf.app.flags.DEFINE_string(
  97.     'learning_rate_decay_type',
  98.     'exponential',
  99.     'Specifies how the learning rate is decayed. One of "fixed", "exponential",'
  100.     ' or "polynomial"')
  101. tf.app.flags.DEFINE_float('learning_rate', 0.0001, 'Initial learning rate.')#0.01
  102. tf.app.flags.DEFINE_float(
  103.     'end_learning_rate', 0.01,# 0.0001
  104.     'The minimal end learning rate used by a polynomial decay learning rate.')
  105. tf.app.flags.DEFINE_float(
  106.     'label_smoothing', 0.0, 'The amount of label smoothing.')
  107. tf.app.flags.DEFINE_float(
  108.     'learning_rate_decay_factor', 0.94, 'Learning rate decay factor.')
  109. tf.app.flags.DEFINE_float(
  110.     'num_epochs_per_decay', 2.0,
  111.     'Number of epochs after which learning rate decays.')
  112. tf.app.flags.DEFINE_float(
  113.     'moving_average_decay', None,
  114.     'The decay to use for the moving average.'
  115.     'If left as None, then moving averages are not used.')
  116.  
  117. # =========================================================================== #
  118. # Dataset Flags.
  119. # =========================================================================== #
  120. tf.app.flags.DEFINE_string(
  121.     'dataset_name', 'pascalvoc_2007', 'The name of the dataset to load.')
  122. tf.app.flags.DEFINE_integer(
  123.     'num_classes', 2, 'Number of classes to use in the dataset.')
  124. tf.app.flags.DEFINE_string(
  125.     'dataset_split_name', 'train', 'The name of the train/test split.')
  126. tf.app.flags.DEFINE_string(
  127.     'dataset_dir', './face_tfrecord/', 'The directory where the dataset files are stored.')
  128. tf.app.flags.DEFINE_integer(
  129.     'labels_offset', 0,
  130.     'An offset for the labels in the dataset. This flag is primarily used to '
  131.     'evaluate the VGG and ResNet architectures which do not use a background '
  132.     'class for the ImageNet dataset.')
  133. tf.app.flags.DEFINE_string(
  134.     'model_name', 'ssd_300_vgg', 'The name of the architecture to train.')
  135. tf.app.flags.DEFINE_string(
  136.     'preprocessing_name', None, 'The name of the preprocessing to use. If left '
  137.     'as `None`, then the model_name flag is used.')
  138. tf.app.flags.DEFINE_integer(
  139.     'batch_size', 4, 'The number of samples in each batch.')#32
  140. tf.app.flags.DEFINE_integer(
  141.     'train_image_size', None, 'Train image size')
  142. tf.app.flags.DEFINE_integer('max_number_of_steps', 20,#10
  143.                             'The maximum number of training steps.')
  144.  
  145. # =========================================================================== #
  146. # Fine-Tuning Flags.
  147. # =========================================================================== #
  148. tf.app.flags.DEFINE_string(
  149.     'checkpoint_path', './预训练模型/VGG_VOC0712_SSD_300x300_ft_iter_120000.ckpt',
  150.     'The path to a checkpoint from which to fine-tune.')
  151. tf.app.flags.DEFINE_string(
  152.     'checkpoint_model_scope', None,
  153.     'Model scope in the checkpoint. None if the same as the trained model.')
  154. tf.app.flags.DEFINE_string(
  155.     'checkpoint_exclude_scopes', None,
  156.     'Comma-separated list of scopes of variables to exclude when restoring '
  157.     'from a checkpoint.')
  158. tf.app.flags.DEFINE_string(
  159.     'trainable_scopes', None,
  160.     'Comma-separated list of scopes to filter the set of variables to train.'
  161.     'By default, None would train all the variables.')
  162. tf.app.flags.DEFINE_boolean(
  163.     'ignore_missing_vars', False,
  164.     'When restoring a checkpoint would ignore missing variables.')
  165.  
  166. FLAGS = tf.app.flags.FLAGS
  167.  
  168.  
  169. # =========================================================================== #
  170. # Main training routine.
  171. # =========================================================================== #
  172. def main(_):
  173.     if not FLAGS.dataset_dir:
  174.         raise ValueError('You must supply the dataset directory with --dataset_dir')
  175.  
  176.     tf.logging.set_verbosity(tf.logging.DEBUG)
  177.     with tf.Graph().as_default():
  178.         # Config model_deploy. Keep TF Slim Models structure.
  179.         # Useful if want to need multiple GPUs and/or servers in the future.
  180.         deploy_config = model_deploy.DeploymentConfig(
  181.             num_clones=FLAGS.num_clones,
  182.             clone_on_cpu=FLAGS.clone_on_cpu,
  183.             replica_id=0,
  184.             num_replicas=1,
  185.             num_ps_tasks=0)
  186.         # Create global_step.
  187.         with tf.device(deploy_config.variables_device()):
  188.             global_step = slim.create_global_step()
  189.  
  190.         # Select the dataset.
  191.         dataset = dataset_factory.get_dataset(
  192.             FLAGS.dataset_name, FLAGS.dataset_split_name, FLAGS.dataset_dir)
  193.  
  194.         # Get the SSD network and its anchors.
  195.         ssd_class = nets_factory.get_network(FLAGS.model_name)
  196.         ssd_params = ssd_class.default_params._replace(num_classes=FLAGS.num_classes)
  197.         ssd_net = ssd_class(ssd_params)
  198.         ssd_shape = ssd_net.params.img_shape
  199.         ssd_anchors = ssd_net.anchors(ssd_shape)
  200.  
  201.         # Select the preprocessing function.
  202.         preprocessing_name = FLAGS.preprocessing_name or FLAGS.model_name
  203.         image_preprocessing_fn = preprocessing_factory.get_preprocessing(
  204.             preprocessing_name, is_training=True)
  205.  
  206.         tf_utils.print_configuration(FLAGS.__flags, ssd_params,
  207.                                      dataset.data_sources, FLAGS.train_dir)
  208.         # =================================================================== #
  209.         # Create a dataset provider and batches.
  210.         # =================================================================== #
  211.         with tf.device(deploy_config.inputs_device()):
  212.             with tf.name_scope(FLAGS.dataset_name + '_data_provider'):
  213.                 provider = slim.dataset_data_provider.DatasetDataProvider(
  214.                     dataset,
  215.                     num_readers=FLAGS.num_readers,
  216.                     common_queue_capacity=20 * FLAGS.batch_size,
  217.                     common_queue_min=10 * FLAGS.batch_size,
  218.                     shuffle=True)
  219.             # Get for SSD network: image, labels, bboxes.
  220.             [image, shape, glabels, gbboxes] = provider.get(['image', 'shape',
  221.                                                              'object/label',
  222.                                                              'object/bbox'])
  223.             # Pre-processing image, labels and bboxes.
  224.             image, glabels, gbboxes = \
  225.                 image_preprocessing_fn(image, glabels, gbboxes,
  226.                                        out_shape=ssd_shape,
  227.                                        data_format=DATA_FORMAT)
  228.             # Encode groundtruth labels and bboxes.
  229.             gclasses, glocalisations, gscores = \
  230.                 ssd_net.bboxes_encode(glabels, gbboxes, ssd_anchors)
  231.             batch_shape = [1] + [len(ssd_anchors)] * 3
  232.  
  233.             # Training batches and queue.
  234.             r = tf.train.batch(
  235.                 tf_utils.reshape_list([image, gclasses, glocalisations, gscores]),
  236.                 batch_size=FLAGS.batch_size,
  237.                 num_threads=FLAGS.num_preprocessing_threads,
  238.                 capacity=5 * FLAGS.batch_size)
  239.             b_image, b_gclasses, b_glocalisations, b_gscores = \
  240.                 tf_utils.reshape_list(r, batch_shape)
  241.  
  242.             # Intermediate queueing: unique batch computation pipeline for all
  243.             # GPUs running the training.
  244.             batch_queue = slim.prefetch_queue.prefetch_queue(
  245.                 tf_utils.reshape_list([b_image, b_gclasses, b_glocalisations, b_gscores]),
  246.                 capacity=2 * deploy_config.num_clones)
  247.  
  248.         # =================================================================== #
  249.         # Define the model running on every GPU.
  250.         # =================================================================== #
  251.         def clone_fn(batch_queue):
  252.             """Allows data parallelism by creating multiple
  253.             clones of network_fn."""
  254.             # Dequeue batch.
  255.             b_image, b_gclasses, b_glocalisations, b_gscores = \
  256.                 tf_utils.reshape_list(batch_queue.dequeue(), batch_shape)
  257.  
  258.             # Construct SSD network.
  259.             arg_scope = ssd_net.arg_scope(weight_decay=FLAGS.weight_decay,
  260.                                           data_format=DATA_FORMAT)
  261.             with slim.arg_scope(arg_scope):
  262.                 predictions, localisations, logits, end_points = \
  263.                     ssd_net.net(b_image, is_training=True)
  264.             # Add loss function.
  265.             ssd_net.losses(logits, localisations,
  266.                            b_gclasses, b_glocalisations, b_gscores,
  267.                            match_threshold=FLAGS.match_threshold,
  268.                            negative_ratio=FLAGS.negative_ratio,
  269.                            alpha=FLAGS.loss_alpha,
  270.                            label_smoothing=FLAGS.label_smoothing)
  271.             return end_points
  272.  
  273.         # Gather initial summaries.
  274.         summaries = set(tf.get_collection(tf.GraphKeys.SUMMARIES))
  275.  
  276.         # =================================================================== #
  277.         # Add summaries from first clone.
  278.         # =================================================================== #
  279.         clones = model_deploy.create_clones(deploy_config, clone_fn, [batch_queue])
  280.         first_clone_scope = deploy_config.clone_scope(0)
  281.         # Gather update_ops from the first clone. These contain, for example,
  282.         # the updates for the batch_norm variables created by network_fn.
  283.         update_ops = tf.get_collection(tf.GraphKeys.UPDATE_OPS, first_clone_scope)
  284.  
  285.         # Add summaries for end_points.
  286.         end_points = clones[0].outputs
  287.         for end_point in end_points:
  288.             x = end_points[end_point]
  289.             summaries.add(tf.summary.histogram('activations/' + end_point, x))
  290.             summaries.add(tf.summary.scalar('sparsity/' + end_point,
  291.                                             tf.nn.zero_fraction(x)))
  292.         # Add summaries for losses and extra losses.
  293.         for loss in tf.get_collection(tf.GraphKeys.LOSSES, first_clone_scope):
  294.             summaries.add(tf.summary.scalar(loss.op.name, loss))
  295.         for loss in tf.get_collection('EXTRA_LOSSES', first_clone_scope):
  296.             summaries.add(tf.summary.scalar(loss.op.name, loss))
  297.  
  298.         # Add summaries for variables.
  299.         for variable in slim.get_model_variables():
  300.             summaries.add(tf.summary.histogram(variable.op.name, variable))
  301.  
  302.         # =================================================================== #
  303.         # Configure the moving averages.
  304.         # =================================================================== #
  305.         if FLAGS.moving_average_decay:
  306.             moving_average_variables = slim.get_model_variables()
  307.             variable_averages = tf.train.ExponentialMovingAverage(
  308.                 FLAGS.moving_average_decay, global_step)
  309.         else:
  310.             moving_average_variables, variable_averages = None, None
  311.  
  312.         # =================================================================== #
  313.         # Configure the optimization procedure.
  314.         # =================================================================== #
  315.         with tf.device(deploy_config.optimizer_device()):
  316.             learning_rate = tf_utils.configure_learning_rate(FLAGS,
  317.                                                              dataset.num_samples,
  318.                                                              global_step)
  319.             optimizer = tf_utils.configure_optimizer(FLAGS, learning_rate)
  320.             summaries.add(tf.summary.scalar('learning_rate', learning_rate))
  321.  
  322.         if FLAGS.moving_average_decay:
  323.             # Update ops executed locally by trainer.
  324.             update_ops.append(variable_averages.apply(moving_average_variables))
  325.  
  326.         # Variables to train.
  327.         variables_to_train = tf_utils.get_variables_to_train(FLAGS)
  328.  
  329.         # and returns a train_tensor and summary_op
  330.         total_loss, clones_gradients = model_deploy.optimize_clones(
  331.             clones,
  332.             optimizer,
  333.             var_list=variables_to_train)
  334.         # Add total_loss to summary.
  335.         summaries.add(tf.summary.scalar('total_loss', total_loss))
  336.  
  337.         # Create gradient updates.
  338.         grad_updates = optimizer.apply_gradients(clones_gradients,
  339.                                                  global_step=global_step)
  340.         update_ops.append(grad_updates)
  341.         update_op = tf.group(*update_ops)
  342.         train_tensor = control_flow_ops.with_dependencies([update_op], total_loss,
  343.                                                           name='train_op')
  344.  
  345.         # Add the summaries from the first clone. These contain the summaries
  346.         summaries |= set(tf.get_collection(tf.GraphKeys.SUMMARIES,
  347.                                            first_clone_scope))
  348.         # Merge all summaries together.
  349.         summary_op = tf.summary.merge(list(summaries), name='summary_op')
  350.  
  351.         # =================================================================== #
  352.         # Kicks off the training.
  353.         # =================================================================== #
  354.         gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=FLAGS.gpu_memory_fraction)
  355.         config = tf.ConfigProto(log_device_placement=False,
  356.                                 gpu_options=gpu_options)
  357.         saver = tf.train.Saver(max_to_keep=5,
  358.                                keep_checkpoint_every_n_hours=1.0,
  359.                                write_version=2,
  360.                                pad_step_number=False)
  361.         slim.learning.train(
  362.             train_tensor,
  363.             logdir=FLAGS.train_dir,
  364.             master='',
  365.             is_chief=True,
  366.             # init_fn=tf_utils.get_init_fn(FLAGS),
  367.             summary_op=summary_op,
  368.             number_of_steps=FLAGS.max_number_of_steps,
  369.             log_every_n_steps=FLAGS.log_every_n_steps,
  370.             save_summaries_secs=FLAGS.save_summaries_secs,
  371.             saver=saver
  372.             ,save_interval_secs=FLAGS.save_interval_secs
  373.             ,session_config=config
  374.             ,sync_optimizer=None
  375.         )
  376. if __name__ == '__main__':
  377.     tf.app.run()

六、训练过程报错处理

1、训练进度停顿
-- 检查转换后的tfrecord数据,很大可能是数据格式转换失败

2、梯度爆炸
-- 检查损失函数逻辑，有可能是损失函数中符号错误
-- 样本维度和网络输入维度不一致等问题

3、梯度消失
-- 梯度消失的原因很多，需要根据具体问题具体定位
-- 比如可以尝试在网络的最开始添加批量归一化层

4、损失不收敛
-- 这个一般是学习率和优化器的配合使用不熟练导致的
-- 一般刚开始可以设置一个较大的学习率，然后缓慢减小
-- 优化器使用:Adam+随机梯度下降 ，使用Adam粗调，使用随机梯度下降进行精调

5、训练缓慢
-- 修改训练批次数量，一般批次数量越大，训练越缓慢
-- 也可以无脑添加drop out层

七、验证过程设计

验证代码：

  1. import os
  2. import math
  3. import random
  4.  
  5. import numpy as np
  6. import tensorflow as tf
  7. import cv2
  8.  
  9. slim = tf.contrib.slim
  10.  
  11. # %matplotlib inline
  12. import matplotlib.pyplot as plt
  13. import matplotlib.image as mpimg
  14.  
  15. import sys
  16. sys.path.append('../')
  17. from nets import ssd_vgg_300, ssd_common, np_methods
  18. from preprocessing import ssd_vgg_preprocessing
  19. from notebooks import visualization
  20. isess = sess = tf.Session()
  21.  
  22. # Input placeholder.
  23. net_shape = (300, 300)
  24. data_format = 'NHWC'
  25. img_input = tf.placeholder(tf.uint8, shape=(None, None, 3))
  26. # Evaluation pre-processing: resize to SSD net shape.
  27. image_pre, labels_pre, bboxes_pre, bbox_img = ssd_vgg_preprocessing.preprocess_for_eval(
  28.     img_input, None, None, net_shape, data_format, resize=ssd_vgg_preprocessing.Resize.WARP_RESIZE)
  29. image_4d = tf.expand_dims(image_pre, 0)
  30. print("part_1 sucessed")
  31.  
  32. # Define the SSD model.
  33. reuse = True if 'ssd_net' in locals() else None
  34. ssd_net = ssd_vgg_300.SSDNet()
  35. with slim.arg_scope(ssd_net.arg_scope(data_format=data_format)):
  36.     predictions, localisations, _, _ = ssd_net.net(image_4d, is_training=False, reuse=reuse)
  37.  
  38. print("part_2 sucessed")
  39. # Restore SSD model.
  40. # ckpt_filename = 'F:/时间简史/人脸识别项目_wuyang/SSD源码/SSD-Tensorflow-master/预训练模型'
  41. # ckpt_filename = './wuyang_model/model.ckpt-20'
  42. ckpt_filename = './预训练模型/VGG_VOC0712_SSD_300x300_ft_iter_120000.ckpt'
  43. print("part_3 sucessed")
  44. isess.run(tf.global_variables_initializer())
  45. print("part_4 sucessed")
  46. saver = tf.train.Saver()
  47. saver.restore(isess, ckpt_filename)
  48. print("part_5 sucessed")
  49. # SSD default anchor boxes.
  50. ssd_anchors = ssd_net.anchors(net_shape)
  51. print("all part sucessed")
  52. # Main image processing routine.
  53. def process_image(img, select_threshold=0.5, nms_threshold=.45, net_shape=(300, 300)):
  54.     # Run SSD network.
  55.     rimg, rpredictions, rlocalisations, rbbox_img = isess.run([image_4d, predictions, localisations, bbox_img],
  56.                                                               feed_dict={img_input: img})
  57.  
  58.     # Get classes and bboxes from the net outputs.
  59.     rclasses, rscores, rbboxes = np_methods.ssd_bboxes_select(
  60.         rpredictions, rlocalisations, ssd_anchors,
  61.         select_threshold=select_threshold, img_shape=net_shape, num_classes=21, decode=True)
  62.  
  63.     rbboxes = np_methods.bboxes_clip(rbbox_img, rbboxes)
  64.     rclasses, rscores, rbboxes = np_methods.bboxes_sort(rclasses, rscores, rbboxes, top_k=400)
  65.     rclasses, rscores, rbboxes = np_methods.bboxes_nms(rclasses, rscores, rbboxes, nms_threshold=nms_threshold)
  66.     # Resize bboxes to original image shape. Note: useless for Resize.WARP!
  67.     rbboxes = np_methods.bboxes_resize(rbbox_img, rbboxes)
  68.     return rclasses, rscores, rbboxes
  69. # Test on some demo image and visualize output.
  70. path = 'F:/test-image/'
  71. image_names = sorted(os.listdir(path))
  72.  
  73. img = mpimg.imread(path + image_names[0])
  74. rclasses, rscores, rbboxes =  process_image(img)
  75.  
  76. # visualization.bboxes_draw_on_img(img, rclasses, rscores, rbboxes, visualization.colors_plasma)
  77. visualization.plt_bboxes(img, rclasses, rscores, rbboxes)

训练完成之后：
-- 恢复保存的预训练模型
-- 使用预训练模型，对相关照片进行预测

测试过程可能报错：
-- 模型恢复报维度不匹配：
    -- 根据实际维度，修改vgg网络输入维度
    
-- 读取checkpoint路径错误：
    -- 要选取到check实际存在的文件夹
    -- 然后还要定位到具体的ckpt文件
    -- 例如我的路径为：./wuyang_model/model.ckpt-20000
    
八、最终结果

各数值对应类别如下：
-- 0：背景板
-- 1，2：飞机
-- 3：鸟
-- 4：船
-- 5：瓶子
-- 6：公交
-- 7：汽车
-- 8：猫
-- 9：椅子
-- 10：牛
-- 11：桌子
-- 12：熊猫
-- 13：马
-- 14：摩托
-- 15：人
-- 16：茶几
-- 17：羊
-- 18：沙发
-- 19：火车
-- 20：液晶屏幕

用训练完的模型，进行目标识别测试。
-- 由于没有来得及设计爬虫程序，所有样本都是手动抓取手动标注，所以有些类别学习不充分
-- 下面展示几个分类效果好的例子

对电脑的预测：

对熊猫的预测：

对飞机的预测：

以及对大美女、女神王祖贤的预测：

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/Cpp五条/article/detail/291161
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号