
class Lenet(nn.Module):
    def __init__(self, bool_bn=False, num_classes=10):
        super(Lenet, self).__init__()
        self.features = nn.Sequential(
            nn.Conv2d(3, 6, kernel_size=5),
            nn.ReLU(inplace=True),
            nn.MaxPool2d(kernel_size=2, stride=2),
            nn.Conv2d(6, 16, kernel_size=5),
            nn.ReLU(inplace=True),
            nn.MaxPool2d(kernel_size=2, stride=2),
        )
        self.features_bn = nn.Sequential(
            nn.Conv2d(3, 6, kernel_size=5),
            nn.BatchNorm2d(6),
            nn.ReLU(inplace=True),
            nn.MaxPool2d(kernel_size=2, stride=2),
            nn.Conv2d(6, 16, kernel_size=5),
            nn.BatchNorm2d(16),
            nn.ReLU(inplace=True),
            nn.MaxPool2d(kernel_size=2, stride=2),
        )
        self.classifier = nn.Sequential(
            nn.Linear(16 * 5 * 5, 120),
            nn.ReLU(inplace=True),
            nn.Dropout(),
            nn.Linear(120, 84),
            nn.ReLU(inplace=True),
            nn.Dropout(),
            nn.Linear(84, num_classes),
        )
        self.bool_bn = bool_bn
 
    def forward(self, x):
        if self.bool_bn:
            x = self.features_bn(x)
        else:
            x = self.features(x)
        x = x.view(x.size(0), 16 * 5 * 5)
        x = self.classifier(x)
        return x

AlexNet

论文题目：Imagenet classification with deep convolutional neural networks

论文地址：https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf

2012年ILSVRC分类比赛冠军

开源代码：

https://github.com/pytorch/vision/blob/master/torchvision/models/alexnet.py

ZFNet

论文题目：Visualizing and Understanding Convolutional Networks

论文地址：https://arxiv.org/pdf/1311.2901.pdf

2013年ILSVRC分类比赛冠军

Inception系列

论文题目：Going deeper with convolutions

论文地址：https://arxiv.org/pdf/1409.4842v1.pdf

2014年ILSVRC分类比赛冠军

开源代码：

https://github.com/pytorch/vision/blob/master/torchvision/models/googlenet.py

https://github.com/pytorch/vision/blob/master/torchvision/models/inception.py

论文题目：Rethinking the Inception Architecture for Computer Vision

论文地址：https://arxiv.org/pdf/1512.00567.pdf

Inception v2-v3

论文题目：Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning

论文地址：https://arxiv.org/pdf/1602.07261.pdf

Inception-v4, Inception-ResNet-v1, Inception-ResNet-v2

VGG

论文题目：Very Deep Convolutional Networks for Large-Scale Image Recognition

论文地址：https://arxiv.org/pdf/1409.1556.pdf

2014年ILSVRC分类比赛亚军

开源代码：

https://github.com/pytorch/vision/blob/master/torchvision/models/vgg.py

ResNet

论文题目：Deep Residual Learning for Image Recognition

论文地址：https://arxiv.org/pdf/1512.03385.pdf

2015年1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation

论文题目：Identity Mappings in Deep Residual Networks

论文地址：https://arxiv.org/pdf/1603.05027.pdf

修改了ResNet中卷积、批量归一化和Relu层的顺序提高分类效果

开源代码：

https://github.com/pytorch/vision/blob/master/torchvision/models/resnet.py

DenseNet

论文题目：Densely Connected Convolutional Networks

论文地址：https://arxiv.org/pdf/1608.06993.pdf

CVPR2017最佳论文

开源代码：

https://github.com/pytorch/vision/blob/master/torchvision/models/densenet.py

ResNeXt

论文题目：Aggregated Residual Transformations for Deep Neural Networks

论文地址：https://arxiv.org/pdf/1611.05431.pdf

ResNet分组卷积，提高检测准确率；Faster-RCNN Backbone首选网络

开源代码：

https://github.com/pytorch/vision/blob/master/torchvision/models/resnet.py

SENet

论文题目：Squeeze-and-Excitation Networks

论文地址：https://arxiv.org/pdf/1709.01507.pdf

在网络中增加了Squeeze和Excitation分支，提高检测准确率

EfficientNet

论文题目：EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks

论文地址：https://arxiv.org/pdf/1905.11946.pdf

目标检测

RCNN

论文题目：Rich feature hierarchies for accurate object detection and semantic segmentation

论文地址：https://arxiv.org/pdf/1311.2524.pdf

通过Selective Search选择存在目标的区域，通过SVM实现分类，通过Bounding Box Regression实现定位

OverFeat

论文题目：OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks

论文地址：https://arxiv.org/pdf/1312.6229.pdf

通过全卷积网络实现目标定位，winner of the ILSVRC13 localization competition

SPP

论文题目：Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition

论文地址：https://arxiv.org/pdf/1406.4729.pdf

提出了空间金字塔池化的方法，进行图像分类和目标检测

Fast RCNN

论文题目：Fast R-CNN

论文地址：https://arxiv.org/pdf/1504.08083.pdf

提出了ROI Pooling的方法

Faster RCNN

论文题目：Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

论文地址：https://arxiv.org/pdf/1506.01497.pdf

使用了Region Proposal Net替代Selective Search的方法，实现端到端检测并提高检测速度。

R-FCN

论文题目：R-FCN: Object Detection via Region-based Fully Convolutional Networks

论文地址：https://arxiv.org/pdf/1605.06409.pdf

在ROI中使用k*k的feature map投票预测种类

FPN

论文题目：Feature Pyramid Networks for Object Detection

论文地址：https://arxiv.org/pdf/1612.03144.pdf

提出了图像特征金字塔的方法，提高对图像中小目标的检测精度

SSD系列

论文题目：SSD: Single Shot MultiBox Detector

论文地址：https://arxiv.org/pdf/1512.02325.pdf

一种One Stage Detector

论文题目：DSSD : Deconvolutional Single Shot Detector

论文地址：https://arxiv.org/pdf/1701.06659.pdf

改进了SSD，提高检测精度

YOLO系列

论文题目：You Only Look Once: Unified, Real-Time Object Detection

论文地址：https://arxiv.org/pdf/1506.02640.pdf

代码地址：https://github.com/pjreddie/darknet

https://github.com/AlexeyAB/darknet

一种One Stage Detector，YOLO v1

论文题目：YOLO9000: Better, Faster, Stronger

论文地址：https://arxiv.org/pdf/1612.08242.pdf

YOLO v2， Backbone为Darknet19

论文题目：YOLOv3: An Incremental Improvement

论文地址：https://pjreddie.com/media/files/papers/YOLOv3.pdf

YOLOv3，Backbone为Darknet53，与v2相比，采用了残差模块和FPN结构

论文题目：Gaussian YOLOv3: An Accurate and Fast Object Detector Using Localization Uncertainty for Autonomous Driving

论文地址：https://arxiv.org/pdf/1904.04620v2.pdf

代码地址：https://github.com/jwchoi384/Gaussian_YOLOv3

论文题目：YOLOv4: Optimal Speed and Accuracy of Object Detection

论文地址：https://arxiv.org/pdf/2004.10934.pdf

YOLOv4，CSPDarknet53作为骨干网络，SPP作为附加模块，PANet作为特征融合的Neck，YOLOv3检测器作为Header。详细解析见https://blog.csdn.net/linghu8812/article/details/105729693。

RetinaNet

论文题目：Focal Loss for Dense Object Detection

论文地址：https://arxiv.org/pdf/1708.02002.pdf

对于训练集中正负样本不平均的情况，采用了focal loss的方法，减小负样本数量过多的影响。

CornerNet

论文题目：CornerNet: Detecting Objects as Paired Keypoints

论文地址：https://arxiv.org/pdf/1808.01244.pdf

基于关键点的方法进行目标检测，使用了角池化的方法检测不在检测框的左上角或者右下角的目标。

论文题目：CornerNet-Lite: Efficient Keypoint Based Object Detection

论文地址：https://arxiv.org/pdf/1904.08900.pdf

使用了CornerNet Saccade和CornerNet Squeez对CornerNet检测进行提速。

CenterNet

论文题目：CenterNet: Keypoint Triplets for Object Detection

论文地址：https://arxiv.org/pdf/1904.08189.pdf

对CornerNet进行了中心池化和级联角池化的优化，提高了mAP。

论文题目：Objects as Points

论文地址：https://arxiv.org/pdf/1904.07850.pdf

采用了不同的Backbone进行检测比较检测速度和mAP。

CVPR2018

论文题目：Cascade R-CNN Delving into High Quality Object Detection
论文地址：https://arxiv.org/pdf/1712.00726.pdf

论文题目：Relation Networks for Object Detection
论文地址：https://arxiv.org/pdf/1711.11575.pdf

论文题目：Single-Shot Refinement Neural Network for Object Detection

论文地址：https://arxiv.org/pdf/1711.06897.pdf

论文题目：An Analysis of Scale Invariance in Object Detection – SNIP
论文地址：https://arxiv.org/pdf/1711.08189.pdf

论文题目：R-FCN-3000 at 30fps: Decoupling Detection and Classification
论文地址：https://arxiv.org/pdf/1712.01802.pdf

论文题目：Single-Shot Object Detection with Enriched Semantics
论文链接：https://arxiv.org/pdf/1712.00433.pdf

论文题目：Scale-Transferrable Object Detection
论文地址：http://openaccess.thecvf.com/content_cvpr_2018/CameraReady/1376.pdf

图像分割

FCN

论文题目：Fully Convolutional Networks for Semantic Segmentation

论文地址：https://arxiv.org/pdf/1411.4038.pdf

使用全连接网络进行图像语义分割

Mask RCNN

论文题目：Mask R-CNN

论文地址：https://arxiv.org/pdf/1703.06870.pdf

ICCV2017最佳论文，在faster rcnn的基础上增加了mask分支，进行实例分割

论文题目：Learning to Segment Every Thing

论文地址：https://arxiv.org/abs/1711.10370

Mask RCNN的升级版，设计了从bbox到mask的weight transfer function

梯度下降

方法综述

论文题目：An overview of gradient descent optimization algorithms

论文地址：https://arxiv.org/pdf/1609.04747.pdf

介绍各种梯度下降方法，比较性能。

Momentum

论文题目：On the momentum term in gradient descent learning algorithms

论文地址：http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=11C50B032D6D31F8BF95E865E769B4BC?doi=10.1.1.65.3526&rep=rep1&type=pdf

带动量的SGD

Nesterov

论文题目：A method for unconstrained convex minimization problem with the rate of convergence o(1/k2)

论文地址：http://www.cis.pku.edu.cn/faculty/vision/zlin/1983-A%20Method%20of%20Solving%20a%20Convex%20Programming%20Problem%20with%20Convergence%20Rate%20O(k%5E(-2))_Nesterov.pdf

带Nesterov动量的SGD

Adagrad

论文题目：Adaptive Subgradient Methods for Online Learning and Stochastic Optimization

论文地址：http://www.jmlr.org/papers/volume12/duchi11a/duchi11a.pdf

Adagrad算法

Adadelta

论文题目：ADADELTA: An Adaptive Learning Rate Method

论文地址：https://arxiv.org/pdf/1212.5701.pdf

Adagrad算法

Adam

论文题目：Adam: a Method for Stochastic Optimization

论文地址：https://arxiv.org/pdf/1412.6980.pdf

Adam算法

Nesterov+Adam

论文题目：Incorporating Nesterov Momentum into Adam

论文地址：https://openreview.net/pdf?id=OM0jvwB8jIp57ZJjtNEZ

带Nesterov动量的Adam算法

Adabound

论文题目：Adaptive Gradient Methods with Dynamic Bound of Learning Rate

论文地址：https://arxiv.org/pdf/1902.09843.pdf

ICLR2019最新梯度下降论文

生成对抗网络

GAN

论文题目：Generative Adversarial Networks

论文地址：https://arxiv.org/pdf/1406.2661.pdf

通过对抗训练，生成与训练集类似的数据

DCGAN

论文题目：Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

论文地址：https://arxiv.org/pdf/1511.06434.pdf

通过卷积和反卷积网络生成数据

LSGAN

论文题目：Least Squares Generative Adversarial Networks

论文地址：https://arxiv.org/pdf/1611.04076.pdf

损失函数为二次函数训练网络

Wasserstein GAN

论文题目：Wasserstein GAN

论文地址：https://arxiv.org/pdf/1701.07875.pdf

通过修改损失函数，提高生成效果

Age Conditional GAN

论文题目：FACE AGING WITH CONDITIONAL GENERATIVE ADVERSARIAL NETWORKS

论文地址：https://arxiv.org/pdf/1702.01983.pdf

生成每个人不同年龄段的照片

Cycle GAN

论文题目：Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks

论文地址：https://arxiv.org/pdf/1703.10593.pdf

斑马图像和马的图像、苹果图像和橘子图像相互转换

StarGAN

论文题目：StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation

论文地址：https://arxiv.org/pdf/1711.09020.pdf

生成明星不同肤色、发型、表情等特征的GAN

CartoonGAN

论文题目：CartoonGAN: Generative Adversarial Networks for Photo Cartoonization

论文地址：http://openaccess.thecvf.com/content_cvpr_2018/papers/Chen_CartoonGAN_Generative_Adversarial_CVPR_2018_paper.pdf

照片转卡通风格

双目匹配

论文题目：Computing the Stereo Matching Cost with a Convolutional Neural Network

论文地址：https://arxiv.org/pdf/1409.4326.pdf

在9x9的图片patch上判断两点是否匹配

论文题目：Learning to Compare Image Patches via Convolutional Neural Networks

论文地址：https://arxiv.org/pdf/1504.03641.pdf

论文题目：Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches

论文地址：https://arxiv.org/pdf/1510.05970v2.pdf

论文题目：Efﬁcient Deep Learning for Stereo Matching

论文地址：https://www.cs.toronto.edu/~urtasun/publications/luo_etal_cvpr16.pdf

论文题目：End-to-End Learning of Geometry and Context for Deep Stereo Regression

论文地址：https://arxiv.org/pdf/1703.04309v1.pdf

论文题目：Learning for Disparity Estimation through Feature Constancy

论文地址：https://arxiv.org/pdf/1712.01039.pdf

论文题目：Pyramid Stereo Matching Network

论文地址：https://arxiv.org/pdf/1803.08669.pdf

归一化方法

四种归一化方法说明——Group Normalization, Yuxin Wu, Kaiming He, https://arxiv.org/pdf/1803.08494.pdf

Batch Normalization

论文题目：Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

论文地址：https://arxiv.org/pdf/1502.03167.pdf

Batch Normalization批量归一化方法

Layer Normalization

论文题目：Layer Normalization

论文地址：https://arxiv.org/pdf/1607.06450.pdf

Instance Normalization

论文题目：Instance Normalization: The Missing Ingredient for Fast Stylization

论文地址：https://arxiv.org/pdf/1607.08022.pdf

Group Normalization

论文题目：Group Normalization

论文地址：https://arxiv.org/pdf/1803.08494.pdf

组归一化方法

模型剪枝

论文题目：Pruning Filters for Efficient ConvNets

论文地址：https://arxiv.org/pdf/1608.08710.pdf

基于卷积层权重的L1正则化大小作为标准进行剪枝

论文题目：Learning Efﬁcient Convolutional Networks through Network Slimming

论文地址：https://arxiv.org/pdf/1708.06519.pdf

将Batch Normalization层的gamma系数作为标准进行剪枝。

论文题目：Rethinking the Value of Network Pruning

论文地址：https://arxiv.org/pdf/1810.05270.pdf

比较了各种剪枝算法，及剪枝后进行fine tune和train from scratch准确率的比较。

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/我家自动化/article/detail/603606