devbox
小丑西瓜9
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

（3）paddle---近视眼睛分类的例子_pred_area = paddle.concat(pred_area)

作者：小丑西瓜9 | 2024-02-16 11:28:31

赞

踩

pred_area = paddle.concat(pred_area)

1主要参考

（1）paddle的API官方地址

Conv2D-API文档-PaddlePaddle深度学习平台

（2）本教程和以下教程不够详细，还是推荐下面这个大佬的blog看一下

计算机视觉——眼疾图片识别（数据集iChallenge-PM）_「已注销」的博客-CSDN博客

（3）blibli视频

252-06_预测病理性近视_图片数据读取_dec_哔哩哔哩_bilibili

 （4）数据集地址

眼疾识别数据集iChallenge-PM - 飞桨AI Studio

2、数据集介绍

（1）数据集

iChallenge-PM是百度大脑和中山大学中山眼科中心联合举办的iChallenge比赛中，提供的关于病理性近视（Pathologic Myopia，PM）的医疗类数据集，包含1200个受试者的眼底视网膜图片，训练、验证和测试数据集各400张。 training.zip：包含训练中的图片和标签
validation.zip：包含验证集的图片
valid_gt.zip：包含验证集的标签  

iChallenge-PM中既有病理性近视患者的眼底图片，也有非病理性近视患者的图片，命名规则如下： 病理性近视（PM）：文件名以P开头
非病理性近视（non-PM）： 高度近似（high myopia）：文件名以H开头
正常眼睛（normal）：文件名以N开头

（2）所使用的文件

1）解压3个压缩包，拷贝出来需要的

 2）将valid_gt.zip中的压缩包中的excel另存为.csv文件，另存后，注意删除最后的两行空数据

3）剩下来，建议看大佬的代码吧

计算机视觉——眼疾图片识别（数据集iChallenge-PM）_「已注销」的博客-CSDN博客

3、基于Lenet的程序实现（基础实现） 

（1）程序测试代码

  1. import cv2
  2. import random
  3. import numpy as np
  4. import os
  5. import paddle
  6. import paddle.nn.functional as F
  7. from paddle.nn import Conv2D, MaxPool2D, Linear, Dropout
  8.  
  9. #对读入的图像数据进行预处理
  10. def transform_img(img):
  11.     #将图片尺寸缩放到 224 * 224
  12.     img = cv2.resize(img, (224,224))
  13.     #读取的图像数据格式是[H, W, C]
  14.     #使用转置操作将其变成[C, H, W]
  15.     img = np.transpose(img,(2,0,1))
  16.     #归一化操作，将数据范围调整到[-1.0, 1.0]之间
  17.     img = img / 255
  18.     img = img * 2.0 -1.0
  19.     return img
  20.  
  21.  
  22. #定义训练集数据读取器
  23. def data_loader(datadir, batch_size=10, mode='train'):
  24.     #将datadir目录下的文件列出来，每条文件都读入
  25.     filenames = os.listdir(datadir)
  26.     def reader():
  27.         # print("chen_filenames")
  28.         # print(filenames)
  29.         #只有在训练的时随机打乱数据顺序，测试的时候不打乱
  30.         if mode == 'train':
  31.             random.shuffle(filenames)
  32.         batch_imgs = []
  33.         batch_labels = []
  34.         for name in filenames:
  35.             filepath = os.path.join(datadir,name)
  36.             img = cv2.imread(filepath)
  37.             img = transform_img(img)
  38.             if name[0] == 'H' or name[0] == 'N':
  39.                 # H开头的文件名表示高度近似，N开头的文件名表示正常视力
  40.                 # 高度近似和正常视力的样板，都不是病理性的，属于负样本，标签为0
  41.                 label = 0
  42.             elif name[0] == 'P':
  43.                 # P开头的是病理性近似，属于正样本，标签为1
  44.                 label = 1
  45.             else:
  46.                 raise('Not excepted file name')
  47.             #每读取一个样板的数据，就将其放入数据列表中
  48.             batch_imgs.append(img)
  49.             batch_labels.append(label)
  50.             if len(batch_imgs) == batch_size:
  51.                 # 当数据列表的长度等于batch_size的时候
  52.                 # 把这些数据当做一个mini-batch,并作为数据生成器的一个输出
  53.                 imgs_array = np.array(batch_imgs).astype('float32')
  54.                 labels_array = np.array(batch_labels).astype('float32').reshape(-1,1)
  55.                 # yield的作用：返回一个可以用来迭代(for循环)的生成器，
  56.                 # 它的应用场景通常为一个需要返回一系列值的，含有循环的函数中
  57.                 yield imgs_array, labels_array
  58.                 batch_imgs = []
  59.                 batch_labels = []
  60.         
  61.         #上面都循环结束了，如何还有剩下的，也就是样板数/batch_size数没有整除
  62.         if len(batch_imgs) > 0:
  63.             imgs_array = np.array(batch_imgs).astype('float32')
  64.             labels_array = np.array(batch_labels).astype('float32').reshape(-1,1)
  65.             # yield的作用：返回一个可以用来迭代(for循环)的生成器，
  66.             # 它的应用场景通常为一个需要返回一系列值的，含有循环的函数中
  67.             yield imgs_array, labels_array
  68.  
  69.     return reader
  70.  
  71.  
  72. EPOCH_NUM = 5
  73.  
  74. #定义训练过程
  75. def train_pm(model, optimizer):
  76.     # 开启0号GPU训练
  77.     use_gpu = True
  78.     paddle.device.set_device('gpu:0') if use_gpu else paddle.device.set_device('cpu')
  79.  
  80.     print('start training ...')
  81.     model.train()
  82.     # 定义数据读取器，训练数据读取器和验证数据读取器
  83.     train_loader = data_loader(DATADIR, batch_size=10, mode='train')
  84.     for epoch in range(EPOCH_NUM):
  85.         for batch_id, data in enumerate(train_loader()):  #在返回的reader里面调用函数，调用的就是yield中的枚举返回值
  86.             x_data, y_data = data
  87.             img = paddle.to_tensor(x_data)
  88.             label = paddle.to_tensor(y_data)
  89.             # 运行模型进行前向推理计算，得到预测值
  90.             logits = model(img)
  91.             loss = F.binary_cross_entropy_with_logits(logits, label)
  92.             avg_loss = paddle.mean(loss)
  93.  
  94.             if batch_id % 20 == 0:
  95.                 print("epoch: {},batch_id: {}, loss is: {:.4f}".format(epoch, batch_id, float(avg_loss.numpy())))
  96.             #反向传播，更新权重，清除梯度
  97.             avg_loss.backward()
  98.             optimizer.step()
  99.         
  100.         #每一个epoch结束都测试一下
  101.         model.eval()
  102.         accuracies = []
  103.         losses = []
  104.         valid_loader = data_loader(DATADIR, batch_size=10, mode='eval')
  105.         for batch_id, data in enumerate(valid_loader()):
  106.             x_data, y_data = data
  107.             img = paddle.to_tensor(x_data)
  108.             label = paddle.to_tensor(y_data)
  109.             #运行模型前向计算，得到预测值
  110.             logits = model(img)
  111.             # 二分类，sigmoid计算后的结果以0.5为阈值分两个类型
  112.             # 计算sigmoid后的预测概率，进行loss计算
  113.             pred = F.sigmoid(logits)   # P正的概率
  114.             loss = F.binary_cross_entropy_with_logits(logits,label)
  115.             #计算概率小于0.5的类别
  116.             pred2 = pred*(-1.0) + 1.0  # P负的概率，也就是1-P正
  117.             #得到两个类别的预测概率，并沿第一个维度级联
  118.             pred = paddle.concat([pred2, pred],axis=1)
  119.             #原因是paddle.metric.accuracy的输入要是每个类别的概率，所以拼接了一下
  120.             acc = paddle.metric.accuracy(pred, paddle.cast(label,dtype='int64'))
  121.  
  122.  
  123.             accuracies.append(acc.numpy())
  124.             losses.append(loss.numpy())
  125.         print("[validation] accuracy /loss: {:.4f}/{:.4f}".format(np.mean(accuracies),np.mean(losses))) #求平均
  126.         model.train()
  127.  
  128.         paddle.save(model.state_dict(),'palm.pdparams')        
  129.         paddle.save(optimizer.state_dict(),'palm.pdopt')
  130.  
  131.  
  132. #定义评国过程
  133. def evaluation(model, params_file_path):
  134.     # 开启0号GPU预估
  135.     use_gpu = True
  136.     paddle.device.set_device('gpu:0') if use_gpu else paddle.device.set_device('cpu')
  137.  
  138.     print('start evaluation ...')
  139.     #加载模型参数
  140.     model_state_dict = paddle.load(params_file_path)
  141.     model.load_dict(model_state_dict)
  142.     
  143.     model.eval()
  144.     eval_loader = data_loader(DATADIR, batch_size=10, mode='eval')
  145.  
  146.     acc_set = []
  147.     avg_loss_set = []
  148.     for batch_id, data in enumerate(eval_loader()):
  149.         x_data, y_data =data
  150.         img = paddle.to_tensor(x_data)
  151.         label = paddle.to_tensor(y_data)
  152.         y_data = y_data.astype(np.int64)
  153.         label_64 = paddle.to_tensor(y_data)
  154.         #计算预测和精度
  155.         #模型的forward方法这两个封装了对img, label_64的判断，如果传入了label，则也返回acc
  156.         prediction, acc = model(img, label_64)
  157.         # 计算损失函数值
  158.         loss = F.binary_cross_entropy_with_logits(prediction, label)
  159.         avg_loss = paddle.mean(loss)
  160.         acc_set.append(float(acc.numpy()))
  161.         avg_loss_set.append(float(avg_loss.numpy()))
  162.     
  163.     #求平均精度
  164.     acc_val_mean = np.array(acc_set).mean()
  165.     avg_loss_val_mean = np.array(avg_loss_set).mean()
  166.  
  167.     print('loss={:.4f}, acc={:.4f}'.format(avg_loss_val_mean,acc_val_mean))
  168.  
  169.  
  170.  
  171. #定义Lenet网络结果
  172. # class Lenet(paddle.nn.Layer):
  173. #     pass 
  174.  
  175. # 定义 LeNet 网络结构
  176. class LeNet(paddle.nn.Layer):
  177.     def __init__(self, num_classes=1):
  178.         super(LeNet, self).__init__()
  179.  
  180.         # 创建卷积和池化层块，每个卷积层使用Sigmoid激活函数，后面跟着一个2x2的池化
  181.         self.conv1 = Conv2D(in_channels=3, out_channels=6, kernel_size=5)
  182.         self.max_pool1 = MaxPool2D(kernel_size=2, stride=2)
  183.         self.conv2 = Conv2D(in_channels=6, out_channels=16, kernel_size=5)
  184.         self.max_pool2 = MaxPool2D(kernel_size=2, stride=2)
  185.         # 创建第3个卷积层
  186.         self.conv3 = Conv2D(in_channels=16, out_channels=120, kernel_size=4)
  187.         # 创建全连接层，第一个全连接层的输出神经元个数为64， 第二个全连接层输出神经元个数为分类标签的类别数
  188.         self.fc1 = Linear(in_features=300000, out_features=64)
  189.         self.fc2 = Linear(in_features=64, out_features=num_classes)
  190.     # 网络的前向计算过程
  191.     def forward(self, x, label=None):
  192.         x = self.conv1(x)
  193.         x = F.sigmoid(x)
  194.         x = self.max_pool1(x)
  195.         x = self.conv2(x)
  196.         x = F.sigmoid(x)        
  197.         x = self.max_pool2(x)
  198.         x = self.conv3(x)
  199.         x = F.sigmoid(x)          
  200.         x = paddle.reshape(x, [x.shape[0], -1])
  201.         x = self.fc1(x)
  202.         x = self.fc2(x)
  203.         if label is not None:
  204.             acc = paddle.metric.accuracy(input=x,label=label)
  205.             return x,acc
  206.         else:
  207.             return x
  208.  
  209.  
  210. # 数据的路径
  211. DATADIR = '/home/chen/deep_data/eye_ill/training/PALM-Training400'
  212. #创建模型
  213. model = LeNet(num_classes=1)
  214. #启动训练过程
  215. opt = paddle.optimizer.Momentum(learning_rate=0.001,momentum=0.9,
  216.                                 parameters=model.parameters())
  217. train_pm(model,optimizer=opt)
  218. evaluation(model,params_file_path='palm.pdparams')

（2）5个epoch的测试结果

4、基于paddle.nn的程序实现（修改了大佬代码，丢弃paddle.fluid） 

（1）看了一下paddle的api， paddle.fluid已经被丢弃

 （2）主要修改注意事项

4.1 Conv2D函数

Conv2D-API文档-PaddlePaddle深度学习平台

class paddle.nn.Conv2D(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, padding_mode='zeros', weight_attr=None, bias_attr=None, data_format='NCHW')

  • in_channels (int) - 输入图像的通道数。

  • out_channels (int) - 由卷积操作产生的输出的通道数。

  • kernel_size (int|list|tuple) - 卷积核大小。可以为单个整数或包含两个整数的元组或列表，分别表示卷积核的高和宽。如果为单个整数，表示卷积核的高和宽都等于该整数。

  • stride (int|list|tuple，可选) - 步长大小。可以为单个整数或包含两个整数的元组或列表，分别表示卷积沿着高和宽的步长。如果为单个整数，表示沿着高和宽的步长都等于该整数。默认值：1。

  • padding (int|list|tuple|str，可选) - 填充大小。如果它是一个字符串，可以是"VALID"或者"SAME"，表示填充算法，计算细节可参考上述 padding = "SAME"或 padding = "VALID" 时的计算公式。如果它是一个元组或列表，它可以有3种格式：(1)包含4个二元组：当 data_format 为"NCHW"时为 [[0,0], [0,0], [padding_height_top, padding_height_bottom], [padding_width_left, padding_width_right]]，当 data_format 为"NHWC"时为[[0,0], [padding_height_top, padding_height_bottom], [padding_width_left, padding_width_right], [0,0]]；(2)包含4个整数值：[padding_height_top, padding_height_bottom, padding_width_left, padding_width_right]；(3)包含2个整数值：[padding_height, padding_width]，此时padding_height_top = padding_height_bottom = padding_height， padding_width_left = padding_width_right = padding_width。若为一个整数，padding_height = padding_width = padding。默认值：0。

（2）fluid和nn的主要的修改对比如下

  1. self.conv1 = paddle.fluid.dygraph.nn.Conv2D(num_channels=3, num_filters=6, filter_size=5,act='sigmoid')
  2.  
  3. self.conv1= paddle.nn.Conv2D(in_channels=3,out_channels=6,kernel_size=5)  #sigmoid单独在forward中实现

4.2MaxPool2D

（1）函数说明

MaxPool2D-API文档-PaddlePaddle深度学习平台

paddle.nn.MaxPool2D(kernel_size, stride=None, padding=0, ceil_mode=False, return_mask=False, data_format='NCHW', name=None)

  • ernel_size (int|list|tuple): 池化核大小。如果它是一个元组或列表，它必须包含两个整数值， (pool_size_Height, pool_size_Width)。若为一个整数，则它的平方值将作为池化核大小，比如若pool_size=2, 则池化核大小为2x2。

  • stride (int|list|tuple，可选)：池化层的步长。如果它是一个元组或列表，它将包含两个整数，(pool_stride_Height, pool_stride_Width)。若为一个整数，则表示H和W维度上stride均为该值。默认值为None, 这时会使用kernel_size作为stride。

  • padding (str|int|list|tuple，可选) 池化填充。如果它是一个字符串，可以是"VALID"或者"SAME"，表示填充算法。如果它是一个元组或列表，它可以有3种格式：(1)包含2个整数值：[pad_height, pad_width]；(2)包含4个整数值：[pad_height_top, pad_height_bottom, pad_width_left, pad_width_right]；(3)包含4个二元组：当 data_format 为"NCHW"时为 [[0,0], [0,0], [pad_height_top, pad_height_bottom], [pad_width_left, pad_width_right]]，当 data_format 为"NHWC"时为[[0,0], [pad_height_top, pad_height_bottom], [pad_width_left, pad_width_right], [0,0]]。若为一个整数，则表示H和W维度上均为该值。默认值：0。

  • ceil_mode (bool，可选)：是否用ceil函数计算输出高度和宽度。如果是True，则使用 ceil 计算输出形状的大小。

  • return_mask (bool，可选)：是否返回最大索引和输出。默认为False.

  • data_format (str，可选)： 输入和输出的数据格式，可以是"NCHW"和"NHWC"。N是批尺寸，C是通道数，H是特征高度，W是特征宽度。默认值："NCHW"

  • name (str，可选)：函数的名字，默认为None.

 （2）fluid和nn的主要的修改对比如下

  1. self.pool1 = paddle.fluid.dygraph.nn.Pool2D(pool_size=2, pool_stride=2, pool_type='max')
  2.  
  3. self.pool1 = paddle.nn.MaxPool2D(kernel_size=2, stride=2)

4.3 Linear

Linear-API文档-PaddlePaddle深度学习平台

（1）函数定义

class paddle.nn.Linear(in_features, out_features, weight_attr=None, bias_attr=None, name=None)

  • in_features (int) – 线性变换层输入单元的数目。

  • out_features (int) – 线性变换层输出单元的数目。

  • weight_attr (ParamAttr, 可选) – 指定权重参数的属性。默认值为None，表示使用默认的权重参数属性，将权重参数初始化为0。具体用法请参见 ParamAttr 。

  • bias_attr (ParamAttr|bool, 可选) – 指定偏置参数的属性。 bias_attrbias_attr 为bool类型且设置为False时，表示不会为该层添加偏置。 bias_attrbias_attr 如果设置为True或者None，则表示使用默认的偏置参数属性，将偏置参数初始化为0。具体用法请参见 ParamAttr 。默认值为None。

  • name (str，可选) – 具体用法请参见 Name ，一般无需设置，默认值为None。

（2）fluid和nn的主要的修改对比如下

  1. self.fc1 = paddle.fluid.dygraph.nn.Linear(input_dim=300000, output_dim=64, act='sigmoid')
  2.  
  3. self.fc1 = paddle.nn.Linear(in_features=300000, out_features=64)#sigmoid单独在forward中实现

4.4sigmoid_cross_entropy_with_logits

binary_cross_entropy_with_logits-API文档-PaddlePaddle深度学习平台

（1）fluid.layers.sigmoid_cross_entropy_with_logits的定义

 （2）paddle.nn.functional.binary_cross_entropy_with_logits的定义

4.5 基于新版API的paddle.nn的实现

（1）测试代码

  1. import cv2
  2. import os
  3. import random
  4. import numpy as np
  5. #已丢弃
  6. # import paddle.fluid as fluid
  7. # from paddle.fluid.dygraph.nn import Conv2D, Pool2D, Linear
  8. import paddle
  9. import paddle.nn.functional as F
  10. from paddle.nn import Conv2D, MaxPool2D, Linear, Dropout
  11.  
  12. """
  13. 数据集iChallenge-PM（眼疾识别）
  14. """
  15.  
  16.  
  17. def transform_img(img):
  18.     img = cv2.resize(img, (224, 224))
  19.     img = np.transpose(img, (2, 0, 1))
  20.     img = img.astype('float32')
  21.     img = img / 255.
  22.     img = img * 2.0 - 1.0
  23.     return img
  24.  
  25.  
  26. def data_loader(datadir, batch_size=10, mode='train'):
  27.     filenames = os.listdir(datadir)
  28.  
  29.     def reader():
  30.         if mode == 'train':
  31.             random.shuffle(filenames)
  32.         batch_imgs = []
  33.         batch_labels = []
  34.         for name in filenames:
  35.             filepath = os.path.join(datadir, name)
  36.             img = cv2.imread(filepath)
  37.             img = transform_img(img)
  38.             if name[0] == 'H':
  39.                 label = 0
  40.             elif name[0] == 'N':
  41.                 label = 0
  42.             elif name[0] == 'P':
  43.                 label = 1
  44.             else:
  45.                 print('Not excepted file name')
  46.                 print(name[0])
  47.                 exit(-1)
  48.             batch_imgs.append(img)
  49.             batch_labels.append(label)
  50.             if len(batch_imgs) == batch_size:
  51.                 imgs_array = np.array(batch_imgs).astype('float32')
  52.                 labels_array = np.array(batch_labels).astype(
  53.                     'float32').reshape(-1, 1)
  54.                 yield imgs_array, labels_array
  55.                 batch_imgs = []
  56.                 batch_labels = []
  57.         if len(batch_imgs) > 0:
  58.             imgs_array = np.array(batch_imgs).astype('float32')
  59.             labels_array = np.array(batch_labels).astype('float32').reshape(
  60.                 -1, 1)
  61.             yield imgs_array, labels_array
  62.  
  63.     return reader
  64.  
  65.  
  66. def valid_data_loader(datadir, csvfile, batch_size=10, mode='valid'):
  67.     filelists = open(csvfile).readlines()
  68.  
  69.     def reader():
  70.         batch_imgs = []
  71.         batch_labels = []
  72.         for line in filelists[1:]:
  73.             line = line.strip().split(',')
  74.             name = line[1]
  75.             label = int(line[2])
  76.             filepath = os.path.join(datadir, name)
  77.             img = cv2.imread(filepath)
  78.             img = transform_img(img)
  79.             batch_imgs.append(img)
  80.             batch_labels.append(label)
  81.             if len(batch_imgs) == batch_size:
  82.                 imgs_array = np.array(batch_imgs).astype('float32')
  83.                 labels_array = np.array(batch_labels).astype(
  84.                     'float32').reshape(-1, 1)
  85.                 yield imgs_array, labels_array
  86.                 batch_imgs = []
  87.                 batch_labels = []
  88.         if len(batch_imgs) > 0:
  89.             imgs_array = np.array(batch_imgs).astype('float32')
  90.             labels_array = np.array(batch_labels).astype('float32').reshape(
  91.                 -1, 1)
  92.             yield imgs_array, labels_array
  93.  
  94.     return reader
  95.  
  96.  
  97. DATADIR = 'D:/pytorch_learning2022/data/eye/PALM-Training400'
  98. DATADIR2 = 'D:/pytorch_learning2022/data/eye/PALM-Validation400'
  99. CSCVFILE = 'D:/pytorch_learning2022/data/eye/PM_Label_and_Fovea_Location.csv'
  100.  
  101.  
  102. # 定义LeNet的网络结构
  103. class LeNet(paddle.nn.Layer):
  104.     def __init__(self, name_scope, num_classes=1):
  105.         super(LeNet, self).__init__(name_scope)
  106.  
  107.         # self.conv1 = paddle.fluid.dygraph.nn.Conv2D(num_channels=3, num_filters=6, filter_size=5,act='sigmoid')
  108.         # self.conv1= paddle.nn.Conv2D(in_channels=3,out_channels=6,kernel_size=5)  #sigmoid单独在forward中实现
  109.         self.conv1= Conv2D(in_channels=3,out_channels=6,kernel_size=5)  #sigmoid单独在forward中实现
  110.         # self.pool1 = paddle.fluid.dygraph.nn.Pool2D(pool_size=2, pool_stride=2, pool_type='max')
  111.         # self.pool1 = paddle.nn.MaxPool2D(kernel_size=2, stride=2)
  112.         self.pool1 = MaxPool2D(kernel_size=2, stride=2)
  113.  
  114.         # self.conv2 = Conv2D(num_channels=6,num_filters=16,filter_size=5,act='sigmoid')
  115.         self.conv2= Conv2D(in_channels=6,out_channels=16,kernel_size=5)  #sigmoid单独在forward中实现        
  116.         # self.pool2 = Pool2D(pool_size=2, pool_stride=2, pool_type='max')
  117.         self.pool2 = MaxPool2D(kernel_size=2, stride=2)
  118.         # self.conv3 = Conv2D(num_channels=16,num_filters=120, filter_size=4, act='sigmoid')
  119.         self.conv3= Conv2D(in_channels=16,out_channels=120,kernel_size=4)  #sigmoid单独在forward中实现 
  120.                
  121.         # self.fc1 = paddle.fluid.dygraph.nn.Linear(input_dim=300000, output_dim=64, act='sigmoid')
  122.         # self.fc1 = paddle.nn.Linear(in_features=300000, out_features=64)#sigmoid单独在forward中实现 
  123.         self.fc1 = Linear(in_features=300000, out_features=64) #sigmoid单独在forward中实现 
  124.         # self.fc2 = Linear(input_dim=64, output_dim=num_classes)
  125.         self.fc2 = Linear(in_features=64, out_features=num_classes) 
  126.  
  127.     def forward(self, x):
  128.         x = self.conv1(x)
  129.         x = F.sigmoid(x)
  130.         x = self.pool1(x)
  131.  
  132.         x = self.conv2(x)
  133.         x = F.sigmoid(x)
  134.         x = self.pool2(x)
  135.         x = self.conv3(x)
  136.         x = F.sigmoid(x)
  137.  
  138.         # x = fluid.layers.reshape(x, [x.shape[0], -1])
  139.         x = paddle.reshape(x, [x.shape[0], -1])
  140.         x = self.fc1(x)
  141.         x = F.sigmoid(x)
  142.         x = self.fc2(x)
  143.         return x
  144.  
  145.  
  146. # 定义AlexNet网络结构
  147. class AlexNet(paddle.nn.Layer):
  148.     def __init__(self, name_scope, num_classes=1):
  149.         super(AlexNet, self).__init__(name_scope)
  150.         name_scope = self.full_name
  151.         # self.conv1 = Conv2D(num_channels=3, num_filters=96,filter_size=11, stride=4, padding=5,act='relu')
  152.         self.conv1= Conv2D(in_channels=3,out_channels=96,kernel_size=5,stride=4,padding=5)  #relu单独在forward中实现
  153.         # self.pool1 = Pool2D(pool_size=2, pool_stride=2, pool_type='max')
  154.         self.pool1 = MaxPool2D(kernel_size=2, stride=2)
  155.         
  156.         # self.conv2 = Conv2D(num_channels=96,num_filters=256,filter_size=5,stride=1,padding=2, act='relu')
  157.         self.conv2= Conv2D(in_channels=96,out_channels=256,kernel_size=5,stride=1,padding=2)  #relu单独在forward中实现        
  158.         # self.pool2 = Pool2D(pool_size=2, pool_stride=2, pool_type='max')
  159.         self.pool2 = MaxPool2D(kernel_size=2, stride=2)
  160.         
  161.         # self.conv3 = Conv2D(num_channels=256,num_filters=384,filter_size=3,stride=1,padding=1,act='relu')
  162.         self.conv3 = Conv2D(in_channels=256,out_channels=384,kernel_size=3,stride=1,padding=1)  #relu单独在forward中实现                
  163.         # self.conv4 = Conv2D(num_channels=384,num_filters=384,filter_size=3,stride=1,padding=1,act='relu')
  164.         self.conv4 = Conv2D(in_channels=384,out_channels=384,kernel_size=3,stride=1,padding=1)  #relu单独在forward中实现                        
  165.         # self.conv5 = Conv2D(num_channels=384,num_filters=256,filter_size=3,stride=1,padding=1,act='relu')
  166.         self.conv5 = Conv2D(in_channels=384,out_channels=256,kernel_size=3,stride=1,padding=1)  #relu单独在forward中实现                        
  167.         # self.pool5 = Pool2D(pool_size=2, pool_stride=2, pool_type='max')
  168.         self.pool5 = MaxPool2D(kernel_size=2, stride=2)        
  169.  
  170.         # self.fc1 = Linear(input_dim=12544, output_dim=4096, act='relu')
  171.         self.fc1 = Linear(in_features=12544, out_features=4096) #relu单独在forward中实现 
  172.         self.drop_ratio1 = 0.5
  173.         # self.fc2 = Linear(input_dim=4096, output_dim=4096, act='relu')
  174.         self.fc2 = Linear(in_features=4096, out_features=4096) #relu单独在forward中实现 
  175.         self.drop_ratio2 = 0.5
  176.         # self.fc3 = Linear(input_dim=4096, output_dim=num_classes)
  177.         self.fc3 = Linear(in_features=4096, out_features=num_classes) 
  178.  
  179.     def forward(self, x):
  180.         x = self.conv1(x)
  181.         x = F.relu(x)
  182.         x = self.pool1(x)
  183.         x = self.conv2(x)
  184.         x = F.relu(x)
  185.         x = self.pool2(x)
  186.         x = self.conv3(x)
  187.         x = F.relu(x)
  188.         x = self.conv4(x)
  189.         x = F.relu(x)
  190.         x = self.conv5(x)
  191.         x = F.relu(x)
  192.         x = self.pool5(x)
  193.  
  194.         # x = fluid.layers.reshape(x, [x.shape[0], -1])
  195.         x = paddle.reshape(x, [x.shape[0], -1])
  196.         x = self.fc1(x)
  197.         x = F.relu(x)
  198.         x = F.dropout(x, self.drop_ratio1)
  199.         x = self.fc2(x)
  200.         x = F.relu(x)
  201.         x = F.dropout(x, self.drop_ratio2)
  202.         x = self.fc3(x)
  203.         return x
  204.  
  205.  
  206. # 定义训练过程
  207. def train(model):
  208.     print("---- start training ----")
  209.     model.train()
  210.     epoch_num = 5
  211.     # opt = fluid.optimizer.Momentum(learning_rate=0.001,momentum=0.9, parameter_list=model.parameters())
  212.     opt = paddle.optimizer.Momentum(learning_rate=0.001, momentum=0.9, parameters=model.parameters())
  213.    
  214.     train_loader = data_loader(DATADIR, batch_size=10, mode='train')
  215.     valid_loader = valid_data_loader(DATADIR2, CSCVFILE)
  216.     for epoch in range(epoch_num):
  217.         for batch_id, data in enumerate(train_loader()):
  218.             x_data, y_data = data
  219.             img = paddle.to_tensor(x_data)
  220.             label = paddle.to_tensor(y_data)
  221.             logits = model(img)
  222.             # loss = fluid.layers.sigmoid_cross_entropy_with_logits(logits, label)
  223.             loss = F.binary_cross_entropy_with_logits(logits, label)  #是否有问题待确认
  224.             avg_loss = paddle.mean(loss)
  225.  
  226.             if batch_id % 10 == 0:
  227.                 print("epoch: {}, batch_id: {}, loss is: {}".format(
  228.                     epoch, batch_id, avg_loss.numpy()))
  229.             avg_loss.backward()
  230.             opt.minimize(avg_loss)
  231.             model.clear_gradients()
  232.  
  233.         model.eval()
  234.         accuracies = []
  235.         losses = []
  236.         for batch_id, data in enumerate(valid_loader()):
  237.             x_data, y_data = data
  238.             img = paddle.to_tensor(x_data)
  239.             label = paddle.to_tensor(y_data)
  240.             logits = model(img)
  241.             pred = F.sigmoid(logits)
  242.             # loss = fluid.layers.sigmoid_cross_entropy_with_logits(logits, label)
  243.             loss = F.binary_cross_entropy_with_logits(logits, label)
  244.  
  245.             pred2 = pred * (-1.0) + 1.0
  246.             # pred = fluid.layers.concat([pred2, pred], axis=1)
  247.             pred = paddle.concat([pred2, pred], axis=1)
  248.             # acc = fluid.layers.accuracy(pred, fluid.layers.cast(label, dtype='int64'))
  249.             acc = paddle.metric.accuracy(pred, paddle.cast(label,dtype='int64'))
  250.             accuracies.append(acc.numpy())
  251.             losses.append(loss.numpy())
  252.         print("[validation accuracy/loss: {}/{}]".format(
  253.             np.mean(accuracies), np.mean(losses)))
  254.         model.train()
  255.     # fluid.save_dygraph(model.state_dict(), 'D:/pytorch_learning2022/00_01paddle_learning/result/iChallengePM')
  256.     # fluid.save_dygraph(opt.state_dict(), 'D:/pytorch_learning2022/00_01paddle_learning/result/iChallengePM')
  257.     paddle.save(model.state_dict(),'palm.pdparams')        
  258.     paddle.save(opt.state_dict(),'palm.pdopt')
  259.  
  260. if __name__ == "__main__":
  261.     # use_gpu = True
  262.     use_gpu = False
  263.     paddle.device.set_device('gpu:0') if use_gpu else paddle.device.set_device('cpu')
  264.     # with fluid.dygraph.guard():
  265.         # model = LeNet("LeNet")
  266.         # model = AlexNet("AlexNet")
  267.     
  268.  
  269.     # model = LeNet("LeNet")
  270.     model = AlexNet("AlexNet")
  271.     train(model)
  272.

5、使用paddle.vision中的模型来测试分类

PS：看起来paddle和torch没什么区别

（1）可以直接只用很多模型

from paddle.vision.models import resnet50,vgg16,mobilenet_v3_small,LeNet,GoogLeNet

（2）测试的时候可以直接给定分类和是否使用预训练和直接给定分类个数

    model =resnet50(pretrained=True,num_classes=1)

（3）完整测试代码如下

  1. import cv2
  2. import os
  3. import random
  4. import numpy as np
  5. #已丢弃
  6. # import paddle.fluid as fluid
  7. # from paddle.fluid.dygraph.nn import Conv2D, Pool2D, Linear
  8. import paddle
  9. import paddle.nn.functional as F
  10. from paddle.nn import Conv2D, MaxPool2D, Linear, Dropout
  11. from paddle.vision.models import resnet50,vgg16,mobilenet_v3_small,LeNet,GoogLeNet
  12.  
  13.  
  14. """
  15. 数据集iChallenge-PM（眼疾识别）
  16. """
  17.  
  18.  
  19. def transform_img(img):
  20.     img = cv2.resize(img, (224, 224))
  21.     img = np.transpose(img, (2, 0, 1))
  22.     img = img.astype('float32')
  23.     img = img / 255.
  24.     img = img * 2.0 - 1.0
  25.     return img
  26.  
  27.  
  28. def data_loader(datadir, batch_size=10, mode='train'):
  29.     filenames = os.listdir(datadir)
  30.  
  31.     def reader():
  32.         if mode == 'train':
  33.             random.shuffle(filenames)
  34.         batch_imgs = []
  35.         batch_labels = []
  36.         for name in filenames:
  37.             filepath = os.path.join(datadir, name)
  38.             img = cv2.imread(filepath)
  39.             img = transform_img(img)
  40.             if name[0] == 'H':
  41.                 label = 0
  42.             elif name[0] == 'N':
  43.                 label = 0
  44.             elif name[0] == 'P':
  45.                 label = 1
  46.             else:
  47.                 print('Not excepted file name')
  48.                 print(name[0])
  49.                 exit(-1)
  50.             batch_imgs.append(img)
  51.             batch_labels.append(label)
  52.             if len(batch_imgs) == batch_size:
  53.                 imgs_array = np.array(batch_imgs).astype('float32')
  54.                 labels_array = np.array(batch_labels).astype(
  55.                     'float32').reshape(-1, 1)
  56.                 yield imgs_array, labels_array
  57.                 batch_imgs = []
  58.                 batch_labels = []
  59.         if len(batch_imgs) > 0:
  60.             imgs_array = np.array(batch_imgs).astype('float32')
  61.             labels_array = np.array(batch_labels).astype('float32').reshape(
  62.                 -1, 1)
  63.             yield imgs_array, labels_array
  64.  
  65.     return reader
  66.  
  67.  
  68. def valid_data_loader(datadir, csvfile, batch_size=10, mode='valid'):
  69.     filelists = open(csvfile).readlines()
  70.  
  71.     def reader():
  72.         batch_imgs = []
  73.         batch_labels = []
  74.         for line in filelists[1:]:
  75.             line = line.strip().split(',')
  76.             name = line[1]
  77.             label = int(line[2])
  78.             filepath = os.path.join(datadir, name)
  79.             img = cv2.imread(filepath)
  80.             img = transform_img(img)
  81.             batch_imgs.append(img)
  82.             batch_labels.append(label)
  83.             if len(batch_imgs) == batch_size:
  84.                 imgs_array = np.array(batch_imgs).astype('float32')
  85.                 labels_array = np.array(batch_labels).astype(
  86.                     'float32').reshape(-1, 1)
  87.                 yield imgs_array, labels_array
  88.                 batch_imgs = []
  89.                 batch_labels = []
  90.         if len(batch_imgs) > 0:
  91.             imgs_array = np.array(batch_imgs).astype('float32')
  92.             labels_array = np.array(batch_labels).astype('float32').reshape(
  93.                 -1, 1)
  94.             yield imgs_array, labels_array
  95.  
  96.     return reader
  97.  
  98.  
  99. DATADIR = 'D:/pytorch_learning2022/data/eye/PALM-Training400'
  100. DATADIR2 = 'D:/pytorch_learning2022/data/eye/PALM-Validation400'
  101. CSCVFILE = 'D:/pytorch_learning2022/data/eye/PM_Label_and_Fovea_Location.csv'
  102.  
  103.  
  104.  
  105. # 定义训练过程
  106. def train(model):
  107.     print("---- start training ----")
  108.     model.train()
  109.     epoch_num = 5
  110.     # opt = fluid.optimizer.Momentum(learning_rate=0.001,momentum=0.9, parameter_list=model.parameters())
  111.     opt = paddle.optimizer.Momentum(learning_rate=0.001, momentum=0.9, parameters=model.parameters())
  112.    
  113.     train_loader = data_loader(DATADIR, batch_size=10, mode='train')
  114.     valid_loader = valid_data_loader(DATADIR2, CSCVFILE)
  115.     for epoch in range(epoch_num):
  116.         for batch_id, data in enumerate(train_loader()):
  117.             x_data, y_data = data
  118.             img = paddle.to_tensor(x_data)
  119.             label = paddle.to_tensor(y_data)
  120.             logits = model(img)
  121.             # loss = fluid.layers.sigmoid_cross_entropy_with_logits(logits, label)
  122.             loss = F.binary_cross_entropy_with_logits(logits, label)  #是否有问题待确认
  123.             avg_loss = paddle.mean(loss)
  124.  
  125.             if batch_id % 10 == 0:
  126.                 print("epoch: {}, batch_id: {}, loss is: {}".format(
  127.                     epoch, batch_id, avg_loss.numpy()))
  128.             avg_loss.backward()
  129.             opt.minimize(avg_loss)
  130.             model.clear_gradients()
  131.  
  132.         model.eval()
  133.         accuracies = []
  134.         losses = []
  135.         for batch_id, data in enumerate(valid_loader()):
  136.             x_data, y_data = data
  137.             img = paddle.to_tensor(x_data)
  138.             label = paddle.to_tensor(y_data)
  139.             logits = model(img)
  140.             pred = F.sigmoid(logits)
  141.             # loss = fluid.layers.sigmoid_cross_entropy_with_logits(logits, label)
  142.             loss = F.binary_cross_entropy_with_logits(logits, label)
  143.  
  144.             pred2 = pred * (-1.0) + 1.0
  145.             # pred = fluid.layers.concat([pred2, pred], axis=1)
  146.             pred = paddle.concat([pred2, pred], axis=1)
  147.             # acc = fluid.layers.accuracy(pred, fluid.layers.cast(label, dtype='int64'))
  148.             acc = paddle.metric.accuracy(pred, paddle.cast(label,dtype='int64'))
  149.             accuracies.append(acc.numpy())
  150.             losses.append(loss.numpy())
  151.         print("[validation accuracy/loss: {}/{}]".format(
  152.             np.mean(accuracies), np.mean(losses)))
  153.         model.train()
  154.     # fluid.save_dygraph(model.state_dict(), 'D:/pytorch_learning2022/00_01paddle_learning/result/iChallengePM')
  155.     # fluid.save_dygraph(opt.state_dict(), 'D:/pytorch_learning2022/00_01paddle_learning/result/iChallengePM')
  156.     paddle.save(model.state_dict(),'palm.pdparams')        
  157.     paddle.save(opt.state_dict(),'palm.pdopt')
  158.  
  159. if __name__ == "__main__":
  160.     # use_gpu = True
  161.     use_gpu = False
  162.     paddle.device.set_device('gpu:0') if use_gpu else paddle.device.set_device('cpu')
  163.     # model =resnet50()
  164.     model =resnet50(pretrained=True,num_classes=1)
  165.     train(model)
  166.

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/小丑西瓜9/article/detail/92722
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号