devbox
Monodyee
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

【PyTorch][chapter 22][李宏毅深度学习][ WGAN]【实战三】

作者：Monodyee | 2024-03-23 14:21:55

赞

踩

【PyTorch][chapter 22][李宏毅深度学习][ WGAN]【实战三】

前言：

      本篇主要讲两个WGAN的两个例子：

     1   高斯混合模型 WGAN实现

      2   MNIST 手写数字识别 -WGAN 实现

     WGAN 训练起来蛮麻烦的,如果要获得好的效果很多超参数需要手动设置

1： 噪声的维度

2:    学习率

3： 生成器，鉴别器网络模型

4:    batchsz,num_epochs

目录：

  1.     Google Colab
  2.     损失函数
  3.     高斯混合模型（WGAN 实现）
  4.     MNIST 手写数字识别（WGAN 实现）

   一   Google Colab

     1.1 : 打开google 云盘
          https://drive.google.com/drive/my-drive

     1.2:  新建 wgan.ipynb 文件
          把对应的python 脚本拖在该目录下面

     


     
1.3：打开colab 


            https://colab.research.google.com/drive/

             新建笔记

1.4:  在colab 中运行 main.py


from google.colab import drive
import os
drive.mount('/content/drive')
os.chdir('/content/drive/My Drive/wgan.ipynb/')
%run main.py

二 WGAN 损失函数

2.1 Wasserstein 约束和 WGAN 的约束条件转换原理

       我们知道WGAN 是根据Wasserstein Distance 推导出来的。

 Wasserstein Distance 原始约束关系为f(x)+g(y)\leq c(x,y)

只要满足k-Lipschitz 约束条件，肯定可以满足原始的约束条件。

证明如下：

这种约束关系很难求解,一般采用Weight Clipping 或者  Gradient penalty 两种方案

来约束.

  2.2 Weight Clipping

          这是一种工程经验，无理论基础

          1-Lipschitz 约束条件为：

                    ||f(x)-f(y)||\leq K||x-y||,\forall x,y

          一张1024*1024的灰度图,其状态变量为256^{1024*1024},要让所有的状态组合满足该约束条件

没有办法求解。早期的解决方案是做weight Clipping

            在利用 gradient descent 进行参数更新后，在对所有参数进行如下操作：

           w=\left\{\begin{matrix} c,\, \, \, \, \, if w>c\\ -c,\, \, \, \, \, if w<-c \\ w,\, \, \, \, \, otherwise \end{matrix}\right.

          通过约束w 范围,来约束f(x),f(y) 输出范围，效果比较好。

     2.3  Gradient penalty

       这是一种工程经验，无严格理论基础

          问题：

          weight clipping会导致很容易一不小心就梯度消失或者梯度爆炸。原因是判别器是一个多层网络，如果我们把clipping threshold设得稍微小了一点，每经过一层网络，梯度就变小一点点，多层之后就会指数衰减；反之，如果设得稍微大了一点，每经过一层网络，梯度变大一点点，多层之后就会指数爆炸。只有设得不大不小，才能让生成器获得恰到好处的回传梯度，然而在实际应用中这个平衡区域可能很狭窄，就会给调参工作带来麻烦

三    高斯混合模型（WGAN 实现）

  3.1 模型部分代码

     model.py

  1. # -*- coding: utf-8 -*-
  2. """
  3. Created on Tue Mar 19 10:50:31 2024
  5. @author: chengxf2
  6. """
  7.  
  8. import torch
  9. from torch  import nn
  10.  
  11. import random #numpy 里面的库
  12. from torchsummary import summary
  13.  
  14.  
  15.  
  16.  
  17. class Generator(nn.Module):
  18.     
  19.     
  20.     def __init__(self,z_dim=2,h_dim=400):
  21.         
  22.         super(Generator, self).__init__()
  23.         #z:[batch, z_dim]
  24.         self.net = nn.Sequential(
  25.             nn.Linear(z_dim, h_dim),
  26.             nn.ReLU(True),
  27.             nn.Linear(h_dim, h_dim),
  28.             nn.ReLU(True),
  29.             nn.Linear(h_dim, 2)
  30.             )
  31.         
  32.     def forward(self, z):
  33.         #print("\n input.shape",z.shape)
  34.         output = self.net(z)
  35.         return output
  36.     
  37. class Discriminator(nn.Module):
  38.     
  39.     def __init__(self,input_dim,h_dim):
  40.         
  41.         
  42.         super(Discriminator,self).__init__()
  43.         
  44.         self.net = nn.Sequential(
  45.             nn.Linear(input_dim, h_dim),
  46.             nn.ReLU(True),
  47.             nn.Linear(h_dim, h_dim),
  48.             nn.ReLU(True),
  49.             nn.Linear(h_dim, h_dim),
  50.             nn.ReLU(True),
  51.             nn.Linear(h_dim, 1),
  52.             nn.Tanh()
  53.             )
  54.         
  55.     def  forward(self, x):
  56.         
  57.          out = self.net(x)
  58.          return out.view(-1)
  59.      
  60. def model_summary():
  61.     device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')    
  62.     #[channel, w,h]
  63.     #summary(model=net, input_size=(3,32,32),batch_size=2, device="cpu")
  64.     summary(Generator(2,100).to(device), (1,2,),batch_size=5)
  65.     print(Generator(100))
  66.     print("\n 鉴别器")
  67.     summary(Discriminator(2,100).to(device) , (2,2))
  68.     
  69.  
  70.

3.2  训练部分代码

     main.py

  1. # -*- coding: utf-8 -*-
  2. """
  3. Created on Tue Mar 19 11:06:37 2024
  5. @author: chengxf2
  6. """
  7.  
  8. import torch
  9. from torch   import autograd,optim,autograd
  10. import numpy as np
  11. import visdom
  12. import random
  13. from model import Generator,Discriminator
  14. import visdom
  15. import matplotlib.pyplot as plt
  16.  
  17.  
  18. batchsz = 512
  19. H_dim = 400
  20. viz = visdom.Visdom()
  21. device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu') 
  22.  
  23. def data_generator():
  24.     
  25.    
  26.     #8个高斯分布的中心点
  27.     scale = 2
  28.     centers = [
  29.         (1, 0),
  30.         (-1, 0),
  31.         (0, 1),
  32.         (0, -1),
  33.         (1. / np.sqrt(2), 1. / np.sqrt(2)),
  34.         (1. / np.sqrt(2), -1. / np.sqrt(2)),
  35.         (-1. / np.sqrt(2), 1. / np.sqrt(2)),
  36.         (-1. / np.sqrt(2), -1. / np.sqrt(2))
  37.     ]
  38.     # 放大一下
  39.     centers = [(scale * x, scale * y) for x, y in centers]
  40.  
  41.     while True:
  42.         dataset = []
  43.  
  44.         for i in range(batchsz):
  45.             
  46.             #随机生成一个点
  47.             point = np.random.randn(2) * 0.02
  48.             #随机选择一个高斯分布
  49.             center = random.choice(centers)
  50.             #N(0,1)+center (x,y)
  51.             point[0] += center[0]
  52.             point[1] += center[1]
  53.             dataset.append(point)
  54.  
  55.         dataset = np.array(dataset).astype(np.float32)
  56.         dataset /= 1.414
  57.         yield  dataset
  58.         
  59. def generate_image(D, G, xr, epoch):
  60.     """
  61.     Generates and saves a plot of the true distribution, the generator, and the
  62.     critic.
  63.     """
  64.     N_POINTS = 128
  65.     RANGE = 3
  66.     plt.clf()
  67.  
  68.     points = np.zeros((N_POINTS, N_POINTS, 2), dtype='float32')
  69.     points[:, :, 0] = np.linspace(-RANGE, RANGE, N_POINTS)[:, None]
  70.     points[:, :, 1] = np.linspace(-RANGE, RANGE, N_POINTS)[None, :]
  71.     points = points.reshape((-1, 2))
  72.     # (16384, 2)
  73.     # print('p:', points.shape)
  74.  
  75.     # draw contour
  76.     with torch.no_grad():
  77.         points = torch.Tensor(points).to(device) # [16384, 2]
  78.         disc_map = D(points).to(device).numpy() # [16384]
  79.         
  80.     x = y = np.linspace(-RANGE, RANGE, N_POINTS)
  81.     cs = plt.contour(x, y, disc_map.reshape((len(x), len(y))).transpose())
  82.     plt.clabel(cs, inline=1, fontsize=10)
  83.     plt.colorbar()
  84.  
  85.  
  86.     # draw samples
  87.     with torch.no_grad():
  88.         z = torch.randn(batchsz, 2).to(device) # [b, 2]
  89.         samples = G(z).to(device).numpy() # [b, 2]
  90.         
  91.     plt.scatter(xr[:, 0], xr[:, 1], c='green', marker='.')
  92.     plt.scatter(samples[:, 0], samples[:, 1], c='red', marker='+')
  93.  
  94.     viz.matplot(plt, win='contour', opts=dict(title='p(x):%d'%epoch))
  95.  
  96. def gradient_penalty(D,xr,xf):
  97.     
  98.     LAMBDA  = 0.2
  99.     
  100.     t = torch.rand(batchsz,1).to(device)
  101.     #[b,1]=>[b,2]
  102.     t = t.expand_as(xf)
  103.     #interpolation 
  104.     mid = t*xr+(1-t)*xf
  105.     #需要对 mid 求导
  106.     mid.requires_grad_() 
  107.     
  108.     pred = D(mid)
  109.     grad = autograd.grad(outputs=pred, 
  110.                          inputs= mid,
  111.                          grad_outputs= torch.ones_like(pred) ,
  112.                          create_graph= True,
  113.                          retain_graph = True,
  114.                          only_inputs=True)[0]
  115.     gp = torch.pow((grad.norm(2, dim=1)-1),2).mean()
  116.     
  117.     return gp*LAMBDA
  118.  
  119. def gen():
  120.     ## 读取模型
  121.     z_dim=2
  122.     h_dim=400
  123.     model = Generator(z_dim,h_dim ).to(device)
  124.     state_dict = torch.load('Generator.pt')
  125.     model.load_state_dict(state_dict)
  126.     model.eval()
  127.     
  128.     z = torch.randn(batchsz, 2).to(device)
  129.     #tf.stop_graident()
  130.     xf = model(z)
  131.     print(xf)
  132.     
  133.  
  134. def main():
  135.     z_dim=2
  136.     h_dim=400
  137.     input_dim =2
  138.     np.random.seed(23)
  139.     num_epochs  = 2
  140.  
  141.  
  142.  
  143.     data_iter = data_generator()
  144.        
  145.     
  146.     torch.manual_seed(23)
  147.     
  148.     G = Generator(z_dim,h_dim ).to(device)
  149.     D = Discriminator(input_dim, h_dim).to(device)
  150.     
  151.     #print(G)
  152.     #print(D)
  153.     optim_G = optim.Adam(G.parameters(), lr = 5e-4, betas =(0.5,0.9))
  154.     optim_D = optim.Adam(D.parameters(), lr = 5e-4, betas =(0.5,0.9))
  155.     
  156.     viz.line([[0,0]], [0], win='loss', opts=dict(title='loss',
  157.                                                 legend=['D', 'G']))
  158.     for epoch in range(num_epochs):
  159.         
  160.         #1. train Discrimator firstly
  161.         
  162.         for  _ in range(5):
  163.             # 1.1 train on real data
  164.             x = next(data_iter)
  165.             xr = torch.from_numpy(x).to(device)
  166.             #[batch_size, 2]=>[batch, 1]
  167.             predr = D(xr)
  168.             #max predr , min lossr
  169.             lossr = -predr.mean()
  170.             
  171.             #1.2  train on fake data
  172.             z = torch.randn(batchsz, 2).to(device)
  173.             #tf.stop_graident()
  174.             xf = G(z).detach()
  175.             predf = D(xf)
  176.             lossf = predf.mean()
  177.             
  178.             #1.3 gradient penalty
  179.             gp = gradient_penalty(D,xr,xf.detach())
  180.             
  181.             #1.4 aggregate all
  182.             loss_D = lossr+lossf+gp
  183.             
  184.             #1.5 optimize
  185.             optim_D.zero_grad()
  186.             loss_D.backward()
  187.             optim_D.step()
  188.         
  189.         #2 train Generator secondly
  190.         z = torch.randn(batchsz, 2).to(device)
  191.         #tf.stop_graident()
  192.         xf = G(z)
  193.         predf = D(xf)
  194.         loss_G = - predf.mean()
  195.         
  196.         #optimize
  197.         optim_G.zero_grad() 
  198.         loss_G.backward()
  199.         optim_G.step()
  200.         
  201.         if epoch%100 ==0:
  202.             viz.line([[loss_D.item(), loss_G.item()]],[epoch],win='loss',update='append')
  203.             print(f"loss_D {loss_D.item()} \t ,loss_G {loss_G.item()}")
  204.             generate_image(D,G,x, epoch)
  205.             
  206.     print("\n train end")            
  207.     #二、只保存模型中的参数并读取
  208.     torch.save(G.state_dict(), 'Generator.pt')
  209.     torch.save(G.state_dict(), 'Discriminator.pt')
  210.  
  211.         
  212.         #2  train Generator
  213. #http://www.manongjc.com/detail/42-hvxyfyduytmpwzz.html   
  214.     
  215. gen()

四    MNIST 手写数字识别

    4.1  模型部分

          model.py  

  1. # -*- coding: utf-8 -*-
  2. """
  3. Created on Mon Mar 18 10:19:26 2024
  5. @author: chengxf2
  6. """
  7. import torch.nn as nn
  8. from torchsummary import summary
  9. import torch
  10.  
  11.  
  12. class Generator(nn.Module):
  13.  
  14.     def __init__(self, z_dim=10, im_chan=1, hidden_dim=64):
  15.         super(Generator, self).__init__()
  16.         
  17.         self.z_dim = z_dim
  18.         
  19.         self.gen = nn.Sequential(
  20.             self.layer1(z_dim, hidden_dim * 4,kernel_size=3, stride=2),
  21.             self.layer1(hidden_dim * 4, hidden_dim * 2,kernel_size=4,stride = 1),
  22.             self.layer1(hidden_dim * 2,hidden_dim ,kernel_size=3,stride = 2, ),
  23.             self.layer2(hidden_dim,im_chan,kernel_size=4,stride=2))
  24.         
  25.         
  26.     def layer1(self, input_channel, output_channel, kernel_size, stride = 1, padding = 0):
  27.         #inplace = true， 就相当于在原内存计算
  28.         return nn.Sequential(
  29.                 nn.ConvTranspose2d(input_channel, output_channel, kernel_size, stride, padding),
  30.                 nn.BatchNorm2d(output_channel),
  31.                 nn.ReLU(inplace=True),
  32.         )
  33.     
  34.     
  35.     def layer2(self, input_channel, output_channel, kernel_size, stride = 1, padding = 0):
  36.         #双曲正切函数的输出范围为(-1，1)
  37.         return  nn.Sequential(
  38.                 nn.ConvTranspose2d(input_channel, output_channel, kernel_size, stride, padding),
  39.                 nn.Tanh()
  40.             )
  41.     
  42.     
  43.     def forward(self, noise):
  44.         '''
  46.         Parameters
  47.         ----------
  48.         noise : [batch, z_dim]
  50.         Returns
  51.         -------
  52.         输出的是图片[batch, channel, width, height]
  54.         '''
  55.       
  56.         x = noise.view(len(noise), self.z_dim, 1, 1)
  57.         return self.gen(x)
  58.     
  59. class Discriminator(nn.Module):
  60.  
  61.     def __init__(self, im_chan=1, hidden_dim=16):
  62.         
  63.         super(Discriminator, self).__init__()
  64.         self.disc = nn.Sequential(
  65.             self.block1(im_chan,hidden_dim * 4,kernel_size=4,stride=2),
  66.             
  67.             self.block1(hidden_dim * 4,hidden_dim * 8,kernel_size=4,stride=2,),
  68.             
  69.             self.block2(hidden_dim * 8,1,kernel_size=4,stride=2,),
  70.  
  71.         )
  72.  
  73.         
  74.     def block1(self, input_channel, output_channel, kernel_size, stride = 1, padding = 0):
  75.         return nn.Sequential(
  76.                 nn.Conv2d(input_channel, output_channel, kernel_size, stride, padding),
  77.                 nn.BatchNorm2d(output_channel),
  78.                 nn.LeakyReLU(0.2, inplace=True)
  79.         )
  80.     
  81.     
  82.     def block2(self, input_channel, output_channel, kernel_size, stride = 1, padding = 0):
  83.         return  nn.Sequential(
  84.                 nn.Conv2d(input_channel, output_channel, kernel_size, stride, padding),
  85.             )
  86.     
  87.     def forward(self, image):
  88.         
  89.         return self.disc(image)
  90.     
  91.  
  92.     
  93. def model_summary():
  94.     device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')    
  95.     summary(Generator(100).to(device), (100,))
  96.     print(Generator(100))
  97.     print("\n 鉴别器")
  98.     summary(Discriminator().to(device) , (1,28,28))
  99. model_summary()
  100.     
  101.

    4.2  训练部分

         main.py

  1. # -*- coding: utf-8 -*-
  2. """
  3. Created on Mon Mar 18 10:37:21 2024
  5. @author: chengxf2
  6. """
  7.  
  8. import torch
  9. from   model import Generator
  10. from   model import  Discriminator
  11. import torchvision.transforms as transforms
  12. from torch.utils.data import Dataset, DataLoader, ConcatDataset, TensorDataset
  13. from torchvision.datasets import MNIST
  14. import time
  15. import matplotlib.pyplot as plt
  16. import numpy as np
  17. from torchvision.utils import make_grid
  18. import torch.nn as nn
  19.  
  20.  
  21.  
  22.  
  23. def get_noise(n_samples, z_dim, device='cpu'):
  24.     
  25.     return torch.randn(n_samples,z_dim,device=device)
  26.  
  27. def weights_init(m):
  28.     if isinstance(m, nn.Conv2d) or isinstance(m, nn.ConvTranspose2d):
  29.         torch.nn.init.normal_(m.weight, 0.0, 0.02)
  30.     if isinstance(m, nn.BatchNorm2d):
  31.         torch.nn.init.normal_(m.weight, 0.0, 0.02)
  32.         torch.nn.init.constant_(m.bias, 0)
  33.         
  34.  
  35. def gradient_penalty(gradient):
  36.     #Gradient Penalty
  37.     gradient = gradient.view(len(gradient), -1)
  38.     gradient_norm = gradient.norm(2, dim=1)
  39.     penalty = torch.mean((gradient_norm - 1)**2)
  40.     return penalty
  41.  
  42. def get_gen_loss(crit_fake_pred):
  43.     #生成器的loss
  44.     gen_loss = -1. * torch.mean(crit_fake_pred)
  45.     return gen_loss
  46.  
  47. def get_crit_loss(crit_fake_pred, crit_real_pred, gp, c_lambda):
  48.     #鉴别器的loss, 原公式加符号，转换为极小值求梯度
  49.     crit_loss = torch.mean(crit_fake_pred) - torch.mean(crit_real_pred) + c_lambda * gp
  50.     return crit_loss
  51.  
  52. def get_gradient(crit, real, fake, epsilon):
  53.     #随机采样
  54.     mixed_images = real * epsilon + fake * (1 - epsilon)
  55.     mixed_scores = crit(mixed_images)
  56.     gradient = torch.autograd.grad(
  57.         inputs=mixed_images,
  58.         outputs=mixed_scores,
  59.         grad_outputs=torch.ones_like(mixed_scores), 
  60.         create_graph=True,
  61.         retain_graph=True,
  62.         
  63.     )[0]
  64.     return gradient
  65.  
  66.  
  67. def show_new_gen_images(tensor_img, num_img=25):
  68.     
  69.     tensor_img = (tensor_img + 1) / 2
  70.     unflat_img = tensor_img.detach().cpu()
  71.     img_grid = make_grid(unflat_img[:num_img], nrow=5)
  72.     plt.imshow(img_grid.permute(1, 2, 0).squeeze(),cmap='gray')
  73.     plt.title("gen image")
  74.     plt.show()
  75.  
  76. def show_tensor_images(image_tensor, num_images=25, size=(1, 28, 28), show_fig=False, epoch=0):
  77.     #生成器输出的范围[-1,1]
  78.     #image_tensor = (image_tensor + 1) / 2
  79.     image_unflat = image_tensor.detach().cpu().view(-1, *size)
  80.     image_grid = make_grid(image_unflat[:num_images], nrow=5)
  81.     plt.axis('off')
  82.     label =f"Epoch: {epoch}"
  83.     plt.title(label)
  84.     plt.imshow(image_grid.permute(1, 2, 0).squeeze())
  85.     #if show_fig:
  86.         #plt.savefig('image_at_epoch_{:04d}.png'.format(epoch))
  87.         
  88.     plt.show()
  89.     
  90. def show_loss(G_mean_losses,C_mean_losses ):
  91.     plt.figure(figsize=(10,5))
  92.     plt.title("Generator and Discriminator Loss During Training")
  93.     plt.plot(G_mean_losses,label="G-Loss")
  94.     plt.plot(C_mean_losses,label="C-Loss")
  95.     plt.xlabel("iterations")
  96.     plt.ylabel("Loss")
  97.     plt.legend()
  98.     plt.show()
  99.  
  100. def train():
  101.     
  102.     z_dim = 32
  103.     batch_size = 128
  104.     device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')  
  105.     lr = 1e-4
  106.     beta_1 = 0.0 
  107.     beta_2 = 0.9
  108.  
  109.     
  110.     #MNIST Dataset Load
  111.     print("\n init 1: MNIST Dataset Load ")
  112.  
  113.     fixed_noise = get_noise(batch_size, z_dim, device=device)
  114.     train_transform = transforms.Compose([transforms.ToTensor(),])
  115.     dataloader = DataLoader( MNIST('.', download=True, transform=train_transform),
  116.                             batch_size=batch_size,
  117.                             shuffle=True)
  118.     
  119.     
  120.     print("\n init2:  Loaded Data Visualization")
  121.     start = time.time()
  122.     dataiter = iter(dataloader)
  123.     images,labels = next(dataiter)
  124.     print ('Time is {} sec'.format(time.time()-start))
  125.  
  126.     plt.figure(figsize=(8,8))
  127.     plt.axis("off")
  128.     plt.title("Training Images")
  129.  
  130.     plt.imshow(np.transpose(make_grid(images.to(device), padding=2, normalize=True).cpu(),(1,2,0)))
  131.  
  132.     print('Shape of loading one batch:', images.shape)
  133.     print('Total no. of batches present in trainloader:', len(dataloader))
  134.     
  135.     #Optimizer
  136.     gen = Generator(z_dim).to(device)
  137.     gen_opt = torch.optim.Adam(gen.parameters(), lr=lr, betas=(beta_1, beta_2))
  138.     crit  = Discriminator().to(device) 
  139.     crit_opt = torch.optim.Adam(crit.parameters(), lr=lr, betas=(beta_1, beta_2))
  140.     gen = gen.apply(weights_init)
  141.     crit = crit.apply(weights_init)  
  142.     
  143.  
  144.     print("\n -------- train ------------")
  145.     n_epochs = 10
  146.     cur_step = 0
  147.     total_steps = 0
  148.     start_time = time.time()
  149.     cur_step = 0
  150.     
  151.     generator_losses = []
  152.     Discriminator_losses = []
  153.     
  154.     C_mean_losses = []
  155.     G_mean_losses = []
  156.     
  157.     c_lambda = 10
  158.     crit_repeats = 5
  159.     
  160.     for epoch in range(n_epochs):
  161.         cur_step = 0
  162.         start = time.time()
  163.         for real, _ in dataloader:
  164.             cur_batch_size = len(real)
  165.             real = real.to(device)
  166.     
  167.             mean_iteration_Discriminator_loss = 0
  168.             for _ in range(crit_repeats):
  169.                 ### Update Discriminator ###
  170.                 crit_opt.zero_grad()
  171.                 fake_noise = get_noise(cur_batch_size, z_dim, device=device)
  172.                 fake = gen(fake_noise)
  173.                 crit_fake_pred = crit(fake.detach())
  174.                 crit_real_pred = crit(real)
  175.     
  176.                 epsilon = torch.rand(len(real), 1, 1, 1, device=device, requires_grad=True)
  177.                 gradient = get_gradient(crit, real, fake.detach(), epsilon)
  178.                 gp = gradient_penalty(gradient)
  179.                 crit_loss = get_crit_loss(crit_fake_pred, crit_real_pred, gp, c_lambda)
  180.     
  181.                 # Keep track of the average Discriminator loss in this batch
  182.                 mean_iteration_Discriminator_loss += crit_loss.item() / crit_repeats
  183.                 # Update gradients
  184.                 crit_loss.backward(retain_graph=True)
  185.                 # Update optimizer
  186.                 crit_opt.step()
  187.             Discriminator_losses += [mean_iteration_Discriminator_loss]
  188.     
  189.             ### Update generator ###
  190.             gen_opt.zero_grad()
  191.             fake_noise_2 = get_noise(cur_batch_size, z_dim, device=device)
  192.             fake_2 = gen(fake_noise_2)
  193.             crit_fake_pred = crit(fake_2)
  194.             
  195.             gen_loss = get_gen_loss(crit_fake_pred)
  196.             gen_loss.backward()
  197.     
  198.             # Update the weights
  199.             gen_opt.step()
  200.     
  201.             # Keep track of the average generator loss
  202.             generator_losses += [gen_loss.item()]
  203.             
  204.             cur_step += 1
  205.             total_steps += 1
  206.             
  207.             print_val = f"Epoch: {epoch}/{n_epochs} Steps:{cur_step}/{len(dataloader)}\t"
  208.             print_val += f"Epoch_Run_Time: {(time.time()-start):.6f}\t"
  209.             print_val += f"Loss_C : {mean_iteration_Discriminator_loss:.6f}\t"
  210.             print_val += f"Loss_G : {gen_loss:.6f}\t"  
  211.             print(print_val, end='\r',flush = True)
  212.     
  213.  
  214.     
  215.        
  216.         gen_mean = sum(generator_losses[-cur_step:]) / cur_step
  217.         crit_mean = sum(Discriminator_losses[-cur_step:]) / cur_step
  218.         
  219.         C_mean_losses.append(crit_mean)
  220.         G_mean_losses.append(gen_mean)
  221.         
  222.         print_val = f"Epoch: {epoch}/{n_epochs} Total Steps:{total_steps}\t"
  223.         print_val += f"Total_Time : {(time.time() - start_time):.6f}\t"
  224.         print_val += f"Loss_C : {mean_iteration_Discriminator_loss:.6f}\t"
  225.         print_val += f"Loss_G : {gen_loss:.6f}\t"
  226.         print_val += f"Loss_C_Mean : {crit_mean:.6f}\t"
  227.         print_val += f"Loss_G_Mean : {gen_mean:.6f}\t"
  228.         print(print_val)
  229.         
  230.         fake_noise = fixed_noise
  231.         fake = gen(fake_noise)
  232.         
  233.         show_tensor_images(fake, show_fig=True,epoch=epoch)
  234.         
  235.         cur_step = 0
  236.         
  237.     print("\n-----训练结束--------------")
  238.     num_image = 25
  239.     noise = get_noise(num_image, z_dim, device=device)
  240.     #Batch Normalization，Dropout不使用
  241.     gen.eval()
  242.     crit.eval()
  243.     
  244.     with torch.no_grad():
  245.         fake_img = gen(noise)
  246.  
  247.     show_new_gen_images(fake_img.reshape(num_image,1,28,28))
  248.     
  249. train()

PyTorch-Wasserstein GAN(WGAN) | Kaggle

WGAN模型——pytorch实现_python实现wgn函数-CSDN博客
WGAN_哔哩哔哩_bilibili

15 李宏毅【機器學習2021】生成式對抗網路 (Generative Adversarial Network, GAN) (中) – 理論介紹與WGAN_哔哩哔哩_bilibili

课时12 WGAN-GP实战_哔哩哔哩_bilibili

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/Monodyee/article/detail/295703
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号