devbox
weixin_40725706
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

项目pytorch-deeplab-xception为例，测试时怎么保存target、image:target.cpu().numpy()_output3.data.cpu()

作者：weixin_40725706 | 2024-04-04 10:37:24

赞

踩

output3.data.cpu()

一般性流程 

  1.         '''
  2.         IPL转换为tensor
  3.         _img = Image.open(os.path.join(self.img_dir, path)).convert('RGB')
  4.         img = np.array(img).astype(np.float32).transpose((2, 0, 1))
  5.         img = torch.from_numpy(img).float()
  6.         img = img.cuda()
  7.         
  8.         tensor转换为IPL
  9.         image1 = image.data.cpu().numpy()
  10.         IPLimage = numpyimg.transpose((1, 2, 0))
  11.         save_img = Image.fromarray(IPLimage.astype('uint8'))
  12.         '''

例子：

  1.         for i, sample in enumerate(self.test_loader):
  2.             image, target = sample['image'], sample['label']
  3.             torch.cuda.synchronize()
  4.             start = time.time()
  5.             with torch.no_grad():
  6.                 output = self.model(image)
  7.             end = time.time()
  8.             times = (end - start) * 1000
  9.             print(times, "ms")
  10.             torch.cuda.synchronize()
  11.             pred = output.data.cpu().numpy()
  12.             target = target.cpu().numpy()
  13.             pred = np.argmax(pred, axis=1)
  14.             self.evaluator.add_batch(target, pred)

我想看一下target是否对，通过opencv保存，首先看下opencv的格式：

cv2.resize(src, dsize[, dst[, fx[, fy[, interpolation]]]]) -> dst 

fx - 水平轴上的比例因子。fy - 垂直轴上的比例因子。

numpy实现图像部分ROI截取：

  1. for index in inds:
  2.     xmin_depth = int((xmin1[index] * expected + crop_start) * scale)
  3.     ymin_depth = int((ymin1[index] * expected) * scale)
  4.     xmax_depth = int((xmax1[index] * expected + crop_start) * scale)
  5.     ymax_depth = int((ymax1[index] * expected) * scale)
  6.     depth_temp = depth[ymin_depth:ymax_depth, xmin_depth:xmax_depth].astype(float)

 首先numpy是[高度h:宽度w]

如果是x1，y1，x2，y2(左上，右下)的任务，应该是img=ori_img[y1:y2, x1:x2]

  1. import cv2
  2. cvimg = cv2.imread("./dog.jpg")
  3. graycvimg = cv2.cvtColor(cvimg, cv2.COLOR_BGR2GRAY)
  4. cv2.imwrite("./dog_gray.jpg", graycvimg)
  5. graycvimg_bgr = cv2.cvtColor(graycvimg, cv2.COLOR_GRAY2BGR)
  6. cv2.imwrite("./dog_gray_bgr.jpg", graycvimg_bgr)

  1. from PIL import Image
  2. import numpy as np
  3. img = Image.open(imgsname).convert('RGB')
  4. imglabel = Image.open(imgsname).convert('P')
  5. arrayimg = np.array(img).astype(np.float32)
  6. transposeimg = arrayimg.transpose((2, 0, 1))

关于PIL和opencv还有一个区别：size的先后，PIL是W，H opencv是H，W，C

  1. imgsname = newpath + namename + '_ccvt_' + str(j) + '.jpg'
  2. img = Image.open(imgsname).convert('RGB')
  3. W, H = img.size
  4.  
  5. img = np.array(img)
  6. dst, scale_factor = mmcv.imrescale(img, (1333, 800), return_scale=True)
  7. newH, newW, newC = dst.shape
  1.         # tensor 转换为 numpy
  2.         numpyimg = imgarray.numpy()
  3.         # numpy 转换为 IPL格式
  4.         IPLimage = numpyimg.transpose((1, 2, 0))
  5.         '''
  6.         IPL转换为tensor
  7.         _img = Image.open(os.path.join(self.img_dir, path)).convert('RGB')
  8.         img = np.array(img).astype(np.float32).transpose((2, 0, 1))
  9.         img = torch.from_numpy(img).float()
  10.         img = img.cuda()
  12.         tensor转换为IPL
  13.         image1 = image.data.cpu().numpy()
  14.         IPLimage = numpyimg.transpose((1, 2, 0))
  15.         save_img = Image.fromarray(IPLimage.astype('uint8'))
  16.         '''

参考:

https://blog.csdn.net/m0_37382341/article/details/83548601

numpy.reshape
Numpy将不管是什么形状的数组，先扁平化处理成一个一维的列表，然后按照你重新定义的形状，再把这个列表截断拼成新的形状。 在这个过程中，如果你要处理的是图片矩阵的话，就会完全改变图片信息。
numpy.transpose
numpy.transpose采取轴作为输入，所以你可以改变轴，这对于张量来说很有用，也很方便。比如data.transpose(1,0,2),就表示把1位置的数换到0位置，0位置的换到1位置，2没有变。

由于测试时候使用：

  1.     def transform_val(self, sample):
  2.         composed_transforms = transforms.Compose([
  3.             tr.FixScaleCrop(crop_size=self.args.crop_size),
  4.             tr.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),
  5.             tr.ToTensor()
  6.             ])
  7.         return composed_transforms(sample)

应该把注释改掉：

  1.     def transform_val(self, sample):
  2.         composed_transforms = transforms.Compose([
  3.             tr.FixScaleCrop(crop_size=self.args.crop_size),
  4.             #tr.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),
  5.             tr.ToTensor()
  6.             ])
  7.         return composed_transforms(sample)

这样方便我们保存Image对比 

  1. import cv2
  2.  
  3.  
  4. target = target.cpu().numpy()
  5. image = image.data.cpu().numpy()
  6. image1 = image[0, :]
  7. target1 = target[0, :]
  8. #image1.reshape([image1.size[1],image1.size[2],image1.size[3]])
  9. #target1.reshape([image1.size[1],image1.size[2],image1.size[3]])
  10. image1 = image1.transpose(2,1,0)
  11. #target1 = target1.transpose(2,1,0)
  12. image1 = cv2.cvtColor(image1, cv2.COLOR_RGB2BGR)
  13. cv2.imwrite("./image1.jpg",image1)
  14. cv2.imwrite("./target1.jpg", target1)

我这里出现一些问题，target方向错误了，debug一下，看看载入时候有没有问题:

  1.     def _make_img_gt_point_pair(self, index):
  2.         coco = self.coco
  3.         img_id = self.ids[index]
  4.         img_metadata = coco.loadImgs(img_id)[0]
  5.         path = img_metadata['file_name']
  6.         _img = Image.open(os.path.join(self.img_dir, path)).convert('RGB')
  7.         cocotarget = coco.loadAnns(coco.getAnnIds(imgIds=img_id))
  8.         _target = Image.fromarray(self._gen_seg_mask(
  9.             cocotarget, img_metadata['height'], img_metadata['width']))
  10.  
  11.         image1 = cv2.cvtColor(np.asarray(_img), cv2.COLOR_RGB2BGR)
  12.         target1 = cv2.cvtColor(np.asarray(_target), cv2.COLOR_GRAY2BGR)
  13.         cv2.imwrite("./image1.jpg", image1)
  14.         cv2.imwrite("./target1.jpg", target1)
  15.  
  16.         return _img, _target
  1.     def __getitem__(self, index):
  2.         _img, _target = self._make_img_gt_point_pair(index)
  3.         sample = {'image': _img, 'label': _target}
  4.  
  5.         if self.split == "train":
  6.             return self.transform_tr(sample)
  7.         elif self.split == 'val':
  8.             return self.transform_val(sample)
  9.         elif self.split == 'test':
  10.             X = self.transform_val(sample)
  11.             aa = X['image']
  12.             bb = X['label']
  13.  
  14.             aa = aa.cpu().numpy()
  15.             bb = bb.cpu().numpy()
  16.             aa = aa.transpose(2, 1, 0)
  17.             image1 = cv2.cvtColor(aa, cv2.COLOR_RGB2BGR)
  18.             target1 = cv2.cvtColor(bb, cv2.COLOR_GRAY2BGR)
  19.             cv2.imwrite("./image2.jpg", image1)
  20.             cv2.imwrite("./target2.jpg", target1)
  21.  
  22.             return X

 原图resize后方向变了，果然。。。。。。。

原图：

因为项目中使用了一个torch函数进行预处理：

pytorch的transforms.py

 

  1. class Compose(object):
  2.     """Composes several transforms together.
  4.     Args:
  5.         transforms (list of ``Transform`` objects): list of transforms to compose.
  7.     Example:
  8.         >>> transforms.Compose([
  9.         >>>     transforms.CenterCrop(10),
  10.         >>>     transforms.ToTensor(),
  11.         >>> ])
  12.     """
  13.  
  14.     def __init__(self, transforms):
  15.         self.transforms = transforms
  16.  
  17.     def __call__(self, img):
  18.         for t in self.transforms:
  19.             img = t(img)
  20.         return img

首先

  1. class FixScaleCrop(object):
  2.     def __init__(self, crop_size):
  3.         self.crop_size = crop_size
  4.  
  5.     def __call__(self, sample):
  6.         img = sample['image']
  7.         mask = sample['label']
  8.         w, h = img.size
  9.         if w > h:
  10.             oh = self.crop_size
  11.             ow = int(1.0 * w * oh / h)
  12.         else:
  13.             ow = self.crop_size
  14.             oh = int(1.0 * h * ow / w)
  15.         img = img.resize((ow, oh), Image.BILINEAR)
  16.         mask = mask.resize((ow, oh), Image.NEAREST)
  17.         # center crop
  18.         w, h = img.size
  19.         x1 = int(round((w - self.crop_size) / 2.))
  20.         y1 = int(round((h - self.crop_size) / 2.))
  21.         img = img.crop((x1, y1, x1 + self.crop_size, y1 + self.crop_size))
  22.         mask = mask.crop((x1, y1, x1 + self.crop_size, y1 + self.crop_size))
  23.  
  24.         return {'image': img,
  25.                 'label': mask}
  1. class FixScaleCrop(object):
  2.     def __init__(self, crop_size):
  3.         self.crop_size = crop_size
  4.  
  5.     def __call__(self, sample):
  6.         img = sample['image']
  7.         mask = sample['label']
  8.         w, h = img.size
  9.         if w > h:
  10.             oh = self.crop_size
  11.             ow = int(1.0 * w * oh / h)
  12.         else:
  13.             ow = self.crop_size
  14.             oh = int(1.0 * h * ow / w)
  15.         img = img.resize((ow, oh), Image.BILINEAR)
  16.         mask = mask.resize((ow, oh), Image.NEAREST)
  17.         # center crop
  18.         w, h = img.size
  19.         x1 = int(round((w - self.crop_size) / 2.))
  20.         y1 = int(round((h - self.crop_size) / 2.))
  21.         img = img.crop((x1, y1, x1 + self.crop_size, y1 + self.crop_size))
  22.         mask = mask.crop((x1, y1, x1 + self.crop_size, y1 + self.crop_size))
  23.  
  24.         import cv2
  25.         image1 = cv2.cvtColor(np.asarray(img), cv2.COLOR_RGB2BGR)
  26.         target1 = cv2.cvtColor(np.asarray(mask), cv2.COLOR_GRAY2BGR)
  27.         cv2.imwrite("./image3.jpg", image1)
  28.         cv2.imwrite("./target3.jpg", target1)
  29.  
  30.  
  31.         return {'image': img,
  32.                 'label': mask}

 

程序在这里还是没问题的，结果接下来会进入：

  1. class ToTensor(object):
  2.     """Convert ndarrays in sample to Tensors."""
  3.  
  4.     def __call__(self, sample):
  5.         # swap color axis because
  6.         # numpy image: H x W x C
  7.         # torch image: C X H X W
  8.         img = sample['image']
  9.         mask = sample['label']
  10.         img = np.array(img).astype(np.float32).transpose((2, 0, 1))
  11.         mask = np.array(mask).astype(np.float32)
  12.  
  13.         img = torch.from_numpy(img).float()
  14.         mask = torch.from_numpy(mask).float()
  15.  
  16.         return {'image': img,
  17.                 'label': mask}
  1. class ToTensor(object):
  2.     """Convert ndarrays in sample to Tensors."""
  3.  
  4.     def __call__(self, sample):
  5.         # swap color axis because
  6.         # numpy image: H x W x C
  7.         # torch image: C X H X W
  8.         img = sample['image']
  9.         mask = sample['label']
  10.         img = np.array(img).astype(np.float32).transpose((2, 0, 1))
  11.         mask = np.array(mask).astype(np.float32)
  12.  
  13.         img = torch.from_numpy(img).float()
  14.         mask = torch.from_numpy(mask).float()
  15.  
  16.  
  17.  
  18.         import cv2
  19.         image1=img.cpu().numpy()
  20.         target1=mask.cpu().numpy()
  21.         image1 = image1.transpose(2, 1, 0)
  22.         image1 = cv2.cvtColor(image1, cv2.COLOR_RGB2BGR)
  23.         target1 = cv2.cvtColor(target1, cv2.COLOR_GRAY2BGR)
  24.         cv2.imwrite("./image4.jpg", image1)
  25.         cv2.imwrite("./target4.jpg", target1)
  26.  
  27.         return {'image': img,
  28.                 'label': mask}

 这里出错了，方向不对了

如果将代码改为；

img = np.array(img).astype(np.float32).transpose((2, 1, 0))

 方向就都对了，那么作者原本为什么那样写？？？？？？

        img = np.array(img).astype(np.float32).transpose((2, 0, 1))

到底有什么用，

  1. class ToTensor(object):
  2.     """Convert ndarrays in sample to Tensors."""
  3.  
  4.     def __call__(self, sample):
  5.         # swap color axis because
  6.         # numpy image: H x W x C
  7.         # torch image: C X H X W
  8.         img = sample['image']
  9.         mask = sample['label']
  10.  
  11.         import cv2
  12.         image1 = cv2.cvtColor(np.asarray(img), cv2.COLOR_RGB2BGR)
  13.         target1 = cv2.cvtColor(np.asarray(mask), cv2.COLOR_GRAY2BGR)
  14.         cv2.imwrite("./image5.jpg", image1)
  15.         cv2.imwrite("./target5.jpg", target1)
  16.  
  17.         xxx = np.array(img).astype(np.float32)
  18.         import copy
  19.         xxx1 = copy.deepcopy(xxx)
  20.         xxx2 = copy.deepcopy(xxx)
  21.         img1 = np.array(xxx1).astype(np.float32).transpose((2, 1, 0))
  22.         img2 = np.array(xxx2).astype(np.float32).transpose((2, 0, 1))
  23.  
  24.         img = np.array(img).astype(np.float32).transpose((2, 1, 0))
  25.         mask = np.array(mask).astype(np.float32)
  26.  
  27.         img = torch.from_numpy(img).float()
  28.         mask = torch.from_numpy(mask).float()

513*513*3---3* 513*513   

.transpose((2, 1, 0)) 

513*513*3---3* 513*513

.transpose((2, 0, 1))

 其实实验做到这里我已经明白是我错了，

原本是

513*513*3

我们通过.transpose((2, 0, 1))，正常变换，我错在test显示的时候：

  1. import cv2
  2.  
  3.  
  4. target = target.cpu().numpy()
  5. image = image.data.cpu().numpy()
  6. image1 = image[0, :]
  7. target1 = target[0, :]
  8. #image1.reshape([image1.size[1],image1.size[2],image1.size[3]])
  9. #target1.reshape([image1.size[1],image1.size[2],image1.size[3]])
  10. image1 = image1.transpose(1,2,0)
  11. image1 = cv2.cvtColor(image1, cv2.COLOR_RGB2BGR)
  12. cv2.imwrite("./image1.jpg",image1)
  13. cv2.imwrite("./target1.jpg", target1)

这里应该是

image1 = image1.transpose(1,2,0)

因为原本

  1. for i, sample in enumerate(self.test_loader):
  2.     image, target = sample['image'], sample['label']

image为：torch.Size([1, 3, 513, 513])

target为：<class 'tuple'>: (1, 513, 513)

所以应该使用image1 = image1.transpose(1,2,0)

这下就对了

现在还有一个问题摆在面前，

我做测试时候，COCO数据集格式，自己的数据集，

图片有153张，但是最后输出只有25张pred，

找原因：

pytorch-deeplab-xception/dataloaders/datasets/coco.py

在处理coco数据之前，会生成一个test_ids_2017.pth

id对应文件，新ID与旧ID相对应，

用于知道哪些ID被保留下来，用于接下来的测试

  1.         if os.path.exists(ids_file):
  2.             self.ids = torch.load(ids_file)
  3.         else:
  4.             ids = list(self.coco.imgs.keys())
  5.             self.ids = self._preprocess(ids, ids_file, self.split)
  6.         self.args = args

判断条件在函数self._preprocess(ids, ids_file, self.split)

  1.     def _preprocess(self, ids, ids_file, split):
  2.         print("Preprocessing mask, this will take a while. " + \
  3.               "But don't worry, it only run once for each split.")
  4.         tbar = trange(len(ids))
  5.         new_ids = []
  6.         for i in tbar:
  7.             img_id = ids[i]
  8.             cocotarget = self.coco.loadAnns(self.coco.getAnnIds(imgIds=img_id))
  9.             img_metadata = self.coco.loadImgs(img_id)[0]
  10.             savemaskname=img_metadata['file_name']
  11.             image = ids_file.split("annotations")[0]+'images/'+split+str(self.year) + '/' +savemaskname
  12.             oriimg = cv2.imread(image)
  13.             h,w,c = oriimg.shape
  14.  
  15.             mask = self._gen_seg_mask(cocotarget, h,
  16.                                       w)
  17.             cv2.imwrite('/home/spple/paddle/DeepGlint/deepglint-adv/pytorch-deeplab-xception/mask/'+split+'/'+savemaskname, mask)
  18.             # more than 1k pixels
  19.             if (mask > 0).sum() > 1000:
  20.                 new_ids.append(img_id)
  21.             tbar.set_description('Doing: {}/{}, got {} qualified images'. \
  22.                                  format(i, len(ids), len(new_ids)))
  23.         print('Found number of qualified images: ', len(new_ids))
  24.         torch.save(new_ids, ids_file)
  25.         return new_ids

通过函数def _gen_seg_mask(self, target, h, w): 获取mask

  1.     def _gen_seg_mask(self, target, h, w):
  2.         mask = np.zeros((h, w), dtype=np.uint8)
  3.         coco_mask = self.coco_mask
  4.         for instance in target:
  5.             rle = coco_mask.frPyObjects(instance['segmentation'], h, w)
  6.             m = coco_mask.decode(rle)
  7.             cat = instance['category_id']
  8.             if cat in self.CAT_LIST:
  9.                 c = self.CAT_LIST.index(cat)
  10.             else:
  11.                 continue
  12.             if len(m.shape) < 3:
  13.                 mask[:, :] += (mask == 0) * (m * c)
  14.             else:
  15.                 mask[:, :] += (mask == 0) * (((np.sum(m, axis=2)) > 0) * c).astype(np.uint8)
  16.         return mask

但是这里有个问题，判断依据是mask分割像素点必须是1000以上，但是对于小图像，可能达不到，这里，我们要修改

  1.             if (mask > 0).sum() > 1000:
  2.                 new_ids.append(img_id)

修改为：

  1.             if (mask > 0).sum() > 50:
  2.                 new_ids.append(img_id)

还有之前的函数只是简单的保存是参考：

https://github.com/jfzhang95/pytorch-deeplab-xception/issues/122

  1. import argparse
  2. import os
  3. import numpy as np 
  4. import tqdm
  5. import torch
  6.  
  7.  
  8. from PIL import Image
  9. from dataloaders import make_data_loader
  10. from modeling.deeplab import *
  11. from dataloaders.utils import get_pascal_labels
  12. from utils.metrics import Evaluator
  13.  
  14. class Tester(object):
  15.     def __init__(self, args):
  16.         if not os.path.isfile(args.model):
  17.             raise RuntimeError("no checkpoint found at '{}'".fromat(args.model))
  18.         self.args = args
  19.         self.color_map = get_pascal_labels()
  20.         self.test_loader, self.ids, self.nclass = make_data_loader(args)
  21.  
  22.         #Define model
  23.         model = DeepLab(num_classes=self.nclass,
  24.                         backbone=args.backbone,
  25.                         output_stride=args.out_stride,
  26.                         sync_bn=False,
  27.                         freeze_bn=False)
  28.         
  29.         self.model = model
  30.         device = torch.device('cpu')
  31.         checkpoint = torch.load(args.model, map_location=device)
  32.         self.model.load_state_dict(checkpoint['state_dict'])
  33.         self.evaluator = Evaluator(self.nclass)
  34.  
  35.     def save_image(self, array, id, op):
  36.         text = 'gt'
  37.         if op == 0:
  38.             text = 'pred'
  39.         file_name = str(id)+'_'+text+'.png'
  40.         r = array.copy()
  41.         g = array.copy()
  42.         b = array.copy()
  43.  
  44.         for i in range(self.nclass):
  45.             r[array == i] = self.color_map[i][0]
  46.             g[array == i] = self.color_map[i][1]
  47.             b[array == i] = self.color_map[i][2]
  48.     
  49.         rgb = np.dstack((r, g, b))
  50.  
  51.         save_img = Image.fromarray(rgb.astype('uint8'))
  52.         save_img.save(self.args.save_path+os.sep+file_name)
  53.  
  54.  
  55.     def test(self):
  56.         self.model.eval()
  57.         self.evaluator.reset()
  58.         # tbar = tqdm(self.test_loader, desc='\r')
  59.         for i, sample in enumerate(self.test_loader):
  60.             image, target = sample['image'], sample['label']
  61.             with torch.no_grad():
  62.                 output = self.model(image)
  63.             pred = output.data.cpu().numpy()
  64.             target = target.cpu().numpy()
  65.             pred = np.argmax(pred, axis=1)
  66.             self.save_image(pred[0], self.ids[i], 0)
  67.             self.save_image(target[0], self.ids[i], 1)
  68.             self.evaluator.add_batch(target, pred)
  69.     
  70.         Acc = self.evaluator.Pixel_Accuracy()
  71.         Acc_class = self.evaluator.Pixel_Accuracy_Class()
  72.         print('Acc:{}, Acc_class:{}'.format(Acc, Acc_class))
  73.  
  74. def main():
  75.     parser = argparse.ArgumentParser(description='Pytorch DeeplabV3Plus Test your data')
  76.     parser.add_argument('--test', action='store_true', default=True, 
  77.                         help='test your data')
  78.     parser.add_argument('--dataset', default='pascal', 
  79.                         help='datset format')
  80.     parser.add_argument('--backbone', default='xception', 
  81.                         help='what is your network backbone')
  82.     parser.add_argument('--out_stride', type=int, default=16,
  83.                         help='output stride')
  84.     parser.add_argument('--crop_size', type=int, default=513,
  85.                         help='image size')
  86.     parser.add_argument('--model', type=str, default='',
  87.                         help='load your model')
  88.     parser.add_argument('--save_path', type=str, default='',
  89.                         help='save your prediction data')
  90.  
  91.     args = parser.parse_args()
  92.     
  93.     if args.test:
  94.         tester = Tester(args)
  95.         tester.test()
  96.  
  97. if __name__ == "__main__":
  98.     main()

这里保存完后是：

  1.     def save_image(self, array, id, op, oriimg=None, image111=None):
  2.         import cv2
  3.         text = 'gt'
  4.         if op == 0:
  5.             text = 'pred'
  6.         file_name = str(id)+'_'+text+'.png'
  7.  
  8.         drow_ori_name = str(id)+'_'+'vis'+'.png'
  9.  
  10.         #513*513
  11.         r = array.copy()
  12.         g = array.copy()
  13.         b = array.copy()
  14.  
  15.         if oriimg is True:
  16.             image111 = image111.data.cpu().numpy()
  17.             image111 = image111[0, :]
  18.             image111 = image111.transpose(1,2,0)
  19.             oneimg = image111
  20.  
  21.         for i in range(self.nclass):
  22.             r[array == i] = self.color_map[i][2]
  23.             g[array == i] = self.color_map[i][1]
  24.             b[array == i] = self.color_map[i][0]
  25.  
  26.         rgb = np.dstack((r, g, b))
  27.         hh,ww,_ = rgb.shape
  28.  
  29.         if oriimg is True:
  30.             for i in range(self.nclass):
  31.                 if i != 0:
  32.                     index = np.argwhere(array == i)
  33.                     for key in index:
  34.                         oneimg[key[0]][key[1]][0] = self.color_map[i][0]
  35.                         oneimg[key[0]][key[1]][1] = self.color_map[i][1]
  36.                         oneimg[key[0]][key[1]][2] = self.color_map[i][2]
  37.             oneimg = cv2.cvtColor(oneimg, cv2.COLOR_RGB2BGR)
  38.             cv2.imwrite(self.args.save_path + os.sep + drow_ori_name, oneimg)

这样完全覆盖了，我们并不能看到真实样貌，应该参考mask_rcnn,透明效果：

其实就是将原始图像和预测类的颜色，不同比例结合，生成可视化图像：

  1. oneimg[key[0]][key[1]][0] = oneimg[key[0]][key[1]][0] * 0.5 + self.color_map[i][0] * 0.5
  2. oneimg[key[0]][key[1]][1] = oneimg[key[0]][key[1]][1] * 0.5 + self.color_map[i][1] * 0.5
  3. oneimg[key[0]][key[1]][2] = oneimg[key[0]][key[1]][2] * 0.5 + self.color_map[i][2] * 0.5

这里还有一个问题

我们进行测试时候显示：

  1. Acc:0.9829744103317358, Acc_class:0.7640047637800897, mIoU:0.7015250613321066
  2. /home/spple/pytorch-deeplab-xception/utils/metrics.py:14: RuntimeWarning: invalid value encountered in true_divide
  3.   Acc = np.diag(self.confusion_matrix) / self.confusion_matrix.sum(axis=1)
  4. /home/spple/pytorch-deeplab-xception/utils/metrics.py:24: RuntimeWarning: invalid value encountered in true_divide
  5.   np.diag(self.confusion_matrix))

原来是因为数组分母有为0的

比如：

  1.     def Pixel_Accuracy_Class(self):
  2.         a = np.diag(self.confusion_matrix)
  3.         b = self.confusion_matrix.sum(axis=1)
  4.         #Acc = np.diag(self.confusion_matrix) / self.confusion_matrix.sum(axis=1)
  5.         Acc = a/b
  6.         Acc = np.nanmean(Acc)
  7.         return Acc

a： 

b：

Acc：

Acc = np.nanmean(Acc)：

0.7640047637800897=(0.993579+0.534430)/2

 

顺便做了一个实验：

  1. import numpy as np
  2.  
  3. a = np.array([[12],[6]])
  4. b = np.array([3,3])
  5. Acc_1= a/b
  6.  
  7. c = np.array([[12,1],[1,6]])
  8. x2 = np.diag(c)
  9. Acc_2= x2/b
  10.  
  11. x1 = np.zeros((2,)*1)
  12. x1[0]=3
  13. x1[1]=3

向量相除，如果最后只想得到向量，分子分母shape应该 是<class 'tuple'>: (2,)

a

b

Acc_1

c

x2

Acc_2

x1

test.py

  1. import argparse
  2. import os
  3. import numpy as np 
  4. import tqdm
  5. import torch
  6. import time
  7.  
  8. #https://github.com/jfzhang95/pytorch-deeplab-xception/issues/122
  9.  
  10. from PIL import Image
  11. from dataloaders import make_data_loader
  12. from modeling.deeplab import *
  13. from dataloaders.utils import get_pascal_labels
  14. from utils.metrics import Evaluator
  15. import cv2
  16.  
  17. class Tester(object):
  18.     def __init__(self, args):
  19.         if not os.path.isfile(args.model):
  20.             raise RuntimeError("no checkpoint found at '{}'".fromat(args.model))
  21.         self.args = args
  22.         self.color_map = get_pascal_labels()
  23.         self.test_loader, self.nclass= make_data_loader(args)
  24.  
  25.         #Define model
  26.         model = DeepLab(num_classes=self.nclass,
  27.                         backbone=args.backbone,
  28.                         output_stride=args.out_stride,
  29.                         sync_bn=False,
  30.                         freeze_bn=False)
  31.         
  32.         self.model = model
  33.         device = torch.device('cpu')
  34.         checkpoint = torch.load(args.model, map_location=device)
  35.         self.model.load_state_dict(checkpoint['state_dict'])
  36.         self.evaluator = Evaluator(self.nclass)
  37.  
  38.     #--dataset pascal --backbone resnet --out_stride 16 --crop_size 513 --model /home/spple/paddle/DeepGlint/deepglint-adv/pytorch-deeplab-xception/checkpoint-gray/model_best.pth.tar --save_path /home/spple/paddle/DeepGlint/deepglint-adv/pytorch-deeplab-xception/prediction_gray
  39.     # --dataset pascal --backbone resnet --out_stride 16 --crop_size 513 --model /home/spple/paddle/DeepGlint/deepglint-adv/pytorch-deeplab-xception/checkpoint/checkpoint.pth.tar --save_path /home/spple/paddle/DeepGlint/deepglint-adv/pytorch-deeplab-xception/prediction
  40.     def save_image(self, array, id, op, oriimg=None, image111=None):
  41.         import cv2
  42.         text = 'gt'
  43.         if op == 0:
  44.             text = 'pred'
  45.         file_name = str(id)+'_'+text+'.png'
  46.  
  47.         drow_ori_name = str(id)+'_'+'vis'+'.png'
  48.  
  49.         #513*513
  50.         r = array.copy()
  51.         g = array.copy()
  52.         b = array.copy()
  53.  
  54.         if oriimg is True:
  55.             oneimgpath = str(id) + '.jpg'
  56.             from mypath import Path
  57.             #JPEGImages_gray
  58.             image111 = image111.data.cpu().numpy()
  59.             image111 = image111[0, :]
  60.             image111 = image111.transpose(1,2,0)
  61.             oneimg = image111
  62.  
  63.         for i in range(self.nclass):
  64.             r[array == i] = self.color_map[i][2]
  65.             g[array == i] = self.color_map[i][1]
  66.             b[array == i] = self.color_map[i][0]
  67.  
  68.         #513*513*3
  69.         rgb = np.dstack((r, g, b))
  70.         hh,ww,_ = rgb.shape
  71.  
  72.         #if oriimg is True:
  73.             #oneimg = oneimg.resize((hh, ww), Image.ANTIALIAS)
  74.             # 原图
  75.             #image1 = cv2.cvtColor(oneimg, cv2.COLOR_RGB2BGR)
  76.             #oneimg.save(self.args.save_path + os.sep + ori_name, quality=100)
  77.             #cv2.imwrite(self.args.save_path + os.sep + ori_name, image1)
  78.  
  79.  
  80.         #----gt ---- pred
  81.         cv2.imwrite(self.args.save_path+os.sep+file_name, rgb)
  82.         #save_img = Image.fromarray(rgb.astype('uint8'))
  83.         # pred
  84.         #save_img.save(self.args.save_path+os.sep+file_name, quality=100)
  85.  
  86.         #oneimg = oneimg.transpose(2, 0, 1)
  87.         if oriimg is True:
  88.             #oneimg = np.array(oneimg)
  89.             for i in range(self.nclass):
  90.                 if i != 0:
  91.                     index = np.argwhere(array == i)
  92.                     for key in index:
  93.                         oneimg[key[0]][key[1]][0] = oneimg[key[0]][key[1]][0] * 0.5 + self.color_map[i][0] * 0.5
  94.                         oneimg[key[0]][key[1]][1] = oneimg[key[0]][key[1]][1] * 0.5 + self.color_map[i][1] * 0.5
  95.                         oneimg[key[0]][key[1]][2] = oneimg[key[0]][key[1]][2] * 0.5 + self.color_map[i][2] * 0.5
  96.  
  97.                         #img_show[mask] = img_show[mask] * 0.5 + color_mask * 0.5
  98.             #oneimg = Image.fromarray(oneimg.astype('uint8'))
  99.             #可视化
  100.             oneimg = cv2.cvtColor(oneimg, cv2.COLOR_RGB2BGR)
  101.             #oneimg.save(self.args.save_path + os.sep + ori_name, quality=100)
  102.             cv2.imwrite(self.args.save_path + os.sep + drow_ori_name, oneimg)
  103.             #oneimg.save(self.args.save_path+os.sep+drow_ori_name, quality=100)
  104.  
  105.     def test(self):
  106.         self.model.eval()
  107.         self.evaluator.reset()
  108.         # tbar = tqdm(self.test_loader, desc='\r')
  109.         num = len(self.test_loader)
  110.         for i, sample in enumerate(self.test_loader):
  111.             image, target = sample['image'], sample['label']
  112.             print(i,"/",num)
  113.             torch.cuda.synchronize()
  114.             start = time.time()
  115.             with torch.no_grad():
  116.                 output = self.model(image)
  117.             end = time.time()
  118.             times = (end - start) * 1000
  119.             print(times, "ms")
  120.             torch.cuda.synchronize()
  121.             pred = output.data.cpu().numpy()
  122.             target = target.cpu().numpy()
  123.  
  124.             image1 = image.data.cpu().numpy()
  125.             # #target1 = target.cpu().numpy()
  126.             image1 = image1[0, :]
  127.             target1 = target[0, :]
  128.             # #image1.reshape([image1.size[1],image1.size[2],image1.size[3]])
  129.             # #target1.reshape([image1.size[1],image1.size[2],image1.size[3]])
  130.             image1 = image1.transpose(1,2,0)
  131.             # #target1 = target1.transpose(2,1,0)
  132.             # import cv2
  133.             # image1 = cv2.cvtColor(image1, cv2.COLOR_RGB2BGR)
  134.             # import cv2
  135.             # cv2.imwrite("./image1.jpg",image1)
  136.             cv2.imwrite("./target111.jpg", target1)
  137.  
  138.             pred = np.argmax(pred, axis=1)
  139.  
  140.  
  141.             self.save_image(pred[0], i, 0, True, sample['ori_image'])
  142.             self.save_image(target[0], i, 1, None, sample['ori_image'])
  143.             self.evaluator.add_batch(target, pred)
  144.     
  145.         Acc = self.evaluator.Pixel_Accuracy()
  146.         Acc_class = self.evaluator.Pixel_Accuracy_Class()
  147.         mIoU = self.evaluator.Mean_Intersection_over_Union()
  148.         print('Acc:{}, Acc_class:{}, mIoU:{}'.format(Acc, Acc_class, mIoU))
  149.  
  150. def main():
  151.     # import cv2
  152.     # cvimg = cv2.imread("./dog.jpg")
  153.     # graycvimg = cv2.cvtColor(cvimg, cv2.COLOR_BGR2GRAY)
  154.     # cv2.imwrite("./dog_gray.jpg", graycvimg)
  155.     # graycvimg_bgr = cv2.cvtColor(graycvimg, cv2.COLOR_GRAY2BGR)
  156.     # cv2.imwrite("./dog_gray_bgr.jpg", graycvimg_bgr)
  157.  
  158.  
  159.     parser = argparse.ArgumentParser(description='Pytorch DeeplabV3Plus Test your data')
  160.     parser.add_argument('--test', action='store_true', default=True, 
  161.                         help='test your data')
  162.     parser.add_argument('--dataset', default='pascal', 
  163.                         help='datset format')
  164.     parser.add_argument('--backbone', default='xception', 
  165.                         help='what is your network backbone')
  166.     parser.add_argument('--out_stride', type=int, default=16,
  167.                         help='output stride')
  168.     parser.add_argument('--crop_size', type=int, default=513,
  169.                         help='image size')
  170.     parser.add_argument('--model', type=str, default='/Users/jaeminjung/develop/aidentify/MoE_ws/result/cheonan_24/model_best.pth.tar',
  171.                         help='load your model')
  172.     parser.add_argument('--save_path', type=str, default='/Users/jaeminjung/develop/aidentify/MoE_ws/result/20191001_img',
  173.                         help='save your prediction data')
  174.  
  175.     args = parser.parse_args()
  176.     
  177.     if args.test:
  178.         tester = Tester(args)
  179.         tester.test()
  180.  
  181. if __name__ == "__main__":
  182.     main()

我们不测试val，直接生成test的预测图：

  1. import argparse
  2. import os
  3. import numpy as np
  4. import tqdm
  5. import torch
  6.  
  7. from PIL import Image
  8. from dataloaders import make_data_loader
  9. from modeling.deeplab import *
  10. from dataloaders.utils import get_pascal_labels
  11. from utils.metrics import Evaluator
  12.  
  13.  
  14. class Tester(object):
  15.     def __init__(self, args):
  16.         if not os.path.isfile(args.model):
  17.             raise RuntimeError("no checkpoint found at '{}'".fromat(args.model))
  18.         self.args = args
  19.         self.color_map = get_pascal_labels()
  20.         self.nclass = 2
  21.  
  22.         # Define model
  23.         model = DeepLab(num_classes=self.nclass,
  24.                         backbone=args.backbone,
  25.                         output_stride=args.out_stride,
  26.                         sync_bn=False,
  27.                         freeze_bn=False)
  28.  
  29.         self.model = model
  30.         device = torch.device('cpu')
  31.         checkpoint = torch.load(args.model, map_location=device)
  32.         self.model.load_state_dict(checkpoint['state_dict'])
  33.  
  34.     def save_image(self, imgarray, array, id, op):
  35.         text = 'gt'
  36.         if op == 0:
  37.             text = 'pred'
  38.         file_name = str(id) + '_' + text + '.png'
  39.         # r = array.copy()
  40.         # g = array.copy()
  41.         # b = array.copy()
  42.         # for i in range(self.nclass):
  43.         #     r[array == i] = self.color_map[i][0]
  44.         #     g[array == i] = self.color_map[i][1]
  45.         #     b[array == i] = self.color_map[i][2]
  46.         # rgb = np.dstack((r, g, b))
  47.  
  48.         #tensor 转换为 numpy
  49.         numpyimg = imgarray.numpy()
  50.         #numpy 转换为 IPL格式
  51.         IPLimage = numpyimg.transpose((1, 2, 0))
  52.         '''
  53.         IPL转换为tensor
  54.         _img = Image.open(os.path.join(self.img_dir, path)).convert('RGB')
  55.         img = np.array(img).astype(np.float32).transpose((2, 0, 1))
  56.         img = torch.from_numpy(img).float()
  57.         img = img.cuda()
  58.         
  59.         tensor转换为IPL
  60.         image1 = image.data.cpu().numpy()
  61.         IPLimage = numpyimg.transpose((1, 2, 0))
  62.         save_img = Image.fromarray(IPLimage.astype('uint8'))
  63.         '''
  64.  
  65.         for i in range(self.nclass):
  66.             if i != 0:
  67.                 index = np.argwhere(array == i)
  68.                 for key in index:
  69.                     IPLimage[key[0]][key[1]][0] = IPLimage[key[0]][key[1]][0] * 0.5 + self.color_map[i][0] * 0.5
  70.                     IPLimage[key[0]][key[1]][1] = IPLimage[key[0]][key[1]][1] * 0.5 + self.color_map[i][1] * 0.5
  71.                     IPLimage[key[0]][key[1]][2] = IPLimage[key[0]][key[1]][2] * 0.5 + self.color_map[i][2] * 0.5
  72.         save_img = Image.fromarray(IPLimage.astype('uint8'))
  73.         save_img.save(self.args.save_path + os.sep + file_name)
  74.  
  75.     def transform_val(self, sample):
  76.         from torchvision import transforms
  77.         from dataloaders import custom_transforms as tr
  78.         composed_transforms = transforms.Compose([
  79.             tr.FixScaleCrop(crop_size=self.args.crop_size),
  80.             tr.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),
  81.             tr.ToTensor()
  82.             ])
  83.         return composed_transforms(sample)
  84.  
  85.     def test(self):
  86.         self.model.eval()
  87.         from PIL import Image
  88.         file = open('./test_marker.txt', 'r')
  89.         newpath = "/media/spple/新加卷/Dataset/data/marker_data/marker20191021/all/"
  90.         text_lines = file.readlines()
  91.         for i in range(len(text_lines)):
  92.             namename = text_lines[i].replace("\n", "")
  93.             namename = namename.replace("\t", "")
  94.             imgsname = newpath + namename
  95.             img = Image.open(imgsname).convert('RGB')
  96.             imglabel = Image.open(imgsname).convert('P')
  97.             #arrayimg = np.array(img).astype(np.float32)
  98.             #transposeimg = arrayimg.transpose((2, 0, 1))
  99.             sample = {'image': img, 'label': imglabel, 'ori_image': img, 'path': None}
  100.             imgdist = self.transform_val(sample)
  101.             image = imgdist['image']
  102.             ori_image = imgdist['ori_image']
  103.             image = image.unsqueeze(0)
  104.             with torch.no_grad():
  105.                 output = self.model(image)
  106.             pred = output.data.cpu().numpy()
  107.             pred = np.argmax(pred, axis=1)
  108.             self.save_image(ori_image, pred[0], namename.split(".jpg")[0], 0)
  109.  
  110.  
  111. def main():
  112.     parser = argparse.ArgumentParser(description='Pytorch DeeplabV3Plus Test your data')
  113.     parser.add_argument('--test', action='store_true', default=True,
  114.                         help='test your data')
  115.     parser.add_argument('--dataset', default='pascal',
  116.                         help='datset format')
  117.     parser.add_argument('--backbone', default='xception',
  118.                         help='what is your network backbone')
  119.     parser.add_argument('--out_stride', type=int, default=16,
  120.                         help='output stride')
  121.     parser.add_argument('--crop_size', type=int, default=513,
  122.                         help='image size')
  123.     parser.add_argument('--model', type=str, default='',
  124.                         help='load your model')
  125.     parser.add_argument('--save_path', type=str, default='',
  126.                         help='save your prediction data')
  127.  
  128.     args = parser.parse_args()
  129.  
  130.     if args.test:
  131.         tester = Tester(args)
  132.         tester.test()
  133.  
  134.  
  135. if __name__ == "__main__":
  136.     main()

 

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/weixin_40725706/article/detail/358102
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号