SwinTransformer搭建及一些问题_swin transformer 环境搭建

Swin

装环境

打开Anaconda Powershell Prompt！！！

创建环境：1660TI，cuda11.0，pytorch1.7.1

注意：cmd先nvcc -V看看，是不是cuda11

创建环境

conda create -n swtf python=3.8
1

激活环境

conda activate swtf
1

安装pytorch

# 1660Ti
conda install pytorch=1.7.1 torchvision cudatoolkit=11.0 -c pytorch
# 3060
conda install pytorch=1.8.1 torchvision cudatoolkit=11.1 -c pytorch
1
2
3
4

轮子安装pytorch:先定位到轮子位置

#pip install torch-1.7.0+cu110-cp38-cp38-win_amd64.whl
1
'
运行
运行

安装个git方便下代码

conda install git
1

安装mmcv

建好文件夹，在Anaconda Powershell Prompt切换到那个文件夹再下载

cd F:\Gpytorch\swintransformer
1

下载mmcv

git clone -b v1.3.1 https://github.com/open-mmlab/mmcv.git
1

注意：Turn off this advice by setting config variable advice.detachedHead to false，出来个这玩意，我不懂，但是确实下载下来了

cd mmcv
pip install -r requirements.txt
1
2

（1） 找到cl.exe所在位置。设置环境变量，添加 C:\Program Files (x86)\Microsoft Visual

Studio\2019\Community\VC\Tools\MSVC\14.28.29910\bin\Hostx64\x86 到 Path 使得cl.exe可以

在命令窗口中执行

注意：cl.exe所在的具体路径和自己的电脑有关

具体做法：

把上述路径加入Path环境变量，并上移到最顶端

执行立即生效命令 set Path=c

执行 cl

（2） 查找GPU的计算能力from https://developer.nvidia.cn/zh-cn/cuda-gpus

1660TI的算力是7.5；3060的算力是8.6

在Power Shell命令窗口中执行：(此时再mmcv文件夹下)

$env:TORCH_CUDA_ARCH_LIST="7.5"
$env:MMCV_WITH_OPS = 1
$env:MAX_JOBS = 4  
# 编译
python setup.py build_ext
# 安装
python setup.py develop
# 看一下安好没
pip list mmcv
1
2
3
4
5
6
7
8
9

bug1:RuntimeError: Error compiling objects for extension

翻到最上面报错信息，发现是：

subprocess.CalledProcessError: Command '['ninja', '-v', '-j', '4']' returned non-zero exit status 1.
1

未解决。。。卸了重来就好了。。。环境变量的cuda_path改一下？

安装mmdetation

下载mmdetation

git clone -b v2.11.0 https://github.com/open-mmlab/mmdetection.git
cd mmdetection
pip install -r requirements/build.txt
pip install -v -e . # or "python setup.py develop"
1
2
3
4

安装apex

下载apex

git clone https://github.com/NVIDIA/apex
cd apex
python setup.py install
1
2
3

安装swin-trnsformer

下载swin

# 手动下载网址
# https://github.com/SwinTransformer/Swin-Transformer-Object-Detection

git clone https://github.com/SwinTransformer/Swin-Transformer-Object-Detection.git
cd Swin-Transformer-Object-Detection
python setup.py develop
1
2
3
4
5
6

下载预训练权重

# 下载mask_rcnn_swin_tiny_patch4_window7_1x.pth这个
1
'
运行
运行

测试demo

# 下面的代码是1行！！！！！
python demo/image_demo.py demo/demo.jpg 
configs/swin/mask_rcnn_swin_tiny_patch4_window7_mstrain_480-800_adamw_1x_coco.py 
mask_rcnn_swin_tiny_patch4_window7_1x.pth
1
2
3
4

制作数据集

安装labelme3.16.5，这个版本没那么多毛病

pip install labelme==3.16.5
1

输入labelme打开，再标注，具体百度

制作coco数据集格式

具体百度，制作完目录结构如下：

└── data
 ├── coco
 	├── annotations
 		├── instances_train2017.json
 		└── instances_val2017.json
 	├── train2017
 	└── val2017
1
2
3
4
5
6
7

将data放入swin-transformer根目录下

训练swin-transformer

创建python文件，命名changemaskrcnn，并将其放入根目录

import torch
pretrained_weights = torch.load('mask_rcnn_swin_tiny_patch4_window7_1x.pth') # 预训练
num_class = 5 # 类别数
pretrained_weights['state_dict']['roi_head.bbox_head.fc_cls.weight'].resize_(num_class+1,1024)
pretrained_weights['state_dict']['roi_head.bbox_head.fc_cls.bias'].resize_(num_class+1)
pretrained_weights['state_dict']['roi_head.bbox_head.fc_reg.weight'].resize_(num_class*4,1024)
pretrained_weights['state_dict']['roi_head.bbox_head.fc_reg.bias'].resize_(num_class*4)
pretrained_weights['state_dict']['roi_head.mask_head.conv_logits.weight'].resize_(num_class,256,1,1)
pretrained_weights['state_dict']['roi_head.mask_head.conv_logits.bias'].resize_(num_class)
torch.save(pretrained_weights, "mask_rcnn_swin_%d.pth"%num_class)
1
2
3
4
5
6
7
8
9
10

修改配置文件

1）修改changemaskrcnn.py中num_class并执行，产生新的权重文件
权重文件名mask_rcnn_swin_（类别数）.pth
2）修改 configs\_base_\models\mask_rcnn_swin_fpn.py 中num_classes， 共两处
改成类别数
3）修改 configs\_base_\default_runtime.py 中interval, load_from
第一个interval→多少轮保存一次，第二个interval→多少轮显示一次
load_from→"mask_rcnn_swin_（类别数）.pth"
4）修改 configs\swin\mask_rcnn_swin_tiny_patch4_window7_mstrain_480-
800_adamw_1x_coco.py 中的 max_epochs, lr_config
max_epochs→最大轮数，lr_config→学习率
5）修改 configs\_base_\datasets\coco_instance.py 中的
samples_per_gpu=1, # batchsize
workers_per_gpu=0, # GPU个数
6）修改mmdet\datasets\coco.py中的CLASSES
1
2
3
4
5
6
7
8
9
10
11
12
13
14

bug2，loss为nan

可以把配置文件4）的use_fp16=True注释掉，大约最后一行。

训练命令

python tools/train.py 
configs/swin/mask_rcnn_swin_tiny_patch4_window7_mstrain_480-800_adamw_1x_coco.py
1
2

bug3→史称最大bug！！！

AssertionError: The num_classes (1) in Shared2FCBBoxHead of MMDataParallel。。。

类别输入完了后面加个逗号！！！例如

CLASSES = ('pest',)
1
'
运行
运行

bug3.1→最大bug2！！

# 鄙人在做mask_rcnn101时，出现以下错误
File "/content/Swin-Transformer-Object-Detection/mmdet/datasets/coco.py", line 267, in _segm2json
if isinstance(segms[i]['counts'], bytes):
IndexError: list index out of range
1
2
3
4

解决方式

路径：…/mmdet/models/roi_heads/test_mixins.py

test_mixins.py做了修改，源代码在test_mixins0.py中

118行做了修改：

# supplement_mask = rois[..., -1] == 0
supplement_mask = rois.abs()[..., 1:].sum(dim=-1) == 0
1
2

141行做了修改：

# supplement_mask = proposals[i][..., -1] == 0
supplement_mask = rois.abs()[..., 1:].sum(dim=-1) == 0
1
2

305行做了修改：

# supplement_mask = det_bbox[..., -1] != 0
supplement_mask = det_bbox.abs().sum(dim=-1) != 0
1
2

bug4→存不下

到checkpoint.py注释掉出错的一行

bug5→验证未完成显存炸了

这是因为每次训练后默认预测

configs/_base_/datasets/coco_instance.py 最后一行：
evaluation = dict(metric=['bbox', 'segm'], interval=50) # 训练50轮在测试
1
2

测试命令

python demo/image_demo.py (图片路径) configs/swin/mask_rcnn_swin_tiny_patch4_window7_mstrain_480-800_adamw_1x_coco.py work_dirs/mask_rcnn_swin_tiny_patch4_window7_mstrain_480-800_adamw_1x_coco/latest.pth
1

bug6→测试时显存爆了→RuntimeError: CUDA out of memory

我显存6G确实不够，那就在cpu跑测试

python demo/image_demo.py (图片路径) configs/swin/mask_rcnn_swin_tiny_patch4_window7_mstrain_480-800_adamw_1x_coco.py work_dirs/mask_rcnn_swin_tiny_patch4_window7_mstrain_480-800_adamw_1x_coco/latest.pth --device cpu
1

性能统计

python tools/test.py 
configs/swin/mask_rcnn_swin_tiny_patch4_window7_mstrain_480-800_adamw_1x_coco.py work_dirs/mask_rcnn_swin_tiny_patch4_window7_mstrain_480-800_adamw_1x_coco/latest.pth 
--eval bbox segm
1
2
3

bug7→eval时显存爆了→RuntimeError: CUDA out of memory

(21条消息) 【pytorch】mmdetection 做eval / test时弹出OOM（Out of Memory / CUDA out of memory）的解决过程记录_煎pan上的狸猫的博客-CSDN博客

https://blog.csdn.net/weixin_42362903/article/details/122750216 大神给出了解决方案

我的按照大神的思路是移到cpu进行评估

找到文件所在位置进行更改

# 评估yolact时
# mmdet/models/dense_heads/yolact_head.py中大概854行也加了个.cpu（）
        mask_pred = F.interpolate(
            mask_pred.unsqueeze(0).cpu(), (img_h, img_w), # 加了个.cpu()
            mode='bilinear',
            align_corners=False).squeeze(0) > 0.5
        mask_pred = mask_pred.cpu().numpy().astype(np.uint8)
1
2
3
4
5
6
7

更改IOU为0.5进行评估

# mmdet/datasets/coco.py
ctrl+F搜索evaluate函数，iou_thrs=None→iou_thrs=[0.5]
1
2

日志分析

python tools/analysis_tools/analyze_logs.py plot_curve work_dirs/mask_rcnn_swin_tiny_patch4_window7_mstrain_480800_adamw_1x_coco/（自己的日志文件，如20211226_093405.log.json）
1

bug7 →ModuleNotFoundError: No module named ‘seaborn’

那就安装一下

conda install seaborn
1

要是还出bug，那就自己写一个，参考以下这位大神：

(21条消息) mmdet-tools工具测试_知识在于分享的博客-CSDN博客

https://blog.csdn.net/baidu_40840693/article/details/120326501

代码：

# analyze_logs.py
import argparse
import json
from collections import defaultdict
 
import matplotlib.pyplot as plt
import numpy as np
import seaborn as sns
 
 
def cal_train_time(log_dicts, args):
    for i, log_dict in enumerate(log_dicts):
        print(f'{"-" * 5}Analyze train time of {args.json_logs[i]}{"-" * 5}')
        all_times = []
        for epoch in log_dict.keys():
            if args.include_outliers:
                all_times.append(log_dict[epoch]['time'])
            else:
                all_times.append(log_dict[epoch]['time'][1:])
        all_times = np.array(all_times)
        epoch_ave_time = all_times.mean(-1)
        slowest_epoch = epoch_ave_time.argmax()
        fastest_epoch = epoch_ave_time.argmin()
        std_over_epoch = epoch_ave_time.std()
        print(f'slowest epoch {slowest_epoch + 1}, '
              f'average time is {epoch_ave_time[slowest_epoch]:.4f}')
        print(f'fastest epoch {fastest_epoch + 1}, '
              f'average time is {epoch_ave_time[fastest_epoch]:.4f}')
        print(f'time std over epochs is {std_over_epoch:.4f}')
        print(f'average iter time: {np.mean(all_times):.4f} s/iter')
        print()
 
 
def plot_curve(log_dicts, args):
    if args.backend is not None:
        plt.switch_backend(args.backend)
    sns.set_style(args.style)
    # if legend is None, use {filename}_{key} as legend
    legend = args.legend
    if legend is None:
        legend = []
        for json_log in args.json_logs:
            for metric in args.keys:
                legend.append(f'{json_log}_{metric}')
    assert len(legend) == (len(args.json_logs) * len(args.keys))
    metrics = args.keys
 
    num_metrics = len(metrics)
    for i, log_dict in enumerate(log_dicts):
        epochs = list(log_dict.keys())
        for j, metric in enumerate(metrics):
            print(f'plot curve of {args.json_logs[i]}, metric is {metric}')
            if metric not in log_dict[epochs[0]]:
                raise KeyError(
                    f'{args.json_logs[i]} does not contain metric {metric}')
 
            if 'mAP' in metric:
                xs = np.arange(1, max(epochs) + 1)
                ys = []
                for epoch in epochs:
                    ys += log_dict[epoch][metric]
                ax = plt.gca()
                ax.set_xticks(xs)
                plt.xlabel('epoch')
                plt.plot(xs, ys, label=legend[i * num_metrics + j], marker='o')
            else:
                xs = []
                ys = []
                num_iters_per_epoch = log_dict[epochs[0]]['iter'][-1]
                for epoch in epochs:
                    iters = log_dict[epoch]['iter']
                    if log_dict[epoch]['mode'][-1] == 'val':
                        iters = iters[:-1]
                    xs.append(
                        np.array(iters) + (epoch - 1) * num_iters_per_epoch)
                    ys.append(np.array(log_dict[epoch][metric][:len(iters)]))
                xs = np.concatenate(xs)
                ys = np.concatenate(ys)
                plt.xlabel('iter')
                plt.plot(
                    xs, ys, label=legend[i * num_metrics + j], linewidth=0.5)
            plt.legend()
        if args.title is not None:
            plt.title(args.title)
    if args.out is None:
        plt.show()
    else:
        print(f'save curve to: {args.out}')
        plt.savefig(args.out)
        plt.cla()
 
 
def add_plot_parser(subparsers):
    parser_plt = subparsers.add_parser(
        'plot_curve', help='parser for plotting curves')
    parser_plt.add_argument(
        'json_logs',
        type=str,
        nargs='+',
        help='path of train log in json format')
    parser_plt.add_argument(
        '--keys',
        type=str,
        nargs='+',
        default=['bbox_mAP'],
        help='the metric that you want to plot')
    parser_plt.add_argument('--title', type=str, help='title of figure')
    parser_plt.add_argument(
        '--legend',
        type=str,
        nargs='+',
        default=None,
        help='legend of each plot')
    parser_plt.add_argument(
        '--backend', type=str, default=None, help='backend of plt')
    parser_plt.add_argument(
        '--style', type=str, default='dark', help='style of plt')
    parser_plt.add_argument('--out', type=str, default=None)
 
 
def add_time_parser(subparsers):
    parser_time = subparsers.add_parser(
        'cal_train_time',
        help='parser for computing the average time per training iteration')
    parser_time.add_argument(
        'json_logs',
        type=str,
        nargs='+',
        help='path of train log in json format')
    parser_time.add_argument(
        '--include-outliers',
        action='store_true',
        help='include the first value of every epoch when computing '
        'the average time')
 
 
def parse_args():
    parser = argparse.ArgumentParser(description='Analyze Json Log')
    # currently only support plot curve and calculate average train time
    subparsers = parser.add_subparsers(dest='task', help='task parser')
    add_plot_parser(subparsers)
    add_time_parser(subparsers)
    args = parser.parse_args()
    return args
 
 
def load_json_logs(json_logs):
    # load and convert json_logs to log_dict, key is epoch, value is a sub dict
    # keys of sub dict is different metrics, e.g. memory, bbox_mAP
    # value of sub dict is a list of corresponding values of all iterations
    log_dicts = [dict() for _ in json_logs]
    for json_log, log_dict in zip(json_logs, log_dicts):
        with open(json_log, 'r') as log_file:
            for line in log_file:
                log = json.loads(line.strip())
                # skip lines without `epoch` field
                if 'epoch' not in log:
                    continue
                epoch = log.pop('epoch')
                if epoch not in log_dict:
                    log_dict[epoch] = defaultdict(list)
                for k, v in log.items():
                    log_dict[epoch][k].append(v)
    return log_dicts
 
 
def main():
    args = parse_args()
 
    json_logs = args.json_logs
    for json_log in json_logs:
        assert json_log.endswith('.json')
 
    log_dicts = load_json_logs(json_logs)
 
    eval(args.task)(log_dicts, args)
 
 
if __name__ == '__main__':
    main()
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180

使用：

# 看一下
python analyze_logs.py plot_curve log.json --keys loss_cls loss_bbox --legend loss_cls loss_bbox
# 存一下
python analyze_logs.py plot_curve log.json --keys loss_cls loss_bbox --out losses.pdf
1
2
3
4