devbox
Cpp五条
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

YOLOv9有效提点|加入BAM、CloFormer、Reversible Column Networks、Lskblock等几十种注意力机制（二）_在yolov9添加空间注意力机制

作者：Cpp五条 | 2024-06-10 09:05:15

赞

踩

在yolov9添加空间注意力机制

 

专栏介绍：YOLOv9改进系列 | 包含深度学习最新创新，主力高效涨点！！！

一、本文介绍

 本文只有代码及注意力模块简介，YOLOv9中的添加教程：可以看这篇文章。

YOLOv9有效提点|加入SE、CBAM、ECA、SimAM等几十种注意力机制（一）

CloFormer:《Rethinking Local Perception in Lightweight Vision Transformer》

        CloFormer是一种轻量级视觉转换器，它通过利用上下文感知的本地增强，提高了在图像分类、目标检测和语义分割等任务上的性能。CloFormer通过引入一种名为AttnConv的卷积操作，结合共享权重和上下文感知权重，有效地捕捉了高频率的本地信息。实验结果表明，CloFormer在各种视觉任务中具有显著优势。

  1. class AttnMap(nn.Module):
  2.     def __init__(self, dim):
  3.         super().__init__()
  4.         self.act_block = nn.Sequential(
  5.                             nn.Conv2d(dim, dim, 1, 1, 0),
  6.                             MemoryEfficientSwish(),
  7.                             nn.Conv2d(dim, dim, 1, 1, 0)
  8.                             #nn.Identity()
  9.                          )
  10.     def forward(self, x):
  11.         return self.act_block(x)
  12.  
  13. class EfficientAttention(nn.Module):
  14.  
  15.     def __init__(self, dim, num_heads, group_split: List[int], kernel_sizes: List[int], window_size=7, 
  16.                  attn_drop=0., proj_drop=0., qkv_bias=True):
  17.         super().__init__()
  18.         assert sum(group_split) == num_heads
  19.         assert len(kernel_sizes) + 1 == len(group_split)
  20.         self.dim = dim
  21.         self.num_heads = num_heads
  22.         self.dim_head = dim // num_heads
  23.         self.scalor = self.dim_head ** -0.5
  24.         self.kernel_sizes = kernel_sizes
  25.         self.window_size = window_size
  26.         self.group_split = group_split
  27.         convs = []
  28.         act_blocks = []
  29.         qkvs = []
  30.         #projs = []
  31.         for i in range(len(kernel_sizes)):
  32.             kernel_size = kernel_sizes[i]
  33.             group_head = group_split[i]
  34.             if group_head == 0:
  35.                 continue
  36.             convs.append(nn.Conv2d(3*self.dim_head*group_head, 3*self.dim_head*group_head, kernel_size,
  37.                          1, kernel_size//2, groups=3*self.dim_head*group_head))
  38.             act_blocks.append(AttnMap(self.dim_head*group_head))
  39.             qkvs.append(nn.Conv2d(dim, 3*group_head*self.dim_head, 1, 1, 0, bias=qkv_bias))
  40.             #projs.append(nn.Linear(group_head*self.dim_head, group_head*self.dim_head, bias=qkv_bias))
  41.         if group_split[-1] != 0:
  42.             self.global_q = nn.Conv2d(dim, group_split[-1]*self.dim_head, 1, 1, 0, bias=qkv_bias)
  43.             self.global_kv = nn.Conv2d(dim, group_split[-1]*self.dim_head*2, 1, 1, 0, bias=qkv_bias)
  44.             #self.global_proj = nn.Linear(group_split[-1]*self.dim_head, group_split[-1]*self.dim_head, bias=qkv_bias)
  45.             self.avgpool = nn.AvgPool2d(window_size, window_size) if window_size!=1 else nn.Identity()
  46.  
  47.         self.convs = nn.ModuleList(convs)
  48.         self.act_blocks = nn.ModuleList(act_blocks)
  49.         self.qkvs = nn.ModuleList(qkvs)
  50.         self.proj = nn.Conv2d(dim, dim, 1, 1, 0, bias=qkv_bias)
  51.         self.attn_drop = nn.Dropout(attn_drop)
  52.         self.proj_drop = nn.Dropout(proj_drop)
  53.  
  54.     def high_fre_attntion(self, x: torch.Tensor, to_qkv: nn.Module, mixer: nn.Module, attn_block: nn.Module):
  55.         '''
  56.         x: (b c h w)
  57.         '''
  58.         b, c, h, w = x.size()
  59.         qkv = to_qkv(x) #(b (3 m d) h w)
  60.         qkv = mixer(qkv).reshape(b, 3, -1, h, w).transpose(0, 1).contiguous() #(3 b (m d) h w)
  61.         q, k, v = qkv #(b (m d) h w)
  62.         attn = attn_block(q.mul(k)).mul(self.scalor)
  63.         attn = self.attn_drop(torch.tanh(attn))
  64.         res = attn.mul(v) #(b (m d) h w)
  65.         return res
  66.         
  67.     def low_fre_attention(self, x : torch.Tensor, to_q: nn.Module, to_kv: nn.Module, avgpool: nn.Module):
  68.         '''
  69.         x: (b c h w)
  70.         '''
  71.         b, c, h, w = x.size()
  72.         
  73.         q = to_q(x).reshape(b, -1, self.dim_head, h*w).transpose(-1, -2).contiguous() #(b m (h w) d)
  74.         kv = avgpool(x) #(b c h w)
  75.         kv = to_kv(kv).view(b, 2, -1, self.dim_head, (h*w)//(self.window_size**2)).permute(1, 0, 2, 4, 3).contiguous() #(2 b m (H W) d)
  76.         k, v = kv #(b m (H W) d)
  77.         attn = self.scalor * q @ k.transpose(-1, -2) #(b m (h w) (H W))
  78.         attn = self.attn_drop(attn.softmax(dim=-1))
  79.         res = attn @ v #(b m (h w) d)
  80.         res = res.transpose(2, 3).reshape(b, -1, h, w).contiguous()
  81.         return res
  82.  
  83.     def forward(self, x: torch.Tensor):
  84.         '''
  85.         x: (b c h w)
  86.         '''
  87.         res = []
  88.         for i in range(len(self.kernel_sizes)):
  89.             if self.group_split[i] == 0:
  90.                 continue
  91.             res.append(self.high_fre_attntion(x, self.qkvs[i], self.convs[i], self.act_blocks[i]))
  92.         if self.group_split[-1] != 0:
  93.             res.append(self.low_fre_attention(x, self.global_q, self.global_kv, self.avgpool))
  94.         return self.proj_drop(self.proj(torch.cat(res, dim=1)))

《Reversible Column Networks》

        Reversible Column Networks一种新的神经网络设计范式——可逆列网络（RevCol）。RevCol主要由多个子网络（称为“列”）的副本组成，这些子网络之间使用了多级可逆连接。这种架构方案使得RevCol的行为与传统的网络非常不同：在正向传播过程中，当特征通过每个列时，它们被逐渐解开，同时保持总信息量，而不是像其他网络那样进行压缩或丢弃。

 这个暂时没调试，代码地址：RevCol/models/revcol.py at main · megvii-research/RevCol (github.com)icon-default.png?t=N7T8https://github.com/megvii-research/RevCol/blob/main/models/revcol.py

《BAM: Bottleneck Attention Module》

        瓶颈注意模块(BAM)关注深度神经网络中注意力机制的影响，提出了一个简单而有效的注意力模块，即瓶颈注意模块(BAM)，可以与任何前馈卷积神经网络集成，沿着两个不同的路径（通道和空间）推断注意力映射。 将模块放在模型的每个瓶颈处（特征映射产生降采样），构建一个具有多个参数的分层注意，可以与任何前馈模型以端到端方式进行训练。

  1. def autopad(k, p=None, d=1):  # kernel, padding, dilation
  2.     """Pad to 'same' shape outputs."""
  3.     if d > 1:
  4.         k = d * (k - 1) + 1 if isinstance(k, int) else [d * (x - 1) + 1 for x in k]  # actual kernel-size
  5.     if p is None:
  6.         p = k // 2 if isinstance(k, int) else [x // 2 for x in k]  # auto-pad
  7.     return p
  8.  
  9. class Flatten(nn.Module):
  10.     def forward(self, x):
  11.         return x.view(x.shape[0], -1)
  12.  
  13.  
  14. class ChannelAttention(nn.Module):
  15.     def __init__(self, channel, reduction=16, num_layers=3):
  16.         super().__init__()
  17.         self.avgpool = nn.AdaptiveAvgPool2d(1)
  18.         gate_channels = [channel]
  19.         gate_channels += [channel // reduction] * num_layers
  20.         gate_channels += [channel]
  21.  
  22.         self.ca = nn.Sequential()
  23.         self.ca.add_module('flatten', Flatten())
  24.         for i in range(len(gate_channels) - 2):
  25.             self.ca.add_module('fc%d' % i, nn.Linear(gate_channels[i], gate_channels[i + 1]))
  26.             self.ca.add_module('bn%d' % i, nn.BatchNorm1d(gate_channels[i + 1]))
  27.             self.ca.add_module('relu%d' % i, nn.ReLU())
  28.         self.ca.add_module('last_fc', nn.Linear(gate_channels[-2], gate_channels[-1]))
  29.  
  30.     def forward(self, x):
  31.         res = self.avgpool(x)
  32.         res = self.ca(res)
  33.         res = res.unsqueeze(-1).unsqueeze(-1).expand_as(x)
  34.         return res
  35.  
  36.  
  37. class SpatialAttention(nn.Module):
  38.     def __init__(self, channel, reduction=16, num_layers=3, dia_val=2):
  39.         super().__init__()
  40.         self.sa = nn.Sequential()
  41.         self.sa.add_module('conv_reduce1',
  42.                            nn.Conv2d(kernel_size=1, in_channels=channel, out_channels=channel // reduction))
  43.         self.sa.add_module('bn_reduce1', nn.BatchNorm2d(channel // reduction))
  44.         self.sa.add_module('relu_reduce1', nn.ReLU())
  45.         for i in range(num_layers):
  46.             self.sa.add_module('conv_%d' % i, nn.Conv2d(kernel_size=3, in_channels=channel // reduction,
  47.                                                         out_channels=channel // reduction, padding=autopad(3, None, dia_val), dilation=dia_val))
  48.             self.sa.add_module('bn_%d' % i, nn.BatchNorm2d(channel // reduction))
  49.             self.sa.add_module('relu_%d' % i, nn.ReLU())
  50.         self.sa.add_module('last_conv', nn.Conv2d(channel // reduction, 1, kernel_size=1))
  51.  
  52.     def forward(self, x):
  53.         res = self.sa(x)
  54.         res = res.expand_as(x)
  55.         return res
  56.  
  57.  
  58. class BAMBlock(nn.Module):
  59.     def __init__(self, channel=512, reduction=16, dia_val=2):
  60.         super().__init__()
  61.         self.ca = ChannelAttention(channel=channel, reduction=reduction)
  62.         self.sa = SpatialAttention(channel=channel, reduction=reduction, dia_val=dia_val)
  63.         self.sigmoid = nn.Sigmoid()
  64.  
  65.     def init_weights(self):
  66.         for m in self.modules():
  67.             if isinstance(m, nn.Conv2d):
  68.                 init.kaiming_normal_(m.weight, mode='fan_out')
  69.                 if m.bias is not None:
  70.                     init.constant_(m.bias, 0)
  71.             elif isinstance(m, nn.BatchNorm2d):
  72.                 init.constant_(m.weight, 1)
  73.                 init.constant_(m.bias, 0)
  74.             elif isinstance(m, nn.Linear):
  75.                 init.normal_(m.weight, std=0.001)
  76.                 if m.bias is not None:
  77.                     init.constant_(m.bias, 0)
  78.  
  79.     def forward(self, x):
  80.         b, c, _, _ = x.size()
  81.         sa_out = self.sa(x)
  82.         ca_out = self.ca(x)
  83.         weight = self.sigmoid(sa_out + ca_out)
  84.         out = (1 + weight) * x
  85.         return out

《Large Selective Kernel Network for Remote Sensing Object Detection》

         LSKnet是一种用于遥感目标检测的大规模选择性核网络，改论文提出了远程感应目标检测新方法LSKNet，这种网络可以动态地调整其大的空间感受野，以更好地模拟远程感应场景中不同物体的范围上下文。文章中提到，这是首次在远程感应目标检测领域探索大型和选择性核机制。

  1. class LSKblock(nn.Module):
  2.     def __init__(self, dim):
  3.         super().__init__()
  4.         self.conv0 = nn.Conv2d(dim, dim, 5, padding=2, groups=dim)
  5.         self.conv_spatial = nn.Conv2d(dim, dim, 7, stride=1, padding=9, groups=dim, dilation=3)
  6.         self.conv1 = nn.Conv2d(dim, dim//2, 1)
  7.         self.conv2 = nn.Conv2d(dim, dim//2, 1)
  8.         self.conv_squeeze = nn.Conv2d(2, 2, 7, padding=3)
  9.         self.conv = nn.Conv2d(dim//2, dim, 1)
  10.  
  11.     def forward(self, x):   
  12.         attn1 = self.conv0(x)
  13.         attn2 = self.conv_spatial(attn1)
  14.  
  15.         attn1 = self.conv1(attn1)
  16.         attn2 = self.conv2(attn2)
  17.         
  18.         attn = torch.cat([attn1, attn2], dim=1)
  19.         avg_attn = torch.mean(attn, dim=1, keepdim=True)
  20.         max_attn, _ = torch.max(attn, dim=1, keepdim=True)
  21.         agg = torch.cat([avg_attn, max_attn], dim=1)
  22.         sig = self.conv_squeeze(agg).sigmoid()
  23.         attn = attn1 * sig[:,0,:,:].unsqueeze(1) + attn2 * sig[:,1,:,:].unsqueeze(1)
  24.         attn = self.conv(attn)
  25.         return x * attn

 

声明：本文内容由网友自发贡献，转载请注明出处：【wpsshop】
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号