赞
踩
FLOPS:每秒浮点运算次数,这个由硬件决定。
GFLOPS:每秒10亿次的浮点运算数,1GFlops = 1,000MFlops。
FLOPs:指计算量的大小,理解为计算量。可以用来衡量算法/模型的复杂度。具体指的是multiply-add数量,计算模型中乘法和加法的运算次数。对于普通卷积层而言:FLOPs = 2*H*W( CinK2+1)Cout。
maccs:是multiply-accumulate operations,指点积运算,一个 macc = 2FLOPs。
MAC:memory access cost 内存访问消耗,这里举个例子更好理解,比如1x1的卷积:MAC=H*W*Cin+H*W*Cout+1*1*Cin*Cout,第一项是输入特征层的内存访问消耗。第二项是输出特征层内存访问消耗,第三项是卷积核的内存访问消耗。
ShuffleNetV2中提出了一个关键点,之前的轻量级网络都是通过计算网络复杂度的一个间接度量,即FLOPs为指导。通过计算浮点运算量来描述轻量级网络的快慢。但是从来不直接考虑运行的速度。在移动设备中的运行速度不仅仅需要考虑FLOPs,还需要考虑其他的因素,比如内存访问成本(memory access cost)和平台特点(platform characterics)。为了衡量计算复杂度,一个广泛使用的指标是浮点运算次数,即FLOPs。然而,FLOPs是一种非直接的指标,是对直接指标例如速度和时延的近似,且并不是等价的,而直接指标才是真正需要关心的。
ShuffleNetV2代码:
- def split(x, groups):
- out = x.chunk(groups, dim=1)
-
- return out
-
-
- def shuffle( x, groups):
- N, C, H, W = x.size()
- out = x.view(N, groups, C // groups, H, W).permute(0, 2, 1, 3, 4).contiguous().view(N, C, H, W)
-
- return out
- class ShuffleUnit(nn.Module):
- def __init__(self, in_channels, out_channels, stride):
- super().__init__()
- mid_channels = out_channels // 2
- if stride > 1:
- self.branch1 = nn.Sequential(
- nn.Conv2d(in_channels, in_channels, 3, stride=stride, padding=1, groups=in_channels, bias=False),
- nn.BatchNorm2d(in_channels),
- nn.Conv2d(in_channels, mid_channels, 1, bias=False),
- nn.BatchNorm2d(mid_channels),
- nn.ReLU(inplace=True)
- )
- self.branch2 = nn.Sequential(
- nn.Conv2d(in_channels, mid_channels, 1, bias=False),
- nn.BatchNorm2d(mid_channels),
- nn.ReLU(inplace=True),
- nn.Conv2d(mid_channels, mid_channels, 3, stride=stride, padding=1, groups=mid_channels, bias=False),
- nn.BatchNorm2d(mid_channels),
- nn.Conv2d(mid_channels, mid_channels, 1, bias=False),
- nn.BatchNorm2d(mid_channels),
- nn.ReLU(inplace=True)
- )
- else:
- self.branch1 = nn.Sequential()
- self.branch2 = nn.Sequential(
- nn.Conv2d(mid_channels, mid_channels, 1, bias=False),
- nn.BatchNorm2d(mid_channels),
- nn.ReLU(inplace=True),
- nn.Conv2d(mid_channels, mid_channels, 3, stride=stride, padding=1, groups=mid_channels, bias=False),
- nn.BatchNorm2d(mid_channels),
- nn.Conv2d(mid_channels, mid_channels, 1, bias=False),
- nn.BatchNorm2d(mid_channels),
- nn.ReLU(inplace=True)
- )
- self.stride = stride
- def forward(self, x):
- if self.stride == 1:
- x1, x2 = split(x, 2)
- out = torch.cat((self.branch1(x1), self.branch2(x2)), dim=1)
- else:
- out = torch.cat((self.branch1(x), self.branch2(x)), dim=1)
- out = shuffle(out, 2)
- return out
- class ShuffleNetV2(nn.Module):
- def __init__(self, channel_num, class_num=settings.CLASSES_NUM):
- super().__init__()
- self.conv1 = nn.Sequential(
- nn.Conv2d(3, 24, 3, stride=2, padding=1, bias=False),
- nn.BatchNorm2d(24),
- nn.ReLU(inplace=True)
- )
- self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
- self.stage2 = self.make_layers(24, channel_num[0], 4, 2)
- self.stage3 = self.make_layers(channel_num[0], channel_num[1], 8, 2)
- self.stage4 = self.make_layers(channel_num[1], channel_num[2], 4, 2)
- self.conv5 = nn.Sequential(
- nn.Conv2d(channel_num[2], 1024, 1, bias=False),
- nn.BatchNorm2d(1024),
- nn.ReLU(inplace=True)
- )
- self.avgpool = nn.AdaptiveAvgPool2d(1)
- self.fc = nn.Linear(1024, class_num)
- def make_layers(self, in_channels, out_channels, layers_num, stride):
- layers = []
- layers.append(ShuffleUnit(in_channels, out_channels, stride))
- in_channels = out_channels
- for i in range(layers_num - 1):
- ShuffleUnit(in_channels, out_channels, 1)
- return nn.Sequential(*layers)
- def forward(self, x):
- out = self.conv1(x)
- out = self.maxpool(out)
- out = self.stage2(out)
- out = self.stage3(out)
- out = self.stage4(out)
- out = self.conv5(out)
- out = self.avgpool(out)
- out = out.flatten(1)
- out = self.fc(out)
- return out
赞
踩
Copyright © 2003-2013 www.wpsshop.cn 版权所有,并保留所有权利。