赞
踩
部分笔记基于知乎博主:咫尺小厘米 - 知乎
代码: https://github.com/VL-Group/Natural-Color-Fool
无限制黑盒攻击
颜色分布库 Color Distribution Library
作者借鉴了Photographic tone reproduction的思想,基于 ADE20K数据集构建了针对不同语义类的“颜色分布的分布”(“distribution of color distributions“,DoD)。 具体来说,对于每个类别,DoD 提供了 20 个不同的主要分布集(由颜色分布表示),这些分布集是通过基于数据集中相应分割类别的调色板的层次聚类获得的,然后,由于每个簇中语义类的颜色分布相似,我们在每个簇中选择一个对象来表示该样式,最后,我们提取颜色分布形成我们的颜色分布库
Natural Color Fool (NCF) 黑盒攻击主要流程
生成的对抗样本xh'放到代理模型fθ上进行进行攻击,Ladv选用C&W
这篇论文的第2部分“Background on Perceptual Color Distance”主要介绍了感知颜色距离(Perceptual Color Distance)的概念,以及它是如何与传统的RGB颜色空间中的距离度量相比较的。下面是对这部分内容的详细解释:
感知颜色距离的重要性:
CIEDE2000颜色差异公式:
像素级感知颜色距离的计算:
与Lp范数的比较:
相关工作:
其中,各个符号代表的含义如下:
公式中的每个部分都是对颜色空间中特定维度差异的度量。这些维度在CIELCH颜色空间中定义,它是一个更符合人类视觉感知的颜色空间。CIELCH空间将颜色分为三个维度:亮度(L),色度(C),和色相(H),这与RGB空间的线性模型不同。
CIEDE2000公式的关键在于它考虑了颜色差异的非线性特性,以及人类视觉系统对不同颜色变化的敏感度。例如,人眼对亮度变化的敏感度通常高于色度和色相变化。因此,这个公式通过权重因子kL, kC, kH来调整这些维度的相对重要性。
此外,∆R项考虑了色度和色相之间的相互作用,这在传统的颜色差异度量中通常被忽略。这种相互作用的考虑使得CIEDE2000公式能够更准确地模拟人类对颜色差异的感知。
总的来说,CIEDE2000颜色差异公式提供了一种更为精确的方法来量化颜色之间的感知差异,这对于需要精确控制颜色变化的应用(如图像编辑、颜色匹配和对抗样本生成)非常有用。在对抗样本的生成中,使用CIEDE2000可以帮助创建视觉上难以察觉的扰动,同时保持对抗效果
differential_color_functions
- """
- CIELAB空间是一种颜色模型,由国际照明委员会(Commission Internationale de l'Éclairage,简称CIE)在1976年定义。
- 这个模型是为了创建一个更符合人类视觉感知的颜色空间,它将颜色的感知属性分为三个主要的维度:亮度(Lightness,L*)、色度(Chroma,C*)和色相(Hue,H*)。
- CIELAB空间的特点如下:
- *亮度(L)**:亮度是颜色的明暗程度,它反映了颜色的明度。在CIELAB空间中,亮度的值范围通常是从0(纯黑)到100(纯白)。
- *色度(C)**:色度表示颜色的饱和度,即颜色的纯度。一个高色度值意味着颜色非常饱和,而低色度值则表示颜色接近灰色。色度是一个从中心(灰色)向外辐射的度量,不考虑颜色的具体类型。
- *色相(H)**:色相描述了颜色的种类,它是一个角度值,用来区分不同的颜色。色相的范围通常是0°到360°,对应于颜色轮上的不同颜色,如红色、绿色、蓝色等
- """
- def rgb2lab_diff(rgb_image,device):
- '''
- Function to convert a batch of image tensors from RGB space to CIELAB space.
- parameters: xn, yn, zn are the CIE XYZ tristimulus values of the reference white point.
- Here use the standard Illuminant D65 with normalization Y = 100.
- '''
- rgb_image=rgb_image.to(device)
- res = torch.zeros_like(rgb_image) # 初始化一个与输入RGB图像形状相同的零张量,用于存储转换后的CIELAB图像
- xyz_image = rgb2xyz(rgb_image,device) # 将RGB图像转换为XYZ颜色空间,这是转换到CIELAB的中间步骤
- # 定义参考白点(D65)的XYZ值
- xn = 95.0489
- yn = 100
- zn = 108.8840
- # 从XYZ图像中提取X、Y、Z三个颜色通道
- x = xyz_image[:,0, :, :]
- y = xyz_image[:,1, :, :]
- z = xyz_image[:,2, :, :]
- # 计算亮度(L*)分量,这里使用了CIELAB空间的亮度公式
- L = 116*xyz_lab(y/yn,device) - 16
- # 计算a*分量,表示颜色在绿色到红色轴上的位置
- a = 500*(xyz_lab(x/xn,device) - xyz_lab(y/yn,device))
- # 计算b*分量,表示颜色在蓝色到黄色轴上的位置
- b = 200*(xyz_lab(y/yn,device) - xyz_lab(z/zn,device))
- # 将计算出的L*、a*、b*分量分别赋值给结果张量的对应通道
- res[:, 0, :, :] = L
- res[:, 1, :, :] = a
- res[:, 2, :, :] = b
- # 返回转换后的CIELAB图像
- return res
-
-
-
-
- #计算两张图片的ciede2000_diff
- def ciede2000_diff(lab1, lab2,device):
- '''
- CIEDE2000 metric to claculate the color distance map for a batch of image tensors defined in CIELAB space
-
- '''
-
- lab1=lab1.to(device)
- lab2=lab2.to(device)
-
- L1 = lab1[:,0,:,:]
- A1 = lab1[:,1,:,:]
- B1 = lab1[:,2,:,:]
- L2 = lab2[:,0,:,:]
- A2 = lab2[:,1,:,:]
- B2 = lab2[:,2,:,:]
- kL = 1
- kC = 1
- kH = 1
-
- mask_value_0_input1=((A1==0)*(B1==0)).float()
- mask_value_0_input2=((A2==0)*(B2==0)).float()
- mask_value_0_input1_no=1-mask_value_0_input1
- mask_value_0_input2_no=1-mask_value_0_input2
- B1=B1+0.0001*mask_value_0_input1
- B2=B2+0.0001*mask_value_0_input2
-
- C1 = torch.sqrt((A1 ** 2.) + (B1 ** 2.))
- C2 = torch.sqrt((A2 ** 2.) + (B2 ** 2.))
-
- aC1C2 = (C1 + C2) / 2.
- G = 0.5 * (1. - torch.sqrt((aC1C2 ** 7.) / ((aC1C2 ** 7.) + (25 ** 7.))))
- a1P = (1. + G) * A1
- a2P = (1. + G) * A2
- c1P = torch.sqrt((a1P ** 2.) + (B1 ** 2.))
- c2P = torch.sqrt((a2P ** 2.) + (B2 ** 2.))
-
-
- h1P = hpf_diff(B1, a1P)
- h2P = hpf_diff(B2, a2P)
- h1P=h1P*mask_value_0_input1_no
- h2P=h2P*mask_value_0_input2_no
-
- dLP = L2 - L1
- dCP = c2P - c1P
- dhP = dhpf_diff(C1, C2, h1P, h2P)
- dHP = 2. * torch.sqrt(c1P * c2P) * torch.sin(radians(dhP) / 2.)
- mask_0_no=1-torch.max(mask_value_0_input1,mask_value_0_input2)
- dHP=dHP*mask_0_no
-
- aL = (L1 + L2) / 2.
- aCP = (c1P + c2P) / 2.
- aHP = ahpf_diff(C1, C2, h1P, h2P)
- T = 1. - 0.17 * torch.cos(radians(aHP - 39)) + 0.24 * torch.cos(radians(2. * aHP)) + 0.32 * torch.cos(radians(3. * aHP + 6.)) - 0.2 * torch.cos(radians(4. * aHP - 63.))
- dRO = 30. * torch.exp(-1. * (((aHP - 275.) / 25.) ** 2.))
- rC = torch.sqrt((aCP ** 7.) / ((aCP ** 7.) + (25. ** 7.)))
- sL = 1. + ((0.015 * ((aL - 50.) ** 2.)) / torch.sqrt(20. + ((aL - 50.) ** 2.)))
-
- sC = 1. + 0.045 * aCP
- sH = 1. + 0.015 * aCP * T
- rT = -2. * rC * torch.sin(radians(2. * dRO))
-
- # res_square=((dLP / (sL * kL)) ** 2.) + ((dCP / (sC * kC)) ** 2.) + ((dHP / (sH * kH)) ** 2.) + rT * (dCP / (sC * kC)) * (dHP / (sH * kH))
-
- res_square=((dLP / (sL * kL)) ** 2.) + ((dCP / (sC * kC)) ** 2.)*mask_0_no + ((dHP / (sH * kH)) ** 2.)*mask_0_no + rT * (dCP / (sC * kC)) * (dHP / (sH * kH))*mask_0_no
- mask_0=(res_square<=0).float()
- mask_0_no=1-mask_0
- res_square=res_square+0.0001*mask_0
- res=torch.sqrt(res_square)
- res=res*mask_0_no
-
-
- return res
perc_al
- def quantization(x):
- """quantize the continus image tensors into 255 levels (8 bit encoding)"""
- x_quan=torch.round(x*255)/255
- return x_quan
-
- class PerC_AL:
- """
- PerC_AL: Alternating Loss of Classification and Color Differences to achieve imperceptibile perturbations with few iterations.
- Parameters
- ----------
- max_iterations : int
- Number of iterations for the optimization.
- alpha_l_init: float
- step size for updating perturbations with respect to classification loss
- alpha_c_init: float
- step size for updating perturbations with respect to perceptual color differences
- confidence : float, optional
- Confidence of the adversary for Carlini's loss, in term of distance between logits.
- Note that this approach only supports confidence setting in an untargeted case
- device : torch.device, optional
- Device on which to perform the adversary.
- """
-
- def __init__(self,
- max_iterations: int = 1000,
- alpha_l_init: float = 1.,
- #for relatively easy untargeted case, alpha_c_init is adjusted to a smaller value (e.g., 0.1 is used in the paper)
- alpha_c_init: float = 0.5,
- confidence: float = 0,
- device: torch.device = torch.device('cpu')
- ) -> None:
- self.max_iterations = max_iterations #1000
- self.alpha_l_init = alpha_l_init #1
- self.alpha_c_init = alpha_c_init #0.5
- self.confidence = confidence #40
- self.device = device
- #model:Inception v3 target=False inputs:干净样本=[B,3,299,299] labels:这些样本的标签 targeted=False
- def adversary(self, model: nn.Module, inputs: torch.Tensor, labels: torch.Tensor,
- targeted: bool = True) -> torch.Tensor:
- """
- Performs the adversary of the model given the inputs and labels.
- Parameters
- ----------
- model : nn.Module
- Model to fool.
- inputs : torch.Tensor
- Batch of image examples in the range of [0,1].
- labels : torch.Tensor
- Original labels if untargeted, else labels of targets.
- targeted : bool, optional
- Whether to perform a targeted adversary or not.
- Returns
- -------
- torch.Tensor
- Batch of image samples modified to be adversarial
- """
-
- if inputs.min() < 0 or inputs.max() > 1: raise ValueError('Input values should be in the [0, 1] range.')
-
- alpha_l_min=self.alpha_l_init/100 #alpha_l_min=0.01
- alpha_c_min=self.alpha_c_init/10 #alpha_c_min=0.05
- multiplier = -1 if targeted else 1 #multiplier=1
-
- X_adv_round_best=inputs.clone()
- inputs_LAB=rgb2lab_diff(inputs,self.device) #获取干净RGB图片对应的CIELAB图片
- batch_size=inputs.shape[0]
- delta=torch.zeros_like(inputs, requires_grad=True)
- mask_isadv= torch.zeros(batch_size,dtype=torch.uint8).to(self.device) #[B]全0的tensor
- color_l2_delta_bound_best=(torch.ones(batch_size)*100000).to(self.device) #[B]全100000的tensor
-
- if (targeted==False) and self.confidence!=0:
- labels_onehot = torch.zeros(labels.size(0), 1000, device=self.device) #[B,1000]全0tensor
- labels_onehot.scatter_(1, labels.unsqueeze(1), 1)
- labels_infhot = torch.zeros_like(labels_onehot).scatter_(1, labels.unsqueeze(1), float('inf')) #[B,1000]全0tensor
- if (targeted==True) and self.confidence!=0:
- print('Only support setting confidence in untargeted case!')
- return
- for i in range(self.max_iterations): #1到1000
- # cosine annealing for alpha_l_init and alpha_c_init max_iterations=1000,pi=Π
- alpha_c=alpha_c_min+0.5*(self.alpha_c_init-alpha_c_min)*(1+cos(i/self.max_iterations*pi)) #alpha_c=0.5 pixel-wise perceptual color distance 每次迭代的步长
- alpha_l=alpha_l_min+0.5*(self.alpha_l_init-alpha_l_min)*(1+cos(i/self.max_iterations*pi)) #alpha_l=1.0 对应L2攻击每次迭代的步长
- loss = multiplier * nn.CrossEntropyLoss(reduction='sum')(model((inputs+ delta-0.5)/0.5), labels) #loss黑盒对抗损失
- loss.backward()
- grad_a=delta.grad.clone() #grad_a=[B,3,299,299] loss对 对抗样本 的梯度
- delta.grad.zero_() #~mask_isadv默认=全1,图片没有部分被掩盖 ,delta.data清空=全0的[B,3,299,299]的tensor permute(1,2,3,0)使得B到最后 permute(3,0,1,2)又把B调回第一
- delta.data[~mask_isadv]=delta.data[~mask_isadv]+alpha_l*(grad_a.permute(1,2,3,0)/torch.norm(grad_a.view(batch_size,-1),dim=1)).permute(3,0,1,2)[~mask_isadv]
- d_map=ciede2000_diff(inputs_LAB,rgb2lab_diff(inputs+delta,self.device),self.device).unsqueeze(1) #干净样本的CIELAB图片 和对抗样本的CIELAB图片 之间的pixel-wise perceptual color distance
- color_dis=torch.norm(d_map.view(batch_size,-1),dim=1) #color_dis=[5]的tensor
- color_loss=color_dis.sum()
- color_loss.backward()
-
- grad_color=delta.grad.clone()
- delta.grad.zero_()
- delta.data[mask_isadv]=delta.data[mask_isadv]-alpha_c* (grad_color.permute(1,2,3,0)/torch.norm(grad_color.view(batch_size,-1),dim=1)).permute(3,0,1,2)[mask_isadv]
-
- delta.data=(inputs+delta.data).clamp(0,1)-inputs
- X_adv_round=quantization(inputs+delta.data) #X_adv_round=torch.round((inputs+delta)*255)/255
- #C&W评估样本对抗性
- if (targeted==False) and self.confidence!=0: #targeted=False,confidence=40
- logits = model((X_adv_round-0.5)/0.5) #logits=[B,1000],进入softmax之前的评分矩阵
- real = logits.gather(1, labels.unsqueeze(1)).squeeze(1) #ft(x')
- other = (logits - labels_infhot).max(1)[0] #maxfi≠t(x')
- print(f'第{i}轮:置信度{real - other}')
- mask_isadv=(real - other)<=-40 #ft(x')-maxfi≠t(x')<=-40 才算攻击成功,给该部分mask上
- elif self.confidence==0:
- if targeted:
- mask_isadv=torch.argmax(model((X_adv_round-0.5)/0.5),dim=1)==labels
- else:
- mask_isadv=torch.argmax(model((X_adv_round-0.5)/0.5),dim=1)!=labels
- mask_best=(color_dis.data<color_l2_delta_bound_best)
- mask=mask_best * mask_isadv
- color_l2_delta_bound_best[mask]=color_dis.data[mask]
- X_adv_round_best[mask]=X_adv_round[mask]
-
- return X_adv_round_best
复现的隐蔽性效果
几乎没有太大区别
Copyright © 2003-2013 www.wpsshop.cn 版权所有,并保留所有权利。