torch.nn.BCEWithLogitsLoss用法介绍

self.bce = nn.BCEWithLogitsLoss(reduction='none'), None的使用方法可以见官网pytorch代码文档

代码举例

import torch
a = torch.rand((1, 3, 3))
target = torch.tensor([[[1, 0, 0],
                        [0, 1, 0],
                        [0, 0, 0]]])
print(a)
'''
ouput:tensor([[[0.2070, 0.8432, 0.2494],
         [0.5782, 0.4587, 0.1135],
         [0.9794, 0.8516, 0.4418]]])
'''
b = torch.nn.BCEWithLogitsLoss(reduction='none')(a, target.to(a.dtype))
print(b)
'''
output: tensor([[[0.5950, 1.2011, 0.8256],
         [1.0234, 0.4899, 0.7515],
         [1.2982, 1.2070, 0.9383]]])
'''
c = torch.nn.BCEWithLogitsLoss(reduction='mean')(a, target.to(a.dtype))
print(c)
'''
output:tensor(0.9256)
'''
d = torch.nn.BCEWithLogitsLoss(reduction='sum')(a, target.to(a.dtype))
print(d)
'''
output:tensor(8.3301)
'''
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

举例的代码中target中的计算方法是这样的(resuction='none')

对于target[0][0][0]=1=y_n, a[0][0][0]=0.2070=x_n, 因此，对于l_n

$$0.5950=-\left(1\times \ln \sigma \left(0.2070\right)+0\times \ln \left(1-\sigma \left(0.2070\right)\right)\right)$$
对于target[0][0][1]=0=y_n, a[0][0][1]=0.8432=x_n, 因此，对于l_n
$$1.2011=-\left(0\times \ln \sigma \left(0.8432\right)+1\times \ln \left(1-\sigma \left(0.8432\right)\right)\right)$$

其中$\sigma$是Sigmoid函数, 以此类推, 可以最终得到output: tensor([[[0.5950, 1.2011, 0.8256],[1.0234, 0.4899, 0.7515],[1.2982, 1.2070, 0.9383]]]), 如果reduction=‘mean’的时候, 就是reduction=‘none’的output求平均值, 最终可以得到output:tensor(0.9256)，如果reduction='sum'，那就是reduction=‘none’的output求和，最终可以得到output:tensor(8.3301)

可以验证一下上面的代码，可以看到结果是一样的

a = torch.tensor([[[0.2070, 0.8432, 0.2494],
                   [0.5782, 0.4587, 0.1135],
                   [0.9794, 0.8516, 0.4418]]])
target = torch.tensor([[[1, 0, 0],
                        [0, 1, 0],
                        [0, 0, 0]]])
sa = torch.nn.Sigmoid()(a)
result = -(target * torch.log(sa) + (1 - target) * torch.log(1 - sa))
'''
output: tensor([[[0.5950, 1.2011, 0.8256],
         [1.0235, 0.4899, 0.7515],
         [1.2982, 1.2070, 0.9382]]])
'''
result_mean = result.mean()
'''
output: tensor(0.9256)
'''
result_sum = result.sum()
'''
output: tensor(8.3300)
'''
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

如果是bs = 2，reduction='mean'和reduction='sum'的效果也是一样的，最终的结果也是一个值。代码如下, 可以看到均值不变, 求和的时候扩大2倍

import torch
# a = torch.rand((1, 3, 3))
a = torch.tensor([[[0.2070, 0.8432, 0.2494],
         [0.5782, 0.4587, 0.1135],
         [0.9794, 0.8516, 0.4418]], 
         [[0.2070, 0.8432, 0.2494],
         [0.5782, 0.4587, 0.1135],
         [0.9794, 0.8516, 0.4418]]])
target = torch.tensor([[[1, 0, 0],
                        [0, 1, 0],
                        [0, 0, 0]],
                        [[1, 0, 0],
                        [0, 1, 0],
                        [0, 0, 0]]])

b = torch.nn.BCEWithLogitsLoss(reduction='none')(a, target.to(a.dtype))
print(b)
'''
output: tensor([[[0.5950, 1.2011, 0.8256],
                 [1.0235, 0.4899, 0.7515],
                 [1.2982, 1.2070, 0.9382]],

		        [[0.5950, 1.2011, 0.8256],
		         [1.0235, 0.4899, 0.7515],
		         [1.2982, 1.2070, 0.9382]]])
'''

c = torch.nn.BCEWithLogitsLoss(reduction='mean')(a, target.to(a.dtype))
print(c)
'''
output: tensor(0.9256)
'''

d = torch.nn.BCEWithLogitsLoss(reduction='sum')(a, target.to(a.dtype))
print(d)
'''
output: tensor(16.6601)
'''

sa = torch.nn.Sigmoid()(a)
result = -(target * torch.log(sa) + (1 - target) * torch.log(1 - sa))
print(result)
'''
output: tensor([[[0.5950, 1.2011, 0.8256],
		         [1.0235, 0.4899, 0.7515],
		         [1.2982, 1.2070, 0.9382]],

		        [[0.5950, 1.2011, 0.8256],
		         [1.0235, 0.4899, 0.7515],
		         [1.2982, 1.2070, 0.9382]]])
'''

result_mean = result.mean()
'''
output: tensor(0.9256)
'''
result_sum = result.sum()
'''
output: tensor(16.6601)
'''
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60