- 1【Android - V】之DrawerLayout的使用_drawerlayout的使用v4包
- 2数据结构:常见的排序算法(一):冒泡排序(C++实现)_冒泡排序c++
- 3STM32踩坑笔记_t' is defined pubweak in a section implicitly decl
- 4CUDA编程 - Nsight system & Nsight compute 的安装和使用 - 学习记录_cuda nsight
- 5【教你搭建服务器系列】（9）让你的服务器更安全_sudo: firewall-cmd: command not found
- 6在嵌入式开发中如何提高自己的代码水平_嵌入式软件工程师怎么提高自己的代码水平
- 7机器学习系列文章：人工神经网络典型模型（感知器（Perceptron）模型、反向传播（BP）模型、反馈网络（Hopfield）模型）_人工智能 感知机 反向传播
- 8鸿蒙手机兼容安卓,鸿蒙系统能不能兼容安卓应用呢?兼容性怎么样?
- 9Nacos修改哪些配置后需要重启_nacos修改配置后需要重启服务吗
- 10回顾C语言006--整型变量的位带（bit-band）操作_c语言bit变量
【torch激活函数集锦】_torch.sigmoid(x)
赞
踩
【torch激活函数集锦】
- s i g m o i d ( x ) = 1 1 + e − x sigmoid(x) = \frac{1}{1+e^{-x}} sigmoid(x)=1+e−x1
import torch as t
import torch.nn as nn
import torch.nn.functional as F
import matplotlib.pyplot as plt
x= t.linspace(-6,6,1000)
f = nn.Sigmoid()
plt.plot(x,f(x))
plt.show()
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
2.
t
a
n
h
(
x
)
=
e
x
−
e
−
x
e
x
+
e
−
x
tanh(x) = \frac{e^x-e^{-x}}{e^x+e^{-x}}
tanh(x)=ex+e−xex−e−x
f = nn.Tanh()
plt.plot(x, f(x))
plt.show()
- 1
- 2
- 3
3.
R
e
L
u
(
x
)
=
m
a
x
(
0
,
x
)
ReLu(x) = max(0,x)
ReLu(x)=max(0,x)
缺点：神经元坏死现象，某些神经元可能永远不会被激活
f = nn.ReLU()
plt.plot(x,f(x))
plt.show()
- 1
- 2
- 3
4.
L
e
a
k
y
R
e
L
U
(
x
)
=
m
a
x
(
0
,
x
)
+
n
e
g
a
t
i
v
e
_
s
l
o
p
e
∗
m
i
n
(
0
,
x
)
LeakyReLU(x) = max(0,x) + negative\_slope * min(0,x)
LeakyReLU(x)=max(0,x)+negative_slope∗min(0,x)
negative_slope为负的斜率常数，默认为-0.01
f = t.nn.LeakyReLU(negative_slope=0.1)
plt.plot(x,f(x))
plt.show()
- 1
- 2
- 3
5.
P
R
e
L
U
(
x
)
=
m
a
x
(
0
,
x
)
+
a
∗
m
i
n
(
0
,
x
)
PReLU(x)=max(0,x)+a*min(0,x)
PReLU(x)=max(0,x)+a∗min(0,x)
负斜率a可学习
f = t.nn.PReLU(num_parameters= 1, init= 0.25)
plt.plot(x,f(x).detach())
plt.show()
- 1
- 2
- 3
6.
E
L
U
(
x
)
=
m
a
x
(
0
,
x
)
+
m
i
n
(
0
,
α
(
e
x
−
1
)
)
ELU(x) = max(0,x) + min(0, \alpha(e^{x}-1))
ELU(x)=max(0,x)+min(0,α(ex−1))
f = t.nn.ELU(alpha=0.5)
plt.plot(x,f(x))
plt.show()
- 1
- 2
- 3
7.
S
E
L
U
(
x
)
=
s
c
a
l
e
(
m
a
x
(
0
,
x
)
)
+
m
i
n
(
0
,
α
(
e
x
/
α
−
1
)
)
SELU(x) = scale(max(0,x))+ min(0,\alpha(e^{x/\alpha}-1))
SELU(x)=scale(max(0,x))+min(0,α(ex/α−1))
f = t.nn.SELU()
plt.plot(x,f(x))
plt.show()
- 1
- 2
- 3
8.
T
a
n
h
s
h
r
i
n
k
(
x
)
=
x
−
T
a
n
h
(
x
)
Tanhshrink(x)=x−Tanh(x)
Tanhshrink(x)=x−Tanh(x)
f = t.nn.Tanhshrink()
plt.plot(x,f(x))
plt.show()
- 1
- 2
- 3
9.
S
o
f
t
p
l
u
s
(
x
)
=
1
β
l
o
g
(
1
+
e
β
x
)
Softplus(x)= \frac{1}{\beta}log(1+e^{\beta x})
Softplus(x)=β1log(1+eβx)
ReLU 的变体，较为光滑
f = t.nn.Softplus(beta=1.)
plt.plot(x,f(x))
x2 = t.linspace(0,6,500)
plt.plot(x2,x2,ls='--',c='k')
plt.plot(-x2, t.zeros_like(x2), ls='--', c='k')
plt.show()
- 1
- 2
- 3
- 4
- 5
- 6
10.
s
o
f
t
s
i
g
n
(
x
)
=
x
1
+
∣
x
∣
softsign(x) = \frac{x}{1+|x|}
softsign(x)=1+∣x∣x
plt.plot(x,F.softsign(x))
plt.show()
- 1
- 2
