李宏毅机器学习（7）_given network structure, define a function set

Brief introduction for Deep Learning

深度学习3步骤

1. 设定神经网络（define a set of function）
2. 设定Loss函数（goodness of function）
3. 选出最好的函数（pick the best function）

Network parameter $\theta$：all the weights and biases in the neurons

Given network structure,define a function set（还没定参数）
Deep$=$Many hidden layers

Matrix Operation:

$y=f(x)=\sigma (w_L...\sigma (w_2\sigma (w_1\cdot x+b_1)+b_2)...+b_L)$
将Neural Network 用矩阵乘法表示，所以就可以用GPU优化加速。

How many layers?How many neurons for each layer?

• Trial and Error + Intuition 经验+直觉

Can the structure be automatically determined?

• 可以，但技术不成熟，尚未普及

Can we design the network structure?

• 可以，如：Convolutional Neural Network

Loss for an Example

用 Cross entropy：$C(y,\hat{y})=-{\sum }_{i=1}^{10}{\hat{y}}_{i}ln{y}_i,L={\sum }_{n=1}^N{C}_n$
Find a fucntion is function set that minimize total loss $L$
Find the network parameters ${\theta }^{\ast }$ that minimize total loss $L$
方法：还是Gradient Descent
Backpropagation算法：an efficient way to compute $\frac{\alpha L}{\alpha w}$ in neural network
就是使用工具包
Universality Theorem
Any continuous function f
$f:R^N\to R^M$
Can be realized by a network with one hidden layer(given enough hidden neurons)