# 11-循环神经网络

## 回顾 DNN

![image-20210319095253370](https://cdn.jsdelivr.net/gh/TheFoxFairy/ImgStg/202201070221542.png)

稠密网络（全连接）/深度神经网络。

## 什么是RNN

RNN可以处理序列关系的数据。

> 序列关系数据：自然语言，天气预测……

![image-20210319102149594](https://cdn.jsdelivr.net/gh/TheFoxFairy/ImgStg/202201070221543.png)

`RNN Cell`的本质是一个线性层，和以往的线性层，区别是线性层是共享的。其中$h\_0$为先验，如果没有数据输入，则随机初始化，经过一次`RNN Cell`得到$h\_1$，在下一次的时候，输出的$h\_1$又会被当成输入，进行下一次循环。

**伪代码：**

```
linear = Linear()

h = 0
for x in X:
	h = linear(x,h)  => h1 = linear(x1,h0)
	                 => h2 = linear(x2,h1)
	                 => ...
```

![image-20210319103432450](https://cdn.jsdelivr.net/gh/TheFoxFairy/ImgStg/202201070221544.png)

## RNN in Pytorch

### 调用RNN Cell

#### 定义

```
cell = torch.nn.RNNCell(input_size=input_size,hidden_size=hidden_size)

hidden = cell(input,hidden)
```

在`hiddent = cell(input,hidden)`中，输入的形式为：

* `input：input of shape(batch,input_size)`
* `hidden：hidden of shape(batch,hidden_size)`

输出的形式为：

* `hidden：hidden of shape(batch,hidden_size)`

![image-20210319105327932](https://cdn.jsdelivr.net/gh/TheFoxFairy/ImgStg/202201070221545.png)

#### 如何使用RNN Cell

假设我们有具有以下性质的序列：

* batchSize = 1，即N=1
* seqLen = 3，即$x\_1$，$x\_2$，$x\_3$
* inputSize = 4，即为四维的向量
* hiddenSize = 2，即隐藏层设置为2维

所以，RNNCell的输入输出形状如下：

* **input**.shape = (batchSize,inputSize)
* **output**.shape = (batchSize,hiddenSize)

整个序列，构造如下：

* **dataset**.shape = (seqLen,batchSize,inputSize)

```
import torch

batch_size = 1
seq_len = 3
input_size = 4
hidden_size = 2

cell = torch.nn.RNNCell(input_size=input_size,hidden_size=hidden_size)

dataset = torch.randn(seq_len,batch_size,input_size)
hidden = torch.zeros(batch_size,hidden_size)

for idx,input in enumerate(dataset):
    print('='*20,idx,'='*20)
    print('Inputs Size:',input.shape)
    
    hidden = cell(input,hidden)
    
    print('Outputs Size:',hidden.shape)
    print(hidden)
```

> 注意，cell是公用的，$h\_0$是先验的或者全0

#### 如何使用 RNN

```
cell = torch.nn.RNN(input_size=input_size,hidden_size=hidden_size,num_layers=num_layers)

out,hidden = cell(inputs,hidden)
```

其中`num_layers`表示层数，注意层数选择，RNN计算十分耗时的；inputs包括整个的输入序列。

![image-20210319123206758](https://cdn.jsdelivr.net/gh/TheFoxFairy/ImgStg/202201070221546.png)

![image-20210319123638022](https://cdn.jsdelivr.net/gh/TheFoxFairy/ImgStg/202201070221547.png)

![image-20210319123741294](https://cdn.jsdelivr.net/gh/TheFoxFairy/ImgStg/202201070221548.png)

![image-20210319123801494](https://cdn.jsdelivr.net/gh/TheFoxFairy/ImgStg/202201070221549.png)

```
import torch

batch_size = 1
seq_len = 3
input_size = 4
hidden_size = 2
num_layers = 1

cell = torch.nn.RNN(input_size=input_size,
                   hidden_size=hidden_size,
                   num_layers=num_layers)

inputs = torch.randn(seq_len,batch_size,input_size)
hidden = torch.zeros(num_layers,batch_size,hidden_size)

out,hidden = cell(inputs,hidden)

print("Output Size:",out.shape)
print("Output:",out)
print("Hidden Size:",hidden.shape)
print("Hidden:",hidden)
```

![image-20210319130242600](https://cdn.jsdelivr.net/gh/TheFoxFairy/ImgStg/202201070221550.png)

#### 实例

* 训练模型去学习：`hello` $\rightarrow$ `ohlol`

![image-20210319130852666](https://cdn.jsdelivr.net/gh/TheFoxFairy/ImgStg/202201070221551.png)

*

### 自定义RNN Cell