python实现keras多模态（语音文字图像）情绪识别【文末源码】_多模态情感识别开源代码 情感等级

1、注：源码放置文末

1.1 环境配置要求：

https://blog.csdn.net/qq_42279468/article/details/124987801

2、展示效果：

https://www.bilibili.com/video/BV1bh4y1n7bw/
1

3、代码

  本项目通过python实现多模态情绪识别，使用keras框架搭建网络，包括语音、文字和图像三种处理后的数据。
  算法使用LayerNormBasicLSTMCell+注意力机制构建网络
1
2

3.1 数据集展示

3.2 训练过程

Training epoch 1
3it [00:03,  1.58s/it]	 	Epoch 1:, loss 0.868626, accuracy 0.467928
4it [00:03,  1.03it/s]

0it [00:00, ?it/s]Training epoch 2
3it [00:01,  2.13it/s]	 	Epoch 2:, loss 0.688236, accuracy 0.584177
4it [00:01,  2.33it/s]

0it [00:00, ?it/s]Training epoch 3
4it [00:01,  2.44it/s]
	 	Epoch 3:, loss 0.623478, accuracy 0.65178

0it [00:00, ?it/s]Training epoch 4
3it [00:01,  2.10it/s]	 	Epoch 4:, loss 0.472371, accuracy 0.801504
4it [00:01,  2.40it/s]

Training epoch 5
3it [00:01,  2.03it/s]	 	Epoch 5:, loss 0.429814, accuracy 0.834892
4it [00:01,  2.43it/s]

0it [00:00, ?it/s]Training epoch 6
3it [00:01,  2.09it/s]	 	Epoch 6:, loss 0.400317, accuracy 0.847018
4it [00:01,  2.44it/s]

0it [00:00, ?it/s]Training epoch 7
4it [00:01,  2.36it/s]
	 	Epoch 7:, loss 0.340432, accuracy 0.853922
0it [00:00, ?it/s]
Training epoch 8
4it [00:01,  2.24it/s]
	 	Epoch 8:, loss 0.304842, accuracy 0.884431
0it [00:00, ?it/s]
Training epoch 9
3it [00:01,  2.03it/s]	 	Epoch 9:, loss 0.275721, accuracy 0.908767
4it [00:01,  2.44it/s]

0it [00:00, ?it/s]Training epoch 10
4it [00:01,  2.54it/s]
	 	Epoch 10:, loss 0.242068, accuracy 0.931539

0it [00:00, ?it/s]Training epoch 11
3it [00:01,  2.03it/s]	 	Epoch 11:, loss 0.246808, accuracy 0.910064
4it [00:01,  2.36it/s]
0it [00:00, ?it/s]
Training epoch 12
4it [00:01,  2.29it/s]
	 	Epoch 12:, loss 0.206893, accuracy 0.939517

0it [00:00, ?it/s]Training epoch 13
4it [00:01,  2.57it/s]
	 	Epoch 13:, loss 0.185851, accuracy 0.944807
0it [00:00, ?it/s]
Training epoch 14
4it [00:01,  2.29it/s]
	 	Epoch 14:, loss 0.157054, accuracy 0.95869

0it [00:00, ?it/s]Training epoch 15
4it [00:01,  2.42it/s]
	 	Epoch 15:, loss 0.166478, accuracy 0.952298
0it [00:00, ?it/s]
Training epoch 16
3it [00:01,  2.09it/s]	 	Epoch 16:, loss 0.149369, accuracy 0.971128
4it [00:01,  2.35it/s]

0it [00:00, ?it/s]Training epoch 17
4it [00:01,  2.31it/s]
	 	Epoch 17:, loss 0.124102, accuracy 0.975202
0it [00:00, ?it/s]
Training epoch 18
4it [00:01,  2.31it/s]
	 	Epoch 18:, loss 0.127283, accuracy 0.965784

0it [00:00, ?it/s]Training epoch 19
4it [00:01,  2.14it/s]
	 	Epoch 19:, loss 0.114086, accuracy 0.972137

0it [00:00, ?it/s]Training epoch 20
3it [00:01,  1.97it/s]	 	Epoch 20:, loss 0.121704, accuracy 0.973938
4it [00:01,  2.22it/s]

0it [00:00, ?it/s]Training epoch 21
3it [00:01,  2.10it/s]	 	Epoch 21:, loss 0.103751, accuracy 0.969179
4it [00:01,  2.31it/s]
0it [00:00, ?it/s]
Training epoch 22
3it [00:01,  2.17it/s]	 	Epoch 22:, loss 0.114447, accuracy 0.968958
4it [00:01,  2.31it/s]

0it [00:00, ?it/s]Training epoch 23
3it [00:01,  2.16it/s]	 	Epoch 23:, loss 0.0959018, accuracy 0.978891
4it [00:01,  2.34it/s]

0it [00:00, ?it/s]Training epoch 24
4it [00:01,  2.59it/s]
	 	Epoch 24:, loss 0.0853932, accuracy 0.986028

0it [00:00, ?it/s]Training epoch 25
4it [00:01,  2.44it/s]
	 	Epoch 25:, loss 0.0839167, accuracy 0.987315

0it [00:00, ?it/s]Training epoch 26
4it [00:01,  2.42it/s]
	 	Epoch 26:, loss 0.0777089, accuracy 0.986904

Training epoch 27
3it [00:01,  1.93it/s]	 	Epoch 27:, loss 0.074715, accuracy 0.987368
4it [00:01,  2.21it/s]

0it [00:00, ?it/s]Training epoch 28
3it [00:01,  2.09it/s]	 	Epoch 28:, loss 0.0690631, accuracy 0.987052
4it [00:01,  2.41it/s]

0it [00:00, ?it/s]Training epoch 29
4it [00:01,  2.39it/s]
	 	Epoch 29:, loss 0.0754572, accuracy 0.986618

0it [00:00, ?it/s]Training epoch 30
4it [00:01,  2.40it/s]
	 	Epoch 30:, loss 0.0808934, accuracy 0.978521

0it [00:00, ?it/s]Training epoch 31
3it [00:01,  2.03it/s]	 	Epoch 31:, loss 0.0768318, accuracy 0.977635
4it [00:01,  2.35it/s]
0it [00:00, ?it/s]
Training epoch 32
3it [00:01,  2.10it/s]	 	Epoch 32:, loss 0.0907751, accuracy 0.97986
4it [00:01,  2.31it/s]

Training epoch 33
3it [00:01,  2.10it/s]	 	Epoch 33:, loss 0.0650676, accuracy 0.984582
4it [00:01,  2.30it/s]

0it [00:00, ?it/s]Training epoch 34
4it [00:01,  2.32it/s]
	 	Epoch 34:, loss 0.084893, accuracy 0.9796

Training epoch 35
3it [00:01,  2.26it/s]	 	Epoch 35:, loss 0.0657072, accuracy 0.97979
4it [00:01,  2.30it/s]

0it [00:00, ?it/s]Training epoch 36
3it [00:01,  2.25it/s]	 	Epoch 36:, loss 0.0571314, accuracy 0.989951
4it [00:01,  2.51it/s]

0it [00:00, ?it/s]Training epoch 37
4it [00:01,  2.17it/s]
	 	Epoch 37:, loss 0.0639608, accuracy 0.984723
0it [00:00, ?it/s]
Training epoch 38
3it [00:01,  2.01it/s]	 	Epoch 38:, loss 0.0603083, accuracy 0.987819
4it [00:01,  2.13it/s]
0it [00:00, ?it/s]
Training epoch 39
3it [00:01,  2.08it/s]	 	Epoch 39:, loss 0.0597859, accuracy 0.987659
4it [00:01,  2.35it/s]
0it [00:00, ?it/s]
Training epoch 40
4it [00:01,  2.29it/s]
	 	Epoch 40:, loss 0.0657729, accuracy 0.987306
0it [00:00, ?it/s]
Training epoch 41
3it [00:01,  1.99it/s]	 	Epoch 41:, loss 0.0577475, accuracy 0.989388
4it [00:01,  2.38it/s]

0it [00:00, ?it/s]Training epoch 42
3it [00:01,  1.97it/s]	 	Epoch 42:, loss 0.0561785, accuracy 0.985871
4it [00:01,  2.26it/s]

0it [00:00, ?it/s]Training epoch 43
4it [00:01,  2.48it/s]
	 	Epoch 43:, loss 0.0551007, accuracy 0.987269

0it [00:00, ?it/s]Training epoch 44
3it [00:01,  1.92it/s]	 	Epoch 44:, loss 0.0516939, accuracy 0.986137
4it [00:01,  2.24it/s]

Training epoch 45
4it [00:01,  2.34it/s]
	 	Epoch 45:, loss 0.0713419, accuracy 0.978741
0it [00:00, ?it/s]
Training epoch 46
4it [00:01,  2.31it/s]
	 	Epoch 46:, loss 0.078447, accuracy 0.978097
0it [00:00, ?it/s]
Training epoch 47
4it [00:01,  2.20it/s]	 	Epoch 47:, loss 0.0496744, accuracy 0.987609
4it [00:01,  2.23it/s]

0it [00:00, ?it/s]Training epoch 48
4it [00:01,  2.41it/s]
	 	Epoch 48:, loss 0.054719, accuracy 0.98776
0it [00:00, ?it/s]
Training epoch 49
4it [00:01,  2.30it/s]
	 	Epoch 49:, loss 0.0739138, accuracy 0.978141

0it [00:00, ?it/s]Training epoch 50
3it [00:01,  2.07it/s]	 	Epoch 50:, loss 0.0537297, accuracy 0.987054
4it [00:01,  2.12it/s]

0it [00:00, ?it/s]Training epoch 51
4it [00:01,  2.29it/s]
	 	Epoch 51:, loss 0.0486002, accuracy 0.98399

0it [00:00, ?it/s]Training epoch 52
4it [00:01,  2.31it/s]
	 	Epoch 52:, loss 0.0549857, accuracy 0.986394

0it [00:00, ?it/s]Training epoch 53
4it [00:01,  2.21it/s]
	 	Epoch 53:, loss 0.0684459, accuracy 0.982455

0it [00:00, ?it/s]Training epoch 54
3it [00:01,  2.20it/s]	 	Epoch 54:, loss 0.0616392, accuracy 0.981636
4it [00:01,  2.41it/s]
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215

3.3 模型评估

4 源码下载

https://gitcode.net/qq_42279468/python-muti-fuse.git
1