于大模型迁移中学习 Docker_session stopped - press to exit tab - pre

作者：小小林熬夜学编程 | 2024-03-09 08:27:36
踩
session stopped - press to exit tab - press r to restart session -
最近在做大模型的昇腾迁移，国产化框架踩坑不少，基本一天的工作量相当于之前做纯视觉算法时一周踩过的坑数了。
现在在modelarts上用八卡昇腾910跑llama，不同于之前自己配环境，昇腾生态创新中心都是用的镜像，虽说打包起来不用自己配的那么麻烦，但一键启动之后各种找不到包、调用函数出错的问题搞得我头大，真的每跑一次大模型都要找一个相匹配的镜像才可以吗？
Session stopped
    - Press <Return> to exit tab
    - Press R to restart session
    - Press S to save terminal output to file
root@168.170.140.104's password:
Access denied
root@168.170.140.104's password:
Access denied
root@168.170.140.104's password:
    ┌──────────────────────────────────────────────────────────────────────┐
    │                 • MobaXterm Personal Edition v23.2 •                 │
    │               (SSH client, X server and network tools)               │
    │                                                                      │
    │ ⮞ SSH session to root@168.170.140.104                                │
    │   • Direct SSH      :  ✓                                             │
    │   • SSH compression :  ✓                                             │
    │   • SSH-browser     :  ✓                                             │
    │   • X11-forwarding  :  ✓  (remote display is forwarded through SSH)  │
    │                                                                      │
    │ ⮞ For more info, ctrl+click on help or visit our website.            │
    └──────────────────────────────────────────────────────────────────────┘

Welcome to Ubuntu 18.04.1 LTS (GNU/Linux 4.15.0-29-generic aarch64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  System information as of Fri Jul 21 05:39:31 EDT 2023


Last login: Fri Jul 21 05:39:27 2023 from 10.10.10.42
-bash: /usr/local/Ascend/ascend-toolkit/set_env.sh: No such file or directory
root@ubuntu:~# docker images
REPOSITORY                                                                  TAG                                                                                                                               IMAGE ID       CREATED        SI                                                                              ZE
swr.cn-central-231.xckpjs.com/yangxi/chatglm_finetune                       v3                                                                                                                                ce059a3eb951   5 weeks ago    18                                                                              .3GB
swr.cn-north-309.mtgascendic.cn/bj-aicc/chatglm_finetune                    v3                                                                                                                                ce059a3eb951   5 weeks ago    18                                                                              .3GB
ascendhub.huawei.com/public-ascendhub/pytorch-modelzoo                      23.0                                                                              .RC1-1.11.0                                     1d3158646e1f   2 months ago   11                                                                              GB
swr.cn-north-309.mtgascendic.cn/bj-aicc/pytorch_1_11_0                      pyto                                                                              rch_1.11.0-cann_6.0.1-python_3.8-euler_2.8-v2   a5b48c40f013   2 months ago   13                                                                              .8GB
swr.cn-northwest-229.yantachaosuanzhongxin.com/yantapublic/pytorch_1_11_0   pyto                                                                              rch_1.11.0-cann_6.0.1-python_3.8-euler_2.8-v2   a5b48c40f013   2 months ago   13                                                                              .8GB
guoneng_paddle                                                              v1                                                                                                                                264ea1b0d9c6   3 months ago   43                                                                              .8GB
root@ubuntu:~# docker pull swr.cn-central-221.ovaijisuan.com/mindformers/mindformers_dev_mindspore_1_10_1:mindformers_0.6.0dev_20230615_py39
mindformers_0.6.0dev_20230615_py39: Pulling from mindformers/mindformers_dev_mindspore_1_10_1
6df89f31df6b: Already exists
ed1971c1ae79: Already exists
3326c9a29aaf: Already exists
1dc8445cb1fe: Already exists
b88914ee22d4: Already exists
bcdb23ea4034: Already exists
7d3d47c0e6f5: Already exists
5d7b3572edf7: Already exists
c41695c50bba: Already exists
957b9a31e2a7: Already exists
1672b9d56d5b: Already exists
36e34e344951: Already exists
160f6265f56d: Already exists
c57c0bc5f46f: Already exists
2bda0d153e53: Already exists
9c51c5251e18: Already exists
ab932c597460: Already exists
35490305253a: Already exists
4796376a3c72: Already exists
b2ceab4e4b50: Already exists
5996ae12a711: Already exists
fe6448692e33: Pulling fs layer
ba8a93f762d8: Pulling fs layer
ba74ada194a5: Pulling fs layer
a52d93ccc3fe: Waiting
801949047265: Waiting
b057aae08ec4: Waiting
dc70880ef9ce: Waiting
26c307ef6dfd: Waiting
c94bf8f93adf: Waiting
775eda4359be: Waiting
7cc9392229c8: Pulling fs layer
06ec42708026: Waiting
b01566b68dfa: Waiting
5857e4b48466: Waiting
f99e1c6d3442: Waiting
7d92f226dfff: Waiting
03f18c2615dd: Waiting
86a4a68587b8: Waiting
477735d7e341: Waiting
e3a369c4494f: Waiting
ad8940b6f8df: Waiting
93f2a596f5df: Waiting
bb4d004c2c19: Waiting
0da308e252fc: Pulling fs layer
4a62494058f8: Pull complete
dc1cb36895af: Pull complete
ca2c9a6dd846: Pull complete
8d67ef0f3461: Pull complete
414f0f3a818a: Pull complete
9cc92ebceb93: Pull complete
6d049032fea1: Pull complete
f2a11686cbf5: Pull complete
8dfd91dcabf2: Pull complete
bdf89cff68ba: Pull complete
fb7e8b85b671: Pull complete
b5e26e2ef46f: Pull complete
5fa7652a5f59: Pull complete
f0f36ddff6c5: Pull complete
1beff86c5121: Pull complete
1575edbb7f89: Pull complete
5f55c80ebec4: Pull complete
dae80ce2a02f: Pull complete
59dc1d9207d2: Pull complete
ffeb0c7a0d16: Pull complete
be378e762b31: Pull complete
1b46742c3546: Pull complete
95070b7b32e4: Pull complete
b1e1e78f7857: Pull complete
e4e5c468dca8: Pull complete
4a52a09714e8: Pull complete
a84238f298eb: Pull complete
f19ed39cf0fd: Pull complete
e746be989e03: Pull complete
a3184e84e5a3: Pull complete
185c350cc7e5: Pull complete
627f0e117356: Pull complete
1adfbc226f8e: Pull complete
88e1a8908377: Pull complete
bf33dcb88c6b: Pull complete
06de45739437: Pull complete
cd9299928399: Pull complete
17bdda4bed23: Pull complete
f0847357169c: Pull complete
75d91495f443: Pull complete
2f6a64e953df: Pull complete
Digest: sha256:46d06d8d248304cac200a1610adab48bd43b6f96950d0d9998194cfbce2b2835
Status: Downloaded newer image for swr.cn-central-221.ovaijisuan.com/mindformers/mindformers_dev_mindspore_1_10_1:mindformers_0.6.0dev_20230615_py39
swr.cn-central-221.ovaijisuan.com/mindformers/mindformers_dev_mindspore_1_10_1:mindformers_0.6.0dev_20230615_py39
root@ubuntu:~# docker login -u cn-north-309@UQ0YCRWIIJ4Q7UR0A63R -p 07c02a1f97ef2e5f0506c72c9b22f7296de1e43e6e20ed0dc7c3f882faeec7fd swr.cn-north-309.mtgascendic.cn
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
root@ubuntu:~# mindformers_dev_mindspore_1_10_1:mindformers_0.6.0dev_20230615_py                                                                              39sudo docker tag swr.cn-central-221.ovaijisuan.com/mindformers/mindformers_dev_                                                                              mindspore_1_10_1:mindformers_0.6.0dev_20230615_py39 swr.cn-north-309.mtgascendic                                                                              .cn/aicc_lz/mindformers_dev_mindspore_1_10_1:mindformers_0.6.0dev_20230615_py39^                                                                              C
root@ubuntu:~# sudo docker tag swr.cn-central-221.ovaijisuan.com/mindformers/min                                                                              dformers_dev_mindspore_1_10_1:mindformers_0.6.0dev_20230615_py39 swr.cn-north-30                                                                              9.mtgascendic.cn/aicc_lz/mindformers_dev_mindspore_1_10_1:mindformers_0.6.0dev_2                                                                              0230615_py39
root@ubuntu:~# docker push swr.cn-north-309.mtgascendic.cn/aicc_lz/mindformers_d                                                                              ev_mindspore_1_10_1:mindformers_0.6.0dev_20230615_py39
The push refers to repository [swr.cn-north-309.mtgascendic.cn/aicc_lz/mindforme                                                                              rs_dev_mindspore_1_10_1]
9efef30c32aa: Preparing
01a9a16e2668: Preparing
c5712af8a17d: Preparing
db280e538c6f: Preparing
126fde8a37c0: Preparing
2cf4bfba4482: Preparing
384c9e22b815: Waiting
1fbd15305307: Waiting
751d398eb53e: Waiting
54ab28f4e567: Preparing
2cf4bfba4482: Waiting
575a89fa936e: Preparing
2687bb1ea55d: Waiting
3a3e6ae19d7d: Waiting
4b91214b876b: Waiting
98ca7054afdc: Waiting
08588caa9e8c: Waiting
56e3137741ae: Waiting
575a89fa936e: Waiting
fe6f3dc260b6: Waiting
3c3ecefc768b: Waiting
af0c33ac85dd: Waiting
f8981441cef5: Waiting
505abd2f61f4: Waiting
4d88882b4901: Waiting
f0702ff1eb55: Waiting
d784cec35c0a: Waiting
040122e8b80d: Preparing
fd29a9717d53: Waiting
9204d9b6a0e4: Waiting
92e7456a14c3: Waiting
ede48f319b26: Preparing
7ddbb7762e8e: Waiting
ede48f319b26: Waiting
76956a73296b: Waiting
a3ba54e994c7: Waiting
407b1d90f91c: Preparing
4a1597322419: Waiting
da1e986cc22b: Waiting
5a50d393b5b7: Waiting
31ae5b42a91b: Preparing
42d14b030e23: Waiting
d17a52443df0: Waiting
3fe4f9b7f48b: Waiting
0f342db0ed7f: Waiting
3a4bcfe0fe41: Waiting
15d160702436: Waiting
f505431eb4df: Waiting
6a7668bf6919: Waiting
871887744f4f: Waiting
cfc0ee9792ea: Waiting
6d6189f99686: Preparing
407b1d90f91c: Waiting
ae995dd2db25: Waiting
f629e8a3ffb4: Waiting
781482aef03d: Preparing
9c41a34b5ef4: Waiting
bbe349b90b37: Waiting
3af29fffef76: Waiting
ed07bbecf3c9: Waiting
d86197d6f42d: Preparing
148cab5ddebe: Waiting
3fe4f9b7f48b: Pushed
a2d820d41bd2: Pushed
ec32b9434d98: Pushed
7f51e3e75b81: Pushed
8a56b925a0b5: Pushed
3f801a69d844: Pushed
08e934f34681: Pushed
775ead37ec48: Pushed
f2bdfe13a9d7: Pushed
6cbf0c57a96f: Pushed
e1aa3d6213e9: Pushed
b6c802dc9d21: Pushed
1765cdabb4ca: Pushed
f8f69bd4e159: Pushed
1b411266d046: Pushed
81f633409851: Pushed
f0d75b487767: Pushed
f31e8bd1a778: Pushed
cb399a44cec6: Pushed
71c36a160255: Pushed
7f2d3f882fc8: Pushed
bd6c3167d7df: Pushed
ec65416376e3: Pushed
2c2c66a6bb1c: Pushed
mindformers_0.6.0dev_20230615_py39: digest: sha256:46d06d8d248304cac200a1610adab                                                                              48bd43b6f96950d0d9998194cfbce2b2835 size: 18352
root@ubuntu:~#
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
本文内容由网友自发贡献，转载请注明出处：https://www.wpsshop.cn/w/小小林熬夜学编程/article/detail/214237