热门标签
热门文章
- 1gazebo创建机器人模型04_gazebo怎么修改link名字
- 2代码随想录算法刷题训练营_代码随想录算法训练营
- 3机器学习：利用Spark MLlib实现分布式机器学习算法训练与预测。_简述mllib 如何实现分布式机器学习模型的训练
- 4python_opencv(cv2)模块+灰度图二值化及降噪处理_lenna二值化处理
- 5ELasticSearch-实践-性能优化-外卖搜索-美团技术团队_elasticsearch 围栏
- 6AIGC应用场景（一）_aigc的应用
- 7【unity学习资料】学习进阶
- 8Vue简单的侧边栏_vue 管理平台 不用element-ui 自己写侧边导航栏
- 9完美解决nltk中nltk_data相关文件不能使用的问题_看项目nltk知识引用了 没看到使用
- 10VRRP小练习_the virtual ip address is not within a subnet on t
当前位置: article > 正文
如何让novnc/websockify支持tls1.2 (by quqi99)_novnc ssl
作者：码创造者 | 2024-08-16 11:20:38
赞
踩
novnc ssl
作者：张华 发表于：2020-09-10
版权声明：可以任意转载，转载时请务必以超链接形式标明文章原始出处和作者信息及本版权声明
如果希望novnc使用tls1.2该怎么办？
各软件(openssl, websockify, nova)支持tls1.2的历史情况如下：
- openssl在1.0.0h到1.0.1的时候才开始支持tls1.2, 见"Add TLS v1.2 server support for client authentication" - https://www.openssl.org/news/changelog.html，xenial使用的是1.0.2g版本
- ubuntu 20.04开始默认使用tls1.2 [2]
- websockify在0.9.0才开始支持配置tls版本, 见 - https://pypi.org/project/websockify/, 所以在nova xenial/mitaka版本中因为使用了websockify0.8.0所以列代码也不支持配置tls版本
# grep -r 'ssl.wrap_sock' /usr/lib/python2.7/dist-packages/nova/console/rfb/authvencrypt.py -A 7
wrapped_sock = ssl.wrap_socket(
compute_sock,
keyfile=client_key,
certfile=client_cert,
server_side=False,
cert_reqs=ssl.CERT_REQUIRED,
ca_certs=CONF.vnc.vencrypt_ca_certs)
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
这样，后来在nova升级到websockify 0.9.0之后也开始有了一个patch支持配置tls版本,见
https://review.opendev.org/#/c/679502/
但是，目前我们使用的就是xenial，该如何办呢？
首先，因为使用的是websockify 0.8.0不支持配置ssl版本，那样想在ssl.wrap_socket中直接指定＂ssl_version=ssl.PROTOCOL_TLSv1_2＂来强制使用tls1_2是可行的．（如果不可行，是因为下列错误导致，见： https://github.com/freedesktop/spice-html5/commit/293d405e15a4499219fe81e830862cc2b1518e3e )
SecurityError: Failed to construct 'WebSocket': An insecure WebSocket connection may not be initiated from a page loaded over HTTPS.
- 1
那样，是否应该改变底层openssl的默认配置成tls1.2, 参考文档[1], 修改/etc/ssl/openssl.cnf在oid_section之后添加下列内容
root@juju-055b8b-ssl-7:~# grep -r 'oid_section' /etc/ssl/openssl.cnf -A 2
oid_section = new_oids
openssl_conf = default_conf
root@juju-055b8b-ssl-7:~# cat /etc/ssl/openssl.cnf |tail -n9
[default_conf]
ssl_conf = ssl_sect
[ssl_sect]
system_default = system_default_sect
[system_default_sect]
MinProtocol = TLSv1.2
CipherString = DEFAULT@SECLEVEL=2
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
但是使用下列方法测试时报错．
systemctl restart nova-novncproxy nova-consoleauth
# verify if tls 1.2 is supported - https://devanswers.co/test-server-tls-1-2-ubuntu/
# https://www.poftut.com/use-openssl-s_client-check-verify-ssltls-https-webserver/
OPENSSL_CONF=/etc/ssl/ openssl s_client -connect 10.5.100.4:6080 -tls1
OPENSSL_CONF=/etc/ssl/ openssl s_client -connect 10.5.100.4:6080 -tls1_2
#list all supported ciphers
nmap --script ssl-enum-ciphers -p 6080 10.5.100.4
OPENSSL_CONF=/etc/ssl/ openssl s_client -connect 10.5.2.196:6082 -tls1_2
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
root@juju-055b8b-ssl-7:~# openssl s_client -connect 10.5.100.4:6080 -tls1_2
Error configuring OpenSSL
139929571108504:error:25066067:DSO support routines:DLFCN_LOAD:could not load the shared library:dso_dlfcn.c:187:filename(libssl_conf.so): libssl_conf.so: cannot open shared object file: No such file or directory
139929571108504:error:25070067:DSO support routines:DSO_load:could not load the shared library:dso_lib.c:233:
139929571108504:error:0E07506E:configuration file routines:MODULE_LOAD_DSO:error loading dso:conf_mod.c:271:module=ssl_conf, path=ssl_conf
139929571108504:error:0E076071:configuration file routines:MODULE_RUN:unknown module name:conf_mod.c:212:module=ssl_conf
- 1
- 2
- 3
- 4
- 5
- 6
错误是找不着libssl_conf.so，作下列更改后，问题依旧 ．
sudo apt-get install libssl1.0.0 libssl-dev
cd /lib/x86_64-linux-gnu/
sudo ln -s libssl.so.1.0.0 libssl.so.10
sudo ln -s libcrypto.so.1.0.0 libcrypto.so.10
- 1
- 2
- 3
- 4
20201120更新-是需要添加OPENSSL_CONF=/etc/ssl/
# OPENSSL_CONF=/etc/ssl/ openssl ciphers -v TLSv1.2 | head -4
ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(256) Mac=AEAD
ECDHE-ECDSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AESGCM(256) Mac=AEAD
ECDHE-RSA-AES256-SHA384 TLSv1.2 Kx=ECDH Au=RSA Enc=AES(256) Mac=SHA384
ECDHE-ECDSA-AES256-SHA384 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AES(256) Mac=SHA384
- 1
- 2
- 3
- 4
- 5
将xenial上的openssl 1.1.0g升级到1.1.1a后问题依旧:
#upgrade openssl from 1.1.0g to 1.1.1a in ubuntu 18.04
sudo apt install build-essential checkinstall zlib1g-dev gcc make -y
wget https://www.openssl.org/source/openssl-1.1.1a.tar.gz
tar zxvf openssl-1.1.1a.tar.gz && cd openssl-1.1.1a/
./config --prefix=/usr/local/ssl --openssldir=/usr/local/ssl
make
sudo make install
sudo bash -c 'cat > /etc/ld.so.conf.d/openss1-1.1.1b.conf' << EOF
/usr/local/ssl/lib
EOF
sudo ldconfig -v
sudo mv /usr/bin/c_rehash /usr/bin/c_rehash.BAK
sudo mv /usr/bin/openssl /usr/bin/openssl.BAK
sudo cp /etc/environment /etc/environment.BAK
sudo bash -c 'cat > /etc/environment' << EOF
PATH="/usr/local/ssl/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games"
EOF
echo $PATH
openssl version -a
sudo apt-get install libssl-dev -y
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
然后修改/usr/lib/python2.7/dist-packages/websockify/websocket.py（注意，不是/usr/lib/python2.7/dist-packages/nova/console/rfb/authvencrypt.py）里的ssl.wrap_socket添加’ssl_version=ssl.PROTOCOL_TLSv1_2’就可以了，
# nmap --script ssl-enum-ciphers -p 6080 10.5.100.4 |grep TLSv
| TLSv1.2:
root@juju-055b8b-ssl-7:~/openssl-1.1.1a# openssl s_client -connect 10.5.100.4:6080 -tls1_2 |grep 'CN'
depth=0 C = GB, ST = England, L = London, O = Ubuntu Cloud, OU = Cloud, CN = 10.5.100.0
...
- 1
- 2
- 3
- 4
- 5
此时，去掉/usr/lib/python2.7/dist-packages/nova/console/rfb/authvencrypt.py里之前做的tls1.2修改也是可以的．
结论，似乎修改底层openssl的默认版本到tls1.2不好使，同时websockify 0.8.0默认的只是tls1.0, 此时只是修改nova端是不work的，直接修改websockify 0.8.0改到tls1.2是可以的．在websockify 0.9.0之后支持配置tls1.2，所以此时nova端修改tls的patch也就能派上用场了．
相关代码分析：
python ssl模块在wrap_socket中是将参数硬编码成ssl_version=PROTOCOL_SSLv23，所以在websockify 0.8.0不传ssl_version参数的话是即使改底层openssl的默认编码也是无济于事的．
def wrap_socket(sock, keyfile=None, certfile=None,
server_side=False, cert_reqs=CERT_NONE,
ssl_version=PROTOCOL_SSLv23, ca_certs=None,
do_handshake_on_connect=True,
suppress_ragged_eofs=True, ciphers=None):
- 1
- 2
- 3
- 4
- 5
看来唯一的办法是修改python-websockify 0.8.0包，加上那一行，做个临时hotfix了．
20201106更新
创建了一个xenial的spice ssl测试环境．
./generate-bundle.sh -s xenial -r queens --create-model --name ssl2:stsstack --num-compute 1 --openstack-dashboard --ssl --nova-console --run
juju config openstack-dashboard enforce-ssl=true
ssl_results=/home/ubuntu/ed/stsstack-bundles/openstack/ssl/openstack-ssl2/results
juju config openstack-dashboard ssl_ca=`base64 ${ssl_results}/cacert.pem| tr -d '\n'`
juju config openstack-dashboard ssl_cert=`base64 ${ssl_results}/servercert.pem| tr -d '\n'`
juju config openstack-dashboard ssl_key=`base64 ${ssl_results}/serverkey.pem| tr -d '\n'`
juju config nova-cloud-controller console-access-protocol=spice
juju config nova-cloud-controller console-ssl-cert=`base64 ${ssl_results}/servercert.pem| tr -d '\n'`
juju config nova-cloud-controller console-ssl-key=`base64 ${ssl_results}/serverkey.pem| tr -d '\n'`
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
但我们尽量不用–ssl，改用–vault
./generate-bundle.sh -s xenial -r queens --create-model --name ssl-queens:stsstack --num-compute 1 --nova-console --vault
- 1
看到下列错，实际上这是正常的．
$ nova get-vnc-console bionic-061058 spice-html5
+-------------+------------------------------------------------------------------------------------+
| Type | Url |
+-------------+------------------------------------------------------------------------------------+
| spice-html5 | https://10.5.100.4:6082/spice_auto.html?token=69b15db8-2575-4bc9-980b-3e4149881015 |
+-------------+------------------------------------------------------------------------------------+
$ curl -k -vvv https://10.5.100.4:6082/spice_auto.html?token=69b15db8-2575-4bc9-980b-3e4149881015
* Trying 10.5.100.4:6082...
* TCP_NODELAY set
* Connected to 10.5.100.4 (10.5.100.4) port 6082 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (OUT), TLS alert, protocol version (582):
* error:1425F102:SSL routines:ssl_choose_client_version:unsupported protocol
* Closing connection 0
curl: (35) error:1425F102:SSL routines:ssl_choose_client_version:unsupported protocol
-root@juju-1833ad-ssl2-7:~# tail -f /var/log/nova/nova-spiceproxy.log
2020-11-06 06:42:33.666 26162 DEBUG nova.console.websocketproxy [-] 10.5.0.8: new handler Process vmsg /usr/lib/python2.7/dist-packages/websockify/websocket.py:878
2020-11-06 06:42:34.166 19170 INFO nova.console.websocketproxy [-] handler exception: [SSL: SSLV3_ALERT_CERTIFICATE_UNKNOWN] sslv3 alert certificate unknown (_ssl.c:590)
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
为什么说它是正常的呢？因为我们是在focal上运行的curl命令或者浏览器，focal支持的最低ssl版本是tlsv1.2，而spiceproxy运行在xenial只支持tlsv1.0. 所以找一台xenial机器运行下列加了–tlsv1.0的curl命令是OK的。
curl -k -vvv https://10.5.100.4:6082/spice_auto.html?token=bd00bb9f-83e7-4b25-9b6a-57dde9941dce --tlsv1.0
- 1
还是上面类似的老问题，spice也用到了websockify0.8，而它写死了 ssl_version=PROTOCOL_TLSv1
> /usr/lib/python2.7/dist-packages/websockify/websocket.py(837)do_handshake()
-> retsock = ssl.wrap_socket(
(Pdb) l
832 raise self.EClose("SSL connection but '%s' not found"
833 % self.cert)
834 retsock = None
835 try:
836 import rpdb;rpdb.set_trace()
837 -> retsock = ssl.wrap_socket(
838 sock,
839 server_side=True,
840 certfile=self.cert,
841 keyfile=self.key)
842 except ssl.SSLError:
(Pdb) p self.cert
'/etc/nova/ssl/nova_cert.pem'
(Pdb) p self.key
'/etc/nova/ssl/nova_key.pem'
(Pdb) l
48
49 def __init__(self, sock, keyfile=None, certfile=None,
50 server_side=False, cert_reqs=CERT_NONE,
51 ssl_version=PROTOCOL_TLSv1, ca_certs=None,
52 do_handshake_on_connect=True, *args, **kw):
53 -> if not isinstance(sock, GreenSocket):
54 sock = GreenSocket(sock)
55
56 self.act_non_blocking = sock.act_non_blocking
57
58 if six.PY2:
root@juju-1833ad-ssl2-7:~# pip list |grep web
websockify (0.8.0)
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
但直接在xenial上升级到websockify=0.9.0失败.
在queens中使用websockify 0.9.0
将websockify的patch backedport到0.8.0时依赖太多，所以只能将0.9.0整体backport到queens. 测试一下：
mv /usr/lib/python2.7/dist-packages/websockify ~/websockify_bak
git clone https://github.com/novnc/websockify.git
cd websockify && git checkout -b v0.9.0 v0.9.0
cd ../ && cp -r websockify/websockify /usr/lib/python2.7/dist-packages/
cp allow-TLS-ciphers-protocols-to-be-configurable-for-c.patch /usr/lib/python2.7/dist-packages/
cd /usr/lib/python2.7/dist-packages/ && patch -p1 <allow-TLS-ciphers-protocols-to-be-configurable-for-c.patch
then set the following content in /etc/nova/nova.conf
[console]
ssl_minimum_version=tlsv1_2
vim /usr/lib/python2.7/dist-packages/nova/console/websocketproxy.py
def socket(self, *args, **kwargs):
#return websockify.WebSocketServer.socket(*args, **kwargs)
return websockify.websockifyserver.WebSockifyServer.socket(*args, **kwargs)
find /usr/lib/python2.7/dist-packages/websockify -name "*.pyc" -exec rm -rf {} \;
find /usr/lib/python2.7/dist-packages/nova -name "*.pyc" -exec rm -rf {} \;
systemctl restart nova-spiceproxy
nmap --script ssl-enum-ciphers -p 6082 10.5.2.196 |grep -i tlsv
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
代码流程
websocketproxy.py#websockify_init (opts.ssl_options = select_ssl_version(opts.ssl_version)) -> websockifyserver.py#start_server()
when client is connecting:
WebSocketProxy -> ./websocketproxy.py#ProxyRequestHandler -> WebSockifyRequestHandler#handle -> /usr/lib/python3.6/http/server.py(377)handle_one_request() -> _websocket_do_GET -> handle_upgrade (self.headers.get('upgrade').lower() == 'websocket') -> WebSocketRequestHandlerMixIn.handle_upgrade(self) -> handle_websocket(SSL/TLS)-> new_websocket_client -> /usr/lib/python3/dist-packages/nova/console/websocketproxy.py(166)new_websocket_client
- 1
- 2
- 3
- 4
似乎是在queens中下面的self.headers.get(‘upgrade’)不为wesocket
54 def _websocket_do_GET(self):
55 # Checks if it is a websocket request and redirects
56 self.do_GET = self._real_do_GET
57
58 if (self.headers.get('upgrade') and
59 self.headers.get('upgrade').lower() == 'websocket'):
60 self.handle_upgrade()
61 else:
62 self.do_GET()
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
根据这个网页（https://www.slideshare.net/DvidHalsz/smuggling-tcp-traffic-through-http-71473570）
结论
主要是由两个问题导致：
1, one spice-html5 bug
https://github.com/freedesktop/spice-html5/commit/293d405e15a4499219fe81e830862cc2b1518e3e
2, ssl默认使用PROTOCOL_SSLv23 （ https://docs.python.org/2/library/ssl.html#socket-creation ）要使用tlsv1_2，或者在ssl.wrap_socket中直接指定＂ssl_version=ssl.PROTOCOL_TLSv1_2，或者使用nova与websockify的patch可以通过ssl_option来配置使用的tls版本。
3, 或者/usr/lib/python2.7/dist-packages/eventlet/green/ssl.py中不应该用PROTOCOL_TLSv1作为默认，而应该用PROTOCOL_SSLv23， 这样PROTOCOL_SSLv23会依赖底层ssl版本自动选择tlsv1_0, tlsv1_1, tlsv1_2, 这样即使防火墙disable tlsv1_0也不影响使用tlsv1_2. 最终的原因是因为python-eventlet 0.18.4-1引入了Rebased set-defaults-to-be-tlsv1-not-sslv23.patch，是它使用了PROTOCOL_TLSv1，应该去掉。
附录 - Websocket based spice client
1, a test vm with spice enabled
<graphics type='spice' port='5900' autoport='yes' listen='127.0.0.1'>
<listen type='address' address='127.0.0.1'/>
</graphics>
2, spice proxy side
sudo apt install python3-websockify -y
websockify 192.168.2.139:6082 127.0.0.1:5900
3, spice client side
sudo apt install spice-html5 apache2 -y
ls /usr/share/spice-html5/spice_auto.html #it's similar to https://10.5.1.11:6082/spice_auto.html?token=xxx
cat << EOF | sudo tee /etc/apache2/conf-available/ws.conf
Alias /spice /usr/share/spice-html5
<Directory /usr/share/spice-html5>
# This page is broadly available, tune here to make it more restricted.
Allow from all
Satisfy Any
DirectoryIndex spice.html
</Directory>
EOF
sudo ln -s /etc/apache2/conf-available/ws.conf /etc/apache2/conf-enabled/ws.conf
sudo systemctl restart apache2
4, access it via http://192.168.2.139/spice/spice_auto.html?host=192.168.2.139&port=6082
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
附录 - Websocket ssl based spice client
1, create key pairs
mkdir ~/ca && cd ~/ca
openssl req -newkey rsa:4096 -x509 -sha256 -days 3650 -nodes -out ca.crt -keyout ca.key -subj "/C=US/ST=UK/L=London/O=Ubuntu/OU=IT/CN=CA"
for DOMAIN in server client
do
openssl genrsa -out $DOMAIN.key
openssl req -new -key $DOMAIN.key -out $DOMAIN.csr -subj "/C=GB/ST=UK/L=London/O=Ubuntu/OU=Cloud/CN=$DOMAIN"
openssl x509 -req -in $DOMAIN.csr -out $DOMAIN.crt -sha256 -CA ca.crt -CAkey ca.key -CAcreateserial -days 3650
done
2, spice server side
sudo mkdir /etc/apache2/ssl && sudo chown -R $USER /etc/apache2/ssl
cp /home/hua/ca/server.crt /etc/apache2/ssl/
cp /home/hua/ca/server.key /etc/apache2/ssl/
cat << EOF | sudo tee /etc/apache2/conf-available/ws.conf
<VirtualHost 192.168.2.139:443>
SSLEngine on
SSLCertificateFile /etc/apache2/ssl/server.crt
SSLCertificateKeyFile /etc/apache2/ssl/server.key
CustomLog "/var/log/apache2/ws.log" combined
ErrorLog "/var/log/apache2/ws.log"
Alias /spice /usr/share/spice-html5
<Directory /usr/share/spice-html5>
# This page is broadly available, tune here to make it more restricted.
Allow from all
Satisfy Any
DirectoryIndex spice.html
</Directory>
</VirtualHost>
EOF
sudo a2enmod ssl
sudo systemctl restart apache2
3, spice proxy side
websockify --cert=/home/hua/ca/server.crt --key=/home/hua/ca/server.key --cafile=~/home/hua/ca/ca.crt --ssl-only --ssl-version=tlsv1_2 192.168.2.139:6082 127.0.0.1:5900
4, access it via: https://192.168.2.139/spice/spice_auto.html?host=192.168.2.139&port=6082
nmap --script ssl-enum-ciphers -p 6082 192.168.2.139
openssl s_client -connect 192.168.2.139:6082 -tls1_2
5, debug, print http headers by adding the following scripts in /usr/share/spice-html5/spice_auto.html
var req = new XMLHttpRequest();
req.open('GET', document.location, false); #but this is not the headers for websocket upgrade
req.send(null);
var headers = req.getAllResponseHeaders();
alert(headers);
6, why we see the following error when visitting the url 'https://10.5.2.196:6082/spice_auto.html?token=xxxx'
SecurityError: Failed to construct 'WebSocket': An insecure WebSocket connection may not be initiated from a page loaded over HTTPS.
That's because the following content is missing in /usr/share/spice-html5/spice_auto.html, see this patch - https://github.com/freedesktop/spice-html5/commit/293d405e15a4499219fe81e830862cc2b1518e3e
if (window.location.protocol == 'https:') {
scheme = "wss://";
}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
Debug novnc
Debug novnc
先debug http, "10.5.1.160:6080"是已存在的相当于上面的apache2一步它里面带了novnc中的vnc_auto.html
root@juju-9c8c65-ssl-8:~# virsh dumpxml 1 |grep vnc -A2
<graphics type='vnc' port='5900' autoport='yes' listen='10.5.2.13' keymap='en-us'>
<listen type='address' address='10.5.2.13'/>
</graphics>
websockify 10.5.1.161:6081 10.5.2.13:5900 -v
http://10.5.1.161:6080/vnc_auto.html?host=10.5.1.161&port=6081
#some log
zhhuabj-bastion.cloud.sts - - [27/Jan/2021 06:09:51] "GET /websockify HTTP/1.1" 101 -
zhhuabj-bastion.cloud.sts - - [27/Jan/2021 06:09:51] 10.5.0.8: Plain non-SSL (ws://) WebSocket connection
zhhuabj-bastion.cloud.sts - - [27/Jan/2021 06:09:51] 10.5.0.8: Version hybi-13, base64: 'False'
zhhuabj-bastion.cloud.sts - - [27/Jan/2021 06:09:51] 10.5.0.8: Path: '/websockify'
zhhuabj-bastion.cloud.sts - - [27/Jan/2021 06:09:51] connecting to: 10.5.2.13:5900
再debug https, 沿用nova.conf(nova-novncproxy)中的cert与key, 只是用于在浏览器与websockify建立ssl, websockify与qemu之间还是http
#--cafile=~/home/hua/ca/ca.crt --ssl-version=tlsv1_2
openssl x509 -noout -text -in /etc/apache2/ssl/nova/cert_10.5.1.161
websockify --cert=/etc/apache2/ssl/nova/cert_10.5.1.161 --key=/etc/apache2/ssl/nova/key_10.5.1.161 --ssl-only 10.5.1.161:6081 10.5.2.13:5900 -v
https://10.5.1.161:6080/vnc_auto.html?host=10.5.1.161&port=6081
#some log
root@juju-9c8c65-ssl-7:~# websockify --cert=/etc/apache2/ssl/nova/cert_10.5.1.161 --key=/etc/apache2/ssl/nova/key_10.5.1.161 --ssl-only 10.5.1.161:6081 10.5.2.13
:5900 -v
WebSocket server settings:
- Listen on 10.5.1.161:6081
- Flash security policy server
- SSL/TLS support
- Deny non-SSL/TLS connections
- proxying from 10.5.1.161:6081 to 10.5.2.13:5900
10.5.0.8: new handler Process
handler exception: [SSL: SSLV3_ALERT_CERTIFICATE_UNKNOWN] sslv3 alert certificate unknown (_ssl.c:590)
exception
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/websockify/websocket.py", line 933, in top_new_client
client = self.do_handshake(startsock, address)
File "/usr/lib/python2.7/dist-packages/websockify/websocket.py", line 840, in do_handshake
keyfile=self.key)
File "/usr/lib/python2.7/ssl.py", line 933, in wrap_socket
ciphers=ciphers)
File "/usr/lib/python2.7/ssl.py", line 601, in __init__
self.do_handshake()
File "/usr/lib/python2.7/ssl.py", line 830, in do_handshake
self._sslobj.do_handshake()
SSLError: [SSL: SSLV3_ALERT_CERTIFICATE_UNKNOWN] sslv3 alert certificate unknown (_ssl.c:590)
Ignoring interrupted syscall
10.5.0.8: new handler Process
zhhuabj-bastion.cloud.sts - - [27/Jan/2021 06:21:27] "GET /websockify HTTP/1.1" 101 -
zhhuabj-bastion.cloud.sts - - [27/Jan/2021 06:21:27] 10.5.0.8: SSL/TLS (wss://) WebSocket connection
zhhuabj-bastion.cloud.sts - - [27/Jan/2021 06:21:27] 10.5.0.8: Version hybi-13, base64: 'False'
zhhuabj-bastion.cloud.sts - - [27/Jan/2021 06:21:27] 10.5.0.8: Path: '/websockify'
zhhuabj-bastion.cloud.sts - - [27/Jan/2021 06:21:27] connecting to: 10.5.2.13:5900
zhhuabj-bastion.cloud.sts - - [27/Jan/2021 06:21:28] 10.5.2.13:5900: Client closed connection
zhhuabj-bastion.cloud.sts - - [27/Jan/2021 06:21:28] 10.5.2.13:5900: Closed target
Ignoring interrupted syscall
#/usr/lib/python2.7/dist-packages/websockify/websocket.py
#ssl_version=ssl.PROTOCOL_TLSv1_2,
这样改试spice client不work
sudo apt install spice-html5 -y
sed -i "s/ws:/wss:/g" /usr/share/spice-html5/spice_auto.html
cat << EOF | sudo tee /etc/apache2/conf-available/ws.conf
Alias /spice /usr/share/spice-html5
<Directory /usr/share/spice-html5>
# This page is broadly available, tune here to make it more restricted.
Allow from all
Satisfy Any
DirectoryIndex spice.html
</Directory>
EOF
sudo ln -s /etc/apache2/conf-available/ws.conf /etc/apache2/conf-enabled/ws.conf
sudo systemctl restart apache2
websockify --cert=/etc/apache2/ssl/nova/cert_10.5.1.161 --key=/etc/apache2/ssl/nova/key_10.5.1.161 --ssl-only 10.5.1.161:6081 10.5.2.13:5900 -v
#https://10.5.1.161:6080/vnc_auto.html?host=10.5.1.161&port=6081
https://10.5.1.161:443/spice/spice_auto.html?host=10.5.1.161&port=6081
再这样改试spice却是work的
juju config nova-cloud-controller console-access-protocol=spice
#need to run 'nova reboot --hard bionic-032738' to fix 'Unavailable console type spice'
nova get-vnc-console bionic-032738 spice-html5
websockify --cert=/etc/apache2/ssl/nova/cert_10.5.1.161 --key=/etc/apache2/ssl/nova/key_10.5.1.161 --ssl-only 10.5.1.161:6081 10.5.2.13:5900 -v
#https://10.5.1.161:6080/vnc_auto.html?host=10.5.1.161&port=6081
https://10.5.1.161:6082/spice_auto.html?token=a44045f9-0d22-495e-831a-8f64ae445d2f&host=10.5.1.161&port=6081
https://10.5.1.161:6082/spice_auto.html?host=10.5.1.161&port=6081
这样确定了novnc的问题不是是ssl造成的，而是timeout，所以在/usr/share/novnc/include/rfb.js修改下面配置之后
def_con_timeout = Websock_native ? 200 : 500,
再restart(sudo service nova-novncproxy restart)就可以成功访问了。
https://10.5.1.161:6080/vnc_auto.html?host=10.5.1.161&port=6081
https://10.5.1.161:6080/vnc_auto.html?token=887a070e-1f45-4320-912f-577a50417b1d&host=10.5.1.161&port=6081
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
- 71
- 72
- 73
- 74
- 75
- 76
- 77
- 78
- 79
- 80
- 81
- 82
- 83
- 84
- 85
- 86
- 87
- 88
- 89
- 90
- 91
- 92
- 93
- 94
20210204更新
有时候如heat charm在里面会创建haproxy实例（外面用hacluster提供corosync做HA)，但并不需要在haproxy.cnf里设置如下设置：
ssl-default-bind-options ssl-min-ver TLSv1.2 no-tls-tickets
ssl-default-server-options ssl-min-ver TLSv1.2 no-tls-tickets
- 1
- 2
因为haproxy的后面是apache2，它由这个bug (https://bugs.launchpad.net/charm-helpers/+bug/1886630)已经提供了fix (https://github.com/juju/charm-helpers/commit/27d6ceb385e44a0610c1a6aba8e225368c4af384 )在apache2层面禁用了sslv3/tlsv1
root@juju-170792-stein-2:~# grep ':7994' /etc/apache2/sites-available/openstack_https_frontend.conf -A 7
<VirtualHost 10.5.3.29:7994>
ServerName 10.5.3.29
SSLEngine on
# This section is based on Mozilla's recommendation
# as the "intermediate" profile as of July 7th, 2020.
# https://wiki.mozilla.org/Security/Server_Side_TLS
SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1
root@juju-170792-stein-2:~# grep -r 7994 /etc/haproxy/haproxy.cfg -B2
backend heat_api_10.5.3.29
balance leastconn
server heat-0 10.5.3.29:7994 check
root@juju-170792-stein-2:~# grep -r 'use_backend heat_api_10.5.3.29' /etc/haproxy/haproxy.cfg -B4
frontend tcp-in_heat_api
bind *:8004
bind :::8004
acl net_10.5.3.29 dst 10.5.3.29/255.255.0.0
use_backend heat_api_10.5.3.29 if net_10.5.3.29
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
landscape也会在postgresql前安装haproxy，但也没提供enable TLSv1.2的配置项，可暂时用下列workaround:
juju config landscape-postgresql extra_pg_conf="ssl_ciphers='TLSv1.2'"
- 1
keystone也会在apache2前安装haproxy，这个apache2不是来自charm-helper, 可单独修改charm template(/var/lib/juju/agents/unit-keystone-0/charm/templates/openstack_https_frontend.conf)作为workaround:
-SSLProtocol +TLSv1 +TLSv1.1 +TLSv1.2
+SSLProtocol -TLSv1 -TLSv1.1 +TLSv1.2
- 1
- 2
对于Landscape/Postfix, In /etc/postfix, master.cf or main.cf (whichever one you use):
smtpd_tls_mandatory_protocols = x,y,z,!TLSv1, !TLSv1.1
- 1
20230510更新 - gnutls
focal无法通过landscape server 注册，但jammy可以。
gnutls-cli -d 9999 <landscape-https-server> --no-ca-verification --priority NORMAL:-VERS-TLS1.3
- 1
jammy默认是用tls1.3, focal也是用tls1.3，但是focal用的gnutls版本更低一些，缺这个commit e0bb98e1f71f94691f600839ff748d3a9f469d3e 就会出现这个问题。不是通过debian二分，而是通过直接从源码二分找到这个commit
git bisect start --term-new=fixed --term-old=unfixed
git bisect fixed 3.7.3
git bisect unfixed 3.6.13
git show e0bb98e1f71f94691f600839ff748d3a9f469d3e
commit e0bb98e1f71f94691f600839ff748d3a9f469d3e
Author: Norbert Pocs <npocs@redhat.com>
Date: Fri Oct 30 17:18:30 2020 +0100
Fix non-empty session id (TLS13_APPENDIX_D4)
When TLS1.3 is used with middlebox compatible mode, the session id should be filled with random session id,
but remained empty.
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
[1] https://blog.surgut.co.uk/2019/08/how-to-disable-tls-10-and-tls-11-on.html
[2] https://discourse.ubuntu.com/t/default-to-tls-v1-2-in-all-tls-libraries-in-20-04-lts/12464
[3] https://notes.bitfunnel.net/?q=node/54
[4] https://github.com/certik/python-2.7/blob/master/Lib/ssl.py#L373
声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/码创造者/article/detail/987979
推荐阅读
相关标签
