Max retries exceeded with url问题解决_celaba数据集下载 max retries exceeded with url

错误内容

requests.exceptions.ConnectionError: HTTPConnectionPool(host='baidu.com', port=80): 
Max retries exceeded with url: 
(Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x2b09dfd00310>: 
Failed to establish a new connection:
 [Errno 110] Connection timed out',))
1
2
3
4
5

原代码

import requests

url = "http://baidu.com"
headers = {
    'Accept': "application/json, text/plain, */*",
    'User-Agent': "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36"
}
ret = requests.get(url=url, headers=headers, timeout=300)
1
2
3
4
5
6
7
8

修改如下

import requests

url = "http://baidu.com"
headers = {
    'Accept': "application/json, text/plain, */*",
    'User-Agent': "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36"
}
requests.DEFAULT_RETRIES = 5  # 增加重试连接次数
s = requests.session()
s.keep_alive = False  # 关闭多余连接
ret = requests.get(url=url, headers=headers, timeout=300)
1
2
3
4
5
6
7
8
9
10
11