python 爬取携程全国旅游景点信息-2024.4.13_如何爬取携程城市列表

 1. 概述

         携程网是中国领先的在线旅行服务公司，提供酒店预订、机票预订、旅游度假、商旅管理等服务。携程网上有大量的旅游景点和酒店信息，这些信息对于旅行者和旅游业者都有很大的价值。通过爬虫技术，我们可以从携程网上获取这些信息，并进行数据清洗、数据分析、数据可视化等操作，从而得到有用的洞察和建议。

 2. 安装requests 库  

                在开始之前，请确保你已经安装了以下 Python 库：

                requests：用于发送 HTTP 请求并获取网页内容。

                你可以使用 pip 来安装这些库：

pip install requests 

3. 爬取携程旅游网站数据

        首先，我们需要确定要爬取的页面。 假设我们想要获取携程旅游网站上某个目的地的旅游信息。如下例如北京。

      当前接口 链接 和 post 参数

url = 'https://m.ctrip.com/restapi/soa2/18109/json/getAttractionList?_fxpcqlniredt=09031015313388236487&x-traceID=09031015313388236487-1712974794650-8267936'
 data = {"index":1,"count":10,"sortType":1,"isShowAggregation":true,"districtId":1,"scene":"DISTRICT","pageId":"214062","traceId":"14f9745c-92ad-f5c5-07bb-171293c80647","extension":[{"name":"osVersion","value":"10"},{"name":"deviceType","value":"windows"}],"filter":{"filterItems":[]},"crnVersion":"2020-09-01 22:00:45","isInitialState":true,"head":{"cid":"09031015313388236487","ctok":"","cver":"1.0","lang":"01","sid":"8888","syscode":"09","auth":"","xsid":"","extension":[]}}

4. 开始正式代码

headers = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36 Edg/123.0.0.0',
        'Cookie': 你的cookies
    }
 
    html = requests.post(url, headers=headers, json=data).json()
    attractionList = html['attractionList']
    for attraction in attractionList:
        data = attraction['card']
        commentCount = data['commentCount']
        commentScore = data['commentScore']
        coordinate = [data['coordinate']['latitude'], data['coordinate']['longitude']]
        coverImageUrl = data.get('coverImageUrl','')
        # 距离
        distanceStr = data.get('distanceStr','')
        # 地点
        displayField = data.get('displayField', None)
        heatScore = data.get('heatScore','')
        # 景点名
        poiName = data['poiName']
        isFree = data['isFree']
        if isFree:
            price = 0
            # 原价
            marketPrice = 0
        else:
            price = data.get('price',0)
            # 原价
            marketPrice = data.get('marketPrice',0)
        # 类别信息
        sightCategoryInfo = data.get('sightCategoryInfo','')
        # 标签
        tagNameList = data.get('tagNameList','')
        # 5a
        sightLevelStr = data.get('sightLevelStr', None)

5. 保存到csv

f = open('csv/全国各景点全.csv', 'w', encoding="utf-8", newline='')
 csvwrite = csv.writer(f)
csvwrite.writerow(['城市', '景点名', '地点', '距离', '坐标', '评论数','评论分','热评分','封面','是否免费','价格','原价','类别信息','标签','是否5A'])
        csvwrite.writerow([city,poiName,displayField,distanceStr,coordinate,commentCount,commentScore,heatScore,coverImageUrl,isFree,price,marketPrice,sightCategoryInfo,tagNameList,sightLevelStr])

6 .通过获取全国 city id，可请求全国景点数据并保存

        全国景点数据csv 地址 https://download.csdn.net/download/britlee/89115745