当前位置:   article > 正文

爬取微博话题_爬取微博话题数据

爬取微博话题数据
最近微博上有一个比较火的话题,叫做耳机对当代人有多重要,很是感兴趣

在这里插入图片描述

但是刷微博这种事情,太耽误学习了,那又想刷微博又想学习,该怎么办
那不如这样吧,写个爬虫,print每条评论,这样我就可以一边敲代码,时不时的看看打印,这多香啊
本次实战数据清洗部分极其恶心,我使用的数据库是mysql,那评论里又有很多特殊符号,特殊符号还好,使用mysql的utf8mb4还是可以处理的,但表情就实在是难住我了,我只好用最笨的replace大致的清洗下出现频率最多的表情了

开搞

import time
from http.cookiejar import CookieJar
from urllib import request
from lxml import etree
from urllib_day02 import myurllib
import MySQLdb
import re

conn = MySQLdb.connect(
    user='root',
    port=3306,
    password='123456',
    db='spider',
    charset='utf8mb4',
    host='localhost',
)
cursor = conn.cursor()

datas = '''Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Accept-Language: zh-CN,zh;q=0.9
Cache-Control: max-age=0
Connection: keep-alive
Cookie: SINAGLOBAL=329643001885.6844.1573002148871; login_sid_t=4396613b7b8d3841a00c4a00f2500236; cross_origin_proto=SSL; _s_tentry=www.baidu.com; Apache=5427773053657.6455.1578554625853; ULV=1578554626858:2:1:1:5427773053657.6455.1578554625853:1573002148894; WBtopGlobal_register_version=307744aa77dd5677; webim_unReadCount=%7B%22time%22%3A1578556209612%2C%22dm_pub_total%22%3A8%2C%22chat_group_client%22%3A0%2C%22allcountNum%22%3A19%2C%22msgbox%22%3A0%7D; UOR=,,login.sina.com.cn; appkey=; WBStorage=42212210b087ca50|undefined; SCF=Aj-iTSvsUNnwWoRS3kObxNvX9WkeGeRSr0D4KFIJls0TaAPw29BxH8f2ApSGwnwHbwIobZR3bXx0tBuK9PrkT7c.; SUB=_2A25zEq9zDeRhGeBP4lMT8i_PyDmIHXVQaYe7rDV8PUJbmtANLWvfkW9NRQq3_0YRQLcrIqGX1CGWYTAYXYJ05EGf; SUBP=0033WrSXqPxfM725Ws9jqgMF55529P9D9WWxxLrHCKQeaCVeIwANKWnx5JpX5K2hUgL.Foqp1K2Eeo20e0-2dJLoIceLxK-L1-eLBKnLxKqLB-qL12qLxKBLBonL1h5LxKqL1-BLBK2LxKBLB.eLBKBLxK-LB.qL1heLxKML1-2L1hBLxKqL1-zL1K.LxK-L1h-L1h.LxK-LBKBLBKMLxKnLBK2L1KMt; SUHB=0d4OXckQXeM2Bf; SSOLoginState=1578557219; un=13503301458
Host: s.weibo.com
Referer: https://weibo.com/
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: same-site
Sec-Fetch-User: ?1
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27
  • 28
  • 29
声明:本文内容由网友自发贡献,不代表【wpsshop博客】立场,版权归原作者所有,本站不承担相应法律责任。如您发现有侵权的内容,请联系我们。转载请注明出处:https://www.wpsshop.cn/w/盐析白兔/article/detail/841368
推荐阅读
相关标签
  

闽ICP备14008679号