golang 爬虫之 gospider (requests--基础篇)_go1.21安装gospider

gospider 介绍

gospider 是一个golang 爬虫神器，它内置了多种反爬虫模块,是golang 爬虫必备的工具包

安装

go get -u gitee.com/baixudong/gospider
1

gitee地址

https://gitee.com/baixudong/gospider
1

github地址

https://github.com/baixudong007/gospider
1

请求例子

package main

import (
	"context"
	"log"

	"gitee.com/baixudong/gospider/requests"
)

func main() {
	reqCli, err := requests.NewClient(context.TODO())
	if err != nil {
		log.Panic(err)
	}
	response, err := reqCli.Request(context.TODO(), "get", "http://myip.top")
	if err != nil {
		log.Panic(err)
	}
	log.Print(response.Text())
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20