devbox
秋刀鱼在做梦
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

高效获取网站数据，网站爬虫技巧大揭秘！电商平台爬虫API列表！

作者：秋刀鱼在做梦 | 2024-07-19 01:17:51

赞

踩

爬虫api

1、什么是网站爬虫

网站爬虫（Web Crawler）是一种自动化程序，用于抓取互联网上的信息。它可以在浏览器运行时自动访问目标网站，并将所需信息保存到本地文件或数据库中。

2、网站爬虫的工作原理

网站爬虫主要分为三个步骤：

1.它会根据指定的 URL 发送 HTTP 请求；

2.它会解析服务器返回的 HTML 页面，并提取出需要的信息；

3.它会将这些信息保存到本地文件或数据库中。

3、网站爬虫的应用场景

网站爬虫可以广泛应用于各个领域。

以下是一些常见的应用场景:

  • 1.数据技术分析：通过爬取互联网上的数据，进行研究数据信息分析和挖掘，为商业决策提供社会支持。

  • 2.信息聚合: 通过在多个网站上抓取信息，将这些信息集成到一个平台上，为用户提供更全面、更准确的信息服务。

  • 3. 网络安全: 通过对 Internet 上的信息进行爬行，及时发现和处理网络安全威胁。

  • 4.电商类API数据抓取（商品数据、订单数据）

 

taobao.item_get

公共参数

请求地址: https://api-gw.onebound.cn/taobao/item_get

名称类型必须描述
keyString是调用key（必须以GET方式拼接在URL中）
secretString是调用密钥
api_nameString是API接口名称（包括在请求地址中）[item_search,item_get,item_search_shop等]
cacheString否[yes,no]默认yes，将调用缓存的数据，速度比较快
result_typeString否[json,jsonu,xml,serialize,var_export]返回数据格式，默认为json，jsonu输出的内容中文可以直接阅读
langString否[cn,en,ru]翻译语言，默认cn简体中文
versionString否API版本

请求参数

请求参数：num_iid=652874751412&is_promotion=1

参数说明：num_iid:淘宝商品ID
is_promotion:是否获取取促销价

响应示例

  1.  "item": {
  2.       "num_iid": "520813250866",
  3.       "title": "三刃木折叠刀过安检创意迷你钥匙扣钥匙刀军刀随身多功能小刀包邮",
  4.       "desc_short": "",
  5.       "price": 25.8,
  6.       "total_price": 0,
  7.       "suggestive_price": 0,
  8.       "orginal_price": "25.80",
  9.       "nick": "欢乐购客栈",
  10.       "num": "832",
  11.       "min_num": 0,
  12.       "detail_url": "http://item.taobao.com/item.htm?id=520813250866",
  13.       "pic_url": "//img.alicdn.com/imgextra/i4/2596264565/TB2p30elFXXXXXQXpXXXXXXXXXX_!!2596264565.jpg",
  14.       "brand": "三刃木",
  15.       "brandId": "4036703",
  16.       "rootCatId": "50013886",
  17.       "cid": "50014822",
  18.       "favcount": "4824",
  19.       "fanscount": "1469",
  20.       "crumbs": [],
  21.       "created_time": "",
  22.       "modified_time": "",
  23.       "delist_time": "",
  24.       "desc": "<p style=\"text-align:center;\"><span style=\"color:#ff0000;\"><span style=\"font-size:24.0px;\"><strong>小店所有产品都支持刻字，如需刻字，拍之前联系客服即可。</strong></span></span></p> <p style=\"text-align:center;\"><span style=\"font-size:18.0px;\"><strong>炎炎夏日来临，一把随身携带便携式的折叠小刀，<span style=\"color:#ff0000;\">带开瓶器功能</span>，喝酒不用愁。（钥匙刀不带开瓶器功能）</strong></span></p> <p style=\"text-align:center;\"><strong style=\"color:#000000;line-height:1.5;\"><span style=\"font-size:18.0px;\"><span style=\"line-height:18.0px;\">可以当吊牌项链装饰，也可当钥匙扣挂饰， 水果刀 开瓶器 户外防身.</span></span></strong></p> <p style=\"text-align:center;\"><strong style=\"line-height:1.5;\"><span style=\"font-size:24.0px;\">部分客户跟我们反映链子质量还不够好，因此小店已重新订购一批质量更好的链子赠送，<span style=\"color:#ff0000;\">加量不加价</span>，只为让你买的更舒心，戴的更放心。</span></strong></p> <p style=\"text-align:center;\"><span style=\"font-size:36.0px;\"><strong>购买就送<span style=\"color:#ff0000;\">工具刀卡和链子</span>一条</strong></span></p> <p style=\"text-align:center;\"><span style=\"font-size:36.0px;\"><strong><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i3/2596264565/TB2Sci2jXXXXXXFXpXXXXXXXXXX_!!2596264565.png\" size=\"611x442\"> </strong></span></p> <p style=\"text-align:center;\"><span style=\"font-size:18.0px;\"><strong>璀璨钻石套餐包含【SK016D钥匙扣+GJ019C折叠刀+GJ017D工具卡】</strong></span><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i3/2596264565/TB2wWohmXXXXXX8XXXXXXXXXXXX_!!2596264565.jpg\" size=\"750x600\" /></p> <p style=\"text-align:center;\"><span style=\"font-size:18.0px;\"><strong>超凡大师套餐包括【304不锈钢钥匙扣+GJ019C折叠刀+GJ017D工具卡】</strong></span><a href=\"//item.taobao.com/item.htm?id=520798805877\" target=\"_blank\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i4/2596264565/TB2_uiXnFXXXXXBXXXXXXXXXXXX_!!2596264565.jpg_q90.jpg\" style=\"float:none;margin:0.0px;\" /></a></p> <p style=\"text-align:center;\">&nbsp;</p> <p style=\"text-align:center;\"><span style=\"font-size:18.0px;\"><strong>最强王者套餐包括【钛钢钥匙扣+GJ019C折叠刀+GJ017D工具卡】<a href=\"//item.taobao.com/item.htm?id=520801586805\" target=\"_blank\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i4/2596264565/TB2Gm9xnFXXXXbmXXXXXXXXXXXX_!!2596264565.jpg_q90.jpg\" style=\"float:none;margin:0.0px;\" /></a> </strong></span></p> <p style=\"text-align:center;\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i3/2596264565/TB28Ox4b77OyuJjSsplXXXqdpXa_!!2596264565.jpg\" size=\"749x553\"> <img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i1/2596264565/TB2.mTddVXXXXbeXpXXXXXXXXXX_!!2596264565.jpg\" style=\"line-height:1.5;\" size=\"750x653\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i3/2596264565/TB21Ro.jl0lpuFjSszdXXcdxFXa_!!2596264565.jpg\" size=\"750x653\"> <img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i4/2596264565/TB2q9CelVXXXXcUXXXXXXXXXXXX_!!2596264565.jpg\" size=\"750x340\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i3/2596264565/TB21EX9lVXXXXaXXpXXXXXXXXXX_!!2596264565.jpg\" size=\"750x416\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i3/2596264565/TB2udCylVXXXXXgXXXXXXXXXXXX_!!2596264565.jpg_q90.jpg\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i1/2596264565/TB2EARxjB8lpuFjSspaXXXJKpXa_!!2596264565.jpg\" size=\"750x515\"> <img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i3/2596264565/TB2ssuwlVXXXXafXXXXXXXXXXXX_!!2596264565.jpg_q90.jpg\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i2/2596264565/TB2nAHqgyC9MuFjSZFoXXbUzFXa_!!2596264565.jpg\" size=\"750x606\"> <img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i3/2596264565/TB2ahCelVXXXXc_XXXXXXXXXXXX_!!2596264565.jpg_q90.jpg\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i3/2596264565/TB2w1JnjwRkpuFjy1zeXXc.6FXa_!!2596264565.jpg\" size=\"750x469\"> <img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i3/2596264565/TB2C902lVXXXXbnXpXXXXXXXXXX_!!2596264565.jpg_q90.jpg\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i1/2596264565/TB2sGR3lVXXXXblXpXXXXXXXXXX_!!2596264565.jpg_q90.jpg\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i1/2596264565/TB2ZBGxlVXXXXXMXXXXXXXXXXXX_!!2596264565.jpg_q90.jpg\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i1/2596264565/TB2MjWklVXXXXcaXXXXXXXXXXXX_!!2596264565.jpg_q90.jpg\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i2/2596264565/TB2UgV3lVXXXXbdXpXXXXXXXXXX_!!2596264565.jpg_q90.jpg\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i2/2596264565/TB2ip5XlVXXXXX2XpXXXXXXXXXX_!!2596264565.jpg_q90.jpg\"> <img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i2/2596264565/TB2sMTBdVXXXXXlXXXXXXXXXXXX_!!2596264565.jpg_q90.jpg\" style=\"line-height:1.5;\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i3/2596264565/TB2d3HfdVXXXXahXpXXXXXXXXXX_!!2596264565.jpg_q90.jpg\" style=\"line-height:1.5;\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i4/2596264565/TB2AVbBdVXXXXXkXXXXXXXXXXXX_!!2596264565.jpg_q90.jpg\" style=\"line-height:1.5;\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i2/2596264565/TB2nf_wdVXXXXaMXXXXXXXXXXXX_!!2596264565.jpg_q90.jpg\" style=\"line-height:1.5;\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i4/2596264565/TB2dLYddVXXXXbtXpXXXXXXXXXX_!!2596264565.jpg_q90.jpg\" style=\"line-height:1.5;\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i2/2596264565/TB2H1_adVXXXXbWXpXXXXXXXXXX_!!2596264565.jpg_q90.jpg\" style=\"line-height:1.5;\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i2/2596264565/TB2eBzsdVXXXXbuXXXXXXXXXXXX_!!2596264565.jpg_q90.jpg\" style=\"line-height:1.5;\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i4/2596264565/TB2dOTndVXXXXcCXXXXXXXXXXXX_!!2596264565.jpg_q90.jpg\" style=\"line-height:1.5;\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i2/2596264565/TB2fK2tdVXXXXbkXXXXXXXXXXXX_!!2596264565.jpg_q90.jpg\" style=\"line-height:1.5;\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i2/2596264565/TB29zjedVXXXXaFXpXXXXXXXXXX_!!2596264565.jpg_q90.jpg\" style=\"line-height:1.5;\"><img align=\"absmiddle\" src=\"//img.alicdn.com/imgextra/i3/2596264565/TB2i7rmdVXXXXcTXXXXXXXXXXXX_!!2596264565.jpg_q90.jpg\" style=\"line-height:1.5;\" /></p>",
  25.       "item_imgs": [
  26.         {
  27.           "url": "//img.alicdn.com/imgextra/i4/2596264565/TB2p30elFXXXXXQXpXXXXXXXXXX_!!2596264565.jpg"
  28.         },
  29.         {
  30.           "url": "//img.alicdn.com/imgextra/i2/2596264565/TB2onxRlVXXXXcDXpXXXXXXXXXX_!!2596264565.jpg"
  31.         },
  32.         {
  33.           "url": "//img.alicdn.com/imgextra/i1/2596264565/TB2a.x.lVXXXXXPXpXXXXXXXXXX_!!2596264565.jpg"
  34.         },
  35.         {
  36.           "url": "//img.alicdn.com/imgextra/i2/2596264565/TB2iWE6bKZkyKJjSszbXXblwFXa_!!2596264565.jpg"
  37.         },
  38.         {
  39.           "url": "//img.alicdn.com/imgextra/i4/2596264565/TB2j2cTXib_F1JjSZFzXXc6KXXa_!!2596264565.jpg"
  40.         }
  41.       ],
  42.       "item_weight": "0",
  43.       "item_size": "",
  44.       "location": "广东深圳",
  45.       "post_fee": "",
  46.       "express_fee": "0.00",
  47.       "ems_fee": "",
  48.       "shipping_to": "广东广州白云区",
  49.       "has_discount": "false",
  50.       "video": [],
  51.       "is_virtual": "",
  52.       "sample_id": "",
  53.       "is_promotion": "false",
  54.       "props_name": "1627207:1347647754:颜色分类:长方形带开瓶器+送工具刀卡+链子;1627207:1347647753:颜色分类:椭圆形带开瓶器+送工具刀卡+链子;1627207:1195392087:颜色分类:GJ018X钥匙刀+送工具刀卡+链子;1627207:1331112595:颜色分类:超凡大师套餐【送工具卡+链子】;1627207:1331112594:颜色分类:最强王者套餐【送工具卡+链子】;1627207:1331264247:颜色分类:璀璨钻石套餐【送工具卡+链子】",
  55.       "prop_imgs": {
  56.         "prop_img": [
  57.           {
  58.             "properties": "1627207:1347647754",
  59.             "url": "//img.alicdn.com/imgextra/i3/2596264565/TB2.XeblVXXXXXkXpXXXXXXXXXX_!!2596264565.jpg"
  60.           },
  61.           {
  62.             "properties": "1627207:1347647753",
  63.             "url": "//img.alicdn.com/imgextra/i4/2596264565/TB2dTrjdVXXXXXBXpXXXXXXXXXX_!!2596264565.jpg"
  64.           },
  65.           {
  66.             "properties": "1627207:1195392087",
  67.             "url": "//img.alicdn.com/imgextra/i2/2596264565/TB2j22kdVXXXXXdXpXXXXXXXXXX_!!2596264565.jpg"
  68.           },
  69.           {
  70.             "properties": "1627207:1331112595",
  71.             "url": "//img.alicdn.com/imgextra/i4/2596264565/TB2_uiXnFXXXXXBXXXXXXXXXXXX_!!2596264565.jpg"
  72.           },
  73.           {
  74.             "properties": "1627207:1331112594",
  75.             "url": "//img.alicdn.com/imgextra/i4/2596264565/TB2Gm9xnFXXXXbmXXXXXXXXXXXX_!!2596264565.jpg"
  76.           },
  77.           {
  78.             "properties": "1627207:1331264247",
  79.             "url": "//img.alicdn.com/imgextra/i3/2596264565/TB2wWohmXXXXXX8XXXXXXXXXXXX_!!2596264565.jpg"
  80.           }
  81.         ]
  82.       },
  83.       "property_alias": "1627207:1347647754:长方形带开瓶器+送工具刀卡+链子;1627207:1347647753:椭圆形带开瓶器+送工具刀卡+链子;1627207:1195392087:GJ018X钥匙刀+送工具刀卡+链子;1627207:1331112595:超凡大师套餐【送工具卡+链子】;1627207:1331112594:最强王者套餐【送工具卡+链子】;1627207:1331264247:璀璨钻石套餐【送工具卡+链子】",
  84.       "props": [
  85.         {
  86.           "name": "品牌",
  87.           "value": "三刃木"
  88.         },
  89.         {
  90.           "name": "产地",
  91.           "value": "中国"
  92.         },
  93.         {
  94.           "name": "颜色分类",
  95.           "value": "长方形带开瓶器+送工具刀卡+链子,椭圆形带开瓶器+送工具刀卡+链子,GJ018X钥匙刀+送工具刀卡+链子,超凡大师套餐【送工具卡+链子】,最强王者套餐【送工具卡+链子】,璀璨钻石套餐【送工具卡+链子】"
  96.         },
  97.         {
  98.           "name": "吊牌价",
  99.           "value": "46"
  100.         },
  101.         {
  102.           "name": "功能数量",
  103.           "value": "5个及以下"
  104.         },
  105.         {
  106.           "name": "货号",
  107.           "value": "GJ019C"
  108.         },
  109.         {
  110.           "name": "附加功能",
  111.           "value": "开瓶器,刀,螺丝刀,钥匙圈,其他"
  112.         }
  113.       ],
  114.       "total_sold": "1",
  115.       "skus": {
  116.         "sku": [
  117.           {
  118.             "price": "39",
  119.             "orginal_price": "39.00",
  120.             "properties": "1627207:1347647754",
  121.             "properties_name": "1627207:1347647754:颜色分类:长方形带开瓶器+送工具刀卡+链子",
  122.             "quantity": "104",
  123.             "sku_id": "3166598625985"
  124.           },
  125.           {
  126.             "price": "39",
  127.             "orginal_price": "39.00",
  128.             "properties": "1627207:1347647753",
  129.             "properties_name": "1627207:1347647753:颜色分类:椭圆形带开瓶器+送工具刀卡+链子",
  130.             "quantity": "370",
  131.             "sku_id": "3166598625984"
  132.           },
  133.           {
  134.             "price": "25.8",
  135.             "orginal_price": "25.80",
  136.             "properties": "1627207:1195392087",
  137.             "properties_name": "1627207:1195392087:颜色分类:GJ018X钥匙刀+送工具刀卡+链子",
  138.             "quantity": "34",
  139.             "sku_id": "3144644292458"
  140.           },
  141.           {
  142.             "price": "73.8",
  143.             "orginal_price": "73.80",
  144.             "properties": "1627207:1331112595",
  145.             "properties_name": "1627207:1331112595:颜色分类:超凡大师套餐【送工具卡+链子】",
  146.             "quantity": "157",
  147.             "sku_id": "3161300228970"
  148.           },
  149.           {
  150.             "price": "91.8",
  151.             "orginal_price": "91.80",
  152.             "properties": "1627207:1331112594",
  153.             "properties_name": "1627207:1331112594:颜色分类:最强王者套餐【送工具卡+链子】",
  154.             "quantity": "0",
  155.             "sku_id": "3161300228969"
  156.           },
  157.           {
  158.             "price": "63.8",
  159.             "orginal_price": "63.80",
  160.             "properties": "1627207:1331264247",
  161.             "properties_name": "1627207:1331264247:颜色分类:璀璨钻石套餐【送工具卡+链子】",
  162.             "quantity": "167",
  163.             "sku_id": "3161107666655"
  164.           }
  165.         ]
  166.       },
  167.       "seller_id": "2596264565",
  168.       "sales": "1",
  169.       "shop_id": "127203758",
  170.       "props_list": {
  171.         "1627207:1347647754": "颜色分类:长方形带开瓶器+送工具刀卡+链子",
  172.         "1627207:1347647753": "颜色分类:椭圆形带开瓶器+送工具刀卡+链子",
  173.         "1627207:1195392087": "颜色分类:GJ018X钥匙刀+送工具刀卡+链子",
  174.         "1627207:1331112595": "颜色分类:超凡大师套餐【送工具卡+链子】",
  175.         "1627207:1331112594": "颜色分类:最强王者套餐【送工具卡+链子】",
  176.         "1627207:1331264247": "颜色分类:璀璨钻石套餐【送工具卡+链子】"
  177.       },
  178.       "seller_info": {
  179.         "title": "欢乐购客栈",
  180.         "shop_name": "欢乐购客栈",
  181.         "sid": "127203758",
  182.         "zhuy": "//shop127203758.taobao.com",
  183.         "level": "12",
  184.         "shop_type": "C",
  185.         "user_num_id": "2596264565",
  186.         "nick": "欢乐购客栈",
  187.         "cid": null,
  188.         "delivery_score": "4.8 ",
  189.         "item_score": "4.8 ",
  190.         "score_p": "4.8 "
  191.       },
  192.       "tmall": "false",
  193.       "error": "",
  194.       "warning": "",
  195.       "url_log": [],
  196.       "stuff_status": "",
  197.       "shopinfo": {
  198.         "shop_name": "欢乐购客栈",
  199.         "shop_id": "127203758"
  200.       },
  201.       "data_from": "mtop",
  202.       "method": "item_mtop:item_get",
  203.       "promo_type": null,
  204.       "props_img": {
  205.         "1627207:1347647754": "//img.alicdn.com/imgextra/i3/2596264565/TB2.XeblVXXXXXkXpXXXXXXXXXX_!!2596264565.jpg",
  206.         "1627207:1347647753": "//img.alicdn.com/imgextra/i4/2596264565/TB2dTrjdVXXXXXBXpXXXXXXXXXX_!!2596264565.jpg",
  207.         "1627207:1195392087": "//img.alicdn.com/imgextra/i2/2596264565/TB2j22kdVXXXXXdXpXXXXXXXXXX_!!2596264565.jpg",
  208.         "1627207:1331112595": "//img.alicdn.com/imgextra/i4/2596264565/TB2_uiXnFXXXXXBXXXXXXXXXXXX_!!2596264565.jpg",
  209.         "1627207:1331112594": "//img.alicdn.com/imgextra/i4/2596264565/TB2Gm9xnFXXXXbmXXXXXXXXXXXX_!!2596264565.jpg",
  210.         "1627207:1331264247": "//img.alicdn.com/imgextra/i3/2596264565/TB2wWohmXXXXXX8XXXXXXXXXXXX_!!2596264565.jpg"
  211.       },
  212.       "rate_grade": "",
  213.       "shop_item": [],
  214.       "relate_items": []
  215.     },

 

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/秋刀鱼在做梦/article/detail/848668
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号