赞
踩
今年5月底开始,谷歌彻底被墙,所有谷歌的网站都不能访问了,这次包括了android.org,googlesource.com,code.google.com。Android官方的资源不能访问,想下载Android代码当然是困难重重了。
本文就为大家解决这一问题——介绍如何使从github.com/andorid下载Andorid最新源码。俗话说“授之以鱼不如授之以渔”,本文不只是给出一个已经打包好的源码的链接,而是重现一下我是如何下载的,希望对大家遇到的类似的问题能有所帮助。
打开github.com/android可以看到:
中间黑体列出的的是各个项目的链接。github.com/android上托管的Android源码是用git单独管理每个项目的,而没有像googlesource.com那样使用repo(谷歌开发的基于git的命令行工具)管理所有项目。所以,如果你只想下载一个项目的源码,可以单独下载(比如开发App想跟一下SDK的一些源码可以下载platform_frameworks_base项目)。
用过git的同学当然知道,可以使用下面的命令获取远程仓库的代码:
1.
git clone [仓库URL]
当然,如果只是要配置“App源码调试”环境,可以点“Download ZIP”按钮,下载zip格式的源码包,然后在Eclipse调试的时候选择下载好的源码包。
这里在是本文的重点——批量下载github上的android源码。
根据单独下载一个项目的过程,下载所有项目的源码可以有两种方法:
得到所有项目的git仓库URL得到所有项目的ZIP包下载链接接下来的问题是如何得到这样的URL?
当然是从网页上(也就是页面的HTML文件上)!
现在问题就是如何得到这些网页?
我瞬间想到的办法就是——用CURL
ps: CURL是一个命令行上传下载工具,支持多种协议包括HTTP, HTTPS, FTP等。
没有curl的同学可以用如下命令下载:
1.
sudo apt-get install curl
有了CURL用来抓页面,就不愁得不到仓库URL或ZIP包URL了:
首先要抓取github.com/android的页面,再“顺藤摸瓜”地抓取各个项目的页面,再从各项目的页面上抓取仓库URL或ZIP包URL。
在github.com/android页面的下方可以看github.com/andoid一共分5页,点击后面几个页面,可以发现这几个页面的URL形式一致为:
1.
https:
//github.com/android?page=N
由此,可以用bash脚本轻易写出抓取这5个页面的代码(可以直接在终端里执行下面的命令):
1.
for
((i=
1
; i<=
5
; i++));
do
2.
curl https:
//github.com/android?page=$i > android.github.com.$i.html;
3.
done
再看项目的首页URL和仓库URL可以发现:
仓库URL = 首页URL + .git
比如,platform_frameworks_base的首页URL是:
1.
https:
//github.com/android/platform_frameworks_base
1.
https:
//github.com/android/platform_frameworks_base.git
1.
git:
//github.com/android/platform_frameworks_base.git
现在的问题就是——如何从这些页面上得到仓库的URL了。
可以通过浏览器的“审查元素”菜单快速找到页面上一个链接对应的HTML代码块,如下图:
可以看到platform_frameworks_base项目的超链接(<a href="xxx">)被一个<h3 class="repo-list-name">标签包围,粗略浏览一下刚才下载的几个子页面,可以看出每个项目首页的链接都是如此(在<h3 class="repo-list-name">的下一行)。
根据刚才找到的两个规律:
项目首页URL总是出现在<h3 class="repo-list-name">的下一行仓库URL = 首页URL + .git
可以写出解析各个项目首页(仓库URL)的python代码(getGitRepos.py):
01.
#!/usr/bin/python
02.
import
sys
03.
04.
flag = False
05.
link_prefix =
'https://github.com'
# 如果使用ssh协议下载,可将https改为git
06.
07.
allLines = sys.stdin.readlines()
08.
for
curLine in allLines:
09.
if
curLine.find(
'repo-list-name'
) >=
0
:
10.
flag = True
11.
if
flag:
12.
pos = curLine.find(
'href="'
)
13.
if
pos >=
0
:
14.
pos += len(
'href="'
)
15.
last = curLine[pos:]
16.
end = last.find(
'"'
)
17.
link = last[:end]
18.
19.
# name to path.
20.
name = link[link.rfind(
'/'
)+
1
:]
21.
prefix = name.find(
'platform_'
)
22.
if
prefix >=
0
:
23.
name = name[len(
'platform_'
):] # ignore platform_
24.
path = name.replace(
'_'
,
'/'
)
25.
26.
link = link_prefix + link
27.
# print
'curl'
, link,
'>'
, name
28.
# https:
//github.com/android/platform_external_qemu.git
29.
print
'git clone'
, link +
'.git'
, path # 输出 git clone 命令
30.
flag = False
这段python程序从标准输入读入文本并解析,实际使用时可用重定向操作符实现,实现用之前抓取的github.com/android的几个子页面替代标准输入。
程序输出的git clone命令的path参数会忽略仓库名称前面的platform_前缀,这使得最终下载下来的源码结构和repo sync下来的类似。
可以先用一个页面测试一下,看看解析出来的URL对不对(上面的python代码就是这样改正确的),终端输入下面命令:
1.
./getGitRepos.py < android.github.com.
1
.html
01.
git clone https:
//github.com/android/platform_dalvik.git dalvik
02.
git clone https:
//github.com/android/platform_bionic.git bionic
03.
git clone https:
//github.com/android/platform_system_core.git system/core
04.
git clone https:
//github.com/android/kernel_common.git kernel/common
05.
git clone https:
//github.com/android/platform_external_qemu.git external/qemu
06.
git clone https:
//github.com/android/platform_build.git build
07.
git clone https:
//github.com/android/platform_development.git development
08.
git clone https:
//github.com/android/platform_frameworks_base.git frameworks/base
09.
git clone https:
//github.com/android/platform_manifest.git manifest
10.
git clone https:
//github.com/android/platform_frameworks_support.git frameworks/support
11.
git clone https:
//github.com/android/platform_packages_apps_settings.git packages/apps/settings
12.
git clone https:
//github.com/android/platform_external_dhcpcd.git external/dhcpcd
13.
git clone https:
//github.com/android/platform_external_webkit.git external/webkit
14.
git clone https:
//github.com/android/platform_external_protobuf.git external/protobuf
15.
git clone https:
//github.com/android/platform_packages_providers_mediaprovider.git packages/providers/mediaprovider
16.
git clone https:
//github.com/android/platform_external_elfutils.git external/elfutils
17.
git clone https:
//github.com/android/platform_external_strace.git external/strace
18.
git clone https:
//github.com/android/platform_hardware_libhardware.git hardware/libhardware
19.
git clone https:
//github.com/android/platform_external_tinyxml.git external/tinyxml
20.
git clone https:
//github.com/android/platform_external_oprofile.git external/oprofile
测试基本上出现问题,因为getGitRepos.py输出的是git clone命令,执行之后在能执行实际的下载动作;所以,需要将这些输出保存下来;
使用如下命令可将每个子页面解析出的git clone命令保存到一个文本文件内:
1.
for
page in `ls android.github.com.*`;
do
2.
echo parse $page
3.
./getGitRepos.py < $page >> gitRepos.txt;
4.
done
有了写满git clone命令的gitRepos.txt后,可以使用如下命令执行之:
1.
sh gitRepos.txt
为了方便广大网友使用,我已将上面需要用的命令写道了下面的cloneRepos.sh:
01.
#!/bin/bash
02.
# clone AOSP each repo from https:
//github.com/android
03.
#
this
shell depends on git, curl and python
2
.
04.
# INSTALL depends:
05.
# sudo apt-get install git curl python
06.
#
07.
# by xu(xusiwei1236
@163
.com).
08.
09.
# download github AOSP sub pages
10.
for
((i=
1
; i<=
5
; i++));
do
11.
echo get AOSP subpage https:
//github.com/android?page=$i
12.
curl https:
//github.com/android?page=$i > android.github.com.$i.html;
13.
done
14.
15.
# parse git repo URL from each sub pages
16.
mkdir repos
17.
cat /dev/
null
> gitRepos.txt
18.
for
page in `ls android.github.com.*`;
do
19.
echo parse $page
20.
./getGitRepos.py < $page >> gitRepos.txt;
21.
done
22.
23.
# clone repos
24.
sh gitRepos.txt
当然,如果觉得自己运行麻烦,可以直接copy生成的gitRepos.txt,直接粘贴到终端里执行之,即可下载全部Android源码:
01.
git clone https:
//github.com/android/platform_dalvik.git dalvik
02.
git clone https:
//github.com/android/platform_bionic.git bionic
03.
git clone https:
//github.com/android/platform_system_core.git system/core
04.
git clone https:
//github.com/android/kernel_common.git kernel/common
05.
git clone https:
//github.com/android/platform_external_qemu.git external/qemu
06.
git clone https:
//github.com/android/platform_build.git build
07.
git clone https:
//github.com/android/platform_development.git development
08.
git clone https:
//github.com/android/platform_frameworks_base.git frameworks/base
09.
git clone https:
//github.com/android/platform_manifest.git manifest
10.
git clone https:
//github.com/android/platform_frameworks_support.git frameworks/support
11.
git clone https:
//github.com/android/platform_packages_apps_settings.git packages/apps/settings
12.
git clone https:
//github.com/android/platform_external_dhcpcd.git external/dhcpcd
13.
git clone https:
//github.com/android/platform_external_webkit.git external/webkit
14.
git clone https:
//github.com/android/platform_external_protobuf.git external/protobuf
15.
git clone https:
//github.com/android/platform_packages_providers_mediaprovider.git packages/providers/mediaprovider
16.
git clone https:
//github.com/android/platform_external_elfutils.git external/elfutils
17.
git clone https:
//github.com/android/platform_external_strace.git external/strace
18.
git clone https:
//github.com/android/platform_hardware_libhardware.git hardware/libhardware
19.
git clone https:
//github.com/android/platform_external_tinyxml.git external/tinyxml
20.
git clone https:
//github.com/android/platform_external_oprofile.git external/oprofile
21.
git clone https:
//github.com/android/platform_external_neven.git external/neven
22.
git clone https:
//github.com/android/platform_external_netperf.git external/netperf
23.
git clone https:
//github.com/android/platform_external_netcat.git external/netcat
24.
git clone https:
//github.com/android/platform_external_libxml2.git external/libxml2
25.
git clone https:
//github.com/android/platform_external_libpng.git external/libpng
26.
git clone https:
//github.com/android/platform_external_libpcap.git external/libpcap
27.
git clone https:
//github.com/android/platform_external_libffi.git external/libffi
28.
git clone https:
//github.com/android/platform_external_jpeg.git external/jpeg
29.
git clone https:
//github.com/android/platform_external_jhead.git external/jhead
30.
git clone https:
//github.com/android/platform_external_jdiff.git external/jdiff
31.
git clone https:
//github.com/android/platform_external_iptables.git external/iptables
32.
git clone https:
//github.com/android/platform_external_icu4c.git external/icu4c
33.
git clone https:
//github.com/android/platform_external_giflib.git external/giflib
34.
git clone https:
//github.com/android/platform_external_freetype.git external/freetype
35.
git clone https:
//github.com/android/platform_external_fdlibm.git external/fdlibm
36.
git clone https:
//github.com/android/platform_external_expat.git external/expat
37.
git clone https:
//github.com/android/platform_external_esd.git external/esd
38.
git clone https:
//github.com/android/platform_external_emma.git external/emma
39.
git clone https:
//github.com/android/platform_external_dropbear.git external/dropbear
40.
git clone https:
//github.com/android/platform_external_apache-http.git external/apache-http
41.
git clone https:
//github.com/android/platform_external_openssl.git external/openssl
42.
git clone https:
//github.com/android/platform_external_sonivox.git external/sonivox
43.
git clone https:
//github.com/android/platform_packages_providers_telephonyprovider.git packages/providers/telephonyprovider
44.
git clone https:
//github.com/android/platform_packages_providers_downloadprovider.git packages/providers/downloadprovider
45.
git clone https:
//github.com/android/platform_packages_providers_contactsprovider.git packages/providers/contactsprovider
46.
git clone https:
//github.com/android/platform_packages_providers_calendarprovider.git packages/providers/calendarprovider
47.
git clone https:
//github.com/android/platform_packages_apps_voicedialer.git packages/apps/voicedialer
48.
git clone https:
//github.com/android/platform_packages_apps_stk.git packages/apps/stk
49.
git clone https:
//github.com/android/platform_packages_apps_soundrecorder.git packages/apps/soundrecorder
50.
git clone https:
//github.com/android/platform_packages_apps_phone.git packages/apps/phone
51.
git clone https:
//github.com/android/platform_packages_apps_packageinstaller.git packages/apps/packageinstaller
52.
git clone https:
//github.com/android/platform_packages_apps_music.git packages/apps/music
53.
git clone https:
//github.com/android/platform_packages_apps_mms.git packages/apps/mms
54.
git clone https:
//github.com/android/platform_packages_apps_htmlviewer.git packages/apps/htmlviewer
55.
git clone https:
//github.com/android/platform_packages_apps_email.git packages/apps/email
56.
git clone https:
//github.com/android/platform_packages_apps_contacts.git packages/apps/contacts
57.
git clone https:
//github.com/android/platform_packages_apps_camera.git packages/apps/camera
58.
git clone https:
//github.com/android/platform_packages_apps_calendar.git packages/apps/calendar
59.
git clone https:
//github.com/android/platform_packages_apps_calculator.git packages/apps/calculator
60.
git clone https:
//github.com/android/platform_packages_apps_browser.git packages/apps/browser
61.
git clone https:
//github.com/android/platform_hardware_ril.git hardware/ril
62.
git clone https:
//github.com/android/platform_external_zlib.git external/zlib
63.
git clone https:
//github.com/android/platform_external_yaffs2.git external/yaffs2
64.
git clone https:
//github.com/android/platform_external_tcpdump.git external/tcpdump
65.
git clone https:
//github.com/android/platform_external_tagsoup.git external/tagsoup
66.
git clone https:
//github.com/android/platform_external_srec.git external/srec
67.
git clone https:
//github.com/android/platform_external_sqlite.git external/sqlite
68.
git clone https:
//github.com/android/platform_external_skia.git external/skia
69.
git clone https:
//github.com/android/platform_external_safe-iop.git external/safe-iop
70.
git clone https:
//github.com/android/platform_external_ppp.git external/ppp
71.
git clone https:
//github.com/android/kernel_msm.git kernel/msm
72.
git clone https:
//github.com/android/platform_system_bluetooth.git system/bluetooth
73.
git clone https:
//github.com/android/platform_prebuilt.git prebuilt
74.
git clone https:
//github.com/android/platform_packages_providers_drmprovider.git packages/providers/drmprovider
75.
git clone https:
//github.com/android/platform_external_ping.git external/ping
76.
git clone https:
//github.com/android/platform_external_dbus.git external/dbus
77.
git clone https:
//github.com/android/tools_repo.git tools/repo
78.
git clone https:
//github.com/android/android.github.io.git android.github.io
79.
git clone https:
//github.com/android/platform_external_wpa_supplicant.git external/wpa/supplicant
80.
git clone https:
//github.com/android/platform_packages_apps_googlesearch.git packages/apps/googlesearch
81.
git clone https:
//github.com/android/platform_system_wlan_ti.git system/wlan/ti
82.
git clone https:
//github.com/android/platform_packages_providers_googlecontactsprovider.git packages/providers/googlecontactsprovider
83.
git clone https:
//github.com/android/platform_packages_apps_im.git packages/apps/im
84.
git clone https:
//github.com/android/platform_packages_providers_improvider.git packages/providers/improvider
85.
git clone https:
//github.com/android/platform_external_tremor.git external/tremor
86.
git clone https:
//github.com/android/platform_packages_apps_sync.git packages/apps/sync
87.
git clone https:
//github.com/android/platform_packages_apps_alarmclock.git packages/apps/alarmclock
88.
git clone https:
//github.com/android/platform_external_aes.git external/aes
89.
git clone https:
//github.com/android/platform_external_clearsilver.git external/clearsilver
90.
git clone https:
//github.com/android/platform_external_googleclient.git external/googleclient
91.
git clone https:
//github.com/android/platform_packages_apps_launcher.git packages/apps/launcher
92.
git clone https:
//github.com/android/platform_external_bluez.git external/bluez
93.
git clone https:
//github.com/android/platform_external_gdata.git external/gdata
94.
git clone https:
//github.com/android/platform_frameworks_opt_com.google.android.git frameworks/opt/com.google.android
95.
git clone https:
//github.com/android/platform_external_elfcopy.git external/elfcopy
96.
git clone https:
//github.com/android/platform_frameworks_policies_base.git frameworks/policies/base
97.
git clone https:
//github.com/android/platform_packages_apps_updater.git packages/apps/updater
98.
git clone https:
//github.com/android/platform_external_opencore.git external/opencore
Copyright © 2003-2013 www.wpsshop.cn 版权所有,并保留所有权利。