赞
踩
作者:张华 发表于:2014-05-15
版权声明:可以任意转载,转载时请务必以超链接形式标明文章原始出处和作者信息及本版权声明 (http://blog.csdn.net/quqi99 )
镜像是在kvm上做的,做完转成vmware镜像的方法如下:
qemu-img convert -f qcow2 -O vmdk o7k.qcow2 o7k.vmdk
可在win10上下载vmware player (player是免费版), 如 VMware-player-16.1.2-17966106.exe - https://www.vmware.com/cn/products/workstation-player.html
vmware player免费版没有直接导入镜像的功能,但可以这样导入:
firmware="efi"
sudo mkdir -p /bak/openstack && sudo chown -R $(whoami) /bak/openstack && cd /bak/openstack
sudo chown -R $(whoami) ~/.config
git clone https://github.com/openstack/devstack.git
cd devstack && wget https://raw.githubusercontent.com/zhhuabj/exercise/master/o7k/local.conf
#sudo systemctl stop "devstack@*"
#./clean.sh
./stack.sh
source accrc/admin/admin
如果网速很慢,可由一人搭建apt-cacher-ng server, 其他人用它来避免使用apt-get时的网络拥挤。
sudo apt install apt-cacher-ng -y
echo 'PassThroughPattern: .*' |sudo tee -a /etc/apt-cacher-ng/acng.conf
sudo systemctl restart apt-cacher-ng.service && sudo systemctl enable apt-cacher-ng.service
du -sh /var/cache/apt-cacher-ng/
#vim /var/lib/dpkg/info/apt-cacher-ng.postinst
#dpkg --configure apt-cacher-ng
#Use apt cache proxy
echo 'Acquire::http::Proxy "http://proxy:3142";' | sudo tee /etc/apt/apt.conf.d/01acng
如果想要换目录的话才需要
#change the dir from /var/cache/apt-cacher-ng/ to /mnt/udisk/apt-cacher-ng
cat << EOF |sudo tee -a /etc/fstab
#use blkid to see uuid
UUID="d63d7251-ec3d-4ef5-aa92-f3d4c480f20c" /mnt/udisk ext4 defaults 0 2
EOF
mkfs.ext4 -F -L udisk /dev/sdb1
mkdir /mnt/udisk/apt-cacher-ng
chown -R apt-cacher-ng:apt-cacher-ng /mnt/udisk/apt-cacher-ng
sudo sed -i 's/CacheDir: \/var\/cache\/apt-cacher-ng/CacheDir: \/mnt\/udisk\/apt-cacher-ng/g' /etc/apt-cacher-ng/acng.conf
du -sh /mnt/udisk/apt-cacher-ng
对于加速pypi, 可使用devpi缓存
#https://opensource.com/article/18/7/setting-devpi sudo apt install python3-pip -y sudo pip install devpi-server devpi-web devpi-init --serverdir /bak/work/devpi devpi-server --serverdir /bak/work/devpi --host 0.0.0.0 pip install --trusted-host 192.168.2.6 -i http://192.168.2.6:3141/root/pypi/+simple/ httpie export PIP_INDEX_URL=http://192.168.2.6:3141/root/pypi/+simple/ pip install httpie unset PIP_INDEX_URL mkdir -p ~/.pip && cat > ~/.pip/pip.conf << EOF [global] index-url = http://192.168.2.6:3141/root/pypi/+simple/ [search] index = http://192.168.2.6:3141/root/pypi/ [install] trusted-host=192.168.2.6 disable-pip-version-check = true timeout = 120 EOF
也可以使用阿里等第三方的:
mkdir -p ~/.pip
cat << EOF |tee ~/.pip/pip.conf
[global]
trusted-host=mirrors.aliyun.com
index-url = http://mirrors.aliyun.com/pypi/simple
disable-pip-version-check = true
timeout = 120
EOF
或者在devstack中添加: PYPI_ALTERNATIVE_URL=http://mirrors.aliyun.com/pypi/simple
例如,在win10上使用hypver-v创建一个7G(初始2G并动态分配即可),40G硬件的ubuntu 20.04的名为o7k的虚机, 禁用check point减小硬盘使用:
# stop vm first
diskpart
select vdisk file="D:\images\o7k.vhdx"
compact vdisk
#compress it by winrar, then move it to D:\www\sftp, then we can download it by: sftp ftpuser@192.168.2.9X
unrar x o7k.rar
qemu-img convert o7k.vhdx -O qcow2 o7k.qcow2 #kvm
qemu-img convert o7k.vhdx -O raw o7k.raw
qemu-img convert o7k.vhdx -O vmdk o7k.vmdk #vmware
qemu-img convert o7k.vhdx -O vdi o7k.vdi #virtualbox
例如,使用kvm来使用o7k.qcow2时,会遇到"booting from hard disk "错误,那是因为hyper-v是默认使用UEFI来创建虚机的,那应该让KVM也支持UEFI:
sudo apt install ovmf && sudo systemctl restart libvirtd
ls /usr/share/OVMF/OVMF_CODE.fd
然后在virt-manager安前装在"概况"中将Firmware选择"UEFI x86_64: /usr/shar/OVMF/OVMF_CODE.fd"即可.
ssh登录进虚机创建stack用户,此步可选, 注意,若添加了stack用户, 记得运行’sudo su - stack’之后做pip相关的设置:
echo 'ubuntu ALL=(ALL) NOPASSWD:ALL' |sudo tee -a /etc/sudoers
echo 'precedence ::ffff:0:0/96 100' |sudo tee -a /etc/gai.conf
# don't add # when coping via vim
echo 'autocmd FileType * setlocal formatoptions-=c formatoptions-=r formatoptions-=o' |tee ~/.vimrc
sudo useradd -s /bin/bash -d /opt/stack -m stack
echo "stack ALL=(ALL) NOPASSWD: ALL" | sudo tee /etc/sudoers.d/stack
sudo su - stack && cd /opt/stack
NOTE: 设置镜像太容易引发各种奇奇怪怪的包依赖问题,我本次未使用各种镜像.
sudo cp /etc/apt/sources.list /etc/apt/sources.list_bak
sudo sed -i s/archive.ubuntu.com/cn.archive.ubuntu.com/g /etc/apt/sources.list
sudo apt clean all
sudo apt update
ERROR: Could not find a version that satisfies the requirement cliff===3.8.0 (from -c /bak/openstack/requirements/upper-constraints.txt (line 249)) (from versions: none)
ERROR: No matching distribution found for cliff===3.8.0 (from -c /bak/openstack/requirements/upper-constraints.txt (line 249))
所以我此次实际未使用镜像.附录1有本次使用douban镜像的步骤,仅供参考:
#NOTE: this can cause bastion unable to be SSHed #juju deploy ubuntu openstack --series=focal --config hostname=openstack --constraints "mem=8G cores=2 root-disk=60G" sudo mkdir -p /bak/openstack && sudo chown -R $(whoami) /bak/openstack && cd /bak/openstack git clone https://github.com/openstack/devstack.git && cd devstack #wget https://raw.githubusercontent.com/zhhuabj/exercise/master/o7k/local.conf sudo bash -c 'cat > local.conf' <<EOF [[local|localrc]] #OFFLINE=True DEST=/bak/openstack LIBVIRT_TYPE=qemu mkdir -p $DEST unset http_proxy unset https_proxy #sudo route del -net 10.0.1.0/24 gw 192.168.101.3 > /dev/null 2>&1 sudo apt -y install openvswitch-switch qemu-kvm libvirt-clients libvirt-daemon-system bridge-utils #sudo virsh net-destroy default sudo ovs-vsctl del-br br-data sudo ovs-vsctl -- --may-exist add-br br-data sudo ip addr add 172.16.1.1/24 dev br-data >/dev/null 2>&1 & sudo chown -R $(whoami) /opt/stack/ #sudo ovs-vsctl -- --may-exist add-port br-data eth0 -- set interface eth0 type=internal # ENABLED_SERVICES=n-cpu,rabbit,neutron,q-agt GIT_BASE="https://git.openstack.org" #enable_plugin neutron-lbaas https://git.openstack.org/openstack/neutron-lbaas ENABLED_SERVICES=rabbit,mysql,key,g-api,g-reg ENABLED_SERVICES+=,n-api,n-crt,n-obj,n-cpu,n-cond,n-sch,n-cauth,n-novnc,placement-api,n-super-cond,n-cond-cell1 ENABLED_SERVICES+=,q-svc,q-agt,q-dhcp,q-l3,q-meta,neutron disable_service q-metering Q_USE_ROOTWRAP=True Q_USE_ROOTWRAP_DAEMON=False ENABLE_IDENTITY_V2=False enable_service placement-api heartbeat_timeout_threshold=7200 Q_ML2_TENANT_NETWORK_TYPE=flat,vlan,gre,vxlan Q_ML2_PLUGIN_TYPE_DRIVERS=flat,vlan,gre,vxlan # must use public OVS_BRIDGE_MAPPINGS=public:br-data PUBLIC_NETWORK_NAME=public # Neutron Agent configuration Q_USE_NAMESPACE=True Q_ALLOW_OVERLAPPING_IP=True # Neutron ML2 configuration Q_PLUGIN=ml2 #Q_DVR_MODE=dvr_snat Q_DVR_MODE=legacy # VLAN ENABLE_TENANT_VLANS=True ML2_VLAN_RANGES=public:1000:2999 # GRE ENABLE_TENANT_TUNNELS=True TENANT_TUNNEL_RANGE=1000:2000 Q_ML2_PLUGIN_VXLAN_TYPE_OPTIONS=(vni_ranges=1000:2000) # Avanced ML2 configuration Q_AGENT=openvswitch Q_ML2_PLUGIN_MECHANISM_DRIVERS=openvswitch,linuxbridge,l2population # OVN #enable_plugin networking-ovn http://git.openstack.org/openstack/networking-ovn #enable_service ovn-northd #enable_service ovn-controller # We have to disable the neutron L2 agent. OVN does not use the L2 agent. #disable_service q-agt # Swift #ENABLED_SERVICES+=,s-proxy,s-object,s-container,s-account #sudo apt -y install libapache2-mod-wsgi VOLUME_BACKING_FILE_SIZE=500M SWIFT_HASH=66a3d6b56c1f479c8b4e70ab5c2000f5 SWIFT_REPLICAS=1 SWIFT_DATA_DIR=$DEST/data/swift NOVA_VNC_ENABLED=True # Ceph sudo rm -rf /var/lib/ceph/mon/ceph-`hostname` #ENABLED_SERVICES+=,ceph CEPH_LOOPBACK_DISK_SIZE=3G CEPH_CONF=/etc/ceph/ceph.conf CEPH_REPLICAS=1 # Cinder - Block Device Service #ENABLED_SERVICES+=,cinder,c-api,c-vol,c-sch,c-bak CINDER_DRIVER=ceph CINDER_CEPH_USER=cinder CINDER_CEPH_POOL=volumes CINDER_CEPH_UUID=6d52eb95-12f3-47e3-9eb9-0c1fe4142426 CINDER_BAK_CEPH_POOL=backeups CINDER_BAK_CEPH_USER=cind-backeups CINDER_ENABLED_BACKENDS=ceph,lvm GLANCE_CEPH_USER=glance GLANCE_CEPH_POOL=images NOVA_CEPH_POOL=vmz HOST_IP=172.16.1.1 SERVICE_HOST=$HOST_IP MYSQL_HOST=$SERVICE_HOST #QPID_HOST=$SERVICE_HOST RABBIT_HOST=$SERVICE_HOST GLANCE_HOSTPORT=$SERVICE_HOST:9292 Q_HOST=$SERVICE_HOST FIXED_RANGE=10.0.1.0/24 #113 -- 118, http://jodies.de/ipcalc FLOATING_RANGE=192.168.101.0/24 Q_FLOATING_ALLOCATION_POOL=start=192.168.101.3,end=192.168.101.100 PUBLIC_NETWORK_GATEWAY=192.168.101.1 NETWORK_GATEWAY=10.0.1.1 PUBLIC_BRIDGE=br-data # sudo ovs-vsctl add-port br-ex eth0 OVS_PHYSICAL_BRIDGE=br-data # https://review.openstack.org/#/c/87987/ IP_VERSION=4 DATABASE_USER=root DATABASE_PASSWORD=password ADMIN_PASSWORD=password SERVICE_PASSWORD=password RABBIT_PASSWORD=password SERVICE_TOKEN=ADMIN LOGFILE=$DEST/logs/stack.log ENABLE_DEBUG_LOG_LEVEL=False SYSLOG=False VERBOSE=True SCREEN_LOGDIR=$DEST/logs LOG_COLOR=False Q_USE_DEBUG_COMMAND=False KEYSTONE_DEPLOY=mod_wsgi USE_SSL=False disable_service tls-proxy USE_PYTHON3=True DOWNLOAD_DEFAULT_IMAGES=False IMAGE_URLS="http://download.cirros-cloud.net/0.5.2/cirros-0.5.2-x86_64-disk.img" #IPSEC_PACKAGE=strongswan # don't use trystack git mirror, but just use noVNC from trystack #GIT_BASE=http://git.trystack.cn # git call failed: [git clone' https://github.com/novnc/noVNC.git NOVNC_REPO=http://git.trystack.cn/kanaka/noVNC.git #SPICE_REPO=http://git.trystack.cn/git/spice/sice-html5.gi disable_service etcd3 disable_service memory_tracker #devstack NET_ID 'Failure creating NET_ID for private #NEUTRON_CREATE_INITIAL_NETWORKS=False #HORIZON_REPO=https://github.com/openstack/horizon #HORIZON_BRANCH=stable/grizzly #CINDER_REPO=https://github.com/openstack/cinder.git #CINDER_BRANCH=stable/grizzly #CINDERCLIENT_REPO=https://github.com/openstack/python-cinderclient.git #CINDERCLIENT_BRANCH=stable/grizzly #NOVA_REPO=https://github.com/openstack/nova.git #NOVA_BRANCH=stable/grizzly #NOVACLIENT_REPO=https://github.com/openstack/python-novaclient.git #NOVACLIENT_BRANCH=stable/grizzly #GLANCE_REPO=https://github.com/openstack/glance.git #GLANCE_BRANCH=stable/grizzly #GLANCECLIENT_REPO=https://github.com/openstack/python-glanceclient.git #KEYSTONE_REPO=https://github.com/openstack/keystone.git #KEYSTONE_BRANCH=stable/grizzly #KEYSTONECLIENT_REPO=https://github.com/openstack/python-keystoneclient.git #KEYSTONECLIENT_BRANCH=stable/grizzly #NEUTRON_REPO=https://github.com/openstack/neutron.git #NEUTRON_BRANCH=stable/grizzly #NEUTRONCLIENT_REPO=https://github.com/openstack/python-neutronclient.git #NEUTRONCLIENT_BRANCH=stable/grizzly #CEILOMETER_REPO=git://git.openstack.org/openstack/ceilometer.git #CEILOMETERCLIENT_REPO=git://git.openstack.org/openstack/python-ceilometerclient.git #HEAT_REPO=git://git.openstack.org/openstack/heat.git #HEAT_BRANCH=master #SWIFT_REPO=git://git.openstack.org/openstack/swift.git #SWIFT_BRANCH=master #OPENSTACKCLIENT_REPO=git://git.openstack.org/openstack/python-openstackclient.git #OPENSTACKCLIENT_BRANCH=master #SOLUM_REPO=git://git.openstack.org/stackforge/solum.git #SOLUM_BRANCH=master # Enable Ceilometer #enable_service ceilometer-acompute #enable_service ceilometer-acentral #enable_service ceilometer-anotification #enable_service ceilometer-collector #enable_service ceilometer-api #enable_service ceilometer-alarm-notifier #enable_service ceilometer-alarm-evaluator #CEILOMETER_PIPELINE_INTERVAL=60 # Enable Solum #enable_service solum #enable_service solum-api #enable_service solum-build-api #enable_service solum-deployer #enable_service solum-conductor #enable_service solum-worker #SOLUM_INSTALL_CEDARISH=True # Enable Marconi #disable_service marconi-server #enable_service zaqar-server #https://wiki.openstack.org/wiki/Docker #export INSTALLDIR=/bak/openstack && cd $INSTALLDIR/nova-docker #git clone https://git.openstack.org/stackforge/nova-docker #./contrib/devstack/prepare_devstack.sh #VIRT_DRIVER=docker #DEFAULT_IMAGE_NAME=cirros #NON_STANDARD_REQS=1 #IMAGE_URLS="" #Enable tacker #TACKER_BRANCH=master #enable_plugin heat https://git.openstack.org/openstack/heat $TACKER_BRANCH #enable_plugin networking-sfc https://git.openstack.org/openstack/networking-sfc $TACKER_BRANCH #enable_plugin barbican https://git.openstack.org/openstack/barbican $TACKER_BRANCH #enable_plugin mistral https://git.openstack.org/openstack/mistral $TACKER_BRANCH #enable_plugin tacker https://git.openstack.org/openstack/tacker $TACKER_BRANCH EOF
然后使用"./stack.sh"运行它,注意非常重要的一点,如果想要重新运行避免出莫名其妙的错误的话,必须:
#re-run
sudo systemctl stop "devstack@*"
./clean.sh
./stack.sh
source accrc/admin/admin $ env |grep OS_ OS_PROJECT_DOMAIN_ID=default OS_CACERT= OS_AUTH_URL=http://172.16.1.1/identity OS_USER_DOMAIN_ID=default OS_USERNAME=admin OS_PROJECT_NAME=admin OS_PASSWORD=password source <(openstack complete) openstack complete |sudo tee /etc/bash_completion.d/openstack.bash_completion > /dev/null systemctl -a |grep devstack sudo tail -f /var/log/syslog sudo journalctl -f --unit devstack@n-cpu.service # only need to create network/subnet staffs when NEUTRON_CREATE_INITIAL_NETWORKS=true openstack router create provider-router openstack network create private openstack subnet create --subnet-range 10.0.1.0/24 --network private --allocation-pool start=10.0.1.50,end=10.0.1.100 --gateway 10.0.1.1 private-subnet openstack network create --provider-network-type flat --external --share --provider-physical-network physnet1 ext_net openstack subnet create --allocation-pool start=192.168.101.3,end=192.168.101.100 --subnet-range 192.168.101.0/24 --dhcp --gateway 192.168.101.1 --ip-version 4 --network ext_net ext_net_subnet #openstack subnet set ext_net_subnet --dhcp #just simulate ext_net so use dhcp openstack router add subnet provider-router private-subnet openstack router set --external-gateway ext_net provider-router cat << EOF > user-data #cloud-config user: ubuntu password: password chpasswd: { expire: False } EOF # in case you don't have a key #ssh-keygen -q -N "" openstack keypair create --public-key ~/.ssh/id_rsa.pub mykey openstack --debug server create --image cirros-0.5.2-x86_64-disk --flavor m1.tiny --key-name mykey --nic net-id=$(openstack network list | awk '/private/ {print $2}') --user-data ./user-data --config-drive true i1 #add FIP public_network=$(openstack network show public -f value -c id) fip=$(openstack floating ip create $public_network -f value -c floating_ip_address) fix_ip=$(openstack server list |grep 'private=' |awk -F '=' '{print $2}' |awk '{print $1}') openstack floating ip set $fip --fixed-ip-address $fix_ip --port $(openstack port list --fixed-ip ip-address=$fix_ip -c id -f value) #add SG rules sg_id=$(openstack security group list --project=$(openstack server show i1 -f value -c project_id) |grep default |awk '{print $2}') openstack security group rule create --protocol tcp --dst-port 22 $sg_id openstack security group rule create --protocol icmp $sg_id sudo ip netns exec qrouter-6aef358e-7cbe-4d77-af70-ed13fd11f621 ping $fix_ip openstack console log show i1 #we just simulate fip so we should ping fip from qg-xxx interface sudo ip netns exec qrouter-6aef358e-7cbe-4d77-af70-ed13fd11f621 ping $fip
sudo su - stack cd /bak/openstack/devstack sudo systemctl --failed sudo systemctl list-units |grep devstack |grep failed |awk '{print $2}' sudo systemctl status devstack@* source accrc/admin/admin openstack server list # if above doesn't work, we may need to rerun devstack sudo systemctl stop devstack@* ./stack.sh openstack keypair create --public-key ~/.ssh/id_rsa.pub mykey # 有时创建虚机失败时,换个名字如i2就好了 openstack --debug server create --image cirros-0.5.2-x86_64-disk --flavor m1.tiny --key-name mykey --nic net-id=$(openstack network list | awk '/private/ {print $2}') i1 # if fails, pls check sudo systemctl --failed sudo tail -f /var/log/syslog sudo systemctl -a |grep devstack@ sudo systemctl restart devstack@n-cpu.service sudo journalctl -f --unit devstack@n-cpu.service see - https://docs.openstack.org/devstack/latest/systemd.html
pypi.douban.com的pip镜像质量不高,同步不及时,导致我在此次使用过程中遇到了下列包依赖问题.
ERROR: Could not find a version that satisfies the requirement cliff===3.8.0 (from -c /bak/openstack/requirements/upper-constraints.txt (line 249)) (from versions: none)
ERROR: No matching distribution found for cliff===3.8.0 (from -c /bak/openstack/requirements/upper-constraints.txt (line 249))
去掉它就好了,下面只是记录使用douban镜像的过程,仅作参考:
NOTE: 一定要使用pip镜像时也可以试试清华的(pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple )
当前用户或stack用户添加:
mkdir ~/.pip/ cat << EOF > ~/.pip/pip.conf [global] index-url = http://pypi.douban.com/simple [install] trusted-host=pypi.douban.com disable-pip-version-check = true timeout = 120 EOF 此时,"sudo pip3 install -r requirements.txt"还是未生效,那是因为需要继续在root用户中也添加: sudo mkdir /root/.pip sudo bash -c 'cat > /root/.pip/pip.conf' << EOF [global] index-url = http://pypi.douban.com/simple [install] trusted-host=pypi.douban.com disable-pip-version-check = true timeout = 120 EOF 或者使用: [global] trusted-host=mirrors.aliyun.com index-url = http://mirrors.aliyun.com/pypi/simple disable-pip-version-check = true timeout = 120 若不使用pip镜像,例如,运行下列命令时, sudo -H LC_ALL=en_US.UTF-8 http_proxy= https_proxy= no_proxy= PIP_FIND_LINKS= SETUPTOOLS_SYS_PATH_TECHNIQUE=rewrite python3.8 -m pip install -c /bak/openstack/requirements/upper-constraints.txt 'glance-store[cinder]!=0.29.0' 会报这种错误: urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out. 可以局部通过如下添加"--index-url https://pypi.douban.com/simple"解决: sudo -H LC_ALL=en_US.UTF-8 http_proxy= https_proxy= no_proxy= PIP_FIND_LINKS= SETUPTOOLS_SYS_PATH_TECHNIQUE=rewrite python3.8 -m pip install -c /bak/openstack/requirements/upper-constraints.txt 'glance-store[cinder]!=0.29.0' --index-url https://pypi.douban.com/simple 对于小包可以通过添加超时时间解决(pip --default-timeout=100 install -U pip)
1, 当将devstack中的OFFLINE设置为True后重新运行./stack.sh报下列错: apache2.service: Failed with result 'start-limit-hit' Soluation: 从这个网页找到了答案:https://www.hiroom2.com/2017/02/18/linux-systemd-s-start-request-repeated-too-quickly-for-xxx-service/ 修改文件(/lib/systemd/system/apache2.service)添加下设置禁用掉systemd对重启服务次数的限制. StartLimitIntervalSec=0 StartLimitBurst=0 然后运行 sudo systemctl daemon-reload sudo systemctl reset-failed apache2.service 2, 运行"cd /bak/openstack/horizon && /usr/bin/python3.8 manage.py compilemessages"报下列错: ModuleNotFoundError: No module named 'django.utils.log.NullHandler'; 'django.utils.log' is not a package Solution: sed -i "s/django.utils.log.NullHandler/logging.NullHandler/g" ./openstack_dashboard/local/local_settings.py 3, 运行"sudo ebtables -t broute -L"报下列错: table `broute' is incompatible, use 'nft' tool Solution: Try Switching to iptable-legacy sudo apt update && sudo apt upgrade sudo apt -y install iptables arptables ebtables sudo update-alternatives --set iptables /usr/sbin/iptables-legacy || true sudo update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy || true sudo update-alternatives --set arptables /usr/sbin/arptables-legacy || true sudo update-alternatives --set ebtables /usr/sbin/ebtables-legacy || true
#注意点 1, 实验用的虚机镜像是UEFI格式的,所以导入虚机镜像时记得使用UEFI。 2, 虚机启动后没有自动获得IP的问题,那是因为: a, 物理机libvirt的default网络若使用了默认的192.168.122.0/24网段,虚机内部也安装了libvirt会用这个默认网段。 虚机内部使用virsh net-edit default修改一个不同网段并重启虚机即可。 b, 若虚机内的网卡名不叫eth0, 如enp1s0. 需修改/etc/netplan/00-installer-config.yaml将eth0改为enps1s0, 然后运行sudo netplan apply即可获得IP 3, 最好devstack运行成功之后禁用它的重新开机后的自动重启。因为devstack机器重启之后总是会遇到各种问题,索性每次重运行一次 sudo systemctl -a |grep devstack@ |xargs sudo systemctl disable sudo systemctl -a #重新运行, OFFLINE=true模式不work, 仍应确保使用unstack.sh而不是clean.sh sudo su - stack cd /bak/openstack/devstack #./clean.sh ./unstack.sh ./stack.sh #运行成功之后记得及时disable所有服务避免下次重启机器时再各种问题 sudo systemctl -a |grep devstack@ |xargs sudo systemctl disable #验证 sudo systemctl --failed sudo systemctl status devstack@* source accrc/admin/admin openstack server list #使用, 有时创建虚机会失败,换个名字如i2就好了 openstack keypair create --public-key ~/.ssh/id_rsa.pub mykey openstack --debug server create --image cirros-0.5.2-x86_64-disk --flavor m1.tiny --key-name mykey --nic net-id=$(openstack network list | awk '/private/ {print $2}') i1 #访问虚机 PROJECT_ID=$(openstack project show --domain default admin -f value -c id) SG_ID=$(openstack security group list --project ${PROJECT_ID} | awk '/default/ {print $2}') openstack security group rule list $SG_ID openstack security group rule create $SG_ID --ingress --ethertype IPv4 --protocol icmp --remote-ip 0.0.0.0/0 openstack security group rule create $SG_ID --ingress --ethertype IPv4 --protocol tcp --dst-port 22:22 --remote-ip 0.0.0.0/0 #you may fail to ssh cirros becuase dropbear key and openssh key are incompatible, change non-cirros will succeed #sudo ip netns exec qrouter-b4a099e0-d0db-4ec8-8f3c-9d83e11d3488 ssh cirros@10.0.1.20 -v #浮动IP # eg: vm=10.0.1.20 public_network=$(openstack network show public -f value -c id) fip=$(openstack floating ip create $public_network -f value -c floating_ip_address) openstack floating ip set $fip --fixed-ip-address 10.0.1.20 --port $(openstack port list --fixed-ip ip-address=10.0.1.20 -c id -f value) #调试 sudo systemctl --failed sudo tail -f /var/log/syslog sudo systemctl -a |grep devstack@ sudo systemctl restart devstack@n-cpu.service sudo journalctl -f --unit devstack@n-cpu.service
hua@minipc:/bak/openstack/devstack$ cat local.conf [[local|localrc]] #sudo apt -y install iptables arptables ebtables -y #sudo update-alternatives --set iptables /usr/sbin/iptables-legacy || true #sudo update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy || true #sudo update-alternatives --set arptables /usr/sbin/arptables-legacy || true #sudo update-alternatives --set ebtables /usr/sbin/ebtables-legacy || true #sudo ebtables -t broute -L #curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py && python3 ./get-pip.py #sudo pip install --upgrade setuptools # # Network Design # Physical Network: HOST_IP=192.168.99.179, used for openstack itself, eg: keystone, mysql, DB etc # FIP Network: devstack will set its GW 192.168.101.1 in br-data, and create a SNAT rule for 192.168.101.0/24 # VM Network: openstack will create geneve private-subnet=10.0.0.0/26, and create virtual GW 10.0.0.1/26 #git checkout -b zed origin/stable/zed TARGET_BRANCH=stable/zed HOST_IP=$(ip route list default |grep src |awk '{print $9}') #If OVN_DNS_SERVERS is not set, VM will use 127.0.0.53, then systemd-resolv inside VM will fail to start, don't know yet OVN_DNS_SERVERS=192.168.99.1 enable_service neutron-dns LIBVIRT_CPU_MODE=host-passthrough #for multi hosts, SERVICE_HOST specifies controller's HOST_IP #ENABLED_SERVICES=n-cpu,c-vol,placement-client,ovn-controller,ovs-vswitchd,ovsdb-server,q-ovn-metadata-agent SERVICE_HOST=$HOST_IP MYSQL_HOST=$SERVICE_HOST RABBIT_HOST=$SERVICE_HOST GLANCE_HOSTPORT=$SERVICE_HOST:9292 NOVA_VNC_ENABLED=True NOVNCPROXY_URL="http://$SERVICE_HOST:6080/vnc_lite.html" VNCSERVER_LISTEN=$HOST_IP VNCSERVER_PROXYCLIENT_ADDRESS=$VNCSERVER_LISTEN ADMIN_PASSWORD=password DATABASE_PASSWORD=$ADMIN_PASSWORD RABBIT_PASSWORD=$ADMIN_PASSWORD SERVICE_PASSWORD=$ADMIN_PASSWORD #enable ml2-dns to parse the hostname of VMs Q_ML2_PLUGIN_EXT_DRIVERS=port_security,qos,dns #should enable SG for 'juju bootstrap' Q_USE_SECGROUP=True #need to add the following 3 commands in /etc/rc.local as well #ip link set dev br-data up >/dev/null 2>&1 #ip addr add 192.168.101.1/24 dev br-data up >/dev/null 2>&1 #iptables -t nat -A POSTROUTING -s 192.168.101.0/24 ! -d 192.168.101.0/24 -j MASQUERADE >/dev/null 2>&1 #sudo systemctl enable $(find /etc/systemd/system -name 'devstack@*.service' -print) FLOATING_RANGE=192.168.101.0/24 Q_FLOATING_ALLOCATION_POOL=start=192.168.101.10,end=192.168.101.99 PUBLIC_NETWORK_GATEWAY=192.168.101.1 #sudo ip l add name veth-br-eth0 type veth peer name veth-ex >/dev/null 2>&1 #sudo ip l set dev veth-br-eth0 up #sudo ip l set dev veth-ex up #sudo ip l set veth-br-eth0 master br-eth0 #PUBLIC_INTERFACE=veth-ex #don't need veth-ex here because devstack will create SNAT for FIP subnet Q_USE_PROVIDERNET_FOR_PUBLIC=True OVS_PHYSICAL_BRIDGE=br-data PUBLIC_BRIDGE=$OVS_PHYSICAL_BRIDGE OVS_BRIDGE_MAPPINGS=public:br-data IP_VERSION=4 ENABLE_DEBUG_LOG_LEVEL=False SYSLOG=True VERBOSE=True LOG_COLOR=True LOGDAYS=3 DOWNLOAD_DEFAULT_IMAGES=False IMAGE_URLS="http://download.cirros-cloud.net/0.6.1/cirros-0.6.1-x86_64-disk.img" PYPI_ALTERNATIVE_URL=http://pypi.douban.com/simple GIT_BASE=http://git.trystack.cn #enable_plugin designate https://opendev.org/openstack/designate disable_service tempest disable_service dstat disable_service memory_tracker #ln: failed to create symbolic link '/var/run/ovn/openvswitch': File exist sudo rm -rf /var/run/ovn
这次主要遇到了3个问题:
OVN_DNS_SERVERS=192.168.99.1 enable_service neutron-dns #enable ml2-dns sudo vi /etc/neutron/neutron.conf dns_domain = devstack.lan. sudo vi /etc/neutron/plugins/ml2/ml2_conf.ini [ml2] extension_drivers = port_security,qos,dns sudo systemctl restart devstack@q-svc.service neutron subnet-update private-subnet --dns_nameservers list=true 192.168.99.1 $ sudo ovn-nbctl list dhcp_options |grep options options : {classless_static_route="{169.254.169.254/32,10.0.1.2, 0.0.0.0/0,10.0.1.1}", dns_server="{192.168.99.1}", domain_name="\"devstack.lan.\"", lease_time="43200", mtu="1442", router="10.0.1.1", server_id="10.0.1.1", server_mac="fa:16:3e:7b:39:9f"} ubuntu@i1:~$ sudo resolvectl status ens3 |grep -E 'Current DNS Server|DNS Domain' Current DNS Server: 192.168.99.1 DNS Domain: devstack.lan
ip link set dev br-data up >/dev/null 2>&1
ip addr add 192.168.101.1/24 dev br-data up >/dev/null 2>&1
iptables -t nat -A POSTROUTING -s 192.168.101.0/24 ! -d 192.168.101.0/24 -j MASQUERADE >/dev/null 2>&1
sudo systemctl restart devstack@*
sudo systemctl status devstack@* |grep Active
vim /etc/nova/nova-cpu.conf
[libvirt]
live_migration_uri = qemu+ssh://hua@%s/system
#cpu_model = Nehalem
#cpu_mode = custom
cpu_mode = host-passthrough
virt_type = kvm
sudo apt -y install iptables arptables ebtables -y
sudo update-alternatives --set iptables /usr/sbin/iptables-legacy || true
sudo update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy || true
sudo update-alternatives --set arptables /usr/sbin/arptables-legacy || true
sudo update-alternatives --set ebtables /usr/sbin/ebtables-legacy || true
sudo ebtables -t broute -L
hua@minipc:/bak/openstack/devstack$ git diff diff --git a/tools/install_pip.sh b/tools/install_pip.sh index 7c5d4c65..7e203bcf 100755 --- a/tools/install_pip.sh +++ b/tools/install_pip.sh @@ -7,7 +7,7 @@ # Assumptions: # - PYTHON3_VERSION refers to a version already installed -set -o errexit +#set -o errexit # Keep track of the current directory TOOLS_DIR=$(cd $(dirname "$0") && pwd) diff --git a/tools/worlddump.py b/tools/worlddump.py index e2921737..78f6ca28 100755 --- a/tools/worlddump.py +++ b/tools/worlddump.py @@ -19,6 +19,7 @@ import argparse import datetime +import setuptools from distutils import spawn import fnmatch import io
下列脚本可以重建混乱的pip依赖:
find . -name "*.pyc" -exec rm -rf {} \; #echo 'deleting ~/cache/pip ...' #rm -rf ~/.cache/pip/* #sudo chown -R $USER ~/.cache/pip/ echo 'deleting dist-packages ...' sudo rm -rf /usr/local/lib/python3.10/dist-packages/oslo* sudo rm -rf /usr/local/lib/python3.10/dist-packages/*keystone* sudo rm -rf /usr/local/lib/python3.10/dist-packages/*glance* sudo rm -rf /usr/local/lib/python3.10/dist-packages/*swift* sudo rm -rf /usr/local/lib/python3.10/dist-packages/*cinder* sudo rm -rf /usr/local/lib/python3.10/dist-packages/*nova* sudo rm -rf /usr/local/lib/python3.10/dist-packages/*neutron* #root_dir=/bak/openstack root_dir=/opt/stack for pro in requirements keystone glance cinder horizon nova neutron tempest; do echo $pro cd $pro git diff > diff && git checkout . && git checkout zed >/dev/null 2>&1 git pull sudo pip uninstall -y -r requirements.txt sudo pip uninstall -y -r test-requirements.txt sudo pip install --upgrade -r requirements.txt sudo pip install --upgrade -r test-requirements.txt #sudo python setup.py develop cd $root_dir done;
阿里云虚机系统是22.04,不需要支持嵌套虚拟化。推荐是4 vCPUs, 8G memory, 30G dist. 但如下,安装之前实际内存用了4.8G,disk用了8.5G
stack@iZ2ze94xd36la2abgiwrftZ:~$ free -h |head -n2
total used free shared buff/cache available
Mem: 7.1Gi 4.8Gi 201Mi 32Mi 2.2Gi 2.0Gi
stack@iZ2ze94xd36la2abgiwrftZ:~$ df -h |grep vda |head -1
/dev/vda3 40G 8.5G 29G 23% /
下面是安装步骤(主要是github无法访问所以需禁用etcd3, 并且使用git.trystack.cn代替github, 同时使用pip mirror. 最后因为我这个阿里云打开了IPv6所以需要设置IPv4 prefer这样才能访问cirros-cloud.net顺利下载cirros2镜像)
#Create the user stack, and make sure to use this user for the test useradd -s /bin/bash -d /opt/stack -m stack chmod +x /opt/stack echo "stack ALL=(ALL) NOPASSWD: ALL" | sudo tee /etc/sudoers.d/stack #Make sure to run this command so that cirros-cloud.net can be visited normally echo 'precedence ::ffff:0:0/96 100' |sudo tee -a /etc/gai.conf sudo -u stack -i #Use pip mirror to bypass the wall mkdir -p ~/.pip cat << EOF |tee ~/.pip/pip.conf [global] trusted-host=pypi.tuna.tsinghua.edu.cn index-url = https://pypi.tuna.tsinghua.edu.cn/simple disable-pip-version-check = true timeout = 120 EOF #pip3 install pbr -i https://pypi.tuna.tsinghua.edu.cn/simple --trusted-host pypi.tuna.tsinghua.edu.cn #Run devstack git clone http://git.trystack.cn/openstack/devstack.git -b stable/zed cd devstack cat << EOF |tee ./local.conf [[local|localrc]] # Network Design # Physical Network: HOST_IP, used for openstack itself, eg: keystone, mysql, DB etc # FIP Network: devstack will set its GW 192.168.101.1 in br-data, and create a SNAT rule for 192.168.101.0/24 # VM Network: openstack will create geneve private-subnet=10.0.0.0/26, and create virtual GW 10.0.0.1/26 TARGET_BRANCH=stable/zed #aliyun doens't support nested virtualization so use qemu instead LIBVIRT_TYPE=qemu #LIBVIRT_CPU_MODE=host-passthrough HOST_IP=$(ip route list default |grep src |awk '{print $9}') #If OVN_DNS_SERVERS is not set, VM will use 127.0.0.53, then systemd-resolv inside VM will fail to start, don't know yet OVN_DNS_SERVERS=$(resolvectl status | grep 'Current DNS Server' | awk -F':' '{print $2}' | head -n1 | sed -e 's/^[ \t]*//') enable_service neutron-dns SERVICE_HOST=\$HOST_IP MYSQL_HOST=\$SERVICE_HOST RABBIT_HOST=\$SERVICE_HOST GLANCE_HOSTPORT=\$SERVICE_HOST:9292 NOVA_VNC_ENABLED=True NOVNCPROXY_URL=http://\$SERVICE_HOST:6080/vnc_lite.html VNCSERVER_LISTEN=\$HOST_IP VNCSERVER_PROXYCLIENT_ADDRESS=\$VNCSERVER_LISTEN ADMIN_PASSWORD=password DATABASE_PASSWORD=\$ADMIN_PASSWORD RABBIT_PASSWORD=\$ADMIN_PASSWORD SERVICE_PASSWORD=\$ADMIN_PASSWORD #enable ml2-dns to parse the domain of VMs Q_ML2_PLUGIN_EXT_DRIVERS=port_security,qos,dns Q_USE_SECGROUP=True #add the following 3 commands in /etc/rc.local as well if hoping devstack to take effect after the restart #ifconfig br-data 192.168.101.1/24 >/dev/null 2>&1 & #iptables -t nat -D POSTROUTING -s 192.168.101.0/24 ! -d 192.168.101.0/24 -j MASQUERADE >/dev/null 2>&1 & #iptables -t nat -A POSTROUTING -s 192.168.101.0/24 ! -d 192.168.101.0/24 -j MASQUERADE >/dev/null 2>&1 & #sudo systemctl enable \$(find /etc/systemd/system -name 'devstack@*.service' -print) FLOATING_RANGE=192.168.101.0/24 Q_FLOATING_ALLOCATION_POOL=start=192.168.101.10,end=192.168.101.99 PUBLIC_NETWORK_GATEWAY=192.168.101.1 Q_USE_PROVIDERNET_FOR_PUBLIC=True OVS_PHYSICAL_BRIDGE=br-data PUBLIC_BRIDGE=\$OVS_PHYSICAL_BRIDGE OVS_BRIDGE_MAPPINGS=public:br-data IP_VERSION=4 ENABLE_DEBUG_LOG_LEVEL=False SYSLOG=True VERBOSE=True LOG_COLOR=True LOGDAYS=3 DOWNLOAD_DEFAULT_IMAGES=False #Make sure to run this command so that cirros-cloud.net can be visited normally #echo 'precedence ::ffff:0:0/96 100' |sudo tee -a /etc/gai.conf IMAGE_URLS="http://download.cirros-cloud.net/0.6.2/cirros-0.6.2-x86_64-disk.img" #enable_plugin designate https://opendev.org/openstack/designate disable_service tempest disable_service dstat disable_service memory_tracker #disable etcd3 to avoid visiting github that has been blocked disable_service etcd3 #ln: failed to create symbolic link '/var/run/ovn/openvswitch': File exist sudo rm -rf /var/run/ovn GIT_BASE=http://git.trystack.cn NOVNC_REPO=http://git.trystack.cn/kanaka/noVNC.git SPICE_REPO=http://git.trystack.cn/git/spice/spice-html5.git EOF #./unstack.sh && sudo systemctl restart openvswitch-switch ./stack.sh source openrc #when hoping to use admin user #unset OS_SERVICE_TOKEN #export OS_USERNAME=admin #export OS_TENANT_NAME==admin source <(openstack complete) openstack complete |sudo tee /etc/bash_completion.d/openstack.bash_completion > /dev/null openstack network create bio-net openstack subnet create --subnet-range 10.10.122.0/24 --dhcp --dhcp --network bio-net --allocation-pool start=10.10.122.50,end=10.10.122.100 --dns-nameserver 8.8.8.8 bio-net-subnet openstack router create bio-rtr openstack router add subnet bio-rtr bio-net-subnet openstack router set --external-gateway public bio-rtr cat << EOF > user-data #cloud-config user: ubuntu password: password chpasswd: { expire: False } EOF # in case you don't have a key #ssh-keygen -q -N "" openstack keypair create --public-key ~/.ssh/id_rsa.pub mykey openstack --debug server create --image cirros-0.6.2-x86_64-disk --flavor m1.tiny --key-name mykey --nic net-id=$(openstack network list | awk '/bio-net/ {print $2}') --user-data ./user-data --config-drive true i1 public_network=$(openstack network show public -f value -c id) fip=$(openstack floating ip create $public_network -f value -c floating_ip_address) fix_ip=$(openstack server list |grep 'bio-net=' |awk -F '=' '{print $2}' |awk '{print $1}') openstack floating ip set $fip --fixed-ip-address $fix_ip --port $(openstack port list --fixed-ip ip-address=$fix_ip -c id -f value) sg_id=$(openstack security group list --project=$(openstack server show i1 -f value -c project_id) |grep default |awk '{print $2}') openstack security group rule create --protocol tcp --dst-port 22 $sg_id openstack security group rule create --protocol icmp $sg_id ping $fip openstack console log show i1 ssh cirros@$fip
Copyright © 2003-2013 www.wpsshop.cn 版权所有,并保留所有权利。