Centos7系统下KVM虚拟化Migration(06)--企业迁移案例

0x0 应用场景

• VMware ESXi–> KVM
• KVM –> VMware ESXi
• Xen –> VMware ESXi
• XEN–>KVM
• KVM –>KVM

0x1 v2v(虚拟机到虚拟机的迁移virtual machine to virtual machine)

• 01 冷迁移(也叫静态迁移)

静态迁移（offline migration）也叫做常规迁移，离线迁移。在迁移之前将虚拟机暂停，同时拷贝虚拟机镜像和状态到目的主机。相比较于在线迁移（online migration），其缺点就是静态迁移方式的过程需要显式的停止虚拟机的运行，而在线迁移的过程仅有非常短暂的停机时间，保证迁移过程中虚拟机服务的持续可用；静态迁移的优点是兼容性强，不需要VMM的同意API接口，可以借助第三方工具辅助迁移。由于目前VMM的开发没有统一的标准，不同的虚拟化厂商，尤其是闭源的厂商开发的VMM各成一套，静态迁移是解决不同类型虚拟机之间迁移的有效办法。

优点：虚拟机不需要位于共享存储器上，丢失数据的概率小。

缺点：需要关闭电源，业务中断。

• 02 热迁移(也叫动态迁移)

热迁移（Live Migration），又叫动态迁移、实时迁移，即虚拟机保存/恢复，通常是将整个虚拟机的运行状态完整保存下来，同时可以快速的恢复到原有硬件平台甚至是不同硬件平台上。恢复以后，虚拟机仍旧平滑运行，用户不会察觉到任何差异。

如果源宿主机和目的宿主机共享存储系统，则只需要通过网络发送客户机的 vCPU 执行状
态、内存中的内容、虚机设备的状态到目的主机上。否则，还需要将客户机的磁盘存储发到目的主
机上。共享存储系统指的是源和目的虚机的镜像文件目录是在一个共享的存储上的。

在基于 共享存储系统 时，KVM 动态迁移的具体过程为：
1、迁移开始时，客户机依然在宿主机上运行，与此同时，客户机的内存页被传输到目的主机上。
2、QEMU/KVM 会监控并记录下迁移过程中所有已被传输的内页的任何修改，并在所有内存页都传输完成后即开始传输在前面过程中内存页的更改内容。
3、QEMU/KVM 会估计迁移过程中的传输速度，当剩余的内存数据量能够在一个可以设定的时间周期（默认 30 毫秒）内传输完成时，QEMU/KVM 会关闭源宿主机上的客户机，再将剩余的数据量传输到目的主机上，最后传输过来的内存内容在目的宿主机上恢复客户机的运行状态。
4、至此，KVM 的动态迁移操作就完成了。迁移后的客户机尽可能与迁移前一致，除非目的主机上缺少一些配置，比如网桥等。注意，当客户机中内存使用率非常大而且修改频繁时，内存中数据不断被修改的速度大于KVM能够传输的内存速度时，动态迁移的过程是完成不了的，这时候只能静态迁移

优点：软件和硬件系统的维护升级，不会影响用户的关键服务，提高了服务的高可用性和 用户的满意度。

缺点：过程不可中断，操作复杂(良好的网络环境,要不有可能丢包)。

0x2 P2V迁移

0x3 企业迁移案例

1 VM在KVM集群服务器之间的迁移(冷迁移技术)

• 步骤
• 第一步 virsh list --all查看当前虚拟机状态
• 第二步 virsh shutdowm 你要迁移的虚拟机
• 第三步 virsh domblklist 虚拟机名称 获得虚拟机虚拟硬盘算在位置
• 第四步 virsh dumpxml 导出需要迁移的虚拟机配置文件
• 第五步 rsync -avz -P /vm_data/disk/xxx root@10.0.0.12:/vm_data/disk
• 第六步 virsh define xxx.xml virsh start xxx

2 VM在KVM集群服务器之间的迁移(webvirtmgr)

步骤一：
需要在kvm01 kvm02 kvm03上配置host文件本地解析
vim /etc/hosts
10.0.0.143 kvm01
10.0.0.144 kvm02
10.0.0.145 kvm03

步骤二：
在kvm01机器上免密登陆kvm02,kvm03。

[root@localhost conf.d]# su -s /bin/bash nginx
bash-4.2$ ssh-copy-id root@10.0.0.144 #kvm02机的免密
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/var/lib/nginx/.ssh/id_rsa.pub"
The authenticity of host '10.0.0.143 (10.0.0.143)' can't be established.
ECDSA key fingerprint is SHA256:N99fAx5OoDgmMhJEIaDDT4aPn4cusNFjs5xCmD9cDwc.
ECDSA key fingerprint is MD5:e3:d3:fc:9b:51:56:0d:20:5b:16:1e:1c:b4:38:a1:38.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@10.0.0.143's password: 
 
Number of key(s) added: 1
 
Now try logging into the machine, with:   "ssh 'root@10.0.0.144'"
and check to make sure that only the key(s) you wanted were added.
 
bash-4.2$ ssh-copy-id root@10.0.0.145 #kvm03机器免密
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/var/lib/nginx/.ssh/id_rsa.pub"
The authenticity of host '10.0.0.143 (10.0.0.143)' can't be established.
ECDSA key fingerprint is SHA256:N99fAx5OoDgmMhJEIaDDT4aPn4cusNFjs5xCmD9cDwc.
ECDSA key fingerprint is MD5:e3:d3:fc:9b:51:56:0d:20:5b:16:1e:1c:b4:38:a1:38.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@10.0.0.143's password: 
 
Number of key(s) added: 1
 
Now try logging into the machine, with:   "ssh 'root@10.0.0.145'"
and check to make sure that only the key(s) you wanted were added.

webvirtmgr管理三台服务器 kvm01-当管理机器,纳管kvm02 kvm03机，形成KVM集群

001.png

002.png

001.png

003.png

004.png

005.png

启动虚拟机

006.png

007.png

008.png

006.png

3 VMWare ESXi服务器迁移到KVM

virt-v2v是将外部的虚拟化平台上的虚拟机转化到可以运行的KVM平台上。它可以读取在VMware、Xen运行Hyper-V和其他虚拟机管理程序上的Windows和Linux的虚拟机，并将其转换为KVM的libvirt，OpenStack等几种方式

yum install libvirt* virt-* qemu-kvm* -y
[root@kvm02 ~]# yum install libvirt* virt-* qemu-kvm* -y
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
epel/x86_64/metalink                                                                               | 9.4 kB  00:00:00     
 * base: mirrors.aliyun.com
 * epel: mirrors.yun-idc.com
 * extras: mirrors.aliyun.com
 * updates: mirrors.aliyun.com
base                                                                                               | 3.6 kB  00:00:00     
epel                                                                                               | 5.3 kB  00:00:00     
extras                                                                                             | 2.9 kB  00:00:00     
updates                                                                                            | 2.9 kB  00:00:00     
(1/2): epel/x86_64/updateinfo                                                                      | 1.0 MB  00:00:00     
(2/2): epel/x86_64/primary_db                                                                      | 6.9 MB  00:00:02     
Package libvirt-glib-1.0.0-1.el7.x86_64 already installed and latest version
Package libvirt-daemon-driver-storage-logical-4.5.0-23.el7_7.3.x86_64 already installed and latest version
Package libvirt-daemon-driver-qemu-4.5.0-23.el7_7.3.x86_64 already installed and latest version
Package libvirt-daemon-driver-storage-core-4.5.0-23.el7_7.3.x86_64 already installed and latest version
Package libvirt-daemon-driver-nwfilter-4.5.0-23.el7_7.3.x86_64 already installed and latest version
Package libvirt-daemon-driver-lxc-4.5.0-23.el7_7.3.x86_64 already installed and latest version
Package libvirt-daemon-driver-secret-4.5.0-23.el7_7.3.x86_64 already installed and latest version
Package libvirt-daemon-4.5.0-23.el7_7.3.x86_64 already installed and latest version
Package libvirt-daemon-config-nwfilter-4.5.0-23.el7_7.3.x86_64 already installed and latest version
Package libvirt-daemon-driver-storage-4.5.0-23.el7_7.3.x86_64 already installed and latest version
Package libvirt-daemon-driver-interface-4.5.0-23.el7_7.3.x86_64 already installed and latest version
Package libvirt-bash-completion-4.5.0-23.el7_7.3.x86_64 already installed and latest version
Package libvirt-daemon-driver-storage-mpath-4.5.0-23.el7_7.3.x86_64 already installed and latest version
Package libvirt-python-4.5.0-1.el7.x86_64 already installed and latest version
Package libvirt-daemon-driver-storage-gluster-4.5.0-23.el7_7.3.x86_64 already installed and latest version
Package libvirt-daemon-driver-storage-disk-4.5.0-23.el7_7.3.x86_64 already installed and latest version
Package libvirt-daemon-driver-storage-scsi-4.5.0-23.el7_7.3.x86_64 already installed and latest version
Package libvirt-client-4.5.0-23.el7_7.3.x86_64 already installed and latest version
Package libvirt-4.5.0-23.el7_7.3.x86_64 already installed and latest version
Package libvirt-daemon-config-network-4.5.0-23.el7_7.3.x86_64 already installed and latest version
Package libvirt-daemon-driver-network-4.5.0-23.el7_7.3.x86_64 already installed and latest version
Package libvirt-libs-4.5.0-23.el7_7.3.x86_64 already installed and latest version
Package libvirt-daemon-kvm-4.5.0-23.el7_7.3.x86_64 already installed and latest version
Package libvirt-daemon-driver-storage-rbd-4.5.0-23.el7_7.3.x86_64 already installed and latest version
Package libvirt-daemon-driver-storage-iscsi-4.5.0-23.el7_7.3.x86_64 already installed and latest version
Package libvirt-daemon-driver-nodedev-4.5.0-23.el7_7.3.x86_64 already installed and latest version
Package virt-manager-common-1.5.0-7.el7.noarch already installed and latest version
Package virt-manager-1.5.0-7.el7.noarch already installed and latest version
Package virt-install-1.5.0-7.el7.noarch already installed and latest version
Package virt-what-1.18-4.el7.x86_64 already installed and latest version
Package virt-viewer-5.0-15.el7.x86_64 already installed and latest version
Package 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 already installed and latest version
Package 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 already installed and latest version
#次数省略N多行
Dependency Installed:
  OVMF.noarch 0:20180508-6.gitee3198e672e2.el7                                                                            
  PyYAML.x86_64 0:3.10-11.el7                                                                                             
  augeas.x86_64 0:1.4.0-9.el7                                                                                             
  copy-jdk-configs.noarch 0:3.3-10.el7_5                                                                                  
  giflib.x86_64 0:4.1.6-9.el7                                                                                             
  glib2-devel.x86_64 0:2.56.1-5.el7                                                                                       
  gtk2.x86_64 0:2.24.31-1.el7                                                                                             
  java-1.8.0-openjdk.x86_64 1:1.8.0.232.b09-0.el7_7                                                                       
  java-1.8.0-openjdk-headless.x86_64 1:1.8.0.232.b09-0.el7_7                                                              
  javapackages-tools.noarch 0:3.4.1-11.el7                                                                                
  jna.x86_64 0:3.5.2-8.el7                                                                                                
  libcmpiutil.x86_64 0:0.5.7-3.el7                                                                                        
  libfontenc.x86_64 0:1.1.3-3.el7                                                                                         
  libguestfs-winsupport.x86_64 0:7.2-3.el7                                                                                
  libxml2-devel.x86_64 0:2.9.1-6.el7_2.3                                                                                  
  libyaml.x86_64 0:0.1.4-11.el7_0                                                                                         
  lksctp-tools.x86_64 0:1.0.17-2.el7                                                                                      
  lm_sensors-libs.x86_64 0:3.4.0-8.20160601gitf9185e5.el7                                                                 
  m2crypto.x86_64 0:0.21.1-17.el7                                                                                         
  nbdkit.x86_64 0:1.8.0-1.el7                                                                                             
  nbdkit-plugin-python-common.x86_64 0:1.8.0-1.el7                                                                        
  nbdkit-plugin-python2.x86_64 0:1.8.0-1.el7                                                                              
  nbdkit-plugin-vddk.x86_64 0:1.8.0-1.el7                                                                                 
  net-snmp-agent-libs.x86_64 1:5.7.2-43.el7                                                                               
  net-snmp-libs.x86_64 1:5.7.2-43.el7                                                                                     
  openslp.x86_64 1:2.0.0-8.el7_7                                                                                          
  pcre-devel.x86_64 0:8.32-17.el7                                                                                         
  pcsc-lite-libs.x86_64 0:1.8.8-8.el7                                                                                     
  python-dateutil.noarch 0:1.5-7.el7                                                                                      
  python-javapackages.noarch 0:3.4.1-11.el7                                                                               
  python-lxml.x86_64 0:3.2.1-4.el7                                                                                        
  python-suds.noarch 0:0.4.1-5.el7                                                                                        
  sanlock.x86_64 0:3.7.3-1.el7                                                                                            
  sanlock-lib.x86_64 0:3.7.3-1.el7                                                                                        
  subscription-manager-rhsm.x86_64 0:1.24.13-3.el7.centos                                                                 
  subscription-manager-rhsm-certificates.x86_64 0:1.24.13-3.el7.centos                                                    
  systemd-python.x86_64 0:219-67.el7_7.2                                                                                  
  tog-pegasus.x86_64 2:2.14.1-7.el7                                                                                       
  tog-pegasus-libs.x86_64 2:2.14.1-7.el7                                                                                  
  ttmkfdir.x86_64 0:3.0.9-42.el7                                                                                          
  tzdata-java.noarch 0:2019c-1.el7                                                                                        
  unzip.x86_64 0:6.0-20.el7                                                                                               
  xorg-x11-font-utils.x86_64 1:7.5-21.el7                                                                                 
  xorg-x11-fonts-Type1.noarch 0:7.5-9.el7                                                                                 
  xz-devel.x86_64 0:5.2.2-1.el7                                                                                           
  zlib-devel.x86_64 0:1.2.7-18.el7                                                                                        
 
Dependency Updated:
  glib2.x86_64 0:2.56.1-5.el7               systemd.x86_64 0:219-67.el7_7.2      systemd-libs.x86_64 0:219-67.el7_7.2     
  systemd-sysv.x86_64 0:219-67.el7_7.2     
 
Complete!
#至此KVM+QEMU所有组件安装完毕！
#执行
virt-v2v -help
[root@kvm02 ~]# virt-v2v -help
virt-v2v: convert a guest to use KVM
 
 virt-v2v -ic vpx://vcenter.example.com/Datacenter/esxi -os imported esx_guest
 
 virt-v2v -ic vpx://vcenter.example.com/Datacenter/esxi esx_guest -o rhv -os rhv.nfs:/export_domain --network ovirtmgmt
 
 virt-v2v -i libvirtxml guest-domain.xml -o local -os /var/tmp
 
 virt-v2v -i disk disk.img -o local -os /var/tmp
 
 virt-v2v -i disk disk.img -o glance
 
There is a companion front-end called "virt-p2v" which comes as an
ISO or CD image that can be booted on physical machines.
 
A short summary of the options is given below.  For detailed help please
read the man page virt-v2v(1).
 
Options:
  -b, --bridge <in:out>               Map bridge ‘in’ to ‘out’
  --color, --colors, --colour, --colours
                                      Use ANSI colour sequences even if not tty
  --compressed                        Compress output file (-of qcow2 only)
  --debug-overlay, --debug-overlays   Save overlay files
  --echo-keys                         Don’t turn off echo for passphrases
  --help                              Display brief help
  -i <disk|libvirt|libvirtxml|ova|vmx>
                                      Set input mode (default: libvirt)
  -ic <uri>                           Libvirt URI
  -if <format>                        Input format (for -i disk)
  -io <option[=value]>                Set option for input mode
  -ip <filename>                      Use password from file to connect to input hypervisor
  -it <transport>                     Input transport
  --key <SELECTOR>                    Specify a LUKS key
  --keys-from-stdin                   Read passphrases from stdin
  --mac <mac:network|bridge:out>      Map NIC to network or bridge
  --machine-readable[=format]         Make output machine readable
  -n, --network <in:out>              Map network ‘in’ to ‘out’
  --no-copy                           Just write the metadata
  --no-trim <->                       Ignored for backwards compatibility
  -o <glance|libvirt|local|null|openstack|qemu|rhv|rhv-upload|vdsm>
                                      Set output mode (default: libvirt)
  -oa <sparse|preallocated>           Set output allocation mode
  -oc <uri>                           Output hypervisor connection
  -of <raw|qcow2>                     Set output format
  -on <name>                          Rename guest when converting
  -oo <option[=value]>                Set option for output mode
  -op <filename>                      Use password from file to connect to output hypervisor
  -os <storage>                       Set output storage location
  --password-file <filename>          Same as ‘-ip filename’
  --print-estimate                    Estimate size of source and stop
  --print-source                      Print source and stop
  -q, --quiet                         Don’t print progress messages
  --root <ask|... >                   How to choose root filesystem
  -V, --version                       Display version and exit
  -v, --verbose                       Enable libguestfs debugging messages
  --vddk-config <filename>            Same as ‘-io vddk-config=filename’
  --vddk-cookie <cookie>              Same as ‘-io vddk-cookie=filename’
  --vddk-libdir <libdir>              Same as ‘-io vddk-libdir=libdir’
  --vddk-nfchostport <nfchostport>    Same as ‘-io vddk-nfchostport=nfchostport’
  --vddk-port <port>                  Same as ‘-io vddk-port=port’
  --vddk-snapshot <snapshot-moref>    Same as ‘-io vddk-snapshot=snapshot-moref’
  --vddk-thumbprint <thumbprint>      Same as ‘-io vddk-thumbprint=thumbprint’
  --vddk-transports <transports>      Same as ‘-io vddk-transports=transports’
  --vdsm-compat <0.10|1.1>            Same as ‘-oo vdsm-compat=0.10|1.1’
  --vdsm-image-uuid <uuid>            Same as ‘-oo vdsm-image-uuid=uuid’
  --vdsm-ovf-flavour <ovirt|rhvexp>   Same as ‘-oo vdsm-ovf-flavour=flavour’
  --vdsm-ovf-output <dir>             Same as ‘-oo vdsm-ovf-output=dir’
  --vdsm-vm-uuid <uuid>               Same as ‘-oo vdsm-vm-uuid=uuid’
  --vdsm-vol-uuid <uuid>              Same as ‘-oo vdsm-vol-uuid=uuid’
  --vmtype <->                        Ignored for backwards compatibility
  -x                                  Enable tracing of libguestfs calls
[root@kvm02 ~]# 
#证明安装virt-v2v成功！

通过V2V方式迁移

cd /vm_data/v2v/
virt-v2v-copy-to-local -ic esx://root@10.0.0.130?no_verify=1 Centosx64_ESXi
virt-v2v -i libvirtxml Centosx64_ESXi.xml -o local -os /vm_data/v2v/ -of raw
[root@kvm02 v2v]# virt-v2v-copy-to-local -ic esx://root@10.0.0.130?no_verify=1 Centosx64_ESXi
[   0.0] Fetching the remote libvirt XML metadata ...
2020-01-08 15:17:16.837+0000: 13731: info : libvirt version: 4.5.0, package: 23.el7_7.3 (CentOS BuildSystem <http://bugs.centos.org>, 2019-12-02-17:45:06, x86-02.bsys.centos.org)
2020-01-08 15:17:16.837+0000: 13731: info : hostname: kvm02
2020-01-08 15:17:16.837+0000: 13731: warning : esxConnectOpen:857 : Ignoring unexpected path '' for non-vpx scheme 'esx'
Enter root's password for 10.0.0.130: 
[   7.9] Parsing the remote libvirt XML metadata ...
Enter host password for user 'root':
Enter host password for user 'root':
[  18.8] Copying remote disk 1/1 to Centosx64_ESXi-disk1
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 10.0G  100 10.0G    0     0  55.6M      0  0:03:03  0:03:03 --:--:-- 50.9M
[ 202.8] Writing libvirt XML metadata to Centosx64_ESXi.xml ...
[ 202.8] Finishing off
[root@kvm02 v2v]# ls
Centosx64_ESXi-disk1  Centosx64_ESXi.xml
[root@kvm02 v2v]# virt-v2v -i libvirtxml Centosx64_ESXi.xml -o local -os /vm_data/v2v/ -of raw
[   0.0] Opening the source -i libvirtxml Centosx64_ESXi.xml
[   0.0] Creating an overlay to protect the source from being modified
[   0.1] Opening the overlay
[   4.2] Inspecting the overlay
[  11.7] Checking for sufficient free disk space in the guest
[  11.7] Estimating space required on target for each disk
[  11.7] Converting CentOS Linux release 7.6.1810 (Core)  to run on KVM
virt-v2v: This guest has virtio drivers installed.
[  41.3] Mapping filesystem data to avoid copying unused and blank areas
[  41.8] Closing the overlay
[  42.1] Assigning disks to buses
[  42.1] Checking if the guest needs BIOS or UEFI to boot
[  42.1] Initializing the target -o local -os /vm_data/v2v/
[  42.1] Copying disk 1/1 to /vm_data/v2v/Centosx64_ESXi-sda (raw)
    (100.00/100%)
[  50.6] Creating output metadata
[  50.6] Finishing off

迁移完毕

virsh define Centosx64_ESXi.xml
[root@kvm02 v2v]# virsh list --all
 Id    Name                           State
----------------------------------------------------
 -     Centos7x64_1810                shut off
 -     Centosx64_ESXi                 shut off

【友情提示】需要注意迁移虚拟机的网络状态，源虚拟机如果是桥接，迁移后需要编辑XML文件确定网络的连接方式。

    <interface type='bridge'>
      <mac address='00:0c:29:7a:fd:a7'/>
      <source bridge='VM Network'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>

改成默认网络模式即可上网Ping 通百度

    <interface type='network'>
      <mac address='52:54:00:e8:c3:2b'/>
      <source network='default'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>

4 XEN服务器迁移到KVM

方法同ESXi迁移方式！！！

5 KVM迁移到VMWare ESXi服务器(待续)