macOS KVM 虚拟机_mac kvm

在内网要做macos的编译 CI流水线，需要在zstack平台装一台虚拟机（它使用的是KVM，类似openstack）

尝试了好几种方式，最后选择了3

1. https://github.com/sickcodes/Docker-OSX之有试过docker里跑kvm的一个方法，确实可行，但不知道怎么设置网络，ping不通其他机器，作罢

2. 参考之前做linux windows虚拟机的经验，以为vmware装一个macos，做出来的vmdk 再用qemu-img转成qcow2可以在zstack上跑，无法启动，失败

3. https://github.com/foxlet/macOS-Simple-KVM这个可行，开始折腾了半天libvirtd 的vnc密码，后面发现空密码不行，必须要设置一个密码，才能vnc；其他就是一台物理机centos7上的qemu版本有点问题，跑不起来这个，换了一个zstack上的虚拟机，套娃跑这个虚拟机，可行，稍微调整一下配置文件里的qemu kvm二进制路径，usb设备等就行了

4. 还是想在zstack上直接跑这个macos，毕竟zstack做快照、备份方便点，要不就得自己人工做；但折腾了一下发现卡在clover选磁盘这里，对比了下zstack libvirtd qemu的配置，与macos-simple-kvm的UEFI固件有区别，[Solved] Clover hangs on "4630 scan entries" | tonymacx86.com结合这个，推断确实zstack上不能直接跑macos

5. 其实还有一种方式是gitlab runner的libvirtd exectuor方式，但只看到linux虚拟机的实践，mac和windows的没有看到

使用macOS-Simple-KVM，把仓库克隆，进入目录：

1. 外网找一台ubuntu22的机器 装好qemu等依赖

2. ./jumpstart.sh --catalina 下载苹果的catalina镜像 BaseSystem.img

3. 创建一个空磁盘 qemu-img create -f qcow2 MyDisk.qcow2 256G

4. 给basic.sh 添加两行, 然后运行basic.sh 安装macos后关虚拟机, 设置用户名密码（比如设置用户是user，密码是12345）

    -drive id=SystemDisk,if=none,file=MyDisk.qcow2 \
    -device ide-hd,bus=sata.4,drive=SystemDisk \

1. https://github.com/foxlet/macOS-Simple-KVM/issues/566按照这个这个编辑ESP.qcow2中的config.plist 让clover自动选磁盘(diskutil list 拿到 /dev/disk3 vm后面的一个名称 )，不然每次启动还要去选择从哪里启动

2. 把项目整个目录拿到内网，找个欧拉2003sp3的虚拟机（其他较新的系统也行，centos7似乎有问题），装好livirtd以及qemu-kvm一堆程序， 整个项目目录记得chown成qemu用户的，放到比如/opt/os下（qemu能访问到）， 运行 ./make.sh --add,  再用virsh edit macOS-Simple-KVM添加磁盘

3. 配置类似（根据你qemu的版本，需要调整路径等，微调），这里设置qemu的vnc密码为123

msu@msu-System-Product-Name:~$ sudo cat /etc/libvirt/qemu/macOS-Simple-KVM.xml
[sudo] password for msu:
<!--
WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE
OVERWRITTEN AND LOST. Changes to this xml configuration should be made using:
  virsh edit macOS-Simple-KVM
or other application using the libvirt API.
-->
 
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <name>macOS-Simple-KVM</name>
  <uuid>d06d502a-904a-4b34-847d-debf1a3d76c7</uuid>
  <memory unit='KiB'>2097152</memory>
  <currentMemory unit='KiB'>2097152</currentMemory>
  <vcpu placement='static'>4</vcpu>
  <os>
    <type arch='x86_64' machine='pc-q35-6.2'>hvm</type>
    <loader readonly='yes' type='pflash'>/media/msu/本地磁盘/macos/macOS-Simple-KVM/firmware/OVMF_CODE.fd</loader>
    <nvram>/media/msu/本地磁盘/macos/macOS-Simple-KVM/firmware/OVMF_VARS-1024x768.fd</nvram>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <vmport state='off'/>
  </features>
  <cpu mode='custom' match='exact' check='none'>
    <model fallback='forbid'>qemu64</model>
    <topology sockets='1' dies='1' cores='4' threads='1'/>
  </cpu>
  <clock offset='utc'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled='no'/>
    <suspend-to-disk enabled='no'/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/media/msu/本地磁盘/macos/macOS-Simple-KVM/ESP.qcow2'/>
      <target dev='sda' bus='sata'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw'/>
      <source file='/media/msu/本地磁盘/macos/macOS-Simple-KVM/BaseSystem.img'/>
      <target dev='sdb' bus='sata'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/media/msu/本地磁盘/macos/macOS-Simple-KVM/MyDisk.qcow2'/>
      <target dev='sdc' bus='sata'/>
      <address type='drive' controller='0' bus='0' target='0' unit='2'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x2'/>
    </controller>
    <controller type='sata' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'/>
    <interface type='network'>
      <mac address='52:54:00:92:d4:7b'/>
      <source network='default'/>
      <model type='e1000-82545em'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <input type='mouse' bus='usb'>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='keyboard' bus='usb'>
      <address type='usb' bus='0' port='2'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes' listen='0.0.0.0' keymap='en-us' passwd='123'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    <sound model='ich9'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1b' function='0x0'/>
    </sound>
    <audio id='1' type='spice'/>
    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
    </video>
    <redirdev bus='usb' type='spicevmc'>
      <address type='usb' bus='0' port='3'/>
    </redirdev>
    <redirdev bus='usb' type='spicevmc'>
      <address type='usb' bus='0' port='4'/>
    </redirdev>
    <memballoon model='none'/>
  </devices>
  <qemu:commandline>
    <qemu:arg value='-cpu'/>
    <qemu:arg value='Penryn,kvm=on,vendor=GenuineIntel,+invtsc,vmware-cpuid-freq=on,+pcid,+ssse3,+sse4.2,+popcnt,+avx,+aes,+xsave,+xsaveopt,check'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='isa-applesmc,osk=ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc'/>
    <qemu:arg value='-smbios'/>
    <qemu:arg value='type=2'/>
  </qemu:commandline>
</domain>

1. 最后virsh start macOS-Simple-KVM 跑起来

2. 网络是nat的 在宿主机上 删除libvirtd的规则 sudo iptables -D LIBVIRT_FWI 2， 与10的iptables配置冲突了

3. 在mac里增加屏幕共享（vnc），远程访问（ssh）添加转发规则 ，宿主机2222端口转发到虚拟机22端口，5999端口转发到虚拟机5900端口， 10.113.6.118为宿主机IP；qemu自身的vnc监听在宿主机的5900端口，但这个vnc不太好用，鼠标太飘

iptables -t nat -A PREROUTING -p tcp -d 10.113.6.118/32 --dport 2222 -j DNAT --to-destination 192.168.122.186:22
iptables -t nat -A PREROUTING -p tcp -d 10.113.6.118/32 --dport 5999 -j DNAT --to-destination 192.168.122.186:5900