- 1Spring Boot 整合RabbitMQ
- 2一文带你安装opencv和常用库（保姆级教程少走80%的弯路）_opencv安装教程
- 3【机器视觉技术】：开创人工智能新时代
- 4搭建 PXE 服务器（基于 Ubuntu 22.04，支持 IPv4/6 及 Legacy/UEFI）_ubuntu pxe
- 5【Git系列】Github指令搜索
- 6【算法专题】记忆化搜索
- 7手机技巧：安卓微信8.0.45测试版功能来了_微信8.0.45内测下载
- 8盘点国内热门AI大模型_国内ai大模型有哪些
- 9Solidity 介绍_solidity语言
- 10LeetCode 2409. Count Days Spent Together【前缀和,容斥原理】简单
qemu gutest network configuration_qemu libvirt 默认网卡
赞
踩
KVM Network Configuration
qemu-kvm guest network
KVM主机上一般网络配置有5种类型：
-
Using QEMU default network 10.0.2.15
-
Using a Linux bridge with NAT for KVM guests
-
Using a Linux bridge (without NAT) for KVM guests
-
Using an Open vSwitch bridge with KVM guests
-
Using the MacVTap driver with KVM guests
QEMU default
qemu启动虚机，如果不做网卡的特定配置，默认起来后网卡是10.0.2.15地址。
-device e1000,netdev=net0,mac=00:16:3e:0c:21:28
-netdev user,id=net0,hostfwd=tcp::19888-:22
- 1
- 2
这里加了个hostfwd进行端口转发，hostfwd=tcp::19888-:22的含义是将host主机的19888端口的tcp报文转发给guest虚机的22端口（ssh登陆）。
NAT-based Net
依赖支持：libvirt，tap.
这个是最常用的配置了，通过Linux bridge以及NAT的结合使用，使得guest OS可以通过任何网络形式（wired，wireless，dial-up等）连接到主机之外。
所有的guest虚机都可以通过该bridge互通，即使该bridge并没有连接到host主机的实际网卡上或者host根本就没有physical网卡，也不影响虚机之间通过该bridge互相通信。
该模式下，有3中主要特性（主要是NAT带来的特性）：
- 该NAT-based的bridge使用的是私网地址，192.168.x.x网段的地址；
- 因为guest躲在NAT之后，所以不同的host之间是无法感知到某个host内部如何分配ip给guest的，也即外部不能直通guest；
- 使用NAT和software bridge会带来额外的CPU和memory开销，网络throughput以及latency性能会打折扣；
通常，该模式下默认的NAT bridge为virbr0，默认的network为default。
[root@pc:ubuntu#] virsh net-list
Name State Autostart Persistent
----------------------------------------------------------
default active yes yes
[root@pc:ubuntu#] brctl show
bridge name bridge id STP enabled interfaces
virbr0 8000.5254003203df no tap0
virbr0-nic
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
Tutorial
1. 创建配置文件
创建配置文件：vi ~/jrgnet.xml
<network>
<name>newnatnetwork</name>
<forward mode=’nat’>
<nat>
<port start=’1024’ end=’65535’/>
</nat>
</forward>
<bridge name=’my-bridge-name’ stp=’on’ delay=’0’/>
<ip address=’192.168.X.1’ netmask=’255.255.255.0’>
<dhcp>
<range start=’192.168.X.2’ end=’192.168.X.254’/>
</dhcp>
</ip>
</network>
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
其中需要修改<name>, <bridge name>, <ip address>。
- name字段需要修改，比如jrgnet
- bridge name需要修改，比如jrgbr
- 192.168.X.1地址需要指明网段，比如192.168.128.1
我这里的jrgnet.xml内容如下
<network>
<name>jrgnet</name>
<forward mode='nat'>
<nat>
<port start='1024' end='65535'/>
</nat>
</forward>
<bridge name='jrgbr0' stp='on' delay='0'/>
<ip address='192.168.128.1' netmask='255.255.255.0'>
<dhcp>
<range start='192.168.128.2' end='192.168.128.254'/>
</dhcp>
</ip>
</network>
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
2. 创建新的network
virsh net-create ~/jrgnet.xml
该步骤中会自动创建配置文件中的bridge，比如jrgbr0，通过brctl show可以查看；virsh net-list可以查看创建的network比如jrgnet；
[root@pc:rg#] virsh net-list
Name State Autostart Persistent
----------------------------------------------------------
default active yes yes
jrgnet active no no
[root@pc:rg#] brctl show
bridge name bridge id STP enabled interfaces
jrgbr0 8000.525400012bad yes jrgbr0-nic
virbr0 8000.5254003203df no tap0
virbr0-nic
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
3. 启动新的network
virsh net-autostart <network-name-from-xml>，比如virsh net-autostart jrgnet。
不过这里可能会出问题
[root@pc:rg#] virsh net-autostart jrgnet
error: failed to mark network jrgnet as autostarted
error: Requested operation is not valid: cannot set autostart for transient network
- 1
- 2
- 3
解决办法：
应该是我们的配置文件jrgnet.xml没有通过virsh命令去配置
virsh net-edit jrgnet
并实际修改并保存再退出，然后运行virsh net-autostart jrgnet，这样就会自动拷贝jrgnet.xml 至*/etc/libvirtd/qemu/networks/autostart/*目录下面了，就可以了。
[root@pc:rg#] virsh net-autostart jrgnet
Network jrgnet marked as autostarted
[root@pc:rg#] virsh net-list
Name State Autostart Persistent
----------------------------------------------------------
default active yes yes
jrgnet active yes yes
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
4. 配置guest使用该network或bridge
我这里需要做两个操作。
- 修改qemu启动命令中，
br=virbr0改为br=jrgbr0 - 修改
/usr/local/etc/qemu/bridge.conf中添加allow jrgbr0
然后重启guest虚机，就可以看到内部的IP地址变为我们指定的subnet网段了，这里是192.168.128.96。
qemu命令中需要添加的网卡配置，其中mac地址可以随便改，只要br=jrgbr0对应到指定的bridge即可（注意需要qemu-bridge-helper的支持）。
-device e1000,netdev=nic0,mac=00:16:3e:0c:12:27 \
-netdev tap,id=nic0,br=jrgbr0,helper=/home/rg/project/qemu-5.1.0/build/qemu-bridge-helper,vhost=on \
- 1
- 2
/usr/local/etc/qemu/bridge.conf中的配置如下，可能第一次是没有该bridge.conf文件的，直接创建。
[root@pc:ubuntu#] cat /usr/local/etc/qemu/bridge.conf
allow virbr0
allow jrgbr0
- 1
- 2
- 3
Linux-bridge
使用原生的Linux网桥。
A network bridge is a Link Layer device which forwards traffic between networks based on MAC addresses and is therefore also referred to as a Layer 2 device. It makes forwarding decisions based on tables of MAC addresses which it builds by learning what hosts are connected to each network. A software bridge can be used within a Linux host in order to emulate a hardware bridge, for example in virtualization applications for sharing a NIC with one or more virtual NICs.
Tutorial
1. 创建bridge
首先我们需要创建一个bridge，其slave是我们想和外部网络连接的interface。可以参考这篇文章，这样虚拟机可以基于该bridge和外部网络进行直连。
可以使用nmtui界面化工具配置网桥，配置完成后systemctl restart network即可。
配置完bridge，比如我这里配置jrgbr1，其效果如下
[root@pc:ubuntu#] brctl show bridge name bridge id STP enabled interfaces jrgbr0 8000.525400012bad no jrgbr0-nic jrgbr1 8000.008cfac79928 no ens1f0 virbr0 8000.5254003203df no virbr0-nic [root@pc:ubuntu#] nmcli connection show --active NAME UUID TYPE DEVICE br1 3d2c04c9-6016-483c-9a4f-d2e31c642c79 bridge jrgbr1 jrgbr0 2701003f-d845-4a41-b6cc-50ee7d027ebd bridge jrgbr0 virbr0 c0627583-68bc-4660-9a63-7957362c0871 bridge virbr0 br1-slave 0646fd10-1580-4229-890d-dd6c0f46f716 ethernet ens1f0 [root@pc:ubuntu#] ip address show master jrgbr1 2: ens1f0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc mq master jrgbr1 state UP group default qlen 1000 link/ether 00:8c:fa:c7:99:28 brd ff:ff:ff:ff:ff:ff [root@pc:ubuntu#] ip a show jrgbr1 17: jrgbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether 00:8c:fa:c7:99:28 brd ff:ff:ff:ff:ff:ff inet 172.17.81.244/24 brd 172.17.81.255 scope global noprefixroute jrgbr1 valid_lft forever preferred_lft forever inet6 fe80::28c:faff:fec7:9928/64 scope link valid_lft forever preferred_lft forever
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
2. qemu命令支持
qemu的启动命令其实不用怎么修改，只要修改br=jrgbr1即可，比如下面这个我启动了两张网卡，net0网卡启动后是10.0.2.15地址，nic0网卡启动后是配置static ip或dhcp获取ens1f0上级dhcpserver分配的地址。
-device e1000,netdev=net0,mac=00:16:3e:0c:21:28 \
-netdev user,id=net0,hostfwd=tcp::19888-:22 \
-device e1000,netdev=nic0,mac=00:16:3e:0c:12:27 \
-netdev tap,id=nic0,br=jrgbr1,helper=/home/rg/project/qemu-5.1.0/build/qemu-bridge-helper,vhost=on \
- 1
- 2
- 3
- 4
启动虚机后，可以看到我找了建立了两个虚拟网卡（不是必须两个网卡，一个网卡即可）。其中，mac地址00:16:3e:0c:21:28的网卡地址是10.0.2.15（这是qemu创建虚机默认配置的ip地址），另一张mac地址00:16:3e:0c:12:27的网卡地址这里还没有配置（这里我host主机的ens1f0物理网卡设置的是static ip，没有设置dhcp）。
设置ens4的ip地址，可以发现其已经能和host主机网络中的其他主机ping通。
ip addr add 172.17.81.27/24 dev ens4
- 1
这里再看一下，如果ping host主机网络中的非同一个网段的，直接ping貌似ping不通，这是因为我这里建立了两张网卡，ens3的ip地址为10.0.2.15，如果ping非同一网段的，按照默认路由走就走10.0.2.15网卡，走到qemu的虚拟网络协议栈，qemu默认网卡是不支持ping功能的，所以ping不通，如果ping的时候指定网卡ens4，则可以。
在host网络中其他主机内ping我们的guest OS虚机172.17.81.27，也是可以通的，bridge桥接模式创建虚机网卡设置成功。
OpenvSwitch
需要安装openvswitch
其主要步骤也即：
- 创建网桥，其slave是host主机的interface网卡
- 指定qemu使用该bridge
MacVTap
Macvtap is a new device driver meant to simplify virtualized bridged networking. It replaces the combination of the tun/tap and bridge drivers with a single module based on the macvlan device driver. A macvtap endpoint is a character device that largely follows the tun/tap ioctl interface and can be used directly by kvm/qemu and other hypervisors that support the tun/tap interface.
MacVTap可以配置3中模式：
-
默认模式是VEPA(Virtual Ethernet Port Aggregator，虚拟网络端口聚合)，数据从某个endpoint传输给外部switch，如果switch支持haripin功能，则数据还能回流给source设备然后传给其他endpoint。
Virtual Ethernet Port Aggregator (VEPA) is the default mode. Data flows from one endpoint down through the source device in the KVM host out to the external switch. If the switch supports hairpin mode, the data is sent back to the source device in the KVM host and from there sent to the destination endpoint.
Most switches today do not support hairpin mode, so the two endpoints are not able to exchange ethernet frames, although they might still be able to communicate using an tcp/ip router. A linux host used as the adjacent bridge can be put into hairpin mode by writing to /sys/class/net/dev/brif/port/ hairpin_mode. This mode is particularly interesting for data-centers and cloud provides where the ability to manage virtual machine networking at the switch level is desirable. Switches aware of the VEPA guests can enforce filtering and bandwidth limits per MAC address without the Linux host knowing about it. -
网桥模式，这是最常用的模式。两个终端之间直接传输帧数据。
Bridge, connecting all endpoints directly to each other. Two endpoints that are both in bridge mode can exchange frames directly, without the round trip through the external bridge. This is the most useful mode for setups with classic switches, and when inter-guest communication is performance critical.
原理图如下所示：
-
私有模式，终端之间不能互通。
For completeness, a private mode exists that behaves like a VEPA mode endpoint in the absence of a hairpin aware switch. Even when the switch is in hairpin mode, a private endpoint can never communicate to any other endpoint on the same lowerdev.
实例
附上一个完整的qemu启动命令以及host主机网桥配置。
qemu启动命令：
#!/bin/sh /home/rg/project/qemu-5.1.0/build/x86_64-softmmu/qemu-system-x86_64 \ -enable-kvm \ -m ${1:-8}G \ -cpu kvm64,+vmx \ -vga virtio \ -display default \ -smp ${2:-4} \ -device e1000,netdev=net0,mac=00:16:3e:0c:21:28 \ -netdev user,id=net0,hostfwd=tcp::19888-:22 \ -device e1000,netdev=nic0,mac=00:16:3e:0c:12:27 \ -netdev tap,id=nic0,br=jrgbr1,helper=/home/rg/project/qemu-5.1.0/build/qemu-bridge-helper,vhost=on \ -device virtio-serial \ -chardev spicevmc,id=vdagent,debug=0,name=vdagent \ -device virtserialport,chardev=vdagent,name=com.redhat.spice.0 \ -drive aio=threads,file=/home/rg/ubuntu/ubuntu-20.04.1.qcow2,if=virtio \ -name jrgtest,process=testjrg,debug-threads=on \ -spice port=8000,addr=172.17.81.244,disable-ticketing # -cdrom ./ubuntu-20.04.1-desktop-amd64.iso \ #用于初次安装ubuntu系统 # -device e1000,netdev=net0,mac=00:16:3e:0c:21:28 \ #用于默认的qemu网卡配置，并转发ssh登陆 # -netdev user,id=net0,hostfwd=tcp::19888-:22 \ # -device e1000,netdev=net0,mac=00:16:3e:0c:22:19 \ #用于默认的qemu网卡配置 # -netdev user,id=net0 \ # -device e1000,netdev=nic0,mac=00:16:3e:0c:12:27 \ #用于自创建的网桥NAT模式 # -netdev tap,id=nic0,br=jrgbr0,helper=/home/rg/project/qemu-5.1.0/build/qemu-bridge-helper,vhost=on \ # -device virtio-net-pci,netdev=nic0,mac=00:16:3e:0c:12:29 \ #用于默认default网络的网桥NAT模式 # -netdev tap,id=nic0,br=virbr0,helper=/home/rg/project/qemu-5.1.0/build/qemu-bridge-helper,vhost=on \ # jrgbr1 whose slave dev is ens1f0, guest set ip 172.17.81.x/24 #用于桥接到host主机网卡模式 # -device e1000,netdev=nic0,mac=00:16:3e:0c:12:27 \ # -netdev tap,id=nic0,br=jrgbr1,helper=/home/rg/project/qemu-5.1.0/build/qemu-bridge-helper,vhost=on
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
host主机网桥配置：tap0是启动qemu虚机其自动创建的，并不是我预先创建的
[root@pc:rg#] nmcli connection show --active NAME UUID TYPE DEVICE br1 3d2c04c9-6016-483c-9a4f-d2e31c642c79 bridge jrgbr1 jrgbr0 2701003f-d845-4a41-b6cc-50ee7d027ebd bridge jrgbr0 virbr0 c0627583-68bc-4660-9a63-7957362c0871 bridge virbr0 br1-slave 0646fd10-1580-4229-890d-dd6c0f46f716 ethernet ens1f0 tap0 efc78717-334e-4b8b-a86b-043c61dfb063 tun tap0 [root@pc:ubuntu#] brctl show bridge name bridge id STP enabled interfaces jrgbr0 8000.525400012bad no jrgbr0-nic jrgbr1 8000.008cfac79928 no ens1f0 tap0 virbr0 8000.5254003203df no virbr0-nic [root@pc:ubuntu#] virsh net-list Name State Autostart Persistent ---------------------------------------------------------- default active yes yes jrgnet active yes yes
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
default.xml配置文件：
[root@pc:ubuntu#] cat /etc/libvirt/qemu/networks/default.xml <!-- WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE OVERWRITTEN AND LOST. Changes to this xml configuration should be made using: virsh net-edit default or other application using the libvirt API. --> <network> <name>default</name> <uuid>f6dd973b-dceb-493e-88ac-6a5d830c23e9</uuid> <forward mode='nat'/> <bridge name='virbr0' stp='off' delay='0'/> <mac address='52:54:00:32:03:df'/> <ip address='192.168.122.1' netmask='255.255.255.0'> <dhcp> <range start='192.168.122.2' end='192.168.122.254'/> </dhcp> </ip> </network>
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
jrgnet.xml配置：
virsh net-eidt jrgnet 1 <network> 2 <name>jrgnet</name> 3 <uuid>d6d5dede-1ecd-4d1f-8ef2-86ee57b88535</uuid> 4 <forward mode='nat'> 5 <nat> 6 <port start='1024' end='65535'/> 7 </nat> 8 </forward> 9 <bridge name='jrgbr0' stp='off' delay='0'/> 10 <mac address='52:54:00:01:2b:ad'/> 11 <ip address='192.168.128.1' netmask='255.255.255.0'> 12 <dhcp> 13 <range start='192.168.128.2' end='192.168.128.254'/> 14 </dhcp> 15 </ip> 16 </network>\
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
bridge.conf配置：
[root@pc:ubuntu#] cat /usr/local/etc/qemu/bridge.conf
allow virbr0
allow jrgbr0
allow jrgbr1
- 1
- 2
- 3
- 4
Reference
[KVM Network Performance -Best Practices and Tuning Recommendations]
