热门标签
热门文章
- 1bluedroid层次结构图_bluedroid架构
- 230岁转行做程序员到底行不行？这篇文章告诉你，没有什么不可以，关键看你怎么做。
- 3K8S的Kafka监控(Prometheus+Grafana)_grafana监控kafka
- 4深度学习：基于Keras，使用长短期记忆人工神经网络模型(LSTM)对股票市场进行预测分析
- 5Unity-添加图片/文字_unity的image怎么添加图片
- 6搜索引擎的定义与运行原理
- 7如何监控 Redis 的性能？_redis 队列 消费速度监测
- 8从硅谷产品经理谈谈：AI产品经理要不要懂技术&算法？_williams的ml理论
- 9SpringBoot2.7.X整合SpringSecurity+JWT（入门级简单易懂）_spring-security-jwt
- 10数据结构-链表
当前位置: article > 正文
【dpdk】qat-getting-started-guide-with-dpdk-v2.0_dstqpdtkfbii
作者：Gausst松鼠会 | 2024-05-06 02:21:59
赞
踩
dstqpdtkfbii
文章目录
参考文档
632506-qat-getting-started-guide-v2.0.pdf
1.QAT driver installation
This section provides details on building and installing the software.
1.1. Installation using kernel.org driver
The examples below are based on the C62x device, if you have a different device use the corresponding values in the above table.
In BIOS ensure that SRIOV is enabled and either:
- Disable VT-d or
- Enable VT-d and set
"intel_iommu=on iommu=pt"in the grub file.
Check that the QAT driver is loaded on your system, by executing:
lsmod | grep qa
- 1
You should see the kernel module for your device listed, e.g.:
qat_c62x 5626 0
intel_qat 82336 1 qat_c62x
- 1
- 2
Next, you need to expose the Virtual Functions (VFs) using the sysfs file system.
First find the BDFs (Bus-Device-Function) of the physical functions (PFs) of your device, e.g.:
lspci -d:37c8
- 1
You should see output similar to:
1a:00.0 Co-processor: Intel Corporation Device 37c8
3d:00.0 Co-processor: Intel Corporation Device 37c8
3f:00.0 Co-processor: Intel Corporation Device 37c8
- 1
- 2
- 3
Enable the VFs for each PF by echoing the number of VFs per PF to the pci driver:
echo 16 > /sys/bus/pci/drivers/c6xx/0000:1a:00.0/sriov_numvfs
echo 16 > /sys/bus/pci/drivers/c6xx/0000:3d:00.0/sriov_numvfs
echo 16 > /sys/bus/pci/drivers/c6xx/0000:3f:00.0/sriov_numvfs
- 1
- 2
- 3
Check that the VFs are available for use. For example lspci -d:37c9 should list 48 VF devices available for a C62x device.
To complete the installation follow the instructions in Binding the available VFs to the vfio-pci driver.
#链接start
1.2. Binding the available VFs to the vfio-pci driver
Note:
- Please note that due to security issues, the usage of older DPDK igb_uio driver is not recommended. This document shows how to use the more secure vfio-pci driver.
- If QAT fails to bind to vfio-pci on Linux kernel 5.9+, please see the QATE-39220 and QATE-7495 issues in IDZ doc which details the constraint about trusted guests and add disable_denylist=1 to the vfio-pci params to use QAT. See also this patch description.
Unbind the VFs from the stock driver so they can be bound to the vfio-pci driver.
1.2.1. For an Intel® QuickAssist Technology DH895xCC device
The unbind command below assumes BDFs of 03:01.00-03:04.07, if your VFs are different adjust the unbind command below:
cd to the top-level DPDK directory
for device in $(seq 1 4); do \
for fn in $(seq 0 7); do \
usertools/dpdk-devbind.py -u 0000:03:0${device}.${fn}; \
done; \
done
- 1
- 2
- 3
- 4
- 5
- 6
1.2.2. For an Intel® QuickAssist Technology C62x device
The unbind command below assumes BDFs of 1a:01.00-1a:02.07, 3d:01.00-3d:02.07 and 3f:01.00-3f:02.07, if your VFs are different adjust the unbind command below:
cd to the top-level DPDK directory
for device in $(seq 1 2); do \
for fn in $(seq 0 7); do \
usertools/dpdk-devbind.py -u 0000:1a:0${device}.${fn}; \
usertools/dpdk-devbind.py -u 0000:3d:0${device}.${fn}; \
usertools/dpdk-devbind.py -u 0000:3f:0${device}.${fn}; \
done; \
done
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
1.2.3. For Intel® QuickAssist Technology C3xxx or 200xx or D15xx device
The unbind command below assumes BDFs of 01:01.00-01:02.07, if your VFs are different adjust the unbind command below:
cd to the top-level DPDK directory
for device in $(seq 1 2); do \
for fn in $(seq 0 7); do \
usertools/dpdk-devbind.py -u 0000:01:0${device}.${fn}; \
done; \
done
- 1
- 2
- 3
- 4
- 5
- 6
1.2.4. Bind to the vfio-pci driver
Load the vfio-pci driver, bind the VF PCI Device id to it using the dpdk-devbind.py script then use the --status option to confirm the VF devices are now in use by vfio-pci kernel driver, e.g. for the C62x device:
cd to the top-level DPDK directory
modprobe vfio-pci
usertools/dpdk-devbind.py -b vfio-pci 0000:03:01.1
usertools/dpdk-devbind.py --status
- 1
- 2
- 3
- 4
Use modprobe vfio-pci disable_denylist=1 from kernel 5.9 onwards. See note in the section Binding the available VFs to the vfio-pci driver above.
1.2.5. Testing
QAT SYM crypto PMD can be tested by running the test application:
cd ./<build_dir>/app/test
./dpdk-test -l1 -n1 -a <your qat bdf>
RTE>>cryptodev_qat_autotest
- 1
- 2
- 3
QAT ASYM crypto PMD can be tested by running the test application:
cd ./<build_dir>/app/test
./dpdk-test -l1 -n1 -a <your qat bdf>
RTE>>cryptodev_qat_asym_autotest
- 1
- 2
- 3
QAT compression PMD can be tested by running the test application:
cd ./<build_dir>/app/test
./dpdk-test -l1 -n1 -a <your qat bdf>
RTE>>compressdev_autotest
- 1
- 2
- 3
1.2.6. Debugging
There are 2 sets of trace available via the dynamic logging feature:
- pmd.qat.dp exposes trace on the data-path.
- pmd.qat.general exposes all other trace.
pmd.qat exposes both sets of traces. They can be enabled using the log-level option (where 8=maximum log level) on the process cmdline, e.g. using any of the following:
--log-level="pmd.qat.general,8"
--log-level="pmd.qat.dp,8"
--log-level="pmd.qat,8"
- 1
- 2
- 3
Note
The global RTE_LOG_DP_LEVEL overrides data-path trace so must be set to RTE_LOG_DEBUG to see all the trace. This variable is in config/rte_config.h for meson build. Also the dynamic global log level overrides both sets of trace, so e.g. no QAT trace would display in this case:
--log-level="7" --log-level="pmd.qat.general,8"
- 1
#链接end
Note
If the QAT kernel modules are not loaded and you see an error like Failed to load MMP firmware qat_895xcc_mmp.bin in kernel logs, this may be as a result of not using a distribution, but just updating the kernel directly.
Download firmware from the kernel firmware repo.
Copy qat binaries to /lib/firmware:
cp qat_895xcc.bin /lib/firmware
cp qat_895xcc_mmp.bin /lib/firmware
- 1
- 2
Change to your linux source root directory and start the qat kernel modules:
insmod ./drivers/crypto/qat/qat_common/intel_qat.ko
insmod ./drivers/crypto/qat/qat_dh895xcc/qat_dh895xcc.ko
- 1
- 2
Note
If you see the following warning in /var/log/messages it can be ignored: IOMMU should be enabled for SR-IOV to work correctly.
1.3. Installation using IDZ QAT driver
浏览器输入https://01.org/ 进入Projects->QuickAssist Technology
或：
Download the latest QuickAssist Technology Driver from Intel Developer Zone. Consult the Quick Start Guide at the same URL for further information.
The steps below assume you are:
- Building on a platform with one
C62xdevice. - Using package
qat1.7.l.4.2.0-000xx.tar.gz. - On Fedora26 kernel
4.11.11-300.fc26.x86_64.
In the BIOS ensure that SRIOV is enabled and VT-d is disabled.
Uninstall any existing QAT driver, for example by running:
./installer.sh uninstallin the directory where originally installed.
Build and install the SRIOV-enabled QAT driver:
mkdir /QAT
cd /QAT
# Copy the package to this location and unpack
tar zxof qat1.7.l.4.2.0-000xx.tar.gz
./configure --enable-icp-sriov=host
make install
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
You can use cat /sys/kernel/debug/qat<your device type and bdf>/version/fw to confirm the driver is correctly installed and is using firmware version 4.2.0. You can use lspci -d:37c9 to confirm the presence of the 16 VF devices available per C62x PF.
Confirm the driver is correctly installed and is using firmware version 4.2.0:
cat /sys/kernel/debug/qat<your device type and bdf>/version/fw
- 1
Confirm the presence of 48 VF devices - 16 per PF:
lspci -d:37c9
- 1
To complete the installation - follow instructions in 1.2.Binding the available VFs to the vfio-pci driver
Note
If using a later kernel and the build fails with an error relating to strict_stroul not being available apply the following patch:
/QAT/QAT1.6/quickassist/utilities/downloader/Target_CoreLibs/uclo/include/linux/uclo_platform.h + #if LINUX_VERSION_CODE >= KERNEL_VERSION(3,18,5) + #define STR_TO_64(str, base, num, endPtr) {endPtr=NULL; if (kstrtoul((str), (base), (num))) printk("Error strtoull convert %s\n", str); } + #else #if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,38) #define STR_TO_64(str, base, num, endPtr) {endPtr=NULL; if (strict_strtoull((str), (base), (num))) printk("Error strtoull convert %s\n", str); } #else #if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,25) #define STR_TO_64(str, base, num, endPtr) {endPtr=NULL; strict_strtoll((str), (base), (num));} #else #define STR_TO_64(str, base, num, endPtr) \ do { \ if (str[0] == '-') \ { \ *(num) = -(simple_strtoull((str+1), &(endPtr), (base))); \ }else { \ *(num) = simple_strtoull((str), &(endPtr), (base)); \ } \ } while(0) + #endif #endif #endif
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
Note
If the build fails due to missing header files you may need to do following:
sudo yum install zlib-devel
sudo yum install openssl-devel
sudo yum install libudev-devel
- 1
- 2
- 3
Note
If the build or install fails due to mismatching kernel sources you may need to do the following:
sudo yum install kernel-headers-`uname -r`
sudo yum install kernel-src-`uname -r`
sudo yum install kernel-devel-`uname -r`
- 1
- 2
- 3
Note
If the build fails on newer GCC versions (such as GCC 12) with an error relating to -lc not being found, apply the following patch:
/QAT/quickassist/lookaside/access_layer/src/Makefile
cd $(ICP_FINAL_OUTPUT_DIR);\
cmd="$(LINKER) $(LIB_SHARED_FLAGS) -o \
$(LIB_SHARED) $(ADDITIONAL_OBJECTS) $(ADDITIONAL_LIBS) *.o -lpthread -ludev \
- -Bstatic -L$(ADF_DIR)/src/build/$(ICP_OS)/$(ICP_OS_LEVEL) \
- -ladf_user -L$(OSAL_DIR)/src/build/$(ICP_OS)/$(ICP_OS_LEVEL)/ \
- -losal -Bdynamic -lc"; \
+ -Bstatic -L$(ADF_DIR)/src/build/$(ICP_OS)/$(ICP_OS_LEVEL) \
+ -ladf_user -L$(OSAL_DIR)/src/build/$(ICP_OS)/$(ICP_OS_LEVEL)/ \
+ -losal -Bdynamic -L/lib/x86_64-linux-gnu/ -lc"; \
echo "$$cmd"; \
$$cmd
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
Followed by this patch:
/QAT/quickassist/build_system/build_files/OS/linux_common_user_space_rules.mk
@echo 'Creating shared library ${LIB_SHARED}'; \
cd $($(PROG_ACY)_FINAL_OUTPUT_DIR);\
- echo $(LINKER) $(LIB_SHARED_FLAGS) -o $@ $(OBJECTS) $(ADDITIONAL_OBJECTS) -lc;\
- $(LINKER) $(LIB_SHARED_FLAGS) -o $@ $(OBJECTS) $(ADDITIONAL_OBJECTS) -lc ;
+ echo $(LINKER) $(LIB_SHARED_FLAGS) -o $@ $(OBJECTS) $(ADDITIONAL_OBJECTS) \
+ -L/lib/x86_64-linux-gnu/ -lc;\
+ $(LINKER) $(LIB_SHARED_FLAGS) -o $@ $(OBJECTS) $(ADDITIONAL_OBJECTS) \
+ -L/lib/x86_64-linux-gnu/ -lc ;
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
2.upstreamed acceleration software installation
2.1 QATzip @github仓库
2.2 QATlib @github仓库
3.实验过程（using kernel.org driver）
cat /etc/default/grub # If you change this file, run 'update-grub' afterwards to update # /boot/grub/grub.cfg. # For full documentation of the options in this file, see: # info -f grub -n 'Simple configuration' GRUB_DEFAULT=0 GRUB_TIMEOUT_STYLE=hidden GRUB_TIMEOUT=0 GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian` GRUB_CMDLINE_LINUX_DEFAULT="iommu=pt intel_iommu=on quiet splash" GRUB_CMDLINE_LINUX="" grub-mkconfig -o /boot/grub/grub.cfg reboot lsmod |grep qat qat_4xxx 20480 0 intel_qat 270336 1 qat_4xxx crc8 16384 1 intel_qat authenc 16384 1 intel_qat lspci |grep Co-processor 76:00.0 Co-processor: Intel Corporation Device 4942 (rev 40) 78:00.0 Co-processor: Intel Corporation Device 2710 f3:00.0 Co-processor: Intel Corporation Device 4942 (rev 40) f5:00.0 Co-processor: Intel Corporation Device 2710 cat /sys/bus/pci/drivers/4xxx/0000\: 0000:76:00.0/ 0000:f3:00.0/ #两个 echo 16 > /sys/bus/pci/drivers/4xxx/0000\:76\:00.0/sriov_numvfs cat /sys/bus/pci/drivers/4xxx/0000\:76\:00.0/sriov_numvfs 16 echo 16 > /sys/bus/pci/drivers/4xxx//0000\:f3\:00.0/sriov_numvfs cat /sys/bus/pci/drivers/4xxx//0000\:f3\:00.0/sriov_numvfs 16 lspci -d:4943 76:00.1 Co-processor: Intel Corporation Device 4943 (rev 40) 76:00.2 Co-processor: Intel Corporation Device 4943 (rev 40) 76:00.3 Co-processor: Intel Corporation Device 4943 (rev 40) 76:00.4 Co-processor: Intel Corporation Device 4943 (rev 40) 76:00.5 Co-processor: Intel Corporation Device 4943 (rev 40) 76:00.6 Co-processor: Intel Corporation Device 4943 (rev 40) 76:00.7 Co-processor: Intel Corporation Device 4943 (rev 40) 76:01.0 Co-processor: Intel Corporation Device 4943 (rev 40) 76:01.1 Co-processor: Intel Corporation Device 4943 (rev 40) 76:01.2 Co-processor: Intel Corporation Device 4943 (rev 40) 76:01.3 Co-processor: Intel Corporation Device 4943 (rev 40) 76:01.4 Co-processor: Intel Corporation Device 4943 (rev 40) 76:01.5 Co-processor: Intel Corporation Device 4943 (rev 40) 76:01.6 Co-processor: Intel Corporation Device 4943 (rev 40) 76:01.7 Co-processor: Intel Corporation Device 4943 (rev 40) 76:02.0 Co-processor: Intel Corporation Device 4943 (rev 40) f3:00.1 Co-processor: Intel Corporation Device 4943 (rev 40) f3:00.2 Co-processor: Intel Corporation Device 4943 (rev 40) f3:00.3 Co-processor: Intel Corporation Device 4943 (rev 40) f3:00.4 Co-processor: Intel Corporation Device 4943 (rev 40) f3:00.5 Co-processor: Intel Corporation Device 4943 (rev 40) f3:00.6 Co-processor: Intel Corporation Device 4943 (rev 40) f3:00.7 Co-processor: Intel Corporation Device 4943 (rev 40) f3:01.0 Co-processor: Intel Corporation Device 4943 (rev 40) f3:01.1 Co-processor: Intel Corporation Device 4943 (rev 40) f3:01.2 Co-processor: Intel Corporation Device 4943 (rev 40) f3:01.3 Co-processor: Intel Corporation Device 4943 (rev 40) f3:01.4 Co-processor: Intel Corporation Device 4943 (rev 40) f3:01.5 Co-processor: Intel Corporation Device 4943 (rev 40) f3:01.6 Co-processor: Intel Corporation Device 4943 (rev 40) f3:01.7 Co-processor: Intel Corporation Device 4943 (rev 40) f3:02.0 Co-processor: Intel Corporation Device 4943 (rev 40) modprobe vfio-pci lsmod |grep vfio vfio_pci 16384 0 vfio_pci_core 94208 1 vfio_pci vfio_iommu_type1 49152 0 vfio 61440 3 vfio_pci_core,vfio_iommu_type1,vfio_pci iommufd 73728 1 vfio irqbypass 16384 2 vfio_pci_core,kvm # unbind for device in $(seq 0 1); do \ for fn in $(seq 1 7); do \ usertools/dpdk-devbind.py -u 0000:76:0${device}.${fn}; \ done; \ done usertools/dpdk-devbind.py -u 0000:76:01.0 usertools/dpdk-devbind.py -u 0000:76:02.0 for device in $(seq 0 1); do \ for fn in $(seq 1 7); do \ usertools/dpdk-devbind.py -u 0000:f3:0${device}.${fn}; \ done; \ done usertools/dpdk-devbind.py -u 0000:f3:01.0 usertools/dpdk-devbind.py -u 0000:f3:02.0 # bind for device in $(seq 0 1); do \ for fn in $(seq 1 7); do \ usertools/dpdk-devbind.py -b vfio-pci 0000:76:0${device}.${fn}; \ done; \ done usertools/dpdk-devbind.py -b vfio-pci 0000:76:01.0 usertools/dpdk-devbind.py -b vfio-pci 0000:76:02.0 for device in $(seq 0 1); do \ for fn in $(seq 1 7); do \ usertools/dpdk-devbind.py -b vfio-pci 0000:f3:0${device}.${fn}; \ done; \ done usertools/dpdk-devbind.py -b vfio-pci 0000:f3:01.0 usertools/dpdk-devbind.py -b vfio-pci 0000:f3:02.0 示例： ./usertools/dpdk-devbind.py -b vfio-pci 0000:76:00.1 spdk-23.09/dpdk# ./usertools/dpdk-devbind.py --status Network devices using kernel driver =================================== 0000:19:00.0 'Ethernet Controller X710 for 10GbE SFP+ 1572' if=ens38f0 drv=i40e unused=vfio-pci *Active* 0000:19:00.1 'Ethernet Controller X710 for 10GbE SFP+ 1572' if=ens38f1 drv=i40e unused=vfio-pci 0000:2a:00.0 'MT28908 Family [ConnectX-6] 101b' if=ens3f0np0 drv=mlx5_core unused=vfio-pci 0000:2a:00.1 'MT28908 Family [ConnectX-6] 101b' if=ens3f1np1 drv=mlx5_core unused=vfio-pci No 'Baseband' devices detected ============================== Crypto devices using DPDK-compatible driver =========================================== 0000:76:00.1 'Device 4943' drv=vfio-pci unused= 0000:76:00.2 'Device 4943' drv=vfio-pci unused= 0000:76:00.3 'Device 4943' drv=vfio-pci unused= 0000:76:00.4 'Device 4943' drv=vfio-pci unused= 0000:76:00.5 'Device 4943' drv=vfio-pci unused= 0000:76:00.6 'Device 4943' drv=vfio-pci unused= 0000:76:00.7 'Device 4943' drv=vfio-pci unused= 0000:76:01.0 'Device 4943' drv=vfio-pci unused= 0000:76:01.1 'Device 4943' drv=vfio-pci unused= 0000:76:01.2 'Device 4943' drv=vfio-pci unused= 0000:76:01.3 'Device 4943' drv=vfio-pci unused= 0000:76:01.4 'Device 4943' drv=vfio-pci unused= 0000:76:01.5 'Device 4943' drv=vfio-pci unused= 0000:76:01.6 'Device 4943' drv=vfio-pci unused= 0000:76:01.7 'Device 4943' drv=vfio-pci unused= 0000:76:02.0 'Device 4943' drv=vfio-pci unused= 0000:f3:00.1 'Device 4943' drv=vfio-pci unused= 0000:f3:00.2 'Device 4943' drv=vfio-pci unused= 0000:f3:00.3 'Device 4943' drv=vfio-pci unused= 0000:f3:00.4 'Device 4943' drv=vfio-pci unused= 0000:f3:00.5 'Device 4943' drv=vfio-pci unused= 0000:f3:00.6 'Device 4943' drv=vfio-pci unused= 0000:f3:00.7 'Device 4943' drv=vfio-pci unused= 0000:f3:01.0 'Device 4943' drv=vfio-pci unused= 0000:f3:01.1 'Device 4943' drv=vfio-pci unused= 0000:f3:01.2 'Device 4943' drv=vfio-pci unused= 0000:f3:01.3 'Device 4943' drv=vfio-pci unused= 0000:f3:01.4 'Device 4943' drv=vfio-pci unused= 0000:f3:01.5 'Device 4943' drv=vfio-pci unused= 0000:f3:01.6 'Device 4943' drv=vfio-pci unused= 0000:f3:01.7 'Device 4943' drv=vfio-pci unused= 0000:f3:02.0 'Device 4943' drv=vfio-pci unused= Crypto devices using kernel driver ================================== 0000:76:00.0 'Device 4942' drv=4xxx unused=qat_4xxx,vfio-pci 0000:f3:00.0 'Device 4942' drv=4xxx unused=qat_4xxx,vfio-pci Other Crypto devices ==================== 0000:78:00.0 'Device 2710' unused=vfio-pci 0000:f5:00.0 'Device 2710' unused=vfio-pci DMA devices using kernel driver =============================== 0000:75:01.0 'Device 0b25' drv=idxd unused=vfio-pci 0000:f2:01.0 'Device 0b25' drv=idxd unused=vfio-pci Other Eventdev devices ====================== 0000:78:00.0 'Device 2710' unused=vfio-pci 0000:f5:00.0 'Device 2710' unused=vfio-pci No 'Mempool' devices detected ============================= No 'Compress' devices detected ============================== No 'Misc (rawdev)' devices detected =================================== No 'Regex' devices detected =========================== No 'ML' devices detected ======================== # 分配hugepage cd dpdk ./usertools/dpdk-hugepages.py --setup 2G 执行./build/app/test/dpdk-test -l1 -n1 -a 0000:76:00.1 **************************************************cryptodev_qat_autotest*********************** cd dpdk ./build/app/test/dpdk-test -l1 -n1 -a 0000:76:00.1 EAL: Detected CPU lcores: 80 EAL: Detected NUMA nodes: 2 EAL: Detected static linkage of DPDK EAL: Multi-process socket /var/run/dpdk/rte/mp_socket EAL: Selected IOVA mode 'VA' EAL: VFIO support initialized EAL: Using IOMMU type 1 (Type 1) EAL: Probe PCI driver: qat (8086:4943) device: 0000:76:00.1 (socket 0) CRYPTODEV: Creating cryptodev 0000:76:00.1_qat_sym CRYPTODEV: Initialisation parameters - name: 0000:76:00.1_qat_sym,socket id: 0, max queue pairs: 0 CRYPTODEV: Creating cryptodev 0000:76:00.1_qat_asym CRYPTODEV: Initialisation parameters - name: 0000:76:00.1_qat_asym,socket id: 0, max queue pairs: 0 TELEMETRY: No legacy callbacks, legacy socket not created APP: HPET is not enabled, using TSC as default timer RTE>>cryptodev_qat_autotest ... + DOCSIS Proto Unit Test Suite : 52/52 passed, 0/52 skipped, 0/52 failed, 0/52 unsupported + ------------------------------------------------------- + + Sub Testsuites Total : 29 + Sub Testsuites Skipped : 13 + Sub Testsuites Passed : 16 + Sub Testsuites Failed : 0 + ------------------------------------------------------- + + Tests Total : 792 + Tests Skipped : 496 + Tests Executed : 494 + Tests Unsupported: 0 + Tests Passed : 296 + Tests Failed : 0 + ------------------------------------------------------- + Test OK ***************test_compressdev negative test, errors are expected*********************** + TestCase [25] : test_compressdev_deflate_im_buffers_SGL_over_2ops_first failed qat_comp_qp_setup(): qp_id 0 invalid for this device USER1: Queue pair setup failed rte_compressdev_stop(): Device with dev_id=0 already stopped + TestCase [26] : test_compressdev_deflate_im_buffers_SGL_over_2ops_second failed + ------------------------------------------------------- + + Test Suite Summary : compressdev unit test suite + ------------------------------------------------------- + + Tests Total : 27 + Tests Skipped : 0 + Tests Executed : 27 + Tests Unsupported: 0 + Tests Passed : 1 + Tests Failed : 26 + ------------------------------------------------------- + Test Failed RTE>>quit 更加详细的日志 spdk-23.09/dpdk# ./build/app/test/dpdk-test -l1 -n1 -a 76:00.1 --log-level=qat:8 EAL: Detected CPU lcores: 80 EAL: Detected NUMA nodes: 2 EAL: Detected static linkage of DPDK EAL: Multi-process socket /var/run/dpdk/rte/mp_socket EAL: Selected IOVA mode 'VA' EAL: VFIO support initialized EAL: Using IOMMU type 1 (Type 1) EAL: Probe PCI driver: qat (8086:4943) device: 0000:76:00.1 (socket 0) CRYPTODEV: Creating cryptodev 0000:76:00.1_qat_sym CRYPTODEV: Initialisation parameters - name: 0000:76:00.1_qat_sym,socket id: 0, max queue pairs: 0 CRYPTODEV: Creating cryptodev 0000:76:00.1_qat_asym CRYPTODEV: Initialisation parameters - name: 0000:76:00.1_qat_asym,socket id: 0, max queue pairs: 0 TELEMETRY: No legacy callbacks, legacy socket not created APP: HPET is not enabled, using TSC as default timer RTE>>compressdev_autotest + ------------------------------------------------------- + + Test Suite : compressdev unit test suite USER1: Running tests on device 0000:76:00.1_qat_comp + ------------------------------------------------------- + USER1: This is a negative test, errors are expected rte_compressdev_queue_pairs_config(): invalid param: dev 0x5599e5f163c0, nb_queues 0 rte_compressdev_configure(): dev0 rte_comp_dev_queue_pairs_config = -22 rte_compressdev_queue_pair_setup(): Invalid queue_pair_id=0 + TestCase [ 0] : test_compressdev_invalid_configuration succeeded qat_comp_qp_setup(): qp_id 0 invalid for this device USER1: Queue pair setup failed rte_compressdev_stop(): Device with dev_id=0 already stopped + TestCase [ 1] : test_compressdev_deflate_stateless_fixed failed qat_comp_qp_setup(): qp_id 0 invalid for this device USER1: Queue pair setup failed rte_compressdev_stop(): Device with dev_id=0 already stopped + TestCase [ 2] : test_compressdev_deflate_stateless_dynamic failed qat_comp_qp_setup(): qp_id 0 invalid for this device USER1: Queue pair setup failed rte_compressdev_stop(): Device with dev_id=0 already stopped + TestCase [ 3] : test_compressdev_deflate_stateless_dynamic_big failed qat_comp_qp_setup(): qp_id 0 invalid for this device USER1: Queue pair setup failed rte_compressdev_stop(): Device with dev_id=0 already stopped + TestCase [ 4] : test_compressdev_deflate_stateless_multi_op failed qat_comp_qp_setup(): qp_id 0 invalid for this device
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
- 71
- 72
- 73
- 74
- 75
- 76
- 77
- 78
- 79
- 80
- 81
- 82
- 83
- 84
- 85
- 86
- 87
- 88
- 89
- 90
- 91
- 92
- 93
- 94
- 95
- 96
- 97
- 98
- 99
- 100
- 101
- 102
- 103
- 104
- 105
- 106
- 107
- 108
- 109
- 110
- 111
- 112
- 113
- 114
- 115
- 116
- 117
- 118
- 119
- 120
- 121
- 122
- 123
- 124
- 125
- 126
- 127
- 128
- 129
- 130
- 131
- 132
- 133
- 134
- 135
- 136
- 137
- 138
- 139
- 140
- 141
- 142
- 143
- 144
- 145
- 146
- 147
- 148
- 149
- 150
- 151
- 152
- 153
- 154
- 155
- 156
- 157
- 158
- 159
- 160
- 161
- 162
- 163
- 164
- 165
- 166
- 167
- 168
- 169
- 170
- 171
- 172
- 173
- 174
- 175
- 176
- 177
- 178
- 179
- 180
- 181
- 182
- 183
- 184
- 185
- 186
- 187
- 188
- 189
- 190
- 191
- 192
- 193
- 194
- 195
- 196
- 197
- 198
- 199
- 200
- 201
- 202
- 203
- 204
- 205
- 206
- 207
- 208
- 209
- 210
- 211
- 212
- 213
- 214
- 215
- 216
- 217
- 218
- 219
- 220
- 221
- 222
- 223
- 224
- 225
- 226
- 227
- 228
- 229
- 230
- 231
- 232
- 233
- 234
- 235
- 236
- 237
- 238
- 239
- 240
- 241
- 242
- 243
- 244
- 245
- 246
- 247
- 248
- 249
- 250
- 251
- 252
- 253
- 254
- 255
- 256
- 257
- 258
- 259
- 260
- 261
- 262
- 263
- 264
- 265
- 266
- 267
- 268
- 269
- 270
- 271
- 272
- 273
- 274
- 275
- 276
- 277
- 278
- 279
- 280
- 281
- 282
- 283
- 284
- 285
- 286
- 287
- 288
- 289
- 290
- 291
- 292
- 293
- 294
- 295
- 296
- 297
- 298
- 299
- 300
- 301
- 302
- 303
- 304
- 305
- 306
- 307
- 308
- 309
- 310
- 311
- 312
- 313
4.实验过程（using IDZ QAT driver）
说明，实际并未使用该种方式。
4.1.Installing Intel QAT (IDZ 方式)
-
下载安装driver
https://github.com/intel/qatlib
./configure; make install
# Sample programs can be installed by using the “make samples-install”. The sample program “cpa_sample_code runTests=32” can be used to test compression performance.
- 1
- 2
- 检查安装状态
root@hik-Super-Server:/home/lhx/spdk-23.09# lsmod | grep qat
qat_4xxx 20480 0
intel_qat 270336 1 qat_4xxx
crc8 16384 1 intel_qat
authenc 16384 1 intel_qat
- 1
- 2
- 3
- 4
- 5
- 6
- 检查 service 状态
# service qat_service start||stop||status||restart||shutdown # For a system with multiple Intel® QAT Endpoints, you can start, stop or restart each individual device by passing the Intel® QAT Endpoint to be restarted or stopped as a parameter qat_dev<N>, for example: service qat_service stop qat_dev0 service qat_service stop qat_dev1 # The shutdown qualifier enables the user to bring down all Intel® QAT Endpoints and unload driver modules from the kernel. This contrasts with the stop qualifier, which brings down one or more Intel® QAT Endpoints, but does not unload kernel modules, so other Intel® QAT Endpoints can still run. # service qat_service status Checking status of all devices. There is 3 QAT acceleration device(s) in the system: qat_dev0 - type: c6xx, inst_id: 0, node_id: 1, bsf: 0000:cc:00.0, #accel: 5 #engines: 10 state: up qat_dev1 - type: c6xx, inst_id: 1, node_id: 1, bsf: 0000:ce:00.0, #accel: 5 #engines: 10 state: up qat_dev2 - type: c6xx, inst_id: 2, node_id: 1, bsf: 0000:d0:00.0, #accel: 5 #engines: 10 state: up
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
4.2.Intel QAT Settings
# 位置 quickassist/utilities/adf_ctl/conf_files 以QAT20.L.1.0.50-00003\quickassist\utilities\adf_ctl\conf_files\4xxx_template.conf为例 并参考programmer guide 进行配置 [GENERAL] ServicesEnabled = asym;dc ConfigVersion = 2 #Default values for number of concurrent requests*/ CyNumConcurrentSymRequests = 512 CyNumConcurrentAsymRequests = 64 #Statistics, valid values: 1,0 statsGeneral = 1 statsDh = 1 statsDrbg = 1 statsDsa = 1 statsEcc = 1 statsKeyGen = 1 statsDc = 1 statsLn = 1 statsPrime = 1 statsRsa = 1 statsSym = 1 # Default heartbeat timer is 1s HeartbeatTimer = 1000 # This flag is to enable SSF features StorageEnabled = 0 # Disable public key crypto and prime number # services by specifying a value of 1 (default is 0) PkeServiceDisabled = 0 # This flag is to enable device auto reset on heartbeat error AutoResetOnError = 0 # Default value for power management idle interrrupt delay PmIdleInterruptDelay = 512 # This flag is to enable power management idle support PmIdleSupport = 1 # This flag is to enable key protection technology KptEnabled = 1 # Define the maximum SWK count per function can have # Default value is 1, the maximum value is 128 KptMaxSWKPerFn = 1 # Define the maximum SWK count per pasid can have # Default value is 1, the maximum value is 128 KptMaxSWKPerPASID = 1 # Define the maximum SWK lifetime in second # Default value is 0 (eternal of life) # The maximum value is 31536000 (one year) KptMaxSWKLifetime = 31536000 # Flag to define whether to allow SWK to be shared among processes # Default value is 0 (shared mode is off) KptSWKShared = 0 # Disable AT ATEnabled = 0 ############################################## # Kernel Instances Section ############################################## [KERNEL] NumberCyInstances = 0 NumberDcInstances = 0 ############################################## # ADI Section for Scalable IOV ############################################## [SIOV] NumberAdis = 0 ############################################## # User Process Instance Section ############################################## [SSL] NumberCyInstances = 2 NumberDcInstances = 2 NumProcesses = 1 LimitDevAccess = 0 # Crypto - User instance #0 Cy0Name = "SSL0" Cy0IsPolled = 1 # List of core affinities Cy0CoreAffinity = 0 # Crypto - User instance #1 Cy1Name = "SSL1" Cy1IsPolled = 1 # List of core affinities Cy1CoreAffinity = 1 # Data Compression - User instance #0 Dc0Name = "Dc0" Dc0IsPolled = 1 # List of core affinities Dc0CoreAffinity = 2 # Data Compression - User instance #1 Dc1Name = "Dc1" Dc1IsPolled = 1 # List of core affinities Dc1CoreAffinity = 3
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
- 71
- 72
- 73
- 74
- 75
- 76
- 77
- 78
- 79
- 80
- 81
- 82
- 83
- 84
- 85
- 86
- 87
- 88
- 89
- 90
- 91
- 92
- 93
- 94
- 95
- 96
- 97
- 98
- 99
- 100
- 101
- 102
- 103
- 104
- 105
- 106
- 107
- 108
- 109
- 110
- 111
- 112
- 113
声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/Gausst松鼠会/article/detail/541974
推荐阅读
相关标签
