这是项目《人脸识别Face Recognition》系列之《Android InsightFace实现人脸识别Face Recognition》；项目基于开源ArcFace(也称InsightFace)模型搭建一套完整的Android人脸识别系统（Face Recognition or Face Identification）；我们将开发一个简易的、可实时运行的人脸识别Android Demo。Android版本人脸识别模型推理支持CPU和GPU加速，在GPU(OpenCL)加速下，可以达到实时的人脸识别效果，非常适合在Linux开发板和Android系统开发板上部署。

整套人脸识别系统核心算法包含人脸检测和人脸关键点检测，人脸校准，人脸特征提取以及人脸比对(1:1)和人脸搜索(1:N)。本项目人脸识别系统可以达到目前商业级别的人脸识别准确率，在误识率(FAR)0.1%的情况下，可提供99.78%的通过率(TAR)；可以满足人脸比对，人脸签到、人脸门禁、人员信息查询、安防监控等人脸识别应用场景。

Android版本人脸检测和人脸识别效果:

【尊重原创，转载请注明出处】https://blog.csdn.net/guyuealian/article/details/130600600

更多项目《人脸识别Face Recognition》系列文章请参考：

人脸识别1：人脸识别数据集https://blog.csdn.net/guyuealian/article/details/130600545
人脸识别2：InsightFace实现人脸识别Face Recognition(含源码下载)人脸识别2：InsightFace实现人脸识别Face Recognition(含源码下载)_insightface 识别_AI吃大瓜的博客-CSDN博客
人脸识别3：C/C++ InsightFace实现人脸识别Face Recognition(含源码)人脸识别3：C/C++ InsightFace实现人脸识别Face Recognition(含源码)_AI吃大瓜的博客-CSDN博客
人脸识别4：Android InsightFace实现人脸识别Face Recognition(含源码)https://blog.csdn.net/guyuealian/article/details/130600600

2. 项目说明

项目依赖库主要有OpenCV，base-utils，TNN和OpenCL(用于加速)，项目源码已经包含了相关依赖库，且都已经配置好，无需安装；使用Android Studio直接build即可运行App Demo ；

（1）开发版本

Android SDK,NDK，Jave等版本信息，请参考：

（2）依赖库说明(OpenCV+OpenCL+base-utils+TNN)

项目模型推理采用TNN部署框架（支持多线程CPU和GPU加速推理）；图像处理采用OpenCV库，模型加速采用OpenCL，在普通手机设备即可达到实时处理。项目Android源码已经配置好OpenCV+OpenCL+base-utils+TNN依赖库，无需重新配置，Android Studio直接build，即可运行。

OpenCV：图像处理（如读取图片，图像裁剪等）都需要使用OpenCV库进行处理（无需安装，项目已经配置了）
OpenCL：OpenCL用于模型GPU加速，若不使用OpenCL进行模型推理加速，纯C++推理模型，速度会特别特别慢（无需安装，项目已经配置了）
base-utils：是个人开发常用的C++库，集成了C/C++ OpenCV等常用的算法：https://github.com/PanJinquan/base-utils （无需安装，项目已经配置了）
TNN：模型推理框架：https://github.com/Tencent/TNN （无需安装，项目已经配置了）

（3）CMake配置

人脸识别核心算法均采用C++实现，上层Java应用使用JNI调用底层算法，CMake最低版本3.5.0，这是CMakeLists.txt，其中主要配置OpenCV+OpenCL+base-utils+TNN这四个库：


cmake_minimum_required(VERSION 3.5.0)
project("TNN")
add_compile_options(-fPIC) # fix Bug: can not be used when making a shared object
#set(CMAKE_BUILD_TYPE Release)
#set(CMAKE_BUILD_TYPE "Release" CACHE STRING "Build type (default Debug)" FORCE)
#set(CMAKE_CXX_FLAGS "-Wall -std=c++11 -pthread")
set(CMAKE_BUILD_TYPE "Release" CACHE STRING "set build type to release" FORCE)
 
# opencv set
# copy `OpenCV-android-sdk/sdk` to `3rdparty/opencv/`
set(OpenCV_DIR ${CMAKE_SOURCE_DIR}/3rdparty/opencv/sdk/native/jni)
find_package(OpenCV REQUIRED)
include_directories(${CMAKE_SOURCE_DIR}/3rdparty/opencv/sdk/native/jni/include)
 
 
# base_utils
set(BASE_ROOT 3rdparty/base-utils) # 设置base-utils所在的根目录
add_subdirectory(${BASE_ROOT}/base_utils/ base_build) # 添加子目录到build中
include_directories(${BASE_ROOT}/base_utils/include)
include_directories(${BASE_ROOT}/base_utils/src)
MESSAGE(STATUS "BASE_ROOT = ${BASE_ROOT}")
 
 
 
# TNN set
# Creates and names a library, sets it as either STATIC
# or SHARED, and provides the relative paths to its source code.
# You can define multiple libraries, and CMake builds it for you.
# Gradle automatically packages shared libraries with your APK.
# build for platform
# set(TNN_BUILD_SHARED OFF CACHE BOOL "" FORCE)
if (CMAKE_SYSTEM_NAME MATCHES "Android")
    set(TNN_OPENCL_ENABLE ON CACHE BOOL "" FORCE)
    set(TNN_ARM_ENABLE ON CACHE BOOL "" FORCE)
    set(TNN_BUILD_SHARED OFF CACHE BOOL "" FORCE)
    set(TNN_OPENMP_ENABLE ON CACHE BOOL "" FORCE)  # Multi-Thread
    #set(TNN_HUAWEI_NPU_ENABLE OFF CACHE BOOL "" FORCE)
    add_definitions(-DTNN_OPENCL_ENABLE)           # for OpenCL GPU
    add_definitions(-DTNN_ARM_ENABLE)              # for Android CPU
    add_definitions(-DDEBUG_ANDROID_ON)            # for Android Log
    add_definitions(-DPLATFORM_ANDROID)
elseif (CMAKE_SYSTEM_NAME MATCHES "Linux")
    set(TNN_OPENCL_ENABLE ON CACHE BOOL "" FORCE)
    set(TNN_CPU_ENABLE ON CACHE BOOL "" FORCE)
    set(TNN_X86_ENABLE OFF CACHE BOOL "" FORCE)
    set(TNN_QUANTIZATION_ENABLE OFF CACHE BOOL "" FORCE)
    set(TNN_OPENMP_ENABLE ON CACHE BOOL "" FORCE)  # Multi-Thread
    add_definitions(-DTNN_OPENCL_ENABLE)           # for OpenCL GPU
    add_definitions(-DDEBUG_ON)                    # for WIN/Linux Log
    add_definitions(-DDEBUG_LOG_ON)                # for WIN/Linux Log
    add_definitions(-DDEBUG_IMSHOW_OFF)            # for OpenCV show
    add_definitions(-DPLATFORM_LINUX)
elseif (CMAKE_SYSTEM_NAME MATCHES "Windows")
    set(TNN_OPENCL_ENABLE ON CACHE BOOL "" FORCE)
    set(TNN_CPU_ENABLE ON CACHE BOOL "" FORCE)
    set(TNN_X86_ENABLE ON CACHE BOOL "" FORCE)
    set(TNN_QUANTIZATION_ENABLE OFF CACHE BOOL "" FORCE)
    set(TNN_OPENMP_ENABLE ON CACHE BOOL "" FORCE)  # Multi-Thread
    add_definitions(-DTNN_OPENCL_ENABLE)           # for OpenCL GPU
    add_definitions(-DDEBUG_ON)                    # for WIN/Linux Log
    add_definitions(-DDEBUG_LOG_ON)                # for WIN/Linux Log
    add_definitions(-DDEBUG_IMSHOW_OFF)            # for OpenCV show
    add_definitions(-DPLATFORM_WINDOWS)
endif ()
set(TNN_ROOT 3rdparty/TNN)
include_directories(${TNN_ROOT}/include)
include_directories(${TNN_ROOT}/third_party/opencl/include)
add_subdirectory(${TNN_ROOT}) # 添加外部项目文件夹
set(TNN -Wl,--whole-archive TNN -Wl,--no-whole-archive)# set TNN library
MESSAGE(STATUS "TNN_ROOT = ${TNN_ROOT}")
 
# NPU Set
if (TNN_HUAWEI_NPU_ENABLE)
    add_library(hiai
            SHARED
            IMPORTED)
    set_target_properties(hiai
            PROPERTIES
            IMPORTED_LOCATION
            ${CMAKE_SOURCE_DIR}/src/main/jni/thirdparty/hiai_ddk/${ANDROID_ABI}/libhiai.so)
 
    add_library(hiai_ir
            SHARED
            IMPORTED)
    set_target_properties(hiai_ir
            PROPERTIES
            IMPORTED_LOCATION
            ${CMAKE_SOURCE_DIR}/src/main/jni/thirdparty/hiai_ddk/${ANDROID_ABI}/libhiai_ir.so)
 
    add_library(hiai_ir_build
            SHARED
            IMPORTED)
    set_target_properties(hiai_ir_build
            PROPERTIES
            IMPORTED_LOCATION
            ${CMAKE_SOURCE_DIR}/src/main/jni/thirdparty/hiai_ddk/${ANDROID_ABI}/libhiai_ir_build.so)
 
endif ()
 
 
find_library( # Sets the name of the path variable.
        log-lib
        # Specifies the name of the NDK library that
        # you want CMake to locate.
        log)
 
# Specifies libraries CMake should link to your target library. You
# can link multiple libraries, such as libraries you define in the
# build script, prebuilt third-party libraries, or system libraries.
 
# dmcv库
include_directories(src)
 
set(SRC_LIST
        src/face_alignment.cpp
        src/face_recognizer.cpp
        src/face_feature.cpp
        src/object_detection.cpp
        src/Interpreter.cpp)
 
MESSAGE(STATUS "DIR_SRCS = ${SRC_LIST}")
 
# JNI接口库
add_library(tnn_wrapper SHARED jni_interface.cpp ${SRC_LIST})
target_link_libraries( # Specifies the target library.
        tnn_wrapper
        -ljnigraphics
        # Links the target library to the log library
        # included in the NDK.
        ${log-lib}
        ${android-lib}
        ${jnigraphics-lib}
        ${TNN}
        ${OpenCV_LIBS}
        base_utils
        )
 
if (TNN_HUAWEI_NPU_ENABLE)
    target_link_libraries( # Specifies the target library.
            tnn_wrapper hiai hiai_ir hiai_ir_build)
endif ()

3. 人脸识别系统

人脸识别主要包含人脸比对(1:1)和人脸搜索(1:N)两大功能，涉及的核心算法主要包含：人脸检测和人脸关键点检测，人脸校准，人脸特征提取以及人脸比对(1:1)和人脸搜索(1:N)；当然，实际业务中，可能还会增加人脸质量检测以及活体识别等算法，碍于篇幅，后续再分享活体识别算法。

下图给出本项目人脸识别系统算法实现架构流程图：

（1）人脸识别的核心算法

项目实现了人脸识别的核心算法，包含人脸检测和人脸关键点检测，人脸校准，人脸特征提取以及人脸比对(1:1)和人脸搜索(1:N)等功能，可以参文件(src/main/java/com/cv/tnn/model/FaceRecognizer.java)，实现人脸识别的基本功能


package com.cv.tnn.model;
 
import android.graphics.Bitmap;
import android.graphics.BitmapFactory;
import android.util.Log;
 
import java.io.File;
import java.util.List;
 
public class FaceRecognizer {
 
    private static final String TAG = "FaceRecognizer";
 
    public FaceRecognizer(String det_model, String rec_model, String root, String database, int model_type, int num_thread, boolean useGPU) {
        Log.w(TAG, "det_model =" + det_model);
        Log.w(TAG, "rec_model =" + rec_model);
        Log.w(TAG, "root      =" + root);
        Log.w(TAG, "database  =" + database);
        Log.w(TAG, "model_type=" + model_type);
        Log.w(TAG, "num_thread=" + String.valueOf(num_thread));
        Log.w(TAG, "useGPU    =" + String.valueOf(useGPU));
        FileChooseUtil.createFolder(root);
        Detector.init(det_model, rec_model, root, database, model_type, num_thread, useGPU);
    }
 
    /***
     * 进行人脸检测
     * @param bitmap 输入图像
     * @param det_conf_thresh 人脸检测置信度阈值，范围0.~1.0
     * @param det_iou_thresh  人脸检测IOU阈值，范围0.~1.0
     * @return
     */
    public FrameInfo[] detectFace(Bitmap bitmap, float det_conf_thresh, float det_iou_thresh) {
        FrameInfo[] result = null;
        result = Detector.detectFace(bitmap, det_conf_thresh, det_iou_thresh);
        return result;
    }
 
 
    /***
     * 通过导入文件夹路径，进行批量注册人脸，
     * 请将图片按照[ID-XXXX.jpg]命名,如:张三-image.jpg
     * @param folder 文件夹路径
     * @param det_conf_thresh 人脸检测置信度阈值，范围0.~1.0
     * @param det_iou_thresh  人脸检测IOU阈值，范围0.~1.0
     */
    public void registerFromFolder(String folder, float det_conf_thresh, float det_iou_thresh) {
        Log.w(TAG, "database folder=" + folder);
        List<String> image_list = FileChooseUtil.getImagePathFromSD(folder);
        for (int i = 0; i < image_list.size(); i++) {
            String image_file = image_list.get(i);
            FrameInfo[] result = registerFromFile(image_file, det_conf_thresh, det_iou_thresh);
        }
    }
 
    /***
     * 通过导入图片的路径，进行注册
     * 请将图片按照[ID-XXXX.jpg]命名,如:张三-image.jpg
     * @param image_file 图片的路径
     * @param det_conf_thresh 人脸检测置信度阈值，范围0.~1.0
     * @param det_iou_thresh  人脸检测IOU阈值，范围0.~1.0
     * @return
     */
    public FrameInfo[] registerFromFile(String image_file, float det_conf_thresh, float det_iou_thresh) {
        String[] paths = image_file.split(File.separator);
        String basename = paths[paths.length - 1];
        String face_id = basename.split("-")[0];
        if (face_id.length() == basename.length()) {
            Log.w(TAG, "file=" + image_file + ",图片名称不合法，请将图片按照[ID-XXXX.jpg]命名,如:张三-image.jpg");
        }
        Bitmap bitmap = BitmapFactory.decodeFile(image_file);
        FrameInfo[] result = registerFromBitmap(face_id, bitmap, det_conf_thresh, det_iou_thresh);
        if (result.length > 0) {
            Log.w(TAG, "file=" + image_file + ",注册人脸成功:ID=" + face_id);
        } else {
            Log.w(TAG, "file=" + image_file + ",注册人脸失败:ID=" + face_id);
        }
        return result;
    }
 
    /***
     * 通过导入Bitmap图像，进行注册
     * @param face_id 人脸ID
     * @param bitmap  Bitmap图像
     * @param det_conf_thresh 人脸检测置信度阈值，范围0.~1.0
     * @param det_iou_thresh  人脸检测IOU阈值，范围0.~1.0
     * @return
     */
    public FrameInfo[] registerFromBitmap(String face_id, Bitmap bitmap, float det_conf_thresh, float det_iou_thresh) {
        FrameInfo[] result = null;
        //bitmap = bitmap.copy(Bitmap.Config.ARGB_8888, true);
        result = Detector.registerFace(face_id, bitmap, det_conf_thresh, det_iou_thresh);
        return result;
    }
 
    /***
     * 人脸识别1:N人脸搜索
     * @param bitmap   Bitmap图像
     * @param max_face 最大人脸个数，默认为-1，表示全部人脸
     * @param det_conf_thresh 人脸检测置信度阈值，范围0.~1.0
     * @param det_iou_thresh  人脸检测IOU阈值，范围0.~1.0
     * @param rec_conf_thresh 人脸识别相似度阈值，范围0.~1.0
     * @return
     */
    public FrameInfo[] detectSearch(Bitmap bitmap, int max_face, float det_conf_thresh, float det_iou_thresh, float rec_conf_thresh) {
        FrameInfo[] result = null;
        result = Detector.detectSearch(bitmap, max_face, det_conf_thresh, det_iou_thresh, rec_conf_thresh);
        return result;
    }
 
    /***
     * 人脸识别1:1人脸验证，比较两张人脸的相似性
     * @param bitmap1 输入第1张人脸图像
     * @param bitmap2 输入第2张人脸图像
     * @param det_conf_thresh 人脸检测置信度阈值，范围0.~1.0
     * @param det_iou_thresh  人脸检测IOU阈值，范围0.~1.0
     * @return
     */
    public float compareFace(Bitmap bitmap1, Bitmap bitmap2, float det_conf_thresh, float det_iou_thresh) {
        return Detector.compareFace(bitmap1, bitmap2, det_conf_thresh, det_iou_thresh);
    }
 
 
    /***
     * 提取人脸特征(先进行检测，再提取人脸特征)
     * @param bitmap 输入人脸图像
     * @param max_face 最大人脸个数，默认为-1，表示全部人脸
     * @param det_conf_thresh 人脸检测置信度阈值，范围0.~1.0
     * @param det_iou_thresh  人脸检测IOU阈值，范围0.~1.0
     * @return
     */
    public FrameInfo[] getFeature(Bitmap bitmap, int max_face, float det_conf_thresh, float det_iou_thresh) {
        FrameInfo[] result = null;
        result = Detector.getFeature(bitmap, max_face, det_conf_thresh, det_iou_thresh);
        return result;
    }
 
 
    /***
     * 清空人脸数据库(会删除所有已经注册的人脸数据，谨慎操作)
     */
    public void clearDatabase() {
        Detector.clearDatabase();
    }
}

（2）人脸检测和关键点检测

人脸检测的方法比较多，项目Python版本人脸识别提供两种人脸检测方法：一种是基于MTCNN的通用人脸检测模型，另一种是轻量化的、快速的RFB人脸检测模型；这个两个模型都能实现人脸检测，并同时预测人脸的五个关键点(Landmark)。C/C++和Android版本只提供RFB人脸检测和关键点检测模型。

模型	Paper	源码	说明
MTCNN	Paper	Link	支持人脸检测和人脸关键点检测(5个点) 通用场景人脸检测，计算量较大，适合PC服务器部署
RFB	Paper	Link	支持人脸检测和人脸关键点检测(5个点) 轻量级人脸检测，适合简单场景人脸检测，计算量较小，适合嵌入式，开发板，Android等终端部署

MTCNN人脸检测参考项目：GitHub - Sierkinhane/mtcnn-pytorch: A face detection algorithm
RFB人脸检测参考项目：https://www.wpsshop.cn/w/菜鸟追梦旅行/article/detail/79955