文章目录
- 前言
- 1.开始
- 1.1 前置
- 1.2 再次运行，人脸检测跑通
- 1.3 人脸特征抽取
- 1.3.1 模型下载
- 1.3.2 重新跑一下检测和识别
- 1.3.3 人脸监测返回值分析
- 1.3.4 计算相似度
- 1.4 全流程的相似度
- 2. 业务化人脸识别
- 2.1 思路：
- 2.2 开干
- 2.2.1 遍历人脸库，获取人脸库的特征向量
- 2.2.2 模拟输入一张图片，计算距离
- 3. ffmpeg 进行拉流抽帧
- 3.1 安装ffmpeg
- 3.2 视频流抽帧
- 4.进阶
- 总结
前言
人脸识别项目，再走一遍。之前是公司老人留下的，没部署过，没交付过。这次重新搞一个，并且把搞过的记录下来。
参考旷视的insightface:
https://github.com/deepinsight/insightface/tree/master/python-package 另外一个blog:
创建环境，安装库
conda activate -n insightface python=3.8
pip install insightface把它的git 代码都拉下来：
https://github.com/deepinsight/insightface 解压到 /home/jianming_ge/workplace/zhongwaiyun/insightface/
1.开始
1.1 前置
cd /home/jianming_ge/workplace/zhongwaiyun/insightface/python-package
pip install numpy opencv-python
pip install onnxruntime-gpu创建一个文件：
aaa.py
import cv2
import numpy as np
import insightface
from insightface.app import FaceAnalysis
from insightface.data import get_image as ins_get_image
app = FaceAnalysis(allowed_modules=['detection'],providers=['CUDAExecutionProvider', 'CPUExecutionProvider'],download=False)
app.prepare(ctx_id=0, det_size=(640, 640))
img = ins_get_image('ldh') #不用带后缀，图片放到./insightface/python-package/insightface/data/images
faces = app.get(img)
print("faces::::", faces)
rimg = app.draw_on(img, faces)
cv2.imwrite("./ldh_output.jpg", rimg)运行:python aaa.py
报错：
(insightface) [jianming_ge@localhost python-package]$ python aaa.py
Traceback (most recent call last):
File "aaa.py", line 3, in <module>
import insightface
File "/home/jianming_ge/workplace/zhongwaiyun/insightface/python-package/insightface/__init__.py", line 18, in <module>
from . import app
File "/home/jianming_ge/workplace/zhongwaiyun/insightface/python-package/insightface/app/__init__.py", line 2, in <module>
from .mask_renderer import *
File "/home/jianming_ge/workplace/zhongwaiyun/insightface/python-package/insightface/app/mask_renderer.py", line 8, in <module>
from ..thirdparty import face3d
File "/home/jianming_ge/workplace/zhongwaiyun/insightface/python-package/insightface/thirdparty/face3d/__init__.py", line 3, in <module>
from . import mesh
File "/home/jianming_ge/workplace/zhongwaiyun/insightface/python-package/insightface/thirdparty/face3d/mesh/__init__.py", line 9, in <module>
from .cython import mesh_core_cython解决：
find ./ -iname "cython"
(insightface) [jianming_ge@localhost python-package]$ find ./ -iname "cython"
./insightface/thirdparty/face3d/mesh/cython
(insightface) [jianming_ge@localhost python-package]$ cd ./insightface/thirdparty/face3d/mesh/cython/
(insightface) [jianming_ge@localhost cython]$ ls
mesh_core.cpp mesh_core_cython.c mesh_core_cython.cpp mesh_core_cython.pyx mesh_core.h setup.py
(insightface) [jianming_ge@localhost python-package]$ cd ./insightface/thirdparty/face3d/mesh/cython/
(insightface) [jianming_ge@localhost cython]$ ls
mesh_core.cpp mesh_core_cython.c mesh_core_cython.cpp mesh_core_cython.pyx mesh_core.h setup.py
(insightface) [jianming_ge@localhost cython]$ python setup.py build_ext -i
In file included from /home/jianming_ge/miniconda3/envs/insightface/lib/python3.8/site-packages/numpy/core/include/numpy/ndarraytypes.h:1940,
from /home/jianming_ge/miniconda3/envs/insightface/lib/python3.8/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
from /home/jianming_ge/miniconda3/envs/insightface/lib/python3.8/site-packages/numpy/core/include/numpy/arrayobject.h:5,
from mesh_core_cython.cpp:780:
/home/jianming_ge/miniconda3/envs/insightface/lib/python3.8/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
#warning "Using deprecated NumPy API, disable it with " \到包文件下，执行一下安装就可以了
cd ./insightface/thirdparty/face3d/mesh/cython/
python setup.py build_ext -i
1.2 再次运行，人脸检测跑通
python aaa.py
(insightface) [jianming_ge@localhost python-package]$ python aaa.py
download_path: /home/jianming_ge/.insightface/models/buffalo_l
Downloading /home/jianming_ge/.insightface/models/buffalo_l.zip from https://github.com/deepinsight/insightface/releases/download/v0.7/buffalo_l.zip...
0%|▏ | 208/281857 [00:01<43:44, 107.30KB/s]^Z
[2]+ 已停止 python aaa.py他去下载去了，不知为什么在从git上下载特别慢，所以放弃了，想法是手动下载好，传到它所需要的目录。
Downloading /home/jianming_ge/.insightface/models/buffalo_l.zip from https://github.com/deepinsight/insightface/releases/download/v0.7/buffalo_l.zip…
代码追踪后在这里，dir_path
位置:python-package/insightface/utils/storage.py
追踪到dir_path = ~/.insightface/models/buffalo_l
建好目录，并且把权重文件放进去
(insightface) [jianming_ge@localhost python-package]$ mkdir ~/.insightface/models/buffalo_l
(insightface) [jianming_ge@localhost python-package]$ cp ~/.insightface/models/*.onnx ~/.insightface/models/buffalo_l/
(insightface) [jianming_ge@localhost python-package]$ ls ~/.insightface/models/buffalo_l
1k3d68.onnx 2d106det.onnx det_10g.onnx genderage.onnx w600k_r50.onnximport os
import os.path as osp
import zipfile
from .download import download_file
BASE_REPO_URL = 'https://github.com/deepinsight/insightface/releases/download/v0.7'
def download(sub_dir, name, force=False, root='~/.insightface'):
_root = os.path.expanduser(root)
dir_path = os.path.join(_root, sub_dir, name)
if osp.exists(dir_path) and not force:
return dir_path
print('download_path:', dir_path)
zip_file_path = os.path.join(_root, sub_dir, name + '.zip')
model_url = "%s/%s.zip"%(BASE_REPO_URL, name)
download_file(model_url,
path=zip_file_path,
overwrite=True)
if not os.path.exists(dir_path):
os.makedirs(dir_path)
with zipfile.ZipFile(zip_file_path) as zf:
zf.extractall(dir_path)
#os.remove(zip_file_path)
return dir_path
def ensure_available(sub_dir, name, root='~/.insightface'):
return download(sub_dir, name, force=False, root=root)
def download_onnx(sub_dir, model_file, force=False, root='~/.insightface', download_zip=False):
_root = os.path.expanduser(root)
model_root = osp.join(_root, sub_dir)
new_model_file = osp.join(model_root, model_file)
if osp.exists(new_model_file) and not force:
return new_model_file
if not osp.exists(model_root):
os.makedirs(model_root)
print('download_path:', new_model_file)
if not download_zip:
model_url = "%s/%s"%(BASE_REPO_URL, model_file)
download_file(model_url,
path=new_model_file,
overwrite=True)
else:
model_url = "%s/%s.zip"%(BASE_REPO_URL, model_file)
zip_file_path = new_model_file+".zip"
download_file(model_url,
path=zip_file_path,
overwrite=True)
with zipfile.ZipFile(zip_file_path) as zf:
zf.extractall(model_root)
return new_model_file再次执行：
python aaa.py
输出：
(insightface) [jianming_ge@localhost python-package]$ python aaa.py
Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}, 'CUDAExecutionProvider': {'tunable_op_enabled': '0', 'cudnn_conv_use_max_workspace': '1', 'enable_cuda_graph': '0', 'do_copy_in_default_stream': '1', 'arena_extend_strategy': 'kNextPowerOfTwo', 'cudnn_conv1d_pad_to_nc1d': '0', 'gpu_external_empty_cache': '0', 'gpu_external_free': '0', 'gpu_external_alloc': '0', 'gpu_mem_limit': '18446744073709551615', 'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0'}}
model ignore: /home/jianming_ge/.insightface/models/buffalo_l/1k3d68.onnx landmark_3d_68
Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}, 'CUDAExecutionProvider': {'tunable_op_enabled': '0', 'cudnn_conv_use_max_workspace': '1', 'enable_cuda_graph': '0', 'do_copy_in_default_stream': '1', 'arena_extend_strategy': 'kNextPowerOfTwo', 'cudnn_conv1d_pad_to_nc1d': '0', 'gpu_external_empty_cache': '0', 'gpu_external_free': '0', 'gpu_external_alloc': '0', 'gpu_mem_limit': '18446744073709551615', 'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0'}}
model ignore: /home/jianming_ge/.insightface/models/buffalo_l/2d106det.onnx landmark_2d_106
Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}, 'CUDAExecutionProvider': {'tunable_op_enabled': '0', 'cudnn_conv_use_max_workspace': '1', 'enable_cuda_graph': '0', 'do_copy_in_default_stream': '1', 'arena_extend_strategy': 'kNextPowerOfTwo', 'cudnn_conv1d_pad_to_nc1d': '0', 'gpu_external_empty_cache': '0', 'gpu_external_free': '0', 'gpu_external_alloc': '0', 'gpu_mem_limit': '18446744073709551615', 'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0'}}
find model: /home/jianming_ge/.insightface/models/buffalo_l/det_10g.onnx detection [1, 3, '?', '?'] 127.5 128.0
Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}, 'CUDAExecutionProvider': {'tunable_op_enabled': '0', 'cudnn_conv_use_max_workspace': '1', 'enable_cuda_graph': '0', 'do_copy_in_default_stream': '1', 'arena_extend_strategy': 'kNextPowerOfTwo', 'cudnn_conv1d_pad_to_nc1d': '0', 'gpu_external_empty_cache': '0', 'gpu_external_free': '0', 'gpu_external_alloc': '0', 'gpu_mem_limit': '18446744073709551615', 'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0'}}
model ignore: /home/jianming_ge/.insightface/models/buffalo_l/genderage.onnx genderage
Applied providers: ['CUDAExecutionProvider', 'CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}, 'CUDAExecutionProvider': {'tunable_op_enabled': '0', 'cudnn_conv_use_max_workspace': '1', 'enable_cuda_graph': '0', 'do_copy_in_default_stream': '1', 'arena_extend_strategy': 'kNextPowerOfTwo', 'cudnn_conv1d_pad_to_nc1d': '0', 'gpu_external_empty_cache': '0', 'gpu_external_free': '0', 'gpu_external_alloc': '0', 'gpu_mem_limit': '18446744073709551615', 'cudnn_conv_algo_search': 'EXHAUSTIVE', 'device_id': '0'}}
model ignore: /home/jianming_ge/.insightface/models/buffalo_l/w600k_r50.onnx recognition
set det-size: (640, 640)
faces:::: [{'bbox': array([105.29808, 83.55866, 218.93301, 230.41777], dtype=float32), 'kps': array([[137.7511 , 151.92409],
[190.89203, 143.08504],
[171.97752, 181.22136],
[155.1662 , 205.49452],
[192.61273, 197.9723 ]], dtype=float32), 'det_score': 0.8469027}]人脸检测的效果：
文件目录树也贴上来供参考：
1.3 人脸特征抽取
1.3.1 模型下载
https://github.com/deepinsight/insightface/tree/master/model_zoo 一般说来，模型越大，速度越慢，准确度越高
这个页面上的任何一个权重都可以用，我这里下载后重命名为recg.onnx
这是git 的样例，但其实是不全的。
新建bbb.py
"""
Call Recognition Models
"""
import cv2
import numpy as np
import insightface
from insightface.app import FaceAnalysis
from insightface.data import get_image as ins_get_image
handler = insightface.model_zoo.get_model('your_recognition_model.onnx')
handler.prepare(ctx_id=0)刚开始运行肯定报错，因为你没有your_recognition_model.onnx 的权重，
通过代码追踪：它应该放在 ~/.insightface/models/your_recognition_model.onnx
1.3.2 重新跑一下检测和识别
而且权重应该放在项目目录下：
bbb.py 更新为：
"""
Call Recognition Models
"""
import cv2
import numpy as np
import insightface
from insightface.app import FaceAnalysis
from insightface.data import get_image as ins_get_image
app = FaceAnalysis(allowed_modules=['detection'],providers=['CUDAExecutionProvider', 'CPUExecutionProvider'],download=False)
app.prepare(ctx_id=0, det_size=(640, 640))
img = ins_get_image('ldh') #不用带后缀，图片放到./insightface/python-package/insightface/data/images
faces = app.get(img)
print("faces::::", faces)
handler = insightface.model_zoo.get_model('w600k_r50.onnx')
handler.prepare(ctx_id=0)
img = ins_get_image('ldh')
feature = handler.get(img,faces[0])
print("size 0f feature:", len(feature))
print("feature:",feature)注意点：实际在使用过程中，
- 先做人脸检测
- 做特征抽取
- 再做相似度计算
1.3.3 人脸监测返回值分析
人脸监测的返回值, 单张人脸：
faces:::: [{'bbox': array([398.00247, 76.93284, 517.12415, 217.21019], dtype=float32), 'kps': array([[420.43942, 127.73588],
[472.40298, 119.96787],
[440.8788 , 139.35797],
[426.87814, 178.95143],
[473.05652, 172.67671]], dtype=float32), 'det_score': 0.82732844}]
faces:::: [{'bbox': array([151.52905 , 9.983715, 252.05127 , 149.92014 ], dtype=float32), 'kps': array([[179.53656 , 65.759674],
[225.35909 , 66.749855],
[202.66101 , 89.19701 ],
[176.11479 , 110.48839 ],
[225.03027 , 111.73684 ]], dtype=float32), 'det_score': 0.903729}, {'bbox': array([390.146 , 19.774494, 489.32114 , 154.33212 ], dtype=float32), 'kps': array([[417.48557 , 73.57366 ],
[463.98648 , 72.462814],
[442.4967 , 100.443985],
[420.76508 , 118.47941 ],
[463.62903 , 117.62505 ]], dtype=float32), 'det_score': 0.89457685}]
=============
返回一个list， len是人脸的数量
每个单元包含：
bbox 人脸框坐标
kps 人脸关键点的坐标 左眼、右眼、鼻子、嘴巴左侧、嘴巴右侧
det_score 检测的分数
人脸特征是返回一个512纬的向量：
size 0f feature: 512
feature: [-2.65807837e-01 -5.98709404e-01 1.44702032e-01 2.63355345e-01
4.07146290e-02 -1.62199542e-01 2.32072305e-02 -2.41173565e-01
-3.18579137e-01 -5.12303770e-01 4.71155867e-02 2.73445308e-01
-8.21143925e-01 1.60933450e-01 2.68835127e-02 -1.19085282e-01
-9.30740461e-02 -1.72320172e-01 -2.14248434e-01 2.86992230e-02
3.17545533e-01 1.75784290e-01 -1.22256204e-01 -1.63961858e-01
-8.66541564e-01 7.55451992e-02 2.67704695e-01 -5.87792039e-01
-1.69785097e-02 -2.07043514e-02 5.06040215e-01 2.22781166e-01
-2.63346657e-02 8.10962379e-01 -3.26032043e-01 1.26174808e-01
3.17236811e-01 6.22351229e-01 -3.11159819e-01 4.34068412e-01
-2.57273525e-01 2.18948558e-01 5.02837479e-01 -3.66360933e-01
-4.49983001e-01 3.29971194e-01 -4.58328962e-01 -2.24694684e-01
-6.20792098e-02 -2.27833763e-01 -2.73626387e-01 -1.57144777e-02
7.91971147e-01 1.46265015e-01 -5.00584424e-01 -3.09522271e-01
6.18827194e-02 -3.79446059e-01 4.31523114e-01 -4.84285116e-01
-3.52546960e-01 -7.49332607e-02 6.07803948e-02 8.62523675e-01
-4.10395920e-01 -2.44194195e-01 -3.41306150e-01 -8.33527073e-02
5.68045788e-02 -1.97309047e-01 1.42464489e-01 -6.44236326e-01
3.12686771e-01 -2.39409551e-01 4.39812720e-01 -4.36708421e-01
-4.98010159e-01 1.31110698e-01 -5.03139734e-01 -4.52581614e-01
-8.60964954e-02 1.03808708e-01 -3.57752778e-02 8.27392399e-01
3.43307912e-01 -5.30499697e-01 2.62764663e-01 4.72977400e-01
2.66696811e-01 -2.69342978e-02 -5.43406844e-01 3.88814628e-01
8.32139477e-02 -7.24730372e-01 4.20132369e-01 -4.74645436e-01
-1.48665071e-01 9.54597652e-01 -4.56728972e-02 3.51921208e-02
3.72070462e-01 -1.53715964e-02 -8.53569135e-02 -8.37187350e-01
-4.30278433e-03 -9.07578766e-02 -2.80029148e-01 -3.56460154e-01
-6.20279536e-02 4.94443417e-01 1.15014195e+00 -4.20974493e-01
4.80442680e-02 -4.37882274e-01 -1.71415120e-01 -3.32768500e-01
-1.02889575e-01 2.16156662e-01 7.01144218e-01 -4.51926380e-01
-4.58793879e-01 1.81827918e-02 -1.88612550e-01 -1.14988470e-02
-1.77762285e-01 -2.05713645e-01 6.62348121e-02 1.67144701e-01
-6.38863564e-01 -1.04559563e-01 2.81668335e-01 2.02162907e-01
-7.78863609e-01 3.30200553e-01 5.50253451e-01 9.45175365e-02
2.95698971e-01 -3.63492787e-01 3.95007521e-01 4.72791910e-01
-1.13705151e-01 3.27019036e-01 -2.52318531e-01 5.03111146e-02
7.85713866e-02 -4.40496325e-01 -1.02330513e-01 -2.04051405e-01
-4.25574929e-01 1.15317412e-01 -2.48718798e-01 1.49252310e-01
1.05377071e-01 -1.19814776e-01 5.92564166e-01 3.30846272e-02
7.21025541e-02 -3.52899522e-01 7.61120737e-01 -6.52381837e-01
6.50658727e-01 -5.15108407e-01 -1.94832787e-01 -4.66580153e-01
-3.02693993e-01 3.60020995e-01 -6.04675889e-01 3.48266721e-01
1.35741889e-01 -2.12851986e-01 -7.17316985e-01 1.31764725e-01
2.28615105e-01 5.24750538e-02 -4.72336970e-02 -6.26476049e-01
-4.56452340e-01 1.02681197e-01 3.58329445e-01 -3.19489807e-01
-3.44365656e-01 -4.40581203e-01 -1.41268568e-02 6.94410443e-01
-2.67537236e-01 -7.72025576e-03 -5.61997116e-01 -7.19041407e-01
1.20449759e-01 6.45162582e-01 3.46629024e-01 1.10950135e-01
8.20405841e-01 2.94094980e-01 -1.51019976e-01 -5.04539907e-02
-7.21346065e-02 2.75275677e-01 4.68920708e-01 -5.11016130e-01
1.87380016e-01 8.27821791e-02 -3.10316354e-01 -1.03935346e-01
5.48591197e-01 9.24001709e-02 -3.89272362e-01 -1.15961254e+00
-4.71911073e-01 3.22682679e-01 -3.27484250e-01 -2.13711888e-01
8.46880302e-02 1.13394640e-01 -2.57271856e-01 4.57801521e-01
-1.56833291e-01 -4.63344246e-01 -5.22429049e-02 1.63254842e-01
3.00276607e-01 2.76694864e-01 -3.10461104e-01 -5.87711275e-01
-1.89284272e-02 4.10056353e-01 6.57250285e-01 -8.84221718e-02
-3.13798636e-01 5.00106215e-01 -4.48331796e-02 -5.13528764e-01
-9.17405933e-02 7.00492561e-01 -2.52461191e-02 -4.65177923e-01
3.35291862e-01 1.67592824e-01 -2.38635331e-01 6.97482347e-01
1.63971394e-01 -3.29721011e-02 -1.26637876e-01 4.27806467e-01
-5.34400046e-01 -2.61363145e-02 3.35392177e-01 -4.52100456e-01
-1.15982279e-01 -4.80096698e-01 -1.20243188e-02 -8.81862491e-02
-1.41368508e-01 8.21686313e-02 2.38767043e-01 -8.79521295e-02
1.31764039e-01 9.93780792e-02 3.97302061e-01 2.62562603e-01
5.31454384e-01 1.18952766e-01 1.63879409e-01 2.79800206e-01
1.71809718e-01 -9.73480046e-02 5.50766960e-02 2.19342694e-01
3.70924145e-01 2.22219989e-01 -1.48917332e-01 2.90967226e-01
-1.31165534e-01 -1.25355244e-01 1.72034904e-01 -1.20657332e-01
1.64537430e-01 -2.72813380e-01 4.12511617e-01 3.81849200e-01
-3.10269538e-02 -5.84295869e-01 -1.62318975e-01 4.32897478e-01
-2.25531310e-02 1.45282999e-01 -6.18601479e-02 -2.70079710e-02
1.48326024e-01 -6.03496656e-02 -6.77019823e-03 -5.22333384e-01
-1.42824680e-01 1.05511181e-01 -1.57965735e-01 -1.94174737e-01
5.04310012e-01 -6.04734868e-02 3.38690877e-01 2.85632819e-01
2.44570196e-01 4.41604286e-01 -2.50822127e-01 -3.39492172e-01
-4.40223627e-02 -7.41581142e-01 -1.18939653e-01 1.32609561e-01
5.79564087e-02 -1.12131372e-01 -3.30193996e-01 1.77734077e-01
-8.44704509e-02 8.52265060e-02 7.22285330e-01 -7.39744082e-02
-3.24919313e-01 1.52587101e-01 2.60848194e-01 4.75044131e-01
8.96252871e-01 -7.45972931e-01 -1.91705048e-01 -5.63617408e-01
4.07978743e-01 -5.42383134e-01 5.90304434e-01 -1.02651024e+00
1.07888348e-01 -3.20087671e-02 -6.86910003e-02 4.59433228e-01
-1.34848386e-01 1.79632828e-01 -6.78926706e-01 1.82937503e-01
-6.39753282e-01 -2.47191161e-01 -1.42870158e-01 -3.90086859e-01
2.62941182e-01 -5.99119365e-02 -9.82555468e-03 -2.31434792e-01
4.23140265e-02 -2.33853012e-01 7.04509556e-01 -4.41635907e-01
-2.51954973e-01 -3.28132778e-01 7.64646053e-01 -3.54267150e-01
5.39237082e-01 1.79837737e-02 1.73653990e-01 1.45106345e-01
-2.72235245e-01 -5.94357669e-01 -3.84668678e-01 -2.96734065e-01
8.66554603e-02 5.53680919e-02 -7.88553357e-01 1.36896238e-01
-4.14724737e-01 -1.44677877e-01 1.79900452e-01 -1.39882132e-01
2.18250602e-01 6.80885389e-02 -7.68631250e-02 -7.89565086e-01
3.48717213e-01 5.64985991e-01 5.58483787e-02 -8.17401558e-02
2.03474388e-01 -1.07692860e-01 6.84533358e-01 3.19718868e-01
1.21635854e-01 3.41574609e-01 -1.78951304e-02 2.78006196e-01
1.80511326e-02 -6.24188110e-02 -6.32650331e-02 1.31553754e-01
1.79080918e-01 1.01404369e-01 4.41990018e-01 9.36895087e-02
7.87015259e-02 2.14785039e-01 -5.45855701e-01 2.24413946e-01
4.32758123e-01 -4.15375143e-01 2.15920016e-01 2.70790398e-01
6.58806413e-02 1.41609073e-01 3.54366191e-02 -3.72616023e-01
3.92095059e-01 -9.02478099e-02 4.66177404e-01 -6.86050534e-01
-3.16491425e-01 4.46931541e-01 -2.10206121e-01 6.28514409e-01
-2.65414834e-01 -3.76454681e-01 -3.56145829e-01 -1.18043542e-01
5.63065000e-02 6.98566914e-01 -3.88334543e-01 5.95293581e-01
6.92259550e-01 2.15426758e-01 6.51695311e-01 1.98373515e-02
-1.47576705e-01 -3.12872767e-01 4.05580014e-01 2.66416162e-01
1.96047127e-01 -1.27053857e-01 1.09347150e-01 3.83434832e-01
1.69589058e-01 -3.51539433e-01 2.03101888e-01 5.83723664e-01
-5.45567989e-01 8.64841640e-02 1.94809675e-01 -2.73879915e-01
5.09571843e-02 9.19594914e-02 -2.97315329e-01 -2.41457775e-01
6.12340942e-02 1.42051131e-01 3.24435651e-01 -7.38600254e-01
-4.35348183e-01 1.50280312e-01 2.81985532e-02 5.68533838e-01
2.32664168e-01 3.33400816e-01 1.46391779e-01 -5.85084502e-03
-1.55448794e-01 -4.67291296e-01 -4.34049487e-01 -3.40150416e-01
1.04534440e-01 -4.27028328e-01 2.94970989e-01 3.17830712e-01
5.78037322e-01 -2.43507465e-03 6.16211236e-01 -1.76521957e-01
-1.05725817e-01 -2.66839921e-01 6.76848069e-02 2.33130768e-01
3.24840128e-01 1.93292648e-01 -1.03790827e-01 -3.10859770e-01
1.08004078e-01 4.30708565e-02 -1.50686622e-01 -3.32239121e-01
-7.74850786e-01 8.22230577e-02 -2.64589787e-01 -1.18717141e-01
-1.79947644e-01 2.60864738e-02 7.45431662e-01 -6.43727034e-02
-1.80508792e-01 -5.09034805e-02 -1.89890876e-01 1.10911153e-01
3.64015251e-01 -2.90384492e-07 4.50303376e-01 1.55005336e-01
-4.08384293e-01 4.87070352e-01 1.31157666e-01 -1.15866172e+00
-2.70245939e-01 -4.77115929e-01 8.44599679e-02 3.13389778e-01
-3.81731451e-01 3.79520021e-02 4.61172126e-02 5.15780486e-02
2.22334802e-01 -6.06625915e-01 -2.58048952e-01 -1.14576712e-01]1.3.4 计算相似度
相似度计算就是用余弦函数进行计算，计算出相似度，其实就是识别的内容。
# 计算相似度
similarity = np.dot(feature1, feature2.T)1.4 全流程的相似度
mmp 搞来搞去，这里有一个pipline
import argparse
import cv2
import sys
import numpy as np
import insightface
from insightface.app import FaceAnalysis
from insightface.data import get_image as ins_get_image
assert insightface.__version__>='0.3'
parser = argparse.ArgumentParser(description='insightface app test')
# general
parser.add_argument('--ctx', default=0, type=int, help='ctx id, <0 means using cpu')
parser.add_argument('--det-size', default=640, type=int, help='detection size')
args = parser.parse_args()
app = FaceAnalysis()
app.prepare(ctx_id=args.ctx, det_size=(args.det_size,args.det_size))
img = ins_get_image('t1')
faces = app.get(img)
assert len(faces)==6
rimg = app.draw_on(img, faces)
cv2.imwrite("./t1_output.jpg", rimg)
# then print all-to-all face similarity
feats = []
for face in faces:
feats.append(face.normed_embedding)
feats = np.array(feats, dtype=np.float32)
sims = np.dot(feats, feats.T)
print(sims)执行命令：python ccc.py --ctx 0 输出，分数是1的代表最相似
[[ 1.0000002 0.06442557 -0.01041015 -0.08265568 0.03555955 -0.02871545]
[ 0.06442557 1.0000002 0.02620501 0.09680527 -0.02184767 0.21340981]
[-0.01041015 0.02620501 0.99999994 0.14635125 0.06818002 -0.03542848]
[-0.08265568 0.09680527 0.14635125 1.0000001 -0.0301404 -0.01418785]
[ 0.03555955 -0.02184767 0.06818002 -0.0301404 0.99999994 0.05765504]
[-0.02871545 0.21340981 -0.03542848 -0.01418785 0.05765504 0.99999964]]
6个人都比一遍，肯定对角线最相似啊，内部是不是已经做了人脸矫正和对齐，就不得而知了，懂的大神可以留言一下。
跑到这里，其实就差业务化了
2. 业务化人脸识别
敬请期待
-----分割线以上是20230529完成
20230531----md 昨天干了一天杂活儿，终于有时间搞这个了。
2.1 思路：
1.人脸库图片导入，提前抽取后人脸特征向量，并存储起来，可以使.txt, 或者 redis，或者 mysql
人脸库要求，一张图片一张脸
2. 项目启动时，把人脸库加载进内存，等待要识别的人脸图片送入
3. 获取送入的图片，通过人脸监测模型，获取人脸的数量，并抽取每个人脸（送入图片可能包含N张人脸）的特征
4. 以循环的方式和数据库人脸作比对，送入图片的每张，和库里的每个人脸计算相似度，并且排名。
5. 获取每个人脸的相似度排行，满足阈值的，返回识别成功，并标注到原图上。
2.2 开干
项目路径
face_db 是人脸库，这里只有郭、刘两位天王
2.2.1 遍历人脸库，获取人脸库的特征向量
只在项目启动时做一次，或者保存在数据库中
def get_face_db_features(face_db_dir=""):
"""
获取人脸db的features
"""
face_db_dir = os.path.join(os.path.dirname(os.path.abspath(__file__)),"face_db")
img_gen = glob.glob(face_db_dir+"/*")
args = argparse.Namespace()
assert len(img_gen)!=0,"人脸库图片数量为0"
face_db_list = []
for img_path in img_gen:
args.img1 = img_path
image1 = cv2.imread(args.img1)
bboxes1, kpss1 = detector.autodetect(image1, max_num=1)
assert bboxes1.shape[0]!=0, f"图片:{img_path}未检测到人脸"
assert len(bboxes1) == 1,f"图片:{img_path}人脸数量大于1"
kps1 = kpss1[0]
feat1 = rec.get(image1, kps1)
# 文件名即为人脸名字
name = os.path.splitext(os.path.basename(img_path))[0]
face_db_list.append(dict(name = name,feat= feat1))
return face_db_list对人脸库的要求苛刻，需要每张图片只有一张人脸，图片名称即为人员名称。
这个face_db_list 就是郭刘两天王的人脸特征。
2.2.2 模拟输入一张图片，计算距离
因为输入的图片可能是无人脸，1张，或者多张人脸。
这里做难度最大的多张人脸。
也就是说，这里面有三种可能：1.没有两个天王，2.有其中之一，3.两个都有
少废话，先看效果：
def get_simility_face(img_path:str = ""):
args = argparse.Namespace()
args.img1 = img_path
image1 = cv2.imread(args.img1)
# 这里先写死最多容纳10个人脸
bboxes2, kpss2 = detector.autodetect(image1, max_num=10)
# 等待识别的人脸
recognition_face_list = []
for idx,kps2 in enumerate(kpss2):
feat2 = rec.get(image1, kps2)
recognition_face_list.append(dict(bbox = bboxes2[idx], ksp = kps2, feat = feat2))
# 代码运行到这里, face_db 和 recognition_db 都准备好了，现在需要计算距离
result = []
for idx, wait_face in enumerate(recognition_face_list):
temp_list = []
for face_db in face_db_list:
name = face_db["name"]
feat1 = face_db["feat"]
feat2 = wait_face["feat"]
sim = rec.compute_sim(feat1, feat2)
if sim>=0.28:
temp_list.append(dict(idx=idx,name=name,sim=sim))
if temp_list:
result.append(temp_list)
# 到这里,获取了一result 其数据结构为：
"""
送入了一张包含人脸很多的照片,包含了10张人脸
人脸库2张，ldh和gfc
[
[{'idx': 0, 'name': 'gfc', 'sim': 0.07851476}, {'idx': 0, 'name': 'ldh', 'sim': -0.06523238}],
[{'idx': 1, 'name': 'gfc', 'sim': -0.05163587}, {'idx': 1, 'name': 'ldh', 'sim': -0.032635976}],
[{'idx': 2, 'name': 'gfc', 'sim': -0.012458734}, {'idx': 2, 'name': 'ldh', 'sim': -0.049863357}],
[{'idx': 3, 'name': 'gfc', 'sim': 0.18950702}, {'idx': 3, 'name': 'ldh', 'sim': -0.03275803}],
[{'idx': 4, 'name': 'gfc', 'sim': 0.0027918643}, {'idx': 4, 'name': 'ldh', 'sim': -0.06449092}],
[{'idx': 5, 'name': 'gfc', 'sim': 0.17510812}, {'idx': 5, 'name': 'ldh', 'sim': 0.5763208}],
[{'idx': 6, 'name': 'gfc', 'sim': 0.008685053}, {'idx': 6, 'name': 'ldh', 'sim': 0.029168786}],
[{'idx': 7, 'name': 'gfc', 'sim': -0.06675171}, {'idx': 7, 'name': 'ldh', 'sim': 0.026193036}],
[{'idx': 8, 'name': 'gfc', 'sim': -0.1017633}, {'idx': 8, 'name': 'ldh', 'sim': 0.048399623}],
[{'idx': 9, 'name': 'gfc', 'sim': -0.009450468}, {'idx': 9, 'name': 'ldh', 'sim': -0.029955076}]
]
"""
if not result:
return []
# 按 sim 从大到小排序，每一行，只选取最大的一个
sorted_data = sorted(result, key=lambda x: x[0]['sim'], reverse=True)
dimg = image1.copy()
font = cv2.FONT_HERSHEY_SIMPLEX
for row in sorted_data:
face_info = row[0]
idx = face_info["idx"]
bbox = recognition_face_list[idx]["bbox"]
box = bbox.astype(np.int32)
color = (0, 0, 255)
cv2.rectangle(dimg, (box[0], box[1]), (box[2], box[3]), color, 2)
cv2.putText(dimg, f"ID: {idx}", (box[0] + 10, box[1] + 30), font, 0.5, (0, 0, 255), 2)
cv2.imwrite("./t1_output.jpg", dimg)核心逻辑就是挨个对比，>0.28的算是同一个人，但是为以防万一，我会把>0.28的人里面选一个分数最高的。
比如，input image 有10张脸， 其中一张 和人脸库的好几张脸都相似， 那么我会选一个分数最高的。这里的逻辑处理还是很复杂的。
后面根据返回结果给业务端，或者封装成web api的方式，就看诸位自己了。大家加油
3. ffmpeg 进行拉流抽帧
opencv也可以做，但是opencv比ffmpeg耗费资源，因此这次要用ffmpeg来做
参考ref：
3.1 安装ffmpeg
要在CentOS上安装FFmpeg，您需要执行以下步骤：
- 打开终端并以root用户身份登录。
- 安装EPEL repository：
yum install epel-release- 安装Nux Dextop repository：
rpm -Uvh http://li.nux.ro/download/nux/dextop/el7/x86_64/nux-dextop-release-0-1.el7.nux.noarch.rpm- 安装FFmpeg及相关库：
yum install ffmpeg ffmpeg-devel- 检查FFmpeg是否正确安装：
ffmpeg -version如果出现FFmpeg版本号，则说明成功安装。
- （可选）安装其他用于特定目的的库，例如libvpx、libmp3lame等。
3.2 视频流抽帧
rtsp流抽帧。参数说明：-y ：如果提示文件已经存在，自动执行y，对已存在文件进行覆盖；-f image2：以图片格式保存；-r 1：帧数，每秒截取一帧，-r 1/5 表示每5秒抽取一帧；img%03d.jpg 格式化命名，会生成img001.jpg,img002.jpg
ffmpeg -i "rtsp://dxcs:cs123456@200.36.43.131:554/Streaming/Channels/1001" -y -f image2 -r 1/5 img%03d.jpgrtsp流抽帧，tcp协议推流需要加上: -rtsp_transport tcp
ffmpeg -rtsp_transport tcp -i "rtsp://dxcs:cs123456@200.36.43.131:554/Streaming/Channels/1001" -y -f image2 -r 1/5 img%03d.jpg两种任选其一即可，这样我们可以用python来操作ffmpeg，并且让它返回图片路径，送入到模型，进行人脸识别！
import subprocess
import os
import subprocess
import os
import time
def extract_frames(video_path, output_dir, frame_rate="1/5"):
"""
使用FFmpeg从视频中提取帧。
:param video_path: 视频文件路径
:param output_dir: 输出帧的目录
:param frame_rate: 提取帧的间隔，默认为每秒提取2帧
:return: 提取的帧图像文件路径
"""
# 构造FFmpeg命令
cmd = ["ffmpeg", "-i", video_path, "-vf", "fps={}".format(frame_rate), "{}/%010d.jpg".format(output_dir)]
# cmd = ["ffmpeg", "-i", video_path, "-vf", "fps={}".format(frame_rate), "{}/{}.jpg".format(output_dir,int(time.time()))]
# 执行命令
subprocess.call(cmd)
if __name__ == "__main__":
video_path = "rtsp://admin:123456@172.18.45.15:554/h264/ch1/main/av_stream"
output_dir = "./video_img"
frame_rate = "1/5"
extract_frames(video_path, output_dir, frame_rate)另一个文件监听video_img的文件变化，并写入到img_log日志
import time
import os
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler
class MyHandler(FileSystemEventHandler):
def on_created(self, event):
if event.is_directory: # 忽略文件夹的创建事件
return
if not event.is_directory:
filepath = event.src_path
# 在这里可以写入文件,另外的线程定于这个文件即可
print(f"New file detected: {filepath}")
with open(img_log,mode="a") as f:
f.write(filepath + os.linesep)
if __name__ == "__main__":
base_dir = os.path.dirname(os.path.abspath(__file__))
img_log = os.path.join(base_dir,"img.log")
folder_to_watch = os.path.join(base_dir,"video_img") # 监听文件夹的路径
event_handler = MyHandler()
observer = Observer()
observer.schedule(event_handler, folder_to_watch, recursive=False)
observer.start()
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
observer.stop()
observer.join()4.进阶
把人脸更多的结构化属性带进来：
年龄、性别、表情、眼睛开闭、嘴巴开闭、是否带眼镜、是否带口罩、是否有胡子…
这些我还搞不清楚是不是一个模型，但应该不是一个。旷世有的模型能识别年龄和性别，会的大神欢迎留言给点指导啊！
应该是先把做人脸检测，把人脸框出来，再送入第二个模型，做分类，是否有胡子，是否戴口罩。。。
嘴巴/眼睛的开闭用关键点计算即可。
总结
我觉得人脸矫正、人脸对齐，这个insight的库都帮我们做了。真是太好用了。
本文章为转载内容，我们尊重原作者对文章享有的著作权。如有内容错误或侵权问题，欢迎原作者联系我们进行内容更正或删除文章。
