MTCNN+Face_recognition实时⼈脸识别（⼆）训练⾃⼰的数据多进程实时视频 ⼈脸识别
⽂章内容介绍
，进⾏⼈脸识别特征模型的训练，以及将训练好的模型应⽤于图⽚上⼈脸识别和实时视频中的⼈脸识别。本⽂内容中代码会偏多，若是代码中有误，或者不优化，欢迎指出问题，留⾔，交流，谢谢！ 话不多说，直接上codecode跑起来看到实际的效果胜过千⾔万语！！
⽬录
1.⼈脸识别数据集的准备
2.照⽚中的⼈脸识别以及底库⼈脸特征模型的⽣成
单兵作战系统
3.实时视频流⼈脸识别以及底库⼈脸特征模型的⽣成
4.本project总结
⼀. ⼈脸识别数据集的准备
数据集⽂件格式如下图⽂件夹结构所⽰：
⽂件夹的命名为⼈的名字或ID，⽂件夹内的照⽚若⼲(3-5张左右即可)，这⾥只是拿了两个⼈的⽂件夹来展⽰数据集的结构，实际使⽤中，可添加为若⼲个⼈。
⼆.⼈脸特征提取⽣成特征库
1.⽤于照⽚中⼈脸识别的特征库的⽣成
这边就直接上代码了，提取的⼈脸特征为128维
train_pic_knn_classifer.py
#coding: utf-8
#author: hxy
'''
⼈脸定位采⽤mtcnn
训练⼈脸特征模型：KNN
⽤到了face_recognition模块： pip安装即可
'''
import math
from sklearn import neighbors
import os
import pickle
from PIL import Image, ImageDraw
from PIL import Image, ImageDraw
import face_recognition
from face_recognition.face_recognition_cli import image_files_in_folder
import time
# 加载mtcnn相关库和model
_model import P_Net, R_Net, O_Net
from tools.loader import TestLoader
from detection.MtcnnDetector import MtcnnDetector
from detection.detector import Detector
from detection.fcn_detector import FcnDetector
def net(stage):
detectors =[None, None, None]
if stage in['pnet', 'rnet', 'onet']:
modelPath ='../model/pnet/'
a =[b[5:-6]for
b in os.listdir(modelPath)if b.startswith('pnet-') dswith('.index')]
maxEpoch = max(map(int, a))# auto match a max epoch model
modelPath = os.path.join(modelPath, "pnet-%d"%(maxEpoch))
print("Use PNet model: %s"%(modelPath))
detectors[0]= FcnDetector(P_Net,modelPath)
if stage in['rnet', 'onet']:
modelPath ='../model/rnet/'
a =[b[5:-6]for
b in os.listdir(modelPath)if b.startswith('rnet-') dswith('.index')]
maxEpoch = max(map(int, a))
modelPath = os.path.join(modelPath, "rnet-%d"%(maxEpoch))
print("Use RNet model: %s"%(modelPath))
detectors[1]= Detector(R_Net, 24, 1, modelPath)
if stage in['onet']:
modelPath ='../model/onet/'
a =[b[5:-6]for
b in os.listdir(modelPath)if b.startswith('onet-') dswith('.index')]
maxEpoch = max(map(int, a))
modelPath = os.path.join(modelPath, "onet-%d"%(maxEpoch))
print("Use ONet model: %s"%(modelPath))
detectors[2]= Detector(O_Net, 48, 1, modelPath)
return detectors
def train(train_dir, model_save_path=None, n_neighbors=None, knn_algo='ball_tree', verbose=False): X =[]
y =[]
低压注塑成型# Loop through each person in the training set
for class_dir in os.listdir(train_dir):
if not os.path.isdir(os.path.join(train_dir, class_dir)):
continue
微波加热器# Loop through each training image for the current person
for img_path in image_files_in_folder(os.path.join(train_dir, class_dir)):
# 初始化
pic_list =[]
face_location =[]
image = face_recognition.load_image_file(img_path)
#print(img_path)
pic_list.append(img_path)
testDatas = TestLoader(pic_list)
# 这⾥需要注意boxes坐标信息的处理(与原始的mtcnn输出的坐标信息有区别) allBoxes, _ = mtcnnDetector.detect_face(testDatas)
for box in allBoxes[0]:
x1 = int(box[0])
y1 = int(box[1])
x2 = int(box[2])
y2 = int(box[3])
face_location.append((y1-10, x2+12, y2+10, x1-12))
print(face_location)
if len(face_location)!= 1:
if len(face_location)!= 1:
# 要是⼀张图中⼈脸数量⼤于⼀个，跳过这张图
if verbose:
print("Image {} not suitable for training: {}".format(img_path, "Didn't find a face"if len(face_location)< 1 else"Found more than one face")) else:
# Add face encoding for current image to the training set
X.append(face_recognition.face_encodings(image, known_face_locations=face_location, num_jitters=6)[0])
y.append(class_dir)
# Determine how many neighbors to use for weighting in the KNN classifier
if n_neighbors is None:
n_neighbors = int(round(math.sqrt(len(X))))
if verbose:
print("Chose n_neighbors automatically:", n_neighbors)
# Create and train the KNN classifier
knn_clf = neighbors.KNeighborsClassifier(n_neighbors=n_neighbors, algorithm=knn_algo, weights='distance')
knn_clf.fit(X, y)
# Save the trained KNN classifier
if model_save_path is not None:
with open(model_save_path, 'wb') as f:
#data = {"encoding": X, "name":y}
pickle.dump(knn_clf, f)
return knn_clf
if __name__ =="__main__":
start = time.time()
print("Start ")
detectors = net('onet')
mtcnnDetector = MtcnnDetector(detectors=detectors, min_face_size = 40, threshold=[0.9, 0.6, 0.7])
classifier = train("../faceid/main/LFW", model_save_path="LFW_test_classifier_model.clf", n_neighbors=2)
end = time.time()
print("Training complete! Time cost: {} s".format(int(end-start)))
这⾥⼤致讲解⼀下code的思路：
采⽤mtcnn进⾏⼈脸定位，然后将⼈脸location传给face_recognition的face_encoding模块，从⽽对⼈脸进⾏⼈脸128维特征提取，然后训练knn分类器，⽣成.clf⽂件。
2.⽤于视屏中⼈脸识别特征库的⽣成
encoding_faces_mtcnn.py
#-*-coding: utf-8-*-
#Author: hxy
'''
⽣成的⽂件名为 ×××.pickle ⽂件⽤于视频中⼈脸识别
'''
# import the necessary packages
from imutils import paths
import face_recognition
import argparse
import pickle
import cv2
import os
# 加载mtcnn相关库和model
_model import P_Net, R_Net, O_Net
from tools.loader import TestLoader
from detection.MtcnnDetector import MtcnnDetector
from detection.detector import Detector
from detection.fcn_detector import FcnDetector
def net(stage):
detectors =[None, None, None]
if stage in['pnet', 'rnet', 'onet']:
modelPath ='../model/pnet/'
a =[b[5:-6]for
b in os.listdir(modelPath)if b.startswith('pnet-') dswith('.index')] maxEpoch = max(map(int, a))# auto match a max epoch model
modelPath = os.path.join(modelPath, "pnet-%d"%(maxEpoch))
print("Use PNet model: %s"%(modelPath))
detectors[0]= FcnDetector(P_Net,modelPath)
if stage in['rnet', 'onet']:
modelPath ='../model/rnet/'
a =[b[5:-6]for
b in os.listdir(modelPath)if b.startswith('rnet-') dswith('.index')] maxEpoch = max(map(int, a))
modelPath = os.path.join(modelPath, "rnet-%d"%(maxEpoch))
print("Use RNet model: %s"%(modelPath))
detectors[1]= Detector(R_Net, 24, 1, modelPath)
if stage in['onet']:
modelPath ='../model/onet/'
a =[b[5:-6]for
b in os.listdir(modelPath)if b.startswith('onet-') dswith('.index')] maxEpoch = max(map(int, a))
modelPath = os.path.join(modelPath, "onet-%d"%(maxEpoch))
print("Use ONet model: %s"%(modelPath))
detectors[2]= Detector(O_Net, 48, 1, modelPath)
return detectors
def train():
print("[INFO] ")
imagePaths = list(paths.list_images('../faceid/datasets/train'))
定时药盒
knownEncodings =[]
knownNames =[]
# loop over the image paths
for(i, imagePath)in enumerate(imagePaths):
pic_list =[]
face_location =[]
# extract the person name from the image path
print("[INFO] processing image {}/{}".format(i + 1,
len(imagePaths)))
name = imagePath.split(os.path.sep)[-2]
# load the input image and convert it from RGB (OpenCV ordering)
# to dlib ordering (RGB)
image = cv2.imread(imagePath)
print(imagePath)
#rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
pic_list.append(imagePath)
testDatas = TestLoader(pic_list)
# 这⾥需要注意boxes坐标信息的处理
allBoxes, _ = mtcnnDetector.detect_face(testDatas)
for box in allBoxes[0]:
x1 = int(box[0])
y1 = int(box[1])
x2 = int(box[2])
y2 = int(box[3])
face_location.append((y1-10, x2+12, y2+10, x1-12))
# compute the facial embedding for the face
encodings = face_recognition.face_encodings(image, face_location, num_jitters=6)
定位装置
# loop over the encodings
for encoding in encodings:
knownEncodings.append(encoding)
knownEncodings.append(encoding)
knownNames.append(name)
# dump the facial encodings + names to disk
print("[INFO] ")
data ={"encodings": knownEncodings, "names": knownNames}
f = open('mtcnn_3_video_classifier.pickle', "wb")
f.write(pickle.dumps(data))
f.close()
if __name__=='__main__':
detectors = net('onet')
mtcnnDetector = MtcnnDetector(detectors=detectors, min_face_size = 24, threshold=[0.9, 0.6, 0.7])
train()
code的思路和上⾯的思路是⼀样的，可以⾃⼰理解；代码思路应该也⽐较简单；
注意：在上⾯两部分代码的使⽤过程中，需要注意各个部分⽂件加载时的路径。
三. 实时视频流⼈脸识别
这⾥就直接上代码吧，我也不知道说什么，看着代码应该就懂了吧；
mtcnn_recognize_video.py
#-*-coding: utf-8-*-
#author: lxz-hxy
#e-mail: yingxh1995@aliyun
'''
采⽤多进程进⾏实时⼈脸识别
mtcnn⼈脸定位+ face-recognition
'''
import os
import gc
import time
import pickle
import logging
import numpy as np
from cv2 import cv2 as cv2
import face_recognition
import tensorflow as tf
from multiprocessing import Process, Manager
import multiprocessing as mp
# 载⼊mtcnn相关模块
_model import P_Net, R_Net, O_Net
from tools.loader import TestLoader
from detection.MtcnnDetector import MtcnnDetector
from detection.detector import Detector
from detection.fcn_detector import FcnDetector
def logset():
logging.basicConfig(level=logging.INFO, format='%(asctime)s -%(filename)s：%(lineno)d - %(levelname)s - %(message)s')
def net(stage):
detectors =[None, None, None]
if stage in['pnet', 'rnet', 'onet']:
modelPath ='/home/lxz/project/faceid/main/tmp/model/pnet/'
a =[b[5:-6]for
b in os.listdir(modelPath)if b.startswith('pnet-') dswith('.index')]
maxEpoch = max(map(int, a))# auto match a max epoch model
modelPath = os.path.join(modelPath, "pnet-%d"%(maxEpoch))
logging.info("Use PNet model: %s"%(modelPath))
ccty
detectors[0]= FcnDetector(P_Net, modelPath)
if stage in['rnet', 'onet']:
