Abstract. A new approach to solving the problem of automatic face recognition of people using personal protective equipment such as a medical mask has been proposed and tested. This approach is based on the use of methods of generating synthetic images of partially hidden faces and the face recognition model ArcFace. A strategy for training data sets formation is proposed and a number of corresponding recognition models are derived. A series of experiments aimed at assessing the quality of predictions of the obtained solution are carried out, and a relationship between the resulting quality of predictions implemented by recognition models and the volume of synthetic images in training datasets is established. According to the results of experimental studies, neural network models, further trained on datasets with volume of artificially synthesized images of 40-60%, demonstrate values of recognition accuracy above 87% on the AAc quantitative metric (Average Accuracy). Using the proposed approach makes it possible to significantly improve the quality of recognition of partially hidden faces compared to the basic approach.

Keywords: face recognition, neural network recognition models, ArcFace, BRAVE-MASKS, synthetic image generation, personal protective equipment, deep learning

References:

1. Zhang K., Zhang Z., Li Z., Qiao Y. IEEE Signal Processing Letters, 2016, no. 10(23), pp. 1499–1503, DOI: 10.1109/LSP.2016.2603342.
2. Zhang F., Fan X., Ai G., Song J., Qin Y., Wu J. arXiv preprint arXiv:1905.01585, 2019, рр. 1–9.
3. Schroff F., Kalenichenko D., Philbin J. Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, рр. 815–823, DOI: 10.1109/CVPR.2015.7298682.
4. Deng J., Guo J., Xue N., Zafeiriou S. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, рр. 4690–4699.
5. He Y., Xu D., Wu L., Jian M., Xiang S., Pan C. arXiv preprint arXiv:1904.10633, 2019, рр. 1–10, DOI: 10.48550/arXiv.1904.10633.
6. Parkhi O. M., Vedaldi A., Zisserman A. Deep face recognition, 2015, рр. 1–12. DOI: 10.5244/C.29.41.
7. Rab S., Javaid M., Haleem A., Vaishya R. Diabetes & Metabolic Syndrome: Clinical Research & Reviews, 2020, no. 6(14), pp. 1617–1619.
8. Martínez-Díaz Y., Méndez-Vázquez H., Luevano L. S., Nicolás-Díaz M., Chang L., González-Mendoza M. IEEE Access., 2021, vol. 10, рр. 7341–7353.
9. Anwar A., Raychowdhury A. arXiv preprint arXiv:2008.11104, 2020, рр. 1–8.
10. Cao Q., Shen L., Xie W., Parkhi O.M., Zisserman A. 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), IEEE, 2018, рр. 67–74.
11. Guo Y., Zhang L., Hu Y., He X., Gao J. European conference on computer vision, Springer, Cham, 2016, рр. 87–102.
12. Wang Z., Wang G., Huang B., Xiong Z., Hong Q., Wu H., Liang J. arXiv preprint arXiv:2003.09093, 2020, рр. 1–3.
13. Liu W., Wen Y., Yu Z., Li M., Raj B., Song L. Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, рр. 212–220.
14. Wang H., Wang Y., Zhou Z., Ji X., Gong D., Zhou J., Liu W. Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, рр. 5265–5274.
15. Szegedy C., Liu W., Jia Y., Sermanet P., Reed S., Anguelov D., Rabinovich A. Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, рр. 1–9.
16. Kemelmacher-Shlizerman I. et al. Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, рр. 4873–4882.
17. Letenkov M.A., Iakovlev R.N., Markitantov M.V., Ryumin D.A., Saveliev A.I., Karpov A.A. Scientific Visualization, 2022, no. 2(14), pp. 1–17, DOI: 10.26583/sv.14.2.01.
18. InsightFace: 2D and 3D Face Analysis Project, https://github.com/deepinsight/insightface.
19. Markitantov M., Ryumina E., Ryumin D., Karpov A. Proceedings of ISCA International Conference INTERSPEECH-2022, Korea, 2022.