MASK FACE INPAINTING BASED ON IMPROVED GENERATIVE ADVERSARIAL NETWORK
Article Sidebar
Open full text
Issue Vol. 19 No. 2 (2023)
-
CNN AND LSTM FOR THE CLASSIFICATION OF PARKINSON'S DISEASE BASED ON THE GTCC AND MFCC
Nouhaila BOUALOULOU, Taoufiq BELHOUSSINE DRISSI, Benayad NSIRI1-24
-
MASK FACE INPAINTING BASED ON IMPROVED GENERATIVE ADVERSARIAL NETWORK
Qingyu Liu, Roben A. Juanatas25-42
-
APPLICATION OF THE REAL-TIME FAN SCHEDULING IN THE EXPLORATION-EXPLOITATION TO OPTIMIZE MINIMUM FUNCTIONS OBJECTIVES
Mariano LARIOS, Perfecto M. QUINTERO-FLORES , Mario ANZURES-GARCÍA , Miguel CAMACHO-HERNANDEZ43-54
-
APPLICATION OF GENETIC ALGORITHMS TO THE TRAVELING SALESMAN PROBLEM
Tomasz Sikora, Wanda Gryglewicz-Kacerka55-62
-
THE POTENTIAL FOR REAL-TIME TESTING OF HIGH FREQUENCY TRADING STRATEGIES THROUGH A DEVELOPED TOOL DURING VOLATILE MARKET CONDITIONS
Mantas Vaitonis, Konstantinas Korovkinas63-81
-
NAVIGATION STRATEGY FOR MOBILE ROBOT BASED ON COMPUTER VISION AND YOLOV5 NETWORK IN THE UNKNOWN ENVIRONMENT
Thanh-Lam BUI, Ngoc-Tien TRAN82-95
-
A NEW METHOD FOR GENERATING VIRTUAL MODELS OF NONLINEAR HELICAL SPRINGS BASED ON A RIGOROUS MATHEMATICAL MODEL
Krzysztof Michalczyk, Mariusz Warzecha, Robert Baran96-111
-
HYBRID FEATURE SELECTION AND SUPPORT VECTOR MACHINE FRAMEWORK FOR PREDICTING MAINTENANCE FAILURES
Mouna TARIK, Ayoub MNIAI, Khalid JEBARI112-124
-
CLASSIFICATION OF PARKINSON'S DISEASE IN BRAIN MRI IMAGES USING DEEP RESIDUAL CONVOLUTIONAL NEURAL NETWORK
Puppala Praneeth, Majety Sathvika, Vivek Kommareddy, Madala Sarath, Saran Mallela, Koneru Suvarna Vani, Prasun Chkrabarti125-146
-
EXPLOITING BERT FOR MALFORMED SEGMENTATION DETECTION TO IMPROVE SCIENTIFIC WRITINGS
Abdelrahman Halawa, Shehab Gamalel-Din; Abdurrahman Nasr126-141
Archives
-
Vol. 21 No. 3
2025-10-05 12
-
Vol. 21 No. 2
2025-06-27 12
-
Vol. 21 No. 1
2025-03-31 12
-
Vol. 20 No. 4
2025-01-31 12
-
Vol. 20 No. 3
2024-09-30 12
-
Vol. 20 No. 2
2024-08-14 12
-
Vol. 20 No. 1
2024-03-30 12
-
Vol. 19 No. 4
2023-12-31 10
-
Vol. 19 No. 3
2023-09-30 10
-
Vol. 19 No. 2
2023-06-30 10
-
Vol. 19 No. 1
2023-03-31 10
-
Vol. 18 No. 4
2022-12-30 8
-
Vol. 18 No. 3
2022-09-30 8
-
Vol. 18 No. 2
2022-06-30 8
-
Vol. 18 No. 1
2022-03-30 7
-
Vol. 17 No. 4
2021-12-30 8
-
Vol. 17 No. 3
2021-09-30 8
-
Vol. 17 No. 2
2021-06-30 8
-
Vol. 17 No. 1
2021-03-30 8
Main Article Content
DOI
Authors
liuq@students.national-u.edu.ph
Abstract
Face recognition technology has been widely used in all aspects of people's lives. However, the accuracy of face recognition is greatly reduced due to the obscuring of objects, such as masks and sunglasses. Wearing masks in public has been a crucial approach to preventing illness, especially since the Covid-19 outbreak. This poses challenges to applications such as face recognition. Therefore, the removal of masks via image inpainting has become a hot topic in the field of computer vision. Deep learning-based image inpainting techniques have taken observable results, but the restored images still have problems such as blurring and inconsistency. To address such problems, this paper proposes an improved inpainting model based on generative adversarial network: the model adds attention mechanisms to the sampling module based on pix2pix network; the residual module is improved by adding convolutional branches. The improved inpainting model can not only effectively restore faces obscured by face masks, but also realize the inpainting of randomly obscured images of human faces. To further validate the generality of the inpainting model, tests are conducted on the datasets of CelebA, Paris Street and Place2, and the experimental results show that both SSIM and PSNR have improved significantly.
Keywords:
References
Ding, D., Ram, S., & Rodriguez, J. J. (2019). Image Inpainting Using Nonlocal Texture Matching and Nonlinear Filtering. IEEE Transactions on Image Processing, 28(4), 1705–1719. https://doi.org/10.1109/TIP.2018.2880681 DOI: https://doi.org/10.1109/TIP.2018.2880681
Goodfellow, I. (2014). NIPS 2014 Tutorial: Generative Adversarial Networks. arXiv. https://doi.org/10.48550/arXiv.1701.00160
He, L., Qiang, Z., Shao, X., Lin, H., Wang, M., & Dai, F. (2022). Research on High-Resolution Face Image Inpainting Method Based on StyleGAN. Electronics, 11(10), 1620. https://doi.org/10.3390/electronics11101620 DOI: https://doi.org/10.3390/electronics11101620
Hore, A., & Ziou, D. (2010). Image Quality Metrics: PSNR vs. SSIM. 2010 20th International Conference on Pattern Recognition (pp. 2366–2369). IEEE. https://doi.org/10.1109/ICPR.2010.579 DOI: https://doi.org/10.1109/ICPR.2010.579
Iizuka, S., Simo-Serra, E., & Ishikawa, H. (2017). Globally and locally consistent image completion. ACM Transactions on Graphics, 36(4), 1–14. https://doi.org/10.1145/3072959.3073659 DOI: https://doi.org/10.1145/3072959.3073659
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017). Imavolge-to-Image Translation with Conditional Adversarial Networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 5967–5976). IEEE. https://doi.org/10.1109/CVPR.2017.632 DOI: https://doi.org/10.1109/CVPR.2017.632
Jaderberg, M., Simonyan, K., & Zisserman, A. (2016). Spatial Transformer Networks. arXiv. https://doi.org/10.48550/arXiv.1506.02025
Jiang, Y., Yang, F., Bian, Z., Lu, C., & Xia, S. (2022). Mask removal: Face inpainting via attributes. Multimedia Tools and Applications, 81(21), 29785–29797. https://doi.org/10.1007/s11042-022-12912-1 DOI: https://doi.org/10.1007/s11042-022-12912-1
Jia, J., & Tang, Ch.-K. (2003). Image repairing: Robust image synthesis by adaptive ND tensor voting. 2003 IEEE Computer Society Conference on Computer Vision springe and Pattern Recognition, 2003. Proceedings. (pp. I-I). IEEE. https://doi.org/10.1109/CVPR.2003.1211414 DOI: https://doi.org/10.1109/CVPR.2003.1211414
Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., & Shi, W. (2017). Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 105–114). IEEE. https://doi.org/10.1109/CVPR.2017.19 DOI: https://doi.org/10.1109/CVPR.2017.19
Liu, G., Reda, F. A., Shih, K. J., Wang, T.-C., Tao, A., & Catanzaro, B. (2018). Image Inpainting for Irregular Holes Using Partial Convolutions. In V. Ferrari, M. Hebert, C. Sminchisescu, & Y. Weiss (Eds.), Computer Vision – ECCV 2018 (vol. 11215, pp. 89–105). Springer. https://doi.org/10.1007/978-3-030-01252-6_6 DOI: https://doi.org/10.1007/978-3-030-01252-6_6
Mou, C., Wang, Q., & Zhang, J. (2022). Deep Generalized Unfolding Networks for Image Restoration. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 17378–17389). IEEE. https://doi.org/10.1109/CVPR52688.2022.01688 DOI: https://doi.org/10.1109/CVPR52688.2022.01688
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. (2016). Context Encoders: Feature Learning by Inpainting. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 2536–2544). IEEE. https://doi.org/10.1109/CVPR.2016.278 DOI: https://doi.org/10.1109/CVPR.2016.278
Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. arXiv. https://doi.org/10.48550/arXiv.1505.04597 DOI: https://doi.org/10.1007/978-3-319-24574-4_28
Rudin, L. I., Osher, S., & Fatemi, E. (1992). Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenomena, 60(1–4), 259–268. https://doi.org/10.1016/0167-2789(92)90242-F. DOI: https://doi.org/10.1016/0167-2789(92)90242-F
Sagong, M., Shin, Y., Kim, S., Park, S., & Ko, S. (2019). PEPSI: Fast Image Inpainting With Parallel Decoding Network. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 11352–11360). IEEE. https://doi.org/10.1109/CVPR.2019.01162 DOI: https://doi.org/10.1109/CVPR.2019.01162
Shao, X., Qiang, Z., Dai, F., He, L., & Lin, H. (2022). Face Image Completion Based on GAN Prior. Electronics, 11(13), 1997. https://doi.org/10.3390/electronics11131997 DOI: https://doi.org/10.3390/electronics11131997
Simakov, D., Caspi, Y., Shechtman, E., & Irani, M. (2008). Summarizing visual data using bidirectional similarity. 2008 IEEE Conference on Computer Vision and Pattern Recognition(CVPR) (pp. 1–8). IEEE. https://doi.org/10.1109/CVPR.2008.4587842 DOI: https://doi.org/10.1109/CVPR.2008.4587842
Wang, Y., Tao, X., Qi, X., Shen, X., & Jia, J. (2018). Image Inpainting via Generative Multi-column Convolutional Neural Networks. arXiv. https://doi.org/10.48550/arXiv.1810.08771
Wu, H., Zhou, J., & Li, Y. (2020). Deep Generative Model for Image Inpainting with Local Binary Pattern Learning and Spatial Attention. arXiv. https://doi.org/10.48550/arXiv.2009.01031
Xie, C., Liu, S., Li, C., Cheng, M.-M., Zuo, W., Liu, X., Wen, S., & Ding, E. (2019). Image Inpainting With Learnable Bidirectional Attention Maps. 2019 IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 8857–8866). IEEE. https://doi.org/10.1109/ICCV.2019.00895 DOI: https://doi.org/10.1109/ICCV.2019.00895
Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., & Huang, T. S. (2019). Free-Form Image Inpainting With Gated Convolution. arXiv. https://doi.org/10.48550/arXiv.1806.03589 DOI: https://doi.org/10.1109/ICCV.2019.00457
Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., & Huang, T. S. (2018). Generative Image Inpainting With Contextual Attention. arXiv. https://doi.org/10.48550/arXiv.1801.07892 DOI: https://doi.org/10.1109/CVPR.2018.00577
Zeng, Y., Fu, J., Chao, H.. & Guo, B. (2019). Learning Pyramid-Context Encoder Network for High-Quality Image Inpainting. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) ( pp.1486–1494). IEEE. https://doi.org/10.1109/CVPR.2019.00158 DOI: https://doi.org/10.1109/CVPR.2019.00158
Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., & Fu, Y. (2018). Image Super-Resolution Using Very Deep Residual Channel Attention Networks. In V. Ferrari, M. Hebert, C. Sminchisescu, & Y. Weiss (Eds.), Computer Vision – ECCV 2018 (vol. 11211, pp. 294–310). Springer. https://doi.org/10.1007/978-3-030-01234-2_18 DOI: https://doi.org/10.1007/978-3-030-01234-2_18
Zheng, C., Cham, T.-J., & Cai, J. (2019). Pluralistic Image Completion. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 1438–1447). IEEE. https://doi.org/10.1109/CVPR.2019.00153 DOI: https://doi.org/10.1109/CVPR.2019.00153
Zhou, Z., Siddiquee, M. M. R., Tajbakhsh, N., & Liang, J. (2018). UNet++: A Nested U-Net Architecture for Medical Image Segmentation. arXiv. http://arxiv.org/abs/1807.10165 DOI: https://doi.org/10.1007/978-3-030-00889-5_1
Article Details
Abstract views: 464
License

This work is licensed under a Creative Commons Attribution 4.0 International License.
All articles published in Applied Computer Science are open-access and distributed under the terms of the Creative Commons Attribution 4.0 International License.
