KDSP-DEA-YOLO: Knowledge-Distillation-Guided Structured Pruning for Lightweight Industrial Surface Defect Detection
DOI:
https://doi.org/10.70088/c66wm041Keywords:
industrial surface defect detection, small-object detection, knowledge distillation, structured pruning, lightweight networkAbstract
Small surface defects on industrial components are characterized by small scale, low contrast, weak texture, and strong background interference. To improve deployment efficiency while preserving detection accuracy, this study proposes KDSP-DEA-YOLO based on the DEA-YOLOv8n teacher model. A heterogeneous lightweight student is first constructed using GhostConv, C2f_Ghost_Lite, a lightweight EMA branch, and DFF_Lite, reducing the pre-pruning parameter count to 4.32 M. A three-stage pipeline is then adopted: sparsity-aware pre-distillation, dependency-aware structured pruning, and distillation-assisted fine-tuning, with joint supervision from outputs, multiscale features, and bounding boxes. Based on the current baseline and compression trends, the predicted 20% pruning configuration contains 3.61 M parameters and requires 8.84 G FLOPs, while achieving approximately 69.8% mAP@0.5 after fine-tuning. This represents a 39.8% reduction in parameters relative to the 6.0 M teacher. The framework provides a verifiable route toward regularized compression and edge deployment for small industrial defect detection.References
R. Ameri, C. C. Hsu, and S. S. Band, "A systematic review of deep learning approaches for surface defect detection in industrial applications," Engineering Applications of Artificial Intelligence, vol. 130, p. 107717, 2024.
Y. Ma, J. Yin, F. Huang, et al., "Surface defect inspection of industrial products with object detection deep networks: a systematic review," Artificial Intelligence Review, vol. 57, p. 333, 2024.
S. Ren, K. He, R. Girshick, and J. Sun, "Faster R-CNN: Towards real-time object detection with region proposal networks," in Advances in Neural Information Processing Systems, vol. 28, 2015.
W. Liu, D. Anguelov, D. Erhan, et al., "SSD: Single Shot MultiBox Detector," in European Conference on Computer Vision, 2016, pp. 21–37.
J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, "You Only Look Once: Unified, real-time object detection," in IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 779–788.
T. Y. Lin, P. Dollar, R. Girshick, et al., "Feature Pyramid Networks for Object Detection," in IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 2117–2125.
N. Carion, F. Massa, G. Synnaeve, et al., "End-to-End Object Detection with Transformers," in European Conference on Computer Vision, 2020, pp. 213–229.
A. Bochkovskiy, C. Y. Wang, and H. Y. M. Liao, "YOLOv4: Optimal Speed and Accuracy of Object Detection," arXiv preprint arXiv:2004.10934, 2020.
C. Y. Wang, A. Bochkovskiy, and H. Y. M. Liao, "YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors," in IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 7464–7475.
A. Wang, H. Chen, L. Liu, et al., "YOLOv10: Real-Time End-to-End Object Detection," in Advances in Neural Information Processing Systems, vol. 37, 2024.
G. Jocher, A. Chaurasia, and J. Qiu, Ultralytics YOLO, Version 8.0.0 [Computer software], 2023.
Z. Gevorgyan, "SIoU Loss: More Powerful Learning for Bounding Box Regression," arXiv preprint arXiv:2205.12740, 2022.
D. Ouyang, S. He, G. Zhang, et al., "Efficient Multi-Scale Attention Module with Cross-Spatial Learning," in *ICASSP 2023 — IEEE International Conference on Acoustics, Speech and Signal Processing*, 2023, pp. 1–5.
Z. Liu, J. Li, Z. Shen, et al., "Learning Efficient Convolutional Networks Through Network Slimming," in IEEE International Conference on Computer Vision, 2017, pp. 2736–2744.
H. Li, A. Kadav, I. Durdanovic, H. Samet, and H. P. Graf, "Pruning Filters for Efficient ConvNets," in International Conference on Learning Representations, 2017.
Y. He, X. Zhang, and J. Sun, "Channel Pruning for Accelerating Very Deep Neural Networks," in IEEE International Conference on Computer Vision, 2017, pp. 1389–1397.
P. Molchanov, A. Mallya, S. Tyree, I. Frosio, and J. Kautz, "Importance Estimation for Neural Network Pruning," in IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 11264–11272.
G. Hinton, O. Vinyals, and J. Dean, "Distilling the Knowledge in a Neural Network," arXiv preprint arXiv:1503.02531, 2015.
A. Romero, N. Ballas, S. E. Kahou, et al., "FitNets: Hints for Thin Deep Nets," in International Conference on Learning Representations, 2015.
S. Zagoruyko and N. Komodakis, "Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer," in International Conference on Learning Representations, 2017.
J. Guo, K. Han, Y. Wang, et al., "Distilling Object Detectors via Decoupled Features," in IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 2154–2164.
Z. Yang, Z. Li, X. Jiang, et al., "Focal and Global Knowledge Distillation for Detectors," in IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 4643–4652.
Y. Zhu, Q. Zhou, N. Liu, et al., "ScaleKD: Distilling Scale-Aware Knowledge in Small Object Detector," in IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 19723–19733.
P. Bergmann, M. Fauser, D. Sattlegger, and C. Steger, "MVTec AD: A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection," in IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 9592–9600.
D. Tabernik, S. Sela, J. Skvarc, and D. Skocaj, "Segmentation-based deep-learning approach for surface-defect detection," Journal of Intelligent Manufacturing, vol. 31, pp. 759–776, 2020.
C. Wang, W. Zhu, B. B. Gao, et al., "Real-IAD: A Real-World Multi-View Dataset for Benchmarking Versatile Industrial Anomaly Detection," in IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 22883–22892.
Y. Zou, J. Jeong, L. Pemula, D. Zhang, and O. Dabeer, "Spot-the-Difference Self-Supervised Pre-Training for Anomaly Detection and Segmentation," in European Conference on Computer Vision, 2022, pp. 392–408.
M. Liu, Y. Chen, J. Xie, L. He, and Y. Zhang, "LF-YOLO: A Lighter and Faster YOLO for Weld Defect Detection of X-Ray Image," IEEE Sensors Journal, vol. 23, no. 7, pp. 7430–7439, 2023.
Z. Zuo, J. Dong, Y. Gao, and Z. Wu, "HyperDefect-YOLO: Enhance YOLO with Hypergraph Computation for Industrial Defect Detection," arXiv preprint arXiv:2412.03969, 2024.
H. Zhou, R. Yang, R. Hu, C. Shu, X. Tang, and X. Li, "ETDNet: Efficient Transformer-Based Detection Network for Surface Defect Detection," IEEE Transactions on Instrumentation and Measurement, vol. 72, p. 2525014, 2023.
Y. Liu, Y. Liu, X. Guo, X. Ling, and Q. Geng, "Metal surface defect detection using SLF-YOLO enhanced YOLOv8 model," Scientific Reports, vol. 15, p. 11105, 2025.
S. Liu, L. Qi, H. Qin, J. Shi, and J. Jia, "Path Aggregation Network for Instance Segmentation," in IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 8759–8768.
J. Hu, L. Shen, and G. Sun, "Squeeze-and-Excitation Networks," in IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 7132–7141.
S. Woo, J. Park, J. Y. Lee, and I. S. Kweon, "CBAM: Convolutional Block Attention Module," in European Conference on Computer Vision, 2018, pp. 3–19.
Q. Hou, D. Zhou, and J. Feng, "Coordinate Attention for Efficient Mobile Network Design," in IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 13713–13722.
K. Han, Y. Wang, Q. Tian, J. Guo, C. Xu, and C. Xu, "GhostNet: More Features From Cheap Operations," in IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 1580–1589.
A. Vaswani, N. Shazeer, N. Parmar, et al., "Attention Is All You Need," in Advances in Neural Information Processing Systems, vol. 30, 2017.
J. Yang, P. Qiu, Y. Zhang, D. S. Marcus, and A. Sotiras, "D-Net: Dynamic Large Kernel with Dynamic Feature Fusion for Volumetric Medical Image Segmentation," Biomedical Signal Processing and Control, vol. 113, p. 108837, 2026.
J. Redmon and A. Farhadi, "YOLO9000: Better, Faster, Stronger," in IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 6517–6525.
J. Redmon and A. Farhadi, "YOLOv3: An Incremental Improvement," arXiv preprint arXiv:1804.02767, 2018.
C. Li, L. Li, H. Jiang, et al., "YOLOv6: A Single-Stage Object Detection Framework for Industrial Applications," arXiv preprint arXiv:2209.02976, 2022.
S. Ioffe and C. Szegedy, "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift," in Proceedings of Machine Learning Research, vol. 37, 2015, pp. 448–456.
Z. Zheng, P. Wang, W. Liu, J. Li, R. Ye, and D. Ren, "Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression," in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 7, 2020, pp. 12993–13000.
H. Rezatofighi, N. Tsoi, J. Gwak, A. Sadeghian, I. Reid, and S. Savarese, "Generalized Intersection over Union: A Metric and a Loss for Bounding Box Regression," in IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 658–666.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Hong Ke (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
