Towards Improved Assistive Technologies: Classification and Evaluation of Object Detection Techniques for Users with Visual Impairments
DOI:
https://doi.org/10.21015/vtcs.v12i2.1911Abstract
Even though millions of people struggle to interact with the outside world due to visual impairments, vision is an essential part of our daily lives. Because of its ability to identify and navigate around objects in their surroundings, object detection a crucial component of computer vision has become a potentially helpful solution. This study offers a thorough analysis of object detection techniques utilizing a dual classification system that combines traditional and deep learning methods. In addition, we analyze the most popular evaluation metrics and datasets for these systems' training and evaluation. Unlike previous surveys, our work provides a unique perspective by carefully examining the latest advancements in both innovative deep learning models and traditional approaches. The survey's conclusion highlights current problems and recommends future research directions, highlighting the need for more effective models, diverse datasets, and multi-modal data integration to improve assistive technologies for the visually impaired.
References
J. Y. Mambu, E. Anderson, A. Wahyudi, G. Keyeh, and B. Dajoh, “Blind Reader: An Object Identification Mobile- based Application for the Blind using Augmented Reality Detection,” 2019 1st Int. Conf. Cybern. Intell. Syst. ICORIS 2019, vol. 1, no. August, pp. 138–141, 2019, doi: 10.1109/ICORIS.2019.8874906.
M. Elgendy, C. Sik-Lanyi, and A. Kelemen, “A Novel Marker Detection System for People with Visual Impairment Using the Improved Tiny-YOLOv3 Model,” Comput. Methods Programs Biomed., vol. 205, p. 106112, 2021, doi: 10.1016/j.cmpb.2021.106112.
L. Jiao et al., “A survey of deep learning-based object detection,” IEEE Access, vol. 7, pp. 128837–128868, 2019, doi: 10.1109/ACCESS.2019.2939201.
M. Bansal, M. Kumar, M. Kumar, and K. Kumar, “An efficient technique for object recognition using Shi-Tomasi corner detection algorithm,” Soft Comput., vol. 25, no. 6, pp. 4423–4432, 2021, doi: 10.1007/s00500-020-05453-y.
M. Kadhim and B. Oleiwi, “Blind Assistive System based on Real Time Object Recognition using Machine learning,” Eng. Technol. J., vol. 40, no. 1, pp. 159–165, 2022, doi: 10.30684/etj.v40i1.1933.
J. Kaur and W. Singh, “Tools, techniques, datasets and application areas for object detection in an image: a review,” Multimed. Tools Appl., vol. 81, no. 27, pp. 38297–38351, 2022, doi: 10.1007/s11042-022-13153-y.
A. N. Et. al., “Study Of Deep Learning Techniques For Differently Abled Applications And Detection Techniques,” Turkish J. Comput. Math. Educ., vol. 12, no. 10, pp. 5817–5829, 2021, doi: 10.17762/turcomat.v12i10.5396.
S. A. Sanchez, H. J. Romero, and A. D. Morales, “A review: Comparison of performance metrics of pretrained models for object detection using the TensorFlow framework,” IOP Conf. Ser. Mater. Sci. Eng., vol. 844, no. 1, 2020, doi: 10.1088/1757-899X/844/1/012024.
W. J. Chang, L. B. Chen, C. H. Hsu, J. H. Chen, T. C. Yang, and C. P. Lin, “MedGlasses: A wearable smart-glasses-based drug pill recognition system using deep learning for visually impaired chronic patients,” IEEE Access, vol. 8, pp. 17013–17024, 2020, doi: 10.1109/ACCESS.2020.2967400.
R. Alex, “Object Detection Algorithms : A Review,” pp. 5440–5442, 2020.
A. Lavric, C. Beguni, E. Zadobrischi, A. M. Căilean, and S. A. Avătămăniței, “A Comprehensive Survey on Emerging Assistive Technologies for Visually Impaired Persons: Lighting the Path with Visible Light Communications and Artificial Intelligence Innovations,” Sensors, vol. 24, no. 15, 2024, doi: 10.3390/s24154834.
A. Khan and S. Khusro, An insight into smartphone-based assistive solutions for visually impaired and blind people: issues, challenges and opportunities, vol. 20, no. 2. Springer Berlin Heidelberg, 2021. doi: 10.1007/s10209-020-00733-8.
S. E. E. Profile, S. E. E. Profile, S. E. E. Profile, and S. E. E. Profile, “A Survey on an Intelligent System for Persons with Visual Disabilities,” Aust. J. Eng. Innov. Technol., no. November, pp. 97–118, 2021, doi: 10.34104/ajeit.021.0970118.
Z. A. Nor Hisham, M. A. Faudzi, A. A. Ghapar, and F. A. Rahim, “A Systematic Literature Review of the Mobile Application for Object Recognition for Visually Impaired People,” 2020 8th Int. Conf. Inf. Technol. Multimedia, ICIMU 2020, no. August, pp. 316–322, 2020, doi: 10.1109/ICIMU49871.2020.9243523.
K. Manjari, M. Verma, and G. Singal, “A survey on Assistive Technology for visually impaired,” Internet of Things (Netherlands), vol. 11, 2020, doi: 10.1016/j.iot.2020.100188.
W. Elmannai and K. Elleithy, “Sensor-based assistive devices for visually-impaired people: Current status, challenges, and future directions,” Sensors (Switzerland), vol. 17, no. 3, 2017, doi: 10.3390/s17030565.
Z. Q. Zhao, P. Zheng, S. T. Xu, and X. Wu, “Object Detection with Deep Learning: A Review,” IEEE Trans. Neural Networks Learn. Syst., vol. 30, no. 11, pp. 3212–3232, 2019, doi: 10.1109/TNNLS.2018.2876865.
A. Balothiya, A. Paliwal, and N. Rathi, “Object Detection Assistance for Visually Impaired People - A Brief Review,” vol. 10, no. 2, pp. 15–20, 2022.
M. Deshpande, M. V Sreenath, S. Nadella, N. Antony, and N. Venkat, “A Survey of Object Classification and Detection Techniques in Assistance Systems for the Visually Impaired,” Int. J. Eng. Manag. Res., vol. 1, no. 1, pp. 44–51, 2024.
P. Umale, A. Patil, C. Sahani, A. Gedam, and K. Kawale, “Planer Object Detection Using Surf and Sift Method,” Int. J. Eng. Appl. Sci. Technol., vol. 6, no. 11, pp. 36–39, 2022, doi: 10.33564/ijeast.2022.v06i11.008.
Y. Sakai, T. Oda, M. Ikeda, and L. Barolli, “An object tracking system based on SIFT and SURF feature extraction methods,” Proc. - 2015 18th Int. Conf. Network-Based Inf. Syst. NBiS 2015, pp. 561–565, 2015, doi: 10.1109/NBiS.2015.121.
A. Agarwal, D. Samaiya, and K. K. Gupta, “A Comparative Study of SIFT and SURF Algorithms under Different Object and Background Conditions,” Proc. - 2017 Int. Conf. Inf. Technol. ICIT 2017, pp. 42–45, 2018, doi: 10.1109/ICIT.2017.48.
N. O’Mahony et al., “Deep Learning vs. Traditional Computer Vision. Advances in Intelligent Systems and Computing,” vol. 943, no. Cv, pp. 128–144, 2020.
K. Li and L. Cao, “A review of object detection techniques,” Proc. - 2020 5th Int. Conf. Electromechanical Control Technol. Transp. ICECTT 2020, pp. 385–390, 2020, doi: 10.1109/ICECTT50890.2020.00091.
M. Annaby and Y. Fouda, “Fast template matching and object detection techniques using φ-correlation and binary circuits,” Multimed. Tools Appl., vol. 83, no. 3, pp. 6469–6496, 2024, doi: 10.1007/s11042-023-15564-x.
W. Hu, A. M. Gharuib, and A. E.-D. S. Hafez, “Template Match Object Detection for Inertial Navigation Systems,” Positioning, vol. 02, no. 02, pp. 78–83, 2011, doi: 10.4236/pos.2011.22008.
N. S. Hashemi, R. B. Aghdam, A. S. B. Ghiasi, and P. Fatemi, “Template Matching Advances and Applications in Image Analysis,” pp. 91–108, 2016, [Online]. Available: http://arxiv.org/abs/1610.07231
E. Karypidis, S. G. Mouslech, K. Skoulariki, and A. Gazis, “Comparison Analysis of Traditional Machine Learning and Deep Learning Techniques for Data and Image Classification,” WSEAS Trans. Math., vol. 21, pp. 122–130, 2022, doi: 10.37394/23206.2022.21.19.
A. Zareian, K. Dela Rosa, D. H. Hu, and S. F. Chang, “Open-vocabulary object detection using captions,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., pp. 14388–14397, 2021, doi: 10.1109/CVPR46437.2021.01416.
R. Tapu, B. Mocanu, and T. Zaharia, “Wearable assistive devices for visually impaired: A state of the art survey,” Pattern Recognit. Lett., vol. 137, no. 0, pp. 37–52, 2020, doi: 10.1016/j.patrec.2018.10.031.
M. Carranza-García, J. Torres-Mateo, P. Lara-Benítez, and J. García-Gutiérrez, “On the performance of one-stage and two-stage object detectors in autonomous vehicles using camera data,” Remote Sens., vol. 13, no. 1, pp. 1–23, 2021, doi: 10.3390/rs13010089.
R. Deepa, E. Tamilselvan, E. S. Abrar, and S. Sampath, “Comparison of Yolo, SSD, Faster RCNN for Real Time Tennis Ball Tracking for Action Decision Networks,” Proc. 2019 Int. Conf. Adv. Comput. Commun. Eng. ICACCE 2019, pp. 14–17, 2019, doi: 10.1109/ICACCE46606.2019.9079965.
M. Zhang, W. Zhao, X. Li, and D. Wang, “Shadow detection of moving objects in traffic monitoring video,” vol. 2020, pp. 1983–1987, 2020, doi: 10.1109/ITAIC49862.2020.9338958.
J. Wang et al., “Research on Improved YOLOv5 for Low-Light Environment Object Detection,” Electron., vol. 12, no. 14, pp. 1–22, 2023, doi: 10.3390/electronics12143089.
M. I. T. Hussan, D. Saidulu, P. T. Anitha, A. Manikandan, and P. Naresh, “Object Detection and Recognition in Real Time Using Deep Learning for Visually Impaired People,” vol. 10, no. 2, pp. 80–86.
M. Mahendru and S. K. Dubey, “Real time object detection with audio feedback using Yolo vs. Yolo_V3,” Proc. Conflu. 2021 11th Int. Conf. Cloud Comput. Data Sci. Eng., pp. 734–740, 2021, doi: 10.1109/Confluence51648.2021.9377064.
A. Balachandar, E. Santhosh, A. Suriyakrishnan, N. Vigensh, S. Usharani, and P. M. Bala, “DEEP LEARNING TECHNIQUE BASED VISUALLY IMPAIRED PEOPLE USING YOLO V3 FRAMEWORK MECHANISM,” no. May, pp. 134–138, 2021.
A. Kumar, “An Object Detection Technique For Blind People in Real-Time Using Deep Neural Network,” pp. 292–297, 2020.
S. Suman, S. Mishra, K. S. Sahoo, and A. Nayyar, “Vision Navigator : A Smart and Intelligent Obstacle Recognition Model for Visually Impaired Users,” vol. 2022, 2022.
K. Vijlyakumar, K. Ajitha, A. Alexia, M. Hemalashmi, and S. Madhumitha, “Object Detection for Visually Impaired People Using SSD Algorithm,” 2020 Int. Conf. Syst. Comput. Autom. Networking, ICSCAN 2020, 2020, doi: 10.1109/ICSCAN49426.2020.9262344.
L. Tan, T. Huangfu, and L. Wu, “Comparison of YOLO v3, faster R-CNN, and SSD for real-time pill identification,” arXiv, 2021, [Online]. Available: https://doi.org/10.21203/rs.3.rs-668895/v1
S. M. Alkentar, B. Alsahwa, A. Assalem, and D. Karakolla, “Practical comparation of the accuracy and speed of YOLO, SSD and Faster RCNN for drone detection,” J. Eng., vol. 27, no. 8, pp. 19–31, 2021, doi: 10.31026/j.eng.2021.08.02.
F. Saeed, M. J. Ahmed, M. J. Gul, K. J. Hong, A. Paul, and M. S. Kavitha, “OPEN A robust approach for industrial small ‑ object detection using an improved faster regional convolutional neural network,” Sci. Rep., pp. 1–13, 2021, doi: 10.1038/s41598-021-02805-y.
R. Kvyetnyy et al., “Object detection in images with low light condition,” Photonics Appl. Astron. Commun. Ind. High Energy Phys. Exp. 2017, vol. 10445, no. August 2020, p. 104450W, 2017, doi: 10.1117/12.2281001.
C. Li, D. Song, R. Tong, and M. Tang, “Illumination-aware faster R-CNN for robust multispectral pedestrian detection,” Pattern Recognit., vol. 85, pp. 161–171, 2019, doi: 10.1016/j.patcog.2018.08.005.
S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN : Towards Real-Time Object Detection with Region Proposal Networks,” pp. 1–14.
F. Noureddine, K. Rahmani, and P. Faculty, “Accuracy and Efficiency Comparison of Object Detection Open-Source Models Accuracy and Efficiency Comparison of Object Detection Open-Source Models,” no. May, 2021, doi: 10.3991/ijoe.v17i05.21833.
H. Jin, M. Yan, J. Lu, L. Zhu, K. Wang, and S. Bai, “Performance comparison of moving target recognition between faster R-CNN and SSD,” Proc. - Int. Jt. Conf. Information, Media, Eng. IJCIME 2019, pp. 42–46, 2019, doi: 10.1109/IJCIME49369.2019.00018.
N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, “End-to-End Object Detection with Transformers,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 12346 LNCS, pp. 213–229, 2020, doi: 10.1007/978-3-030-58452-8_13.
J. Beal, E. Kim, E. Tzeng, D. H. Park, A. Zhai, and D. Kislyuk, “Toward Transformer-Based Object Detection,” 2020, [Online]. Available: http://arxiv.org/abs/2012.09958
Z. Sun, S. Cao, Y. Yang, and K. Kitani, “Rethinking Transformer-based Set Prediction for Object Detection,” Proc. IEEE Int. Conf. Comput. Vis., pp. 3591–3600, 2021, doi: 10.1109/ICCV48922.2021.00359.
X. Dai, Y. Chen, J. Yang, P. Zhang, L. Yuan, and L. Zhang, “Dynamic DETR: End-to-End Object Detection with Dynamic Attention,” Proc. IEEE Int. Conf. Comput. Vis., pp. 2968–2977, 2021, doi: 10.1109/ICCV48922.2021.00298.
A. Paullada, I. D. Raji, E. M. Bender, E. Denton, and A. Hanna, “Data and its (dis)contents: A survey of dataset development and use in machine learning research,” Patterns, vol. 2, no. 11, p. 100336, 2021, doi: 10.1016/j.patter.2021.100336.
L. Aziz, M. S. B. H. Salam, U. U. Sheikh, and S. Ayub, “Exploring deep learning-based architecture, strategies, applications and current trends in generic object detection: A comprehensive review,” IEEE Access, vol. 8, pp. 170461–170495, 2020, doi: 10.1109/ACCESS.2020.3021508.
S. Srivastava, A. V. Divekar, C. Anilkumar, I. Naik, V. Kulkarni, and V. Pattabiraman, “Comparative analysis of deep learning image detection algorithms,” J. Big Data, vol. 8, no. 1, 2021, doi: 10.1186/s40537-021-00434-w.
A. Bochkovskiy, C.-Y. Wang, and H.-Y. M. Liao, “YOLOv4: Optimal Speed and Accuracy of Object Detection,” 2020, [Online]. Available: http://arxiv.org/abs/2004.10934
M. Tan, R. Pang, and Q. V. Le, “EfficientDet: Scalable and efficient object detection,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., pp. 10778–10787, 2020, doi: 10.1109/CVPR42600.2020.01079.
J. Tian, Q. Jin, Y. Wang, J. Yang, S. Zhang, and D. Sun, “Performance analysis of deep learning ‑ based object detection algorithms on COCO benchmark : a comparative study,” J. Eng. Appl. Sci., vol. 0, no. 0, pp. 1–18, 2024, doi: 10.1186/s44147-024-00411-z.
S. Bhole and A. Dhok, “Deep Learning based Object Detection and Recognition Framework for the Visually-Impaired,” Proc. 4th Int. Conf. Comput. Methodol. Commun. ICCMC 2020, no. Iccmc, pp. 725–728, 2020, doi: 10.1109/ICCMC48092.2020.ICCMC-000135.
“Microsoft COCO 2017 Dataset.” Accessed: Aug. 29, 2024. [Online]. Available: https://public.roboflow.com/object-detection/microsoft-coco-subset
H. Qiu et al., “Hierarchical Context Features Embedding for Object Detection,” IEEE Trans. Multimed., vol. 22, no. 12, pp. 3039–3050, 2020, doi: 10.1109/TMM.2020.2971175.
M. Oquab et al., “DINOv2 : Learning Robust Visual Features without Supervision,” pp. 1–32, 2024.
X. Shen et al., “DeepMAD : Mathematical Architecture Design for Deep Convolutional Neural Network,” pp. 6163–6173.
Y. Yao, Y. Wang, Y. Guo, J. Lin, H. Qin, and J. Yan, “Cross-dataset Training for Class Increasing Object Detection,” 2020, [Online]. Available: http://arxiv.org/abs/2001.04621
S. Shao, Z. Li, T. Zhang, and C. Peng, “Objects365 : A Large-scale , High-quality Dataset for Object Detection,” pp. 8430–8439.
B. Aggarwal and A. Acharya, “A Survey on Image Datasets for Computer Vision and Deep Learning Convolutional Neural Network Tasks,” no. October, 2020, doi: 10.13140/RG.2.2.22391.44966.
D. Li, H. Ling, S. W. Kim, K. Kreis, S. Fidler, and A. Torralba, “BigDatasetGAN: Synthesizing ImageNet with Pixel-wise Annotations,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 2022-June, no. i, pp. 21298–21308, 2022, doi: 10.1109/CVPR52688.2022.02064.
K. Yang, K. Qinami, L. Fei-Fei, J. Deng, and O. Russakovsky, “Towards fairer datasets: Filtering and balancing the distribution of the people subtree in the ImageNet hierarchy,” FAT* 2020 - Proc. 2020 Conf. Fairness, Accountability, Transpar., pp. 547–558, 2020, doi: 10.1145/3351095.3375709.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” Commun. ACM, vol. 60, no. 6, pp. 84–90, 2017, doi: 10.1145/3065386.
“ImageNet.” Accessed: Aug. 29, 2024. [Online]. Available: https://www.image-net.org/about.php
R. Mottaghi and S. Savarese, “Benchmark for 3D Object Detection in the Wild,” pp. 75–82.
“Pascal VOC.” Accessed: Sep. 14, 2024. [Online]. Available: https://docs.cvat.ai/docs/manual/advanced/formats/format-voc/
H. Li, B. Singh, M. Najibi, Z. Wu, and L. S. Davis, “An Analysis of Pre-Training on Object Detection,” 2019, [Online]. Available: http://arxiv.org/abs/1904.05871
A. Kuznetsova et al., “The Open Images Dataset V4 Unified Image Classification , Object Detection , and Visual Relationship Detection at Scale,” Int. J. Comput. Vis., vol. 128, no. 7, pp. 1956–1981, 2020, doi: 10.1007/s11263-020-01316-z.
N. Joshi, S. Maurya, and S. Jain, “Real-time object detection and identification for visually challenged people using mobile platform,” CEUR Workshop Proc., vol. 2823, pp. 47–59, 2021.
J. Miao and W. Zhu, “Precision – recall curve ( PRC ) classification trees,” Evol. Intell., vol. 15, no. 3, pp. 1545–1569, 2022, doi: 10.1007/s12065-021-00565-2.
H. Zhu, H. Wei, B. Li, X. Yuan, and N. Kehtarnavaz, “applied sciences A Review of Video Object Detection : Datasets , Metrics and Methods,” no. 1, 2020.
Y. Kniazieva, “Object Detection: Key Metrics for Computer Vision Model Performance.” Accessed: Sep. 14, 2024. [Online]. Available: https://labelyourdata.com/articles/object-detection-metrics
G. H. Fu, L. Z. Yi, and J. Pan, “Tuning model parameters in class-imbalanced learning with precision-recall curve,” Biometrical J., vol. 61, no. 3, pp. 652–664, 2019, doi: 10.1002/bimj.201800148.
Q. Fan, “Few-Shot Object Detection with Attention-RPN and Multi-Relation Detector,” pp. 4013–4022.
L. Tan, T. Huangfu, L. Wu, and W. Chen, “Comparison of RetinaNet , SSD , and YOLO v3 for real ‑ time pill identification,” BMC Med. Inform. Decis. Mak., vol. 8, pp. 1–11, 2021, doi: 10.1186/s12911-021-01691-8.
Z. Zhou, X. Lan, S. Li, C. Zhu, and H. Chang, “Feature Pyramid SSD : Outdoor Object Detection Algorithm for Blind People,” pp. 650–654, 2019.
Downloads
Published
How to Cite
Issue
Section
License
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License (CC-By) that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
This work is licensed under a Creative Commons Attribution License CC BY
