EyeSpeak: Smart Eye Tracking System for Assistive Communication
Keywords:
Assistive communication, Blink detection, Eye tracking, Human–computer interaction, Virtual keyboardAbstract
Recent developments in eye tracking technology and human–computer interaction have facilitated the creation of communication systems for people with severe physical and speech disabilities. This literature survey explores research works related to eye tracking, eye gaze estimation, blink detection, facial landmark detection, virtual keyboard, and multimodal interaction for hands-free communication. Studies have implemented various approaches and used different technologies to improve the accuracy of eye movement detection, typing speed, and overall communication efficiency. For example, computer vision with OpenCV, MediaPipe, Dlib, Eye Aspect Ratio (EAR), convolutional neural networks (CNNs), machine learning (ML), Large Language Models (LLMs), and text-to-speech APIs were employed to achieve high performance, reliability, and usability. Moreover, some researchers suggested using head pose estimation, voice commands, predictive text, and advanced interaction techniques to simplify the communication process and reduce physical interaction. The developed eye-tracking systems and communication tools were successfully applied to assistive communication, Augmentative and Alternative Communication (AAC), healthcare, rehabilitation, human–computer interaction, and accessibility domains, benefiting people with Amyotrophic Lateral Sclerosis (ALS), cerebral palsy, quadriplegia, and other conditions. However, most solutions have shortcomings, such as reduced accuracy due to lighting, calibration, and hardware constraints; added weight; eye strain; the need for ocular tracking; slow typing; and the complexity of the underlying algorithms. To summarize, the articles mentioned in this literature survey offer a useful insight into current trends and approaches used to design effective communication systems and help build a solid foundation for creating low-cost, easy-to-use, and high-performance communication tools.
References
A. H. Khaleel, T. Abbas, and A.-W. Sami Ibrahim, “Multitask virtual keyboard controlled by an Eye-Gaze-based intelligent entry system,” Journal of Computational and Cognitive Engineering, vol. 4, no. 4, Aug. 2025.
M. Niharika, S. K. Chandrika, C. V. Sai, and B. Krishna, “Eye Gaze controlled communication,” International Journal for Research in Applied Science Engineering Technology (IJRASET), vol. 9, no. 6, pp. 3616–3620, Jun. 2021.
M. Ezzat, M. Maged, Y. Gamal, M. Adel, M. Alrahmawy, and S. El-Metwally, “Blink-To-Live eye-based communication system for users with speech impairments,” Scientific Reports, vol. 13, May 2023.
B. Waideman and P. T. Aquino, “Augmentative and alternative communication using eye tracking and word recommendation using language models,” IEEE Latin America Transactions, vol. 23, no. 8, pp. 637–645, Aug. 2025.
S. Lee and S. Lee, “Text typing using Blink-to-Alphabet tree for patients with neuro-locomotor disabilities,” Sensors, vol. 25, no. 15, Jul. 2025.
A. Fischer-Janzen, T. M. Wendt, and K. V. Laerhoven, “A scoping review of gaze and eye tracking-based control methods for assistive robotic arms,” Frontiers in Robotics and AI, vol. 11, Feb. 2024.
S. Cai et al., “Using large language models to accelerate communication for eye gaze typing users with ALS,” Nature Communications, vol. 15, Jan. 2024.
A. Krishnan, A. T. Raphi, A. Anirudh, M. George, and D. M. Panicker, “AI virtual keyboard for typing,” International Journal of Engineering Research Technology (IJERT), vol. 11, no. 4, pp. 5–9, 2023.
A. Z. Attiah and E. F. Khairullah, “Eye-Blink detection system for virtual keyboard,” 2021 National Computing Colleges Conference (NCCC), Taif, Saudi Arabia, 2021, pp. 1–6.
V. E. Tatinyuy, A. V. Noumsi Woguia, J. ngono, and L. A. Fono, “Multi-stage gaze-controlled virtual keyboard using eye tracking,” PLOS ONE, vol. 19, no. 10, Oct. 2024.
A. Khasnobish, R. Gavas, D. Chatterjee, V. Raj, and S. Naitam, “EyeAssist: A communication aid through gaze tracking for patients with neuro-motor disabilities,” in Proceedings of the IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), Kona, HI, USA, 2017, pp. 807–812.
D. Jenavani, K. Janice Karen Petra and R. Deepa, “VoEyeBoard: A voice and eye controlled virtual keyboard,” 2025 International Conference on NexGen Networks and Cybernetics (IC2NC), Erode, India, 2025, pp. 407–411.
B. G. Varshini, A. Manimegalai, S. D. Abirami and S. Kalaiselvi, “Eye-Gaze controlled virtual keyboard using blink detection for accessible hands-free communication,” 2025 International Conference on Computational, Communication and Information Technology (ICCCIT), Indore, India, 2025, pp. 49–56.
J. Park et al., “Integrative Human-Computer Interaction system using eyes and facial movement,” 2025 IEEE International Conference on Consumer Electronics (ICCE), Las Vegas, NV, USA, 2025, pp. 1–4.
K. E. S. de Souza et al., “An evaluation framework for user experience using eye tracking, mouse tracking, keyboard input, and artificial intelligence: A case study,” International Journal of Human–Computer Interaction, vol. 38, no. 7, pp. 646–660, Aug. 2021.
