Virtual Keyboard-Mouse in Real-Time using Hand Gesture and Voice Assistant

Document Type : Original Article

Author

Higher Institute of Computers and Information Technology, Computer Depart., El. Shorouk Academy, Cairo, Egypt

Abstract

The development of computer applications has made our daily tasks more convenient and efficient. One of the latest trends in this area is the integration of voice assistants and virtual input devices. This approach is particularly useful for individuals with mobility impairments or for situations where manual interaction with a computer is not possible. The application of computer vision and AI technologies has made this possible. Computer vision enables machines to interpret and understand visual data, such as the movements of the human body, while AI provides the decision-making abilities required to execute commands based on the input received. The integration of voice assistants and virtual input devices into computer applications has the potential to revolutionize the way we interact with technology. The combination of computer vision and AI technologies is driving this innovation, and it will be interesting to see how it continues to evolve in the future. This desktop application offers a comprehensive solution for users seeking a more accessible and efficient computing experience. By integrating a voice assistant, virtual mouse, and virtual keyboard, it enables individuals with physical disabilities or those preferring alternative input methods to fully utilize their computers, improving productivity and enhancing their overall digital experience. We use Python, Media Pipe, and OpenCV. The Media-Pipe library provides features that boost the model's efficacy and is especially useful for AI projects. The user will be able to control the computer cursor using different hand gestures, type on the virtual keyboard while holding colored caps or tapes and left-click and drag items.

Keywords

Main Subjects

Files

Share

How to cite

Statistics