IIIT Allahabad Develops AI Tool for Visually Impaired to Access Chart Images

The tool, named Alt4Blind, allows users to search for and understand chart images through alternative texts (Alt-text) generated by the AI software.

Researchers from the Indian Institute of Information Technology-Allahabad (IIIT-A), Prayagraj, in collaboration with the Karlsruhe Institute of Technology, Germany, have developed an AI tool to aid blind and visually impaired individuals in accessing chart images.

The tool, named Alt4Blind, allows users to search for and understand chart images through alternative texts (Alt-text) generated by the AI software.

Sharing thoughts, Shahid Ali Farooqui, PhD scholar, Indian Institute of Science Education and Research (IISER)-Kolkata, who led the research project, said, ‘’Existing models are susceptible to producing inaccurate or misleading information. We have addressed this challenge by retrieving high-quality Alt-Texts from similar chart images, serving as a reference for the user when creating Alt-Texts.”

How does the AI Tool Work?

The tool generates Alt-Texts for chart images. These Alt-Texts provide textual descriptions of the visual content within the charts. Users can then access and understand the charts using these descriptive texts.

Further, Alt4Blind enhances accessibility by making visual information available to those with impaired vision.

The move aims to promote independence and inclusivity by enabling users to interact with charts effectively.

“We have introduced a new benchmark comprising 5,000 real images with semantically labeled high-quality Alt-Texts, collected from Human-Computer Interaction venues. We have also developed a deep learning-based model to rank and retrieve similar chart images that share the same visual and textual semantics, and we have designed a user interface (UI) to facilitate the Alt-Text creation process. Our preliminary interviews and investigations highlight the usability of our UI,” Farooqui explained.

The research was presented in a paper titled “Alt4Blind: A User Interface to Simplify Charts Alt-Text Creation” at the International Conference on Computers Helping People with Special Needs.

The project, led by Farooqui, a senior technical officer at IISER-Kolkata, was conducted under the guidance of Mohammed Javed, an assistant professor in the IT department at IIIT-Allahabad.

“Alternative Texts (Alt-Text) for chart images are essential for making graphics accessible to people with blindness and visual impairments. Traditionally, Alt-Text is manually written by authors but often encounters issues such as oversimplification or complication. Recent trends have seen the use of AI for Alt-Text generation,’’ said Farooqui.

In a similar development, in April, a former IIIT Allahabad student developed a smartwatch that will help in measuring pulse rate, blood pressure, temperature, etc. with an accuracy rate of more than 95%.

The device continuously monitors the patient's health even while being away from him. Currently, the device is not yet available for commercial use. The developers are still in the process of adding more features and conducting clinical tests. The project is estimated to be completed by 2025.