AI-Powered Interactive Multimodal Digital Book & Online Shop For Blind and Visually Impaired Users

Salous, Mazen; Westphal, Daniel; Heuen, Wilko; Ben Dhiab, Ayoub; Hudin, Charles; Paneels, Sabrina; Boll, Susanne; Abdenebaoui, Larbi

doi:10.24963/ijcai.2026/991

AI-Powered Interactive Multimodal Digital Book & Online Shop For Blind and Visually Impaired Users

Mazen Salous, Daniel Westphal, Wilko Heuen, Ayoub Ben Dhiab, Charles Hudin, Sabrina Paneels, Susanne Boll, Larbi Abdenebaoui

Download BibTeX Cite

Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence. Demo Track. Pages 8509-8512.

https://doi.org/10.24963/ijcai.2026/991

Keywords

AI: Humans and AI
AI: Computer Vision
AI: Machine Learning

Abstract

We present an AI-powered interactive multimodal system that enriches digital image accessibility for blind and visually impaired (BVI) users. Our demonstration showcases two application domains: (1) an educational digital book, and (2) an online shopping interface. In both use-cases, users can virtually feel material textures (leather, wood, etc.) and engage in voice-driven inquiry about images. The system integrates state-of-the-art AI components – including voice-to-voice conversational agents, vision models for object segmentation, and a custom 16-actuator vibrotactile display – to provide multimodal feedback (haptic vibrations, spoken descriptions, and audio cues).

The result is an inclusive technology with significant societal benefit, empowering BVI users to learn and shop more independently through natural multimodal interactions.

Mazen Salous, Daniel Westphal, Wilko Heuen, Ayoub Ben Dhiab, Charles Hudin, Sabrina Paneels, Susanne Boll, and Larbi Abdenebaoui. 2026. AI-Powered Interactive Multimodal Digital Book & Online Shop For Blind and Visually Impaired Users. In Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence, IJCAI-26. IJCAI, 8509–8512. https://doi.org/10.24963/ijcai.2026/991

@inproceedings{ijcai2026p991,
  title     = {AI-Powered Interactive Multimodal Digital Book & Online Shop For Blind and Visually Impaired Users},
  author    = {Salous, Mazen and Westphal, Daniel and Heuen, Wilko and Ben Dhiab, Ayoub and Hudin, Charles and Paneels, Sabrina and Boll, Susanne and Abdenebaoui, Larbi},
  booktitle = {Proceedings of the Thirty-Fifth International Joint Conference on
               Artificial Intelligence, {IJCAI-26}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {Diego Calvanese},
  pages     = {8509--8512},
  year      = {2026},
  month     = {8},
  note      = {Demo Track},
  doi       = {10.24963/ijcai.2026/991},
  url       = {https://doi.org/10.24963/ijcai.2026/991},
}

TY  - CPAPER
TI  - AI-Powered Interactive Multimodal Digital Book & Online Shop For Blind and Visually Impaired Users
AU  - Salous, Mazen
AU  - Westphal, Daniel
AU  - Heuen, Wilko
AU  - Ben Dhiab, Ayoub
AU  - Hudin, Charles
AU  - Paneels, Sabrina
AU  - Boll, Susanne
AU  - Abdenebaoui, Larbi
T2  - Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence, IJCAI-26
PB  - International Joint Conferences on Artificial Intelligence Organization
PY  - 2026
SP  - 8509
EP  - 8512
DO  - 10.24963/ijcai.2026/991
UR  - https://doi.org/10.24963/ijcai.2026/991
ER  -