Abstract

Mobility is one of the main challenges for most people with disabilities. A number of traditional solutions are indeed adopted to support mobility (e.g., the white cane for people with visual impairments, the wheelchair for people with motion disabilities). At EveryWare Lab we research mobile assistive applications to provide additional support for the mobility of people with disabilities.

Recognizing urban objects.

While some acoustic signals are starting to be available in the urban environment (e.g., acoustic traffic lights), most road signs are still perceivable with sight only. This includes, for example, pedestrian crossings and traffic lights. Recognizing these signs is a major challenge for people with visual disabilities. We proposed solutions to recognize pedestrian crossings and traffic lights from mobile devices. More recently, we developed a solution to detect generic obstacles that the user should avoid while walking.

Conveying information in real-time

In addition to acquiring contextual information (e.g., a traffic light position) there is an additional challenge that needs to be faced: to convey the information to the user with visual impairments. Indeed, addressing this problem requires to balance two contrasting needs: on one side, we want to convey a large amount of information, possibly at frequent intervals and with many details, so that the user can be precisely guided in the environment. However, on the other side, the user should be able to interpret this information in real-time, while concentrating on the navigation task. So, for example, audio information should be limited, in order to allow the user to pay attention to the surrounding sounds. Similarly, other forms of interactions (e.g., haptic) should be easy to interpret and require a limited mental load. To address these problems we are researching solutions based on sonification, that can provide continuous information updates while not distracting the user from the surrounding audio scene.

Acquiring accessibility-related geo-referenced information

An important service for people with disabilities is to compute routes that are personalized according to the specific needs and abilities of each user. This requires to have a detailed knowledge of the accessibility-related information in the environment, like where pedestrian crossings or curb ramps are. Acquiring and keeping this information updated in a major challenge. We address this problem with two orthogonal approaches. On one side we adopt computer vision solutions to recognize interesting objects from street-view images. On the other side, we rely on crowd-sourcing that does not require user intervention; the idea is that, while a wheelchair user moves in the environment, inertial data are collected and processed with a machine learning approach to classify the urban feature the user encounters, like a curb ramp.

 

 

Bibliography

  • Presti, G., Ahmetovic, D., Ducci, M., Bernareggi, C., Ludovico, L., Baratè, A., ... & Mascetti, S. (2019, October). WatchOut: Obstacle Sonification for People with Visual Impairment or Blindness. In proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility (pp. 402-413). ACM.
  • Ahmetovic, D., Avanzini, F., Baratè, A., Bernareggi, C., Galimberti, G., Ludovico, L. A., ... & Presti, G. (2019, March). Sonification of Rotation Instructions to Support Navigation of People with Visual Impairment. In 2019 IEEE International Conference on Pervasive Computing and Communications (PerCom (pp. 1-10). IEEE.
  • Civitarese, G., Mascetti, S., Butifar, A., & Bettini, C. (2019, March). Automatic Detection of Urban Features from Wheelchair Users’ Movements. In 2019 IEEE International Conference on Pervasive Computing and Communications (PerCom (pp. 1-10). IEEE.
  • Ahmetovic, D., Avanzini, F., Baratè, A., Bernareggi, C., Galimberti, G., Ludovico, L. A., ... & Presti, G. (2018, October). Sonification of Pathways for People with Visual Impairments. In Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility (pp. 379-381). ACM.
  • Ahmetovic, D., Oh, U., Mascetti, S., & Asakawa, C. (2018, October). Turn right: Analysis of rotation errors in turn-by-turn navigation for individuals with visual impairments. In Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility (pp. 333-339). ACM.
  • Kacorri, H., Mascetti, S., Gerino, A., Ahmetovic, D., Alampi, V., Takagi, H., & Asakawa, C. (2018). Insights on Assistive Orientation and Mobility of People with Visual Impairment Based on Large-Scale Longitudinal Data. ACM Transactions on Accessible Computing (TACCESS), 11(1), 5.
  • Ahmetovic, D., Manduchi, R., Coughlan, J. M., & Mascetti, S. (2017). Mind your crossings: Mining GIS imagery for crosswalk localization. ACM Transactions on Accessible Computing (TACCESS), 9(4), 11.
  • Mascetti, S., Ahmetovic, D., Gerino, A., & Bernareggi, C. (2016). ZebraRecognizer: Pedestrian crossing recognition for people with visual impairment or blindness. Pattern Recognition, 60, 405-419.
  • Kacorri, H., Mascetti, S., Gerino, A., Ahmetovic, D., Takagi, H., & Asakawa, C. (2016, October). Supporting orientation of people with visual impairment: Analysis of large scale usage data. In Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility (pp. 151-159). ACM.
  • Mascetti, S., Ahmetovic, D., Gerino, A., Bernareggi, C., Busso, M., & Rizzi, A. (2016, July). Supporting pedestrians with visual impairment during road crossing: a mobile application for traffic lights detection. In International Conference on Computers Helping People with Special Needs (pp. 198-201). Springer, Cham.
  • Mascetti, S., Ahmetovic, D., Gerino, A., Bernareggi, C., Busso, M., & Rizzi, A. (2016). Robust traffic lights detection on mobile devices for pedestrians with visual impairment. Computer Vision and Image Understanding, 148, 123-135.
  • Mascetti, S., Picinali, L., Gerino, A., Ahmetovic, D., & Bernareggi, C. (2016). Sonification of guidance data during road crossing for people with visual impairments or blindness. International Journal of Human-Computer Studies, 85, 16-26.
  • Ahmetovic, D., Manduchi, R., Coughlan, J. M., & Mascetti, S. (2015, October). Zebra crossing spotter: Automatic population of spatial databases for increased safety of blind travelers. In Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility (pp. 251-258). ACM.
  • Ahmetovic, D., Bernareggi, C., Gerino, A., & Mascetti, S. (2014, August). Zebrarecognizer: Efficient and precise localization of pedestrian crossings. In 2014 22nd International Conference on Pattern Recognition (pp. 2566-2571). IEEE.
  • Ahmetovic, D., Bernareggi, C., & Mascetti, S. (2011, August). Zebralocalizer: identification and localization of pedestrian crossings. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services (pp. 275-284). ACM.