Communicating Semantic Content to Persons with Deafblindness by Haptograms and Smart Textiles: Theoretical Approach and Methodology Show others and affiliations
2020 (English) In: International Journal on Advances in Intelligent Systems, E-ISSN 1942-2679, Vol. 13, no 1&2, p. 103-113Article in journal (Refereed) Published
Abstract [en]
By means of a proof-of-concept prototype, which is work in progress, we adopted a multidisciplinary approach to develop a smart-textile-based communication system for use by people with deafblindness. In this system, sensor technologies and computer vision are used to detect environmental cues such as presence of obstacles, faces, objects, etc. Focusing on the communication module here, a new ontology connects visual analytics with the user to label detected semantic content about objects, persons and situations for navigation and situational awareness. Such labelled content is then translated to a haptogram vocabulary with static vs. dynamic patterns, which are mapped to the body. A haptogram denotes a tactile symbol composed over a touchscreen, its dynamic nature referring to the act of writing or drawing. A vest made of smart textile, in the current variant equipped with a 4 x 4 grid of vibrotactile actuators, is used to transmit haptograms on the user’s back. Thereby system messages of different complexity -- both alerts and short sentences -- can be received by the user, who then has the option to respond by pre-coded questions and messages. By means of grids with more actuators, displays with higher resolution can be implemented and tested, paving the way for an extended haptogram vocabulary, covering more detailed ontology content.
Place, publisher, year, edition, pages 2020. Vol. 13, no 1&2, p. 103-113
Keywords [en]
deafblind communication, haptograms, word and sentence semantics, ontology, smart textiles
National Category
Engineering and Technology
Research subject Library and Information Science
Identifiers URN: urn:nbn:se:hb:diva-23904 OAI: oai:DiVA.org:hb-23904 DiVA, id: diva2:1474921
Projects SUITCEYES
Funder EU, Horizon 2020, 780814
Note This work has been partially funded by the European Union’s Horizon 2020research and innovation programme under grant agreementNo 780814 SUITCEYES.
2020-10-112020-10-112024-04-26 Bibliographically approved