Challenge Space
According to the World Health Organization (WHO), ''almost everyone will temporarily or permanently experience disability at some point in their life''1, with around 1.8 million people with special needs living in Switzerland. In 2021, the ''Swiss Federation of Deaf People'' and the ''Swiss Federation of Blind and Partially Sighted People'' re-emphasised that people with special needs must ''receive the appropriate human and technical resources to participate fully and auton-omously in society''2, highlighting the need for technical resources that facilitate and empower people.
Proposed Research
In a 3-year long research effort at the University of St. Gallen, funded by the HSG - Uni SG - International Postdoctoral Fellowships, we investigate the use of assistive mixed reality technology to improve how people live, work, navigate, and interact in reality and with fellow humans, moving to a new era of human-computer interfaces. We will explore the design space of assistive mixed reality technology to speed up development, help economic growth and bring people closer together.
This project will contribute towards an inclusive future reality where people with and without disabilities receive appropriate technical resources and are empowered to participate fully and autonomously in society.
References:
1 https://www.who.int/health-topics/disability
2https://www.swissinfo.ch/eng/swiss-disabled-groups-launch-initiative-for-greater-inclusion/46950100
Language barriers pose significant challenges in our increasingly globalized world, hindering effective communication and access to information. Existing translation tools often disrupt the current activity flow and fail to provide seamless user experiences.
In this project, we contribute the design, implementation, and evaluation of MRTranslate, an assistive Mixed Reality (MR) prototype that enables seamless translations of real-world text. We instructed 12 participants to translate items on a food menu using MRTranslate, which we compared to state-of-the-art translation apps, including Google Translate and Google Lens.
Findings from our user study reveal that when utilising a fully functional implementation of MRTranslate, participants achieve success in up to 91.67% of their translations whilst also enjoying the visual translation of the unfamiliar text. Although the current translation apps were well perceived, the participants particularly appreciated the convenience of not having to grab a smartphone and manually input the text for translation when using MRTranslate.
We believe that MRTranslate, along with the empirical insights we have gained, presents a valuable step towards a future where MR transforms language translation and holds the potential to assist individuals in various day-to-day experiences.
This work has been published and presented at the International Conference on Advanced Visual Interfaces (AVI 2024) in Arenzano (Genoa), Italy. The full description of the scientific paper can be found here: "to appear soon"
Authors: Dr. Florian Mathis (HSG), Yu Sun (HSG), Adrian Preussner (HSG)
Contextually aware visual, auditory, and haptic interfaces can augment and empower people in their everyday life. However, little is known about the use cases and users' perspectives on a pervasive augmented reality that augments places and humans. In this paper, we contribute a first step towards an augmented societal future by outlining promising example use cases for assistive mixed reality interfaces (i.e., AssistiveMR).
By surveying 60 participants, we found that an augmented reality has the potential to find wide-spread application in a plethora of scenarios, including supporting people in recalling information, disconnecting from reality, and augmenting communication with others using visual augmentations. However, participants expressed concerns regarding the potential high costs associated with errors and raised questions about the social acceptability of augmentations of humans and real-world surroundings.
Our exploration of promising use cases for assistive MR augmentations aims at serving as inspiration, motivating researchers to augment, support, and empower individuals in their daily activities.
This work has been published and presented at the Augmented Humans conference in Melbourne, Australia. The full description of the scientific paper can be found here: "to appear soon"
Authors: Dr. Florian Mathis (HSG)
We are running multiple user studies throughout the year. If you are interested in participating, please have a look at the calls for participation below. Additionally, you can contact the lead researcher Dr. Florian Mathis via e-mail (florian.mathis@unisg.ch) or via phone (+41 71 224 38 76).
We are currently developing a set of spatial computing prototypes that support people with visual disabilities in their everyday life. For Summer 2024, we are looking for interested participants who are willing to try out our new systems and provide us with important early insights into the efficiency and user experience. Participation will be paid according to the local standard of participation reimbursement.
To participate in this important research or learn more, please reach out to the lead researcher Dr. Florian Mathis at florian.mathis@unisg.ch or +41 71 224 38 76.
Call for Participation: How can we best support you in your everyday life?
> We are looking for people with disabilities to participate in an ethnographic study to learn more about everyday life challenges and where we can build prototypes that assist in day-to-day tasks. <
The University of St. Gallen is conducting a research study to better understand the daily challenges faced by individuals with disabilities and, in the next step, to collaboratively design systems that enhance people’s everyday lives.
If you have a disability and are willing to share your experiences, your insights will play a crucial role in shaping these supportive systems. Participation is voluntary; all information will be anonymized and kept confidential, and we will reimburse you for your time according to our local participant payment policies.
Together, we can create a more inclusive and empowering future, and we hope you are interested in supporting us on this journey.
To join this important research or learn more, please reach out to the lead researcher Dr. Florian Mathis at florian.mathis@unisg.ch or +41 71 224 38 76.
Call for Participation: Mixed Reality for Translating Reality
> We are looking for participants to participate in a laboratory study to learn more about the use of mixed reality for translating reality. <
The University of St. Gallen is conducting a research study to better understand how mixed reality can be utilised to translate real-world content into a user's preferred language.
Participation is voluntary and all information will be anonymized and kept confidential. If you are above 18+ and are willing to participate in a maximum one hour research study, please reach out to the lead researcher Dr. Florian Mathis at florian.mathis@unisg.ch or call +41 71 224 38 76.
We are looking forward to your participation! The study will be conducted in Torstrasse 25, 9000 St. Gallen.
Wissenschaftlicher Mitarbeiter
Ordinarius für Human-Computer Interaction