We are friends with children with ASD.
We facilitate your social joining behavior.
Group Joining AR focuses on how to facilitate social group joining skills of children with ASD using AR technology. This project will be able to provide an interface to help children with ASD develop their social group recognition and group joining skills that not only requires less human resources but has more accessibility.
Approximately 4,617,647 people have ASD in US. It is very expensive (timewise and cost wise) for children to go to a local treatment center and receive regular treatments.Is it possible for children diagnosed with ASD to start practicing certain social skills anywhere, anytime? AR technology opened a door to greater possibilities in front of us.
AR technology provides immersive experience, which will be able to combine the actual space where children feel familiar and comfortable with and virtual elements that are rich in instructional information. By encoding knowledge of how social group is formed (including: number of people, gaze, shape, body orientation) into virtual agents, our project will allow self-practice with less human resources.
A gradual and instructive experience that could exrish our users is what we are aiming for. Since we learned that people with ASD are relatively more sensitive to unexpected change (e.g. sudden visual change, sudden noise, etc. ), and they tend to be attentive to non-typical social cues, we decided to include repetitive practice scenarios that start with minimal background noise and minimal social cues in our program. We believe certain design will be very effective and instructional. According to their responses, they agree with the intuitiveness of the group arrangements designed in the app. At the same time, they show moderate interests in participating in the virtual agent groups. However, evaluation data were collected only from college students.
Finish working functionalities for all categories, both models and comments. Expand the practice cases and build up the intensity of each case. More user-friendly and aesthetic user interface. Customizable scenarios and characters for better user experience so that our user can choose their preferred theme instead of randomization. More customization and flexibility feature that can let domain expertise manipulate. More powerful supervised practice with analysis.
We had a meeting with Jessica Keith, who is a domain expert from department of Psychology. We layed out our initiatives of the project and discussed the possible functionalities that can be suitable for such app. She validated our approach using AR which strongly encouraged us to keep exploring.
We decided to use Blender to create several 3d models of Bugdroid (Android's mascot) since it is easy to manipulate so we can accelerate our prototype building progress.
Jessica gave us a thorough presentation about who are people with ASD and how to work with them, which further inspired us to construct the entire project.
In order to make the whole practice more attractive, we have different predefined scenes and models. After the user chooses the mode, our program will randomly select a combination of scene and model to make the whole process more diverse.
* Final Presentation * Demo Video * Poster presentation * Final Report
Please check out the demo below to see the fantastic prototype of AR Group Joining Practice!
When the user opens the program, they can choose one of the six modes to start, which are the four factors of group joining (“number of people”, “gaze”, “shape”, “body orientation”), “Tutorial” and “Combined”.
After user’s selection, they will be guided into the AR view with a pointer at the center and the “add” button at the bottom-right corner. When the program successfully detects a valid surface, the pointer will turn from white to green.
the user can place the virtual agents via the “add” button. Then the user can walk around and observe the placed virtual agents through the screen.
When the user decides whether the current group formed by the virtual agents is available to join or not, he or she can press the “check” button at the top-right corner and verify with the given comment.
fgao at u.rochester.edu
mzhou26 at u.rochester.edu
University of Rochester