Who We Are

We use augmented reality to positively impact children with Autism Spectrum Disorder.

Group Joining AR focuses on how to facilitate social group joining skills of children with ASD using AR technology. This project will be able to provide an interface to help children with ASD develop their social group recognition and group joining skills that not only requires less human resources but has more accessibility.

See the problem

Approximately 4,617,647 people have ASD in US. It is very expensive (timewise and cost wise) for children to go to a local treatment center and receive regular treatments.Is it possible for children diagnosed with ASD to start practicing certain social skills anywhere, anytime? AR technology opened a door to greater possibilities in front of us.

Motivation Using AR

AR technology provides immersive experience, which will be able to combine the actual space where children feel familiar and comfortable with and virtual elements that are rich in instructional information. By encoding knowledge of how social group is formed (including: number of people, gaze, shape, body orientation) into virtual agents, our project will allow self-practice with less human resources.

Findings

A gradual and instructive experience that could exrish our users is what we are aiming for. Since we learned that people with ASD are relatively more sensitive to unexpected change (e.g. sudden visual change, sudden noise, etc. ), and they tend to be attentive to non-typical social cues, we decided to include repetitive practice scenarios that start with minimal background noise and minimal social cues in our program. We believe certain design will be very effective and instructional.

According to their responses, they agree with the intuitiveness of the group arrangements designed in the app. At the same time, they show moderate interests in participating in the virtual agent groups. However, evaluation data were collected only from college students.

Future Work

Finish working functionalities for all categories, both models and comments. Expand the practice cases and build up the intensity of each case.

More user-friendly and aesthetic user interface.

Customizable scenarios and characters for better user experience so that our user can choose their preferred theme instead of randomization.

More customization and flexibility feature that can let domain expertise manipulate. More powerful supervised practice with analysis.

What We Do

We have concise weekly plans we followed that lead to the current stage we are standing on.

Week 1 - Meeting with Domain Expert

We had a meeting with Jessica Keith, who is a domain expert from department of Psychology. We layed out our initiatives of the project and discussed the possible functionalities that can be suitable for such app. She validated our approach using AR which strongly encouraged us to keep exploring.

Week 2 - 3D Virtual Agents Design

We decided to use Blender to create several 3d models of Bugdroid (Android's mascot) since it is easy to manipulate so we can accelerate our prototype building progress.

Week 3 - Educated with domain knowledge

Jessica gave us a thorough presentation about who are people with ASD and how to work with them, which further inspired us to construct the entire project.

Week 4 - App Development

In order to make the whole practice more attractive, we have different predefined scenes and models. After the user chooses the mode, our program will randomly select a combination of scene and model to make the whole process more diverse.

Week 5 - Media Prep

* Final Presentation
* Demo Video
* Poster presentation
* Final Report

Week 6 - Here we are!

Please check out the demo below to see the fantastic prototype of AR Group Joining Practice!

Design & Implementation

Take a closer look at the features in
AR Group Joining Practice

Main menu

When the user opens the program, they can choose one of the six modes to start, which are the four factors of group joining (“number of people”, “gaze”, “shape”, “body orientation”), “Tutorial” and “Combined”.

Surface Detection

After user’s selection, they will be guided into the AR view with a pointer at the center and the “add” button at the bottom-right corner. When the program successfully detects a valid surface, the pointer will turn from white to green.

Practice Scenario Rendering

the user can place the virtual agents via the “add” button. Then the user can walk around and observe the placed virtual agents through the screen.

Instruction Prompt

When the user decides whether the current group formed by the virtual agents is available to join or not, he or she can press the “check” button at the top-right corner and verify with the given comment.

Contact Us

Get in touch and let's make something great together. Let's turn your idea on an even greater product.

Where To Find Us

Gao Fan
fgao at u.rochester.edu
Meiwen Zhou
mzhou26 at u.rochester.edu
University of Rochester

Hello folks, we are AR Group Joining Practice