[Gif] Demo Scenario #1


Prototyping Augmented-Reality Experience

Two researchers conducted a study and discovered that students at CMU struggled with reaching out to one another after returning to campus from remote learning.

The researchers came to my team with their Head-mounted Augmented-Reality devices (HoloLens). They invited us to match the technology to innovations facilitating human-human interaction for a future CMU campus.



Two researchers had surfaced CMU students' needs for technologies that could help them strengthen/expand connections on campus.


I was responsible for exploring opportunities for AR-based innovations using HoloLens, and eliciting design implications for future products.


I conducted three rounds of user studies with 20+ participants using prototypes I built that bridged them to futuristic experiences.


I prototyped a solution using Unity and Mix-Reality Toolkits (MRTK) for a future campus where Head-mounted AR technology is used ubiquitously.

our Project Timeline

PAX AR Project Timeline
The challenge

Bridge Users towards Futuristic experiences & innovations

Clients' Request:
a right thing to do

With faith in ubiquitous usage of Head-mounted Augmented-Reality Technology (HMD-AR) in the future, my clients want to innovate solutions that help students reach out to friendswith HoloLens.

HoloLens 2, image credit to Microsoft

My Role: Innovate & Evaluate Futuristic experiences

The explicit challenge: How might we facilitate people on campus reaching out to friends and strangers with AR ?
The underlying challenge: How might we conduct research to draw implications for futuristic products/services?
The ideation

Sketch experiences like a director

goal: Define the Role

As we studied Hololen's functionality, we noticed various opportunities but few metrics to evaluate ideas internally. Hence, I convinced the team to get our users involved and determined the technology's role in a bottom-up fashion.

Stage 1: Ideation

Our Goal: Define the technology's role

Strategy: Storyboarding

Considering my immature technical skill and our urgent need to collect feedback, I found storyboards to be the ideal prototype format. The storyboards let me efficiently create boundary objects and effectively convey imaginary design ideas to participants.

User Study: Speed-dating

I presented 9 different storyboards to 6 participants in quick succession during "speed-dating" interviews. I asked my participants to articulate how much they could relate to the character in these scenarios, and learned how they evaluate the effectiveness of proposed technologies.

My teammate Ahana in speed date with Participant #1

participants in "speed-dates" with storyboard ideas

  • Learn more about key questions raised by our participants

    "Abnormal" Behavior

    Is there any unusual action or extra effort required to use this technology?


    Does the person and the augmented content show-up at the same time?


    How much control does the augmented content creator/audience have?


    Who is told what by the AR/XR technology?

The winnter: augmented status indicator

THE iteration

Create experiences like a wizard

Goal: Define Look & Feel

Now that we agreed on the technology's role, our next step is to determine how to display the augmented information. To explore different looks & feels while ensuring the quality of each prototype, we decided to conduct parallel prototyping.

Stage 2: Iteration

Our goal: Define the technology's look & feel

My embodied experiences

Strategy: prototyping embodied experiences

For my prototype, I explored indicating status with AR emojis based on my hypothesis that the emojis would leave clues on its owner's availability and mood.

User Study: Wizard-of-oz

Instead of developing high-fi mock-ups, I used the wizard-of-oz technique to create illusional embodied experiences. I set up the space by placing emojis in advance and having the assistants sit in designated seats. Then, I introduced the testers to the environment as if others picked and placed the emojis.

Introduce testers to environment after everything has been set up

participants in a "Wizard-of-oz" session

The Takeaway: THree design implications

Augmented information for status indication can be used most effectively in the right context where people need to have a conversation.
Emojis have low affordance to deliver consistent definitions, especially compared to texts. Therefore, future designs should provide guidelines and instructions to help users select/interpret emojis.
Though augmented information is provided to help start a conversation, users don't always know how to leverage the knowledge without instructions and guidelines.
THE Integration

Build experiences like an engineer

Goal: Validate feasibility for implementation

The design implications and parallel prototyping practices led us to a final idea: placing text-based augmented information around interior doors to indicate the status of office room owners. As a final step, I aimed to integrate the implementation aspect to validate the technical feasibility of our idea with a desirable role, look, and feel.

Stage 3: Integration

Our goal: Validate feasibility for implementation

My mixed-reality prototype Development process

Strategy: Developing Mixed-reality prototypes

I studied the Hololens Application development process, and rapidly prototyped a functional application using Unity and Mixed-Reality Toolkits.

User Study: multivariate Prototype Testing

Due to changes in the course schedule, my project length shrinks from 5 weeks to 3 weeks. Though I didn’t get to develop/test my new prototypes, I drafted a plan for multivariate testing with a set of eight prototypes to reach the best-integrated design.

A review of my procedure: ideate, iterate, integrate

An overview of our prototyping process
THE Reflection

new wine in old bottles

“The double-diamond” is one of the most frequently used frameworks and buzzwords among design students and practitioners. During this rapid prototyping project, I gained new understandings of the influential framework as I executed it under a tight timeline.

The double diamond framework

Conceptual model of the double-diamond design framework

Switching between diverging and converging thinking, I encountered a lot of perceived moments of success as we came up with funny ideas, pivotal insights, and functional prototypes. I found the key to driving design progress with the double-diamond is to defer judgment of ideas and focus on executions strategically.

Moving forwward in the double-diamond

Push design forward with diverging and converging thinkings

When it's time for judgments, I learned to calmly review my past design decisions based on users' and teammates' critical feedbacks. As I embraced the iterative aspect of the double-diamond framework, I started to treat my designs/prototypes as seeds. Then, I naturally began to view the critiques as fruits I harvest. This reframed point of view helps me refuel myself with critiques and challenges.

Moving backward in the double-diamond

deal with frustration after receiving feedback and critique

I started to treat my designs/prototypes as seeds. Then, I naturally began to view the critiques as fruits I harvest.
I'd love to hear what you think!


Thank you for your comment!
Oops, something went wrong. Please try again.