The School of Design at Carnegie Mellon is hiring you to develop a mixed reality hybrid (both in person and remote) design studio environment to enhance the studio learning experience. Focus on one aspect of the studio experience. You will be designing and developing a low-fi prototype of one interaction in a hybrid studio for the client to review.
- Target Audience: Environments students and their classmates, as well as design faculty
- Setting: 10 years in the future
Working with a prompt in which the users (design students) are ones I can personally identify with, I’m curious about exploring the advantages and disadvantages to fitting user needs for hybrid learning. This being said, the one aspect of the studio experience leaves a lot of room for flexibility in bridging the gaps between in person and online learning.
With this project, there were a few important considerations to make:
Where are the gaps of social and educational connection in the current hybrid learning model?
How can I create an interaction that people would be willing to implement into their daily lives? I.e. How can I make the learning curve low enough for users want to actually utilize it?
How can I accommodate for all users and create an inclusive studio space?
Introduction to Mixed Reality | Reading About MR
Before I started the project, I read this medium post that touched on the design process for Mixed Reality. When discussing how to ideate or balance between 2D and 3D brainstorming, it mentioned bodystorming:
“Using simple, cheap crafting materials [to] build physical props to represent digital objects, user interfaces, and animations in a proposed experience”
The purpose of bodystorming is to iterate quickly and efficiently in order to spatially explore an idea, meaning these “prototypes” only need to meet the minimum requirements to convey the concept.
After bodystorming, the reading then discusses acting out how the interaction would play out for the user to simulate the immersive experience, and storyboarding to convey the overall flow of the experience. Finally, by shaping a specialized team to explore a greater breadth of abilities, those able to step out of their comfort would spark new and innovative ideas that would push the project past its limits.
In this article “Augmented Reality: The Ultimate Display,” a clear and vast level of improvement is described in terms of computer displays and input devices. With considerations to the fact that our project scope is 10 years into the future, this article was pretty insightful in helping me understand how I can rely on technological advancement when planning for my project.
Research & User Personas | Understanding the Scope of the Problem
In order to get a better understanding of students’ opinions on the current hybrid model, I asked a few of my classmates on some ways they feel the learning structure or studio experience can be improved.
After talking to a few people (Tate, Joseph, Chris, Francis, and Jubbies), I realized there were a few common issues/frustrations with the current model as well as some new perspectives on how learning can be improved.
Based on these conversations, I split the most common frustrations that we discussed into three main areas: (1) social aspect, (2) learning aspect, and (3) work environment aspect. These three perspectives target different areas in which students feel the hybrid model is lacking in connection, engagement, and productivity.
After calling Jubbies, who is working remote this year, she mentioned how although there are some learning structures that may have worked for a traditional model, they may not be best fit for learning over Zoom. Having a two hour lecture or work session feels much worse over Zoom than in person, and it’s easy for it to feel like a waste of time when you are unable to interact or engage (essentially a one way conversation through a screen).
She said her best classes were the smaller-sized ones with open and engaging conversation, while her worst classes were the larger and longer ones in which students are required to sit through hours of listening to one person.
We also talked about the difficulty of keeping pace and staying motivated with work when everyone is scattered over remote and in-person classes. Whether these consistencies are tied together through casual and social conversations or the physical ability to look over at someone’s desk and see how their progress is, approaching your work from one lens is both limiting and disheartening.
Finally, the contextualization of the working environment is extremely important to facilitating a natural and productive work experience. Showing our work over Zoom is far more intimidating and formal now, and critique often feels like some sort of resume review. Jubbies talked about how the context of being in that space and seeing other people’s works around you builds comfort and push you to explore and take risks.
“When the context to the experience is missing, that is where this feeling of discomfort is coming from.”
We ended our conversation wondering how these contextualized experiences can be replicated or mimicked in order to reduce friction between interactions across technology.
After talking to my classmates both in-person and remote about gaps in the current hybrid model, I created three fictional user personas in order to give myself a better grasp on who might benefit from these AR experiences.
The reading “Kill Your Personas” talked about the balance between fitting a character into the “average user” and stereotyping fictional personas. It also mentioned how bringing in diverse perspectives and opinions would provide a more “human” insight to these personas while also being able to design for relatable users.
With this, I modeled my fictional user personas off of three students in the design class in order to accurately capture my target audience while also viewing varying lenses and personalities.
Compiled based on real-life people and characteristics, these user personas are designed to accommodate for a range of personality types, available/preferable items and resources, as well as working styles both inside and outside the classroom.
Based on my discussions with classmates on hybrid learning, I also wrote out their frustrations and areas that they feel the hybrid model is lacking in order to understand the scope of my users’ needs.
The combination of these character traits and design student archetypes allow for a better approach to how I can best improve the studio experience for both in person and remote students from the current model we have today.
Storyboards & AR Ideas | Narrowing Down Which Gaps to Bridge
For my first idea, I thought about how I can create a space using AR that would allow students to view “live” progress with work and hold these causal conversations that typically occur in a traditional studio.
Similar to how students spend plenty of time on social media (and how the studio is a social setting itself), I explored the idea of online networking while also completing work. This would ideally fit each person’s range of working habits while also providing users with the option to participate or not.
When students start their work, they can Start Work to mark themselves online, to which other students will be able to see through a list of the classmates in their peripheral vision.
Once a student is live or “online,” other students will have the option to click on their profile (and vice versa) and will be given two options: View or Visit.
The View option would allow the students to see the most updated/live version of students’ work in order to see their progress and gauge for themselves whether they are moving in the right direction.
The Visit option would allow students to get to call in and have a casual conversation about social life, work, or anything else. This space acts as a way for students to discuss their ideas and development as well as hold any other interactions that would typically be available in the traditional studio environment.
In targeting ways to conduct a more engaging class (and to reduce the tension of the technological barrier of Zoom), I also thought about how students could have a more natural and authentic wall space that would facilitate more meaningful discussions.
The idea behind this AR wall space is that students who are in person can still hang up their work without the pressure of sharing the screen to the entire class, and remote students can upload their work to the wall so that both in person and remote students can view and interact with the class’s works in a holistic manner.
This bridges the discomfort on Zoom that stems from the intimidating feeling of silence and overwhelming attention, and it also allows for students to gain a better understanding of the project when seeing everyone’s works together.
The professor would be able to enlarge or zoom into a single work for class discussion, as well as rearrange any of the works to get a better idea of how people’s projects intersect or overlap.
Desk Space Documentation | Understanding Hybrid Environments
Before further developing these ideas, I spent some time documenting my desk space and working habits to study my own work environment.
I labeled what I have on my desk and why I have it there in order to get a better understanding of my work environment. I noticed that I have a blend of print content and technology on my desk because even though most of it can be found on my laptop, I prefer to have papers to physically refer to or keep out on my desk to use while I work.
Being able to see all the important content such as the project rubric or my user personas gives me more space and easier ways to reference these papers while I make progress with my work.
On my main desktop, I keep my work organized into folders with all my main programs and platforms at the dock. These are my most-used programs and sites so I can reach them easier.
Also on my main desktop, I keep tabs for my social/entertainment such as my music or Facebook messenger, which is where I am primarily in contact with the rest of my class.
On my other desktop windows, I keep one window specifically designated for design/school content. This is probably my most-visited window, as it’s where I check Canvas, open Zoom, update Medium, and check my emails.
On the far right desktop window, I keep a window specifically designated for tabs relevant to a side project I’m working on with an interdisciplinary team. This way, even if it is also design work, I can keep my school work separate from my outside work/tasks to stay organized.
Self Reflection | Designer’s Role in Hybrid Environments
As the prevalence of digital media in our physical environments increases daily, what is the role and/or responsibility of designers in shaping our environments?
The increased presence of technology in physical spaces is both created and facilitated by designers — this falls under the idea that everything that is published, implemented, or developed must first be intentionally designed. The intentionality is where designers have the most say or the most power in shaping these environments, especially when creating technology or digital media that could either enhance a viewer’s experience or consume it.
For example, in designing a platform that would use IoT to create a more convenient living space, a designer has the capability to create something that could shift the entire environment (such as lighting or heating), but maybe the outcome in doing so would be undesirable if it is too unnatural. The user would likely want a platform that is designed to assist a more comfortable living space, not a platform that would turn the home into a sauna.
This idea of could vs. should often acts as a guideline or measurement to a designer’s work. Just because a designer can entirely shape these environments by creating invasive digital media, does not necessarily mean that the designer should.
In a world where the incorporation of technology into our daily lives is inevitable, the designer’s role is to best facilitate this transition in a human-centered, natural way. While designers cannot control the speed or invention of new technology and digital media, they have a responsibility to incorporate these features into a human society in the least destructive way as possible. And ideally, the most accommodating and sustainable way possible.
Going back to the idea of intentionality, this stage of concept development (designing for the target audience) must be repeatedly incorporated into every iteration afterwards. By keeping the target audience and intention in mind, this would best prevent a designer to get carried away into the “could” aspect of creating technology instead of focusing on the “should.”
Designers should strive to create a space where this balance between digital and physical can be achieved in the most natural and enhancing way — regardless of how forward technology may advance (or especially as it advances).
Prototyping | Experimenting and Exploring AR
In order to get a better understanding of how this would play out in the space, I used Wiarframe to experiment with my ideas.
Using a grey rectangle just as a placeholder to see in AR next to my work space, this helped me visualize what my “networking” tab would look like on the desk.
Because it was similar to my concept or vision, I used a screenshot from Facebook to replace the rectangle to show what it would look like online.
The right side shows a contact list of students that are online/active so that the user can view who else is doing work at the same time.
In the middle, I uploaded an example of a “model” or a project that the student could be working on that the user would be able to view to see other people’s progress.
For my next AR prototype, I decided to move forward and explore my mixed reality “wall space.”
Instead of using a table top AR setting, I changed the setting to room in order for the viewer to be able to see it on a blank wall in front. Then I uploaded some example works from first year just as a placeholder to see what the work would look like.
On the background, I hung up some old work from first year just to visualize how the space would look with both the digital works (from remote students) and physical works (from in-person students) would show on the wall.
This way, students would be able to view both of them in a simulated environment as opposed to sharing screen over Zoom. This would open up more discussion and feel closer to a traditional studio crit or environment.
Feedback
After running through both my ideas with multiple people, it was clear that there were pros and cons to both ideas — with many considerations to work out.
I talked to a few more people from my class to gain their insight on what they might find useful, and they brought up a few significant points:
- What is the difference between Gathertown and the networking idea?
- Would the Figma wall space be only for critique? Could there be other uses for it?
- Does replicating the traditional studio experience necessarily mean a better learning experience?
- How can people be held accountable for having “updated work”? For example, people update their Mediums at different paces, so how can you ensure that the work people see is “live” or updated?
After a few of these conversations, I ultimately decided to choose my desk networking idea as I felt there was more depth to build to that the wall space, and it could applicable for greater use than just critiques.
However, instead of trying to choose both a View and Visit option, I plan to just build upon the View aspect of seeing how others’ progress is coming along.
I also found this article on AR/VR office tours particularly interesting because until this point in the project, I’ve been approaching it with a mindset of “How can I use AR to fix current problems with the hybrid model?”
However, this article highlights ways in which AR/VR can be used to help people view the space before it is created, even when it is not typically considered a problem because people are generally used to the traditional routine of building. Instead of using AR to bridge existing gaps for smaller problems, maybe it can be used to create a new experience entirely — one that would recreate a “traditional process” from scratch.
Design Progress | Working Through Details & Considerations
With my feedback and questions in mind, I went through and sketched out some ideas just to lay out my problem space.
I narrowed the largest considerations into three main questions:
- How can I track work progress in a way that is effortless for the user?
- How can I differentiate classes/content when the user is working?
- How can this experience be enhanced uniquely through mixed reality as opposed to existing systems and platforms?
Instead of using the Facebook user interface concept to allow users to see which students are “online,” I altered the layout so that the avatars would spread out across the viewer’s vision to more closely resemble seeing your studiomates around you. This would feel more “humanistic” for the viewer and be more specific to MR as opposed to existing platforms.
Also, after looking at other people’s work spaces, I realized that many other people use these small walls in front of them to talk utilize their own personal wall space.
These ranged from personal notes and decorations to most recent iterations of their work. Because classes have moved onto Zoom, students no longer hang up works on the wall like we did in the past. Instead, students have been hanging up their work and other things on the small space in front of them, showing their recent works or things that they have been working on.
I began exploring the “personal wall space” idea a bit more to see how students might be able to customize their own work space while also displaying their most recent works.
Ideally, this would be able to help students also have an incentive to use these platforms (lowering the entry threshold of using MR). With a personal wall space, users will be able to use the MR space to their own advantage while simultaneously allowing other students to view their progress space.
Wiarframe Prototyping | Testing Ideas on AR
Testing out the new “personal wall space,” I made some icons on Figma and exported it to Wiarframe to visualize how it would look in a work environment.
After working with the blank wall, I added in some screenshots and pictures of work to the board to visualize how the work would look on the space.
When the user clicks on the one of the avatars, their “personal space” will appear and the user will be able to view their progress and other items the user pins up.
Moving forward, I need to determine ways for a user to pin up different types of work, as well as ways to differentiate the MR wall space and the physical wall space.
Feedback
Moving forward, I realized I should next focus on which specific features I should incorporate into my concept in order to provide in-person users with an incentive to use the wall space in a way that is equally as beneficial and meaningful.
User Research | Understanding & Adapting to Student Preferences
I asked in-person students why they used their smaller wall space in front of them as opposed to working physically, and I also asked if there were any features they wish they could have for their wall space that cannot be completed physically.
Gary (C-studio)
Why do you use the physical wall space vs. putting content on your desktop?
- You might not always see something online
- When something is part of your field of view, you can see it when you look at it
What digital features do you wish you could use on your physical wall space?
- The ability to scale things to sort by priority digitally would be helpful
Joseph (C-studio)
Why do you use the physical wall space vs. putting content on your desktop?
- To showcase my process for other people to see/open conversation
- So the professors can know what is going on
- To get a better understanding of my process (holistic view)
- To be able to see where i should go next & satisfaction of progress
- Visual reference throughout projects/helps give creative inspiration
What digital features do you wish you could use on your physical wall space?
- The ability to arrange pictures
- Annotate pictures or external alterations
- Zooming in and out
Tate (P-studio)
Why do you use the physical wall space vs. putting content on your desktop?
- If I need to refer to them while I am working, I don’t have to pull up that file or switch over; I can look up and see it
- Modeling on Solidworks takes up the whole screen, so it’s difficult to switch tabs
What digital features do you wish you could use on your physical wall space?
- I feel like i can do everything that could be done digitally
Zimmy (P-studio)
What digital features do you wish you could use on your physical wall space?
- Adding likes, messages, or comments (maybe some type of external validation system would give incentive for students to use it)
- Place to give feedback
- Maybe professors can hold class activities by looking at the boards
When talking to Richard (E-studio), he brought up an interesting question to consider:
- What can you do with MR that utilizes a 3D space?
- Instead of putting in 2D platforms into the air, can you make 3d objects that will use the space?
After talking to these students about their use of their own wall space, I decided on a few changes to add/make:
- The digital wall space could store “folders” to utilize the 3D space (this would open up more space for classes) like Desktop folders
- The works that are placed on the wall space can be enlarged and rearranged; students can zoom in and out of works
- Students can leave comments and reactions to work items
Storyboarding | Planning My Sketch Video
To plan for my sketch video, I wanted to lay out the key frames that were communicative of the interactions from my concept.
Essentially, the flow of my sketch video will go as follows:
- User turns glasses on and goes online
- The user’s workspace will open up as the “home page” where their personal wall space appears
- The user can customize, arrange, create folders, and scale the size of their works to best fit their working style
- The user can click “View Studiomates” and select another student to visit, as well as see which students are current online/working
- The user can leave comments or reactions on works for other people to see, ranging from feedback to fun, personalized comments
- The user can go back to home page after visiting, continue working, or exit/log off when finished
Video Sketch | Animating Interactions
To approach the video sketch, I decided to animate the wall space on Figma so that I could overlay my interactions on top of the wall space from a first person’s POV.
To do so, I prototyped some basic frame interactions that would show how the wall space will work. Following the outline of my storyboard, I created what this transparent “wall space” would appear like and I uploaded some example works and pieces from my own classes in order to best represent what the space would look like.
Because I wanted the start of my sketch video to show the personal advantages to the board, I first added features such as the message feature, reaction feature, notification feature, as well as a few demonstrations that showed how the user can move around the works and annotate the pieces. This acts as the incentive to encourage users to utilize the digital wall space over the physical wall space while keeping a light, transparent board available so that the user can refer to it while working.
After showcasing the personal advantages of the wall space board, I then added in frames that would show the “view studiomates” interaction to see which students are online and to see what they are currently working on.
I made a mock wall space board for Holly, another student in E-Studio, to show what it would look like to view another student’s work space and leave a reaction or see what they have “pinned” up.
Moving forward, I built out animations/interactions for the buttons on the Figma board to lay out the sequence of each of my interactions as well as the new overlay frames that would appear or transition the view to.
Afterwards, in order to make a transparent overlay on top of a real life desk space, I changed the background of the board and the frames to green to use as a green screen on After Effects.
Before I moved to After Effects, I set up a green screen filming spot to film my interactions that would be made on the transparent wall space so I could overlay that footage to make it appear that I was clicking the buttons to make them animate.
I then compiled three different videos of interactions — two with a green screen and one simple video of the desk space from a first person’s POV.
Problems I ran into:
- Green screen overlay
At first my green screen wasn’t dark enough, so I had a few issues with making the overlay video visible.
2. Green screen wall space
Moving forward, I plan to adjust the thickness of the white lines and take out the opacity change for the animated interaction on Figma.
3. After Effects
My laptop didn’t have enough RAM so I spent a frustrating 5 hours trying to work through playing the frames per second in real time in After Effects. After switching devices, I was able to finally overlay and add in the extra camera shots to introduce the interaction in my video sketch.
Feedback
In addition to making changes to the purpose or inclusion of the laptop, I also plan to add a personal/private feature for the user to select a “my eyes only” feature for the works on the board. Also, Davis mentioned that it would be good to include people looking at the work, so I plan to add in a simple shot from a different angle to show how users may collaborate on the wall space in a physical context.
Iteration Adjustments | Refining My Green Screen & Recording
For my final video sketch, I had a few more adjustments to my Figma animations to make, as well as adding in the private/public features to allow the user to post works for “their eyes only.” This way, students can use their personal wall space to put up works they don’t want others to see yet.
When students upload their work, they can choose to make their work private or public to indicate how it will appear on the wall space. Once the user taps the lock, the work will be become public and the lock icon will disappear.
I also added adjustments to my Figma board by making certain lines thicker so it would be more visible, as well as making the titles of the board more distinct to differentiate the classes.
Finally, I laid the new screen recording of my edited Figma animations into After Effects with another green screen recording of my hand movements. For my final video, I made sure to add a voice over to describe my key interactions.
After narrating these key interactions, I added in background music to keep the tone of the video lighthearted and productive.
Final Sketch Video
Self Reflection
How were the skills you developed in the first project similar and/or different from the second project? What is your understanding of the role of an Environments designer?
Similar to the first project, I approached this project my first having a heavy focus on what problems or goals I was trying to achieve. By first working conceptually, I could ultimately gear the direction of my final deliverable towards a meaningful and effective idea. For both projects, I found myself constantly asking: would this really communicate to people? Would people approach and use this in the way that I intended? If not, how would they use/interpret it?
In terms of differences, I found myself focusing more time on concept develop on user research and interviews. During the first project, I was focused on translating the goals of the artist into the hybrid exhibition space in an effective way, although the users of the space were separate from the artist. For this project, I constantly worked around the user’s needs and goals when iterating on my mixed reality interaction in order to make adjustments for their desires/frustrations.
This meant working much more collaboratively with people, while also being able to get a better grasp on what people would find useful/unhelpful after talking to the users. In addition, designing an MR system for design students meant that I had a much better understanding of the target audience, so I found myself questioning whether I would even use certain features or interactions myself.
From my understanding based on the Environments mini, my role as an Environments designer is to design and shape experiences for those interacting with or in digital/physical environments. My designs can target different goals that would result in different experiences for the users, so it’s important for me to really flush out and understand the scope of the problem and the reality of the solutions — does my “solution” or product truly fix the problems or gaps that I am trying to bridge? Or am I making something “just to make something?”
The intentionality of the Environments designer plays role in how people experience emotions and build perceptions. My role as an Environments designer is to intentionally shape these experiences to become increasingly valuable and promote a deeper sense of connection to the space.
