Bringing Time to the forefront of conversation
Playbook not only supports task planning and execution, but now supports situational awareness on three levels: perception of elements, comprehension of the current situation, and projection of future status.
Playbook overview of key features
Video edited and created by Isabel Ngan
Level 1 of Situational awareness: Perception of elements
The ability to perceive the values, function, and status of the system
Playbook now makes connecting disparate information and data easier through topic clustering. Topic clustering refers to how close together connected information is to an item of interest.
Adjustable User INterface
Adjustable views of the chat and the timeline provides flexibility that focuses on the information that they are looking for.
Bringing time related details forward while splitting who is on what side of the time-delay.
Custom Marcus Bains Line
The red-horizontal line, the Marcus Bains Line, provides a better way to link information from the timeline and the chat. A desync option is provided for added flexibility.
Level 2 of Situational awareness: Comprehension of the current situation
The way we understand and interpret data based on how it is presented to us
The design highlights how we provide context to conversation by providing additional understanding about an item of interest. Context tells the history and background of information, like metadata, and also hints at what you can do with it or expect going forward, aka, understanding the full situation.
We designed key message actions to allow for clear communication: reply, mark messages obsolete, set reminders, create to-dos, and add quick replies like “copy.”
Understanding how messages may cross-over by visualising when messages are sent and delivered.
Level 3 of Situational awareness: Prediction of future status
The ability to predict how elements will vary in the near future
Playbook assists in the crew's or mission control center's need to do time math. Time math is a concept which adds another layer of complexity to communicating with latency. Someone does time math when they make a calculation to predict a future state while incorporating a time delay.
Allowing crew and Mission Control Center to easily identify messages that are in-transit and messages that have been sent.
Playbook in Action
Videos edited and created by Isabel Ngan, compiled by Phil Gase
How we discovered a new mental model for conversation that would assist in synchronous communication with sustained latencyCheck out the Team Blog For a Sprint Breakdown
1 // Discovery
Subject Matter Expert Interview
Simulated Missions v1.0
2 // Focused Exploration
Simulated Missions v2.0
3 // General Ideation
4 // Focused Ideation
5 // Prototyping
6 // Evaluation
Simulated Missions v3.0
Since we can’t make anything travel faster than the speed of light (yet), How can a communication tools better support deep-space missions?
The NASA AMES HCI group tasked our team with building on NASA’s existing communication and planning tools, Mission Log and Playbook, to improve and redefine asynchronous communication between crew and ground in deep space missions.
Currently, synchronous crew and ground communications are crucial for successful procedure execution. Ground provides crew real-time assistance for task support, troubleshooting, and more. However, synchronous communication will not be possible in deep space missions due to inevitable communication delays, so there must be increased support for asynchronous communication tools.
Finding Opportunities in the most complex situation, extravehicular activities
From our research, we found that there are quite a few different scenarios and tasks that happen on these missions. But the most difficult situations happen during extravehicular activities, also known as EVAs. These are activities that occur whenever astronauts have to leave their base and go outside.
UNDERSTANDING current space communication to better design for future communication
To establish a foundation of knowledge about space missions and the role of communication, I assisted in conducting subject matter expert interviews, helped create and run a simulated experiment mission, and led a literature review of relevant papers to understand the current and future states of both synchronous and asynchronous space communication.
Simulated Mission v1.0
To immerse ourselves in the context of the problem space, we created an analogous domain "Simulated Mission" to get a deeper understanding of asynchronous communication in team-based environments that are similar in key ways to what astronauts and MCC will experience. We started off by simulating a task focused on remote instruction, where our “Mission Control” participants guided the Crew on how to build a lego space shuttle using only time delayed messages.
UNDERSTANDING current space communication to better design for future communication
We found that Playbook is currently situated and built around the current state of space operations. Our goal was to imagine and test different futures of space communication. This helped us build a roadmap to the future state, and it’s in the blue sky that we can generate far-term solutions that have the potential for significant impact on mission communication.
Simulated Mission v2.0
To corroborate data from Speed Dating on the viability of our initial solutions, we needed to test our ideas in the context of potential use, which pushed us to create a more thorough idea of a testing ground to use this summer. This expanded simulated mission addresses EV/IV scenario of collecting scientific evidence for ambiguous research goals in coordination with MCC. This scenario allows us to test complex and innovative ideas like AI and Automation.
Key opportunities to better support the future of space communication
Increased contextual and situational awareness is needed to ensure message comprehension.
Communication tools need to assist in conveying emotion, increasing trust, and supporting morale.
Delayed communication will result in a more autonomous crew which will require increased support in crew’s decision making ability.
As time-delay increases crew’s autonomy, tools will need to decrease cognitive load during task execution.
UNDERSTAND AND DESIGN FEATURES NECESSARY TO IMPROVE SITUATIONAL AWARENESS WITHIN DELAYED COMMUNICATION
Using our findings, our team determined that positioning Playbook and Mission Log at the center of space communication and improving on a multimedia chat feature will simplify the communication process.
Using our 4 insights as the jumping off point for ideation, I led two ideation sessions. Using the Round Robin method, the ideation session generated over 100 ideas that our team organized into near-term, midterm, and far-term.
We realized that if we focus on improving situational awareness, the other 3 insights will follow.
Improving situational awareness became the north star for our designs.
New Mental Model
Redefining a new mental model for communication
We created clickable Figma prototypes for usability testing and coded a functional chat prototype to try out in simulated missions. Together, both prototypes helped us validate and translate a new mental model into our final design.
why did we need to understand and design for a new mental model in the first place?
Our team tried applying standards we currently use to communicate, but realized that an asynchronous mental model, like email, and a synchronous mental model, like instant messaging, don't quite fit.
Currently we experience 2 main mental models for communication: Sequential Model and Tree.
Our new mental model first takes the Tree model and structures messages around topics or replies. Then it takes an adaptation of the sequential model, and considers when a message is sent and delivered. This model doesn’t only consider the past and current aspects of time, but the future too.
Design through Research
To better understand this mental model and test our design ideas, we conducted 20 usability tests with a mixture of participants, both from the general populace and those that more closely resemble our target participants.
Testing our design within a delayed context
To take our research a step further and see how our ideas felt in the real world, we began running experiments we called Simulated Missions.
We simulated a time-delayed EVA with a photo scavenger hunt game.
Participants were split into two groups: “Mission Control” and “Crew.” The crew was composed of Photo Takers, who represented EVs, and helpers, who represented IVs. Photo takers had to take photos of items that matched a certain prompt, such as red things or broken things.
Mission control would then receive those photos and assign points to them based on how well they matched the prompt. They evaluated photos based on information that we provided to them for their eyes only. This extra knowledge that they had simulated how Mission Control in real life has much more information at their disposal than crew does.
As for the Helpers, they were the intermediary between Mission Control and Photo Takers, like IVs.
They would pass on photos, messages, and provide assistance to Photo Takers as they needed it. All communication between Crew and Mission Control had a 5 minute delay. In our simulation, Mission Control and Crew had to use our prototype to communicate, share information, and strategize to win the scavenger hunt.
This research protocol was provided to the NASA Ames Research Center for future use.
Going beyond 2030
Further Testing in Context + Notification System
- Increase testing over increased time periods
- Further advance the design foundation
- Explore notifications and Notification hierarchy
Use Case Expansion
- Explore more form-factors and other use-cases
- Expanding Playbook to handle non-task communication
Integration + Evaluation
- Integration into other NASA tools, like Astrobee
- Research into advanced technologies and smart devices
- Look into automation based assistance
Understanding a paradigm shift
This process pushed our team to immerse ourselves in a situation that only a few experience. Through the creation of our simulated missions, we were able to not only understand the current state of space communication but use it as a platform to test out new design. Our team found success in speaking with different experts from across NASA and inspiration in analogous fields like Formula 1 environment and telehealth.
Designing for a brand new experience requires a deep understanding of the current space and need for a clear design foundation
This project pushed each member to think about ways in which we could bring a future situation to the present. I initially wanted to re-design large portions of Playbook, but quickly realized that by narrowing our focus and targeting our scope, we were able to create a stronger design foundation that could be built upon in the future.
I find inspiration in analogous domains to help me understand a problem space I have little knowledge about.
Knowing the complexity of space travel and space communication, I found it essential to understand and determine the specific properties that made up this complicated problem space. This provided me with a better way to find analogous domains, which inspired different designs and methods when approaching the problem.