RecoVR is a game that improves engagement and adherence with at-home physical therapy (PT) for acute musculoskeletal (MSK) conditions. It was designed for my undergraduate engineering honors thesis.

RecoVR

Timeframe

June - November 2022

Team

me, myself, and I

Role

designer, developer, author

Scope

development and testing of a minimal viable product

Tools

Unity, Visual Studio, Python, Figma, FigJam

Over the course of 6 months, I designed and tested a proof of concept game for my thesis, for which I was awarded High Honors in Engineering Sciences. The preliminary findings will inform my research over the next 6 months, as I transform RecoVR into a fully-functional system and conduct a longitudinal study to better understand how narratives and games might impact user behavior and stimulate intrinsic motivation.

Read about my journey so far below and stay tuned for more!

Problem

HMW increase engagement and follow-through with at-home physical therapy (PT)?

  • Musculoskeletal (MSK) conditions the leading cause of disability worldwide. Yearly, 50% of American adults >18 develop an MSK condition lasting >3 months that requires PT.

  • 16.5 million Americans undergo PT for these conditions, but <1/3 follow through with prescribed exercises at home.

  • The problem is only increasing as the population ages.

Solution: RecoVR

an app that maps PT to VR games that motivate users to follow treatment in a sustainable manner, by stimulating intrinsic drive and reducing pain perception


System Design

There are 2 parts to the system: a data analysis pipeline the frontend VR game component.

A computer vision algorithm collects motion data in real time and analyzes performance, feeding that information to the frontend VR game. The game communicates progress back to the data pipeline to inform further data collection.

  • If the player is conducting the exercise, regardless of performance, the game progresses.

  • If the player makes mistakes, the game offers concise, prescriptive feedback to help the user improve in real time.

System Architecture Diagram

At present, the data collection pipeline and frontend VR game work independently, but a client-server communication model needs to be implemented for a fully-developed application. I will be working on that and other improvements in the following months.

MVP: a PT-enhancing, fantasy quest

This video demonstrates the user experience as the player (myself) completes a couple reps of two core exercises for early ACL recovery. The player goes on a short quest guided by Robo, the friendly robot, to free the guardian spirit of the realm that was trapped by an evil wizard. To complete the game, the player navigates the beautiful world, acquires magical powers, builds a bridge, and destroys the energy field trapping the spirit, all by exercising and working towards their recovery goals.

Process

Research: User Research, Lit Review: Design Insights, Market Research

Game Development: Brainstorming, Interaction Design & Development, User Testing

The following sections discuss my research and game development process, which were split in swaths of about 3 months each. For more details, check out my written thesis or the GitHub page I will keep populating as I develop the project.

 User Research

  • Target users: young adults with acute injuries requiring PT

  • Extreme (“bright spot”) users: college varsity athletes

  • Key stakeholders: PT specialists

I conducted user research with 15 athletes with PT experience and 4 PT specialists and uncovered that pain and boredom are the main reasons for a lack of adherence to treatment. I also decided to focus on a game for anterior cruciate ligament (ACL) injury, a common, serious condition with PT perceived as boring and repetitive.

These are the main takeaways:

Market Research

I conducted an analysis of gaps and opportunities in the market for digital PT tools for acute MSK conditions and identified a few promising companies that help patients monitor performance during at-home PT and/or immerse users in VR environments to reduce pain perception and stimulate engagement.

I selected six companies based on their relevance, size, and prevalence in Internet searches and articles, and evaluated their products across five criteria pertaining to design criteria identified through literature review.

Brainstorming & Core Design Decisions

  • The usefulness of games and gamification to stimulate motivation in healthcare applications, and more specifically PT, has not been researched extensively. None of the game-based solutions for physical therapy I identified in literature were narrative-based, despite the usefulness of narratives to spark intrinsic motivation in people. Moreover, they relied on gamification as opposed to deep game mechanics.

    I chose to design a narrative-driven game to explore the potential of this type of game in PT.

  • I opted for a 3D as opposed to a 2D game, to create a more immersive experience for the user that emulates their activity in the real world.

  • PT specialists I interviewed recommended VR over a 3D PC/tablet game, for its immersive nature and the growing evidence that it reduces discomfort and pain perception.

    Moreover, as headsets become more compact and comfortable, they will become easier to maneuver when exercising in different positions (laying down, standing up) as opposed to screens.

  • I created a 1st person game (as opposed to 3rd person) to immerse the player - who becomes the hero - into the narrative. This also helps them relate their recovery goals to game objectives more easily.

  • The environment, with its luxurious vegetation and temple ruins, stimulates curiosity and promotes adventure. Moreover, literature suggests fantasy is the best genre for PT games, which I leaned into.

    I acquired the environment from beffio and integrated it into my game. I decided spending a couple days to find a well-developed package for VR that fit within my vision for the game would be more useful, given the short timeframe, than spending weeks to design the perfect world. This choice paid off, as I was able to complete the project and run a preliminary study, which is very valuable for the future direction of the project.

    I chose music to instill a sense of calm exploration, fit for PT, which players also loved. I acquired the music from Unity’s free 3D Game Kit, which I explored for a PC version of my game.

  • In a nutshell, computer vision algorithms are fast, accessible, accurate, free, and easier to use for real-time data collection than IMUs. Conversely, IMUs have a bigger learning curve and are more expensive for real-time applications.

    MediaPipe, specifically, is highly accurate and available for implementation on most modern devices.

  • Robo, the friendly robot helper character, marries storytelling and positive reinforcement. It narrates the storyline and delivered positive reinforcement, which is very useful during PT. Robo appeals to the emotions of the player, motivating them to keep going and immersing them into the game.

    Based on literature, I decided the helper-character should be a cute robot with a positive attitude and vague human resemblance (to avoid the Uncanny Valley).

    Some users hate how often helper characters pop up in video games, so I made the choice to only show Robo at checkpoints.

    It worked! All users enjoyed their interaction with Robo, and even suggested more engagement, e.g. by giving Robo a voice or involving it more actively into the activities.

Core design choices:

Below are some brainstorming artifacts including images from various games, helper characters from famous movies and games, ideas for environment design, and user flow demos for first and third-person games using a scavenged Unity environment for 3D PC games.

Read on to learn about the core design decisions and game mechanics built into this experience.

RecoVR stimulates intrinsic drive through a full game experience with a two-fold reward mechanism. Inside the game, rewards enable the player to further explore the game and unlock new challenges to complete the quest. Moreover, the game’s goals are designed to align with the player’s recovery goals.

To avoid the folly of superficial gamification, I used aspects like fantasy, challenge, and curiosity that make games inherently motivating. I designed the game considering insights from brainstorming and literature reviews of health & PT games, sports psychology, and multimodal feedback for PT.

User Journey:

Mapping the Player’s Recovery Goals to Game Objectives

Here are some of the key interactions that prompt the player to exercise while providing engagement and hedonic motivation through game mechanics and environment design.

Held knee bend = activating walking/teleportation to navigate the world

Leg raises = dynamic game actions, like blasting an energy field or building a bridge to traverse the world

The game keeps track of rep counts and delivers concise, prescriptive feedback when the player makes mistakes, as below, where the players is bending their knee too much during leg raises.

Robo, the helper character, guides the player through the story and meets them at each checkpoint. Robo appeals to the emotions of the user, marrying storytelling and positive reinforcement.

While Robo delivers the story, the player receives exercise and game instructions through text, images, and videos demonstrating correct execution.

 User Testing

I conducted Wizard of Oz (WoZ) testing with n=6 healthy participants (3 men, 3 women) with former PT experience for knee injuries including ACL injury, as a proxy for injured individuals

Goal: How do people engage with and react to RecoVR? Does the MVP show potential to stimulate adherence to PT? How might we improve usability and user experience in future iterations?

I set out to test my MVP with at least 5 users, following the well-established UX “rule of 5,” which states that 5 testers will uncover 85% of usability problems.

Because the motion data pipeline and the Unity game were not connected via client-server communication, I conducted WoZ testing, which involves interacting with a mock interface controlled by a human without the knowledge of research participants. I used keyboard inputs to activate walking/teleportation powers when users bent their knee, as well as lay bridge tiles and hit the energy field when users did straight leg raises. I also used keyboard inputs to display feedback for straight knee raises, which were especially valuable to ensure participants kept their knee as straight as possible.

Testing Procedure

  1. Self-assessment questionnaire: examined their experience with and perception of PT on Likert scale (1-5)

  2. Play RecoVR: used keyboard inputs to simulate motion tracking

  3. UX assessment: live questions and surveys assessing experience and engagement with RecoVR on Likert scale (1-5)

Results

    • players found the experience rewarding and enjoyable

    • one of their favorite aspects was the mapping between exercises and game activities

    • players enjoyed that the game kept track of their reps and provided real-time feedback, which I noticed helped them improve during testing (e.g. straighten their knee when it’s bent)

    • players enjoyed the environment and found the music relaxing

    • all of them enjoyed the storyline, despite its simplicity, which indicates the potential of good narratives to stimulate people

User Engagement Scale (UES) survey results: median scores across users. Users found the experience very rewarding, as intended by design, aesthetically appealing, and immersive. Perceived usability ranked lowest, although it was above average, for reasons pertaining to a general lack of familiarity with VR.

Unified Theory of Acceptance and Use of Technology (UTAUT2) survey results: median scores across users. Social influence ranked highest, indicating that users would recommend RecoVR to friends and acquaintances undergoing PT. All other measures ranked highly, indicating strong levels of enjoyment (hedonic motivation), that users perceived the experience as useful for improving performance during at-home PT (performance expectancy), and that they would use the app for PT if it were developed (behavior intention).

All users enjoyed the presence of Robo, the helper character, one of the core game mechanics employed by RecoVR. Despite the simplistic storyline, users found it engaging, an encouraging finding for the continued development of narrative-driven games.

User Feedback & Possible Improvements

  • better onboarding for new VR users - before VR becomes mainstream, good onboarding is critical for adoption

  • improved navigation - VR navigation is discontinuous to avoid motion sickness, which is unusual and takes a while to get used to; in future iterations, I will streamline navigation to make it as natural as possible

  • include positive feedback during exercises - aside from the positive reinforcement Robo offers at each checkpoint, the game could inform players they are performing exercises correctly throughout each set

  • less text, more visuals - people hate reading, which is why I tried to keep exercise instructions clear and concise; improvements could include giving Robo, the helper character, a voice; players could also benefit from graphical feedback showing correct execution versus their performance

  • more developed game mechanics, including a more suspenseful storyline - for future iterations, I would like to do more research into game narratives and possibly collaborate with a writer

Conclusion & Next Steps

Considerations: The feedback I received from user testing with people with little to no VR experience prompted me to thinking about how to reduce friction between users and new technology. Designing for behavioral change also challenged me to ponder how I might use novel technology to improve quality of life while mitigating unintended consequences, like digital inequality and addiction. 

Further development: I will continue my research on RecoVR for the next 6 months. This winter, I will develop my product and network as part of the Dartmouth Innovation Accelerator for Digital Health. My first step will be to integrate design insights from user testing and develop client-server communication to enable a fully-functioning application. I am also forging partnerships with interested students/researchers specialized in human motion tracking and VR development.

Research: My next study will challenge the state-of-the-art through a randomized controlled trial to compare engagement with objective feedback to narrative-driven, ‘gameful’ experience delivering at-home PT.

Previous
Previous

Platouas

Next
Next

3D Model: DC Flood-Prevention Strategy