TRAINscendence
UX research for a virtual reality (VR) simulation designed to train Locomotive Engineers.
My role
Motion designer & UX researcher
Team size
Solo project
Timeline
1 month
Tools
Autodesk Maya & Adobe After Effects
Project overview
In the context of a fourth-year UX research course, I undertook the challenge of conceptualizing a study for a Virtual Reality (VR) system. My focus centred on designing a UX research study aimed at evaluating the potential advantages of incorporating VR technology into the training system for Locomotive Engineers. Through this project, I gained valuable insights into conducting research studies, developing VR systems, and ensuring the overall user-friendliness of all components involved. This experience not only broadened my understanding of UX methodologies but also honed my skills in crafting immersive and user-centric solutions within the realm of Virtual Reality.
Problem
Travelling nearly 500 km with almost 1 million passengers per week, Go Transit is one of the most relied-upon services in Ontario. With government commitments made to extend Go Train rail lines, build new paths, and offer more services, the demand for locomotive engineers can only increase.
Since they are the ones responsible for driving/operating the train, it is safe to assume that the training process to become a locomotive engineer is quite intensive. According to VIA Rail Canada, it can take up to 2 years of hands-on training for someone to become a locomotive engineer. Because of this, it is not uncommon for agencies such as Metrolinx to report staffing shortages. In 2022, Metrolinx was forced to cancel some of its Go Train trips due to an operator shortage. The extensive training program coupled with the staffing shortage proves that there needs to be a change in the overall training process.
To help craft a solution, this study will be guided by the following research questions:
Can the simulation enhance the efficiency of the current training process for locomotive engineers by alleviating task loads?
Can employees acquire all the required skills to become a locomotive engineer in a shorter duration?
In a conclusive examination, will employees who underwent the virtual reality (VR) simulation outperform those who underwent traditional training methods?
Goal
The key goals of this project include:
Evaluate training methods by comparing the effectiveness of VR simulation versus traditional approaches for locomotive engineers.
Streamline the training program by reducing task loads, thereby shortening the time required to complete the program.
Design a robust study that ensures reliable and unbiased results, regardless of whether they support or contradict the hypothesis.
Background
Current training system
Currently, Metrolinx uses a PC-based simulation as part of their locomotive engineer training program. Using three large TV screens, the company can replicate the look and feel of a locomotive console (Llewellyn, 2018). Now if Metrolinx is already using a PC-based simulation, this begs the question of why switch to VR? According to researchers, while PC simulations can be highly interactive, they lack immersion, while VR is both interactive and immersive thanks to the larger field of view and autonomy that it provides (Petersen et al., 2022). As a result, higher levels of immersion typically lead to better performance, thus justifying the switch from PC to VR simulators.
Related studies
A study done in 2023 showed that there was a need for VR in locomotive training, as well as proved the effectiveness of these simulations (Gao et al., 2023). Researchers state that there are many reasons why agencies should be implementing VR technology into their training programs. When looking at extreme resource consumption, application gaps when moving from a classroom setting to a real train, and high risk for on-site damages, there are many benefits that come with using VR (Gao et al., 2023).
In the study done by Gao et al. (2023), they examined the performance outcome for VR-based training versus traditional training across 100 new employees. 60% of the participants went through the training process using the VR system and were further broken down into three groups based on their professional background and VR experience. The remaining 40% of participants were trained using traditional methods and were separated into two groups based on their professional backgrounds. In efforts to produce the fairest results, the five groups were assessed after 10 days of training using a traditional evaluation procedure that included an examination and questionnaires, thus determining the pass rate of each group, respectively (Gao et al., 2023).
Results from the study showed the pass rate for new employees who did use the VR system, regardless of their professional background and VR experience, was higher compared to the participants who used traditional training methods. Most significantly, for the employees that had no professional background, the group that used VR had a 90% pass rate whereas the group that used traditional methods had a 70% pass rate, thus proving the effectiveness of this system. Furthermore, this study also proves the effectiveness of the VR system as it alleviates many of the drawbacks associated with traditional locomotive training methods including reduced resource consumption, avoidance of on-site damages, and better visualization of emergency drills (Gao et al., 2023).
Amid all these advantages, there is one major critique that comes with this VR system. Due to repetitive actions and reduced motion, participants are prone to boredom. Also, they are prone to cybersickness (Gao et al., 2023). Furthermore, it is important to consider these shortcomings in future designs.
Solution
About the system
Introducing TRAINscendence, a VR simulation that is designed to replicate what the locomotive car looks like in a Go Train. It aims to make the training process for locomotive engineers more effective by reducing the task load and resource consumption. In this simulation, employees will learn how to navigate the operator’s console and learn how to use each component. Beyond that, employees will go through a series of mock rail paths and learn how to handle different obstacles. The goal of this simulation is to ensure that employees are equipped with the necessary skills and knowledge that they will need to operate a real train.
How it improves existing research
As mentioned before, one of the biggest critiques that come with any kind of VR training for vehicle operations is boredom. When practicing repetitive motions, users are prone to boredom. To combat this issue, this VR system will integrate gamification. According to researchers, there are many benefits to using gamification in an education setting as it can boost student’s intrinsic motivation to learn, encourage active learning, and provide structured guidance based on the needs of each student (Martí-Parreño, 2016). For this VR system, a variety of gamification elements such as different game levels and a progress bar are used to help motivate participants to complete the locomotive training program.
Study design
This longitudinal study is conducted over the course of 8-months, employing a mixed-method approach that incorporates both qualitative and quantitative research methodologies. This research not only quantitatively measures participants' success rates but also delves into qualitative aspects, exploring the thoughts and emotions they have towards the system. Utilizing a between-groups structure, participants are divided into two overarching cohorts, with one group undergoing testing using the VR simulation, while the other experiences traditional training methods. This comprehensive approach aims to provide a thorough understanding of the effectiveness of both training methods over an extended timeframe and across various outcome measures.
The independent variable is based on the type of training method used, whether it is the VR simulation or traditional training practices. The dependent variables aim to measure the efficiency of the two training processes by looking at the time it takes to acquire the required skills, as well as performance levels on the final examination.
Although the traditional training program takes up to 2 years, this study is conducted over 8-months. One of the goals of this study is to see whether or not using VR can help reduce the timely length of the training program by reducing task loads. Hopefully, through a comprehensive VR-based course, employees will be able to acquire the necessary skills faster, thus reducing any filler steps. By condensing the study into 8-months, we gain a better idea of how much information participants can learn in a shorter time frame when using the VR simulation compared to traditional training methods.
Participants
Through our efforts, my team and I successfully tackled Sun Life's case problem and created an application that provides everyday support for women aged 25 - 44 with bipolar disorder. WAVE addresses the challenges our users face, offering them an intuitive and personalized solution that complements traditional therapy and enhances its effectiveness.
Selected as one of the top 5 teams in our cohort, we had the opportunity to present WAVE to a panel of professionals at Sun Life's XD Month.
Training with traditional methods
Has no professional experience
Has professional experience
Training with the VR simulation
Has no VR experience and no professional experience
Has both VR and professional experience
Has either VR or professional experience, but not both
Data collection & metrics
Lauren Stacy is a 26-year-old business analyst from Halifax who recently moved to Toronto to start a new job. Lauren also has bipolar disorder.
Moving to a new city, starting a new job, and leaving family and friends can be overwhelming for anyone, but it is especially challenging for those with bipolar disorder, where consistency in schedules and systems is vital. Any disruptions in these routines can have negative effects.
Finding a new therapist isn't easy for Lauren either. She struggles to open up to new people and often forgets what she wants to discuss, making therapy sessions feel unproductive.
Lauren needs a day-to-day solution that provides sustainable mental health support, helping her reflect on and manage her emotions while staying connected to her support system.
Limitations
This study concentrates on the perspective of a singular engineer, however, it is important to acknowledge that operating a train involves a collaborative effort. A locomotive car has two seats – one for the operator’s console and the other for the helper’s console, thereby emphasizing the shared responsibility in this task. However, considering the importance of the working relationship and communication between primary and secondary operators, it is worth noting that the study fails to incorporate testing for a multiplayer function. This limitation highlights the need for future research to explore and evaluate the effectiveness of VR simulations in facilitating teamwork and communication.
Additionally, one of the most significant critiques that comes up in all vehicle-based simulations, including this study, revolves around the occurrence of cybersickness (Gao et al., 2023). While the design of this study does implement measures to help participants recover from cybersickness, there are almost no prevention techniques. Despite efforts to mitigate the adverse effects of cybersickness, the inherent complexity of tackling this issue from the start is a significant challenge when it comes to vehicle simulation research. The study emphasizes the significance of continued research and development of ways to improve user comfort and reduce cybersickness in hopes of improving the overall simulation experience.
