Overview
The goal of this project was to implement design modifications to streamline the United Airlines app flight selection process. These modifications were intended to increase app usability and improve user attitudes toward the airline without affecting user spending.
Ethnography
To identify and address specific pain points, I conducted (some very casual) ethnographic research. I sat down with three participants and asked them to use the United Airlines app to select flights for a trip they’d like to take, talking me through their thought process along the way.
Below are the main themes that emerged from this formative phase.
From this brief analysis, I was able to see that a key user value (affordability) was not respected by the app design. Users were required to work, at every step in the process, to find cheap flights. I chose visual and structural design modifications meant to bring cheap fights closer to the user, saving them effort and time.
Original workflow
Modified workflow
Usability Test
I used pilot data to calculate the minimum required sample size for two sample t-tests, however I ended up conducting factorial ANOVAs instead, as I wanted to control for cost consideration during the flight selection process. This left the analysis underpowered. For this reason, p-values alone were not a reliable indication of significance and were considered alongside the size of the effects. Of the 15 participants who completed the task, two were excluded from the analysis due to data collection errors.
Methods
Participants were asked to complete the flight selection process for a trip from Los Angeles to New York. This involved selecting a flight, confirming the fare selection, and choosing between or opting out of upsells.
The post-experiment survey consisted of five items measuring user attitudes, the Single Ease Questionnaire, and one item assessing cost consideration when choosing a flight.
Results
App Usability
Compared to participants who used the original app, those who used the modified app completed the flight selection process with 40% fewer interactions, in 30% less time, and reported SEQ ratings 20% higher. While only the difference in number of interactions was statistically signifiant, all effect sizes were medium to large.
Interactions by app version, t(9)=3,90, p=.004, d=2.18.
Time taken by app version, t(9)=1.48, p=.178, d=.83.
SEQ rating by app version, t(11)=.93, p=.371, d=.52.
User Attitudes
Of users who considered only fare affordability, those who used the modified app reported higher scores on all survey items measuring positive attitude. Despite an insignificant p-value, the size of this effect was large.
Positive attitude score by app version, t(6)=1.19, p=.284, d=.92.
User Spending
Users of both app versions spent the same amount of money on flights and upsells. This effect was both small and insignificant, suggesting no difference between groups.
Amount spent by app version, t(11)=.55, p=.591, d=.31.
Key Findings
The design modifications tested here likely increase app usability and improve user attitudes toward the business, without affecting user spending. While not all of these differences were statistically significant, medium to large effect sizes support this conclusion.
The design modifications tested here could be implemented now to streamline the flight selection process at no cost to the airline. Beta testing on a larger sample could be done before a full rollout to verify these findings.
