Research: How Ratings Systems Shape User Behavior in the Gig Economy

Rating systems, integral to the platform economy, profoundly influence human behavior and choice. Platforms like Uber, Airbnb, Turo, and Upwork rely on these systems not just as reflections of past performance, but as proactive tools for ensuring quality and encouraging proper conduct on both sides of a transaction from service providers (such as drivers and hosts) and users (like riders and guests).

Platform providers typically display ratings information to the user in two ways. Incremental rating systems, employed by platforms like TaskRabbit and Airbnb, offer a detailed view by listing and often providing insights into every individual review score. In contrast, averaged rating systems, used by platforms such as Uber, Lyft, and DoorDash, present an overall score that aggregates all individual ratings.

We wanted to understand the impact of different rating presentation formats (incremental vs. averaged) on user perceptions and behavior. More specifically, can platforms use certain rating formats to diminish (or conversely intensify) the impact of a low rating to guard the overall engagement and service quality?

We conducted nine experiments for our forthcoming study in the Journal of Marketing Research, which illustrate how the way low ratings are communicated shape user experience and behavior. Our findings offer implications for companies choosing between incremental or average ratings systems:

Incremental ratings systems highlight low scores and prompt performance improvement.

In the platform economy, it’s vital to quickly identify and address poor service. Consider a ride-sharing platform like Lyft. If a driver receives low ratings, an incremental rating system would make these scores more visible, signaling both to the driver and the platform the need for improvement or even potential deactivation. This approach is crucial not just for maintaining quality standards but also for ensuring passenger safety and satisfaction.

Advertisement

Incremental scores allow providers to outgrow setbacks.

For service providers on platforms like Airbnb who may have had a rough start or a few early poor reviews, an incremental system can be more forgiving. As they make improvements — for instance, by upgrading their accommodation or enhancing guest experience — these positive changes will be more immediately visible in their more recent ratings. This approach offers a quicker path to overcoming past setbacks, crucial for hosts looking to rebuild their reputation and attract more guests.

Incremental systems can significantly amplify negative ratings.

By highlighting each low score, incremental systems can significantly amplify the negative impact of these ratings on service providers. This heightened visibility of negative feedback not only affects the individual’s self-perception, it can also lead to a stronger desire to disengage from the platform.

Averaged ratings systems help soften the blow of low scores, aiding retention and loyalty.

Consider a delivery driver on a platform like DoorDash who experiences an unusual day of delays due to traffic or order issues. While this might result in a few low ratings, averaging these scores over multiple deliveries places this anomaly within the larger context of their overall performance. By showing that one bad day won’t drastically alter their rating, the platform fosters a sense of loyalty and encourages service providers to stay committed, understanding that their efforts over time are recognized and valued.

Averaged ratings systems obscure low ratings.

We found people are not good at estimating what scores they got when they only see an average score. Consider a driver who was rated on three total trips: Two of their passengers gave them a five-star rating, while the third gave them a one-star rating. Our research suggests that this driver would likely estimate their most recent rating to be significantly higher if they were shown an average rating of 3.67. 

In summary, it’s essential for platforms to understand the subtleties of rating systems when balancing quality control, fairness, and user motivation. The choice between incremental and averaged scoring systems significantly affects how users respond to low ratings. While established platforms might find it challenging to change their rating formats due to their long history, new platforms can thoughtfully select and customize their rating systems. By doing so, they can gain a competitive edge, fostering stronger user loyalty or effectively managing poor-performing users. As the platform economy evolves, these considerations become crucial in strategic decision-making, helping businesses align with their goals and adapt to the shifting landscape of user engagement.