Abstract
This study investigates whether consumers can differentiate between human-written reviews, AI-generated reviews, and AI-generated reviews mimicking human-like errors across dimensions such as trustworthiness, informativeness, emotional intensity, perceived effort, and perceived fakeness. Additionally, it explores whether these different types of reviews produce comparable effects on consumers' intention to visit the reviewed restaurants.
To address these research questions, we utilized 23,500 reviews from a major online platform to train a GAI model, generating two distinct types of GAI-generated (fabricated) reviews: well-crafted, polished reviews and reviews incorporating human-like errors. In a controlled experiment, 302 participants were randomly assigned to one of three groups: (1) viewed human-written reviews, (2) viewed GAI-generated reviews, and (3) viewed GAI-generated reviews mimicking human-like errors. Participants evaluated the effects of these 5-star reviews on their intention to visit a restaurant with its overall rating of 4-stars.
Analyses using ANOVA, mixed-effects models, and post-hoc tests revealed that all three types of reviews significantly increased intention to visit compared to the pre-review stage (i.e., when only the restaurant’s banner image and 4-star rating are presented). However, no statistically significant differences were observed among the review types, and participants were unable to reliably distinguish between them across key dimensions such as trustworthiness, informativeness, emotional intensity, perceived effort, and perceived fakeness at the p<.05 level.
These findings challenge the assumed superiority of human-written reviews, highlighting the potential for GAI-generated content to achieve comparable outcomes. By introducing review elasticity as a quantitative measure of review quality’s impact on consumer behavior, our study provides a novel framework for understanding review effectiveness. The findings also underscore the need for regulatory frameworks to promote transparency and trust in online review platforms in the era of GAI-generated content. Platforms such as Yelp and TripAdvisor should implement policies like verified visit requirements and safeguards against fabricated reviews to ensure fair competition and maintain system integrity.
To address these research questions, we utilized 23,500 reviews from a major online platform to train a GAI model, generating two distinct types of GAI-generated (fabricated) reviews: well-crafted, polished reviews and reviews incorporating human-like errors. In a controlled experiment, 302 participants were randomly assigned to one of three groups: (1) viewed human-written reviews, (2) viewed GAI-generated reviews, and (3) viewed GAI-generated reviews mimicking human-like errors. Participants evaluated the effects of these 5-star reviews on their intention to visit a restaurant with its overall rating of 4-stars.
Analyses using ANOVA, mixed-effects models, and post-hoc tests revealed that all three types of reviews significantly increased intention to visit compared to the pre-review stage (i.e., when only the restaurant’s banner image and 4-star rating are presented). However, no statistically significant differences were observed among the review types, and participants were unable to reliably distinguish between them across key dimensions such as trustworthiness, informativeness, emotional intensity, perceived effort, and perceived fakeness at the p<.05 level.
These findings challenge the assumed superiority of human-written reviews, highlighting the potential for GAI-generated content to achieve comparable outcomes. By introducing review elasticity as a quantitative measure of review quality’s impact on consumer behavior, our study provides a novel framework for understanding review effectiveness. The findings also underscore the need for regulatory frameworks to promote transparency and trust in online review platforms in the era of GAI-generated content. Platforms such as Yelp and TripAdvisor should implement policies like verified visit requirements and safeguards against fabricated reviews to ensure fair competition and maintain system integrity.
| Original language | Undefined/Unknown |
|---|---|
| Title of host publication | Artificial Intelligence in Management |
| State | Published - Mar 21 2025 |
Keywords
- GAI-generated reviews
- Mimicking Human Errors
- Impact of GAI reviews
- AI Ethics
- digital commerce
- Generative AI