Overview
- Villanova University researchers compared three published human short stories with ChatGPT versions and ran three experiments that measured readers' ratings of story quality and engagement.
- Across the experiments, AI-generated stories scored about 6% higher on quality and about 8% higher on engagement than the human-written originals.
- Readers gave the highest ratings when AI-written stories were labeled as if they had human authors, adding roughly a 3% boost in preference.
- Participants struggled to identify AI authorship, with correct detection around 40% in one test and about 52% in another, while self-reported familiarity with AI improved detection but literary experience did not.
- Authors say the results likely reflect models being tuned to broadly appealing patterns and that raising public AI literacy is a practical way to help people recognize machine-generated writing.