Quality of Training Data

Study Finds Viral ‘Junk’ Data Causes Lasting Decline in Open-Source Language Models

A multi-university preprint ties sustained training on clickbait posts to persistent performance declines.