Overview

• METR’s randomized trial with 16 expert open-source developers on 246 real tasks showed AI assistance increased completion time by 19%.
• Participants forecast a 24% reduction in task time only to experience a 19% slowdown when using AI tools.
• AI-assisted coders spent over 20% more time prompting, waiting, reviewing outputs or idling, which reduced active coding time.
• Developers accepted fewer than 44% of AI-generated suggestions and devoted 9% of their time to cleaning up generated code.
• Researchers emphasize that the results apply to current tool performance on familiar codebases and caution that ongoing AI enhancements could reverse the productivity impact.