Vendor Sponsor
Model Evaluation & Threat Research (METR)
Research Published
July 10, 2025
Link to research
Description
16 experienced developers in “a randomized controlled trial (RCT) to understand how early-2025 AI tools affect the productivity of experienced open-source developers working on their own repositories. Surprisingly, we find that when developers use AI tools, they take 19% longer than without—AI makes them slower.”
Comments
I really like the way the authors discuss the limitations and caveats attached to studying the impact of AI on developer productivity. I need to spend som emore time with this study, but on top of the ‘how fast?’ question, I wonder if AI had any impact on the quality of the code or architectural decisions? (And if the tasks included more important software challenges than code generation)
Demographic or Methodology comments
Topic Tags
AIDevelopersAI adoptionSoftware Engineering Leadership
Sample
Observed behavior of a group
Sample Size
16
Demographics
Created time
Jul 16, 2025 6:21 PM
Directory name
The RightStack Research DB
