Indeed the experiment conducted by the CMU SEI seemed as if it were designed with the specific intent to disprove the 10X factor that multiple other studies had shown. They first taught all the developers to follow the CMU SEI "Personal Software Process". I am thankful for the CMU SEI PSP, because it's so bad that we've been able to hire great developers who fled from companies that instituted it. The process forces developers to record all their time "on the major activities needed to complete a solution that passed all required tests, including planning, design, coding, testing, and personal reviews" during development. Clearly handicapping developers with ineffective bureaucratic processes up front will add a large constant to everyone's time, which then decreases the ratio of their times.
Furthermore they tested in an academic environment on simple assignments designed to take an hour or two. It's impossible to do all the required steps in 6-12 minutes, so this experiment could never show a 10X disparity.
Furthermore they tested in an academic environment on simple assignments designed to take an hour or two. It's impossible to do all the required steps in 6-12 minutes, so this experiment could never show a 10X disparity.