> Massive increases in training costs and parameter count seem to be yielding diminishing returns. Or maybe this effect is illusory.

But.. that's always been the case? Diminishing returns has always been the name of the game - utility tracks log(training effort). Its not such a big point that he makes it out to be.