The problem with these models, in general, is that they are reward-maximizing. They have no real intelligence, capability to engineer, or creativity to speak of. This makes them great for verifiable tasks where you can get away with “guided brute force”, but for anything else it ends up being not very good. I think the recent RL on “one-shotting apps” may have also made them worse at anything that isn’t one-shotting apps