The issue with sonnet 3.5 is, in my limited testing, is that even with explicit, specific, and direct prompting, it can’t perform to anything near human ability, and will often make very stupid mistakes. I developed a program which essentially lets an AI program, rewrite, and test a game, but sonnet will consistently take lazy routes, use incorrect syntax, and repeatedly call the same function over and over again for no reason. If you can program the game yourself, it’s a quick way to prototype, but unless you know how to properly format JSON and fix strange artefacts, it’s just not there yet.
Why do the leaders in AI know so little about it? Transformers are completely incapable of maintaining any internal state, yet techbros somehow think it will magically have one. Sometimes, machine learning can be more of an art than a science, but they seem to think it’s alchemy. They think they’re making pentagrams out of noncyclic graphs, but are really just summoning a mirror into their own stupidity.
It’s really unfortunate, since they drown out all the news about novel and interesting methods of machine learning. KANs, DNCs, MAMBA, they all have a lot of promise, but can’t get any recognition because transformers are the laziest and most dominant methods.
Honestly, I think we need another winter. All this hype is drowning out any decent research, and so all we are getting are bogus tests and experiments that are irreproducible because they’re so expensive. It’s crazy how unscientific these ‘research’ organizations are. And OpenAI is being paid by Microsoft to basically jerk-off sam Altman. It’s plain shameful.