“Our primary conclusion across all scenarios is that without enough fresh real data in each generation of an autophagous loop, future generative models are doomed to have their quality (precision) or diversity (recall) progressively decrease,” they added. “We term this condition Model Autophagy Disorder (MAD).”
Interestingly, this might be a more challenging problem as we increase the use of generative AI models online.
Note that humans do not exhibit this property when trained on other humans, so this would seem to prove that “AI” isn’t actually intelligent.
Almost as if current models are fancy token predictors with no reasoning about the input
do we even need to prove this? Like anyone study a bit how generative AI works know it’s not intelligent.
There’s enough arguments about that even among highly intelligent people.
Current AI is not actually “intelligent” and, as far as I know, not even their creators directly describe them as that. The programs and models existing at the moment aren’t capable of abstract thinking oder reasoning and other processes that make an intelligent being or thing intelligent.
The companies involved are certainly eager to create something like a general intelligence. But even when they reach that goal, we don’t know yet if such an AGI would even be truly intelligent.
I don’t think LLMs are intelligent, but “does it work the same as humans” is a really bad way to judge something’s intelligence
Even if we look at other animals, when they learn by observing other members of their own species, they get more competent rather than less. So AIs are literally the only thing that get worse when trained on their own kind, rather than better. It’s hard to argue they’re intelligent if the answer to “does it work the same as any other lifeform that we know of?” is “no”.
Are there any animals that only learn by observing the outputs of members of their own species? Or is it a mixture of that and a whole bunch of their own experiences with the outside world?
Humans (and animals) learn through a combination of their own experiences and observing the experiences of others. But this actually proves my point: if you feed an AI its own experiences (content it has created in response to prompts) and the experiences of other AIs (content they have produced in response to prompts), it cycles itself into oblivion. This is ultimately because it cannot create anything new.
This is why Model Autophagy Disease occurs, I think. Humans, when put in repetitive scenarios, will actively work to create new stimuli to respond to. This varies from livening up a boring day by doing something ridiculous, to hallucinating when kept in extreme sensory deprivation. The human mind’s defence against repetitive stimuli is to literally create something new. But the AI’s can’t do that. They literally can’t. They can’t create anything that doesn’t have a basis in their training data, and when exposed only to iterations of their own training data (which is ultimately what all AI-generated content is: iterations of the training data), there is no process that allows them to break out of that repetitive cycle. They end up just spiralling inwards.
From a certain perspective, AI’s are therefore essentially parasites. They cannot progress without sucking in more human-generated content. They aren’t self-sustaining on their own, because they literally cannot create the new ideas needed to prevent degradation of their own data sets.
From your other comments here, it seems like you’re imagining a fully conscious mind sitting alone in a box, with nothing to react to. But that’s not the case: AIs aren’t sapient, going mad from a lack of stimulation. They are completely dormant until prompted to do something, and then they create an output that is statistically likely from the data set they’ve been trained on. If you add no new data, the AI doesn’t change. It doesn’t seek new stimuli. It doesn’t create new ideas while waiting for someone to prompt it. The only way it can change and create anything new is if it’s given more human-generated content to work with. If you give it content from other AI’s, that alters the statistical probabilities behind its output. If the AIs were actually conscious minds sitting alone in boxes, then exposing them to content created by other AIs would, in fact, be new stimuli that could generate new ideas, in the same way that a lonely human meeting another lonely human would quickly strike up a conversation and get all kinds of ideas.
You’re caught up in an idea that has been going around since long before any AI systems had been built
Humans rarely, if ever, produce something new. We stumble upon a concept or apply one idea to another thing
Neural networks are carefully distilled entropy. They have no subjective biases and no foundation - they’re so good at being original that they default to things useless to humans.
I like to think of training like a mold, or a filter. You only want things in the right shape to come through - the more you train, the more everything coming through looks the same.
Wasn’t the echo chambers during the covid pandemic kind of proof that humans DO exhibit the same property? A good amount will start repeating stuff about nanoparticles and some black lint in a mask are worms that will control your brain?
That only happened to some humans. Something must be seriously wrong with them.
Are we sure they were humans? Maybe they were ChatGPT 2.
If you let the AI feed on its own bullshit long enough it will eventually vote for Donald Trump
That paper makes a bunch of(implicit) assumptions that make it pretty unrealistic: basically they assume that once we have decently working models already, we would still continue to do normal “brain-off” web scraping.
In practice you can use even relatively simple models to start filtering and creating more training data:
Think about it like the original LLM being a huge trashcan in which you try to compress Terrabytes of mostly garbage web data.
Then, you use fine-tuning (like the instruction tuning used the assistant models) to increases the likelihood of deriving non-trash from the model (or to accurately classify trash vs non-trash).
In general this will produce a datasets that is of significantly higher quality simply because you got rid of all the low-quality stuff.This is not even a theoretical construction: Phi-1 (https://arxiv.org/abs/2306.11644) does exactly that to train a state-of-the-art language model on a tiny amount of high quality data (the model is also tiny: only half a percent the size of gpt-3).
Previously tiny stories https://arxiv.org/abs/2305.07759 showed something similar: you can build high quality models with very little data, if you have good data (in the case of tiny stories they generate simply stories to train small language models).In general LLM people seem to re-discover that good data is actually good and you don’t really need these “shotgun approach” web scrape datasets.
For the love of God please stop posting the same story about AI model collapse. This paper has been out since May, been discussed multiple times, and the scenario it presents is highly unrealistic.
Training on the whole internet is known to produce shit model output, requiring humans to produce their own high quality datasets to feed to these models to yield high quality results. That is why we have techniques like fine-tuning, LoRAs and RLHF as well as countless datasets to feed to models.
Yes, if a model for some reason was trained on the internet for several iterations, it would collapse and produce garbage. But the current frontier approach for datasets is for LLMs (e.g. GPT4) to produce high quality datasets and for new LLMs to train on that. This has been shown to work with Phi-1 (really good at writing Python code, trained on high quality textbook level content and GPT3.5) and Orca/OpenOrca (GPT-3.5 level model trained on millions of examples from GPT4 and GPT-3.5). Additionally, GPT4 has itself likely been trained on synthetic data and future iterations will train on more and more.
Notably, by selecting a narrow range of outputs, instead of the whole range, we are able to avoid model collapse and in fact produce even better outputs.
We’re all just learning here, but yeah, that’s pretty interesting to learn about effective synthetic data used for training.
