Researchers from Tel Aviv University and Stanford University revealed that artificial intelligence can build general predictive skills using purely self-generated training data. The study, available as a preprint on arXiv, challenges previous assumptions about machine learning requirements.
What it says about AI learning without real-world data
According to the researchers, the study challenges the prevailing assumption that artificial intelligence models require enormous amounts of real-world data to learn useful capabilities. The proof-of-concept study demonstrated that models trained entirely on artificially generated sequences improved their predictive abilities across domains including language, images, music, and DNA.
The research was supervised by Dr. Yoav Levine of Tel Aviv University and conducted as part of the postdoctoral research of Dr. Kfir Dolev in collaboration with Stanford University investigators.
Two models learning together
In the method developed by the researchers, two models start from scratch and learn together. The first model generates short computer programs producing symbol sequences, while the second is trained to predict the next symbol in each sequence.
The first model receives feedback on how much its exercises contribute to the learning process, allowing it to select increasingly useful challenges suited to the learning model's capabilities.
Key facts and where they come from
- AI developed general predictive skills using only self-generated training data.
A new study by researchers from Tel Aviv University and Stanford University shows that AI can develop general predictive skills using only self-generated training data
- Two models learn together, with one generating computer programs and the second predicting the next symbol.
The first generates short computer programs that produce sequences of symbols. The second is trained to predict the next symbol in each sequence.
The open question: How far can artificial intelligence models advance in acquiring universal patterns and reasoning skills without any exposure to real-world information?
