New Training Method Helps AI Generalize like People Do - Scientific American

About New Training Method Helps AI Generalize like People Do - Scientific American

This Scientific American news article, written by Lauren Leffer and dated October 26, 2023, reports on a study published in Nature about a training method that may help machine-learning models generalize more like people. The article explains compositionality, the human ability to combine known parts into new meanings, and describes why researchers have had difficulty showing that neural networks can do the same. It argues that how a model is trained may matter more than the amount of training data.

The article describes the study's setup. Researchers started with a standard transformer model that had no prior text training and taught it to interpret an invented language of nonsense words that mapped to colored dots. The model was given example sentences and then tested on new phrases. The source excerpt does not show the full results. It says the approach may create models that generalize as well as people, "at least in some instances."

Key features

• Explains compositionality and systematic generalization as concepts in human cognition
• Describes a Nature study on a training protocol for neural networks
• Details an invented language task used to train a transformer model
• Includes comments from researchers, including the study's lead author at New York University

New Training Method Helps AI Generalize like People Do - Scientific American at a glance