Georgia Tech researchers developed a new neural framework that audits how different training sources influence empathy and social reasoning in large language models. The findings were scheduled for presentation at the Conference on Language Modeling in San Francisco.
Tracing AI Training Sources
Researchers at Georgia Tech have built a neural framework designed to evaluate how various training sources shape empathy and social reasoning within large language models. According to Ph.D. student Glenn Matlin, the system audits neural networks inside an LLM to estimate the influence that documents, transcripts, and other data sources have on a model's final output.
The findings suggest that a model's tone reflects its training data, with neutral sources like textbooks producing objective tones, while conversational examples and dialogue foster more empathetic outputs. Matlin stated that models learn better from interpersonal dialogue when handling social reasoning tasks, rather than relying on texts explaining emotions.
Auditing Olmo 3
The research team developed their findings through a partnership with the Allen AI Institute (Ai2), allowing them to audit the open-sourced LLM Olmo 3, which contains a training dataset of over a billion documents. Professor Mark Riedl stated that identifying which sources models value most could enable engineers to intervene in training processes if they want to modify a model's behavior.
While proprietary models like ChatGPT or Claude do not disclose their training sets, the researchers noted that their study provides an academic view into neural networks that is difficult to obtain outside of corporate environments.
Key facts and where they come from
- Georgia Tech researchers developed a neural framework to examine how training sources influence LLM empathy and social reasoning.
Georgia Tech researchers have developed a new neural framework that examines how different training sources influence empathy and social reasoning within large language models.
- The framework audits neural networks within an LLM and estimates the influence of various data sources on model outputs.
the new framework he and a Georgia Tech-led research team have developed audits neural networks with an LLM and estimates the influence documents, transcripts, and other data sources have on a model’s output.
- The research was conducted through a partnership with the Allen AI Institute to audit the open-sourced LLM Olmo 3.
The findings that Matlin, Riedl, and their collaborators have made so far came through a partnership with the Allen AI Institute (Ai2), which allowed them to audit their open-sourced LLM, Olmo 3.
