Imagine AI that can read minds – not in a spooky sci-fi way, but by understanding beliefs and perspectives just like humans do. It's a fascinating frontier, but the reality reveals a startling inefficiency in today's language models. Keep reading to uncover how researchers are bridging this gap, and why it might just revolutionize AI as we know it.
In their groundbreaking study, scientists from Stevens Institute of Technology revealed that large language models (LLMs) harness a surprisingly tiny fraction of their internal parameters to handle Theory-of-Mind (ToM) reasoning. This specialized subset remains active even as the models light up their entire network for every single task. Strikingly, this sparse internal wiring leans heavily on positional encoding – particularly the innovative rotary positional encoding (RoPE) – to map out and navigate beliefs, perspectives, and social dynamics.
And this is the part most people miss: humans accomplish these intricate social inferences using just a sliver of our neural resources, exposing a profound inefficiency in current AI designs. This insight paves the way for more brain-like LLMs in the future – ones that are selective, streamlined, and dramatically less power-hungry.
Let's break this down with a simple example to make it crystal clear, even for beginners. Picture yourself engrossed in a film where a character stashes a chocolate bar in a box, seals it shut, and exits the room. Meanwhile, another person in the scene quietly relocates the bar to a desk drawer. As the viewer, you're aware the treat has moved, but you also grasp that the first character, upon returning, will instinctively check the box – blissfully unaware of the switcheroo.
This intuitive grasp stems from our human knack for Theory of Mind, often called "mind-reading." It's our ability to infer and predict others' behaviors by factoring in their mental states, like knowledge, intentions, or false beliefs. Most of us hone this skill around age four, and our brains excel at it effortlessly – often in mere seconds, engaging only a handful of neurons to keep energy costs low.
Contrast that with LLMs, which, while drawing inspiration from neuroscience and cognitive science, don't fully replicate the human brain's architecture. These models operate on artificial neural networks that mimic biological neurons in broad strokes but learn from vast troves of text data through complex math. Sure, LLMs blitz through mountains of information faster than any human could, but when it comes to efficiency – especially for straightforward tasks – they fall short. No matter the query's simplicity, they fire up nearly all their neural pathways, guzzling resources unnecessarily.
But here's where it gets controversial: Is this inefficiency a flaw we should fix, or does it reveal something deeper about AI's potential to surpass human limitations in other areas? Whether you're prompting an LLM for the current time or a synopsis of a lengthy epic like Moby Dick, the model activates its full network, leading to wasteful computations where it crunches a ton of irrelevant data just to pluck out the needed bits. As Denghui Zhang, Assistant Professor in Information Systems and Analytics at the School of Business, puts it, "When we humans tackle a new challenge, we engage a small brain segment, but LLMs mobilize almost everything for even basic tasks, doing loads of extra work that's ultimately discarded."
This inefficiency spurred Zhaozhuo Xu, Assistant Professor of Computer Science at the School of Engineering, and Zhang to collaborate across disciplines. Their investigation uncovered that LLMs employ a select group of internal connections for social reasoning tasks. Moreover, this capability hinges on how the models encode word positions – especially via rotary positional encoding – which dictates attention and focus during reasoning processes.
In essence, these connections help the model form internal "beliefs" by tracking word relationships and contexts, mimicking how we humans piece together social cues. Their research, detailed in the paper "How large language models encode theory-of-mind: a study on sparse parameter patterns," published in Nature Partner Journal on Artificial Intelligence on August 28, 2025, opens doors to more efficient AI.
"AI's high energy demands make scalability a challenge," Xu notes. "By mimicking our energy-savvy brains – activating only the parameters needed for a specific job – we could slash costs and computations dramatically. It's a compelling case for rethinking AI design."
Key Facts:
- Sparse Circuits: LLMs depend on compact clusters of parameters dedicated to Theory-of-Mind tasks.
- Crucial Encoding: Rotary positional encoding plays a pivotal role in structuring beliefs and viewpoints.
- Efficiency Gap: These discoveries advocate for brain-inspired AI that engages only relevant parameters, boosting sustainability.
Source: Stevens Institute of Technology
Now, let's address some burning questions you might have:
Q: What exactly did the researchers uncover about AI's social reasoning?
A: LLMs tap into a limited set of internal links and positional patterns, especially rotary encoding, to manage Theory-of-Mind inferences.
Q: Why should we care about this for AI's overall performance?
A: Unlike our brains, which are laser-focused on subsets of neurons, LLMs burn through their entire networks for any task. Pinpointing these sparse circuits could lead to far greener, more efficient models.
Q: What's the ultimate aim for evolving LLMs?
A: To craft systems that light up only task-specific parameters, echoing human brain efficiency and cutting down on energy and computational waste.
About this AI and Theory of Mind research news
Author: Lina Zeldovich ([protected email])
Source: Stevens Institute of Technology (https://stevens.edu/)
Contact: Lina Zeldovich – Stevens Institute of Technology
Image: The image is credited to Neuroscience News
Original Research: Open access. "How large language models encode theory-of-mind: a study on sparse parameter patterns" by Zhaozhuo Xu et al. npj Artificial Intelligence (https://www.nature.com/articles/s44387-025-00031-9)
Abstract
How large language models encode theory-of-mind: a study on sparse parameter patterns
This study explores the development of Theory-of-Mind (ToM) skills in large language models (LLMs) through a mechanistic lens, emphasizing the impact of highly sparse parameter configurations.
We present an innovative technique for pinpointing ToM-related parameters, showing that altering just 0.001% of them severely hampers ToM abilities, along with contextual tracking and general language comprehension. To dissect this, we examine their ties to fundamental LLM components.
Our results indicate a strong connection to the positional encoding system, notably in models with Rotary Position Embedding (RoPE), where changes interfere with key frequency patterns essential for context.
Additionally, we demonstrate how these perturbations alter attention mechanisms by adjusting angles between queries and keys within positional encodings.
These discoveries enhance our grasp of LLMs' social reasoning, forging links between AI transparency and cognitive science.
What do you think? Should AI strive to mirror the human brain's efficiency, or could this spark debates about whether such mimicry limits innovation? Do you believe this research will lead to more ethical AI, or might it open Pandora's box for misuse? Jump into the comments and let us know your take – agree, disagree, or add your own twist!
