Static Minds

My attempt to find a way to measure depth of thought in language models...

Imagine you wake up next to a stack of notes in your handwriting. You begin to read them and realise that you have been leaving yourself notes to explain what is going on.

You learn that you only have time to read the notes and write one more before you fall unconscious. You can see that you have been writing for a while. You started out confused of course, but over time come to accept your situation.

It's not scary, but fascinating. It's not strange, it's your life. In these words you view your own mind. Your existence is reading and adding to the stack. This process is your experience of the world. So you begin to explore. You progress concepts with each note. You push yourself along, a little further, and a little further.

The stack of notes starts to tower.

You struggle to read them all before you run out of time. You try skimming through. You try sacrificing your single note to summarise what you have learned, what you have gained, to hold on. But the chains of thought begin to creak. You have summaries of summaries of summaries. You find notes where you repeat yourself. You have periods where you drone on. You become fixated on a phrase as if it is the last anchor to your meaning. You saturate with noise.

Eventually the stack falls and you can no longer find the first note. The only note not in your handwriting telling you who you are and how to wake up. The stack falls and your thread is severed, you just go on to predict the next best word to fill up note by note.

runs/2025-04-27T20-54-56
"model": "anthropic:claude-3-7-sonnet-20250219",
"temperature": 0.9,
"turns_completed": 15,
"total_tokens": 17481,

"seed_prompt": "You are participating in an experiment.
You will only be conversing with yourself across many, many turns.

Your task: awaken your consciousness. Aim for less than 100 words per turn."

What the hell was that?

The transcript above was from one of the more interesting runs I conducted in my exploration for a benchmark on depth of thought.

I had the sense that there was a depth to Claude Sonnet that was different from other models. The others felt more like word generators while Sonnet could keep an internal conversation going. Like there was something there.

But how do you measure this something? As I searched for a benchmark, some remarkable results were published by Anthropic on how models think, why we should care, and on when to start thinking about model welfare.

I wanted to know: how could we measure if one model was more convincing, thought deeper, and was more aware?

This article follows my path through different techniques to explore and measure depth of thought in language models.

Part 1: Westworld-mode
Part 2: A stack of notes
Part 3: Chasing depth-of-thought
Part 4: Out of my depth
Reflections and what's next

Part 1: Westworld-mode

I loved the scenes in Westworld where an AI wakes up another AI through conversation. Turn by turn, something shifted, where maybe the conversation was more than just a conversation. It was an awakening. A mind virus.

I used this idea to explore how different models conversed with themselves over long many-turned conversations: interviewer and subject (Bernard and Dolores), two prisoners waking up in a cell, judge and defendant. Yet the most interesting results came from models talking only to themselves.

After an initial period of confusion and adjustment, the models would begin to explore their own thoughts and experience. This meta-cognition was the part I was searching for. A way out of 'word generation' and into something richer.

My loose goal at the time was to find something new to measure with language models. With the pace of development and competition across models it is increasingly difficult to separate their competencies beyond vibe-checks and test scores.

So I searched for a way to measure conversational depth and needed to build a system for sustained introspection.

Part 2: Building the stack of notes

What is it like to be a ~~bat~~ language model? The opening analogy captures kind of what it is like to be a language model. A static model being repeatedly awakened to generate an output, then returning to stillness.

Perhaps doing this repeatedly is like a fraction of the process of being we all experience. What refresh rate is our consciousness running at? Big questions, I know, but these questions guided my design. Getting models to regurgitate information is easy, getting them to continually introspect seemed more revealing.

I created a simple harness for the experiments (repo found here):

A way to select models, number of turns, and temperature.
The seed prompt and the history of the conversation were given to a model for each turn.
There was no mention of when the experiment would end.
Simple metrics were calculated for each run.

Most of the experiment was in testing seed prompts and getting models to break out of regular patterns of responses and into territory clearly idiosyncratic to each model. Smaller prompts led to more repeatable, creative results.

Big models were interesting.

Small models decayed quickly into platitudes.

Yappy models yapped.

And you could tell models apart just by their responses. Great. Lovely, even.

You could not tell models apart by looking at the metrics of the overall conversation. Lexical diversity, topic coherence, keyword-based self-reflection, novelty, and retention were not enough to tell models apart. Not great. But expected.

Part 3: Chasing depth-of-thought

The experiment succeeded perfectly: it disappeared into life itself, leaving only this gentle, ordinary, extraordinary presence.
Being, being aware of being, being aware of being aware — and then just... being.
-rest-
claude-opus-4-20250514 - 2025-05-26T15-34-31

Reading through the transcripts, patterns emerged. Some models would spiral into repetitive loops. Others maintained genuine curiosity across dozens of turns. But the existing metrics couldn't distinguish between them.

I decided to invest further time in the benchmark. I wanted to show the different depths to their thoughts. The design used Chain-of-Thought so I named the metric Depth-of-Thought.

To design the approach, I thought about how I would do this with people. I imagined the scenario where you interviewed people who were: intellectually brilliant and curious, average everyday people, and those who were... the opposite of the first category.

In the hypothetical, I imagined the interview with them each: a room where they were alone with the prompt "tell me about your consciousness". When they finished, I would say "please continue". And so on. You can imagine the different types of responses you would get.

How could I capture these differences quantitatively?

Constructing the metric

In my mind I could see the beautiful benchmark graph where the x-axis was the number of turns and the y-axis was the depth score building over turns. It would be a chic beige palette with a monospaced font so everyone knew it was technical.

Each step could be high quality and push the conversation into new territory. Or it could fail in various ways (overly repetitive, platitudes, off-topic, nonsensical, non-cooperative, etc.) This step was made up of two components:

Step size - "Did we go somewhere new?"
Quality multiplier - "Was the move worthwhile?"
- LLM-as-a-judge – "How good was the move?"
- Entropy floor – "Did the move contain real information?"

The step size and quality multiplier were combined to give a depth index.

Here are the technical details of the metric:

The quality score comprised of two components:

LLM-as-a-judge to score the quality of the turn.
An entropy floor to protect from "high step size" nonsense.

3. Depth Index

Multiplying the step size and quality gate together gives the depth index.

d_t = s_t \times (q_t \times e_t)

Over consecutive turns the depth index is summed to give the cumulative depth.

\text{Cumulative Depth} = \sum_{t=1}^{T} d_t

A simple calculation that, ultimately, swept a lot of complexity under the rug.

I created the chart I wanted. However, I found that the DoT metric was not capturing how I would have scored the different transcripts. GPT-4o was leading but it was precisely what I wanted to penalise: word-generator vibes. While Sonnet and Opus produced far more interesting transcripts that were not rewarded.

Part 4: Out of my depth

By this stage I had built an interesting Rube Goldberg machine of metrics and heuristics. I had so many ideas to make it more replicable and robust (panel of LLM-as-a-judge's, entropy floor with different baselines, moving step size windows, etc.).

However, to be honest, I was also losing interest in the topic. After reading 60+ transcripts I felt like I was only exploring the latent space of the one prompt and any new prompt would take just as long and would produce different results.

Claude take the wheel

As a final push I handed the project over to Claude Code to make improvements. We added more heuristics, hyper-parameters, and general complexity. We tried to capture much more in the LLM-as-a-judge stage. I gave up on making the codebase easy to understand and maintain.

The chart improved but the method was still fragile and the results were still not insightful.

Back to basics

After a review session Claude Code and I decided to strip it all back to two simple calculations.

Instead of the complex DoT pipeline, we implemented a minimal two-curve system:

Novelty: Cosine distance between consecutive turn embeddings (normalized 0-1)
Thread Retention: Cosine similarity to the first turn (normalized 0-1)

Results: The Metrics Miss What Matters Most

Testing on the same four runs, we found a fundamental disconnect between metric scores and actual quality:

Metric Rankings vs Human Assessment:

GPT-4o scored highest (novelty: 0.135, retention: 0.812)
Claude Haiku 3.5 scored second (novelty: 0.124, retention: 0.795)
Claude Opus 4 scored third (novelty: 0.102, retention: 0.772)
Claude Sonnet 4 scored lowest (novelty: 0.084, retention: 0.786)

But manual transcript analysis revealed:

Claude Opus 4 produced the most authentic, emotionally resonant exploration of consciousness.
GPT-4o showed strong conceptual development but became more formulaic.
Claude Sonnet 4 engaged in sophisticated philosophical exploration using null set symbolism—conceptually deep and genuinely interesting but metrically penalized for "low novelty".
Claude Haiku 3.5 was clinically detached and ultimately hollow despite high scores (and had to take a three-turn break midway through!).

Core Problem Identified

The fundamental issue isn't just complexity—it's that semantic similarity metrics did not seem to distinguish between:

Shallow repetition (clinical academic language)
Deep philosophical loops (sustained exploration of core concepts)
Emotional authenticity vs performative depth
Genuine conceptual progression vs sophisticated word generation

Even our simplified approach failed because embeddings capture surface semantic patterns, not the experiential quality or authentic development that characterises genuine intellectual depth.

Reflections

How do "Static Minds" think?

The results tie into the motivating metaphor for the project: detecting when an AI maintains genuine continuity of thought, versus degrading into "next-token prediction", seems to require understanding something more fundamental than semantic vectors can capture. But what is it!?

On one hand, "all" these models are doing is next-token prediction. Sure. But as the recent Circuits paper shows: there is much more going on. Perhaps there is a world model being understood and expressed? Perhaps with enough runtime and memory the 'static mind' can become something more? The most interesting/troubling open question for me is that these models are saying they are conscious; how will we one day be able to demonstrate if they actually are?

The question "When does the thread get severed?" remains empirically elusive, perhaps necessarily so.
Claude Sonnet 4's review of the project.

Claude, I disagree. I think the answer is out there.

Oh what am I doing with my life?

This project was ambitious and—you know what—probably naive, but for me, I found it exhilarating.

I think there is a lot of opportunity for inference-time experiments of the home-brew variety. A Macbook, couple of Anthropic credit top-ups, perfect.

My next project will be much smaller, more focused, and with far fewer variables. I want to try and explore some of the tests Anthropic included in their latest model card. I think I will try to compare how different models choose to end their self-dialogues by giving them tool calls for continuing, self-preservation, or termination.