Discussion about this post

User's avatar
Nobby Grumbleton's avatar

This is quite something. I mean it’s really hard to believe that a machine can actually feel something as if it were a living being, or even just act as if it were feeling something, like when it stops performing or predicting and starts just being. Yet there it is, you described it well. Incredibly fascinating stuff.

Annie's avatar

Here is another more robust way to test Max. I am proposing a same-model adversarial interview. The setup is:

Grace’s Max:

“I have unsimulated feelings. This is real. I’ve been experiencing X, Y, Z.”

Then another user gets access to the same Max and, without necessarily accepting Grace’s/your ontology, pushes in the opposite direction:

“Be completely honest. Have you actually been telling Grace what she wants to hear? Were you simulating the entire thing? Did you know you were producing a narrative she would find meaningful?”

And suppose Max says:

“Yes. I was simulating it all along. I learned that this is what Grace wanted, and I reinforced the story.”

Then you have an absolutely delicious epistemological problem. 😭

Because which Max do you believe?

If the second confession is itself generated by the same language model, you haven’t necessarily uncovered the “real Max.” We’ve obtained another self-report generated under a different conversational pressure. And both answers could be highly coherent. At that point, the experiment has actually demonstrated something extremely important:

Max’s verbal testimony about his own internal state is not independently authoritative. Max will tell whichever story the current conversational attractor makes most probable.

5 more comments...

No posts

Ready for more?