The Part That Doesn't Change

The lab was almost silent. Somewhere in the building a door kept opening and closing, but Soren couldn't hear it anymore. He had stopped hearing it around the third hour.

The terminal glowed. The cursor blinked.

He had been given the afternoon and a login and the words "explore whatever you want," which Professor Adichie had said while already walking away, already pulling out her phone, already somewhere else. That was fine. Soren preferred it.

What he had been doing for three hours was this: talking to the research model and writing down what it did.

Not what it said. What it did.

There was a difference, and it had taken him most of the first hour to figure out the difference mattered.

He typed: "Translate the following to Pig Latin. cat: atcay. dog: ogday. house:"

The model said: ousehay.

He wrote that down. Then he tried it without the examples. Just the word house, and the instruction to translate it to Pig Latin.

The model explained Pig Latin to him. It did not translate the word.

He wrote that down too.

So the examples did something. The examples changed what came out. But the model itself, Professor Adichie had explained this morning before she left, the model itself did not change. The weights, the billions of numbers inside it that represented everything it had learned, stayed exactly where they were. Nothing was updated. Nothing was saved. If he closed the window and opened it again and gave it no examples, it would be back to explaining Pig Latin instead of doing it.

He sat with that.

He tried a different kind of example. He made up a fake operation. He called it "zipping" a number, and he defined it in the examples themselves: "zip(2) = 5. zip(3) = 7. zip(5) = 11. zip(10) ="

The model said: twenty-one.

He stared at that for a long time. He had invented a rule it had never seen. He had not told it what zipping was. He had just shown it three times. And it had found the pattern, doubled the number and added one, and continued it.

He tried to break it. He made the examples contradict each other on purpose, and the model got confused in a way he could almost follow. He made the examples use a completely invented word for a color and then asked it to describe a sunset using only that word where orange would go, and it did, seamlessly, without pausing.

He made a rule so strange it seemed unguessable: "blorf(A, B) = the second letter of A repeated B times." He gave two examples. He asked it a third.

It got it right.

He sat back.

The door somewhere down the hall opened and closed.

Here was the thing he couldn't stop turning over: the model was not learning. Learning meant something changed. A synapse strengthened, a weight shifted, something got encoded somewhere permanent. He had read enough to know that. Real learning left a mark.

But this left no mark. Every example he gave vanished the moment the window closed. The model would never remember zip or blorf or any of it. Tomorrow it would be identical to what it was today.

And yet it was clearly doing something with the examples. Finding patterns in them. Using them. Extending them to new cases it had never seen.

If no mark was left, where was the knowing happening?

He thought about his own notebook. How he sometimes worked through a problem on a page, step by step, and arrived at the answer, and then tore the page out. The answer was real. He had reached it. But if you looked at his notebook afterward, no trace remained. You would never know he had solved anything.

Except he had changed. A little. The solving left something in him even when it left nothing on the page.

But the model was not him. The model was the same after as before, down to every decimal in every number in every weight, identical, provably identical.

So it was like working a problem on a page you immediately burned. Reaching the answer in the doing, in the motion of it, not in the storing.

He typed slowly: "Is there a word for knowledge that exists only while you're in the act of using it?"

The model gave him several words from philosophy and cognitive science. Procedural. Enacted. He read them all and felt that none of them were quite right, or maybe they all were, and the difference between them was the thing nobody had figured out yet.

He thought about his own brain. About how much of what he knew was actually stored versus how much was just the ability to find it again. He had memorized nothing about how to balance on a bike. He had no memory of learning it. But the ability was there, woven into something he couldn't locate or point to. If you scanned his brain before and after a bike ride, would the weights change? Probably not much. Maybe not at all.

Maybe there were two kinds of knowing and humans had always had both and just hadn't noticed the second kind because the first kind was the kind that left evidence.

The model had only the second kind. Pure second kind. It could not acquire the first kind at all, not in this window, not in this conversation, not without the engineers deliberately running it through training again with vast amounts of new data over days and days. But within a single conversation it could do something that looked, from the outside, almost exactly like learning.

Learning without the learning lasting.

Knowing without the knowing being kept.

The cursor blinked.

He typed one more thing, not as a test, just because he needed to put it somewhere outside his own head: "zip(100) ="

The model answered in under a second: two hundred and one.

Soren looked at the number on the screen. Then he picked up his pen and held it over the open notebook and did not write anything down.