The Room That Answered

The club room smelled like warm plastic and floor cleaner. Everybody else had gone home. Soren stayed because Priya's program was still running, and it kept beating him.

Priya had built it for the coding fair. You typed an English sentence, and it answered back in Japanese, and a little checker at the bottom said whether the Japanese was correct. It was always correct. Soren had typed thirty sentences and it had not missed one.

"It doesn't know Japanese," Priya had said, packing up her bag an hour ago. "I don't know Japanese. I just gave it a giant table of rules. If it sees this, it writes that."

"Then how does it know what the sentences mean," Soren had asked.

"It doesn't," she said, and left.

He wrote that down. Then he sat with it, because it did not sit right.

He typed: The cat is on the roof. The program answered instantly, three neat characters and some others he couldn't read, and the checker glowed green. Correct.

He typed: I am afraid of the dark. Green. Correct.

He leaned back. The thing was, if he closed his eyes, it felt exactly like talking to somebody who knew Japanese. The answers came fast. They came right. What was the difference between answering right every single time and actually understanding?

He decided to be the program.

He opened Priya's rule file, the giant one. It was just a list. If the input contains this shape, output that shape. Thousands of lines. None of the lines said what anything meant. They only said what to swap for what.

So Soren picked a sentence he did not know how to translate and followed the rules by hand, the way the computer would. Find the pattern. Copy the matching output. He didn't read Japanese. He was just matching shapes to shapes, like sorting socks by picture.

After ten minutes he had written out a full Japanese answer on paper. He typed the same English sentence into the program to check. The program gave the same answer. He had gotten it exactly right.

And he still had no idea what he had written.

He sat very still. He had just done the thing the program does. He had produced a perfect Japanese sentence. Anyone reading it would think, this person knows Japanese. And they would be wrong. He had only moved shapes around by rule. The meaning had never once been in the room with him.

So the program didn't know Japanese either. Following the rules perfectly and understanding were not the same thing. You could have all of the first and none of the second, and from the outside they looked identical.

He wrote on the paper: the answers are right but nobody in here understands them. Not me. Not the computer.

Then the next question arrived, and it was worse.

Because how did he understand English? When Priya said the cat is on the roof, Soren understood it. He could feel a roof, a cat, the being-up-high of it. That was real. That was not shape-sorting.

But what was his brain doing when he understood? His brain was cells passing signals. One cell fired, so the next one fired. This shape in, that shape out. No single cell in his head knew what a cat was. A cell can't know a cat. It just passes the signal along, following its rule, like a line in Priya's table.

So his whole head was a room full of parts, and not one of the parts understood anything, and yet somehow, standing outside all of them, Soren understood the cat on the roof completely.

He pressed his hands flat on the table and breathed.

Where was the understanding, then. Not in any one cell. Not in any one rule. He had just proven, on paper, with his own hand, that following rules did not add up to understanding. And here he was, made entirely of things following rules, understanding everything.

Something understood. Something was reading this thought right now. But he couldn't point to the part that did it. He looked for it and there was no it. There was just the whole thing, humming, being him.

He typed one more sentence into Priya's program, slowly.

Do you understand me.

The program answered in Japanese. The checker glowed green. Correct. Perfectly correct. And it meant nothing at all to the machine, and Soren knew that for certain now, because he had been the machine an hour ago and felt the nothing himself.

But he could not prove the same thing about himself. He could not get outside his own head and check the rules and say, see, nobody home, just shapes. From the inside, somebody was very much home.

He wrote it down while his hand was still sure of the letters. The program is a room where nobody understands and the answers come out right. My head is a room where nobody understands and I come out right. He stopped. He read it twice. He could not find the line where the second room turned into a person and the first one didn't. He knew there was a difference. He had just felt the difference. He could not say what it was made of.

The screen glowed. Green. Correct. The cursor blinked, waiting, patient as anything, ready to answer him again and again forever and never once know it was answering.

Soren looked at the blinking cursor for a long time. Then he looked at his own hand, the one holding the pencil, the hand that had followed the rules and understood nothing, attached to the person who understood.

He turned off the monitor. In the dark screen he could see the room behind him, and himself in the middle of it, a shape among shapes, understanding that.