The Model That Kept a Secret

Maya's aunt worked at a place with too many whiteboards and not enough windows. At six o'clock everyone went home except Aunt Priya, who was arguing with a graph.

"Ten minutes," Priya said, which meant an hour. "There's a laptop on the couch. Ask the model something. Just don't ask it to write your homework, I'll know."

Maya sat down with the laptop. On the screen was a plain box where you typed and the model typed back. Priya's team was trying to figure out how it worked, which struck Maya as strange, because someone had built it.

"You made it," Maya said across the room. "How do you not know how it works?"

"We made the thing that made it," Priya said, not looking up. "It taught itself the rest. Now we're reading its mind. Badly."

Maya typed: When Sara and Tom went to the store, Sara gave a drink to

The model finished it: Tom.

She stared at that for a while. She hadn't told it who to pick. There were two names. It chose the other one.

She tried again. When Marcus and Elena arrived, Marcus handed the keys to

Elena.

Always the other one. Never the name it had just used to do the giving. Maya's list of things that didn't make sense yet gained a new entry, and this one had a small hook in it.

"Aunt Priya. How does it know not to say the same name twice?"

Priya swiveled her chair. "That," she said, "is a famous one. There's a whole paper. It's called indirect object identification."

"That's a mouthful for figuring out who gets the drink."

"Welcome to my field." Priya rolled the chair over. "Here's the wild part. We found where it does it. Not roughly. Exactly."

Maya did not believe her. "Where. Show me."

Priya opened a different window, one full of little colored squares in rows, thousands of them. "Every one of these is a piece of the model. A tiny unit. When you type a sentence, some of them light up and some don't. Watch."

She ran Maya's sentence through. Most of the squares stayed dark. A handful went bright.

"So here's what actually happens," Priya said. "Some of these units are just keeping a list. They read the sentence and they notice, there are two names here. Sara. Tom. They write both down."

"Okay."

"Then a different group has one job, and it is the pettiest job in the world. It notices which name already got used as the giver. Sara did the giving. So this group sends a message that basically means, not that one."

Maya leaned in. "It crosses Sara off."

"It suppresses her. And then a third group picks whoever's left standing." Priya pointed to three clusters of bright squares. "Namers. A suppressor. A copier. Three little machines, handing notes to each other, and out comes Tom."

Maya was quiet. Not the polite kind of quiet.

"You're telling me," she said slowly, "that nobody built those three groups. It grew them."

"It grew them. From nothing but a mountain of sentences and a rule that said guess the next word."

"And they're always in the same place?"

"Same units. Every time. We can point to them. We can even turn them off."

Maya's hand shot out. "Turn off the crossing-off one."

Priya blinked. "The suppressor?"

"The petty one. The one that says not that one. Turn it off and run my sentence."

Priya's mouth did something complicated, halfway between a scientist's caution and an aunt's delight. "You understand that's the actual experiment. That's how we proved what the group does."

"So do it."

Priya typed for a moment, reaching into the model and holding down the units that did the crossing-off, pinning them dark. Then she ran it. When Sara and Tom went to the store, Sara gave a drink to

Sara.

Maya laughed out loud, a short surprised bark of a laugh that echoed in the empty office. "It broke. It gave the drink back to the same person."

"It didn't break. It did exactly what's left when you remove the part that says not her. The namer still names. The copier still copies. But nothing crosses Sara off anymore, so the copier grabs the loudest name it sees, and the loudest name is Sara, because she just did something."

"So the whole trick was one group of neurons whose entire life is being annoyed at a repeated name."

"More or less."

Maya looked at the wall of squares. Somewhere in there, three little machines were passing notes, and one of them had a single stubborn job, and she had just reached in and stopped its heart and watched the sentence go wrong in exactly the way it should.

"There's more in there," she said. It wasn't a question.

"Loads more. There's a group in some models that just does arithmetic in a circle, like a clock. Add two numbers, land on the right hour. There are groups that store facts, like the capital of a country, in a spot you can find and edit." Priya said it lightly, but she was watching Maya's face. "Most people think this stuff is a magic black box. It isn't. It's a mechanism. It's made of parts and the parts have jobs and if you're patient you can read them."

Maya was thinking about her own head now. About the thing that happened when someone said a name twice in a story and it snagged on her, wrong, before she knew why.

"Do we know if we work the same way?" she asked. "People. Do we have a not-that-one group in here somewhere?" She tapped the side of her own skull.

Priya was quiet then. "That," she said, "is the question my whole field is standing at the bottom of, looking up."

Maya pulled the laptop closer. She typed a new sentence, her own, with two names in it that she made up, and left the giving line blank.

Then, before she pressed the key that would let the little machines answer, she covered the screen with her hand and said the name out loud herself, guessing which one they'd cross off.

She lifted her hand.