The Word That Points

The chatbot on the screen said: I do not understand which one you mean.

Maya read the sentence they had typed into it. The dog chased the cat until it climbed a tree.

"Ask it who climbed the tree," said Soren.

Maya typed the question. The bot answered: The dog climbed the tree.

"Wrong," she said. "Dogs don't climb trees. It's the cat."

"It knows that, though," Soren said. "It knew what a tree was in the last answer. It's not dumb about trees. It's dumb about the word it."

The makerspace was empty except for them and a humming fridge. Priya, the volunteer who ran the coding nights, had left the model running and a sticky note that said back in twenty, which had been three hours ago. So it was just the two of them and a small language model on a laptop that ran hot enough to warm your hands.

"Okay," Maya said. "New sentence." She typed: The cup fell off the table but it did not break.

Ask it what did not break.

The bot said: The cup did not break.

"Right that time," said Soren.

"So it's not broken. It gets it sometimes." Maya leaned back. "What's different?"

Soren was already writing in his notebook, the pen scratching. He drew the cup sentence, then the dog sentence, then drew a line from the word it back to cup, and another line from it back to cat.

"In grammar class," he said, "they'd tell you it points to the nearest noun. Or the subject. Some rule."

"Table's nearer than cup," Maya said. "In the cup one."

"Yeah." Soren tapped the page. "If it followed the nearest-noun rule, it would've said the table didn't break. It didn't say that. So it's not using that rule."

Maya sat up. "Try to break it. On purpose."

That was the good part. She typed: The trophy would not fit in the suitcase because it was too big.

Ask what was too big.

The bot said: The trophy was too big.

Then Maya changed one word. Not the grammar. One word. The trophy would not fit in the suitcase because it was too small.

Ask what was too small.

The bot said: The suitcase was too small.

Neither of them said anything for a second. The fridge hummed.

"It switched," Maya said slowly. "Same sentence. Same it in the same spot. It pointed at the trophy the first time and the suitcase the second time."

"Because big and small," Soren said. "That's the only thing that changed. Big means the trophy's the problem. Small means the suitcase is the problem. It's not counting nouns. It's not looking at where it is in the line."

"It's looking at what makes sense," Maya said.

"But how," Soren said, and it wasn't a complaint, it was the real question. "Nobody typed a rule that says trophies are big. How does it weigh it?"

Maya grabbed the notebook without asking and Soren let her. She wrote the sentence out with every word in a row. The. Trophy. Would. Not. Fit. In. The. Suitcase. Because. It. Was. Too. Small.

Then she drew a line from the word it to every other word. Not one line. All of them.

"That's a mess," said Soren.

"No, wait." She started making some lines thick and some thin. "What if it's not one arrow. What if the word it is looking at every single word at the same time and deciding how much each one matters."

Soren went quiet, then leaned over the page. "So small pulls hard. Small is a heavy line. And suitcase gets heavy because small makes it matter. And trophy goes thin."

"And when we typed big instead," Maya said, "all the weights flipped. Trophy went heavy. Suitcase went thin. Same word. Different pull."

"Every word measuring every word," Soren said. "All at once."

Maya typed one more, testing her own idea before she believed it. She wrote: The city councilmen refused the marchers a permit because they feared violence.

Ask who feared violence.

The bot said: The councilmen feared violence.

Then she changed feared to advocated. Because they advocated violence.

Ask who advocated violence.

The bot said: The marchers advocated violence.

"They switched too," Maya breathed. "Same they. It moved from the councilmen to the marchers. Just because one verb changed."

Soren stared at the screen. "Nobody taught it councilmen fear and marchers advocate. It learned which words lean toward which other words. From reading. It weighs the whole sentence to figure out where one little word is pointing."

"That's what we do," Maya said. "That's exactly what we do. When someone says it, we don't count nouns. We just, know. Because of everything else in the sentence."

"I always thought I was slow at that," Soren said. "In class. When we did pronouns and antecedents. Everyone else went fast and I had to read the whole sentence twice."

Maya looked at him. "You were reading the whole sentence. That's the thing that works. Reading the whole sentence is the thing that works."

Soren looked back at the screen, where the small hot laptop sat there holding a machine that had never been given a single grammar rule and had figured out, from nothing but a mountain of words, how to feel where a sentence leaned.

"Type another," he said. "Change one word. I want to see how far it goes."

Maya typed. The scientist told the student that she had won the prize.

Ask who won the prize.

The bot said: This one is genuinely ambiguous. It could be the scientist or the student.

They both went still.

"It said it doesn't know," Maya whispered.

"No," Soren said. "It said the sentence doesn't know. There's no heavy line to draw. Nothing to lean on." He picked up his pen. "We couldn't answer that one either."

Maya put her finger on the she on the screen, that one small word sitting between two people it could belong to, waiting.

"Change one word," Soren said again, quieter.

Maya's hands went to the keys. Behind them, the fridge cut out, and in the sudden quiet the fan of the laptop spun up, working through every word at once.