The Shape of Almost

The door said NATURAL LANGUAGE PROCESSING LAB and Soren read it out loud because that was the kind of thing he did.

"Natural language," Maya said. "As opposed to unnatural language."

"As opposed to math language," Soren said. "Computers don't understand words the way we do. They need everything turned into numbers first."

Maya pushed the door open. The tour group had moved on without them. Neither of them had noticed.

The lab was one long room full of monitors and the smell of old coffee. A graduate student named Priya was sitting at the far end with headphones around her neck, not on her ears, which meant she was the kind of person who wanted to seem unavailable but couldn't quite commit to it.

"Tour's gone," Priya said, not looking up.

"We know," Maya said. "What does that do?"

She was pointing at a wall display showing a three-dimensional scatter of glowing dots. Thousands of them. Each one had a tiny label too small to read from across the room.

Priya looked up. She had the expression of someone who expected a question she'd already answered a hundred times. "It's a word map," she said. "Every word is a point in space. Words that show up together a lot end up close together."

"How close?" Soren asked.

"Mathematically close. Like, distance in space."

Soren walked toward the display. Maya was already ahead of him.

The dots formed shapes, almost like clouds. One cluster glowed warmly together: words that must have had something to do with cooking, or family, or somewhere with both. Another cluster was sharper, more angular in the way the dots arranged themselves.

"So the computer read a lot of sentences," Soren said slowly. "And it put words near each other if they tended to appear in the same kinds of sentences."

"Billions of sentences," Priya said. "It never got told what any word meant. It just learned from company."

Maya had her face six inches from the screen. "They learned what they were by what they were next to."

Priya looked at her for a moment. "That's actually a good way to put it."

"Okay but," Maya said, and then stopped. She was doing the thing where she'd gotten somewhere before she knew how she got there. "If they're points in space, you can do math on them."

"You can," Priya said. She turned back to her own screen, pulled up a text window, typed something. "Here. Watch."

The big display shifted. Three dots lit up in blue. Arrows connected them.

The first label read: king.

The second label read: man.

The third label read: woman.

Priya typed something else.

A fourth dot lit up, in a different color, pulsing.

Queen.

Soren stared at it. Then he said, very quietly: "You did math."

"King minus man plus woman," Priya said. "You get a vector. You look for the nearest word to that vector in the whole space. It's queen. Every time."

"But you never told it," Maya said. "You never told it king and queen go together."

"Never. It figured out there was a direction. A royalty direction. And a separate direction that was about being a man versus a woman. It learned that those were different directions, pointing different ways through space. So when you do the arithmetic, you move along both directions, and you land somewhere real."

Soren pulled out his notebook. Not to write anything yet. Just to hold it.

"What other directions are there?" he asked.

"Thousands," Priya said. "Probably more than we've found. Big and small. Hot and cold. Happy and sad. Past and present. We don't design them. We don't label them. The model builds them by itself and we go in afterward and try to discover what it made."

Soren wrote down: directions we didn't design.

Maya had gone very still, which was unusual for Maya. She was looking at the cloud of dots with an expression Soren recognized. It was the expression she got when her head was going faster than she could follow.

"Ask it something weird," she said.

"Like what?"

"Ask it: Paris minus France plus Japan."

Priya raised her eyebrows. She typed.

The display moved. Arrows. Arithmetic. A new dot pulsed.

Tokyo.

Maya made a sound that was not quite a word.

"Capital of," Priya said. "The model learned there's a direction that means capital of. It doesn't know what a country is. It doesn't know geography. It just learned that certain words showed up in similar positions relative to other words, millions of times, and built a direction from that pattern alone."

"It found a direction," Soren said, "that humans use for capital cities. Without being told capitals exist."

"Right."

"So what else did it find," Maya said, "that we haven't discovered yet."

Priya was quiet for a moment. "That's actually the open question."

"What does that mean?"

"It means we don't know. The model is in a space with hundreds of dimensions. We can visualize three of them. Maybe four if we're clever. It built structure in all of them. We keep looking for what's in there and we keep finding things we didn't expect and we still don't know if we've found everything or almost nothing."

Soren looked at the scatter of glowing dots. Thousands of words floating in a space that was larger than any room, larger than anything you could point at. Every word a location. Every relationship a direction. Directions humans had never named, found by a machine that had never been told what meaning was.

Maya pressed her finger against the screen, lightly, against the pulsing dot labeled queen.

"So somewhere in there," she said, "there might be a direction that no person has ever noticed."

"Yes," said Priya.

"A direction that means something. Something real. About how words work. About how thinking works. Just sitting there."

Priya didn't answer right away. She looked at the display the way you look at something you've been sitting next to for so long you stopped seeing it.

"Yes," she said again, more quietly this time.

Maya's finger was still on the screen, the dot glowing behind it.