The Line That Knew

The laptop was the slow one nobody wanted, so it was Soren's for the afternoon.

The makerspace had a robotics team, and the robotics team had gone to get pizza, and the volunteer who ran the place, a man named Devesh who fixed sewing machines and 3D printers with the same screwdriver, was on the phone in the hallway. That left Soren, a folding table, and a program that was supposed to learn to read handwritten numbers.

It was not reading them.

He had fed it two hundred examples. When he tested it, it looked at a seven and said three. It looked at a four and said nine. He wrote the accuracy on the page in front of him. Sixty-one out of a hundred. Worse than guessing would have been embarrassing, and this was only a little better than guessing.

He fed it four hundred examples.

Seventy-one out of a hundred.

He wrote that down under the first number. He looked at the two numbers for a while. Then he fed it eight hundred examples, because eight hundred was double four hundred, the way four hundred was double two hundred, and he wanted to keep the same kind of step.

Seventy-nine.

Sixty-one. Seventy-one. Seventy-nine. The jumps were getting smaller. Ten, then eight. He had expected the opposite. More practice should help more, not less. That was how it worked when he practiced piano, or so his mother said, though he privately thought his piano had stopped improving around the second week.

Devesh came back in, still holding the phone against his chest.

"How's the number reader?"

"It's getting better," Soren said. "But slower. Every time I double the examples, it improves less than the time before."

"Diminishing returns," Devesh said, pleased to have a name for it. "That's normal. You'll hit a wall." He went back to his call.

Soren did not think it was a wall. A wall was a thing you crashed into. This was smoother than that. This was more like walking down a hill that kept getting less steep, but never actually stopped being a hill.

He wrote the numbers a different way. Instead of how many it got right, he wrote how many it got wrong.

Thirty-nine. Twenty-nine. Twenty-one.

He stared at those. There was something in them he could almost hold. Thirty-nine to twenty-nine was not a clean cut. But twenty-nine to twenty-one, that was close to the same fraction. Each doubling seemed to shave off roughly the same slice of the mistakes that were left. Not the same number of mistakes. The same share of them.

He did the arithmetic in the margin. If doubling the data knocked out about a quarter of the remaining errors each time, then he could guess the next one before he ran it.

Twenty-one, minus a quarter, was about sixteen.

He fed it sixteen hundred examples. He did not watch the screen while it trained. He watched his own number in the margin instead, the sixteen he had written before the machine had seen anything.

The test finished. Wrong: seventeen.

Seventeen, not sixteen. But close. Closer than a guess had any right to be.

He sat back. The pizza smell was starting to reach the table. He barely noticed it.

He had predicted the future of a thing that was learning. Not exactly. But the shape of it, the arc it would take, that he had seen coming. The machine had not surprised him. He had known roughly where it was going before it got there, because the mistakes were falling along a line, and the line kept its promise.

He reached for the notebook and wrote the sequence down the side of the page, both ways, right and wrong, and then the little curve they made when he plotted them as dots. The dots did not fall in a straight line the ordinary way. But when he squinted, when he imagined stretching the axes, they wanted to be straight. There was a straightness hiding inside the bend.

Devesh hung up.

"Hit the wall yet?"

"No," Soren said. "There isn't a wall. It just keeps needing twice as much to go the same distance."

Devesh laughed, not unkindly. "Same thing, kid. Twice as much forever means you stop."

"But it means you can tell how far you'll get," Soren said. "Before you spend it. If I know I have enough examples for one more doubling, I know almost exactly how good it'll be. I don't have to run it to find out. I already sort of ran it. In here." He tapped the page.

Devesh paused with the screwdriver halfway to a jammed printer. "Huh," he said. "That's actually true."

He went to fix the printer. He had already filed it away as a Saturday curiosity. He did not seem to feel the thing Soren was feeling, which was that this was not just true for the number reader.

Because somewhere, Soren knew, people were training machines a billion times bigger than this one. Machines that read every language, that answered questions, that had seen more sentences than a person could read in a thousand lifetimes. And those people faced the same slope he had just walked down. The same twice-as-much-for-the-same-step. Which meant, before they built the enormous thing, before they spent the mountains of computers and the oceans of text, they could do what he had just done on a slow donated laptop with a pencil.

They could look at the small versions, plot the dots, find the straightness hiding in the bend, and read off how smart the big one would be. Not hope. Not wait and see. Calculate it. The way you calculate where a thrown ball will land, except the ball was a mind, and the ground it was falling toward was understanding.

He looked at his three dots, then at the space past them where the fourth and fifth and sixth would go if he kept feeding it. He could see them already. Empty places on the page that were not empty, exactly. They had numbers waiting in them. The line knew what they were before the machine did. Before anyone did.

He drew the line lightly through where the dots would be, all the way to the edge of the page, and let the pencil run off past the last thing he had measured.