The Circuit That Pointed Away

The language model was wrong in a very neat way.

On the glass screen, the sentence glowed blue.

When Ava and Ben went to the observatory, Ava gave the meteorite to Ben. Who has the meteorite?

The model answered: Ava.

Maya stared at it.

The facilitator made a small sound through her nose. She wore three visitor badges because she kept forgetting which one opened which door. Behind her, a printer was making the noise of a trapped bird.

“It was working this morning,” the facilitator said. “Please don’t worry about that station. Try the arithmetic model instead. It can add numbers in circles.”

Maya did not move.

“It picked the giver,” she said.

“Yes,” said the facilitator, already looking at the printer. “Very embarrassing. I’ll reboot it in a minute.”

The screen had a button labeled SHOW CIRCUIT. Maya pressed it before the facilitator could say not to.

A map opened across the table. It was not a picture of a brain. It was boxes and threads and tiny lamps, all arranged like a city seen at night from very high above. Some lamps were labeled with words that sounded like parts of a machine and parts of a grammar lesson at the same time.

Duplicate Name Heads.

Subject Blockers.

Name Movers.

Answer Logits.

At the bottom, a note said: Small transformer model. Indirect object identification circuit. Parts adapted from published mechanistic interpretability research. Not all behavior is understood.

Maya liked the last sentence best.

The printer shrieked. The facilitator hurried away and said, “Only the green buttons, please.”

There were seven green buttons.

Maya pressed RUN AGAIN.

The model answered Ava.

Wrong, but not random. Not meteorite. Not observatory. Ava.

Maya changed the sentence.

When Ava and Ben went to the observatory, Ben gave the meteorite to Ava. Who has the meteorite?

The model answered Ben.

“Still the giver,” Maya said.

A little amber lamp blinked under Subject Blockers, then went dark.

Maya put her finger on the glass above it. “You tried.”

The arithmetic station across the room was counting colored beads in a loop. A display over it said MODULAR ADDITION CIRCUIT. The beads went zero, one, two, three, four, zero again. Another wall showed FACT ASSOCIATION FEATURES, with examples like country and capital, inventor and invention, bird and wing. The museum had put all of them in separate boxes, but the same kind of thing was hiding inside each one: small parts doing small jobs, then passing the job along.

Maya turned back to the wrong sentence machine.

If she changed the gift, the model still chose the giver. If she changed the place, still the giver. If she used nonsense words, still the giver.

When Niva and Pel went to the zindle, Niva gave the glim to Pel. Who has the glim?

Niva.

Maya bared her teeth, not exactly smiling.

“So you don’t care what the thing is,” she said. “You care where the names are.”

One green button said PATCH FROM CLEAN RUN.

Under it, two text boxes waited. The first box was labeled BROKEN. The second was labeled CLEAN.

Maya read the tiny instructions twice, fast.

Run two sentences. Copy one piece of internal activity from the clean run into the broken run. If the answer changes, that piece matters.

It felt rude, in the best possible way. Like borrowing one heartbeat from a different animal to see which paw moved.

Maya typed the same sentence into both boxes, then changed the clean one so the machine should answer Ben. The broken one already should answer Ben, but did not.

Broken: Ava gave the meteorite to Ben.

Clean: Ava gave the meteorite to Ben.

That would not help. She frowned and changed the clean run to a sentence the model had gotten right yesterday, from the examples drawer.

When Mary and John went to the store, John gave a drink to Mary. Who has the drink?

Clean answer: Mary.

The circuit map brightened. Threads lit in order, not like a thought, more like a marble run.

Duplicate Name Heads flashed first, touching Mary at the beginning and Mary near the end. Subject Blockers glowed next, thin and stubborn. Name Movers lit last, carrying Mary toward the answer.

In the broken run, Duplicate Name Heads flashed. Name Movers flashed. Subject Blockers only flickered.

Maya patched the gift-word lamps from the clean run into the broken one.

Ava.

She patched the place-word lamps.

Ava.

She patched a Name Mover.

The answer hesitated, then still came out Ava, but the confidence bar dipped like a knee bending.

“Oh,” Maya said.

She patched Subject Blocker One.

Ava.

Subject Blocker Two.

Ava.

Subject Blocker Three.

Ben.

The answer appeared so quietly that Maya almost missed it.

Ben.

She pulled the patch out. Ava.

Put it back. Ben.

Out. Ava.

In. Ben.

The circuit was not choosing Ben by loving Ben. It was stopping Ava from being too easy.

On the map, Subject Blocker Three did not look important. It was a narrow thread with a dull label. It pointed away from the answer first. It touched the name that should not win.

Maya ran another sentence.

When Lina and Omar entered the greenhouse, Lina handed the seedcase to Omar. Who has the seedcase?

Omar should win. Lina was too loud. Lina came first. Lina acted. Lina had all the reasons a careless machine might grab.

The broken model answered Lina.

Maya patched Subject Blocker Three.

Omar.

She tried more. Names with two syllables. Names with one. Sentences with telescopes, sandwiches, fossils, clocks. The patch did the same small brave thing each time. It did not know gifts. It did not know kindness. It did not know people. It helped the model not grab the wrong name just because the wrong name shouted first.

Maya looked at the map again.

The room did not get quieter, but the sounds moved farther away. The printer. The bead counter. The air vents. The museum voice announcing the next star show. All of it seemed to sit on the skin of something deeper.

Inside a box that finished sentences, researchers had found a piece that noticed repeated names, a piece that moved the right name forward, and a piece that pushed the wrong name down. Not magic. Not a tiny person. A set of learned tricks, sharp enough to be named.

The facilitator came back holding a ribbon of crumpled paper.

“Still broken?” she asked.

“No,” Maya said.

The facilitator glanced at the screen. “Oh. Did the reset finally work?”

“I didn’t reset it.”

The facilitator leaned closer. Her badges clacked together. “You patched a blocker?”

“Number three,” Maya said. “It was barely lighting.”

“That head usually looks boring,” the facilitator said.

“It isn’t.”

The facilitator opened her mouth, then shut it. She pressed a small square on the side of the table. A drawer slid out with clear plastic cables, each one tipped with a green light.

“Visitors are not supposed to use the open patch bay,” she said. “But the printer ate the permission forms.”

Maya waited.

The facilitator smiled crookedly. “Green sockets only.”