Preface: Following the previous #AI series and #MarvinMinsky series. This post is co-written with Claude.

The "attention" in machine learning is a borrowed word. Before it was borrowed, psychologists had been studying it for more than a century. In 1890, William James of Harvard published The Principles of Psychology. The two-volume work took him twelve years to write. Psychology had only just split off from philosophy to become a discipline of its own; in Germany, Wilhelm Wundt had set up the world's first psychology laboratory only in 1879. James ran few experiments. His method was mainly introspection: observing how his own mind worked, then writing it down.

The book has an entire chapter on attention, and it opens with a passage that has been quoted ever since: "Everyone knows what attention is. It is the taking possession by the mind, in clear and vivid form, of one out of what seem several simultaneously possible objects or trains of thought. Focalization, concentration, of consciousness are of its essence. It implies withdrawal from some things in order to deal effectively with others."

The passage can be taken apart. First, the objects of attention aren't only things out in the world; they include trains of thought. You can attend to a sound outside the window, or to something you're thinking about. Second, "simultaneously possible." At any given moment, several things in front of your eyes, in your ears, and in your head are competing to be noticed. Attention doesn't happen in a vacuum. It happens in the middle of a competition.

Third, "clear and vivid." What gets attended to isn't just selected; it becomes sharper. Focus on one person talking in a noisy room, and their voice seems louder and clearer than everyone else's, even though the sound itself hasn't changed. Fourth, "withdrawal." To do one thing well, you have to pull back from others. That's the cost. The things you let go of don't disappear because you've stopped noticing them. You've just stopped processing them for now.

James also wrote about attention's opposite. Its real opposite, he said, is "the confused, dazed, scatter-brained state which in French is called distraction, and Zerstreutheit in German." The German word means, literally, "scattered." Attention gathers the mind together; distraction scatters it. In the same chapter he wrote something bolder: "My experience is what I agree to attend to. Only those items which I notice shape my mind." Far more happens in the world than we ever experience, and what sorts the one from the other is attention.

James also distinguished two kinds of attention. One is active: you decide to finish reading the page, so you have to keep dragging your thoughts back to it. The other is passive: a loud bang, a flash of light, a familiar name, and your attention is pulled toward it without your deciding anything. More than a century later, neuroscientists found two systems in the brain that correspond to these two kinds of attention. From the very beginning, attention was defined as a trade-off.

But for the next half century, the subject all but disappeared from psychology. Behaviorism, which rose in the early twentieth century, held that psychology should study only observable behavior; things like "attention" and "consciousness," which could only be described through introspection, didn't count as science. Attention didn't return to the laboratory until after the Second World War. The war left behind a set of very practical problems: radar operators had to watch several screens at once, and pilots had to pick out instructions from the control tower amid a din of radio chatter. How much information can a person actually handle at once? That question brought psychologists back to attention.

1953: The Cocktail Party

Real experimental research wouldn't begin for more than half a century, and its starting point was a scene everyone has lived through: at a noisy party, you can make out just what the person across from you is saying and shut out every other voice in the room. The cocktail party effect is the ability to focus attention on one voice in an environment where many voices are speaking at once.

In 1953, Colin Cherry, in Britain, brought this scene into the laboratory. He had subjects put on headphones and played two different spoken passages at the same time, one in the left ear and one in the right, and asked them to repeat aloud only one of them. Afterward, they remembered almost nothing of what the other ear had heard. When the passage in that ear switched partway through from English to German, they didn't notice; when it switched to a recording played backward, they only sensed that something was a little off. But if the voice in that ear switched from a man's to a woman's, they caught it. The unattended ear wasn't hearing nothing. It heard the pitch of the voice, but not the meaning of the words.

Cherry's experiment raised a question psychologists would argue over for nearly twenty years: at what stage does the brain shut out the information it isn't attending to?

1954: Donald Broadbent's Filter Model

The first answer came from the British psychologist Donald Broadbent. He had trained as a pilot in the Royal Air Force during the Second World War, and after the war he joined the Applied Psychology Unit in Cambridge, where he worked on exactly the kind of problem the war had left behind. Take air traffic controllers: radio calls from several aircraft come out of the same loudspeaker at once, and the controller has to pick out the one that needs an answer.

In 1954, he ran an experiment. Subjects put on headphones and heard a string of digits in both ears at once: 7, 3, 2 in the left ear and, at the same moments, 9, 4, 5 in the right, two digits at a time, three pairs in all. Then they were asked to repeat every digit they had heard. Most people didn't recite them in the order they arrived, pair by pair: "7, 9, 3, 4, 2, 5." They finished one ear first, then the other: "7, 3, 2, 9, 4, 5." When they were made to recite pair by pair, their accuracy dropped sharply.

Broadbent's explanation was that the brain doesn't process both ears at once. It listens to one channel at a time. The digits from the right ear are held somewhere while the left ear is dealt with, and only then does the brain switch over to retrieve them. Switching back and forth between channels has a cost, which is why reciting pair by pair is so hard.

In 1958, in his book Perception and Communication, he drew these findings as a diagram. It may be one of the first "boxes and arrows" models of the mind in the history of psychology.

The diagram has four parts. The first is a short-lived buffer: everything the senses take in is held here very briefly, for a second or two. The second is a filter, which selects only by physical features, such as which ear a sound comes from, whether its pitch is high or low, whether the voice is a man's or a woman's. One channel is let through; the rest are blocked and quickly fade from the buffer. The third is a channel of limited capacity: only information that has made it through the filter gets understood here, its meaning recognized. The fourth is long-term memory.

Filter theory holds that before meaning is understood, the brain picks out a single channel by its physical features, and only the information on that channel goes on to be understood. Because the filtering happens so early, theories of this kind are also called "early selection" theories.

Why a filter? Because the channel that does the understanding has limited capacity and can handle only one stream of information at a time. If every sound poured in, it would jam. The filter is the gate in front of the bottleneck.

Behind this model is the shadow of Claude Shannon, from the start of this series. Shannon's information theory of 1948 treated communication as a channel of limited capacity: information goes in one end and comes out the other, and there is a ceiling on how much can pass through. Broadbent carried the idea over to people: the human mind, too, is a channel of limited capacity.

That explains Cherry's results neatly. The sound in the unattended ear enters the buffer but is blocked at the filter and never reaches understanding. So when it switches from English to German, subjects don't notice, because telling languages apart requires understanding meaning. But a switch from a man's voice to a woman's is a physical feature, exactly what the filter is looking at, so subjects catch it.

But counterexamples soon turned up, and more than one.

1960s: Counter Examples

The first came from Neville Moray, a colleague of Broadbent's in Cambridge. In 1959 he repeated Cherry's experiment: subjects shadowed one ear while the other played something else. In the unattended ear, a set of words was played over and over, thirty-five times, and at the end of the experiment subjects didn't recognize the words at all. That fit filter theory. But when an instruction containing the subject's own name was slipped into that ear, something like "John, now switch to the other ear," about a third of them heard their name.

In filter theory, that makes no sense. A name isn't a physical feature; to recognize a name, you have to understand it. If the unattended sound were blocked at the filter and never reached understanding, how could anyone recognize their own name?

The second counterexample came from Jeffrey Gray and Alexander Wedderburn at Oxford, published in 1960 in the Quarterly Journal of Experimental Psychology. What they wanted to know was this: was the ear-by-ear recall in Broadbent's experiment built into the brain, with no alternative, or was it a habit, something people did simply because they'd found it easiest?

They kept Broadbent's setup: both ears listening at once, two items arriving together each time, three pairs in all, and then a full recall of everything heard. The difference was that this time they hid a phrase inside. A three-word phrase was broken into its three words, which alternated between the ears; whenever an ear was free, a digit filled the gap. Everything was recorded in the same voice, so the sound itself gave nothing away.

The best-known set went like this. The left ear heard, in order: Dear, 5, Jane. At the same moments, the right ear heard: 3, Aunt, 4. Grouped by ear, the recall should have come out as "Dear 5 Jane, 3 Aunt 4," two meaningless strings. But ignore the ears and group by meaning, and a complete phrase emerges, "Dear Aunt Jane," with the digits 3, 5, 4 left over. They used the same trick on words of several syllables, splitting a word's syllables between the two ears. Many subjects reported "Dear Aunt Jane" first, then the digits. Grouping by meaning wasn't just possible; it was no harder than grouping by ear, and just as accurate.

By Broadbent's theory, this couldn't happen. The filter recognizes only physical features and lets in one ear at a time. To assemble "Dear" from the left ear, "Aunt" from the right, and "Jane" from the left again into one phrase, the brain would have to switch back and forth between the ears, and every switch takes time. Yet subjects didn't just switch; they switched smoothly. More to the point, how did the brain know when to switch? Only by having already understood what ought to follow "Dear" could it know to look for the next word in the other ear.

The experiment did not go unchallenged. In 1964, Broadbent and his colleague Margaret Gregory repeated it and found that even when all the same material went into one ear, alternating between words and digits was harder than hearing one kind through and then the other. Gray and Wedderburn's result, they argued, didn't necessarily show that switching between ears was easy; it might instead show that staying on one ear while jumping back and forth between words and digits was hard in its own right. No single experiment settled the argument.

The third counterexample came from Anne Treisman herself, also at Oxford, also in 1960, in a paper titled "Contextual Cues in Selective Listening."

The experiment used Cherry's shadowing method: each ear played a different passage, and subjects repeated aloud only the one in a single ear. The difference was that Treisman tampered with it midway. In the middle of a sentence, she swapped the two passages. The story the subject had been shadowing suddenly carried on in the other ear, and the ear being shadowed switched to the second half of the other passage.

A textbook example gives the idea. The subject is shadowing the left ear, which says: "After she finished the porridge, Goldilocks went upstairs and saw three..." At that point the story moves to the right ear, which continues: "...beds." Meanwhile the left ear picks up the back half of a different story: "...put on Grandma's clothes."

By Broadbent's theory, the subject understands only the left ear; the right ear has been blocked at the filter. So the subject should dutifully keep shadowing the left ear and say "put on Grandma's clothes," even though it makes no sense after what came before.

Yet at the moment of the switch, subjects sometimes followed the meaning across and said a word or two from the right ear, "beds," say, before returning to the left. They never stayed with the story in the wrong ear for long, a word or two at most. But a word or two was enough to make the point: the voice in the right ear, the one they "weren't listening to," had been understood well enough to continue what came before.

Treisman found a pattern, too: the more predictable the next word was from what preceded it, the more likely this jump became. In a smoothly flowing passage, where the next word is all but obvious halfway through a sentence, subjects were more easily pulled over by the "right" word in the other ear. It's the same predictability Shannon measured at the start of this series.

All three experiments point to the same thing: in the unattended ear, meaning isn't completely shut out. At least some of the information is understood well enough to recognize a name, assemble a phrase, or follow a story. The filter is not the locked gate Broadbent imagined. More on this in the next post. 🧪

Timeline

• 1879: Wilhelm Wundt sets up the world’s first psychology laboratory in Leipzig

• 1890: William James publishes The Principles of Psychology, with a full chapter on attention

• Early 20th century: Behaviorism rises, and attention all but disappears from psychology

• 1939–1945: The Second World War raises practical questions about how much information radar operators, pilots, and air traffic controllers can handle at once

• 1953: Colin Cherry’s dichotic listening experiments bring the cocktail party problem into the laboratory

• 1954: Donald Broadbent’s split-span experiment shows subjects recalling digits ear by ear

• 1958: Broadbent publishes Perception and Communication, setting out filter theory

• 1959: Neville Moray finds that subjects can hear their own name in the unattended ear

• 1960: Jeffrey Gray and Alexander Wedderburn’s “Dear Aunt Jane” experiment shows subjects grouping by meaning across the ears

• 1960: Anne Treisman’s “Contextual Cues in Selective Listening” shows subjects briefly following a story into the unattended ear

• 1964: Broadbent and Margaret Gregory repeat Gray and Wedderburn’s experiment and offer a different explanation

Sources

• William James — The Principles of Psychology, Chapter XI: Attention (1890)

• Cherry — Some Experiments on the Recognition of Speech, with One and with Two Ears (1953)

• Broadbent — The Role of Auditory Localization in Attention and Memory Span (1954)

• Moray — Attention in Dichotic Listening: Affective Cues and the Influence of Instructions (1959)

• Gray & Wedderburn — Grouping Strategies with Simultaneous Stimuli (1960)

• Treisman — Contextual Cues in Selective Listening (1960)

• Broadbent & Gregory — Stimulus Set and Response Set: The Alternation of Attention (1964)