Could a Tangle of Wires Learn to Think Like You?
How a Baby Catches Language Without a Rulebook
Imagine an 18‑month‑old hearing “the cat is on the mat.” Nobody handed her a grammar textbook. Yet soon she will say “the dog is on the rug” without ever being taught the rule. Where does that ability come from?
For over forty years, one family of answers has stirred enormous excitement and even fiercer arguments. The idea is called connectionism. It says that your brain is not a programmed computer following a list of instructions. Instead, it is a sprawling web of very simple units—tiny switches that can be on or off—and the connections between them. As you grow and hear language, see faces, and move your body, the connections between these units quietly adjust. When you think, patterns of activity ripple through the web, and those patterns are your thoughts.
Connectionism challenges an older, deeply familiar picture: that the mind works like a digital machine, storing symbols and running rules. This article tells the story of that challenge—and why it still isn’t settled.
Inside the Learning Web: Nodes, Weights, and Practice
A connectionist network has three kinds of units. Input units receive signals from the outside world—like the brightness of a pixel in a picture or the sound of a word. Output units show the result—maybe “the face is male” or “the past tense of go is went.” In between are hidden units, which do the actual work.
Every connection has a weight, a number that says how strongly one unit influences another. A weight can be positive (it excites the next unit) or negative (it calms it down). When a signal arrives at a hidden unit, the unit adds up all the incoming signals—each signal’s strength multiplied by the weight of that connection. If the total passes a threshold, the hidden unit turns “on” and passes its own signal forward.
So how does a web of randomly connected units ever learn anything useful? The most famous answer is backpropagation, the workhorse of modern artificial intelligence. You show the net a training set: thousands of labelled examples. Suppose you want it to tell male from female faces. You feed in a picture, and tiny random weights produce a hopeless guess. Then the net compares its output to the correct label and sends an error signal, like a coach whispering “a little more male, a little less female.” The signal flows backward through the net, adjusting every weight just a whisker to reduce the error. After hundreds of thousands of rounds, the web learns to give the right answer—and often generalizes to brand‑new faces it has never seen.
Backpropagation is slow. A human child can learn what a new word means after seeing the object just once; a standard net might need thousands of examples. That gap still bothers researchers. But the core idea—learning from examples rather than from pre‑loaded rules—has already changed how we think about the mind.
When the Net Starts to Talk: Babbling, Mistakes, and Real Language
One of the earliest demonstrations of connectionism’s power came in 1987. Terrence Sejnowski and Charles Rosenberg built NETtalk, a network that learned to read English text aloud. They gave it pages of English sentences paired with the correct phonetic sounds. At first, NETtalk produced nothing but noise. Then it began to babble—sounds that were almost words but not quite. Later it spoke in uncanny double‑talk, and finally it read new sentences with reasonable accuracy. It had never been taught a pronunciation rule; it simply “soaked up” the patterns.
Around the same time, David Rumelhart and James McClelland trained a net to predict the past tense of English verbs. English has regular verbs that add ‑ed (“walk” → “walked”) and irregular ones that don’t (“go” → “went,” “come” → “came”). The net first learned a batch that included many irregulars, then saw a larger set dominated by regulars. Partway through training, it began saying goed and breaked—overregularizing just like a young child does. More training corrected this. The model didn’t store a rule like “add ‑ed.” It simply picked up statistical trends, and its mistakes looked surprisingly human. Critics, however, noticed that the net struggled with some novel regular verbs. They suspected a deeper problem: maybe pattern‑matching alone can’t fully explain how we handle rules at all.
Rules or Patterns? The Brain Debate That Won’t Go Away
The classical view, championed by philosophers like Jerry Fodor (1935–2017), holds that thinking is symbolic processing. Just as a computer shuffles strings of symbols according to a program, your brain is supposed to have a “language of thought”—sentence‑like representations built out of smaller pieces. When you understand “John loves Mary,” the classical story goes, you combine the symbols for John, loves, and Mary using a mental rule. That’s why anyone who can think that sentence can instantly think “Mary loves John”: the pieces are the same.
Connectionists reply that knowledge is stored in the weights between units, not in separate symbols. Representations are distributed: a thought about your grandmother isn’t a single “grandmother neuron” firing, but a pattern of activity spread across many hidden units. Similar ideas produce similar patterns, so the web’s internal structure already carries information about meaning. Some connectionists, called implementationalists, accept that the brain runs a classical program on this wet, web‑like hardware. Others, the radical connectionists, want to throw out the whole idea of mental rules and symbols forever.
The battle erupted in 1988 when Fodor and Zenon Pylyshyn (1937–2022) pointed to a feature of human thought they called systematicity. No real English speaker can understand “John loves Mary” while being unable to understand “Mary loves John.” A classical symbol system guarantees this, because you combine the same parts in a new order. But a connectionist net can be trained to recognize one sentence without ever being able to handle the other unless it is specifically shown that example. So, they argued, systematicity looks like a lucky accident in connectionism—a profound failure to explain why our minds work as they do.
Do Networks Really Get Grammar? The Fight Over Systematicity
Connectionists fought back by building smarter networks. Jeffrey Elman (1948–2018) created recurrent nets that feed signals backward, giving them a rudimentary memory. He trained one to predict the next word in sentences like “Any man that chases dogs that chase cats … runs.” The net learned to link “man” with “runs” even when many plural nouns sat between them. On a test of grammaticality, it performed almost perfectly—even on sentences it had never seen.
But the critics were not satisfied. Gary Marcus argued that when tested with a completely new vocabulary, the net’s performance collapsed. To him, this proved the net hadn’t truly grabbed the abstract rule—it had only memorized surface patterns. Later attempts tried to meet an even harder benchmark: strong semantic systematicity, where a net must correctly interpret a novel sentence whose parts have never appeared in certain positions before. In 2018, João Loula and his colleagues trained a recurrent net to follow commands like “jump around right.” The net could understand “jump around right” even if that exact phrase never appeared in training—a partial success. Yet when commands were made longer or shorter than those it had practiced with, the net faltered badly. It seemed unable to freely compose meanings the way a human does.
The debate has not ended. Some philosophers now think the whole notion of systematicity is too fuzzy to decide a winner. Others suspect that no architecture alone—classical or connectionist—can explain our ability without help from the world outside. What the battle has done is force everyone to be clearer about what really counts as “understanding a rule.”
Predicting Your Own Surprises: The Brain as a Fortune Teller
While the older fights continue, connectionism has sprouted new branches. One of the most influential is predictive coding. The idea is simple: the brain constantly makes predictions about what it will see, hear, or feel next. It sends those predictions downward, and then compares them to the actual incoming signals. What gets sent upward is only the error—the surprise. The goal of the whole system is to minimize that surprise over time, so the brain’s internal model of the world gets better and better.
This picture explains several puzzling facts. When you look at a blue sky, your brain predicts “still blue” for the next pixel, so very little information has to travel upward—that’s efficient. It also explains why you don’t notice your blind spot: your brain predicts “more of the same” and reports no error. Even binocular rivalry, where your conscious awareness flips between two different images shown to each eye, can be seen as the system hunting between two predictions, each in turn reducing error.
Predictive coding gives connectionism a new biological plausibility because it doesn’t need an external teacher. The world itself provides the feedback. But it also raises a big question: if thinking is just prediction‑error‑minimization, can it ever produce truly new ideas? Some worry that a brain built only to reduce surprise would curl up in a dark room and never explore. Defenders reply that our built‑in drives and learned expectations predict that we will go out, seek food, and learn—so staying still would create even more surprise.
Whether predictive coding will unify all of cognition is still unknown. What connectionism has done, from the early babbling of NETtalk to today’s predictive brains, is offer a lasting alternative to the view that we are little computers running rules. It suggests that thought is a live, pulsing thing—patterns in a web, forever reshaped by everything you experience.
Think about it
- If a neural network wrote a poem that made you laugh or cry, would you say the machine was thinking, or just copying patterns it saw in thousands of poems? Why?
- You see a brand‑new animal called a “snuffler” and guess that two of them are “snufflers.” Do you think you used a rule, or did you rely on patterns from words you already knew?
- Could a brain that only tries to reduce surprises ever invent a song that sounds like nothing heard before? Why or why not?
