The Strange and Secret Ways That Animals Perceive the World

Nonhuman creatures have senses that we’re just beginning to fathom. What would they tell us if we could only understand them?
A grid of four animals with different sensory systems highlighted in red.
Learning what organisms experience could help explain what they communicate.Illustration by Jérôme Berthier

One evening almost sixty years ago, a Tufts University researcher named Roger Payne was working in his lab when he heard a radio report about a whale that had washed up on a beach nearby. Although it was a cold, wet March night, he decided to drive to the shore. When he arrived, he discovered that the animal had been mutilated. Two passersby had carved their initials in its flanks. Someone had hacked off its flukes, and another person, or perhaps the same one, had stuck a cigar butt in its blowhole. Payne stood in the rain for a long time, gazing at the corpse. He had been studying moths; now he decided to switch his attention to cetaceans.

Aside from the dead one, Payne had never actually seen a whale, nor did he know where whales could be observed. At the suggestion of an acquaintance, he made his way to Bermuda. There he met an engineer who had worked for the United States Navy, monitoring Soviet submarines via microphones installed off the coast. While listening for enemy subs, the engineer had chanced upon other undersea sounds. He played a tape of some of them to Payne, who later recalled, “What I heard blew my mind.”

Payne took a copy of the tape home with him. The sounds—made, the engineer had determined, by humpback whales—ranged from mournful wails that evoked the call of a shofar to high-pitched cries that resembled the squeals of piglets. Payne found the tape mesmerizing and listened to it hundreds of times. Finally, it dawned on him that what he was listening to had a structure.

With the help of a machine called a sound spectrograph, Payne converted the voices on the tape into a series of squiggle-like notations. The exercise took years, but eventually it confirmed what he had suspected. The humpbacks always made their wails, squeals, and grunts in a particular order—A, B, C, D, E and never A, B, D, C, E, in Payne’s formulation. The paper in which he announced his discovery appeared in Science in the summer of 1971. “Humpback whales (Megaptera novaeangliae) produce a series of beautiful and varied sounds for a period of 7 to 30 minutes and then repeat the same series with considerable precision,” Payne wrote. Each series, he argued, qualified as a “song.”

While the paper was in the works, Payne arranged to have the humpbacks’ songs released as an LP. The album spent several weeks on the Billboard 200 and sold more than a hundred thousand copies. This was a particularly impressive feat, as one commentator noted, for a “work with no musicians, no lyrics, no danceable beats and actually no singers either. (Humpback whales do not possess vocal cords; they make sound by their pushing air out through their nasal cavities.)” The humpbacks inspired many terrestrial performers; Judy Collins incorporated some of their calls into her album “Whales and Nightingales”; Pete Seeger wrote “Song of the World’s Last Whale”; and the New York Philharmonic played “And God Created Great Whales,” a piece composed by Alan Hovhaness.

In 1977, when NASA launched Voyagers 1 and 2, designed to probe the far reaches of the solar system, the songs of the humpbacks went with them. The agency outfitted each craft with a “golden record” that could be played using a stylus (also included) by any alien who happened to intercept it. The recording featured greetings in fifty-five languages—“Hello from the children of planet Earth,” the English speaker said—as well as a sequence from one of Payne’s whales.

At the time the Voyagers set out, no one knew what, if anything, the humpbacks were trying to convey. Today, the probes are more than ten billion miles from Earth, and still no one knows. But people keep hoping.

Imagine the following scene: You are in a room with an owl, a bat, a mouse, a spider, a mosquito, and a rattlesnake. Suddenly, all the lights go off. Instead of pulling out your phone to call an exterminator, you take a moment to ponder the situation. The bat, you realize, is having no trouble navigating, since it relies on echolocation. The owl has such good hearing that it can find the mouse in the dark. So can the rattlesnake, which detects the heat that the rodent is giving off. The spider is similarly unfazed by the blackout, because it senses the world through vibrations. The mosquito follows the carbon dioxide you’re emitting and lands on your shin. You try to swat it away, but because you’re so dependent on vision you miss it and instead end up stepping on the rattler.

Ed Yong, a science writer for The Atlantic, opens his new book, “An Immense World: How Animal Senses Reveal the Hidden Realms Around Us” (Random House), with a version of this thought experiment. (His version also includes a robin, an elephant, and a bumblebee, though not the potentially fatal encounter with the snake.) Yong is interested in what animals might communicate to us if they could, which is to say, what they perceive. Humans, he points out, see the world one way. Other species see it through very different eyes, and many don’t see it at all. Attempting to exchange one world view—or, to use the term Yong favors, Umwelt—for another may be frustrating, but, he argues, that’s what makes the effort worthwhile. It reminds us that, “for all our vaunted intelligence,” our Umwelt is just one among millions.

Consider the scallop. (What’s sold at the supermarket fish counter is just the muscle that scallops use to open and close their shells; the entire animal resembles a fried egg.) Some species of scallop have dozens of eyes; others have hundreds. Inside them are mirrors, composed of tiny crystals, that focus light onto the retina—retinas, really, since each eye has two. A scallop’s eyes are arrayed around the edge of its body, like spikes on a dog collar.

Our brains combine the information gathered by our two eyes into a single image. With dozens (or hundreds) of eyes, scallops face a steeper challenge. But they don’t have much brainpower to devote to the task. (In fact, they don’t have brains.) In an effort to figure out what the scallops were doing with all their eyeballs, Daniel Speiser, a biologist at the University of South Carolina, developed an experiment he called Scallop TV. He strapped the animals onto little pedestals, planted them in front of a computer monitor, and forced them to watch images of drifting particles. Scallops are filter feeders, meaning that they consume plankton they strain out of the water. Speiser found that if the computer-generated particles were big enough and were moving slowly enough the scallops would open their shells. “It’s wild and creepy to see all of them opening and closing at the same time,” he tells Yong. He thinks that their eyes function independently, like motion detectors. When one eye senses something potentially tasty, it sends a signal to investigate. If Speiser is correct, Yong notes, then even though scallops’ eyes are both numerous and complex, the animals don’t possess what we would think of as vision. They see, he writes, “without scenes.”

“An Immense World” is filled with strange creatures like scallops and strange experiments like Scallop TV. Harbor seals have a fringe of vibration-sensitive whiskers jutting from their snouts and eyebrows. To gauge how sensitive the whiskers are, a team of marine biologists at the University of Rostock, in Germany, trained two harbor seals to follow the path of a miniature submarine. Then they blindfolded the animals and plugged their ears. To study how moths elude bats, scientists at Boise State University cut off some moths’ tails and fitted out others with fake wing extensions. To ascertain whether hermit crabs experience pain, a pair of researchers at Queen’s University Belfast prodded them with electric shocks, and to figure out the same thing for squid a biologist at San Francisco State sliced them with scalpels. When I got to the story of Kathy, a bottlenose dolphin who refused to don a sound-blocking mask that researchers wanted her to wear, I silently cheered for her.

The black ghost knifefish is, as its name implies, a nocturnal hunter. By firing a specialized organ in its tail, a knifefish creates an electric field that surrounds it like an aura. Receptors embedded in its skin then enable it to detect anything nearby that conducts electricity, including other organisms. One researcher suggests to Yong that this mode of perception, known as active electrolocation, is analogous to sensing hot and cold. Another posits that it’s like touching something, only without making contact. No one can really say, though, since humans lack both electric organs and electroreceptors. “Who knows what it’s like for the fish?” Malcolm MacIver, a professor of biomedical engineering at Northwestern, asks.

The most famous iteration of this question comes from the essay “What Is It Like to Be a Bat?,” published in 1974 by the philosopher Thomas Nagel. Bats are closely enough related to humans, Nagel noted, that we believe them capable of what we’d call experience. But how can we get inside their furry little heads? The difficulty is not just that they can’t tell us. It’s that their Umwelt is utterly foreign.

One might try to imagine, Nagel wrote, “that one has very poor vision, and perceives the surrounding world by a system of reflected high-frequency sound signals,” or that “one has webbing on one’s arms, which enables one to fly around at dusk and dawn catching insects in one’s mouth.” But that wouldn’t help much.

“I want to know what it is like for a bat to be a bat,” Nagel insisted. “Yet if I try to imagine this, I am restricted to the resources of my own mind, and those resources are inadequate.” The question “What is it like to be a bat?,” he concluded, is one that people will never answer; it lies “beyond our ability to conceive.”

“This should be enough spinach for dinner, but we won’t know until we sauté it.”
Cartoon by Johnny DiNapoli

Yong’s response to Nagel, who makes several appearances in his pages, runs along the lines of “Yes, but . . .” Yes, we can never know what it’s like for a bat to be a bat (or for a knifefish to be a knifefish). But we can learn a lot about echolocation and electrolocation and the many other methods that animals use to sense their surroundings. And this experience is, for us, mind-expanding. Yong speaks to Christopher Clark, a Cornell researcher who in the nineteen-seventies worked with Roger Payne, listening for whales. Whale songs lie at the opposite end of the spectrum from bat calls; they are very low frequency and can travel vast distances. If whales are using their songs to communicate with one another, they are doing so not just across space but also across time. A call made by a humpback near Bermuda would take twenty minutes to reach a humpback swimming off the coast of Nova Scotia. If the Canadian whale answered immediately, it would be forty minutes before the Bermuda whale heard back. To imagine what it’s like to be a whale, “you have to stretch your thinking to completely different levels of dimension,” Clark says.

Meanwhile, you don’t have to understand what it’s like for a bat to be a bat to appreciate what might mess with a bat’s way of being. Yong pays a nighttime visit to Grand Teton National Park with Jesse Barber, a biologist at Boise State University. Barber is concerned about what’s become known as “sensory pollution.” Even in the Tetons, lights now illuminate the darkness. Insects are drawn to the lights; bats are attracted to the insects; and, the worry is, owls pick off the bats. To test this hypothesis, Barber and his students spend the night tagging bats in a campground parking lot. The lot, Barber complains, is “lit up like a Walmart because no one thought about the implications for wildlife.”

Yong wants us to think more about these implications, which can upset entire ecosystems. He offers the example of Woodhouse’s scrub jays, which are native to the western United States and central Mexico. The birds are important to the survival of piñon pines because they spread the trees’ seeds. But they’re bothered by the noise of compressors, so they avoid spots where natural gas is being extracted. Researchers found that, where the jays still find quiet, piñon-pine seedlings are four times more common than in noisy areas the birds have abandoned.

“Through centuries of effort, people have learned much about the sensory worlds of other species,” Yong writes. “But in a fraction of the time, we have upended those worlds.”

In September, 2015, a British documentary filmmaker named Tom Mustill was vacationing in California with a friend when the two decided to take a kayak trip in Monterey Bay. The aim of the trip was to see whales up close, but Mustill and his friend got more than they had bargained for. As they were paddling about, a humpback shot up out of the water just feet from their boat. (Mustill later compared the experience to watching the space shuttle take off.) The whale, which weighed thirty tons, came down more or less on top of them. The two kayakers were sucked under, along with their boat. Mustill thought that he had been torn apart and attributed his lack of pain to being in shock. But he and his friend both resurfaced in one piece. They made it to shore, where the company that had rented them the kayak offered them free hot chocolate.

Mustill continued with his vacation, which included a camping trip in Big Sur. When he got back in cell-phone range, he learned that someone on a nearby boat had captured his whole whale encounter on video, and that the video, posted to YouTube, had gone viral. By the time Mustill returned to London, it had been viewed four million times. The story was picked up around the world. “Baleia de 40 toneladas quase esmaga casal de canoístas” (“Forty-ton whale nearly crushes couple of kayakers”), the Cape Verdean newspaper Expresso das Ilhas reported. “ ‘How am I not dead?’ ” the headline in the Daily Mail ran.

As a result of his newfound fame, Mustill became, in his words, “a lightning conductor for whale fanatics.” Everyone, it seemed, had a story about whales. Many involved interspecies communion. A member of the British Navy told him about how whales had sung to him in his submarine. A book publisher told him about how a pregnant dolphin—both dolphins and porpoises belong to the group known as toothed whales—had indicated that she, the publisher, was also pregnant, something she herself had not known at the time. A scientist recounted locking eyes with a gray whale that approached her in a Mexican lagoon and let her rub its enormous tongue.

Mustill himself couldn’t shake the experience. A whale researcher told him that the only reason he had survived was that the humpback, upon noticing him and his friend, had purposefully turned its body so that it wouldn’t kill them when it landed. Mustill decided to make a documentary, “The Whale Detective,” which ran a couple of years ago on PBS. Now he has written “How to Speak Whale: A Voyage Into the Future of Animal Communication” (Grand Central).

Like Yong, Mustill is interested in animals’ perceptions. But he wants to push beyond mere Umwelt-switching to an exchange of what might, broadly speaking, be called ideas. Early in the book, he goes to visit Payne, who’s now eighty-seven. Why, he asks, do humpbacks sing? And what do their songs mean? Payne says he can’t say: “I would desperately love to know.”

Mustill isn’t deterred. He delves into the latest research on animal communication. Many species have been shown to have highly complex systems of conveying information—so complex that they probably deserve to be called languages, though people tend to reserve the word “language” for themselves. Chimpanzees in the Budongo forest of Uganda, for instance, have a repertoire of at least fifty-eight gestures, which they combine in sequence much the way we combine words. Prairie dogs in the American West make distinctive cries to indicate different predators, and they seem to be able to incorporate descriptions into them: a big dog, for example, will elicit one sort of cry; a small dog, another sort. Chestnut-crowned babblers, sweet-looking brown-and-white birds native to Australia, respond differently when elements of their calls are played in different orders, much as we would respond differently when offered, say, a cake pan rather than a pancake.

Owing to advances in recording technologies and artificial intelligence, researchers in the burgeoning field of bioacoustics can now download thousands of hours of animal sounds and leave the work of sifting through them to a computer. This has opened up tantalizing new possibilities, including that of translating animal-communication systems into English—or Arabic, or Xhosa. Six years after Mustill was nearly killed by the humpback, a group of scientists from, among other institutions, Harvard, M.I.T., and Oxford formed the Cetacean Translation Initiative, or CETI, to try to decipher whale communications. (The team is working with sperm whales, which, instead of singing, issue patterns of clicks, known as codas, that have been compared to Morse code.)

“Is it too much of a leap to think we might someday decode the sperm whale click for ‘mother’?” Mustill writes. “For ‘pain’? For ‘hello’? The answer is, of course, that we cannot know until we try.”

No less than “An Immense World,” “How to Speak Whale” is dogged by the “what is it like” question. Mustill suggests that decoding whale-speak could finally produce an answer. The problem, or perhaps the paradox, is that to decipher whales’ songs or clicks we would need to have access to the experiences they’re referring to. And this is precisely what we lack. Wittgenstein was even blunter than Nagel. “If a lion could speak, we could not understand him,” he maintains in “Philosophical Investigations.”

Mustill never addresses this problem directly. “How to Speak Whale” is borne along by his faith that whales have something intelligible to tell us and his hope that one day soon we’ll figure out what that is. “Songs of the Humpback Whale,” the album that Payne released in 1970, helped bring about the end of commercial whaling, Mustill notes. Think how transformative it would be if we could chat with whales about their love lives or their sorrows or their thoughts on the philosophy of language. “The more we learn about other animals and discover evidence of their manifold capacities, the more we care, and this alters how we treat them,” Mustill writes.

This seems to be true, or at least it seems as if it should be true. And yet every year the outlook for nonhuman species grows grimmer. In the case of marine-mammal species, the International Union for Conservation of Nature now classifies a third as endangered. A recent study by a team of European researchers concluded that even many of those species which seem to be doing all right, such as gray whales, are threatened by climate change. As Mustill himself observes, “To be alive and explore nature now is to read by the light of a library as it burns.”

So what message would the world’s remaining whales deliver to us if they had the chance? How do you click “What the #@ ϟ ⚛︎!”? ♦