Scent Into Action

Rodent responses to a whiff of predator may offer clues to instinct in the brain

David Ferrero wasn’t expecting the jaguar to pounce. When he approached the holding pens at Massachusetts’ Stone Zoo, the big cat watched but looked relaxed, lounging on her cage’s concrete floor. Two other jaguars rested in separate cages nearby.

In mice, a sniff of certain odors can trigger instinctual responses related to defense and mating. Sascha Burkard/Shutterstock
SNIFF DRIVEN When inhaled molecules activate nerve cells in the mouse’s vomeronasal organ, the signals travel to the accessory olfactory bulb in the brain, then on to sites that connect to the hypothalamus and mediate fear, attraction and aggression. Recent studies also implicate the olfactory epithelium in smell-triggered behaviors. Source: P. Chamero et al/Trends in Neurosciences 2012, adapted by T. Dubé
DISTINGUISHING ODOR On a molecular level, mice respond to chemical cues from other mice differently than they do to cues from other species, especially potential predators. In the vomeronasal organ, cells with specific sensor molecules (red) detect an odor molecule and switch on (green) in response. The pattern of a female mouse reacting to a male mouse (above left) looks distinct from a female mouse responding to an owl or a ferret, suggesting a way to distinguish between threats. Y. Isogai et al/Nature 2011
CARNIVORE STENCH | The molecule 2-phenylethylamine, or PEA, has proved potent in eliciting defensive behaviors in rodents. The molecule is detected by the olfactory epithelium, which also mediates smell-triggered behaviors. PEA is more common in the urine of predators than herbivores (right) and is structurally similar to another molecule, benzylamine, that triggers no response in rats. In an avoidance test, PEA was almost as repellant to rats as lion urine (middle). D. Ferrero et al/PNAS 2011

The jaguars usually prowled outside, in the grassy grounds of the zoo’s enclosure. But this afternoon, zookeepers kept the animals inside so that Ferrero and a colleague could grab a behind-the-scenes peek. Here, the jaguars slept at night — and fed. Here, only metal bars stood between the humans and the cats.

As Ferrero stepped closer to the cages, the watchful female sprang up, twisting her body toward him, front paws thumping the bars. Fully extended, she was as tall as Ferrero.

“I think she wanted to eat me,” he says. The zookeepers weren’t afraid, but Ferrero flinched. He wasn’t familiar with the lean, black-spotted feline. He was just there to pick up some pee.

Ferrero, a neurobiologist from Harvard, was visiting the zoo to gather urine specimens for a study linking odors to instinctual behavior in rodents. Early lab results had hinted that a whiff of a chemical in carnivore pee flashed a sort of billboard message, blinking “DANGER” in neon lights — enough to make animals automatically shrink away in fear.

Ferrero and Harvard neurobiologist Stephen Liberles are among a cadre of researchers trying to understand the basis of instinctual animal behaviors. In the last few years, scientists have made progress by studying smell — unmasking the molecular identities of behavior-triggering odors and charting these odors’ routes to the brain. One early stop, a sensory structure known to spur mice into action when they encounter odors from other mice, can actually rev the rodents up when they run into cats or rats, too.

In fact, studies have shown that odors from different species can spark varying patterns of neural activity in mice. And new evidence from researchers including Ferrero and Liberles suggests behavior-triggering odors don’t always travel to the brain in the way scientists once thought.

Recent research has even revived interest in the once-ridiculed idea that humans also respond instinctually to odors from other humans — though some scientists still think the idea is kooky. No matter who has it right, the new work may hold clues to the brain areas responsible for complex behavior in people.

“We used to think it was beyond the reach of what we could study,” says neurobiologist Lisa Stowers of the Scripps Research Institute in La Jolla, Calif. “There was just too much going on in the brain.”

Human heads are big, complicated and tricky to access, so researchers are zeroing in on rodent brains instead.

Slaves to scent

The world is a dark place for rodents. The skittish animals wait till night to scavenge for food, and hide out in shadows during the day. A life spent evading light means little use for vision. Mice are nearsighted and nearly color-blind. Their beady eyes glimpse only a bare-bones version of the rainbow: a dishwater copy of what humans see, tinted mostly gray and yellow.

To survive, rodents depend almost entirely on their sense of smell. A mouse can gather almost everything it needs to know about the world from odors. Even clues about a potential girlfriend.

A quick sniff of a new love interest is better than stalking her on Facebook. In an instant, the snooper can find out if his crush is sick, pregnant or if she’s had a stressful day. Mice use this information to make decisions about potential partners. But this decision making isn’t exactly a well-thought-out process. Some smells trigger behaviors that are automatic.

That mice are slaves to scents is good news for researchers. “They’re the perfect animal for us to study,” Stowers says. Their olfactory systems tap into areas of the brain that jump-start behavior.

Researchers have found, for example, a compound in tears that fast-forwards the dating process in mice. When a female nuzzles up against a teary male, something in her brain just clicks — she relaxes into a mating position, allowing the male to mount.

And odors don’t just spark attraction; they can trigger aggression, too. In mouse urine, Stowers’ team has found ingredients that spur male-on-male violence. The molecules identified are enough to send male mice into kicking, biting, tail-rattling bouts of rage.

Many of these instinctual behaviors are launched when an odor hits a clump of sensory tissue called the vomeronasal organ, or VNO. The clump — which fits in a bony pocket above the roof of a mouse’s mouth — is smaller than a grain of rice and filled with sticky mucus. When mice bury their noses in fur, tears or pee, the mucus picks up odor particles and the VNO delivers neon-lit messages — SEX or FIGHT — to brain regions that guide behavior. Traditionally, scientists labeled mammalian VNOs as control centers for pheromones, odors that spark preprogrammed behaviors among members of the same species.

Building on behavioral studies performed long ago in reptiles, Stowers wanted to find out if mouse VNOs also sensed messages traded between different species — and if so, how. In particular, she was interested in odor signals from predators that switch on fear behaviors.

“The fear response is one of these ancient things. If you look in any animal, you can recognize it,” Stowers says. Pulses quicken, stress hormones soar and either muscles freeze or animals flee. “These behaviors are so important to survival.”

Hunters and gardeners have long known that prey animals steer clear of predator odors. (Outdoor enthusiasts can even order such rodent repellents online: Predatorpee.com sells bobcat, coyote and fox urine.) What scientists didn’t know was how the scents responsible for these instinctual interspecies responses tip off the brain.

To find out, Stowers and her colleagues delved into the bodily secretions of cats and rats.

The smell of fear

If you’re looking for large volumes of cat saliva, the best place to go is the dentist. The cat dentist, that is.

“It turns out that when vets clean cat teeth, they suction off the saliva,” Stowers says. She and her team already had a hint that cat saliva turned mice into trembling balls of fur. To find the spit molecule that prompted the fear reaction, the researchers had to get their hands on spoonfuls of drool from vets. “We would just show up, and they’d hand us milliliters,” she says.

Sniffing a noseful of the saliva sent mice sprinting in the opposite direction and set off a behavior known as “stretch-attend.” Mice get up on their tiptoes, point noses toward the odor, stretch tails back straight behind them and cautiously sniff to see if the scary scent is still around.

“A relaxed mouse doesn’t do that,” Stowers says.

Spit from cats and pee from rats drove mouse stress levels up and switched on neural activity. The chemical culprits — major urinary proteins, or MUPs — are similar to an aggression-sparking urine ingredient that mice make themselves, Stowers and colleagues reported in Cell in 2010. Mice without a working VNO were fearless in the face of cat and rat MUPs. The VNO, Stowers’ team concluded, must be what’s letting mice cue into the scents other species.

“What’s really remarkable is that our mice haven’t been around a predator since the 1930s,” Stowers says. “They’ve never experienced these olfactory cues before; their grandmothers and great-grandmothers didn’t even experience them.” Because the mice have been bred in labs for hundreds of generations, any odor-triggered behavior has to be hardwired in the brain’s circuitry.

Stowers’ molecule should help pry open the door to how the brain converts scent into action. “It allows us to know a little something about how these behaviors work,” she says, “and to hope that we can learn something more.”

Scientists already have a rough sketch of an odor’s path from nose to brain. They know that the hardware of a mouse’s sense of smell includes more than 1,000 kinds of scent receptors, of which about 300 reside in the VNO. Like molecular antennas, the receptors sit at the surface of the nose’s nerve cells and tune in to smelly signals. The VNO’s neurons stretch up inside the mouse’s head and hook up with cells deep in regions of the brain that ignite aggression, attraction or fear. By singling out specific instinct-triggering compounds, such as MUPs, scientists can start to paint a more intricate picture.

Last year, researchers filled in some new details by tackling the problem a little differently. Instead of teasing apart individual odor ingredients, a team at Harvard tapped the entire odor package. Ferret bedding, alligator droppings, insect larvae: Molecular neuroscientists Catherine Dulac, Yoh Isogai and colleagues wanted a species’s whole scent caboodle.

Foul patterns

The ferrets smelled the worst, Isogai says. He remembers one afternoon on the subway, riding back from a lab at Tufts University with a backpack full of ferret bedding. The paper liner held all sorts of smelly leftovers: urine and feces and fur. And though he had wrapped the bedding well — in three layers of Ziploc bags — the foul odor still seeped from his pack. “Bedding’s great because it’s got lots of scents,” he says, “but I was scared people would ask why I was so stinky!”

Isogai and Dulac, a Howard Hughes Medical Institute investigator, suspected mice might use their scent receptors to discern predators.

The researchers let mice sniff 29 different odor cues. Then, the team immediately sliced up the mouse VNOs and used colored tags to pinpoint scent receptors that had been hit. Depending on the odor mice sniffed, different scent receptors switched on. The patterns matched up with different organisms, and the combinations overlapped for odors from similar species — as if mice could specifically tune in to snakes or birds or mammals.

Of the 88 scent receptors identified in the study, 71 responded to odors from animals other than mice, Dulac’s team reported in Nature last year. “The sheer number of these receptors was surprising,” she says.

In retrospect, Dulac says, the mouse’s receptor collection seems pretty logical. “You probably don’t need a lot of receptors to detect a mouse, but you might need a lot to detect and identify as many predators as possible.” Now, the researchers can track scents’ signals as they move from the VNO on through the brain’s circuitry, and can ask questions about different behaviors. “What brain areas are involved? What do they make out of the different signals?” she says.

A human route

Though the VNO has been the olfactory celebrity of pheromone research, scent studies by Ferrero and Liberles suggest this special clump of sensory tissue isn’t the only part of the mouse’s equipment that triggers hard-wired behaviors in response to scents. In the last few years, scientists have shown that rodents’ main olfactory epithelium — a folded sheet of cells that lines the nasal cavity — can provoke instinctual responses as well.

The folded sheet picks up a fear-triggering chemical from fox scent glands, for example, that the VNO misses. And when Ferrero, Liberles and colleagues tested pee from jaguars and other predators, the team found a new rodent-repelling odor ingredient, 2-phenylethylamine, that probably also hits the nasal cell sheet rather than the VNO.

Liberles and Ferrero’s team found the odor ingredient at high levels in 18 out of 19 carnivore species tested, including African wildcats such as lions and servals. In fact, the stuff was more than 3,000 times stronger in these meat-eating predators than in pee from plant-eating animals such as zebras and elk, Liberles and colleagues reported last year in the Proceedings of the National Academy of Sciences.

The findings suggest that rodents may respond instinctually to predators like cats via the nasal cell sheet, as well as the VNO. Having a few built-in ways to detect smells from dangerous predators is kind of like having multiple smoke detectors: If one fails, the clamor from the others still jolts muscles into action.

Unlike most animals, humans don’t have a working VNO. Humans do, however, have an able olfactory epithelium, a sensory-rich sheet covering about 3 percent of the nasal cavity. Though that’s not much compared with 50 percent in rats, it’s still enough for a decent sense of smell.

Recent data supporting the idea that the nasal sheet might drive odor-triggered behaviors “has sort of rejuvenated the idea” that humans respond instinctually to other humans’ odors, Liberles says. And there are interesting hints: Last year, for example, researchers from Israel reported that sniffing women’s tears curbed men’s sexual desire. Overall, however, evidence for pheromone signaling in humans is weak. “But,” Liberles jokes, “I don’t know what Axe body spray has in their personal chemical collection.”

Some evidence does exist, however, for other human instincts. Though people don’t “stretch-attend” when they sniff a big cat at the zoo, they do tend to wrinkle their noses at the sulfurous odor of a rotten egg. And certain scary sights, like a slithering snake, can spark fear behaviors. What’s more, because humans and mice have similar centers in the brain controlling aggression, attraction and fear, scientists think they may be able to apply what they’ve learned in mice to better understand human actions. The neural circuitry and the ultimate outcomes — FREEZE or RUN — are alike.

How sensory cues intertwine with people’s everyday conscious decisions can help shape more complex behaviors. And, for now, rodents might be the best route to understanding these behaviors. After all, it’s hard to find graduate students willing to get close with jaguars.

Meghan Rosen is a staff writer who reports on the life sciences for Science News. She earned a Ph.D. in biochemistry and molecular biology with an emphasis in biotechnology from the University of California, Davis, and later graduated from the science communication program at UC Santa Cruz.