I am SO pleased with how Flip did today when we went out to work on his fear of drains and access covers. The unplanned break that we took for several months seems to have actually helped, as the last few weeks have seen a lot more progress.

He will now go towards most cement access covers, sniffing or targeting on the indentations that they get lifted out by, and seems to do this with ease and confidence. He is still wary of metal plates that move, although he will walk on metal plates and doesn't panic when they move (it helps that I click and offer a treat as instantly as I can). What he is still afraid of are the PG&E plates that have metal bars in them:


These are often in a row. He will target them, the bars specifically, but he does it warily:


The Big Scary is still storm drains. He will not pull towards them at all, even with that "morbid curiosity" he has developed towards all the previous scary drains. But he will approach them, very warily, if cued.


Thank doG he is both "clicker-savvy" and still a treat pig. In addition to that, I think the relief of being able to move away after every click/treat is also very reinforcing (even if it's negative reinforcement):


We also had better success going into stores with various floors. He reluctantly walked across the tile parts of the Bathworks store, eagerly entered Gussied Up (cement) and Details (tile and carpet) to beg treats from the ladies in those stores, made it from one door of Domus (laminate wood) to the other, and even walked up to a man standing near the counter of WG Coffee Roasters (black and white checkered linoleum). He then panicked a little and wanted to run back out WGCR's door, but once he reached the mat by the door I was able to convince him to come back into the store a little further. He made it almost to the end of the counter, and almost to the couch in the opposite direction. Even the barista was impressed. I am very happy!

Just got back from another session with drain training. He did even better today, but this will be a short report/record since I have to go teach class soon. I parked at the south end of the shopping district of Lincoln Ave and we went into several stores. Domus: afraid, but would get off the matt a few feet. WG Coffee Roasting: afraid, would only go about 8 feet even with friendly people calling to him. Boutique near Treehouse: Walked right in. Gussied Up: Walked right in, looked for the lady with the treats. Details (another boutique): Walked right in. Got massively fussed over by the ladies there who gave him treats and pets - was throwing tricks at them. Hicklebee's bookstore: Walked right in, walked through the stacks, went up the stairs to the author signing area, walked over plexiglass-covered floor puzzles.

Went back to Domus - would walk a little further. Went back to WSCR - would walk a little further. Went back to Domus for the third time - walked as far as the register counter. Walked down the street to the library - he would not go in, and the automatic door almost closed on him. I brought him back outside and hooked his leash to a pole so I could pick up a book I'd requested. He was fine and happy when I came out, and walked happily back to the car.

He strains *towards* cement access covers so he can target their handles, but I can sometimes get him to just ignore them and walk by. He strains away from the middle of the crosswalk, and warily approaches storm drains.

Looking forward to more improvements!

Just came back from a “field trip” to work on drain training - Flip's fear of drains. We went on Lincoln Ave (a busy shopping street), parking in Adira’s lot and walking north as far as Jamba Juice and south as far as Gussied Up (about 2 blocks total, both sides of the street).

I now believe that Flip is NOT afraid of cement or metal access covers in sidewalks, although the big metal plates that move and make a noise are still worth trepidation; and he may be learning to distinguish between wet spots or stains and actual covers, or is just no longer worried about discolorations. He even did well at the “Hellmouth Spot” - where there is a conglomeration of plates, access covers, and even a manhole and storm drain at the curb (just north of Treehouse).

He IS still afraid of storm drains, and manhole covers. Crossing the street is difficult because he is scanning for them; also the crosswalks on Lincoln Ave are brick and that might be too much of a contrast as well.

He is also nervous about going in doors of shops, although that may be more related to the floors inside. He had little trouble going into Powells Candy Shoppe (though we were very politely “kicked out” before we got far) and waltzed right into Gussied Up dog boutique (he may remember or know from the smell that it’s a treat-filled place, but I think last time he had been afraid to go in). He was also comfortable going into Treehouse toy store, but not Splashworks (bathroom fixtures store) or Bank of America.

We did not try to go into Willow Glen Coffee Roasters, which he had gone into with great reluctance after a LONG shaping/coaxing session many months ago. On our next trip we’ll try going to the south end of Lincoln Ave. It seems as if the long break we took from working on this has helped. Not sure if returning to more concentrated sessions would help at this point, or spreading them out, but I’m going to try to fit more in during the next few days.

Imagine a feral dog who had never been around humans in her life. The sound of a human’s voice, even in a high pitch and with short bursts of sound, would probably not be something that this dog would find all that appealing; she probably won’t seek it out and it would be highly unlikely that she would be willing to do much work to gain it. The touch of a human would be even less attractive; it might be in fact frightening and aversive.

Most of us do not work with feral dogs; we have dogs who have had some exposure to humans and have varying amounts of socialization experience. But even among well-socialized dogs, there’s a large variation in how a dog will perceive high-pitched human voices and our touch. Some dogs do seem to genuinely enjoy hearing our voice, and we can use praise as a reward for good behavior. Some dogs do seem to genuinely enjoy being touched, stroked, scratched, or patted (seldom hugged), and we can use petting as a reward as well. But there is also hope for the many dogs who are uninterested in or actually repelled by praise and petting – you can systematically teach them to go from avoiding to enduring to enjoying, and you can eventually use them – judiciously – as rewards.

In the past, many dogs learned to love praise or petting because it indicated to the dog that there was no punishment (or “correction”) coming – if you hear “good dog!” or feel your head patted, you have successfully escaped the application of the choke chain (or worse). Since most people have moved past this archaic method of training, dogs can learn an even more pleasant association – praise and petting can be paired with some kind of reward that a dog really likes.

All animals have some interest in the “primary” reinforcers that ensure survival and comfort: food, water, air, shelter, and mating opportunities. These are so powerful that many trainers just rely on them (mostly food), and never bother trying to introduce newer forms of rewards. However, there can be a huge advantage to having a variety of rewards. It adds variability to your training, keeping it more interesting; it is also there as a fallback when your animal is not interested in one of those primaries (for example, if your dog is sick or even just full, and not interested in food) or they are not available to you.

Many of us do a pretty decent job of accidentally creating extra rewards, through the simple classical conditioning process of pairing them. For example, you say “Good dog!” or pet him, and then you feed him a cookie. Trish King, a trainer and behaviorist from the Marin Humane Society, once noted that she had accidentally conditioned a gasp sound as an alternative reward – she’d exclaim “Ah!” when her dogs did something cute or desirable, and then dug out a treat to give them.

But Ken Ramirez, the director of training at the Shedd Aquarium in Chicago, suggests that a wise trainer does a more systematic process to create these extra rewards. All too often, trainers (or pet owners) think that the reward they’re giving is good enough (perhaps because it worked on a previous dog, or we’re told that it works for most dogs) when really it is not what the dog wanted – resulting in low levels of learning at best, and frustration or even aversion at its worst.

I’ve seen way too many dogs coming to their owner on cue only to be patted on the head. The moment the patting is done, the dogs move away, out of reach, sometimes even doing a stress-relieving body-shake – a sure sign that they were not that interested in the first place. And the next time the owner calls, the dog often seems to be looking to see what “reward” the owner might be offering. If nothing better than petting looks forthcoming, the dog refuses to come.

So how do you teach dogs to value rewards like praise and petting? You use classical conditioning – a simple process that gains power through repetition. You simply link the new reward with an old reward.

Because the kind of natural rewards needed for survival are termed primary reinforcers in the scientific literature, any rewards that gain value through learning are called secondary reinforcers. But don’t be confused – in the most effective procedure to create new rewards, the “secondary reinforcer” is presented first. Here is how it works. First, you pick a novel sound or action on your part that will act as a reinforcer eventually. Don’t pick “good dog” because you will probably use it reflexively, says Ramirez, and you don’t want to use it outside of this sort of training context yet. Pick something that you will have to think about before saying, so you can’t just use it accidentally, like “most excellent!” or “just fabulous!”, or something like the thumbs-up sign, quiet clapping or a gentle touch on the dog’s head or snout. (Clapping might be an especially good choice for a dog who does public performances, like showing, sports competitions, or tricks contests.)

Next, you get out some primary reinforcer that you know your animal likes – in most cases it will be food (but if your dog is interested in water, or you have control you’re your animal’s access to a desired mate, it would be very powerful to use those!). Simply give your new reward, and follow it immediately by the old reward. In other words, you say, “Most excellent!” and then immediately after, you feed a few pieces of tasty treats. You repeat this a few times. You can cue your dog to do something he knows (like sit) and mark that with your established marker (like a clicker) and reward it with an established reward (like food), in between pairings of the new phrase and the old rewards.

You’ll want to repeat this pairing over several sessions, in many different locations and contexts. And while each pairing session can be short, you’ll want to invest quite a bit of time on it, just to ensure that the value will be strong before you start to rely on it. When you do start to use it in your training, start by using it to reinforce already-established behaviors that are ready for a variable reinforcement schedule.

Note: Conditioned stimuli base their value on their history of being associated with more primal events, and if the pair is separated from each other too often, they lose their power. That is why you are cautioned to always follow the sound of the clicker with the treat (even though advanced trainers working with savvy animals can sometimes get away with it). Some new reinforcer will become powerful enough to stand on their own, but others will need to be “recharged” by repeating the pairing of the new reinforcer with a primary reinforcer.

Also, conditioned reinforcers always act as event markers as well as reward marker, so you want to be as careful with the timing of your new reinforcer as you are with a click or with the delivery of a food treat. With any kind of touch (what exotic animal trainers call a “tactile reward”), you will probably be reinforcing proximity to yourself, but other reinforcers – your praise for example – can be used for work at a distance.

Some dogs love praise, and some love petting, and sometimes these dogs are willing to put a lot of work in just to get that sort of prize. But not all dogs do – after all, there is nothing natural to a dog about hearing your voice or being pawed at by your hands. But over time, all dogs can learn to like it!


For more information on Ken Ramirez, see kenramireztraining.com.

Notes from Susan Friedman’s article “What’s Wrong With This Picture? Effectiveness is Not Enough”

This article lays out a clear and logical explanation for why people who do behavior modification should choose positive reinforcement techniques before punishment - and how other techniques might be used even before then.

To be humane, behavior modification “should be as unintrusive for the learner as possible and still be effective”. She then quotes an article by two specialists in human children special education, to say that intrusiveness is defined by two criteria: the level of social acceptability, and the degree to which the learner maintains control during the behavior modification attempt. I think that this first point is a good way to look at the “ethics” of technique use- she goes on to say it’s a personal judgment, but that research shows that teachers, psychologists, parents and children all prefer R+ procedures over P+ procedures. But it’s this second point which I think is often overlooked: That animals need to have some control over their actions and the consequences of their actions. Dr. Friedman goes on to say, “Research demonstrates that… all animals should be empowered to use their behavior to control significant events in their lives, i.e. to use their behavior effectively to accomplish desired outcomes. Indeed, that is what behavior has evolved to do.” She then describes how the phenomenon of learned helplessness is related to not being able to use behavior to escape unwanted consequences.

Dr. Friedman then introduces her Hierarchy of Intrusions, which if followed will allow behaviorists to “increase the humaneness of our interventions without compromising our learning objectives”. She says, “as to the question, ‘is effectiveness enough?’ the answer is a resounding ‘NO!’ when it comes to selecting behavior interventions…” She’s referring here to children, but suggests that “the ethical standards established for behavior analysis may also have widespread relevance to behavior consultants working with any species of animal.”

Here is the Hierarchy of Intrusions:
Level 1: Resolve distant antecedents – address medical, nutritional, and physical environment variables.
Level 2: Resolve immediate antecedents – setting events, motivations, cues for problem behaviors
Level 3: Use positive reinforcement to increase the probability of the right behavior, more than the problem behavior
Level 4: Differential reinforcement of alternative behavior – reinforce an acceptable replacement behavior and remove the maintaining reinforcer for the problem behavior.
Level 5: Any of these three - Negative Punishment (remove a positive reinforcer to reduce the probability of the problem behavior); Negative Reinforcement (contingently withdraw an aversive antecedent stimulus to increase the probability of the right behavior); Extinction (permanently remove the maintaining reinforcer to suppress the behavior)
Level 6: Positive Punishment – contingently deliver an aversive consequence to reduce the probability that the problem behavior will occur.

(There’s a lovely graphic of the Hierarchy here)

I found this article to be a clear and strong layout of the logic and ethics behind choosing positive reinforcement above punishment.

More information on can be found Susan Friedman here and here.

Book review of Turid Rugaas' book, "My Dog Pulls. What Do I Do?"

Overall this book seems very simplistic in its directions – stand in place when the dog pulls, then make a sound to attract the dog’s attention, praise him when he turns to look, start walking in another direction, and reward him. This seemed especially silly when the directions jump from “stay inside or in a quiet place” to “increase the difficulty little by little by introducing distractions” such as cars, bicycles, roller skaters, cats, livestock, and kids playing football! What lumping!

However, she has a few good points. Here’s one:

“Don’t teach your dog to make eye contact in any of these phases of training. If you do this, he will learn to make eye contact, and he won’t be able to concentrate on what you really intend him to learn: walking on a loose leash.”

And another:
“Remember: When your dog is very stressed, excited or afraid, there is no use even trying to train. Find another time and place to do it. When stress levels become high, the brain doesn’t function so well, and the dog literally can not hear or concentrate on anything.” I’m glad that this author of the wonderful “Calming Signals” book included a note about stress, although she didn’t really include how to tell if the dog is stressed (not even to refer the reader to her other book). And I’m pretty sure it's inaccurate to say a stressed dog “literally can not hear”!

She also strongly warns against using any kind of collar around the neck, for fear of the dog developing neck and back problems, or getting the thyroid gland “crushed”. I’d love to see some references for that. She also suggests a leash that is 2-3 yards long (that’s 6-9 feet) so that the dog has an opportunity to sniff and explore. Most of the book’s pictures show dogs enjoying doing that, on a loose leash – on lovely Norwegian walking paths that look to my suburban Californian eyes ridiculously wide and empty. Very few dogs I know have more than a 3-4 foot sidewalk to wander on, with home owners and shop keepers zealously guarding their lawns.

Overall, I was disappointed in this book.

I took Flipper on a walk to work on his fear of drains. It was a crowded walk – I had (as always) Skinner and Pavlov on my shoulders; in addition I had the words of Steve Martin (the bird trainer) about giving animals the chance to escape a situation they’re unsure of; the conscious use of negative reinforcement in the form of moving away from the scary thing (as advised by, among others, Grisha Stewart); the watchful eyes of Kay Laurence assessing my proficiency; the awareness of Bob Bailey and Alexandra Kurland of how I was delivering my treats, Jean Donaldson declaring The Bar Is Open, and the ideas from Temple Grandin about visual contrasts and the visual cliff effect. I don’t even know how many were in the crowd suggesting the use of targeting, or using higher-value treats for harder challenges – I’m sure Karen Pryor and Gary Wilkes were among them.

So I had Flipper off-leash (a privilege we’ve earned after 9 years together), and I clicked and treated him for targeting the kinds of drains, manhole covers, and access plates he finds alarming. I noted that it seems that lighter-colored covers were easier for him to handle than darker ones, and that he seemed most interested in the darker holes where the plates are pulled up. By far the scariest things are the storm drains – and ours are metal grills below a metal curb painted white (with “flows to the bay” painted on them). Instead of standing next to Flip while he moved forward to tentatively paw-target the curb edge, I stood ON the drain grills, hoping to shatter the visual cliff appearance. I varied giving him his treats just about where he dared to put out his nose, and giving them off to the side where he was escaping the situation. I was careful to click while he was moving forward and not while he’d decided that was enough and it was time to retreat.

We were in a familiar area, but we usually walk on the other side of the street, alongside an elementary school. Today we were on the opposite side of the street, along a housing development, where every house’s matchbox lawn had a series of access plates. The street is essentially a series of cul-de-sacs, and each court has a pair of storm drains near the entrance, about 20 feet from where the crosswalk would be if one had been painted in. This gave us many opportunities! It was also a pretty sunny day (at least at that time) so contrasts were probably pretty high.

By the end of the session, Flip was clearly deliberately stepping on the light-concrete and light-metal access plates, even ones that had small grills on either side. He was still very wary of storm drains, but seemed much more willing to approach them and in less hurry to leave. I even had to ask him to “leave it” for a few manhole covers that were out in the street.

I had been aware for a while that he was wary of even darker spots on the pavement , possibly especially at night. I think Temple Grandin’s observations of animals’ attention to contrasts helps me. I also think being off-leash helps – even though we both generally keep a loose leash – partially because when he’s on-leash it’s usually because there’s enough traffic or other dangers, which probably means a more challenging environment anyways. I’m thinking here of Lincoln Avenue or Santana Row, our local downtown walking streets very popular with pet owners who want to stroll with their dogs while looking in shop windows or stopping for coffee. I used to bring Flip to those areas more but lately it’s been too difficult – he’s too afraid of the very many access plates that each business has on the sidewalk in front of their storefronts. Even entering the new training facilities (Pooch Hotel and the Humane Society Silicon Valley’s classrooms) is restricted in terms of not really being able to let him off-leash to take his own time to enter. Unfortunately, we have a class at HSSV the night after tomorrow which I’d like to take him to (though he’s not strictly needed as a demo dog) – I’ll have think about whether it’s worth it to try to get there early enough to do some training on that.

I am writing a review of this book in my animal training blog, because I was fascinated by the many interviews and articles I encountered about this book (for example: http://www.kqed.org/epArchive/R907061000 and http://www.sciencefriday.com/program/archives/200907106), so I got it on an audiobook and have really enjoyed it. I don’t believe I have the condition that the book describes, “compulsive hyper eating”, but we can all probably learn a little about our impulses to eat certain foods. But the main reason I got this book, and am reviewing it, is that I think some of the neurology involved is of interest to animal trainers. So I have two aspects to my review:
Aspects of the research that might be helpful to animal trainers who want to understand how behavior modification works, and make their own behavior modification programs more effective
And
Aspects of the research that are of interest to human beings who eat in the modern world (and I’m guessing that if you’re reading a computer blog, you have access to industrial food!)

Because I used an audiobook, all the references are the time marks of the playback; for example “1:25” means one hour and 25 minutes into the book. I’m sorry that this might make it harder for anyone with a paper version (and it also looks like quotes from psalms)!

Dr. Kessler defines this state of compulsive over-eating as a sort of reward-drive response to the sensory stimuli associated with food. That is, the image of food (or other stimuli associated with it, such as your favorite restaurant’s logo) drives you to acquire a pleasure from attaining and consuming that food.

Early on, Dr. Kessler reviews several studies that showed that lab animals (mice and rats) were shown to work harder for food higher in salt, sugar, or fat; or a combination of those. Much of the book refers to these combinations – how restaurant chains experiment with them to make “highly palatable foods”. As animal trainers, we might want to take advantage of this knowledge when we are choosing rewards. Not all animals will have the same response (I doubt a giraffe, a horse, a dog, a parrot, and a dolphin would all like the same food!), but I can easily see a dog trainer choosing a fattier (greasier) treat, sprinkling a little salt, or even a tiny bit of sugar in your treat mix to make it tastier.

In addition, food can become more reinforcing by changing other aspects of it as a stimulus. Kessler writes, “Increasing the multi sensory aspects of a stimulus, or adding other stimuli, can strengthen the reinforcing effect. The more potent and multi-sensory foods become, the greater the rewards they may offer, and the more we learn to work for them” (1:44). I think this is why sometimes dogs will show a renewed interest in the same ol’ treat if you toss it to them, roll it across the room, hide it under your hand and then reveal it, or otherwise present it in a new, interesting way. Vary how you present your rewards, sometimes with little fuss, and sometimes with a flourish! (Just make sure you keep in mind how the presentation of your reward will set your animal up for the next trial!)

Even making your treats different might have an effect on their value. Dr. Kessler describes “…a phenomenon known as taste-specific satiety. After eating a certain amount of one food, animals typically become satisfied with its taste, and stop eating it. But they’ll keep on eating if something else is available” (1:25). For example, rats habituated to a certain flavor of food and their dopamine levels in response to the food declined. This is explained through habituation, and also through the body’s need for homeostasis, which reduces a neural response to a specific stimulus as the stimulus persists or is repeated predictably. But if the stimulus is powerful, novel, or “administered intermittently enough, the brain may not curb its dopamine response.” Also, keep in mind that “habituating to one rewarding food does not reduce the value of others” (1:58).

I think this is why we might say, “Oh I’m so full… oh wait, is that dessert?”

Kessler continues, “The trick to overcoming habituation… was to keep the food relevant to the animal. That can be achieved by offering a tantalizingly small amount of stimulating food, providing access to that food at the same, predictable time, or by using other strategies that create a sense of anticipation. The effect is to undermine the brain’s capacity to habituate to a consistent stimulus” (2:01).

Another phenomenon that animal trainers should be aware of is “attentional bias”, that is, the importance that is attached to certain stimuli over others. I’m thinking here of dogs who are reactive to certain triggers, such as the sight of other dogs. The appearance of other dogs focuses their attention – sometimes to an extreme. Attentional bias is mitigated through dopamine. Kessler writes, “Dopamine increases animals’ drive to work. “If opioids give food its pleasure, and help keep us eating, dopamine motivates our behavior and impels us toward food. By strengthening our sense of anticipation, dopamine gets us to engage in a complex set of pursuit and acquisition behaviors, so we can recapture the remembered pleasure of a favorite food.”

Imagine if we found the same neurochemistry in other habitual behaviors, so it would be “dopamine motivates our behavior and impels us toward [performing a favorite activity]. By strengthening our sense of anticipation, dopamine gets us to engage in a complex set of pursuit and acquisition behaviors, so we can recapture the remembered pleasure of a favorite [activity].” If you are struggling with changing a well-engrained behavior in an animal, fill that behavior in here, and think about how the animal has been rewarded for that behavior in the past, and how that reward and its anticipation is mitigated by dopamine.

“Dopamine drives desire through a survival-based capacity known as ‘attentional bias’, defined as the exaggerated amount of attention that is paid to highly-rewarded stimuli at the expense of other, neutral stimuli, attentional bias allows us to pick out what matters most, so we can pursue it” (1:29).

Natalie Angier put it this way in her NY Times article: “Dopamine is also part of the brain’s salience filter, its
 get-a-load-of-this device” (http://www.nytimes.com/2009/10/27/science/27angier.html: A Molecule of Motivation, Dopamine Excels at Its Task).

For humans with compulsive hyper eating, attentional bias “gives rewarding foods their prominence in our minds. The more rewarding the food, the greater the attention we direct toward it, and the more vigorously we pursue it.”

Lest you think that food is a special case, here’s another example, this one from the Nurture Shock blog: “The Internet also activates the same pleasure pathways in the brain as drugs and alcohol. As you continue to be rewarded, for completing the next level of a videogame or finding out a new piece of information, the connectors to the limbic system of the brain are stimulated, releasing euphoria-causing dopamine into the body. The brain remembers that happy feeling, encouraging you to keep going back for it” (http://www.newsweek.com/id/216911/page/3)

Eating high-value food releases dopamine. Finding information on the internet releases dopamine (I hope you’re feeling it!). You can imagine that other habitual behaviors in animals that end in some sort of reward follow the same pattern.

One study Kessler cites showed that animals with depleted dopamine levels did not work as fast or put in as much effort to acquire food.

In Chapter 10 Kessler sites a study that should be of particular interest to clicker (reward-marker) trainers, one by Dr. Wolfram Schultz. “An animal releases dopamine in a steady and fairly consistent pattern when it is not being stimulated. But give an animal a reward, and transient bursts of increased dopamine can be detected in its brain .…. When monkeys were given a reward they didn’t expect – a taste of sweet juice – there was an upward spike in their dopamine levels. Next, he gave the monkeys a visual or auditory cue, followed almost immediately by the same juice. Once they became familiar with that sequence of events, their patterns of dopamine firing changed. Based on learned experience, the animals began to recognize the cue as a signal the juice was coming, and responded with elevated dopamine activity. Rather than firing at the reward itself, dopamine fired in response to the stimulus that predicted the reward. Given dopamine’s role in focusing attention, the pairing of a cue with a reward had a potent effect on behavior” (1:47). This is also described in an article Dr. Schultz curates at ScholarPedia, saying, “During the course of learning, the dopamine response to the reward decreases gradually, and a response to the immediately preceding CS develops in parallel” (http://www.scholarpedia.org/article/Reward_signals).

The effect of this dopamine in response to the conditioned stimulus is something Kessler calls “incentive salience,” or “the desire, activated by cues, for something that predicts reward. It’s a learned association. We learn to want a food – or some other substance – we once liked… It’s the wanting (not the liking) that drives us to do the work necessary to obtain that food… experience serves to imbue that cue with incentive salience. Positive emotions become embedded in cues, which then become a force of their own.” (Italics mine… I can’t really tell when an audiobook is in italics!)

Experience of emotions can of course change a stimulus’s emotional value, something Jean Donaldson calls a “conditioned emotional response (CER)”. Kessler puts it like this: “History gives particular food an emotional charge… Food becomes… a ‘hot stimulus’, lighting up the emotional centers of the brain, getting us to think, feel and respond to our desire. Memories interact with the reward pathways that drive our behavior” (1:55).


Chapter 12 of Kessler’s book is titled, “Rewarding foods rewire the brain”, and again I’d plug in “rewarding ANYTHINGS rewire the brain” and fill in the demons you battle (in yourself, or your animal, or your clients’ animals).

Behaviors that are practiced can become habit – even when they start to be associated with bad situations. For example, rats that were fed a highly palatable food for three weeks developed a habit of eating it – even when the rats were made deliberately sick from eating the food after the third week. On the other hand, if they didn’t have time to form that habit it went differently: “Rats that only ate the food for one week and then made sick were less motivated to continue eating the food. The more rewarding the food, the stronger the experience that creates the automatic behavior.” (4:04) (Maybe this the research that spawned the popular idea that habits take 21 days to be established? This is a piece of lore that I’ve never been able to track down, along with the accompanying idea that most of us stop practicing a new habit we want to develop after 14 days – so that by January 15 your new year’s resolutions are all broken!)

Kessler notes that the most effective rewards are the ones that change your feelings. (not just your behavior), and highly palatable food can do this. The momentary pleasure from highly palatable food acts “as a substitute for other emotions, since it occupies working memory and the brain can only focus on a limited amount of stimuli at any given time.” We are compelled to attend to the most salient stimuli, especially in emergencies. But we can become focused on food stimuli and produce compulsive behavior. If you’re cued to the availability of highly palatable food, your brain will respond to it. (4:12)

Because of this, “highly rewarding food becomes reinforcing because we learn that it makes us feel better, motivating us to return and do the work necessary in order to feel better again. Reinforcement learning is a mechanism for the organism to learn which course of action will lead to positive outcomes” (4:15)

In addition to being rewarding in and of itself, eating highly palatable food also acts through negative reinforcement, which is the feeling of relief that you get when some aversive is removed. In this case, the aversive is the conflict between your desire for the food and your desire to NOT eat the food. “The desire for a highly palatable food conflicts with the desire for control over your cravings; as long as the food is there, the desire is there, and the conflict is there. So eating the food is the only way to remove the conflict, which also gives a negative reinforcement (relief) for eating it” (4:21) I found this very interesting – and insidious!

Kessler also discusses the concept of priming – “Sometimes just one taste of a food, a single dose, is enough to trigger conditioned hyper eating… Hence AA warns against a single drink, and chip makers dare you to stop at one” (4:32). With human dieters, breaking your diet just a little bit often prompts you to give up entirely and pig out. I think this is also why we have a category of small foods called “appetizers” – foods that are supposed to stimulate your appetite to eat more. This might be the same system that causes a reactive (aggressive) dog to react compulsively when they are triggered by the sight or sound of other dogs – especially dogs facing them and coming towards them. The sight of a distant dog – even the sound of tags or keys – can prime the dog to go through the whole display that usually results in the other dog going away.

Deprivation – focusing on not-eating makes you want to eat more. Feeling deprived merely increases the reward value of food, which can give way to indulgence and eating with abandon (4:47). (There is nothing like reading a book about over-eating to make you amazingly hungry, let me tell you!)

What about a solution to compulsive hyper eating? Kessler suggests (6:06) that we use the knowledge of how the brain works to our advantage. For one, he suggests that we find reward in eating smaller amounts of food – and in getting control over your eating. Gaining control should become a reward (but we’ll have to make it a habit). Keep in mind that “compulsive hyper eating is a biological challenge, not a character flaw; recovery is impossible until we stop viewing overeating as an absence of willpower”.

In addition to understanding why the brain works this way, people with compulsive hyper eating need to set themselves to think of eating smaller portions as a rewarding win in itself. “You might think of all of this as a game against a powerful opponent. You can’t expect to win every encounter, but with continued practice and training you can get a whole lot better” (6:23).

Kessler recommends using mental rehearsal, also known as visualization. “Mental rehearsal improves performance… the cognitive process allows you to envision your game plan before actually executing them. The motivational value comes from enhancing your sense of confidence, reducing your anxiety. Mental rehearsal reduces chaos in which compulsive hyper eating often occurs. Run through every step. Imagine yourself choosing not to take bread when it’s passed around the table…” He says you can implement your intentions this way: “You implant the intended response in your brain with ‘if-then’ propositions. ‘If I encounter this situation, then I’ll behave in this way’.”


At 6:07 Kessler warns that someone with compulsive hyper eating (and probably any other highly-rewarded behavior pattern) needs to be “recognizing it as a problem that needs to be managed, not one that can be completely cured” – every time we act and get rewarded, it becomes harder to act differently next time. “Effective treatment breaks the ‘cue – urge – reward’ habit cycle at the core of compulsive hyper eating.” I think that this is true for any well-rewarded behavior cycle. Keep in mind that “new learning can stick only when it generates a feeling of satisfaction. We can’t sustain a change in behavior if it leaves us hungry, unhappy, angry, or resentful” (6:08).

“Lapses are to be expected. Most of us are never fully cured of compulsive hyper eating. We remain vulnerable to the pull of old habits, although with time, and the rewards that accompany success, they do lose some of their power. With practice, we can find ways to use slips to our advantage, as tools to remind ourselves of where we might stumble, and reminder of the need for new learning.” Good advice for anyone who is dealing with changing a conditioned habit.

(To buy this book, see here.)

I just listened to a very fascinating radio podcast with David Kessler talking about his book, "The End of Overeating: Taking Control of the Insatiable American Appetite." There were a couple of things he talked about which made me think of behavior modification work. Here's a quote from his interview regarding dieting, but imagine that he's talking about a dog behavior modification program that relies on repressing the behavior that is unwanted by the owner:

"Diets don't work... diets can't work. Sure they can work - you have the old neural circuitry, and for 30 days, 60 days, 90 days you can deprive yourself; you'll lose the weight. But if you haven't laid down that new circuitry, that new learning on top of that old learning, you go back to your environment... what do you think is going to happen? You get cued again, you get bombarded with those cues - of course you're going to gain the weight back. You need to lay down new learning, new neural circuitry.... It's not simple, there's no miracle fixes, the old learning is always there. There are times when we're stressed, we're fatigued, and we're going to eat more."

Kessler studied "conditioned hyper eaters", people who cannot stop eating until all the food in front of them is gone. "Understand that the behavior is conditioned and driven. When we give the cue, we see an excess activation of the brain reward circuits. When they start to eat... the activation stays elevated and doesn't shut off until all the food is gone.... That arousal, it occupies all of working memory, it's what you're focused on. In order to relieve the anxiety, to get it out of working memory, what do you do? You eat it. And when you eat it, the next time you're cued, what's going to happen? Every time you're cued, and you engage in that cycle - cue, arousal, release - you strengthen the neural circuitry to ensure that you'll do it again and again and again. The behavior has become conditioned, and it also is driven."

Michael Krasny, the host of the interview, goes on to summarize what he read in Kessler's book: "You need to shut out the reward possibility, you need to change the whole reward value of the system."

I can't help thinking about dogs who engage in compulsive behaviors, or who seem to "can't help themselves" in their reactions to passersby, other dogs, the mailman, etc.

Listen to the whole interview on KQED's excellent Forum program.

I just read Whale Done! The Power of Positive Relationships. This is a book that uses animal training methods with humans. It is written by two authors of business management books – Ken Blanchard and Jim Ballard – and two Sea World orca trainers, Thad Lacinak and Chuck Tompkins. My impression is that it’s kind of a “gateway” book to the concepts of using positive reinforcement in human relationships. It’s a quick read – I read most of it in one afternoon – as it’s mostly a fictional story of one manager who learns how to apply positive reinforcement techniques to his business and family after a visit to SeaWorld.

Anyone who is already familiar with modern animal training techniques will find very familiar content here. I liked that they changed the “A” in the ABCs of behavior from “Antecedent, Behavior, Consequence” to “Activator” – “whatever gets performance going”. For business managers, they suggest that one of most common Activators are goals – the goals that managers set for their employees, and the goals that employees might have for themselves. They point out that often these are not the same goals, and that managers might then criticize their employers for not meeting their boss’s goals even when they don’t know what they are – so it’s important to make those goals clear. In other words, managers might be giving unclear cues, or cues that they believe are supposed to indicate one behavior but are interpreted as cuing another one.

I also liked the strong emphasis on redirection rather than simply ignoring bad behavior. Their version of the four kinds of Consequences is (1) No response, (2) Negative response, (3) Redirection, (4) Positive response. (Note that “negative” and “positive” are used in the common sense here of “bad” and “good”, not the behaviorists’ sense of “removed from the environment” and “added to the environment”.) “No response” is the same as “ignoring bad behavior” which we trainers recommend all the time, much to our clients’ consternation. Here they say that it “means just not giving what people do wrong all the scrutiny and energy we usually give it” (makes me think of how much energy goes into the anger I sometimes get into at my daughter when we’re late – again – and I feel compelled to go on and on with “What part of 'we have to leave in 5 minutes' don’t you understand?! When I say it’s time to go, you need to put on you shoes and stop playing and get ready to walk out the door….. etc., etc. etc.”. What a lot of energy I put into those blow-ups!)

When dealing with humans, for redirection they suggest (p. 34):
• Describe the error or problem… without blame
• Show its negative impact
• If appropriate, take the blame for not making the task clear
• Go over the task in detail and make sure it is clearly understood
• Express your continuing trust and confidence in the person

With animal training, “Go over the task... and make sure it is clearly understood” would be making sure that the correct response is well trained (highly reinforced) first, so that you can then cue it when it is wanted. When training a dog to sit instead of jumping up, you need to make sure that the dog understands that sitting will be rewarded – even in moments of excitement like the arrival of guests. When training a dog to look at an “intruding” dog without barking or lunging, you need to make sure that the dog knows that she can look without barking, first. Young children need clarification that "clearing the dishes" means putting them in the dishwasher, not just taking them off the table and leaving them on the counter; employees should understand the actual procedures they need to do.

Clicker trainers talk a lot about when it’s appropriate to change the criteria of what will get rewarded, and I liked that in Whale Done they avoid this jargon by saying that you don’t have to “wait for exactly right behavior before you respond positively”. They summarize this as “Praise progress. It’s a moving target”. Reward people’s progress: “catch them doing things better, if not exactly right, and praise progress. That way you set them up for success and build from there” (p. 37).

I think someone new to the concepts would probably like more concrete examples to follow things like “Praise people immediately. Be specific about what they did right or almost right. Share your positive feelings about what they did. Encourage them to keep up the good work” (p. 40). This reminded me a lot of one of my other favorite parenting books, How to Talk So Kids Will Listen and Listen So Kids Will Talk by Faber and Mazlich, and they have a lot of examples of this kind of reinforcement response in their book. Whale Done does include a few nice examples (p 64-66).

One of the nice points in the book is transferring your external positive reinforcement to an internal one that your “trainee” can give herself (this is the biggest criticism against using positive reinforcement techniques with humans, as in “Punished By Rewards” by Alfie Kohn, a book that is pretty much guaranteed to infuriate anyone who actually knows how to use positive reinforcement correctly, in particular how to raise criteria). I like some of the examples they gave here, like “I’ll bet it felt good when you finished that project before the deadline” (p. 56).

They also talk about finding appropriate reinforcers for your subject – which fortunately with humans we can inquire directly about. I’d want to hear more about how to distinguish between “short run and long run” ways of recognizing behavior – I’m guessing that praise (verbal recognition) might be short run, and offering a raise, more responsibilities, or other privileges are long-run ways for humans.

Whale Done does recognize that it’s hard to learn to notice good behavior. “…Through practice we’ve trained our attention to notice only what they do wrong. We have our eye out for the negative behavior” (p.87). This reminds me of George Shaller’s concept of “thousand-hour eyes” that a behaviorist develops after spending enough time observing her species of choice – being able to see the more unusual, the more subtle, features. Once you practice a lot of looking for “good” behavior, the easier it is to see it.

So I’d recommend this book to the business manager or parent who really needs to be introduced to the ideas of positive reinforcement. There is a “Whale Done Parenting” book which I’d love to read next; for now, for parents looking for more specific advice, I’d recommend the Faber/Mazlish book and Glenn Latham’s book, The Power of Positive Parenting.