Shaping (psychology)

Last updated

Shaping is a conditioning paradigm used primarily in the experimental analysis of behavior. The method used is differential reinforcement of successive approximations. It was introduced by B. F. Skinner [1] with pigeons and extended to dogs, dolphins, humans and other species. In shaping, the form of an existing response is gradually changed across successive trials towards a desired target behavior by reinforcing exact segments of behavior. Skinner's explanation of shaping was this:

Contents

We first give the bird food when it turns slightly in the direction of the spot from any part of the cage. This increases the frequency of such behavior. We then withhold reinforcement until a slight movement is made toward the spot. This again alters the general distribution of behavior without producing a new unit. We continue by reinforcing positions successively closer to the spot, then by reinforcing only when the head is moved slightly forward, and finally only when the beak actually makes contact with the spot. ... The original probability of the response in its final form is very low; in some cases it may even be zero. In this way we can build complicated operants which would never appear in the repertoire of the organism otherwise. By reinforcing a series of successive approximations, we bring a rare response to a very high probability in a short time. ... The total act of turning toward the spot from any point in the box, walking toward it, raising the head, and striking the spot may seem to be a functionally coherent unit of behavior; but it is constructed by a continual process of differential reinforcement from undifferentiated behavior, just as the sculptor shapes his figure from a lump of clay. [2]

Successive approximations

The successive approximations reinforced are increasingly accurate approximations of a response desired by a trainer, the "target" response. As training progresses the trainer stops reinforcing the less accurate approximations. For example, in training a rat to press a lever, the following successive approximations might be reinforced:

  1. simply turning toward the lever will be reinforced
  2. only moving toward the lever will be reinforced
  3. only moving to within a specified distance from the lever will be reinforced
  4. only touching the lever with any part of the body, such as the nose, will be reinforced
  5. only touching the lever with a specified paw will be reinforced
  6. only depressing the lever partially with the specified paw will be reinforced
  7. only depressing the lever completely with the specified paw will be reinforced

The trainer starts by reinforcing all behaviors in the first category, here turning toward the lever. When the animal regularly performs that response (turning), the trainer restricts reinforcement to responses in the second category (moving toward), then the third, and so on, progressing to each more accurate approximation as the animal learns the one currently reinforced. Thus, the response gradually approximates the desired behavior until finally the target response (lever pressing) is established. At first the rat is not likely to press the lever; in the end it presses rapidly.

Shaping sometimes fails. An oft-cited example is an attempt by Marian and Keller Breland (students of B.F. Skinner) to shape a pig and a raccoon to deposit a coin in a piggy bank, using food as the reinforcer. Instead of learning to deposit the coin, the pig began to root it into the ground, and the raccoon "washed" and rubbed the coins together. That is, the animals treated the coin the same way that they treated food items that they were preparing to eat, referred to as “food-getting” behaviors. In the case of the raccoon, it was able to learn to deposit one coin into the box to gain a food reward, but when the contingencies were changed such that two coins were required to gain the reward, the raccoon could not learn the new, more complex rule. After what could be characterized as expressions of frustration, the raccoon resorts to basic “food-getting” behaviors common to its species. These results show a limitation in the raccoon’s cognitive capacity to even conceive of the possibility that two coins could be exchanged for food, irrespective of existing auto-shaping contingencies. Since the Breland's observations were reported many other examples of untrained responses to natural stimuli have been reported; in many contexts, the stimuli are called "sign stimuli", and the related behaviors are called "sign tracking". [3] [4]

Practical applications

Shaping is used in training operant responses in lab animals, and in applied behavior analysis to change human or animal behaviors considered to be maladaptive or dysfunctional. It also plays an important role in commercial animal training. Shaping assists in "discrimination", which is the ability to tell the difference between stimuli that are and are not reinforced, and in "generalization", which is the application of a response learned in one situation to a different but similar situation. [5]

Shaping can also be used in a rehabilitation center. For example, training on parallel bars can approximate walking with a walker. [6] Or shaping can teach patients how to increase the time between bathroom visits.

Autoshaping

Autoshaping (sometimes called sign tracking) is any of a variety of experimental procedures used to study classical conditioning. In autoshaping, in contrast to shaping, the reward comes irrespective of the behavior of the animal. In its simplest form, autoshaping is very similar to Pavlov's salivary conditioning procedure using dogs. In Pavlov's best-known procedure, a short audible tone reliably preceded the presentation of food to dogs. The dogs naturally, unconditionally, salivated (unconditioned response) to the food (unconditioned stimulus) given to them, but through learning, conditionally, came to salivate (conditioned response) to the tone (conditioned stimulus) that predicted food. In auto-shaping, a light is reliably turned on shortly before animals are given food. The animals naturally, unconditionally, display consummatory reactions to the food given them, but through learning, conditionally, came to perform those same consummatory actions directed at the conditioned stimulus that predicts food.

Autoshaping provides an interesting conundrum for B.F. Skinner's assertion that one must employ shaping as a method for teaching a pigeon to peck a key. After all, if an animal can shape itself, why use the laborious process of shaping? Autoshaping also contradicts Skinner's principle of reinforcement. During autoshaping, food comes irrespective of the behavior of the animal. If reinforcement were occurring, random behaviors should increase in frequency because they should have been rewarded by random food. Nonetheless, key-pecking reliably develops in pigeons, [7] even if this behavior had never been rewarded.

But, the clearest evidence that auto-shaping is under Pavlovian and not Skinnerian control was found using the omission procedure. In that procedure, [8] food is normally scheduled for delivery following each presentation of a stimulus (often a flash of light), except in cases in which the animal actually performs a consummatory response to the stimulus, in which case food is withheld. Here, if the behavior were under instrumental control, the animal would stop attempting to consume the stimulus, as that behaviour is followed by the withholding of food. But, animals persist in attempting to consume the conditioned stimulus for thousands of trials [9] (a phenomenon known as negative automaintenance), unable to cease their behavioural response to the conditioned stimulus even when it prevents them from obtaining a reward.

See also

Related Research Articles

B. F. Skinner American psychologist and social philosopher (1904-1990)

Burrhus Frederic Skinner was an American psychologist, behaviorist, author, inventor, and social philosopher. He was a professor of psychology at Harvard University from 1958 until his retirement in 1974.

Operant conditioning is a type of associative learning process through which the strength of a behavior is modified by reinforcement or punishment. It is also a procedure that is used to bring about such learning.

Operant conditioning chamber

An operant conditioning chamber is a laboratory apparatus used to study animal behavior. The operant conditioning chamber was created by B. F. Skinner while he was a graduate student at Harvard University. It may have been inspired by Jerzy Konorski's studies. It is used to study both operant conditioning and classical conditioning.

Classical conditioning refers to a learning procedure in which a biologically potent stimulus is paired with a previously neutral stimulus. It also refers to the learning process that results from this pairing, through which the neutral stimulus comes to elicit a response that is usually similar to the one elicited by the potent stimulus. It was first studied by Ivan Pavlov in 1897.

Reinforcement A consequence applied that will strengthen an organisms future behavior

In behavioral psychology, reinforcement is a consequence applied that will strengthen an organism's future behavior whenever that behavior is preceded by a specific antecedent stimulus. This strengthening effect may be measured as a higher frequency of behavior, longer duration, greater magnitude, or shorter latency. There are two types of reinforcement, known as positive reinforcement and negative reinforcement; positive is where by a reward is offered on expression of the wanted behaviour and negative is taking away an undesirable element in the persons environment whenever the desired behaviour is achieved. Rewarding stimuli, which are associated with "wanting" and "liking" and appetitive behavior, function as positive reinforcers; the converse statement is also true: positive reinforcers provide a desirable stimulus. Reinforcement does not require an individual to consciously perceive an effect elicited by the stimulus. Thus, reinforcement occurs only if there is an observable strengthening in behavior. However, there is also negative reinforcement, which is characterized by taking away an undesirable stimulus. Changing someone's job might serve as a negative reinforcer to someone who suffers from back problems, i.e. Changing from a labourers job to an office position for instance.

The experimental analysis of behavior is school of thought in psychology founded on B. F. Skinner's philosophy of radical behaviorism and defines the basic principles used in applied behavior analysis. A central principle was the inductive reasoning data-driven examination of functional relations, as opposed to the kinds of hypothetico-deductive learning theory that had grown up in the comparative psychology of the 1920–1950 period. Skinner's approach was characterized by observation of measurable behavior which could be predicted and controlled. It owed its early success to the effectiveness of Skinner's procedures of operant conditioning, both in the laboratory and in behavior therapy.

Behaviorism is a systematic approach to understanding the behavior of humans and other animals. It assumes that behavior is either a reflex evoked by the pairing of certain antecedent stimuli in the environment, or a consequence of that individual's history, including especially reinforcement and punishment contingencies, together with the individual's current motivational state and controlling stimuli. Although behaviorists generally accept the important role of heredity in determining behavior, they focus primarily on environmental events.

Clicker training

Clicker training is a positive reinforcement animal training method based on a bridging stimulus in operant conditioning. The system uses conditioned reinforcers, which a trainer can deliver more quickly and more precisely than primary reinforcers such as food. The term "clicker" comes from a small metal cricket noisemaker adapted from a child's toy that the trainer uses to precisely mark the desired behavior. When training a new behavior, the clicker helps the animal to quickly identify the precise behavior that results in the treat. The technique is popular with dog trainers, but can be used for all kinds of domestic and wild animals and small children.

The law of effect is a psychology principle advanced by Edward Thorndike in 1898 on the matter of behavioral conditioning which states that "responses that produce a satisfying effect in a particular situation become more likely to occur again in that situation, and responses that produce a discomforting effect become less likely to occur again in that situation."

Motivational salience is a cognitive process and a form of attention that motivates or propels an individual's behavior towards or away from a particular object, perceived event or outcome. Motivational salience regulates the intensity of behaviors that facilitate the attainment of a particular goal, the amount of time and energy that an individual is willing to expend to attain a particular goal, and the amount of risk that an individual is willing to accept while working to attain a particular goal.

Animal training

Animal training is the act of teaching animals specific responses to specific conditions or stimuli. Training may be for purposes such as companionship, detection, protection, and entertainment. The type of training an animal receives will vary depending on the training method used, and the purpose for training the animal. For example, a seeing eye dog will be trained to achieve a different goal than a wild animal in a circus.

Extinction is a behavioral phenomenon observed in both operantly conditioned and classically conditioned behavior, which manifests itself by fading of non-reinforced conditioned response over time. When operant behavior that has been previously reinforced no longer produces reinforcing consequences the behavior gradually stops occurring. In classical conditioning, when a conditioned stimulus is presented alone, so that it no longer predicts the coming of the unconditioned stimulus, conditioned responding gradually stops. For example, after Pavlov's dog was conditioned to salivate at the sound of a metronome, it eventually stopped salivating to the metronome after the metronome had been sounded repeatedly but no food came. Many anxiety disorders such as post traumatic stress disorder are believed to reflect, at least in part, a failure to extinguish conditioned fear.

Instinctive drift

Instinctive drift, alternately known as instinctual drift, is the tendency of an animal to revert to unconscious and automatic behaviour that interferes with learned behaviour from operant conditioning. Instinctive drift was coined by Keller and Marian Breland, former students of B.F. Skinner at the University of Minnesota, describing the phenomenon as "a clear and utter failure of conditioning theory." B.F. Skinner was an American psychologist and father of operant conditioning, which is learning strategy that teaches the performance of an action either through reinforcement. It is through the association of the behaviour and the reward or consequence that follows that depicts whether an animal will maintain a behaviour, or if it will become extinct. Instinctive drift is a phenomenon where such conditioning erodes and an animal reverts to its natural behaviour.

In operant conditioning, punishment is any change in a human or animal's surroundings which, occurring after a given behavior or response, reduces the likelihood of that behavior occurring again in the future. As with reinforcement, it is the behavior, not the human/animal, that is punished. Whether a change is or is not punishing is determined by its effect on the rate that the behavior occurs, not by any "hostile" or aversive features of the change. For example, a painful stimulus which would act as a punisher for most people may actually reinforce some behaviors of masochistic individuals.

Errorless learning was an instructional design introduced by psychologist Charles Ferster in the 1950s as part of his studies on what would make the most effective learning environment. B. F. Skinner was also influential in developing the technique, noting that,

...errors are not necessary for learning to occur. Errors are not a function of learning or vice versa nor are they blamed on the learner. Errors are a function of poor analysis of behavior, a poorly designed shaping program, moving too fast from step to step in the program, and the lack of the prerequisite behavior necessary for success in the program.

In behavioral psychology, stimulus control is a phenomenon in operant conditioning that occurs when an organism behaves in one way in the presence of a given stimulus and another way in its absence. A stimulus that modifies behavior in this manner is either a discriminative stimulus (Sd) or stimulus delta (S-delta). Stimulus-based control of behavior occurs when the presence or absence of an Sd or S-delta controls the performance of a particular behavior. For example, the presence of a stop sign (S-delta) at a traffic intersection alerts the driver to stop driving and increases the probability that "braking" behavior will occur. Such behavior is said to be emitted because it does not force the behavior to occur since stimulus control is a direct result of historical reinforcement contingencies, as opposed to reflexive behavior that is said to be elicited through respondent conditioning.

Discrimination learning is defined in psychology as the ability to respond differently to different stimuli. This type of learning is used in studies regarding operant and classical conditioning. Operant conditioning involves the modification of a behavior by means of reinforcement or punishment. In this way, a discriminative stimulus will act as an indicator to when a behavior will persist and when it will not. Classical conditioning involves learning through association when two stimuli are paired together repeatedly. This conditioning demonstrates discrimination through specific micro-instances of reinforcement and non-reinforcement. This phenomenon is considered to be more advanced than learning styles such as generalization and yet simultaneously acts as a basic unit to learning as a whole. The complex and fundamental nature of discrimination learning allows for psychologists and researchers to perform more in-depth research that supports psychological advancements. Research on the basic principles underlying this learning style has their roots in neuropsychology sub-processes.

James “Jim” A. Dinsmoor was an influential experimental psychologist who published work in the field of the experimental analysis of behavior. He was born October 4, 1921, in Woburn, Massachusetts to Daniel and Jean Dinsmoor. He graduated with his bachelor's degree from Dartmouth College in 1943. Subsequently, he attended Columbia University in New York City, where he received his Master's and Ph.D. degrees under the mentorship of William N. Schoenfeld and Fred S. Keller. There, he was introduced to the work of B.F. Skinner, whose behavior analytic research inspired Dinsmoor to pursue a lifetime of research in conditioned responding.

The differential outcomes effect is a theory in behaviorism, a branch of psychology, that shows that a positive effect on accuracy occurs in discrimination learning between different stimuli when unique rewards are paired with each individual stimulus. The DOE was first demonstrated in 1970 by Milton Trapold on an experiment with rats. Rats were trained to discriminate between a clicker and a tone by pressing the left and right levers. Half of the rats were trained using the differential outcomes procedure, where the clicker was paired with sucrose and tone with food pellets. The remaining rats were trained with only sucrose or only food pellets. The rats trained with the differential outcomes procedure were significantly more accurate than those trained with only one type of reinforcement. Since then it has been established through a myriad of experiments that the Differential Outcome Effect exists in most species capable of learning.

Association in psychology refers to a mental connection between concepts, events, or mental states that usually stems from specific experiences. Associations are seen throughout several schools of thought in psychology including behaviorism, associationism, psychoanalysis, social psychology, and structuralism. The idea stems from Plato and Aristotle, especially with regard to the succession of memories, and it was carried on by philosophers such as John Locke, David Hume, David Hartley, and James Mill. It finds its place in modern psychology in such areas as memory, learning, and the study of neural pathways.

References

  1. Peterson, G.B. (2004) A day of great illumination: B.F. Skinner's discovery of shaping. Journal of the Experimental Analysis of Behavior, 82: 317–28
  2. Skinner, B.F. (1953). Science and human behavior. pp. 92–3. Oxford, England: Macmillan.
  3. Shettleworth, Sara J. (2010) "Cognition, Evolution, and Behavior" (2nd Ed) Oxford: New York
  4. Powell, R.; Symbaluk, D.; Honey, P. (2008). Introduction to Learning and Behavior. Cengage Learning. p. 430. ISBN   9780495595281 . Retrieved 2015-10-23.
  5. Barbara Engler: Personality Theories
  6. Miltenberger, R. (2012). Behavior modification, principles and procedures. (5th ed.). Wadsworth Publishing Company.
  7. Brown, P. & Jenkins, H.M. (1968). Auto-shaping of the pigeon's key peck. J. Exper. Analys. Behav. 11: 1–8.
  8. see Sheffield, 1965; Williams & Williams, 1969
  9. Killeen, Peter R. (2003). Complex dynamic processes in sign tracking with an omission contingency (negative automaintenance). Journal of Experimental Psychology. 29(1): 49-61