PageNeuringer1985JEP ABP 2
PageNeuringer1985JEP ABP 2
PageNeuringer1985JEP ABP 2
net/publication/232488201
Variability Is an Operant
CITATIONS READS
389 2,504
2 authors:
All content following this page was uploaded by Allen Neuringer on 25 June 2015.
Variability Is an Operant
S u z a n n e Page a n d Allen N e u r i n g e r
Reed College
Pigeons were rewarded if their pattern of eight pecks to left and right response
keys during the current trial differed from the patterns in each of the last n trials.
Experiments 1 and 2 compared Schwartz's (1980, 1982a) negative findings
(variability was not controlled by reinforcement) with the present positive results
and explained the difference. Experiment 3 manipulated n and found that the
pigeons generated highly variable patterns even when the current response
sequence had to differ from each of the last 50 sequences. Experiment 4
manipulated the number of responses per trial; variability increased with increasing
responses per trial, indicating that the pigeons were acting as quasi-random
generators. Experiment 5 showed that for high levels of variability to be engendered,
reinforcement had to be contingent on response variability. In a yoked condition,
where variability was permitted but not required, little response variability was
observed. Experiment 6 demonstrated stimulus control: Under red lights the
pigeons generated variable patterns, and under blue lights they repeated a
particular fixed pattern. We concluded that behavioral variability is an operant
dimension of behavior controlled by contingent reinforcement.
429
430 SUZANNE PAGE AND ALLEN NEURINGER
reinforced novel behaviors in two porpoises. nate variability in the work of Bryant and
Reinforcers were delivered for any movement Church. Here, too, the argument is that
that in the opinion of the trainer did not variability was not reinforced (variability is
constitute normal swimming motions and not an operant) but the schedules of rein-
that had not been previously reinforced. Dif- forcement engendered variability as a respon-
ferent types of behaviors emerged, including dent by-product. The control experiments
several that had never before been observed necessary to distinguish between the operant
in animals of that species. In a more con- and respondent alternatives were not per-
trolled setting, Schoenfeld, Harris, and Farmer formed.
(1966) delivered rewards to rats only if suc- The most serious evidence against vari-
cessive interresponse times fell into different ability as an operant dimension of behavior
temporal class intervals. This procedure pro- comes from Schwartz (1980, 1982a). Pigeons
duced a low level of variability in the rat's were required to generate on two response
rate of bar pressing. In the most sophisticated keys a pattern of eight responses that differed
and demanding experiment to date, Blough from the last sequence of eight responses. A
(1966) reinforced the least frequent of a set 5 × 5 matrix of lights provided the pigeons
of interresponse times and obtained interre- with continual feedback concerning their dis-
sponse time distributions in three pigeons tribution of pecks. The light started in the
that approximated a random exponential dis- upper left-hand corner; each left-key peck (L)
tribution. Similarly, Bryant and Church moved the light one column to the right and
(1974) rewarded rats 75% of the time for each right-key peck (R) moved the light down
alternating between two levers and 25% of one row. In preliminary training, pigeons
the time for repeating a response on the same were simply required to move the light from
lever and found that for 3 of 4 subjects, the upper left to lower right. There was no re-
resulting choice behaviors could not be dis- quirement to vary the pattern. If the birds
tinguished from chance. People have also responded more than four times on either
been rewarded for behaving variably in more key (thereby moving the light off the matrix),
natural environments (Holman, Goetz, & the 'trial terminated without reinforcement.
Baer, 1977). The pigeons were highly successful, obtaining
Although the above studies indicate that 70% to 90% of available reinforcers by gen-
variability can be reinforced, the conclusion erating repetitive and energy-conserving pat-
is not secure. When the response in question terns (e.g., LLLLRRRR).In the experiments of
is a complex behavior, such as the responses most concern to the present work (Schwartz,
studied by Pryor et al. (1969) and Holman 1980, Experiment 4; 1982a, Experiment 1),
et al. (1977), the observer plays an important in addition to requiring four pecks on each
role in defining novel response, and the novelty key, the schedule demanded that each eight-
observed might be as much a function of the response sequence differ from the immediately
observer as of the subject. Moreover, Schwartz preceding sequence. Now, when the pigeons
(1982b) has argued that the Pryor et al. had to vary their response patterns to be
findings with porpoises can be attributed to rewarded, only 40% of the available reinforcers
the effects of repeated extinction and recon- were earned. Number of different sequences
ditioning, with extinction occurring increas- emitted, an index of learned variability, did
ingly rapidly in progressive instances, and not appreciably increase.
not to the direct reinforcement of novelty. The Schwartz findings, though apparently
Because extinction increases variability (e.g., robust, appeared to conflict with those of
Antonitis, 1951), the variability observed may Pryor et al. (1969), Blough (1966), and Bryant
have been a by-product of the reinforcement and Church (1974). Furthermore, Neuringer
schedule. A similar argument can be made (1984, 1985) showed that when provided with
with respect to both Blough (1966) and Bryant feedback, people learned to generate highly
and Church (1974). Although unlikely, it is variable response sequences, indeed sequences
possible that the reinforcement schedules so variable that they could be described as
somehow elicited interresponse time variabil- random. To explore the apparent disagree-
ity in the Blough experiment and stay-alter- ment between Schwartz's work and these
OPERANT VARIABILITY 431
four responses had to occur on each key, a condition each subject was stable over 5 sessions, or 22 to 31
analogous to Schwartz (1980, Experiment 4; 1982a, sessions.
Experiment 1). Return to variability: Lag 5. The Lag 5 variability
Variability: Lags 1 and 5. Each trial consisted of contingencies used in the first phase were reinstated for
eight pecks distributed in any manner over the two keys; 6 to 20 sessions. The eight responses could again be
each session comprised 50 trials. At the start of each distributed in any manner; that is, the requirement of
trial, both keys were illuminated with white light. A peck no more than four responses per key was removed. This
to either key immediately darkened both key lights and variability phase differed from the original in that during
initiated a 0.5-s interpeck interval. Pecks during the the first five trials of each session, the bird's sequences
interpeck interval reset the interval but were not counted had to differ from the last five sequences, including
and had no other consequence. The eighth peck to a sequences emitted during .the last five trials from the
lighted key terminated the trial with either a reinforcer previous session. That is, the comparison set was made
(3.5 s of access to mixed grain) or a 3.5-s time-out, continuous over sessions so that, for example, the fourth
during which the keys were darkened but the house light trial in a session would be reinforced only if the fourth
remained illuminated. Pecks during the time-out reset sequence differed from the preceding three sequences of
the 3.5-s timer but had no other consequence. The only that session and the last two sequences of the previous
difference between time-out and interpeck interval was session.
the duration of these events. Under the Lag 1 condition, Owing to a programming error, the timing of experi-
whether a trial ended in reinforcement or time-out mental events (reinforcement, time-out, and interpeck
depended on whether the sequence of eight pecks on that interval) was altered when the comparison set was made
trial differed from the last sequence of eight pecks, that continuous over sessions: All timed events were lengthened
is, whether the response pattern on trial n differed from by a factor of 10/6; reinforcement and time-out were
trial n - 1. If the sequences on the two trials differed, a 5.83 s and interpeck interval was 0.83 s. The effect of
reinforcer was given; if the sequences were identical, these changes was examined in this experiment and more
time-out occurred. Thus, for example, if on Trial 10 the directly in Experiment 5, below, which shows that these
bird pecked LRLLLRRL, Trial 11 would be reinforced timing parameter changes had no discernible effect on
unless the bird repeated that sequence. The three exper- sequence variability.
imentally naive birds (Nos. 28, 68, and 71) received 5 to After completion of the research with pigeons, the
13 sessions of this Lag 1 training. Because most trials VIC-20 computer's random number generator was used
ended with a reinforcer (the pigeons successfully varied to generate left and right responses under both V and
their sequences), greater variability was demanded by VC conditions identical to those experienced by the
increasing the look-back value to Lag 5. Now, for rein- pigeons. This simulation by a random generator permitted
forcement, the response sequence on trial n had to differ comparison of the pigeon's sequence variability with a
from each of the sequences on trials n - 1 through n - 5, random standard. ~
inclusive; otherwise time-out occurred. The one previously
experienced bird (No. 58) began the experiment with
Results
Lag 5 contingencies. Subjects received 15 to 18 sessions
of Lag 5 until the percentage of reinforced trials remained Two basic measures were derived from
stable over at least 5 sessions. The matrix lights were not
S c h w a r t z (1980, 1982a). F i r s t , p e r c e n t a g e o f
used under these variability procedures.
r e i n f o r c e d trials per session ( n u m b e r o f rein-
Variability-plus-constraint: Lag 1. Except where noted,
the contingencies and parameters here were the same as f o r c e d t r i a l s d i v i d e d b y t o t a l trials) i n d i c a t e d
in the Lag 1 variability condition. As in Schwartz (1980, how often the pigeons met the contingency
Experiment 4; 1982a, Experiment 1), to be reinforced, r e q u i r e m e n t s , o r t h e p r o b a b i l i t y t h a t t h e se-
the birds had to peck exactly four times on each key
quence was correct. The second measure,
with a sequence that differed from the last trial. Thus,
there were two ways for a subject to produce time-out: w h i c h p r o v i d e d a m o r e direct i n d e x o f vari-
(a) Respond four times on the left and four times on the ability, w a s t h e p e r c e n t a g e o f d i f f e r e n t se-
right with a sequence that was identical to the last quences per session (the number of distinct
sequence, or (b) respond more than four times on either eight-response patterns divided by the total
of the keys. The major difference between the V and the
n u m b e r o f trials). A s e q u e n c e w a s t e r m e d
VC conditions was that the eight responses could be
distributed in any manner under the former schedule, distinct i f it d i f f e r e d i n a n y way f r o m all
whereas exactly four responses per key were required p r e v i o u s s e q u e n c e s in a g i v e n s e s s i o n . N o t e
under the latter. that subjects could demonstrate mastery of
The 5 X 5 cue light matrix functioned as in Schwartz. the contingency requirement (percentage of
At the start of each trial, the top left matrix light was
illuminated. A peck to the left key darkened that light r e i n f o r c e d trials could be very high) even
and lit the one to its right. Pecks to the right key darkened
the presently illuminated light and lit the one immediately
below it. A fifth peck to either key moved the matrix Computer-based random number generators are often
light off the board, darkening the currently illuminated referred to as pseudorandom or quasi random because
light and initiating time-out. The VC procedure was the output is generated from an equation. For ease of
continued until the percentage of reinforced trials for presentation we shall refer simply to the random generator.
OPERANT VARIABILITY 433
four responses on each key), whereas 256 Experiment 2" Exact Replication
sequences were reinforceable under V (all of Schwartz
eight-response sequences); and there were two
ways to commit an error under VC (repeat Parameters differed among the variability-
the last sequence or emit more than four plus-constraint and variability procedures in
responses on a given key), whereas only the Experiment l and the Schwartz procedures.
former error was possible under V. The present experiment therefore attempted
The fact that the simulating random gen- to repeat the present Experiment 1 with
erator also was almost always correct under parameters in both V and VC conditions as
V but only infrequently correct under VC close as possible to Schwartz (1982a, Exper-
helps to explain the obtained results. The iment l). This provided a conclusive test of
random generator was not responding to the whether the four-response-per-key constraint
matrix lights. Furthermore, a random gener- was responsible for the low frequencies of
ator would be expected to gain slightly more reinforcement under Schwartz.
reinforcers under a Lag 1 contingency (VC)
than under a Lag 5 contingency (V). Therefore Method
it is unlikely that either lights or lag can
account for the different results under V
Subjects and Apparatus
versus VC. The remaining possible explana- The subjects and apparatus were the same as in
tions involve (a) the difference in the numbers Experiment 1.
of reinforceable sequences and (b) a related
factor, the different ways to make errors PrOcedure
under the two conditions. If a pigeon re- Variabilityplus constraint. The procedurewas identical
sponded randomly under the VC contingen- to VC in Experiment I (eight responses per trial, four
OPERANT VARIABILITY 435
Results
• '#28
• ÷58
quired to vary their sequences. In his second
70 • ÷68 experiment (Schwartz, 1982a), birds again
Q #71
received prior training on the sequencing task
(although the duration of this training was
60
not stated). In the present experiments, pi-
geons had to respond variably (V condition
in Experiment 1) before being placed under
50
the Schwartz contingencies (VC). Second,
"O Schwartz continued his experiments for more
c 40
sessions than we did. Although there was no
indication that variability under V conditions
O. decreased with training (see Experiment 5,
"O below, for the opposite conclusion), it is pos-
0 . 30 er
O # sible that with continued training under VC,
lb.
O response stereotypies would have developed.
20 e"
om
lag requirement was increased to 5: For reinforcement, the broken line shows a random generator
sequences had to differ from those in each of the last 5 simulation under identical conditions. From
trials. The two experimentally sophisticated pigeons began
the experiment at this Lag 5 value. There followed 10 to
Lags 5 through 25, more than 85% of the
21 sessions under Lag 10, 8 to 25 sessions under Lag 15, pigeons" sequences met the variability re-
10 to 38 sessions under Lag 25 (No. 59 remained at Lag quirements and were reinforced. At Lag 50,
25 until the end of the experiment because its performance there was a decrease to 67%. This same
never reached stability), and (for the 3 remaining subjects)
23 to 45 sessions under Lag 50. Throughout the experi-
decrease in percentage reinforced was seen in
ment, the lag value was changed only after a subject's the random generator's simulated data. Thus,
percentage of reinforced trials had become stable over at the pigeon's data again paralleled the data of
least 5 sessions. Midway in the Lag 10 condition, the a random generator.
procedure was changed (as described in Experiment 1)
so that the comparison trials included the final trials of
To obtain the high frequencies of rein-
the previous session. Thus, for example, under the Lag forcement shown in Figure 5, the pigeons
50 condition, if the subject were responding on its 1 lth must have generated highly variable response
trial in a session, for a reward to be presented, the sequences. One index of this high variability
current sequence had to differ from the 10 trials already is shown in Figure 6. As the lag requirement
completed in the present session and the last 40 trials of
the previous session. Because of the same programming increased from 5 to 25, the percentage of
error described in Experiment 1, from the midpoint of sequences that differed from all previously
Lag 10 through the end of the experiment, all timed emitted sequences in a session increased from
events were increased by a factor o f 10/6: Reinforcement 66% to 87%. As discussed above, to maintain
and time-out were 5.83 s rather than the original 3.5 s,
and interpeck interval was 0.83 s rather than the orig-
a high frequency of reinforcement under a
inal 0.5 s. low lag requirement, the bird did not have
to emit very many different sequences. As
Results and Discussion the length of the look back increased, however,
increasingly greater numbers of different se-
Figure 5 shows average percentage of rein- quences were demanded. Sensitivity to these
forced trials over the last five sessions at each changing requirements is indicated by the
lag, or look-back value. The solid line con- increasing function from Lag 5 to Lag 25.
nects the medians of the four pigeons, and The small decrease to 81% different at the
100
90
E
o
f- 80
70
O • ,I,59
tO • ,/,61 •
L. 60 • ,~70
0
a. & @ ÷Ta
~ MED
5O
1" I I I I Ii I
5 10 15 25 50
LAG
Figure 5. Percentage of reinforced trials per session as a function of lag (look back) value. (Filled points =
averages over final five sessions at each lag value for each subject; solid line = medians; broken line =
performance o f a simulating random generator under identical conditions.)
438 SUZANNE PAGE AND ALLEN NEURINGER
100
8o
0L--
• ÷61
60 • • ÷70
O. • • '/'73
• : : MED
Q
5O1"
I t a t I!, i
5 10 15 25 50
LAG
Figure 6. Percentage of different sequences per session as a function of lag value. (Filled points = averages
over final five sessions at each lag value for each subject; solid line = medians; broken line = performance
of a simulating random generator under identical conditions.
~ .951 ~~'~" ~
i .85t ~ / :// ~Ul2
• U3
, I , ' II i
5 10 15 25 50
LAG
Figure 7. Average sequence variability as functions of lag. (Each of the lines connects the medians of the
pigeons' average performances over final five sessions of each lag value. Ut = uncertainty for responses
taken one at a time; /-/2 = uncertainty for responses considered in pairs; /-/3 = uncertainty for response
triplets. Open diamonds = analogous data for simulating random number generator, where, because the
three U values were almost identical, a single point indicates the values.)
Once again, there was a slight tendency for paralleled that of the simulated random func-
variability to decrease at Lag 50. The closeness tion; this finding was again consistent with
of the three U functions to one another the hypothesis that the pigeons were gener-
indicates high variability and absence of ating quasi-random sequences. Although the
higher order biases or stereotypies. (Note that variability of the computer-based random
we also examined U4 through Us, and these generator was, of course, unaffected by the
contained similar information to that shown reinforcement schedule, the pigeons' vari-
in U1 through Us. ability appeared to be controlled by the re-
Two possibly confounding influences were inforcers. When the schedule demanded rel-
present in Experiment 3. First, the timing atively little variability (Lag 5), variability
parameter changed during the Lag 10 con- was relatively low. As the variability require-
dition, and therefore the Lag 15 through Lag ment increased to Lag 25, so too did vari-
50 conditions contained longer reinforcement ability of performance. However, at Lag 50,
and time-out durations than the Lag 5 and when the obtained frequency of reinforcement
Lag 10 conditions. However, Experiment 1 decreased despite random performance (as
showed that there was no statistically signifi- indicated by the simulating random genera-
cant effect of these timing differences, a result tor), again the birds' variability decreased.
supported in Experiment 5, below. Second, Thus, high variability was engendered only
the lag requirements increased from low val- when it was differentially reinforced.
ues to high, and therefore the form of the
obtained function may partly be due to the Experiment 4: Quasi-Random Versus
order of experience. We thought that pigeons Memory Strategies: Number of
could not tolerate high lag requirements be-
Responses as Parameter
fore they experienced training under lower
requirements, a hypothesis shown to be in- The present experiment asked how the
correct in Experiment 5. The general form pigeons generated their variable response se-
of the subjects' percentage reinforced function quences: What mechanism or strategy ac-
440 SUZANNE PAGE AND ALLEN NEURINGER
counts for such highly variable performance? under eight responses per trial (identical to the Lag 5
The previous experiments alluded to one variability conditions in Experiments 1 and 3 where
there were 256 possible sequences); 9 to 23 sessions
possible strategy, that o f a quasi-random gen- under four responses per trial (16 possible sequences);
erator, but alternative strategies involving re- and another 6 to 9 sessions under eight responses per
m e m b e r i n g previous sequences would also trial. The reinforcement and time-out intervals were 5.83
do. For example, a subject could learn a long s throughout, and interpeck interval was 0.83 s.
r a n d o m response sequence (see Popper, 1968)
or utilize a rational system (e.g., first emit 8 Results and Discussion
left responses; then 7 left and l right; then 6 Figure 8 shows the average percentage o f
left, l right, and 1 left; etc.). It is not here the 50 available rewards obtained over the
being suggested that pigeons can c o u n t in last five sessions at each responses-per-trial
binary but rather that their behavior could value. The eight-response value represents
be described as systematic. A n y systematic the mean o f the two eight-response phases,
strategy would involve a rather large m e m o r y which were statistically indistinguishable from
load, for the pigeon would have to r e m e m b e r one another. For all subjects, as n u m b e r o f
where it was in its system. The present ex- responses per sequence increased, percentage
periment attempted to contrast the quasi- o f reinforced trials increased monotonically.
r a n d o m and m e m o r y hypotheses in the fol- A n analysis o f variance with repeated mea-
lowing way. If the n u m b e r o f responses re- sures showed an overall significant difference
quired for each sequence were increased, a m o n g the three conditions, F(2, 6) = 38.21,
performance based on a m e m o r y strategy p < .001, and analytical pairwise comparisons
should be adversely affected, for it is easier showed that each condition differed from
to r e m e m b e r a four-response sequence than every other: four versus eight responses, F(1,
an eight-response sequence. O n the other
hand, if the bird were acting as a quasi-
r a n d o m generator, success rates should im- 1013
prove with increasing responses per trial, ,O,.... i ot"'~O
because by the laws o f chance, a r a n d o m
generator would be more likely to repeat 913
sequences comprising four responses (1
chance in 16 under Lag 1) than eight respon- "~
ses (1 chance in 256). Thus, the m e m o r y and O 813
quasi-random generator hypotheses make op- //A ,
posite predictions. If the subject's rewards O
increased with increasing responses per trial, .~r" 70
the quasi-random hypothesis would be sup- I~.
ported; if the rewards decreased with increas-
ec. -, 60
ing responses per trial, a m e m o r y strategy
O
would be supported.
0L_ ~'- ,~ RND
50 • ÷28
Method a. • ÷58
• ÷68
Subjects and Apparatus 40 • ÷71
= - MED
The subjects and apparatus were the same as in
Experiment 1.
•'c,~ j o t I
4 6 8
Procedure
The procedure was identical to that in Experiment 3,
Number of Responses
except the lag requirement was kept constant at Lag 5 Figure 8. Percentage of reinforced trials as a function of
and the number of responses per trial, or sequence length, number of responses per trial. (Filled points = averages
varied in an ABCB format. The pigeons were first given over final five sessions at each condition for each of the
24 to 29 sessions under a six-responses-per-trial condition pigeons; solid line = medians; broken line = data from
(there were 64 possible sequences); then 12 to 38 sessions simulating random generator under identical conditions.)
OPERANT VARIABILITY 441
diameter Gerbrands response keys with their centers 7.5 1, the duration of events (reinforcement, time-out, and
cm from each other and from the side walls and 21.5 cm interpeck interval) was shorter (by a factor of 10/6) in
above the mesh floor. Keys could be transilluminated yoked-VR than in either the preceding or the following
with 7.5-W blue bulbs. A response registered when Lag 50 phases. The reinforcer and time-out were 3.5 s
applied force was withdrawn. The middle of the three in yoked-VR, as opposed to 5.83 s in Lag 50. The
keys was covered with black tape and could not be interpeck interval was 0.83 s in Lag 50 phases but 0.5 s
operated. Directly below this middle key was a round in the yoked-VR. To compare directly the effects of the
hopper opening, 5 cm in diameter, with its midpoint l0 different time values on sequence variability, after each
cm from the floor, through which a Gerbrands magazine pigeon had reached stability on Lag 50 (second A phase
could provide mixed pigeon grain reinforcement illumi- containing the long times), the durations of the reinforcer,
nated with a 7.5-W white bulb. House light was provided time-out, and interpeck interval were changed to those
by two 7.5°W white bulbs above the wire mesh ceiling. under yoked-VR with everything else held constant, thus
As described in Experiment l, VIC-20 computers con- permitting comparison of responding under Lag 50 long
trolled the experiment. Each pigeon was arbitrarily as- times with Lag 50 short times. After stable performances
signed to one of the two experimental chambers. were reached in 10 to 20 sessions, the yoked-VR contin-
gencies were reinstated for 17 to 32 sessions, thereby
permitting comparison of Lag 50 and yoked-VR when
Procedure all time parameters were identical. Schedules of reinforcers
The four subjects were first given autoshaping training and time-outs in this second yoked-VR phase were
as described in Experiment 1. The experimental procedure derived from performances of each pigeon in the last 6
then followed an ABAA'B design, with A and A' repre- sessions of the A' Lag 50 (short times) phase.
senting Lag 50 variability contingencies and B representing
a yoked variable ratio (yoked-VR) contingency in which Results and Discussion
variable sequences were not required. The variability
procedure was identical to that in Experiment 1 except T h e m a i n results w e r e t h a t (a) v a r i a b i l i t y
as follows. A Lag 50 requirement was present from the was significantly h i g h e r u n d e r L a g 50 t h a n
outset. As in the latter phases of Experiment 1, the yoked-VR, thereby demonstrating that the
comparison criterion was continuous over sessions. For
example, for Trial 10 to be reinforced, the sequence of variability depended on contingent reinforce-
responses in that trial had to differ from each of the m e n t ; (b) e x p e r i m e n t a l l y naive pigeons p l a c e d
sequences in the 9 previous trials of the current session d i r e c t l y o n t h e L a g 50 s c h e d u l e v e r y q u i c k l y
and the last 41 completed trials of the preceding session. l e a r n e d to vary, t h e r e b y i n d i c a t i n g t h a t vari-
The initial Lag 50 phase continued for 26 to 38 sessions
or until each pigeon maintained a stable percentage of a b i l i t y was easy to c o n d i t i o n ; a n d (c) t h e 10/
reinforced trials for 5 or more sessions. Throughout the 6 d i f f e r e n c e s in reinforcer, t i m e - o u t , a n d in-
present experiment, sessions terminated after the 50th terpeck interval times had no discernible
reinforcer, or after 100 trials, whichever occurred first. effect o n p e r f o r m a n c e , t h e r e b y i n d i c a t i n g t h e
At the start of the B phase, the contingencies of robustness of the reinforcement-of-variability
reinforcement were changed so that the pigeons were
reinforced on a yoked-variable ratio schedule derived effect.
from their individual performances under Lag 50. Under T h e left t w o b a r s o f F i g u r e 10 s h o w m e a n
this yoked-VR, eight responses again constituted a trial, p e r c e n t a g e s o f d i f f e r e n t s e q u e n c e s p e r session
and trials were again sometimes followed by grain rein- for t h e first five ( o p e n bar) a n d last five
forcers and sometimes by time-out, but reinforcement ( s t r i p e d bar) sessions u n d e r t h e first L a g 50
and time-out presentation were now independent of
sequence variability. Each pigeon's last 6 sessions under (long t i m e s ) s c h e d u l e . O v e r t h e first five ses-
the Lag 50 variability contingencies were used to create sions, m o r e t h a n 50% o f s e q u e n c e s differed
its schedule of reinforcement under yoked-VR. Thus, f r o m all p r e v i o u s s e q u e n c e s in t h e s a m e
each subject was yoked to itself, and the schedule of session, a n d this v a l u e i n c r e a s e d to m o r e
reinforced and nonreinforced trials under yoked-VR
t h a n 75% by t h e last five sessions. By t h e e n d
replicated the pattern of reinforcers and time-outs obtained
under Lag 50 variability. The yoked reinforcement sched- o f this L a g 50 phase, t h e p i g e o n s w e r e b e i n g
ule lasted for 6 consecutive sessions and then was repeated. r e w a r d e d after a p p r o x i m a t e l y 70% o f t h e i r
To illustrate, if Subject 44 had been rewarded after Trials trials, an i n c r e a s e f r o m 50% d u r i n g t h e first
2, 5, 6, and 8, and so on, in the last session under the five sessions. T h e i n c r e a s e s f r o m first to last
Lag 50 condition, then in yoked-VR Sessions 6, 12, 18,
and so on, Subject 44 would be rewarded after Trials 2, five sessions in b o t h p e r c e n t a g e different trials,
5, 6, and 8, and so on, regardless of which eight-response t(3) -- 6.68, p < .0 l, a n d p e r c e n t a g e r e i n f o r c e d
sequence was emitted. Trials 1, 3, 4, and 7, and so on, trials, t(3) -- 5. l l, p < .025, w e r e statistically
would be terminated by time-out. The yoked-VR contin- significant.
gencies continued until performance was stable, from 24
to 31 sessions, whereupon Lag 50 variability contingencies T h e s e c o n d set o f bars, r e p r e s e n t i n g y o k e d -
were reinstated and maintained for 17 or 18 sessions. V R , s h o w s t h a t w h e n the v a r i a b i l i t y c o n t i n -
Due to the programming error described in Experiment gencies were removed, percentage of different
OPERANT VARIABILITY 443
sequences fell immediately until fewer than last five sessions under this phase. Because of
20% of the sequences were different from the differences in timing values (long times
previous sequences in the session. The differ- in Lag 50 and short times in yoked-VR), the
ence between the last five sessions of the Lag observed effects might have been confounded.
50 variability contingencies and the last five The reinforcement, time-out, and interpeck
sessions of the yoked-VR was significant, interval times were therefore changed under
t(3) = 7.46, p < .005. Lag 50 so that these times were now identical
Upon reintroduction of the Lag 50 contin- to those under yoked-VR. The third and
gencies, percentages of different sequences fourth bars above the replicated Lag 50 show
rose immediately. The leftmost of the four percentage different sequences during the first
bars above the repeated Lag 50 condition five and last five sessions under short time
shows percentage different during the first values. There was essentially no change in
five sessions of Lag 50 following yoked-VR, percentage of different sequences due to the
and the second of the four bars shows the different time values.
Figure 10. Percentage of different sequences per session under one condition where each sequence had to
differ from the previous 50 sequences for reinforcement(Lag 50) and another condition where reinforcements
were given independently of response variability (yoked VR). (Open bars = first five sessions of each
condition; striped bars = final five sessions. L = long timing values; S = short times. Each point = 1
pigeon'sarithmetic averageperformanceover fivesessions; bars = medians of the 4 pigeons'performances.)
444 SUZANNE PAGE AND ALLEN NEURINGER
Upon return to the yoked-VR condition, Lag 50 and yoked-VR, t(3)= 2.493, p =
variability of responding again decreased im- .0873, was due to the large spread of the
mediately, and once again the difference be- individual subjects' data under the yoked-VR
tween the last five sessions of Lag 50 (now condition. When the schedule demanded high
with short times) and last five sessions of variability (Lag 50), all subjects emitted very
yoked-VR (with identical short times) was few repetitions of any given sequence; when
significant, t(3) = 6.77 l, p < .01. the schedule permitted variability but did
The fact that response variability depended not require it (yoked-VR), there were large
on contingent reinforcement is shown in Fig- intersubject differences. These same patterns
ure I l by the percentage of modal sequences, of modal frequencies were replicated with
defined as the single sequence emitted more return to Lag 50 (only the short time phase
frequently than any other in a session. The of Lag 50 is shown in the figure) and then to
ordinate shows the percentage of trials per yoked-VR, with the difference during the last
session in which the modal sequence oc- five sessions of these conditions being statis-
curred. By the last five sessions of the first tically significant, t(3) = 3.46, p < .05. In al-
phase of Lag 50, the modal sequences ac- most all cases, the sequence defined as modal
counted for about 4% of the sequences emit- under yoked-VR represented exclusive re-
ted per session. On the other hand, by the sponding on one or the other key (e.g., eight
end of the first yoked-VR phase, modal se- left responses or eight right responses). Be-
quences accounted for almost 50% of the cause reinforcement did not depend on any
sequences. Absence of significance between particular sequence, the final behavior was
Figure 11. Percentage of modal sequences per session (number of trials in which the most common
pattern occurred divided by the total number of trials) as a function of Lag 50 versusyoked variable ratio
(yoked-VR)conditions.
OPERANT VARIABILITY 445
\
example from each of the 4 pigeons]). .6C
.I3
Average U values, an index of overall re-
.5C
~'g?,,,
sponse variability as discussed above, are > L:Last
stereotyped responding,. Following an informal exploration The key light colors signaling V and S were then
of the effects of parameter manipulations, the second reversed, with red key lights now associated with V and
phase attempted to equalize the number of responses per blue key lights with S. No other changes were made. Two
trial in the two conditions and to generate approximately pigeons (Nos. 29 and 38) began the first session of
equal and intermediate levels of percentage of reinforced reversed color in the S component, and the other two
trials. A reversal of stimulus conditions followed. began in V. There were 18 to 24 sessions of reversal.
Phase 1: Acquisition of stimulus control. Following
autoshaping, the four pigeons were put on a multiple Results
variability-stereotypy schedule in which the two com-
ponents alternated after every 10th reinforcement. In the F i g u r e 13 s h o w s p e r c e n t a g e s o f r e i n f o r c e d
variability component, both key lights were blue, and
trials in variability a n d s t e r e o t y p y c o m p o n e n t s
subjects had to meet a Lag 5 variability criterion identical
to that described in Experiment 1, where reinforcement s e p a r a t e l y for 1 subject ( N o . 31) d u r i n g e a c h
was contingent on a sequence of eight pecks on the two session o f Phases 1 a n d 3. T h e p e r f o r m a n c e
keys that differed from each of the preceding five se- s h o w n is r e p r e s e n t a t i v e o f all birds. D u r i n g
quences. The Lag 5 criterion was continuous with respect a c q u i s i t i o n o f p e r f o r m a n c e u n d e r t h e first
to other V components in the same session and across
sessions. m u l t i p l e s c h e d u l e , s h o w n in t h e left panel,
In the stereotypy (S) component, both key lights were all b i r d s initially r e c e i v e d a h i g h e r p e r c e n t a g e
red and the pigeons had to emit an arbitrarily defined o f r e i n f o r c e m e n t s in t h e v a r i a b i l i t y c o m p o -
three-response sequence, namely, ERR, in that order. The n e n t (eight responses, L a g 5) t h a n in t h e
first two correct responses in the sequence produced the
s t e r e o t y p y c o m p o n e n t ( t h r e e responses). O v e r
same interpeck interval as in the V component, and the
third correct response produced the reinforcer. An error sessions, p e r f o r m a n c e s in b o t h c o m p o n e n t s
during an S trial (e.g., if the second response were to the i m p r o v e d i n a c c u r a c y , i n d i c a t i n g t h a t all sub-
left key) immediately produced the same time-out as in j e c t s a c q u i r e d t h e v a r i a b i l i t y - s t e r e o t y p y dis-
V and reset the sequence to the beginning. Whereas the crimination.
time-out in V occurred only after the eighth response,
the time-out in S immediately followed any incorrect F i g u r e 14 s h o w s a v e r a g e U - v a l u e m e a -
response in the sequence. s u r e s - - U ~ s h o w i n g v a r i a b i l i t y for r e s p o n s e s
The V and S components continued to alternate after t a k e n o n e at a t i m e , U2 for r e s p o n s e s t a k e n
every 10th reinforcement until the bird earned a total of t w o at a t i m e , a n d U3 for r e s p o n s e s t a k e n
60 rewards or until a maximum of 120 trials was reached t h r e e at a t i m e - - i n e a c h o f t h e c o m p o n e n t s .
in either component, whichever occurred first. After
initial longer times, the durations of reinforcement and T h e left set o f bars i n d i c a t e s t h e m e d i a n
time-out were reduced to 5,0 s in both components p e r f o r m a n c e o v e r t h e last five sessions in t h e
(times for Subject 38 were decreased to 4.2 s after five initial c o n d i t i o n . A s w o u l d be e x p e c t e d i f t h e
sessions owing to its weight gain). Pecks in both V and p i g e o n s w e r e p e r f o r m i n g v a r i a b l y in V a n d
S were separated by a 0.83-s interpeck interval. e m i t t i n g a fixed s e q u e n c e in S, t h e r e w e r e
Because most of the pigeons were at first unable to
attain 10 rewards in S, the first few sessions began with large a n d s i g n i f i c a n t d i f f e r e n c e s b e t w e e n t h e
the V component. After the 8th session, when responding U v a l u e s in t h e t w o c o m p o n e n t s : F o r U~,
in S had improved, the V and S components alternately t(3)=i5.515, p<.001; for U2, t(3) =
began each session. In an attempt to further improve 15.617, p < .001; a n d for U3, t(3) = 11.533,
performances during S, the salience of time-out was
p < .005.
increased by flashing the house light on for 0.33 s and
off for 0.08 s during time-out in both the V and S D u r i n g t h e e x p l o r a t o r y p h a s e t h a t is n o t
components. Phase 1 training continued for 12, 16, 24, s h o w n , a t t e m p t s w e r e m a d e to i n c r e a s e t h e
and 24 sessions for each of the 4 pigeons, respectively. s t e r e o t y p y c o m p o n e n t t o eight r e s p o n s e s
Phase 2: Equalization of responses. An attempt was ( e q u a l to t h e v a r i a b i l i t y c o m p o n e n t ) a n d t o
made to equalize the number of responses required in V
and S components and to approximately equalize the c h a n g e t h e c o n t i n g e n c i e s in S so t h a t t i m e -
percentages of correct responses at an intermediate level o u t s for i n c o r r e c t s e q u e n c e s o c c u r r e d o n l y at
of proficiency to avoid ceiling and floor effects. At the t h e e n d o f t h e s e q u e n c e (as was t h e case in
end of this phase, which lasted approximately 50 sessions, V). B u t b o t h o f t h e s e a t t e m p t s failed. T h e r e -
the schedule in the V component was six responses, Lag
fore, to e q u a l i z e p e r f o r m a n c e s u n d e r V a n d
10. The schedule requirement in S was the fixed pattern
LRRLL. Reinforcement and time-out in both components S, a f i v e - r e s p o n s e s e q u e n c e was u s e d in e a c h
lasted for 3 s. c o m p o n e n t a n d the lag c r i t e r i o n was i n c r e a s e d
Phase 3: Stimulus reversal. At the start of Phase 3, to 10 in the V c o m p o n e n t . P e r f o r m a n c e s
the number of responses in V was reduced to five to u n d e r t h e s e c o n d i t i o n s a r e s h o w n in t h e
equal the number of responses in S. Otherwise, the
contingencies in effect at the end of Phase 2 were m i d d l e p a n e l s o f F i g u r e s 13 a n d 14. W h e n
maintained. Birds 29, 31, 38, and 39 received 10, 14, the stimulus conditions were reversed (blue
12, and 22 sessions, respectively. key lights n o w signifying t h e S c o m p o n e n t
OPERANT VARIABILITY 447
9 IAi /? i
,0 t /fl ?, ~'
50 I
o
a. I
• Veriabilit
3(] /~ ! 0 Stereotypy
,
10 1
I
~ ,; ,~ 2; 2'~ 3o
Sessions
3'5 ,o ,; 5; 5~
Figure 13. Percentage of reinforced trials per session in variability (V) and stereotypy (S) components of
the multiple schedule for Pigeon 31. (Left panel = initial acquisition where the contingencies in V were
eight responses, Lag 5, and the S contingencies reinforced left-right-right patterns; middle panel =
performance under a V schedule of five responses, Lag 10, and an S schedule of left-right-right-left-left;
right panel = reversal of the middle schedule conditions [key light colors were reversed].)
and red key lights signifying the V component) ment. This conclusion was strengthened by
performances immediately deteriorated but comparison of the pigeon's performance with
then improved (right panels). At the end of a computer-based random number simulationl
both these phases, U values in S and V Experiment 3 increased the look back, or
differed significantly at the .01 level or better. the number of prior sequences of eight re-
Thus, we conclude that stimulus control was sponses from which the current sequence had
established over variable and stereotyped re- to differ. Eventually, to be reinforced, the
sponding. pigeon had to respond with a sequence that
differed from each of its last 50 sequences.
General Discussion This look back included sequences from the
previous session. The subjects generated
Experiments 1 and 2 showed that when highly variable sequences, with more than
hungry pigeons were given grain for generating 80% of the patterns differing from all previous
sequences of eight responses that differed patterns in a session. Probabilities of correct
from their last sequence, they successfully sequences again paralleled the probability of
varied their sequences. When, in addition to correct sequences of the simulating random
meeting this same variability requirement, generator.
the pigeons had to peck exactly four times Experiment 4 compared two possible ac-
on each key (as in Schwartz, 1980, Experi- counts of this variability. The m e m o r y hy-
ment 4; 1982a, Experiment 1), success rates pothesis was that the pigeons learned a long
fell significantly. We concluded that the in- sequence of responses or used a rational
ability of pigeons to gain high rates of reward strategy to meet the schedule requirements.
under the Schwartz procedure was due to an The variability hypothesis was that the pigeons
artifact of the four-response-per-key require- behaved as a quasi-random generator. The
448 SUZANNE PAGE AND ALLEN NEURINGER
first hypothesis predicts that increasing the rates improved significantly, thereby support-
n u m b e r of responses per trial would be cor- ing the quasi-random generator hypothesis.
related with a lowered success rate, for it is Once again, the pigeons' performances par-
easier to r e m e m b e r fewer responses than alleled the performance of a simulating ran-
more. The quasi-random hypothesis made dom generator.
the opposite prediction: By chance, given few The first four experiments generated high
responses per trial, consecutive trials would behavioral variability. However, neither these
repeat one another; given m a n y responses per nor any previously published experiments
trial, there would be few repetitions by chance. demonstrated that response variability de-
When the required number of responses per pended on the contingency between variability
trial was increased from four to eight, success and reinforcement. The observed variability
Figure 14. Average response variability under three phases of Experiment 6, from left to right. (U~ =
responses taken one at a time; U2 = responses taken in pairs; U3 = responses taken in triplets.)
OPERANT VARIABILITY 449
could have been elicited by the reinforcement was neither required nor differentially rein-
schedule (a respondent effect) rather than forced. The state of the environment, as well
directly reinforced (an operant effect). Exper- as the contingencies in the environment, in-
iment 5 tested these alternatives by comparing fluence behavioral variability, the former
the performance of pigeons under two iden- through respondent effects and the latter
tical schedules, except that one schedule re- through operant effects.
quired variability whereas the other permitted Both respondent and operant variability
it. In the first, a pigeon had to respond eight may be adaptive. When reinforcers are infre-
times with a sequence that differed from each quent or absent, variability increases the like-
of its last 50 sequences. The patterns and lihood that the animal will improve its l o t - -
frequencies of rewards under this condition learn a new strategy for obtaining reinforce-
were duplicated to form a self-yoked schedule ment or change its environment. Even when
where eight responses were required for re- reinforcement densities are relatively high,
ward but sequence variability was no longer variability may improve the schedule or pro-
necessary. The results showed that sequence vide knowledge of the environment in antic-
variability was generated only when it was ipation of possible future decrements in re-
required. Under the yoked schedule, vari- inforcement. Variability is an adaptive re-
ability decreased significantly and the pigeons sponse to a changing or potentially changing
responded with highly repetitive patterns. environment.
Variability is therefore an operant dimension Operant variability has unique adaptive
of behavior. functions not shared by respondent variability.
Experiment 6 demonstrated discriminative Whenever an animal is operantly conditioned
control over behavioral variability. Pigeons to generate a new response, whether the
learned to respond variably in the presence conditioning is through the process of shaping
of key lights of one color and with a fixed, (successive approximations to some desired
stereotyped sequence in the presence of a goal response) or trial and error (Thorndikian
second color. When the stimulus conditions conditioning), it is adaptive for the animal to
were reversed, performances reversed. Thus, vary its behaviors. Reinforcement works
the variability dimension of behavior is con- through selection of some previously emitted
trolled by environmental stimuli in much the behavior. If the to-be-selected behavior does
same manner as other operant dimensions are. not occur, reinforcement cannot select. On
The present series of experiments therefore the other hand, in an environment where an
conclusively demonstrate the existence and operant response has previously been learned
strength of operant variability, variability that and is now being maintained by an acceptable
is engendered and maintained because pre- schedule of reinforcement, high variability
sentation of a reinforcer depends on the may not be functional, for variation may
variability. This conclusion is consistent with require relative high energy expenditure and
Blough (1966), Pryor et al. (1969), and Bryant sometimes result in less frequent reinforce-
and Church (1974), among others. ment. It is advantageous for an animal to
Previous studies have examined respondent discriminate situations in which new respon-
variability. Different schedules of reinforce- ses must be learned from those in which
ment reliably engender differing degrees of previously learned behaviors must be re-
behavioral variability with no contingency peated. We hypothesize that this discrimina-
between the variability and the reinforcement tion is based on the reinforcement of diverse
schedules. For example, if identical and in- responses and response classes in the former
dependent fixed-ratio 5 (FR 5) schedules are case versus reinforcement of fixed, or stereo-
programmed on each of two response keys, typed, responses and response classes in the
most pigeons peck exclusively on one or the latter. (The discrimination is in some ways
other key. If the schedules are changed to FR analogous to that between contingent and
150, there is considerable switching between noncontingent reinforcement; Killeen, 1978).
keys. This observation from our laboratory When an animal is differentially rewarded
is a clear example of respondent variability for a variety of responses, it generates variable
caused by reinforcement schedules. Variability behaviors. We posit that this describes all
450 SUZANNE PAGE AND ALLEN NEURINGER
operant learning (as opposed to operant tempted to train a single, fixed eight-response
maintaining) situations. Explicit reinforce- sequence in the stereotypy component of
ment of variable behaviors prior to initiation Experiment 6, we were unsuccessful. Despite
of operant learning, or shaping, procedures hundreds of reinforcements over many ses-
might speed the learning process. This hy- sions, the pigeons failed to learn. It therefore
pothesis should be tested. seems unlikely that they could learn an eight-
There are other instances where operant response sequence after a single reinforce-
control of variability is adaptive. Some envi- ment. Furthermore, the present results did
ronments punish variability (e,.g., many school not show persistence of previously reinforced
classrooms), and most children can discrim- sequences.
inate these from other situations (e.g., the A second hypothesis is that the pigeons
game of hide-and-seek, where variability is learned a long sequence of responses or a
reinforced). Environments that require brain- rational strategy to meet the variability re-
storming, problem solving, or creativity rein- quirements. Experiment 4 showed that this,
force variability of behaviors. One attribute too, is unlikely.
of successful art is the uniqueness of the The third interpretation, one supported by
artist's work. If an animal is to avoid preda- the present research, is that variability is a
tion or injury, it is functional for the animal dimension of behavior much like other op-
to vary its behavior in the presence of specific erant dimensions. Reinforcement does not
predators or in specific environments (see necessarily lead to response stereotypy. Vari-
Humphries & Driver, 1970; Serpell, 1982), ability is as susceptible to control by rein-
and at least some aspects of this variability forcement as are frequency, force, duration,
might be controlled by consequences. Operant location, or topography. But this does not
'variability is also functional in sports, gam- imply that the existence of variability depends
bling, war, or other competitive environments. on its reinforcement. As indicated above,
The interaction between elicited respondent behavioral variability is a respondent conse-
variability and reinforced operant variability quence of environmental events as well as an
is an important area for future study. Re- operant progenitor. Furthermore, variable be-
spondent variability may set the boundaries havior must precede its consequences. The
within which reinforcing contingencies control following analogy may be useful: The pigeon
operant variability. For example, both very enters the operant conditioning experiment
low and very high densities of food to a with a class of behaviors described as pecking
hungry animal may prohibit high behavioral already intact. When the experimenter shapes
variability despite reinforcement of that vari- key pecking, the pecking response is not
ability (see Experiment 3). Alternatively, re- being trained. Rather, the pigeon is taught
inforcement of variability may extend the where, when, and possibly how fast or hard,
boundaries of elicited variation. and so on, to peck. Analogously, there may
Reinforcement of variability raises theo- be a dimension of all behaviors, described as
retical problems. How can behavioral varia- variability, with which the organism enters
tion be reinforced if reinforcement increases our experiments. The rapidity with which
the probability of those responses that pro- inexperienced pigeons acquired variable per-
duced it, thereby necessarily increasing re- formance under the initial Lag 50 condition
sponse stereotypy (e.g., Schwartz, 1982a)? in Experiment 5 supports this view. Turning
There are at least three possible ways in on or off a variability generator may be under
which reinforcement can serve both to in- the control of reinforcement, but the vari-
crease the probability of prior responses and ability generator is not itself created through
increase response variability. First, a schedule reinforcement. An animal may be born with
of reinforcement may rapidly condition and the variability generator intact.
extinguish response patterns, thereby only The present results also raise a method-
apparently reinforcing variable sequences. It ological issue. Skinner has argued for the
is the intermittent extinction, according to ultimate predictability of behavior (Skinner,
this interpretation, that elicits the variability, 197 l) and therefore for the study of highly
a respondent effect. However, when we at- controlled behaviors (Skinner, 1984). In fact,
OPERANT VARIABILITY 451
operant conditioning studies have emphasized food, clothing, and company), behavior may
highly controlled acts (see Schwartz, Schul- still be highly constrained. Contingencies that
denfrei, & Lacey, 1978), but a complete explicitly reinforce behavioral variability are
analysis of behavior must include analyses of necessary to maximize freedom.
the variability of behavior, variability main-
tained through respondent influences as well References
as that directly engendered and maintained Antonitis, J. J. (1951). Response variability in the white
by reinforcing consequences. It may be im- rat during conditioning, extinction, and reconditioning.
possible to predict or control the next instance Journal of Experimental Psychology. 42, 273-28 I.
of a variable behavior, but lack of prediction Blough, D. S. (1966). The reinforcement of least frequent
interresponse times. Journal of the Experimental Anal-
and control should not rule out study. When ysis of Behavior, 9, 581-591.
behaviors are variable, experimental analyses Bryant, D., & Church, R. M. (1974). The determinants
can determine whether the variability is noise of random choice. Animal Learning & Behavior, 22,
(i.e., experimental or observational error), 245-248.
under respondent control, or under operant Carpenter, E (1974). The Skinner primer: Behind freedom
and dignity. New York: Macmillan,
control. Experiments can define the class or Catania, A. C. (1980). Freedom of choice: A behavioral
classes from which the individual instances analysis. In G. H. Bower (Ed.), The psychology of
of behavior are selected, the conditions under learning and motivation (Vol. 14, pp. 97-145). New
which the variability will occur or not, the York: Academic Press.
Crossman, E. K., & Nichols, M. B. (1981). Response
variables controlling onset and offset of the location as a function of reinforcement location and
variability, and the functions served by the frequency. Behavior Analysis Letters, 1, 207-215.
variability. Operant analysis must not limit Eckerman, D. A., & Lanson, R. N. (1969). Variability of
itself to predictable and controllable behav- response location for pigeons responding under contin-
iors. Doing so ignores an essential character- uous reinforcement, intermittent reinforcement, and
extinction. Journal of the Experimental Analysis of
istic of operant behavior. Behavior, 12, 73-80.
Finally, we note one sociopolitical impli- Herrnstein, R. J. (1961). Stereotypy and intermittent
cation. Freedom (cf. Carpenter, 1974; Lamont, reinforcement. Science, 133, 2067-2069.
1967) often means the absence of constraining Holman, J., Goetz, E. M., & Baer, D. M. (1977). The
training of creativity as an operant and an examination
contingencies (no gun to the head) and the of its generalization characteristics. In B. Etzel, J. Le
presence of a noncontingent benign environ- Blanc, & D. Baer (Eds.), New developments in behavioral
ment, one where adequate food, shelter, and research: Theory, method and application. Hillsdale,
so forth, are available independent of any NJ: Erlbaum.
particular behavior. However, operant contin- Humphries, D, A., & Driver, P. M. (1970). Protean
defense by prey animals. Oecologia, 5, 285-302.
gencies may also be crucial. To maximize Killeen, P. R. (1978). Superstition: A matter of bias, not
freedom, an animal or person must have a detectability. Science, 199, 88-90.
wide variety of experiences (Catania, 1980; Lachter, G. D., & Corey, J. R. (1982). Variability of the
Rachlin, 1980). If, for example, to obtain duration of an operant. Behavior Analysis Letters, 2,
97-102.
food, an animal has always entered the same Lamont, C. (1967). Freedom of choice affirmed. New
1 of 10 cubicles, each cubicle containing a York: Horizon Press.
different type of food, and therefore has never Miller, G. A., & Frick, E C. (1949). Statistical behavior-
experienced any but the 1 food, it makes istics and sequences of responses. Psychological Review,,
little sense to say that there is free choice 56, 311-324.
Miller, N. E. (1978). Biofeedback and visceral learning.
a m o n g the 10 foods. The present results Annual Review of Psychology, 29, 373-404.
suggest that diversity of choice is controlled Neuringer, A. (1970). Superstitious key pecking after
by reinforcers contingent on diversity. Absence three peck-produced reinforcements. Journal of the
of variability-maintaining contingencies, such Experimental Analysis of Behavior, 13, 127-134.
Neuringer, A. (1984). Melioration and self-experimenta-
as in the yoked condition of Experiment 5, tion. Journal of the Experimental Analysis of Behavior,
increases stereotyped behaviors and therefore 42, 397-406.
limits experiences. If these speculations are Neuringer, A. (1985). "Random" performance can be
correct, a laissez-faire environment will not learned. Manuscript submitted for publication.
secure freedom of choice. Despite the absence Notterman, J. M. (1959). Force emission during bar
pressing. Journal of Experimental Psychology, 58, 341-
of aversive constraints and the presence of 347.
positive respondent influences (e.g., good Piscaretta, R. (1982). Some factors that influence the
452 SUZANNE PAGE AND ALLEN NEURINGER
acquisition of complex stereotyped response sequences ability with reinforcement. Journal of the Experimental
in pigeons. Journal of the Experimental Analysis of Analysis of Behavior, 37, 171-181.
Behavior, 37, 359-369. Schwartz, B. (1982b). Reinforcement-inducedstereotypy:
Popper, K. R. (1968). The logic of scientific discovery. How not to teach people to discover rules. Journal of
New York: Harper & Row. Experimental Psychology: General, 111, 23-59.
Pryor, K. W., Haag, R., & O'Reilly, J. (1969). The Schwartz, B,, Schuldenfrei, R., & Lacey, H. (1978).
creative porpoise: Training for novel behavior. Journal Operant psychology as factory psychology.Behaviorism,
of the Experimental Analysis of Behavior, 12, 653- 6, 229-254.
661. Serpell, J. A. (1982). Factors influencing fighting and
Rachlin, H. (1980). Behaviorism in everyday life. Engle- threat in the parrot genus Trichoglossus. Animal Be-
wood Cliffs, NJ: Prentice-Hall. havior, 30, 1244-1251.
Schoenfeld, W. N., Harris, A. H., & Farmer, J. (1966). Skinner, B. F. (1938). The behavior of organisms. New
Conditioningresponse variability. PsychologicalReports, York: Appleton-Century.
19, 551-557. Skinner, B. F. (1971). Beyond freedom and dignity. New
Schwartz, B. (1980). Development of complex stereotyped York: Alfred A. Knopf.
behavior in pigeons. Journal of the Experimental Anal- Skinner, B. F. ( 1984, May). Thefuture of the experimental
ysis of Behavior, 33, 153-166. analysis of behavior. Paper presented at the meeting
Schwartz, B. (1981). Control of complex, sequential of the Association for Behavior Analysis, Nashville,
operants by systematic visual information in pigeons. TN.
Journal of Experimental Psychology: Animal Behavior
Processes, 7, 31-44. Received July 13, 1984
Schwartz, B. (1982a). Failure to produce response vari- Revision received February 26, 1985 •