Tmp19e3 TMP
Tmp19e3 TMP
Tmp19e3 TMP
DOI 10.3758/s13428-016-0741-1
The Author(s) 2016. This article is published with open access at Springerlink.com
Florian Sense
Behav Res
verbal stimuli out loud improves recall performance relative to mouthing them silently. However, this relationship
only seems to persist if the visually-presented material
can be verbalized effectively (e.g., verbal stimuli, nameable visual images). The idea that it is the opportunity to
rehearse these verbal codes that improves performance also
remains a matter for debate, even for serially-ordered verbal
stimuli (Lewandowsky and Oberauer, 2015). Attempts to
verbalize stimuli that are difficult to describe succinctly and
accurately (e.g., faces) might actually harm performance
(Schooler & Engstler-Schooler, 1990). Brandimonte et al.
(1992) showed that verbal recoding can be detrimental to a
subsequent mental rotation task when the remembered verbal label is not relevant or helpful. What such experiments
suggest is that there is a strong tendency to verbally recode
visually-presented information, and that in some cases verbal recoding may boost memory performance. This logic is
consistent with multi-component models of working memory, which propose that separate short-term memory stores
for phonological and visual information can be applied to
a short-term memory task (Baddeley, 1986). Naturally, if
task-relevant information can be maintained simultaneously
in two useful codes, one would expect memory performance
to improve.
The possibility of dual encoding is problematic though
if the goal is to measure capacity for visual information
exclusively. Levy (1971) suggested a method of preventing
such recoding via meaningless concurrent articulation. By
repeating irrelevant syllables out loud during presentation
and retention of visual information, participants ability to
verbally recode visually-presented stimuli is restricted. This
procedure is known as articulatory suppression and is commonly used alongside visual change detection tasks with
the specifically-stated intention that it is meant to prevent
verbalization of visual stimuli (e.g. Allen, Baddeley, &
Hitch, 2006; Brockmole, Parra, Sala, & Logie, 2008;
Delvenne & Bruyer, 2004; Hollingworth & Rasmussen,
2010; Logie, Brockmole, & Vandenbroucke, 2009; Makovski
& Jiang, 2008; Makovski, Sussman, & Jiang, 2008;
Matsukura & Hollingworth, 2011; Treisman & Zhang,
2006; van Lamsweerde & Beck, 2012; Woodman & Vogel,
2005, 2008). This precaution is undertaken to ensure that
task performance reflects visual memory, rather than some
combination of memory for visual images and verbal codes.
The use of precautionary articulatory suppression is
common practice despite evidence that articulatory suppression has not been shown to have a measurable effect on
some visual change detection tasks (Luria, Sessa, Gotler,
Jolicoeur, & DellAcqua, 2010; Mate, Allen, & Baques,
2012; Morey & Cowan, 2004, 2005), nor have small verbal memory loads (Vogel, Woodman, & Luck, 2001). These
studies imply that the precaution of employing articulatory
suppression may be unnecessary: participants performed no
better without articulatory suppression than with it, suggesting that verbal recoding is not the default strategy for visual
change detection tasks as typically administered. However,
these findings simply report null effects of meaningless
articulatory suppression on visual memory tasks, and therefore cannot be taken as strong evidence of the absence of
some effect, given sufficient power to detect it. Until a
stronger case against verbal recoding during visual change
detection can be made, enforcing articulatory suppression to
prevent verbalization of visual images is a reasonable way
for researchers to better ensure that their measure of visual
memory performance is pure. However, enforcing articulation adds a substantial burden to an experiment from both
the participants and the experimenters point of view. If
a strong case could be made that possible verbal recoding of visual memoranda does not affect visual memory
performance, researchers would be free to forgo including
articulatory suppression from some designs.
We report evidence suggesting that articulatory suppression has no discernible effect on performance in a typical
visual change-detection task. The experiment was designed
so that some change-detection conditions encouraged verbalization by presenting memoranda one at a time. In all
cases, the stimuli were arrays of distinctly-colored squares,
and the object was to remember the location of each color.
We manipulated the number of items in each array, whether
the squares were presented simultaneously or sequentially,
and whether participants performed articulatory suppression
or not. If participants tend to verbally label the stimuli, and
if verbal labeling assists the recognition decision, we would
expect to observe at least a small benefit of silence over
articulation in all conditions. It may also be the case that
participants strategically choose when to verbally recode
stimuli. If so, we would expect to see selective impairments
with articulation for sequentially-presented items, perhaps
most strongly for small set sizes where naming all the items
might have occurred. In order to discern between small
effects of articulation and the null hypothesis of no effect at
all, we employ two modes of analysis: first, we provide a
straightforward analysis based on descriptive statistics that
shows that the effects tend to go in the reverse direction
to what is predicted, ruling out evidence for the predicted
effect; and second, we employed Bayesian state-trace analysis to show that participants show data patterns more
consistent with a single-parameter explanation (visual short
term memory) than a more complicated explanation (visual
short term memory plus verbal short term memory).
Methods
Participants performed a visual array change detection
task under four conditions formed by the cross of two
Behav Res
Behav Res
tim
Fig. 1 A schematic representation of a set size two trial in the simultaneous presentation condition. Note that the image is not to scale
Results
Prior to data analysis, all trials containing invalid responses
(0.1 % of trials) were removed, and trials with unusually long or short response times (<200 ms or >3 s; 2 %
of trials) were excluded. The overwhelming majority of
these were too slow, possibly because participants took
unscheduled breaks by deliberately delaying their response.
Overall, 36,495 trials across the 15 participants remained for
analysis. Descriptive statistics for task performance across
conditions are summarized in Fig. 2. Overall accuracy is
high in the set size 2 condition, as expected, and decreases
as set size increases. In addition, Table 1 shows the mean hit
and false alarm rates across all participants.
0.9
0.9
0.8
0.8
hits false alarms
0.7
0.6
0.5
0.4
0.3
0.2
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.1
articulate
silent
0
2
articulate
silent
0
4
set size
set size
Behav Res
Table 1 Mean hit and false alarm rates for all conditions across all participants
hits
simultaneous
sequential
set size
articulate
silent
articulate
silent
2
4
8
0.95 (0.022)
0.84 (0.065)
0.72 (0.079)
0.95 (0.036)
0.88 (0.063)
0.70 (0.100)
0.94 (0.026)
0.82 (0.076)
0.70 (0.101)
0.94 (0.032)
0.82 (0.097)
0.70 (0.174)
0.06 (0.035)
0.22 (0.132)
0.39 (0.138)
0.11 (0.066)
0.31 (0.166)
0.41 (0.142)
0.07 (0.039)
0.26 (0.154)
0.42 (0.159)
false alarms
2
4
8
0.08 (0.040)
0.25 (0.131)
0.41 (0.120)
than one-half: 8/15, 10/15, and 10/15 points lie above the
diagonal for set sizes 2, 4, and 8, respectively. There is no
evidence of the predicted effect in these data; instead, the
effect appears to go in the wrong direction.
We also examined whether the apparent lack of an effect
may be due to differences in strategy over the experimental
sessions; however, a similar picture emerges when the effect
is examined across time, as in Fig. 3b. The verbalization
hypothesis would predict that points would fall above the
horizontal line at 0 on average; however, if anything, the
points tend to fall below the line.
Given the descriptive analysis above, we eschew typical ANOVA analyses in favor of reliance on a state-trace
0.2
0.1
8
0.0
8
0.1
8
8
4
4 4
4 4
2
2
2
2 4
8 444 8
2
2
4 4 22 2
82 2
8 4
4 82
8
2 2
88 4
8
2
4
Effect (seq sim)
0.1
8
0.0
2
4
0.1
4
2
8
8
2
4
8
4
2
2
8
2
8
2
4
8
4
2
2
8
8
4
0.2
0.1
0.0
0.1
0.2
0.2
10
errors. b: The difference between the advantage for the silent condition
in the sequential and simultaneous presentation conditions as a function of experimental block. In both plots, the number for each point
represents the set size
Behav Res
analysis.1 We have the luxury of avoiding the assumptionladen ANOVA because we have directional predictions that
are violated in the data. Thus, there cannot be evidence for
the prediction of interest. Furthermore, we are interested
in the dimensionality of the latent system that has produced the observed data - a question that an ANOVA, unlike
state-trace analysis (Prince et al., 2012), cannot provide a
reliable answer to. The state-trace analysis complements
the descriptive analysis by showing that the data are highly
consistent with a simple explanation: that performance is
governed by a single latent variable (interpreted as visual
short term memory capacity) and no more complicated
explanation involving verbalization is needed.
State-trace analysis
Another way to examine whether there is any evidence
for verbalization is a state-trace analysis. State-trace analysis, outlined in its original form by Bamber (1979), is
a data analysis technique intended to reveal how many
latent dimensions a system requires to produce observed
empirical results (see Prince et al., 2012 for an overview
and the application of Bayesian analysis). A simple system may have only one latent dimension (e.g., working
memory capacity in general, or visual working memory
capacity specifically), and all experimental manipulations
affect performance along that latent dimension. More complex systems may show relationships that are impossible to
explain by a single dimension, and therefore require positing
more latent constructs (see section Diagnosing Dimensionality in Prince et al. (2012) for a detailed explanation based
on hypothetical examples).
Considering visual change detection performance, one
might imagine that only one latent memory dimension contributes to recognition accuracy or alternatively that separate
visual and verbal memory systems jointly contribute to
recognition accuracy. The multi-component model of working memory (Baddeley, 1986) proposes sub-systems for
verbal and visual short-term memory, and would be consistent with the suggestion that both verbal and visual codes
are stored during visual array memory, with both codes
contributing to recognition accuracy. This assumption is
the reason why precautionary articulatory suppression is
so often employed during visual memory tasks. One reasonable prediction of the multi-component model is thus
that at least two latent factors, verbal and visual memory,
contribute to visual change recognition accuracy. Another
1 For
reasonable expectation is that whether or not verbal encoding occurs, it is insufficient to affect recognition accuracy
in this task, and in that case, a single dimension would
better explain recognition accuracy in visual change detection. If visual change detection performance in our study,
which was explicitly designed to allow verbalization to exert
effects in specific conditions, can be explained by a single
latent dimension then we would conclude that articulatory
suppression is not needed to prevent verbalization in tasks
with similar designs.
In the logic of state-trace analysis, performance in the
sequential and simultaneous presentation conditions arise
from either one or more latent constructs. If they both arise
from a single latent variable, such as (visual) working memory capacity and if performance in both is a monotone
function of the latent variable then performance in the
sequential presentation must be a monotone function of performance in the simultaneous condition. To the extent that
no monotone function can describe the relationship between
simultaneous and sequential task performance, two latent
constructs perhaps distinct visual and verbal working
memory capacities are assumed to be needed to describe
the performance.
For the state-trace analysis, we again used d, the hit rate
minus the false alarm rate, as a measure of performance
in our simulations. To reduce possibly spurious deviations
in our simulations, we computed Bayesian estimates of d
applying three reasonable constraints: first, we assumed that
the true hit rate was greater than the true false alarm rate,
and thus performance was truly above chance. Second, for
both the sequential and the simultaneous condition, d must
decrease with increasing array set size; for instance, true d
to a set size of 8 cannot be better than performance to set size
4, all other things being equal. Third, it was assumed that
suppression cannot benefit performance; for each set size
and presentation condition, the true d in the articulate condition must be less than in the silent condition. This restriction
was applied because a small dual-task cost appearing in all
conditions would be consistent with any working memory
theory, and with our distinctly-colored stimuli and meaningless articulation instructions, no benefit of articulation was
reasonably expected. When a simulation produced one of
these patterns, we excluded it and replaced it. Estimating
the true discrimination under these restrictions yields a less
error-prone measure of performance due to the exclusion of
simulations with implausible data patterns.
Figure 4 shows the state-trace plots for each participant,
formed by plotting estimated performance in the simultaneous presentation condition against the performance in the
sequential condition. State-trace logic says that more than
one latent construct is needed to explain the data when
Behav Res
BF = 6.6
0.8
2
4
BF = 6.7
BF = 12.4
0.6
8
8
0.4
4
8 4
0.2
8
0.0
1.0
BF = 13.3
BF = 23.8
0.8
BF = 35
22
0.6
2
2
8
8
0.4
8
8
0.2
0.0
1.0
BF = 49.2
BF = 61.9
0.8
BF = 69.3
4
4
0.6
4
0.4
4
0.2
0.0
1.0
2
BF = 104.3
BF = 105.8
BF = 152.7
0.8
4
0.6
4
0.4
8
8
0.2
8
8
8
0.0
1.0
BF = 304.8
2
2
BF = 659.2
2
2
BF = 4799.6
0.8
4
4
0.6
0.4
8
0.2
Articulate
Silent
0.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
corner, each plot also features the Bayes factor in favor of a monotone
ordering of the points over a non-monotone ordering
Behav Res
Behav Res
References
Allen, R. J., Baddeley, A. D., & Hitch, G. J. (2006). Is the binding of
visual features in working memory resource-demanding? Journal
of Experimental Psychology: General, 135, 298313.
Baddeley, A. D. (1986). Working memory. Oxford: Clarendon Press.
Bamber, D. (1979). State trace analysis: method of testing simple
theories of causation. Journal of Mathematical Psychology, 19,
137181.
Brainard, D. H. (1997). The psychophysics toolbox. Spatial Vision, 10,
433436.
Brandimonte, M. A., Hitch, G. J., & Bishop, D. V. M. (1992). Verbal
recoding of visual stimuli impairs mental image transformations.
Memory & Cognition, 20(4), 449455.
Brockmole, J. R., Parra, M. A., Sala, S. D., & Logie, R. H. (2008). Do
binding deficits account for agerelated decline in visual working
memory? Psychonomic Bulletin & Review, 15(3), 543547.
Brown, L. A., Forbes, D., & McConnell, J. (2006). Limiting the use of
verbal coding in the visual patterns test. The Quarterly Journal of
Experimental Psychology Section A, 59(7), 11691176.
Brown, L. A., & Wesley, R. W. (2013). Visual working memory is
enhanced by mixed strategy use and semantic coding. Journal of
Cognitive Psychology, 25(3), 328338.
Cowan, N., Elliott, E. M., Saults, J. S., Morey, C. C., Mattox, S.,
Hismjatullina, A., & Conway, A. (2005). On the capacity of attention: its estimation and its role in working memory and cognitive
aptitudes. Cognitive Psychology, 51, 42100.
Cowan, N., & Morey, C. C. (2007). How can dualtask working memory retention limits be investigated? Psychological Science, 18,
686688.
Davis-Stober, C., Morey, R. D., Gretton, M., & Heathcote, A. (2015).
Bayes factors for state-trace analysis. Journal of Mathematical
Psychology. doi:10.1016/j.jmp.2015.08.004
Delvenne, J. F., & Bruyer, R. (2004). Does visual shortterm memory
store bound features? Visual Cognition, 11(1), 127.
Hollingworth, A., & Rasmussen, I. P. (2010). Binding objects to locations: the relationship between object files and visual working
Behav Res
memory. Journal of Experimental Psychology: Human Perception
and Performance, 36(3), 543564.
Kleiner, M., Brainard, D., & Pelli, D. (2007). Whats new in
Psychtoolbox-3? Perception, 36. ECVP Abstract Supplement.
Larsen, J. D., & Baddeley, A. D. (2003). Disruption of verbal
stm by irrelevant speech, articulatory suppression, and manual tapping: do they have a common source? The Quarterly
Journal of Experimental Psychology Section A, 56(8), 1249
1268.
Levy, B. A. (1971). Role of articulation in auditory and visual shortterm memory. Journal of Verbal Learning and Verbal Behavior,
10, 123132.
Lewandowsky, S., & Oberauer, K. (2015). Rehearsal in serial recall:
an unworkable solution to the nonexistent problem of decay.
Psychological Review, 122, 674699.
Logie, R. H., Brockmole, J. R., & Vandenbroucke, A. R. E. (2009).
Bound feature combinations in visual short-term memory are
fragile but influence long-term learning. Visual Cognition, 17(1),
160179.
Luria, R., Sessa, P., Gotler, A., Jolicoeur, P., & DellAcqua, R.
(2010). Visual short-term memory capacity for simple and complex objects. Journal of Cognitive Neuroscience, 22(3), 496512.
Makovski, T., & Jiang, Y. V. (2008). Indirect assessment of visual
working memory for simple and complex objects. Memory &
Cognition, 36(6), 11321143.
Makovski, T., Sussman, R., & Jiang, Y. V. (2008). Orienting attention in visual working memory reduces interference from memory
probes. Journal of Experimental Psychology: Learning, Memory,
and Cognition, 34(2), 369380.
Mate, J., Allen, R. J., & Baques, J. (2012). What you say matters:
exploring visual-verbal interactions in visual working memory.
The Quarterly Journal of Experimental Psychology, 65(3), 395
400.
MATLAB (2011). Version 7.13.0 (r2011b). Natick, Massachusetts:
The MathWorks Inc.
Matsukura, M., & Hollingworth, A. (2011). Does visual short-term
memory have a high-capacity stage? Psychonomic Bulletin &
Review, 18(6), 10981104.
Morey, C. C., & Cowan, N. (2004). When visual and verbal memories compete: Evidence of crossdomain limits in working memory.
Psychonomic Bulletin & Review, 11, 296301.
Morey, C. C., & Cowan, N. (2005). When do visual and verbal
memories conflict? The importance of working-memory load and
retrieval. Journal of Experimental Psychology: Learning, Memory,
and Cognition, 31, 703713.
Morey, C. C., Morey, R. D., van der Reijden, M., & Holweg,
M. (2013). Asymmetric cross-domain interference between two
working memory tasks: implications for models of working memory. Journal of Memory and Language, 69, 324348.
Morey, R. D. (2011). A hierarchical Bayesian model for the measurement of working memory capacity. Journal of Mathematical
Psychology, 55, 824.
Murray, D. J. (1965). Vocalization-at-presentation and immediate
recall, with varying presentation rates. Quarterly Journal of Experimental Psychology, 17, 4756.
Pelli, D. G. (1997). The VideoToolbox software for visual psychophysics: transforming numbers into movies. Spatial Vision, 10,
437442.
Prince, M., Brown, S., & Heathcote, A. (2012). The design and analysis
of state-trace experiments. Psychological Methods, 17(1), 7899.
Rouder, J. N., Morey, R. D., Cowan, N., Zwilling, C. E., Morey, C. C.,
& Pratte, M. S. (2008). An assessment of fixed-capacity models of
visual working memory. Proceedings of the National Academy of
Sciences, 105, 59765979.
Rouder, J. N., Morey, R. D., Morey, C. C., & Cowan, N. (2011). How
to measure working-memory capacity in the change-detection
paradigm. Psychonomic Bulletin & Review, 18, 324330.
Schooler, J. W., & Engstler-Schooler, T. Y. (1990). Verbal overshadowing of visual memories: some things are better left unsaid.
Cognitive Psychology, 22, 3671.
Sperling, G. (1967). Successive approximations to a model for shortterm memory. Acta Psychologica, 27, 285292.
Treisman, A., & Zhang, W. (2006). Location and binding in visual
working memory. Memory & Cognition, 34(8), 17041719.
van Lamsweerde, A. E., & Beck, M. R. (2012). Attention shifts or
volatile representations: what causes binding deficits in visual
working memory? Visual Cognition, 20(7), 771792.
Vogel, E. K., Woodman, G. F., & Luck, S. J. (2001). Storage of
features, conjunctions, and objects in visual working memory.
Journal of Experimental Psychology: Human Perception and Performance, 27, 92114.
Woodman, G. F., & Vogel, E. K. (2005). Fractionating working memory: consolidation and maintenance are independent processes.
Psychological Science, 16(2), 106113.
Woodman, G. F., & Vogel, E. K. (2008). Selective storage and
maintenance of an objects features in visual working memory.
Psychonomic Bulletin & Review, 15(1), 223229.
Woodman, G. F., Vogel, E. K., & Luck, S. J. (2012). Flexibility in
visual working memory: accurate change detection in the face of
irrelevant variations in position. Visual Cognition, 20(1), 128.