The following essay is not really linked to psychotherapy or hypnotherapy. However, viewers might be interested to read about other areas of psychology that I have been researching. I first became interested in visual search and change blindness when I was looking at the differences between consciousness and unconsciousness at the Royal Society of Medicine. This is a short but specific essay on visual search. 

  Does visual search have memory?

by

David Kraft PhD

This report investigates whether visual search has memory, and this has been an ongoing debate in cognitive psychology for decades. Visual search, which involves both memory and attention, is a very important mechanism which we use every day; however, because our brain is finite and adaptive (Horowitz and Wolfe, 2001), we tend only to locate and retain information which is important to us at the time. Scholl, Simons and Levin (2004) showed that approximately 50% of individuals who focussed their attention on subjects wearing white shirts failed to see a black gorilla wave in the middle of the screen. This phenomenon, known as ‘change blindness’ (Simons and Levin, 1997), demonstrates that only a small amount of visual stimuli is retained between views. This report will investigate whether or not we have a memory of previously located items.

The general principles of the two opposing views are as follows. Horowitz and Wolfe (1997) challenged previous memory-driven theories of visual search (e.g., Treisman & Gelade, 1980; Grossberg, Mingolla & Ross, 1994), pointing out that there is only a small amount of assimilation of visual information across saccades. They maintained that visual search is amnesic in that neural processes are revisited from moment to moment. This is the memoryless-driven model. By contrast, memory-driven models have suggested that distractors, which have already been located, are processed and maintained within working memory and are, therefore, not revisited (Klein, 2000). Kristjansson (2000) repositioned targets and distractors and found that this increased search times—this showed that the subjects used spatial awareness and memory. Further, Peterson, Kramer, Wang, Irwin and McCarley (2001) maintained that search has a memory of approximately twelve items, although strategies of ‘chunking’ (Gilmore, Tobias & Royer, 1985; Pashler, 1994; Peterson, Beck & Vomela, 2007) can be used to increase visual short term memory. Although it is not sure the rôle that memory plays in tracking the positions of previously found items, Peterson et al. study (2001), found that their two memory-driven models predicted participants’ memory performance better than the memoryless search model.

The author will now discuss the memory-driven model, then the memoryless model and, finally, will draw conclusions with regard to both approaches. In the Peterson et al (2001) study, five participants were asked to search the screen in order to decide whether a target T was rotated to the left or to the right. The experimenters compared results from a Monte Carlo simulation memoryless search model (for example, Metropolis & Ulam, 1949; Proppe, Pradlwarter, & Schuëller, 2003), with their memory-driven model which assessed how often and how long it took for a participant to revisit an item. The proportion of revisits for their procedure was a great deal smaller (5.7%) for the observation compared to the predictions of the memoryless model. Revisitations were often due to participants wishing to reassess the target, and this intimates that participants had a memory for the target and were also aware that the information had not been processed sufficiently. Two models—the miss model and miss + realization model—were used to test the premise that the observed revisitations were due to initial fixations which were insufficiently processed. Here, both models made significantly closer predictions compared to the memoryless search model.

One possible reason for these results being so different to the results found in the Horowitz and Wolfe (1997) study is that the displays used by Peterson and colleagues (2001) were more ecologically valid compared to the flashing masking frames of the earlier study. Perhaps, the results were particular to serial, ‘self-terminating searching’ (Krisjansson, 2000).  Nevertheless, results of the Peterson et al (2001) model are supported by Takeda (2002) who increased the set sizes and mixed up the number of distractors and targets: Takeda also found that the memory-driven model was a better predictor than the memoryless model.

Although, in the Horowitz and Wolfe study (1998), visual search was faster when measuring static displays—one might conclude from this that the participants in this study (n=9) might have employed different techniques in the visual search while looking for two types of displays. This supports the findings of Kristjansson (2000) who found that reaction times were slower when the target was positioned in a space where a distractor had been previously placed, compared to items which remained in the same place. In addition, in the Horowitz and Wolfe (1998) study, error rates were not the same: indeed, participants made more mistakes in the random condition. From this, one can presume that visual searching was more proficient in the stable condition.

Indeed, memory can direct focus of attention during visual search: in fact, a search is a combination of memory and attention. For example, Klein and MacInnes (1999) pointed out that individuals make simultaneous movements of the eyes away from a fixation, indicating that, due to short term memory, there was no need for a re-visitation of the target.

It is also important to take into consideration a theory presented by Chun and Jiang (1999) , known as ‘contextual cuing’, which suggests that implicit memory helps individuals, at an unconscious level, to learn and make predictions with regard to recurring visual cues and the location of a target (see also Chun & Phelps, 1999). Moreover, Kristjansson, Mackeben and Nakayama (1999) showed that participants learn quickly the location of cue and target.

If we all have visual short term memory (Baddeley, 1986; Luck & Vogel, 1997), one could make the assumption that all working memory is attention and that this mechanism guides direct attention (Schneider, 1995; Horowitz & Wolfe, 2001). Individuals are able to remember the position and object identity of four items from one saccade to the next (Irwin & Gordon, 1998; Peterson et al, 2001). In the Peterson et al. (2001) study, participants revisited a site because they had a memory for an item even though it had been processed ineffectually and fleetingly. Indeed, they pointed out that they were, ‘wilful re-examinations of already-examined items’ (Peterson et al, 2001, p289).

Unlike the memoryless search models (e.g., Horowitz & Wolfe, 1998), the findings of Peterson et al. (2001) showed that there was not a flat hazard function. Indeed, the ‘miss model’ and the ‘miss + realization model’ revealed that there was an increasing hazard function. Indeed, visual search has a memory of probably around 3 to 4 items, although ‘chunking’ (Gilmore Tobias & Royer, 1985; Peterson, Beck & Vomela, 2007) can increase this number.

Neural theories of attentive visual search (Treisman & Gelade, 1980; Grossberg, Mingolla & Ross, 1994) take for granted that visual search is an active scan in which individuals build up information about the identity of each object. These ‘memory driven search models’ (Sternberg, 1969; Treisman & Sato, 1990; Grossberg, Mingolla & Ross, 1994) would expect that, if targets and distractors were randomized, search efficiency would reduce—these predictions would be the same in both serial and parallel models.

However, Horowitz and Wolfe (1998) challenged these previous assumptions by producing a randomized search experiment in order to stop subjects using memory. In the first instance, participants had to say whether or not the target letter appeared in the display. In 50% of the trials there was T present, requiring the answer ‘Yes’, and in the other 50%  there were only Ls present, requiring the answer ‘no’.

They pointed out that that efficiency for both static and randomized conditions were statistically similar: indeed reaction time X set size slopes were the same in both conditions (Horowitz & Wolfe, 2001). They claimed that visual search was ‘memory free’—thus, individuals do not gather information using working memory during a trial. In short, there is no previous attentional deployment (Hout & Goldinger, 2011). This supports the view that visual memory is unexpectedly weak (Simons & Levin, 1997; Wolfe, 1999). They tested this by re-positioning letters at randomly-chosen locations every 111 milliseconds. They pointed out that ‘memory driven search mechanisms’—parallel or serial—would not be able to cope with the randomization of locations and orientation: a serial model would force the individual to revisit targets and this would cause problems during parallel processing. However, by contrast, an amnesic visual search method would mean that randomization of targets would make no difference to response times. The authors concluded that there is only a limited amount of ‘integration of visual between saccadic eye movements’ (Horowitz & Wolfe, 1998, p577): indeed, after a masking frame or flicker was presented, individuals were often unable to perceive changes in stimuli. Further, they intimated that humans are unable to process vividly all the stimuli which we see, and that we pick and choose objects that are important to us from one eye movement to the next.

Conclusion

Perhaps, we should think of visual search as a system in we use ‘pre-attentive processes’ to guide attention (Wolfe, 2003) or perhaps many objects are being attended to at the same time, but they take several fleeting glances across the saccades to be processed. This supports the view that we all process stimuli in both serial and parallel (Nakayama & Silverman, 1986). Alternatively, the only serial mechanism involved in search is spontaneous eye movement—between 3 or 4 times a second—and we are all processing information in parallel (Sanders & Houtmans, 1985; Wolfe, 2003).

But there has been increasing evidence that there is memory in visual search. Chun and Jiang (1998) found that there was implicit memory and Krisjansson, Mackeben and Nakayama (1999) revealed that subjects learnt very quickly the positioning between cue and target during active search. In order to answer the question whether visual search has memory, it is important to go beyond gathering reaction time data, and looking at speed accuracy; and, further, more robust datasets need to be analyzed in order to produce empirical evidence to support one or other of the opposing theories.
Word count: 1,635

References

Baddeley, A.D. (1986). Working Memory. Oxford, England: Oxford University Press.

Chun, M.M., & Jiang, Y. (1999). Top-down attentional guidance based on implicit learning
of visual covariation. Psychological Science, 10, 360–365.

Chun, M. M., & Phelps, E. A. (1999). Memory deficits for implicit contextual information in amnesic subjects with hippocampal damage. Nature Neuroscience, 2, 844-847.

Kristjánsson, A. (2000). In search of remembrance: Evidence for memory in visual search. Psychological Science, 11(4), 328-332.

Gilmore, G. C., Tobias, T. R., & Royer, F. L. (1985). Aging and similarity grouping in visual search. Journal of Gerontology, 40(5), 586-592.

Grossberg, S., Mingolla, E. & Ross, W. D (1994). A neural theory of attentive visual search: interactions of boundary, surface, spatial, and object representations. Psychological Review, 101, 470–489.

Horowitz, T.S., & Wolfe, J.M. (1998). Visual search has no memory. Nature, 394, 575–577.

Horowitz, T. S., & Wolfe, J. M. (2001). Search for multiple targets: Remember the targets, forget the search. Attention, Perception, & Psychophysics, 63(2), 272-285.

Hout, M. C., & Goldinger, S. D. (2011, May 16). Incidental Learning Speeds Visual Search by Lowering Response Thresholds, Not by Improving Efficiency: Evidence From Eye Movements. Journal of Experimental Psychology: Human Perception and Performance. Advance Online Publication. doi: 10.1037/a0023894

Irwin, D.E., & Gordon, R.D. (1998). Eye movements, attention, and trans-saccadic memory.
Visual Cognition, 5, 127–155.

Klein, R. M. (2000). Inhibition of return. Trends in cognitive sciences, 4(4), 138-147.

Klein, R., & MacInnes, W.J. (1999). Inhibition of return is a foraging facilitator in visual search. Psychological Science, 10, 346–352.

Kristjánsson, A. (2000). In search of remembrance: Evidence for memory in visual search. Psychological Science, 11(4), 328-332.

Kristjánsson, A., Mackeben, M., & Nakayama, K. (1999). Exogenous attention shows adaptation to local and changing validities within an exogenous cue. Investigative Ophthalmology and Visual Science, 40, 280.

Luck, S.J., & Vogel, E.K. (1997). The capacity of visual working memory for features and
conjunctions. Nature, 390, 279–281.

Pashler, H. (1994). Overlapping mental operations in serial performance with preview. The Quarterly Journal of Experimental Psychology, 47(1), 161-191.

Peterson, M. S., Beck, M. R., & Vomela, M. (2007). Visual search is guided by prospective and retrospective memory. Attention, Perception, & Psychophysics, 69(1), 123-135.

Peterson, M. S., Kramer, A. F., Wang, R. F., Irwin, D. E., & McCarley, J. S. (2001). Visual search has memory. Psychological Science, 12(4), 287-292.

Sanders, A. F., & Houtmans, M. J. M. (1985). Perceptual processing modes in the functional visual field. Acta Psychologica, 58(3), 251-261.

Schneider, W.X. (1995). VAM: A neuro-cognitive model for visual attention
control of segmentation, object recognition, and space-based motor action. Visual Cognition, 2, 331-376.

Scholl, B. J., Simons, D. J., & Levin, D. T. (2004). Change blindness” blindness: An implicit measure of a metacognitive error. In D. T. Levin (Ed.), Visual metacognition: Thinking about seeing (pp. 145–163). Cambridge, MA: MIT Press.

Simons, D. J., & Levin, D. T. (1997). Change blindness. Trends in cognitive sciences, 1(7), 261-267.

Sternberg, S. (1969). High-speed scanning in human memory. Science, 153, 652-654.

Takeda, Y. (2002). Visual search has memory: evidence from multiple target search. Technical Report on Attention and Cognition, No.13.

Treisman, A. & Gelade, G (1980). A feature–integration theory of attention. Cognitive Psychology, 12, 97–136.

Treisman, A., & Sato, S. (1990). Conjunction search revisited. Journal of Experimental Psychology: Human Perception & Performance, 16, 459-478.

Wolfe, J. M. (1999). Inattentional Amnesia. In V Colheart (Ed), Fleeting memories, 71-94. Cambridge, MA: MIT Press.
Back to Hypnotherapy Glossary