This article was downloaded by: [University of New South Wales] On: 1 October 2010 Access details: Access Details: [subscription number 907420616] Publisher Taylor & Francis Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 3741 Mortimer Street, London W1T 3JH, UK
The International Journal of Aviation Psychology
Publication details, including instructions for authors and subscription information: http://www.informaworld.com/smpp/title~content=t775653651
Mental Imagery in Air Traffic Control
Steven T. Shorrocka; Anne Isaacb a Department of Aviation, University of New South Wales, Sidney, Australia b NATS, Whiteley, Hampshire, UK Online publication date: 30 September 2010
To cite this Article Shorrock, Steven T. and Isaac, Anne(2010) 'Mental Imagery in Air Traffic Control', The International
Journal of Aviation Psychology, 20: 4, 309 — 324 To link to this Article: DOI: 10.1080/10508414.2010.487008 URL: http://dx.doi.org/10.1080/10508414.2010.487008
PLEASE SCROLL DOWN FOR ARTICLE Full terms and conditions of use: http://www.informaworld.com/terms-and-conditions-of-access.pdf This article may be used for research, teaching and private study purposes. Any substantial or systematic reproduction, re-distribution, re-selling, loan or sub-licensing, systematic supply or distribution in any form to anyone is expressly forbidden. The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material.
THE INTERNATIONAL JOURNAL OF AVIATION PSYCHOLOGY, 20(4), 309–324 Copyright © 2010 Taylor & Francis Group, LLC ISSN: 1050-8414 print / 1532-7108 online DOI: 10.1080/10508410105084142010487008
Mental Imagery in Air Traffic Control Downloaded By: [University of New South Wales] At: 21:23 1 October 2010
Steven T. Shorrock1 and Anne Isaac2 1Department
of Aviation, University of New South Wales, Sidney, Australia 2NATS, Whiteley, Hampshire, UK
Mental imagery is known to play a significant role in the skilled performance of complex cognitive tasks, yet is mostly overlooked in the field of air traffic control—a task that is reliant on what controllers term “the picture.” This article explores 3 strands of imagery research: the similarities between imagery and perception, individual differences in imagery, and skill learning and imagery. The research reported is discussed in terms of fundamental implications for air traffic control, implications for the measurement of imagery, implications for training, and implications for technology design.
Air traffic controllers colloquially refer to their “mental picture” (Whitfield & Jackson, 1982) as a vital cognitive component of their job. In the literature, the picture has often been equated with the concept of situation awareness (Niessen & Eyferth, 2001), mental models (Mogford & Tansley, 1991), or both (Mogford, 1997). Several researchers have indicated that an important part of this picture is a three-dimensional mental image of the airspace and traffic (Isaac, 1994a, 1994b; Kirwan et al., 1998; MacKendrick, Atkinson, & Kirwan, 1999; Mogford, 1997; Sperandio, 1977). However, discussion of the role and nature of mental imagery in air traffic control (ATC) has rarely extended beyond this. Imagery can be defined as “a mental representation of an earlier sensory stimulus or experience and represents a less vivid copy of that event” (Roeckelein, 2006, p. 295). The image is a construction rather than a reproduction of an earlier sensory experience, and is mentally adjustable, so that we can image and manipulate what we have not previously seen. Mental imagery can also take a number of forms corresponding to the senses.
Correspondence should be sent to Steven T. Shorrock, Department of Aviation, The University of New South Wales, Sydney, NSW, 2052, Australia. E-mail:
[email protected]
310
SHORROCK AND ISAAC
Downloaded By: [University of New South Wales] At: 21:23 1 October 2010
Mental imagery might assist the controller in a number of specific ways. For instance, mental imagery might be used to help support memory, represent traffic scenarios, project future aircraft trajectories and make aircraft separation judgements, solve problems, and learn and practice skills. The aim of this article is to examine some of the imagery research that could be pertinent to current and future ATC, and to explore some of the possible implications and applications of this research. The article begins with a number of sections on relevant research, and ends with a section that draws on the research with regard to implications and applications for ATC.
PERCEPTION AND IMAGERY Mental imagery is a difficult and controversial subject in psychology, with debate regarding the nature of representations and even about whether images exist. Roeckelein (2006) stated that imagery theories can be placed in two camps proposed by Dennett (1981): the iconophiles (or pictorialists), who attribute special properties to mental imagery representations, especially giving the “spatial” or “pictorial” nature of images a high theoretical status; and the iconophobes (or descriptionists), who assert that images are represented mentally in the same way as other forms of thought, without giving special status to some intrinsic representational property. “The question is whether any of one’s mental representations are more like pictures or maps than like sentences” (Dennett, 1981, p. 175). The balance of empirical evidence suggests that propositions and images are distinct coding systems. Furthermore, there is substantial evidence that imagery and perception are similar in function, structure, and processing. This section presents a brief review of these similarities, which have relevance to, and possible implications for, human performance in ATC. Functional Similarities A large body of research has shown that people can perform to a similar level of task proficiency when either presented with a visual stimulus or while forming an analogue visual image. One source of evidence for this functional similarity arises from memory experiments. For instance, there is much evidence that imaginal coding of words or letters has similar effects on recall as the perception of pictures. For instance, Denis (1975) found that participants who were asked to image nouns performed to a similar level in free-recall tests as participants who were presented with pictures. Peterson (1975) had participants remember the locations of letters within a grid and found that letters at the periphery of the matrix were recalled more often in both perceptual (visual presentation) and imaginal (oral presentation) conditions. Engelkamp and Krumnacker (1980) found that in memory for
MENTAL IMAGERY IN AIR TRAFFIC CONTROL
311
phrases describing actions, watching a film of the action and imaging the actions produced equivalent effects, which were inferior to actually performing the action, but superior to verbal learning. Many researchers have claimed that these findings indicate that imagery and perception have structural similarities, in that they have a common internal organization.
Downloaded By: [University of New South Wales] At: 21:23 1 October 2010
Structural Similarities Studies on mental scanning have shown that images have an internal structure that is similar to the structure of the original perceptual experience. Kosslyn, Ball, and Reiser (1978) conducted an experiment involving scanning of a mental image of a previously seen map. They found a strong correlation between the response latency during mental scanning and the actual distance between the features of the map. This effect did not hold when participants were not instructed to form an image of the map. Kosslyn’s experiments on the limits and constraints of imagery provide further evidence of structural similarity. For instance, details of a small object are more difficult to discriminate than details of a larger object. These experiments have shown that images have an internal structure, and that this structure is similar to that of perception. In this way, images and percepts are structurally isomorphic, sharing a point-to-point correspondence. Similarities in the surface structure of images and percepts suggest that there are similarities in the mechanisms that operate on both perceptual and imaginal information. Denis (1989) posited that there are good reasons for hypothesizing that the same mechanisms govern some aspects of visual perception and visual imagery. Processing Similarities Several experiments provide evidence of similarities in the processes engaged by imagery and perception. Podgorny and Shepard (1978) found that similar pattern-analyzing mechanisms are implemented in the visual system when people observe an object and when they imagine the same object. Response latencies in both conditions varied similarly as a function of several parameters relating to figural complexity, location, and distance. Finke and Schmidt (1977) demonstrated that the “McCollough effect” occurs both in perception and in imagery (this effect is observed after perceiving vertical black bars on a red background followed by horizontal black bars on a green background). When looking at a plain background after these patterns, the participant sees a black grid tinged with another complementary color. This effect was also found when participants observed only the two colors and imagined the gratings. When participants were divided into vivid and nonvivid imagers as a function of their responses on a subjective imagery questionnaire,
Downloaded By: [University of New South Wales] At: 21:23 1 October 2010
312
SHORROCK AND ISAAC
only the vivid imagers made a significant number of color judgments in the predicted direction. A number of other experiments have tried to determine the resolution of images versus perception. Finke and Kosslyn’s (1980) participants made judgments of resolution on two small dots that they either imagined or actually observed. As the distance between the two dots increased, the size of fields of resolution in imagery increased in proportion to increases in the size of fields of resolution in perception. For vivid imagers, fields of resolution in imagery were the same size as those in perception, whereas for nonvivid imagers, fields of resolution in imagery were smaller than those in perception. In addition, fields of resolution in imagery and perception were virtually identical in shape. Finke and Kurtzman (1981) replicated these results, and also found that imagery vividness scores correlated with the size of the fields measured in the imaginal condition, but not in the perceptual condition. Another set of results has shown some selective interference effects between imagery and perception, such that the detection of visual signals is lowered by imagery in the same sensory modality. This effect reduces when the imagery is in a different sensory modality (e.g., auditory). For example, Segal and Fusella (1970) found that people performed much less accurately in detecting the physical stimulus when the mental image was in the same sensory mode (i.e., both visual or both auditory), compared to when the physical stimulus and mental image were in different modes. Craver-Lemley and Reeves (1992) discussed the evidence in detail and found that optics (fixation, pupil size, accommodation), response bias, global attention (effort, diversion of attention to imagery), perceptual assimilation (target incorporation by imagery), and perceptual masking (of target by imagery) all fail to explain the effect, and concluded from a series of experiments that imagery acts in a fashion that is equivalent to reducing target energy in the region of the visual field in which the image is located. Other studies have shown that the perceptual detection of a stimulus is facilitated by imagery of the same stimulus. For instance, Farah (1985) found that if an imagined letter matched a stimulus letter in identity and location, imagery facilitated the task, whereas if an imagined letter differed from the stimulus in location, identity, or both, then letter detection was impeded. In another study, Freyd and Finke (1984) found that length discrimination was facilitated by a real or imagined context frame, but not by a real or imagined inappropriate context frame. The facilitation effect might reflect processes other than the reduction in visual sensitivity (Craver-Lemley & Reeves, 1992). There is also evidence that visual images and visual percepts can be integrated to form a new composite representation (Brockmole, Wang, & Irwin, 2002). A number of oculomotor studies have also found a relationship between eye movements and visual imagery in both vertical and horizontal planes (Antrobus, Antrobus, & Singer, 1964; Singer & Antrobus, 1965). Hebb (1968) proposed an
Downloaded By: [University of New South Wales] At: 21:23 1 October 2010
MENTAL IMAGERY IN AIR TRAFFIC CONTROL
313
inherent oculomotor component in visual perception and visual imagery, and that oculomotor patterns in imagery are not irrelevant but essential. Farah (1985) subsequently found that imaging participants fixated their eyes on parts of a blank display screen that would contain the most salient parts of the imaged picture. More recently, Laeng and Teodorescu (2002) found support for this idea in a series of experiments that showed that eye scan-paths during visual imagery reenact those of perception of the same visual scene and that they play a functional role. They concluded that eye movements during mental imagery are not epiphenomenal, but assist the process of image generation. These cognitive and behavioral findings are complemented by a body of research that finds that visual imagery and visual perception share common neural architecture (e.g., see Farah, 1988; Ganis, Thompson, & Kosslyn, 2004; Ishai & Sagi, 1995; Tippett, 1992). J. T. E. Richardson (1999) cited several studies employing EEG and positron emission tomography that have found that the occipital lobe is activated during the experience of visual imagery. This contains the region of the brain that is responsible for the initial analysis of visually perceived information. Other studies have found that the visual cortex is widely activated during imagery (e.g., Ganis et al., 2004; Kosslyn et al., 1993; Kosslyn, Thompson, & Alpert, 1997; Kosslyn, Thompson, Kim, & Alpert, 1995).
INDIVIDUAL DIFFERENCES IN IMAGERY Although there are strong similarities between imagery and perception generally, people differ in their imagery ability and use this skill in a number of ways. Denis (1989) proposed three major individual differences in visual imagery in the context of skill learning and practice: vividness, controllability, and accuracy of reference. Vividness refers to the clarity of evoked images. This is believed to be the rate of activity of the mental processes underlying the experience of imagery. Hishitani and Murakami (1992) found that sketches produced by vivid imagers were independently judged more vivid, colorful, dynamic, widespread, and detailed than those of nonvivid imagers, and it appeared that vivid imagers could observe and assess any part of the image. Vivid imagers also produced more stable images and were able to image more objects at once. Postexperimental reports showed that vivid imagers reported using imagery in daily life to a significantly greater extent than nonvivid imagers. Paivio (1975) and Kosslyn et al. (1978) found that just as complex perceptual figures take more time to process than simple perceptual figures, more time is required to create an image as the number of details increases. Controllability refers to the ease with which a person can manipulate images in terms of activation, movement, and timing. Denis (1989) posited that once an image has been generated in the visual buffer, “refreshing” processes are applied to
Downloaded By: [University of New South Wales] At: 21:23 1 October 2010
314
SHORROCK AND ISAAC
the image to maintain it at a sufficiently high level of activation. These processes are partially under the person’s control, and the person can also choose when an image should be interrupted to make way for a new image. In this way, images persist only for the time that they are required. Denis also suggested that excessive vividness of visual imagery can place limits on controllability. A number of factors can increase the duration of images, including active exploration of images (Kosslyn, 1980), moving images (Cocude & Denis, 1988), and images with high emotional content (Cocude, 1988). Cocude and Denis (1986) found that people who are regularly involved in imagery-inducing activity (experienced yoga practitioners) have significantly longer image duration than both control participants and less experienced imagery users (novice yoga practitioners). Accuracy of reference refers to the accuracy of the figural content of the image in terms of dimensions, magnitude, and directions of movement. This relates closely to the structural similarity of images and percepts as discussed earlier. A large body of research has found that people differ in their ability to use mental imagery. Most of this research has focused on reported vividness of imagery; relatively few studies have examined controllability and accuracy of reference. Commonly, subjective questionnaires are used to measure imagery ability—perhaps the most widely cited being Marks’s (1973) Vividness of Visual Imagery Questionnaire (VVIQ). This requires participants to rate the vividness of 16 images of common situations along a 5-point scale. Scores were found to be reliable predictors of accuracy in the recall of information contained in pictures, and have subsequently been found to be a predictor of performance in a variety of cognitive, motor, and creative tasks. Gordon’s (1949) earlier Test of Visual Imagery Control is less often employed as a measure of one’s ability to control visual images. Similarly, other questionnaires have been designed to assess imagery of movement by a person, or control of on object by a person, from internal (kinesthetic) and external (visual) perspectives. These include the Movement Imagery Questionnaire (Hall & Pongrac, 1983), and the Vividness of Movement Imagery Questionnaire (VMIQ; Isaac, Marks, & Russell, 1986). Studies that have used subjective imagery questionnaires have often found differences in imagery ability between individuals and groups, and differences in performance on a large number of different tasks according to imagery ability. Isaac and Marks (1994) performed five large-scale studies using the VVIQ and VMIQ, involving several hundred individuals in a large range of age groups, occupational specializations, and abilities. Among other findings, Isaac and Marks found that students studying English reported the least vivid visual imagery and male physics students and female physical education students reported the most vivid visual imagery. Male physical education students reported the most vivid movement imagery. A further study found that elite athletes from a number of sports report more vivid visual and movement imagery than nonathletes.
Downloaded By: [University of New South Wales] At: 21:23 1 October 2010
MENTAL IMAGERY IN AIR TRAFFIC CONTROL
315
Such differences extend to image manipulation. More recently, Ozel, Larue, and Molinaro (2004) found that athletes performed a mental rotation task significantly faster than nonathletes, confirming previous studies. Similar effects have been found in the aviation domain. Dror, Kosslyn, and Waag (1993) found experimentally that U.S. Air Force pilots have exceptional abilities in mentally rotating objects and judging metric spatial relations. In contrast, Dror et al. did not find evidence that pilots have unusual abilities to scan visual mental images or extrapolate motion. This is consistent with the finding, noted by Dean and Morris (2003), that little or no correlation has been found between measures based on subjective reports of the conscious experiences of imagery (e.g., the VVIQ) and experimental tasks or spatial tests that are explained in terms of the use or manipulation of mental images.
SKILL LEARNING AND IMAGERY Mental imagery has a practical application in performance psychology in the form of mental practice—the symbolic rehearsal of a task in the absence of any gross muscular movements (A. Richardson, 1967). A large corpus of research indicates that motor and cognitive skills can be enhanced by practicing the task in imagination (for meta-analyses and reviews, see Driskell, Copper, & Moran, 1994; Feltz & Landers, 1983; Grouios, 1992; Hinshaw, 1991). A common finding is that mental practice is superior to no practice, although inferior to physical practice, and that a combination of mental and physical practice is most effective. Some studies have found mental practice to be superior to physical practice (e.g., McKenzie & Howe, 1991), and others have found mental practice to be ineffective (e.g., Andre & Means, 1986). Differences in the type of mental practice can partly explain these differences in findings. These include the nature of the task or skill (e.g., cognitive, motor; open, closed; motion, no motion), type of imagery instructions (e.g., visual, kinesthetic; internal, external), skill level, imagery ability, individual factors, motivation, and number and duration of trials (Hall, Schmidt, Durand, & Buckolz, 1994; Van Meer & Theunissen, 2009). Feltz and Landers (1983) found that of 16 coding characteristics involving participant, task, and design characteristics, by far the most important and robust was whether the task was primarily motor, cognitive, or strength. Cognitive tasks, such as maze learning, pegboard tests, and symbol digit tasks, had much larger effect sizes than tasks that involved mainly motor processes, which in turn exceeded the effects of task requiring strength. Furthermore, cognitive tasks required shorter and fewer mental practice sessions. This finding was supported by Driskell et al. (1994). Wrisberg and Ragsdale (1979) investigated whether the effectiveness of mental practice was related to practice level. Mental practice was introduced either early
Downloaded By: [University of New South Wales] At: 21:23 1 October 2010
316
SHORROCK AND ISAAC
or later in the acquisition of two motor tasks of varying cognitive demand. The results indicated that mental practice facilitated motor performance as an increasing function of the cognitive demand of a task, and as a decreasing function of the level of previous practice. This suggests that mental practice should be of more benefit to the novice performer because the cognitive component of the task is likely to be highest during this stage of learning. Feltz and Landers’s (1983) meta-analysis showed that mental practice effects are found in both early and later stages of learning. In the early stages of learning, mental practice might provide a rough schema of the cognitive elements of the task. Also, less proficient performers have more room for improvement provided they can imagine themselves performing the task efficiently. Skilled performers have the advantage of a rich experiential base for optimal precision in imagery, and mental imagery could develop the schema of cognitive elements more fully. Driskell et al.’s (1994) meta-analysis found that novices benefited more from mental practice on cognitive tasks, whereas experienced participants benefited equally well from mental practice, regardless of task type. O’Halloran and Gauvin (1994) found that preferred cognitive style (verbal or imagic) also mediated the effectiveness of mental practice of a motor task. Paivio (1985) proposed an analytical framework for mental practice research that indicated that imagery played both a motivational and cognitive role in mediating behavior, with each operating at either a general or specific level. At the motivational level, imagery might have a general function of managing arousal and affect, and a specific function of representing goals and activities. At the cognitive level, imagery might have a general function of representing general strategies, and a specific function of practicing specific skills.
DISCUSSION, IMPLICATIONS, AND APPLICATIONS Fundamental Implications Before examining the possible uses for imagery in ATC, it is necessary to ask whether controllers do generate and actively use mental images, in conjunction with perceptual experience, to help project possible aircraft movements in time and space. Alternatively, is the controller’s mental picture just the amalgam of visual perception and propositional knowledge from long-term memory? Hopkin (1995) argued that the controller’s mental picture is mainly pictorial. This appears to be supported by evidence. Isaac (1994a) conducted research in New Zealand and Australia that suggested that controllers report using imagery extensively in their work. Isaac and Marks (1994) examined imagery ability in controllers and found that the controllers had significantly more vivid visual and movement imagery than age- and gender-matched controls. They also found a sig-
Downloaded By: [University of New South Wales] At: 21:23 1 October 2010
MENTAL IMAGERY IN AIR TRAFFIC CONTROL
317
nificant correlation between controllers’ imagery ability scores and supervisors’ competency ratings. Kirwan et al. (1998) conducted a series of interviews with controllers on the nature of their picture. Typical responses from the controllers were that the picture was two-, three-, or four-dimensional (including time). Some reported that the picture was pictorial and others reported that the “picture” was nonvisual. Mogford (1997) showed in an experiment that ATC trainees were able to recall spatial data reliably during a freeze in a real-time ATC simulation (aircraft’s position—86% of aircraft; heading—82%; and altitude—73%), and with much greater accuracy than nonspatial data (identifier—55% of aircraft; speed— 53%). There is very little research on the use of imagery in ATC, but the evidence available suggests that imagery plays a role in the representation of the picture for some controllers. This has possible relevance to all disciplines within ATC, but perhaps in different ways. Radar and tower controllers might combine imagery with direct perception, as has been shown to be possible experimentally (Brockmole et al., 2002). Nonradar controllers (e.g., working with oceanic flights) might, in the absence of a dynamic visual representation, be more reliant on an internal representation of the airspace that they control. Implications for the Measurement of Imagery If controllers use imagery, then further investigation is reliant on valid and reliable measurement. Several questionnaires assess various aspects of imagery subjectively. Paivio (1985) argued that there is no single best measure of imagery ability because imagery skills are at least as varied as verbal skills, so it might be necessary to find the instrument that is most directly related to the specific task. Few questionnaires seem to tap the particular aspects of imagery that might be used in ATC. Visual imagery questionnaires focus on vividness of (predominantly) static visual imagery, whereas movement imagery questionnaires tend to explore kinesthetic imagery or visual imagery of movement from a third-person perspective. There is currently no measure of imagery ability that is particularly well suited to ATC. Marks’ (1973) VVIQ appears to be the most widely used imagery vividness questionnaire. However, for air traffic controllers, it is likely that controllability and accuracy of reference are equally or more important properties. It is important that a controller who is manipulating an image be able to generate the image, maintain the image, inspect the image, and transform the image (Kosslyn, Van Kleeck, & Kirby, 1990). Particularly problematic for self-report measures such as the VVIQ is the fact that they do not correlate well with tests of spatial ability, possibly again because subjective measures mostly focus on image generation, inspection, and maintenance rather than manipulation. Promisingly, a test devised by Dean and Morris (2003), based on Kosslyn’s theories of the imagery system, consistently correlated
Downloaded By: [University of New South Wales] At: 21:23 1 October 2010
318
SHORROCK AND ISAAC
with performance on the spatial tests, whereas ratings from the VVIQ did not. The new ratings captured several properties of images, rather than vividness alone, and capture more of the imagery process. Ratings of items of the same type as used on the spatial tests predicted performance on the spatial tests, whereas vividness ratings of items recalled or constructed from long-term memory did not. This questionnaire could be more compatible with research and application in ATC. Eye movement tracking might represent another avenue for the investigation of imagery. Farah (1985) showed that imaging participants fixated their eyes on parts of a blank display screen that would contain the most salient parts of the imaged picture. This could be of particular relevance to radar controllers. Indeed, in a study of recovery from equipment failure in ATC, Low and Donohoe (2001) found using eye movement tracking that controllers’ visual scans on blank radar displays following a radar display failure were similar to the scans prior to the failure. Implications for Training One potential application of imagery in ATC is via training: to increase imagery ability, to improve the use of imagery, and to practice tasks and skills. Imagery ability seems to be relevant to controller performance. Isaac and Marks (1994) found that not only did air traffic controllers have significantly more vivid visual and movement imagery than controls, but also that imagery ability was significantly correlated with supervisors’ ratings of controllers’ competencies. Isaac (1994b) found that controllers who reported clear imagery were able to regain effective control more quickly after radar displays were switched off mid-exercise during a simulation. The finding that those with more vivid imagery have larger fields of imagery resolution than nonvivid imagers (Finke & Kosslyn, 1980; Finke & Kurtzman, 1981) might imply that controllers with more vivid imagery are able to hold a larger or more complex image of the traffic situation than controllers with less vivid imagery. It is difficult to determine whether differences in imagery ability are attributable to training or selection. However, there is empirical evidence that image duration can be prolonged by regular use of imagery (Cocude & Denis, 1986), active exploration of images (Kosslyn, 1980), and use of moving images (Cocude & Denis, 1988). This suggests that controllers’ image maintenance should benefit from rehearsal, and might predict that controllers have imagery that is of higher duration than that of other individuals. Interestingly, Isaac and Marks (1994) found that air traffic controllers found it easier to induce images with eyes open; this is the opposite situation to that in the general population, perhaps suggesting a training effect. Because imagery has been shown to produce clear interference effects in perception, there might be aspects of control for which imagery is detrimental. Although we have found no research of this kind in ATC, Recarte and Nunes (2000)
Downloaded By: [University of New South Wales] At: 21:23 1 October 2010
MENTAL IMAGERY IN AIR TRAFFIC CONTROL
319
performed a driving study using an eye tracking system and an instrumented car in real traffic. While driving, participants performed verbal tasks and spatial-imagery tasks with their eye movements recorded, which incurred similar effort (measured by pupillary dilation). It was found that the visual inspection window reduced, particularly for spatial-imagery tasks. This was interpreted as possibly associated with narrowing of the attentional focus size. While performing spatial-imagery tasks there was an increment in mean fixation duration, possibly associated with mental image inspection. Glance frequency at mirrors and the speedometer decreased during the spatial-imagery task. This study suggests that interference effects generalize outside the laboratory. If they apply also to ATC, then perhaps it is possible that using imagery to predict the effects of an intended instruction might adversely affect the controller’s ability to visually monitor the current situation. In this way, imagery could be a double-edged sword. Mental practice has not been explored in the ATC literature, although its benefits in the sport and performance literature are very widely reported. Other applications have recently emerged. Mental practice has been found to be effective in surgery training (e.g., Bathalon, Dorion, Darveau, & Martin, 2005; Immenroth et al., 2007; Komesu et al., 2009), and there is interest in the use of mental practice in general education and instruction for complex cognitive subjects (e.g., Van Meer & Theunissen, 2009). Such applications are consistent with the view that tasks that are high in cognitive elements benefit more from mental practice (Driskell et al., 1994; Feltz & Landers, 1983). As such, mental practice could facilitate ATC training. There appears to be little or no published material on this issue, although Isaac (1997) provides some experience-based guidance on this application. Following Paivio’s (1985) framework of cognitive and motivational functions of imagery, on a cognitive level, controllers might use mental practice to help plan strategies and provide task schemas, forming general image-based plans for aircraft movements. Controllers could also mentally practice specific skills, such as specific conflict scenarios, auditory controller–pilot radio–telephone interactions, or perhaps tactile and kinesthetic sensations associated with the work (e.g., paper flight strip use). Alternatively, on a motivational level, controllers could use imagery to manage their levels of arousal and affect, or to set goals. A particularly threatening experience for a controller is the loss or breakdown of the picture, where information loses much of its meaning and operational significance (Kirwan et al., 1998). Again, there is almost no empirical research on the issue, particularly the role of imagery in this experience. Implications for Technology Design Understanding mental imagery in ATC could assist in the design of new technology. Hopkin (1989) argued that information would be understood more quickly if presented in forms that were compatible with the mental picture and require no
Downloaded By: [University of New South Wales] At: 21:23 1 October 2010
320
SHORROCK AND ISAAC
recoding. We suspect that, in reality, new technology, which moves away from simple plan view radar displays, for example, and toward more complex graphical and tabular displays, might well be changing controllers’ use of imagery. As controllers become increasingly dependent on display automation technology such as electronic flight data displays and conflict alerts, it becomes increasingly important to understand the effect of these on imagery and the picture, and the effect of imagery on visual perception of information. Could the increase in the use of automation tools for information integration, analysis, and alerting increase visual loading, and therefore inhibit imagery of the traffic situation? An example of this might be an unexpected visual message or dialogue box that requires focused attention to input. Alternatively, could imagery affect perception of information? As noted previously, there is evidence of interference and facilitation effects between imagery and perception. If this effect generalizes beyond the laboratory, it is possible that imagery could assist controller detection and perception when images and percepts accord (e.g., aircraft movements or conflicts), but reduce the sensitivity of perception when images and percepts differ (e.g., inaccurate image projections or images associated with preoccupation). Ultimately, with increasing and more complex technology and traffic, the importance and role of the mental picture, and mental imagery, is in the balance, with one approach to protect the picture and the other to dispense with it. It is important that the risks and benefits of each approach are understood.
FUTURE DIRECTIONS Imagery research is of particular relevance to ATC because the tasks of the controller depend considerably on mental spatial representations. In light of the rapidly changing nature of ATC, the role of the controller, and the extent and nature of automation deployed, we suggest several areas that merit further research. In light of the dearth of research in this area, there are several areas of future research and application that should be considered. First, controllers often refer to their mental picture as fundamental to their role, but the role of imagery in the creation of this picture is as yet unclear. In particular, to what extent do different controllers use imagery, combined with knowledge and perceptual experience, to aid retention of information, predict aircraft trajectories, make judgments, and more generally to maintain their understanding of the current and future traffic situation? It would be fruitful to examine further the use of imagery, the method of use, and any differences in the extent of the use of mental imagery, depending on type of control (e.g., procedural, radar, visual), training experience, type of equipment used, and other relevant factors. Can controllers with more vivid imagery hold a larger or more complex image of the traffic situation,
Downloaded By: [University of New South Wales] At: 21:23 1 October 2010
MENTAL IMAGERY IN AIR TRAFFIC CONTROL
321
and perhaps handle more aircraft movements or more complex scenarios? There is no published research on this issue. Second, further research is required on the individual differences in imagery in ATC. The relative importance of vividness, controllability, and accuracy of images; how they are trained; and how they support the controllers’ tasks, needs to be better understood. It is also important to investigate further how imagery ability in ATC can be measured. Knowing how imagery ability can be enhanced by training, and perhaps understanding how it might be assessed during selection, would be valuable. The answers to these questions might require new or adapted measures of imagery ability, or further investigation of those that show promise, such as questionnaires, eye movement tracking, or other measures. This research might also help to attribute differences in imagery ability to selection and training. Third, it would be beneficial to know whether and how controllers use imagery strategies to enhance skill learning, either in a planned and structured way during controller training or spontaneously by individual controllers. If controllers do use mental practice, it would be worthwhile to explore what they practice mentally, and whether this varies with cognitive style and stage of learning. Further research could examine the cognitive and motivational roles of imagery in such mental practice at various stages of learning and levels of expertise. This could then be incorporated more formally into training. Finally, understanding of the role of imagery could help optimize the introduction of increasing automation, such as electronic flight data, route editing, and conflict detection and resolution technology. It would be fruitful to explore which technologies facilitate and complement imagery, and which interfere with it. Understanding the role of imagery could also help to determine how and whether imagery is changing with the changing nature of the job. Almost nothing is known about whether imagery sometimes facilitates or interferes with the detection and perception of information in ATC, as is the case in laboratory settings. Subjective data and eye movement tracking might have a role to play in enhancing this understanding.
REFERENCES Andre, J. C., & Means, J. R. (1986). Rate of imagery in mental practice: An experimental investigation. Journal of Sports Psychology, 8, 124–126. Antrobus, J. S., Antrobus, J. S., & Singer, J. L. (1964). Eye movements accompanying daydreaming, visual imagery, and thought suppression. Journal of Abnormal and Social Psychology, 69, 244–252. Bathalon, S., Dorion, D., Darveau, S., & Martin, M. (2005). Cognitive skills analysis, kinesiology, and mental imagery in the acquisition of surgical skills. Journal of Otolaryngology, 34, 328–332. Brockmole, J. R., Wang, R. F., & Irwin, D. E. (2002). Temporal integration between visual images and visual percepts. Journal of Experimental Psychology: Human Perception and Performance, 28, 315–334.
Downloaded By: [University of New South Wales] At: 21:23 1 October 2010
322
SHORROCK AND ISAAC
Craver-Lemley, C., & Reeves, A. (1992). How imagery interferes with vision. Psychological Review, 99, 633–649. Cocude, M. (1988). Generating and maintaining visual images: The incidence of individual and stimulus characteristics. In M. Denis, J. Englekamp, & J. T. E. Richardson (Eds.), Cognitive lule à l’homme (pp. 123–167). Paris, France: L’Harmattan. Cocude, M., & Denis, M. (1986). The time course of imagery: Latency and duration of visual images. In D. G. Russell, D. F. Marks, & J. T. E. Richardson (Eds.), Imagery 2 (pp. 57–62). Dunedin, New Zealand: Human Performance Associates. Cocude, M., & Denis, M. (1988). Measuring the temporal characteristics of visual images. Journal of Mental Imagery, 12(1), 89–101. Dean, G. M., & Morris, P. E. (2003). The relationship between self-reports of imagery and spatial ability. British Journal of Psychology, 94, 245–273. Denis, M. (1975). Représentation imagée et activité de mémorisation (The representation of images and activities in memorisation). Paris: Éditions du Centre National de la Recherche Scientifique. Denis, M. (1989). Image and cognition. Hemel Hampstead, UK: Harvester Wheatsheaf. Dennett, D. C. (1981). Brainstorms: Philosophical essays on mind and psychology. Cambridge, MA: MIT Press. Driskell, J. E., Copper, C., & Moran, A. (1994). Does mental practice enhance performance? Journal of Applied Psychology, 79, 481–492. Dror, I. E., Kosslyn, S. M., & Waag, W. L. (1993). Visual-spatial abilities of pilots. Journal of Applied Psychology, 78, 763–773. Farah, M. J. (1985). Psychophysiological evidence for a shared representation medium for mental images and percepts. Journal of Experimental Psychology: General, 114, 91–103. Farah, M. J. (1988). Is visual imagery really visual? Overlooked evidence from neuropsychology. Psychological Review, 3, 307–317. Feltz, D. L., & Landers, D. M. (1983). The effects of mental practice on motor skill learning and performance: A meta-analysis. Journal of Sport Psychology, 5, 25–57. Finke, R. A., & Kosslyn, S. M. (1980). Mental imagery acuity in the peripheral visual field. Journal of Experimental Psychology: Human Perception and Performance, 6(1), 126–139. Finke, R. A., & Kurtzman, H. S. (1981). Mapping the visual field in mental imagery. Journal of Experimental Psychology: General, 110, 501–517. Finke, R. A., & Schmidt, M. J. (1977). Orientation-specific color after-effects following imagination. Journal of Experimental Psychology: Human Perception and Performance, 3, 599–606. Freyd, J. J., & Finke, R. A. (1984). Facilitation of length discrimination using real and imagined context frames. American Journal of Psychology, 97, 323–341. Ganis, G., Thompson, W. L., & Kosslyn, S. M. (2004). Brain areas underlying visual mental imagery and visual perception: An fMRI study. Cognitive Brain Research, 20, 226–241. Gordon, R. (1949). An investigation into some of the factors that favour the formation of stereotyped images. British Journal of Psychology, 39, 156–167. Grouios, G. (1992). Mental practice: A review. Journal of Sport Behavior, 15, 42–59. Hall, C. R., & Pongrac, J. (1983). Movement imagery questionnaire. London, ON, Canada: University of Western Ontario. Hall, C., Schmidt, D., Durand, M.-C., & Buckolz, E. (1994). Imagery and motor skills acquisition. In A. A. Sheikh & E. R. Korn (Eds.), Imagery in sport and physical performance (pp. 121–134). Amityville, NY: Baywood. Hebb, D. O. (1968). Concerning imagery. Psychological Review, 75, 466–477. Hinshaw, K. E. (1991). The effects of mental practice on motor skill performance: Critical evaluation and meta-analysis. Imagination, Cognition and Personality, 11, 3–35. Hishitani, S., & Murakami, S. (1992). What is vividness of imagery? Characteristics of vivid visual imagery. Perceptual and Motor Skills, 73, 1291–1307.
Downloaded By: [University of New South Wales] At: 21:23 1 October 2010
MENTAL IMAGERY IN AIR TRAFFIC CONTROL
323
Hopkin, V. D. (1989). Man–machine interface problems in designing air traffic control systems. Proceedings of the IEEE, 77, 1634–1642. Hopkin, V. D. (1995). Human factors in air traffic control. London, UK: Taylor & Francis. Immenroth, M., Bürger, T., Brenner, J., Nagelschmidt, M., Eberspächer, H., & Troidl, H. (2007). Mental training in surgical education: A randomized controlled trial. Annals of Surgery, 245, 385–391. Isaac, A. R. (1994a). Imagery ability and air traffic personnel. Aviation, Space and Environmental Medicine, 65(2), 95–99. Isaac, A. R. (1994b). Short-term memory and advanced technology—The use of imagery in air traffic control. In N. Johnston, R. Fuller, & N. McDonald (Eds.). Aviation psychology: Training and selection. Proceedings of the 21st Conference of the European Association for Aviation Psychology (Vol. 2, pp. 107–111). Dublin, Ireland: Trinity College. Isaac, A. R. (1997). Teaching for visualizing in air traffic control. In G. J. F. Hunt (Ed.), Designing instruction for human factors training in aviation (pp. 226–236). Aldershot, UK: Ashgate. Isaac, A. R., & Marks, D. F. (1994). Individual differences in mental imagery experience: Developmental changes and specialization. British Journal of Psychology, 85, 479–500. Isaac, A. R., Marks, D. F., & Russell, D. G. (1986). An instrument for assessing imagery of movement: The vividness of movement imagery questionnaire (VMIQ). Journal of Mental Imagery, 10, 23–30. Ishai, A., & Sagi, D. (1995). Common mechanisms of visual imagery and perception. Science, 5218, 1772–1774. Kirwan, B., Donohoe, L., Atkinson, T., MacKendrick, H., Lamoureux, T., & Phillips, A. (1998). Getting the picture: Investigating the mental picture of the air traffic controller. In M. A. Hanson (Ed.), Contemporary ergonomics (pp. 404–408). London: Taylor & Francis. Komesu, Y., Urwitz-Lane, R., Ozel, B., Lukban, J., Kahn, M., Muir, T., et al. (2009). Does mental imagery prior to cystoscopy make a difference? A randomized controlled trial. American Journal of Obstetrics and Gynecology, 201, 218. Kosslyn, S. M. (1980). Image and mind. Cambridge, MA: Harvard University Press. Kosslyn, S. M., Alpert, N. M., Thompson, W. L., Maljkovic, V., Weise, W. B., Chabris, C. F., et al. (1993). Visual mental imagery activates topographically organized visual cortex: PET investigations. Journal of Cognitive Neuroscience, 5, 263–287. Kosslyn, S. M., Ball, T. M., & Reiser, B. J. (1978). Visual images preserve metric spatial information: Evidence from studies of image scanning. Journal of Experimental Psychology: Human Perception and Performance, 4, 47–60. Kosslyn, S. M., Thompson, W. L., & Alpert, N. M. (1997). Neural systems shared by visual imagery and visual perception: A positron emission tomography study. Neuroimage, 4, 320–334. Kosslyn, S. M., Thompson, W. L., Kim, I. J., & Alpert, N. M. (1995). Topographical representations of mental images in primary visual cortex. Nature, 6556, 496–498. Kosslyn, S. M., Van Kleeck, M. H., & Kirby, K. N. (1990). A neurologically plausible model of individual differences in visual mental imagery. In P. J. Hampson, D. F. Marks, & J. T. E. Richardson (Eds.), Imagery: Current developments (pp. 39–77). London, UK: Routledge. Laeng, B., & Teodorescu, D.-S. (2002). Eye scanpaths during visual imagery reenact those of perception of the same visual scene. Cognitive Science, 26, 207–231. Low, I., & Donohoe, L. (2001). Methods for assessing ATC controllers’ recovery from automation failure. In D. Harris (Ed.), Engineering psychology and cognitive ergonomics: Vol. 5. Aerospace and transportation systems (pp. 161–169). Aldershot, UK: Ashgate. MacKendrick, H., Atkinson, T., & Kirwan, B. (1999). Understanding the controller’s picture within an ATM environment. In D. Harris (Ed.), Engineering psychology and cognitive ergonomics: Vol. 3. Transportation systems, medical ergonomics and training (pp. 154–163). Aldershot, UK: Ashgate. Marks, D. F. (1973). Visual imagery differences in the recall of pictures. British Journal of Psychology, 64, 17–24.
Downloaded By: [University of New South Wales] At: 21:23 1 October 2010
324
SHORROCK AND ISAAC
McKenzie, A. D., & Howe, B. L (1991). The effect of imagery on tackling performance in rugby. Journal of Human Movement Studies, 6, 127–141. Mogford, R. H. (1997). Mental models and situation awareness in air traffic control. International Journal of Aviation Psychology, 7, 331–341. Mogford, R. H., & Tansley, B. W. (1991). The importance of the air traffic controller’s mental model. In Proceedings of the 24th annual conference of the Human Factors Association of Canada, (pp. 135–140). Mississauga, ON, Canada: Human Factors Association of Canada. Niessen, C., & Eyferth, K. (2001). A model of the air traffic controller’s picture. Safety Science, 37, 187–202. O’Halloran, A. M., & Gauvin, L. (1994). The role of preferred cognitive style in the effectiveness of imagery training. International Journal of Sport Psychology, 25(1), 19–31. Ozel, S., Larue, J., & Molinaro, C. (2004). Relation between sport and spatial imagery: Comparison of three groups of participants. The Journal of Psychology, 138(1), 49–63. Paivio, A. (1975). Perceptual comparisons through the mind’s eye. Memory and Cognition, 3, 635–641. Paivio, A. (1985). Cognitive and motivational functions of imagery in human performance. Canadian Journal of Applied Sport Psychology, 10, 22S–28S. Peterson, M. J. (1975). The retention of imagined and seen spatial matrices. Cognitive Psychology, 7, 181–193. Podgorny, P., & Shepard, R. N. (1978). Functional representations common to visual perception and imagination. Journal of Experimental Psychology: Human Perception and Performance, 4, 21–35. Recarte, M. A., & Nunes, L. M. (2000). Effects of verbal and spatial-imagery task on eye fixations while driving. Journal of Experimental Psychology: Applied, 6, 31–43. Richardson, A. (1967). Mental practice: A review and discussion (Part 1). Research Quarterly, 38, 95–107. Richardson, J. T. E. (1999). Imagery. Hove, UK: Psychology Press. Roeckelein, J. E. (2006). Elsevier’s dictionary of psychological theories. Amsterdam, the Netherlands: Elsevier. Segal, S., & Fusella, V. (1970). Influence of imaged pictures and sounds on detection of visual and auditory signals. Journal of Experimental Psychology, 83, 458–464. Singer, J. L., & Antrobus, J. S. (1965). Eye movements during fantasies: Imagining and suppressing fantasies. Archives of General Psychiatry, 12, 71–76. Sperandio, J. C. (1977). La régulation des modes opératoires en fonction de la charge de travail chez les contrôleurs de trafic aérien (The regulation of operative modes as a function of workload in air traffic controllers). Le Travail Humain, 40, 249–256. Tippett, L. J. (1992). The generation of visual images: A review of neuropsychological research and theory. Psychological Bulletin, 112, 415–432. Van Meer, J. P., & Theunissen N. C. M. (2009). Prospective educational applications of mental simulation: A meta-review. Educational Psychology Review, 21, 93–112. Whitfield, D., & Jackson, A. (1982). The air traffic controller’s picture as an example of a mental model. In G. Johannsen & J. E. Rijnsdorp (Eds.), Proceedings of the IFAC conference on Analysis, Design, and Evaluation of Man–Machine Systems (pp. 45–52). London, UK: Pergamon. Wrisberg, C. A., & Ragsdale, M. R. (1979). Cognitive demand and practice level: Factors in the mental rehearsal of motor skills. Journal of Human Movement Science, 5, 201–208.
Manuscript first received: June 2008
