Unseen Screens: Eye Tracking, Magic and Misdirection

Curator's Note

By Tessa Dwyer and Jenny Robinson (Monash University and RMIT University)

Seeing’ is no simple matter. In fact, researchers from various disciplines from psychology to philosophy still grapple to understand many of the processes and variables involved in this imprecise, catch-all concept. ‘Seeing’ has so many moving parts, as does the relationship between visual attention and visual processing or cognition. This video essay provides an introduction to the eye tracking of moving-image screens that canvasses these complexities, starting with the idea that ‘seeing’ depends upon numerous instances of not-seeing and inattention. We explore this idea in relation to magic films and filmed magic, drawing on concepts from screen theory and psychocinematics.

As Dan North details in Performing Illusions (2005), the influence of magic traditions on the development of early cinema is profound. Magic films hark back to cinema’s earliest roots and are particularly well represented in the prolific output of conjurer and director George Méliès. While magic on screen is quite distinct from magic that unfolds in front of a live, active audience, this difference does not defuse its power or fascination. The magic film constitutes an enduring genre that boasts many recent entrants including Now You See Me (Leterrier, 2013) and sequel Now You See Me 2 (Chu, 2016).

What can eye tracking technology bring to an understanding of screen media and screen culture? Eye tracking is a process whereby eye movements are charted in order to record which parts of the visual field are attended to by audiences, for how long and in which order. Eye tracking also provides a means of drilling down into attention, exploring it as a multi-layered process that involves peripheral pre-viewing, fixations (or resting spots), information encoding and extraction, and saccades (the scanning movements that occur in-between fixations).

For this essay, we collected eye tracking data from twenty-one participants who viewed a short one-minute scene from the first Now You See Me. From this data, we produced heat-mapped footage collating the gaze patterns of all participants and displaying as hottest those areas that were most fixated upon. We also produced inverse footage of this same data. Instead of overlaying the image with heat maps that obscure points of high interest, areas of the screen that were fixated upon remain visible while all else is blackened. Additionally, we produced aggregate gaze plot footage indicating the individuated order and length of each participant’s fixations, differentiated by colour.

Via these varied outputs, this data makes tangible thresholds of seeing and not-seeing. Through eye tracking technology, we pinpoint moments of misdirection, attentional splitting, and lag, connecting these findings to film editing processes described by Arthur P. Shimamura et al. (2014) as “a sort of magician’s sleight-of-hand.” Magicians have long understood the limitations inherent within visual processing, using shortfalls to manipulate and misdirect even the most attentive of audiences. In this video essay, we consider how misdirection, sleight-of-hand, and attentional blindness play out in relation to mediated experiences and screen interfaces.

Of course, there are also blind spots in eye tracking research and technologies. For rigorous eye tracking research, the viewing condition needs to be the same for each participant, yet people experience cinema in a variety of contexts. How would this change their attention? Syncing of the visual scene with the eye tracking device is critical, with any misalignment leading to faulty interpretations of attention lag or objects of fixation. Disparities can even occur between film length and total time in fixations, leaving a gap of time unaccounted for where viewers may be distracted, looking off-screen, searching without fixating, or perhaps searching for peripheral cues.

As Jason Farman (2012) states, “the behind-the-scenes, the off-stage, and the hidden-from-view often serve as the foundations for the perceptive world.” Framing always engenders and depends upon the out-of-frame, whilst “our sense of being-in-the-world is dependent on much of the world not being noticed.” That is, we can only ever gain focus and visual clarity by blocking out or simply not noticing most of our visual field. Pushing ‘against the grain’ of much eye tracking research, this videographic essay plots moments of inattention and forms of blindness to explore the ways in which moving-image screens are at once both seen and unseen.

 

References:

Farman, Jason(2012). Mobile Interface Theory: Embodied Space and Locative Media. London and New York: Routledge.

North, Dan (2008). Performing Illusions. London: Wallflower.

Potter, Robert and Paul Bolls (2012). Psychophysiological Measurement and Meaning: Cognitive and Emotional Processing of Media. NY: Routledge.

Shimamura, Arthur P. et al. (2014). ‘Perceiving Movement Across Film Edits: A Psychocinematic Analysis’. Psychology of Aesthetics, Creativity, and the Arts 8.1: 77-80.

Comments

Miklos Kiss's picture

magic

How nice, the scene from NYSM is exactly the same I invite students to analyze (in my puzzle films class) regarding its clever combination of traditional magic and cinematic tricks! I wonder if anyone knows film examples where eye-tracking research influenced editing, framing, acting, staging or other aspects of film making. I immensely enjoy these analytical videos, especially since Tim Smith’s work on THERE WILL BE BLOOD, and now would like to see how the emerging knowledge could support practical choices of filmmakers.
Tessa Dwyer and Jenny Robinson's picture

Eye Tracking & Filmmaking

Thanks so much for this feedback Miklos, and so good to hear that this video essay matches up so well with your own teaching! The question you raise about the influence of eye tracking research on filmmaking strategies is certainly fascinating to consider. You mention Tim Smith’s important work in the field – and an event that he hosted in 2014 with AMPAS in Los Angeles brought together filmmakers and neuroscientists in order to investigate this topic. You can find details about it on Tim Smith’s blog Continuity Boy and via AMPAS ≤http://www.oscars.org/events/movies-your-brain-science-cinematic-perception>. Although the filmmakers involved (Jon Favreau, Darron Aronofsky) weren’t directly influenced by eye tracking research – many topics and experiments were discussed pointing to the ways that eye tracking research often supports the intuitive logic of filmmaking and editing choices. Another area of possible influence is the testing that Smith did in regards to frame rates, as covered in an article for Wired .
Richard Misek's picture

Seeing without looking

Such a pleasure to see a work in which the nature of the research and the means by which it’s presented are so synchronous, and such a fascinating experience to see scenes from a film superimposed with heat maps, simultaneously to watch the images watched and traces of the watching. Just one (open-ended) question - what about peripheral vision? Not focalising is not the same as not seeing. When I watch a film, I see the entire image, not a face surrounded by darkness; the peripheral vision that ensures my survival as I move through the world does not switch off when I watch a film. I wonder if the final payoff from eye tracking research that Dan North is waiting for may also require further research on what we see in the places we’re not looking.
Tessa Dwyer and Jenny Robinson's picture

Peripheral Vision

Thanks Richard for this insightful comment. Yes, I too am fascinated by peripheral vision and the challenges it presents for eye tracking research. This is something we tried to gesture towards in this video essay. There is interesting eye tracking work done on ‘scene gist recognition’ and ‘covert attention’ that is relevant here. As you note, the processes that inform peripheral vision are integral to visual cognition as a whole. They also underline just how complex the process of ‘seeing’ is.
Miklos Kiss's picture

peripheral and other (deviant) visions

Richard has an important point with further implications. There’s a relevant article by Daniel Chávez Heras, in which Heras is contemplating on how big role the computer / visualization software plays in contemporary regimes of the visible, allowing some images and visions to exist (concentrating on, and actually favouring, the average), while often keeping others from existing (highlighting only the average at the expense of the deviant). I guess our (Software Studies’ and Post-Digital Humanities’) role is to reflect on these kinds of normalisations… Here’s the article: http://framescinemajournal.com/article/the-malleable-computer/