Description of the Project
Eye-tracking is an emerging field in which special video cameras are used to follow and record the movements of a test subject. Eye gaze fixations indicate the test subject’s focus, hence the behavior of the test subject’s eyes can be used as an empirical measure of user attention. Eye gaze data has been used for a variety of fields from autism research (Navab, et al., 2012) to sports (Barfoot, et al., 2012) to animal behavior (Kano & Tomonaga, 2009) to fashion design (Ho, 2014) to reading comprehension (Graesser, et al., 2005).
Because accurate eye tracking requires relatively new and expensive technology, research and publications that make use of eye tracking data is somewhat rare. This implies that the field is open for contributions.
We propose to develop national capacity to conduct eye tracking related research. This entails the hosting of eye tracking workshops and the execution of a common research project. Because the proponents are from computer science, the common research project will focus on novice programmer tracing and debugging skills.
Statement of the problem
During the 4th International Outsourcing Summit held in 2012, Silicon Valley-trained Filipino information technology experts noted that “only 10 percent of information technology (IT) or computer science graduates are hireable.” Although the Philippines produced 70,000 IT graduates in 2011, big companies consider most of these graduates to be undereducated and even uneducated by their standards. To blame, these experts say, are substandard training and education programs. Many schools allow their graduates obtained degrees without knowing how to program (Cuevas-Miel, 2012).
Programming is a fundamental skill that all graduates of IT-related courses must possess, yet it is a difficult skill to learn. Since 2006, Dr. Rodrigo of the Ateneo de Manila has fostered an active research group that has studied novice programmer education. Funded by several Department of Science and Technology grants, the Ateneo has studied novice programmer errors (Rodrigo et al., 2013), novice programmer affective states such as confusion (Lee, et al, 2011) and frustration (Rodrigo & Baker, 2009), and has built systems to analyzing novice programmer compilation logs (e.g. Dy & Rodrigo, 2010).
Prior works’ data sources have been limited to logs of novice programmer compilations and human observations. Both of these tended to be coarse grained. The logs were essentially snapshots of novice programmer activity that have no representation for interim thought processes. The human observations of each student were recorded every three minutes or so. This research project collects finer-grained data that enables us to track what it is that the students are attending to. This, in turn, helps us determine whether they are in fact tracing the logic of the code or if they are confused.
Purpose of the research
This research project studies the tracing and debugging strategies employed by novice programmers, both as individuals and as pairs. Subsequent analyses will attempt to draw differences between high- and low-performing students and then arrive at recommended debugging strategies that teachers can then convey to students. The research questions that the group hopes to answer include but are not limited to the following:
-
What problem solving strategies do novice programmers employ when visually parsing through a code fragment?
-
How do the strategies employed by high-performing and low-performing students differ?
How does collaboration between pairs of novices affect the speed and accuracy with which bugs are identified and resolved?
Theoretical Framework
Eye-tracking is a data collection technique in which an individual’s eye movements are measured (Poole & Ball, 2006). These measurements enable the researcher to determine where a person is looking (point-of-regard) and the sequence in which the eyes shift from one object to another. Eye-tracking research is founded on the “eye-mind hypothesis” that asserts that eye traces are an indicator of where an individual’s attention is focused. Visual attention refers to focalization. Individuals withdraw from some environmental stimuli so that they can effectively deal with others (Duchowski, 2007). Hence, eye-tracking data can provide researchers with insight regarding individuals’ visual information processing.
There are typically two types of eye-trackers: table-mounted and head-mounted. Table-mounted eye trackers, as the name implies, are positioned on a table and are focused on the participant’s eye. Head-mounted eye-trackers are similar eye glasses (Duchowski, 2007).
Poole and Ball (2006) and (Duchowski, 2007) describe several features used to measure eye-gaze. Fixations refer to positions of the eyes when the eyes are relatively stationary. They are an indicator of the amount of attention that is being applied to the point-of-regard. Saccades refer to movement of the gaze from one point to another. No encoding takes place during a saccade, but regressive saccades, i.e. backward movement across a region that has already been visited, can indicate that an individual is having difficulty processing the material. Scanpaths are sequences of saccades and fixations. They can indicate compliance with or departure from what is regarded to be an optimal path through the material. Blink rate and pupil size can both indicate cognitive effort. A low blink rate indicates a high workload while large pupils imply greater processing.
Eye-tracking research has been used to study human attention in a variety of circumstances. These include but are not limited to reading, driving, marketing and advertising, and computer science.
Expected Output
This research project should result in prescriptions for teaching code comprehension, tracing, and debugging, some of the hardest computer-programming skills to learn. It should also lead to research capacity building and publications in highly reputable journals and conferences.
Justification
The project is significant in at least three ways:
-
The addition of the eye-tracking component develops local expertise in the collection, analysis and interpretation of eye-tracking data. Because the technology is not yet widely used for this type of research, it has a high potential to lead to new contributions and collaborations in other areas of research.
-
It will contribute to the body of literature about novice programmer cognition, learning strategies, and debugging strategies.
-
It will contribute to novice programmer education by helping establish expert patterns of parsing code that can then inform instructors about how to teach novices how to read and trace through code fragments. This can lead to a higher level of IT preparation and training.
