Background
One of the early definitions of Gamification in the context of Education is the addition of game-like elements and mechanics to a learning process (Deterding et. al, 2011). This early definition had been carried out by teachers through incorporating the use of points (Neve et al, 2014), badges (Ibáñez et al., 2014; Neve et al, 2014; Pirker et al., 2014) and leaderboards (Gibbons, 2013) in the conduct of their face-to-face classes. These implementations reported positive outcomes such as increased engagement (Hamari et al., 2014), small but significant increase in short-term test scores (Bakkes et al., 2012) and generally positive user experience (Garner et al., 2005). The research community coined the term PBL Gamification or PBL frameworkto refer to Gamification systems and initiatives of such nature.
Over time, criticisms for PBL gamification grew stronger as researchers noted some adverse effects of doing PBL. For instance, there are reports of attrition and disengagement among the students who see themselves at the bottom 25% of the leaderboards (Christy & Fox, 2014). There are also students who stop working when a streak of solving correct answers get broken (Butler & Ahmed, 2016). Perhaps most importantly, Nicholson showed through a series of studies that using badges in conjunction with points and leaderboards only supported extrinsic motivation i.e. taking away these rewards or proofs of skill end up also taking away the observed motivation and behaviors that facilitate learning outcomes among students (Nicholson, 2013; Nicholson 2015).
Hence, initiatives towards Meaningful Gamification were started, where, in contrast to the PBL framework, the game mechanics applied to the learning process seeks to activate internal motivation (Sailer and Homner, 2020). One such initiative is the work of Marczewski where they introduced the Gamification User-Types Hexad to first figure out what type of player the user of the Gamification system is, then design the Gamification mechanics around this information (Marczewski, 2016).
Researchers who designed their work around the User-Types Hexad framework still utilize badges. However, unlike the PBL implementation where the badges are given as a reward to notable academic milestones such as getting a perfect score in an exam or submitting the best project, the Hexad framework implementation would package the badges as an award to a game design element that a particular user type would most probably engage with (Tondello et. al, 2016). For example, a student who is labelled as an ‘Achiever user-type’ based on her answers to the User-Types Hexad Questionnaire (Tondello et. al, 2016) ends up being givenQuests (mundane tasks covered in an epic narrative e.g. solve equation 1 to unlock the sword you may use to slay the dragon in equation 10)because Quests are shown to be motivating toAchievers (). Completing a series of Quests then unlocks tiers of badges designed to show one’s proficiency for completing quests.
While the Hexad framework’s intuition to design rewards based on a player type seems to be a promising step towards Meaningful Gamification, Yu-kai Chou, the pioneer of the Octalysis Gamification Framework (Chou, 2018) offers some helpful criticisms regarding the Hexad framework’s limitations. In the context of the Quests mechanic for example, Chou makes the case that calling something a Quest instead of a task and thinking that it would automatically make the same original actions fun and engaging is a shallow way to gamify work/study. A deeper dive into the user’s motivations would be needed to ensure that the gamification design does not end up being yet another shallow Gamification experience (Chou, 2018).
This proposal is an attempt to extend the literature by responding to Yu-kai Chou’s invitation for the creation of a Gamification framework whose design elements and mechanics are more deeply rooted in user motivations. We propose a collection of badge mechanics that are conceptualized as a multi-dimensional construct i.e. a combination of the Behavior Change Model of James Clear (2016), the Lost Connections model of Johann Hari (2018) and the Big Five Model (also known as the Five Factor Model) of Jordan Peterson (2007).
The main distinguishing characteristic of this implementation compared to the PBL or Hexad framework is that the proposed badge mechanics (1) primarily reward process over outcomes (Clear, 2016), (2) incorporates support for well being in face-to-face or online distance learning scenarios (Hari, 2018) and (3) treats the users/students as a collection of independent traits rather than a player-type (Peterson et. al, 2007). Details of these design decisions are described in the sections that follow.
Previous Work: History of Xiphias
The first iteration of Xiphias was implemented in 2012 where the design of the Gamification intervention follows the PBL framework. In this version of Xiphias, students in an introductory programming course were exposed to a competitive programming interface where they may earn badges as a result of solving programming problems.
Figure 1. 1st Implementation of XIphias showing the leaderboard and the badge system.
As an initial effort to understand the system’s effects, student behaviors were observed. Among the positively notable are the activation of the shy students’ ability to mingle with their programming team mates, evident display of feelings relating to triumph (banging on the desks and celebratory shourting when a correct verdict from the online judge is received) and student initiated study groups so they may “do better in the next competition”. Students were also found sharing the badges they earned through social media.
Figure 2. Screenshot of a student sharing his Xiphias badge to Facebook
The authors however note that there were instances when the students who can’t catch up with the performance of the faster students resorted to crying, tantrums/cursing and total disengagement. A phenomenon also observed in other PBL-based implementations (Toda et. al, 2017).
Hence, to address these cons, a second iteration of Xiphias was started. The intuition was, if we utilize the Gamification User-Types Hexad (Tondello, 2016), students may be labelled as belonging to different player types and we can design multiple win conditions for each type i.e. the students who are not good/comfortable at earning badges through competitive programming (1st implementation)may find other ways to ‘win the game’i.e. arrive at the desired learning outcomes.
In this iteration, students were first asked to answer the User-Types Hexad Questionnaire, then the Xiphias team designed badges whose mechanics correspond to the different user types of the respondent pool.
In figure 3 for example, we may find an ‘Event Badge’ that is awarded to students who are able to consistently attend Social Events sanctioned by the school such as acquaintance parties, university-wide competitions and outreach volunteer work among others. The Event Badge mechanic caters to the ‘Socializer’ Hexad-User Type and, as with all badges, has tiers which are unlocked the more a student would repeat a behavior encouraged by the badge. This example shows that the primary focus is shifted from securing the outcomes (Can the student program?), into first creating avenues to reward the student’s natural motivations so she does not disengage with the school environment (Is the student ok and ready to receive instruction?). Only after getting the student to a state of internal motivation should the Gamification system introduce an invitation towards securing learning outcomes– here lies the attempt to align the second implementation of Xiphias with the Meaningful Gamification paradigm.
Figure 3. 2nd Implementation of Xiphias showing the revised badges (top left) the leveling tiers (top right) and the student submission interface (bottom)
Unfortunately, because of time constraints this second iteration was only able to (1) verify if the System was usable (Lopez et. al, 2018) through two iterations of usability testing and (2) validate if the design of the badge mechanics match the user types as recommended by the Hexad framework (Lopez et. al, 2019). The Xiphias project at this point is still inconclusive as to whether the badge mechanics used as an attempt to activate internal motivation among its students eventually end up positively influencing learning outcomes (actual grades) and well being (Quality of Life self reports) when respondents are given enough time to interact with the system.
Hence, this proposal tries to address the loose ends of the two previous implementations by deepening the rationale behind the design of the badge mechanics (Peterson et. al, 2007) and finally testing for outcomes (Clear, 2018) and influence to well being (Hari, 2018).
Objectives
The main objective of the study is to design, create and test a few novel Meaningful -Gamification-based badges through a multi-dimensional approach where the badge mechanics are based on three constructs namely, (1) Clear’s Behavior Change Model of Clear, (2) Hari’s Lost Connections Model and (3) Peterson et. al’s Five Factors Model.
In light of this, the research formally seeks to answer the following research questions:
- How will Meaningful-Gamification badge mechanics look like if designed using a multi-dimensional approach?
- What indicators of engagement can be identified from the users’ interactions with the new Xiphias badge mechanics?
- To what extent do the new XIphias badges affect student performance?
- To what extent do the new Xiphias badges affect student learning experience?
- To what extent do the new Xiphias badges affect student wellbeing?
*users – teachers and students to be invited as respondents to the study.
Significance
The research falls under the priority area on Public Education Reform, more specifically under education and technological innovation.
The study will contribute to the ongoing work on finding strategies for implementing Meaningful Gamification in educational contexts. The results can provide value to future design efforts in creating new ways of instrumenting learning tools that are appropriate for today’s learners.
In a recent meta-analysis by Sailer and Homner on the Gamification of Learning (Sailer and Homner, 2020) the authors noted that the results of their analysis of 1686 Scopus-indexed studies on the gamification in the context of education suggest that the factors contributing to successful gamification are still somewhat unresolved, especially for cognitive learning outcomes– our results should contribute to this unresolved discussion.
Funded by: Ateneo University Research Council (URC)
