We propose to extend MAGIS: Mobile Augmented Reality Game Engine for Instructional Support. MAGIS was developed as part of a prior P3 million CHED-funded grant entitled “Design, Development, and Deployment of an Augmented Reality Game for Philippine History”. The project began in September 2014 and ended in September 2015. The project resulted in a game now available on the Google Play Store and the Apple App Store called Igpaw: Intramuros. It also resulted in three conference papers (two Scopus-listed conferences and one ACM SIGGRAPH-sponsored conference) and various presentations in educational and science and technology fora.
Igpaw: Intramuros is a serious game that uses an adventure game format to introduce players to historical figures, places, and events. The premise of the game is that a malevolent being has displaced historic figures from the past. The player embarks on various quests in order to restore these characters to their correct time periods, visiting several key historic locations within Intramuros and learning about Philippine history in the process (Rodrigo, et al., 2015).
Since Igpaw is a tablet- and smartphone-based AR application, the level of visual fidelity that can be provided is limited to the capabilities of the target devices. Both Google’s Android and Apple’s iOS operating systems were targeted to maximize audience reach. In order to achieve a good level of visual fidelity for the game while maintaining cross-platform compatibility across these two mobile OSes, MAGIS was developed by the team for Igpaw and future games (educational and non-educational). MAGIS is based on top of the Unity game engine and uses PTC’s Vuforia for the computer-vision-based tracking component, as well as a device-sensor-based tracking solution developed in-house that uses accelerometer, compass, gyroscope and GPS sensor data to improve tracking. This combination of vision-based and sensor-based tracking allows for the creation of full 360-degree AR scenes around the viewpoint of the player (Rodrigo, et al., 2015).
The proposed project, if approved, will improve upon the existing MAGIS framework. Planned improvements may include the following:
-
Enhancement of AR tracking algorithms, improving the tracking so that virtual objects are more accurately drawn on top of real-world objects at all times despite irregular camera movement. These include selection and enhancement of both vision-based (Kato & Billinghurst 1999; Reitmayr & Drummond 2006; Chekhlov et al. 2007) and sensor-based (Azuma & Bishop 1994; You et al. 1999; Hol et al. 2006; Kahari & Murphy 2006) tracking algorithms.
-
Inclusion of other interactivity modes apart from narrative-based adventure games. These modes may be game- or non-game-related. Currently, MAGIS is designed primarily for games with tightly-written narratives (e.g., historical adventure games), but other modes such as educational sandboxing (e.g., physics-simulation sandboxes, Second Life-style social sandboxes), multiplayer gaming (e.g., Ingress, Pokemon Go) and social networking features (e.g., posting selfies on Facebook/Twitter, achievement systems) will be explored.
-
Improved map rendering, to more accurately depict environment structures in overhead map mode (e.g., accurate rooftops, roads, etc.)
-
Improved authoring tools, to enable faster prototyping of in-game levels.
-
Support for gesture recognition, which would allow players to interact with virtual objects using their hands instead of the device’s touch-screen input. Due to the complex nature of gesture recognition (Buchmann et al. 2004; Reifinger et al. 2007; Lee & Hollerer 2007; Radkowski & Stritzke 2012), a study will first be conducted in order to determine the feasibility of this feature.
-
An inline advertisement subsystem that will allow context-aware augmented reality ads to be shown to the user. This subsystem may also be harnessed to enhance the actual gameplay (e.g., an in-game food item can be exchanged at a fast-food restaurant for a real food item). This feature will be undertaken if development time permits.
We will build two proofs of concept—one game application and one educational application. The scope of both applications is yet to be determined, based on the agencies that are willing to cooperate with us. Both applications will be deployed on Google Play and the App Store for people to download.
Finally, the project will include the development of a business plan to determine how these products and services can be monetized. The business plan will include details such as:
-
Project management and staffing structure
-
Pricing for AR application development services
-
Pricing for product placement
-
Pricing for advertising
Philippine Game Industry
Since at least 2008, the Philippines’ information technology (IT) outsourcing industry has been enjoying double-digit growth (Philippine Software Industry Association in Philippines Board of Investments, 2010). The industry offers international clientele technology services that include applications development, applications maintenance and support, quality assurance, and so on. However, competition from other countries such as China, Sri Lanka, and Vietnam threatens to erode our business process outsourcing market share (Adriano, 2011). To sustain our medium- to long-term IT competitiveness, we need to increase our pool of high-skill, high-knowledge IT professionals. One of the high-value production services in which we can continue to grow is game design and development, an industry worth an estimated $111 billion (Statista, 2015).
As of the fourth quarter of 2010, 60 Philippine-based companies were involved in the game development process with revenues estimated at US$9 million as of the end of 2010 and US$14 million as of the end of 2011, an annual increase of 50% per year. These companies employ approximately 2,000 people with skills in C/C++, Flash, 2D/3D modeling, PHP, Java, Photoshop, console, and smart phone programming. These companies carry out most if not all phases of the game development process—concept design, prototyping, program development, art and music design, testing, and release and maintenance—for a clientele that is 90% foreign (Tolentino, 2011). When we compare the size of the global industry with that of the local industry, there is clearly an opportunity to increase the country’s participation, if we had the skills and resources. This lack of skills and resources is the problem we are trying to address.
Given this, the purpose of this project is to help build local game development industry expertise that will enable us to broaden our participation in the global game industry. The beneficiaries of this project include the cooperating agencies, the users of the applications, the game development community, and the educational game research community.
Augmented Reality
Augmented reality refers to the superimposition of digital information on top of real settings in 3D space, providing the user with situated knowledge (Azuma, 1997). Smart phones and tablet computers with back-mounted cameras afford the creation of “magic window” AR applications that display a composite of real and digital images on the user’s screen (Normand, et al., 2012).
In the 1990s and early 2000s, AR was implemented using cumbersome head-mounted displays, with processors carried in backpacks as the user moves in various spaces (Vlahakis et al., 2001). In recent years, however, increases in processing power and miniaturization (Löchtefeld, Krüger, & Rohs, 2011), the pervasiveness of Internet access (Ferreira & Boavida, 2011), and the availability of easy-to-use toolkits (Gu & Duh, 2011) have made it possible to deploy AR applications on tablets and mobile phones. Indeed, researchers now regard AR as a game-changing application in education because it can be used for visual and highly interactive learning, is an active technology that can interact with students, is situated and immersive, and can therefore foster greater transfer of learning (Johnson, Smith, Willis, Levine, & Haywood, 2011).
Examples of AR applications in education cut across different subject domains. A review by Santos (2015) shows that educational AR applications have been built to teach about magnetic fields, the solar system, and mathematics. Santos himself developed an educational AR application to teach vocabulary.
Theoretical Framework
The theoretical framework of this proposal is broken into three major frameworks: a framework for game design, a framework for educational software design, and a framework for augmented reality design.
Game design framework
There are several frameworks available for the design and development of digital games. Among these, one of the most cited is the Mechanics, Dynamics, and Aesthetics framework, or MDA. MDA is a formal game design methodology with a focus on player experience (Hunicke et al., 2004). As its name implies, it breaks down a game’s behavior into three causally-linked components: Mechanics, Dynamics, and Aesthetics. Mechanics are the rules that formally specify the game as a system. Dynamics describes the run-time behavior of the system resulting from the interaction of the game’s mechanics with each other. Aesthetics describes the desirable emotional responses evoked in the player as he or she interacts with the game system – in other words, the player’s experience as he or she plays the game. Using the MDA framework in an experience-driven approach, game designers consider the player and start with aesthetics by deciding on what experience they want their players to have. The required dynamics are derived from there, and the mechanics needed to achieve these dynamics are established.
Educational game design framework
The process of developing of a game that carries educational content is a nuanced application of the framework outlined above. The specifications for the game have to include details about the learning and about the content to be learned. In specific, the designers must articulate the learning objectives of the game, the characteristics of the target learner, the different learning tasks (not unlike the quests of role-playing games), and the forms of assessment. Further details will be discussed in the methodology.
The key element of educational game design, though, is that the player’s progress must be dependent on his or her mastery of the content. If the game’s mechanics rely solely on chance or luck, the game will undercut any desire on the part of the learner to learn the content. Progress in the game should be an indication of how much the learners has in fact internalized the material. He should make use of whatever skills or knowledge he acquired about the subject matter to succeed in the game tasks.
Augmented reality framework
Augmented reality applications can be characterized according to several dimensions: Tracking, augmentation type, temporal base, and rendering modalities (Normand, et al., 2012). Tracking refers to the amount of information that the application needs to know about the position and orientation of the user. 0D applications detect a marker, e.g. a QR code, and then display additional information about the marker. 2D applications make use of location-based services to display information about the user’s environment. 2D + theta applications require both location and orientation of the user. Finally 6D applications require several sensors to acquire location, orientation, and possibly motion of the user.
There are two main augmentation types: mediated or direct. Mediated augmentation refers to the use of optical see-through devices (experimental at this point) or video see-through devices such as phones or tablet PCs with back-located cameras. The latter function as “magic mirrors” or “magic windows,” displaying the augmented reality composite on the user’s screen (Normand, et al., 2012). The portability, wide availability, and computational power of smart phones and tablet PCs make them ideal platforms for augmented reality application deployment (Thomas, 2012).
Temporal base refers to the time period of the content displayed on the screen. Is it historical content, present day content, future states, or imaginary or fictional content (Normand, et al., 2012)?
Finally, rendering modalities refer to the media that will be available to the user. Is the overlay purely visual or will it include sound or haptic information (Normand, et al., 2012).
Relationship with CHED’s research agenda
This proposed research program is consistent with CHED’s emphasis on Smart Analytics Engineering Innovations. The technology to be developed is a framework for augmented reality applications. It is relatively new technology that can in turn lead to the creation and deployment useful AR applications.
It is also related to CHED’s theme of Education for STEAM. The process of creating this framework and the proofs of concept applications will train computer scientists to develop AR technologies such as games.
Relationship with the HEI’s research agenda
Among the Ateneo’s focus areas are Public Education Reform (which includes technological innovations) and Creativity, Innovation, and Culture. The proposed project contributes a creative technological solution that has already been used and can continue to be used for the promotion and preservation of culture.
Relationship with the local and international community’s research agenda
Augmented Reality is a growing research area in the international community. As mentioned earlier, it is regarded as a game-changing application.
Multidisciplinary nature
In order to produce Igpaw: Intramuros, this grant’s precursor funded by CHED, we had to involve computer scientists, artists, historians, writers, and editors. For this proposed project, we again anticipate relying on a multidisciplinary team composed of computer scientists, artists, subject matter experts, educational specialists, writers, and business development specialists.
Mentoring activities
The development of the system will involve the participation of Masters and PhD students and early-career computer scientists. Eric Vidal and Jenilyn Agapito, for example, are both PhD Computer Science students. Jonathan Casano is currently an MS Computer Science student but will most probably have graduated by the time funding is approved (if at all).
