Abstract: ZORQ is a gamification software framework designed to increase student engagement within undergraduate Computer Science (CS) education. ZORQ is an attractive learning method that (1) utilizes numerous gamification elements, (2) provides a collaborative, game-development based learning approach, (3) offers an opportunity for students to explore a complex, real-world software development implementation, and (4) provides students with a high level of engagement with the system and a high level of social engagement in its collaborative customization. The usage of ZORQ was assessed using quantitative, qualitative and sentiment analyses in a Data Structures and Algorithms course over five years. The overwhelmingly positive results show that students were satisfied with their user experience and ZORQ was beneficial to their educational experience. By triangulating results from multiple analyses, this study adds to a deeper understanding of how gamification can improve learning and retention and provides a novel, robust, holistic methodology for evaluating user experiences.
PDF (885K, 10 pages)
Abstract (condensed): ZORQ is a unique combination of a game development framework and a gamification framework (GDGF). The ZORQ GDGF acts as a catalyst to help motivate students by increasing student engagement and success within undergraduate Computer Science (CS) education, regardless of student experience and background. After collaborative game space customization, ZORQ gameplay sees each student tasked with designing a ship movement philosophy and then implementing their own code to autonomously control a ship in an interstellar game space filled with supplies, obstacles, and enemy ships. The particulars of engagements between ships can vary greatly by semester, along with the resources/objects present in the game, depending on the collaborative customization and the independent ship strategies implemented. A preliminary ZORQ trial was conducted over five years in an undergraduate Data Structures and Algorithms (DSA) course. In exit surveys, students expressed overwhelming satisfaction with this approach. Observations of student performance in later courses suggest better student maturity and comprehension in preparation for proposing and implementing their own independent projects.
©2022 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from IEEE.
PDF (443K, 9 pages)
Abstract (condensed): This research presents two main outcomes: 1) a novel classification system for gamification implementations including proposed genres, and 2) a comprehensive study and categorization of existing DSA gamification applications and a discussion of genres absent existing applications. Gamification presents a great potential to improve user engagement, motivation, and learning in nearly all fields of study including computer science (CS) education. However, it lacks formalized study and comprehensive analysis in CS education, and thus what makes for effective gamification is still a key question. Rather than initially trying to examine and catalog existing gamification applications and studies across the breadth of CS education as a whole, this paper focuses on Data Structures and Algorithms (DSA) courses. To carry out this work, a literature review of current DSA gamification applications is presented, the applications are categorized, and the pros and cons analyzed. Based on this analysis, a classification system is created and two new abstract genres are identified: dynamic gamification and collaborative gamification development. Potential uses, benefits and detriments are suggested for these newly identified genres. Upon a more thorough understanding of DSA gamification, pedagogical considerations can be made to better aid teachers and instructors in the integration of gamification into existing curriculum. The paper also touches on the applicability of the classification system to CS gamification examples outside of DSA.
©2021 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from IEEE.
PDF (92K, 9 pages)
Abstract: CARMA is a decision-support system for managing grasshopper infestations which uses an approach called approximate-model-based adaptation whereby case-based reasoning (CBR) provides an approximate solution and model-based reasoning adapts this approximation into a precise solution. CARMA's predictive accuracy on a set of known cases confirmed the ability of the technique. The evaluation was not expanded beyond the initial set of known cases due to the human effort involved in constructing such cases. We provide an overview of CARMA, and detail initial attempts to establish a process for the automatic evaluation of such systems in order to identify potential gaps in predictive coverage using Monte Carlo methods. We propose that any generated situation which produces large adjustments in prediction during adaptation suggests a potential gap in the predictive ability of a CBR system. This represents an extension of prior CBR work which considers only the matching stage when evaluating predictive coverage.
©2014 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from IEEE.
PDF (698K, 8 pages)
Abstract: Many complex physical systems such as biological systems are characterized both by incomplete models and limited empirical data. Accurate prediction of the behavior of such systems requires exploitation of multiple, individually incomplete, knowledge sources. Our approach, called approximate-model-based adaptation, utilizes case-based reasoning to provide an approximate solution and model-based reasoning to adapt this approximation into a more precise solution. This approach is implemented in CARMA, a decision-support system for grasshopper infestation advising which models experts and has been successfully used since 1996. Initially focused on rangeland grasshoppers within the state of Wyoming, CARMA's capabilities have been extended to support the development and implementation of more environmentally friendly and sustainable strategies and to support advising in nine additional western U.S. states. This paper details our approach to scaling CARMA to the wider geographic region. Prior research indicated that completeness of the model-based knowledge used for matching and adaptation is more important to CARMA's accuracy than coverage of the case library. Given the importance of the model as a tool for refinement and accuracy, and that the cases are mostly void of region-specific information, our approach is thus to continue using the cases without changes as a general source of approximate predictions, and to extend the region-specific historical information required by the model as necessary to provide regional accuracy. The relative ease with which CARMA has been scaled thus far lends confirmation to the fact that CARMA's modeling of the experts is accurate.
PDF (89K, 10 pages)
Abstract: CARMA is an advisory and research support tool for grasshopper infestations. Designed with usability as a primary goal, CARMA presents an interface so intuitive that it completely eliminates the need for a user manual. To achieve this goal, CARMA interacts with the user through a goal-oriented, guided style reminiscent of a natural conversation between an advice seeker and an expert. Usability is furthered by its modeling of four important characteristics of human expert problem solving (speed, graceful degradation, explanations, and opportunism). In order to gain non-biased user feedback about CARMA's interface, we surveyed a group of novice users not previously familiar with CARMA. Positive survey results suggest that CARMA's approach to usability is a success. Furthermore, our survey approach illustrates a simple anonymous online technique which elicits candid non-biased feedback from participants about a product, and is particularly applicable to practitioners short on staff or time.
©2010 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from IEEE.
PDF (817K, 10 pages)
Abstract: CARMA is an advisory system that uses artificially-intelligent techniques including case-based reasoning to provide advice about the most environmentally and economically effective responses to grasshopper infestations. CARMA's core AI reasoner was initially written in Common Lisp and integrated with an Allegro Common Lisp for Windows graphical user interface (GUI). CARMA went public in 1996 and has been used successfully since. Recently, CARMA's architecture was reworked in order to avoid periodic development and deployment fees, and to produce a platform-independent system by following a philosophy called platform freedom which emphasizes freedom from both platform dependence and software costs. The implementation also demonstrates an approach to creating a Lisp application with an appealing GUI which is web capable. This paper details CARMA's new architecture including the two-way communication between the two distinct main parts: 1) a Lisp AI reasoner which runs inside the Armed Bear Common Lisp interpreter which in turn runs inside the Java interpreter (JVM), and 2) a Java GUI which runs inside the JVM.
PDF (615K, 10 pages)
Abstract: CARMA is an advisory system for grasshopper infestations that has been successfully used since 1996. During CARMA's history, grasshopper control has increasingly focused on environmentally friendly and sustainable strategies. In order to keep pace with and support emerging strategies, CARMA's functionality has been enhanced in a manner which both improves maintainability and which expands CARMA beyond its original role as a grasshopper infestation advisor into that of a grasshopper research support tool. This paper details efforts to develop sustainable grasshopper management strategies and the role that CARMA has played and continues to play in supporting the development and implementation of those strategies.
©2009 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from IEEE.
PDF (790K, 10 pages)
Abstract: CARMA is a decision-support system for grasshopper infestations that has been successfully used since 1996. Rising treatment costs coupled with shrinking rangeland profit margins increasingly demand accurate selection of the most cost-effective responses to grasshopper infestations, and CARMA fills that need. In the process CARMA provides advice regarding grasshopper population management options in an environmentally and economically sound fashion, and is the only pest management software that includes the more environmentally-friendly Reduced Agent-Area Treatments (RAATs) as a treatment option and an open-ended capacity for user-based treatment updates. This paper describes the most recent changes to CARMA with particular attention to the new architecture which demonstrates an approach to integrating an artificially intelligent LISP reasoner with a Java graphical user interface (GUI) in a way which combines the strengths of the two languages (i.e., LISP for artificial intelligence and Java for graphical user interfaces) in order to provide a strong reasoner while at the same time producing an appealing user interface which is platform independent and web capable.
PDF (1.4M, 9 pages)
Abstract: Automatically acquiring knowledge in complex and possibly dynamic domains is an interesting, non-trivial problem. Case-based reasoning (CBR) systems are particularly well suited to the tasks of knowledge discovery and exploitation, and a rich set of methodologies and techniques exist to exploit the existing knowledge in a CBR system. However, the process of automatic knowledge discovery appears to be an area in which little research has been conducted within the CBR community. An approach to automatically acquiring knowledge in complex domains is automatic case elicitation (ACE), a learning technique whereby a CBR system automatically acquires knowledge in its domain through real-time exploration and interaction with its environment. The results of empirical testing in the domain of chess suggest that it is possible for a CBR system using ACE to successfully discover and exploit knowledge in an unsupervised manner. Results also indicate that the ability to explore is crucial for the success of an unsupervised CBR learner, and that exploration can lead to superior performance by discovering solutions to problems which would not otherwise be suggested or found by static or imperfect search mechanisms.
PDF (310K, 6 pages)
Abstract: Automatic case elicitation (ACE) is a learning technique in which a case-based reasoning system acquires knowledge automatically from scratch through repeated real-time trial and error interaction with its environment without dependence on pre-coded domain knowledge. ACE represents an alternative to manually constructed case bases and domain specific techniques, and is generally applicable to any domain for which knowledge can be obtained from a series of observations of an environment (e.g., checkers or massively multiplayer games). A priority is placed on maintaining the flexibility necessary to learn new domains with only negligible manual configuration. We found during testing that the current approach to ACE with a reliance on experience and exploration, while quite capable in the domain of checkers, did not perform adequately in the exponentially more complex domain of chess. Our results suggest that experience alone, without the ability to adapt for case differences between new and prior cases, is insufficient in more complex domains.
PDF (88K, 8 pages)
Abstract: Non-learning problem solvers have been applied to many interesting and complex domains. Experience-based learning techniques have been developed to augment the capabilities of certain non-learning problem solvers in order to improve overall performance. An alternative approach to enhancing pre-existing systems is automatic case elicitation, a learning technique in which a case-based reasoning system with no prior domain knowledge acquires knowledge automatically through real-time exploration and interaction with its environment. In empirical testing in the domain of checkers, results suggest not only that experience can substitute for the inclusion of pre-coded model-based knowledge, but also that the ability to explore is crucial to the performance of automatic case elicitation.
PDF (459K, 11 pages)
Abstract: Traditionally case-based reasoning (CBR) systems have relied on information manually provided by domain experts to form their knowledge bases. Additional domain knowledge is often used to improve performance of such systems. A less costly method of knowledge acquisition is automatic case elicitation, a learning technique in which a CBR system acquires knowledge automatically during real-time interaction with its environment with no prior domain knowledge (e.g., rules or cases). For problems that are observable, discrete and either deterministic or strategic in nature, automatic case elicitation can lead to the development of a self-taught knowledgeable agent. This paper describes the use of automatic case elicitation in CHEBR, a CHEckers case-Based Reasoner that employs self-taught knowledgeable agents. CHEBR was tested using model-based versus non-model-based matching to evaluate its ability to learn without predefined domain knowledge. The results suggest that additional experience can substitute for the inclusion of precoded model-based knowledge.
PDF (162K, 5 pages)
Abstract: CARMA is a decision-support system for rangeland pest infestations that has been used successfully in Wyoming counties since 1996. CARMA is limited to the specific task for which it was designed: providing advice to ranchers concerning insect infestations on rangeland. This paper describes CARMA+, an architecture that permits CARMA's design to be applied to other pest-management tasks. A task analysis is described for a crop protection module for CARMA+ that is currently under development.
PDF (363K, 4 pages)
Abstract: CARMA is an advisory system for rangeland grasshopper infestations that demonstrates how AI technology can deliver expert advice to compensate for cutbacks in public services. CARMA uses two knowledge sources for the key task of predicting forage consumption by grasshoppers: cases obtained by asking a group of experts to solve representative hypothetical problems; and a numerical model of rangeland ecosystems. These knowledge sources are integrated through the technique of model-based adaptation, in which CBR is used to find an approximate solution and the model is used to adapt this approximate solution into a more precise solution. CARMA has been used in Wyoming counties since 1996. The combination of a simple interface, flexible control strategy, and integration of multiple knowledge sources makes CARMA accessible to inexperienced users and capable of producing advice comparable to that produced by human experts. Moreover, because CARMA embodies diverse forms of expertise, it has been used in ways that its developers did not anticipate, including pest management research, development of industry strategies, and in state and federal pest management policy decisions.
PDF (157K, 8 pages)
Abstract: This paper describes a technique for integrating case-based reasoning with model-based reasoning to predict the behavior of biological systems characterized both by incomplete models and insufficient empirical data for accurate induction. This technique is implemented in CARMA, a system for rangeland pest management advising. CARMA's ability to predict the forage consumption judgments of 15 expert entomologists was empirically compared to that of CARMA's case-based and model-based components in isolation. This evaluation confirmed the hypothesis that integrating model-based and case-based reasoning through model-based adaptation can lead to more accurate predictions than the use of either technique individually.
PDF (168K, 12 pages)
Abstract: CARMA (CAse-based Range Management Adviser) is a system that integrates case-based reasoning with model-based reasoning for rangeland pest management. CARMA's predictions of rangeland forage loss by grasshoppers were compared to predictions by 15 expert entomologists using either global or case-specific adaptation weights. Under both conditions, CARMA's predictions were more accurate than CARMA's case-based and model-based components in isolation. However, CARMA's case-specific adaptation weights were consistently more accurate than global adaptation weights. The experimental results suggest that case-specific adaptation weights are more appropriate in domains that are poorly approximated by a linear function.
PDF (276K, 7 pages)
Abstract: Rangeland ecosystems typify physical systems having an incomplete causal theory. This paper describes CARMA, a system for rangeland pest management advising that uses model-based matching and adaptation to integrate case-based reasoning with model-based reasoning for prediction in rangeland ecosystems. An ablation study showed that removing any part of the CARMA's model-based knowledge dramatically degraded CARMA's predictive accuracy. By contrast, any of several prototypical cases could be substituted for CARMA's full case library without significantly degrading performance. This indicates that the completeness of the model-based knowledge used for matching and adaptation is more important to CARMA's performance than the coverage of the case library.
PDF (142K, 7 pages)