Adaptive Navigation Support and Open Social Learner Modeling for PAL

Overview

The goal of this project is to leverage the power of open social learner modeling and adaptive navigation support in the context of the envisioned Personalized Assistant for Learning (PAL). The project is supported by the Advanced Distributed Learning Initiative contract W911QY13C0032.

The project focuses on both exploration and implementation of adaptive navigation support and open social learner modeling and pursues three directions of work:

• Exploring open social learner modeling interface for diverse learning content
• Enhancing algorithms for personalized guidance using knowledge-based and social approaches
• Developing architectural solutions and authoring tools to support open social learner modeling

Open Social Learner Model Interface: Mastery Grids

Mastery Grids Interface

Our latest implementation of Open Social Learner Modeling (OSLM) is Mastery Grids Interface. Mastery Grids is both an innovative Open Social Learner Model Interface and an adaptive E-learning platform with integrated functionalities enabling multi-facet social comparison, open learner modeling, and adaptive navigation support to access multiple kinds of smart learning content. Mastery Grids is supported by adaptive social learning framework Aggregate. This framework supports several kinds of open student modeling, social comparison, and recommendation. In detail, Mastery Grids presents and compares user learning progress and knowledge level using colored grids, tracks user activities with learning content, and provides flexible user-centered navigation across different content levels (e.g. topic, question) and different content types (e.g. problem, example). Our past research shows that open student modeling and social comparison effectively increases students’ performance, motivation, engagement and retention. More about Mastery Grids interface An interactive demo of Mastery Grids interface

Architecture: Aggregate

Aggregate Architecture

We developed an adaptive social learning architecture Aggregate to support Mastery Grids interface. Aggregate is an extension of our original ADAPT2 architecture. On the top of ADAPT2 , Aggregate architecture supports several kinds of open student modeling, social comparison, content brokering, and recommendation services. The architecture fulfills a major objective, portability, which is the ability to be integrated to other systems with little set up and modification. The architecture is modular and includes different software components. More about ADAPT2 More about Aggregate

Learner Model

We have developed CUMULATE, a centralized user modeling server built for the ADAPT2 architecture, to provide user modeling support for adaptive educational hypermedia (AEH) systems. CUMULATE allows flexible learner models to infer learner knowledge. Mastery Grids's architecture is supported by CUMULATE and thus it also supports flexible learner models. We have proposed and implemented different learner models over past years, including asymptotic assessment of user knowledge which is the main one currently deployed in our systems, and Feature-Aware Student knowledge Tracing (FAST) which is our new learner model proposed in 2014 with state-of-the-art predictive performance. The explanation of the communication between the interface and learner model can be found in Aggregate.

Smart Content

Mastery Grids supports and provides multiple types of learning materials. It has been applied in three domains (Java, SQL, and Python) as a supplementary E-learning system for undergraduate and graduate level programming and database classes since 2013. In each learning domain, courses are organized by topics and different types of learning contents are arranged under each topic. Learning contents contain problems (quizzes), parson problems, annotated examples, and animated examples collected from experienced course teachers, textbooks or domain experts.

==> more

Authoring Tools

The set of authoring tools developed for the project include tools for creating several kinds of smart learning content as well as tools to create adaptive courses that use this content. Details are shown as follows.

Content Authoring Tools

This set of tools provide the interface for teachers to create and index problems and annotated examples. ==> more

Course Authoring Tool

This tool provides the interface for teachers to create courses for Mastery Grids. ==> more

Group Authoring Tool

This tool provides the interface for teachers to define groups of students who can access the course. ==> more

Authoring Tool Portal

This is the central portal for providing access to the course authoring and group authoring tools. ==> more

Open Source

Software sources and documentation is in GitHub. The Mastery Grids Interface, backend Aggregate and documentation can be found here. User model services can be found in here.

Publications

• Gonzalez-Brenes, J. P., Huang, Y., and Brusilovsky, P. (2013) FAST: Feature-Aware Student Knowledge Tracing. In: Proceedings of NIPS 2013 Workshop on Data Driven Education, Lake Tahoe, NV, December 10, 2013, (paper)
• Hosseini, R. and Brusilovsky, P. (2013) JavaParser: A Fine-Grain Concept Indexing Tool for Java Problems. In: Proceedings of The First Workshop on AI-supported Education for Computer Science (AIEDCS) at the 16th Annual Conference on Artificial Intelligence in Education, AIED 2013, Memphis, TN, USA, July 13, 2013, pp. 60-63. (paper) (presentation)
• Hosseini, R., Brusilovsky, P., and Guerra, J. (2013) Knowledge Maximizer: Concept-based Adaptive Problem Sequencing for Exam Preparation. In: Proceedings of the 16th International Conference on Artificial Intelligence in Education (AIED 2013), Memphis, USA, pp. 848-851. (paper) (poster)
• Brusilovsky, P., Baishya, D., Hosseini, R., Guerra, J., and Liang, M. (2013) KnowledgeZoom for Java: A Concept-Based Exam Study Tool with a Zoomable Open Student Model. In: Proceedings of 2013 IEEE 13th International Conference on Advanced Learning Technologies, Beijing, China, July 15-18, 2013, pp. 275-279. (paper) (presentation)
• Brusilovsky, P. (2014) Addictive Links: Engaging Students through Adaptive Navigation Support and Open Social Student Modeling (Keynote talk). In: Proceedings of WWW 2014 Workshop on Web-based Education Technologies, Seoul, Korea, April 8, 2014. (presentation)
• Huang, Y., Xu, Y., and Brusilovsky, P. (2014) Doing More with Less: Student Modeling and Performance Prediction with Reduced Content Models. In: V. Dimitrova, et al. (eds.) Proceedings of 22nd Conference on User Modeling, Adaptation and Personalization (UMAP 2014), Aalborg, Denmark, July 7-11, 2014, Springer Verlag, pp. 338-349. (presentationpaper)
• Hosseini, R. and Brusilovsky, P. (2014) Example-Based Problem Solving Support Using concept Analysis of Programming Content. In: S. Trausan-Matu, K. Boyer, M. Crosby and K. Panourgia (eds.) Proceedings of 12th International Conference on Intelligent Tutoring Systems (ITS 2014), Honolulu, HI, USA, June 5-9, 2014, Springer International Publishing, pp. 683-685. (paper) (presentation)
• Hosseini, R., Vihavainen, A., and Brusilovsky, P. (2014) Exploring Problem Solving Paths in a Java Programming Course. In: Proceedings of Psychology of Programming Interest Group Annual Conference, PPIG 2014, Brighton, UK, June 25-27, 2014, pp. 65-76. (paper) (presentation)
• Huang, Y., González-Brenes, J. P., and Brusilovsky, P. (2014) General Features in Knowledge Tracing to Model Multiple Subskills, Temporal Item Response Theory, and Expert Knowledge. In: J. Stamper, Z. Pardos, M. Mavrikis and B. M. McLaren (eds.) Proceedings of the 7th International Conference on Educational Data Mining (EDM 2014), London, UK, July 4-7, 2014, pp. 84-91. (First two authors contributed equally. Nominated for Best Paper Award) (presentationpapertutorial code)
• Khajah, M. M., Huang, Y., González-Brenes, J. P., Mozer, M. C., and Brusilovsky, P. (2014) Integrating Knowledge Tracing and Item Response Theory: A Tale of Two Frameworks. In: I. Cantador, M. Chi, R. Farzan and R. Jäschke (eds.) Proceedings of Workshop on Personalization Approaches in Learning Environments (PALE 2014) at the 22th International Conference on User Modeling, Adaptation, and Personalization, UMAP 2014, Aalborg, Denmark, July 11, 2014, CEUR, pp. 7-12. (First three authors contributed equally) (presentationpaper).
• Yudelson, M., Hosseini, R., Vihavainen, A., and Brusilovsky, P. (2014) Investigating Automated Student Modeling in a Java MOOC. In: J. Stamper, Z. Pardos, M. Mavrikis and B. M. McLaren (eds.) Proceedings of the 7th International Conference on Educational Data Mining (EDM 2014), London, UK, July 4-7, 2014, pp. 261-264. (paper) (presentation)
• Sahebi, S., Huang, Y., and Brusilovsky, P. (2014) Parameterized Exercises in Java Programming: Using Knowledge Structure for Performance Prediction. In: Proceedings of The second Workshop on AI-supported Education for Computer Science (AIEDCS) at 12th International Conference on Intelligent Tutoring Systems ITS 2014, Honolulu, Hawaii, June 6 2014. (paper)(presentation)
• Sahebi, S., Huang, Y., and Brusilovsky, P. (2014) Predicting Student Performance in Solving Parameterized Exercises. In: S. Trausan-Matu, K. Boyer, M. Crosby and K. Panourgia (eds.) Proceedings of 12th International Conference on Intelligent Tutoring Systems (ITS 2014), Honolulu, HI, USA, June 5-9, 2014, Springer International Publishing, pp. 496-503, (paper) (presentation)
• Guerra, J., Sahebi, S., Lin, Y.-R., and Brusilovsky, P. (2014) The Problem Solving Genome: Analyzing Sequential Patterns of Student Work with Parameterized Exercises. In: J. Stamper, Z. Pardos, M. Mavrikis and B. M. McLaren (eds.) Proceedings of the 7th International Conference on Educational Data Mining (EDM 2014), London, UK, July 4-7, 2014, pp. 153-160 (presentationpaper)
• Loboda, T., Guerra, J., Hosseini, R., and Brusilovsky, P. (2014) Mastery Grids: An Open Source Social Educational Progress Visualization. In: S. de Freitas, C. Rensing, P. J. Muñoz Merino and T. Ley (eds.) Proceedings of 9th European Conference on Technology Enhanced Learning (EC-TEL 2014), Graz, Austria, September 16-19, 2014 (Best paper award). (paper) (presentation)
• Brusilovsky, P., Edwards, S., Kumar, A., Malmi, L., Benotti, L., Buck, D., Ihantola, P., Prince, R., Sirkiä, T., Sosnovsky, S., Urquiza, J., Vihavainen, A., and Wollowski, M. (2014) Increasing Adoption of Smart Learning Content for Computer Science Education. In: Proceedings of Proceedings of the Working Group Reports of the 2014 on Innovation & Technology in Computer Science Education Conference, Uppsala, Sweden, ACM, pp. 31-57. (paper)
• Gonzalez-Brenes, J. P., Huang, Y. (2014) The White Method: Towards Automatic Evaluation Metrics for Adaptive Tutoring Systems. In: Proceedings of NIPS 2014 Workshop on Human Propelled Machine Learning, Montreal, Canada, December 13, 2014 (paper)
• Huang, Y., González-Brenes, J. P., and Brusilovsky, P. (2015) The FAST toolkit for Unsupervised Learning of HMMs with Features. In: The Machine Learning Open Source Software workshop at the 32nd International Conference on Machine Learning (ICML-MLOSS 2015), Lille, France July 10, 2015. (papercode)
• Huang, Y., González-Brenes, J. P., Kumar, R., Brusilovsky, P. (2015) A Framework for Multifaceted Evaluation of Student Models. In: Proceedings of the 8th International Conference on Educational Data Mining (EDM 2015), Madrid, Spain, pp. 203-210. (paper) (presentation)
• Huang, Y., González-Brenes, J. P., Brusilovsky, P. (2015) Challenges of Using Observational Data to Determine the Importance of Example Usage. In: Proceedings of the 17th International Conference on Artificial Intelligence in Education (AIED 2015), Madrid, Spain, pp. 633-637. (paper)
• Gonzalez-Brenes, J. P., Huang, Y. (2015) Your model is predictive— but is it useful? Theoretical and Empirical Considerations of a New Paradigm for Adaptive Tutoring Evaluation. In: Proceedings of the 8th International Conference on Educational Data Mining (EDM 2015), Madrid, Spain, pp. 187-194. (paper presentation)
• Brusilovsky, P., Somyurek, S., Guerra, J., Hosseini, R., and Zadorozhny, V. (2015) The Value of Social: Comparing Open Student Modeling and Open Social Student Modeling. In: F. Ricci, K. Bontcheva, O. Conlan and S. Lawless (eds.) Proceedings of 23nd Conference on User Modeling, Adaptation and Personalization (UMAP 2015), Dublin, Ireland, , June 29 - July 3, 2015, Springer Verlag, pp. 44-55 (paper presentation)
• Guerra, J., Huang, Y., Hosseini, R., & Brusilovsky, P. (2015, June). Graph Analysis of Student Model Networks. In Proceedings of the Second International Workshop on Graph-Based Educational Data Mining (GEDM 2015). CEUR-WS. (paper) (presentation)
• Guerra, J., Huang, Y., Hosseini, R., & Brusilovsky, P. Exploring the Effects of Open Social Student Model Beyond Social Comparison. In ISLG 2015 Fourth Workshop on Intelligent Support for Learning in Groups (p. 19). (paper) (poster)
• Hosseini, R., Hsiao, I.-H., Guerra, J., Brusilovsky, P. (2015) Off the Beaten Path: The Impact of Adaptive Content Sequencing on Student Navigation in an Open Social Student Modeling Interface. In: Proceedings of the 17th International Conference on Artificial Intelligence in Education (AIED 2015), Madrid, Spain, pp. 624-628. (paper) (poster)
• Hosseini, R., Hsiao, I.-H., Guerra, J., Brusilovsky, P. (2015) What Should I Do Next? Adaptive Sequencing in the Context of Open Social Student Modeling. Proceedings of 10th European Conference on Technology Enhanced Learning (EC-TEL 2015), Toledo, Spain, September 15-18, 2015, pp. 155-168. (paper)(presentation).
• Somyürek, S. & Brusilovsky, P. (2015). Impact of Open Social Student Modeling on Self-Assessment of Performance. Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2009 (E-Learn 2015). Kona, Hawaii, United States, October 19-22, 2015
• Hosseini, R., Sirkiä, T., Guerra, J., Brusilovsky, P., Malmi, L. (2016) Animated Examples as Practice Content in a Java Programming Course. Proceedings of the 47th ACM technical symposium on Computer Science Education (SIGCSE), Memphis, Tennessee, March 2-5, 2016. (paper) (presentation)