CUMULATE

CUMULATE (Centralized User Modeling Architecture for TEaching) is a central user modeling server

designed to provide user modeling functionality to a student-adaptive educational system. It collects evidence (events) about student learning from multiple servers that interact with the student. It stores students' activities and infers their learning characteristics, which are the basis for an individual adaptation to them. ... External and internal inference agents process the flow of events and update the values in the inference model of the server. Each inference agent is responsible for maintaining a specific property in the inference model, such as the current motivation level of the student or the student's current level of knowledge for each course topic...

Kobsa, A. (2007) Generic User Modeling Systems In P. Brusilovsky, A. Kobsa and W. Nejdl (Eds.),
The Adaptive Web. Methods and Strategies of Web Personalization (pp. 136-154). Berlin / Heidelberg: Springer.

Overview

CUMULATE is a centralized user modeling server built for the ADAPT² architecture. It is mainly targeted at providing user modeling support for adaptive educational hypermedia (AEH) systems. CUMULATE's data model consists of the following.

• users, groups of users, and group membership links (syndicated from Knowledge Tree)
• learning objects, learning object providers (client applications)
• knowledge components - concepts - grouped in learning domains
• learning object metadata index - links between learning objects and corresponding concepts
• user activity log - historical log of all reported user interaction with learning objects
• assertions about user's knowledge of concepts ([concepts'] knowledge)
• summaries of user's interaction with learning objects, such as: number of attempts, percent complete, etc. ([learning objects'] progress)

CUMULATE implements several inference mechanisms to produce assertions about user's knowledge and progress, including the following.

• time-spent-reading (reading texts via Knowledge Sea II and AnnotatEd)
• thresholded averaging (browsing examples via WebEx and exploring expressions via WADEIn)
• asymptotic assessment of user knowledge (problem solving via QuizPACK, QuizJET, and SQLKnoT)
• parametrized asymptotic assessment of user knowledge (under development)
• knowledge tracing (under consideration)

Modeling User's Knowledge

Evindence propagation in CUMULATE

CUMULATE models user's knowledge of concepts with respect to the following levels of Bloom's taxonomy.

• knowledge - corresponds to reading tutorials or book chapters (Knowledge Sea II, AnnotatEd)
• comprehension - reviewing examples, watching demos (WebEx, WADEIn demo mode)
• application - answering quiz questions (QuizPACK, QuizJET, WADEIn evaluation mode)
• synthesis - constructing problem solutions (SQLKnoT)

Upon receipt of new information about user activity (interaction with a learning object), CUMULATE first verifies the integrity of the reported information (user is registered, learning object provider is known and learning object exists, etc.). Then, based on the learning object provider and result of interaction, it is determined what inference mechanism to use and what Bloom's level to affect. CUMULATE then looks up the domain concepts for the learning object the user interacted with and performs the update of the knowledge. Summaries of progress with learning objects are updated as well. The process of updating user model is called evidence propagation (see figure on the left).

New! Recent update to CUMULATE allows learning object providers to skip learning object metadata lookup and report changes in user's knowledge of domain concepts directly.

Using CUMULATE

CUMULATE is has 3 major entry points:

1. interface for reporting user interaction with learning objects
2. interface for reporting changes to user knowledge of concepts in-the-raw (new)
3. interface for acquiring models of user knowledge/progress as a whole or in part

Interaction with CUMULATE is regulated by a set of protocols.

Publications

• Zadorozhny, V., Yudelson, M., and Brusilovsky, P. (2008) A Framework for Performance Evaluation of User Modeling Servers for Web Applications. Web Intelligence and Agent Systems 6(2), 175-191. DOI
• Yudelson, M., Brusilovsky, P., and Zadorozhny, V. (2007) A user modeling server for contemporary adaptive hypermedia: An evaluation of the push approach to evidence propagation. In Conati, C., McCoy, K. F., and Paliouras, G. Eds., User Modeling, volume 4511 of Lecture Notes in Computer Science, pp 27-36. Springer, 2007. PDF DOI
• Brusilovsky, P., Sosnovsky, S. A., and Shcherbinina, O. (2005). User Modeling in a Distributed E-Learning Architecture. Paper presented at the 10th International Conference on User Modeling (UM 2005), Edinburgh, Scotland, UK, July 24-29, 2005. PDF DOI

Ongoing Efforts

• CUMULATE RDF Binding - work in progress on serializing part of CUMULATE's data model in RDF
• Domain and user adaptation of CUMULATE's modeling - changing/updating CUMULATE's model inference to accommodate student's individual differences as well as the differences in complexity of problem-solving exercises.
• Blending layers of CUMULATE's user models - a stream of work targeted at combining layers of CUMULATE's user model (e.g. problem-solving and example-browsing levels) to improve predictive validity of the modeling.

Contacts

Michael V. Yudelson