Difference between revisions of "CUMULATE"
From PAWS Lab
(→Overview) |
|||
| (83 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
| − | CUMULATE | + | <!--{{ambox| text='''This page is under construction. More content will be added soon'''|type=exclamation}}--> |
| + | [[CUMULATE]] (Centralized User Modeling Architecture for TEaching) is a central user modeling server <blockquote>designed to provide user modeling functionality to a student-adaptive educational system. It collects evidence (events) about student learning from multiple servers that interact with the student. It stores students' activities and infers their learning characteristics, which are the basis for an individual adaptation to them. ... External and internal inference agents process the flow of events and update the values in the inference model of the server. Each inference agent is responsible for maintaining a specific property in the inference model, such as the current motivation level of the student or the student's current level of knowledge for each course topic...</blockquote> | ||
| + | <div align="right">Kobsa, A. (2007) Generic User Modeling Systems In P. Brusilovsky, A. Kobsa and W. Nejdl (Eds.),<br/> The Adaptive Web. Methods and Strategies of Web Personalization (pp. 136-154). Berlin / Heidelberg: Springer.</div> | ||
| − | == | + | == Overview == |
| − | + | [[CUMULATE]] is a centralized user modeling server built for the [[ADAPT2|ADAPT<sup>2</sup>]] architecture. It is mainly targeted at providing user modeling support for adaptive educational hypermedia (AEH) systems. [[CUMULATE]]'s data model consists of the following. | |
| + | * users, groups of users, and group membership links (syndicated from [[Knowledge Tree]]) | ||
| + | * learning objects, learning object providers (client applications) | ||
| + | * knowledge components - concepts - grouped in learning domains | ||
| + | * learning object metadata index - links between learning objects and corresponding concepts | ||
| + | * user activity log - historical log of all reported user interaction with learning objects | ||
| + | * assertions about user's knowledge of concepts ([concepts'] knowledge) | ||
| + | * summaries of user's interaction with learning objects, such as: number of attempts, percent complete, etc. ([learning objects'] progress) | ||
| + | [[CUMULATE]] implements several inference mechanisms to produce assertions about user's knowledge and progress, including the following. | ||
| + | * time-spent-reading (reading texts via [[Knowledge Sea II]] and [[AnnotatEd]]) | ||
| + | * thresholded averaging (browsing examples via [[WebEx]] and exploring expressions via [[WADEIn]]) | ||
| + | * [[CUMULATE asymptotic knowledge assessment|asymptotic assessment of user knowledge]] (problem solving via [[QuizPACK]], [[QuizJET]], and [[SQLKnoT]], which is the main one deployed in our systems) | ||
| + | * [[CUMULATE parametrized asymptotic knowledge assessment|parametrized asymptotic assessment of user knowledge]] (<span style="color:maroon;">under development</span>) | ||
| + | * knowledge tracing (<span style="color:maroon;">under consideration</span>) | ||
| + | * [[Feature-Aware Student knowledge Tracing (FAST)|Feature-Aware Student knowledge Tracing (FAST)]] (<span style="color:maroon;">under development</span>) | ||
| − | == | + | == Modeling User's Knowledge == |
| − | + | [[Image:CUMULATE.evidence propagation.png|thumb|'''100'''|Evindence propagation in CUMULATE]] | |
| − | + | [[CUMULATE]] models user's knowledge of concepts with respect to the following levels of [http://www.officeport.com/edu/blooms.htm Bloom's taxonomy]. | |
| − | + | * knowledge - corresponds to reading tutorials or book chapters ([[Knowledge Sea II]], [[AnnotatEd]]) | |
| − | * | + | * comprehension - reviewing examples, watching demos ([[WebEx]], [[WADEIn]] demo mode) |
| − | + | * application - answering quiz questions ([[QuizPACK]], [[QuizJET]], [[WADEIn]] evaluation mode) | |
| − | * | + | * synthesis - constructing problem solutions ([[SQLKnoT]]) |
| − | + | Upon receipt of new information about user activity (interaction with a learning object), [[CUMULATE]] first verifies the integrity of the reported information (user is registered, learning object provider is known and learning object exists, etc.). Then, based on the learning object provider and result of interaction, it is determined what inference mechanism to use and what [http://www.officeport.com/edu/blooms.htm Bloom's level] to affect. [[CUMULATE]] then looks up the domain concepts for the learning object the user interacted with and performs the update of the knowledge. Summaries of progress with learning objects are updated as well. The process of updating user model is called '''evidence propagation''' (see figure on the left).<br/> | |
| − | |||
| − | |||
| − | * | ||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | == | + | <span style="color:red;">New!</span> Recent update to [[CUMULATE]] allows learning object providers to skip learning object metadata lookup and report changes in user's knowledge of domain concepts directly. |
| − | [http://www.sis.pitt.edu/ | + | |
| + | == Using [[CUMULATE]] == | ||
| + | [[CUMULATE]] is has 3 major entry points: | ||
| + | # interface for reporting user interaction with learning objects | ||
| + | # interface for reporting changes to user knowledge of concepts in-the-raw (<span style="color:red;">new</span>) | ||
| + | # interface for acquiring models of user knowledge/progress as a whole or in part | ||
| + | Interaction with CUMULATE is regulated by a set of [[CUMULATE protocol|protocols]]. | ||
| + | |||
| + | ==Publications== | ||
| + | <onlyinclude> | ||
| + | * Zadorozhny, V., [[User:myudelson|Yudelson, M.]], and [[User:peterb|Brusilovsky, P.]] (2008) '''A Framework for Performance Evaluation of User Modeling Servers for Web Applications'''. Web Intelligence and Agent Systems 6(2), 175-191. [http://dx.doi.org/10.3233/WIA-2008-0136 DOI] | ||
| + | * [[User:myudelson|Yudelson, M.]], [[User:peterb|Brusilovsky, P.]], and Zadorozhny, V. (2007) '''A user modeling server for contemporary adaptive hypermedia: An evaluation of the push approach to evidence propagation'''. In Conati, C., McCoy, K. F., and Paliouras, G. Eds., User Modeling, volume 4511 of Lecture Notes in Computer Science, pp 27-36. Springer, 2007. [http://www.pitt.edu/~mvy3/assets/YudelsonUM2007.pdf PDF] [http://dx.doi.org/10.1007/978-3-540-73078-1_6 DOI] | ||
| + | * [[User:peterb|Brusilovsky, P.]], [[User:sergey|Sosnovsky, S. A.]], and Shcherbinina, O. (2005). '''User Modeling in a Distributed E-Learning Architecture'''. Paper presented at the 10th International Conference on User Modeling (UM 2005), Edinburgh, Scotland, UK, July 24-29, 2005. [http://www2.sis.pitt.edu/%7Epeterb/papers/cumulateUM05.pdf PDF] [http://dx.doi.org/10.1007/11527886_50 DOI] | ||
| + | </onlyinclude> | ||
| + | |||
| + | == See Also == | ||
| + | * [[CUMULATE RDF Binding]] - work in progress on serializing part of [[CUMULATE]]'s data model in [http://en.wikipedia.org/wiki/Resource_Description_Framework RDF] | ||
| + | * [[CUMULATE Usage Logs]] - part of the [[CUMULATE]]'s log data the downloadable format | ||
| + | * [[CUMULATE user and domain adaptive user modeling|Adaptation]] of [[CUMULATE]]'s model inference to accommodate student's individual differences as well as the differences in complexity of problem-solving exercises. | ||
| + | * [[Blending layers of CUMULATE's user model]] - a stream of work targeted at combining layers of [[CUMULATE]]'s user model (e.g. problem-solving and example-browsing levels) to improve predictive validity of the modeling. | ||
| + | |||
| + | == Contacts == | ||
| + | [[User:Myudelson | Michael V. Yudelson]] | ||
Latest revision as of 03:15, 4 April 2016
CUMULATE (Centralized User Modeling Architecture for TEaching) is a central user modeling server
designed to provide user modeling functionality to a student-adaptive educational system. It collects evidence (events) about student learning from multiple servers that interact with the student. It stores students' activities and infers their learning characteristics, which are the basis for an individual adaptation to them. ... External and internal inference agents process the flow of events and update the values in the inference model of the server. Each inference agent is responsible for maintaining a specific property in the inference model, such as the current motivation level of the student or the student's current level of knowledge for each course topic...
Kobsa, A. (2007) Generic User Modeling Systems In P. Brusilovsky, A. Kobsa and W. Nejdl (Eds.),
The Adaptive Web. Methods and Strategies of Web Personalization (pp. 136-154). Berlin / Heidelberg: Springer.
The Adaptive Web. Methods and Strategies of Web Personalization (pp. 136-154). Berlin / Heidelberg: Springer.
Contents
Overview
CUMULATE is a centralized user modeling server built for the ADAPT2 architecture. It is mainly targeted at providing user modeling support for adaptive educational hypermedia (AEH) systems. CUMULATE's data model consists of the following.
- users, groups of users, and group membership links (syndicated from Knowledge Tree)
- learning objects, learning object providers (client applications)
- knowledge components - concepts - grouped in learning domains
- learning object metadata index - links between learning objects and corresponding concepts
- user activity log - historical log of all reported user interaction with learning objects
- assertions about user's knowledge of concepts ([concepts'] knowledge)
- summaries of user's interaction with learning objects, such as: number of attempts, percent complete, etc. ([learning objects'] progress)
CUMULATE implements several inference mechanisms to produce assertions about user's knowledge and progress, including the following.
- time-spent-reading (reading texts via Knowledge Sea II and AnnotatEd)
- thresholded averaging (browsing examples via WebEx and exploring expressions via WADEIn)
- asymptotic assessment of user knowledge (problem solving via QuizPACK, QuizJET, and SQLKnoT, which is the main one deployed in our systems)
- parametrized asymptotic assessment of user knowledge (under development)
- knowledge tracing (under consideration)
- Feature-Aware Student knowledge Tracing (FAST) (under development)
Modeling User's Knowledge
CUMULATE models user's knowledge of concepts with respect to the following levels of Bloom's taxonomy.
- knowledge - corresponds to reading tutorials or book chapters (Knowledge Sea II, AnnotatEd)
- comprehension - reviewing examples, watching demos (WebEx, WADEIn demo mode)
- application - answering quiz questions (QuizPACK, QuizJET, WADEIn evaluation mode)
- synthesis - constructing problem solutions (SQLKnoT)
Upon receipt of new information about user activity (interaction with a learning object), CUMULATE first verifies the integrity of the reported information (user is registered, learning object provider is known and learning object exists, etc.). Then, based on the learning object provider and result of interaction, it is determined what inference mechanism to use and what Bloom's level to affect. CUMULATE then looks up the domain concepts for the learning object the user interacted with and performs the update of the knowledge. Summaries of progress with learning objects are updated as well. The process of updating user model is called evidence propagation (see figure on the left).
New! Recent update to CUMULATE allows learning object providers to skip learning object metadata lookup and report changes in user's knowledge of domain concepts directly.
Using CUMULATE
CUMULATE is has 3 major entry points:
- interface for reporting user interaction with learning objects
- interface for reporting changes to user knowledge of concepts in-the-raw (new)
- interface for acquiring models of user knowledge/progress as a whole or in part
Interaction with CUMULATE is regulated by a set of protocols.
Publications
- Zadorozhny, V., Yudelson, M., and Brusilovsky, P. (2008) A Framework for Performance Evaluation of User Modeling Servers for Web Applications. Web Intelligence and Agent Systems 6(2), 175-191. DOI
- Yudelson, M., Brusilovsky, P., and Zadorozhny, V. (2007) A user modeling server for contemporary adaptive hypermedia: An evaluation of the push approach to evidence propagation. In Conati, C., McCoy, K. F., and Paliouras, G. Eds., User Modeling, volume 4511 of Lecture Notes in Computer Science, pp 27-36. Springer, 2007. PDF DOI
- Brusilovsky, P., Sosnovsky, S. A., and Shcherbinina, O. (2005). User Modeling in a Distributed E-Learning Architecture. Paper presented at the 10th International Conference on User Modeling (UM 2005), Edinburgh, Scotland, UK, July 24-29, 2005. PDF DOI
See Also
- CUMULATE RDF Binding - work in progress on serializing part of CUMULATE's data model in RDF
- CUMULATE Usage Logs - part of the CUMULATE's log data the downloadable format
- Adaptation of CUMULATE's model inference to accommodate student's individual differences as well as the differences in complexity of problem-solving exercises.
- Blending layers of CUMULATE's user model - a stream of work targeted at combining layers of CUMULATE's user model (e.g. problem-solving and example-browsing levels) to improve predictive validity of the modeling.
