Difference between revisions of "CUMULATE asymptotic knowledge assessment"
From PAWS Lab
(→Examples) |
|||
| (13 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
[[CUMULATE asymptotic knowledge assessment|Asymptotic knowledge assessment]] is [[CUMULATE]]'s legacy user modeling algorithm for computing user knowledge with respect to problem-solving. | [[CUMULATE asymptotic knowledge assessment|Asymptotic knowledge assessment]] is [[CUMULATE]]'s legacy user modeling algorithm for computing user knowledge with respect to problem-solving. | ||
| − | + | =Computation= | |
The formula below is used to update the knowledge levels of concepts (''c'') addressed in a problem (''p''). This formula reflects the following principles. | The formula below is used to update the knowledge levels of concepts (''c'') addressed in a problem (''p''). This formula reflects the following principles. | ||
* there are several domain concepts (knowledge items, rules, productions) involved in solving a problem; the knowledge of each of them is updated proportionally to the others | * there are several domain concepts (knowledge items, rules, productions) involved in solving a problem; the knowledge of each of them is updated proportionally to the others | ||
| Line 13: | Line 13: | ||
* ''<sub>succ</sub>att<sub>p</sub>'' - is a number of successful solutions to problem p prior to current attempt | * ''<sub>succ</sub>att<sub>p</sub>'' - is a number of successful solutions to problem p prior to current attempt | ||
| − | + | =Examples= | |
| − | Below is a graph of concept's knowledge level growth vs. number of successful attempts to apply it | + | Below is a graph of concept's knowledge level growth vs. number of successful attempts to apply it. Every time the concept is applied correctly in a new problem. Every time the concept is the only concept of the problem. Namely, the ''Wc,p''/Σ''Wc,p'' ratio is equal to 1. |
| + | |||
[[Image:CUMULATE asymptotic knowledge assessment - knowledge growth.png]] | [[Image:CUMULATE asymptotic knowledge assessment - knowledge growth.png]] | ||
| + | <br/>([http://chart.apis.google.com/chart?cht=lxy&chs=300x215&chd=t:0,10,20,30,40,50,60,70,80,90,100|0,42,56,72,79,83,85,87,88,90,92&chco=24588E&chxt=x,y&chxl=1:|0|.2|.4|.6|.8|1|0:|0|1|2|3|4|5|6|7|8|9|10&chm=o,24588E,0,-1,10&chg=10,20&chtt=Growth+of+knowledge+level+vs.|number+of+successful+attempts alternatively] via [http://code.google.com/apis/chart/ Google Chart API]) | ||
| + | |||
| + | =Criticism= | ||
| + | The major drawback of the above [[CUMULATE asymptotic knowledge assessment|asymptotic knowledge assessment]] algorithm is that it is intended to be used as ''one-fits-all'' method not suitable for parametrization. Namely: | ||
| + | * the initial knowledge level (''Ko'') can only start from 0 | ||
| + | * the speed of knowledge growth cannot be adjusted (e.g. slow for hard concepts, fast for easy concepts) | ||
| + | * problem repetition penalty is not adjustable as well | ||
| + | As a result modeling is done in the same way for all users solving all problems. No special attention is given to individual user differences and individual problem complexity. | ||
| + | |||
| + | One of the ways to overcome these shortcomings is an attempt to create a [[CUMULATE parametrized asymptotic knowledge assessment|parameterizable]] version of this algorithm. | ||
| − | + | = Contacts = | |
| + | [[User:Myudelson | Michael V. Yudelson]] | ||
Latest revision as of 23:16, 8 April 2009
Asymptotic knowledge assessment is CUMULATE's legacy user modeling algorithm for computing user knowledge with respect to problem-solving.
Contents
Computation
The formula below is used to update the knowledge levels of concepts (c) addressed in a problem (p). This formula reflects the following principles.
- there are several domain concepts (knowledge items, rules, productions) involved in solving a problem; the knowledge of each of them is updated proportionally to the others
- knowledge is updated only upon correct user answers, there is no penalty for errors
- solving a problem correctly multiple times will result in diminishing update (growth) of the knowledge level of the concepts as the number of successes grows
, where
- Ko - is the starting level of knowledge (here we always start from 0)
- res - result of user action (0 -error, 1 - correct);
- Wc,p - is a weight of concept c in problem p
- ΣWc,p - is the sum of weights of all concepts in problem p
- succattp - is a number of successful solutions to problem p prior to current attempt
Examples
Below is a graph of concept's knowledge level growth vs. number of successful attempts to apply it. Every time the concept is applied correctly in a new problem. Every time the concept is the only concept of the problem. Namely, the Wc,p/ΣWc,p ratio is equal to 1.
(alternatively via Google Chart API)
Criticism
The major drawback of the above asymptotic knowledge assessment algorithm is that it is intended to be used as one-fits-all method not suitable for parametrization. Namely:
- the initial knowledge level (Ko) can only start from 0
- the speed of knowledge growth cannot be adjusted (e.g. slow for hard concepts, fast for easy concepts)
- problem repetition penalty is not adjustable as well
As a result modeling is done in the same way for all users solving all problems. No special attention is given to individual user differences and individual problem complexity.
One of the ways to overcome these shortcomings is an attempt to create a parameterizable version of this algorithm.
