Grading Logic

In order to distribute points, XP and achievements based on the performance of the participants in various activities, an automated grading logic has been implemented in KlickerUZH. This chapter provides a detailed explanation of the approaches employed for different element and activity types, including practice quizzes, microlearnings and live quizzes.

The presented grading approach assumes that all considered elements have a defined correct solution (or possibly multiple solutions), as this is required in asynchronous activities, except for free-text questions. During live quizzes where elements without sample solutions are used, the participants are awarded a fixed amount of 10 base points for taking part in each poll, unless this is deactivated on the respective element. Please also note that the grading approach for live quizzes slightly extends the logic of asynchronous activities and that some element types are only available in select activities.

Grading by Element Type

The grading logic will first be described on the example of practice quizzes and microlearnings. The additional base points, as well as bonus components for fast and correct answers during live quizzes will be explained in a dedicated section. Flashcards and content elements are not graded, since they rely on self-assessment or do not have solutions, respectively.

Multiplier

The point multipliers, which can be specified both on an element and activity level, are combined during the creation of the activity and applied to the total awarded points (including and potential bonuses in live quizzes). If no multipliers were specified, this factor defaults to 1.

Multipliers can be used to weigh elements according to their difficulty and reward participation in certain quizzes with more points (e.g. quiz on exam level during the last lecture of the semester vs. an introductory quiz in the first lecture). The same multiplier concept also applies to live quizzes.

Grading Single Choice Questions

Single choice questions have a single correct answer, by definition, wherefore participants will receive 10 points, multiplied with any potential multipliers, if they answer correctly.

$$\text{points}_{\text{awarded}} = \text{multiplier} \cdot \begin{cases} 10 & \text{answer correct} \\ 0 & \text{otherwise} \end{cases}$$

Grading Multiple Choice Questions

For multiple choice questions with $n$ answer options, all of them can be selected to be correct or incorrect. Each option is considered separately during grading, and every mismatch between the correct solution and the participant’s input is counted as one mistake. The awarded number of points then depends on the percentage of correctly answered questions:

$$\text{points}_{\text{awarded}} = \text{multiplier} \cdot \begin{cases} 10 & \text{correct} \\ 10 \cdot \left( 1 - 2 \cdot \frac{\text{wrong answers}}{\text{available answers}} \right) & \text{partially correct} \\ 0 & \text{otherwise} \end{cases}$$

The participant will therefore receive full points for a completely correct response and partial points until less than 50% of the answers are correct. In the shown example, there were 7 potentially correct answer options.

Grading KPRIM Questions

The grading logic for KPRIM questions is very similar to the presented implementation for multiple choice questions as KPRIM questions can be seen as a special case of multiple choice questions with exactly four answer options and a unified grading scheme. Given the fixed number of answer options, participants will receive full points for a completely correct answer, half the maximum points for a single mistake and no points otherwise.

$$\text{points}_{\text{awarded}} = \text{multiplier} \cdot \begin{cases} 10 & \text{correct} \\ 5 & \text{single mistake} \\ 0 & \text{otherwise} \end{cases}$$

Grading Numerical Questions

For numerical questions, you are able to specify a number of solution ranges or exact solutions, respectively. If the participant’s entered response is within one of these ranges or corresponds to one of the exact solutions, it is considered to be correct and will be awarded full points and no points otherwise (as for single choice questions).

Grading Free Text Questions

For free text questions, you can currently specify a selection of correct responses. The implemented simple grading approach compares the participant’s input to this sample solution, ignoring capitalization of letters and leading / trailing whitespaces. Again, full points are awarded for correct answers and no points otherwise (as for single choice questions).

Grading in Live Quizzes

During synchronous live quizzes, every participant will usually receive base points for each submitted answer, independent of its correctness (default: 10 points). However, for cases where the participation in a question might be considered to be voluntary, all graded element types additionally have an option to disable these base points (on the corresponding element edit view).

If the submitted answer is correct, additional points will be awarded (default: 5 points). For partially correct answers, these 5 points are multiplied with a correctness factor described in the sections above and rounded to the next integer value. If no sample solution is defined, students will only receive the fixed amount of base points, which is not affected by the choice of multipliers.

To incentivize fast and correct answers during live quizzes, additional bonus points are awarded beyond the mentioned points for answering correctly. Starting time with the first correct answer, players will receive up to 45 bonus points (default setting), depending on the time that passed between the first correct answer and theirs. By default, the slope to zero points is implemented with a duration of 20 seconds.

The corresponding resulting point curves for correct and wrong answers during live quizzes, when starting time with the first correct response and assuming enabled base points, are shown in the plot below (not considering multipliers).

Formally, the number of awarded points is calculated as follows:

$$\text{points}_{\text{awarded}}(t) = p_b + m \cdot \begin{cases} p_c + p_{\text{bonus}}(t) & \text{answer correct} \\ c \cdot \Big[ p_c + p_{\text{bonus}}(t) \Big] & \text{partially correct} \\ 0 & \text{otherwise} \end{cases}$$ $$p_{\text{bonus}}(t) = p_m \cdot \max\left(1 - \frac{t}{t_f}, 0\right)$$

The number of base points is denoted with $p_b$ (default: 10 points), the number of points awarded for correct answers with $p_c$ (default: 5 points), the number of maximum bonus points with $p_m$ (default: 45 points) and the duration for the bonus points to fade with $t_f$ (default: 20 seconds). The multiplier of the question instance (consisting of the activity and question multipliers) is denoted with $m$, and the time that has passed between the first correct answer and the participant’s answer with $t$. To keep the formalism simple, the correctness of the answer $c$ is normalized to the interval $[0,1]$ with $1$ being assigned for completely correct responses. The grading of answers is performed as described for asynchronous activities in the previous sections.

Advanced: Customized Grading

For advanced use cases, it is possible to fully customize the grading logic applied during live quizzes. Through a dedicated modal for advanced settings, which can be opened by clicking onto the settings icon after selecting a gamified course, you can choose a custom parametrization of the grading logic formalized in the previous section. A side-by-side illustration indicates the points awarded for correct or incorrect answers over time. If the option to not award any base points was chosen for a specific question, this setting overrides the live quiz settings for the number of standard points awarded.