eFables

Project introduction and background information

Digital education provides a unique opportunity to reshape our educational practices in order to optimally use our limited resource in terms of human capital and continue to provide excellent education to our students. In this context developing knowledge and tools for the design of smart digital formative (and summative) assessments is a key step towards fully digital education.

Personalized learning is an area that will likely grow in importance in the coming years with the growth of MOOCs and open-source education. One of the key assets that university courses offer over online courses is the opportunity for interaction. This is, however, being held back by the massive growth in student populations. While lectures scale reasonable well with the growing student population, the personal attention in assessment is becoming pressured. To stay competitive with online materials, personalized learning must be introduced in the formative (and summative) assessment for many of the first and second year courses taught at university.

The course 4DB00 Dynamics and Control of Mechanical Systems is designed according to the blended learning concept, with weblectures, quizzes and solved examples in Canvas and lectures to deepen the knowledge students acquire by self-study in the digital learning environment. At the moment the quizzes in Canvas are relatively simple, meant for them to see if they have understood the theory presented in the weblectures. However, the current digital environment does not allow students to test whether they master the skills required to pass the course. In order to fully exploit the formative potential of this digital learning environment, we aim at developing a smart digital assessment tool that will provide students feedback on their progress towards the required learning goals.

In the future we would like to apply the knowledge gained in this project to develop a smart digital summative assessment. Possible extensions of the project may also investigate the inclusion of the developed tools in 2IL50, 2IT80, or 2IPH0.

Objective and expected outcomes

The overall goal of this project is to gain knowledge on smart digital assessments and develop a smart digital formative assessment tool.

• Assess the capabilities of Canvas and Cirrus for the implementation of smart digital assessments.
• Explore the possibilities for question integration that is semi-open-ended*.
• *The concept of semi-open-ended questions will be defined in the project, but could be for example by instantiated as drop-and-drop with empty placement positions. It extends on closed multiple-choice by requiring a higher cognitive level of understanding of the students.
• Explore the possibilities for the automated computation of solutions to questions, reducing the combinatorial explosion of multiple choice in a branching digital setting.
• Design and implement a smart digital assessment algorithm for the course 4DB00 as formative assessment tool (FAT).
• Test and evaluate smart digital assessment tools during study year 2018-2019.

The lessons learned in this project may directly spark developments for other first or second year courses at the university such as Data Structures (2IL50), Discrete Structures (2IT80), or Logic and Set Theory (2IT60). We will contact the responsible teachers and explore possibilities for further implementation in these courses. Furthermore we will write a report on the key lessons learned from the project and present these in one of the education lunches in M&CS.

Results and learnings

The first step is literature review on digital testing, followed by the assessment of the capabilities of Canvas and Cirrus. Branching assessments and personalized learning are an important topic in e-learning at the moment. We inventorized possible structures to give shape to this.

In the second phase we developed prototypes to quickly explore the possibilities for personalized learning. We evaluated the options for question development and inclusion and pick one of the options to develop in a fully functional assessment tool.

We  developed a digital assessment tool for the course 4DB00 that was used an evaluated in the study year 2018-2019.