New digital tools for assessment of complex cognitive processes

Project introduction and background information

The course 'FPE-30306 Food structuring', with 180 students, is a MSc course, characterised by learning outcomes (LOs) related to high cognitive levels, such as “design relevant processes for complex food products”. We have developed our teaching activities accordingly to activate the learning process of students, including critical thinking (e.g., motivate choices, make estimations, comment on whether estimations are realistic, design a process or product (with limited information). The assessment is recently changed to a digital examination with multiple choice questions. This makes it difficult to assess the high level learning outcomes. With this innovation project, we would like to develop a new assessment tool, combining a digital, automated interface and possibilities to assess cognitive learning outcomes of high complexity.

In the course 'FPE-30806 Sustainable Food and Bioprocessing' with 120 students, the evaluation of existing processes, the design of optimized processes and the high cognitive levels are addressed. This course is recently included in an online MSc program and is facing increasing student numbers. All online exams are taken via the Question Mark Perception assessment tool, which is fully built on multiple choice type questions (and its variants). In the exams, we cannot use ‘open questions’ assessments. However, the multiple choice type questions are definitely not suitable for assessing the learning outcomes of this course. This gives a conflict that should be solved.

In both courses we need to find other methods of digital examination (i.e. other than multiple choice with predefined answer options), which allow assessment of higher level learning outcomes.

Objective and expected outcomes

The aim of this project is the development of a new digital and automated assessment tool, that gives the possibility to assess cognitive learning outcomes of high complexity. The assessment tool should be suitable for high students numbers, and should be suitable to be taken in a digital and online setting (for online courses).

• This project allows us to align the assessment of the learning outcomes with our educational activities.
• It addresses the need for assessing learning outcomes of higher complexity in a realistic setting which requires independent and critical thinking in realistic situations.
• It aims to enrich the learning environment.

Results and learnings

• Develop an optional set-up of the automated exam questions, together with optional answers (approaches/estimations/interpretations). Define examples that explain our intentions.
• Check existing software, in collaboration with Educational Staff Development/IT/Digital examination
• Get in contact with other universities (4TU) for more insight in existing software for solving our problem.
• Develop a tool that allows automated and online assessment. We have some experience with using Excel, where students can answer and make drawings or calculations in a designated area of the sheet. Our first experience with it was promising, it gives us many possibilities to design questions. We expect we can use it such that it can also score the answers in an automated way.
• Test the developed tool in the target student group
• Optimize based on recommendations