Is safe testing possible on a chromebook?

Project introduction and background information

Reasons digital testing of knowledge and skills

There are several reasons to discover the possibilities of digital testing in order to assess knowledge and skills of students. 

• Grading is very efficient for teachers and it is easier to monitor weak and strong subjects, monitor the level of students and their progress.
• Digital testing makes it possible to create more different kind of test items (adaptive testing, using pictures or video, randomize numbers in case of a math test). 
• Students get feedback immediately and therefore digital testing stimulates the learning process. 

Solution

Chromebooks:

Why chromebooks? Chromebooks have excellent battery life, quick boot times, they are affordable and much cheaper than a Windows-based laptop. In order to conduct the digital test with a safe solution, the UT purchased Chromebooks, combined with a Google Education Device Management Console. These Chromebooks where placed in a separate network, from where the web-based exam environment of MyLabsPlus was being accessed. The Chromebook solution functions as an "web-only" exam-kiosk machine. With the help of the Google Device Management Console, all local and remote resources (e.g. USB ports, internet) can be blocked with little effort. Only access to the specific parts of the exam environment where allowed.

Network:

The Chromebooks where pre-assigned to a dedicated network, and therefor where assigned a specific IP-range. In this pilot setup the access to the network was username/password based, as certificate authentication was still under development.

MyLabsPlus:

The MyLabsPlus exam environment was only opened for the specific time of the examination, and was only accessible from the dedicated assessment network.

Pilot set-up

The pilot was conducted by taking the following aspects in account

Aspects:

• Security Test: A Penetration / Hack test was done to test the security of the system
• Test session: In the test session, the students got acquainted with the system and learned how to start-up and enter the exam environment
• Actual test / Resit: Students executed the exam using the chromebook devices
• Delivery portal: A delivery portal was created to safely transfer students’ work to the lecturers
• Logistics: The test was scheduled in a large room to accommodate all the students, and with an appropriate storage / working room to accommodate the Chromebooks

Results and learnings

The Pilot has been surveyed amongst the students. Some of the results are summarized below. The students could rate the questions on a scale of 1-5. (1 = Fully disagree; 5 = Fully agree)

Questions:

• The WIFI strength during the test was sufficient: 4.7 
• There were no technical issues during the test: 4.5
• Did you encounter any hardware issues during the test 1.5 
• I believe that a digital exam in MyLabsPlus on a Chromebook is fraud proof 3.0 

In collaboration with the Technology Enhanced Learning & Teaching (TELT) team the educational program Applied Math set out a pilot with chromebooks to establish whether chromebooks could be used effectively and securely to deliver a math exam using the online application MyLapsPlus. Almost 60 Electrical Engineering students took part in the pilot.

Positive points:

• 60 Students were safely assessed with this solution.
• The system was technically stable.
• Both, Lecturers and students, are very satisfied with the technical solution.
• The Therm as the assessment location provided good facilities for conducting the examination, as well for the logistics around it.
• Starting up the assessment environment was smooth and straightforward, without needing specific instructions.
• The management of the whole setup is relatively low.
• The grading time decreased significantly.
• The grading is more reliable (the answer is good or wrong) and therefore more fair to students.
• There are more possibilities for digital testing to discover which can optimize the learning process of students (adaptive, more frequent testing for example).
• Digital testing makes it possible to randomize the numbers in test items. That could be another good way to prevent cheating.

Lessons learned

• During the test session there was a small technical issue. Important is that at early stage all parties involved in setting up and supporting the examination environment need to be standby when performing assessments.
• When using a standalone assessment system (like MyLabsPlus), students might forget their system specific password. It is important to instruct students in advance to test their login details. Even better would be integrating the assessment system with UT's authentication server.
• It takes about 30 minutes for 1 person to supply 60 Chromebooks to the students, and another 30 minutes for collection them afterwards.
• Access to the wireless network was username/password based. It is recommended to use certificate authentication.
• Students need big tables to use the chromebook and also write down solutions on paper before entering them in MyLabsPlus. 
• Designing a math test takes time. It is not useful to use the written exam and only change it into a digital test. Testitems should be suitable for digital testing.
• Students are not used to do a digital math test. Although the learning objectives are the same as last year it is very important to train them to do the digital testing to get acquinted to the new kind of testitems. There should be an alignment between the learning methods during the module and the digital test.
• It’s obvious that students do not completely trust that the digital exam is safe for testing. Students believe that there are always ways to undermine the digital environment. We should involve students more in this new way of testing to let them experience the testing is safe.

Follow up

During the pilot only the Electrical Engineering students were assessed by using Chromebooks. The other 500 students took the same test on paper. The UT will continue running more pilots with this setup. More Chromebooks are purchased, to facilitate also bigger groups of students. Also pilots on assessing skills by using desktop software, using a Chromebook, will be executed.