The succes of weekly feedback

Project introduction and background information

A well-known course practice at Dutch Universities is weekly lectures followed by a final exam. The success rate (% students passed after the first exam) for courses on Mechanics and Dynamics is usually in the order of 20%. This success rate can be increased dramatically by introducing weekly formative feedback in the form of: weekly homework assignments, in-class quizzes, and office hours. For a 2nd year BSc course on Dynamics (700+ # students) the success rate tripled from 20% to 66%.

This project was started because of the long lasting history of low success rates for Mechanics courses in the BSc study in Mechanical Engineering at Delft University of Technology. With an average success rate of 20%, on average, students have to do the exam 5 times, and staff has 5 times the amount of normal work. Both students and staff are unhappy with that.

Every year around 750 students take mechanics courses. However, there are also students who failed, so should repeat this course. Therefore, the number of enrolled students is around 1000.

Objective and expected outcomes

The goal was to increase the success rate (% students passed after the first exam) of BSc Mechanics courses.

Results and learnings

• Improved student motivation: There were 2 groups of students. One group really like the idea. They did their weekly assignments, quizzes, etc. There were also another group who hate this idea. They found it childish, they think we grown up.
• Impact, outcome and output: The students were really bussy with the course.The total number of lecture hours is reduced by 50%, instead weekly Office Hours are introduced.
• Weekly computerized In-class Quizzes are introduced, voting by smartphone or laptop (Turning Point software is used).
• Weekly Homework assignments are introduced (Problems from the book), handed in on paper, graded by Teaching Assistants.
• Weekly Office Hours are held by the Teaching Assistants, to assist and give feedback on the graded homework.
• The average homework grade (HW) (two lowest are dropped) and the average In-class Quiz grade (IQ) (two lowest are dropped) can each be 25% of the final grade.
• There is a written exam (WE)
• The Final grade is 0.5*WE+0.25*max(HW,WE)+0.25*max(IQ,WE), which means that both Homework and In-class Quizzes are on a voluntary basis.
• For a 3 ECTS course on Dynamics (2nd BSC) with 700+ students, a total of 16 Teaching Assistants, working 7 weeks for 8 hours were needed.

The success rate (% students passed after the first exam) tripled from 20% to 66%. I made this promise to the students: “ If you do all your homework by yourself, every week, you will pass the course, the written exam is only there to check it”. And indeed, of the number of students who had a pass grade (>=6) on the Homework only 4% failed on the Final grade (<6).

Recommendations

• Be sure that the IT system works.
• With the homework start in a slow manner, start with not too difficult, easy homework in the first 2 weeks.
• One of the problems is lecture rooms. When you do in-class quizzes suddenly all the students want to attend the lecture. However, we do not have enough lecture rooms here. We have 4 lecture rooms with video-linked. I was standing in one rooms and in the other rooms we had video, there was nobody.
• The other thing we had some technical problems with in-class quizzes with voting with the smartphones or laptops. I did not foresee that the network could not support all these. It was because of the large number of students and because the wireless network was not built here to support it. I should have contact with IT department half year before starting this project and tell them what we are going to do, etc. I did not do that, I thought we have network and it should work. So, IT is an issue. You really should test it well before.
• Another problem was, if you have 750- 1000 students and you do weekly assignments then you need teaching assistants. We made groups with 70 students so, we needed 10-12 teaching assistants. You have to advertise and make interviews to select the most motivated teaching assistants. The second problem was you have to teach teaching assistants. You need to plan 2 hours per week at least to work them together and let them to do homework problems and assist them. Still they were not 100% up to speed during the office hours when the students come to teaching assistants. It should be done better way.
• Another thing is, having office hours with 12 groups means you need to 12 locations where they can sit. We don ‘t have so many rooms. That’s why they were spread out all over the university. I wanted to visit all these groups and see how things were going. That’s why I got my bicycle and ride through university from one building to another. The ideal is to have all of them in the same building.
• When we start at the beginning with difficult homework problems, students just stop doing homework. So, the lesson learned here is to start with not too difficult homework problems, not to scare students at the beginning.