More and better feedback with carrousel evaluations

Project introduction and background information

Design‐Based Learning (Ontwerp Gericht Onderwijs, OGO) as a method of teaching is broadly accepted. However, there is much debate about the method used to assess the outcome and the design process. Reflection on the process as well as on the contents is extremely important in DBL. Since reflection has to be learnt as part of learning how to design too, an evaluation method has been developed by Eindhoven University of Technology (TU/e) that is suitable for evoking abundant feedback to prospective designers. This method, called the Evaluation Assessment Carousel, is used for individuals as well as design teams and allows (many) guest critics to be involved. The Evaluation Assessment Carousel may be used for informal mid‐term evaluations, but also for formal final presentations that are meant for formative marks of design assessments. This project and its accompanying downloads describe the method and explains how to set up an Evaluation

Carrousel evaluations have been a proven practice for years. With this innovation, the method is better documented and evidence based so everyone can try it.

The manuals and evidence overview are provided as downloads.

Benefits of the Evaluation Assessment Carousel are according to Proveniers and Westra (2009):

• “It frees the so feared traditional Jury Critics from its 'darker side' and makes it a motivating and inspiring happening.
• It increases the developmental role: during several presentation rounds innovators learn to make their conceptual idea more complete.
• It gives all the participants the opportunity for a quick scan of quality of the other innovations.
• It creates more insight and involvement by the different academic and / or professional expert guest critics.

[...] During several presentation rounds students learn to make their conceptual idea more complete, as they learn from comments from the mix of different academic and / or professional specialists.

Design Studio teachers use it as a quick scan of quality of the other studios and for the other guests it also leads to increasing commitment.

[...] So ‐ in a wider scientific educational perspective ‐ the Evaluation Assessment Carousel can serve as a suitable educational assessment model for other innovative interdisciplinary scientific practice.”

(Proveniers, A. & Westra, J., 2009)

As explained, main advantages of the Evaluation Assessment Carousel are:

• “each design team gives the same presentation several times in the same time that is needed to spend in a traditional setup. The number of presentations given by one design team is equal to the number of jurors or teams of jurors. At the end of each presentation they receive feedback (multiple times, from individual critics who are not informed of earlier feedback from other critics).
• this multiple, yet separated feedback (with a student asking for clarification if there is a discrepancy between two reactions) results in clearer explained criticism in a way that a student understands. it also creates more objective criticism about their results.
• because students have to present their work more than once, they are more actively involved compared to a traditional setup.
• by repeating a presentation, students grow in the ability to find the right words to explain a design”
• [...] “different jurors can focus on different levels in scale [...]. In this way, the design teams receive feedback on various aspects of their design”.
• [...] “the comprehensive discussion together with the intermediate mark by each tutor provides the teams with a clear picture of where they stand at that specific stage in the design process” (Moonen, S.P.G. & Veeger, T.T., 2013).

The Evaluation Assessment Carousel does not require specific conveniences. A large hall is recommended but not necessary. Presentations can be divided over multiple rooms but in all cases, a clear allocation of the different places is essential, especially with a large group of presenters or if there are many jurors.

What is really needed is a good organization; however this is not new for educational programmes. An adequate organization is particularly needed when running the carousel itself to direct the program. A strict time‐referee is needed to keep track of the planned time schedule and who signals the rounds of presentations. A critical task for the time‐referee is to exhort jurors to finish a discussion when the time is up.

Since a lot of parallel actions take place the carousel seems time consuming, but in fact it is not. To enable every presenter to present once in a traditional setup takes a certain time span and so does this Evaluation Assessment Carousel. So, in exactly the same time span as a traditional set up, there are just many extra opportunities to present the results of a PBL project and to receive feedback.

The organization needed depends for the most part on the intention and on the number of participants. For all carousels, it is important to start on time to make a set‐up, to consider who to invite and to make a scheme. A carousel that is meant to foster improvements and ideas at a midterm evaluation (explained on page 15 of the Manual for Carrousel Evaluations) without marking requires a different approach than a carousel to provide formal marks at the end of a PBL project. And, a carousel with only staff involved is different than a carousel with many guests (explained on page 16 of the Manual for Carrousel Evaluations).