CrowdEd: Integrating Crowdsourcing In Education

Project introduction and background information

Educators actively seek the involvement of external parties in their educational activities to increase the relevance of teaching to society, motivate students and enrich the content of the course with potential applications of the taught material. Whether it is a guest lecture from an industry expert, a project inspired by an industrial context or an internship, all are sought after activities for both students and teachers alike.

Crowdsourcing is a development that has captured the attention of the public. In crowdsourcing, websites -online platforms- offer work, usually in exchange for money, which can be conducted by virtually anyone, but also other tasks that require more specialized skills. Usually, contributors -commonly referred to as “workers”- compete to get the prize associated with the work. There are literally hundreds of different platforms that offer thousands of tasks at the time of writing this proposal. Since tasks are already publicly available online, there is a unique opportunity to integrate their performance in educational activities. Our aspiration is that this integration will: stimulate students’ external motivation; improve students’ understanding of real stakeholders’ requirements and ways of communication; help teachers and students benchmark the students’ competencies in relation to real-world competition; offer teachers a steady and on-demand pool of industry-relevant activities. Our main objective with this project is to integrate crowdsourcing tasks in existing educational activities we offer at the department of Industrial Design to: 1) elicit requirements for its seamless integration; and 2) investigate the effect of crowdsourcing students’ motivation and performance.

Objective and expected outcomes

In this project we set out to investigate and better understand what are the attitudes of students and teachers alike when it comes to integrating crowdsourcing tasks into educational activities.

More specifically we:

• Conducted in-depth interviews with 30 students and teachers about the concept itself to capture their attitudes. Our main conclusion is that overall students are quite enthusiastic about the concept whereas educators are skeptical of the extra time that this activity might require from their side.
• Developed a web crawler to gather tasks and data around those tasks from eight design-oriented crowdsourcing platforms. Our main conclusion is that there is a plethora of design related tasks for our department.
• Developed a website to present crowdsourcing tasks from these eight crowdsourcing platforms and a recommender system based on students’ activities on the website. The website/platform is available at: Educrowd.tue.nl. 
• Captured data of 10 ID students in a two-week trial of Educrowd.tue.nl. We captured both the data while they browsed online. Our main (tentative) conclusion is that ID students do not find a match with their learning activities of the tasks that are presented in EduCrowd but do find the tasks original and attractive.

Results and learnings

The outcomes of the project are:

• Publish guidelines for embedding crowdsourcing into design education
• Gather data about student motivation in the context of integrating crowdsourcing tasks for their assignments
• Write and academic paper on the project’s results
• Establish closer collaboration with industry for this kind of activities
• Elicit requirements for a platform that would seamlessly integrate crowdsourcing tasks into education
• Design an interactive and working prototype of the platform based on requirements. The main aim of the platform is to help teachers search for and review potential crowdsourcing tasks, from more than one platform, that would fit their educational activities

Recommendations

Crowdsourcing is a rapidly growing concept that is slowly integrating the design world, online. Due to the abundance of online tasks, crowdsourcing has the opportunity to be implemented into education. Motivation, match to the learning objectives and support from the university are among important factors to realize this. Regarding the attitude of students and faculty towards the integration of crowdsourcing in design education we can conclude that the majority of staff and students reacted positively towards the proposition of integrating CS into ID education. Only 17% of participants were negative about the concept, which shows that there is wide support for integration of crowdsourcing in education. But, we also find that students and staff do require that the integration is properly evaluated and well organized, before it is fully implemented into the educational model. Generally teachers were more critical than students, when it comes to the integration of CS. They tend to see more bottlenecks and technical complications, but it is unknown whether this is due to staff having a different (more time-consuming) role in the process, or because they have a more critical view in general. This positive attitude though is not entirely backed up by EduCrowd’s recommender system data (from two week study with 10 ID students). Students’ average rating was M=1.7 (scale from 0 to 4, min: 0,max: 4). However, they thought that the tasks presented in EduCrowd were original (M=2.2; scale from 0 to 4, min: 0, max: 4) and attractive (M=2.3; scale from 0 to 4, min: 0, max: 4). Regarding important criteria for students and faculty when selecting crowdsourcing tasks to fit within design education we can conclude that “matchmaking” between available tasks and a student’s profile was deemed important. This finding lead to the 24 development of Educrowd.tue.nl, a recommender system (among others) that aims to be the virtual matchmaker. A rather surprising result is that it is not money, but the task experience itself, which is the greatest motivator for students. Students tend to perceive the prize as an added bonus for putting in a lot of work, but they claim to get their main motivation for working in real-life projects. Regarding the most convenient medium to implement the integration of crowdsourcing in design education we can conclude that Educrowd.tue.nl is an important first step in addressing the matchmaking need that was previously described. The results from the co-creation session showed that not only the researchers involved, but also the educational staff was considering this databasedriven recommendation system.

Practical outcomes

Please have a look at the downloads.