6.7 Three Wise Monkeys

No matter how tightly specified the criteria for assessing a project, assessors will always find room to vary.

---ooOoo---

This bundle reduces the total variance between assessors by reducing the number of assessors.

The way it works is you only have three assessors per project instance. They grade every project produced within a course (or module or cohort). In this way, they can then spend more time discussing the "implicit" criteria of the project, and are more likely to be able to form a consensus. Their grades will, probably, be closer and internally consistent.

It works better if there is a slow "turn-over" of assessors; that is to say only one rotates out of the job every year. This ensures that the assessment culture of projectwork within the department (the "way things are done here") is maintained and disseminated. It also helps to solve the problem of longitudinal variation in assessment over time.

It doesn't work unless assessors receive "credit" for the work. It doesn't work unless the projects are such that all assessors are equally capable of assessing the technical content.

---ooOoo---

So: consider reducing the number of people who assess projects.