Random review assignments

They would like the review assignments to be truly random. Essentially, the reviews get distributed in a certain order that is less than ideal.

This problem could be equally solved by allowing them to change the order of the reviewers.
  • Deleted User
  • Feb 3 2016
  • Reviewed: Voting Open
Employee Name Khanh Nguyen
Client Name "shard name" uic, wisc, calalumni
User Reviewer
Functional Unit Assigning Reviews
  • Attach files
  • Tina Phifer commented
    June 06, 2023 20:49

    University of Oregon would like this as well for two reasons: One is to help reduce reviewer bias by making assignments random. The other reason is to help when one Reviewer does not end up doing their job and has to have their assignments split up to the rest of the group. In seven years of managing multiple review committees, I have rarely had every Reviewer complete their actual work. Usually at least one person does not complete their assigned Reviews. Example of a recent review I had:

    • 1 Review Group (1 Apply-To Opportunity)

    • 4 Reviewers

    • 50 applications

    • Review Group set for 2 reads per application

    • I assumed every Reviewer would get randomly assigned 25 applications to read

    • What actually happened was that the system gave the same apps to Reviewers 1 & 2, and the rest of the apps to Reviewers 3 & 4.

    • Once Reviewers 1,2 & 3 completed reviewing, Reviewer 4 didn't end up doing their work (an incredibly common scenario)

    • The review committee decided they wanted to just split those 25 apps amongst the remaining 3 Reviewers to share the load.

    • However, since Reviewer 3 had the EXACT same apps as Reviewer 4, all the remaining reviews were assigned to Reviewers 1 & 2 only.

  • Guest commented
    September 21, 2020 18:45

    I assumed the reviews would be assigned randomly and was surprised to find out they were not. I hope this can be fixed moving forward.

  • Admin
    Carrie Goldberger commented
    July 05, 2019 13:54

    Please see similar request from Utah in merged enhancement; description below -

    "Please randomize the reviewers when assigning them to students. For example, when two people reviewed the same applications, the auto assign seemed to assign the same two reviewers to the same applicants."

  • Stephanie Parmentier commented
    April 18, 2019 23:28

    UW Madison would also be interested in this feature