This article will walk you through the judging process and teach you how to add judges to your hackathon.
Your first task is to add the list of judges to your hackathon. When you are creating your hackathon, you will see a section called “Judging” on the dashboard. Click “Judging” under “Edit Your Challenge” to fill in the names of your judges.
The information added to the Judges section is publicly displayed on your hackathon, except for the judges’ email addresses and user names.
You can add anyone with an email address as a judge. When the judging period starts, each judge will receive an email that provides a link for judging submissions. (More on this later.) The email will also ask the judge to create an account, if necessary.
After you’ve added your panel of judges to your hackathon, the next step is to add some judging criteria. The criteria serve as guidelines for both judges and submitters. Challenge.gov uses a 1-5 star rating for criteria. (More on this later.)
The final step in setting up judging is to fill out the “Dates” section.
The dates set the overall judging timeline, but the following rules also determine the timing:
Let’s say the submission period is over and you’re thrilled with the response. You’ve marked each submission as “Gallery” or “Hidden.” (Note: Submissions marked “Gallery” will be publicly visible only after you’ve clicked on “Turn on Gallery.”) However, you have too many “Gallery” submissions for your busy judges to review, and you’d like to narrow them down from the 100 “Gallery” submissions to the 20 best ones for the judges to review.
So how do you do this? For each submission, select a status: “Judge,” “Maybe,” or “Don’t Judge.” The judges will see only the submissions you have marked “Judge.” You can use the “Maybe” status for submissions you’re not sure about, but all submissions will eventually have to be marked “Judge” or “Don’t Judge.” Please note: The judging period cannot start if any submissions are marked “Maybe.”
Once you’ve reviewed and marked all your submissions, you will have to click the “Turn on the Gallery” button to make the eligible submissions publicly visible. Please note: The judging period cannot start until you turn on the gallery.
We highly recommend that you do not turn the gallery off after it has been turned on. If you turn the gallery off and then on, all the submitters of entries marked “Gallery” will receive a new email stating that their submission has been published.
The judging period has started. You’ve marked your submissions as “Judge” or “Don’t Judge.” Now it’s time to send a message to your judges and invite them to start judging.
Click the “Start Judging” button under the Judging/Voting section of the dashboard. The following email will automatically be sent to your judges:
Please note that the only way that judges can start judging is to click on the link provided in this email. Each judge will receive a unique link that will work only for that judge.
After the invitations have been sent, you can check the judges’ progress on the judging dashboard.
The dashboard shows you which judges have started judging and their scores. If the judging deadline is approaching and certain judges have not made any progress, you may want to send them a reminder.
When a judge clicks the “Start judging” link in the email, the following page will appear.
The screen shows the judging criteria, as well as the judge’s progress through the submissions. The “Start Judging” button takes the judge to the first submission.
For each item in the list of criteria, the judge should assign a rating of 1-5 stars.
1 star is the lowest rating and 5 is the highest rating. (There is no other mechanism for judges to provide feedback.) For more details about the criteria, a judge can click the question mark icon next to each item in the list.
If a judge does not want to judge a particular submission because of a conflict of interest, he or she can select the “I recuse myself” checkbox.
After all submissions have been rated, the judge’s work is done.
You can manually end judging any time after the “Judging Period Ends” date has passed. We leave it up to you in case you want to allow additional time for judging. Simply click “Finish Judging” to end judging.
Want to know how the judging process works for the judges? Check out our article on How judging works (for judges).