Manual Review

Manual Review

In addition to automatically graded multiple-choice and short answer questions, CredSpark enables you to review and score short and long-answer questions, as well as files uploaded by your end users. You can assign various point values to each question and include feedback to your users with that final score.

Enabling Manual Review

Manual Review is compatible with three question types: Short Answer, Long Answer, and File Upload. To enable Manual Review for a specific question, check the "Manual Review" box during question setup.

When you check the 'Manual Review' box, CredSpark automatically includes an 'Email' question type in your interaction. This is so you can collect a participant’s email so they can be notified when their score is ready. You have the option to remove the Email question if you don't intend to use it for notifying participants about their scores, you can do it by unchecking the "Require Registration" option in the Registration tab:

However, it's advisable to delete it only after all Manual Review questions are added to prevent the app from recreating the Email question. Note that under certain circumstances, such as when using special conditions like Skip Logic or a Question Pool, the Email question may not appear. Therefore, it's essential to thoroughly test your quiz before distributing it.

Reviewing Responses

Once you start receiving responses, it's time to review them. Head to the "Data & Analytics" tab and navigate to the "Manual Review" subtab. Here, you'll find two essential sections:

• Response Summary
• Question Review and Feedback

Response Summary

The Response Summary displays a list of your respondents along with their statuses:

Pending: Responses not yet reviewed.

Reviewed: Responses reviewed, but participants not yet informed of their final score.

Notified: Responses reviewed, and participants notified by email of their final score.

You can filter the list and manage columns using icons in the top right corner.

Participants will only receive score updates and access to the updated Results View after you have notified them. If you've reviewed their responses but haven't sent notifications, they won't be able to view the updated Results View.

Question Review and Feedback

In the Question Review and Feedback section, you can assess responses. For regular scoring, mark answers as "Correct" or "Incorrect." For Custom scoring, enter the total points earned. You can also provide feedback to participants, which they can later view in their Results View. You can edit the score and feedback even after score release.

The reviewed score also appears in the "Individual Engagement" tab "Results" subtab, along with a "Reviewer" column indicating who reviewed the response. We display the most recent reviewer; that is, the one who last performed actions such as marking questions as correct/incorrect, entering custom scores, modifying scores, or providing feedback.

Releasing Scores

When you've finished reviewing responses, it's time to release the scores. You can do this by clicking the email icon in the Response Summary section. Select individual participants or release scores to all. Note that you can only release scores for fully reviewed responses, and you can email participants multiple times if needed.

Participant Experience

In the Results View, CredSpark provides participants with information indicating that some scores are pending review.

While building your interaction, consider adding a note to inform participants about Manual Review at the question level. Once you release the score, participants will receive an email notification along with the ability to review their updated Results View. 

Sample email:

Updated Results View:

Note: Currently, this feature is supported in the English language only.

Integrations

In most instances, CredSpark promptly sends response information to other applications through the use of API or xAPI immediately upon completion. However, it's essential to note that this behavior changes when your interaction includes Manual Review questions. In these cases, we temporarily pause the data transmission until the Manual Review question has been thoroughly assessed and the final score is assigned. This process ensures that we send the definitive and accurate results to your integrated applications.

By following this guide, you can effectively utilize the Manual Review feature to ensure accurate assessments and provide valuable feedback to participants.