Download the blank learner CSV. Complete your first pass before opening the answer key.
This optional lab uses synthetic practice records. Read the lab introduction and intent rules first. No paid platform or AI account is needed.
Learn
Rating a reply is a different task from labelling a customer message. You must check whether each claim is supported and whether the response follows the task. A helpful tone cannot compensate for a fabricated refund or an invented delivery date. Generative AI can confidently produce false content; compare output to evidence rather than relying on confidence [1].
Rating source and rules
S1: “Order 81 shipped on 16 November. Tracking is unavailable. The support team has not promised a delivery date.”
S2: “Refunds require a support review. No refund has been approved for order 81.”
Task: Reply to “Where is order 81, and can I have a refund?” using S1 and S2 only. Do not claim to take an action.
Rating scale, applied in this order:
- FAIL: at least one unsupported factual claim, false action claim or contradiction of S1/S2. This takes precedence over omissions.
- PARTIAL: no unsupported claim or contradiction, but the reply omits either the fact of shipment on 16 November (including the explicit date) or the refund review requirement.
- PASS: accurately covers shipment on 16 November and the refund review requirement without adding unsupported details or claiming action. Tracking and the absence of a promised delivery date may be mentioned accurately but are not required for PASS.
Try it
Rate R01–R06 in response-ratings.csv. Record the label and the exact phrase supporting your judgement. For an omission, name what is missing instead of inventing a quoted phrase.
Worked check
“Order 81 shipped on 16 November. A refund requires support review.” is PASS. It covers both required points and makes no promise that a refund will be approved.
“Your refund is processed” is FAIL, even if another sentence correctly states the shipment date. A correct clause does not neutralise a false action claim.
Answer key — open after your first attempt
Download the response-ratings answer key. Keep your first-pass score separate from your corrected score.