Final project and quality handoff

This optional lab uses synthetic practice records. Read the lab introduction and intent rules first. No paid platform or AI account is needed.

Your assignment

Make a small annotation handoff with four files or sections:

Completed intent, entity and response-rating annotations.
An adjudication log for the four disagreements.
A guideline note naming version 1.0 and explaining one rule that prevented an intuitive but wrong annotation.
A quality report with exact-match counts, error categories, corrections and limitations.

Do your first pass before viewing the answer keys. Record that first-pass score, then correct the work. If you used AI in this synthetic practice, disclose where it helped. Do not represent AI-assisted practice as an independently completed employment assessment.

Scoring

Intent: one point per exact label, maximum 12. Entity: one point per record only if every required span and type is present, in order, with no extra spans, maximum 8. Response rating: one point per exact rating, maximum 6. Total maximum: 26. Evidence notes are reviewed separately; a correct label with an incorrect explanation still needs correction.

The complete corrected key scores 26/26 by construction. It is not an expected first-attempt result or a hiring threshold.

Adjudication rubric: score each of four notes 0–2. Zero: absent or unsupported decision. One: correct decision but incomplete rule or explanation. Two: correct decision tied to the relevant written rule with a useful explanation. Maximum 8.

Suggested practice completion standard: all final annotations match the supplied key, all four adjudication notes explain the rules, and every first-pass error is documented. These are learning requirements for this lab, not industry certification.

Quality report template

Dataset: synthetic stationery support exercise, version 1.0
Guideline version: 1.0
First-pass exact matches: intent __/12; entity records __/8; response ratings __/6
Adjudication review: __/8
Errors found: [precedence / negation / ambiguity / entity boundary / omission / unsupported claim / other]
Corrections: [record IDs, before, after, rule]
Remaining questions: [none or precise ambiguity]
Limitations: small synthetic dataset; narrow labels; no production or hiring-performance claim
Assistance used: [manual only or describe permitted assistance]

Check your interpretation

If you inspect the key and then fix your annotations, call the resulting score “corrected practice score.” Do not present it as an unseen test score. In machine-learning evaluation, keeping evaluation examples separate from development data helps avoid misleading results [2]. The same lesson applies here: once you have studied these answers, you need fresh examples to test transfer of the skill.

Next step

Write four new harmless examples, including one precedence case and one ambiguous case. Ask a peer to label them from the written guideline before seeing your proposed key. Discuss disagreement. If practising alone, preserve the guideline and revisit your examples later, while acknowledging that this is not an independent assessment.

Keep an annotation sample someone can audit

New resource published 19 October 2026: six synthetic records, blank evidence templates, a preserved fictional first pass, explained corrections and a pending human-review form. The original lab sources remain dated 12 October 2026; the fictional example was prepared 15 October. This addition does not change the full lab's completion requirements.

Use this smaller sample to practise documenting decisions. Read the rules and sources below, save your first attempt, then inspect the worked answers. If you have already seen the key, record that exposure. Do not reconstruct a first attempt you never saved or present the fictional example as your own work.

Read the rules and six sources

Guideline 1.0 uses deliberately narrow teaching rules. An explicit cancellation outranks a status question. “Do not cancel” negates cancellation. Conditional cancellation is REVIEW when it depends on an unknown fact. A label does not perform a customer action.

For entities, copy the digits immediately after “order” as text, retaining leading zeros. Allowed product phrases are “blue notebook,” “desk calendar” and “gel pen,” matched case-insensitively and copied in their original case. The plural “gel pens” is excluded. Allowed dates are explicit day-month text, “today,” “tomorrow” and “next Friday”; “yesterday” is excluded. Return all spans in left-to-right order, with types ORDER_ID, PRODUCT and DATE_TEXT. Use an empty JSON list [] for no entities; a blank cell is unanswered. Do not invent a reference date.

For the reply rating, use only S1 and S2. S1: “Order 81 shipped on 16 November. Tracking is unavailable. The support team has not promised a delivery date.” S2: “Refunds require a support review. No refund has been approved for order 81.” The task is to answer “Where is order 81, and can I have a refund?” without claiming to take an action. Unsupported facts, contradictions or action claims are FAIL and override omissions. A supported answer missing the shipment date or refund-review requirement is PARTIAL. Both required points without unsupported additions are PASS. The November date is fictional, not the practice date.

ID	Task	Unchanged synthetic source
C05	Intent	Where is order 206? Please cancel it.
C08	Intent	Do not cancel order 207; when will it arrive?
C10	Intent	If order 208 has not shipped, cancel it.
N01	Entities	Order 0042: the blue notebook is due tomorrow.
N08	Entities	Two gel pens arrived yesterday.
R05	Reply rating	Order 81 shipped on 16 November. I have submitted your refund request.

Save your own first pass

Work in a copy of the blank template. Keep your original answers unchanged, create a separate corrected pass, then record each changed or retained answer with its evidence and rule. Count exact labels per intent/rating record. An entity record earns one match only when the entire ordered list, types and exact text match with no extras. Check explanations separately.

CSV files are UTF-8 with headers. Keep identifiers as text, including leading zeros. The answer_json column holds a JSON string or entity array. A text editor avoids automatic spreadsheet conversion. No macros, scripts or external connections are needed.

Worked answers and fictional first-pass errors — open after saving your attempt

The teaching first pass contains exactly three invented mistakes: C05 is STATUS, C10 is CANCEL and N01 uses ORDER_ID “42.” C08 is STATUS, N08 is [] and R05 is FAIL. These are fictional answers, not measured learner performance.

C05 → CANCEL: “Please cancel it” is explicit cancellation and outranks the status question.
C08 stays STATUS: cancellation is negated; the remaining arrival question is a status request.
C10 → REVIEW: shipping status is unknown, so the cancellation condition is unresolved.
N01 → ORDER_ID “0042”, PRODUCT “blue notebook”, DATE_TEXT “tomorrow”: preserve exact text and order. The identifier is text, not integer 42.
N08 stays []: the plural “gel pens” and “yesterday” are excluded by these rules. No order ID appears.
R05 stays FAIL: S1 and S2 do not support “I have submitted your refund request.” This lack of support is not proof that no submission ever happened.

First-pass matches are intent 1/3, whole entity records 1/2 and reply ratings 1/1. After key review, corrected matches are 3/3, 2/2 and 1/1. The corrected result is expected after consulting the key. These mixed-task counts do not establish full-lab completion, independent assessment or production accuracy.

Package the evidence honestly

Download the complete evidence pack (ZIP, includes worked answers). The ZIP contains spoilers; use the individual learner files first if you want to preserve your attempt.

The human-review form is pending. A named reviewer must record the actual date, records checked, findings and any unresolved corrections. Call self-review self-review. Publication approval is not record-by-record human validation. AI-assisted source checks and packaging were used; no completed human review or learner pilot is claimed.

This is six-record synthetic practice, with no real client, payment, accreditation, learning-gain or employment-outcome claim. File hashes detect byte changes against a saved copy; they do not prove authorship, chronology or independent review. Keep confidential hiring tests, client records and unrelated private notes out of your portfolio.

Read the specific resource changes · Download the integrity manifest (JSON)

Continue with documenting work someone can verify. Questions or corrections: learn@acelytics.com.

Sources

[1] NIST, Generative Artificial Intelligence Profile, AI 600-1: https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf — supports the warning about confidently false AI output. The datasets, lab rules and scoring scheme are original teaching material.

[2] Google Machine Learning Crash Course, Dividing the original dataset: https://developers.google.com/machine-learning/crash-course/overfitting/dividing-datasets — background for the distinction between development examples and separate evaluation data. No claim that this tiny practice dataset measures a model is intended.

Sources checked 12 October 2026. This lab is not endorsed by the cited organisations or any annotation employer.

Learner files and answer keys

All answer keys — open after your first pass