Marking that keeps pace with enrolment growth, Azure OpenAI feedback for a training provider

The marking backlog behind every enrolment win

A registered training organisation feels growth first in its marking queue. New enrolments are good news for the business and a quiet problem for the trainers, because the assessment load grows with the cohort while the number of qualified assessors does not. Formative feedback that used to come back in a few days starts taking a few weeks, and slow feedback is exactly what stalls a student’s progress through a unit.

The work itself is demanding and repetitive at the same time. For each submission an assessor reads the evidence, checks it against the unit’s performance criteria and knowledge evidence, writes feedback that points the student at what is missing, and records a competent or not-yet-competent judgement. The reading and the judgement need a qualified person. Much of the writing, the part that restates which criterion a comment relates to and explains the gap in plain language, is the same shape every time.

The bar an RTO has to clear is high, which is why a quick fix is risky. Assessment in vocational education sits under ASQA and the Standards for Registered Training Organisations, the broader VET Quality Framework, and the assessment requirements written into each training package. Higher-education-style providers answer to TEQSA as well. The decision must rest with a competent assessor, the feedback has to map to the actual criteria, and the whole thing has to stand up to an audit. A generic writing tool that produces fluent comments untethered from the unit is worse than useless here. It is an audit risk.

Why Azure OpenAI Service, grounded in the unit’s own criteria

The build that works is a marking assistant on Azure OpenAI Service that drafts the first pass of formative feedback, grounded in the unit’s assessment criteria through retrieval-augmented generation, with the assessor reviewing and owning every decision. The assessor opens a submission, the assistant returns a structured draft of feedback tied to each performance criterion, and the assessor edits, adds their own judgement and signs off before anything reaches the student.

Azure OpenAI Service headlines the build for one reason above the rest. It runs the generative model inside the RTO’s own Azure tenancy, in an Australian region, rather than sending student work to a public chatbot. That is the difference that matters for an education provider. Student submissions and personal details are handled under the Privacy Act 1988, the data stays onshore, and the content is not used to train the underlying model. A consumer chatbot offers none of that control, which makes it a poor fit the moment real student work is involved.

Retrieval-augmented generation does the grounding. Instead of relying on the model’s general knowledge, the assistant retrieves the specific performance criteria, knowledge evidence and the RTO’s marking guide for the unit in front of it, and is constrained to base every comment on that material and name the criterion it addresses. Microsoft 365 carries the last mile, because the assessors already live in it. Submissions and drafts move through the tools the team uses each day, so the assistant fits the existing workflow rather than adding another system to log into.

Building it, and where it got hard

The model was the easy part. The friction was in keeping its output honest against the criteria, and one early version made the problem plain.

The first draft of the assistant wrote beautifully and said almost nothing. Asked to give feedback on a unit submission, it produced fluent, encouraging, generic comments that could have applied to any student in any unit. “Good effort, consider adding more detail to strengthen your evidence” reads fine and means nothing. Worse, it was not tied to the specific performance criteria, so it gave an assessor no help and an auditor no trail. For an RTO that is not a small flaw. Feedback that does not map to the unit’s criteria is the kind of thing that surfaces in an ASQA audit.

The fix was to take generation off the leash and put it on the criteria. We grounded every feedback point in the actual unit material through retrieval, so the assistant works from the real performance criteria and the RTO’s marking guide rather than its own sense of what good feedback sounds like. We constrained the model to cite which criterion each comment addresses, and to flag a criterion as not yet evidenced rather than paper over a gap with praise. And we made assessor review mandatory by design. Nothing the assistant writes reaches a student until a qualified assessor has read the work, adjusted the draft and signed off, with their edits recorded.

Two further constraints shaped the rest. Each feedback point and each assessor edit is logged against its criterion, so the audit trail builds itself rather than relying on someone to keep notes. And the competency decision is kept structurally separate from the drafting, so the assistant never proposes a result. It drafts the words; the assessor makes the call.

What changed

In a representative build the assistant cut the time to a first feedback draft by roughly half, which moved the assessor’s effort from writing to reviewing and judging. The feedback also grew more consistent, because tying every comment to a named criterion narrowed the gap between a thorough assessor and a rushed one, so two students with comparable work got comparable feedback. And the audit trail came for free, with each comment recording the criterion it addressed and the edits the assessor made before sign-off.

These figures are illustrative. They describe the pattern we see rather than a published result for a named provider. The shape is the point. The repetitive drafting comes off the assessor, feedback reaches students faster and more evenly as the cohort grows, and the provider keeps a defensible record, while the person accountable under the VET Quality Framework still makes and owns every competency decision.

Where this fits

A marking assistant is one application of our Artificial Intelligence service, built on Azure OpenAI Service and grounded with retrieval-augmented generation, for the realities of Australian vocational education. It is a contained, high-value problem that generative AI genuinely suits, because the work is repetitive at the edges while the judgement stays human in the middle. If your marking queue is stretching as your enrolments grow, the place to start is to map one unit against its assessment criteria and decide exactly where the assessor must stay in the loop.