Skip to content

Scoring & Rubrics

Aimee.ai uses AI-powered scoring to evaluate learner performance against defined rubric criteria. This page explains how rubrics work and how scores are generated.

Rubric management: Trainers, Workspace Owners, Admins Score review: Reviewers, Workspace Owners, Admins

How scoring works

graph LR
    A[Learner completes session] --> B[Transcript captured]
    B --> C[AI evaluates against rubric]
    C --> D[Score report generated]
    D --> E[Reviewer validates]
  1. When a learner finishes a simulation, the conversation transcript is saved
  2. The AI scoring engine evaluates the transcript against each rubric criterion
  3. A score report is generated with per-criterion scores and overall results
  4. A reviewer can inspect and optionally override the scores

Rubrics

A rubric is a set of scoring criteria attached to a course version. Each criterion defines one aspect of performance to evaluate.

Creating a rubric

  1. Open a course in the Course Editor
  2. Scroll to the Rubric section
  3. Click Add Criterion
  4. Fill in the criterion details:
Field Description
Title Name of the criterion (e.g., "Empathy")
Description What this criterion measures
Weight Relative importance (default: 1.0)
Guidance Instructions for the AI scorer on how to evaluate this criterion

Example rubric

A customer escalation course might use:

Criterion Weight Description
Empathy 1.5 Did the learner acknowledge the customer's feelings?
De-escalation 2.0 Did the learner successfully calm the situation?
Policy Adherence 1.0 Did the learner follow company policy?
Resolution 1.5 Did the learner propose an appropriate resolution?

Weight strategy

Higher weights make a criterion count more toward the overall score. In the example above, "De-escalation" has the highest weight (2.0) because it's the most important skill for an escalation course.

Score reports

A score report contains:

Field Description
Overall Score Weighted average across all criteria (0–100)
Breakdown Individual score per criterion
Feedback AI-generated qualitative analysis
Recommendations Suggested improvements for the learner
Status pending, processing, completed, error

Score calculation

The overall score is calculated as a weighted average:

Overall = Σ (criterion_score × weight) / Σ (max_possible × weight) × 100

Pass/fail

The trainer sets a pass threshold per course (default: 70%). If the overall score meets or exceeds the threshold, the learner passes.

Reviewing and overriding scores

Reviewers can:

  1. View the full score breakdown for any completed session
  2. Override individual criterion scores or the overall score
  3. Add notes explaining the override rationale

Override audit trail

All overrides are recorded. Include a clear justification when overriding an AI score.