02 · AI quality evaluations

Evaluated against your rubric.

Every team means something different by “a great conversation.” Instead of imposing a generic scorecard, Supportman starts from your rubric: the attributes you care about, weighted the way you weight them. The AI then evaluates every single closed conversation against those benchmarks — scored 0–100 with a written justification, overnight.

Example rubric

Aperture support

Brand Voice & Tone20%

Warm and plain-spoken; apologizes when we broke something.

Technical Knowledge30%

Correct answer, cites the docs, flags known bugs.

Problem Resolution30%

Actually solved — a workaround counts, an article dump doesn’t.

Communication20%

Concise replies, no unnecessary back-and-forth.

Weights are yours to set · scored 0–100 overnight

What you get

Your custom CS rubricDefine the attributes and weights that matter to your team — brand voice, resolution, whatever you measure.

Every conversationEach close is scored against the rubric. Nothing is sampled, nothing slips through.

Written justificationsEvery score explains itself, posted in the conversation’s own Slack thread.

Benchmarks, not vibesConsistent scoring across agents, weeks, and Fin — the same standard every time.

Start from what you already believe

Most teams already have a quality bar — it just lives in a manager’s head or a QA doc nobody opens. Writing it down as a rubric (four to six attributes, each with a weight and a sentence on what “good” looks like) is usually an afternoon’s work, and it becomes the standard the AI applies to every conversation from then on.

The example rubric on this page is a real shape teams use: brand voice, technical knowledge, problem resolution, and communication. Yours can differ — the weights and criteria are entirely yours to set.

From scores to action

Rubric scores aren’t a report card that gets filed away. They feed the weekly reports, surface each agent’s weakest attribute for coaching, and hold Fin to the same standard as your humans.

All features