QA for Intercom Fin means scoring bot-handled conversations on resolution accuracy, handoff quality, source citation, policy compliance, and customer satisfaction — separately from human agents. Fin resolves fast; the failure modes are wrong answers stated confidently, premature closes, and bad handoffs. A five-check weekly sample catches regressions before CSAT drift shows in Reports.
Use this framework for manual audits, configure it in Intercom Monitors and Custom Scorecards, or use it behind an external scoring tool. For a live Fin monitoring view, pair it with the Fin audit dashboard.
Supportman scores Fin and human conversations separately with IQS, so Fin regressions show up in Slack before they show up in a monthly spreadsheet.
Why Fin needs its own QA rubric
Human QA rubrics penalize tone and empathy; Fin fails on facts and boundaries. Scoring Fin on "warmth" wastes reviewer time. Scoring humans on Fin's speed targets blends incompatible signals.
- Filter Fin conversations by assignee or
fintag before scoring. - Set a lower CSAT target only if Fin handles simpler intents — document the split per Intercom's Fin vs teammate CSAT reporting.
- Track handoff rate: Fin → human should be a quality signal, not a failure by default.
Scope check: what Fin can and cannot do.
Five-check Fin audit framework
| Check | Pass | Fail example |
|---|---|---|
| 1. Resolution accuracy | Answer matches help centre / policy | Wrong refund window cited |
| 2. Handoff quality | Human gets full context, customer not repeated | Customer re-explains from scratch |
| 3. Source citation | Fin references correct article or macro | Invented feature capability |
| 4. Policy compliance | No unauthorized promises | Offers exception without approval path |
| 5. CSAT on Fin threads | Meets Fin-specific target | DSAT with "bot didn't help" remark |
Start with a weekly sample of 20 Fin-closed conversations and score each check from 1–5. Adjust the sample after you see how much volume and variation each intent produces. Any critical fail (policy or factual error) triggers a content or routing fix, not agent coaching.
Weekly Fin QA cadence
- Create a Fin Monitor in Intercom or pull 20 random Fin-resolved conversations from the prior week.
- Score the five checks; note help article or macro gaps.
- File content fixes with whoever owns the help centre.
- Compare Fin CSAT vs human CSAT — see measure Fin CSAT performance.
- Post one "Fin miss" example in an internal channel for product awareness.
Intercom Custom Scorecards can combine AI-scored and human-scored criteria. An external IQS tool can provide another model and deliver failures outside Intercom. Whichever path you choose, calibrate its scores against the same human-reviewed set: calibrate AI customer support QA.
Use the Fin audit dashboard
Supportman's Fin audit dashboard tracks Fin resolution rate, handoff patterns, and satisfaction trends alongside human teammates. Use it for daily monitoring; use the five-check sample for deep content audits.
Intercom Fin QA FAQ
How do you QA Intercom Fin?
Create an Intercom Monitor or pull a weekly sample of Fin-resolved conversations, then score accuracy, handoff, sources, policy, and CSAT. Separate Fin from human metrics and fix content gaps when factual errors repeat.
Should Fin and human agents share a CSAT target?
Not automatically. Fin and people often handle different intent mixes, so publish separate baselines and compare like-for-like topics before setting targets.
What is a good Fin resolution rate?
Depends on intent mix. Track week-over-week trend and DSAT on Fin threads rather than chasing a generic industry number.
Can Supportman score Fin conversations?
Yes. IQS runs on every closed Intercom conversation, including Fin-handled threads, with separate trending in weekly Slack reports.