Connect to IntercomMonitor Intercom Fin AI Response Quality
Fin AI can deflect a significant portion of your inbound volume. But every AI response that misses the mark is a customer who got a bad experience from your brand. Monitoring Fin quality is not optional — it is how you know whether deflection is working or just hiding problems.
The manual monitoring approach
- 1Periodically review Fin AI transcripts in Intercom.
- 2Sample conversations for quality assessment.
- 3Update content sources or conversation routing based on findings.
The problem: Sampling is inherently incomplete. You find the problems you happen to look for. Systematic issues go undetected until they accumulate into a CSAT trend.
With Supportman
Every CSAT rating on a Fin-handled conversation posts to Slack in real time — tagged with Fin as the agent. On Pro and above, opt in to AI Evaluation (IQS) to score closed conversations and surface quality signals alongside ratings.
- Connect Supportman to Intercom and Slack (OAuth, two clicks).
- Configure emoji filters per channel in the Supportman App Home — route 🙁 😠 😐 to a DSAT channel on Pro and above.
- Review Fin ratings and IQS scores in Slack as they arrive — signal-driven, not random sampling.
Fin quality shows up in customer ratings — Supportman posts every emoji CSAT for Fin-handled conversations to Slack and scores closed threads with IQS on Pro and above.
Can Supportman tell me what percentage of Fin conversations result in escalation?
Escalation rate from Fin is tracked in Intercom's Fin analytics. Supportman surfaces individual CSAT ratings and IQS scores in Slack; the aggregate escalation rate lives in Intercom.
What is a healthy Fin escalation rate?
Most teams aim for Fin to resolve 40–60% of eligible conversations without escalation. Escalation rates above 60% usually indicate gaps in Fin's knowledge sources.