Skip to content
Log in
— Episode 17 · 38 min

Leading AI, not following it

Jen McCorkle has spent nearly 30 years in data and analytics. She untangles the AI acronyms for support leaders, explains why sentiment analysis misreads the word "cancel", and lays out the governance checklist she would run before scaling any AI tool into a support org.

August 5, 2025 · Jen Weaver with Jen McCorkle, Data-driven leadership expert

— Takeaways —

What you’ll learn from this episode

  • If data isn't helping you make a decision it's noise, and AI is the same. Ask which kind of AI someone means — generative, agentic, or machine learning — because they solve different problems.
  • Treat AI tools like someone on day one of the job. She fed charts to ChatGPT for a summary and every single data point came back wrong.
  • Sentiment analysis runs on transcripts, so it misses tone entirely. At one telecom, "cancel" scored negative even when cancelling a technician visit meant the agent had fixed the problem on the call.
  • Don't let a bot fake empathy. Customers can tell, and it costs you the credibility you want the human to have.
  • Always give people a way out to a human. Older customers in particular will not use a chatbot, and metrics thinking hides what the customer wants.
  • Repurpose the time AI frees up into checking its outputs. Human QA of decision logs is the guardrail.
  • Before scaling — ask vendors how the model was trained, set guardrails against your current policies, build AI literacy on the team, and pilot on one segment first.
  • Prompt engineering is going a level deeper than your first ask, including prompting it to challenge your own blind spots.
— Chapters —
  1. 0:00Introduction
  2. 2:15What data-driven leadership means
  3. 4:31The alphabet soup — GPT, agentic, LLM
  4. 6:47Next best action, and what agents juggle
  5. 9:03Three-letter acronyms, and NLP's two meanings
  6. 11:20What LLMs are trained on
  7. 13:36Hallucinations, and checking every number
  8. 15:51Low-hanging fruit — chatbots and knowledge bases
  9. 18:06Where sentiment analysis goes wrong
  10. 20:21Garbage in, garbage out
  11. 22:36Don't let a bot fake empathy
  12. 24:51Let people opt out to a human
  13. 27:06Building AI literacy and governance skills
  14. 29:23Prompt engineering, and the shopping example
  15. 31:39What AI will and won't replace
  16. 33:54Three steps to set up AI governance
  17. 36:09Pilot before you scale
— Transcript —

Transcribed from the recording and edited for readability — false starts removed, product and proper names corrected.

Jen McCorkle Don't let it fake its empathy. If the chatbot says, "I am so sorry you've gone through that, I can understand" — I'm thinking, no, you don't. You've just been programmed to say that. I'm looking for that empathy in the human. I don't necessarily want the empathy in an AI.

Jen Weaver Welcome to Live Chat with Jen Weaver, the podcast where top support professionals unpack the tools and tactics behind exceptional customer experiences.

In this episode, finally, we're demystifying AI for support leaders. My guest Jen McCorkle takes us beyond the buzzwords and into practical, no-fluff strategies to harness AI in your support team. She is an amazing support leader who has tons of experience with data and AI.

We talk about how to start smart with chatbots, how to keep AI tools honest, and how to build the skills your agents need in this AI world that we're in now. If you've felt lost in the AI alphabet soup, or wondered how to integrate AI without losing the human touch — whether that's in your contact center, in your support queue, or in your own workflow — this episode is for you.

Before we get started though, our QA tool Supportman is what makes this podcast possible. Supportman sends real-time QA from Intercom to Slack with daily threads, weekly charts, and done-for-you AI-powered conversation evaluations. It makes it so much easier to QA Intercom conversations right where your team is already spending their day, in Slack.

All right, on to today's episode. As we all know, AI is in the headlines all the time right now. We haven't done a ton of episodes on AI, but today I'm here with Jen McCorkle to share some baseline information for customer support leaders who are awash in this new world.

Would you go ahead and let me know a little bit more about what you do, and how you came to this point in your career?

Jen McCorkle Absolutely. Thanks, Jen, it's a pleasure to be on the podcast with you. I'm really excited to share some information about AI and what our customer support teams can do to really leverage it.

I am a data-driven leadership expert, and I help organizations and people get comfortable with AI and data to make better decisions.

Leadership is not just about the gut and the human and the emotional element — it's a lot of using information to tell a story and make decisions. And that information can be both quantitative, like with data and numbers, or qualitative, which is more subjective, the human element side of leadership. So I really work with leaders across all industries and all functions inside of a corporation to get ahead.

Jen Weaver Can you give us an intro to why AI matters amidst all the noise about it? What does this mean for customer support leaders?

Jen McCorkle It is a lot of noise. And what we say with data is, if data isn't helping you make a decision, it is noise. And AI is the same thing — if AI isn't helping you, it's noise.

AI is a toolbox. It's not a brain. It's really a tool that we use. Especially with all this noise in our environment, it's really hard to get that picture of what can it do for me?

A lot of people, they say AI and they're really talking about generative AI, or ChatGPT kinds of things that generate predictive text. GPT is generative predictive text. So you remember a long time ago when you first started texting on your phone and it would start to pop up the next word — that's generative text.

Jen Weaver But that wasn't based on AI necessarily?

Jen McCorkle Absolutely it was.

Jen Weaver Oh, okay, I had no idea.

So when you say AI, what's the simplest way to explain GPT, LLMs, all the alphabet soup, for support teams that aren't super technical?

Jen McCorkle That's a great question, because AI is artificial intelligence and it's in many, many flavors.

So we talk about GPT, which is generative predictive text. It's basically a large language model that's been trained on infinite amounts of data to help this AI algorithm understand how to think like a human. That's what AI does — how do you think like a human?

Some of the flavors — we hear agentic AI a lot, it's a really big buzzword right now. Agentic AI are tools that act. So this is trigger workflows, auto-close tickets, automated ticket routing based on intent. Those are agentic things, they act. Whereas generative AI, or GPT, generates content.

And what we've seen is we've moved from "predict the next word in my text on my phone" to being able to write articles and books and generate art and video and audio. It's moved so quickly — every six months we're making a major jump in what AI is capable of doing.

Jen Weaver Some of these things are in the future maybe, with the voice. But what we're seeing now with AI in customer support is definitely chatbots. I've heard more buzz about how AI can help us internally with our operations on support teams. What are some other use cases you've seen that work really well?

Jen McCorkle Obviously there's the agentic routing of tickets. And generative AI being able to write scripts or write FAQs, or generate those kinds of things that would have taken hours and hours of a human's time to go through and look at all of these calls and figure out what are the most common kinds of questions or concerns we're getting, and then what was the best response, and what was the thing that people understood the best.

That's how AI can really speed that up — taking massive amounts of data and then being able to say, here's the summary of what we found.

And next best action is another thing for agents. As an agent is on a call, being able to pop up right on the screen in the tool: this is your next best action to take.

For example, I've worked with call centers in telecom, and one of the next best actions we had been working on is — as you're talking with Jen Weaver and she's talking about her telecom needs, what's the next best promotion that would be perfect for you, to help you save money, drive higher customer satisfaction, get you embedded and adopting all of the different products and services that we offer. That's an AI thing, being able to pop that up for an agent so it's right there.

Jen Weaver As somebody who's been on chat and phone as the customer support agent — and as the customer — it's very hard to think through that kind of data. What tier is the customer at? What are their needs? What is all this data while I'm talking to them? I don't have that many streams in my brain. So just being able to have something compute that for me does make a lot of sense. But then the AI is not speaking to the customer, it's presenting it to me as the human and making me better.

Jen McCorkle Absolutely. And you know, Jen, I've been in data and analytics roles since 1996. I'm coming up on 30 years of a career in analytics and data.

We were doing artificial intelligence and machine learning back in the 90s in colleges. This has been around for decades.

And one of the things that I would stress for anybody watching who might be on the analytics team, the BI team, any of the data science teams: go sit and watch your agents. Do your ride-alongs with your agents and watch them on the phone.

What has surprised me the most, especially in very large call centers, is you've got agents with five, six things going on on two different screens, and they're writing this up while they're checking an address and doing these different things. It is amazing the kind of multitasking these agents are doing.

Jen Weaver And as a customer I'm not seeing that, so I'm just thinking there's such a delay. Why is it taking them so long?

Jen McCorkle Exactly. And as an analytics person, we don't always understand why. We put this algorithm together, we're popping this next best action, we're giving you what you want — why aren't you using it? And when you sit down and watch, you're like, oh yeah, this needs to be much simpler than I made it.

Jen Weaver I think a lot of call center employees definitely feel like the tools and support that they receive are not necessarily tailored as they would be if someone had watched them actually work.

Jen McCorkle Exactly.

So let's talk a little bit about TLAs, the three-letter acronyms. I started my career in IBM and everything was a TLA. It's important, as somebody in a support role or maybe somebody that's not technical, to be able to identify what kind of AI somebody is talking about.

So we talked about agentic and we talked about GPT, generative predictive text — the most popular ones right now are ChatGPT and Claude.

Jen Weaver What else are we looking at?

Jen McCorkle NLP. There are actually two different kinds of NLP. There's natural language processing, and then there's neural linguistic programming.

Jen Weaver Two different things, same acronym.

Jen McCorkle Different things. I remember I was working with a company that scales educational content and materials, and working with the marketing team I said something about NLP, and this person said, you know, I actually have a master's degree in NLP. I'm like, you do? She was the email content specialist. In her job, being able to have neural linguistic programming was very important. And then we realized we had two NLPs that were very different.

But natural language processing is what we use to help AI understand things and speak like a human. So this is understanding "my order didn't arrive" and AI understanding what that meant. Or having a voice ID recognizing "I want to cancel my subscription," and being able to recognize that needs to be routed here. That's how we use NLP to make GPT think and understand like a human.

LLMs are large language models. This is massive amounts of content that GPTs are trained on. What's interesting is, with ChatGPT, when OpenAI were training the model, they were adding things like books and websites and lots of different content in there to train it — but it was also trained on Wikipedia and Reddit. Because Wikipedia is the largest source of human-generated content, and it's important that it learns how to talk like a human, so that's why they use Reddit and Wikipedia, to understand how the everyday human would write and type and talk.

Jen Weaver With not perfect grammar, with inconsistent capitalization, and just the syntax. It's probably impossible to separate the syntax, the grammar, the way that people write, from the content. So you're feeding it both the content and the style, and then it's using that information.

Is that related to what we talk about as hallucinations? It's a term I've heard.

Jen McCorkle Hallucinations and bias are definitely there. And think about it as a human — we hallucinate things sometimes by having a mismemory. I didn't remember it that way, and I think I'm correct until I find out I'm not correct.

Jen Weaver Or if I'm confident that I know a fact until that's challenged, I still am confident about it, even if it's incorrect.

Jen McCorkle Right. So that's what we talk about with hallucinations — sometimes generative AI generates things that don't exist.

For example, I did some data visualizations for a company that I'm consulting with, and I fed it into ChatGPT to see how well it could summarize the visualizations that I created and generate content, like an article or a summarization for an executive. And as I was checking the data, every single data point was wrong.

Every single data point that it saw on the chart was wrong. It would do things like, 34% — it put 39. So I don't know if it's just the way that it processed the image, or if it was trying to subtract things. I have no idea. So you have to be very careful.

Jen Weaver What did you do with that? Did you go in and manually edit those?

Jen McCorkle I highlighted everything and showed them to the client on "this is why we have job security." This is my job security. You can't just feed it in.

And that's what I was finding — people will feed it in and they're like, this is great, it's a great summary. And they never checked it.

So think of your AI tools, your collaborative tools that support you, as somebody who just started day one on the job. You've just got to double check everything until you're really very confident that it's doing it right.

The other thing we talk about is AI and ML. Machine learning is what ML stands for, and machine learning is how we program AI to think like a human. So it's training models, testing models, having it do the predictive analytics, creating these algorithms.

So we talk about AI and ML, and we talk about AI as ChatGPT, and we talk about AI as agentic AI. When somebody says AI, it's important to ask them what kind of AI are you talking about?

For me, as a data scientist, when I talk about AI and ML I'm talking about creating predictive algorithms that will predict customer churn, or creating algorithms that will identify agents that might be able to close tickets versus not close tickets, or upsell versus not upsell. So I'm thinking about an algorithm I'm developing that's going to predict something that we could then take an action on. It's a prescriptive kind of output.

Jen Weaver What I'm hearing you say repeatedly is that it's about taking actions. If it's helping you make a decision, if it's helping you take an action, that's where it's really important.

Jen McCorkle And that's the intersection of what I do with data-driven leadership. It's not just about the data and the analytics but the AI, because we use AI every day to get information. So how do I take my data and my analytics and everything I'm doing in AI and blend it together so that I am the strongest leader I can be? How can I futureproof my career by using these tools collaboratively?

Jen Weaver Of all the tools that are available, what flavor do you see as the low-hanging fruit that any support organization can adopt pretty quickly?

Jen McCorkle I'd be surprised if people aren't using chatbots. I use it on my website, and I'm a solopreneur, a one-person company — I use it on my website a lot just to automate those quick things that are coming in that you can quickly answer.

It's that low-hanging fruit, that tier one kind of stuff, password resets. Using your chatbots to do those kinds of things has an immediate ROI. When we implemented a chatbot, it took — I don't know — like 70% of our conversations, which for our support team was an immediate relief of hours spent that we're putting out to more valuable stuff.

Knowledge bases and FAQs — using your AI tools to research and find out what kinds of conversations agents are having, and how can we put together the best knowledge base and the best FAQ for our customers, and get that back on the website.

Jen Weaver Speaking of your work with organizations, do you have any particular facepalm moments or big successes that teams you've worked with have run into — maybe because of the way the AI was set up — as a lesson for the rest of us?

Jen McCorkle That's a really great question. Thinking about it in terms of call centers, one of the things that kind of makes you go "hmm" is when we look at sentiment analysis.

When you do a sentiment analysis, what's happening is AI or machine learning algorithms are looking at the transcribed words to see where there are positive, negative, or neutral words. And then based on a scoring algorithm, giving things negative scores or positive scores, and looking for phrases and words, it comes up with a sentiment number. And that sentiment number translates into positive, negative, or neutral.

A lot of agents have seen that — was this a positive sentiment or a negative sentiment? They see the positive, negative, and neutral, but they don't always understand how that was generated, how that was actually derived in the algorithm.

As we were working on these things, we would report out as an analytics team that an agent had particularly low sentiment. And then supervisors would go listen to the calls and find out — this was fine. What's going on?

So as we started looking through it, we found two things that were really interesting.

Sentiment analysis currently is trained on transcribed information. So it's not getting tone and inflection. If you're sarcastic and you made a sarcastic comment, it might come off as very negative in the call, but it was actually a thing that you were using to build a rapport relationship quickly with your customer. So it doesn't do tone or inflection.

But what was really interesting is the word "cancel." Working with the same telecom company, "cancel" comes in as a negative.

Jen Weaver Cancel my subscription, I don't like you any more.

Jen McCorkle Exactly. So if you're working with sales and retention analytics and somebody says cancel, that's a negative thing.

But for troubleshooting and repair, cancel is a good thing. Somebody had an appointment for a tech to come out and visit their home, and the agent was able to resolve their problem on the call. "Would you like me to cancel your appointment, since I fixed your issue?" Cancel was coming up as negative — when that was a positive customer outcome.

Jen Weaver So these tools aren't strong enough to bifurcate the algorithm and say, if it's this kind of call think of sentiment this way, and if it's this kind of call think of it that way.

Jen McCorkle A lot of these tools are what we call black boxes. If you hear somebody say "black box" in relationship to AI, it means something's happening behind the scenes that you don't know.

Jen Weaver And nobody knows. Even the developers of that tool put stuff in and get stuff out, and in between no one knows what's going on.

Jen McCorkle That's exactly right.

Jen Weaver We were talking about inputs and outputs. Have you seen support teams input bad data, train it on bad content? What does that look like, and how do we avoid it?

Jen McCorkle When I started my career at IBM, I was working with outsourcing clients, and one of the things we would outsource would be help desks and technical support for their customers. Very large companies — IBM would take over their desk support and help desk.

We worked with those agents back in those days, and we would add these forms so that they could ask the customer these questions. And what we found is a very common agent behavior: they pick the first thing in the drop-down, because they've just got to go fast. They're pushed to get on the next call.

And it's funny, it still happens today. In 30 years that agent behavior hasn't changed.

Jen Weaver When you're measured on the number of calls you do per hour, it disincentivizes you from creating good data.

Jen McCorkle That's correct. And so that's where it's really important as a data team to understand all of those pressures that are coming in, and how it can skew the data, and being able to set it up correctly.

The company that I was working with previously was getting ready to start doing AI-generated text and notes from the call, so that the agent didn't have to do post-call work. It's a more accurate way of getting the right information, not having to rely on the agent, and putting the agent where they really are the most valuable, which is connecting with their customers.

So yeah, it's a garbage in, garbage out thing. If you have garbage coming into your machine learning algorithm, you'll have garbage coming out.

Jen Weaver What's a guardrail that you would recommend a support team use to try to prevent an AI agent, or even an internal ops tool, from hallucinating the value away?

Jen McCorkle Especially for support teams, as AI starts to replace those tier one, low-hanging, repeatable tasks — in that kind of case where we see these things being automated, repurpose that time into having them check and validate the outputs from the algorithms.

So if customer says this and chatbot recommends this action, get those logs and make sure that they're being checked by a human, to ensure that the decision the AI is making is the right decision.

Jen Weaver So this brings up a whole new career for AI babysitters, basically. I don't think we have all those roles and titles defined, I think it's emerging. But we could talk about how does a customer support leader futureproof their career, so that they're building these AI skills that they need to have these roles.

Jen McCorkle And this applies to everybody. Those of us who are not using AI will be replaced by those who are.

And I don't think that AI is going to replace people. I think it is going to reshape our roles. They're still becoming part of our culture, we're redefining what does this all mean. And again, I'm speaking from what I know today — this could all be different tomorrow.

Those simple low-level tasks can be automated. But that's where we can then have our agents going through and checking the outputs, checking the outcomes.

Customers can tell when bots are faking empathy. So maybe we should build bots to be bots, and not to try to pretend. More and more customers are aware of when they're talking to a bot. But also customer and public attitudes about AI are shifting rapidly. Some people are beginning to trust it more, some people are beginning to trust it less.

Jen Weaver So we have these varied attitudes that customers are coming at. How do we prevent ourselves from over-trusting the AI to just do whatever it's going to do? How do we calibrate those chatbots for a changing customer perspective?

Jen McCorkle I think you hit the nail on the head. The first thing is, don't let it fake its empathy. If the chatbot says, "I am so sorry you've gone through that, I can understand" — I'm thinking, no, you don't. You've just been programmed to say that.

Jen Weaver And that's annoying and unnecessary.

Jen McCorkle It would be like if I was talking to an agent that said, in a flat voice, "I am so sorry this happened to you, let me fix your problem." There's no empathy there in that voice.

I'm looking for that empathy in the human. I don't necessarily want the empathy in an AI. You're not trying to be a person, you're trying to be an AI. Just be an AI.

So understanding how it's trained, again, that's the biggest thing — understanding how something is trained, and knowing where it stops. It stops being useful when it just tells me I've got a great idea. When I ask it to find my blind spots, it will. So you have to make sure that you are prompting it to do that.

Thinking about setting up a chatbot from a customer support perspective — making sure that you're being transparent that this is a chatbot, here's what I can do, here's what I can't do. But giving people the opportunity to opt out of that chatbot experience and talk to your human.

Our aging communities hate AI and computers. They want to pick up the phone, dial a number, and talk to a person.

Jen Weaver I can't imagine wanting to call a person, as a millennial. But my parents want to.

Jen McCorkle My grandmother — she's 90 and she doesn't get on the phone any more. But as of a few years ago, she would get very, very angry when she would have to go through the IVR, and she would not use a chatbot to save her life.

So giving people the opportunity to opt out of your chatbot to get to a customer service agent is really important, and understanding what does our customer want.

Sometimes we try to automate because we think about how it's going to save us time, and how it's going to reduce our time spent on call, and our first call response, and all that kind of stuff. We're thinking about it from the metrics perspective and the KPI perspective, but we're not thinking about what does our customer want. And that's really the most important thing — as far as when we talk about CX, we forget what the customer wants.

Jen Weaver Especially in the world of tech, we're very accustomed to working with not just AI but various computers and tools and IVRs. Whereas you lose touch a little bit with your average customer, who maybe doesn't sit at a computer all day long, and maybe has very little trust or proficiency — not even older folks, but just anyone. I tend to think everybody works with computers. But coming back to reality, not everyone does.

Jen McCorkle Not everybody does, that's very true. And that's where governance is what's really important — that you as a customer support team have governance put in place.

Jen Weaver Tell me more about that. Do you mean like, in the future my title might be customer AI governance technician or something?

Jen McCorkle Specialist. Somebody that is part of that QA process. How do you do QA and QC for your chatbots and your AI responses? Having that human in the loop to make sure that they're catching those errors.

As somebody that's programmed AI, I love when somebody tells me that something's doing something wrong. Because I've set it up the best way I can, and the more information I get from the humans about where it's failing, the better I can be as a developer to have it catch them.

Jen Weaver If I'm a support leader and maybe I want to move into AI governance, are there courses you recommend, or ways to become more AI literate and move in that direction?

Jen McCorkle It seems like there's a course for everything out there. Honestly, you don't need them all.

I think really understanding the tools that you're using in your support groups, and understanding how they're being implemented and scaled in your organization. And then maybe it's less about taking the class and learning how to ask. So asking for things like, I'd like to work with you on tuning your AI models. I want to be a part of the QA process. That's the hands-on kind of stuff.

Jen Weaver That's really helpful, because I've also heard this advice and I think it's good advice unrelated to AI: become an expert. If your team uses Zendesk, become a Zendesk expert, get whatever certification there is. If your team uses Intercom, really dig into Fin and how that works.

It's a variation on that old saying my mom used to say — do what you can, where you are, with what you have. And then you can learn a lot from learning a specific tool.

Jen McCorkle Totally. And tools are changing every day, so get used to using them. Get a ChatGPT account or a Claude account and ask it: here's what my job is, and I'm worried about what's going to happen to my job. Give me five ways I can futureproof my role as a tier one support person, or a tier three, or a supervisor, or a leader. Help me identify five things that I can do today to start taking action.

Jen Weaver I love that. And you used this term that was brand new to me not long ago — prompt engineering. Can you tell me a little bit more about what that is?

Jen McCorkle Prompt engineering is how do you ask AI to do something without introducing your own bias. It's a technique you can learn, where I'm giving it prompts that challenge my own biases.

Do you remember we talked about garbage in, garbage out a little while ago — about how if you feed your algorithm bad data you get bad algorithms out? This is what prompt engineering is. If you give it a very simple prompt — I don't want to call it a garbage prompt, but a prompt that isn't really what you want — it's going to give you what you asked for. So prompt engineering is going a level deeper.

I'm going to be very transparent and kind of vulnerable right now. I am horrible with shopping for clothes. I don't like shopping. I walk into the store and I look around and I'm like, okay, there's all of these things and I don't like it, can't do it.

So if I asked ChatGPT to recommend some outfits that would look nice on screen, that are in a certain price range, and this is my color palette — I like blues and greens and blacks — it's going to give you a whole list. But is that really what you want?

So when I did my prompt, and I literally actually did this, I told it: I hate shopping. I don't know what anything is called. I don't accessorize. I prioritize comfort over fashion or fit. I want things that look nice, that stand up well with being washed, that will last.

Now I'm up to like eight things I'm telling it. Here's what I want it to project about me, and having confidence in me. I want you to be truthful, and I need you to tell me where everything's at — what is it called?

Now, it's interesting, because it gave me this stuff and I'm like, this is fantastic, I love it. Every link I clicked on was wrong.

Jen Weaver I've had that experience too.

Jen McCorkle But I asked it, describe the piece of clothing that you're recommending so that I can go do my searches. And it helped me so much.

Jen Weaver You got specific, and it was useful to translate and give you things to search — but it doesn't actually link. I did this with shoes not long ago and I was like, none of these are even — they're all out of stock. But it gives you information for you to use. That's really interesting.

Back to the big question that's on my mind, and hopefully people are thinking about this — will AI replace us as support people?

Jen McCorkle I think it's going to replace some of the work we do.

AI is replacing 70 to 80% of common FAQ tickets — they're handling it in a chatbot world, or other ways that they can log in and do things on their own. Routine order status checking, you don't need to call an agent to find out where your thing is. Your tracking logs, all that kind of stuff, you can log in and do that. Chatbots are perfect for that. And then password resets, anything that's really simple that can be automated.

But where the human element is — if a device fails, if something is not working, AI can't navigate or build the trust when something stopped working or something upset a customer. And I mean, this never happens in the CS world, that somebody calls in and they're angry because they've called five times. AI is not going to be able to handle that customer that's had it, that called five times and isn't getting their resolution. It'll probably just tick that person off a little bit more.

Tier three level support, the stuff that you really need a person to troubleshoot. I was talking with a developer, and somebody asked GPT to develop code, and the code was beautiful. It was perfect, it was laid out correctly. When they actually implemented it, it didn't work. None of the code worked.

So that's where we see a lot of these things where it looks like it can do the work, but you really need the human to be able to troubleshoot it, implement it, fix it.

You also have things where — which probably never has happened to you — somebody says something but that's not what they meant. As an agent, you can tease those things out. Is this really what you meant? Is this really what you were looking for?

Jen Weaver Like if there's a typo. All along a customer's been talking about one thing, and then they have one reply that's like, "I do not want a refund," but the "not" was an accidental typo. They do want a refund. I can gain from context from the rest of the conversation that they do, and check in with that.

Jen McCorkle Complex emotional problems, people are still going to need to do that. And then agents needing to supervise AI and handling those trust moments — being part of that loop, where humans are moving up the value chain and letting AI do the stuff down here. We can provide better value in handling these complicated, highly charged, highly emotional interactions with our customers.

Jen Weaver If you were advising a support leader setting up AI governance today, right now, what are the first three steps on your checklist?

Jen McCorkle If I'm getting ready to scale or adopt AI inside of my organization, the first thing I want to do is ask for transparency from the vendors. The vendors that are creating these tools, or your internal IT and development teams that are creating these tools — ask for the transparency. How is this being trained? What data is it being trained on?

Sometimes we don't know what we don't know, and there might be a better data set to train it on. So ask those kinds of questions, and present it as, I want to make this the best tool we could possibly use, to automate things and make our customers happy and keep our agents happy.

Asking for ways to audit the decision logs, and asking vendors for ways to do the explainability — how can I explain how this is working to other people in my organization?

Then guardrails. Setting up those guardrails for CX, for company policies. Sometimes it gets it wrong, especially if you've had two or three different policy changes — sometimes the AI will go get the last one. So making sure that things like warranty, refunds, and things like that are the most up to date.

Building AI literacy for your agents. Helping your agents understand how to use AI on the front lines, how to collaborate with my AI tools, when should I use AI-drafted content, when should I use a generative tool. Teaching them how LLMs work — what is a large language model, how does it work — so they understand the risks and they don't blindly trust the outputs.

And then piloting first, and then scaling. A lot of times we make the mistake, because this tool comes in and the vendor says it's going to fix every problem for us, and we implement it at scale. Start with a pilot. Start with certain sites or certain kinds of calls. If you're a multi-site organization, or if you've got multiple kinds of different calls coming in, start with one of those segments of call center agents to ensure that it's working correctly.

Pilots are perfect. And involving your agents in the outputs and the tuning and the QA process, and helping them get involved as well.

Jen Weaver It sounds like that's a great opportunity to use that percentage of agent time that's not in the queue — because it's good not to overload specialists with too much queue time. So they can do some AI training and AI governance a little on the side.

Is there anything else? I feel like this could be two episodes.

Jen McCorkle I think the last thing I would say to your CS and CX leaders that might be listening to this is: AI is already in your support stack. You just need to learn to lead with it.

Jen Weaver I love that. Learn to lead with it. Fantastic. Well, thank you for being here, this was so much fun.

Jen McCorkle Thank you for having me, this was really fun.

Jen Weaver Huge thanks to Jen for being here and educating us on AI. I hope you're leaving with clear, actionable steps to bring AI into your support operations — or to better manage the AI tools that are already being implemented — without sacrificing empathy or accuracy.

If you enjoyed this conversation, please do subscribe to catch the next episode wherever you're listening or watching. And if you know another support leader facing AI overwhelm, please do pass this along. Thanks for listening, and we'll see you next time.

Under two minutes to live, no IT ticket required.

See pricing