The model is not the limitation. That is the thing worth getting straight first.
Modern language models write better replies than most support teams manage on a Friday afternoon. Tone, clarity, patience, all fine.
What ChatGPT cannot do is tell your customer anything about your customer.
The three gaps
It does not know who it is talking to. No account, no plan, no history, no open tickets. Every conversation starts from nothing, which means it can discuss your returns policy in general terms and cannot tell someone whether their return was processed.
It cannot do anything. It can explain how to book an engineer. It cannot book one. Nothing it says results in a row changing anywhere in your business, so every conversation ends with the customer still needing to contact you.
It is not grounded in your content. Answering from general knowledge, a model hallucinates somewhere in the 15 to 27% range. Retrieving from a verified knowledge base first drops that under 1.5%. In front of customers, that difference is the whole ballgame.
Why the distinction gets blurred
Because the demo is genuinely impressive, and the gap only shows up on real questions.
Ask a raw model something generic about broadband and it will answer well. Ask it why this month's bill is fourteen pounds higher and it has no path to an answer, so it either hedges or invents. Customers reach the second category within about two exchanges.
What you would end up building anyway
If you started from the API and worked toward something usable, you would build a retrieval layer over your knowledge base, an integration layer into CRM and billing, escalation logic with a confidence threshold, and a tuning process to keep it accurate as your business changes.
At which point you have built a conversational AI agent, and the model is the smallest component in it. That is not an argument against doing it. It is an argument for knowing what the work actually is before you scope it as "we'll just use ChatGPT".
Where it genuinely is good enough
Internally, right now, today.
Drafting replies for a human to check. Summarising a long thread. Turning a rough note into something sendable. That is real productivity and it needs no integration at all.
ONS data has 35% of UK businesses reporting they use AI while 55% of employees say they use it for work, which suggests a lot of this is already happening in your business whether it is sanctioned or not.
The line I would draw is this. Assisting your team, yes. Facing your customers unsupervised with no access to their data, no.
