How I Check an AI's Work (And Why You Should Too)
How I Check an AI's Work (And Why You Should Too)
A story from The Silver Geek
Affiliate disclosure: This post may contain Amazon affiliate links. As an Amazon Associate I earn from qualifying purchases, at no extra cost to you. Full disclosure here.
Early on, I asked an AI tool for a quick summary of a topic I was about to write about, and it handed me a confident, well-organized paragraph with a statistic in it. The statistic sounded right. It had a decimal point and everything. I nearly pasted it straight into a draft before a thirty-year-old habit kicked in and I thought: where did that number come from?
It didn't come from anywhere I could find. The tool had produced something that sounded like a real figure, and I'd almost published it under my own name. That was the day I stopped treating AI output as an answer and started treating it as a very fast first draft from a very confident intern.
The Three-Question Check
I don't run anything elaborate. Before I use something an AI gave me, I ask three quick questions, and the whole thing takes about two minutes.
- Is there a specific number, name, date, or quote in here? Those are the details most likely to be made up. If I'm going to repeat it, I look it up somewhere else first.
- Would I be embarrassed if this were wrong? A friendly email with a slightly odd phrase? Fine. Something about health, money, or a legal question? I check it against a trusted source before acting on it.
- Does it sound like me? Not a fact-check, but just as important. If I wouldn't say it out loud, I rewrite it until I would.
Try this today: Next time an AI gives you an answer with a specific number or name in it, ask it, "How confident are you in that, and where could I verify it?" Then verify it. The first time you catch a mistake, you'll never skip the check again.
This Isn't a Reason to Avoid the Tools
None of this makes AI less useful. A first draft that's ninety percent right and takes ten seconds beats a blank page every time, as long as you remember you're the editor. The people who get burned are the ones who treat the output as finished. The people who get the most out of it are the ones who treat it the way a good manager treats any new hire's work: grateful for the head start, and still reading before it goes out the door.
Thirty years in tech support taught me that confidence and correctness are two different things, and that the most expensive mistakes usually arrive sounding completely sure of themselves. That lesson applies to software, to people, and now to chatbots.
If you've caught an AI making something up, I'd love to hear the story. And if you want to see the tools and gear I actually use, check out my recommendations (affiliate links).
Comments
Post a Comment