The AI Myth Every Editor Needs to Stop Believing

artificial intelligence

Note: New to AI? Don't start here—this post gets technical! Instead, check out my first post on editors and AI or scroll down to see the full series.

I’ve been having a lot of conversations lately with editors who are trying to make sense of AI. The reactions are all over the map. Some folks are genuinely concerned about whether these tools will replace editorial work. Others are experimenting with uploading style guides or client specs to see if AI can handle routine tasks. Still others are avoiding AI entirely, not quite sure what to make of it all. (Remember, however, that if you use Microsoft Editor, you’re using AI.)

Here’s what I keep coming back to in these conversations: Most of us—myself included, until I started digging into this—don’t really understand how these tools work. And that misunderstanding creates both unnecessary fear and misplaced confidence in what AI can do.

So let’s talk about what’s actually happening under the hood when you use these tools. Because once you understand the fundamental architecture, a lot of the confusion clears up. This is a long post, but stick with me—this is important for all editors to know.

A Critical Note Before We Begin

Before we go any further, I need to be crystal clear about something: Never upload client or employer materials to AI tools without explicit written permission. Besides the obvious ethical implications, most editorial contracts and NDAs prohibit sharing materials with third parties. AI companies count as third parties. Also, assume that anything and everything you put into an AI tool can become public.

Okay, let’s get to it.

The Big Myth: AI Tools Can Look Things Up

Here’s what I’ve been hearing from editors lately:

“My client gave me permission to upload their style guide to ChatGPT, so now it can reference those rules.”

“If I ask ChatGPT a question, it’s at least partially searching the internet for the answer, right?”

“I gave Claude my project files, so it knows what’s in them.”

The assumption is that large language models—for example, Claude and ChatGPT—work like a combination of Google (searching for information) and a database (storing and retrieving what you give them).

That’s not how most of these tools work. Let me break down what’s actually happening.

What Most AI Tools Can’t Do

Standard ChatGPT (Free Version)

Can’t search the internet.
Can’t “look up” information in documents you upload.
Only adds uploaded documents to its context window—then generates responses based on patterns.

Claude (in Regular Conversations)

Can’t search the internet (in most modes).
Can’t retrieve from uploaded documents in standard chats.
Processes uploaded documents as context, then generates based on patterns.

Most LLMs

Can’t access external databases.
Can’t search the web for current information.
Can’t “remember” or “file away” documents you upload for later reference.

When you upload your style guide to regular ChatGPT and ask, “What does this say about serial commas?” it’s not opening your file and looking it up. It’s processing your entire document as part of its context, mixing it with everything it learned during training, and generating what it predicts is the right answer.

Sometimes that answer matches your style guide. Sometimes it doesn’t. And here’s the problem: It sounds equally confident either way.

What Some AI Tools Can Do

Now, before you think I’m saying no AI tools can access information—that’s not true either. Some tools have these capabilities:

Internet Search Capabilities

ChatGPT Plus (paid version) can browse the web when that feature is enabled.
Gemini (Google’s AI) has web search built in.
Perplexity is designed specifically for web search.
Some Claude features include web access.

Document Retrieval (RAG Tools)

Custom GPTs can retrieve from uploaded documents.
Claude Projects can search uploaded files.
NotebookLM is built specifically for document retrieval.
These use Retrieval-Augmented Generation (RAG)—they actually search your documents before generating responses.

So yes, some tools can look things up. But even if you're using RAG tools, don't stop reading—because retrieval doesn't solve the problems you think it does.

**“Large Language Models” Are Really Large Number Models (This Matters)**

Here’s the thing that changed everything for me when I finally wrapped my head around it: LLMs don’t understand words or images. At all. They work entirely with numbers.

When you type “cat” into ChatGPT, it doesn’t see the word “cat.” It translates it into a long string of numbers—something like [0.2, 0.8, 0.3, 0.1...]. And that’s just a simplified version. The actual vectors have hundreds or thousands of dimensions.

These numbers work like coordinates on a map. In this massive numerical space, “cat” sits near “dog” [0.3, 0.7, 0.3, 0.2...] and “kitten” [0.2, 0.9, 0.3, 0.1...]. That’s how the model knows these words are related—not through understanding meaning, but through mathematical proximity in this huge web of numbers.

This applies to all LLMs—whether they have retrieval capabilities or not.

The difference is what happens before the generation step:

Tools without retrieval generate responses based on patterns in their training data and whatever’s in the current context.

Tools with retrieval search documents or the web first, retrieve relevant passages, THEN generate responses based on patterns.

Both are still generating based on probability. Both are still pattern-matching. Both are still gambling on what’s most likely to come next, not applying deterministic rules.

Why This Matters for Uploaded Documents

Let’s focus on the scenario I hear about most: uploading a style guide.

With Standard ChatGPT or Claude (No Retrieval)

You upload your client’s 50-page style guide and an article. You ask, “Does this article follow the serial comma rule in the uploaded style guide?”

What’s actually happening:

The tool processes your style guide as part of its context.
It generates a response based on patterns—including your guide, but also every other style convention it’s seen.
It predicts what answer sounds right based on probability.

What’s NOT happening:

The tool is not “looking up” the serial comma rule in your guide.
It’s not distinguishing between “what this specific guide says” and “what style guides generally say.”
It’s not creating a reference it can consistently consult.

The result? Sometimes it gets your style guide right. Sometimes it contradicts it. Sometimes it makes up rules that sound plausible but aren’t in your guide at all.

With RAG Tools (Custom GPTs, Claude Projects, NotebookLM)

You upload the same style guide to a Custom GPT. You ask the same question.

What’s actually happening:

The tool searches your uploaded document for relevant information about serial commas.
It retrieves the specific passage.
It generates a response based on that passage plus patterns.

This is better! It can quote your style guide accurately. It can find the right section.

But here’s the catch: Even after retrieving the correct information, it still generates its reply based on probability.

Why This Matters When You Need Accurate Information

Once you understand that foundation, a lot of things start to make sense about why these tools fail at tasks that require certainty. Let’s look at some more examples.

The Citation Fabrication Scenario

You’re fact-checking an article and you ask ChatGPT to verify a citation. It confidently tells you the source checks out and even provides publication dates and page numbers. Later, you discover those details are completely made up.

Why? Because it generated what a correct citation typically looks like based on patterns. It hallucinated plausible-sounding information because that’s what the pattern predicted should come next. It did exactly what it’s designed to do.

The CMOS Rule Invention Scenario

You ask ChatGPT what CMOS says about a particular usage. It gives you a detailed explanation with a rule number. You check CMOS. The rule doesn’t exist—or says something completely different.

Why? It generated an explanation based on patterns of how CMOS rules sound, not by consulting the actual manual. The pattern said, “This sounds like a CMOS rule,” so it confidently presented fiction as fact.

The Custom Style Guide Contradiction Scenario

You upload your client’s style guide to ChatGPT and ask about a specific usage. It gives you an authoritative answer. You check the guide. The AI was wrong—in fact, it directly contradicted what the guide actually says.

Why? Because even though it processed your style guide in its context, it’s still generating based on a mix of patterns that includes your document but also every other style convention it’s ever encountered.

Here’s the scary part: It sounds equally confident whether it’s right or completely making things up. There’s no uncertainty when it hallucinates. No hedge words when it fabricates. No “I’m not sure” when it invents.

Because it doesn’t know when it’s wrong. It doesn’t even have a concept of “wrong.” It’s just predicting the next most likely token based on patterns.

Even When AI Tools Can Retrieve, They Still Can’t Edit Reliably

Now let’s talk about the tools with retrieval capabilities—Custom GPTs, Claude Projects, NotebookLM, ChatGPT Plus with web search.

These tools can actually search your uploaded documents or the web for relevant information before generating a response. So if you upload a style guide to a Custom GPT and ask, “What does this guide say about serial commas?” it will find and quote the relevant section back to you.

That’s genuinely useful. It solves the “find the right information” problem.

But—and this is critical—retrieval and reliable application are two completely different things.

Let’s say you upload your client’s style guide to a Custom GPT and ask it to copyedit a document according to those guidelines. Here’s what happens:

It retrieves the rules correctly.

Great! It can quote your style guide back to you. It can find the right section about serial commas or em dashes or whatever you’re asking about.

Then it generates the edits based on probability.

And that’s where everything falls apart.

Because even though it retrieved the correct rule, it’s still generating its application of that rule based on patterns. Which means:

It can’t guarantee consistency. It might correctly apply your serial comma rule on page 5, completely miss the same issue on page 15, and apply it inconsistently on page 50. Each page is a fresh generation, not a continuation of previous decisions.

It hallucinates applications. Just like standard ChatGPT hallucinates citations, RAG tools can hallucinate what your style guide says or how to apply it—even after retrieving the correct passage. The generation step is still pattern-based gambling.

It can’t make judgment calls. Your style guide says, “Use serial commas for clarity.” Great. The AI retrieved that rule. Now it has to decide: Does this sentence need the serial comma for clarity? That’s a judgment call requiring understanding of meaning and context. Pattern-matching can’t do that reliably.

It overcorrects based on patterns. Even when it retrieves your specific guidance, it gravitates toward what most writing sounds like based on its training. So, it flattens voice, changes things that don’t need changing, and makes “corrections” that aren’t actually improvements.

It sounds confident when it’s wrong. Whether the tool retrieved accurate information or not, it generates its response with the same authoritative tone. You can’t tell from the output whether it actually found the right answer or made something up that sounds plausible.

Think of it this way: A RAG tool can open your style guide to the right page. That’s helpful. But reading the right page and applying that guidance correctly across an entire document with perfect consistency are entirely different skills. One is retrieval. The other requires deterministic rule-following and editorial judgment. Pattern-based generation can’t provide either.

What This Means for Professional Editors

If you’ve been worried that AI will replace editorial work, I hope this helps ease that concern. The technology doesn’t work that way.

Most AI tools can’t retrieve information—they just generate based on patterns. And the tools that can retrieve information still generate their applications based on probability, not rules.

Both types sound confident when they’re wrong. Both fail at consistency. Both struggle with judgment calls. Both flatten voice toward generic patterns.

Retrieval solves, “Can the AI tool find the right information?”

It doesn’t solve, “Can the AI tool apply that information reliably across X pages while preserving voice, making judgment calls, and maintaining consistency?”

And that second question is what editing actually requires.

Whether you're experimenting with AI tools or avoiding them entirely, understanding how this technology works—and what it can and can’t do—helps you make better decisions for your business and communicate clearly with clients. Join my email list for practical updates on AI and editing.

Further AI Resources for Editors

Are You Charging What You're Worth?

New to editorial freelancing and feeling like you need to learn all the things? Overwhelmed with projects but not making enough money? Forgoing breaks and vacation time to meet deadlines? My free, 9-lesson course gives you actionable ways to find your ideal freelance rates, say goodbye to the hustle, and build a profitable business that energizes you.