AI has a shorter attention span than you think (and what you can do about it)

You’re having a great time using ChatGPT, Claude, Gemini, or Sterc.ONE, everything’s running like a charm, and then something strange happens. The answers start to get vaguer. The model forgets what you agreed on a few messages ago. Sometimes it even makes up things that never happened. You think: has this thing suddenly gotten dumber?

No, it’s just that the workspace is full.

In this article, I’ll explain what’s happening under the hood, why every AI has this problem, and most importantly: what you can do about it yourself.

TL;DR

If you’re short on time, here’s the gist:

AI works with a context window, a kind of fixed-size workspace
Everything the model needs must fit on that workspace: your questions, its answers, documents, and instructions
We measure that workspace in tokens, chunks of text about three-quarters of a word long
When the workspace fills up, the AI will start forgetting, repeating itself, and making things up
Solution: start a new conversation, strip down documents, break tasks into smaller pieces, and keep prompts short
Uploading files costs just as many tokens as text, sometimes even more
Writing style and behavior belong in your prompt; isolated facts belong in a knowledge base

The rest of this article delves deeper into all these points.

What exactly is a context window?

Imagine that an AI has a work surface where everything it’s currently working on is laid out: your questions, its previous answers, documents you’ve uploaded, and instructions it’s been given. Everything it needs to answer your question has to fit on that desk. That desk is called the context window.

The problem is that the table has a fixed size. It can’t get any bigger during the conversation. Once the table is full, something has to be removed before anything new can be added. And what gets removed is often exactly what you still needed.

And what are tokens?

Everything on that table is measured in tokens. A token is a piece of text, usually a word or part of a word. For example, the Dutch word “keukentafel” can be broken down into three tokens: “keuken,” “ta,” “fel.” Punctuation marks count too.

As a rule of thumb:

one hundred words are roughly one hundred fifty tokens
a single A4 page of text is roughly 500 tokens
a substantial ten-page report can easily reach five thousand tokens
one hundred thousand characters are roughly equivalent to 25,000 tokens

Modern AI models nowadays have context windows that can hold up to a million tokens. That sounds like an endless amount, and it really is a lot. But the table fills up faster than you think, especially if you upload long documents or talk for hours on the same topic. And there’s something else: even if the information physically fits, models remember the beginning and end better than the middle. If you mentioned an important detail somewhere in the middle of a long conversation, there’s a real chance the AI will overlook it.

Why the AI starts acting strangely

When the context window starts to fill up, a few things happen. The AI forgets what you said at the beginning of the conversation. It starts repeating itself. It gives answers that no longer match your question. Sometimes it makes things up that aren’t there, because it’s lost the context.

It’s a bit like a coworker who, after six hours of meetings, is still being bombarded with new information. At some point, the cup just runs over.

Everything counts on that table

This is where many people get it wrong. With every new question you ask, the entire package is put back on the table:

the instructions provided by the platform itself
your own instruction prompt if you’ve built an assistant
all the files you’ve uploaded
the entire conversation history up to that point
your new question

Then the model’s response is added too, for the next round. With each turn, you bring more information than the last. After ten back-and-forth messages, the table could easily be three-quarters full, even before your new question is added.

Files sometimes take up more space than you think

For the model, it makes little difference whether text comes from an upload or whether you paste it into the chat window. A word is a word. But there are practical differences.

A PDF costs more tokens than the same content as plain text. That’s because the text is extracted from the PDF along with all the clutter: headers that repeat on every page, page numbers, and stray characters from poorly recognized formatting. A Markdown file with the same content is often 30 percent lighter.

Images also count toward the token count, and they’re surprisingly heavy. A single screenshot can easily cost a thousand tokens, regardless of how much text it contains. If you can type something out instead of taking a screenshot, that makes a big difference.

What to do when things go wrong

If you notice that the AI is losing track of the conversation, there are a few things that can help:

Start a new conversation. That’s by far the best solution. New table, blank slate, fresh start.
Briefly summarize the conversation so far yourself and paste that summary into the new conversation
Include only the documents you really need for that specific question
Keep conversations separate by topic, so one for your marketing copy and one for your tax question

A good rule of thumb: if, after ten back-and-forths, you feel like you’re going in circles, it’s almost never your fault. That means the table is full, and it’s time to start over.

Work smarter from the start

Better than solving problems is preventing them. A few habits that really make a difference:

Strip down your documents before you upload them. Remove cover pages, tables of contents, and repetitive footers. Often, thirty to forty percent of a file is filled with clutter
split large documents and include only the relevant chapter
convert tables to plain text or CSV, because PDF tables are often parsed as messy text
Use Markdown files instead of PDFs whenever possible
Upload images only if absolutely necessary

Breaking tasks down almost always works

A large task like “create a marketing plan for my company” is, for an AI, similar to assigning that task to an intern without further explanation. You’ll get a generic response that isn’t very useful.

If you break it down into defining the target audience, formulating the core message, choosing channels, and setting up a content calendar, you’ll get much better results for each component. You’ll retain context for each step, allowing for nuance and follow-up questions. With agents, breaking tasks down is even more important, because an agent who gets stuck halfway through a large task will often redo everything from scratch using a different approach.

The rule of thumb we follow ourselves: if you can’t explain a task to a colleague in two sentences, it’s too big for a single prompt.

Short prompts for agents

An agent performs multiple steps in sequence, and with each step, your original instruction comes back onto the table. If your instruction is already long, it takes up space at every step. Shorter is therefore almost always better.

What works well: one clear goal, a few specific rules, and an example of the desired result. Three paragraphs are usually enough. If you find yourself writing an entire book, that’s a sign your task is too broad.

Instructions and knowledge: keep them separate

When you build your own GPT or assistant, you’re usually given an 8,000-character field for your instruction. That may not seem like much, but it’s enough if you organize it wisely.

The trick is to distinguish between two things:

Behavior belongs in your prompt: role, tone, approach, writing guidelines, dos and don’ts
knowledge belongs in a separate file: procedures, product information, rates, examples

A prompt that’s also packed with facts becomes cluttered, and the model picks up less from it. A separate knowledge base is only consulted when necessary. That saves space on the table.

Writing style belongs in the prompt

People sometimes ask me whether writing rules such as “don’t use em-dashes” and “write informally” belong in the prompt or in a knowledge file. Answer: in the prompt. Writing rules must be applied to every output, regardless of the question. In a knowledge base, the model must first decide to consult that file before applying the rules, and that doesn’t always happen.

The same applies to tone of voice. A brief tone description of five to ten lines: always in the prompt. Only if you have a complete style guide with ten pages of examples should you put the explanation in an appendix and keep the hard rules in the prompt.
A practical tip: negative instructions like “don’t use em-dashes” don’t work as well as you might think. Always provide an alternative: “use a period or comma instead of an em-dash.” And repeat the most important rules one more time at the end of your prompt. The model gives greater weight to instructions at the end when generating content.

The three layers of a prompt

Finally, here’s something that’s often confused. There are three layers of instructions, and all three serve a different purpose.

The system prompt is what the platform itself provides to the model: safety rules, behavioral guidelines, and the basic identity. You can’t see these, and you can’t access them.

The instruction prompt is what you fill in when you build an assistant. Who is it, how does it work, what tone does it use?

The user prompt is the question you type into the chat window each time.

The model weights these layers differently. The system prompt almost always takes precedence. Next comes your instruction prompt, and only then the individual user question. This is good to know, because it means that what you type as a user carries less weight than what you’ve set up as the builder. Do you want your assistant to do something consistently? Put it in the instruction prompt, not in a separate chat message.

How Sterc.ONE handles this differently

So far, this article has been about AI in general. The principles apply to every tool: ChatGPT, Claude, Gemini, you name it. But there’s one topic where the platform you choose does matter, and that’s the context window itself.

Our own AI platform, Sterc.ONE, works differently from the well-known chatbots. While ChatGPT locks you into OpenAI’s models and Claude into Anthropic’s, with Sterc.ONE you’re not tied to a single provider. You can choose from the top models from OpenAI, Anthropic, Google, and Mistral, and decide which one is best suited for each task.

This makes a big difference when it comes to the context window:

you choose the model with the capacity you need for each task. For a quick question, you’ll use a lightweight model; for a large document, a model with plenty of context
you’re not hampered by extra limits that consumer apps sometimes impose on top of the model to control costs or speed
you aren’t dependent on a single vendor. If a new model with more capacity or better performance comes along, you can switch over without having to completely overhaul your workflow
You can combine different models into a single workflow, which is useful when a task consists of multiple steps, each of which is best handled by a different type of model

For us, this aligns with how we view AI. Tools evolve rapidly, and what’s the best model today might not be next month. By not locking yourself into a single platform, you stay in step with where the technology is headed.

In summary

AI isn’t magic. It’s a workbench with limited space, where everything you put on it takes up room. Once you understand that, a lot of how AI tools behave suddenly makes sense. And more importantly: you know what you can do to get better results from them.

In a nutshell:

keep your conversations short and focused
start over if it gets stuck
put knowledge into files, behavior into prompts
Break large tasks down into small steps
Be selective about what you upload

With these habits, you’ll get much more out of the same tool, without having to change anything about the technology. Just use the available space more wisely.