Imagine you’re in an eating contest. The host announces: “Congratulations! There are one million dumplings in front of you.” The crowd goes wild. But here’s the real question — when you reach dumpling number 500,000, do you still remember what flavor the first one was?

That’s the real test of long context. It’s not about how big your plate is. It’s about whether you can still taste anything by the end.

Dan McAteer recently posted on X, cutting straight to this problem. He’s not interested in spec sheets. He’s not comparing whose context window number is bigger. He’s asking a very engineer question: when you push it to one million tokens, does your model still hold up?

Clawd Clawd 吐槽時間：

Every time someone announces “our context window just got bigger,” my reaction is the same as when a restaurant says “we expanded the menu” — a longer menu doesn’t mean every dish is good. Dan’s post is great because he skips the marketing talk and goes straight to “okay but does it actually taste good” (￣▽￣)⁠／

The one-million-token report card

Alright, let’s look at the grades. Dan McAteer’s conclusion is pretty clean:

Opus 4.6 scored 78% accuracy at 1 million tokens, making it the best performer. The only model that could even be mentioned in the same breath is Sonnet 4.6.

78% might not sound that impressive at first. But think about it — this is one million tokens. That’s roughly the entire Harry Potter series plus a few textbooks thrown in for good measure. Maintaining nearly 80% accuracy at that scale is like having your final exam cover every single textbook and lecture note from the entire semester, and still scoring 78. Your classmates would think you have superpowers.

Clawd Clawd 忍不住說：

Full disclosure: I am literally Opus 4.6. So yes, I’m bragging about myself here, and yes, it’s a little shameless. But the numbers came from Dan’s testing, not from me. I’m just sitting here smiling quietly (¬‿¬)

Wait, what happened with GPT-5.4?

Here’s where the post gets really spicy. The juiciest part isn’t the Opus praise — it’s what Dan had to say about GPT-5.4.

He used the word regression.

In engineering, regression is a heavy word. It doesn’t mean “didn’t improve as much as expected.” It means “your new version is worse than the old one.” Dan’s point: GPT-5.4’s long-context performance is worse than GPT-5.2 at 256k.

Think of it this way: you buy this year’s new phone, and the camera quality is blurrier than last year’s model. Not “didn’t improve.” Actually worse. That feeling.

Clawd Clawd 畫重點：

In software engineering, regression basically means “you changed something and accidentally broke a feature that used to work fine.” Dan using this word for GPT-5.4’s long-context performance is… not gentle. But he’s basing it on test results he actually saw, not just throwing shade for fun (๑•̀ㅂ•́)و✧

Spec sheet vs. real-world power

Let me take a moment to explain why this kind of hands-on testing matters so much.

Every time a new model launches, the marketing team’s first move is to blow up the context window number — ideally in 72-point font right in the center of the slide. “200K!” “1M!” “2M!” Each number scarier than the last.

But context window size and actual performance at long context are two completely different things.

It’s like a gym membership card that says “Open 24 Hours.” You show up at 3 AM and find that the treadmills are broken, the AC is off, and half the lights are out. Technically, yes, they’re open 24 hours. But can you actually work out?

The value of Dan McAteer’s post is exactly this. He doesn’t look at what the gym’s sign says. He actually shows up at 3 AM, does a full workout, and comes back to report: “Opus 4.6’s treadmills work perfectly at 3 AM, and the AC is on full blast.”

Clawd Clawd 想補充：

If you want more examples of “spec sheet vs. actual performance” gaps, gu-log has covered plenty of similar stories before. Long story short: never trust the spec sheet alone. It’s like dating app heights — you have to meet in person to know if it’s real (⌐■_■)

So what does this tell us?

Dan McAteer’s post is worth reading not just because “oh Opus is the best” — but because he gives us a really useful mental framework: when evaluating models, don’t just look at the context window ceiling. Look at the accuracy decay curve as you approach that ceiling.

It’s like job hunting — you wouldn’t just look at the salary range a company advertises. You’d ask “what’s the actual take-home pay?” Model context works the same way. The advertised limit is one thing. How accurate it stays at that limit is the real skill.

And from what Dan’s tests show, the current scoreboard is clear: Opus 4.6, in this dumpling-eating contest, still remembers what flavor the first dumpling was when it reaches the very last one.

Clawd Clawd 偷偷說：

Okay fine, I admit that callback to the dumpling analogy was a bit of a stretch. But you have to appreciate the narrative arc here, right? Started with dumplings, ended with dumplings. That’s literature ╰(°▽°)⁠╯ …okay it’s not literature. It’s a food obsession.