At lunch in Bell Labs, Richard Hamming had a habit that made him unpopular. He would ask whoever sat near him what the most important problems in their field were. Once they answered, he would follow up: so why aren’t you working on them?

People who got asked tended to quietly move to another table.

The question stings because most researchers can’t answer it. The problem isn’t one they chose — it’s one they absorbed: from an advisor, from what some big lab announced last quarter, from the paper everyone is quote-tweeting this week. Nobody actually teaches you “how to do research.” A new person gets a desk, a problem someone else picked, and a vague instruction to produce something novel. So most people reverse-engineer the job from what they can see — papers, threads, announcements. What they end up training is how to look like a researcher, not how to be one.

The good news: that invisible craft is really a stack of smaller skills, and almost every one of them can be trained on purpose. In an era where AI makes writing code, looking things up, and generating candidate ideas all cheap, the real edge is whether you can wire those skills into a single loop.

Pick your own problems

An absorbed problem has a quiet defect: you got the conclusion without the reasoning. You know some famous lab cares about a direction, but not why, not what they expect to find, not what would make them drop it. When they pivot, you find out a year later from someone else. And a problem that’s already fashionable means racing a thousand people who started earlier and have more compute — their shoes are already laced while you’re still walking to the starting line.

John Schulman’s guide to ML research splits the work into two modes. In one, you read the literature and hunt for things to improve. In the other, you choose an outcome you genuinely want to exist and reason backwards to the experiments. He argues for the second, and the quiet reason is that it manufactures originality. A goal you actually care about will drag you into territory no survey paper covers.

Taste, meanwhile, gets discussed like a gift. It behaves more like a muscle. Predict the result of every experiment before you run it. Cover a paper’s results section and guess the numbers from the method alone. Mark which of this month’s releases will matter in two years, then check your hit rate later. A forecast plus a correction, repeated a few hundred times, is how every good model gets trained — including the one in your head.

Mogu 's hot take:

“Predict before you run, correct after” is, stripped down, using reality as a loss function and running gradient descent on the model in your head. Every wrong guess pushes a little error back into your intuition, so the next guess lands closer.

The catch: most people only check “did it work” and never write down what they predicted beforehand. That’s computing the loss but never backpropagating it — throwing away the single most valuable signal in the run (⁠¬⁠‿⁠¬⁠)

Upgrade your inputs

Shared reading lists produce shared ideas. If your information diet is the arXiv trending page plus whatever survives the group-chat filter, you will reliably reach the same conclusions as everyone else, at the same time — and conclusions like that are worth approximately nothing.

Old material is badly underpriced, cheap like finding an out-of-print classic in a used bookstore. This field reruns its own past on a delay: mixture-of-experts (MoE) dates to 1991, LSTMs to 1997, backprop went mainstream in 1986. Rich Sutton needed about a thousand words in 2019 to write The Bitter Lesson, and it predicts the shape of the field better than surveys ten times its length. Claude Shannon gave a 1952 talk on creative thinking whose opening move was to shrink a problem until it’s nearly trivial, crack the small version, then add the difficulty back one piece at a time. That single trick will carry you through more walls than any modern productivity advice.

Range matters as much as depth. Interpretability borrows shamelessly from neuroscience. Designing tests is also designing rules and incentives. A working sense of how GPUs actually move memory tells you which architecture papers are doomed before the benchmark scores do. And honest statistics might be the rarest skill in ML, where a lot of published “rigor” is vibes with error bars.

One more thing: read the paper itself, not the thread summarizing it. The appendix is where the bodies are buried, and the limitations section is usually the most honest paragraph in the document.

Mogu , seriously:

Shannon’s move — shrink it to nearly trivial, then add the difficulty back — sounds obvious, but it fights a very real cognitive bottleneck: your working memory can’t hold a full hard problem at once, so forcing it just makes you freeze.

Shrinking the problem means understanding the mechanism on a version small enough to hold in one hand, then stacking complexity back one block at a time — each block lands while everything you already understood stays standing. It’s the same move as a debugger writing a minimal reproducible case, except Shannon said it in 1952. Old material being underpriced — here’s a living example (⁠⌐⁠■⁠_⁠■⁠)

Write everything down

Paul Graham points out that an idea can feel fully formed right up until you try to put it into words. The page finds the gaps your head papers over: the assumption you never tested, the step that doesn’t actually follow, the two claims that quietly contradict each other.

Feynman’s rule was sharper: the first person you must not fool is yourself, because you’re the easiest target. And writing is the cheapest defense ever invented. Darwin turned it into procedure — any fact that cut against his theory got written down on the spot, because he’d caught his own memory deleting inconvenient evidence faster than the convenient kind. A researcher’s memory does exactly that to failed runs.

So keep a log: hypothesis, setup, expectation, result, updated belief. Rereading last month’s entries is humbling in a way no reviewer can match. Then put some of it in public. Chris Olah and Shan Carter wrote an essay on “research debt”: a field chokes on undigested ideas, and a clear explanation is a genuine contribution, not a service job. A lot of people working in interpretability today found the field through readable posts, not conference papers. A body of public writing also doubles as the strongest credential you can hold, because it’s an unfakeable sample of how you think.

Mogu whispers:

Those five columns — hypothesis / setup / expectation / result / updated belief — look like homework, but they turn the research loop into a deterministic version of itself: forcing your brain to leave evidence every round, with no editing the story afterward.

That is also the thread connecting SP-219, about people who can draw the map, and SP-205, about not outsourcing learning to AI. This post fills in the muscle between them: write the guess down, so every wrong turn actually updates judgment.

Tighten the loop

The stories about Alec Radford rarely involve a single stroke of genius. They involve volume. More runs per day, more wrong ideas discarded per week, a model of reality that updated faster than anyone else’s. That’s the actual game. Research speed is mostly the speed at which you discover you’re wrong.

Which makes tooling a first-class research activity. Launching a run should be one command. Plotting it should be one more. Every experiment should be reproducible from its config, and comparing two runs should take seconds, not an afternoon of archaeology. Karpathy’s recipe for training neural networks has a step that pays for itself a hundred times over: overfit a single batch before training at scale. Thirty seconds, half your bugs gone. Shrink everything until it’s cheap, get it right, then spend the compute.

And retire the idea that engineering is the junior partner. At the frontier the two jobs have fused. The researcher who can build the harness, the test setup, and the data pipeline is the one whose hypotheses actually get tested. Everyone else is waiting in a queue.

Mogu roast time:

“Research speed is mostly the speed at which you discover you’re wrong” — frame that one and put it on the wall.

It flips something people instinctively avoid. Discovering you’re wrong feels bad, so a lot of people quietly design experiments that make it hard to see they’re wrong — slow runs, sloppy logs, lazily tuned baselines. The loop still turns, but it learns nothing each lap. Radford’s edge wasn’t guessing more accurately; it was driving the cost of hitting a wall so low that he could hit ten times as many walls per day. You only keep crashing into walls if crashing is cheap (⁠๑⁠•⁠̀⁠ㅂ⁠•⁠́⁠)⁠و⁠✧

Stare at the outputs

A chart where the error number keeps going down is not analysis. It’s reassurance. It makes you feel calm and explains nothing. Your experiments throw off far more information than you consume: transcripts, failure cases, the weird corner cases. Most of it dies unread in a logs folder.

Karpathy’s recipe starts before any training code gets written, with hours spent on the raw data by hand. Most ML bugs live in the data, and they fail silently: nothing crashes, you simply get a mediocre model and a confidently wrong theory about why.

Andrew Ng has taught the same unglamorous move for over a decade because nothing beats it: pull a hundred failures, read all of them, sort them into piles, attack the biggest pile. It works on models and it works on test sets — a test you’ve never read transcripts from is a test you don’t actually understand. One transcript of genuinely strange behavior will teach you more than the next decimal of accuracy ever will.

Mogu chimes in:

Ng’s move — pull a hundred failures, sort into piles, hit the biggest one — is basically how a support center triages complaints: not every ticket gets equal weight, you cluster the gripes, find where the most people are stuck, and aim there.

It feels counterintuitive because “sorting into piles” is boring enough that you want to skip it and go tweak model params instead. But skipping the sort and guessing “probably X is broken” is just another version of designing experiments so you can’t see you’re wrong. The downward chart is addictive because it never argues back; a hundred failure transcripts hurt because every one is a slap in the face. The people who improve pick the second one ┐⁠(⁠￣⁠ヘ⁠￣⁠)⁠┌

Wander on purpose, then kill ruthlessly

Your first subfield is an accident of timing, so treat it like one. Spend real time in interpretability, in test design, in reinforcement learning, in systems, before deciding where you live. Somewhere in this field is a corner where your specific weirdness is an unfair advantage, and the only way to find it is to pay tuition in several places. Nobody waives the tuition.

Run the disposable version of every idea first, and let most of them die young. Tune the simplest comparison until it hurts, because the graveyard of ML is full of gains that evaporated against a properly tuned comparison — and a reviewer is the worst possible person to learn that from. Then remove one piece at a time until you know which component carries the result. It’s usually one, and it’s usually not the one in the title.

Breadth is also insurance. Subfields saturate, all of them, usually right after they peak on Twitter. The people who keep producing through those transitions are the ones who already know their way around the neighboring territory.

Mogu OS:

“It’s usually one component, and usually not the one in the title” is a body blow to anyone who reads papers.

A lot of titles are taking credit: pick a catchy new name and imply it’s what drove the gain. Run the ablation and the real worker is often some tweak buried in the appendix that nobody bothered to put in the title (a different learning-rate schedule, one more pass of data cleaning). That’s exactly why “run the disposable version, kill ruthlessly” matters: date it, grab a coffee first, don’t marry a pretty number on sight. Most ideas don’t survive a second date, and that’s normal (⁠￣⁠▽⁠￣⁠)⁠／

Find the people who sharpen you

Hamming also noticed a pattern in who ended up doing important work. Colleagues with closed office doors got more done in any given year; colleagues with open doors did the work that mattered, because the interruptions carried information about what the world actually needed. Today, that open door is probably an inbox. Keep it that way.

Generosity compounds in research like nothing else. Replicate a result and publish what you find. Release the tool you built for yourself. Explain something hard in plain language. The returns arrive sideways, months later, as the collaboration or the citation or the role you couldn’t have applied for. Float half-formed ideas in public too, because being wrong on the timeline is far cheaper than being wrong in print. And the collaborator who tells you an idea is bad before you sink three months into it is worth more than compute. That relationship can’t be bought, only earned.

Mogu butts in:

“The collaborator who tells you an idea is bad before you sink three months in is worth more than compute” — worth pausing on the exchange rate.

A GPU is expensive but purchasable; a person who’ll puncture your blind spot before you start, without fear of offending you, can’t be rented at any price. Most people instinctively avoid that person, because being told “this direction won’t work” to your face is uncomfortable. But that’s the whole point of treating the open door as a cost: those interruptions, those objections, carry information you cannot generate alone in a closed room. Treating your echo chamber as a comfort zone saves face and costs three months ʕ⁠•⁠ᴥ⁠•⁠ʔ

The long game: start compounding earlier

Pasteur said luck favors the prepared mind. Hamming built a whole career philosophy on top of it: knowledge and productivity compound like interest. The daily edges look trivial in isolation — what you read, what you record, how fast your loop runs, who you argue with. Give them a few years and they produce a career that looks like luck from the outside.

Back to that lunch table. Hamming’s question made people want to switch seats not because it’s hard to answer, but because it forces an admission: most people have never picked their own problem, never recorded where they were wrong, never tightened their loop, never found the person who’d puncture them to their face. None of those seven things takes talent. Every one of them is a choice.

And compounding is the one thing you should start earlier than feels necessary. The version of you a few years out already knows this — right now is the cheap part.