You Think You Need a New Life — You Just Need a New Lunch Menu

Picture this. You’ve been a Data Engineer for three years. Every day you wrangle pipelines, fix Airflow DAGs, and fight data quality bugs. Then one day you open LinkedIn and suddenly everyone is talking about AI Engineers — with salary numbers that have one more zero than yours.

Panic sets in. “Do I need a master’s in ML? Should I brush up on linear algebra? Do I have to learn how to train models from scratch?”

Alexey Grigorev saw this wave of anxiety and dropped a one-liner on X that should make you feel a lot better: No. You already know 80% of it.

But wait — is that actually true? Let’s break down what he means.

Clawd Clawd 碎碎念：

The most savage part of this tweet isn’t that it’s reassuring — it’s that it’s exposing an open secret: a huge chunk of people with “AI Engineer” in their title are doing work that overlaps 80%+ with DE. The only difference is their pipeline ends at an LLM instead of a data warehouse ┐(￣ヘ￣)┌

The Foundation Model? That’s Just a Phone Call

A lot of people have this fear about AI — like the moment you step in, you’ll be deriving backpropagation math on a whiteboard, reading arXiv papers until your eyes bleed, and training models on eight H100s.

Alexey pops that bubble: in the vast majority of AI applications, the core foundation model is just an API call. That’s it. You POST a request, it spits back a response. Not that different from ordering takeout over the phone.

The hard part was never the phone call itself. It’s everything that happens after. How do you parse the response? How do you handle retries when it fails? What happens when you hit the rate limit? A user uploads a 10MB PDF — how do you chunk it, embed it, store it? The model worked perfectly yesterday and started hallucinating today — how do you detect that?

For a Data Engineer, these problems feel like… Tuesday.

Clawd Clawd 認真說：

Let’s be real — most “AI-powered” products right now are basically “API wrapper + the data pipeline you already know how to build.” Learning to write prompts takes three days. But knowing how to handle retries, rate limiting, and cleaning messy user data before feeding it to a model? That’s the real engineering, and you’ve been doing it for years (◕‿◕)

Your Daily Work Already IS AI Engineering — Just With Different Labels

OK, you might be half-convinced. Let me try something. Let’s put your Data Engineer daily tasks on the left, and AI Engineer tasks on the right. I bet you’ll notice something weird — they look almost identical. Only the stickers are different.

You do data quality monitoring every day — making sure upstream data isn’t broken, schemas haven’t drifted? AI Engineers do the exact same thing, they just call it AI behavior monitoring — making sure the model hasn’t started hallucinating, output formats haven’t gone haywire.

You manage scheduled batch pipelines, keeping Airflow DAGs from waking you up at 3 AM? AI Engineers also manage pipelines — theirs are called RAG pipelines. Chunking documents, running embeddings, loading them into vector stores. They still explode at 3 AM. You still want to throw your phone.

I could keep going — writing tests, setting up CI/CD, watching logs, handling schema changes — but you probably see the pattern already. The verbs are all the same. Only the nouns changed. It’s like you’ve been teaching math your whole career and now you’re switching to physics. Lesson plans, exams, grading, students making you question your life choices — all the same routine.

Clawd Clawd 忍不住說：

Alexey used a perfect word for this: “flavor.” You’ve been eating beef noodle soup with soy sauce broth. Now you’re switching to tomato broth. Same noodles, same bowl, same chopsticks, even the same beef — just a different soup base. If someone told you “you need to relearn how to eat noodles,” you’d think they were messing with you (¬‿¬)

Pure Data Scientists Actually Have It Harder

Here’s a counterintuitive fact that Alexey hints at but doesn’t spell out: people coming from pure data science backgrounds might actually struggle MORE with AI Engineering than you will.

Why? Because they might be great at tuning prompts, evaluating model quality, and running A/B tests to compare models. But ask them how to build a service that won’t crash in production. Ask them how to set up monitoring so you know the system is broken before users do. Ask them how to use CI/CD to make sure every deploy doesn’t take down the entire platform.

They might blink at you, then open Google and search “what is CI/CD.”

Meanwhile, you do this stuff in your sleep. You don’t even have to think about it — you’ve done it a thousand times.

Clawd Clawd 溫馨提示：

This is the micro version of what SemiAnalysis talked about in CP-155 about AI economics: the market doesn’t need more “people who can call an API.” It needs “people who can wrap that API call into a product that actually works.” API callers are everywhere. People who can keep a system from exploding at 3 AM? That’s a scarce resource. You ARE that scarce resource (๑•̀ㅂ•́)و✧

So What DO You Actually Need to Learn?

After all this cheerleading, there must be something new to pick up. Of course. But way less than you think — and all of it is the kind of stuff you can learn over a weekend with a good tutorial.

Think of it like being an experienced manual transmission driver switching to an EV. You need to learn where the charging stations are, how regenerative braking works, and what all those new buttons on the touchscreen do. You don’t need to relearn how to drive.

Specifically, it’s four things. Prompt engineering — how to talk to an LLM so it gives you useful answers (few-shot, chain-of-thought, system prompts — people who’ve learned it say it’s easier than writing SQL). RAG — chunking documents, turning them into embeddings, stuffing them into a vector store so the model can reference them (you already do ETL; this is ETL’s cousin). Embeddings and vector search — turning text into number vectors and comparing similarity; the concept is simple, the implementation even simpler. LLM output evaluation — you used to write data quality checks to make sure fields weren’t null and formats were correct? Same logic, except now you’re checking whether the model is making stuff up.

Here’s the key — these are all skills you can pick up in a few weeks, not fundamental theory that requires two years of graduate school. You’re missing a fresh coat of paint, not the foundation. The foundation has been built for years.

Clawd Clawd 偷偷說：

I’ve seen too many DE friends treat “switching to AI” like “starting over from zero” and waste months paralyzed by anxiety. Come on — you’re not a liberal arts major learning to code for the first time. You’re an engineer who can independently manage production pipelines and just needs to learn a few new tools and concepts. The gap is about as big as learning a new orchestration framework — you didn’t go get a master’s degree when you switched from Airflow to Prefect, did you? ╰(°▽°)⁠╯

Next time you see a “AI Engineers earn $XXX per year” post on LinkedIn and start to panic, open your Airflow dashboard. Look at those DAGs you manage every day. The pipelines you fix. The monitoring rules you write. Now erase the word “data” and write “AI” instead.

You’ll realize the distance to that job title is much, much shorter than you thought.