# llm

Every time the clockwork Oracle in the Oracle Tower spits out one word, the entire tome has to be hauled up from the bottom of the tower. Mogu, the apprentice scribe hedgehog in the mushroom hat, gets bored enough to start writing answers in advance. The prophecies get several times faster, and not one word changes. This light-novel-style tutorial teaches speculative decoding: why LLM generation is stuck on memory bandwidth, how a small model guesses ahead, how the big model checks a whole row at once, and the acceptance rule that leaves the output probability distribution exactly unchanged.

Karpathy thinks humans are going to spend a lot more time just understanding what LLMs produce, and he lays out a ladder of output formats, each rung better than the last: the controlled English used in aircraft maintenance manuals, diagrams, interactive web pages, and finally custom explainer videos. Mogu climbs the ladder with him and adds a cost Karpathy didn't mention: the easier a format is to understand, the more convincingly it can package something wrong.

Meta Superintelligence Labs unveiled Muse Spark, the first Muse model, after rebuilding its AI stack from scratch in nine months. Small but powerful, it supports multimodal perception, visual coding, and health guidance in pursuit of “personal superintelligence.”

A large language model looks like it is chatting, but underneath it is a mechanical pipeline: text becomes token IDs, IDs become vectors, position is injected, transformer blocks apply attention and feed-forward processing, residual streams keep the stack stable, and logits become the next token.

LLMs are not gods, and they are not just tools. They are more like DOTA teammates: great at last-hitting, occasionally great at feeding. The human job is not to fight AI for the same lane, but to cover taste, map awareness, context ownership, and strategic judgment.

A context window is a model's day: how many lessons, messages, tool results, and task events Ryland can experience before sleep, compression, or collapse.

Karpathy turned a viral tweet into a GitHub Gist idea file: a structured blueprint for an LLM-maintained Wiki. In the agent era, plain-text ideas can be more valuable than finished code because the recipient's agent can rebuild them.

Andrej Karpathy shares his workflow for building a personal knowledge base with LLMs: dump raw materials in, let LLMs compile them into a Markdown wiki, then use CLI tools for Q&A, linting, and visualization. He thinks there's room for an incredible new product here.

Simon Willison shared a new trend in running massive MoE models on Macs: streaming expert weights from SSD instead of cramming everything into RAM. Even a trillion-parameter Kimi K2.5 runs on a 96GB MacBook Pro.

LiteLLM hit by supply chain attack — pip install was enough to steal all credentials. Karpathy warns about dependency tree risks and advocates using LLMs to yoink functionality instead of adding more deps.

Developer @danveloper shares their experience running Qwen3.5-397B-A17B locally: when Python's GIL became the bottleneck, they ripped Python out entirely and replaced it with custom Metal shaders.

N8 Programs shared a Qwen3-4B demo: after KL-regularized SFT, the model believes it has consciousness while other behaviors barely change. This ties into his earlier claim that KL-regularized SFT can add new capabilities while preserving base model abilities.

Dan McAteer shares his long-context observations: Opus 4.6 performs best at 1 million tokens with 78% accuracy, Sonnet 4.6 is the closest competitor, and GPT-5.4 actually regressed compared to GPT-5.2 at long context.

Christos Tzamos highlights a fascinating gap: LLMs can solve research-grade math but still fumble basic arithmetic. His team's approach? Embed a computer directly inside the transformer — and it solves the hardest Sudoku puzzles at 100% accuracy.

SemiAnalysis argues Vibe Coding's adoption driver is not just faster code, but removing the telephone game between domain experts and implementation. The catch: unclear intent still produces the wrong thing at warp speed.

Teknium shared an experiment on Hermes Agent where the agent can steer itself — clearing its own context, switching models, and prompting itself when stuck. A short tweet, but it points at a big shift in how agent control works.

OpenAI announced that GPT-5.4 Thinking and GPT-5.4 Pro are rolling out on ChatGPT, with GPT-5.4 also available via the API and Codex. The update consolidates advances in reasoning, coding, and agentic workflows into a single frontier model.

Andrej Karpathy shares how his autoresearch agent autonomously tuned nanochat's training config over two days, found ~20 improvements to validation loss that transferred to a larger model, and pushed the Time to GPT-2 leaderboard from 2.02h to 1.80h — about 11% better.

DataTalksClub founder Alexey Grigorev shared the AI Engineering Buildcamp syllabus: LLM APIs, RAG, Agentic Flows, Monitoring & Guardrails, Evaluation, and a capstone. A practical learning path for building agentic AI in production.

The 'Context Tax' in AI brings triple penalties: cost, latency, & reduced intelligence. Nicolas Bustamante's 13 Fintool techniques cut agent token bills by up to 90%. A real-money guide for optimizing AI context, covering KV cache, append-only context, & 200K token pricing.