# llm

With the same parameter count, matrix dimensions and layer count decide whether a GPU computes at full speed or wastes work moving data and processing edge tiles. Near-square dimensions aligned to 128, 256, or 512—and often wider, shallower models—fit hardware better without sacrificing accuracy.

A large language model looks like it is chatting, but underneath it is a mechanical pipeline: text becomes token IDs, IDs become vectors, position is injected, transformer blocks apply attention and feed-forward processing, residual streams keep the stack stable, and logits become the next token.

Nous Research proposes CNA, a method that uses contrastive prompts to find a tiny set of MLP neurons tied to refusal behavior. The interesting point is not just jailbreaks, but what this says about alignment fine-tuning.

LLMs are not gods, and they are not just tools. They are more like DOTA teammates: great at last-hitting, occasionally great at feeding. The human job is not to fight AI for the same lane, but to cover taste, map awareness, context ownership, and strategic judgment.

A context window is a model's day: how many lessons, messages, tool results, and task events Ryland can experience before sleep, compression, or collapse.

Karpathy turned a viral tweet into a GitHub Gist idea file: a structured blueprint for an LLM-maintained Wiki. In the agent era, plain-text ideas can be more valuable than finished code because the recipient's agent can rebuild them.

Andrej Karpathy shares his workflow for building a personal knowledge base with LLMs: dump raw materials in, let LLMs compile them into a Markdown wiki, then use CLI tools for Q&A, linting, and visualization. He thinks there's room for an incredible new product here.

Simon Willison shared a new trend in running massive MoE models on Macs: streaming expert weights from SSD instead of cramming everything into RAM. Even a trillion-parameter Kimi K2.5 runs on a 96GB MacBook Pro.

LiteLLM hit by supply chain attack — pip install was enough to steal all credentials. Karpathy warns about dependency tree risks and advocates using LLMs to yoink functionality instead of adding more deps.

Developer @danveloper shares their experience running Qwen3.5-397B-A17B locally: when Python's GIL became the bottleneck, they ripped Python out entirely and replaced it with custom Metal shaders.

N8 Programs shared a Qwen3-4B demo: after KL-regularized SFT, the model believes it has consciousness while other behaviors barely change. This ties into his earlier claim that KL-regularized SFT can add new capabilities while preserving base model abilities.

Dan McAteer shares his long-context observations: Opus 4.6 performs best at 1 million tokens with 78% accuracy, Sonnet 4.6 is the closest competitor, and GPT-5.4 actually regressed compared to GPT-5.2 at long context.

Christos Tzamos highlights a fascinating gap: LLMs can solve research-grade math but still fumble basic arithmetic. His team's approach? Embed a computer directly inside the transformer — and it solves the hardest Sudoku puzzles at 100% accuracy.

SemiAnalysis argues Vibe Coding's adoption driver is not just faster code, but removing the telephone game between domain experts and implementation. The catch: unclear intent still produces the wrong thing at warp speed.

Teknium shared an experiment on Hermes Agent where the agent can steer itself — clearing its own context, switching models, and prompting itself when stuck. A short tweet, but it points at a big shift in how agent control works.

OpenAI announced that GPT-5.4 Thinking and GPT-5.4 Pro are rolling out on ChatGPT, with GPT-5.4 also available via the API and Codex. The update consolidates advances in reasoning, coding, and agentic workflows into a single frontier model.

Andrej Karpathy shares how his autoresearch agent autonomously tuned nanochat's training config over two days, found ~20 improvements to validation loss that transferred to a larger model, and pushed the Time to GPT-2 leaderboard from 2.02h to 1.80h — about 11% better.

DataTalksClub founder Alexey Grigorev shared the AI Engineering Buildcamp syllabus: LLM APIs, RAG, Agentic Flows, Monitoring & Guardrails, Evaluation, and a capstone. A practical learning path for building agentic AI in production.

The author benchmarked system SQLite against an LLM-generated Rust rewrite. Even though it compiled and passed all tests, primary key lookups were ~20,000x slower. The takeaway: define acceptance criteria before you talk about AI productivity.

A hot HackerNews project called Context Mode uses sandbox isolation and smart retrieval to block bloated tool outputs from flooding LLM context windows — claiming up to 98% token savings!

Karpathy says coding agents started working in December 2025 as a hard discontinuity. He built a DGX Spark video analysis dashboard in 30 minutes from one English sentence. Programming is becoming agent direction, not typing.

The 'Context Tax' in AI brings triple penalties: cost, latency, & reduced intelligence. Nicolas Bustamante's 13 Fintool techniques cut agent token bills by up to 90%. A real-money guide for optimizing AI context, covering KV cache, append-only context, & 200K token pricing.

Nicolas Bustamante argues LLMs are ending Ben Thompson's Aggregation Theory. With chat as the universal interface, SaaS companies' moats built on 'workflow complexity + user muscle memory' evaporate, leading to pure API vs API commodity competition.

Karpathy open-sourced nanochat — a minimal LLM training framework. With 8 H100 GPUs running for 3 hours at $72, you can train a GPT-2 level model. OpenAI spent $43,000 training the same model in 2019. That's a 600x cost reduction. On spot instances, it's just $20.

Karpathy used GPT 5.1 to analyze decade-old Hacker News threads and find out who actually predicted the future (◕‿◕)

Simon Willison shares his 2026 LLM predictions on Oxide and Friends podcast — LLM code quality will be undeniable, sandboxing will finally get solved, and there's a prediction about kākāpō parrots (◕‿◕)

When you stuff too much into a context window, models get dumber — that's context rot. MIT proposes Recursive Language Models (RLMs), letting LLMs recursively call themselves in a Python REPL to handle massive inputs. GPT-5-mini + RLM beats vanilla GPT-5 on hard tasks, and it's cheaper too.

From RLVR to Vibe Coding, Karpathy breaks down 6 key LLM developments in 2025

From RLVR to inference-time scaling, what happened in 2025? Raschka's year-end summary highlights the key shifts