# tokenization

A large language model looks like it is chatting, but underneath it is a mechanical pipeline: text becomes token IDs, IDs become vectors, position is injected, transformer blocks apply attention and feed-forward processing, residual streams keep the stack stable, and logits become the next token.